2024-09-10T21:44:36.204 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T21:44:36.225 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T21:44:36.314 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899705' kernel: flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' mgr: client mount timeout: 30 debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOM+g29lHpSCxgRzgZ2SkoMjXZ8tkwMtW4Rko10j4iY8exzxkFWClPtAlnFQUgkYDS7K4bBits9Jp9Ff6YoBWEw= smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMUss4KSGVsNmsqLtCP5YU8u8vTtpGLblwcDOS0jV528pxaQ9clrxUOHT4VidxwBec+XfoC9a2eLL3m77abOXlk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgressReporter teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T21:44:36.314 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T21:44:36.314 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T21:44:36.315 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T21:44:36.317 INFO:teuthology.task.internal:Checking packages... 2024-09-10T21:44:36.341 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T21:44:36.341 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T21:44:36.341 INFO:teuthology.packaging:ref: None 2024-09-10T21:44:36.341 INFO:teuthology.packaging:tag: None 2024-09-10T21:44:36.341 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T21:44:36.341 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:44:36.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T21:44:36.498 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:44:36.500 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T21:44:36.506 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T21:44:36.506 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T21:44:36.513 INFO:teuthology.task.internal:Saving configuration 2024-09-10T21:44:36.530 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T21:44:36.536 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T21:44:36.569 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 21:38:19.233359', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHaFF2S0ajXMmSFtRxxGRSexRt992zqO3i47j15jJ/Td'} 2024-09-10T21:44:36.603 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 21:38:19.234764', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOgpbey4pB3GG33KKqpxr9qR40Q/FJZaVjqB/CWN2g57'} 2024-09-10T21:44:36.603 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T21:44:36.609 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-10T21:44:36.610 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-10T21:44:36.610 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T21:44:36.691 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa5f92afbe0>, signals=[15]) 2024-09-10T21:44:36.692 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T21:44:36.697 INFO:teuthology.task.internal:Opening connections... 2024-09-10T21:44:36.697 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T21:44:36.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:44:36.794 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T21:44:36.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:44:36.884 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T21:44:36.891 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-09-10T21:44:36.897 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-09-10T21:44:36.897 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-09-10T21:44:36.943 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-10T21:44:36.944 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="Ubuntu" 2024-09-10T21:44:36.944 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="22.04" 2024-09-10T21:44:36.944 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-10T21:44:36.944 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_CODENAME=jammy 2024-09-10T21:44:36.944 INFO:teuthology.orchestra.run.smithi066.stdout:ID=ubuntu 2024-09-10T21:44:36.944 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE=debian 2024-09-10T21:44:36.944 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T21:44:36.944 INFO:teuthology.orchestra.run.smithi066.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T21:44:36.944 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T21:44:36.944 INFO:teuthology.orchestra.run.smithi066.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T21:44:36.944 INFO:teuthology.orchestra.run.smithi066.stdout:UBUNTU_CODENAME=jammy 2024-09-10T21:44:36.945 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-09-10T21:44:37.014 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-09-10T21:44:37.019 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-09-10T21:44:37.019 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-09-10T21:44:37.066 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-10T21:44:37.066 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="Ubuntu" 2024-09-10T21:44:37.066 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="22.04" 2024-09-10T21:44:37.067 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-10T21:44:37.067 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_CODENAME=jammy 2024-09-10T21:44:37.067 INFO:teuthology.orchestra.run.smithi094.stdout:ID=ubuntu 2024-09-10T21:44:37.067 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE=debian 2024-09-10T21:44:37.067 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T21:44:37.067 INFO:teuthology.orchestra.run.smithi094.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T21:44:37.067 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T21:44:37.067 INFO:teuthology.orchestra.run.smithi094.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T21:44:37.067 INFO:teuthology.orchestra.run.smithi094.stdout:UBUNTU_CODENAME=jammy 2024-09-10T21:44:37.067 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-09-10T21:44:37.097 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T21:44:37.105 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T21:44:37.110 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T21:44:37.110 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-09-10T21:44:37.113 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-09-10T21:44:37.117 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T21:44:37.123 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T21:44:37.123 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-09-10T21:44:37.161 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-09-10T21:44:37.380 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T21:44:37.450 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T21:44:37.464 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T21:44:37.464 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'mon.b': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, timeout 300 2024-09-10T21:44:37.464 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T21:44:37.465 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T21:44:37.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:44:37.470 DEBUG:teuthology.task.kernel:sha1 for mon.a is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:44:37.470 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T21:44:37.470 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-09-10T21:44:37.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:44:37.471 DEBUG:teuthology.task.kernel:sha1 for mon.b is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:44:37.471 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T21:44:37.471 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-09-10T21:44:37.517 INFO:teuthology.orchestra.run.smithi066.stdout:5.15.0-56-generic 2024-09-10T21:44:37.517 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:44:37.517 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T21:44:37.518 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:44:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:5.15.0-56-generic 2024-09-10T21:44:37.530 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:44:37.530 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T21:44:37.531 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:44:37.680 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2024-09-10T21:44:37.680 DEBUG:teuthology.orchestra.run.smithi066:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-10T21:44:37.685 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2024-09-10T21:44:37.686 DEBUG:teuthology.orchestra.run.smithi094:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-10T21:44:37.699 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-10T21:44:37.718 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-10T21:44:37.823 INFO:teuthology.orchestra.run.smithi066.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-10T21:44:37.823 INFO:teuthology.orchestra.run.smithi066.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-10T21:44:37.823 INFO:teuthology.orchestra.run.smithi066.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-10T21:44:37.823 INFO:teuthology.orchestra.run.smithi066.stderr:hint: 2024-09-10T21:44:37.823 INFO:teuthology.orchestra.run.smithi066.stderr:hint: git config --global init.defaultBranch 2024-09-10T21:44:37.823 INFO:teuthology.orchestra.run.smithi066.stderr:hint: 2024-09-10T21:44:37.824 INFO:teuthology.orchestra.run.smithi066.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-10T21:44:37.824 INFO:teuthology.orchestra.run.smithi066.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-10T21:44:37.824 INFO:teuthology.orchestra.run.smithi066.stderr:hint: 2024-09-10T21:44:37.824 INFO:teuthology.orchestra.run.smithi066.stderr:hint: git branch -m 2024-09-10T21:44:37.824 INFO:teuthology.orchestra.run.smithi066.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-10T21:44:37.826 DEBUG:teuthology.orchestra.run.smithi066:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-10T21:44:37.831 INFO:teuthology.orchestra.run.smithi094.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-10T21:44:37.832 INFO:teuthology.orchestra.run.smithi094.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-10T21:44:37.832 INFO:teuthology.orchestra.run.smithi094.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-10T21:44:37.832 INFO:teuthology.orchestra.run.smithi094.stderr:hint: 2024-09-10T21:44:37.832 INFO:teuthology.orchestra.run.smithi094.stderr:hint: git config --global init.defaultBranch 2024-09-10T21:44:37.832 INFO:teuthology.orchestra.run.smithi094.stderr:hint: 2024-09-10T21:44:37.832 INFO:teuthology.orchestra.run.smithi094.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-10T21:44:37.832 INFO:teuthology.orchestra.run.smithi094.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-10T21:44:37.832 INFO:teuthology.orchestra.run.smithi094.stderr:hint: 2024-09-10T21:44:37.832 INFO:teuthology.orchestra.run.smithi094.stderr:hint: git branch -m 2024-09-10T21:44:37.833 INFO:teuthology.orchestra.run.smithi094.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-10T21:44:37.835 DEBUG:teuthology.orchestra.run.smithi094:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-10T21:44:37.891 DEBUG:teuthology.orchestra.run.smithi066:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-10T21:44:37.914 DEBUG:teuthology.orchestra.run.smithi094:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-10T21:44:37.944 DEBUG:teuthology.orchestra.run.smithi066:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-10T21:44:37.969 DEBUG:teuthology.orchestra.run.smithi094:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-10T21:45:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-10T21:45:14.653 INFO:teuthology.orchestra.run.smithi094.stderr: * [new branch] main -> origin/main 2024-09-10T21:45:14.689 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190312 -> 20190312 2024-09-10T21:45:14.689 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190416 -> 20190416 2024-09-10T21:45:14.689 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190514 -> 20190514 2024-09-10T21:45:14.689 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190618 -> 20190618 2024-09-10T21:45:14.689 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190717 -> 20190717 2024-09-10T21:45:14.689 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190815 -> 20190815 2024-09-10T21:45:14.689 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190923 -> 20190923 2024-09-10T21:45:14.689 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20191022 -> 20191022 2024-09-10T21:45:14.689 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20191215 -> 20191215 2024-09-10T21:45:14.689 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200122 -> 20200122 2024-09-10T21:45:14.690 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200316 -> 20200316 2024-09-10T21:45:14.692 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200421 -> 20200421 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200519 -> 20200519 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200619 -> 20200619 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200721 -> 20200721 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200817 -> 20200817 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200918 -> 20200918 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20201022 -> 20201022 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20201118 -> 20201118 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20201218 -> 20201218 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210208 -> 20210208 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210315 -> 20210315 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210511 -> 20210511 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210716 -> 20210716 2024-09-10T21:45:14.693 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210818 -> 20210818 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210919 -> 20210919 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20211027 -> 20211027 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20211216 -> 20211216 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220209 -> 20220209 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220310 -> 20220310 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220411 -> 20220411 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220509 -> 20220509 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220610 -> 20220610 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220708 -> 20220708 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220815 -> 20220815 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220913 -> 20220913 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20221012 -> 20221012 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20221109 -> 20221109 2024-09-10T21:45:14.694 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20221214 -> 20221214 2024-09-10T21:45:14.695 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230117 -> 20230117 2024-09-10T21:45:14.696 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230210 -> 20230210 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230310 -> 20230310 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230404 -> 20230404 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230515 -> 20230515 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230625 -> 20230625 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230804 -> 20230804 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230919 -> 20230919 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20231030 -> 20231030 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20231111 -> 20231111 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20231211 -> 20231211 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240115 -> 20240115 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240220 -> 20240220 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240312 -> 20240312 2024-09-10T21:45:14.697 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240410 -> 20240410 2024-09-10T21:45:14.698 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240513 -> 20240513 2024-09-10T21:45:14.698 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240610 -> 20240610 2024-09-10T21:45:14.698 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240709 -> 20240709 2024-09-10T21:45:14.698 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240811 -> 20240811 2024-09-10T21:45:14.698 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240909 -> 20240909 2024-09-10T21:45:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-10T21:45:17.481 INFO:teuthology.orchestra.run.smithi066.stderr: * [new branch] main -> origin/main 2024-09-10T21:45:17.519 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190312 -> 20190312 2024-09-10T21:45:17.519 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190416 -> 20190416 2024-09-10T21:45:17.519 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190514 -> 20190514 2024-09-10T21:45:17.519 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190618 -> 20190618 2024-09-10T21:45:17.520 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190717 -> 20190717 2024-09-10T21:45:17.520 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190815 -> 20190815 2024-09-10T21:45:17.520 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190923 -> 20190923 2024-09-10T21:45:17.520 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20191022 -> 20191022 2024-09-10T21:45:17.520 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20191215 -> 20191215 2024-09-10T21:45:17.521 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200122 -> 20200122 2024-09-10T21:45:17.522 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200316 -> 20200316 2024-09-10T21:45:17.522 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200421 -> 20200421 2024-09-10T21:45:17.522 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200519 -> 20200519 2024-09-10T21:45:17.522 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200619 -> 20200619 2024-09-10T21:45:17.522 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200721 -> 20200721 2024-09-10T21:45:17.522 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200817 -> 20200817 2024-09-10T21:45:17.522 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200918 -> 20200918 2024-09-10T21:45:17.522 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20201022 -> 20201022 2024-09-10T21:45:17.522 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20201118 -> 20201118 2024-09-10T21:45:17.524 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20201218 -> 20201218 2024-09-10T21:45:17.524 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20210208 -> 20210208 2024-09-10T21:45:17.524 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20210315 -> 20210315 2024-09-10T21:45:17.524 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20210511 -> 20210511 2024-09-10T21:45:17.524 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20210716 -> 20210716 2024-09-10T21:45:17.524 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20210818 -> 20210818 2024-09-10T21:45:17.524 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20210919 -> 20210919 2024-09-10T21:45:17.525 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20211027 -> 20211027 2024-09-10T21:45:17.525 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20211216 -> 20211216 2024-09-10T21:45:17.525 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220209 -> 20220209 2024-09-10T21:45:17.525 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220310 -> 20220310 2024-09-10T21:45:17.525 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220411 -> 20220411 2024-09-10T21:45:17.525 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220509 -> 20220509 2024-09-10T21:45:17.525 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220610 -> 20220610 2024-09-10T21:45:17.526 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220708 -> 20220708 2024-09-10T21:45:17.526 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220815 -> 20220815 2024-09-10T21:45:17.527 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220913 -> 20220913 2024-09-10T21:45:17.527 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20221012 -> 20221012 2024-09-10T21:45:17.527 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20221109 -> 20221109 2024-09-10T21:45:17.527 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20221214 -> 20221214 2024-09-10T21:45:17.527 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230117 -> 20230117 2024-09-10T21:45:17.528 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230210 -> 20230210 2024-09-10T21:45:17.528 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230310 -> 20230310 2024-09-10T21:45:17.528 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230404 -> 20230404 2024-09-10T21:45:17.528 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230515 -> 20230515 2024-09-10T21:45:17.528 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230625 -> 20230625 2024-09-10T21:45:17.528 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230804 -> 20230804 2024-09-10T21:45:17.529 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230919 -> 20230919 2024-09-10T21:45:17.529 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20231030 -> 20231030 2024-09-10T21:45:17.529 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20231111 -> 20231111 2024-09-10T21:45:17.529 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20231211 -> 20231211 2024-09-10T21:45:17.529 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20240115 -> 20240115 2024-09-10T21:45:17.529 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20240220 -> 20240220 2024-09-10T21:45:17.530 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20240312 -> 20240312 2024-09-10T21:45:17.530 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20240410 -> 20240410 2024-09-10T21:45:17.530 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20240513 -> 20240513 2024-09-10T21:45:17.530 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20240610 -> 20240610 2024-09-10T21:45:17.530 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20240709 -> 20240709 2024-09-10T21:45:17.531 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20240811 -> 20240811 2024-09-10T21:45:17.531 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20240909 -> 20240909 2024-09-10T21:45:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:Updating files: 44% (1569/3501) Updating files: 45% (1576/3501) Updating files: 46% (1611/3501) Updating files: 47% (1646/3501) Updating files: 48% (1681/3501) Updating files: 49% (1716/3501) Updating files: 50% (1751/3501) Updating files: 51% (1786/3501) Updating files: 52% (1821/3501) Updating files: 53% (1856/3501) Updating files: 54% (1891/3501) Updating files: 55% (1926/3501) Updating files: 56% (1961/3501) Updating files: 57% (1996/3501) Updating files: 58% (2031/3501) Updating files: 59% (2066/3501) Updating files: 59% (2070/3501) Updating files: 60% (2101/3501) Updating files: 61% (2136/3501) Updating files: 62% (2171/3501) Updating files: 63% (2206/3501) Updating files: 64% (2241/3501) Updating files: 65% (2276/3501) Updating files: 66% (2311/3501) Updating files: 67% (2346/3501) Updating files: 68% (2381/3501) Updating files: 69% (2416/3501) Updating files: 69% (2427/3501) Updating files: 70% (2451/3501) Updating files: 71% (2486/3501) Updating files: 72% (2521/3501) Updating files: 73% (2556/3501) Updating files: 74% (2591/3501) Updating files: 75% (2626/3501) Updating files: 76% (2661/3501) Updating files: 77% (2696/3501) Updating files: 78% (2731/3501) Updating files: 79% (2766/3501) Updating files: 80% (2801/3501) Updating files: 81% (2836/3501) Updating files: 82% (2871/3501) Updating files: 82% (2891/3501) Updating files: 83% (2906/3501) Updating files: 84% (2941/3501) Updating files: 85% (2976/3501) Updating files: 86% (3011/3501) Updating files: 87% (3046/3501) Updating files: 88% (3081/3501) Updating files: 89% (3116/3501) Updating files: 90% (3151/3501) Updating files: 91% (3186/3501) Updating files: 92% (3221/3501) Updating files: 93% (3256/3501) Updating files: 94% (3291/3501) Updating files: 95% (3326/3501) Updating files: 96% (3361/3501) Updating files: 97% (3396/3501) Updating files: 98% (3431/3501) Updating files: 99% (3466/3501) Updating files: 100% (3501/3501) Updating files: 100% (3501/3501), done. 2024-09-10T21:45:19.198 INFO:teuthology.orchestra.run.smithi094.stdout:HEAD is now at ce299c0f Merge branch 'rt1320' into 'main' 2024-09-10T21:45:19.218 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.b... 2024-09-10T21:45:19.219 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:45:19.373 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:45:19.374 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-10T21:45:19.376 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi094.front.sepia.ceph.com to complete... 2024-09-10T21:45:20.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10 21:45:20 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-10T21:45:20.537 INFO:teuthology.orchestra.run.smithi094.stderr:FINISHED --2024-09-10 21:45:20-- 2024-09-10T21:45:20.538 INFO:teuthology.orchestra.run.smithi094.stderr:Total wall clock time: 1.1s 2024-09-10T21:45:20.538 INFO:teuthology.orchestra.run.smithi094.stderr:Downloaded: 1 files, 43M in 0.9s (45.7 MB/s) 2024-09-10T21:45:20.540 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.b... 2024-09-10T21:45:20.541 DEBUG:teuthology.orchestra.run.smithi094:> sudo dpkg -i /tmp/linux-image.deb 2024-09-10T21:45:20.763 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-10T21:45:22.285 INFO:teuthology.orchestra.run.smithi066.stderr:Updating files: 29% (1021/3501) Updating files: 30% (1051/3501) Updating files: 31% (1086/3501) Updating files: 32% (1121/3501) Updating files: 33% (1156/3501) Updating files: 34% (1191/3501) Updating files: 35% (1226/3501) Updating files: 36% (1261/3501) Updating files: 37% (1296/3501) Updating files: 38% (1331/3501) Updating files: 39% (1366/3501) Updating files: 40% (1401/3501) Updating files: 41% (1436/3501) Updating files: 42% (1471/3501) Updating files: 43% (1506/3501) Updating files: 44% (1541/3501) Updating files: 45% (1576/3501) Updating files: 46% (1611/3501) Updating files: 47% (1646/3501) Updating files: 48% (1681/3501) Updating files: 49% (1716/3501) Updating files: 50% (1751/3501) Updating files: 51% (1786/3501) Updating files: 52% (1821/3501) Updating files: 53% (1856/3501) Updating files: 54% (1891/3501) Updating files: 55% (1926/3501) Updating files: 56% (1961/3501) Updating files: 57% (1996/3501) Updating files: 58% (2031/3501) Updating files: 58% (2048/3501) Updating files: 59% (2066/3501) Updating files: 60% (2101/3501) Updating files: 61% (2136/3501) Updating files: 62% (2171/3501) Updating files: 63% (2206/3501) Updating files: 64% (2241/3501) Updating files: 65% (2276/3501) Updating files: 66% (2311/3501) Updating files: 67% (2346/3501) Updating files: 68% (2381/3501) Updating files: 69% (2416/3501) Updating files: 70% (2451/3501) Updating files: 71% (2486/3501) Updating files: 72% (2521/3501) Updating files: 73% (2556/3501) Updating files: 74% (2591/3501) Updating files: 75% (2626/3501) Updating files: 76% (2661/3501) Updating files: 77% (2696/3501) Updating files: 78% (2731/3501) Updating files: 79% (2766/3501) Updating files: 80% (2801/3501) Updating files: 81% (2836/3501) Updating files: 82% (2871/3501) Updating files: 82% (2875/3501) Updating files: 83% (2906/3501) Updating files: 84% (2941/3501) Updating files: 85% (2976/3501) Updating files: 86% (3011/3501) Updating files: 87% (3046/3501) Updating files: 88% (3081/3501) Updating files: 89% (3116/3501) Updating files: 90% (3151/3501) Updating files: 91% (3186/3501) Updating files: 92% (3221/3501) Updating files: 93% (3256/3501) Updating files: 94% (3291/3501) Updating files: 95% (3326/3501) Updating files: 96% (3361/3501) Updating files: 97% (3396/3501) Updating files: 98% (3431/3501) Updating files: 99% (3466/3501) Updating files: 100% (3501/3501) Updating files: 100% (3501/3501), done. 2024-09-10T21:45:22.290 INFO:teuthology.orchestra.run.smithi066.stdout:HEAD is now at ce299c0f Merge branch 'rt1320' into 'main' 2024-09-10T21:45:22.313 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.a... 2024-09-10T21:45:22.313 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:45:22.472 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:45:22.472 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-10T21:45:22.476 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi066.front.sepia.ceph.com to complete... 2024-09-10T21:45:22.774 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-10T21:45:22.778 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-10T21:45:22.882 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T21:45:23.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10 21:45:23 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-10T21:45:23.482 INFO:teuthology.orchestra.run.smithi066.stderr:FINISHED --2024-09-10 21:45:23-- 2024-09-10T21:45:23.482 INFO:teuthology.orchestra.run.smithi066.stderr:Total wall clock time: 1.0s 2024-09-10T21:45:23.482 INFO:teuthology.orchestra.run.smithi066.stderr:Downloaded: 1 files, 43M in 0.8s (51.0 MB/s) 2024-09-10T21:45:23.487 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.a... 2024-09-10T21:45:23.487 DEBUG:teuthology.orchestra.run.smithi066:> sudo dpkg -i /tmp/linux-image.deb 2024-09-10T21:45:23.705 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-10T21:45:25.656 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-10T21:45:25.661 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-10T21:45:25.790 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T21:45:26.542 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T21:45:26.652 INFO:teuthology.orchestra.run.smithi094.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:29.821 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T21:45:29.931 INFO:teuthology.orchestra.run.smithi066.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:48.957 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub' 2024-09-10T21:45:48.973 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T21:45:48.999 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-09-10T21:45:49.313 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:49.337 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T21:45:49.621 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T21:45:49.686 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T21:45:49.691 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T21:45:49.842 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T21:45:49.842 INFO:teuthology.orchestra.run.smithi094.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T21:45:49.842 INFO:teuthology.orchestra.run.smithi094.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T21:45:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-09-10T21:45:50.050 DEBUG:teuthology.orchestra.run.smithi094:> dpkg-deb -c /tmp/linux-image.deb 2024-09-10T21:45:50.058 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-10T21:45:50.058 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-10T21:45:50.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:50.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:50.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:50.100 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-10T21:45:50.100 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-10T21:45:50.100 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-10T21:45:50.100 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-10T21:45:50.100 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-10T21:45:50.100 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-10T21:45:50.100 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-10T21:45:50.100 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-10T21:45:50.100 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:45:50.101 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-10T21:45:50.101 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-10T21:45:50.101 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-10T21:45:50.101 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-10T21:45:50.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-10T21:45:50.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T21:45:50.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T21:45:50.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T21:45:50.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T21:45:50.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T21:45:50.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T21:45:50.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T21:45:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-10T21:45:50.103 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-10T21:45:50.103 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-10T21:45:50.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T21:45:50.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-10T21:45:50.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-10T21:45:50.103 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-10T21:45:50.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T21:45:50.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T21:45:50.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T21:45:50.122 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-10T21:45:50.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-10T21:45:50.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-10T21:45:50.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-10T21:45:50.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T21:45:50.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T21:45:50.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T21:45:50.123 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-10T21:45:50.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-10T21:45:50.123 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-10T21:45:50.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T21:45:50.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T21:45:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T21:45:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T21:45:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T21:45:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T21:45:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T21:45:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T21:45:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T21:45:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T21:45:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T21:45:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T21:45:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T21:45:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T21:45:50.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T21:45:50.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T21:45:50.127 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T21:45:50.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T21:45:50.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T21:45:50.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T21:45:50.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T21:45:50.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T21:45:50.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T21:45:50.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T21:45:50.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-10T21:45:50.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T21:45:50.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-10T21:45:50.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T21:45:50.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T21:45:50.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T21:45:50.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T21:45:50.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T21:45:50.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T21:45:50.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T21:45:50.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T21:45:50.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T21:45:50.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T21:45:50.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T21:45:50.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T21:45:50.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T21:45:50.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T21:45:50.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T21:45:50.133 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-10T21:45:50.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T21:45:50.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T21:45:50.134 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-10T21:45:50.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T21:45:50.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T21:45:50.134 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-10T21:45:50.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T21:45:50.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T21:45:50.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T21:45:50.134 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-10T21:45:50.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T21:45:50.134 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T21:45:50.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T21:45:50.136 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T21:45:50.136 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T21:45:50.136 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T21:45:50.136 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T21:45:50.136 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T21:45:50.136 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-10T21:45:50.136 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T21:45:50.136 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-10T21:45:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T21:45:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T21:45:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-10T21:45:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T21:45:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-10T21:45:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-10T21:45:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-10T21:45:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T21:45:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T21:45:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-10T21:45:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-10T21:45:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-10T21:45:50.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T21:45:50.139 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T21:45:50.189 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-10T21:45:50.189 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-10T21:45:50.189 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-10T21:45:50.190 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T21:45:50.190 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-10T21:45:50.190 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T21:45:50.190 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T21:45:50.190 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T21:45:50.190 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-10T21:45:50.190 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T21:45:50.190 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-10T21:45:50.190 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T21:45:50.190 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-10T21:45:50.190 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-10T21:45:50.190 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T21:45:50.191 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-10T21:45:50.192 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-10T21:45:50.193 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-10T21:45:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-10T21:45:50.195 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T21:45:50.196 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T21:45:50.197 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T21:45:50.198 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T21:45:50.199 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T21:45:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T21:45:50.201 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-10T21:45:50.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T21:45:50.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T21:45:50.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T21:45:50.202 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-10T21:45:50.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T21:45:50.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T21:45:50.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T21:45:50.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T21:45:50.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T21:45:50.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T21:45:50.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T21:45:50.202 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T21:45:50.203 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T21:45:50.204 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T21:45:50.205 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T21:45:50.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T21:45:50.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T21:45:50.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T21:45:50.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T21:45:50.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T21:45:50.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-10T21:45:50.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T21:45:50.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T21:45:50.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T21:45:50.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T21:45:50.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T21:45:50.206 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T21:45:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T21:45:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T21:45:50.209 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T21:45:50.210 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T21:45:50.211 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-10T21:45:50.212 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T21:45:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T21:45:50.214 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T21:45:50.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T21:45:50.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-10T21:45:50.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T21:45:50.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T21:45:50.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-10T21:45:50.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-10T21:45:50.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T21:45:50.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-10T21:45:50.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T21:45:50.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T21:45:50.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T21:45:50.215 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T21:45:50.216 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T21:45:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T21:45:50.218 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-10T21:45:50.219 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-10T21:45:50.220 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T21:45:50.221 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T21:45:50.222 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-10T21:45:50.223 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T21:45:50.224 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-10T21:45:50.225 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-10T21:45:50.225 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-10T21:45:50.225 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-10T21:45:50.225 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T21:45:50.225 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T21:45:50.225 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T21:45:50.225 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T21:45:50.225 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T21:45:50.225 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T21:45:50.225 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T21:45:50.225 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T21:45:50.225 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-10T21:45:50.226 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T21:45:50.227 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-10T21:45:50.228 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T21:45:50.229 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-10T21:45:50.230 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-10T21:45:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-10T21:45:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T21:45:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-10T21:45:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T21:45:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-10T21:45:50.236 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T21:45:50.237 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T21:45:50.238 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-10T21:45:50.238 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T21:45:50.238 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T21:45:50.238 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-10T21:45:50.238 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T21:45:50.238 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T21:45:50.238 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-10T21:45:50.238 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T21:45:50.238 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T21:45:50.238 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-10T21:45:50.238 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-10T21:45:50.238 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T21:45:50.272 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-10T21:45:50.272 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-10T21:45:50.272 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T21:45:50.272 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-10T21:45:50.272 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T21:45:50.272 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-10T21:45:50.272 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T21:45:50.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-10T21:45:50.274 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-10T21:45:50.275 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T21:45:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-10T21:45:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T21:45:50.278 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-10T21:45:50.279 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T21:45:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-10T21:45:50.281 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T21:45:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T21:45:50.283 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T21:45:50.284 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-10T21:45:50.285 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T21:45:50.286 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-10T21:45:50.287 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-10T21:45:50.288 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-10T21:45:50.289 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-10T21:45:50.289 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T21:45:50.289 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-10T21:45:50.289 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T21:45:50.289 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-10T21:45:50.289 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-10T21:45:50.289 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-10T21:45:50.289 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T21:45:50.289 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T21:45:50.289 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-10T21:45:50.289 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-10T21:45:50.289 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-10T21:45:50.290 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-10T21:45:50.291 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T21:45:50.291 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T21:45:50.291 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T21:45:50.291 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T21:45:50.291 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T21:45:50.291 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T21:45:50.291 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T21:45:50.291 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T21:45:50.291 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T21:45:50.291 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T21:45:50.291 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T21:45:50.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T21:45:50.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-10T21:45:50.292 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-10T21:45:50.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T21:45:50.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-10T21:45:50.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T21:45:50.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T21:45:50.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T21:45:50.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T21:45:50.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T21:45:50.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-10T21:45:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-10T21:45:50.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-10T21:45:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-10T21:45:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T21:45:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-10T21:45:50.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T21:45:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T21:45:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-10T21:45:50.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-10T21:45:50.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T21:45:50.303 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T21:45:50.304 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T21:45:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-10T21:45:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T21:45:50.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T21:45:50.308 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T21:45:50.308 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T21:45:50.308 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-10T21:45:50.308 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-10T21:45:50.308 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T21:45:50.308 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-10T21:45:50.308 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-10T21:45:50.308 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T21:45:50.344 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-10T21:45:50.344 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T21:45:50.344 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T21:45:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T21:45:50.346 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T21:45:50.347 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-10T21:45:50.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-10T21:45:50.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-10T21:45:50.350 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-10T21:45:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-10T21:45:50.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-10T21:45:50.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-10T21:45:50.354 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T21:45:50.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-10T21:45:50.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T21:45:50.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T21:45:50.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T21:45:50.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T21:45:50.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-10T21:45:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T21:45:50.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T21:45:50.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T21:45:50.363 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-10T21:45:50.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T21:45:50.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T21:45:50.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T21:45:50.363 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-10T21:45:50.363 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-10T21:45:50.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T21:45:50.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T21:45:50.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-10T21:45:50.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-10T21:45:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T21:45:50.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T21:45:50.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T21:45:50.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T21:45:50.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T21:45:50.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T21:45:50.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T21:45:50.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T21:45:50.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T21:45:50.367 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-10T21:45:50.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-10T21:45:50.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-10T21:45:50.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T21:45:50.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T21:45:50.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-10T21:45:50.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-10T21:45:50.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-10T21:45:50.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-10T21:45:50.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-10T21:45:50.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-10T21:45:50.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T21:45:50.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T21:45:50.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T21:45:50.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-10T21:45:50.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T21:45:50.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T21:45:50.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-10T21:45:50.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T21:45:50.373 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-10T21:45:50.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-10T21:45:50.374 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-10T21:45:50.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T21:45:50.374 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-10T21:45:50.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T21:45:50.374 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-10T21:45:50.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-10T21:45:50.374 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-10T21:45:50.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T21:45:50.374 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-10T21:45:50.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T21:45:50.374 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T21:45:50.375 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T21:45:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T21:45:50.377 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T21:45:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T21:45:50.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T21:45:50.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-10T21:45:50.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-10T21:45:50.382 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-10T21:45:50.382 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-10T21:45:50.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T21:45:50.382 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-10T21:45:50.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T21:45:50.382 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-10T21:45:50.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T21:45:50.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T21:45:50.417 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T21:45:50.417 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T21:45:50.417 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T21:45:50.417 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-10T21:45:50.417 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-10T21:45:50.417 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-10T21:45:50.417 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T21:45:50.417 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T21:45:50.418 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T21:45:50.419 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-10T21:45:50.420 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-10T21:45:50.421 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-10T21:45:50.422 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-10T21:45:50.423 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-10T21:45:50.424 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T21:45:50.425 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T21:45:50.426 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T21:45:50.427 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T21:45:50.428 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-10T21:45:50.429 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T21:45:50.430 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T21:45:50.431 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T21:45:50.432 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T21:45:50.433 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T21:45:50.434 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T21:45:50.435 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-10T21:45:50.436 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T21:45:50.437 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T21:45:50.438 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T21:45:50.439 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T21:45:50.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T21:45:50.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-10T21:45:50.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T21:45:50.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T21:45:50.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T21:45:50.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T21:45:50.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T21:45:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T21:45:50.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T21:45:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T21:45:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T21:45:50.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T21:45:50.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T21:45:50.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T21:45:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-10T21:45:50.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T21:45:50.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T21:45:50.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T21:45:50.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T21:45:50.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-10T21:45:50.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-10T21:45:50.463 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:50.463 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:50.463 DEBUG:teuthology.orchestra.run.smithi094:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-10T21:45:50.504 INFO:teuthology.orchestra.run.smithi094.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:45:50.504 INFO:teuthology.orchestra.run.smithi094.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:45:50.504 INFO:teuthology.orchestra.run.smithi094.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:45:50.505 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-10T21:45:50.505 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:50.505 DEBUG:teuthology.orchestra.run.smithi094:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-10T21:45:50.548 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi094.front.sepia.ceph.com to complete... 2024-09-10T21:45:50.947 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub' 2024-09-10T21:45:50.948 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-09-10T21:45:51.170 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:51.175 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:51.413 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T21:45:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T21:45:51.461 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T21:45:51.468 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T21:45:51.593 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T21:45:51.593 INFO:teuthology.orchestra.run.smithi094.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T21:45:51.593 INFO:teuthology.orchestra.run.smithi094.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T21:45:51.614 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-09-10T21:45:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:Sourcing file `/etc/default/grub' 2024-09-10T21:45:52.547 INFO:teuthology.orchestra.run.smithi066.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T21:45:52.574 INFO:teuthology.orchestra.run.smithi066.stderr:Generating grub configuration file ... 2024-09-10T21:45:52.874 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:52.896 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T21:45:53.217 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T21:45:53.286 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T21:45:53.292 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T21:45:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T21:45:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T21:45:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T21:45:53.460 INFO:teuthology.orchestra.run.smithi066.stderr:done 2024-09-10T21:45:53.620 DEBUG:teuthology.orchestra.run.smithi066:> dpkg-deb -c /tmp/linux-image.deb 2024-09-10T21:45:53.628 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-10T21:45:53.628 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-10T21:45:53.628 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:53.647 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:53.647 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:53.668 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-10T21:45:53.668 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-10T21:45:53.668 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-10T21:45:53.668 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-10T21:45:53.668 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-10T21:45:53.668 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-10T21:45:53.668 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-10T21:45:53.669 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-10T21:45:53.669 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:45:53.669 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-10T21:45:53.669 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-10T21:45:53.669 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-10T21:45:53.669 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-10T21:45:53.669 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-10T21:45:53.669 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T21:45:53.669 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T21:45:53.669 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T21:45:53.670 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T21:45:53.670 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T21:45:53.670 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T21:45:53.672 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T21:45:53.672 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T21:45:53.672 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T21:45:53.672 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T21:45:53.672 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T21:45:53.672 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T21:45:53.672 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T21:45:53.673 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T21:45:53.673 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T21:45:53.673 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T21:45:53.673 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T21:45:53.673 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T21:45:53.673 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T21:45:53.673 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-10T21:45:53.673 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-10T21:45:53.673 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-10T21:45:53.673 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T21:45:53.673 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-10T21:45:53.673 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-10T21:45:53.674 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-10T21:45:53.674 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T21:45:53.674 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T21:45:53.674 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T21:45:53.690 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-10T21:45:53.690 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-10T21:45:53.690 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-10T21:45:53.690 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-10T21:45:53.690 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T21:45:53.690 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T21:45:53.690 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T21:45:53.690 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T21:45:53.691 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T21:45:53.692 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T21:45:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-10T21:45:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-10T21:45:53.695 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T21:45:53.695 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T21:45:53.695 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-10T21:45:53.695 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-10T21:45:53.695 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T21:45:53.695 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T21:45:53.695 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T21:45:53.695 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T21:45:53.695 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T21:45:53.696 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-10T21:45:53.696 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T21:45:53.696 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-10T21:45:53.696 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T21:45:53.696 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-10T21:45:53.696 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T21:45:53.696 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T21:45:53.696 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T21:45:53.697 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T21:45:53.697 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T21:45:53.697 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T21:45:53.697 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T21:45:53.697 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T21:45:53.697 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T21:45:53.697 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T21:45:53.697 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T21:45:53.697 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T21:45:53.697 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T21:45:53.697 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T21:45:53.697 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T21:45:53.698 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T21:45:53.699 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T21:45:53.700 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T21:45:53.700 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T21:45:53.700 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T21:45:53.700 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T21:45:53.700 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T21:45:53.700 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T21:45:53.700 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T21:45:53.700 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T21:45:53.700 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T21:45:53.700 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T21:45:53.700 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T21:45:53.700 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T21:45:53.701 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T21:45:53.733 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-10T21:45:53.733 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T21:45:53.734 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T21:45:53.734 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T21:45:53.734 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-10T21:45:53.734 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T21:45:53.734 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-10T21:45:53.734 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T21:45:53.734 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-10T21:45:53.734 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-10T21:45:53.734 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T21:45:53.734 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-10T21:45:53.734 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T21:45:53.734 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T21:45:53.735 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T21:45:53.735 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T21:45:53.735 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T21:45:53.735 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-10T21:45:53.735 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T21:45:53.735 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T21:45:53.735 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T21:45:53.735 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T21:45:53.735 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T21:45:53.735 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T21:45:53.735 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T21:45:53.735 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-10T21:45:53.736 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T21:45:53.736 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-10T21:45:53.736 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T21:45:53.736 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T21:45:53.736 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-10T21:45:53.736 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T21:45:53.736 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-10T21:45:53.736 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-10T21:45:53.736 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-10T21:45:53.736 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T21:45:53.736 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T21:45:53.736 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T21:45:53.737 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T21:45:53.738 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-10T21:45:53.738 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-10T21:45:53.738 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T21:45:53.738 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T21:45:53.738 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-10T21:45:53.738 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-10T21:45:53.738 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-10T21:45:53.738 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T21:45:53.738 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-10T21:45:53.738 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T21:45:53.738 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T21:45:53.738 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T21:45:53.739 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-10T21:45:53.740 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-10T21:45:53.741 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-10T21:45:53.742 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-10T21:45:53.743 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-10T21:45:53.744 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-10T21:45:53.744 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-10T21:45:53.744 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-10T21:45:53.744 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T21:45:53.744 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T21:45:53.744 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T21:45:53.744 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-10T21:45:53.744 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-10T21:45:53.744 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-10T21:45:53.744 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-10T21:45:53.744 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-10T21:45:53.744 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T21:45:53.745 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T21:45:53.746 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-10T21:45:53.746 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T21:45:53.746 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T21:45:53.746 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T21:45:53.746 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T21:45:53.746 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T21:45:53.746 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T21:45:53.748 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T21:45:53.749 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T21:45:53.749 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T21:45:53.749 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T21:45:53.749 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T21:45:53.749 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T21:45:53.749 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T21:45:53.749 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T21:45:53.749 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T21:45:53.749 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T21:45:53.749 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T21:45:53.749 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T21:45:53.749 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T21:45:53.750 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T21:45:53.751 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T21:45:53.751 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T21:45:53.751 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T21:45:53.751 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T21:45:53.751 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T21:45:53.751 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T21:45:53.752 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T21:45:53.752 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T21:45:53.752 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T21:45:53.752 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-10T21:45:53.752 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T21:45:53.752 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-10T21:45:53.752 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-10T21:45:53.752 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T21:45:53.753 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T21:45:53.754 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T21:45:53.755 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T21:45:53.755 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T21:45:53.755 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T21:45:53.755 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T21:45:53.755 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T21:45:53.755 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T21:45:53.755 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-10T21:45:53.755 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-10T21:45:53.755 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-10T21:45:53.755 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T21:45:53.755 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T21:45:53.755 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T21:45:53.756 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T21:45:53.757 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T21:45:53.758 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T21:45:53.758 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T21:45:53.758 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T21:45:53.758 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T21:45:53.758 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T21:45:53.758 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T21:45:53.758 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T21:45:53.758 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T21:45:53.758 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T21:45:53.758 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T21:45:53.758 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T21:45:53.759 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T21:45:53.759 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T21:45:53.759 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T21:45:53.759 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T21:45:53.759 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T21:45:53.759 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T21:45:53.759 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T21:45:53.759 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T21:45:53.759 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T21:45:53.759 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T21:45:53.759 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T21:45:53.759 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T21:45:53.760 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T21:45:53.761 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T21:45:53.762 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T21:45:53.762 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T21:45:53.762 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T21:45:53.762 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T21:45:53.762 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T21:45:53.762 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T21:45:53.763 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T21:45:53.763 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T21:45:53.763 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T21:45:53.763 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T21:45:53.763 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T21:45:53.763 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T21:45:53.763 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T21:45:53.763 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T21:45:53.763 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T21:45:53.763 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T21:45:53.763 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T21:45:53.763 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T21:45:53.764 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T21:45:53.765 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T21:45:53.766 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-10T21:45:53.767 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T21:45:53.767 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T21:45:53.767 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-10T21:45:53.767 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T21:45:53.767 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T21:45:53.767 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T21:45:53.767 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-10T21:45:53.767 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T21:45:53.767 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T21:45:53.767 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-10T21:45:53.767 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-10T21:45:53.767 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T21:45:53.768 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-10T21:45:53.768 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T21:45:53.768 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T21:45:53.768 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T21:45:53.768 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T21:45:53.768 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-10T21:45:53.768 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T21:45:53.768 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T21:45:53.768 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T21:45:53.768 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-10T21:45:53.768 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-10T21:45:53.768 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-10T21:45:53.769 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T21:45:53.770 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-10T21:45:53.770 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T21:45:53.770 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-10T21:45:53.770 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T21:45:53.770 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-10T21:45:53.770 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T21:45:53.795 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-10T21:45:53.795 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-10T21:45:53.795 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-10T21:45:53.795 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-10T21:45:53.795 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-10T21:45:53.795 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-10T21:45:53.795 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T21:45:53.795 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T21:45:53.796 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-10T21:45:53.797 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-10T21:45:53.798 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-10T21:45:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-10T21:45:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-10T21:45:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T21:45:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T21:45:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T21:45:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T21:45:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-10T21:45:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-10T21:45:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T21:45:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T21:45:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T21:45:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T21:45:53.800 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-10T21:45:53.800 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T21:45:53.800 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-10T21:45:53.800 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T21:45:53.800 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T21:45:53.800 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-10T21:45:53.800 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T21:45:53.800 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-10T21:45:53.800 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-10T21:45:53.800 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-10T21:45:53.800 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T21:45:53.800 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-10T21:45:53.801 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-10T21:45:53.802 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T21:45:53.803 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T21:45:53.804 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-10T21:45:53.805 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-10T21:45:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-10T21:45:53.807 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-10T21:45:53.807 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-10T21:45:53.807 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-10T21:45:53.807 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-10T21:45:53.807 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-10T21:45:53.807 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-10T21:45:53.807 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-10T21:45:53.807 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T21:45:53.807 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T21:45:53.807 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-10T21:45:53.807 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-10T21:45:53.807 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T21:45:53.808 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-10T21:45:53.809 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-10T21:45:53.810 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-10T21:45:53.811 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-10T21:45:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T21:45:53.813 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-10T21:45:53.814 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T21:45:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T21:45:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T21:45:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-10T21:45:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-10T21:45:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T21:45:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-10T21:45:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T21:45:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-10T21:45:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T21:45:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-10T21:45:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T21:45:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T21:45:53.816 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-10T21:45:53.817 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-10T21:45:53.817 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T21:45:53.817 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-10T21:45:53.817 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T21:45:53.817 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-10T21:45:53.817 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T21:45:53.817 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-10T21:45:53.817 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T21:45:53.817 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-10T21:45:53.817 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T21:45:53.817 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-10T21:45:53.817 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T21:45:53.818 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T21:45:53.819 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-10T21:45:53.820 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T21:45:53.821 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T21:45:53.822 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-10T21:45:53.822 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-10T21:45:53.822 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T21:45:53.849 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-10T21:45:53.849 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-10T21:45:53.850 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-10T21:45:53.851 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T21:45:53.852 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-10T21:45:53.853 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T21:45:53.854 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T21:45:53.855 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-10T21:45:53.855 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T21:45:53.855 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-10T21:45:53.855 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-10T21:45:53.855 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-10T21:45:53.855 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-10T21:45:53.855 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T21:45:53.855 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-10T21:45:53.855 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T21:45:53.855 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-10T21:45:53.855 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T21:45:53.855 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T21:45:53.856 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T21:45:53.857 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T21:45:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T21:45:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T21:45:53.860 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-10T21:45:53.861 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-10T21:45:53.862 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T21:45:53.862 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-10T21:45:53.862 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T21:45:53.862 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-10T21:45:53.862 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-10T21:45:53.862 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-10T21:45:53.862 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T21:45:53.862 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T21:45:53.862 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-10T21:45:53.862 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-10T21:45:53.863 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T21:45:53.864 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T21:45:53.865 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T21:45:53.865 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T21:45:53.865 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-10T21:45:53.865 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-10T21:45:53.865 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T21:45:53.865 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-10T21:45:53.865 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T21:45:53.865 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T21:45:53.865 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T21:45:53.865 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-10T21:45:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-10T21:45:53.867 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-10T21:45:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T21:45:53.869 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T21:45:53.870 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-10T21:45:53.871 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T21:45:53.872 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T21:45:53.873 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T21:45:53.874 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-10T21:45:53.875 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-10T21:45:53.876 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-10T21:45:53.877 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-10T21:45:53.878 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T21:45:53.908 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-10T21:45:53.908 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T21:45:53.909 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T21:45:53.910 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T21:45:53.911 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T21:45:53.912 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-10T21:45:53.913 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-10T21:45:53.914 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-10T21:45:53.915 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-10T21:45:53.916 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-10T21:45:53.917 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-10T21:45:53.918 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-10T21:45:53.919 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-10T21:45:53.920 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-10T21:45:53.921 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-10T21:45:53.922 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-10T21:45:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-10T21:45:53.924 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T21:45:53.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T21:45:53.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T21:45:53.927 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-10T21:45:53.928 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T21:45:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T21:45:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T21:45:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T21:45:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T21:45:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-10T21:45:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T21:45:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T21:45:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T21:45:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-10T21:45:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-10T21:45:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T21:45:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T21:45:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T21:45:53.931 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T21:45:53.931 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-10T21:45:53.931 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T21:45:53.931 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T21:45:53.931 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T21:45:53.931 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T21:45:53.931 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-10T21:45:53.931 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-10T21:45:53.931 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-10T21:45:53.931 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-10T21:45:53.931 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-10T21:45:53.932 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-10T21:45:53.932 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-10T21:45:53.932 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T21:45:53.932 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-10T21:45:53.932 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-10T21:45:53.932 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-10T21:45:53.932 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-10T21:45:53.932 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-10T21:45:53.932 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-10T21:45:53.932 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T21:45:53.932 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-10T21:45:53.932 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-10T21:45:53.933 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-10T21:45:53.933 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-10T21:45:53.933 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-10T21:45:53.933 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-10T21:45:53.933 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T21:45:53.933 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-10T21:45:53.933 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-10T21:45:53.933 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-10T21:45:53.933 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T21:45:53.933 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T21:45:53.933 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-10T21:45:53.934 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-10T21:45:53.934 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-10T21:45:53.934 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T21:45:53.934 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T21:45:53.934 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T21:45:53.934 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T21:45:53.934 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T21:45:53.934 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T21:45:53.934 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T21:45:53.934 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T21:45:53.934 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T21:45:53.934 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T21:45:53.935 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T21:45:53.935 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T21:45:53.935 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-10T21:45:53.935 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-10T21:45:53.935 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-10T21:45:53.935 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T21:45:53.935 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T21:45:53.935 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-10T21:45:53.935 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-10T21:45:53.935 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-10T21:45:53.935 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-10T21:45:53.935 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-10T21:45:53.936 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-10T21:45:53.936 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T21:45:53.936 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T21:45:53.936 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T21:45:53.936 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T21:45:53.936 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T21:45:53.936 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T21:45:53.936 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T21:45:53.936 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T21:45:53.936 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T21:45:53.936 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T21:45:53.936 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T21:45:53.937 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-10T21:45:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-10T21:45:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T21:45:53.940 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-10T21:45:53.940 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-10T21:45:53.940 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-10T21:45:53.940 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T21:45:53.940 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-10T21:45:53.940 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-10T21:45:53.940 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-10T21:45:53.940 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-10T21:45:53.940 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-10T21:45:53.940 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T21:45:53.940 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-10T21:45:53.940 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-10T21:45:53.941 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T21:45:53.942 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-10T21:45:53.942 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T21:45:53.950 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-10T21:45:53.950 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-10T21:45:53.950 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-10T21:45:53.950 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T21:45:53.950 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-10T21:45:53.950 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T21:45:53.950 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-10T21:45:53.950 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-10T21:45:53.950 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-10T21:45:53.951 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T21:45:53.952 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T21:45:53.953 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T21:45:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T21:45:53.955 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T21:45:53.956 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-10T21:45:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-10T21:45:53.958 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T21:45:53.959 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-10T21:45:53.960 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-10T21:45:53.960 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T21:45:53.960 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-10T21:45:53.960 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T21:45:53.960 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T21:45:53.960 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-10T21:45:53.960 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T21:45:53.960 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-10T21:45:53.960 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T21:45:53.960 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-10T21:45:53.960 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T21:45:53.963 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-10T21:45:53.963 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T21:45:53.963 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-10T21:45:53.963 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T21:45:53.963 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-10T21:45:53.963 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-10T21:45:53.963 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-10T21:45:53.963 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-10T21:45:53.963 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-10T21:45:53.963 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T21:45:53.963 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-10T21:45:53.963 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-10T21:45:53.964 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T21:45:53.965 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-10T21:45:53.965 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-10T21:45:53.965 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-10T21:45:53.965 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-10T21:45:53.965 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-10T21:45:53.965 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T21:45:53.965 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-10T21:45:53.965 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-10T21:45:53.965 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T21:45:53.965 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-10T21:45:53.965 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-10T21:45:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-10T21:45:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-10T21:45:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T21:45:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T21:45:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T21:45:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-10T21:45:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-10T21:45:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-10T21:45:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T21:45:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-10T21:45:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-10T21:45:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T21:45:53.969 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T21:45:53.970 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-10T21:45:53.970 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T21:45:53.970 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-10T21:45:53.970 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T21:45:53.970 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T21:45:53.970 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T21:45:53.970 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T21:45:53.970 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-10T21:45:53.970 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T21:45:53.970 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-10T21:45:53.970 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T21:45:53.970 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T21:45:53.973 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T21:45:53.974 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-10T21:45:53.977 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-10T21:45:53.977 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-10T21:45:53.977 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T21:45:53.977 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T21:45:53.999 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T21:45:54.000 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-10T21:45:54.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T21:45:54.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T21:45:54.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T21:45:54.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T21:45:54.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T21:45:54.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T21:45:54.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T21:45:54.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T21:45:54.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T21:45:54.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T21:45:54.000 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T21:45:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T21:45:54.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T21:45:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T21:45:54.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T21:45:54.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T21:45:54.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T21:45:54.007 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-10T21:45:54.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-10T21:45:54.008 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-10T21:45:54.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T21:45:54.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-10T21:45:54.008 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-10T21:45:54.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T21:45:54.008 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-10T21:45:54.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T21:45:54.008 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-10T21:45:54.008 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-10T21:45:54.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T21:45:54.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T21:45:54.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T21:45:54.010 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-10T21:45:54.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T21:45:54.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T21:45:54.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T21:45:54.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T21:45:54.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T21:45:54.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T21:45:54.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T21:45:54.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T21:45:54.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T21:45:54.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T21:45:54.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T21:45:54.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T21:45:54.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T21:45:54.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T21:45:54.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T21:45:54.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T21:45:54.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T21:45:54.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-10T21:45:54.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T21:45:54.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T21:45:54.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T21:45:54.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T21:45:54.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T21:45:54.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T21:45:54.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T21:45:54.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T21:45:54.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T21:45:54.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-10T21:45:54.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-10T21:45:54.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-10T21:45:54.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T21:45:54.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T21:45:54.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T21:45:54.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T21:45:54.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T21:45:54.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T21:45:54.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T21:45:54.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T21:45:54.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T21:45:54.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T21:45:54.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T21:45:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T21:45:54.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T21:45:54.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T21:45:54.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T21:45:54.021 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-10T21:45:54.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T21:45:54.021 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-10T21:45:54.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-10T21:45:54.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-10T21:45:54.021 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-10T21:45:54.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T21:45:54.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T21:45:54.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T21:45:54.021 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T21:45:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T21:45:54.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T21:45:54.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T21:45:54.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T21:45:54.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T21:45:54.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T21:45:54.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T21:45:54.025 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-10T21:45:54.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T21:45:54.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T21:45:54.028 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-10T21:45:54.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T21:45:54.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-10T21:45:54.031 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:45:54.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-10T21:45:54.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-10T21:45:54.036 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:54.036 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:54.036 DEBUG:teuthology.orchestra.run.smithi066:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-10T21:45:54.076 INFO:teuthology.orchestra.run.smithi066.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:45:54.077 INFO:teuthology.orchestra.run.smithi066.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:45:54.077 INFO:teuthology.orchestra.run.smithi066.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:45:54.077 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-10T21:45:54.077 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:54.077 DEBUG:teuthology.orchestra.run.smithi066:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-10T21:45:54.121 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi066.front.sepia.ceph.com to complete... 2024-09-10T21:45:54.479 INFO:teuthology.orchestra.run.smithi066.stderr:Sourcing file `/etc/default/grub' 2024-09-10T21:45:54.480 INFO:teuthology.orchestra.run.smithi066.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T21:45:54.491 INFO:teuthology.orchestra.run.smithi066.stderr:Generating grub configuration file ... 2024-09-10T21:45:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:54.703 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:45:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T21:45:54.931 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T21:45:54.975 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T21:45:54.981 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T21:45:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T21:45:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T21:45:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T21:45:55.142 INFO:teuthology.orchestra.run.smithi066.stderr:done 2024-09-10T21:45:56.288 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-09-10T21:46:22.749 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T21:46:22.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T21:46:22.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:46:26.293 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T21:46:26.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T21:46:26.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:47:00.797 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-10T21:47:09.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T21:47:09.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:47:12.893 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-10T21:47:22.764 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T21:47:24.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T21:47:24.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:47:31.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T21:47:31.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:47:31.784 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-09-10T21:47:43.297 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-10T21:47:43.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T21:47:43.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:47:44.024 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-09-10T21:47:44.666 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T21:47:44.666 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T21:47:44.667 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-09-10T21:47:44.720 INFO:teuthology.orchestra.run.smithi094.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:47:44.720 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:47:44.721 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T21:47:44.721 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T21:47:45.721 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T21:47:45.722 DEBUG:teuthology.orchestra.run.smithi094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T21:47:45.798 INFO:teuthology.orchestra.run.smithi094.stdout:ttyS1 2024-09-10T21:47:45.856 DEBUG:teuthology.parallel:result is None 2024-09-10T21:47:58.298 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T21:47:58.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:47:58.327 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-10T21:48:16.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T21:48:16.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:48:16.590 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-09-10T21:48:17.302 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T21:48:17.303 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T21:48:17.303 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-09-10T21:48:17.353 INFO:teuthology.orchestra.run.smithi066.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:48:17.353 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:48:17.354 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T21:48:17.354 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T21:48:18.355 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T21:48:18.355 DEBUG:teuthology.orchestra.run.smithi066:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T21:48:18.453 INFO:teuthology.orchestra.run.smithi066.stdout:ttyS1 2024-09-10T21:48:18.507 DEBUG:teuthology.parallel:result is None 2024-09-10T21:48:18.508 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T21:48:18.515 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T21:48:18.515 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T21:48:18.519 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T21:48:18.528 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T21:48:18.534 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T21:48:18.541 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T21:48:18.541 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T21:48:18.572 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T21:48:18.599 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T21:48:18.607 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T21:48:18.607 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T21:48:18.641 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T21:48:18.668 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:48:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:48:18.682 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:48:18.683 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:48:18.686 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T21:48:18.693 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T21:48:18.693 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T21:48:18.733 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T21:48:18.754 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T21:48:18.763 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T21:48:18.763 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T21:48:18.796 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T21:48:18.804 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:48:18.854 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:48:18.906 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:48:18.907 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T21:48:18.985 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:48:18.994 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:48:19.046 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:48:19.046 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T21:48:19.115 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-09-10T21:48:19.118 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-09-10T21:48:19.243 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T21:48:19.251 INFO:teuthology.task.internal:Starting timer... 2024-09-10T21:48:19.251 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T21:48:19.261 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T21:48:19.268 DEBUG:teuthology.task.selinux:Excluding smithi066: OS 'ubuntu' does not support SELinux 2024-09-10T21:48:19.268 DEBUG:teuthology.task.selinux:Excluding smithi094: OS 'ubuntu' does not support SELinux 2024-09-10T21:48:19.268 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T21:48:19.268 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-10T21:48:19.268 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T21:48:19.269 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T21:48:19.276 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T21:48:19.279 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T21:48:19.424 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T21:48:19.434 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T21:48:19.435 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi066.front.sepia.ceph.com,smithi094.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T21:56:32.621 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi094.front.sepia.ceph.com')] 2024-09-10T21:56:32.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T21:56:32.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:56:32.719 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-09-10T21:56:32.848 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T21:56:32.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T21:56:32.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:56:32.937 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-09-10T21:56:33.047 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T21:56:33.048 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T21:56:33.058 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T21:56:33.058 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:56:33.059 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:56:33.062 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:56:33.062 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:56:33.098 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-10T21:56:33.098 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: Command line: ntpd -gq 2024-09-10T21:56:33.099 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: ---------------------------------------------------- 2024-09-10T21:56:33.099 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: ntp-4 is maintained by Network Time Foundation, 2024-09-10T21:56:33.099 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-10T21:56:33.099 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: corporation. Support and training for ntp-4 are 2024-09-10T21:56:33.099 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: available at https://www.nwtime.org/support 2024-09-10T21:56:33.099 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: ---------------------------------------------------- 2024-09-10T21:56:33.101 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: proto: precision = 0.045 usec (-24) 2024-09-10T21:56:33.101 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: basedate set to 2022-02-04 2024-09-10T21:56:33.101 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: gps base set to 2022-02-06 (week 2196) 2024-09-10T21:56:33.101 INFO:teuthology.orchestra.run.smithi066.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:56:33.102 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:56:33.102 INFO:teuthology.orchestra.run.smithi066.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:56:33.104 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:56:33.104 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T21:56:33.105 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T21:56:33.105 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T21:56:33.105 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: Listen normally on 3 enp3s0f1 172.21.15.66:123 2024-09-10T21:56:33.105 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: Listen normally on 4 lo [::1]:123 2024-09-10T21:56:33.105 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:f8bd%5]:123 2024-09-10T21:56:33.106 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:33 ntpd[18448]: Listening on routing socket on fd #22 for interface updates 2024-09-10T21:56:33.127 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-10T21:56:33.127 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: Command line: ntpd -gq 2024-09-10T21:56:33.127 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: ---------------------------------------------------- 2024-09-10T21:56:33.127 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: ntp-4 is maintained by Network Time Foundation, 2024-09-10T21:56:33.127 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-10T21:56:33.127 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: corporation. Support and training for ntp-4 are 2024-09-10T21:56:33.127 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: available at https://www.nwtime.org/support 2024-09-10T21:56:33.127 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: ---------------------------------------------------- 2024-09-10T21:56:33.128 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: proto: precision = 0.041 usec (-24) 2024-09-10T21:56:33.128 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: basedate set to 2022-02-04 2024-09-10T21:56:33.128 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: gps base set to 2022-02-06 (week 2196) 2024-09-10T21:56:33.129 INFO:teuthology.orchestra.run.smithi094.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:56:33.129 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:56:33.130 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:56:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:56:33.131 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T21:56:33.131 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T21:56:33.131 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T21:56:33.131 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: Listen normally on 3 enp3s0f1 172.21.15.94:123 2024-09-10T21:56:33.131 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: Listen normally on 4 lo [::1]:123 2024-09-10T21:56:33.131 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d763%5]:123 2024-09-10T21:56:33.131 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:33 ntpd[18482]: Listening on routing socket on fd #22 for interface updates 2024-09-10T21:56:34.098 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:34 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:34.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:34 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:35.127 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:35 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:35.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:35 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:35.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:35 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:35.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:35 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:35.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:35 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:35.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:35 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:35.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:35 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:35.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:35 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:36.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:36 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:36.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:36 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:36.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:36 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:36.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:36 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:36.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:36 ntpd[18448]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:36.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:36 ntpd[18448]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:37.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:37 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:37.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:37 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:37.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:37 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:37.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:37 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:37.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:37 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:37.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:37 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:37.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:37 ntpd[18448]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:37.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:37 ntpd[18448]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:37.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:37 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:37.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:37 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:37.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:37 ntpd[18448]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:37.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:37 ntpd[18448]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:37.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:37 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:37.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:37 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:37.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:37 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:37.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:37 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:37.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:37 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:37.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:37 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:37.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:37 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:37.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:37 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:37.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:37 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:37.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:37 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:37.129 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:37 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:37.129 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:37 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:38 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:38 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:38 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:38 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:38 ntpd[18448]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:38 ntpd[18448]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:38.127 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:38 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:38.127 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:38 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:38.127 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:38 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:38.127 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:38 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:38.127 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:38 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:38.127 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:38 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:39.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:39 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:39.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:39 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:39.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:39 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:39.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:39 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:39.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:39 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:39.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:39 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:39.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:39 ntpd[18448]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:39.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:39 ntpd[18448]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:39.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:39 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:39.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:39 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:39.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:39 ntpd[18448]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:39.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:39 ntpd[18448]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:39.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:39 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:39.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:39 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:39.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:39 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:39.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:39 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:39.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:39 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:39.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:39 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:39.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:39 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:39.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:39 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:39.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:39 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:39.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:39 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:39.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:39 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:39.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:39 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:40.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:40 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:40.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:40 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:40.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:40 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:40.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:40 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:40.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:40 ntpd[18448]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:40.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:40 ntpd[18448]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:40.127 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:40 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:40.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:40 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:40.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:40 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:40.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:40 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:40.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:40 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:40.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:40 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:41 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:41 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:41 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:41 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:41 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:41 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:41 ntpd[18448]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:41 ntpd[18448]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:41 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:41 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:41 ntpd[18448]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:41 ntpd[18448]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:41.129 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:41.129 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:41.129 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 21:56:41 ntpd[18482]: ntpd: time slew -0.000442 s 2024-09-10T21:56:41.129 INFO:teuthology.orchestra.run.smithi094.stdout:ntpd: time slew -0.000442s 2024-09-10T21:56:41.129 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T21:56:41.129 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 21:56:41 ntpd[18482]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T21:56:41.231 INFO:teuthology.orchestra.run.smithi094.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T21:56:41.232 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================== 2024-09-10T21:56:41.232 INFO:teuthology.orchestra.run.smithi094.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:56:41.232 INFO:teuthology.orchestra.run.smithi094.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:56:41.232 INFO:teuthology.orchestra.run.smithi094.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:56:41.232 INFO:teuthology.orchestra.run.smithi094.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:56:42.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:42 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:42.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:42 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:42.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:42 ntpd[18448]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:56:42.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:42 ntpd[18448]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:56:42.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:42 ntpd[18448]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:56:42.099 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:42 ntpd[18448]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:56:42.100 INFO:teuthology.orchestra.run.smithi066.stdout:10 Sep 21:56:42 ntpd[18448]: ntpd: time slew -0.001891 s 2024-09-10T21:56:42.100 INFO:teuthology.orchestra.run.smithi066.stdout:ntpd: time slew -0.001891s 2024-09-10T21:56:42.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:42 ntpd[18448]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T21:56:42.100 INFO:teuthology.orchestra.run.smithi066.stderr:10 Sep 21:56:42 ntpd[18448]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T21:56:42.197 INFO:teuthology.orchestra.run.smithi066.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T21:56:42.197 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================== 2024-09-10T21:56:42.197 INFO:teuthology.orchestra.run.smithi066.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:56:42.197 INFO:teuthology.orchestra.run.smithi066.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:56:42.197 INFO:teuthology.orchestra.run.smithi066.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:56:42.197 INFO:teuthology.orchestra.run.smithi066.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:56:42.198 INFO:teuthology.run_tasks:Running task install... 2024-09-10T21:56:42.206 DEBUG:teuthology.task.install:project ceph 2024-09-10T21:56:42.206 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T21:56:42.207 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T21:56:42.207 INFO:teuthology.task.install:Using flavor: default 2024-09-10T21:56:42.212 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T21:56:42.212 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T21:56:42.213 DEBUG:teuthology.orchestra.run.smithi066:> sudo apt-key list | grep Ceph 2024-09-10T21:56:42.213 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-key list | grep Ceph 2024-09-10T21:56:42.317 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-10T21:56:42.350 INFO:teuthology.orchestra.run.smithi066.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-10T21:56:42.368 INFO:teuthology.orchestra.run.smithi094.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T21:56:42.368 INFO:teuthology.orchestra.run.smithi094.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T21:56:42.369 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-10T21:56:42.369 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-10T21:56:42.370 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:56:42.402 INFO:teuthology.orchestra.run.smithi066.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T21:56:42.402 INFO:teuthology.orchestra.run.smithi066.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T21:56:42.412 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-10T21:56:42.412 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-10T21:56:42.412 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:56:42.532 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default/ 2024-09-10T21:56:42.532 INFO:teuthology.task.install.deb:Package version is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:56:42.560 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default/ 2024-09-10T21:56:42.560 INFO:teuthology.task.install.deb:Package version is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:56:42.659 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:56:42.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T21:56:42.674 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:56:42.675 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T21:56:42.678 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get update 2024-09-10T21:56:42.694 DEBUG:teuthology.orchestra.run.smithi066:> sudo apt-get update 2024-09-10T21:56:42.912 INFO:teuthology.orchestra.run.smithi094.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy InRelease 2024-09-10T21:56:42.933 INFO:teuthology.orchestra.run.smithi066.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy InRelease 2024-09-10T21:56:42.934 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-10T21:56:42.954 INFO:teuthology.orchestra.run.smithi066.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-10T21:56:42.955 INFO:teuthology.orchestra.run.smithi094.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-10T21:56:42.976 INFO:teuthology.orchestra.run.smithi066.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-10T21:56:42.977 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-10T21:56:42.997 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-10T21:56:42.997 INFO:teuthology.orchestra.run.smithi066.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-10T21:56:42.999 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-10T21:56:43.019 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-10T21:56:43.019 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-10T21:56:43.019 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-10T21:56:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-10T21:56:43.060 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-10T21:56:43.084 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-10T21:56:43.142 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-10T21:56:43.166 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-10T21:56:43.594 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 29.3 kB in 1s (42.5 kB/s) 2024-09-10T21:56:43.637 INFO:teuthology.orchestra.run.smithi066.stdout:Fetched 29.3 kB in 1s (41.5 kB/s) 2024-09-10T21:56:44.621 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-09-10T21:56:44.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4854-g8f219c47-1jammy cephadm=19.3.0-4854-g8f219c47-1jammy ceph-mds=19.3.0-4854-g8f219c47-1jammy ceph-mgr=19.3.0-4854-g8f219c47-1jammy ceph-common=19.3.0-4854-g8f219c47-1jammy ceph-fuse=19.3.0-4854-g8f219c47-1jammy ceph-test=19.3.0-4854-g8f219c47-1jammy ceph-volume=19.3.0-4854-g8f219c47-1jammy radosgw=19.3.0-4854-g8f219c47-1jammy python3-rados=19.3.0-4854-g8f219c47-1jammy python3-rgw=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy python3-rbd=19.3.0-4854-g8f219c47-1jammy libcephfs2=19.3.0-4854-g8f219c47-1jammy libcephfs-dev=19.3.0-4854-g8f219c47-1jammy librados2=19.3.0-4854-g8f219c47-1jammy librbd1=19.3.0-4854-g8f219c47-1jammy rbd-fuse=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy cephfs-shell=19.3.0-4854-g8f219c47-1jammy cephfs-top=19.3.0-4854-g8f219c47-1jammy cephfs-mirror=19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:56:44.704 INFO:teuthology.orchestra.run.smithi066.stdout:Reading package lists... 2024-09-10T21:56:44.711 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-09-10T21:56:44.720 DEBUG:teuthology.orchestra.run.smithi066:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4854-g8f219c47-1jammy cephadm=19.3.0-4854-g8f219c47-1jammy ceph-mds=19.3.0-4854-g8f219c47-1jammy ceph-mgr=19.3.0-4854-g8f219c47-1jammy ceph-common=19.3.0-4854-g8f219c47-1jammy ceph-fuse=19.3.0-4854-g8f219c47-1jammy ceph-test=19.3.0-4854-g8f219c47-1jammy ceph-volume=19.3.0-4854-g8f219c47-1jammy radosgw=19.3.0-4854-g8f219c47-1jammy python3-rados=19.3.0-4854-g8f219c47-1jammy python3-rgw=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy python3-rbd=19.3.0-4854-g8f219c47-1jammy libcephfs2=19.3.0-4854-g8f219c47-1jammy libcephfs-dev=19.3.0-4854-g8f219c47-1jammy librados2=19.3.0-4854-g8f219c47-1jammy librbd1=19.3.0-4854-g8f219c47-1jammy rbd-fuse=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy cephfs-shell=19.3.0-4854-g8f219c47-1jammy cephfs-top=19.3.0-4854-g8f219c47-1jammy cephfs-mirror=19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:56:44.799 INFO:teuthology.orchestra.run.smithi066.stdout:Reading package lists... 2024-09-10T21:56:44.892 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2024-09-10T21:56:44.893 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2024-09-10T21:56:44.992 INFO:teuthology.orchestra.run.smithi066.stdout:Building dependency tree... 2024-09-10T21:56:44.992 INFO:teuthology.orchestra.run.smithi066.stdout:Reading state information... 2024-09-10T21:56:45.077 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:56:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:56:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:56:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:56:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:56:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:56:45.078 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:56:45.079 INFO:teuthology.orchestra.run.smithi094.stdout:The following additional packages will be installed: 2024-09-10T21:56:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T21:56:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-10T21:56:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:56:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-10T21:56:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-10T21:56:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-10T21:56:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:56:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:56:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-10T21:56:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-10T21:56:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-10T21:56:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-10T21:56:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-10T21:56:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-10T21:56:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T21:56:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-10T21:56:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-10T21:56:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-10T21:56:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-10T21:56:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-10T21:56:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-10T21:56:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-10T21:56:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-10T21:56:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-10T21:56:45.083 INFO:teuthology.orchestra.run.smithi094.stdout:Suggested packages: 2024-09-10T21:56:45.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-10T21:56:45.083 INFO:teuthology.orchestra.run.smithi094.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-10T21:56:45.083 INFO:teuthology.orchestra.run.smithi094.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-10T21:56:45.083 INFO:teuthology.orchestra.run.smithi094.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-10T21:56:45.083 INFO:teuthology.orchestra.run.smithi094.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-10T21:56:45.083 INFO:teuthology.orchestra.run.smithi094.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T21:56:45.083 INFO:teuthology.orchestra.run.smithi094.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-10T21:56:45.083 INFO:teuthology.orchestra.run.smithi094.stdout:Recommended packages: 2024-09-10T21:56:45.083 INFO:teuthology.orchestra.run.smithi094.stdout: btrfs-tools 2024-09-10T21:56:45.187 INFO:teuthology.orchestra.run.smithi066.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:56:45.188 INFO:teuthology.orchestra.run.smithi066.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:56:45.188 INFO:teuthology.orchestra.run.smithi066.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:56:45.188 INFO:teuthology.orchestra.run.smithi066.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:56:45.188 INFO:teuthology.orchestra.run.smithi066.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:56:45.189 INFO:teuthology.orchestra.run.smithi066.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:56:45.189 INFO:teuthology.orchestra.run.smithi066.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:56:45.189 INFO:teuthology.orchestra.run.smithi066.stdout:The following additional packages will be installed: 2024-09-10T21:56:45.189 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T21:56:45.190 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-10T21:56:45.190 INFO:teuthology.orchestra.run.smithi066.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:56:45.190 INFO:teuthology.orchestra.run.smithi066.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-10T21:56:45.190 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-10T21:56:45.191 INFO:teuthology.orchestra.run.smithi066.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-10T21:56:45.191 INFO:teuthology.orchestra.run.smithi066.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:56:45.191 INFO:teuthology.orchestra.run.smithi066.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:56:45.191 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-10T21:56:45.191 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-10T21:56:45.191 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-10T21:56:45.191 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-10T21:56:45.191 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-10T21:56:45.192 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-10T21:56:45.192 INFO:teuthology.orchestra.run.smithi066.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T21:56:45.192 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-10T21:56:45.192 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-10T21:56:45.192 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-10T21:56:45.192 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-10T21:56:45.192 INFO:teuthology.orchestra.run.smithi066.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-10T21:56:45.192 INFO:teuthology.orchestra.run.smithi066.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-10T21:56:45.192 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-10T21:56:45.192 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-10T21:56:45.192 INFO:teuthology.orchestra.run.smithi066.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-10T21:56:45.193 INFO:teuthology.orchestra.run.smithi066.stdout:Suggested packages: 2024-09-10T21:56:45.193 INFO:teuthology.orchestra.run.smithi066.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-10T21:56:45.193 INFO:teuthology.orchestra.run.smithi066.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-10T21:56:45.193 INFO:teuthology.orchestra.run.smithi066.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-10T21:56:45.193 INFO:teuthology.orchestra.run.smithi066.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-10T21:56:45.193 INFO:teuthology.orchestra.run.smithi066.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-10T21:56:45.193 INFO:teuthology.orchestra.run.smithi066.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T21:56:45.193 INFO:teuthology.orchestra.run.smithi066.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-10T21:56:45.193 INFO:teuthology.orchestra.run.smithi066.stdout:Recommended packages: 2024-09-10T21:56:45.193 INFO:teuthology.orchestra.run.smithi066.stdout: btrfs-tools 2024-09-10T21:56:45.236 INFO:teuthology.orchestra.run.smithi094.stdout:The following NEW packages will be installed: 2024-09-10T21:56:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T21:56:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T21:56:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T21:56:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-10T21:56:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:56:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-10T21:56:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-10T21:56:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-10T21:56:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-10T21:56:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T21:56:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-10T21:56:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-10T21:56:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-10T21:56:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-10T21:56:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: socat unzip xmlstarlet zip 2024-09-10T21:56:45.307 INFO:teuthology.orchestra.run.smithi094.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-10T21:56:45.307 INFO:teuthology.orchestra.run.smithi094.stdout:Need to get 251 MB of archives. 2024-09-10T21:56:45.307 INFO:teuthology.orchestra.run.smithi094.stdout:After this operation, 936 MB of additional disk space will be used. 2024-09-10T21:56:45.307 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-10T21:56:45.324 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-10T21:56:45.334 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-10T21:56:45.355 INFO:teuthology.orchestra.run.smithi066.stdout:The following NEW packages will be installed: 2024-09-10T21:56:45.356 INFO:teuthology.orchestra.run.smithi066.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T21:56:45.356 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T21:56:45.356 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T21:56:45.356 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-10T21:56:45.356 INFO:teuthology.orchestra.run.smithi066.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:56:45.356 INFO:teuthology.orchestra.run.smithi066.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-10T21:56:45.356 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-10T21:56:45.357 INFO:teuthology.orchestra.run.smithi066.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-10T21:56:45.357 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:56:45.357 INFO:teuthology.orchestra.run.smithi066.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:56:45.357 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-10T21:56:45.358 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-10T21:56:45.359 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-10T21:56:45.359 INFO:teuthology.orchestra.run.smithi066.stdout: socat unzip xmlstarlet zip 2024-09-10T21:56:45.376 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-10T21:56:45.389 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4854-g8f219c47-1jammy [3,633 kB] 2024-09-10T21:56:45.397 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-10T21:56:45.399 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-10T21:56:45.410 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-10T21:56:45.426 INFO:teuthology.orchestra.run.smithi066.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-10T21:56:45.426 INFO:teuthology.orchestra.run.smithi066.stdout:Need to get 251 MB of archives. 2024-09-10T21:56:45.426 INFO:teuthology.orchestra.run.smithi066.stdout:After this operation, 936 MB of additional disk space will be used. 2024-09-10T21:56:45.426 INFO:teuthology.orchestra.run.smithi066.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-10T21:56:45.444 INFO:teuthology.orchestra.run.smithi066.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-10T21:56:45.448 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-10T21:56:45.451 INFO:teuthology.orchestra.run.smithi094.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-10T21:56:45.454 INFO:teuthology.orchestra.run.smithi066.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-10T21:56:45.457 INFO:teuthology.orchestra.run.smithi094.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-10T21:56:45.460 INFO:teuthology.orchestra.run.smithi094.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-10T21:56:45.461 INFO:teuthology.orchestra.run.smithi094.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-10T21:56:45.462 INFO:teuthology.orchestra.run.smithi094.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-10T21:56:45.468 INFO:teuthology.orchestra.run.smithi094.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-10T21:56:45.469 INFO:teuthology.orchestra.run.smithi094.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-10T21:56:45.471 INFO:teuthology.orchestra.run.smithi094.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-10T21:56:45.472 INFO:teuthology.orchestra.run.smithi094.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-10T21:56:45.485 INFO:teuthology.orchestra.run.smithi094.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-10T21:56:45.486 INFO:teuthology.orchestra.run.smithi094.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-10T21:56:45.488 INFO:teuthology.orchestra.run.smithi094.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-10T21:56:45.490 INFO:teuthology.orchestra.run.smithi094.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-10T21:56:45.490 INFO:teuthology.orchestra.run.smithi094.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-10T21:56:45.496 INFO:teuthology.orchestra.run.smithi066.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-10T21:56:45.501 INFO:teuthology.orchestra.run.smithi094.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-10T21:56:45.503 INFO:teuthology.orchestra.run.smithi094.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-10T21:56:45.503 INFO:teuthology.orchestra.run.smithi094.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-10T21:56:45.503 INFO:teuthology.orchestra.run.smithi094.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-10T21:56:45.503 INFO:teuthology.orchestra.run.smithi094.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-10T21:56:45.508 INFO:teuthology.orchestra.run.smithi066.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4854-g8f219c47-1jammy [3,633 kB] 2024-09-10T21:56:45.517 INFO:teuthology.orchestra.run.smithi066.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-10T21:56:45.518 INFO:teuthology.orchestra.run.smithi094.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-10T21:56:45.519 INFO:teuthology.orchestra.run.smithi094.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-10T21:56:45.519 INFO:teuthology.orchestra.run.smithi094.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-10T21:56:45.520 INFO:teuthology.orchestra.run.smithi066.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-10T21:56:45.521 INFO:teuthology.orchestra.run.smithi094.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-10T21:56:45.522 INFO:teuthology.orchestra.run.smithi094.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-10T21:56:45.531 INFO:teuthology.orchestra.run.smithi066.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-10T21:56:45.535 INFO:teuthology.orchestra.run.smithi094.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-10T21:56:45.536 INFO:teuthology.orchestra.run.smithi094.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-10T21:56:45.537 INFO:teuthology.orchestra.run.smithi094.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T21:56:45.537 INFO:teuthology.orchestra.run.smithi094.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-10T21:56:45.537 INFO:teuthology.orchestra.run.smithi094.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-10T21:56:45.553 INFO:teuthology.orchestra.run.smithi094.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-10T21:56:45.553 INFO:teuthology.orchestra.run.smithi094.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-10T21:56:45.553 INFO:teuthology.orchestra.run.smithi094.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-10T21:56:45.558 INFO:teuthology.orchestra.run.smithi094.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-10T21:56:45.558 INFO:teuthology.orchestra.run.smithi094.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-10T21:56:45.564 INFO:teuthology.orchestra.run.smithi094.stdout:Get:44 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4854-g8f219c47-1jammy [3,330 kB] 2024-09-10T21:56:45.569 INFO:teuthology.orchestra.run.smithi066.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-10T21:56:45.570 INFO:teuthology.orchestra.run.smithi094.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-10T21:56:45.570 INFO:teuthology.orchestra.run.smithi094.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-10T21:56:45.571 INFO:teuthology.orchestra.run.smithi094.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-10T21:56:45.571 INFO:teuthology.orchestra.run.smithi066.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-10T21:56:45.573 INFO:teuthology.orchestra.run.smithi094.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-10T21:56:45.573 INFO:teuthology.orchestra.run.smithi094.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-10T21:56:45.578 INFO:teuthology.orchestra.run.smithi066.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-10T21:56:45.581 INFO:teuthology.orchestra.run.smithi066.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-10T21:56:45.582 INFO:teuthology.orchestra.run.smithi066.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-10T21:56:45.583 INFO:teuthology.orchestra.run.smithi066.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-10T21:56:45.586 INFO:teuthology.orchestra.run.smithi094.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-10T21:56:45.587 INFO:teuthology.orchestra.run.smithi094.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-10T21:56:45.589 INFO:teuthology.orchestra.run.smithi066.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-10T21:56:45.589 INFO:teuthology.orchestra.run.smithi094.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-10T21:56:45.590 INFO:teuthology.orchestra.run.smithi094.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-10T21:56:45.590 INFO:teuthology.orchestra.run.smithi094.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-10T21:56:45.591 INFO:teuthology.orchestra.run.smithi066.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-10T21:56:45.592 INFO:teuthology.orchestra.run.smithi066.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-10T21:56:45.593 INFO:teuthology.orchestra.run.smithi066.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-10T21:56:45.603 INFO:teuthology.orchestra.run.smithi094.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-10T21:56:45.605 INFO:teuthology.orchestra.run.smithi066.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-10T21:56:45.607 INFO:teuthology.orchestra.run.smithi066.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-10T21:56:45.609 INFO:teuthology.orchestra.run.smithi066.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-10T21:56:45.611 INFO:teuthology.orchestra.run.smithi066.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-10T21:56:45.612 INFO:teuthology.orchestra.run.smithi066.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-10T21:56:45.618 INFO:teuthology.orchestra.run.smithi094.stdout:Get:56 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4854-g8f219c47-1jammy [721 kB] 2024-09-10T21:56:45.623 INFO:teuthology.orchestra.run.smithi066.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-10T21:56:45.624 INFO:teuthology.orchestra.run.smithi066.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-10T21:56:45.624 INFO:teuthology.orchestra.run.smithi066.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-10T21:56:45.625 INFO:teuthology.orchestra.run.smithi066.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-10T21:56:45.625 INFO:teuthology.orchestra.run.smithi066.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-10T21:56:45.629 INFO:teuthology.orchestra.run.smithi094.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4854-g8f219c47-1jammy [356 kB] 2024-09-10T21:56:45.635 INFO:teuthology.orchestra.run.smithi094.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4854-g8f219c47-1jammy [32.8 kB] 2024-09-10T21:56:45.635 INFO:teuthology.orchestra.run.smithi094.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4854-g8f219c47-1jammy [180 kB] 2024-09-10T21:56:45.640 INFO:teuthology.orchestra.run.smithi066.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-10T21:56:45.641 INFO:teuthology.orchestra.run.smithi066.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-10T21:56:45.641 INFO:teuthology.orchestra.run.smithi066.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-10T21:56:45.643 INFO:teuthology.orchestra.run.smithi066.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-10T21:56:45.643 INFO:teuthology.orchestra.run.smithi094.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4854-g8f219c47-1jammy [71.2 kB] 2024-09-10T21:56:45.644 INFO:teuthology.orchestra.run.smithi066.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-10T21:56:45.644 INFO:teuthology.orchestra.run.smithi094.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4854-g8f219c47-1jammy [338 kB] 2024-09-10T21:56:45.649 INFO:teuthology.orchestra.run.smithi094.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-10T21:56:45.650 INFO:teuthology.orchestra.run.smithi094.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-10T21:56:45.650 INFO:teuthology.orchestra.run.smithi094.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4854-g8f219c47-1jammy [7,988 kB] 2024-09-10T21:56:45.651 INFO:teuthology.orchestra.run.smithi094.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T21:56:45.652 INFO:teuthology.orchestra.run.smithi094.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-10T21:56:45.653 INFO:teuthology.orchestra.run.smithi094.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-10T21:56:45.657 INFO:teuthology.orchestra.run.smithi066.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-10T21:56:45.658 INFO:teuthology.orchestra.run.smithi066.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-10T21:56:45.658 INFO:teuthology.orchestra.run.smithi066.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T21:56:45.659 INFO:teuthology.orchestra.run.smithi066.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-10T21:56:45.659 INFO:teuthology.orchestra.run.smithi066.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-10T21:56:45.672 INFO:teuthology.orchestra.run.smithi094.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-10T21:56:45.674 INFO:teuthology.orchestra.run.smithi094.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-10T21:56:45.675 INFO:teuthology.orchestra.run.smithi066.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-10T21:56:45.675 INFO:teuthology.orchestra.run.smithi094.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-10T21:56:45.675 INFO:teuthology.orchestra.run.smithi066.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-10T21:56:45.675 INFO:teuthology.orchestra.run.smithi066.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-10T21:56:45.680 INFO:teuthology.orchestra.run.smithi066.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-10T21:56:45.680 INFO:teuthology.orchestra.run.smithi066.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-10T21:56:45.692 INFO:teuthology.orchestra.run.smithi066.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-10T21:56:45.692 INFO:teuthology.orchestra.run.smithi094.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-10T21:56:45.692 INFO:teuthology.orchestra.run.smithi094.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-10T21:56:45.693 INFO:teuthology.orchestra.run.smithi066.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-10T21:56:45.693 INFO:teuthology.orchestra.run.smithi066.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-10T21:56:45.694 INFO:teuthology.orchestra.run.smithi094.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-10T21:56:45.694 INFO:teuthology.orchestra.run.smithi094.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-10T21:56:45.694 INFO:teuthology.orchestra.run.smithi094.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-10T21:56:45.695 INFO:teuthology.orchestra.run.smithi094.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-10T21:56:45.695 INFO:teuthology.orchestra.run.smithi066.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-10T21:56:45.696 INFO:teuthology.orchestra.run.smithi066.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-10T21:56:45.703 INFO:teuthology.orchestra.run.smithi094.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-10T21:56:45.704 INFO:teuthology.orchestra.run.smithi094.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-10T21:56:45.707 INFO:teuthology.orchestra.run.smithi094.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-10T21:56:45.708 INFO:teuthology.orchestra.run.smithi066.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-10T21:56:45.709 INFO:teuthology.orchestra.run.smithi066.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-10T21:56:45.711 INFO:teuthology.orchestra.run.smithi066.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-10T21:56:45.711 INFO:teuthology.orchestra.run.smithi066.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-10T21:56:45.712 INFO:teuthology.orchestra.run.smithi066.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-10T21:56:45.720 INFO:teuthology.orchestra.run.smithi094.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-10T21:56:45.722 INFO:teuthology.orchestra.run.smithi094.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-10T21:56:45.722 INFO:teuthology.orchestra.run.smithi094.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-10T21:56:45.726 INFO:teuthology.orchestra.run.smithi066.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-10T21:56:45.726 INFO:teuthology.orchestra.run.smithi094.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-10T21:56:45.729 INFO:teuthology.orchestra.run.smithi094.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-10T21:56:45.729 INFO:teuthology.orchestra.run.smithi094.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-10T21:56:45.730 INFO:teuthology.orchestra.run.smithi094.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-10T21:56:45.737 INFO:teuthology.orchestra.run.smithi094.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-10T21:56:45.741 INFO:teuthology.orchestra.run.smithi094.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-10T21:56:45.742 INFO:teuthology.orchestra.run.smithi094.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-10T21:56:45.754 INFO:teuthology.orchestra.run.smithi094.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-10T21:56:45.754 INFO:teuthology.orchestra.run.smithi094.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T21:56:45.755 INFO:teuthology.orchestra.run.smithi094.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-10T21:56:45.756 INFO:teuthology.orchestra.run.smithi094.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-10T21:56:45.757 INFO:teuthology.orchestra.run.smithi094.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-10T21:56:45.758 INFO:teuthology.orchestra.run.smithi094.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-10T21:56:45.765 INFO:teuthology.orchestra.run.smithi094.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-10T21:56:45.771 INFO:teuthology.orchestra.run.smithi094.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-10T21:56:45.772 INFO:teuthology.orchestra.run.smithi094.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-10T21:56:45.772 INFO:teuthology.orchestra.run.smithi066.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-10T21:56:45.773 INFO:teuthology.orchestra.run.smithi066.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-10T21:56:45.774 INFO:teuthology.orchestra.run.smithi094.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-10T21:56:45.774 INFO:teuthology.orchestra.run.smithi066.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T21:56:45.775 INFO:teuthology.orchestra.run.smithi066.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-10T21:56:45.776 INFO:teuthology.orchestra.run.smithi066.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-10T21:56:45.788 INFO:teuthology.orchestra.run.smithi094.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-10T21:56:45.795 INFO:teuthology.orchestra.run.smithi066.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-10T21:56:45.797 INFO:teuthology.orchestra.run.smithi066.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-10T21:56:45.798 INFO:teuthology.orchestra.run.smithi066.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-10T21:56:45.805 INFO:teuthology.orchestra.run.smithi094.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4854-g8f219c47-1jammy [112 kB] 2024-09-10T21:56:45.806 INFO:teuthology.orchestra.run.smithi094.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4854-g8f219c47-1jammy [473 kB] 2024-09-10T21:56:45.812 INFO:teuthology.orchestra.run.smithi094.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4854-g8f219c47-1jammy [28.3 MB] 2024-09-10T21:56:45.814 INFO:teuthology.orchestra.run.smithi066.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-10T21:56:45.814 INFO:teuthology.orchestra.run.smithi066.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-10T21:56:45.815 INFO:teuthology.orchestra.run.smithi066.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-10T21:56:45.816 INFO:teuthology.orchestra.run.smithi066.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-10T21:56:45.816 INFO:teuthology.orchestra.run.smithi066.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-10T21:56:45.817 INFO:teuthology.orchestra.run.smithi066.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-10T21:56:45.827 INFO:teuthology.orchestra.run.smithi066.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-10T21:56:45.828 INFO:teuthology.orchestra.run.smithi066.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-10T21:56:45.831 INFO:teuthology.orchestra.run.smithi066.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-10T21:56:45.845 INFO:teuthology.orchestra.run.smithi066.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-10T21:56:45.846 INFO:teuthology.orchestra.run.smithi066.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-10T21:56:45.847 INFO:teuthology.orchestra.run.smithi066.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-10T21:56:45.850 INFO:teuthology.orchestra.run.smithi066.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-10T21:56:45.853 INFO:teuthology.orchestra.run.smithi066.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-10T21:56:45.854 INFO:teuthology.orchestra.run.smithi066.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-10T21:56:45.855 INFO:teuthology.orchestra.run.smithi066.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-10T21:56:45.862 INFO:teuthology.orchestra.run.smithi066.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-10T21:56:45.866 INFO:teuthology.orchestra.run.smithi066.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-10T21:56:45.867 INFO:teuthology.orchestra.run.smithi066.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-10T21:56:45.879 INFO:teuthology.orchestra.run.smithi066.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-10T21:56:45.879 INFO:teuthology.orchestra.run.smithi066.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T21:56:45.880 INFO:teuthology.orchestra.run.smithi066.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-10T21:56:45.881 INFO:teuthology.orchestra.run.smithi066.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-10T21:56:45.882 INFO:teuthology.orchestra.run.smithi066.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-10T21:56:45.883 INFO:teuthology.orchestra.run.smithi066.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-10T21:56:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-10T21:56:45.897 INFO:teuthology.orchestra.run.smithi066.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-10T21:56:45.897 INFO:teuthology.orchestra.run.smithi066.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-10T21:56:45.899 INFO:teuthology.orchestra.run.smithi066.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-10T21:56:45.914 INFO:teuthology.orchestra.run.smithi066.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-10T21:56:45.988 INFO:teuthology.orchestra.run.smithi066.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4854-g8f219c47-1jammy [3,330 kB] 2024-09-10T21:56:46.318 INFO:teuthology.orchestra.run.smithi066.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4854-g8f219c47-1jammy [721 kB] 2024-09-10T21:56:46.365 INFO:teuthology.orchestra.run.smithi094.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4854-g8f219c47-1jammy [5,206 kB] 2024-09-10T21:56:46.375 INFO:teuthology.orchestra.run.smithi066.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4854-g8f219c47-1jammy [356 kB] 2024-09-10T21:56:46.424 INFO:teuthology.orchestra.run.smithi066.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4854-g8f219c47-1jammy [32.8 kB] 2024-09-10T21:56:46.424 INFO:teuthology.orchestra.run.smithi066.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4854-g8f219c47-1jammy [180 kB] 2024-09-10T21:56:46.427 INFO:teuthology.orchestra.run.smithi066.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4854-g8f219c47-1jammy [71.2 kB] 2024-09-10T21:56:46.432 INFO:teuthology.orchestra.run.smithi066.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4854-g8f219c47-1jammy [338 kB] 2024-09-10T21:56:46.464 INFO:teuthology.orchestra.run.smithi094.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4854-g8f219c47-1jammy [251 kB] 2024-09-10T21:56:46.466 INFO:teuthology.orchestra.run.smithi094.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4854-g8f219c47-1jammy [124 kB] 2024-09-10T21:56:46.468 INFO:teuthology.orchestra.run.smithi094.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4854-g8f219c47-1jammy [1,562 kB] 2024-09-10T21:56:46.481 INFO:teuthology.orchestra.run.smithi066.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4854-g8f219c47-1jammy [7,988 kB] 2024-09-10T21:56:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4854-g8f219c47-1jammy [6,308 kB] 2024-09-10T21:56:46.634 INFO:teuthology.orchestra.run.smithi094.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4854-g8f219c47-1jammy [23.1 MB] 2024-09-10T21:56:47.030 INFO:teuthology.orchestra.run.smithi066.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4854-g8f219c47-1jammy [112 kB] 2024-09-10T21:56:47.039 INFO:teuthology.orchestra.run.smithi066.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4854-g8f219c47-1jammy [473 kB] 2024-09-10T21:56:47.083 INFO:teuthology.orchestra.run.smithi066.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4854-g8f219c47-1jammy [28.3 MB] 2024-09-10T21:56:47.141 INFO:teuthology.orchestra.run.smithi094.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4854-g8f219c47-1jammy [14.0 kB] 2024-09-10T21:56:47.141 INFO:teuthology.orchestra.run.smithi094.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4854-g8f219c47-1jammy [880 kB] 2024-09-10T21:56:47.163 INFO:teuthology.orchestra.run.smithi094.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4854-g8f219c47-1jammy [2,454 kB] 2024-09-10T21:56:47.222 INFO:teuthology.orchestra.run.smithi094.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4854-g8f219c47-1jammy [813 kB] 2024-09-10T21:56:47.239 INFO:teuthology.orchestra.run.smithi094.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4854-g8f219c47-1jammy [160 kB] 2024-09-10T21:56:47.242 INFO:teuthology.orchestra.run.smithi094.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4854-g8f219c47-1jammy [61.4 MB] 2024-09-10T21:56:48.301 INFO:teuthology.orchestra.run.smithi066.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4854-g8f219c47-1jammy [5,206 kB] 2024-09-10T21:56:48.457 INFO:teuthology.orchestra.run.smithi066.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4854-g8f219c47-1jammy [251 kB] 2024-09-10T21:56:48.460 INFO:teuthology.orchestra.run.smithi066.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4854-g8f219c47-1jammy [124 kB] 2024-09-10T21:56:48.477 INFO:teuthology.orchestra.run.smithi066.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4854-g8f219c47-1jammy [1,562 kB] 2024-09-10T21:56:48.527 INFO:teuthology.orchestra.run.smithi066.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4854-g8f219c47-1jammy [6,308 kB] 2024-09-10T21:56:48.734 INFO:teuthology.orchestra.run.smithi066.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4854-g8f219c47-1jammy [23.1 MB] 2024-09-10T21:56:49.059 INFO:teuthology.orchestra.run.smithi094.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4854-g8f219c47-1jammy [8,625 kB] 2024-09-10T21:56:49.329 INFO:teuthology.orchestra.run.smithi094.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4854-g8f219c47-1jammy [14.2 kB] 2024-09-10T21:56:49.330 INFO:teuthology.orchestra.run.smithi094.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4854-g8f219c47-1jammy [55.8 MB] 2024-09-10T21:56:49.496 INFO:teuthology.orchestra.run.smithi066.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4854-g8f219c47-1jammy [14.0 kB] 2024-09-10T21:56:49.496 INFO:teuthology.orchestra.run.smithi066.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4854-g8f219c47-1jammy [880 kB] 2024-09-10T21:56:49.523 INFO:teuthology.orchestra.run.smithi066.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4854-g8f219c47-1jammy [2,454 kB] 2024-09-10T21:56:49.602 INFO:teuthology.orchestra.run.smithi066.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4854-g8f219c47-1jammy [813 kB] 2024-09-10T21:56:49.636 INFO:teuthology.orchestra.run.smithi066.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4854-g8f219c47-1jammy [160 kB] 2024-09-10T21:56:49.638 INFO:teuthology.orchestra.run.smithi066.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4854-g8f219c47-1jammy [61.4 MB] 2024-09-10T21:56:51.076 INFO:teuthology.orchestra.run.smithi094.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4854-g8f219c47-1jammy [130 kB] 2024-09-10T21:56:51.077 INFO:teuthology.orchestra.run.smithi094.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4854-g8f219c47-1jammy [233 kB] 2024-09-10T21:56:51.079 INFO:teuthology.orchestra.run.smithi094.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4854-g8f219c47-1jammy [33.9 kB] 2024-09-10T21:56:51.080 INFO:teuthology.orchestra.run.smithi094.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4854-g8f219c47-1jammy [23.4 kB] 2024-09-10T21:56:51.080 INFO:teuthology.orchestra.run.smithi094.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4854-g8f219c47-1jammy [39.0 kB] 2024-09-10T21:56:51.081 INFO:teuthology.orchestra.run.smithi094.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4854-g8f219c47-1jammy [15.9 MB] 2024-09-10T21:56:51.570 INFO:teuthology.orchestra.run.smithi094.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4854-g8f219c47-1jammy [91.2 kB] 2024-09-10T21:56:51.687 INFO:teuthology.orchestra.run.smithi066.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4854-g8f219c47-1jammy [8,625 kB] 2024-09-10T21:56:51.811 INFO:teuthology.orchestra.run.smithi066.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4854-g8f219c47-1jammy [14.2 kB] 2024-09-10T21:56:51.811 INFO:teuthology.orchestra.run.smithi066.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4854-g8f219c47-1jammy [55.8 MB] 2024-09-10T21:56:52.223 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 251 MB in 6s (39.8 MB/s) 2024-09-10T21:56:52.367 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-10T21:56:52.430 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-10T21:56:52.433 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-10T21:56:52.474 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-10T21:56:52.741 INFO:teuthology.orchestra.run.smithi066.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4854-g8f219c47-1jammy [130 kB] 2024-09-10T21:56:52.742 INFO:teuthology.orchestra.run.smithi066.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4854-g8f219c47-1jammy [233 kB] 2024-09-10T21:56:52.744 INFO:teuthology.orchestra.run.smithi066.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4854-g8f219c47-1jammy [33.9 kB] 2024-09-10T21:56:52.744 INFO:teuthology.orchestra.run.smithi066.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4854-g8f219c47-1jammy [23.4 kB] 2024-09-10T21:56:52.745 INFO:teuthology.orchestra.run.smithi066.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4854-g8f219c47-1jammy [39.0 kB] 2024-09-10T21:56:52.745 INFO:teuthology.orchestra.run.smithi066.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4854-g8f219c47-1jammy [15.9 MB] 2024-09-10T21:56:52.810 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T21:56:52.823 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-10T21:56:52.868 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:56:52.996 INFO:teuthology.orchestra.run.smithi066.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4854-g8f219c47-1jammy [91.2 kB] 2024-09-10T21:56:53.138 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cmd2. 2024-09-10T21:56:53.151 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-10T21:56:53.213 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-10T21:56:53.516 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-10T21:56:53.529 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-10T21:56:53.608 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:56:53.674 INFO:teuthology.orchestra.run.smithi066.stdout:Fetched 251 MB in 8s (33.0 MB/s) 2024-09-10T21:56:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-10T21:56:53.881 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-10T21:56:53.884 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-10T21:56:53.930 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-10T21:56:53.970 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T21:56:53.983 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-10T21:56:54.020 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:56:54.299 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T21:56:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-10T21:56:54.356 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T21:56:54.366 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:56:54.370 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-10T21:56:54.440 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:56:54.744 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cmd2. 2024-09-10T21:56:54.759 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-10T21:56:54.802 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T21:56:54.810 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-10T21:56:54.815 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:56:54.859 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:56:55.213 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-10T21:56:55.228 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-10T21:56:55.297 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T21:56:55.310 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:56:55.355 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:56:55.505 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:56:55.725 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T21:56:55.739 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:56:55.776 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:56:55.975 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T21:56:55.991 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-10T21:56:56.034 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:56:56.320 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T21:56:56.336 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-10T21:56:56.353 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-10T21:56:56.368 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-10T21:56:56.370 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:56:56.579 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:56:56.774 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T21:56:56.789 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:56:56.857 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:56:56.932 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librados2. 2024-09-10T21:56:56.946 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../010-librados2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:56:56.990 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:56:57.260 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T21:56:57.275 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:56:57.318 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:56:57.410 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libnbd0. 2024-09-10T21:56:57.423 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-10T21:56:57.501 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-10T21:56:57.646 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T21:56:57.661 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:56:57.713 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:56:57.829 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librbd1. 2024-09-10T21:56:57.843 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../012-librbd1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:56:57.887 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:56:58.023 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-10T21:56:58.039 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-10T21:56:58.082 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:56:58.324 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T21:56:58.337 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:56:58.382 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:56:58.393 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package librados2. 2024-09-10T21:56:58.408 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../010-librados2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:56:58.451 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:56:58.643 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rados. 2024-09-10T21:56:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:56:58.710 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:56:58.921 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libnbd0. 2024-09-10T21:56:58.935 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-10T21:56:58.979 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-10T21:56:59.013 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T21:56:59.027 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:56:59.071 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:56:59.232 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package librbd1. 2024-09-10T21:56:59.246 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../012-librbd1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:56:59.290 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:56:59.349 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T21:56:59.363 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:56:59.407 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:56:59.718 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T21:56:59.726 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T21:56:59.732 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:56:59.741 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:56:59.776 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:56:59.785 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:00.054 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-rados. 2024-09-10T21:57:00.068 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:00.112 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:00.121 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T21:57:00.135 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-10T21:57:00.163 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-10T21:57:00.426 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T21:57:00.444 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:00.457 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T21:57:00.472 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:00.500 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:00.506 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:00.726 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T21:57:00.740 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:00.784 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:00.844 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T21:57:00.857 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-10T21:57:00.902 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:57:01.103 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T21:57:01.118 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:01.161 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:01.217 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-10T21:57:01.227 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-10T21:57:01.280 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:57:01.506 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T21:57:01.521 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-10T21:57:01.558 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-10T21:57:01.564 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-10T21:57:01.571 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-10T21:57:01.616 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:57:01.816 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T21:57:01.831 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:01.866 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:01.928 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua5.1. 2024-09-10T21:57:01.942 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-10T21:57:01.986 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:57:02.186 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T21:57:02.201 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-10T21:57:02.244 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:57:02.320 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua-any. 2024-09-10T21:57:02.334 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-10T21:57:02.382 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-10T21:57:02.513 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-10T21:57:02.528 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-10T21:57:02.572 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:57:02.626 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package zip. 2024-09-10T21:57:02.639 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-10T21:57:02.684 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking zip (3.0-12build2) ... 2024-09-10T21:57:02.853 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-10T21:57:02.865 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-10T21:57:02.924 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:57:02.979 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package unzip. 2024-09-10T21:57:02.993 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-10T21:57:03.038 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:57:03.227 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package lua5.1. 2024-09-10T21:57:03.243 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-10T21:57:03.277 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:57:03.442 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package luarocks. 2024-09-10T21:57:03.455 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-10T21:57:03.500 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:57:03.580 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package lua-any. 2024-09-10T21:57:03.596 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-10T21:57:03.638 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-10T21:57:03.908 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librgw2. 2024-09-10T21:57:03.921 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../028-librgw2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:03.966 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package zip. 2024-09-10T21:57:03.966 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:03.982 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-10T21:57:04.033 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking zip (3.0-12build2) ... 2024-09-10T21:57:04.411 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package unzip. 2024-09-10T21:57:04.426 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-10T21:57:04.477 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:57:04.519 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T21:57:04.534 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:04.577 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:04.886 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-10T21:57:04.901 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-10T21:57:04.939 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:57:04.956 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package luarocks. 2024-09-10T21:57:04.972 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-10T21:57:05.014 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:57:05.242 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T21:57:05.257 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:05.300 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:05.397 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package librgw2. 2024-09-10T21:57:05.411 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../028-librgw2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:05.455 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:05.754 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-common. 2024-09-10T21:57:05.779 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:05.812 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:06.066 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T21:57:06.081 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:06.124 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:06.435 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-10T21:57:06.451 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-10T21:57:06.485 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:57:06.788 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T21:57:06.803 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:06.846 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:07.084 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-base. 2024-09-10T21:57:07.099 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:07.299 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-common. 2024-09-10T21:57:07.315 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:07.357 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:07.730 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-10T21:57:07.744 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-10T21:57:07.789 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:57:08.125 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cheroot. 2024-09-10T21:57:08.139 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-10T21:57:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:57:08.478 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-10T21:57:08.479 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-base. 2024-09-10T21:57:08.492 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-10T21:57:08.495 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:08.536 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:57:08.547 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:08.848 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-10T21:57:08.861 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-10T21:57:08.906 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:57:09.125 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-10T21:57:09.139 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-10T21:57:09.183 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:57:09.192 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-10T21:57:09.206 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-10T21:57:09.276 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:57:09.469 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cheroot. 2024-09-10T21:57:09.483 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-10T21:57:09.527 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:57:09.579 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-tempora. 2024-09-10T21:57:09.592 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-10T21:57:09.637 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-10T21:57:09.889 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-10T21:57:09.903 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-10T21:57:09.939 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:57:09.940 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-portend. 2024-09-10T21:57:09.954 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-10T21:57:09.998 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-10T21:57:10.208 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-10T21:57:10.223 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-10T21:57:10.266 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:57:10.268 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-10T21:57:10.281 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-10T21:57:10.318 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-10T21:57:10.552 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-10T21:57:10.568 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-10T21:57:10.610 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:57:10.646 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T21:57:10.659 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-10T21:57:10.704 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:57:10.938 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-tempora. 2024-09-10T21:57:10.954 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-10T21:57:10.991 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T21:57:10.996 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-10T21:57:11.005 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-10T21:57:11.041 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-10T21:57:11.299 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-portend. 2024-09-10T21:57:11.314 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-10T21:57:11.353 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T21:57:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-10T21:57:11.366 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-10T21:57:11.411 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-10T21:57:11.677 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-10T21:57:11.692 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-10T21:57:11.727 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-10T21:57:11.747 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T21:57:11.761 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-10T21:57:11.805 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:57:12.005 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T21:57:12.019 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-10T21:57:12.063 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:57:12.117 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-mako. 2024-09-10T21:57:12.130 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-10T21:57:12.175 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:57:12.399 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T21:57:12.414 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-10T21:57:12.466 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-10T21:57:12.495 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T21:57:12.508 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T21:57:12.545 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:57:12.727 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T21:57:12.742 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-10T21:57:12.786 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-10T21:57:12.832 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T21:57:12.845 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-10T21:57:12.915 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:57:13.097 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T21:57:13.112 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-10T21:57:13.155 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:57:13.218 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-webob. 2024-09-10T21:57:13.232 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-10T21:57:13.276 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:57:13.483 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-mako. 2024-09-10T21:57:13.498 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-10T21:57:13.541 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:57:13.646 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T21:57:13.660 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-10T21:57:13.707 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:57:13.877 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T21:57:13.893 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T21:57:13.936 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:57:13.991 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T21:57:14.005 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-10T21:57:14.040 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:57:14.255 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T21:57:14.271 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-10T21:57:14.305 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:57:14.361 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-paste. 2024-09-10T21:57:14.375 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-10T21:57:14.419 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:57:14.608 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-webob. 2024-09-10T21:57:14.623 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-10T21:57:14.666 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:57:14.705 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T21:57:14.720 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-10T21:57:14.772 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:57:15.036 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T21:57:15.051 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-10T21:57:15.083 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T21:57:15.097 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:57:15.098 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-10T21:57:15.142 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:57:15.390 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T21:57:15.406 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-10T21:57:15.448 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:57:15.461 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T21:57:15.477 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-10T21:57:15.511 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-10T21:57:15.701 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-paste. 2024-09-10T21:57:15.717 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-10T21:57:15.759 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:57:15.831 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T21:57:15.846 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-10T21:57:15.889 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:57:16.113 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T21:57:16.128 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-10T21:57:16.185 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:57:16.311 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-certifi. 2024-09-10T21:57:16.326 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-10T21:57:16.369 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-10T21:57:16.607 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T21:57:16.622 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-10T21:57:16.665 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:57:16.823 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-idna. 2024-09-10T21:57:16.837 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-10T21:57:16.881 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:57:17.001 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T21:57:17.017 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-10T21:57:17.060 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-10T21:57:17.372 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T21:57:17.386 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-10T21:57:17.429 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:57:17.817 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-certifi. 2024-09-10T21:57:17.832 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-10T21:57:17.875 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-10T21:57:17.959 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-urllib3. 2024-09-10T21:57:17.975 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-10T21:57:18.017 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:57:18.363 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-requests. 2024-09-10T21:57:18.377 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-10T21:57:18.420 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:57:18.741 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T21:57:18.756 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-10T21:57:18.799 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:57:18.878 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-idna. 2024-09-10T21:57:18.894 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-10T21:57:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:57:19.085 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T21:57:19.099 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:19.143 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:19.231 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-urllib3. 2024-09-10T21:57:19.246 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-10T21:57:19.289 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:57:19.541 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T21:57:19.554 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:19.599 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:19.609 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-requests. 2024-09-10T21:57:19.624 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-10T21:57:19.667 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:57:19.902 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T21:57:19.916 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-10T21:57:19.960 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:57:19.978 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T21:57:19.993 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-10T21:57:20.036 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:57:20.296 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T21:57:20.297 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T21:57:20.310 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-10T21:57:20.313 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:20.354 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-10T21:57:20.355 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:20.676 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T21:57:20.691 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:20.752 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T21:57:20.759 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:20.767 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:20.810 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:21.063 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T21:57:21.078 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-10T21:57:21.113 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:57:21.163 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T21:57:21.178 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:21.221 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:21.449 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T21:57:21.465 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-10T21:57:21.507 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-10T21:57:21.783 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T21:57:21.797 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:21.810 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T21:57:21.826 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:21.833 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:21.860 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:22.238 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T21:57:22.254 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:22.296 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:22.645 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph. 2024-09-10T21:57:22.660 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../069-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:22.703 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:22.841 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T21:57:22.856 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:22.900 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:23.022 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T21:57:23.037 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:23.080 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:23.434 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T21:57:23.449 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:23.484 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:23.653 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph. 2024-09-10T21:57:23.668 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../069-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:23.702 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:23.879 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephadm. 2024-09-10T21:57:23.893 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../072-cephadm_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:23.937 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T21:57:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:24.231 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-10T21:57:24.246 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-10T21:57:24.289 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-10T21:57:24.425 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T21:57:24.440 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:24.483 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:24.626 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-10T21:57:24.641 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:57:24.676 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:57:24.885 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package cephadm. 2024-09-10T21:57:24.901 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../072-cephadm_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:24.944 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:25.255 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-10T21:57:25.270 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-10T21:57:25.313 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-10T21:57:25.590 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-10T21:57:25.605 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:57:25.647 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-babel. 2024-09-10T21:57:25.661 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:57:25.674 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:57:25.697 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:57:25.993 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T21:57:26.006 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-10T21:57:26.042 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:57:26.412 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T21:57:26.426 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:26.470 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:26.687 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-babel. 2024-09-10T21:57:26.701 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:57:26.745 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:57:26.782 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T21:57:26.796 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T21:57:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T21:57:27.064 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T21:57:27.079 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-10T21:57:27.122 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:57:27.369 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-routes. 2024-09-10T21:57:27.383 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-10T21:57:27.459 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T21:57:27.460 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:57:27.473 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:27.517 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:27.772 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T21:57:27.786 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:27.821 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:27.845 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T21:57:27.860 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T21:57:27.904 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T21:57:28.156 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-routes. 2024-09-10T21:57:28.171 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-10T21:57:28.206 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:57:28.543 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T21:57:28.557 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:28.592 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:30.228 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-10T21:57:30.243 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-10T21:57:30.286 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:57:30.814 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T21:57:30.828 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-10T21:57:30.847 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-10T21:57:30.863 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-10T21:57:30.905 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:57:30.958 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:57:31.312 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-10T21:57:31.326 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-10T21:57:31.387 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:57:31.417 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T21:57:31.431 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-10T21:57:31.552 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:57:31.691 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T21:57:31.705 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-10T21:57:31.749 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:57:32.181 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-10T21:57:32.196 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-10T21:57:32.239 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:57:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T21:57:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:32.500 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T21:57:32.515 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-10T21:57:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:32.550 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:57:33.233 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T21:57:33.249 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:33.292 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:33.537 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T21:57:33.553 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-10T21:57:33.595 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-10T21:57:33.931 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T21:57:33.947 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-10T21:57:33.989 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:57:34.228 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T21:57:34.245 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-10T21:57:34.278 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-10T21:57:34.326 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T21:57:34.342 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-10T21:57:34.385 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:57:34.564 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T21:57:34.581 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-10T21:57:34.647 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:57:34.721 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T21:57:34.737 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-10T21:57:34.779 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-10T21:57:34.976 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T21:57:34.992 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-10T21:57:35.034 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:57:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T21:57:35.109 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-10T21:57:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-10T21:57:35.345 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T21:57:35.362 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-10T21:57:35.403 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-10T21:57:35.461 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T21:57:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-10T21:57:35.520 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:57:35.748 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T21:57:35.764 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-10T21:57:35.807 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-10T21:57:35.865 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T21:57:35.881 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-10T21:57:35.923 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-10T21:57:36.118 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T21:57:36.135 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-10T21:57:36.176 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:57:36.218 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T21:57:36.234 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-10T21:57:36.276 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:57:36.479 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T21:57:36.496 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-10T21:57:36.537 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-10T21:57:36.874 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T21:57:36.891 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-10T21:57:36.932 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:57:37.082 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T21:57:37.097 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:37.140 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:37.460 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T21:57:37.475 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-10T21:57:37.543 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:57:37.871 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T21:57:37.872 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T21:57:37.886 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:57:37.886 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:37.929 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:37.930 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:57:38.199 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T21:57:38.213 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-10T21:57:38.241 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package jq. 2024-09-10T21:57:38.256 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:57:38.273 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:57:38.299 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-10T21:57:38.552 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package socat. 2024-09-10T21:57:38.567 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-10T21:57:38.602 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:57:38.618 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T21:57:38.633 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:57:38.677 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:57:38.888 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T21:57:38.903 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-10T21:57:38.937 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package jq. 2024-09-10T21:57:38.938 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:57:38.952 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:57:38.995 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-10T21:57:39.248 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package socat. 2024-09-10T21:57:39.263 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-10T21:57:39.268 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-test. 2024-09-10T21:57:39.282 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:39.306 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:57:39.349 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:39.634 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T21:57:39.650 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-10T21:57:39.692 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:57:40.080 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-test. 2024-09-10T21:57:40.095 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:40.138 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:41.481 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T21:57:41.495 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:41.530 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:41.935 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-10T21:57:41.950 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:41.995 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:42.176 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T21:57:42.193 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:42.234 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:42.264 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-colorama. 2024-09-10T21:57:42.278 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-10T21:57:42.313 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-10T21:57:42.613 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-10T21:57:42.629 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:42.671 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:42.807 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephfs-shell. 2024-09-10T21:57:42.822 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:42.866 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:43.007 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-colorama. 2024-09-10T21:57:43.024 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-10T21:57:43.065 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-10T21:57:43.168 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephfs-top. 2024-09-10T21:57:43.182 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:43.235 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:43.326 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package cephfs-shell. 2024-09-10T21:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:43.384 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:43.487 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T21:57:43.501 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:43.546 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:43.637 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package cephfs-top. 2024-09-10T21:57:43.653 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:57:43.695 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:43.849 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-10T21:57:43.863 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-10T21:57:43.899 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:57:43.931 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T21:57:43.947 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:43.988 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:44.237 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-10T21:57:44.255 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-10T21:57:44.295 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:57:44.299 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-10T21:57:44.316 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-10T21:57:44.358 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:57:44.691 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T21:57:44.707 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-10T21:57:44.712 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-10T21:57:44.729 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-10T21:57:44.749 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:57:44.770 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:57:45.091 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T21:57:45.108 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-10T21:57:45.149 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:57:45.180 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package pkg-config. 2024-09-10T21:57:45.195 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-10T21:57:45.230 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:57:45.449 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-10T21:57:45.464 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-10T21:57:45.508 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:57:45.612 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package pkg-config. 2024-09-10T21:57:45.629 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-10T21:57:45.671 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:57:45.919 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-10T21:57:45.931 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-10T21:57:45.934 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-10T21:57:45.948 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-10T21:57:45.969 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-10T21:57:45.990 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:57:46.289 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T21:57:46.304 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T21:57:46.347 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T21:57:46.426 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-10T21:57:46.443 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-10T21:57:46.484 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-10T21:57:46.667 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T21:57:46.683 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-10T21:57:46.725 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:57:46.754 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T21:57:46.769 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T21:57:46.803 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T21:57:47.070 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T21:57:47.086 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-10T21:57:47.129 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-10T21:57:47.140 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T21:57:47.156 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-10T21:57:47.198 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:57:47.490 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-py. 2024-09-10T21:57:47.506 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-10T21:57:47.526 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T21:57:47.542 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-10T21:57:47.548 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-10T21:57:47.584 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-10T21:57:47.937 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-py. 2024-09-10T21:57:47.953 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-10T21:57:47.995 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-10T21:57:48.085 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pygments. 2024-09-10T21:57:48.101 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-10T21:57:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:57:48.324 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pygments. 2024-09-10T21:57:48.340 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-10T21:57:48.382 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:57:48.589 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T21:57:48.604 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-10T21:57:48.664 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:57:48.819 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T21:57:48.834 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-10T21:57:48.877 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:57:48.933 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-toml. 2024-09-10T21:57:48.949 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-10T21:57:48.992 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-10T21:57:49.213 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-toml. 2024-09-10T21:57:49.228 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-10T21:57:49.271 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-10T21:57:49.337 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T21:57:49.353 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-10T21:57:49.395 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:57:49.557 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T21:57:49.573 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-10T21:57:49.615 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:57:49.690 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T21:57:49.705 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-10T21:57:49.748 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:57:49.969 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T21:57:49.984 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-10T21:57:50.026 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T21:57:50.027 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:57:50.042 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-10T21:57:50.084 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:57:50.296 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T21:57:50.308 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-10T21:57:50.346 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:57:50.789 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package radosgw. 2024-09-10T21:57:50.796 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../123-radosgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:50.830 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:51.058 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package radosgw. 2024-09-10T21:57:51.074 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../123-radosgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:51.108 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:51.525 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T21:57:51.537 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:51.592 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:51.786 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T21:57:51.801 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:57:51.845 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:51.892 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-10T21:57:52.140 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-10T21:57:52.164 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:57:52.265 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:57:52.454 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:57:52.528 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-10T21:57:52.576 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:57:52.846 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-10T21:57:52.904 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-10T21:57:53.269 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-10T21:57:53.518 INFO:teuthology.orchestra.run.smithi094.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-10T21:57:53.562 INFO:teuthology.orchestra.run.smithi094.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-10T21:57:53.600 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:53.842 INFO:teuthology.orchestra.run.smithi066.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-10T21:57:53.889 INFO:teuthology.orchestra.run.smithi066.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-10T21:57:53.929 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:54.136 INFO:teuthology.orchestra.run.smithi094.stdout:Adding system user cephadm....done 2024-09-10T21:57:54.282 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:57:54.481 INFO:teuthology.orchestra.run.smithi066.stdout:Adding system user cephadm....done 2024-09-10T21:57:54.592 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:57:54.635 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:57:54.868 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:57:54.920 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:57:55.019 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:57:55.212 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:57:55.296 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T21:57:55.338 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:57:55.580 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:57:55.598 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T21:57:55.689 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-10T21:57:55.899 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:57:55.992 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:57:56.025 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-10T21:57:56.343 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:57:56.368 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-10T21:57:56.628 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-10T21:57:56.720 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-10T21:57:56.897 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-10T21:57:57.013 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-10T21:57:57.181 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:57:57.281 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-10T21:57:57.482 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:57:57.632 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:57:57.784 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:57:57.891 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:57:58.043 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:57:58.143 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:57:58.311 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:58.419 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:57:58.612 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:57:58.720 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:57:58.738 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-10T21:57:58.839 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:57:58.940 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-10T21:57:58.988 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:57:59.105 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-10T21:57:59.217 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:57:59.231 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:57:59.317 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:57:59.382 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-10T21:57:59.568 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:57:59.658 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:57:59.784 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:57:59.974 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-10T21:58:00.041 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-10T21:58:00.051 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:58:00.097 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:58:00.223 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-10T21:58:00.474 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-10T21:58:00.532 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-10T21:58:00.583 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:58:00.597 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:58:00.709 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-10T21:58:00.722 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-10T21:58:00.985 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:58:01.107 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:58:01.233 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-10T21:58:01.278 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:58:01.501 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:58:01.580 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up zip (3.0-12build2) ... 2024-09-10T21:58:01.739 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:58:01.777 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:58:02.103 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up zip (3.0-12build2) ... 2024-09-10T21:58:02.229 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:58:02.432 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:58:02.700 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:58:02.851 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:58:02.948 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:58:03.011 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:58:03.223 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:58:03.330 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-10T21:58:03.349 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:58:03.475 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:58:03.732 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-10T21:58:03.810 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-10T21:58:03.999 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:58:04.228 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-10T21:58:04.417 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:58:04.487 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:58:04.719 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-10T21:58:04.905 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:58:05.020 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:58:05.215 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-10T21:58:05.363 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:58:05.541 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:58:05.656 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:58:05.781 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:05.900 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:58:06.124 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:58:06.168 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:58:06.319 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:06.505 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:58:06.646 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:58:06.989 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:58:07.731 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:58:07.949 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:58:08.092 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-10T21:58:08.285 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:58:08.368 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:58:08.493 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:58:08.520 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:58:08.619 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:58:08.638 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-10T21:58:08.922 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:58:08.954 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:58:09.048 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:58:09.174 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:58:09.214 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-10T21:58:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:58:09.484 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:58:09.582 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:58:09.760 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-10T21:58:09.900 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:58:10.036 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:58:10.168 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-10T21:58:10.170 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:58:10.436 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-10T21:58:10.480 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:58:10.686 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:58:10.773 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-10T21:58:10.987 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-10T21:58:11.049 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-10T21:58:11.264 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:58:11.317 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:58:11.432 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-10T21:58:11.589 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-10T21:58:11.716 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:58:11.847 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:58:11.861 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:58:12.001 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-10T21:58:12.135 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:58:12.321 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:58:12.444 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:58:12.446 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:58:12.703 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:58:12.748 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:58:12.804 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:58:12.930 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:58:13.019 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-10T21:58:13.057 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:58:13.240 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:58:13.358 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:58:13.383 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:58:13.509 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:58:13.635 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:58:13.724 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-10T21:58:13.790 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:58:13.979 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-10T21:58:14.129 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:58:14.170 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-10T21:58:14.420 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:58:14.521 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:58:14.531 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:58:14.798 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-10T21:58:14.839 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:58:14.924 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-10T21:58:15.148 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:58:15.175 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:58:15.301 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:58:15.518 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-10T21:58:15.636 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:58:15.652 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-10T21:58:15.912 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-10T21:58:15.945 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:58:16.172 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:58:16.298 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-10T21:58:16.424 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-10T21:58:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:58:16.709 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-10T21:58:16.866 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T21:58:16.993 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:58:17.159 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:58:17.270 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:58:17.503 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:58:17.604 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:17.663 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T21:58:17.722 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:17.848 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:58:17.964 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:58:18.209 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:18.359 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:58:18.493 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:18.753 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:58:18.759 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:18.885 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-10T21:58:19.064 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:19.154 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:19.232 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:19.254 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:19.358 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-10T21:58:19.406 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:19.615 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:58:19.616 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:58:19.651 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:19.777 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:19.903 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:20.125 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:58:20.125 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:58:20.240 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:20.366 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:20.517 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:20.643 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:20.730 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:20.761 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:58:20.847 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:20.998 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:21.124 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:21.250 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:58:22.006 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:22.132 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:22.258 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:22.417 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:22.543 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:22.579 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:22.669 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:22.713 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:22.822 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:22.948 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:23.074 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:23.172 INFO:teuthology.orchestra.run.smithi094.stdout:Adding group ceph....done 2024-09-10T21:58:23.200 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:23.573 INFO:teuthology.orchestra.run.smithi094.stdout:Adding system user ceph....done 2024-09-10T21:58:23.750 INFO:teuthology.orchestra.run.smithi094.stdout:Setting system user ceph properties....done 2024-09-10T21:58:23.767 INFO:teuthology.orchestra.run.smithi094.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T21:58:23.769 INFO:teuthology.orchestra.run.smithi066.stdout:Adding group ceph....done 2024-09-10T21:58:23.911 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T21:58:24.078 INFO:teuthology.orchestra.run.smithi066.stdout:Adding system user ceph....done 2024-09-10T21:58:24.223 INFO:teuthology.orchestra.run.smithi066.stdout:Setting system user ceph properties....done 2024-09-10T21:58:24.242 INFO:teuthology.orchestra.run.smithi066.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T21:58:24.304 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T21:58:24.397 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T21:58:24.780 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T21:58:24.995 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:25.194 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:58:25.195 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:58:25.509 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:25.726 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:58:25.726 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:58:25.880 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:26.022 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:26.411 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:26.547 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:58:26.548 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:58:26.605 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:27.233 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:27.270 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:58:27.270 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:58:27.590 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T21:58:27.906 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:28.278 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T21:58:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:28.548 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:58:28.549 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:58:29.185 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:29.253 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:29.422 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:58:29.423 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:58:29.448 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:58:29.448 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:58:30.070 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:30.104 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:30.325 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:58:30.325 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:58:30.495 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:58:30.496 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:58:30.955 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:31.139 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:31.265 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:31.278 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:58:31.278 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:58:31.408 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:31.642 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:58:31.642 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:58:31.898 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:32.024 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:32.200 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:32.267 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:32.410 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:32.416 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:58:32.416 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:58:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:32.998 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:58:33.068 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:58:33.211 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:33.275 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:58:33.345 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:33.496 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:58:33.876 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:58:33.991 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:58:34.288 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:58:34.510 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:58:35.038 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:58:35.205 INFO:teuthology.orchestra.run.smithi094.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:58:35.210 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-10T21:58:35.286 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-09-10T21:58:35.472 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2024-09-10T21:58:35.473 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2024-09-10T21:58:35.657 INFO:teuthology.orchestra.run.smithi094.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-10T21:58:35.657 INFO:teuthology.orchestra.run.smithi094.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-10T21:58:35.658 INFO:teuthology.orchestra.run.smithi094.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-10T21:58:35.658 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:58:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:58:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:58:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:58:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:58:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:58:35.659 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:58:35.659 INFO:teuthology.orchestra.run.smithi094.stdout:The following additional packages will be installed: 2024-09-10T21:58:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-10T21:58:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-10T21:58:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-10T21:58:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-10T21:58:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:58:35.661 INFO:teuthology.orchestra.run.smithi094.stdout:Suggested packages: 2024-09-10T21:58:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-10T21:58:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-10T21:58:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-10T21:58:35.718 INFO:teuthology.orchestra.run.smithi094.stdout:The following NEW packages will be installed: 2024-09-10T21:58:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-10T21:58:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-10T21:58:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-10T21:58:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-10T21:58:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-10T21:58:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-10T21:58:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:58:35.720 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages will be upgraded: 2024-09-10T21:58:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-10T21:58:35.759 INFO:teuthology.orchestra.run.smithi066.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:58:35.763 DEBUG:teuthology.orchestra.run.smithi066:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-10T21:58:35.794 INFO:teuthology.orchestra.run.smithi094.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-10T21:58:35.795 INFO:teuthology.orchestra.run.smithi094.stdout:Need to get 55.8 MB of archives. 2024-09-10T21:58:35.795 INFO:teuthology.orchestra.run.smithi094.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-10T21:58:35.795 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-10T21:58:35.842 INFO:teuthology.orchestra.run.smithi066.stdout:Reading package lists... 2024-09-10T21:58:35.925 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-10T21:58:35.929 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-10T21:58:35.956 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-10T21:58:35.956 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-10T21:58:35.957 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-10T21:58:35.964 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-10T21:58:35.965 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-10T21:58:35.966 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-10T21:58:35.968 INFO:teuthology.orchestra.run.smithi094.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-10T21:58:35.968 INFO:teuthology.orchestra.run.smithi094.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-10T21:58:35.969 INFO:teuthology.orchestra.run.smithi094.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-10T21:58:35.977 INFO:teuthology.orchestra.run.smithi094.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-10T21:58:35.977 INFO:teuthology.orchestra.run.smithi094.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-10T21:58:35.978 INFO:teuthology.orchestra.run.smithi094.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-10T21:58:35.998 INFO:teuthology.orchestra.run.smithi094.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-10T21:58:36.029 INFO:teuthology.orchestra.run.smithi066.stdout:Building dependency tree... 2024-09-10T21:58:36.029 INFO:teuthology.orchestra.run.smithi066.stdout:Reading state information... 2024-09-10T21:58:36.206 INFO:teuthology.orchestra.run.smithi094.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-10T21:58:36.208 INFO:teuthology.orchestra.run.smithi094.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-10T21:58:36.208 INFO:teuthology.orchestra.run.smithi094.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-10T21:58:36.211 INFO:teuthology.orchestra.run.smithi094.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-10T21:58:36.211 INFO:teuthology.orchestra.run.smithi094.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-10T21:58:36.215 INFO:teuthology.orchestra.run.smithi094.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-10T21:58:36.217 INFO:teuthology.orchestra.run.smithi094.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-10T21:58:36.219 INFO:teuthology.orchestra.run.smithi066.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-10T21:58:36.220 INFO:teuthology.orchestra.run.smithi066.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-10T21:58:36.220 INFO:teuthology.orchestra.run.smithi066.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-10T21:58:36.220 INFO:teuthology.orchestra.run.smithi066.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:58:36.220 INFO:teuthology.orchestra.run.smithi066.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:58:36.220 INFO:teuthology.orchestra.run.smithi066.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:58:36.220 INFO:teuthology.orchestra.run.smithi066.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:58:36.221 INFO:teuthology.orchestra.run.smithi066.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:58:36.221 INFO:teuthology.orchestra.run.smithi066.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:58:36.221 INFO:teuthology.orchestra.run.smithi066.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:58:36.221 INFO:teuthology.orchestra.run.smithi066.stdout:The following additional packages will be installed: 2024-09-10T21:58:36.221 INFO:teuthology.orchestra.run.smithi066.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-10T21:58:36.222 INFO:teuthology.orchestra.run.smithi066.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-10T21:58:36.222 INFO:teuthology.orchestra.run.smithi066.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-10T21:58:36.223 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-10T21:58:36.223 INFO:teuthology.orchestra.run.smithi066.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:58:36.223 INFO:teuthology.orchestra.run.smithi066.stdout:Suggested packages: 2024-09-10T21:58:36.224 INFO:teuthology.orchestra.run.smithi066.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-10T21:58:36.224 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-10T21:58:36.224 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-10T21:58:36.224 INFO:teuthology.orchestra.run.smithi094.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-10T21:58:36.224 INFO:teuthology.orchestra.run.smithi094.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-10T21:58:36.246 INFO:teuthology.orchestra.run.smithi094.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-10T21:58:36.246 INFO:teuthology.orchestra.run.smithi094.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-10T21:58:36.256 INFO:teuthology.orchestra.run.smithi094.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-10T21:58:36.276 INFO:teuthology.orchestra.run.smithi094.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-10T21:58:36.279 INFO:teuthology.orchestra.run.smithi094.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-10T21:58:36.280 INFO:teuthology.orchestra.run.smithi094.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-10T21:58:36.281 INFO:teuthology.orchestra.run.smithi094.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-10T21:58:36.281 INFO:teuthology.orchestra.run.smithi094.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-10T21:58:36.282 INFO:teuthology.orchestra.run.smithi066.stdout:The following NEW packages will be installed: 2024-09-10T21:58:36.282 INFO:teuthology.orchestra.run.smithi066.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-10T21:58:36.282 INFO:teuthology.orchestra.run.smithi066.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-10T21:58:36.282 INFO:teuthology.orchestra.run.smithi066.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-10T21:58:36.283 INFO:teuthology.orchestra.run.smithi066.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-10T21:58:36.283 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-10T21:58:36.283 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-10T21:58:36.283 INFO:teuthology.orchestra.run.smithi066.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:58:36.284 INFO:teuthology.orchestra.run.smithi066.stdout:The following packages will be upgraded: 2024-09-10T21:58:36.284 INFO:teuthology.orchestra.run.smithi066.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-10T21:58:36.292 INFO:teuthology.orchestra.run.smithi094.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-10T21:58:36.294 INFO:teuthology.orchestra.run.smithi094.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-10T21:58:36.341 INFO:teuthology.orchestra.run.smithi066.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-10T21:58:36.341 INFO:teuthology.orchestra.run.smithi066.stdout:Need to get 55.8 MB of archives. 2024-09-10T21:58:36.341 INFO:teuthology.orchestra.run.smithi066.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-10T21:58:36.342 INFO:teuthology.orchestra.run.smithi066.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-10T21:58:36.438 INFO:teuthology.orchestra.run.smithi094.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-10T21:58:36.438 INFO:teuthology.orchestra.run.smithi094.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-10T21:58:36.438 INFO:teuthology.orchestra.run.smithi094.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-10T21:58:36.438 INFO:teuthology.orchestra.run.smithi094.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-10T21:58:36.439 INFO:teuthology.orchestra.run.smithi094.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-10T21:58:36.439 INFO:teuthology.orchestra.run.smithi094.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-10T21:58:36.451 INFO:teuthology.orchestra.run.smithi094.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-10T21:58:36.452 INFO:teuthology.orchestra.run.smithi094.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-10T21:58:36.474 INFO:teuthology.orchestra.run.smithi066.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-10T21:58:36.478 INFO:teuthology.orchestra.run.smithi066.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-10T21:58:36.506 INFO:teuthology.orchestra.run.smithi066.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-10T21:58:36.506 INFO:teuthology.orchestra.run.smithi066.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-10T21:58:36.507 INFO:teuthology.orchestra.run.smithi066.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-10T21:58:36.514 INFO:teuthology.orchestra.run.smithi066.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-10T21:58:36.515 INFO:teuthology.orchestra.run.smithi066.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-10T21:58:36.516 INFO:teuthology.orchestra.run.smithi066.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-10T21:58:36.517 INFO:teuthology.orchestra.run.smithi066.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-10T21:58:36.518 INFO:teuthology.orchestra.run.smithi066.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-10T21:58:36.519 INFO:teuthology.orchestra.run.smithi066.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-10T21:58:36.527 INFO:teuthology.orchestra.run.smithi066.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-10T21:58:36.527 INFO:teuthology.orchestra.run.smithi066.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-10T21:58:36.527 INFO:teuthology.orchestra.run.smithi066.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-10T21:58:36.544 INFO:teuthology.orchestra.run.smithi066.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-10T21:58:36.756 INFO:teuthology.orchestra.run.smithi066.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-10T21:58:36.758 INFO:teuthology.orchestra.run.smithi066.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-10T21:58:36.759 INFO:teuthology.orchestra.run.smithi066.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-10T21:58:36.761 INFO:teuthology.orchestra.run.smithi066.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-10T21:58:36.762 INFO:teuthology.orchestra.run.smithi066.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-10T21:58:36.766 INFO:teuthology.orchestra.run.smithi066.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-10T21:58:36.768 INFO:teuthology.orchestra.run.smithi066.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-10T21:58:36.775 INFO:teuthology.orchestra.run.smithi066.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-10T21:58:36.775 INFO:teuthology.orchestra.run.smithi066.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-10T21:58:36.797 INFO:teuthology.orchestra.run.smithi066.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-10T21:58:36.797 INFO:teuthology.orchestra.run.smithi066.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-10T21:58:36.807 INFO:teuthology.orchestra.run.smithi066.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-10T21:58:36.827 INFO:teuthology.orchestra.run.smithi066.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-10T21:58:36.830 INFO:teuthology.orchestra.run.smithi066.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-10T21:58:36.831 INFO:teuthology.orchestra.run.smithi066.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-10T21:58:36.832 INFO:teuthology.orchestra.run.smithi066.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-10T21:58:36.832 INFO:teuthology.orchestra.run.smithi066.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-10T21:58:36.833 INFO:teuthology.orchestra.run.smithi094.stdout:Preconfiguring packages ... 2024-09-10T21:58:36.843 INFO:teuthology.orchestra.run.smithi066.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-10T21:58:36.845 INFO:teuthology.orchestra.run.smithi066.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-10T21:58:36.985 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 55.8 MB in 1s (77.1 MB/s) 2024-09-10T21:58:36.991 INFO:teuthology.orchestra.run.smithi066.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-10T21:58:36.991 INFO:teuthology.orchestra.run.smithi066.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-10T21:58:36.991 INFO:teuthology.orchestra.run.smithi066.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-10T21:58:36.991 INFO:teuthology.orchestra.run.smithi066.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-10T21:58:36.991 INFO:teuthology.orchestra.run.smithi066.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-10T21:58:36.991 INFO:teuthology.orchestra.run.smithi066.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-10T21:58:37.004 INFO:teuthology.orchestra.run.smithi066.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-10T21:58:37.005 INFO:teuthology.orchestra.run.smithi066.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-10T21:58:37.270 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:58:37.274 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:58:37.409 INFO:teuthology.orchestra.run.smithi066.stdout:Preconfiguring packages ... 2024-09-10T21:58:37.416 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:58:37.565 INFO:teuthology.orchestra.run.smithi066.stdout:Fetched 55.8 MB in 1s (78.3 MB/s) 2024-09-10T21:58:37.851 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:58:37.855 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:58:37.969 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:58:38.057 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-10T21:58:38.636 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-10T21:58:39.113 INFO:teuthology.orchestra.run.smithi094.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-10T21:58:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package flex. 2024-09-10T21:58:39.441 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:58:39.445 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-10T21:58:39.495 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-10T21:58:39.750 INFO:teuthology.orchestra.run.smithi066.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-10T21:58:39.981 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:58:40.049 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package flex. 2024-09-10T21:58:40.122 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:58:40.126 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-10T21:58:40.172 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-10T21:58:40.175 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-10T21:58:40.523 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:58:40.776 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:58:41.137 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-10T21:58:41.162 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:58:41.166 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:58:41.324 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:58:41.571 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:58:41.737 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:58:41.906 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:58:42.152 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:58:42.155 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:58:42.222 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:58:42.269 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:58:42.606 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package bison. 2024-09-10T21:58:42.682 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:58:42.683 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:58:42.689 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-10T21:58:42.725 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:58:42.809 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:58:43.116 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:58:43.159 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-10T21:58:43.174 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-10T21:58:43.217 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:58:43.486 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package bison. 2024-09-10T21:58:43.545 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package dump. 2024-09-10T21:58:43.561 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-10T21:58:43.563 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:58:43.567 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-10T21:58:43.603 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking dump (0.4b47-3) ... 2024-09-10T21:58:43.613 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:58:43.966 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package indent. 2024-09-10T21:58:43.972 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-10T21:58:43.982 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:58:43.987 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-10T21:58:44.025 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:58:44.030 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:58:44.277 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-10T21:58:44.293 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-10T21:58:44.343 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:58:44.400 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package dump. 2024-09-10T21:58:44.416 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-10T21:58:44.458 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking dump (0.4b47-3) ... 2024-09-10T21:58:44.639 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-10T21:58:44.655 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-10T21:58:44.697 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:58:44.753 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package indent. 2024-09-10T21:58:44.770 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:58:44.811 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:58:44.967 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-10T21:58:44.982 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-10T21:58:45.025 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:58:45.073 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-10T21:58:45.089 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-10T21:58:45.148 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:58:45.361 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-10T21:58:45.377 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-10T21:58:45.420 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:58:45.443 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-10T21:58:45.460 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-10T21:58:45.493 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:58:45.697 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-10T21:58:45.713 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-10T21:58:45.746 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-10T21:58:45.755 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:58:45.762 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-10T21:58:45.804 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:58:45.983 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libjson-perl. 2024-09-10T21:58:45.999 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-10T21:58:46.033 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-10T21:58:46.157 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-10T21:58:46.174 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-10T21:58:46.216 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:58:46.378 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-10T21:58:46.394 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-10T21:58:46.436 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:58:46.460 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-10T21:58:46.476 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-10T21:58:46.518 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:58:46.746 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libjson-perl. 2024-09-10T21:58:46.762 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-10T21:58:46.804 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-10T21:58:47.149 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-10T21:58:47.165 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-10T21:58:47.207 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:58:47.799 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-10T21:58:47.821 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:58:47.856 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:58:48.133 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-10T21:58:48.149 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:58:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:58:48.519 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-10T21:58:48.535 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:58:48.560 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-10T21:58:48.577 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:58:48.577 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:58:48.618 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:58:48.862 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-10T21:58:48.878 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:58:48.912 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:58:48.938 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-10T21:58:48.955 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-10T21:58:48.996 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:58:49.290 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-10T21:58:49.307 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:58:49.348 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:58:49.685 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-10T21:58:49.702 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-10T21:58:49.734 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-10T21:58:49.743 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:58:49.749 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:58:49.783 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:58:50.087 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-10T21:58:50.103 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:58:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-10T21:58:50.137 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:58:50.153 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:58:50.194 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:58:50.473 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-10T21:58:50.489 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:58:50.540 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:58:50.547 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-10T21:58:50.564 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:58:50.605 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:58:50.867 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-10T21:58:50.883 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:58:50.901 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-10T21:58:50.917 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:58:50.925 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:58:50.959 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:58:51.152 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-10T21:58:51.169 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:58:51.202 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:58:51.254 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-10T21:58:51.271 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:58:51.312 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:58:51.523 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-10T21:58:51.540 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:58:51.581 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:58:51.697 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-10T21:58:51.713 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-10T21:58:51.772 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:58:52.134 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package modemmanager. 2024-09-10T21:58:52.143 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-10T21:58:52.150 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:58:52.159 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-10T21:58:52.192 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:58:52.201 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:58:52.571 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package modemmanager. 2024-09-10T21:58:52.588 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:58:52.629 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:58:52.815 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package network-manager. 2024-09-10T21:58:52.831 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:58:52.881 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:58:53.261 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package network-manager. 2024-09-10T21:58:53.278 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:58:53.328 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:58:53.564 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ppp. 2024-09-10T21:58:53.581 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-10T21:58:53.919 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:58:54.001 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ppp. 2024-09-10T21:58:54.019 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-10T21:58:54.229 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package pptp-linux. 2024-09-10T21:58:54.244 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-10T21:58:54.279 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-10T21:58:54.371 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:58:54.659 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-10T21:58:54.674 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-10T21:58:54.717 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-10T21:58:54.731 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package pptp-linux. 2024-09-10T21:58:54.746 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-10T21:58:54.789 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-10T21:58:55.025 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-10T21:58:55.041 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-10T21:58:55.054 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-10T21:58:55.070 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-10T21:58:55.100 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-10T21:58:55.137 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-client-common (238) ... 2024-09-10T21:58:55.370 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-10T21:58:55.386 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-10T21:58:55.428 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql-client-common (238) ... 2024-09-10T21:58:55.450 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-10T21:58:55.466 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:58:55.500 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:58:55.715 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-10T21:58:55.731 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:58:55.773 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:58:56.114 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-common. 2024-09-10T21:58:56.129 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-10T21:58:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-10T21:58:56.221 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-common (238) ... 2024-09-10T21:58:56.378 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql-common. 2024-09-10T21:58:56.394 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-10T21:58:56.455 INFO:teuthology.orchestra.run.smithi066.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-10T21:58:56.485 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql-common (238) ... 2024-09-10T21:58:56.723 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-14. 2024-09-10T21:58:56.738 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:58:56.783 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:58:56.951 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql-14. 2024-09-10T21:58:56.968 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:58:57.012 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:58:57.961 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql. 2024-09-10T21:58:57.976 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-10T21:58:58.019 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql (14+238) ... 2024-09-10T21:58:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql. 2024-09-10T21:58:58.187 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-10T21:58:58.221 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql (14+238) ... 2024-09-10T21:58:58.263 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-client. 2024-09-10T21:58:58.278 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-10T21:58:58.322 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-client (14+238) ... 2024-09-10T21:58:58.448 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql-client. 2024-09-10T21:58:58.469 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-10T21:58:58.506 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql-client (14+238) ... 2024-09-10T21:58:58.591 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-10T21:58:58.607 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-10T21:58:58.649 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-10T21:58:58.767 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-10T21:58:58.783 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-10T21:58:58.825 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-10T21:58:58.944 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package tcl8.6. 2024-09-10T21:58:58.960 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-10T21:58:59.029 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:58:59.161 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package tcl8.6. 2024-09-10T21:58:59.177 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-10T21:58:59.219 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:58:59.322 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package tcl. 2024-09-10T21:58:59.338 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-10T21:58:59.385 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-10T21:58:59.530 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package tcl. 2024-09-10T21:58:59.547 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-10T21:58:59.592 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-10T21:58:59.768 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package wpasupplicant. 2024-09-10T21:58:59.784 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-10T21:58:59.817 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:59:00.164 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-10T21:59:00.179 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-10T21:59:00.256 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-10T21:59:00.407 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package wpasupplicant. 2024-09-10T21:59:00.424 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-10T21:59:00.465 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:59:00.634 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-10T21:59:00.650 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-10T21:59:00.700 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:59:00.803 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-10T21:59:00.820 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-10T21:59:00.861 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-10T21:59:01.003 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-client-common (238) ... 2024-09-10T21:59:01.216 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-10T21:59:01.264 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-10T21:59:01.282 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-10T21:59:01.331 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:59:01.351 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:59:01.536 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:59:01.610 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql-client-common (238) ... 2024-09-10T21:59:01.645 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:59:01.755 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:59:01.861 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-10T21:59:01.881 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:59:02.021 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:59:02.147 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:59:02.273 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:59:02.399 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:59:02.533 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:59:02.834 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-10T21:59:02.938 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:59:03.299 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:59:03.300 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:59:03.505 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-10T21:59:03.605 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:59:03.958 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:59:04.016 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:59:04.016 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:59:04.084 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:59:04.219 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:59:04.345 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-10T21:59:04.446 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:59:04.614 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:59:04.733 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:59:04.757 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:59:04.859 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:59:04.985 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:59:04.991 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:59:05.119 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-10T21:59:05.220 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:59:05.346 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:59:05.497 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:59:05.656 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:59:05.723 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:59:05.816 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up dump (0.4b47-3) ... 2024-09-10T21:59:05.911 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-10T21:59:05.976 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:59:06.102 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:59:06.228 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-10T21:59:06.371 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-10T21:59:06.404 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:59:06.489 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:59:06.513 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up dump (0.4b47-3) ... 2024-09-10T21:59:06.579 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-10T21:59:06.608 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-10T21:59:06.649 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:59:06.673 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:59:06.775 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:59:06.799 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:59:06.933 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-10T21:59:06.969 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:59:07.059 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-10T21:59:07.095 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:59:07.210 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:59:07.213 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-10T21:59:07.300 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-10T21:59:07.364 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-client (14+238) ... 2024-09-10T21:59:07.370 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:59:07.479 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:59:07.490 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:59:07.639 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:59:07.765 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:59:07.890 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-10T21:59:08.084 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql-client (14+238) ... 2024-09-10T21:59:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-10T21:59:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:which means they are currently configured by ifupdown: 2024-09-10T21:59:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:- enp3s0f1 2024-09-10T21:59:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-10T21:59:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:remove their configuration from /etc/network/interfaces. 2024-09-10T21:59:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:08.210 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:59:08.236 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-10T21:59:08.685 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-10T21:59:08.771 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:08.771 INFO:teuthology.orchestra.run.smithi066.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-10T21:59:08.771 INFO:teuthology.orchestra.run.smithi066.stdout:which means they are currently configured by ifupdown: 2024-09-10T21:59:08.771 INFO:teuthology.orchestra.run.smithi066.stdout:- enp3s0f1 2024-09-10T21:59:08.771 INFO:teuthology.orchestra.run.smithi066.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-10T21:59:08.771 INFO:teuthology.orchestra.run.smithi066.stdout:remove their configuration from /etc/network/interfaces. 2024-09-10T21:59:08.771 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:08.913 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-10T21:59:09.119 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-10T21:59:09.321 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-10T21:59:09.726 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-10T21:59:10.198 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:59:10.400 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:59:10.616 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-10T21:59:10.654 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:59:10.788 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:59:10.907 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-common (238) ... 2024-09-10T21:59:10.914 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-10T21:59:11.040 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql-common (238) ... 2024-09-10T21:59:11.951 INFO:teuthology.orchestra.run.smithi094.stdout:Adding user postgres to group ssl-cert 2024-09-10T21:59:12.037 INFO:teuthology.orchestra.run.smithi066.stdout:Adding user postgres to group ssl-cert 2024-09-10T21:59:12.185 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:12.185 INFO:teuthology.orchestra.run.smithi094.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-10T21:59:12.285 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:12.285 INFO:teuthology.orchestra.run.smithi066.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-10T21:59:12.429 INFO:teuthology.orchestra.run.smithi094.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-10T21:59:12.430 INFO:teuthology.orchestra.run.smithi094.stdout:Removing obsolete dictionary files: 2024-09-10T21:59:12.585 INFO:teuthology.orchestra.run.smithi066.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-10T21:59:12.585 INFO:teuthology.orchestra.run.smithi066.stdout:Removing obsolete dictionary files: 2024-09-10T21:59:13.163 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-10T21:59:13.321 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-10T21:59:13.613 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:59:13.808 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:59:13.989 INFO:teuthology.orchestra.run.smithi094.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-10T21:59:14.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-10T21:59:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-10T21:59:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:This user must also own the server process. 2024-09-10T21:59:14.069 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:The database cluster will be initialized with locale "en_US". 2024-09-10T21:59:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-10T21:59:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:The default text search configuration will be set to "english". 2024-09-10T21:59:14.069 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:Data page checksums are disabled. 2024-09-10T21:59:14.070 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-10T21:59:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:creating subdirectories ... ok 2024-09-10T21:59:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:selecting dynamic shared memory implementation ... posix 2024-09-10T21:59:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:selecting default max_connections ... 100 2024-09-10T21:59:14.127 INFO:teuthology.orchestra.run.smithi094.stdout:selecting default shared_buffers ... 128MB 2024-09-10T21:59:14.141 INFO:teuthology.orchestra.run.smithi094.stdout:selecting default time zone ... Etc/UTC 2024-09-10T21:59:14.142 INFO:teuthology.orchestra.run.smithi094.stdout:creating configuration files ... ok 2024-09-10T21:59:14.164 INFO:teuthology.orchestra.run.smithi066.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-10T21:59:14.182 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-10T21:59:14.239 INFO:teuthology.orchestra.run.smithi066.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-10T21:59:14.240 INFO:teuthology.orchestra.run.smithi066.stdout:This user must also own the server process. 2024-09-10T21:59:14.240 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:14.240 INFO:teuthology.orchestra.run.smithi066.stdout:The database cluster will be initialized with locale "en_US". 2024-09-10T21:59:14.240 INFO:teuthology.orchestra.run.smithi066.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-10T21:59:14.240 INFO:teuthology.orchestra.run.smithi066.stdout:The default text search configuration will be set to "english". 2024-09-10T21:59:14.240 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:14.240 INFO:teuthology.orchestra.run.smithi066.stdout:Data page checksums are disabled. 2024-09-10T21:59:14.240 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:14.240 INFO:teuthology.orchestra.run.smithi066.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-10T21:59:14.241 INFO:teuthology.orchestra.run.smithi066.stdout:creating subdirectories ... ok 2024-09-10T21:59:14.241 INFO:teuthology.orchestra.run.smithi066.stdout:selecting dynamic shared memory implementation ... posix 2024-09-10T21:59:14.265 INFO:teuthology.orchestra.run.smithi066.stdout:selecting default max_connections ... 100 2024-09-10T21:59:14.300 INFO:teuthology.orchestra.run.smithi066.stdout:selecting default shared_buffers ... 128MB 2024-09-10T21:59:14.303 INFO:teuthology.orchestra.run.smithi094.stdout:running bootstrap script ... ok 2024-09-10T21:59:14.314 INFO:teuthology.orchestra.run.smithi066.stdout:selecting default time zone ... Etc/UTC 2024-09-10T21:59:14.315 INFO:teuthology.orchestra.run.smithi066.stdout:creating configuration files ... ok 2024-09-10T21:59:14.486 INFO:teuthology.orchestra.run.smithi066.stdout:running bootstrap script ... ok 2024-09-10T21:59:14.934 INFO:teuthology.orchestra.run.smithi094.stdout:performing post-bootstrap initialization ... ok 2024-09-10T21:59:15.105 INFO:teuthology.orchestra.run.smithi066.stdout:performing post-bootstrap initialization ... ok 2024-09-10T21:59:15.425 INFO:teuthology.orchestra.run.smithi094.stdout:syncing data to disk ... ok 2024-09-10T21:59:15.620 INFO:teuthology.orchestra.run.smithi066.stdout:syncing data to disk ... ok 2024-09-10T21:59:15.937 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-10T21:59:16.135 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-10T21:59:18.889 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:59:18.998 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-10T21:59:19.090 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:59:19.124 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql (14+238) ... 2024-09-10T21:59:19.216 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-10T21:59:19.283 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:59:19.367 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql (14+238) ... 2024-09-10T21:59:19.410 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:59:19.468 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:59:19.594 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:59:19.670 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:59:19.670 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:59:19.863 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:59:19.864 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:59:20.463 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:59:20.555 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-10T21:59:20.588 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:59:20.680 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-10T21:59:20.882 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:59:20.974 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:59:21.076 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:59:21.151 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:59:22.129 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-10T21:59:22.271 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:59:22.578 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-10T21:59:22.679 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:59:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:59:22.825 DEBUG:teuthology.parallel:result is None 2024-09-10T21:59:23.273 INFO:teuthology.orchestra.run.smithi066.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:59:23.278 DEBUG:teuthology.parallel:result is None 2024-09-10T21:59:23.279 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:59:23.430 DEBUG:teuthology.orchestra.run.smithi066:> dpkg-query -W -f '${Version}' ceph 2024-09-10T21:59:23.453 INFO:teuthology.orchestra.run.smithi066.stdout:19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:59:23.453 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:59:23.453 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854-g8f219c47-1jammy is installed. 2024-09-10T21:59:23.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:59:23.610 DEBUG:teuthology.orchestra.run.smithi094:> dpkg-query -W -f '${Version}' ceph 2024-09-10T21:59:23.633 INFO:teuthology.orchestra.run.smithi094.stdout:19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:59:23.633 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:59:23.633 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854-g8f219c47-1jammy is installed. 2024-09-10T21:59:23.635 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T21:59:23.635 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:23.635 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:59:23.655 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:23.656 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:59:23.696 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T21:59:23.696 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:23.696 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:59:23.717 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:59:23.781 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:23.781 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:59:23.801 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:59:23.863 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T21:59:23.863 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:23.863 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:59:23.882 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:59:23.941 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:23.941 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:59:23.961 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:59:24.023 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T21:59:24.023 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:24.023 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:59:24.043 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:59:24.105 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:24.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:59:24.125 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:59:24.187 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T21:59:24.276 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T21:59:24.276 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /var/log/ceph 2024-09-10T21:59:24.280 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /var/log/ceph 2024-09-10T21:59:24.299 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T21:59:24.299 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T21:59:24.341 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T21:59:24.361 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T21:59:24.361 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T21:59:24.401 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T21:59:24.428 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T21:59:24.429 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T21:59:24.429 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T21:59:24.429 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:24.429 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T21:59:24.466 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T21:59:24.556 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:24.557 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T21:59:24.565 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T21:59:24.662 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T21:59:24.662 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T21:59:24.662 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705', 'branch': 'wip-hemanth-testing-2024-09-10-0723', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}}', 'email': 'hyelloji@redhat.com', 'first_in_suite': False, 'job_id': '7899705', 'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-hemanth-testing-2024-09-10-0723'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-hemanth-testing-2024-09-10-0723', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'owner': 'scheduled_hyelloji@teuthology', 'priority': 100, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6190, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'sleep_before_teardown': 0, 'subset': '479/1024', 'suite': 'fs', 'suite_branch': 'wip-hemanth-testing-2024-09-10-0723', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi066.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOM+g29lHpSCxgRzgZ2SkoMjXZ8tkwMtW4Rko10j4iY8exzxkFWClPtAlnFQUgkYDS7K4bBits9Jp9Ff6YoBWEw=', 'smithi094.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMUss4KSGVsNmsqLtCP5YU8u8vTtpGLblwcDOS0jV528pxaQ9clrxUOHT4VidxwBec+XfoC9a2eLL3m77abOXlk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgressReporter']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_16:59:47', 'tube': 'smithi', 'user': 'hyelloji', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T21:59:24.663 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T21:59:24.666 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T21:59:24.675 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T21:59:24.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T21:59:24.739 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:24.739 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:59:24.783 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:59:24.783 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-09-10T21:59:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:59:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:59:24.835 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 6h/6d Inode: 886 Links: 1 2024-09-10T21:59:24.835 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:59:24.835 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 21:52:45.175797888 +0000 2024-09-10T21:59:24.835 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 21:52:44.867800639 +0000 2024-09-10T21:59:24.835 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 21:52:44.867800639 +0000 2024-09-10T21:59:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 21:52:44.867800639 +0000 2024-09-10T21:59:24.835 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:59:24.895 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T21:59:24.896 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T21:59:24.896 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000341541 s, 1.5 MB/s 2024-09-10T21:59:24.897 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:59:24.948 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-09-10T21:59:25.002 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:59:25.003 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:59:25.003 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 6h/6d Inode: 903 Links: 1 2024-09-10T21:59:25.003 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:59:25.003 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 21:52:45.743792814 +0000 2024-09-10T21:59:25.003 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 21:52:45.475795208 +0000 2024-09-10T21:59:25.003 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 21:52:45.475795208 +0000 2024-09-10T21:59:25.003 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 21:52:45.475795208 +0000 2024-09-10T21:59:25.003 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:59:25.063 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T21:59:25.064 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T21:59:25.064 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000338247 s, 1.5 MB/s 2024-09-10T21:59:25.065 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:59:25.116 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-09-10T21:59:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:59:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:59:25.171 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 6h/6d Inode: 922 Links: 1 2024-09-10T21:59:25.171 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:59:25.171 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 21:52:46.315787705 +0000 2024-09-10T21:59:25.171 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 21:52:46.043790135 +0000 2024-09-10T21:59:25.171 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 21:52:46.043790135 +0000 2024-09-10T21:59:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 21:52:46.043790135 +0000 2024-09-10T21:59:25.171 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:59:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T21:59:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T21:59:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000334036 s, 1.5 MB/s 2024-09-10T21:59:25.233 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:59:25.288 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-09-10T21:59:25.343 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:59:25.343 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:59:25.343 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 6h/6d Inode: 940 Links: 1 2024-09-10T21:59:25.343 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:59:25.344 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 21:52:46.887782596 +0000 2024-09-10T21:59:25.344 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 21:52:46.603785133 +0000 2024-09-10T21:59:25.344 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 21:52:46.603785133 +0000 2024-09-10T21:59:25.344 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 21:52:46.603785133 +0000 2024-09-10T21:59:25.344 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:59:25.404 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T21:59:25.404 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T21:59:25.404 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000346557 s, 1.5 MB/s 2024-09-10T21:59:25.405 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:59:25.456 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:59:25.456 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:25.456 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:59:25.465 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:59:25.465 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-09-10T21:59:25.516 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:59:25.516 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:59:25.516 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 902 Links: 1 2024-09-10T21:59:25.516 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:59:25.516 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-10 21:52:42.862520527 +0000 2024-09-10T21:59:25.516 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-10 21:52:42.598522885 +0000 2024-09-10T21:59:25.516 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-10 21:52:42.598522885 +0000 2024-09-10T21:59:25.516 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-10 21:52:42.598522885 +0000 2024-09-10T21:59:25.517 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:59:25.578 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T21:59:25.578 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T21:59:25.578 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000277863 s, 1.8 MB/s 2024-09-10T21:59:25.580 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:59:25.634 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-09-10T21:59:25.688 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:59:25.688 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:59:25.689 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 920 Links: 1 2024-09-10T21:59:25.689 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:59:25.689 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-10 21:52:43.466515132 +0000 2024-09-10T21:59:25.689 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-10 21:52:43.190517598 +0000 2024-09-10T21:59:25.689 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-10 21:52:43.190517598 +0000 2024-09-10T21:59:25.689 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-10 21:52:43.190517598 +0000 2024-09-10T21:59:25.689 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:59:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T21:59:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T21:59:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000275563 s, 1.9 MB/s 2024-09-10T21:59:25.752 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:59:25.806 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-09-10T21:59:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:59:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:59:25.857 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 936 Links: 1 2024-09-10T21:59:25.857 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:59:25.857 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-10 21:52:44.046509951 +0000 2024-09-10T21:59:25.857 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-10 21:52:43.778512346 +0000 2024-09-10T21:59:25.857 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-10 21:52:43.778512346 +0000 2024-09-10T21:59:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-10 21:52:43.778512346 +0000 2024-09-10T21:59:25.857 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:59:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T21:59:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T21:59:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000266398 s, 1.9 MB/s 2024-09-10T21:59:25.920 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:59:25.974 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-09-10T21:59:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:59:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:59:26.024 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 954 Links: 1 2024-09-10T21:59:26.024 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:59:26.024 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-10 21:52:44.570505271 +0000 2024-09-10T21:59:26.024 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-10 21:52:44.318507522 +0000 2024-09-10T21:59:26.024 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-10 21:52:44.318507522 +0000 2024-09-10T21:59:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-10 21:52:44.318507522 +0000 2024-09-10T21:59:26.025 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:59:26.086 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T21:59:26.086 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T21:59:26.086 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000275942 s, 1.9 MB/s 2024-09-10T21:59:26.088 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:59:26.143 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:59:26.144 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T21:59:26.144 INFO:tasks.ceph:Generating config... 2024-09-10T21:59:26.145 INFO:tasks.ceph:[client] pid file = 2024-09-10T21:59:26.145 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-10T21:59:26.145 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T21:59:26.145 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-10T21:59:26.145 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T21:59:26.145 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T21:59:26.145 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T21:59:26.146 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T21:59:26.146 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T21:59:26.146 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T21:59:26.146 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-10T21:59:26.146 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-10T21:59:26.146 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-10T21:59:26.146 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T21:59:26.146 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T21:59:26.146 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-10T21:59:26.146 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-10T21:59:26.146 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T21:59:26.146 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T21:59:26.146 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-10T21:59:26.146 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T21:59:26.147 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T21:59:26.147 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T21:59:26.147 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T21:59:26.147 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T21:59:26.147 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T21:59:26.147 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T21:59:26.192 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T21:59:26.204 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T21:59:26.299 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:59:26.320 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.66'), ('mon.b', '172.21.15.94'), ('mon.c', '[v2:172.21.15.94:3301,v1:172.21.15.94:6790]')] 2024-09-10T21:59:26.321 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-10T21:59:26.322 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:26.322 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T21:59:26.371 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.66 --add b 172.21.15.94 --addv c '[v2:172.21.15.94:3301,v1:172.21.15.94:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:59:26.463 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:59:26.463 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool: generated fsid 944cb5ba-80f3-427e-9dcb-45be8cb10513 2024-09-10T21:59:26.463 INFO:teuthology.orchestra.run.smithi066.stdout:setting min_mon_release = quincy 2024-09-10T21:59:26.463 INFO:teuthology.orchestra.run.smithi066.stdout:epoch 0 2024-09-10T21:59:26.463 INFO:teuthology.orchestra.run.smithi066.stdout:fsid 944cb5ba-80f3-427e-9dcb-45be8cb10513 2024-09-10T21:59:26.463 INFO:teuthology.orchestra.run.smithi066.stdout:last_changed 2024-09-10T21:59:26.460842+0000 2024-09-10T21:59:26.463 INFO:teuthology.orchestra.run.smithi066.stdout:created 2024-09-10T21:59:26.460842+0000 2024-09-10T21:59:26.463 INFO:teuthology.orchestra.run.smithi066.stdout:min_mon_release 17 (quincy) 2024-09-10T21:59:26.464 INFO:teuthology.orchestra.run.smithi066.stdout:election_strategy: 1 2024-09-10T21:59:26.464 INFO:teuthology.orchestra.run.smithi066.stdout:0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-09-10T21:59:26.464 INFO:teuthology.orchestra.run.smithi066.stdout:1: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-09-10T21:59:26.464 INFO:teuthology.orchestra.run.smithi066.stdout:2: [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] mon.c 2024-09-10T21:59:26.464 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T21:59:26.466 DEBUG:teuthology.orchestra.run.smithi066:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T21:59:26.474 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 944cb5ba-80f3-427e-9dcb-45be8cb10513... 2024-09-10T21:59:26.475 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:59:26.517 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:59:26.557 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-09-10T21:59:26.557 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-09-10T21:59:26.557 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:59:26.557 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-09-10T21:59:26.557 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.557 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-09-10T21:59:26.557 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:59:26.558 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-09-10T21:59:26.559 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 944cb5ba-80f3-427e-9dcb-45be8cb10513 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.560 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-09-10T21:59:26.561 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-09-10T21:59:26.561 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.561 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-09-10T21:59:26.561 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-09-10T21:59:26.561 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:59:26.561 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.561 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-09-10T21:59:26.561 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-09-10T21:59:26.561 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.561 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-09-10T21:59:26.561 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-09-10T21:59:26.561 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-09-10T21:59:26.562 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: bdev async discard = True 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: bdev enable discard = True 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore allocator = bitmap 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 1/20 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 1/20 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 4/10 2024-09-10T21:59:26.563 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-09-10T21:59:26.564 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 30 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: debug finisher = 20 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:59:26.565 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: mon down mkfs grace = 300 2024-09-10T21:59:26.566 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-09-10T21:59:26.567 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-09-10T21:59:26.568 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-09-10T21:59:26.568 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2024-09-10T21:59:26.568 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:59:26.568 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2024-09-10T21:59:26.568 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.568 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2024-09-10T21:59:26.568 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:59:26.569 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.570 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = 944cb5ba-80f3-427e-9dcb-45be8cb10513 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2024-09-10T21:59:26.571 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug trim objects = true 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2024-09-10T21:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: bdev async discard = True 2024-09-10T21:59:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: bdev enable discard = True 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore allocator = bitmap 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 1/20 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 1/20 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 4/10 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2024-09-10T21:59:26.574 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout: client mount timeout = 30 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout: debug finisher = 20 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2024-09-10T21:59:26.575 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: mon down mkfs grace = 300 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2024-09-10T21:59:26.577 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2024-09-10T21:59:26.578 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:59:26.578 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:59:26.578 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = "" 2024-09-10T21:59:26.578 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2024-09-10T21:59:26.578 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2024-09-10T21:59:26.578 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2024-09-10T21:59:26.578 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-09-10T21:59:26.578 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-09-10T21:59:26.578 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-09-10T21:59:26.578 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-09-10T21:59:26.579 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T21:59:26.580 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T21:59:26.681 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T21:59:26.681 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:26.681 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T21:59:26.690 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:26.690 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T21:59:26.738 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T21:59:26.738 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:26.738 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T21:59:26.738 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:59:26.814 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:26.815 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:59:26.870 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T21:59:26.870 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:26.870 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T21:59:26.871 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:59:26.905 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:26.906 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:59:26.956 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T21:59:26.956 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T21:59:26.956 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T21:59:27.013 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T21:59:27.018 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T21:59:27.077 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T21:59:27.097 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T21:59:27.097 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T21:59:27.152 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T21:59:27.157 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T21:59:27.177 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T21:59:27.277 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T21:59:27.281 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T21:59:27.301 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T21:59:27.361 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T21:59:27.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T21:59:27.387 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T21:59:27.486 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T21:59:27.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T21:59:27.509 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T21:59:27.510 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T21:59:27.553 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T21:59:27.570 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-10T21:59:27.652 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-10T21:59:27.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T21:59:27.712 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-10T21:59:27.728 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-10T21:59:27.816 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-10T21:59:27.833 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T21:59:27.833 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T21:59:27.833 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T21:59:27.852 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:59:27.852 INFO:tasks.ceph:role: osd.0 2024-09-10T21:59:27.853 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T21:59:27.853 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T21:59:27.925 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:59:27.925 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:59:27.925 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:59:27.925 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:59:27.925 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:59:27.925 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:59:27.925 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:59:27.925 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:59:27.926 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:59:27.926 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:59:28.338 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-09-10T21:59:28.341 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-09-10T21:59:28.341 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T21:59:28.484 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T21:59:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:59:28.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:59:28.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T21:59:28.562 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:59:28.562 INFO:tasks.ceph:role: osd.1 2024-09-10T21:59:28.562 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T21:59:28.562 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T21:59:28.629 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:59:28.629 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:59:28.629 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:59:28.630 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:59:28.630 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:59:28.630 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:59:28.630 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:59:28.630 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:59:28.630 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:59:28.630 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:59:28.991 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-09-10T21:59:28.994 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-09-10T21:59:28.994 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T21:59:29.066 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T21:59:29.127 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:59:29.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:59:29.128 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T21:59:29.196 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:59:29.196 INFO:tasks.ceph:role: osd.2 2024-09-10T21:59:29.197 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T21:59:29.197 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T21:59:29.266 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:59:29.266 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:59:29.266 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:59:29.266 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:59:29.266 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:59:29.266 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:59:29.266 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:59:29.266 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:59:29.266 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:59:29.266 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:59:29.660 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-09-10T21:59:29.663 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-09-10T21:59:29.663 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T21:59:29.767 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T21:59:29.784 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:59:29.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:59:29.785 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T21:59:29.851 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:59:29.851 INFO:tasks.ceph:role: osd.3 2024-09-10T21:59:29.852 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T21:59:29.852 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T21:59:29.925 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:59:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:59:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:59:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:59:29.925 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:59:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:59:29.925 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:59:29.925 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:59:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:59:29.925 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:59:30.315 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-09-10T21:59:30.317 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-09-10T21:59:30.318 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T21:59:30.459 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T21:59:30.476 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:59:30.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:59:30.478 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:59:30.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:30.604+0000 7fd6823197c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T21:59:30.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:30.608+0000 7fd6823197c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T21:59:30.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:30.608+0000 7fd6823197c0 -1 bdev(0x56240383d800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T21:59:30.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:30.608+0000 7fd6823197c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T21:59:33.878 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T21:59:33.899 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:59:33.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:33.992+0000 7f6913ab87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T21:59:33.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:33.996+0000 7f6913ab87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T21:59:33.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:33.996+0000 7f6913ab87c0 -1 bdev(0x561b811c9800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T21:59:34.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:33.996+0000 7f6913ab87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T21:59:37.210 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T21:59:37.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:59:37.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:37.320+0000 7fd86cb967c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T21:59:37.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:37.324+0000 7fd86cb967c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T21:59:37.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:37.324+0000 7fd86cb967c0 -1 bdev(0x55da19b7f800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T21:59:37.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:37.324+0000 7fd86cb967c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T21:59:40.946 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T21:59:40.966 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:59:41.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:41.056+0000 7fc7841197c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T21:59:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:41.056+0000 7fc7841197c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T21:59:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:41.056+0000 7fc7841197c0 -1 bdev(0x561d161cd800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-10T21:59:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T21:59:41.056+0000 7fc7841197c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-10T21:59:44.321 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T21:59:44.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T21:59:44.364 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:59:44.364 INFO:tasks.ceph:role: osd.4 2024-09-10T21:59:44.364 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T21:59:44.365 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T21:59:44.428 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:59:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:59:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:59:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:59:44.428 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:59:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:59:44.428 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:59:44.428 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:59:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:59:44.429 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:59:44.457 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-10T21:59:44.460 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-10T21:59:44.460 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T21:59:44.529 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T21:59:44.591 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:59:44.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:59:44.592 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T21:59:44.654 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:59:44.655 INFO:tasks.ceph:role: osd.5 2024-09-10T21:59:44.655 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T21:59:44.655 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T21:59:44.714 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:59:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:59:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:59:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:59:44.714 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:59:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:59:44.715 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:59:44.715 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:59:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:59:44.715 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:59:44.743 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-10T21:59:44.746 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-10T21:59:44.746 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T21:59:44.815 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T21:59:44.878 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:59:44.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:59:44.879 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T21:59:44.938 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:59:44.938 INFO:tasks.ceph:role: osd.6 2024-09-10T21:59:44.938 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T21:59:44.938 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T21:59:44.998 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:59:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:59:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:59:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:59:44.998 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:59:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:59:44.998 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:59:44.998 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:59:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:59:44.998 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:59:45.028 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-10T21:59:45.030 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-10T21:59:45.030 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T21:59:45.100 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T21:59:45.162 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:59:45.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:59:45.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T21:59:45.230 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:59:45.230 INFO:tasks.ceph:role: osd.7 2024-09-10T21:59:45.230 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T21:59:45.230 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T21:59:45.294 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:59:45.294 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:59:45.294 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:59:45.295 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:59:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:59:45.295 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:59:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:59:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:59:45.295 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:59:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:59:45.323 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-10T21:59:45.326 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-10T21:59:45.326 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T21:59:45.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T21:59:45.471 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:59:45.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:59:45.472 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:59:45.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:45.619+0000 7f3e690547c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T21:59:45.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:45.623+0000 7f3e690547c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T21:59:45.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:45.623+0000 7f3e690547c0 -1 bdev(0x55e61936f800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-10T21:59:45.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:45.623+0000 7f3e690547c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-10T21:59:48.855 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T21:59:48.876 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:59:48.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:48.967+0000 7f592170b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T21:59:48.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:48.967+0000 7f592170b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T21:59:48.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:48.967+0000 7f592170b7c0 -1 bdev(0x560b861d3800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-10T21:59:48.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:48.967+0000 7f592170b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-10T21:59:52.210 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T21:59:52.230 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:59:52.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:52.323+0000 7f76acb7c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T21:59:52.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:52.323+0000 7f76acb7c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T21:59:52.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:52.323+0000 7f76acb7c7c0 -1 bdev(0x5620dc0e3800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-10T21:59:52.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:52.323+0000 7f76acb7c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-10T21:59:54.665 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:59:54.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:59:55.577 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T21:59:55.598 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:59:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:55.691+0000 7fcd89d2b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T21:59:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:55.691+0000 7fcd89d2b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T21:59:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:55.691+0000 7fcd89d2b7c0 -1 bdev(0x558e33775800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-10T21:59:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T21:59:55.691+0000 7fcd89d2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-10T21:59:58.906 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T21:59:58.927 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T21:59:58.928 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:58.928 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T21:59:58.950 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:58.951 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T21:59:59.019 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:59.020 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T21:59:59.087 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:59.087 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T21:59:59.151 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:59.151 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T21:59:59.215 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:59.216 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T21:59:59.279 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:59.279 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T21:59:59.343 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:59.343 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T21:59:59.365 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:59.365 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T21:59:59.432 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:59.432 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T21:59:59.495 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:59.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T21:59:59.560 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:59.560 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T21:59:59.624 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:59.624 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T21:59:59.692 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:59.692 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T21:59:59.760 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:59.760 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T21:59:59.769 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T21:59:59.769 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-10T21:59:59.825 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:59.826 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-10T21:59:59.835 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T21:59:59.836 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-10T21:59:59.888 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T21:59:59.888 DEBUG:teuthology.orchestra.run.smithi066:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T21:59:59.891 DEBUG:teuthology.orchestra.run.smithi094:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T21:59:59.936 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.y] 2024-09-10T21:59:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmItWpABAAlJ+6dFjvtBV3g+jYmUh0tQ== 2024-09-10T21:59:59.936 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-09-10T21:59:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmxQD7CBAAttZP0KtdjTI7QWbD7CVinQ== 2024-09-10T21:59:59.937 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-09-10T21:59:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmQnVoEBAAvsZ/TSOOP/JgJBr/aHwI/g== 2024-09-10T21:59:59.937 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.0] 2024-09-10T21:59:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBCweBmDJkxJBAAVSQNiMAQl5bR3RUwVOuGYw== 2024-09-10T21:59:59.937 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.1] 2024-09-10T21:59:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBFweBmwcdsOxAAPPQyzktcfiF8pJNP8uAjwg== 2024-09-10T21:59:59.938 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.2] 2024-09-10T21:59:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBJweBmXQhZExAAnEf8mjGX6nnYwWygKjo/2A== 2024-09-10T21:59:59.938 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.3] 2024-09-10T21:59:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBNweBmlauHAxAA/k9xRqbHZsqJq9OYbJKfNQ== 2024-09-10T21:59:59.938 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.x] 2024-09-10T21:59:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmLiy1BBAADnraML8IUet7Sw0j83TUVQ== 2024-09-10T21:59:59.938 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-09-10T21:59:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmI7ufFRAANelhvqylKyfGPVfBuY40kg== 2024-09-10T21:59:59.938 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmRQkTHRAAHPIdwqtkkyh31D7IO2vxPA== 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.4] 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBRweBmHnFDJRAAgF1ZAPvmoNzeUftZQyL0eg== 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.5] 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBUweBm/hXpORAA6jpOI/I+QyB2mZsEoAWF0w== 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.6] 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBYweBmYu2MExAAPD9WcxrBLzwwJQm19z7YcQ== 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.7] 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBbweBmXfJ9KRAA/AbWlli+Qfg7gy0SblJ7fg== 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw== 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout:[client.2] 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmQ47KJhAAI50ZaSQIhH6FxRM48JZICA== 2024-09-10T21:59:59.939 INFO:teuthology.orchestra.run.smithi066.stdout:[client.1] 2024-09-10T21:59:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmbS6IKhAAaEmK9irxMipALgkAct/+OQ== 2024-09-10T21:59:59.940 INFO:teuthology.orchestra.run.smithi066.stdout:[client.3] 2024-09-10T21:59:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmUcLCMBAAdf4tfcpVaZoaKjS1UzrxhA== 2024-09-10T21:59:59.950 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.y] 2024-09-10T21:59:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA/weBmItWpABAAlJ+6dFjvtBV3g+jYmUh0tQ== 2024-09-10T21:59:59.950 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-09-10T21:59:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA/weBmxQD7CBAAttZP0KtdjTI7QWbD7CVinQ== 2024-09-10T21:59:59.950 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-09-10T21:59:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA/weBmQnVoEBAAvsZ/TSOOP/JgJBr/aHwI/g== 2024-09-10T21:59:59.950 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.0] 2024-09-10T21:59:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBCweBmDJkxJBAAVSQNiMAQl5bR3RUwVOuGYw== 2024-09-10T21:59:59.950 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.1] 2024-09-10T21:59:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBFweBmwcdsOxAAPPQyzktcfiF8pJNP8uAjwg== 2024-09-10T21:59:59.951 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.2] 2024-09-10T21:59:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBJweBmXQhZExAAnEf8mjGX6nnYwWygKjo/2A== 2024-09-10T21:59:59.951 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.3] 2024-09-10T21:59:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBNweBmlauHAxAA/k9xRqbHZsqJq9OYbJKfNQ== 2024-09-10T21:59:59.951 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.x] 2024-09-10T21:59:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA/weBmLiy1BBAADnraML8IUet7Sw0j83TUVQ== 2024-09-10T21:59:59.951 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-09-10T21:59:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA/weBmI7ufFRAANelhvqylKyfGPVfBuY40kg== 2024-09-10T21:59:59.951 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-09-10T21:59:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA/weBmRQkTHRAAHPIdwqtkkyh31D7IO2vxPA== 2024-09-10T21:59:59.952 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.4] 2024-09-10T21:59:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBRweBmHnFDJRAAgF1ZAPvmoNzeUftZQyL0eg== 2024-09-10T21:59:59.952 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.5] 2024-09-10T21:59:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBUweBm/hXpORAA6jpOI/I+QyB2mZsEoAWF0w== 2024-09-10T21:59:59.952 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.6] 2024-09-10T21:59:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBYweBmYu2MExAAPD9WcxrBLzwwJQm19z7YcQ== 2024-09-10T21:59:59.952 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.7] 2024-09-10T21:59:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBbweBmXfJ9KRAA/AbWlli+Qfg7gy0SblJ7fg== 2024-09-10T21:59:59.952 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T21:59:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw== 2024-09-10T21:59:59.953 INFO:teuthology.orchestra.run.smithi094.stdout:[client.2] 2024-09-10T21:59:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA/weBmQ47KJhAAI50ZaSQIhH6FxRM48JZICA== 2024-09-10T21:59:59.953 INFO:teuthology.orchestra.run.smithi094.stdout:[client.1] 2024-09-10T21:59:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA/weBmbS6IKhAAaEmK9irxMipALgkAct/+OQ== 2024-09-10T21:59:59.953 INFO:teuthology.orchestra.run.smithi094.stdout:[client.3] 2024-09-10T21:59:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA/weBmUcLCMBAAdf4tfcpVaZoaKjS1UzrxhA== 2024-09-10T21:59:59.954 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:59:59.985 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T22:00:00.057 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:00:00.061 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:00:00.124 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:00:00.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:00:00.185 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.188 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.241 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.298 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.301 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.353 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.409 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T22:00:00.412 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T22:00:00.465 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:00:00.468 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:00:00.523 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:00:00.526 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:00:00.582 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.585 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.638 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.641 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.695 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.698 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.751 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.754 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:00:00.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:00:00.811 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:00:00.866 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:00:00.870 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:00:00.925 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:00:00.929 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:00:00.985 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:00:00.988 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:00:01.043 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T22:00:01.044 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T22:00:01.064 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T22:00:01.174 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T22:00:01.195 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T22:00:01.215 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T22:00:01.328 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T22:00:01.347 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T22:00:01.410 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T22:00:01.523 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T22:00:01.543 DEBUG:teuthology.orchestra.run.smithi066:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:00:01.546 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:00:01.599 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T22:00:01.599 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T22:00:01.599 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T22:00:01.602 INFO:tasks.ceph.mon.a:Started 2024-09-10T22:00:01.602 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T22:00:01.602 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T22:00:01.642 INFO:tasks.ceph.mon.b:Started 2024-09-10T22:00:01.642 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T22:00:01.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T22:00:01.645 INFO:tasks.ceph.mon.c:Started 2024-09-10T22:00:01.646 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T22:00:01.646 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T22:00:01.646 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T22:00:01.649 INFO:tasks.ceph.mgr.y:Started 2024-09-10T22:00:01.649 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T22:00:01.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T22:00:01.654 INFO:tasks.ceph.mgr.x:Started 2024-09-10T22:00:01.654 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-10T22:00:01.654 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-10T22:00:01.658 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-10T22:00:01.659 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-10T22:00:01.705 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-10T22:00:01.706 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-10T22:00:01.757 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-10T22:00:01.757 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-10T22:00:01.805 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-10T22:00:01.805 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-10T22:00:01.853 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-10T22:00:01.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-10T22:00:01.901 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-10T22:00:01.902 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-10T22:00:01.949 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-10T22:00:01.949 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-10T22:00:01.993 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-10T22:00:01.993 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-10T22:00:02.041 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-10T22:00:02.041 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-10T22:00:02.085 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-10T22:00:02.085 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-10T22:00:02.129 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-10T22:00:02.129 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-10T22:00:02.178 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-10T22:00:02.178 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-10T22:00:02.226 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T22:00:02.226 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-10T22:00:02.278 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T22:00:02.278 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-10T22:00:02.329 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-10T22:00:02.329 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-10T22:00:02.377 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T22:00:02.377 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T22:00:02.426 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-10T22:00:02.426 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-10T22:00:02.477 DEBUG:tasks.ceph:set 18 configs 2024-09-10T22:00:02.477 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi066') 2024-09-10T22:00:07.902 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi066') 2024-09-10T22:00:07.902 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi066') 2024-09-10T22:00:07.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:07.976+0000 7f1bd9600640 1 -- 172.21.15.66:0/4109136577 <== mon.1 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v14)=0 v14) ==== 109+0+0 (secure 0 0 0) 0x7f1bd00a67a0 con 0x7f1bbc003b60 2024-09-10T22:00:07.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:07.980+0000 7f1be4600640 1 -- 172.21.15.66:0/4109136577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc003b60 msgr2=0x7f1bdc0c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:07.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:07.980+0000 7f1be4600640 1 --2- 172.21.15.66:0/4109136577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc003b60 0x7f1bdc0c86c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00ae000 tx=0x7f1bd00a6f90 comp rx=0 tx=0).stop 2024-09-10T22:00:07.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:07.980+0000 7f1be4600640 1 -- 172.21.15.66:0/4109136577 shutdown_connections 2024-09-10T22:00:07.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:07.980+0000 7f1be4600640 1 --2- 172.21.15.66:0/4109136577 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1bbc007610 0x7f1bdc13fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:07.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:07.980+0000 7f1be4600640 1 --2- 172.21.15.66:0/4109136577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1bbc006bf0 0x7f1bdc15cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:07.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:07.980+0000 7f1be4600640 1 --2- 172.21.15.66:0/4109136577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc003b60 0x7f1bdc0c86c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:07.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:07.980+0000 7f1be4600640 1 -- 172.21.15.66:0/4109136577 >> 172.21.15.66:0/4109136577 conn(0x7f1bdc0b26c0 msgr2=0x7f1bdc070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:07.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:07.980+0000 7f1be4600640 1 -- 172.21.15.66:0/4109136577 shutdown_connections 2024-09-10T22:00:07.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:07.980+0000 7f1be4600640 1 -- 172.21.15.66:0/4109136577 wait complete. 2024-09-10T22:00:08.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 -- 172.21.15.66:0/2133323515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb4006200 msgr2=0x7f9fb4008640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 --2- 172.21.15.66:0/2133323515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb4006200 0x7f9fb4008640 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4008bb0 tx=0x7f9fc401d730 comp rx=0 tx=0).stop 2024-09-10T22:00:08.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 -- 172.21.15.66:0/2133323515 shutdown_connections 2024-09-10T22:00:08.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 --2- 172.21.15.66:0/2133323515 >> v2:172.21.15.94:3300/0 conn(0x7f9fb400a940 0x7f9fb400cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 --2- 172.21.15.66:0/2133323515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb4006200 0x7f9fb4008640 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 -- 172.21.15.66:0/2133323515 >> 172.21.15.66:0/2133323515 conn(0x7f9fd40cfcf0 msgr2=0x7f9fd40d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:08.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 -- 172.21.15.66:0/2133323515 shutdown_connections 2024-09-10T22:00:08.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 -- 172.21.15.66:0/2133323515 wait complete. 2024-09-10T22:00:08.006 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 Processor -- start 2024-09-10T22:00:08.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 -- start start 2024-09-10T22:00:08.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fb4003b20 0x7f9fd409c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb4006200 0x7f9fd4153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9fb400a940 0x7f9fd409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4073750 con 0x7f9fb4006200 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd40735d0 con 0x7f9fb4003b20 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.000+0000 7f9fd9c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd40b2ca0 con 0x7f9fb400a940 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fb4003b20 0x7f9fd409c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb4006200 0x7f9fd4153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fb4003b20 0x7f9fd409c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:50088/0 (socket says 172.21.15.66:50088) 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd3e00640 1 -- 172.21.15.66:0/1258763132 learned_addr learned my addr 172.21.15.66:0/1258763132 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd9200640 1 --2- 172.21.15.66:0/1258763132 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9fb400a940 0x7f9fd409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd3e00640 1 -- 172.21.15.66:0/1258763132 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9fb400a940 msgr2=0x7f9fd409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd3e00640 1 --2- 172.21.15.66:0/1258763132 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9fb400a940 0x7f9fd409aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd3e00640 1 -- 172.21.15.66:0/1258763132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb4006200 msgr2=0x7f9fd4153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd3e00640 1 --2- 172.21.15.66:0/1258763132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb4006200 0x7f9fd4153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd3e00640 1 -- 172.21.15.66:0/1258763132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd409c970 con 0x7f9fb4003b20 2024-09-10T22:00:08.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd3400640 1 --2- 172.21.15.66:0/1258763132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb4006200 0x7f9fd4153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:08.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd3e00640 1 --2- 172.21.15.66:0/1258763132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fb4003b20 0x7f9fd409c430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9fc401b360 tx=0x7f9fc40a7f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:08.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fcbe00640 1 -- 172.21.15.66:0/1258763132 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40af030 con 0x7f9fb4003b20 2024-09-10T22:00:08.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fcbe00640 1 -- 172.21.15.66:0/1258763132 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f9fc40b4420 con 0x7f9fb4003b20 2024-09-10T22:00:08.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fcbe00640 1 -- 172.21.15.66:0/1258763132 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40ac040 con 0x7f9fb4003b20 2024-09-10T22:00:08.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd9c00640 1 -- 172.21.15.66:0/1258763132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd40c9510 con 0x7f9fb4003b20 2024-09-10T22:00:08.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd9c00640 1 -- 172.21.15.66:0/1258763132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd40c9ad0 con 0x7f9fb4003b20 2024-09-10T22:00:08.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fcbe00640 1 -- 172.21.15.66:0/1258763132 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f9fc40b47f0 con 0x7f9fb4003b20 2024-09-10T22:00:08.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fcbe00640 1 -- 172.21.15.66:0/1258763132 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9fc40a8810 con 0x7f9fb4003b20 2024-09-10T22:00:08.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.004+0000 7f9fd9c00640 1 -- 172.21.15.66:0/1258763132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd40b30b0 con 0x7f9fb4003b20 2024-09-10T22:00:08.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.008+0000 7f9fcbe00640 1 -- 172.21.15.66:0/1258763132 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f9fd409c970 con 0x7f9fb4003b20 2024-09-10T22:00:08.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.068+0000 7f9fd9c00640 1 -- 172.21.15.66:0/1258763132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f9fd409dee0 con 0x7f9fb4003b20 2024-09-10T22:00:08.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.076+0000 7f9fcbe00640 1 -- 172.21.15.66:0/1258763132 <== mon.1 v2:172.21.15.94:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fc40cc020 con 0x7f9fb4003b20 2024-09-10T22:00:08.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.076+0000 7f9fcbe00640 1 -- 172.21.15.66:0/1258763132 <== mon.1 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v15)=0 v15) ==== 123+0+0 (secure 0 0 0) 0x7f9fc4017480 con 0x7f9fb4003b20 2024-09-10T22:00:08.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.076+0000 7f9fd9c00640 1 -- 172.21.15.66:0/1258763132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fb4003b20 msgr2=0x7f9fd409c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.076+0000 7f9fd9c00640 1 --2- 172.21.15.66:0/1258763132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fb4003b20 0x7f9fd409c430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9fc401b360 tx=0x7f9fc40a7f70 comp rx=0 tx=0).stop 2024-09-10T22:00:08.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.076+0000 7f9fd9c00640 1 -- 172.21.15.66:0/1258763132 shutdown_connections 2024-09-10T22:00:08.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.076+0000 7f9fd9c00640 1 --2- 172.21.15.66:0/1258763132 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9fb400a940 0x7f9fd409aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.076+0000 7f9fd9c00640 1 --2- 172.21.15.66:0/1258763132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb4006200 0x7f9fd4153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.076+0000 7f9fd9c00640 1 --2- 172.21.15.66:0/1258763132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fb4003b20 0x7f9fd409c430 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.076+0000 7f9fd9c00640 1 -- 172.21.15.66:0/1258763132 >> 172.21.15.66:0/1258763132 conn(0x7f9fd40cfcf0 msgr2=0x7f9fd40acd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:08.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.076+0000 7f9fd9c00640 1 -- 172.21.15.66:0/1258763132 shutdown_connections 2024-09-10T22:00:08.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.076+0000 7f9fd9c00640 1 -- 172.21.15.66:0/1258763132 wait complete. 2024-09-10T22:00:08.098 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.098 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.098 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.099 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.136+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1600645456 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c4006670 msgr2=0x7ff5c4003b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.136+0000 7ff5e8a00640 1 --2- 172.21.15.66:0/1600645456 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c4006670 0x7ff5c4003b60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4005fd0 tx=0x7ff5d401e4b0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.136+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1600645456 shutdown_connections 2024-09-10T22:00:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.136+0000 7ff5e8a00640 1 --2- 172.21.15.66:0/1600645456 >> v2:172.21.15.94:3300/0 conn(0x7ff5c40040a0 0x7ff5c40044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.136+0000 7ff5e8a00640 1 --2- 172.21.15.66:0/1600645456 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c4006670 0x7ff5c4003b60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.136+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1600645456 >> 172.21.15.66:0/1600645456 conn(0x7ff5e40d0b60 msgr2=0x7ff5e40d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.136+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1600645456 shutdown_connections 2024-09-10T22:00:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.136+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1600645456 wait complete. 2024-09-10T22:00:08.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.136+0000 7ff5e8a00640 1 Processor -- start 2024-09-10T22:00:08.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e8a00640 1 -- start start 2024-09-10T22:00:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c40040a0 0x7ff5e4150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c4004aa0 0x7ff5e41530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e8a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c4006670 0x7ff5e4155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e8a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e40cb440 con 0x7ff5c4004aa0 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e8a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e40cbce0 con 0x7ff5c40040a0 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e8a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e40cb5c0 con 0x7ff5c4006670 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c4004aa0 0x7ff5e41530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c40040a0 0x7ff5e4150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c4004aa0 0x7ff5e41530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48424/0 (socket says 172.21.15.66:48424) 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c40040a0 0x7ff5e4150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:50094/0 (socket says 172.21.15.66:50094) 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e1400640 1 -- 172.21.15.66:0/1296924190 learned_addr learned my addr 172.21.15.66:0/1296924190 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e2800640 1 --2- 172.21.15.66:0/1296924190 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c4006670 0x7ff5e4155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e1e00640 1 -- 172.21.15.66:0/1296924190 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c4006670 msgr2=0x7ff5e4155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e1e00640 1 --2- 172.21.15.66:0/1296924190 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c4006670 0x7ff5e4155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e1e00640 1 -- 172.21.15.66:0/1296924190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c4004aa0 msgr2=0x7ff5e41530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e1e00640 1 --2- 172.21.15.66:0/1296924190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c4004aa0 0x7ff5e41530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e1e00640 1 -- 172.21.15.66:0/1296924190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e41463d0 con 0x7ff5c40040a0 2024-09-10T22:00:08.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e1400640 1 --2- 172.21.15.66:0/1296924190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c4004aa0 0x7ff5e41530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:08.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e1e00640 1 --2- 172.21.15.66:0/1296924190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c40040a0 0x7ff5e4150730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40ac000 tx=0x7ff5d4005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5da000640 1 -- 172.21.15.66:0/1296924190 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d40ba020 con 0x7ff5c40040a0 2024-09-10T22:00:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5da000640 1 -- 172.21.15.66:0/1296924190 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5d40049e0 con 0x7ff5c40040a0 2024-09-10T22:00:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1296924190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e41525d0 con 0x7ff5c40040a0 2024-09-10T22:00:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5da000640 1 -- 172.21.15.66:0/1296924190 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d40b1040 con 0x7ff5c40040a0 2024-09-10T22:00:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1296924190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e4150c70 con 0x7ff5c40040a0 2024-09-10T22:00:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5da000640 1 -- 172.21.15.66:0/1296924190 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7ff5d4004020 con 0x7ff5c40040a0 2024-09-10T22:00:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5da000640 1 -- 172.21.15.66:0/1296924190 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff5d4003a50 con 0x7ff5c40040a0 2024-09-10T22:00:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.140+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1296924190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e4073900 con 0x7ff5c40040a0 2024-09-10T22:00:08.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.144+0000 7ff5da000640 1 -- 172.21.15.66:0/1296924190 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7ff5d40c4070 con 0x7ff5c40040a0 2024-09-10T22:00:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.204+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1296924190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7ff5e40ca3e0 con 0x7ff5c40040a0 2024-09-10T22:00:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.212+0000 7ff5da000640 1 -- 172.21.15.66:0/1296924190 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v16)=0 v16) ==== 106+0+0 (secure 0 0 0) 0x7ff5d4015080 con 0x7ff5c40040a0 2024-09-10T22:00:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.212+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1296924190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c40040a0 msgr2=0x7ff5e4150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.212+0000 7ff5e8a00640 1 --2- 172.21.15.66:0/1296924190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c40040a0 0x7ff5e4150730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40ac000 tx=0x7ff5d4005700 comp rx=0 tx=0).stop 2024-09-10T22:00:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.212+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1296924190 shutdown_connections 2024-09-10T22:00:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.212+0000 7ff5e8a00640 1 --2- 172.21.15.66:0/1296924190 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c4006670 0x7ff5e4155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.212+0000 7ff5e8a00640 1 --2- 172.21.15.66:0/1296924190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c4004aa0 0x7ff5e41530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.212+0000 7ff5e8a00640 1 --2- 172.21.15.66:0/1296924190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c40040a0 0x7ff5e4150730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.212+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1296924190 >> 172.21.15.66:0/1296924190 conn(0x7ff5e40d0b60 msgr2=0x7ff5e40709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.212+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1296924190 shutdown_connections 2024-09-10T22:00:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.212+0000 7ff5e8a00640 1 -- 172.21.15.66:0/1296924190 wait complete. 2024-09-10T22:00:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.232+0000 7f216ba00640 1 -- 172.21.15.66:0/3338560291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2144005200 msgr2=0x7f21440056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.232+0000 7f216ba00640 1 --2- 172.21.15.66:0/3338560291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2144005200 0x7f21440056b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f21640cb6d0 tx=0x7f215c01c810 comp rx=0 tx=0).stop 2024-09-10T22:00:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.232+0000 7f216ba00640 1 -- 172.21.15.66:0/3338560291 shutdown_connections 2024-09-10T22:00:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.232+0000 7f216ba00640 1 --2- 172.21.15.66:0/3338560291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2144005200 0x7f21440056b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.232+0000 7f216ba00640 1 --2- 172.21.15.66:0/3338560291 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2164152b90 0x7f214400b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.232+0000 7f216ba00640 1 -- 172.21.15.66:0/3338560291 >> 172.21.15.66:0/3338560291 conn(0x7f21640b26c0 msgr2=0x7f21640b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.232+0000 7f216ba00640 1 -- 172.21.15.66:0/3338560291 shutdown_connections 2024-09-10T22:00:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.232+0000 7f216ba00640 1 -- 172.21.15.66:0/3338560291 wait complete. 2024-09-10T22:00:08.238 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.238 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.232+0000 7f216ba00640 1 Processor -- start 2024-09-10T22:00:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.232+0000 7f216ba00640 1 -- start start 2024-09-10T22:00:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f216ba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2144005200 0x7f21641f5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f216ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2144006640 0x7f21641f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f216ba00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2164152b90 0x7f216414e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f216ba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f216406f7d0 con 0x7f2144006640 2024-09-10T22:00:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f216ba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f216406f650 con 0x7f2144005200 2024-09-10T22:00:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f216ba00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f216406f950 con 0x7f2164152b90 2024-09-10T22:00:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2163e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2144006640 0x7f21641f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2163e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2144006640 0x7f21641f85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48436/0 (socket says 172.21.15.66:48436) 2024-09-10T22:00:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2163e00640 1 -- 172.21.15.66:0/175627503 learned_addr learned my addr 172.21.15.66:0/175627503 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2169a00640 1 --2- 172.21.15.66:0/175627503 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2164152b90 0x7f216414e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2169000640 1 --2- 172.21.15.66:0/175627503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2144005200 0x7f21641f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2163e00640 1 -- 172.21.15.66:0/175627503 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2164152b90 msgr2=0x7f216414e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2163e00640 1 --2- 172.21.15.66:0/175627503 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2164152b90 0x7f216414e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2163e00640 1 -- 172.21.15.66:0/175627503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2144005200 msgr2=0x7f21641f5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2163e00640 1 --2- 172.21.15.66:0/175627503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2144005200 0x7f21641f5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2163e00640 1 -- 172.21.15.66:0/175627503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21641f8b30 con 0x7f2144006640 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2169000640 1 --2- 172.21.15.66:0/175627503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2144005200 0x7f21641f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2163e00640 1 --2- 172.21.15.66:0/175627503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2144006640 0x7f21641f85f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2150004c50 tx=0x7f2150014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2160c00640 1 -- 172.21.15.66:0/175627503 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150012650 con 0x7f2144006640 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2160c00640 1 -- 172.21.15.66:0/175627503 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21500127f0 con 0x7f2144006640 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2160c00640 1 -- 172.21.15.66:0/175627503 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150012c00 con 0x7f2144006640 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f216ba00640 1 -- 172.21.15.66:0/175627503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21640cfb30 con 0x7f2144006640 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f216ba00640 1 -- 172.21.15.66:0/175627503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21641f4730 con 0x7f2144006640 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2160c00640 1 -- 172.21.15.66:0/175627503 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f215001b070 con 0x7f2144006640 2024-09-10T22:00:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f2160c00640 1 -- 172.21.15.66:0/175627503 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f215001d780 con 0x7f2144006640 2024-09-10T22:00:08.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.236+0000 7f216ba00640 1 -- 172.21.15.66:0/175627503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f216406cf70 con 0x7f2144006640 2024-09-10T22:00:08.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.240+0000 7f2160c00640 1 -- 172.21.15.66:0/175627503 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f2150026440 con 0x7f2144006640 2024-09-10T22:00:08.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.300+0000 7f216ba00640 1 -- 172.21.15.66:0/175627503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f21640b2f40 con 0x7f2144006640 2024-09-10T22:00:08.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.304+0000 7f2160c00640 1 -- 172.21.15.66:0/175627503 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v17)=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f215001d980 con 0x7f2144006640 2024-09-10T22:00:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.304+0000 7f216ba00640 1 -- 172.21.15.66:0/175627503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2144006640 msgr2=0x7f21641f85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.304+0000 7f216ba00640 1 --2- 172.21.15.66:0/175627503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2144006640 0x7f21641f85f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2150004c50 tx=0x7f2150014830 comp rx=0 tx=0).stop 2024-09-10T22:00:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.304+0000 7f216ba00640 1 -- 172.21.15.66:0/175627503 shutdown_connections 2024-09-10T22:00:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.304+0000 7f216ba00640 1 --2- 172.21.15.66:0/175627503 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2164152b90 0x7f216414e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.304+0000 7f216ba00640 1 --2- 172.21.15.66:0/175627503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2144006640 0x7f21641f85f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.304+0000 7f216ba00640 1 --2- 172.21.15.66:0/175627503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2144005200 0x7f21641f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.304+0000 7f216ba00640 1 -- 172.21.15.66:0/175627503 >> 172.21.15.66:0/175627503 conn(0x7f21640b26c0 msgr2=0x7f216415e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.304+0000 7f216ba00640 1 -- 172.21.15.66:0/175627503 shutdown_connections 2024-09-10T22:00:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.304+0000 7f216ba00640 1 -- 172.21.15.66:0/175627503 wait complete. 2024-09-10T22:00:08.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.324+0000 7f6fdbe00640 1 -- 172.21.15.66:0/2499872752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb4007d40 msgr2=0x7f6fb4008140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.324+0000 7f6fdbe00640 1 --2- 172.21.15.66:0/2499872752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb4007d40 0x7f6fb4008140 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0005fd0 tx=0x7f6fc001e4b0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.324+0000 7f6fdbe00640 1 -- 172.21.15.66:0/2499872752 shutdown_connections 2024-09-10T22:00:08.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.324+0000 7f6fdbe00640 1 --2- 172.21.15.66:0/2499872752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb4007d40 0x7f6fb4008140 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.324+0000 7f6fdbe00640 1 -- 172.21.15.66:0/2499872752 >> 172.21.15.66:0/2499872752 conn(0x7f6fd40d1c60 msgr2=0x7f6fd40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:08.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.324+0000 7f6fdbe00640 1 -- 172.21.15.66:0/2499872752 shutdown_connections 2024-09-10T22:00:08.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.324+0000 7f6fdbe00640 1 -- 172.21.15.66:0/2499872752 wait complete. 2024-09-10T22:00:08.330 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.330 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.324+0000 7f6fdbe00640 1 Processor -- start 2024-09-10T22:00:08.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.324+0000 7f6fdbe00640 1 -- start start 2024-09-10T22:00:08.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fdbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fb4003b40 0x7f6fd40ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fdbe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6fb4007d40 0x7f6fd40c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fdbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb40086e0 0x7f6fd40c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fdbe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd40cdd10 con 0x7f6fb40086e0 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fdbe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd40cdb90 con 0x7f6fb4003b40 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fdbe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd40cde90 con 0x7f6fb4007d40 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb40086e0 0x7f6fd40c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb40086e0 0x7f6fd40c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48448/0 (socket says 172.21.15.66:48448) 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd9e00640 1 -- 172.21.15.66:0/3672913774 learned_addr learned my addr 172.21.15.66:0/3672913774 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd9400640 1 --2- 172.21.15.66:0/3672913774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fb4003b40 0x7f6fd40ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd8a00640 1 --2- 172.21.15.66:0/3672913774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6fb4007d40 0x7f6fd40c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd9e00640 1 -- 172.21.15.66:0/3672913774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6fb4007d40 msgr2=0x7f6fd40c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd9e00640 1 --2- 172.21.15.66:0/3672913774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6fb4007d40 0x7f6fd40c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd9e00640 1 -- 172.21.15.66:0/3672913774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fb4003b40 msgr2=0x7f6fd40ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd9e00640 1 --2- 172.21.15.66:0/3672913774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fb4003b40 0x7f6fd40ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd9e00640 1 -- 172.21.15.66:0/3672913774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd41fef70 con 0x7f6fb40086e0 2024-09-10T22:00:08.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd9400640 1 --2- 172.21.15.66:0/3672913774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fb4003b40 0x7f6fd40ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:08.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd9e00640 1 --2- 172.21.15.66:0/3672913774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb40086e0 0x7f6fd40c9450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40cf030 tx=0x7f6fcc00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:08.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd1600640 1 -- 172.21.15.66:0/3672913774 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc00a810 con 0x7f6fb40086e0 2024-09-10T22:00:08.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fdbe00640 1 -- 172.21.15.66:0/3672913774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd41ff250 con 0x7f6fb40086e0 2024-09-10T22:00:08.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd1600640 1 -- 172.21.15.66:0/3672913774 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fcc018070 con 0x7f6fb40086e0 2024-09-10T22:00:08.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd1600640 1 -- 172.21.15.66:0/3672913774 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc00ad40 con 0x7f6fb40086e0 2024-09-10T22:00:08.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fdbe00640 1 -- 172.21.15.66:0/3672913774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd41ff810 con 0x7f6fb40086e0 2024-09-10T22:00:08.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd1600640 1 -- 172.21.15.66:0/3672913774 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f6fcc01d460 con 0x7f6fb40086e0 2024-09-10T22:00:08.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fd1600640 1 -- 172.21.15.66:0/3672913774 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6fcc0126e0 con 0x7f6fb40086e0 2024-09-10T22:00:08.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.328+0000 7f6fdbe00640 1 -- 172.21.15.66:0/3672913774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd40ce390 con 0x7f6fb40086e0 2024-09-10T22:00:08.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.332+0000 7f6fd1600640 1 -- 172.21.15.66:0/3672913774 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f6fcc01d910 con 0x7f6fb40086e0 2024-09-10T22:00:08.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.392+0000 7f6fdbe00640 1 -- 172.21.15.66:0/3672913774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f6fd40bc200 con 0x7f6fb40086e0 2024-09-10T22:00:08.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.396+0000 7f6fd1600640 1 -- 172.21.15.66:0/3672913774 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v18)=0 v18) ==== 119+0+0 (secure 0 0 0) 0x7f6fcc01daf0 con 0x7f6fb40086e0 2024-09-10T22:00:08.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.400+0000 7f6fdbe00640 1 -- 172.21.15.66:0/3672913774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb40086e0 msgr2=0x7f6fd40c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.400+0000 7f6fdbe00640 1 --2- 172.21.15.66:0/3672913774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb40086e0 0x7f6fd40c9450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40cf030 tx=0x7f6fcc00cb30 comp rx=0 tx=0).stop 2024-09-10T22:00:08.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.400+0000 7f6fdbe00640 1 -- 172.21.15.66:0/3672913774 shutdown_connections 2024-09-10T22:00:08.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.400+0000 7f6fdbe00640 1 --2- 172.21.15.66:0/3672913774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb40086e0 0x7f6fd40c9450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.400+0000 7f6fdbe00640 1 --2- 172.21.15.66:0/3672913774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6fb4007d40 0x7f6fd40c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.400+0000 7f6fdbe00640 1 --2- 172.21.15.66:0/3672913774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fb4003b40 0x7f6fd40ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.400+0000 7f6fdbe00640 1 -- 172.21.15.66:0/3672913774 >> 172.21.15.66:0/3672913774 conn(0x7f6fd40d1c60 msgr2=0x7f6fd406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:08.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.400+0000 7f6fdbe00640 1 -- 172.21.15.66:0/3672913774 shutdown_connections 2024-09-10T22:00:08.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.400+0000 7f6fdbe00640 1 -- 172.21.15.66:0/3672913774 wait complete. 2024-09-10T22:00:08.422 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.422 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 -- 172.21.15.66:0/29157083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c00c290 msgr2=0x7f262c00e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 --2- 172.21.15.66:0/29157083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c00c290 0x7f262c00e680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f264c0d1c30 tx=0x7f264401d800 comp rx=0 tx=0).stop 2024-09-10T22:00:08.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 -- 172.21.15.66:0/29157083 shutdown_connections 2024-09-10T22:00:08.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 --2- 172.21.15.66:0/29157083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c00c290 0x7f262c00e680 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 -- 172.21.15.66:0/29157083 >> 172.21.15.66:0/29157083 conn(0x7f264c0b26c0 msgr2=0x7f264c0b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:08.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 -- 172.21.15.66:0/29157083 shutdown_connections 2024-09-10T22:00:08.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 -- 172.21.15.66:0/29157083 wait complete. 2024-09-10T22:00:08.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 Processor -- start 2024-09-10T22:00:08.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 -- start start 2024-09-10T22:00:08.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c003b60 0x7f264c1450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f262c007dd0 0x7f264c0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f262c00c290 0x7f264c1f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0cb9b0 con 0x7f262c003b60 2024-09-10T22:00:08.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0cb830 con 0x7f262c00c290 2024-09-10T22:00:08.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2653e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f264c0cbb30 con 0x7f262c007dd0 2024-09-10T22:00:08.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2651400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c003b60 0x7f264c1450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2651400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c003b60 0x7f264c1450c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48466/0 (socket says 172.21.15.66:48466) 2024-09-10T22:00:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2651400640 1 -- 172.21.15.66:0/3400050052 learned_addr learned my addr 172.21.15.66:0/3400050052 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2650a00640 1 --2- 172.21.15.66:0/3400050052 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f262c007dd0 0x7f264c0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2651e00640 1 --2- 172.21.15.66:0/3400050052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f262c00c290 0x7f264c1f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2651400640 1 -- 172.21.15.66:0/3400050052 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f262c007dd0 msgr2=0x7f264c0c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2651400640 1 --2- 172.21.15.66:0/3400050052 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f262c007dd0 0x7f264c0c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2651400640 1 -- 172.21.15.66:0/3400050052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f262c00c290 msgr2=0x7f264c1f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2651400640 1 --2- 172.21.15.66:0/3400050052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f262c00c290 0x7f264c1f4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.424+0000 7f2651400640 1 -- 172.21.15.66:0/3400050052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c1f45e0 con 0x7f262c003b60 2024-09-10T22:00:08.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.428+0000 7f2651400640 1 --2- 172.21.15.66:0/3400050052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c003b60 0x7f264c1450c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2640000f70 tx=0x7f2640014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:08.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.428+0000 7f2649600640 1 -- 172.21.15.66:0/3400050052 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640012860 con 0x7f262c003b60 2024-09-10T22:00:08.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.428+0000 7f2649600640 1 -- 172.21.15.66:0/3400050052 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2640012a00 con 0x7f262c003b60 2024-09-10T22:00:08.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.428+0000 7f2653e00640 1 -- 172.21.15.66:0/3400050052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c145600 con 0x7f262c003b60 2024-09-10T22:00:08.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.428+0000 7f2649600640 1 -- 172.21.15.66:0/3400050052 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640012e30 con 0x7f262c003b60 2024-09-10T22:00:08.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.428+0000 7f2653e00640 1 -- 172.21.15.66:0/3400050052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c145bb0 con 0x7f262c003b60 2024-09-10T22:00:08.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.428+0000 7f2649600640 1 -- 172.21.15.66:0/3400050052 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f264001b070 con 0x7f262c003b60 2024-09-10T22:00:08.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.428+0000 7f2649600640 1 -- 172.21.15.66:0/3400050052 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2640026820 con 0x7f262c003b60 2024-09-10T22:00:08.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.428+0000 7f2653e00640 1 -- 172.21.15.66:0/3400050052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c0cf220 con 0x7f262c003b60 2024-09-10T22:00:08.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.428+0000 7f2649600640 1 -- 172.21.15.66:0/3400050052 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f264002b070 con 0x7f262c003b60 2024-09-10T22:00:08.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.492+0000 7f2653e00640 1 -- 172.21.15.66:0/3400050052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f264c1ea730 con 0x7f262c003b60 2024-09-10T22:00:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.496+0000 7f2649600640 1 -- 172.21.15.66:0/3400050052 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v19)=0 v19) ==== 120+0+0 (secure 0 0 0) 0x7f264c1f45e0 con 0x7f262c003b60 2024-09-10T22:00:08.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.496+0000 7f2653e00640 1 -- 172.21.15.66:0/3400050052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c003b60 msgr2=0x7f264c1450c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.496+0000 7f2653e00640 1 --2- 172.21.15.66:0/3400050052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c003b60 0x7f264c1450c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2640000f70 tx=0x7f2640014830 comp rx=0 tx=0).stop 2024-09-10T22:00:08.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.496+0000 7f2653e00640 1 -- 172.21.15.66:0/3400050052 shutdown_connections 2024-09-10T22:00:08.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.496+0000 7f2653e00640 1 --2- 172.21.15.66:0/3400050052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f262c00c290 0x7f264c1f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.496+0000 7f2653e00640 1 --2- 172.21.15.66:0/3400050052 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f262c007dd0 0x7f264c0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.496+0000 7f2653e00640 1 --2- 172.21.15.66:0/3400050052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c003b60 0x7f264c1450c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.496+0000 7f2653e00640 1 -- 172.21.15.66:0/3400050052 >> 172.21.15.66:0/3400050052 conn(0x7f264c0b26c0 msgr2=0x7f264c06d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:08.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.496+0000 7f2653e00640 1 -- 172.21.15.66:0/3400050052 shutdown_connections 2024-09-10T22:00:08.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.496+0000 7f2653e00640 1 -- 172.21.15.66:0/3400050052 wait complete. 2024-09-10T22:00:08.521 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.521 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.521 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.521 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi066') 2024-09-10T22:00:08.521 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T22:00:08.521 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T22:00:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.708+0000 7faf90600640 1 -- 172.21.15.66:0/3140804492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf70003c70 msgr2=0x7faf70004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.708+0000 7faf90600640 1 --2- 172.21.15.66:0/3140804492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf70003c70 0x7faf70004070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faf7c006f00 tx=0x7faf7c01cea0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.708+0000 7faf8e600640 1 -- 172.21.15.66:0/3140804492 reap_dead start 2024-09-10T22:00:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.708+0000 7faf90600640 1 -- 172.21.15.66:0/3140804492 shutdown_connections 2024-09-10T22:00:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.708+0000 7faf90600640 1 -- 172.21.15.66:0/3140804492 >> 172.21.15.66:0/3140804492 conn(0x7faf8809aa50 msgr2=0x7faf8809ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.708+0000 7faf90600640 1 -- 172.21.15.66:0/3140804492 shutdown_connections 2024-09-10T22:00:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.708+0000 7faf90600640 1 -- 172.21.15.66:0/3140804492 wait complete. 2024-09-10T22:00:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf90600640 1 Processor -- start 2024-09-10T22:00:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf90600640 1 -- start start 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf90600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faf70003c70 0x7faf8814ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf90600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf8809f7d0 0x7faf8814b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf90600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf880c7210 0x7faf8814b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf90600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faf88167a60 con 0x7faf880c7210 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf90600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf881678e0 con 0x7faf8809f7d0 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf90600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7faf88167be0 con 0x7faf70003c70 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8e600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf880c7210 0x7faf8814b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf8809f7d0 0x7faf8814b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8dc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faf70003c70 0x7faf8814ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8e600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf880c7210 0x7faf8814b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48492/0 (socket says 172.21.15.66:48492) 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf8809f7d0 0x7faf8814b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:50134/0 (socket says 172.21.15.66:50134) 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8e600640 1 -- 172.21.15.66:0/3999584566 learned_addr learned my addr 172.21.15.66:0/3999584566 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8e600640 1 -- 172.21.15.66:0/3999584566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faf70003c70 msgr2=0x7faf8814ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8e600640 1 --2- 172.21.15.66:0/3999584566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faf70003c70 0x7faf8814ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8e600640 1 -- 172.21.15.66:0/3999584566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf8809f7d0 msgr2=0x7faf8814b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8e600640 1 --2- 172.21.15.66:0/3999584566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf8809f7d0 0x7faf8814b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8e600640 1 -- 172.21.15.66:0/3999584566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf880c7720 con 0x7faf880c7210 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8dc00640 1 --2- 172.21.15.66:0/3999584566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faf70003c70 0x7faf8814ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8d200640 1 --2- 172.21.15.66:0/3999584566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf8809f7d0 0x7faf8814b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf8e600640 1 --2- 172.21.15.66:0/3999584566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf880c7210 0x7faf8814b6f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faf80006fd0 tx=0x7faf80006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf85600640 1 -- 172.21.15.66:0/3999584566 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80011700 con 0x7faf880c7210 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf85600640 1 -- 172.21.15.66:0/3999584566 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf80016020 con 0x7faf880c7210 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf85600640 1 -- 172.21.15.66:0/3999584566 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80011b60 con 0x7faf880c7210 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf90600640 1 -- 172.21.15.66:0/3999584566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf880c79e0 con 0x7faf880c7210 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf90600640 1 -- 172.21.15.66:0/3999584566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf880c7fa0 con 0x7faf880c7210 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf85600640 1 -- 172.21.15.66:0/3999584566 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7faf800107b0 con 0x7faf880c7210 2024-09-10T22:00:08.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.712+0000 7faf85600640 1 -- 172.21.15.66:0/3999584566 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faf8001bd70 con 0x7faf880c7210 2024-09-10T22:00:08.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.716+0000 7faf90600640 1 -- 172.21.15.66:0/3999584566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf88148950 con 0x7faf880c7210 2024-09-10T22:00:08.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.716+0000 7faf85600640 1 -- 172.21.15.66:0/3999584566 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7faf88148950 con 0x7faf880c7210 2024-09-10T22:00:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.784+0000 7faf90600640 1 -- 172.21.15.66:0/3999584566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7faf880b4e80 con 0x7faf880c7210 2024-09-10T22:00:08.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.788+0000 7faf85600640 1 -- 172.21.15.66:0/3999584566 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7faf80020020 con 0x7faf880c7210 2024-09-10T22:00:08.790 INFO:teuthology.orchestra.run.smithi066.stderr:adjusted tunables profile to default 2024-09-10T22:00:08.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.788+0000 7faf90600640 1 -- 172.21.15.66:0/3999584566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf880c7210 msgr2=0x7faf8814b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:08.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.788+0000 7faf90600640 1 --2- 172.21.15.66:0/3999584566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf880c7210 0x7faf8814b6f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faf80006fd0 tx=0x7faf80006430 comp rx=0 tx=0).stop 2024-09-10T22:00:08.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.788+0000 7faf90600640 1 -- 172.21.15.66:0/3999584566 shutdown_connections 2024-09-10T22:00:08.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.788+0000 7faf90600640 1 --2- 172.21.15.66:0/3999584566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf880c7210 0x7faf8814b6f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.788+0000 7faf90600640 1 --2- 172.21.15.66:0/3999584566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf8809f7d0 0x7faf8814b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.788+0000 7faf90600640 1 --2- 172.21.15.66:0/3999584566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faf70003c70 0x7faf8814ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:08.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.788+0000 7faf90600640 1 -- 172.21.15.66:0/3999584566 >> 172.21.15.66:0/3999584566 conn(0x7faf8809aa50 msgr2=0x7faf88071940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:08.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.788+0000 7faf90600640 1 -- 172.21.15.66:0/3999584566 shutdown_connections 2024-09-10T22:00:08.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:08.788+0000 7faf90600640 1 -- 172.21.15.66:0/3999584566 wait complete. 2024-09-10T22:00:08.810 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T22:00:08.810 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T22:00:08.810 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:00:08.810 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T22:00:08.830 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:00:08.830 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T22:00:08.893 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:00:08.893 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T22:00:08.953 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:00:08.953 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T22:00:09.013 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:00:09.013 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T22:00:09.033 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:00:09.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T22:00:09.094 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:00:09.095 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T22:00:09.154 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:00:09.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T22:00:09.215 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 7c2f6b27-f225-42a8-8c2a-34078bba9b13 0 2024-09-10T22:00:09.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.390+0000 7f0e90200640 1 -- 172.21.15.94:0/261028146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e88151840 msgr2=0x7f0e8815f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.390+0000 7f0e90200640 1 --2- 172.21.15.94:0/261028146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e88151840 0x7f0e8815f050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e8815bc60 tx=0x7f0e7c01c820 comp rx=0 tx=0).stop 2024-09-10T22:00:09.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.390+0000 7f0e90200640 1 -- 172.21.15.94:0/261028146 shutdown_connections 2024-09-10T22:00:09.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.390+0000 7f0e90200640 1 --2- 172.21.15.94:0/261028146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e88151840 0x7f0e8815f050 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.390+0000 7f0e90200640 1 -- 172.21.15.94:0/261028146 >> 172.21.15.94:0/261028146 conn(0x7f0e8806dd30 msgr2=0x7f0e8806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:09.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.390+0000 7f0e90200640 1 -- 172.21.15.94:0/261028146 shutdown_connections 2024-09-10T22:00:09.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.390+0000 7f0e90200640 1 -- 172.21.15.94:0/261028146 wait complete. 2024-09-10T22:00:09.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e90200640 1 Processor -- start 2024-09-10T22:00:09.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e90200640 1 -- start start 2024-09-10T22:00:09.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e90200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e880b4c10 0x7f0e88144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e90200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e88151840 0x7f0e881599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e90200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e88151c20 0x7f0e88159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e90200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e88168ea0 con 0x7f0e880b4c10 2024-09-10T22:00:09.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e90200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e88168d20 con 0x7f0e88151840 2024-09-10T22:00:09.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e90200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0e88169020 con 0x7f0e88151c20 2024-09-10T22:00:09.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e8ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e88151840 0x7f0e881599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e8e200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e88151c20 0x7f0e88159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e8e200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e88151c20 0x7f0e88159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:40016/0 (socket says 172.21.15.94:40016) 2024-09-10T22:00:09.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e8e200640 1 -- 172.21.15.94:0/3301612202 learned_addr learned my addr 172.21.15.94:0/3301612202 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:00:09.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e8d800640 1 --2- 172.21.15.94:0/3301612202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e880b4c10 0x7f0e88144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e8e200640 1 -- 172.21.15.94:0/3301612202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e88151840 msgr2=0x7f0e881599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e8e200640 1 --2- 172.21.15.94:0/3301612202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e88151840 0x7f0e881599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e8e200640 1 -- 172.21.15.94:0/3301612202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e880b4c10 msgr2=0x7f0e88144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e8e200640 1 --2- 172.21.15.94:0/3301612202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e880b4c10 0x7f0e88144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e8e200640 1 -- 172.21.15.94:0/3301612202 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e88154040 con 0x7f0e88151c20 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e8e200640 1 --2- 172.21.15.94:0/3301612202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e88151c20 0x7f0e88159f20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000cbc0 tx=0x7f0e8000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e85600640 1 -- 172.21.15.94:0/3301612202 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e80016020 con 0x7f0e88151c20 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e85600640 1 -- 172.21.15.94:0/3301612202 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e8000ab00 con 0x7f0e88151c20 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e85600640 1 -- 172.21.15.94:0/3301612202 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e80012420 con 0x7f0e88151c20 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e90200640 1 -- 172.21.15.94:0/3301612202 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e88153d60 con 0x7f0e88151c20 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.394+0000 7f0e90200640 1 -- 172.21.15.94:0/3301612202 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8815c450 con 0x7f0e88151c20 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.398+0000 7f0e85600640 1 -- 172.21.15.94:0/3301612202 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f0e80012950 con 0x7f0e88151c20 2024-09-10T22:00:09.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.398+0000 7f0e85600640 1 -- 172.21.15.94:0/3301612202 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f0e8001bcc0 con 0x7f0e88151c20 2024-09-10T22:00:09.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.398+0000 7f0e90200640 1 -- 172.21.15.94:0/3301612202 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8815f050 con 0x7f0e88151c20 2024-09-10T22:00:09.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.398+0000 7f0e85600640 1 -- 172.21.15.94:0/3301612202 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f0e80020020 con 0x7f0e88151c20 2024-09-10T22:00:09.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.470+0000 7f0e90200640 1 -- 172.21.15.94:0/3301612202 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7c2f6b27-f225-42a8-8c2a-34078bba9b13", "id": 0} v 0) -- 0x7f0e881446d0 con 0x7f0e88151c20 2024-09-10T22:00:09.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.474+0000 7f0e85600640 1 -- 172.21.15.94:0/3301612202 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7c2f6b27-f225-42a8-8c2a-34078bba9b13", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f0e80004a50 con 0x7f0e88151c20 2024-09-10T22:00:09.478 INFO:teuthology.orchestra.run.smithi094.stdout:0 2024-09-10T22:00:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.474+0000 7f0e90200640 1 -- 172.21.15.94:0/3301612202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e88151c20 msgr2=0x7f0e88159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.474+0000 7f0e90200640 1 --2- 172.21.15.94:0/3301612202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e88151c20 0x7f0e88159f20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000cbc0 tx=0x7f0e8000ccb0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.474+0000 7f0e90200640 1 -- 172.21.15.94:0/3301612202 shutdown_connections 2024-09-10T22:00:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.474+0000 7f0e90200640 1 --2- 172.21.15.94:0/3301612202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e88151c20 0x7f0e88159f20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.474+0000 7f0e90200640 1 --2- 172.21.15.94:0/3301612202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e88151840 0x7f0e881599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.474+0000 7f0e90200640 1 --2- 172.21.15.94:0/3301612202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e880b4c10 0x7f0e88144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.474+0000 7f0e90200640 1 -- 172.21.15.94:0/3301612202 >> 172.21.15.94:0/3301612202 conn(0x7f0e8806dd30 msgr2=0x7f0e8814aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.474+0000 7f0e90200640 1 -- 172.21.15.94:0/3301612202 shutdown_connections 2024-09-10T22:00:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.474+0000 7f0e90200640 1 -- 172.21.15.94:0/3301612202 wait complete. 2024-09-10T22:00:09.497 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 5dca926d-7e4e-423f-b413-f69dfcc207fc 1 2024-09-10T22:00:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.634+0000 7f76ce600640 1 -- 172.21.15.94:0/681349358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c815ef20 msgr2=0x7f76c80cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.634+0000 7f76ce600640 1 --2- 172.21.15.94:0/681349358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c815ef20 0x7f76c80cecc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009900 tx=0x7f76bc01c870 comp rx=0 tx=0).stop 2024-09-10T22:00:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.634+0000 7f76ce600640 1 -- 172.21.15.94:0/681349358 shutdown_connections 2024-09-10T22:00:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.634+0000 7f76ce600640 1 --2- 172.21.15.94:0/681349358 >> v2:172.21.15.66:3300/0 conn(0x7f76c815f300 0x7f76c80b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.634+0000 7f76ce600640 1 --2- 172.21.15.94:0/681349358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c815ef20 0x7f76c80cecc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.634+0000 7f76ce600640 1 -- 172.21.15.94:0/681349358 >> 172.21.15.94:0/681349358 conn(0x7f76c80d1b10 msgr2=0x7f76c80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.634+0000 7f76ce600640 1 -- 172.21.15.94:0/681349358 shutdown_connections 2024-09-10T22:00:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.634+0000 7f76ce600640 1 -- 172.21.15.94:0/681349358 wait complete. 2024-09-10T22:00:09.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.634+0000 7f76ce600640 1 Processor -- start 2024-09-10T22:00:09.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.634+0000 7f76ce600640 1 -- start start 2024-09-10T22:00:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76ce600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76c80b15d0 0x7f76c814e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76ce600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c815ef20 0x7f76c80b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76ce600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c815f300 0x7f76c806f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76ce600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8168d60 con 0x7f76c815ef20 2024-09-10T22:00:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76ce600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76c80ce0b0 con 0x7f76c815f300 2024-09-10T22:00:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76ce600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f76c8168ee0 con 0x7f76c80b15d0 2024-09-10T22:00:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76cd200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76c80b15d0 0x7f76c814e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76cdc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c815f300 0x7f76c806f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76cd200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76c80b15d0 0x7f76c814e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:40024/0 (socket says 172.21.15.94:40024) 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76c7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c815ef20 0x7f76c80b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76cd200640 1 -- 172.21.15.94:0/1289552192 learned_addr learned my addr 172.21.15.94:0/1289552192 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76cd200640 1 -- 172.21.15.94:0/1289552192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c815f300 msgr2=0x7f76c806f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76cd200640 1 --2- 172.21.15.94:0/1289552192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c815f300 0x7f76c806f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76cd200640 1 -- 172.21.15.94:0/1289552192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c815ef20 msgr2=0x7f76c80b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76cd200640 1 --2- 172.21.15.94:0/1289552192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c815ef20 0x7f76c80b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76cd200640 1 -- 172.21.15.94:0/1289552192 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c8163d90 con 0x7f76c80b15d0 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76cd200640 1 --2- 172.21.15.94:0/1289552192 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76c80b15d0 0x7f76c814e060 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009850 tx=0x7f76bc004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76c4c00640 1 -- 172.21.15.94:0/1289552192 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0b2040 con 0x7f76c80b15d0 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76c4c00640 1 -- 172.21.15.94:0/1289552192 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76bc005020 con 0x7f76c80b15d0 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76c4c00640 1 -- 172.21.15.94:0/1289552192 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc005470 con 0x7f76c80b15d0 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76ce600640 1 -- 172.21.15.94:0/1289552192 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c8164070 con 0x7f76c80b15d0 2024-09-10T22:00:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76ce600640 1 -- 172.21.15.94:0/1289552192 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c8164630 con 0x7f76c80b15d0 2024-09-10T22:00:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76c4c00640 1 -- 172.21.15.94:0/1289552192 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f76c80d0100 con 0x7f76c80b15d0 2024-09-10T22:00:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76c4c00640 1 -- 172.21.15.94:0/1289552192 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f76bc003d70 con 0x7f76c80b15d0 2024-09-10T22:00:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.638+0000 7f76ce600640 1 -- 172.21.15.94:0/1289552192 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c80cecc0 con 0x7f76c80b15d0 2024-09-10T22:00:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.642+0000 7f76c4c00640 1 -- 172.21.15.94:0/1289552192 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f76bc016490 con 0x7f76c80b15d0 2024-09-10T22:00:09.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.710+0000 7f76ce600640 1 -- 172.21.15.94:0/1289552192 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5dca926d-7e4e-423f-b413-f69dfcc207fc", "id": 1} v 0) -- 0x7f76c8072e00 con 0x7f76c80b15d0 2024-09-10T22:00:09.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.714+0000 7f76c4c00640 1 -- 172.21.15.94:0/1289552192 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5dca926d-7e4e-423f-b413-f69dfcc207fc", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f76bc004b50 con 0x7f76c80b15d0 2024-09-10T22:00:09.718 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T22:00:09.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.714+0000 7f76ce600640 1 -- 172.21.15.94:0/1289552192 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76c80b15d0 msgr2=0x7f76c814e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.714+0000 7f76ce600640 1 --2- 172.21.15.94:0/1289552192 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76c80b15d0 0x7f76c814e060 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009850 tx=0x7f76bc004040 comp rx=0 tx=0).stop 2024-09-10T22:00:09.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.714+0000 7f76ce600640 1 -- 172.21.15.94:0/1289552192 shutdown_connections 2024-09-10T22:00:09.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.714+0000 7f76ce600640 1 --2- 172.21.15.94:0/1289552192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76c815f300 0x7f76c806f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.714+0000 7f76ce600640 1 --2- 172.21.15.94:0/1289552192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c815ef20 0x7f76c80b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.714+0000 7f76ce600640 1 --2- 172.21.15.94:0/1289552192 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76c80b15d0 0x7f76c814e060 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.714+0000 7f76ce600640 1 -- 172.21.15.94:0/1289552192 >> 172.21.15.94:0/1289552192 conn(0x7f76c80d1b10 msgr2=0x7f76c80b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:09.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.714+0000 7f76ce600640 1 -- 172.21.15.94:0/1289552192 shutdown_connections 2024-09-10T22:00:09.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.714+0000 7f76ce600640 1 -- 172.21.15.94:0/1289552192 wait complete. 2024-09-10T22:00:09.744 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 440ebb37-778a-4842-acf2-159f17fd29d3 2 2024-09-10T22:00:09.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.930+0000 7fc038600640 1 -- 172.21.15.94:0/2108146601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0300b0f60 msgr2=0x7fc0300d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.930+0000 7fc038600640 1 --2- 172.21.15.94:0/2108146601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0300b0f60 0x7fc0300d0310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009840 tx=0x7fc01c01c700 comp rx=0 tx=0).stop 2024-09-10T22:00:09.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.930+0000 7fc038600640 1 -- 172.21.15.94:0/2108146601 shutdown_connections 2024-09-10T22:00:09.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.930+0000 7fc038600640 1 --2- 172.21.15.94:0/2108146601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0300b0f60 0x7fc0300d0310 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.930+0000 7fc038600640 1 -- 172.21.15.94:0/2108146601 >> 172.21.15.94:0/2108146601 conn(0x7fc0300d1d00 msgr2=0x7fc0300d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:09.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.930+0000 7fc038600640 1 -- 172.21.15.94:0/2108146601 shutdown_connections 2024-09-10T22:00:09.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.930+0000 7fc038600640 1 -- 172.21.15.94:0/2108146601 wait complete. 2024-09-10T22:00:09.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.930+0000 7fc038600640 1 Processor -- start 2024-09-10T22:00:09.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.930+0000 7fc038600640 1 -- start start 2024-09-10T22:00:09.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc038600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0300b0f60 0x7fc0301f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc038600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0300b32a0 0x7fc0300c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc038600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc03015f460 0x7fc03006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc038600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc030169df0 con 0x7fc0300b0f60 2024-09-10T22:00:09.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc038600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc030169c70 con 0x7fc03015f460 2024-09-10T22:00:09.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc038600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc030169f70 con 0x7fc0300b32a0 2024-09-10T22:00:09.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc035200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0300b32a0 0x7fc0300c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc036600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc03015f460 0x7fc03006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc035200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0300b32a0 0x7fc0300c5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:40036/0 (socket says 172.21.15.94:40036) 2024-09-10T22:00:09.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc035200640 1 -- 172.21.15.94:0/368848600 learned_addr learned my addr 172.21.15.94:0/368848600 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:00:09.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc036600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc03015f460 0x7fc03006eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56336/0 (socket says 172.21.15.94:56336) 2024-09-10T22:00:09.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc035c00640 1 --2- 172.21.15.94:0/368848600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0300b0f60 0x7fc0301f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc036600640 1 -- 172.21.15.94:0/368848600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0300b32a0 msgr2=0x7fc0300c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc036600640 1 --2- 172.21.15.94:0/368848600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0300b32a0 0x7fc0300c5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc036600640 1 -- 172.21.15.94:0/368848600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0300b0f60 msgr2=0x7fc0301f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc036600640 1 --2- 172.21.15.94:0/368848600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0300b0f60 0x7fc0301f8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc036600640 1 -- 172.21.15.94:0/368848600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0300c7150 con 0x7fc03015f460 2024-09-10T22:00:09.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc035200640 1 --2- 172.21.15.94:0/368848600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0300b32a0 0x7fc0300c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:09.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc036600640 1 --2- 172.21.15.94:0/368848600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc03015f460 0x7fc03006eeb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc02800c3a0 tx=0x7fc028007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:09.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc02d600640 1 -- 172.21.15.94:0/368848600 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028017070 con 0x7fc03015f460 2024-09-10T22:00:09.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc02d600640 1 -- 172.21.15.94:0/368848600 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc028003600 con 0x7fc03015f460 2024-09-10T22:00:09.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc02d600640 1 -- 172.21.15.94:0/368848600 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028003a50 con 0x7fc03015f460 2024-09-10T22:00:09.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc038600640 1 -- 172.21.15.94:0/368848600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0300c7430 con 0x7fc03015f460 2024-09-10T22:00:09.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc038600640 1 -- 172.21.15.94:0/368848600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0300c79f0 con 0x7fc03015f460 2024-09-10T22:00:09.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc02d600640 1 -- 172.21.15.94:0/368848600 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fc028003bf0 con 0x7fc03015f460 2024-09-10T22:00:09.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc02d600640 1 -- 172.21.15.94:0/368848600 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fc02801c860 con 0x7fc03015f460 2024-09-10T22:00:09.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.934+0000 7fc038600640 1 -- 172.21.15.94:0/368848600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0300c7c30 con 0x7fc03015f460 2024-09-10T22:00:09.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:09.938+0000 7fc02d600640 1 -- 172.21.15.94:0/368848600 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fc028021070 con 0x7fc03015f460 2024-09-10T22:00:10.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.010+0000 7fc038600640 1 -- 172.21.15.94:0/368848600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "440ebb37-778a-4842-acf2-159f17fd29d3", "id": 2} v 0) -- 0x7fc030156c50 con 0x7fc03015f460 2024-09-10T22:00:10.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.014+0000 7fc02d600640 1 -- 172.21.15.94:0/368848600 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "440ebb37-778a-4842-acf2-159f17fd29d3", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fc02801c440 con 0x7fc03015f460 2024-09-10T22:00:10.017 INFO:teuthology.orchestra.run.smithi094.stdout:2 2024-09-10T22:00:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.014+0000 7fc038600640 1 -- 172.21.15.94:0/368848600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc03015f460 msgr2=0x7fc03006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.014+0000 7fc038600640 1 --2- 172.21.15.94:0/368848600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc03015f460 0x7fc03006eeb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc02800c3a0 tx=0x7fc028007f90 comp rx=0 tx=0).stop 2024-09-10T22:00:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.014+0000 7fc038600640 1 -- 172.21.15.94:0/368848600 shutdown_connections 2024-09-10T22:00:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.014+0000 7fc038600640 1 --2- 172.21.15.94:0/368848600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc03015f460 0x7fc03006eeb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.014+0000 7fc038600640 1 --2- 172.21.15.94:0/368848600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0300b32a0 0x7fc0300c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.014+0000 7fc038600640 1 --2- 172.21.15.94:0/368848600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0300b0f60 0x7fc0301f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.014+0000 7fc038600640 1 -- 172.21.15.94:0/368848600 >> 172.21.15.94:0/368848600 conn(0x7fc0300d1d00 msgr2=0x7fc0300b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.014+0000 7fc038600640 1 -- 172.21.15.94:0/368848600 shutdown_connections 2024-09-10T22:00:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.014+0000 7fc038600640 1 -- 172.21.15.94:0/368848600 wait complete. 2024-09-10T22:00:10.036 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new a06c9eae-f712-4076-96e3-05a60e6c30b7 3 2024-09-10T22:00:10.114 INFO:tasks.ceph.mgr.x.smithi094.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T22:00:10.114 INFO:tasks.ceph.mgr.x.smithi094.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T22:00:10.114 INFO:tasks.ceph.mgr.x.smithi094.stderr: from numpy import show_config as show_numpy_config 2024-09-10T22:00:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 -- 172.21.15.94:0/195588449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbba415bd40 msgr2=0x7fbba40cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 --2- 172.21.15.94:0/195588449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbba415bd40 0x7fbba40cffa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbba4073a90 tx=0x7fbb9401c830 comp rx=0 tx=0).stop 2024-09-10T22:00:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 -- 172.21.15.94:0/195588449 shutdown_connections 2024-09-10T22:00:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 --2- 172.21.15.94:0/195588449 >> v2:172.21.15.66:3300/0 conn(0x7fbba40aec10 0x7fbba40af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 --2- 172.21.15.94:0/195588449 >> v2:172.21.15.94:3300/0 conn(0x7fbba40cf830 0x7fbba40d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 --2- 172.21.15.94:0/195588449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbba415bd40 0x7fbba40cffa0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 -- 172.21.15.94:0/195588449 >> 172.21.15.94:0/195588449 conn(0x7fbba40d1c60 msgr2=0x7fbba40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 -- 172.21.15.94:0/195588449 shutdown_connections 2024-09-10T22:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 -- 172.21.15.94:0/195588449 wait complete. 2024-09-10T22:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 Processor -- start 2024-09-10T22:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 -- start start 2024-09-10T22:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbba40aec10 0x7fbba40c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.174+0000 7fbba9c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba40cf830 0x7fbba414f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbba415bd40 0x7fbba41f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba9c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbba40cead0 con 0x7fbba415bd40 2024-09-10T22:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba9c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbba40ce950 con 0x7fbba40cf830 2024-09-10T22:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba9c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbba40cec50 con 0x7fbba40aec10 2024-09-10T22:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba2000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba40cf830 0x7fbba414f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba2000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba40cf830 0x7fbba414f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56354/0 (socket says 172.21.15.94:56354) 2024-09-10T22:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba2000640 1 -- 172.21.15.94:0/2579992529 learned_addr learned my addr 172.21.15.94:0/2579992529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba2a00640 1 --2- 172.21.15.94:0/2579992529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbba40aec10 0x7fbba40c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba3400640 1 --2- 172.21.15.94:0/2579992529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbba415bd40 0x7fbba41f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba2000640 1 -- 172.21.15.94:0/2579992529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbba40aec10 msgr2=0x7fbba40c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba2000640 1 --2- 172.21.15.94:0/2579992529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbba40aec10 0x7fbba40c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba2000640 1 -- 172.21.15.94:0/2579992529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbba415bd40 msgr2=0x7fbba41f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba2000640 1 --2- 172.21.15.94:0/2579992529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbba415bd40 0x7fbba41f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba2000640 1 -- 172.21.15.94:0/2579992529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba40c77d0 con 0x7fbba40cf830 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba2000640 1 --2- 172.21.15.94:0/2579992529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba40cf830 0x7fbba414f660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000e940 tx=0x7fbb9000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbb9aa00640 1 -- 172.21.15.94:0/2579992529 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000cd80 con 0x7fbba40cf830 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbb9aa00640 1 -- 172.21.15.94:0/2579992529 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb90005020 con 0x7fbba40cf830 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbb9aa00640 1 -- 172.21.15.94:0/2579992529 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90005450 con 0x7fbba40cf830 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba9c00640 1 -- 172.21.15.94:0/2579992529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba40c7ab0 con 0x7fbba40cf830 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba9c00640 1 -- 172.21.15.94:0/2579992529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba40c8070 con 0x7fbba40cf830 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbb9aa00640 1 -- 172.21.15.94:0/2579992529 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fbb90004b40 con 0x7fbba40cf830 2024-09-10T22:00:10.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbb9aa00640 1 -- 172.21.15.94:0/2579992529 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fbb900218d0 con 0x7fbba40cf830 2024-09-10T22:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.178+0000 7fbba9c00640 1 -- 172.21.15.94:0/2579992529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba40af040 con 0x7fbba40cf830 2024-09-10T22:00:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.182+0000 7fbb9aa00640 1 -- 172.21.15.94:0/2579992529 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fbb900055f0 con 0x7fbba40cf830 2024-09-10T22:00:10.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.250+0000 7fbba9c00640 1 -- 172.21.15.94:0/2579992529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a06c9eae-f712-4076-96e3-05a60e6c30b7", "id": 3} v 0) -- 0x7fbba40b0df0 con 0x7fbba40cf830 2024-09-10T22:00:10.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.254+0000 7fbb9aa00640 1 -- 172.21.15.94:0/2579992529 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a06c9eae-f712-4076-96e3-05a60e6c30b7", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fbb90021440 con 0x7fbba40cf830 2024-09-10T22:00:10.260 INFO:teuthology.orchestra.run.smithi094.stdout:3 2024-09-10T22:00:10.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.258+0000 7fbba9c00640 1 -- 172.21.15.94:0/2579992529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba40cf830 msgr2=0x7fbba414f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.258+0000 7fbba9c00640 1 --2- 172.21.15.94:0/2579992529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba40cf830 0x7fbba414f660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000e940 tx=0x7fbb9000ee00 comp rx=0 tx=0).stop 2024-09-10T22:00:10.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.258+0000 7fbba9c00640 1 -- 172.21.15.94:0/2579992529 shutdown_connections 2024-09-10T22:00:10.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.258+0000 7fbba9c00640 1 --2- 172.21.15.94:0/2579992529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbba415bd40 0x7fbba41f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.258+0000 7fbba9c00640 1 --2- 172.21.15.94:0/2579992529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba40cf830 0x7fbba414f660 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.258+0000 7fbba9c00640 1 --2- 172.21.15.94:0/2579992529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbba40aec10 0x7fbba40c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.258+0000 7fbba9c00640 1 -- 172.21.15.94:0/2579992529 >> 172.21.15.94:0/2579992529 conn(0x7fbba40d1c60 msgr2=0x7fbba40b6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:10.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.258+0000 7fbba9c00640 1 -- 172.21.15.94:0/2579992529 shutdown_connections 2024-09-10T22:00:10.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.258+0000 7fbba9c00640 1 -- 172.21.15.94:0/2579992529 wait complete. 2024-09-10T22:00:10.279 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 0ba4966e-0371-445c-9660-1c8605713886 4 2024-09-10T22:00:10.347 INFO:tasks.ceph.mgr.y.smithi066.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T22:00:10.347 INFO:tasks.ceph.mgr.y.smithi066.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T22:00:10.347 INFO:tasks.ceph.mgr.y.smithi066.stderr: from numpy import show_config as show_numpy_config 2024-09-10T22:00:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.414+0000 7f1cc7600640 1 -- 172.21.15.94:0/2925609937 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cc015bd40 msgr2=0x7f1cc00cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.414+0000 7f1cc7600640 1 --2- 172.21.15.94:0/2925609937 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cc015bd40 0x7f1cc00cffa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009900 tx=0x7f1cb401c850 comp rx=0 tx=0).stop 2024-09-10T22:00:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.414+0000 7f1cc7600640 1 -- 172.21.15.94:0/2925609937 shutdown_connections 2024-09-10T22:00:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.414+0000 7f1cc7600640 1 --2- 172.21.15.94:0/2925609937 >> v2:172.21.15.66:3300/0 conn(0x7f1cc00cf180 0x7f1cc00d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.414+0000 7f1cc7600640 1 --2- 172.21.15.94:0/2925609937 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cc015bd40 0x7f1cc00cffa0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.414+0000 7f1cc7600640 1 -- 172.21.15.94:0/2925609937 >> 172.21.15.94:0/2925609937 conn(0x7f1cc00d1c60 msgr2=0x7f1cc00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.414+0000 7f1cc7600640 1 -- 172.21.15.94:0/2925609937 shutdown_connections 2024-09-10T22:00:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.414+0000 7f1cc7600640 1 -- 172.21.15.94:0/2925609937 wait complete. 2024-09-10T22:00:10.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc7600640 1 Processor -- start 2024-09-10T22:00:10.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc7600640 1 -- start start 2024-09-10T22:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc7600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00cf180 0x7f1cc014fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc7600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc00cf830 0x7f1cc01fb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc7600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cc015bd40 0x7f1cc0152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc7600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0168ba0 con 0x7f1cc00cf180 2024-09-10T22:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc7600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0168a20 con 0x7f1cc00cf830 2024-09-10T22:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc7600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc0168d20 con 0x7f1cc015bd40 2024-09-10T22:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc00cf830 0x7f1cc01fb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc5600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cc015bd40 0x7f1cc0152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00cf180 0x7f1cc014fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc00cf830 0x7f1cc01fb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56358/0 (socket says 172.21.15.94:56358) 2024-09-10T22:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbfe00640 1 -- 172.21.15.94:0/2236506675 learned_addr learned my addr 172.21.15.94:0/2236506675 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbfe00640 1 -- 172.21.15.94:0/2236506675 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cc015bd40 msgr2=0x7f1cc0152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbfe00640 1 --2- 172.21.15.94:0/2236506675 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cc015bd40 0x7f1cc0152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbfe00640 1 -- 172.21.15.94:0/2236506675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00cf180 msgr2=0x7f1cc014fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbfe00640 1 --2- 172.21.15.94:0/2236506675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00cf180 0x7f1cc014fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbfe00640 1 -- 172.21.15.94:0/2236506675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc0150320 con 0x7f1cc00cf830 2024-09-10T22:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc4c00640 1 --2- 172.21.15.94:0/2236506675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00cf180 0x7f1cc014fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbfe00640 1 --2- 172.21.15.94:0/2236506675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc00cf830 0x7f1cc01fb650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00c500 tx=0x7f1cac00c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbcc00640 1 -- 172.21.15.94:0/2236506675 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac017070 con 0x7f1cc00cf830 2024-09-10T22:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc7600640 1 -- 172.21.15.94:0/2236506675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc0150600 con 0x7f1cc00cf830 2024-09-10T22:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbcc00640 1 -- 172.21.15.94:0/2236506675 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cac0053c0 con 0x7f1cc00cf830 2024-09-10T22:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbcc00640 1 -- 172.21.15.94:0/2236506675 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac012890 con 0x7f1cc00cf830 2024-09-10T22:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cc7600640 1 -- 172.21.15.94:0/2236506675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01fbb90 con 0x7f1cc00cf830 2024-09-10T22:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.418+0000 7f1cbcc00640 1 -- 172.21.15.94:0/2236506675 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1cac012a30 con 0x7f1cc00cf830 2024-09-10T22:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.422+0000 7f1cbcc00640 1 -- 172.21.15.94:0/2236506675 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f1cac012d50 con 0x7f1cc00cf830 2024-09-10T22:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.422+0000 7f1cc7600640 1 -- 172.21.15.94:0/2236506675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc0150ab0 con 0x7f1cc00cf830 2024-09-10T22:00:10.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.422+0000 7f1cbcc00640 1 -- 172.21.15.94:0/2236506675 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f1cac021070 con 0x7f1cc00cf830 2024-09-10T22:00:10.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.494+0000 7f1cc7600640 1 -- 172.21.15.94:0/2236506675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0ba4966e-0371-445c-9660-1c8605713886", "id": 4} v 0) -- 0x7f1cc00aec10 con 0x7f1cc00cf830 2024-09-10T22:00:10.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.498+0000 7f1cbcc00640 1 -- 172.21.15.94:0/2236506675 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0ba4966e-0371-445c-9660-1c8605713886", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f1cac0037f0 con 0x7f1cc00cf830 2024-09-10T22:00:10.501 INFO:teuthology.orchestra.run.smithi094.stdout:4 2024-09-10T22:00:10.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.498+0000 7f1cc7600640 1 -- 172.21.15.94:0/2236506675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc00cf830 msgr2=0x7f1cc01fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.498+0000 7f1cc7600640 1 --2- 172.21.15.94:0/2236506675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc00cf830 0x7f1cc01fb650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00c500 tx=0x7f1cac00c9c0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.498+0000 7f1cc7600640 1 -- 172.21.15.94:0/2236506675 shutdown_connections 2024-09-10T22:00:10.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.498+0000 7f1cc7600640 1 --2- 172.21.15.94:0/2236506675 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cc015bd40 0x7f1cc0152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.498+0000 7f1cc7600640 1 --2- 172.21.15.94:0/2236506675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc00cf830 0x7f1cc01fb650 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.498+0000 7f1cc7600640 1 --2- 172.21.15.94:0/2236506675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00cf180 0x7f1cc014fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.498+0000 7f1cc7600640 1 -- 172.21.15.94:0/2236506675 >> 172.21.15.94:0/2236506675 conn(0x7f1cc00d1c60 msgr2=0x7f1cc0165cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:10.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.498+0000 7f1cc7600640 1 -- 172.21.15.94:0/2236506675 shutdown_connections 2024-09-10T22:00:10.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.498+0000 7f1cc7600640 1 -- 172.21.15.94:0/2236506675 wait complete. 2024-09-10T22:00:10.526 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new e308cc33-fb34-4a88-a7e3-cdbb2929478f 5 2024-09-10T22:00:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 -- 172.21.15.94:0/526545815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe79006d4b0 msgr2=0x7fe790073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 --2- 172.21.15.94:0/526545815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe79006d4b0 0x7fe790073f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe7900cd670 tx=0x7fe78c01cb30 comp rx=0 tx=0).stop 2024-09-10T22:00:10.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 -- 172.21.15.94:0/526545815 shutdown_connections 2024-09-10T22:00:10.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 --2- 172.21.15.94:0/526545815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe79006d4b0 0x7fe790073f40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 --2- 172.21.15.94:0/526545815 >> v2:172.21.15.66:3300/0 conn(0x7fe7900737c0 0x7fe79006cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 -- 172.21.15.94:0/526545815 >> 172.21.15.94:0/526545815 conn(0x7fe7900b26c0 msgr2=0x7fe7900b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:10.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 -- 172.21.15.94:0/526545815 shutdown_connections 2024-09-10T22:00:10.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 -- 172.21.15.94:0/526545815 wait complete. 2024-09-10T22:00:10.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 Processor -- start 2024-09-10T22:00:10.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 -- start start 2024-09-10T22:00:10.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe79006d4b0 0x7fe7900c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:10.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7900737c0 0x7fe7901fb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe790152b90 0x7fe79014e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe790168cc0 con 0x7fe79006d4b0 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe790168b40 con 0x7fe790152b90 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.666+0000 7fe798800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe790168e40 con 0x7fe7900737c0 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe795400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7900737c0 0x7fe7901fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe796800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe790152b90 0x7fe79014e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe795e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe79006d4b0 0x7fe7900c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe795e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe79006d4b0 0x7fe7900c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.94:56384/0 (socket says 172.21.15.94:56384) 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe795e00640 1 -- 172.21.15.94:0/1465251118 learned_addr learned my addr 172.21.15.94:0/1465251118 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe796800640 1 -- 172.21.15.94:0/1465251118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7900737c0 msgr2=0x7fe7901fb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe796800640 1 --2- 172.21.15.94:0/1465251118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7900737c0 0x7fe7901fb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe796800640 1 -- 172.21.15.94:0/1465251118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe79006d4b0 msgr2=0x7fe7900c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe796800640 1 --2- 172.21.15.94:0/1465251118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe79006d4b0 0x7fe7900c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe796800640 1 -- 172.21.15.94:0/1465251118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7901fabe0 con 0x7fe790152b90 2024-09-10T22:00:10.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe796800640 1 --2- 172.21.15.94:0/1465251118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe790152b90 0x7fe79014e6c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0ac000 tx=0x7fe78c005a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:10.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe78a000640 1 -- 172.21.15.94:0/1465251118 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0b2040 con 0x7fe790152b90 2024-09-10T22:00:10.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe78a000640 1 -- 172.21.15.94:0/1465251118 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe78c004130 con 0x7fe790152b90 2024-09-10T22:00:10.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe78a000640 1 -- 172.21.15.94:0/1465251118 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c004560 con 0x7fe790152b90 2024-09-10T22:00:10.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe798800640 1 -- 172.21.15.94:0/1465251118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7900b5030 con 0x7fe790152b90 2024-09-10T22:00:10.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe798800640 1 -- 172.21.15.94:0/1465251118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7901fb160 con 0x7fe790152b90 2024-09-10T22:00:10.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe78a000640 1 -- 172.21.15.94:0/1465251118 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fe78c0b5070 con 0x7fe790152b90 2024-09-10T22:00:10.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe78a000640 1 -- 172.21.15.94:0/1465251118 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fe78c0bfd60 con 0x7fe790152b90 2024-09-10T22:00:10.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.670+0000 7fe798800640 1 -- 172.21.15.94:0/1465251118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe790073f40 con 0x7fe790152b90 2024-09-10T22:00:10.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.674+0000 7fe78a000640 1 -- 172.21.15.94:0/1465251118 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fe78c016750 con 0x7fe790152b90 2024-09-10T22:00:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.742+0000 7fe798800640 1 -- 172.21.15.94:0/1465251118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e308cc33-fb34-4a88-a7e3-cdbb2929478f", "id": 5} v 0) -- 0x7fe790152310 con 0x7fe790152b90 2024-09-10T22:00:10.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.746+0000 7fe78a000640 1 -- 172.21.15.94:0/1465251118 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e308cc33-fb34-4a88-a7e3-cdbb2929478f", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fe78c004750 con 0x7fe790152b90 2024-09-10T22:00:10.750 INFO:teuthology.orchestra.run.smithi094.stdout:5 2024-09-10T22:00:10.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.746+0000 7fe798800640 1 -- 172.21.15.94:0/1465251118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe790152b90 msgr2=0x7fe79014e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.746+0000 7fe798800640 1 --2- 172.21.15.94:0/1465251118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe790152b90 0x7fe79014e6c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0ac000 tx=0x7fe78c005a60 comp rx=0 tx=0).stop 2024-09-10T22:00:10.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.746+0000 7fe798800640 1 -- 172.21.15.94:0/1465251118 shutdown_connections 2024-09-10T22:00:10.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.746+0000 7fe798800640 1 --2- 172.21.15.94:0/1465251118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe790152b90 0x7fe79014e6c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.746+0000 7fe798800640 1 --2- 172.21.15.94:0/1465251118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7900737c0 0x7fe7901fb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.746+0000 7fe798800640 1 --2- 172.21.15.94:0/1465251118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe79006d4b0 0x7fe7900c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.746+0000 7fe798800640 1 -- 172.21.15.94:0/1465251118 >> 172.21.15.94:0/1465251118 conn(0x7fe7900b26c0 msgr2=0x7fe790150d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:10.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.746+0000 7fe798800640 1 -- 172.21.15.94:0/1465251118 shutdown_connections 2024-09-10T22:00:10.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.746+0000 7fe798800640 1 -- 172.21.15.94:0/1465251118 wait complete. 2024-09-10T22:00:10.769 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new b6dea904-68b7-41e4-a0f5-c68fa404e5cc 6 2024-09-10T22:00:10.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.906+0000 7f1cf3600640 1 -- 172.21.15.94:0/297209169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec06d4b0 msgr2=0x7f1cec073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.906+0000 7f1cf3600640 1 --2- 172.21.15.94:0/297209169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec06d4b0 0x7f1cec073f40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0cd670 tx=0x7f1ce401cd30 comp rx=0 tx=0).stop 2024-09-10T22:00:10.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.906+0000 7f1cf3600640 1 -- 172.21.15.94:0/297209169 shutdown_connections 2024-09-10T22:00:10.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.906+0000 7f1cf3600640 1 --2- 172.21.15.94:0/297209169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec06d4b0 0x7f1cec073f40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.906+0000 7f1cf3600640 1 --2- 172.21.15.94:0/297209169 >> v2:172.21.15.66:3300/0 conn(0x7f1cec0737c0 0x7f1cec06cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.906+0000 7f1cf3600640 1 -- 172.21.15.94:0/297209169 >> 172.21.15.94:0/297209169 conn(0x7f1cec0b26c0 msgr2=0x7f1cec0b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:10.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.906+0000 7f1cf3600640 1 -- 172.21.15.94:0/297209169 shutdown_connections 2024-09-10T22:00:10.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.906+0000 7f1cf3600640 1 -- 172.21.15.94:0/297209169 wait complete. 2024-09-10T22:00:10.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf3600640 1 Processor -- start 2024-09-10T22:00:10.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf3600640 1 -- start start 2024-09-10T22:00:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf3600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cec06d4b0 0x7f1cec154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf3600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec0737c0 0x7f1cec1fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf3600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec152b90 0x7f1cec1510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf3600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec168d50 con 0x7f1cec152b90 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf3600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec168bd0 con 0x7f1cec0737c0 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf3600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1cec168ed0 con 0x7f1cec06d4b0 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cebe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec0737c0 0x7f1cec1fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec152b90 0x7f1cec1510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cebe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec0737c0 0x7f1cec1fbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56382/0 (socket says 172.21.15.94:56382) 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec152b90 0x7f1cec1510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.94:56402/0 (socket says 172.21.15.94:56402) 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cebe00640 1 -- 172.21.15.94:0/510801194 learned_addr learned my addr 172.21.15.94:0/510801194 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf0c00640 1 --2- 172.21.15.94:0/510801194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cec06d4b0 0x7f1cec154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf1600640 1 -- 172.21.15.94:0/510801194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cec06d4b0 msgr2=0x7f1cec154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf1600640 1 --2- 172.21.15.94:0/510801194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cec06d4b0 0x7f1cec154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf1600640 1 -- 172.21.15.94:0/510801194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec0737c0 msgr2=0x7f1cec1fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf1600640 1 --2- 172.21.15.94:0/510801194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec0737c0 0x7f1cec1fbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf1600640 1 -- 172.21.15.94:0/510801194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec15f160 con 0x7f1cec152b90 2024-09-10T22:00:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf0c00640 1 --2- 172.21.15.94:0/510801194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cec06d4b0 0x7f1cec154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:10.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf1600640 1 --2- 172.21.15.94:0/510801194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec152b90 0x7f1cec1510c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0b3520 tx=0x7f1ce4009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:10.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1ce8c00640 1 -- 172.21.15.94:0/510801194 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40bb070 con 0x7f1cec152b90 2024-09-10T22:00:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1ce8c00640 1 -- 172.21.15.94:0/510801194 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ce4004d40 con 0x7f1cec152b90 2024-09-10T22:00:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf3600640 1 -- 172.21.15.94:0/510801194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec154a20 con 0x7f1cec152b90 2024-09-10T22:00:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1ce8c00640 1 -- 172.21.15.94:0/510801194 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40b2040 con 0x7f1cec152b90 2024-09-10T22:00:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1cf3600640 1 -- 172.21.15.94:0/510801194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec0c4e80 con 0x7f1cec152b90 2024-09-10T22:00:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1ce8c00640 1 -- 172.21.15.94:0/510801194 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1ce40b5030 con 0x7f1cec152b90 2024-09-10T22:00:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.910+0000 7f1ce8c00640 1 -- 172.21.15.94:0/510801194 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f1ce40a5c10 con 0x7f1cec152b90 2024-09-10T22:00:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.914+0000 7f1cf3600640 1 -- 172.21.15.94:0/510801194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cec0b6260 con 0x7f1cec152b90 2024-09-10T22:00:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:10.914+0000 7f1ce8c00640 1 -- 172.21.15.94:0/510801194 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f1ce4016950 con 0x7f1cec152b90 2024-09-10T22:00:11.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.002+0000 7f1cf3600640 1 -- 172.21.15.94:0/510801194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b6dea904-68b7-41e4-a0f5-c68fa404e5cc", "id": 6} v 0) -- 0x7f1cec15e960 con 0x7f1cec152b90 2024-09-10T22:00:11.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.002+0000 7f1ce8c00640 1 -- 172.21.15.94:0/510801194 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b6dea904-68b7-41e4-a0f5-c68fa404e5cc", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f1ce40c6020 con 0x7f1cec152b90 2024-09-10T22:00:11.007 INFO:teuthology.orchestra.run.smithi094.stdout:6 2024-09-10T22:00:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.006+0000 7f1cf3600640 1 -- 172.21.15.94:0/510801194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec152b90 msgr2=0x7f1cec1510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.006+0000 7f1cf3600640 1 --2- 172.21.15.94:0/510801194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec152b90 0x7f1cec1510c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0b3520 tx=0x7f1ce4009330 comp rx=0 tx=0).stop 2024-09-10T22:00:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.006+0000 7f1cf3600640 1 -- 172.21.15.94:0/510801194 shutdown_connections 2024-09-10T22:00:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.006+0000 7f1cf3600640 1 --2- 172.21.15.94:0/510801194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec152b90 0x7f1cec1510c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.006+0000 7f1cf3600640 1 --2- 172.21.15.94:0/510801194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec0737c0 0x7f1cec1fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.006+0000 7f1cf3600640 1 --2- 172.21.15.94:0/510801194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cec06d4b0 0x7f1cec154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.006+0000 7f1cf3600640 1 -- 172.21.15.94:0/510801194 >> 172.21.15.94:0/510801194 conn(0x7f1cec0b26c0 msgr2=0x7f1cec070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.006+0000 7f1cf3600640 1 -- 172.21.15.94:0/510801194 shutdown_connections 2024-09-10T22:00:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.006+0000 7f1cf3600640 1 -- 172.21.15.94:0/510801194 wait complete. 2024-09-10T22:00:11.027 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new a291da02-ca76-4af0-9b97-7ef8e2a97799 7 2024-09-10T22:00:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.162+0000 7f9e67200640 1 -- 172.21.15.94:0/4285628294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e6815f100 msgr2=0x7f9e680d1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.162+0000 7f9e67200640 1 --2- 172.21.15.94:0/4285628294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e6815f100 0x7f9e680d1960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009930 tx=0x7f9e5801c770 comp rx=0 tx=0).stop 2024-09-10T22:00:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.162+0000 7f9e67200640 1 -- 172.21.15.94:0/4285628294 shutdown_connections 2024-09-10T22:00:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.162+0000 7f9e67200640 1 --2- 172.21.15.94:0/4285628294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e6815f100 0x7f9e680d1960 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e67200640 1 -- 172.21.15.94:0/4285628294 >> 172.21.15.94:0/4285628294 conn(0x7f9e6809aa50 msgr2=0x7f9e6809ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e67200640 1 -- 172.21.15.94:0/4285628294 shutdown_connections 2024-09-10T22:00:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e67200640 1 -- 172.21.15.94:0/4285628294 wait complete. 2024-09-10T22:00:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e67200640 1 Processor -- start 2024-09-10T22:00:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e67200640 1 -- start start 2024-09-10T22:00:11.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e67200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e680b3110 0x7f9e680c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:11.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e67200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e680d0110 0x7f9e680c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:11.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e67200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e6815f100 0x7f9e68150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:11.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e67200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9e680d1ea0 con 0x7f9e6815f100 2024-09-10T22:00:11.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e67200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9e680b0c50 con 0x7f9e680b3110 2024-09-10T22:00:11.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e67200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9e680d2020 con 0x7f9e680d0110 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e65e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e680b3110 0x7f9e680c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e65400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e680d0110 0x7f9e680c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e65e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e680b3110 0x7f9e680c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56392/0 (socket says 172.21.15.94:56392) 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e65e00640 1 -- 172.21.15.94:0/3450540928 learned_addr learned my addr 172.21.15.94:0/3450540928 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e66800640 1 --2- 172.21.15.94:0/3450540928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e6815f100 0x7f9e68150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e65e00640 1 -- 172.21.15.94:0/3450540928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e680d0110 msgr2=0x7f9e680c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e65e00640 1 --2- 172.21.15.94:0/3450540928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e680d0110 0x7f9e680c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e65e00640 1 -- 172.21.15.94:0/3450540928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e6815f100 msgr2=0x7f9e68150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e65e00640 1 --2- 172.21.15.94:0/3450540928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e6815f100 0x7f9e68150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e65e00640 1 -- 172.21.15.94:0/3450540928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e680c8670 con 0x7f9e680b3110 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.166+0000 7f9e65e00640 1 --2- 172.21.15.94:0/3450540928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e680b3110 0x7f9e680c58a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9e580b0040 tx=0x7f9e580042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.170+0000 7f9e5e000640 1 -- 172.21.15.94:0/3450540928 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e580b2040 con 0x7f9e680b3110 2024-09-10T22:00:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.170+0000 7f9e5e000640 1 -- 172.21.15.94:0/3450540928 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e580044e0 con 0x7f9e680b3110 2024-09-10T22:00:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.170+0000 7f9e5e000640 1 -- 172.21.15.94:0/3450540928 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58003aa0 con 0x7f9e680b3110 2024-09-10T22:00:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.170+0000 7f9e67200640 1 -- 172.21.15.94:0/3450540928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e680c80b0 con 0x7f9e680b3110 2024-09-10T22:00:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.170+0000 7f9e67200640 1 -- 172.21.15.94:0/3450540928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e680cd5c0 con 0x7f9e680b3110 2024-09-10T22:00:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.170+0000 7f9e5e000640 1 -- 172.21.15.94:0/3450540928 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f9e580ad440 con 0x7f9e680b3110 2024-09-10T22:00:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.170+0000 7f9e5e000640 1 -- 172.21.15.94:0/3450540928 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f9e580addc0 con 0x7f9e680b3110 2024-09-10T22:00:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.170+0000 7f9e67200640 1 -- 172.21.15.94:0/3450540928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e680d1960 con 0x7f9e680b3110 2024-09-10T22:00:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.170+0000 7f9e5e000640 1 -- 172.21.15.94:0/3450540928 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f9e58016430 con 0x7f9e680b3110 2024-09-10T22:00:11.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.242+0000 7f9e67200640 1 -- 172.21.15.94:0/3450540928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a291da02-ca76-4af0-9b97-7ef8e2a97799", "id": 7} v 0) -- 0x7f9e680d2970 con 0x7f9e680b3110 2024-09-10T22:00:11.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.246+0000 7f9e5e000640 1 -- 172.21.15.94:0/3450540928 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a291da02-ca76-4af0-9b97-7ef8e2a97799", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f9e580ad760 con 0x7f9e680b3110 2024-09-10T22:00:11.249 INFO:teuthology.orchestra.run.smithi094.stdout:7 2024-09-10T22:00:11.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.246+0000 7f9e67200640 1 -- 172.21.15.94:0/3450540928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e680b3110 msgr2=0x7f9e680c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:11.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.246+0000 7f9e67200640 1 --2- 172.21.15.94:0/3450540928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e680b3110 0x7f9e680c58a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9e580b0040 tx=0x7f9e580042f0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.246+0000 7f9e67200640 1 -- 172.21.15.94:0/3450540928 shutdown_connections 2024-09-10T22:00:11.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.246+0000 7f9e67200640 1 --2- 172.21.15.94:0/3450540928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e6815f100 0x7f9e68150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.246+0000 7f9e67200640 1 --2- 172.21.15.94:0/3450540928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e680d0110 0x7f9e680c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.246+0000 7f9e67200640 1 --2- 172.21.15.94:0/3450540928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e680b3110 0x7f9e680c58a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.246+0000 7f9e67200640 1 -- 172.21.15.94:0/3450540928 >> 172.21.15.94:0/3450540928 conn(0x7f9e6809aa50 msgr2=0x7f9e680706b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:11.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.246+0000 7f9e67200640 1 -- 172.21.15.94:0/3450540928 shutdown_connections 2024-09-10T22:00:11.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:11.246+0000 7f9e67200640 1 -- 172.21.15.94:0/3450540928 wait complete. 2024-09-10T22:00:11.268 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T22:00:11.268 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T22:00:11.271 INFO:tasks.ceph.osd.0:Started 2024-09-10T22:00:11.271 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T22:00:11.272 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T22:00:11.274 INFO:tasks.ceph.osd.1:Started 2024-09-10T22:00:11.274 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T22:00:11.275 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T22:00:11.277 INFO:tasks.ceph.osd.2:Started 2024-09-10T22:00:11.278 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T22:00:11.278 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T22:00:11.281 INFO:tasks.ceph.osd.3:Started 2024-09-10T22:00:11.282 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T22:00:11.282 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T22:00:11.285 INFO:tasks.ceph.osd.4:Started 2024-09-10T22:00:11.285 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T22:00:11.285 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T22:00:11.288 INFO:tasks.ceph.osd.5:Started 2024-09-10T22:00:11.288 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T22:00:11.288 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T22:00:11.291 INFO:tasks.ceph.osd.6:Started 2024-09-10T22:00:11.291 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T22:00:11.291 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T22:00:11.295 INFO:tasks.ceph.osd.7:Started 2024-09-10T22:00:11.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:00:11.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.477+0000 7f6958a00640 1 -- 172.21.15.66:0/3019439774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6938004810 msgr2=0x7f6938004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:11.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.477+0000 7f6958a00640 1 --2- 172.21.15.66:0/3019439774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6938004810 0x7f6938004c90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f693c009b30 tx=0x7f693c01d270 comp rx=0 tx=0).stop 2024-09-10T22:00:11.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.477+0000 7f6952800640 1 -- 172.21.15.66:0/3019439774 reap_dead start 2024-09-10T22:00:11.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.477+0000 7f6958a00640 1 -- 172.21.15.66:0/3019439774 shutdown_connections 2024-09-10T22:00:11.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.477+0000 7f6958a00640 1 -- 172.21.15.66:0/3019439774 >> 172.21.15.66:0/3019439774 conn(0x7f69540d1c60 msgr2=0x7f69540d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:11.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.477+0000 7f6958a00640 1 -- 172.21.15.66:0/3019439774 shutdown_connections 2024-09-10T22:00:11.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.477+0000 7f6958a00640 1 -- 172.21.15.66:0/3019439774 wait complete. 2024-09-10T22:00:11.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6958a00640 1 Processor -- start 2024-09-10T22:00:11.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6958a00640 1 -- start start 2024-09-10T22:00:11.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6958a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6938004810 0x7f695414a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6958a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69540cbaa0 0x7f69541f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6958a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69541f9160 0x7f69540b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6958a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6954166a10 con 0x7f69541f9160 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6958a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69540cf500 con 0x7f69540cbaa0 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6958a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6954166b90 con 0x7f6938004810 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6952800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69541f9160 0x7f69540b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6951e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6938004810 0x7f695414a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6952800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69541f9160 0x7f69540b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48530/0 (socket says 172.21.15.66:48530) 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6952800640 1 -- 172.21.15.66:0/2083294872 learned_addr learned my addr 172.21.15.66:0/2083294872 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f694be00640 1 --2- 172.21.15.66:0/2083294872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69540cbaa0 0x7f69541f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6951e00640 1 -- 172.21.15.66:0/2083294872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69540cbaa0 msgr2=0x7f69541f8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6951e00640 1 --2- 172.21.15.66:0/2083294872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69540cbaa0 0x7f69541f8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6951e00640 1 -- 172.21.15.66:0/2083294872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69541f9160 msgr2=0x7f69540b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:11.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6951e00640 1 --2- 172.21.15.66:0/2083294872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69541f9160 0x7f69540b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6951e00640 1 -- 172.21.15.66:0/2083294872 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69540d27c0 con 0x7f6938004810 2024-09-10T22:00:11.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6952800640 1 --2- 172.21.15.66:0/2083294872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69541f9160 0x7f69540b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:11.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6951e00640 1 --2- 172.21.15.66:0/2083294872 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6938004810 0x7f695414a170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6944001870 tx=0x7f69440039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:11.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f694a000640 1 -- 172.21.15.66:0/2083294872 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944015070 con 0x7f6938004810 2024-09-10T22:00:11.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f694a000640 1 -- 172.21.15.66:0/2083294872 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6944011900 con 0x7f6938004810 2024-09-10T22:00:11.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f6958a00640 1 -- 172.21.15.66:0/2083294872 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69541f96a0 con 0x7f6938004810 2024-09-10T22:00:11.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.481+0000 7f694a000640 1 -- 172.21.15.66:0/2083294872 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944011d50 con 0x7f6938004810 2024-09-10T22:00:11.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.485+0000 7f6958a00640 1 -- 172.21.15.66:0/2083294872 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f69540cb2a0 con 0x7f6938004810 2024-09-10T22:00:11.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.485+0000 7f694a000640 1 -- 172.21.15.66:0/2083294872 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f69440103c0 con 0x7f6938004810 2024-09-10T22:00:11.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.485+0000 7f694a000640 1 -- 172.21.15.66:0/2083294872 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6944023620 con 0x7f6938004810 2024-09-10T22:00:11.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.485+0000 7f6958a00640 1 -- 172.21.15.66:0/2083294872 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69540ce170 con 0x7f6938004810 2024-09-10T22:00:11.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.485+0000 7f694a000640 1 -- 172.21.15.66:0/2083294872 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f69440238f0 con 0x7f6938004810 2024-09-10T22:00:11.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.549+0000 7f6958a00640 1 -- 172.21.15.66:0/2083294872 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69540b0df0 con 0x7f6938004810 2024-09-10T22:00:11.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.549+0000 7f694a000640 1 -- 172.21.15.66:0/2083294872 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f6944023ad0 con 0x7f6938004810 2024-09-10T22:00:11.551 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:11.552 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":10,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:11.244473+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:11.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.549+0000 7f6958a00640 1 -- 172.21.15.66:0/2083294872 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6938004810 msgr2=0x7f695414a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:11.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.549+0000 7f6958a00640 1 --2- 172.21.15.66:0/2083294872 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6938004810 0x7f695414a170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6944001870 tx=0x7f69440039d0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.549+0000 7f6958a00640 1 -- 172.21.15.66:0/2083294872 shutdown_connections 2024-09-10T22:00:11.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.549+0000 7f6958a00640 1 --2- 172.21.15.66:0/2083294872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69541f9160 0x7f69540b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.549+0000 7f6958a00640 1 --2- 172.21.15.66:0/2083294872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69540cbaa0 0x7f69541f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.549+0000 7f6958a00640 1 --2- 172.21.15.66:0/2083294872 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6938004810 0x7f695414a170 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:11.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.549+0000 7f6958a00640 1 -- 172.21.15.66:0/2083294872 >> 172.21.15.66:0/2083294872 conn(0x7f69540d1c60 msgr2=0x7f69540b71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:11.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.549+0000 7f6958a00640 1 -- 172.21.15.66:0/2083294872 shutdown_connections 2024-09-10T22:00:11.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:11.549+0000 7f6958a00640 1 -- 172.21.15.66:0/2083294872 wait complete. 2024-09-10T22:00:11.573 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T22:00:11.573 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T22:00:12.380 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T22:00:12.518 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.513+0000 7fd721800640 1 -- 172.21.15.66:0/3135595253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd71c0ccca0 msgr2=0x7fd71c166780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:12.518 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.513+0000 7fd721800640 1 --2- 172.21.15.66:0/3135595253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd71c0ccca0 0x7fd71c166780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd714009bc0 tx=0x7fd71401f3b0 comp rx=0 tx=0).stop 2024-09-10T22:00:12.518 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.513+0000 7fd721800640 1 -- 172.21.15.66:0/3135595253 shutdown_connections 2024-09-10T22:00:12.518 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.513+0000 7fd721800640 1 --2- 172.21.15.66:0/3135595253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd71c0ccca0 0x7fd71c166780 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:12.519 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.513+0000 7fd721800640 1 -- 172.21.15.66:0/3135595253 >> 172.21.15.66:0/3135595253 conn(0x7fd71c09aa10 msgr2=0x7fd71c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:12.519 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.513+0000 7fd721800640 1 -- 172.21.15.66:0/3135595253 shutdown_connections 2024-09-10T22:00:12.519 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.513+0000 7fd721800640 1 -- 172.21.15.66:0/3135595253 wait complete. 2024-09-10T22:00:12.519 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd721800640 1 Processor -- start 2024-09-10T22:00:12.519 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd721800640 1 -- start start 2024-09-10T22:00:12.522 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd721800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd71c0ccca0 0x7fd71c15a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:12.522 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd721800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd71c0d0270 0x7fd71c148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:12.522 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd721800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd71c15bc80 0x7fd71c06eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:12.522 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd721800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c166ed0 con 0x7fd71c15bc80 2024-09-10T22:00:12.522 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd721800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c0b25a0 con 0x7fd71c0ccca0 2024-09-10T22:00:12.522 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd721800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd71c167050 con 0x7fd71c0d0270 2024-09-10T22:00:12.522 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd71b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd71c15bc80 0x7fd71c06eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:12.522 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd71aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd71c0ccca0 0x7fd71c15a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd71b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd71c15bc80 0x7fd71c06eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33190/0 (socket says 172.21.15.66:33190) 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd71b400640 1 -- 172.21.15.66:0/224676590 learned_addr learned my addr 172.21.15.66:0/224676590 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd713e00640 1 --2- 172.21.15.66:0/224676590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd71c0d0270 0x7fd71c148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd71b400640 1 -- 172.21.15.66:0/224676590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd71c0d0270 msgr2=0x7fd71c148ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd71b400640 1 --2- 172.21.15.66:0/224676590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd71c0d0270 0x7fd71c148ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd71b400640 1 -- 172.21.15.66:0/224676590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd71c0ccca0 msgr2=0x7fd71c15a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd71b400640 1 --2- 172.21.15.66:0/224676590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd71c0ccca0 0x7fd71c15a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd71b400640 1 -- 172.21.15.66:0/224676590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c15ad40 con 0x7fd71c15bc80 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd71b400640 1 --2- 172.21.15.66:0/224676590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd71c15bc80 0x7fd71c06eb90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd71401cfe0 tx=0x7fd714009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd712a00640 1 -- 172.21.15.66:0/224676590 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7140b4040 con 0x7fd71c15bc80 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd712a00640 1 -- 172.21.15.66:0/224676590 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7140be070 con 0x7fd71c15bc80 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd721800640 1 -- 172.21.15.66:0/224676590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71c06ddd0 con 0x7fd71c15bc80 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.517+0000 7fd712a00640 1 -- 172.21.15.66:0/224676590 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7140af8a0 con 0x7fd71c15bc80 2024-09-10T22:00:12.523 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.521+0000 7fd721800640 1 -- 172.21.15.66:0/224676590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd71c06e390 con 0x7fd71c15bc80 2024-09-10T22:00:12.524 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.521+0000 7fd721800640 1 -- 172.21.15.66:0/224676590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd71c0cc590 con 0x7fd71c15bc80 2024-09-10T22:00:12.524 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.521+0000 7fd712a00640 1 -- 172.21.15.66:0/224676590 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 2) ==== 48237+0+0 (secure 0 0 0) 0x7fd7140af540 con 0x7fd71c15bc80 2024-09-10T22:00:12.524 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.521+0000 7fd712a00640 1 -- 172.21.15.66:0/224676590 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd7140ec740 con 0x7fd71c15bc80 2024-09-10T22:00:12.525 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.521+0000 7fd712a00640 1 -- 172.21.15.66:0/224676590 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fd714018f10 con 0x7fd71c15bc80 2024-09-10T22:00:12.587 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.585+0000 7fd721800640 1 -- 172.21.15.66:0/224676590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd71c0cd220 con 0x7fd71c15bc80 2024-09-10T22:00:12.587 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.585+0000 7fd712a00640 1 -- 172.21.15.66:0/224676590 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd714004750 con 0x7fd71c15bc80 2024-09-10T22:00:12.588 INFO:teuthology.misc.health.smithi066.stdout: 2024-09-10T22:00:12.588 INFO:teuthology.misc.health.smithi066.stdout:{"epoch":10,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:11.244473+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:12.589 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.585+0000 7fd721800640 1 -- 172.21.15.66:0/224676590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd71c15bc80 msgr2=0x7fd71c06eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:12.589 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.585+0000 7fd721800640 1 --2- 172.21.15.66:0/224676590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd71c15bc80 0x7fd71c06eb90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd71401cfe0 tx=0x7fd714009200 comp rx=0 tx=0).stop 2024-09-10T22:00:12.589 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.585+0000 7fd721800640 1 -- 172.21.15.66:0/224676590 shutdown_connections 2024-09-10T22:00:12.589 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.585+0000 7fd721800640 1 --2- 172.21.15.66:0/224676590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd71c15bc80 0x7fd71c06eb90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:12.589 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.585+0000 7fd721800640 1 --2- 172.21.15.66:0/224676590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd71c0d0270 0x7fd71c148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:12.589 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.585+0000 7fd721800640 1 --2- 172.21.15.66:0/224676590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd71c0ccca0 0x7fd71c15a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:12.589 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.585+0000 7fd721800640 1 -- 172.21.15.66:0/224676590 >> 172.21.15.66:0/224676590 conn(0x7fd71c09aa10 msgr2=0x7fd71c070910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:12.589 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.585+0000 7fd721800640 1 -- 172.21.15.66:0/224676590 shutdown_connections 2024-09-10T22:00:12.589 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:12.585+0000 7fd721800640 1 -- 172.21.15.66:0/224676590 wait complete. 2024-09-10T22:00:12.606 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T22:00:12.736 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-09-10T22:00:12.733+0000 7f6032d807c0 -1 Falling back to public interface 2024-09-10T22:00:12.907 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-09-10T22:00:12.905+0000 7f41fe0cd7c0 -1 Falling back to public interface 2024-09-10T22:00:12.931 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-09-10T22:00:12.929+0000 7fc9908957c0 -1 Falling back to public interface 2024-09-10T22:00:12.955 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-09-10T22:00:12.953+0000 7f3003e5e7c0 -1 Falling back to public interface 2024-09-10T22:00:13.033 INFO:tasks.ceph.osd.5.smithi094.stderr:2024-09-10T22:00:13.030+0000 7f9aeee2b7c0 -1 Falling back to public interface 2024-09-10T22:00:13.053 INFO:tasks.ceph.osd.6.smithi094.stderr:2024-09-10T22:00:13.050+0000 7fc6b48f07c0 -1 Falling back to public interface 2024-09-10T22:00:13.061 INFO:tasks.ceph.osd.4.smithi094.stderr:2024-09-10T22:00:13.058+0000 7f8f7da367c0 -1 Falling back to public interface 2024-09-10T22:00:13.245 INFO:tasks.ceph.osd.7.smithi094.stderr:2024-09-10T22:00:13.242+0000 7f8462d3c7c0 -1 Falling back to public interface 2024-09-10T22:00:13.318 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:00:13.314+0000 7f6013000640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T22:00:13.318 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:00:13.314+0000 7f6013000640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T22:00:16.957 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-09-10T22:00:16.949+0000 7f6032d807c0 -1 osd.2 0 log_to_monitors true 2024-09-10T22:00:17.088 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-09-10T22:00:17.081+0000 7f41fe0cd7c0 -1 osd.1 0 log_to_monitors true 2024-09-10T22:00:17.146 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-09-10T22:00:17.141+0000 7fc9908957c0 -1 osd.0 0 log_to_monitors true 2024-09-10T22:00:17.254 INFO:tasks.ceph.osd.5.smithi094.stderr:2024-09-10T22:00:17.250+0000 7f9aeee2b7c0 -1 osd.5 0 log_to_monitors true 2024-09-10T22:00:17.426 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-09-10T22:00:17.421+0000 7f3003e5e7c0 -1 osd.3 0 log_to_monitors true 2024-09-10T22:00:17.461 INFO:tasks.ceph.osd.7.smithi094.stderr:2024-09-10T22:00:17.458+0000 7f8462d3c7c0 -1 osd.7 0 log_to_monitors true 2024-09-10T22:00:17.809 INFO:tasks.ceph.osd.4.smithi094.stderr:2024-09-10T22:00:17.806+0000 7f8f7da367c0 -1 osd.4 0 log_to_monitors true 2024-09-10T22:00:17.915 INFO:tasks.ceph.osd.6.smithi094.stderr:2024-09-10T22:00:17.910+0000 7fc6b48f07c0 -1 osd.6 0 log_to_monitors true 2024-09-10T22:00:19.178 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-09-10T22:00:19.169+0000 7f602dc00640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T22:00:19.316 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-09-10T22:00:19.309+0000 7f6023c00640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:00:19.411 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T22:00:19.590 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 -- 172.21.15.66:0/2277311151 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65cc0ccc40 msgr2=0x7f65cc0cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:19.590 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 --2- 172.21.15.66:0/2277311151 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65cc0ccc40 0x7f65cc0cd020 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009900 tx=0x7f65c001c850 comp rx=0 tx=0).stop 2024-09-10T22:00:19.591 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 -- 172.21.15.66:0/2277311151 shutdown_connections 2024-09-10T22:00:19.591 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 --2- 172.21.15.66:0/2277311151 >> v2:172.21.15.94:3300/0 conn(0x7f65cc0afe40 0x7f65cc0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.591 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 --2- 172.21.15.66:0/2277311151 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65cc0ccc40 0x7f65cc0cd020 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.591 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 -- 172.21.15.66:0/2277311151 >> 172.21.15.66:0/2277311151 conn(0x7f65cc074220 msgr2=0x7f65cc074630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:19.591 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 -- 172.21.15.66:0/2277311151 shutdown_connections 2024-09-10T22:00:19.591 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 -- 172.21.15.66:0/2277311151 wait complete. 2024-09-10T22:00:19.591 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 Processor -- start 2024-09-10T22:00:19.592 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 -- start start 2024-09-10T22:00:19.593 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc0afe40 0x7f65cc0b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:19.593 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65cc0b0400 0x7f65cc1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:19.593 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65cc0ccc40 0x7f65cc157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:19.593 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc169170 con 0x7f65cc0b0400 2024-09-10T22:00:19.593 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc168ff0 con 0x7f65cc0afe40 2024-09-10T22:00:19.593 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d2800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f65cc1692f0 con 0x7f65cc0ccc40 2024-09-10T22:00:19.593 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc0afe40 0x7f65cc0b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:19.594 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65cc0b0400 0x7f65cc1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:19.594 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc0afe40 0x7f65cc0b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:56646/0 (socket says 172.21.15.66:56646) 2024-09-10T22:00:19.594 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d1400640 1 -- 172.21.15.66:0/1456076436 learned_addr learned my addr 172.21.15.66:0/1456076436 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:19.595 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.585+0000 7f65d1e00640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65cc0ccc40 0x7f65cc157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:19.595 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.589+0000 7f65d1400640 1 -- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65cc0ccc40 msgr2=0x7f65cc157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:19.595 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.589+0000 7f65d1400640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65cc0ccc40 0x7f65cc157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.595 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.589+0000 7f65d1400640 1 -- 172.21.15.66:0/1456076436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65cc0b0400 msgr2=0x7f65cc1577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:19.595 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.589+0000 7f65d1400640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65cc0b0400 0x7f65cc1577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.595 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.589+0000 7f65d1400640 1 -- 172.21.15.66:0/1456076436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc0c7870 con 0x7f65cc0afe40 2024-09-10T22:00:19.595 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.589+0000 7f65d1400640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc0afe40 0x7f65cc0b4270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65c00b0040 tx=0x7f65c00a68e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:19.595 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.589+0000 7f65c9600640 1 -- 172.21.15.66:0/1456076436 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00bb070 con 0x7f65cc0afe40 2024-09-10T22:00:19.595 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.589+0000 7f65c9600640 1 -- 172.21.15.66:0/1456076436 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65c00a6dc0 con 0x7f65cc0afe40 2024-09-10T22:00:19.595 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.589+0000 7f65d2800640 1 -- 172.21.15.66:0/1456076436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65cc0c7b50 con 0x7f65cc0afe40 2024-09-10T22:00:19.595 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.589+0000 7f65c9600640 1 -- 172.21.15.66:0/1456076436 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00b2040 con 0x7f65cc0afe40 2024-09-10T22:00:19.596 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.589+0000 7f65d2800640 1 -- 172.21.15.66:0/1456076436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65cc0c8110 con 0x7f65cc0afe40 2024-09-10T22:00:19.596 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.589+0000 7f65d2800640 1 -- 172.21.15.66:0/1456076436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65cc0c75b0 con 0x7f65cc0afe40 2024-09-10T22:00:19.598 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.593+0000 7f65c9600640 1 -- 172.21.15.66:0/1456076436 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 96338+0+0 (secure 0 0 0) 0x7f65c0005290 con 0x7f65cc0afe40 2024-09-10T22:00:19.599 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.593+0000 7f65c9600640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65ac072550 0x7f65ac074a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:19.599 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.593+0000 7f65c9600640 1 -- 172.21.15.66:0/1456076436 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f65c012c490 con 0x7f65cc0afe40 2024-09-10T22:00:19.600 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.593+0000 7f65d0a00640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65ac072550 0x7f65ac074a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:19.600 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.593+0000 7f65d0a00640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65ac072550 0x7f65ac074a10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f65cc15f740 tx=0x7f65b8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:19.605 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.597+0000 7f65c9600640 1 -- 172.21.15.66:0/1456076436 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f65c00164d0 con 0x7f65cc0afe40 2024-09-10T22:00:19.836 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.829+0000 7f65d2800640 1 -- 172.21.15.66:0/1456076436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65cc0748b0 con 0x7f65cc0afe40 2024-09-10T22:00:19.837 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.829+0000 7f65c9600640 1 -- 172.21.15.66:0/1456076436 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f65c0101270 con 0x7f65cc0afe40 2024-09-10T22:00:19.837 INFO:teuthology.misc.health.smithi066.stdout: 2024-09-10T22:00:19.837 INFO:teuthology.misc.health.smithi066.stdout:{"epoch":13,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:19.014942+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:19.840 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.833+0000 7f65d2800640 1 -- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65ac072550 msgr2=0x7f65ac074a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:19.840 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.833+0000 7f65d2800640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65ac072550 0x7f65ac074a10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f65cc15f740 tx=0x7f65b8009040 comp rx=0 tx=0).stop 2024-09-10T22:00:19.840 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.833+0000 7f65d2800640 1 -- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc0afe40 msgr2=0x7f65cc0b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:19.840 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.833+0000 7f65d2800640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc0afe40 0x7f65cc0b4270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65c00b0040 tx=0x7f65c00a68e0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.840 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.833+0000 7f65d2800640 1 -- 172.21.15.66:0/1456076436 shutdown_connections 2024-09-10T22:00:19.840 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.833+0000 7f65d2800640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65ac072550 0x7f65ac074a10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.840 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.833+0000 7f65d2800640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65cc0ccc40 0x7f65cc157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.840 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.833+0000 7f65d2800640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65cc0b0400 0x7f65cc1577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.840 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.833+0000 7f65d2800640 1 --2- 172.21.15.66:0/1456076436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65cc0afe40 0x7f65cc0b4270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.841 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.833+0000 7f65d2800640 1 -- 172.21.15.66:0/1456076436 >> 172.21.15.66:0/1456076436 conn(0x7f65cc074220 msgr2=0x7f65cc0b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:19.841 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.833+0000 7f65d2800640 1 -- 172.21.15.66:0/1456076436 shutdown_connections 2024-09-10T22:00:19.841 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:19.833+0000 7f65d2800640 1 -- 172.21.15.66:0/1456076436 wait complete. 2024-09-10T22:00:19.861 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T22:00:20.770 INFO:tasks.ceph.osd.4.smithi094.stderr:2024-09-10T22:00:20.766+0000 7f8f78800640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T22:00:20.784 INFO:tasks.ceph.osd.6.smithi094.stderr:2024-09-10T22:00:20.778+0000 7fc6af800640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T22:00:20.797 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-09-10T22:00:20.789+0000 7fc98b800640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T22:00:20.809 INFO:tasks.ceph.osd.4.smithi094.stderr:2024-09-10T22:00:20.806+0000 7f8f6e800640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:00:20.821 INFO:tasks.ceph.osd.6.smithi094.stderr:2024-09-10T22:00:20.818+0000 7fc6a5800640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:00:20.835 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-09-10T22:00:20.829+0000 7fc981800640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:00:20.848 INFO:tasks.ceph.osd.7.smithi094.stderr:2024-09-10T22:00:20.846+0000 7f845e600640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T22:00:20.856 INFO:tasks.ceph.osd.5.smithi094.stderr:2024-09-10T22:00:20.850+0000 7f9ae9200640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T22:00:20.868 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-09-10T22:00:20.861+0000 7f2ffee00640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T22:00:20.877 INFO:tasks.ceph.osd.7.smithi094.stderr:2024-09-10T22:00:20.874+0000 7f8453c00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:00:20.892 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-09-10T22:00:20.885+0000 7f41f9a00640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T22:00:20.897 INFO:tasks.ceph.osd.5.smithi094.stderr:2024-09-10T22:00:20.894+0000 7f9adf200640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:00:20.899 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-09-10T22:00:20.893+0000 7f2ff4e00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:00:20.919 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-09-10T22:00:20.913+0000 7f41ef000640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:00:24.750 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:00:24.753 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:00:26.668 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T22:00:26.817 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.809+0000 7f2d26a00640 1 -- 172.21.15.66:0/747168334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d08003c40 msgr2=0x7f2d08004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:26.817 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.809+0000 7f2d26a00640 1 --2- 172.21.15.66:0/747168334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d08003c40 0x7f2d08004040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d14001860 tx=0x7f2d1401c840 comp rx=0 tx=0).stop 2024-09-10T22:00:26.817 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.809+0000 7f2d26a00640 1 -- 172.21.15.66:0/747168334 shutdown_connections 2024-09-10T22:00:26.817 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.809+0000 7f2d26a00640 1 --2- 172.21.15.66:0/747168334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d08003c40 0x7f2d08004040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.818 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.809+0000 7f2d26a00640 1 -- 172.21.15.66:0/747168334 >> 172.21.15.66:0/747168334 conn(0x7f2d20074220 msgr2=0x7f2d20074630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:26.818 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.809+0000 7f2d26a00640 1 -- 172.21.15.66:0/747168334 shutdown_connections 2024-09-10T22:00:26.818 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.809+0000 7f2d26a00640 1 -- 172.21.15.66:0/747168334 wait complete. 2024-09-10T22:00:26.818 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d26a00640 1 Processor -- start 2024-09-10T22:00:26.819 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d26a00640 1 -- start start 2024-09-10T22:00:26.821 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d08003c40 0x7f2d201fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:26.821 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d200c5cd0 0x7f2d2009d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:26.821 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d200c6150 0x7f2d2009d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:26.821 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d26a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20160dc0 con 0x7f2d200c5cd0 2024-09-10T22:00:26.821 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d26a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20160c40 con 0x7f2d200c6150 2024-09-10T22:00:26.821 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d26a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2d20160f40 con 0x7f2d08003c40 2024-09-10T22:00:26.822 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d24c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d200c5cd0 0x7f2d2009d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:26.822 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d24c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d200c5cd0 0x7f2d2009d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35544/0 (socket says 172.21.15.66:35544) 2024-09-10T22:00:26.822 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d24c00640 1 -- 172.21.15.66:0/2555383096 learned_addr learned my addr 172.21.15.66:0/2555383096 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:26.822 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d25600640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d08003c40 0x7f2d201fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:26.822 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d26000640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d200c6150 0x7f2d2009d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:26.822 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d24c00640 1 -- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d08003c40 msgr2=0x7f2d201fc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:26.822 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d24c00640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d08003c40 0x7f2d201fc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.822 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d24c00640 1 -- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d200c6150 msgr2=0x7f2d2009d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:26.822 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d24c00640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d200c6150 0x7f2d2009d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.822 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d24c00640 1 -- 172.21.15.66:0/2555383096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2009e130 con 0x7f2d200c5cd0 2024-09-10T22:00:26.822 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d24c00640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d200c5cd0 0x7f2d2009d220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c006630 tx=0x7f2d0c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:26.822 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d1d600640 1 -- 172.21.15.66:0/2555383096 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c015070 con 0x7f2d200c5cd0 2024-09-10T22:00:26.823 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d1d600640 1 -- 172.21.15.66:0/2555383096 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d0c011860 con 0x7f2d200c5cd0 2024-09-10T22:00:26.823 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d1d600640 1 -- 172.21.15.66:0/2555383096 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c011cb0 con 0x7f2d200c5cd0 2024-09-10T22:00:26.823 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d26a00640 1 -- 172.21.15.66:0/2555383096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2009de50 con 0x7f2d200c5cd0 2024-09-10T22:00:26.823 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.813+0000 7f2d26a00640 1 -- 172.21.15.66:0/2555383096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d201fbf50 con 0x7f2d200c5cd0 2024-09-10T22:00:26.823 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.817+0000 7f2d26a00640 1 -- 172.21.15.66:0/2555383096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2015e220 con 0x7f2d200c5cd0 2024-09-10T22:00:26.823 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.817+0000 7f2d1d600640 1 -- 172.21.15.66:0/2555383096 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) ==== 96338+0+0 (secure 0 0 0) 0x7f2d0c0103c0 con 0x7f2d200c5cd0 2024-09-10T22:00:26.825 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.817+0000 7f2d1d600640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2d00072550 0x7f2d00074a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:26.825 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.817+0000 7f2d1d600640 1 -- 172.21.15.66:0/2555383096 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f2d0c093f90 con 0x7f2d200c5cd0 2024-09-10T22:00:26.825 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.821+0000 7f2d25600640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2d00072550 0x7f2d00074a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:26.826 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.821+0000 7f2d25600640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2d00072550 0x7f2d00074a10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d14001830 tx=0x7f2d14019680 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:26.829 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.821+0000 7f2d1d600640 1 -- 172.21.15.66:0/2555383096 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2d0c05f430 con 0x7f2d200c5cd0 2024-09-10T22:00:26.842 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:26.837+0000 7f2d1d600640 1 -- 172.21.15.66:0/2555383096 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2d0c05b6a0 con 0x7f2d200c5cd0 2024-09-10T22:00:27.006 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 -- 172.21.15.66:0/2555383096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d20076170 con 0x7f2d200c5cd0 2024-09-10T22:00:27.007 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d1d600640 1 -- 172.21.15.66:0/2555383096 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12824 (secure 0 0 0) 0x7f2d0c07c080 con 0x7f2d200c5cd0 2024-09-10T22:00:27.007 INFO:teuthology.misc.health.smithi066.stdout: 2024-09-10T22:00:27.007 INFO:teuthology.misc.health.smithi066.stdout:{"epoch":18,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:24.816593+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:27.009 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 -- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2d00072550 msgr2=0x7f2d00074a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:27.009 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2d00072550 0x7f2d00074a10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d14001830 tx=0x7f2d14019680 comp rx=0 tx=0).stop 2024-09-10T22:00:27.009 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 -- 172.21.15.66:0/2555383096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d200c5cd0 msgr2=0x7f2d2009d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:27.009 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d200c5cd0 0x7f2d2009d220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c006630 tx=0x7f2d0c0039d0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.010 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 -- 172.21.15.66:0/2555383096 shutdown_connections 2024-09-10T22:00:27.010 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2d00072550 0x7f2d00074a10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.010 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d200c6150 0x7f2d2009d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.010 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d200c5cd0 0x7f2d2009d220 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.010 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 --2- 172.21.15.66:0/2555383096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d08003c40 0x7f2d201fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.010 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 -- 172.21.15.66:0/2555383096 >> 172.21.15.66:0/2555383096 conn(0x7f2d20074220 msgr2=0x7f2d20070250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:27.010 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 -- 172.21.15.66:0/2555383096 shutdown_connections 2024-09-10T22:00:27.010 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:00:27.001+0000 7f2d26a00640 1 -- 172.21.15.66:0/2555383096 wait complete. 2024-09-10T22:00:27.027 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T22:00:27.027 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T22:00:27.027 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:00:27.027 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:00:27.031 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:00:27.031 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:00:27.031 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:00:27.034 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:00:27.034 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:00:27.034 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:00:27.037 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:00:27.038 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:00:27.038 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:00:27.042 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:00:27.042 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T22:00:27.043 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:00:27.127 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:00:27.128 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:00:27.138 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:00:27.141 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:00:27.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.189+0000 7f5d8f600640 1 -- 172.21.15.66:0/2251109598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d88160280 msgr2=0x7f5d880b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:27.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.189+0000 7f5d8f600640 1 --2- 172.21.15.66:0/2251109598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d88160280 0x7f5d880b3170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009900 tx=0x7f5d7401d280 comp rx=0 tx=0).stop 2024-09-10T22:00:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.189+0000 7f5d8f600640 1 -- 172.21.15.66:0/2251109598 shutdown_connections 2024-09-10T22:00:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.189+0000 7f5d8f600640 1 --2- 172.21.15.66:0/2251109598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d88160280 0x7f5d880b3170 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.189+0000 7f5d8f600640 1 --2- 172.21.15.66:0/2251109598 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d880b1e10 0x7f5d880aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.189+0000 7f5d8f600640 1 -- 172.21.15.66:0/2251109598 >> 172.21.15.66:0/2251109598 conn(0x7f5d880d1c60 msgr2=0x7f5d880d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.189+0000 7f5d8f600640 1 -- 172.21.15.66:0/2251109598 shutdown_connections 2024-09-10T22:00:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.189+0000 7f5d8f600640 1 -- 172.21.15.66:0/2251109598 wait complete. 2024-09-10T22:00:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8f600640 1 Processor -- start 2024-09-10T22:00:27.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8f600640 1 -- start start 2024-09-10T22:00:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8f600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d880b1e10 0x7f5d880cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8f600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d8815f450 0x7f5d88153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8f600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d88160280 0x7f5d881fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8f600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5d880aff30 con 0x7f5d880b1e10 2024-09-10T22:00:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8f600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d880afdb0 con 0x7f5d8815f450 2024-09-10T22:00:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8f600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5d88168eb0 con 0x7f5d88160280 2024-09-10T22:00:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d880b1e10 0x7f5d880cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d880b1e10 0x7f5d880cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35576/0 (socket says 172.21.15.66:35576) 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8cc00640 1 -- 172.21.15.66:0/4099778542 learned_addr learned my addr 172.21.15.66:0/4099778542 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d87e00640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d8815f450 0x7f5d88153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8d600640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d88160280 0x7f5d881fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8cc00640 1 -- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d88160280 msgr2=0x7f5d881fadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8cc00640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d88160280 0x7f5d881fadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8cc00640 1 -- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d8815f450 msgr2=0x7f5d88153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8cc00640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d8815f450 0x7f5d88153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8cc00640 1 -- 172.21.15.66:0/4099778542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d880cd940 con 0x7f5d880b1e10 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8cc00640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d880b1e10 0x7f5d880cd360 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00c500 tx=0x7f5d7c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d84c00640 1 -- 172.21.15.66:0/4099778542 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c018070 con 0x7f5d880b1e10 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d84c00640 1 -- 172.21.15.66:0/4099778542 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d7c003600 con 0x7f5d880b1e10 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d84c00640 1 -- 172.21.15.66:0/4099778542 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c003a90 con 0x7f5d880b1e10 2024-09-10T22:00:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8f600640 1 -- 172.21.15.66:0/4099778542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8806de60 con 0x7f5d880b1e10 2024-09-10T22:00:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.193+0000 7f5d8f600640 1 -- 172.21.15.66:0/4099778542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d8806e420 con 0x7f5d880b1e10 2024-09-10T22:00:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.197+0000 7f5d8f600640 1 -- 172.21.15.66:0/4099778542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8806e620 con 0x7f5d880b1e10 2024-09-10T22:00:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.197+0000 7f5d84c00640 1 -- 172.21.15.66:0/4099778542 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5d7c003c30 con 0x7f5d880b1e10 2024-09-10T22:00:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.197+0000 7f5d84c00640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5d680725e0 0x7f5d68074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.197+0000 7f5d84c00640 1 -- 172.21.15.66:0/4099778542 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f5d7c092230 con 0x7f5d880b1e10 2024-09-10T22:00:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.197+0000 7f5d87e00640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5d680725e0 0x7f5d68074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.201+0000 7f5d87e00640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5d680725e0 0x7f5d68074aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5d740af000 tx=0x7f5d7401b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.201+0000 7f5d84c00640 1 -- 172.21.15.66:0/4099778542 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5d8806e620 con 0x7f5d880b1e10 2024-09-10T22:00:27.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.413+0000 7f5d8f600640 1 -- 172.21.15.66:0/4099778542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d880d2a40 con 0x7f5d880b1e10 2024-09-10T22:00:27.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.413+0000 7f5d84c00640 1 -- 172.21.15.66:0/4099778542 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7f5d7c05d640 con 0x7f5d880b1e10 2024-09-10T22:00:27.420 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:27.420 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":2,"btime":"2024-09-10T22:00:27:134165+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2533443317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2533443317},{"type":"v1","addr":"172.21.15.66:6834","nonce":2533443317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T22:00:27.420 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 2 2024-09-10T22:00:27.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.413+0000 7f5d8f600640 1 -- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5d680725e0 msgr2=0x7f5d68074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.413+0000 7f5d8f600640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5d680725e0 0x7f5d68074aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5d740af000 tx=0x7f5d7401b060 comp rx=0 tx=0).stop 2024-09-10T22:00:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.413+0000 7f5d8f600640 1 -- 172.21.15.66:0/4099778542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d880b1e10 msgr2=0x7f5d880cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.413+0000 7f5d8f600640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d880b1e10 0x7f5d880cd360 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00c500 tx=0x7f5d7c00c9c0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.417+0000 7f5d8f600640 1 -- 172.21.15.66:0/4099778542 shutdown_connections 2024-09-10T22:00:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.417+0000 7f5d8f600640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5d680725e0 0x7f5d68074aa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.417+0000 7f5d8f600640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d88160280 0x7f5d881fadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.417+0000 7f5d8f600640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d8815f450 0x7f5d88153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.417+0000 7f5d8f600640 1 --2- 172.21.15.66:0/4099778542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d880b1e10 0x7f5d880cd360 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.417+0000 7f5d8f600640 1 -- 172.21.15.66:0/4099778542 >> 172.21.15.66:0/4099778542 conn(0x7f5d880d1c60 msgr2=0x7f5d880b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.417+0000 7f5d8f600640 1 -- 172.21.15.66:0/4099778542 shutdown_connections 2024-09-10T22:00:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:27.417+0000 7f5d8f600640 1 -- 172.21.15.66:0/4099778542 wait complete. 2024-09-10T22:00:29.441 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:00:29.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.593+0000 7fa255c00640 1 -- 172.21.15.66:0/730651954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2500d06b0 msgr2=0x7fa2500ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:29.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.593+0000 7fa255c00640 1 --2- 172.21.15.66:0/730651954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2500d06b0 0x7fa2500ce170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009900 tx=0x7fa23c01c850 comp rx=0 tx=0).stop 2024-09-10T22:00:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.593+0000 7fa255c00640 1 -- 172.21.15.66:0/730651954 shutdown_connections 2024-09-10T22:00:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.593+0000 7fa255c00640 1 --2- 172.21.15.66:0/730651954 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2500afc20 0x7fa2500ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.593+0000 7fa255c00640 1 --2- 172.21.15.66:0/730651954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2500d06b0 0x7fa2500ce170 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.593+0000 7fa255c00640 1 -- 172.21.15.66:0/730651954 >> 172.21.15.66:0/730651954 conn(0x7fa2500d1c60 msgr2=0x7fa2500d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.593+0000 7fa255c00640 1 -- 172.21.15.66:0/730651954 shutdown_connections 2024-09-10T22:00:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.593+0000 7fa255c00640 1 -- 172.21.15.66:0/730651954 wait complete. 2024-09-10T22:00:29.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.593+0000 7fa255c00640 1 Processor -- start 2024-09-10T22:00:29.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.593+0000 7fa255c00640 1 -- start start 2024-09-10T22:00:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa255c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2500afc20 0x7fa25015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa255c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2500d06b0 0x7fa2500b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa255c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa25015bd40 0x7fa2501568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa255c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa250169250 con 0x7fa25015bd40 2024-09-10T22:00:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa255c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501690d0 con 0x7fa2500afc20 2024-09-10T22:00:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa255c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa2501693d0 con 0x7fa2500d06b0 2024-09-10T22:00:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa24f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa25015bd40 0x7fa2501568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa24f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa25015bd40 0x7fa2501568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35604/0 (socket says 172.21.15.66:35604) 2024-09-10T22:00:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa24f400640 1 -- 172.21.15.66:0/2097766988 learned_addr learned my addr 172.21.15.66:0/2097766988 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa24ea00640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2500afc20 0x7fa25015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:29.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa24e000640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2500d06b0 0x7fa2500b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:29.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa24f400640 1 -- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2500d06b0 msgr2=0x7fa2500b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:29.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa24f400640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2500d06b0 0x7fa2500b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa24f400640 1 -- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2500afc20 msgr2=0x7fa25015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:29.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa24f400640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2500afc20 0x7fa25015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa24f400640 1 -- 172.21.15.66:0/2097766988 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2500ca490 con 0x7fa25015bd40 2024-09-10T22:00:29.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa24f400640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa25015bd40 0x7fa2501568e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa24800cd10 tx=0x7fa24800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:29.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa246a00640 1 -- 172.21.15.66:0/2097766988 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24800f450 con 0x7fa25015bd40 2024-09-10T22:00:29.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa246a00640 1 -- 172.21.15.66:0/2097766988 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa24800f700 con 0x7fa25015bd40 2024-09-10T22:00:29.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa246a00640 1 -- 172.21.15.66:0/2097766988 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24800fb50 con 0x7fa25015bd40 2024-09-10T22:00:29.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa255c00640 1 -- 172.21.15.66:0/2097766988 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2500ca770 con 0x7fa25015bd40 2024-09-10T22:00:29.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa255c00640 1 -- 172.21.15.66:0/2097766988 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2500cad30 con 0x7fa25015bd40 2024-09-10T22:00:29.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.597+0000 7fa255c00640 1 -- 172.21.15.66:0/2097766988 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2500cffa0 con 0x7fa25015bd40 2024-09-10T22:00:29.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.601+0000 7fa246a00640 1 -- 172.21.15.66:0/2097766988 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa24800e3b0 con 0x7fa25015bd40 2024-09-10T22:00:29.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.601+0000 7fa246a00640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa2300725e0 0x7fa230074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:29.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.601+0000 7fa246a00640 1 -- 172.21.15.66:0/2097766988 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fa248093a30 con 0x7fa25015bd40 2024-09-10T22:00:29.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.601+0000 7fa24ea00640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa2300725e0 0x7fa230074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:29.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.601+0000 7fa24ea00640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa2300725e0 0x7fa230074aa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa2400098e0 tx=0x7fa240009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:29.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.605+0000 7fa246a00640 1 -- 172.21.15.66:0/2097766988 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa248014030 con 0x7fa25015bd40 2024-09-10T22:00:29.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 -- 172.21.15.66:0/2097766988 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2500b0df0 con 0x7fa25015bd40 2024-09-10T22:00:29.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa246a00640 1 -- 172.21.15.66:0/2097766988 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3903 (secure 0 0 0) 0x7fa24805ed70 con 0x7fa25015bd40 2024-09-10T22:00:29.819 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:29.819 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":3,"btime":"2024-09-10T22:00:28:136644+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2728893456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2728893456},{"type":"v1","addr":"172.21.15.94:6835","nonce":2728893456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2533443317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2533443317},{"type":"v1","addr":"172.21.15.66:6834","nonce":2533443317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/317149387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":317149387},{"type":"v1","addr":"172.21.15.66:6835","nonce":317149387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/4271184466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":4271184466},{"type":"v1","addr":"172.21.15.94:6837","nonce":4271184466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T22:00:29.820 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 3 2024-09-10T22:00:29.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 -- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa2300725e0 msgr2=0x7fa230074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:29.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa2300725e0 0x7fa230074aa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa2400098e0 tx=0x7fa240009060 comp rx=0 tx=0).stop 2024-09-10T22:00:29.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 -- 172.21.15.66:0/2097766988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa25015bd40 msgr2=0x7fa2501568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:29.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa25015bd40 0x7fa2501568e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa24800cd10 tx=0x7fa24800a620 comp rx=0 tx=0).stop 2024-09-10T22:00:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 -- 172.21.15.66:0/2097766988 shutdown_connections 2024-09-10T22:00:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa2300725e0 0x7fa230074aa0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa25015bd40 0x7fa2501568e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2500d06b0 0x7fa2500b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 --2- 172.21.15.66:0/2097766988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2500afc20 0x7fa25015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 -- 172.21.15.66:0/2097766988 >> 172.21.15.66:0/2097766988 conn(0x7fa2500d1c60 msgr2=0x7fa25015add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 -- 172.21.15.66:0/2097766988 shutdown_connections 2024-09-10T22:00:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.813+0000 7fa255c00640 1 -- 172.21.15.66:0/2097766988 wait complete. 2024-09-10T22:00:29.840 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:00:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 -- 172.21.15.66:0/1820763140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05415a310 msgr2=0x7fd0540c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 --2- 172.21.15.66:0/1820763140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05415a310 0x7fd0540c48b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd044009900 tx=0x7fd04401c850 comp rx=0 tx=0).stop 2024-09-10T22:00:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 -- 172.21.15.66:0/1820763140 shutdown_connections 2024-09-10T22:00:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 --2- 172.21.15.66:0/1820763140 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0540aeec0 0x7fd0540c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 --2- 172.21.15.66:0/1820763140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05415a310 0x7fd0540c48b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 -- 172.21.15.66:0/1820763140 >> 172.21.15.66:0/1820763140 conn(0x7fd0540cfa30 msgr2=0x7fd0540cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 -- 172.21.15.66:0/1820763140 shutdown_connections 2024-09-10T22:00:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 -- 172.21.15.66:0/1820763140 wait complete. 2024-09-10T22:00:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 Processor -- start 2024-09-10T22:00:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 -- start start 2024-09-10T22:00:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0540aeec0 0x7fd05414c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd05415a310 0x7fd054149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd05414ff70 0x7fd054150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd05415edb0 con 0x7fd0540aeec0 2024-09-10T22:00:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd05415ec30 con 0x7fd05414ff70 2024-09-10T22:00:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd05b000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd054167300 con 0x7fd05415a310 2024-09-10T22:00:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd053e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0540aeec0 0x7fd05414c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd053e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0540aeec0 0x7fd05414c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35626/0 (socket says 172.21.15.66:35626) 2024-09-10T22:00:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.981+0000 7fd053e00640 1 -- 172.21.15.66:0/3760894856 learned_addr learned my addr 172.21.15.66:0/3760894856 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd04be00640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd05415a310 0x7fd054149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:29.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd059000640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd05414ff70 0x7fd054150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:29.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd053e00640 1 -- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd05415a310 msgr2=0x7fd054149370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:29.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd053e00640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd05415a310 0x7fd054149370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd053e00640 1 -- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd05414ff70 msgr2=0x7fd054150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:29.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd053e00640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd05414ff70 0x7fd054150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd053e00640 1 -- 172.21.15.66:0/3760894856 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05414c6f0 con 0x7fd0540aeec0 2024-09-10T22:00:29.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd053e00640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0540aeec0 0x7fd05414c1b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd04401a480 tx=0x7fd0440a64a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:29.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd050c00640 1 -- 172.21.15.66:0/3760894856 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0440ac070 con 0x7fd0540aeec0 2024-09-10T22:00:29.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd050c00640 1 -- 172.21.15.66:0/3760894856 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd044004340 con 0x7fd0540aeec0 2024-09-10T22:00:29.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd050c00640 1 -- 172.21.15.66:0/3760894856 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044005320 con 0x7fd0540aeec0 2024-09-10T22:00:29.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd05b000640 1 -- 172.21.15.66:0/3760894856 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0541498b0 con 0x7fd0540aeec0 2024-09-10T22:00:29.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd05b000640 1 -- 172.21.15.66:0/3760894856 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05415ae90 con 0x7fd0540aeec0 2024-09-10T22:00:29.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd05b000640 1 -- 172.21.15.66:0/3760894856 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd054070420 con 0x7fd0540aeec0 2024-09-10T22:00:29.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd050c00640 1 -- 172.21.15.66:0/3760894856 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd0440054c0 con 0x7fd0540aeec0 2024-09-10T22:00:29.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.985+0000 7fd050c00640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd0300725e0 0x7fd030074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:29.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.989+0000 7fd050c00640 1 -- 172.21.15.66:0/3760894856 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fd04412d2e0 con 0x7fd0540aeec0 2024-09-10T22:00:29.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.989+0000 7fd04be00640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd0300725e0 0x7fd030074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:29.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.989+0000 7fd04be00640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd0300725e0 0x7fd030074aa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd0541f9a50 tx=0x7fd03c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:29.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:29.993+0000 7fd050c00640 1 -- 172.21.15.66:0/3760894856 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd05414c6f0 con 0x7fd0540aeec0 2024-09-10T22:00:30.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 -- 172.21.15.66:0/3760894856 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd054076650 con 0x7fd0540aeec0 2024-09-10T22:00:30.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd050c00640 1 -- 172.21.15.66:0/3760894856 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7fd0440164d0 con 0x7fd0540aeec0 2024-09-10T22:00:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:30.175 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:00:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:00:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:00:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:00:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:00:30.175 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:00:30.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 -- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd0300725e0 msgr2=0x7fd030074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:30.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd0300725e0 0x7fd030074aa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd0541f9a50 tx=0x7fd03c009040 comp rx=0 tx=0).stop 2024-09-10T22:00:30.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 -- 172.21.15.66:0/3760894856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0540aeec0 msgr2=0x7fd05414c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:30.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0540aeec0 0x7fd05414c1b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd04401a480 tx=0x7fd0440a64a0 comp rx=0 tx=0).stop 2024-09-10T22:00:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 -- 172.21.15.66:0/3760894856 shutdown_connections 2024-09-10T22:00:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd0300725e0 0x7fd030074aa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd05414ff70 0x7fd054150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd05415a310 0x7fd054149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 --2- 172.21.15.66:0/3760894856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0540aeec0 0x7fd05414c1b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 -- 172.21.15.66:0/3760894856 >> 172.21.15.66:0/3760894856 conn(0x7fd0540cfa30 msgr2=0x7fd0540cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 -- 172.21.15.66:0/3760894856 shutdown_connections 2024-09-10T22:00:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.169+0000 7fd05b000640 1 -- 172.21.15.66:0/3760894856 wait complete. 2024-09-10T22:00:30.196 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:00:30.196 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:00:30.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.341+0000 7f1d81200640 1 -- 172.21.15.66:0/2836341732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d7c0ce8d0 msgr2=0x7f1d7c0cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:30.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.341+0000 7f1d81200640 1 --2- 172.21.15.66:0/2836341732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d7c0ce8d0 0x7f1d7c0cecb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009900 tx=0x7f1d6c01c8e0 comp rx=0 tx=0).stop 2024-09-10T22:00:30.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.341+0000 7f1d81200640 1 -- 172.21.15.66:0/2836341732 shutdown_connections 2024-09-10T22:00:30.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.341+0000 7f1d81200640 1 --2- 172.21.15.66:0/2836341732 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1d7c0cdf90 0x7f1d7c0cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:30.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.341+0000 7f1d81200640 1 --2- 172.21.15.66:0/2836341732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d7c0ce8d0 0x7f1d7c0cecb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:30.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.341+0000 7f1d81200640 1 -- 172.21.15.66:0/2836341732 >> 172.21.15.66:0/2836341732 conn(0x7f1d7c0d1c60 msgr2=0x7f1d7c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:30.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.341+0000 7f1d81200640 1 -- 172.21.15.66:0/2836341732 shutdown_connections 2024-09-10T22:00:30.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.341+0000 7f1d81200640 1 -- 172.21.15.66:0/2836341732 wait complete. 2024-09-10T22:00:30.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d81200640 1 Processor -- start 2024-09-10T22:00:30.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d81200640 1 -- start start 2024-09-10T22:00:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d81200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1d7c0cdf90 0x7f1d7c0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d81200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d7c0ce8d0 0x7f1d7c15bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d81200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c0cf1f0 0x7f1d7c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d81200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c1691a0 con 0x7f1d7c0ce8d0 2024-09-10T22:00:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d81200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c169020 con 0x7f1d7c0cf1f0 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d81200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7c169320 con 0x7f1d7c0cdf90 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d73e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d7c0ce8d0 0x7f1d7c15bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d73e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d7c0ce8d0 0x7f1d7c15bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35638/0 (socket says 172.21.15.66:35638) 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d73e00640 1 -- 172.21.15.66:0/2043158983 learned_addr learned my addr 172.21.15.66:0/2043158983 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d79e00640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1d7c0cdf90 0x7f1d7c0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d7a800640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c0cf1f0 0x7f1d7c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d73e00640 1 -- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1d7c0cdf90 msgr2=0x7f1d7c0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d73e00640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1d7c0cdf90 0x7f1d7c0afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d73e00640 1 -- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c0cf1f0 msgr2=0x7f1d7c15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d73e00640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c0cf1f0 0x7f1d7c15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d73e00640 1 -- 172.21.15.66:0/2043158983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7c0d1a40 con 0x7f1d7c0ce8d0 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d79e00640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1d7c0cdf90 0x7f1d7c0afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d73e00640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d7c0ce8d0 0x7f1d7c15bd00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400c970 tx=0x7f1d6400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d72000640 1 -- 172.21.15.66:0/2043158983 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64007c70 con 0x7f1d7c0ce8d0 2024-09-10T22:00:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d72000640 1 -- 172.21.15.66:0/2043158983 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d6400f450 con 0x7f1d7c0ce8d0 2024-09-10T22:00:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d72000640 1 -- 172.21.15.66:0/2043158983 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400f8a0 con 0x7f1d7c0ce8d0 2024-09-10T22:00:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d81200640 1 -- 172.21.15.66:0/2043158983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7c0b7af0 con 0x7f1d7c0ce8d0 2024-09-10T22:00:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.345+0000 7f1d81200640 1 -- 172.21.15.66:0/2043158983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d7c0b7290 con 0x7f1d7c0ce8d0 2024-09-10T22:00:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.349+0000 7f1d81200640 1 -- 172.21.15.66:0/2043158983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d7c0cecb0 con 0x7f1d7c0ce8d0 2024-09-10T22:00:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.349+0000 7f1d72000640 1 -- 172.21.15.66:0/2043158983 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1d6400fa40 con 0x7f1d7c0ce8d0 2024-09-10T22:00:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.349+0000 7f1d72000640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1d580725e0 0x7f1d58074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:30.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.349+0000 7f1d72000640 1 -- 172.21.15.66:0/2043158983 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f1d64093820 con 0x7f1d7c0ce8d0 2024-09-10T22:00:30.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.349+0000 7f1d79e00640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1d580725e0 0x7f1d58074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:30.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.349+0000 7f1d79e00640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1d580725e0 0x7f1d58074aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0af000 tx=0x7f1d6c01b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:30.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.353+0000 7f1d72000640 1 -- 172.21.15.66:0/2043158983 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1d64014030 con 0x7f1d7c0ce8d0 2024-09-10T22:00:30.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:30.529+0000 7f1d81200640 1 -- 172.21.15.66:0/2043158983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1d7c070550 con 0x7f1d7c0ce8d0 2024-09-10T22:00:31.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.149+0000 7f1d72000640 1 -- 172.21.15.66:0/2043158983 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f1d6405f1b0 con 0x7f1d7c0ce8d0 2024-09-10T22:00:31.155 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:00:31.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.149+0000 7f1d81200640 1 -- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1d580725e0 msgr2=0x7f1d58074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:31.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.149+0000 7f1d81200640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1d580725e0 0x7f1d58074aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0af000 tx=0x7f1d6c01b120 comp rx=0 tx=0).stop 2024-09-10T22:00:31.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.149+0000 7f1d81200640 1 -- 172.21.15.66:0/2043158983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d7c0ce8d0 msgr2=0x7f1d7c15bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:31.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.149+0000 7f1d81200640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d7c0ce8d0 0x7f1d7c15bd00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400c970 tx=0x7f1d6400ce30 comp rx=0 tx=0).stop 2024-09-10T22:00:31.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.149+0000 7f1d81200640 1 -- 172.21.15.66:0/2043158983 shutdown_connections 2024-09-10T22:00:31.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.149+0000 7f1d81200640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1d580725e0 0x7f1d58074aa0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:31.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.149+0000 7f1d81200640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c0cf1f0 0x7f1d7c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:31.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.149+0000 7f1d81200640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d7c0ce8d0 0x7f1d7c15bd00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:31.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.149+0000 7f1d81200640 1 --2- 172.21.15.66:0/2043158983 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1d7c0cdf90 0x7f1d7c0afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:31.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.149+0000 7f1d81200640 1 -- 172.21.15.66:0/2043158983 >> 172.21.15.66:0/2043158983 conn(0x7f1d7c0d1c60 msgr2=0x7f1d7c06f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:31.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.149+0000 7f1d81200640 1 -- 172.21.15.66:0/2043158983 shutdown_connections 2024-09-10T22:00:31.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.153+0000 7f1d81200640 1 -- 172.21.15.66:0/2043158983 wait complete. 2024-09-10T22:00:31.186 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:00:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.337+0000 7f3f55800640 1 -- 172.21.15.66:0/1592475061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f500b32a0 msgr2=0x7f3f500af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.337+0000 7f3f55800640 1 --2- 172.21.15.66:0/1592475061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f500b32a0 0x7f3f500af390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009c80 tx=0x7f3f4801cbf0 comp rx=0 tx=0).stop 2024-09-10T22:00:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.337+0000 7f3f55800640 1 -- 172.21.15.66:0/1592475061 shutdown_connections 2024-09-10T22:00:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.337+0000 7f3f55800640 1 --2- 172.21.15.66:0/1592475061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f500b32a0 0x7f3f500af390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.337+0000 7f3f55800640 1 --2- 172.21.15.66:0/1592475061 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f500b0f60 0x7f3f500d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.337+0000 7f3f55800640 1 -- 172.21.15.66:0/1592475061 >> 172.21.15.66:0/1592475061 conn(0x7f3f500d1d00 msgr2=0x7f3f500d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.337+0000 7f3f55800640 1 -- 172.21.15.66:0/1592475061 shutdown_connections 2024-09-10T22:00:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.337+0000 7f3f55800640 1 -- 172.21.15.66:0/1592475061 wait complete. 2024-09-10T22:00:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.337+0000 7f3f55800640 1 Processor -- start 2024-09-10T22:00:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f55800640 1 -- start start 2024-09-10T22:00:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f55800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f500b0f60 0x7f3f50146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f55800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f500b32a0 0x7f3f501f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f55800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f5015f460 0x7f3f501f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f55800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f500b0600 con 0x7f3f500b0f60 2024-09-10T22:00:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f55800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f500b0480 con 0x7f3f5015f460 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f55800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3f50168c60 con 0x7f3f500b32a0 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f500b0f60 0x7f3f50146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f500b0f60 0x7f3f50146d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35646/0 (socket says 172.21.15.66:35646) 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4ea00640 1 -- 172.21.15.66:0/4243396946 learned_addr learned my addr 172.21.15.66:0/4243396946 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4f400640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f5015f460 0x7f3f501f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4e000640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f500b32a0 0x7f3f501f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4ea00640 1 -- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f500b32a0 msgr2=0x7f3f501f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4ea00640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f500b32a0 0x7f3f501f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4ea00640 1 -- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f5015f460 msgr2=0x7f3f501f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4ea00640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f5015f460 0x7f3f501f69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4ea00640 1 -- 172.21.15.66:0/4243396946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f500c8660 con 0x7f3f500b0f60 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4f400640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f5015f460 0x7f3f501f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4e000640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f500b32a0 0x7f3f501f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:31.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f4ea00640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f500b0f60 0x7f3f50146d20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c3c0 tx=0x7f3f4000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:31.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f46a00640 1 -- 172.21.15.66:0/4243396946 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40017070 con 0x7f3f500b0f60 2024-09-10T22:00:31.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f46a00640 1 -- 172.21.15.66:0/4243396946 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f400053c0 con 0x7f3f500b0f60 2024-09-10T22:00:31.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f46a00640 1 -- 172.21.15.66:0/4243396946 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40003860 con 0x7f3f500b0f60 2024-09-10T22:00:31.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f55800640 1 -- 172.21.15.66:0/4243396946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f501f6180 con 0x7f3f500b0f60 2024-09-10T22:00:31.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f55800640 1 -- 172.21.15.66:0/4243396946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f500c9230 con 0x7f3f500b0f60 2024-09-10T22:00:31.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.341+0000 7f3f55800640 1 -- 172.21.15.66:0/4243396946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f500af390 con 0x7f3f500b0f60 2024-09-10T22:00:31.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.345+0000 7f3f46a00640 1 -- 172.21.15.66:0/4243396946 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3f40003a00 con 0x7f3f500b0f60 2024-09-10T22:00:31.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.345+0000 7f3f46a00640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3f30072610 0x7f3f30074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:31.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.345+0000 7f3f46a00640 1 -- 172.21.15.66:0/4243396946 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f3f40092660 con 0x7f3f500b0f60 2024-09-10T22:00:31.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.345+0000 7f3f4e000640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3f30072610 0x7f3f30074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:31.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.345+0000 7f3f4e000640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3f30072610 0x7f3f30074ad0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3f500c8260 tx=0x7f3f3c00b2c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:31.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.349+0000 7f3f46a00640 1 -- 172.21.15.66:0/4243396946 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3f4005df10 con 0x7f3f500b0f60 2024-09-10T22:00:31.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:31.525+0000 7f3f55800640 1 -- 172.21.15.66:0/4243396946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3f50074aa0 con 0x7f3f500b0f60 2024-09-10T22:00:32.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.153+0000 7f3f46a00640 1 -- 172.21.15.66:0/4243396946 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7f3f4005d810 con 0x7f3f500b0f60 2024-09-10T22:00:32.161 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:00:32.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.157+0000 7f3f55800640 1 -- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3f30072610 msgr2=0x7f3f30074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:32.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.157+0000 7f3f55800640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3f30072610 0x7f3f30074ad0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3f500c8260 tx=0x7f3f3c00b2c0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.157+0000 7f3f55800640 1 -- 172.21.15.66:0/4243396946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f500b0f60 msgr2=0x7f3f50146d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:32.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.157+0000 7f3f55800640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f500b0f60 0x7f3f50146d20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c3c0 tx=0x7f3f4000c880 comp rx=0 tx=0).stop 2024-09-10T22:00:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.161+0000 7f3f55800640 1 -- 172.21.15.66:0/4243396946 shutdown_connections 2024-09-10T22:00:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.161+0000 7f3f55800640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3f30072610 0x7f3f30074ad0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.161+0000 7f3f55800640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f5015f460 0x7f3f501f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.161+0000 7f3f55800640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f500b32a0 0x7f3f501f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.161+0000 7f3f55800640 1 --2- 172.21.15.66:0/4243396946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f500b0f60 0x7f3f50146d20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.161+0000 7f3f55800640 1 -- 172.21.15.66:0/4243396946 >> 172.21.15.66:0/4243396946 conn(0x7f3f500d1d00 msgr2=0x7f3f500b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:32.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.161+0000 7f3f55800640 1 -- 172.21.15.66:0/4243396946 shutdown_connections 2024-09-10T22:00:32.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.161+0000 7f3f55800640 1 -- 172.21.15.66:0/4243396946 wait complete. 2024-09-10T22:00:32.204 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:00:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 -- 172.21.15.66:0/710099106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6628071230 msgr2=0x7f662806d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 --2- 172.21.15.66:0/710099106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6628071230 0x7f662806d630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6614009840 tx=0x7f662815d120 comp rx=0 tx=0).stop 2024-09-10T22:00:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 -- 172.21.15.66:0/710099106 shutdown_connections 2024-09-10T22:00:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 --2- 172.21.15.66:0/710099106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6628071230 0x7f662806d630 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 -- 172.21.15.66:0/710099106 >> 172.21.15.66:0/710099106 conn(0x7f66280d0b30 msgr2=0x7f66280d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 -- 172.21.15.66:0/710099106 shutdown_connections 2024-09-10T22:00:32.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 -- 172.21.15.66:0/710099106 wait complete. 2024-09-10T22:00:32.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 Processor -- start 2024-09-10T22:00:32.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 -- start start 2024-09-10T22:00:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6628071230 0x7f66280b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6628156430 0x7f66280ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6628160370 0x7f66280b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f662816a1a0 con 0x7f6628156430 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f662816a020 con 0x7f6628071230 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f662d400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f662816a320 con 0x7f6628160370 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f6625e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6628156430 0x7f66280ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f6625e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6628156430 0x7f66280ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35654/0 (socket says 172.21.15.66:35654) 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f6625e00640 1 -- 172.21.15.66:0/2102573652 learned_addr learned my addr 172.21.15.66:0/2102573652 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f6627200640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6628160370 0x7f66280b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f6626800640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6628071230 0x7f66280b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f6625e00640 1 -- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6628160370 msgr2=0x7f66280b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f6625e00640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6628160370 0x7f66280b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f6625e00640 1 -- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6628071230 msgr2=0x7f66280b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f6625e00640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6628071230 0x7f66280b6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.349+0000 7f6625e00640 1 -- 172.21.15.66:0/2102573652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662815d790 con 0x7f6628156430 2024-09-10T22:00:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.353+0000 7f6625e00640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6628156430 0x7f66280ca0b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f661401a2b0 tx=0x7f66140a7780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:32.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.353+0000 7f661ea00640 1 -- 172.21.15.66:0/2102573652 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66140bc070 con 0x7f6628156430 2024-09-10T22:00:32.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.353+0000 7f661ea00640 1 -- 172.21.15.66:0/2102573652 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66140a7d20 con 0x7f6628156430 2024-09-10T22:00:32.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.353+0000 7f662d400640 1 -- 172.21.15.66:0/2102573652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66281f7f70 con 0x7f6628156430 2024-09-10T22:00:32.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.353+0000 7f661ea00640 1 -- 172.21.15.66:0/2102573652 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614005280 con 0x7f6628156430 2024-09-10T22:00:32.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.353+0000 7f662d400640 1 -- 172.21.15.66:0/2102573652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66281f8530 con 0x7f6628156430 2024-09-10T22:00:32.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.353+0000 7f662d400640 1 -- 172.21.15.66:0/2102573652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f662806d630 con 0x7f6628156430 2024-09-10T22:00:32.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.353+0000 7f661ea00640 1 -- 172.21.15.66:0/2102573652 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6614005420 con 0x7f6628156430 2024-09-10T22:00:32.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.353+0000 7f661ea00640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f66080725e0 0x7f6608074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:32.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.357+0000 7f661ea00640 1 -- 172.21.15.66:0/2102573652 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f661412ea70 con 0x7f6628156430 2024-09-10T22:00:32.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.357+0000 7f6626800640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f66080725e0 0x7f6608074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:32.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.357+0000 7f6626800640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f66080725e0 0x7f6608074aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6618001230 tx=0x7f6618009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:32.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.357+0000 7f661ea00640 1 -- 172.21.15.66:0/2102573652 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6614016330 con 0x7f6628156430 2024-09-10T22:00:32.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:32.569+0000 7f662d400640 1 -- 172.21.15.66:0/2102573652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f662815a260 con 0x7f6628156430 2024-09-10T22:00:33.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.169+0000 7f661ea00640 1 -- 172.21.15.66:0/2102573652 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f66140fa220 con 0x7f6628156430 2024-09-10T22:00:33.176 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T22:00:33.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.181+0000 7f662d400640 1 -- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f66080725e0 msgr2=0x7f6608074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.181+0000 7f662d400640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f66080725e0 0x7f6608074aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6618001230 tx=0x7f6618009040 comp rx=0 tx=0).stop 2024-09-10T22:00:33.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.181+0000 7f662d400640 1 -- 172.21.15.66:0/2102573652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6628156430 msgr2=0x7f66280ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.181+0000 7f662d400640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6628156430 0x7f66280ca0b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f661401a2b0 tx=0x7f66140a7780 comp rx=0 tx=0).stop 2024-09-10T22:00:33.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.181+0000 7f662d400640 1 -- 172.21.15.66:0/2102573652 shutdown_connections 2024-09-10T22:00:33.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.181+0000 7f662d400640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f66080725e0 0x7f6608074aa0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.181+0000 7f662d400640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6628160370 0x7f66280b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.181+0000 7f662d400640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6628156430 0x7f66280ca0b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.181+0000 7f662d400640 1 --2- 172.21.15.66:0/2102573652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6628071230 0x7f66280b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.181+0000 7f662d400640 1 -- 172.21.15.66:0/2102573652 >> 172.21.15.66:0/2102573652 conn(0x7f66280d0b30 msgr2=0x7f66280c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:33.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.181+0000 7f662d400640 1 -- 172.21.15.66:0/2102573652 shutdown_connections 2024-09-10T22:00:33.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.181+0000 7f662d400640 1 -- 172.21.15.66:0/2102573652 wait complete. 2024-09-10T22:00:33.219 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:00:33.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.357+0000 7fc11d800640 1 -- 172.21.15.66:0/2833127571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1180cf180 msgr2=0x7fc1180d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.357+0000 7fc11d800640 1 --2- 172.21.15.66:0/2833127571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1180cf180 0x7fc1180d04e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc110009c80 tx=0x7fc11001cd10 comp rx=0 tx=0).stop 2024-09-10T22:00:33.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.357+0000 7fc11d800640 1 -- 172.21.15.66:0/2833127571 shutdown_connections 2024-09-10T22:00:33.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.357+0000 7fc11d800640 1 --2- 172.21.15.66:0/2833127571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1180cf180 0x7fc1180d04e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.357+0000 7fc11d800640 1 --2- 172.21.15.66:0/2833127571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc11815bd40 0x7fc1180cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.357+0000 7fc11d800640 1 -- 172.21.15.66:0/2833127571 >> 172.21.15.66:0/2833127571 conn(0x7fc1180d1c60 msgr2=0x7fc1180d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:33.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.357+0000 7fc11d800640 1 -- 172.21.15.66:0/2833127571 shutdown_connections 2024-09-10T22:00:33.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.357+0000 7fc11d800640 1 -- 172.21.15.66:0/2833127571 wait complete. 2024-09-10T22:00:33.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc11d800640 1 Processor -- start 2024-09-10T22:00:33.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc11d800640 1 -- start start 2024-09-10T22:00:33.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc11d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1180cf180 0x7fc1181fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:33.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc11d800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1180cf830 0x7fc1180c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:33.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc11d800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc11815bd40 0x7fc118156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:33.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc11d800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc118168ba0 con 0x7fc1180cf180 2024-09-10T22:00:33.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc11d800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc118168a20 con 0x7fc1180cf830 2024-09-10T22:00:33.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc11d800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc118168d20 con 0x7fc11815bd40 2024-09-10T22:00:33.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc116a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1180cf180 0x7fc1181fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:33.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc10fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1180cf830 0x7fc1180c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:33.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc117400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc11815bd40 0x7fc118156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:33.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc116a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1180cf180 0x7fc1181fbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60030/0 (socket says 172.21.15.66:60030) 2024-09-10T22:00:33.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc116a00640 1 -- 172.21.15.66:0/1227190158 learned_addr learned my addr 172.21.15.66:0/1227190158 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:33.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc117400640 1 -- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1180cf830 msgr2=0x7fc1180c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc117400640 1 --2- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1180cf830 0x7fc1180c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc117400640 1 -- 172.21.15.66:0/1227190158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1180cf180 msgr2=0x7fc1181fbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc117400640 1 --2- 172.21.15.66:0/1227190158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1180cf180 0x7fc1181fbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc117400640 1 -- 172.21.15.66:0/1227190158 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1180c9270 con 0x7fc11815bd40 2024-09-10T22:00:33.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc117400640 1 --2- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc11815bd40 0x7fc118156130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc1100ac000 tx=0x7fc110009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:33.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc10ea00640 1 -- 172.21.15.66:0/1227190158 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100bb070 con 0x7fc11815bd40 2024-09-10T22:00:33.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc10ea00640 1 -- 172.21.15.66:0/1227190158 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc110004d40 con 0x7fc11815bd40 2024-09-10T22:00:33.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc11d800640 1 -- 172.21.15.66:0/1227190158 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11815a830 con 0x7fc11815bd40 2024-09-10T22:00:33.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc10ea00640 1 -- 172.21.15.66:0/1227190158 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100b2040 con 0x7fc11815bd40 2024-09-10T22:00:33.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.361+0000 7fc11d800640 1 -- 172.21.15.66:0/1227190158 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1181fb2a0 con 0x7fc11815bd40 2024-09-10T22:00:33.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.365+0000 7fc11d800640 1 -- 172.21.15.66:0/1227190158 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1180aec10 con 0x7fc11815bd40 2024-09-10T22:00:33.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.365+0000 7fc10ea00640 1 -- 172.21.15.66:0/1227190158 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc1100b5030 con 0x7fc11815bd40 2024-09-10T22:00:33.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.365+0000 7fc10ea00640 1 --2- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc0f40725e0 0x7fc0f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:33.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.365+0000 7fc10ea00640 1 -- 172.21.15.66:0/1227190158 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc11012cd30 con 0x7fc11815bd40 2024-09-10T22:00:33.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.365+0000 7fc116a00640 1 --2- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc0f40725e0 0x7fc0f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:33.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.369+0000 7fc116a00640 1 --2- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc0f40725e0 0x7fc0f4074aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc108009c00 tx=0x7fc1080091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:33.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.373+0000 7fc10ea00640 1 -- 172.21.15.66:0/1227190158 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc110132020 con 0x7fc11815bd40 2024-09-10T22:00:33.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.545+0000 7fc11d800640 1 -- 172.21.15.66:0/1227190158 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1180b0df0 con 0x7fc11815bd40 2024-09-10T22:00:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc10ea00640 1 -- 172.21.15.66:0/1227190158 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16606 (secure 0 0 0) 0x7fc110016990 con 0x7fc11815bd40 2024-09-10T22:00:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:33.554 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":21,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:32.573738+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:30.545203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:31.538729+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:33.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc11d800640 1 -- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc0f40725e0 msgr2=0x7fc0f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc11d800640 1 --2- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc0f40725e0 0x7fc0f4074aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc108009c00 tx=0x7fc1080091e0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc11d800640 1 -- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc11815bd40 msgr2=0x7fc118156130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc11d800640 1 --2- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc11815bd40 0x7fc118156130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc1100ac000 tx=0x7fc110009330 comp rx=0 tx=0).stop 2024-09-10T22:00:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc11d800640 1 -- 172.21.15.66:0/1227190158 shutdown_connections 2024-09-10T22:00:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc11d800640 1 --2- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc0f40725e0 0x7fc0f4074aa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc11d800640 1 --2- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc11815bd40 0x7fc118156130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc11d800640 1 --2- 172.21.15.66:0/1227190158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1180cf830 0x7fc1180c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc11d800640 1 --2- 172.21.15.66:0/1227190158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1180cf180 0x7fc1181fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc11d800640 1 -- 172.21.15.66:0/1227190158 >> 172.21.15.66:0/1227190158 conn(0x7fc1180d1c60 msgr2=0x7fc1180b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc11d800640 1 -- 172.21.15.66:0/1227190158 shutdown_connections 2024-09-10T22:00:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.549+0000 7fc11d800640 1 -- 172.21.15.66:0/1227190158 wait complete. 2024-09-10T22:00:33.576 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:00:33.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.713+0000 7f8dbe000640 1 -- 172.21.15.66:0/2610352744 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8db80b0f60 msgr2=0x7f8db80d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.713+0000 7f8dbe000640 1 --2- 172.21.15.66:0/2610352744 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8db80b0f60 0x7f8db80d0310 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009840 tx=0x7f8da001c6d0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 -- 172.21.15.66:0/2610352744 shutdown_connections 2024-09-10T22:00:33.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 --2- 172.21.15.66:0/2610352744 >> v2:172.21.15.94:3300/0 conn(0x7f8db80b32a0 0x7f8db80af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 --2- 172.21.15.66:0/2610352744 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8db80b0f60 0x7f8db80d0310 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 -- 172.21.15.66:0/2610352744 >> 172.21.15.66:0/2610352744 conn(0x7f8db80d1d00 msgr2=0x7f8db80d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:33.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 -- 172.21.15.66:0/2610352744 shutdown_connections 2024-09-10T22:00:33.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 -- 172.21.15.66:0/2610352744 wait complete. 2024-09-10T22:00:33.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 Processor -- start 2024-09-10T22:00:33.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 -- start start 2024-09-10T22:00:33.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db80b0f60 0x7f8db8151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:33.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8db80b32a0 0x7f8db814efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:33.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8db815f460 0x7f8db80cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:33.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8db80b0550 con 0x7f8db80b0f60 2024-09-10T22:00:33.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8db80b03d0 con 0x7f8db815f460 2024-09-10T22:00:33.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dbe000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8db814f960 con 0x7f8db80b32a0 2024-09-10T22:00:33.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8db7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db80b0f60 0x7f8db8151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:33.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8db7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db80b0f60 0x7f8db8151bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60046/0 (socket says 172.21.15.66:60046) 2024-09-10T22:00:33.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8db7400640 1 -- 172.21.15.66:0/1849306723 learned_addr learned my addr 172.21.15.66:0/1849306723 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:33.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dafe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8db80b32a0 0x7f8db814efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:33.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8db7e00640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8db815f460 0x7f8db80cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:33.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dafe00640 1 -- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8db815f460 msgr2=0x7f8db80cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dafe00640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8db815f460 0x7f8db80cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dafe00640 1 -- 172.21.15.66:0/1849306723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db80b0f60 msgr2=0x7f8db8151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dafe00640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db80b0f60 0x7f8db8151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dafe00640 1 -- 172.21.15.66:0/1849306723 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db8157ff0 con 0x7f8db80b32a0 2024-09-10T22:00:33.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.717+0000 7f8dafe00640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8db80b32a0 0x7f8db814efa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8da00ac000 tx=0x7f8da00a56a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:33.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.721+0000 7f8daf400640 1 -- 172.21.15.66:0/1849306723 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da00b2040 con 0x7f8db80b32a0 2024-09-10T22:00:33.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.721+0000 7f8daf400640 1 -- 172.21.15.66:0/1849306723 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8da0004400 con 0x7f8db80b32a0 2024-09-10T22:00:33.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.721+0000 7f8dbe000640 1 -- 172.21.15.66:0/1849306723 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db81582d0 con 0x7f8db80b32a0 2024-09-10T22:00:33.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.721+0000 7f8daf400640 1 -- 172.21.15.66:0/1849306723 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0005280 con 0x7f8db80b32a0 2024-09-10T22:00:33.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.721+0000 7f8dbe000640 1 -- 172.21.15.66:0/1849306723 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db8158f60 con 0x7f8db80b32a0 2024-09-10T22:00:33.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.721+0000 7f8dbe000640 1 -- 172.21.15.66:0/1849306723 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db80af390 con 0x7f8db80b32a0 2024-09-10T22:00:33.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.721+0000 7f8daf400640 1 -- 172.21.15.66:0/1849306723 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8da00b5070 con 0x7f8db80b32a0 2024-09-10T22:00:33.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.721+0000 7f8daf400640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d940725e0 0x7f8d94074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:33.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.721+0000 7f8daf400640 1 -- 172.21.15.66:0/1849306723 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8da012c870 con 0x7f8db80b32a0 2024-09-10T22:00:33.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.721+0000 7f8db7400640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d940725e0 0x7f8d94074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:33.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.725+0000 7f8db7400640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d940725e0 0x7f8d94074aa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8da8000f70 tx=0x7f8da8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:33.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.725+0000 7f8daf400640 1 -- 172.21.15.66:0/1849306723 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8da0016410 con 0x7f8db80b32a0 2024-09-10T22:00:33.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.901+0000 7f8dbe000640 1 -- 172.21.15.66:0/1849306723 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8db81584a0 con 0x7f8db80b32a0 2024-09-10T22:00:33.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.901+0000 7f8daf400640 1 -- 172.21.15.66:0/1849306723 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16606 (secure 0 0 0) 0x7f8da00f78f0 con 0x7f8db80b32a0 2024-09-10T22:00:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:33.907 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":21,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:32.573738+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:30.545203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:31.538729+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:33.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.905+0000 7f8dbe000640 1 -- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d940725e0 msgr2=0x7f8d94074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.905+0000 7f8dbe000640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d940725e0 0x7f8d94074aa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8da8000f70 tx=0x7f8da8009040 comp rx=0 tx=0).stop 2024-09-10T22:00:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.905+0000 7f8dbe000640 1 -- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8db80b32a0 msgr2=0x7f8db814efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.905+0000 7f8dbe000640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8db80b32a0 0x7f8db814efa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8da00ac000 tx=0x7f8da00a56a0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.905+0000 7f8dbe000640 1 -- 172.21.15.66:0/1849306723 shutdown_connections 2024-09-10T22:00:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.905+0000 7f8dbe000640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d940725e0 0x7f8d94074aa0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.905+0000 7f8dbe000640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8db815f460 0x7f8db80cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.905+0000 7f8dbe000640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8db80b32a0 0x7f8db814efa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.905+0000 7f8dbe000640 1 --2- 172.21.15.66:0/1849306723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db80b0f60 0x7f8db8151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.905+0000 7f8dbe000640 1 -- 172.21.15.66:0/1849306723 >> 172.21.15.66:0/1849306723 conn(0x7f8db80d1d00 msgr2=0x7f8db8070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.905+0000 7f8dbe000640 1 -- 172.21.15.66:0/1849306723 shutdown_connections 2024-09-10T22:00:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:33.905+0000 7f8dbe000640 1 -- 172.21.15.66:0/1849306723 wait complete. 2024-09-10T22:00:33.928 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:00:34.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 -- 172.21.15.66:0/3393681074 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53fc0b32a0 msgr2=0x7f53fc0d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:34.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 --2- 172.21.15.66:0/3393681074 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53fc0b32a0 0x7f53fc0d0310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009bf0 tx=0x7f53f401c9e0 comp rx=0 tx=0).stop 2024-09-10T22:00:34.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 -- 172.21.15.66:0/3393681074 shutdown_connections 2024-09-10T22:00:34.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 --2- 172.21.15.66:0/3393681074 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53fc0b32a0 0x7f53fc0d0310 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:34.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 -- 172.21.15.66:0/3393681074 >> 172.21.15.66:0/3393681074 conn(0x7f53fc0d1d00 msgr2=0x7f53fc0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:34.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 -- 172.21.15.66:0/3393681074 shutdown_connections 2024-09-10T22:00:34.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 -- 172.21.15.66:0/3393681074 wait complete. 2024-09-10T22:00:34.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 Processor -- start 2024-09-10T22:00:34.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 -- start start 2024-09-10T22:00:34.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53fc0b0f60 0x7f53fc06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:34.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53fc0b32a0 0x7f53fc06f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:34.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53fc15f460 0x7f53fc1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc169e80 con 0x7f53fc15f460 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc0b06c0 con 0x7f53fc0b32a0 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5403c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f53fc16a000 con 0x7f53fc0b0f60 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5401c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53fc15f460 0x7f53fc1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5401c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53fc15f460 0x7f53fc1539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60052/0 (socket says 172.21.15.66:60052) 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5401c00640 1 -- 172.21.15.66:0/2175135874 learned_addr learned my addr 172.21.15.66:0/2175135874 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5401200640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53fc0b0f60 0x7f53fc06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f53fbe00640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53fc0b32a0 0x7f53fc06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5401c00640 1 -- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53fc0b0f60 msgr2=0x7f53fc06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5401c00640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53fc0b0f60 0x7f53fc06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5401c00640 1 -- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53fc0b32a0 msgr2=0x7f53fc06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5401c00640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53fc0b32a0 0x7f53fc06f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5401c00640 1 -- 172.21.15.66:0/2175135874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53fc0b2900 con 0x7f53fc15f460 2024-09-10T22:00:34.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.073+0000 7f5401200640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53fc0b0f60 0x7f53fc06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:34.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.077+0000 7f5401c00640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53fc15f460 0x7f53fc1539f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f53f40b0040 tx=0x7f53f4005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:34.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.077+0000 7f53f8c00640 1 -- 172.21.15.66:0/2175135874 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40bb070 con 0x7f53fc15f460 2024-09-10T22:00:34.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.077+0000 7f53f8c00640 1 -- 172.21.15.66:0/2175135874 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53f4004e00 con 0x7f53fc15f460 2024-09-10T22:00:34.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.077+0000 7f53f8c00640 1 -- 172.21.15.66:0/2175135874 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40b2040 con 0x7f53fc15f460 2024-09-10T22:00:34.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.077+0000 7f5403c00640 1 -- 172.21.15.66:0/2175135874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53fc15b6e0 con 0x7f53fc15f460 2024-09-10T22:00:34.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.077+0000 7f5403c00640 1 -- 172.21.15.66:0/2175135874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53fc15bca0 con 0x7f53fc15f460 2024-09-10T22:00:34.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.077+0000 7f5403c00640 1 -- 172.21.15.66:0/2175135874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53fc0af390 con 0x7f53fc15f460 2024-09-10T22:00:34.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.077+0000 7f53f8c00640 1 -- 172.21.15.66:0/2175135874 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f53f40b5070 con 0x7f53fc15f460 2024-09-10T22:00:34.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.081+0000 7f53f8c00640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53dc0725e0 0x7f53dc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:34.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.081+0000 7f53f8c00640 1 -- 172.21.15.66:0/2175135874 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f53f412c8d0 con 0x7f53fc15f460 2024-09-10T22:00:34.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.081+0000 7f5401200640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53dc0725e0 0x7f53dc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:34.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.081+0000 7f5401200640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53dc0725e0 0x7f53dc074aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53f0001250 tx=0x7f53f0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:34.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.085+0000 7f53f8c00640 1 -- 172.21.15.66:0/2175135874 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f53f40166f0 con 0x7f53fc15f460 2024-09-10T22:00:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:34.297+0000 7f5403c00640 1 -- 172.21.15.66:0/2175135874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f53fc074aa0 con 0x7f53fc15f460 2024-09-10T22:00:35.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.185+0000 7f53f8c00640 1 -- 172.21.15.66:0/2175135874 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f53f40f7f10 con 0x7f53fc15f460 2024-09-10T22:00:35.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.189+0000 7f5403c00640 1 -- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53dc0725e0 msgr2=0x7f53dc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:35.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.189+0000 7f5403c00640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53dc0725e0 0x7f53dc074aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53f0001250 tx=0x7f53f0009040 comp rx=0 tx=0).stop 2024-09-10T22:00:35.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.189+0000 7f5403c00640 1 -- 172.21.15.66:0/2175135874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53fc15f460 msgr2=0x7f53fc1539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:35.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.189+0000 7f5403c00640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53fc15f460 0x7f53fc1539f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f53f40b0040 tx=0x7f53f4005ac0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.189+0000 7f5403c00640 1 -- 172.21.15.66:0/2175135874 shutdown_connections 2024-09-10T22:00:35.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.189+0000 7f5403c00640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53dc0725e0 0x7f53dc074aa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.189+0000 7f5403c00640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53fc15f460 0x7f53fc1539f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.189+0000 7f5403c00640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53fc0b32a0 0x7f53fc06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.189+0000 7f5403c00640 1 --2- 172.21.15.66:0/2175135874 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53fc0b0f60 0x7f53fc06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.189+0000 7f5403c00640 1 -- 172.21.15.66:0/2175135874 >> 172.21.15.66:0/2175135874 conn(0x7f53fc0d1d00 msgr2=0x7f53fc0b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:35.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.189+0000 7f5403c00640 1 -- 172.21.15.66:0/2175135874 shutdown_connections 2024-09-10T22:00:35.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.189+0000 7f5403c00640 1 -- 172.21.15.66:0/2175135874 wait complete. 2024-09-10T22:00:35.216 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:00:35.217 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:00:35.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.361+0000 7f2a0c600640 1 -- 172.21.15.66:0/1501728470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a04159900 msgr2=0x7f2a040ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:35.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.361+0000 7f2a0c600640 1 --2- 172.21.15.66:0/1501728470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a04159900 0x7f2a040ca080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2a041608e0 tx=0x7f29f801c570 comp rx=0 tx=0).stop 2024-09-10T22:00:35.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.361+0000 7f2a0c600640 1 -- 172.21.15.66:0/1501728470 shutdown_connections 2024-09-10T22:00:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.361+0000 7f2a0c600640 1 --2- 172.21.15.66:0/1501728470 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a040aec50 0x7f2a0415f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.361+0000 7f2a0c600640 1 --2- 172.21.15.66:0/1501728470 >> v2:172.21.15.94:3300/0 conn(0x7f2a0415cc80 0x7f2a040ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.361+0000 7f2a0c600640 1 --2- 172.21.15.66:0/1501728470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a04159900 0x7f2a040ca080 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.361+0000 7f2a0c600640 1 -- 172.21.15.66:0/1501728470 >> 172.21.15.66:0/1501728470 conn(0x7f2a040cfa30 msgr2=0x7f2a040cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.361+0000 7f2a0c600640 1 -- 172.21.15.66:0/1501728470 shutdown_connections 2024-09-10T22:00:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.361+0000 7f2a0c600640 1 -- 172.21.15.66:0/1501728470 wait complete. 2024-09-10T22:00:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0c600640 1 Processor -- start 2024-09-10T22:00:35.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0c600640 1 -- start start 2024-09-10T22:00:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0c600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a040aec50 0x7f2a0414d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0c600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a04159900 0x7f2a040c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0c600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0415cc80 0x7f2a040c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0c600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a040cde50 con 0x7f2a0415cc80 2024-09-10T22:00:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0c600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a040cdcd0 con 0x7f2a04159900 2024-09-10T22:00:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0c600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2a040cdfd0 con 0x7f2a040aec50 2024-09-10T22:00:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0415cc80 0x7f2a040c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0415cc80 0x7f2a040c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60072/0 (socket says 172.21.15.66:60072) 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0a600640 1 -- 172.21.15.66:0/337129863 learned_addr learned my addr 172.21.15.66:0/337129863 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a09c00640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a040aec50 0x7f2a0414d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a09200640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a04159900 0x7f2a040c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0a600640 1 -- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a040aec50 msgr2=0x7f2a0414d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0a600640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a040aec50 0x7f2a0414d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0a600640 1 -- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a04159900 msgr2=0x7f2a040c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0a600640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a04159900 0x7f2a040c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0a600640 1 -- 172.21.15.66:0/337129863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a04158f40 con 0x7f2a0415cc80 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a09200640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a04159900 0x7f2a040c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0a600640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0415cc80 0x7f2a040c9260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00e9c0 tx=0x7f29fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a01600640 1 -- 172.21.15.66:0/337129863 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc00cd80 con 0x7f2a0415cc80 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a01600640 1 -- 172.21.15.66:0/337129863 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29fc004b30 con 0x7f2a0415cc80 2024-09-10T22:00:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a01600640 1 -- 172.21.15.66:0/337129863 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc005020 con 0x7f2a0415cc80 2024-09-10T22:00:35.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0c600640 1 -- 172.21.15.66:0/337129863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0415a0c0 con 0x7f2a0415cc80 2024-09-10T22:00:35.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0c600640 1 -- 172.21.15.66:0/337129863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0414dbf0 con 0x7f2a0415cc80 2024-09-10T22:00:35.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.365+0000 7f2a0c600640 1 -- 172.21.15.66:0/337129863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a040ca080 con 0x7f2a0415cc80 2024-09-10T22:00:35.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.369+0000 7f2a01600640 1 -- 172.21.15.66:0/337129863 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f29fc00f040 con 0x7f2a0415cc80 2024-09-10T22:00:35.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.369+0000 7f2a01600640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29e40725e0 0x7f29e4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:35.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.369+0000 7f2a01600640 1 -- 172.21.15.66:0/337129863 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f29fc093730 con 0x7f2a0415cc80 2024-09-10T22:00:35.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.369+0000 7f2a09c00640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29e40725e0 0x7f29e4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:35.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.369+0000 7f2a09c00640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29e40725e0 0x7f29e4074aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29f80af000 tx=0x7f29f801adb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:35.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.373+0000 7f2a01600640 1 -- 172.21.15.66:0/337129863 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f29fc016030 con 0x7f2a0415cc80 2024-09-10T22:00:35.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:35.581+0000 7f2a0c600640 1 -- 172.21.15.66:0/337129863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2a04070470 con 0x7f2a0415cc80 2024-09-10T22:00:36.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.185+0000 7f2a01600640 1 -- 172.21.15.66:0/337129863 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f29fc05e830 con 0x7f2a0415cc80 2024-09-10T22:00:36.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.189+0000 7f2a0c600640 1 -- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29e40725e0 msgr2=0x7f29e4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.189+0000 7f2a0c600640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29e40725e0 0x7f29e4074aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29f80af000 tx=0x7f29f801adb0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.189+0000 7f2a0c600640 1 -- 172.21.15.66:0/337129863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0415cc80 msgr2=0x7f2a040c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.189+0000 7f2a0c600640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0415cc80 0x7f2a040c9260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00e9c0 tx=0x7f29fc00ee80 comp rx=0 tx=0).stop 2024-09-10T22:00:36.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.189+0000 7f2a0c600640 1 -- 172.21.15.66:0/337129863 shutdown_connections 2024-09-10T22:00:36.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.189+0000 7f2a0c600640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29e40725e0 0x7f29e4074aa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.189+0000 7f2a0c600640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0415cc80 0x7f2a040c9260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.189+0000 7f2a0c600640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a04159900 0x7f2a040c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.189+0000 7f2a0c600640 1 --2- 172.21.15.66:0/337129863 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a040aec50 0x7f2a0414d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.189+0000 7f2a0c600640 1 -- 172.21.15.66:0/337129863 >> 172.21.15.66:0/337129863 conn(0x7f2a040cfa30 msgr2=0x7f2a040b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:36.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.189+0000 7f2a0c600640 1 -- 172.21.15.66:0/337129863 shutdown_connections 2024-09-10T22:00:36.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.189+0000 7f2a0c600640 1 -- 172.21.15.66:0/337129863 wait complete. 2024-09-10T22:00:36.215 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:00:36.215 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:00:36.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.353+0000 7fbafc600640 1 -- 172.21.15.66:0/3035194356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaf40b2bf0 msgr2=0x7fbaf40aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.353+0000 7fbafc600640 1 --2- 172.21.15.66:0/3035194356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaf40b2bf0 0x7fbaf40aeea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009c80 tx=0x7fbaec01cda0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.353+0000 7fbafc600640 1 -- 172.21.15.66:0/3035194356 shutdown_connections 2024-09-10T22:00:36.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.353+0000 7fbafc600640 1 --2- 172.21.15.66:0/3035194356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaf40b2bf0 0x7fbaf40aeea0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.353+0000 7fbafc600640 1 --2- 172.21.15.66:0/3035194356 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbaf40d0260 0x7fbaf40ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.353+0000 7fbafc600640 1 -- 172.21.15.66:0/3035194356 >> 172.21.15.66:0/3035194356 conn(0x7fbaf40d1c50 msgr2=0x7fbaf40d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:36.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.353+0000 7fbafc600640 1 -- 172.21.15.66:0/3035194356 shutdown_connections 2024-09-10T22:00:36.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.353+0000 7fbafc600640 1 -- 172.21.15.66:0/3035194356 wait complete. 2024-09-10T22:00:36.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbafc600640 1 Processor -- start 2024-09-10T22:00:36.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbafc600640 1 -- start start 2024-09-10T22:00:36.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbafc600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaf40b2bf0 0x7fbaf40b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:36.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbafc600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbaf40d0260 0x7fbaf414da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:36.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbafc600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbaf415f340 0x7fbaf4159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:36.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbafc600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4169300 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbafc600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4169180 con 0x7fbaf40d0260 2024-09-10T22:00:36.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbafc600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf4169480 con 0x7fbaf415f340 2024-09-10T22:00:36.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbaf9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaf40b2bf0 0x7fbaf40b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:36.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbaf9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaf40b2bf0 0x7fbaf40b6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60098/0 (socket says 172.21.15.66:60098) 2024-09-10T22:00:36.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbaf9c00640 1 -- 172.21.15.66:0/915291703 learned_addr learned my addr 172.21.15.66:0/915291703 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:36.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbaf9c00640 1 -- 172.21.15.66:0/915291703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbaf415f340 msgr2=0x7fbaf4159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbaf9c00640 1 --2- 172.21.15.66:0/915291703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbaf415f340 0x7fbaf4159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbaf9c00640 1 -- 172.21.15.66:0/915291703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbaf40d0260 msgr2=0x7fbaf414da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbaf9c00640 1 --2- 172.21.15.66:0/915291703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbaf40d0260 0x7fbaf414da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbaf9c00640 1 -- 172.21.15.66:0/915291703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf414e0f0 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbaf9c00640 1 --2- 172.21.15.66:0/915291703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaf40b2bf0 0x7fbaf40b6ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c3c0 tx=0x7fbae800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:36.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbaf1600640 1 -- 172.21.15.66:0/915291703 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8017070 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbaf1600640 1 -- 172.21.15.66:0/915291703 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbae80053c0 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbafc600640 1 -- 172.21.15.66:0/915291703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf40b75d0 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbaf1600640 1 -- 172.21.15.66:0/915291703 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8003860 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.357+0000 7fbafc600640 1 -- 172.21.15.66:0/915291703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf414e3b0 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.361+0000 7fbafc600640 1 -- 172.21.15.66:0/915291703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf40cfb50 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.361+0000 7fbaf1600640 1 -- 172.21.15.66:0/915291703 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbae8003a00 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.361+0000 7fbaf1600640 1 --2- 172.21.15.66:0/915291703 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbad4072610 0x7fbad4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.361+0000 7fbaf1600640 1 -- 172.21.15.66:0/915291703 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbae8092880 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.361+0000 7fbaf9200640 1 --2- 172.21.15.66:0/915291703 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbad4072610 0x7fbad4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.361+0000 7fbaf9200640 1 --2- 172.21.15.66:0/915291703 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbad4072610 0x7fbad4074ad0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbae00058d0 tx=0x7fbae0005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.365+0000 7fbaf1600640 1 -- 172.21.15.66:0/915291703 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbae805df50 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 -- 172.21.15.66:0/915291703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaf40c5500 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbaf1600640 1 -- 172.21.15.66:0/915291703 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7fbae805d850 con 0x7fbaf40b2bf0 2024-09-10T22:00:36.579 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:36.579 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:00:36:190941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2728893456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2728893456},{"type":"v1","addr":"172.21.15.94:6835","nonce":2728893456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2533443317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2533443317},{"type":"v1","addr":"172.21.15.66:6834","nonce":2533443317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/317149387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":317149387},{"type":"v1","addr":"172.21.15.66:6835","nonce":317149387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:00:32.573702+0000","modified":"2024-09-10T22:00:35.587262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/4271184466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":4271184466},{"type":"v1","addr":"172.21.15.94:6837","nonce":4271184466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-10T22:00:36.579 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:00:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 -- 172.21.15.66:0/915291703 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbad4072610 msgr2=0x7fbad4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 --2- 172.21.15.66:0/915291703 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbad4072610 0x7fbad4074ad0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbae00058d0 tx=0x7fbae0005860 comp rx=0 tx=0).stop 2024-09-10T22:00:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 -- 172.21.15.66:0/915291703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaf40b2bf0 msgr2=0x7fbaf40b6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 --2- 172.21.15.66:0/915291703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaf40b2bf0 0x7fbaf40b6ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c3c0 tx=0x7fbae800c880 comp rx=0 tx=0).stop 2024-09-10T22:00:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 -- 172.21.15.66:0/915291703 shutdown_connections 2024-09-10T22:00:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 --2- 172.21.15.66:0/915291703 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbad4072610 0x7fbad4074ad0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 --2- 172.21.15.66:0/915291703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbaf415f340 0x7fbaf4159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 --2- 172.21.15.66:0/915291703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbaf40d0260 0x7fbaf414da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 --2- 172.21.15.66:0/915291703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaf40b2bf0 0x7fbaf40b6ed0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 -- 172.21.15.66:0/915291703 >> 172.21.15.66:0/915291703 conn(0x7fbaf40d1c50 msgr2=0x7fbaf4071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:36.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 -- 172.21.15.66:0/915291703 shutdown_connections 2024-09-10T22:00:36.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.573+0000 7fbafc600640 1 -- 172.21.15.66:0/915291703 wait complete. 2024-09-10T22:00:36.600 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:00:36.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.741+0000 7f29aa000640 1 -- 172.21.15.66:0/1510011370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29a40b2e60 msgr2=0x7f29a4160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.741+0000 7f29aa000640 1 --2- 172.21.15.66:0/1510011370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29a40b2e60 0x7f29a4160a20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f299c009bc0 tx=0x7f299c01cac0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.741+0000 7f29aa000640 1 -- 172.21.15.66:0/1510011370 shutdown_connections 2024-09-10T22:00:36.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.741+0000 7f29aa000640 1 --2- 172.21.15.66:0/1510011370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29a40b2e60 0x7f29a4160a20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.741+0000 7f29aa000640 1 -- 172.21.15.66:0/1510011370 >> 172.21.15.66:0/1510011370 conn(0x7f29a40d3e90 msgr2=0x7f29a40d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:36.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.741+0000 7f29aa000640 1 -- 172.21.15.66:0/1510011370 shutdown_connections 2024-09-10T22:00:36.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.741+0000 7f29aa000640 1 -- 172.21.15.66:0/1510011370 wait complete. 2024-09-10T22:00:36.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.741+0000 7f29aa000640 1 Processor -- start 2024-09-10T22:00:36.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.741+0000 7f29aa000640 1 -- start start 2024-09-10T22:00:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.741+0000 7f29aa000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29a40b2e60 0x7f29a41599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.741+0000 7f29aa000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a40d49b0 0x7f29a4156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.741+0000 7f29aa000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29a41611d0 0x7f29a414ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f29aa000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29a416c080 con 0x7f29a40d49b0 2024-09-10T22:00:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f29aa000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29a416bf00 con 0x7f29a41611d0 2024-09-10T22:00:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f29aa000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f29a416c200 con 0x7f29a40b2e60 2024-09-10T22:00:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a40d49b0 0x7f29a4156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a40d49b0 0x7f29a4156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60110/0 (socket says 172.21.15.66:60110) 2024-09-10T22:00:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299be00640 1 -- 172.21.15.66:0/3034252014 learned_addr learned my addr 172.21.15.66:0/3034252014 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f29a3400640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29a40b2e60 0x7f29a41599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f29a3e00640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29a41611d0 0x7f29a414ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:36.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299be00640 1 -- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29a40b2e60 msgr2=0x7f29a41599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299be00640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29a40b2e60 0x7f29a41599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299be00640 1 -- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29a41611d0 msgr2=0x7f29a414ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299be00640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29a41611d0 0x7f29a414ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299be00640 1 -- 172.21.15.66:0/3034252014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a4156dd0 con 0x7f29a40d49b0 2024-09-10T22:00:36.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f29a3400640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29a40b2e60 0x7f29a41599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:36.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299be00640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a40d49b0 0x7f29a4156890 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f298c001870 tx=0x7f298c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:36.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299b400640 1 -- 172.21.15.66:0/3034252014 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c016020 con 0x7f29a40d49b0 2024-09-10T22:00:36.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299b400640 1 -- 172.21.15.66:0/3034252014 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f298c00a900 con 0x7f29a40d49b0 2024-09-10T22:00:36.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299b400640 1 -- 172.21.15.66:0/3034252014 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c00ad50 con 0x7f29a40d49b0 2024-09-10T22:00:36.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f29aa000640 1 -- 172.21.15.66:0/3034252014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a415a1a0 con 0x7f29a40d49b0 2024-09-10T22:00:36.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f29aa000640 1 -- 172.21.15.66:0/3034252014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a414fa70 con 0x7f29a40d49b0 2024-09-10T22:00:36.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f29aa000640 1 -- 172.21.15.66:0/3034252014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a414fc70 con 0x7f29a40d49b0 2024-09-10T22:00:36.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.745+0000 7f299b400640 1 -- 172.21.15.66:0/3034252014 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f298c01b400 con 0x7f29a40d49b0 2024-09-10T22:00:36.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.749+0000 7f299b400640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29800725e0 0x7f2980074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:36.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.749+0000 7f299b400640 1 -- 172.21.15.66:0/3034252014 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f298c0935e0 con 0x7f29a40d49b0 2024-09-10T22:00:36.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.749+0000 7f29a3400640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29800725e0 0x7f2980074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:36.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.749+0000 7f29a3400640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29800725e0 0x7f2980074aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2994000f70 tx=0x7f2994009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:36.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.753+0000 7f299b400640 1 -- 172.21.15.66:0/3034252014 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f29a414fc70 con 0x7f29a40d49b0 2024-09-10T22:00:36.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.925+0000 7f29aa000640 1 -- 172.21.15.66:0/3034252014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29a41615b0 con 0x7f29a40d49b0 2024-09-10T22:00:36.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.925+0000 7f299b400640 1 -- 172.21.15.66:0/3034252014 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16593 (secure 0 0 0) 0x7f298c020020 con 0x7f29a40d49b0 2024-09-10T22:00:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:36.933 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":22,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:34.173529+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:30.545203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:31.538729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:36.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.929+0000 7f29aa000640 1 -- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29800725e0 msgr2=0x7f2980074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.929+0000 7f29aa000640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29800725e0 0x7f2980074aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2994000f70 tx=0x7f2994009040 comp rx=0 tx=0).stop 2024-09-10T22:00:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.929+0000 7f29aa000640 1 -- 172.21.15.66:0/3034252014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a40d49b0 msgr2=0x7f29a4156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.929+0000 7f29aa000640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a40d49b0 0x7f29a4156890 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f298c001870 tx=0x7f298c00cbb0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.929+0000 7f29aa000640 1 -- 172.21.15.66:0/3034252014 shutdown_connections 2024-09-10T22:00:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.929+0000 7f29aa000640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29800725e0 0x7f2980074aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.929+0000 7f29aa000640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29a41611d0 0x7f29a414ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.929+0000 7f29aa000640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a40d49b0 0x7f29a4156890 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.929+0000 7f29aa000640 1 --2- 172.21.15.66:0/3034252014 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29a40b2e60 0x7f29a41599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.929+0000 7f29aa000640 1 -- 172.21.15.66:0/3034252014 >> 172.21.15.66:0/3034252014 conn(0x7f29a40d3e90 msgr2=0x7f29a4160830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.929+0000 7f29aa000640 1 -- 172.21.15.66:0/3034252014 shutdown_connections 2024-09-10T22:00:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:36.929+0000 7f29aa000640 1 -- 172.21.15.66:0/3034252014 wait complete. 2024-09-10T22:00:36.954 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:00:36.954 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:00:37.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.093+0000 7f181e400640 1 -- 172.21.15.66:0/3066391347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17fc003a10 msgr2=0x7f17fc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.093+0000 7f181e400640 1 --2- 172.21.15.66:0/3066391347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17fc003a10 0x7f17fc003e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f18180cf760 tx=0x7f180801d2d0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.093+0000 7f1817e00640 1 -- 172.21.15.66:0/3066391347 reap_dead start 2024-09-10T22:00:37.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f181e400640 1 -- 172.21.15.66:0/3066391347 shutdown_connections 2024-09-10T22:00:37.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f181e400640 1 -- 172.21.15.66:0/3066391347 >> 172.21.15.66:0/3066391347 conn(0x7f18180d1c60 msgr2=0x7f18180d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:37.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f181e400640 1 -- 172.21.15.66:0/3066391347 shutdown_connections 2024-09-10T22:00:37.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f181e400640 1 -- 172.21.15.66:0/3066391347 wait complete. 2024-09-10T22:00:37.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f181e400640 1 Processor -- start 2024-09-10T22:00:37.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f181e400640 1 -- start start 2024-09-10T22:00:37.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f181e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17fc003a10 0x7f1818156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:37.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f181e400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18181646d0 0x7f1818164b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:37.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f181e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18181630f0 0x7f1818163570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:37.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f181e400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1818166d50 con 0x7f18181630f0 2024-09-10T22:00:37.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f181e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1818166bd0 con 0x7f17fc003a10 2024-09-10T22:00:37.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f181e400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1818166ed0 con 0x7f18181646d0 2024-09-10T22:00:37.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f1817e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18181630f0 0x7f1818163570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:37.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f1817e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18181630f0 0x7f1818163570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60136/0 (socket says 172.21.15.66:60136) 2024-09-10T22:00:37.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f1817e00640 1 -- 172.21.15.66:0/1366469145 learned_addr learned my addr 172.21.15.66:0/1366469145 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:37.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f1817400640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17fc003a10 0x7f1818156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:37.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f180ea00640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18181646d0 0x7f1818164b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:37.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f1817e00640 1 -- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18181646d0 msgr2=0x7f1818164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f1817e00640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18181646d0 0x7f1818164b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f1817e00640 1 -- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17fc003a10 msgr2=0x7f1818156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f1817e00640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17fc003a10 0x7f1818156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f1817e00640 1 -- 172.21.15.66:0/1366469145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1818050ef0 con 0x7f18181630f0 2024-09-10T22:00:37.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f180ea00640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18181646d0 0x7f1818164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:37.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f1817e00640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18181630f0 0x7f1818163570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18180cc240 tx=0x7f1810006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:37.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.097+0000 7f180fe00640 1 -- 172.21.15.66:0/1366469145 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181000f9a0 con 0x7f18181630f0 2024-09-10T22:00:37.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.101+0000 7f180fe00640 1 -- 172.21.15.66:0/1366469145 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1810018070 con 0x7f18181630f0 2024-09-10T22:00:37.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.101+0000 7f181e400640 1 -- 172.21.15.66:0/1366469145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1818163f60 con 0x7f18181630f0 2024-09-10T22:00:37.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.101+0000 7f180fe00640 1 -- 172.21.15.66:0/1366469145 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181000fde0 con 0x7f18181630f0 2024-09-10T22:00:37.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.101+0000 7f181e400640 1 -- 172.21.15.66:0/1366469145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18180b4b30 con 0x7f18181630f0 2024-09-10T22:00:37.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.101+0000 7f181e400640 1 -- 172.21.15.66:0/1366469145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18180d04e0 con 0x7f18181630f0 2024-09-10T22:00:37.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.101+0000 7f180fe00640 1 -- 172.21.15.66:0/1366469145 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f181001d670 con 0x7f18181630f0 2024-09-10T22:00:37.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.101+0000 7f180fe00640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f17f4072610 0x7f17f4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:37.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.101+0000 7f180fe00640 1 -- 172.21.15.66:0/1366469145 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1810093be0 con 0x7f18181630f0 2024-09-10T22:00:37.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.101+0000 7f1817400640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f17f4072610 0x7f17f4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:37.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.105+0000 7f1817400640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f17f4072610 0x7f17f4074ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18080ae000 tx=0x7f180801a070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:37.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.105+0000 7f180fe00640 1 -- 172.21.15.66:0/1366469145 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f181005f220 con 0x7f18181630f0 2024-09-10T22:00:37.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.277+0000 7f181e400640 1 -- 172.21.15.66:0/1366469145 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18180b0df0 con 0x7f17f4072610 2024-09-10T22:00:37.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.301+0000 7f180fe00640 1 -- 172.21.15.66:0/1366469145 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240840 (secure 0 0 0) 0x7f18180b0df0 con 0x7f17f4072610 2024-09-10T22:00:37.309 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:37.311 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:00:37.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.305+0000 7f181e400640 1 -- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f17f4072610 msgr2=0x7f17f4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.305+0000 7f181e400640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f17f4072610 0x7f17f4074ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18080ae000 tx=0x7f180801a070 comp rx=0 tx=0).stop 2024-09-10T22:00:37.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.305+0000 7f181e400640 1 -- 172.21.15.66:0/1366469145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18181630f0 msgr2=0x7f1818163570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.305+0000 7f181e400640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18181630f0 0x7f1818163570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18180cc240 tx=0x7f1810006720 comp rx=0 tx=0).stop 2024-09-10T22:00:37.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.305+0000 7f181e400640 1 -- 172.21.15.66:0/1366469145 shutdown_connections 2024-09-10T22:00:37.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.305+0000 7f181e400640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f17f4072610 0x7f17f4074ad0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.305+0000 7f181e400640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18181630f0 0x7f1818163570 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.305+0000 7f181e400640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18181646d0 0x7f1818164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.305+0000 7f181e400640 1 --2- 172.21.15.66:0/1366469145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17fc003a10 0x7f1818156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.305+0000 7f181e400640 1 -- 172.21.15.66:0/1366469145 >> 172.21.15.66:0/1366469145 conn(0x7f18180d1c60 msgr2=0x7f1818071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:37.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.305+0000 7f181e400640 1 -- 172.21.15.66:0/1366469145 shutdown_connections 2024-09-10T22:00:37.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.305+0000 7f181e400640 1 -- 172.21.15.66:0/1366469145 wait complete. 2024-09-10T22:00:37.330 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-10T22:00:36.310810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":116,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":70,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231340,"kb_used_data":2636,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743380,"statfs":{"total":773094113280,"available":772857221120,"internally_reserved":0,"allocated":2699264,"data_stored":1655558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":234016308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.154708"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663811+0000","last_change":"2024-09-10T22:00:33.192188+0000","last_active":"2024-09-10T22:00:34.663811+0000","last_peered":"2024-09-10T22:00:34.663811+0000","last_clean":"2024-09-10T22:00:34.663811+0000","last_became_active":"2024-09-10T22:00:33.191616+0000","last_became_peered":"2024-09-10T22:00:33.191616+0000","last_unstale":"2024-09-10T22:00:34.663811+0000","last_undegraded":"2024-09-10T22:00:34.663811+0000","last_fullsized":"2024-09-10T22:00:34.663811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:03.715049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663157+0000","last_change":"2024-09-10T22:00:33.213581+0000","last_active":"2024-09-10T22:00:34.663157+0000","last_peered":"2024-09-10T22:00:34.663157+0000","last_clean":"2024-09-10T22:00:34.663157+0000","last_became_active":"2024-09-10T22:00:33.213202+0000","last_became_peered":"2024-09-10T22:00:33.213202+0000","last_unstale":"2024-09-10T22:00:34.663157+0000","last_undegraded":"2024-09-10T22:00:34.663157+0000","last_fullsized":"2024-09-10T22:00:34.663157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:03.339805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163552+0000","last_change":"2024-09-10T22:00:31.163552+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163552+0000","last_undegraded":"2024-09-10T22:00:31.163552+0000","last_fullsized":"2024-09-10T22:00:31.163552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.159305+0000","last_change":"2024-09-10T22:00:31.159305+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.159305+0000","last_undegraded":"2024-09-10T22:00:31.159305+0000","last_fullsized":"2024-09-10T22:00:31.159305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163295+0000","last_change":"2024-09-10T22:00:31.163295+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163295+0000","last_undegraded":"2024-09-10T22:00:31.163295+0000","last_fullsized":"2024-09-10T22:00:31.163295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.164731+0000","last_change":"2024-09-10T22:00:31.164731+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.164731+0000","last_undegraded":"2024-09-10T22:00:31.164731+0000","last_fullsized":"2024-09-10T22:00:31.164731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.160703+0000","last_change":"2024-09-10T22:00:31.160703+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.160703+0000","last_undegraded":"2024-09-10T22:00:31.160703+0000","last_fullsized":"2024-09-10T22:00:31.160703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163225+0000","last_change":"2024-09-10T22:00:31.163225+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163225+0000","last_undegraded":"2024-09-10T22:00:31.163225+0000","last_fullsized":"2024-09-10T22:00:31.163225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.170537+0000","last_change":"2024-09-10T22:00:31.170537+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.170537+0000","last_undegraded":"2024-09-10T22:00:31.170537+0000","last_fullsized":"2024-09-10T22:00:31.170537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.159538+0000","last_change":"2024-09-10T22:00:31.159538+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.159538+0000","last_undegraded":"2024-09-10T22:00:31.159538+0000","last_fullsized":"2024-09-10T22:00:31.159538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.162179+0000","last_change":"2024-09-10T22:00:31.162179+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.162179+0000","last_undegraded":"2024-09-10T22:00:31.162179+0000","last_fullsized":"2024-09-10T22:00:31.162179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163836+0000","last_change":"2024-09-10T22:00:31.163836+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163836+0000","last_undegraded":"2024-09-10T22:00:31.163836+0000","last_fullsized":"2024-09-10T22:00:31.163836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.159305+0000","last_change":"2024-09-10T22:00:31.159305+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.159305+0000","last_undegraded":"2024-09-10T22:00:31.159305+0000","last_fullsized":"2024-09-10T22:00:31.159305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.662989+0000","last_change":"2024-09-10T22:00:32.182584+0000","last_active":"2024-09-10T22:00:34.662989+0000","last_peered":"2024-09-10T22:00:34.662989+0000","last_clean":"2024-09-10T22:00:34.662989+0000","last_became_active":"2024-09-10T22:00:32.182289+0000","last_became_peered":"2024-09-10T22:00:32.182289+0000","last_unstale":"2024-09-10T22:00:34.662989+0000","last_undegraded":"2024-09-10T22:00:34.662989+0000","last_fullsized":"2024-09-10T22:00:34.662989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:41.773634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.159340+0000","last_change":"2024-09-10T22:00:31.159340+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.159340+0000","last_undegraded":"2024-09-10T22:00:31.159340+0000","last_fullsized":"2024-09-10T22:00:31.159340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663916+0000","last_change":"2024-09-10T22:00:32.184130+0000","last_active":"2024-09-10T22:00:34.663916+0000","last_peered":"2024-09-10T22:00:34.663916+0000","last_clean":"2024-09-10T22:00:34.663916+0000","last_became_active":"2024-09-10T22:00:32.182524+0000","last_became_peered":"2024-09-10T22:00:32.182524+0000","last_unstale":"2024-09-10T22:00:34.663916+0000","last_undegraded":"2024-09-10T22:00:34.663916+0000","last_fullsized":"2024-09-10T22:00:34.663916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:26.056799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169683+0000","last_change":"2024-09-10T22:00:31.169683+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169683+0000","last_undegraded":"2024-09-10T22:00:31.169683+0000","last_fullsized":"2024-09-10T22:00:31.169683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663655+0000","last_change":"2024-09-10T22:00:32.195097+0000","last_active":"2024-09-10T22:00:34.663655+0000","last_peered":"2024-09-10T22:00:34.663655+0000","last_clean":"2024-09-10T22:00:34.663655+0000","last_became_active":"2024-09-10T22:00:32.194103+0000","last_became_peered":"2024-09-10T22:00:32.194103+0000","last_unstale":"2024-09-10T22:00:34.663655+0000","last_undegraded":"2024-09-10T22:00:34.663655+0000","last_fullsized":"2024-09-10T22:00:34.663655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:57.181450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-10T22:00:31.167874+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:00:31.167874+0000","last_peered":"2024-09-10T22:00:31.167874+0000","last_clean":"2024-09-10T22:00:31.167874+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:00:31.167874+0000","last_undegraded":"2024-09-10T22:00:31.167874+0000","last_fullsized":"2024-09-10T22:00:31.167874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.164958+0000","last_change":"2024-09-10T22:00:31.164958+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.164958+0000","last_undegraded":"2024-09-10T22:00:31.164958+0000","last_fullsized":"2024-09-10T22:00:31.164958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163045+0000","last_change":"2024-09-10T22:00:31.163045+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163045+0000","last_undegraded":"2024-09-10T22:00:31.163045+0000","last_fullsized":"2024-09-10T22:00:31.163045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663264+0000","last_change":"2024-09-10T22:00:32.178337+0000","last_active":"2024-09-10T22:00:34.663264+0000","last_peered":"2024-09-10T22:00:34.663264+0000","last_clean":"2024-09-10T22:00:34.663264+0000","last_became_active":"2024-09-10T22:00:32.178053+0000","last_became_peered":"2024-09-10T22:00:32.178053+0000","last_unstale":"2024-09-10T22:00:34.663264+0000","last_undegraded":"2024-09-10T22:00:34.663264+0000","last_fullsized":"2024-09-10T22:00:34.663264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:26.949229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163215+0000","last_change":"2024-09-10T22:00:31.163215+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163215+0000","last_undegraded":"2024-09-10T22:00:31.163215+0000","last_fullsized":"2024-09-10T22:00:31.163215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169542+0000","last_change":"2024-09-10T22:00:31.169542+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169542+0000","last_undegraded":"2024-09-10T22:00:31.169542+0000","last_fullsized":"2024-09-10T22:00:31.169542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.164658+0000","last_change":"2024-09-10T22:00:31.164658+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.164658+0000","last_undegraded":"2024-09-10T22:00:31.164658+0000","last_fullsized":"2024-09-10T22:00:31.164658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169404+0000","last_change":"2024-09-10T22:00:31.169404+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169404+0000","last_undegraded":"2024-09-10T22:00:31.169404+0000","last_fullsized":"2024-09-10T22:00:31.169404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.160191+0000","last_change":"2024-09-10T22:00:31.160191+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.160191+0000","last_undegraded":"2024-09-10T22:00:31.160191+0000","last_fullsized":"2024-09-10T22:00:31.160191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163616+0000","last_change":"2024-09-10T22:00:31.163616+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163616+0000","last_undegraded":"2024-09-10T22:00:31.163616+0000","last_fullsized":"2024-09-10T22:00:31.163616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169413+0000","last_change":"2024-09-10T22:00:31.169413+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169413+0000","last_undegraded":"2024-09-10T22:00:31.169413+0000","last_fullsized":"2024-09-10T22:00:31.169413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169328+0000","last_change":"2024-09-10T22:00:31.169328+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169328+0000","last_undegraded":"2024-09-10T22:00:31.169328+0000","last_fullsized":"2024-09-10T22:00:31.169328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163027+0000","last_change":"2024-09-10T22:00:31.163027+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163027+0000","last_undegraded":"2024-09-10T22:00:31.163027+0000","last_fullsized":"2024-09-10T22:00:31.163027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163713+0000","last_change":"2024-09-10T22:00:31.163713+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163713+0000","last_undegraded":"2024-09-10T22:00:31.163713+0000","last_fullsized":"2024-09-10T22:00:31.163713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.166355+0000","last_change":"2024-09-10T22:00:31.166355+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.166355+0000","last_undegraded":"2024-09-10T22:00:31.166355+0000","last_fullsized":"2024-09-10T22:00:31.166355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.164132+0000","last_change":"2024-09-10T22:00:31.164132+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.164132+0000","last_undegraded":"2024-09-10T22:00:31.164132+0000","last_fullsized":"2024-09-10T22:00:31.164132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.162408+0000","last_change":"2024-09-10T22:00:31.162408+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.162408+0000","last_undegraded":"2024-09-10T22:00:31.162408+0000","last_fullsized":"2024-09-10T22:00:31.162408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:00:37.334 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:00:37.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 -- 172.21.15.66:0/3185679240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c80cb620 msgr2=0x7f89c80ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 --2- 172.21.15.66:0/3185679240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c80cb620 0x7f89c80ceff0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009c80 tx=0x7f89c001cd70 comp rx=0 tx=0).stop 2024-09-10T22:00:37.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 -- 172.21.15.66:0/3185679240 shutdown_connections 2024-09-10T22:00:37.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 --2- 172.21.15.66:0/3185679240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c80cb620 0x7f89c80ceff0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 --2- 172.21.15.66:0/3185679240 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89c815a8a0 0x7f89c80cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 -- 172.21.15.66:0/3185679240 >> 172.21.15.66:0/3185679240 conn(0x7f89c80b26c0 msgr2=0x7f89c80b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:37.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 -- 172.21.15.66:0/3185679240 shutdown_connections 2024-09-10T22:00:37.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 -- 172.21.15.66:0/3185679240 wait complete. 2024-09-10T22:00:37.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 Processor -- start 2024-09-10T22:00:37.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 -- start start 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89c80cb620 0x7f89c81f96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c815a8a0 0x7f89c80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89c815ac80 0x7f89c80b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89c8167d70 con 0x7f89c815a8a0 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89c8167bf0 con 0x7f89c815ac80 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89cf200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f89c8167ef0 con 0x7f89c80cb620 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.517+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c815a8a0 0x7f89c80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c815a8a0 0x7f89c80b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60154/0 (socket says 172.21.15.66:60154) 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89bfe00640 1 -- 172.21.15.66:0/3976922634 learned_addr learned my addr 172.21.15.66:0/3976922634 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89c7e00640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89c80cb620 0x7f89c81f96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89cd200640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89c815ac80 0x7f89c80b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89bfe00640 1 -- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89c80cb620 msgr2=0x7f89c81f96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89bfe00640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89c80cb620 0x7f89c81f96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89bfe00640 1 -- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89c815ac80 msgr2=0x7f89c80b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89bfe00640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89c815ac80 0x7f89c80b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89bfe00640 1 -- 172.21.15.66:0/3976922634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c814e000 con 0x7f89c815a8a0 2024-09-10T22:00:37.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89bfe00640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c815a8a0 0x7f89c80b4fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c3c0 tx=0x7f89b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:37.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89c4c00640 1 -- 172.21.15.66:0/3976922634 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0017070 con 0x7f89c815a8a0 2024-09-10T22:00:37.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89c4c00640 1 -- 172.21.15.66:0/3976922634 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89b00053c0 con 0x7f89c815a8a0 2024-09-10T22:00:37.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89c4c00640 1 -- 172.21.15.66:0/3976922634 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0003860 con 0x7f89c815a8a0 2024-09-10T22:00:37.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89cf200640 1 -- 172.21.15.66:0/3976922634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c814e2e0 con 0x7f89c815a8a0 2024-09-10T22:00:37.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89cf200640 1 -- 172.21.15.66:0/3976922634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c80b5b20 con 0x7f89c815a8a0 2024-09-10T22:00:37.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89cf200640 1 -- 172.21.15.66:0/3976922634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c80ceff0 con 0x7f89c815a8a0 2024-09-10T22:00:37.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.521+0000 7f89c4c00640 1 -- 172.21.15.66:0/3976922634 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f89b0003a00 con 0x7f89c815a8a0 2024-09-10T22:00:37.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.525+0000 7f89c4c00640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f89a40725e0 0x7f89a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:37.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.525+0000 7f89c4c00640 1 -- 172.21.15.66:0/3976922634 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f89b00927b0 con 0x7f89c815a8a0 2024-09-10T22:00:37.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.525+0000 7f89c7e00640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f89a40725e0 0x7f89a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:37.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.525+0000 7f89c7e00640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f89a40725e0 0x7f89a4074aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89c8154690 tx=0x7f89b80091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:37.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.525+0000 7f89c4c00640 1 -- 172.21.15.66:0/3976922634 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f89b005d830 con 0x7f89c815a8a0 2024-09-10T22:00:37.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.697+0000 7f89cf200640 1 -- 172.21.15.66:0/3976922634 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f89c8152e10 con 0x7f89a40725e0 2024-09-10T22:00:37.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.701+0000 7f89c4c00640 1 -- 172.21.15.66:0/3976922634 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240840 (secure 0 0 0) 0x7f89c8152e10 con 0x7f89a40725e0 2024-09-10T22:00:37.712 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:37.714 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:00:37.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.705+0000 7f89cf200640 1 -- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f89a40725e0 msgr2=0x7f89a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.705+0000 7f89cf200640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f89a40725e0 0x7f89a4074aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89c8154690 tx=0x7f89b80091e0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.705+0000 7f89cf200640 1 -- 172.21.15.66:0/3976922634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c815a8a0 msgr2=0x7f89c80b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.705+0000 7f89cf200640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c815a8a0 0x7f89c80b4fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c3c0 tx=0x7f89b000c880 comp rx=0 tx=0).stop 2024-09-10T22:00:37.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.705+0000 7f89cf200640 1 -- 172.21.15.66:0/3976922634 shutdown_connections 2024-09-10T22:00:37.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.705+0000 7f89cf200640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f89a40725e0 0x7f89a4074aa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.705+0000 7f89cf200640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89c815ac80 0x7f89c80b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.705+0000 7f89cf200640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c815a8a0 0x7f89c80b4fb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.705+0000 7f89cf200640 1 --2- 172.21.15.66:0/3976922634 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89c80cb620 0x7f89c81f96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.705+0000 7f89cf200640 1 -- 172.21.15.66:0/3976922634 >> 172.21.15.66:0/3976922634 conn(0x7f89c80b26c0 msgr2=0x7f89c8073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.705+0000 7f89cf200640 1 -- 172.21.15.66:0/3976922634 shutdown_connections 2024-09-10T22:00:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.705+0000 7f89cf200640 1 -- 172.21.15.66:0/3976922634 wait complete. 2024-09-10T22:00:37.733 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-10T22:00:36.310810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":116,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":70,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231340,"kb_used_data":2636,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743380,"statfs":{"total":773094113280,"available":772857221120,"internally_reserved":0,"allocated":2699264,"data_stored":1655558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":234016308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.154708"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663811+0000","last_change":"2024-09-10T22:00:33.192188+0000","last_active":"2024-09-10T22:00:34.663811+0000","last_peered":"2024-09-10T22:00:34.663811+0000","last_clean":"2024-09-10T22:00:34.663811+0000","last_became_active":"2024-09-10T22:00:33.191616+0000","last_became_peered":"2024-09-10T22:00:33.191616+0000","last_unstale":"2024-09-10T22:00:34.663811+0000","last_undegraded":"2024-09-10T22:00:34.663811+0000","last_fullsized":"2024-09-10T22:00:34.663811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:03.715049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663157+0000","last_change":"2024-09-10T22:00:33.213581+0000","last_active":"2024-09-10T22:00:34.663157+0000","last_peered":"2024-09-10T22:00:34.663157+0000","last_clean":"2024-09-10T22:00:34.663157+0000","last_became_active":"2024-09-10T22:00:33.213202+0000","last_became_peered":"2024-09-10T22:00:33.213202+0000","last_unstale":"2024-09-10T22:00:34.663157+0000","last_undegraded":"2024-09-10T22:00:34.663157+0000","last_fullsized":"2024-09-10T22:00:34.663157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:03.339805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163552+0000","last_change":"2024-09-10T22:00:31.163552+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163552+0000","last_undegraded":"2024-09-10T22:00:31.163552+0000","last_fullsized":"2024-09-10T22:00:31.163552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.159305+0000","last_change":"2024-09-10T22:00:31.159305+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.159305+0000","last_undegraded":"2024-09-10T22:00:31.159305+0000","last_fullsized":"2024-09-10T22:00:31.159305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163295+0000","last_change":"2024-09-10T22:00:31.163295+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163295+0000","last_undegraded":"2024-09-10T22:00:31.163295+0000","last_fullsized":"2024-09-10T22:00:31.163295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.164731+0000","last_change":"2024-09-10T22:00:31.164731+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.164731+0000","last_undegraded":"2024-09-10T22:00:31.164731+0000","last_fullsized":"2024-09-10T22:00:31.164731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.160703+0000","last_change":"2024-09-10T22:00:31.160703+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.160703+0000","last_undegraded":"2024-09-10T22:00:31.160703+0000","last_fullsized":"2024-09-10T22:00:31.160703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163225+0000","last_change":"2024-09-10T22:00:31.163225+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163225+0000","last_undegraded":"2024-09-10T22:00:31.163225+0000","last_fullsized":"2024-09-10T22:00:31.163225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.170537+0000","last_change":"2024-09-10T22:00:31.170537+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.170537+0000","last_undegraded":"2024-09-10T22:00:31.170537+0000","last_fullsized":"2024-09-10T22:00:31.170537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.159538+0000","last_change":"2024-09-10T22:00:31.159538+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.159538+0000","last_undegraded":"2024-09-10T22:00:31.159538+0000","last_fullsized":"2024-09-10T22:00:31.159538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.162179+0000","last_change":"2024-09-10T22:00:31.162179+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.162179+0000","last_undegraded":"2024-09-10T22:00:31.162179+0000","last_fullsized":"2024-09-10T22:00:31.162179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163836+0000","last_change":"2024-09-10T22:00:31.163836+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163836+0000","last_undegraded":"2024-09-10T22:00:31.163836+0000","last_fullsized":"2024-09-10T22:00:31.163836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.159305+0000","last_change":"2024-09-10T22:00:31.159305+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.159305+0000","last_undegraded":"2024-09-10T22:00:31.159305+0000","last_fullsized":"2024-09-10T22:00:31.159305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.662989+0000","last_change":"2024-09-10T22:00:32.182584+0000","last_active":"2024-09-10T22:00:34.662989+0000","last_peered":"2024-09-10T22:00:34.662989+0000","last_clean":"2024-09-10T22:00:34.662989+0000","last_became_active":"2024-09-10T22:00:32.182289+0000","last_became_peered":"2024-09-10T22:00:32.182289+0000","last_unstale":"2024-09-10T22:00:34.662989+0000","last_undegraded":"2024-09-10T22:00:34.662989+0000","last_fullsized":"2024-09-10T22:00:34.662989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:41.773634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.159340+0000","last_change":"2024-09-10T22:00:31.159340+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.159340+0000","last_undegraded":"2024-09-10T22:00:31.159340+0000","last_fullsized":"2024-09-10T22:00:31.159340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663916+0000","last_change":"2024-09-10T22:00:32.184130+0000","last_active":"2024-09-10T22:00:34.663916+0000","last_peered":"2024-09-10T22:00:34.663916+0000","last_clean":"2024-09-10T22:00:34.663916+0000","last_became_active":"2024-09-10T22:00:32.182524+0000","last_became_peered":"2024-09-10T22:00:32.182524+0000","last_unstale":"2024-09-10T22:00:34.663916+0000","last_undegraded":"2024-09-10T22:00:34.663916+0000","last_fullsized":"2024-09-10T22:00:34.663916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:26.056799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169683+0000","last_change":"2024-09-10T22:00:31.169683+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169683+0000","last_undegraded":"2024-09-10T22:00:31.169683+0000","last_fullsized":"2024-09-10T22:00:31.169683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663655+0000","last_change":"2024-09-10T22:00:32.195097+0000","last_active":"2024-09-10T22:00:34.663655+0000","last_peered":"2024-09-10T22:00:34.663655+0000","last_clean":"2024-09-10T22:00:34.663655+0000","last_became_active":"2024-09-10T22:00:32.194103+0000","last_became_peered":"2024-09-10T22:00:32.194103+0000","last_unstale":"2024-09-10T22:00:34.663655+0000","last_undegraded":"2024-09-10T22:00:34.663655+0000","last_fullsized":"2024-09-10T22:00:34.663655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:57.181450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-10T22:00:31.167874+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:00:31.167874+0000","last_peered":"2024-09-10T22:00:31.167874+0000","last_clean":"2024-09-10T22:00:31.167874+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:00:31.167874+0000","last_undegraded":"2024-09-10T22:00:31.167874+0000","last_fullsized":"2024-09-10T22:00:31.167874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.164958+0000","last_change":"2024-09-10T22:00:31.164958+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.164958+0000","last_undegraded":"2024-09-10T22:00:31.164958+0000","last_fullsized":"2024-09-10T22:00:31.164958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163045+0000","last_change":"2024-09-10T22:00:31.163045+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163045+0000","last_undegraded":"2024-09-10T22:00:31.163045+0000","last_fullsized":"2024-09-10T22:00:31.163045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663264+0000","last_change":"2024-09-10T22:00:32.178337+0000","last_active":"2024-09-10T22:00:34.663264+0000","last_peered":"2024-09-10T22:00:34.663264+0000","last_clean":"2024-09-10T22:00:34.663264+0000","last_became_active":"2024-09-10T22:00:32.178053+0000","last_became_peered":"2024-09-10T22:00:32.178053+0000","last_unstale":"2024-09-10T22:00:34.663264+0000","last_undegraded":"2024-09-10T22:00:34.663264+0000","last_fullsized":"2024-09-10T22:00:34.663264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:26.949229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163215+0000","last_change":"2024-09-10T22:00:31.163215+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163215+0000","last_undegraded":"2024-09-10T22:00:31.163215+0000","last_fullsized":"2024-09-10T22:00:31.163215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169542+0000","last_change":"2024-09-10T22:00:31.169542+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169542+0000","last_undegraded":"2024-09-10T22:00:31.169542+0000","last_fullsized":"2024-09-10T22:00:31.169542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.164658+0000","last_change":"2024-09-10T22:00:31.164658+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.164658+0000","last_undegraded":"2024-09-10T22:00:31.164658+0000","last_fullsized":"2024-09-10T22:00:31.164658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169404+0000","last_change":"2024-09-10T22:00:31.169404+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169404+0000","last_undegraded":"2024-09-10T22:00:31.169404+0000","last_fullsized":"2024-09-10T22:00:31.169404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.160191+0000","last_change":"2024-09-10T22:00:31.160191+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.160191+0000","last_undegraded":"2024-09-10T22:00:31.160191+0000","last_fullsized":"2024-09-10T22:00:31.160191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163616+0000","last_change":"2024-09-10T22:00:31.163616+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163616+0000","last_undegraded":"2024-09-10T22:00:31.163616+0000","last_fullsized":"2024-09-10T22:00:31.163616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169413+0000","last_change":"2024-09-10T22:00:31.169413+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169413+0000","last_undegraded":"2024-09-10T22:00:31.169413+0000","last_fullsized":"2024-09-10T22:00:31.169413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169328+0000","last_change":"2024-09-10T22:00:31.169328+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169328+0000","last_undegraded":"2024-09-10T22:00:31.169328+0000","last_fullsized":"2024-09-10T22:00:31.169328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163027+0000","last_change":"2024-09-10T22:00:31.163027+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163027+0000","last_undegraded":"2024-09-10T22:00:31.163027+0000","last_fullsized":"2024-09-10T22:00:31.163027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163713+0000","last_change":"2024-09-10T22:00:31.163713+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163713+0000","last_undegraded":"2024-09-10T22:00:31.163713+0000","last_fullsized":"2024-09-10T22:00:31.163713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.166355+0000","last_change":"2024-09-10T22:00:31.166355+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.166355+0000","last_undegraded":"2024-09-10T22:00:31.166355+0000","last_fullsized":"2024-09-10T22:00:31.166355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.164132+0000","last_change":"2024-09-10T22:00:31.164132+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.164132+0000","last_undegraded":"2024-09-10T22:00:31.164132+0000","last_fullsized":"2024-09-10T22:00:31.164132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.162408+0000","last_change":"2024-09-10T22:00:31.162408+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.162408+0000","last_undegraded":"2024-09-10T22:00:31.162408+0000","last_fullsized":"2024-09-10T22:00:31.162408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:00:37.738 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:00:37.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 -- 172.21.15.66:0/604007870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8415f340 msgr2=0x7f2b840cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 --2- 172.21.15.66:0/604007870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8415f340 0x7f2b840cfb50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009870 tx=0x7f2b7801c7f0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 -- 172.21.15.66:0/604007870 shutdown_connections 2024-09-10T22:00:37.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 --2- 172.21.15.66:0/604007870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8415f340 0x7f2b840cfb50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 -- 172.21.15.66:0/604007870 >> 172.21.15.66:0/604007870 conn(0x7f2b840d1c50 msgr2=0x7f2b840d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:37.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 -- 172.21.15.66:0/604007870 shutdown_connections 2024-09-10T22:00:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 -- 172.21.15.66:0/604007870 wait complete. 2024-09-10T22:00:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 Processor -- start 2024-09-10T22:00:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 -- start start 2024-09-10T22:00:37.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b840af970 0x7f2b841f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:37.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b840b2bf0 0x7f2b84150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:37.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8415f340 0x7f2b841568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:37.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8416a7a0 con 0x7f2b8415f340 2024-09-10T22:00:37.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8416a620 con 0x7f2b840b2bf0 2024-09-10T22:00:37.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.917+0000 7f2b8c200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8416a920 con 0x7f2b840af970 2024-09-10T22:00:37.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b8a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8415f340 0x7f2b841568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:37.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b8a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8415f340 0x7f2b841568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60174/0 (socket says 172.21.15.66:60174) 2024-09-10T22:00:37.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b8a200640 1 -- 172.21.15.66:0/102466273 learned_addr learned my addr 172.21.15.66:0/102466273 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:37.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b89800640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b840af970 0x7f2b841f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:37.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b88e00640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b840b2bf0 0x7f2b84150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:37.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b8a200640 1 -- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b840af970 msgr2=0x7f2b841f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b8a200640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b840af970 0x7f2b841f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b8a200640 1 -- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b840b2bf0 msgr2=0x7f2b84150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:37.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b8a200640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b840b2bf0 0x7f2b84150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:37.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b8a200640 1 -- 172.21.15.66:0/102466273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b84156140 con 0x7f2b8415f340 2024-09-10T22:00:37.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b8a200640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8415f340 0x7f2b841568d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00cbc0 tx=0x7f2b7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:37.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b81600640 1 -- 172.21.15.66:0/102466273 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c016020 con 0x7f2b8415f340 2024-09-10T22:00:37.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b8c200640 1 -- 172.21.15.66:0/102466273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b841f6a90 con 0x7f2b8415f340 2024-09-10T22:00:37.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b81600640 1 -- 172.21.15.66:0/102466273 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b7c00aae0 con 0x7f2b8415f340 2024-09-10T22:00:37.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b81600640 1 -- 172.21.15.66:0/102466273 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c012420 con 0x7f2b8415f340 2024-09-10T22:00:37.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b8c200640 1 -- 172.21.15.66:0/102466273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b841563b0 con 0x7f2b8415f340 2024-09-10T22:00:37.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.921+0000 7f2b8c200640 1 -- 172.21.15.66:0/102466273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b840cfb50 con 0x7f2b8415f340 2024-09-10T22:00:37.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.925+0000 7f2b81600640 1 -- 172.21.15.66:0/102466273 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b7c0125c0 con 0x7f2b8415f340 2024-09-10T22:00:37.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.925+0000 7f2b81600640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2b640725e0 0x7f2b64074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:37.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.925+0000 7f2b81600640 1 -- 172.21.15.66:0/102466273 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2b7c093630 con 0x7f2b8415f340 2024-09-10T22:00:37.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.925+0000 7f2b89800640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2b640725e0 0x7f2b64074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:37.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.925+0000 7f2b89800640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2b640725e0 0x7f2b64074aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2b780af000 tx=0x7f2b7801b030 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:37.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:37.929+0000 7f2b81600640 1 -- 172.21.15.66:0/102466273 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b7c020030 con 0x7f2b8415f340 2024-09-10T22:00:38.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.097+0000 7f2b8c200640 1 -- 172.21.15.66:0/102466273 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b840aff10 con 0x7f2b640725e0 2024-09-10T22:00:38.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.101+0000 7f2b81600640 1 -- 172.21.15.66:0/102466273 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240840 (secure 0 0 0) 0x7f2b840aff10 con 0x7f2b640725e0 2024-09-10T22:00:38.109 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:38.111 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:00:38.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.105+0000 7f2b8c200640 1 -- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2b640725e0 msgr2=0x7f2b64074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.105+0000 7f2b8c200640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2b640725e0 0x7f2b64074aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2b780af000 tx=0x7f2b7801b030 comp rx=0 tx=0).stop 2024-09-10T22:00:38.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.105+0000 7f2b8c200640 1 -- 172.21.15.66:0/102466273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8415f340 msgr2=0x7f2b841568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.105+0000 7f2b8c200640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8415f340 0x7f2b841568d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00cbc0 tx=0x7f2b7c00ccb0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.105+0000 7f2b8c200640 1 -- 172.21.15.66:0/102466273 shutdown_connections 2024-09-10T22:00:38.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.105+0000 7f2b8c200640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2b640725e0 0x7f2b64074aa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.105+0000 7f2b8c200640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8415f340 0x7f2b841568d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.105+0000 7f2b8c200640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b840b2bf0 0x7f2b84150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.105+0000 7f2b8c200640 1 --2- 172.21.15.66:0/102466273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b840af970 0x7f2b841f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.105+0000 7f2b8c200640 1 -- 172.21.15.66:0/102466273 >> 172.21.15.66:0/102466273 conn(0x7f2b840d1c50 msgr2=0x7f2b8415b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:38.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.105+0000 7f2b8c200640 1 -- 172.21.15.66:0/102466273 shutdown_connections 2024-09-10T22:00:38.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.105+0000 7f2b8c200640 1 -- 172.21.15.66:0/102466273 wait complete. 2024-09-10T22:00:38.129 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-10T22:00:36.310810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":116,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":70,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231340,"kb_used_data":2636,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743380,"statfs":{"total":773094113280,"available":772857221120,"internally_reserved":0,"allocated":2699264,"data_stored":1655558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":234016308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.154708"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663811+0000","last_change":"2024-09-10T22:00:33.192188+0000","last_active":"2024-09-10T22:00:34.663811+0000","last_peered":"2024-09-10T22:00:34.663811+0000","last_clean":"2024-09-10T22:00:34.663811+0000","last_became_active":"2024-09-10T22:00:33.191616+0000","last_became_peered":"2024-09-10T22:00:33.191616+0000","last_unstale":"2024-09-10T22:00:34.663811+0000","last_undegraded":"2024-09-10T22:00:34.663811+0000","last_fullsized":"2024-09-10T22:00:34.663811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:03.715049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663157+0000","last_change":"2024-09-10T22:00:33.213581+0000","last_active":"2024-09-10T22:00:34.663157+0000","last_peered":"2024-09-10T22:00:34.663157+0000","last_clean":"2024-09-10T22:00:34.663157+0000","last_became_active":"2024-09-10T22:00:33.213202+0000","last_became_peered":"2024-09-10T22:00:33.213202+0000","last_unstale":"2024-09-10T22:00:34.663157+0000","last_undegraded":"2024-09-10T22:00:34.663157+0000","last_fullsized":"2024-09-10T22:00:34.663157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:03.339805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163552+0000","last_change":"2024-09-10T22:00:31.163552+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163552+0000","last_undegraded":"2024-09-10T22:00:31.163552+0000","last_fullsized":"2024-09-10T22:00:31.163552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.159305+0000","last_change":"2024-09-10T22:00:31.159305+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.159305+0000","last_undegraded":"2024-09-10T22:00:31.159305+0000","last_fullsized":"2024-09-10T22:00:31.159305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163295+0000","last_change":"2024-09-10T22:00:31.163295+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163295+0000","last_undegraded":"2024-09-10T22:00:31.163295+0000","last_fullsized":"2024-09-10T22:00:31.163295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.164731+0000","last_change":"2024-09-10T22:00:31.164731+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.164731+0000","last_undegraded":"2024-09-10T22:00:31.164731+0000","last_fullsized":"2024-09-10T22:00:31.164731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.160703+0000","last_change":"2024-09-10T22:00:31.160703+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.160703+0000","last_undegraded":"2024-09-10T22:00:31.160703+0000","last_fullsized":"2024-09-10T22:00:31.160703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163225+0000","last_change":"2024-09-10T22:00:31.163225+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163225+0000","last_undegraded":"2024-09-10T22:00:31.163225+0000","last_fullsized":"2024-09-10T22:00:31.163225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.170537+0000","last_change":"2024-09-10T22:00:31.170537+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.170537+0000","last_undegraded":"2024-09-10T22:00:31.170537+0000","last_fullsized":"2024-09-10T22:00:31.170537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.159538+0000","last_change":"2024-09-10T22:00:31.159538+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.159538+0000","last_undegraded":"2024-09-10T22:00:31.159538+0000","last_fullsized":"2024-09-10T22:00:31.159538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.162179+0000","last_change":"2024-09-10T22:00:31.162179+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.162179+0000","last_undegraded":"2024-09-10T22:00:31.162179+0000","last_fullsized":"2024-09-10T22:00:31.162179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163836+0000","last_change":"2024-09-10T22:00:31.163836+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163836+0000","last_undegraded":"2024-09-10T22:00:31.163836+0000","last_fullsized":"2024-09-10T22:00:31.163836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.159305+0000","last_change":"2024-09-10T22:00:31.159305+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.159305+0000","last_undegraded":"2024-09-10T22:00:31.159305+0000","last_fullsized":"2024-09-10T22:00:31.159305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.662989+0000","last_change":"2024-09-10T22:00:32.182584+0000","last_active":"2024-09-10T22:00:34.662989+0000","last_peered":"2024-09-10T22:00:34.662989+0000","last_clean":"2024-09-10T22:00:34.662989+0000","last_became_active":"2024-09-10T22:00:32.182289+0000","last_became_peered":"2024-09-10T22:00:32.182289+0000","last_unstale":"2024-09-10T22:00:34.662989+0000","last_undegraded":"2024-09-10T22:00:34.662989+0000","last_fullsized":"2024-09-10T22:00:34.662989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:41.773634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.159340+0000","last_change":"2024-09-10T22:00:31.159340+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.159340+0000","last_undegraded":"2024-09-10T22:00:31.159340+0000","last_fullsized":"2024-09-10T22:00:31.159340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663916+0000","last_change":"2024-09-10T22:00:32.184130+0000","last_active":"2024-09-10T22:00:34.663916+0000","last_peered":"2024-09-10T22:00:34.663916+0000","last_clean":"2024-09-10T22:00:34.663916+0000","last_became_active":"2024-09-10T22:00:32.182524+0000","last_became_peered":"2024-09-10T22:00:32.182524+0000","last_unstale":"2024-09-10T22:00:34.663916+0000","last_undegraded":"2024-09-10T22:00:34.663916+0000","last_fullsized":"2024-09-10T22:00:34.663916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:26.056799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169683+0000","last_change":"2024-09-10T22:00:31.169683+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169683+0000","last_undegraded":"2024-09-10T22:00:31.169683+0000","last_fullsized":"2024-09-10T22:00:31.169683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663655+0000","last_change":"2024-09-10T22:00:32.195097+0000","last_active":"2024-09-10T22:00:34.663655+0000","last_peered":"2024-09-10T22:00:34.663655+0000","last_clean":"2024-09-10T22:00:34.663655+0000","last_became_active":"2024-09-10T22:00:32.194103+0000","last_became_peered":"2024-09-10T22:00:32.194103+0000","last_unstale":"2024-09-10T22:00:34.663655+0000","last_undegraded":"2024-09-10T22:00:34.663655+0000","last_fullsized":"2024-09-10T22:00:34.663655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:57.181450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-10T22:00:31.167874+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:00:31.167874+0000","last_peered":"2024-09-10T22:00:31.167874+0000","last_clean":"2024-09-10T22:00:31.167874+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:00:31.167874+0000","last_undegraded":"2024-09-10T22:00:31.167874+0000","last_fullsized":"2024-09-10T22:00:31.167874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.164958+0000","last_change":"2024-09-10T22:00:31.164958+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.164958+0000","last_undegraded":"2024-09-10T22:00:31.164958+0000","last_fullsized":"2024-09-10T22:00:31.164958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163045+0000","last_change":"2024-09-10T22:00:31.163045+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163045+0000","last_undegraded":"2024-09-10T22:00:31.163045+0000","last_fullsized":"2024-09-10T22:00:31.163045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663264+0000","last_change":"2024-09-10T22:00:32.178337+0000","last_active":"2024-09-10T22:00:34.663264+0000","last_peered":"2024-09-10T22:00:34.663264+0000","last_clean":"2024-09-10T22:00:34.663264+0000","last_became_active":"2024-09-10T22:00:32.178053+0000","last_became_peered":"2024-09-10T22:00:32.178053+0000","last_unstale":"2024-09-10T22:00:34.663264+0000","last_undegraded":"2024-09-10T22:00:34.663264+0000","last_fullsized":"2024-09-10T22:00:34.663264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:26.949229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163215+0000","last_change":"2024-09-10T22:00:31.163215+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163215+0000","last_undegraded":"2024-09-10T22:00:31.163215+0000","last_fullsized":"2024-09-10T22:00:31.163215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169542+0000","last_change":"2024-09-10T22:00:31.169542+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169542+0000","last_undegraded":"2024-09-10T22:00:31.169542+0000","last_fullsized":"2024-09-10T22:00:31.169542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.164658+0000","last_change":"2024-09-10T22:00:31.164658+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.164658+0000","last_undegraded":"2024-09-10T22:00:31.164658+0000","last_fullsized":"2024-09-10T22:00:31.164658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169404+0000","last_change":"2024-09-10T22:00:31.169404+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169404+0000","last_undegraded":"2024-09-10T22:00:31.169404+0000","last_fullsized":"2024-09-10T22:00:31.169404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.160191+0000","last_change":"2024-09-10T22:00:31.160191+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.160191+0000","last_undegraded":"2024-09-10T22:00:31.160191+0000","last_fullsized":"2024-09-10T22:00:31.160191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163616+0000","last_change":"2024-09-10T22:00:31.163616+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163616+0000","last_undegraded":"2024-09-10T22:00:31.163616+0000","last_fullsized":"2024-09-10T22:00:31.163616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169413+0000","last_change":"2024-09-10T22:00:31.169413+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169413+0000","last_undegraded":"2024-09-10T22:00:31.169413+0000","last_fullsized":"2024-09-10T22:00:31.169413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.169328+0000","last_change":"2024-09-10T22:00:31.169328+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.169328+0000","last_undegraded":"2024-09-10T22:00:31.169328+0000","last_fullsized":"2024-09-10T22:00:31.169328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163027+0000","last_change":"2024-09-10T22:00:31.163027+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163027+0000","last_undegraded":"2024-09-10T22:00:31.163027+0000","last_fullsized":"2024-09-10T22:00:31.163027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.163713+0000","last_change":"2024-09-10T22:00:31.163713+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.163713+0000","last_undegraded":"2024-09-10T22:00:31.163713+0000","last_fullsized":"2024-09-10T22:00:31.163713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.166355+0000","last_change":"2024-09-10T22:00:31.166355+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.166355+0000","last_undegraded":"2024-09-10T22:00:31.166355+0000","last_fullsized":"2024-09-10T22:00:31.166355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.164132+0000","last_change":"2024-09-10T22:00:31.164132+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.164132+0000","last_undegraded":"2024-09-10T22:00:31.164132+0000","last_fullsized":"2024-09-10T22:00:31.164132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:00:32.154625+0000","last_change":"2024-09-10T22:00:32.154625+0000","last_active":"2024-09-10T22:00:32.154625+0000","last_peered":"2024-09-10T22:00:32.154625+0000","last_clean":"2024-09-10T22:00:32.154625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:32.154625+0000","last_undegraded":"2024-09-10T22:00:32.154625+0000","last_fullsized":"2024-09-10T22:00:32.154625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T22:00:31.162408+0000","last_change":"2024-09-10T22:00:31.162408+0000","last_active":"2024-09-10T22:00:31.148076+0000","last_peered":"2024-09-10T22:00:31.148076+0000","last_clean":"2024-09-10T22:00:31.148076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:00:31.162408+0000","last_undegraded":"2024-09-10T22:00:31.162408+0000","last_fullsized":"2024-09-10T22:00:31.162408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:00:38.133 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-10T22:00:38.133 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.133 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-10T22:00:38.133 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.134 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-10T22:00:38.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.134 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-10T22:00:38.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.134 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-10T22:00:38.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.134 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-10T22:00:38.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.134 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-10T22:00:38.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.135 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-10T22:00:38.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.135 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-10T22:00:38.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.135 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-10T22:00:38.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.135 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-10T22:00:38.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.135 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-10T22:00:38.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.135 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-10T22:00:38.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.136 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-10T22:00:38.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.136 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-10T22:00:38.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.136 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-10T22:00:38.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.136 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-10T22:00:38.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.136 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-10T22:00:38.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.137 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-10T22:00:38.137 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.137 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-10T22:00:38.137 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.137 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-10T22:00:38.137 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.137 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-10T22:00:38.137 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.137 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-10T22:00:38.137 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.137 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-10T22:00:38.137 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.138 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-10T22:00:38.138 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.138 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-10T22:00:38.138 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.138 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-10T22:00:38.138 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.138 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-10T22:00:38.138 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.138 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-10T22:00:38.138 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.138 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-10T22:00:38.139 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.139 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-10T22:00:38.139 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.139 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1b is not active+clean 2024-09-10T22:00:38.139 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.163552+0000', 'last_change': '2024-09-10T22:00:31.163552+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.163552+0000', 'last_undegraded': '2024-09-10T22:00:31.163552+0000', 'last_fullsized': '2024-09-10T22:00:31.163552+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T22:00:38.139 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-10T22:00:38.139 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.139 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1a is not active+clean 2024-09-10T22:00:38.139 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.159305+0000', 'last_change': '2024-09-10T22:00:31.159305+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.159305+0000', 'last_undegraded': '2024-09-10T22:00:31.159305+0000', 'last_fullsized': '2024-09-10T22:00:31.159305+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T22:00:38.139 INFO:tasks.ceph.ceph_manager.ceph:PG 2.19 is not active+clean 2024-09-10T22:00:38.139 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.163295+0000', 'last_change': '2024-09-10T22:00:31.163295+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.163295+0000', 'last_undegraded': '2024-09-10T22:00:31.163295+0000', 'last_fullsized': '2024-09-10T22:00:31.163295+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T22:00:38.140 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-10T22:00:38.140 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.140 INFO:tasks.ceph.ceph_manager.ceph:PG 2.18 is not active+clean 2024-09-10T22:00:38.140 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.164731+0000', 'last_change': '2024-09-10T22:00:31.164731+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.164731+0000', 'last_undegraded': '2024-09-10T22:00:31.164731+0000', 'last_fullsized': '2024-09-10T22:00:31.164731+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T22:00:38.140 INFO:tasks.ceph.ceph_manager.ceph:PG 3.19 is not active+clean 2024-09-10T22:00:38.140 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.140 INFO:tasks.ceph.ceph_manager.ceph:PG 2.17 is not active+clean 2024-09-10T22:00:38.140 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.160703+0000', 'last_change': '2024-09-10T22:00:31.160703+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.160703+0000', 'last_undegraded': '2024-09-10T22:00:31.160703+0000', 'last_fullsized': '2024-09-10T22:00:31.160703+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T22:00:38.140 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-10T22:00:38.140 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.140 INFO:tasks.ceph.ceph_manager.ceph:PG 2.16 is not active+clean 2024-09-10T22:00:38.140 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.163225+0000', 'last_change': '2024-09-10T22:00:31.163225+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.163225+0000', 'last_undegraded': '2024-09-10T22:00:31.163225+0000', 'last_fullsized': '2024-09-10T22:00:31.163225+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T22:00:38.141 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-10T22:00:38.141 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.141 INFO:tasks.ceph.ceph_manager.ceph:PG 2.15 is not active+clean 2024-09-10T22:00:38.141 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.170537+0000', 'last_change': '2024-09-10T22:00:31.170537+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.170537+0000', 'last_undegraded': '2024-09-10T22:00:31.170537+0000', 'last_fullsized': '2024-09-10T22:00:31.170537+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T22:00:38.141 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-10T22:00:38.141 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.141 INFO:tasks.ceph.ceph_manager.ceph:PG 2.14 is not active+clean 2024-09-10T22:00:38.141 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.159538+0000', 'last_change': '2024-09-10T22:00:31.159538+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.159538+0000', 'last_undegraded': '2024-09-10T22:00:31.159538+0000', 'last_fullsized': '2024-09-10T22:00:31.159538+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T22:00:38.141 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-10T22:00:38.141 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.142 INFO:tasks.ceph.ceph_manager.ceph:PG 2.13 is not active+clean 2024-09-10T22:00:38.142 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.162179+0000', 'last_change': '2024-09-10T22:00:31.162179+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.162179+0000', 'last_undegraded': '2024-09-10T22:00:31.162179+0000', 'last_fullsized': '2024-09-10T22:00:31.162179+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T22:00:38.142 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-10T22:00:38.142 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.142 INFO:tasks.ceph.ceph_manager.ceph:PG 2.12 is not active+clean 2024-09-10T22:00:38.142 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.163836+0000', 'last_change': '2024-09-10T22:00:31.163836+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.163836+0000', 'last_undegraded': '2024-09-10T22:00:31.163836+0000', 'last_fullsized': '2024-09-10T22:00:31.163836+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T22:00:38.142 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-10T22:00:38.142 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.142 INFO:tasks.ceph.ceph_manager.ceph:PG 2.11 is not active+clean 2024-09-10T22:00:38.143 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.159305+0000', 'last_change': '2024-09-10T22:00:31.159305+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.159305+0000', 'last_undegraded': '2024-09-10T22:00:31.159305+0000', 'last_fullsized': '2024-09-10T22:00:31.159305+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T22:00:38.143 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-10T22:00:38.143 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.143 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-10T22:00:38.143 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.143 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-09-10T22:00:38.143 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.159340+0000', 'last_change': '2024-09-10T22:00:31.159340+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.159340+0000', 'last_undegraded': '2024-09-10T22:00:31.159340+0000', 'last_fullsized': '2024-09-10T22:00:31.159340+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T22:00:38.143 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-10T22:00:38.143 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.143 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-10T22:00:38.143 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.144 INFO:tasks.ceph.ceph_manager.ceph:PG 2.d is not active+clean 2024-09-10T22:00:38.144 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.169683+0000', 'last_change': '2024-09-10T22:00:31.169683+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.169683+0000', 'last_undegraded': '2024-09-10T22:00:31.169683+0000', 'last_fullsized': '2024-09-10T22:00:31.169683+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T22:00:38.144 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-10T22:00:38.144 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.144 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-10T22:00:38.144 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.144 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2024-09-10T22:00:38.144 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.164958+0000', 'last_change': '2024-09-10T22:00:31.164958+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.164958+0000', 'last_undegraded': '2024-09-10T22:00:31.164958+0000', 'last_fullsized': '2024-09-10T22:00:31.164958+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T22:00:38.144 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2 is not active+clean 2024-09-10T22:00:38.145 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.145 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-09-10T22:00:38.145 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.163045+0000', 'last_change': '2024-09-10T22:00:31.163045+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.163045+0000', 'last_undegraded': '2024-09-10T22:00:31.163045+0000', 'last_fullsized': '2024-09-10T22:00:31.163045+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:00:38.145 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-10T22:00:38.145 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.145 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-10T22:00:38.145 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.145 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2024-09-10T22:00:38.145 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.163215+0000', 'last_change': '2024-09-10T22:00:31.163215+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.163215+0000', 'last_undegraded': '2024-09-10T22:00:31.163215+0000', 'last_fullsized': '2024-09-10T22:00:31.163215+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T22:00:38.145 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-10T22:00:38.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.146 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2024-09-10T22:00:38.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.169542+0000', 'last_change': '2024-09-10T22:00:31.169542+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.169542+0000', 'last_undegraded': '2024-09-10T22:00:31.169542+0000', 'last_fullsized': '2024-09-10T22:00:31.169542+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T22:00:38.146 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-10T22:00:38.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.146 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-09-10T22:00:38.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.164658+0000', 'last_change': '2024-09-10T22:00:31.164658+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.164658+0000', 'last_undegraded': '2024-09-10T22:00:31.164658+0000', 'last_fullsized': '2024-09-10T22:00:31.164658+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:00:38.147 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-10T22:00:38.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.147 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2024-09-10T22:00:38.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.169404+0000', 'last_change': '2024-09-10T22:00:31.169404+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.169404+0000', 'last_undegraded': '2024-09-10T22:00:31.169404+0000', 'last_fullsized': '2024-09-10T22:00:31.169404+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T22:00:38.147 INFO:tasks.ceph.ceph_manager.ceph:PG 3.7 is not active+clean 2024-09-10T22:00:38.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.147 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2024-09-10T22:00:38.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.160191+0000', 'last_change': '2024-09-10T22:00:31.160191+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.160191+0000', 'last_undegraded': '2024-09-10T22:00:31.160191+0000', 'last_fullsized': '2024-09-10T22:00:31.160191+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T22:00:38.147 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-10T22:00:38.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.147 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-09-10T22:00:38.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.163616+0000', 'last_change': '2024-09-10T22:00:31.163616+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.163616+0000', 'last_undegraded': '2024-09-10T22:00:31.163616+0000', 'last_fullsized': '2024-09-10T22:00:31.163616+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:00:38.148 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-10T22:00:38.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.148 INFO:tasks.ceph.ceph_manager.ceph:PG 2.9 is not active+clean 2024-09-10T22:00:38.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.169413+0000', 'last_change': '2024-09-10T22:00:31.169413+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.169413+0000', 'last_undegraded': '2024-09-10T22:00:31.169413+0000', 'last_fullsized': '2024-09-10T22:00:31.169413+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T22:00:38.148 INFO:tasks.ceph.ceph_manager.ceph:PG 3.8 is not active+clean 2024-09-10T22:00:38.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.148 INFO:tasks.ceph.ceph_manager.ceph:PG 2.a is not active+clean 2024-09-10T22:00:38.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.169328+0000', 'last_change': '2024-09-10T22:00:31.169328+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.169328+0000', 'last_undegraded': '2024-09-10T22:00:31.169328+0000', 'last_fullsized': '2024-09-10T22:00:31.169328+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T22:00:38.149 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-10T22:00:38.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.149 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-09-10T22:00:38.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.163027+0000', 'last_change': '2024-09-10T22:00:31.163027+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.163027+0000', 'last_undegraded': '2024-09-10T22:00:31.163027+0000', 'last_fullsized': '2024-09-10T22:00:31.163027+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:00:38.149 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-10T22:00:38.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.149 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-10T22:00:38.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.149 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-10T22:00:38.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.163713+0000', 'last_change': '2024-09-10T22:00:31.163713+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.163713+0000', 'last_undegraded': '2024-09-10T22:00:31.163713+0000', 'last_fullsized': '2024-09-10T22:00:31.163713+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:00:38.149 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-10T22:00:38.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.150 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-09-10T22:00:38.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.166355+0000', 'last_change': '2024-09-10T22:00:31.166355+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.166355+0000', 'last_undegraded': '2024-09-10T22:00:31.166355+0000', 'last_fullsized': '2024-09-10T22:00:31.166355+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:00:38.150 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-10T22:00:38.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.150 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1e is not active+clean 2024-09-10T22:00:38.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.164132+0000', 'last_change': '2024-09-10T22:00:31.164132+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.164132+0000', 'last_undegraded': '2024-09-10T22:00:31.164132+0000', 'last_fullsized': '2024-09-10T22:00:31.164132+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T22:00:38.150 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-10T22:00:38.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:00:32.154625+0000', 'last_change': '2024-09-10T22:00:32.154625+0000', 'last_active': '2024-09-10T22:00:32.154625+0000', 'last_peered': '2024-09-10T22:00:32.154625+0000', 'last_clean': '2024-09-10T22:00:32.154625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:32.154625+0000', 'last_undegraded': '2024-09-10T22:00:32.154625+0000', 'last_fullsized': '2024-09-10T22:00:32.154625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:32.154625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:00:38.150 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1f is not active+clean 2024-09-10T22:00:38.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:00:31.162408+0000', 'last_change': '2024-09-10T22:00:31.162408+0000', 'last_active': '2024-09-10T22:00:31.148076+0000', 'last_peered': '2024-09-10T22:00:31.148076+0000', 'last_clean': '2024-09-10T22:00:31.148076+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:00:31.162408+0000', 'last_undegraded': '2024-09-10T22:00:31.162408+0000', 'last_fullsized': '2024-09-10T22:00:31.162408+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'last_clean_scrub_stamp': '2024-09-10T22:00:31.148076+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T22:00:38.151 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T22:00:38.152 INFO:tasks.ceph.mds.b.smithi094.stderr:2024-09-10T22:00:38.145+0000 7fe5bfe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:00:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.313+0000 7f6bdfe00640 1 -- 172.21.15.66:0/3470589738 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6bd8160490 msgr2=0x7f6bd80d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.313+0000 7f6bdfe00640 1 --2- 172.21.15.66:0/3470589738 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6bd8160490 0x7f6bd80d0300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009870 tx=0x7f6bcc01f050 comp rx=0 tx=0).stop 2024-09-10T22:00:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.313+0000 7f6bdfe00640 1 -- 172.21.15.66:0/3470589738 shutdown_connections 2024-09-10T22:00:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.313+0000 7f6bdfe00640 1 --2- 172.21.15.66:0/3470589738 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6bd8160490 0x7f6bd80d0300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.313+0000 7f6bdfe00640 1 -- 172.21.15.66:0/3470589738 >> 172.21.15.66:0/3470589738 conn(0x7f6bd80d1d20 msgr2=0x7f6bd80d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.313+0000 7f6bdfe00640 1 -- 172.21.15.66:0/3470589738 shutdown_connections 2024-09-10T22:00:38.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.313+0000 7f6bdfe00640 1 -- 172.21.15.66:0/3470589738 wait complete. 2024-09-10T22:00:38.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.313+0000 7f6bdfe00640 1 Processor -- start 2024-09-10T22:00:38.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.313+0000 7f6bdfe00640 1 -- start start 2024-09-10T22:00:38.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdfe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6bd80af1e0 0x7f6bd80b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:38.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd80b2a20 0x7f6bd815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:38.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bd8160490 0x7f6bd815b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:38.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdfe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd80b3580 con 0x7f6bd80b2a20 2024-09-10T22:00:38.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdfe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd80b3400 con 0x7f6bd8160490 2024-09-10T22:00:38.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdfe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd80b11f0 con 0x7f6bd80af1e0 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd80b2a20 0x7f6bd815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd80b2a20 0x7f6bd815b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60186/0 (socket says 172.21.15.66:60186) 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdca00640 1 -- 172.21.15.66:0/2407333651 learned_addr learned my addr 172.21.15.66:0/2407333651 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdd400640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6bd80af1e0 0x7f6bd80b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdca00640 1 -- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6bd80af1e0 msgr2=0x7f6bd80b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdca00640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6bd80af1e0 0x7f6bd80b73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdca00640 1 -- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bd8160490 msgr2=0x7f6bd815b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdde00640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bd8160490 0x7f6bd815b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdca00640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bd8160490 0x7f6bd815b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdca00640 1 -- 172.21.15.66:0/2407333651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd8155170 con 0x7f6bd80b2a20 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdde00640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bd8160490 0x7f6bd815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdca00640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd80b2a20 0x7f6bd815b3a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4001ec0 tx=0x7f6bc400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:38.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bd5600640 1 -- 172.21.15.66:0/2407333651 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4016020 con 0x7f6bd80b2a20 2024-09-10T22:00:38.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bd5600640 1 -- 172.21.15.66:0/2407333651 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bc4012940 con 0x7f6bd80b2a20 2024-09-10T22:00:38.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bd5600640 1 -- 172.21.15.66:0/2407333651 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4012d90 con 0x7f6bd80b2a20 2024-09-10T22:00:38.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdfe00640 1 -- 172.21.15.66:0/2407333651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd81543b0 con 0x7f6bd80b2a20 2024-09-10T22:00:38.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdfe00640 1 -- 172.21.15.66:0/2407333651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd8154970 con 0x7f6bd80b2a20 2024-09-10T22:00:38.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bdfe00640 1 -- 172.21.15.66:0/2407333651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd80b2710 con 0x7f6bd80b2a20 2024-09-10T22:00:38.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.317+0000 7f6bd5600640 1 -- 172.21.15.66:0/2407333651 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6bc401b400 con 0x7f6bd80b2a20 2024-09-10T22:00:38.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.321+0000 7f6bd5600640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6bb80725e0 0x7f6bb8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:38.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.321+0000 7f6bd5600640 1 -- 172.21.15.66:0/2407333651 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6bc40934d0 con 0x7f6bd80b2a20 2024-09-10T22:00:38.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.321+0000 7f6bdd400640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6bb80725e0 0x7f6bb8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:38.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.321+0000 7f6bdd400640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6bb80725e0 0x7f6bb8074aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0b2000 tx=0x7f6bcc01d890 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:38.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.325+0000 7f6bd5600640 1 -- 172.21.15.66:0/2407333651 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6bc4020020 con 0x7f6bd80b2a20 2024-09-10T22:00:38.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.557+0000 7f6bdfe00640 1 -- 172.21.15.66:0/2407333651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f6bd80d00e0 con 0x7f6bd80b2a20 2024-09-10T22:00:38.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.557+0000 7f6bd5600640 1 -- 172.21.15.66:0/2407333651 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1210 (secure 0 0 0) 0x7f6bc405eba0 con 0x7f6bd80b2a20 2024-09-10T22:00:38.564 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:38.564 INFO:teuthology.orchestra.run.smithi066.stdout:{"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1726005621,"num_in_osds":8,"osd_in_since":1726005611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":62},{"state_name":"creating+peering","count":28},{"state_name":"active+clean","count":7}],"num_pgs":97,"num_pools":3,"num_objects":7,"data_bytes":590368,"bytes_used":236892160,"bytes_avail":772857221120,"bytes_total":773094113280,"unknown_pgs_ratio":0.63917523622512817,"inactive_pgs_ratio":0.28865978121757507,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":8,"btime":"2024-09-10T22:00:36:190941+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4378}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-10T22:00:06.939671+0000","services":{}},"progress_events":{}} 2024-09-10T22:00:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.561+0000 7f6bdfe00640 1 -- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6bb80725e0 msgr2=0x7f6bb8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.561+0000 7f6bdfe00640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6bb80725e0 0x7f6bb8074aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0b2000 tx=0x7f6bcc01d890 comp rx=0 tx=0).stop 2024-09-10T22:00:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.561+0000 7f6bdfe00640 1 -- 172.21.15.66:0/2407333651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd80b2a20 msgr2=0x7f6bd815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.561+0000 7f6bdfe00640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd80b2a20 0x7f6bd815b3a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4001ec0 tx=0x7f6bc400c960 comp rx=0 tx=0).stop 2024-09-10T22:00:38.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.561+0000 7f6bdfe00640 1 -- 172.21.15.66:0/2407333651 shutdown_connections 2024-09-10T22:00:38.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.561+0000 7f6bdfe00640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6bb80725e0 0x7f6bb8074aa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.561+0000 7f6bdfe00640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bd8160490 0x7f6bd815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.561+0000 7f6bdfe00640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd80b2a20 0x7f6bd815b3a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.561+0000 7f6bdfe00640 1 --2- 172.21.15.66:0/2407333651 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6bd80af1e0 0x7f6bd80b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.561+0000 7f6bdfe00640 1 -- 172.21.15.66:0/2407333651 >> 172.21.15.66:0/2407333651 conn(0x7f6bd80d1d20 msgr2=0x7f6bd80cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:38.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.561+0000 7f6bdfe00640 1 -- 172.21.15.66:0/2407333651 shutdown_connections 2024-09-10T22:00:38.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.561+0000 7f6bdfe00640 1 -- 172.21.15.66:0/2407333651 wait complete. 2024-09-10T22:00:38.586 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T22:00:38.586 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:00:38.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.725+0000 7f3d06000640 1 -- 172.21.15.66:0/2685940754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d000afc20 msgr2=0x7f3d000ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.725+0000 7f3d06000640 1 --2- 172.21.15.66:0/2685940754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d000afc20 0x7f3d000ce880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009bc0 tx=0x7f3cf801cb10 comp rx=0 tx=0).stop 2024-09-10T22:00:38.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.725+0000 7f3d06000640 1 -- 172.21.15.66:0/2685940754 shutdown_connections 2024-09-10T22:00:38.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.725+0000 7f3d06000640 1 --2- 172.21.15.66:0/2685940754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d000afc20 0x7f3d000ce880 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.725+0000 7f3d06000640 1 -- 172.21.15.66:0/2685940754 >> 172.21.15.66:0/2685940754 conn(0x7f3d000d1c60 msgr2=0x7f3d000d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:38.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.725+0000 7f3d06000640 1 -- 172.21.15.66:0/2685940754 shutdown_connections 2024-09-10T22:00:38.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.725+0000 7f3d06000640 1 -- 172.21.15.66:0/2685940754 wait complete. 2024-09-10T22:00:38.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3d06000640 1 Processor -- start 2024-09-10T22:00:38.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3d06000640 1 -- start start 2024-09-10T22:00:38.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3d06000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d000afc20 0x7f3d00150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:38.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3d06000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d000d06b0 0x7f3d000cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:38.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3d06000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0015bd40 0x7f3d0015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:38.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3d06000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0016a1b0 con 0x7f3d000d06b0 2024-09-10T22:00:38.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3d06000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0016a030 con 0x7f3d0015bd40 2024-09-10T22:00:38.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3d06000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0016a330 con 0x7f3d000afc20 2024-09-10T22:00:38.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cfea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d000d06b0 0x7f3d000cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:38.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cfea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d000d06b0 0x7f3d000cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60198/0 (socket says 172.21.15.66:60198) 2024-09-10T22:00:38.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cfea00640 1 -- 172.21.15.66:0/1049820262 learned_addr learned my addr 172.21.15.66:0/1049820262 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:38.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cff400640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d000afc20 0x7f3d00150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:38.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cffe00640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0015bd40 0x7f3d0015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:38.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cfea00640 1 -- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d000afc20 msgr2=0x7f3d00150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cfea00640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d000afc20 0x7f3d00150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cfea00640 1 -- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0015bd40 msgr2=0x7f3d0015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cfea00640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0015bd40 0x7f3d0015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cfea00640 1 -- 172.21.15.66:0/1049820262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d00150680 con 0x7f3d000d06b0 2024-09-10T22:00:38.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cffe00640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0015bd40 0x7f3d0015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:00:38.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cfea00640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d000d06b0 0x7f3d000cb440 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00c460 tx=0x7f3cec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:38.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cf7400640 1 -- 172.21.15.66:0/1049820262 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec016020 con 0x7f3d000d06b0 2024-09-10T22:00:38.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cf7400640 1 -- 172.21.15.66:0/1049820262 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cec012920 con 0x7f3d000d06b0 2024-09-10T22:00:38.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3cf7400640 1 -- 172.21.15.66:0/1049820262 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec012d50 con 0x7f3d000d06b0 2024-09-10T22:00:38.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3d06000640 1 -- 172.21.15.66:0/1049820262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d001fbee0 con 0x7f3d000d06b0 2024-09-10T22:00:38.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3d06000640 1 -- 172.21.15.66:0/1049820262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d000cac40 con 0x7f3d000d06b0 2024-09-10T22:00:38.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.729+0000 7f3d06000640 1 -- 172.21.15.66:0/1049820262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d000cffa0 con 0x7f3d000d06b0 2024-09-10T22:00:38.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.733+0000 7f3cf7400640 1 -- 172.21.15.66:0/1049820262 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3cec01b400 con 0x7f3d000d06b0 2024-09-10T22:00:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.733+0000 7f3cf7400640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3ce0072610 0x7f3ce0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.733+0000 7f3cf7400640 1 -- 172.21.15.66:0/1049820262 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3cec093500 con 0x7f3d000d06b0 2024-09-10T22:00:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.733+0000 7f3cff400640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3ce0072610 0x7f3ce0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:38.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.733+0000 7f3cff400640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3ce0072610 0x7f3ce0074ad0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0000f70 tx=0x7f3cf0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:38.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.737+0000 7f3cf7400640 1 -- 172.21.15.66:0/1049820262 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3cec020020 con 0x7f3d000d06b0 2024-09-10T22:00:38.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.905+0000 7f3d06000640 1 -- 172.21.15.66:0/1049820262 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d000b0df0 con 0x7f3ce0072610 2024-09-10T22:00:38.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.913+0000 7f3cf7400640 1 -- 172.21.15.66:0/1049820262 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7f3d000b0df0 con 0x7f3ce0072610 2024-09-10T22:00:38.922 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:38.923 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:00:38.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.917+0000 7f3d06000640 1 -- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3ce0072610 msgr2=0x7f3ce0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.917+0000 7f3d06000640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3ce0072610 0x7f3ce0074ad0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0000f70 tx=0x7f3cf0009040 comp rx=0 tx=0).stop 2024-09-10T22:00:38.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.917+0000 7f3d06000640 1 -- 172.21.15.66:0/1049820262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d000d06b0 msgr2=0x7f3d000cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.917+0000 7f3d06000640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d000d06b0 0x7f3d000cb440 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00c460 tx=0x7f3cec00c920 comp rx=0 tx=0).stop 2024-09-10T22:00:38.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.917+0000 7f3d06000640 1 -- 172.21.15.66:0/1049820262 shutdown_connections 2024-09-10T22:00:38.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.917+0000 7f3d06000640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3ce0072610 0x7f3ce0074ad0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.917+0000 7f3d06000640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0015bd40 0x7f3d0015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.917+0000 7f3d06000640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d000d06b0 0x7f3d000cb440 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.917+0000 7f3d06000640 1 --2- 172.21.15.66:0/1049820262 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d000afc20 0x7f3d00150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.917+0000 7f3d06000640 1 -- 172.21.15.66:0/1049820262 >> 172.21.15.66:0/1049820262 conn(0x7f3d000d1c60 msgr2=0x7f3d0015a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:38.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.917+0000 7f3d06000640 1 -- 172.21.15.66:0/1049820262 shutdown_connections 2024-09-10T22:00:38.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:38.917+0000 7f3d06000640 1 -- 172.21.15.66:0/1049820262 wait complete. 2024-09-10T22:00:38.941 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-10T22:00:38.311227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235848,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738872,"statfs":{"total":773094113280,"available":772852604928,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.148433"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665723+0000","last_change":"2024-09-10T22:00:33.198521+0000","last_active":"2024-09-10T22:00:34.665723+0000","last_peered":"2024-09-10T22:00:34.665723+0000","last_clean":"2024-09-10T22:00:34.665723+0000","last_became_active":"2024-09-10T22:00:33.197869+0000","last_became_peered":"2024-09-10T22:00:33.197869+0000","last_unstale":"2024-09-10T22:00:34.665723+0000","last_undegraded":"2024-09-10T22:00:34.665723+0000","last_fullsized":"2024-09-10T22:00:34.665723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:48.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183587+0000","last_change":"2024-09-10T22:00:33.182512+0000","last_active":"2024-09-10T22:00:34.183587+0000","last_peered":"2024-09-10T22:00:34.183587+0000","last_clean":"2024-09-10T22:00:34.183587+0000","last_became_active":"2024-09-10T22:00:33.182230+0000","last_became_peered":"2024-09-10T22:00:33.182230+0000","last_unstale":"2024-09-10T22:00:34.183587+0000","last_undegraded":"2024-09-10T22:00:34.183587+0000","last_fullsized":"2024-09-10T22:00:34.183587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:08.694915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663553+0000","last_change":"2024-09-10T22:00:33.182382+0000","last_active":"2024-09-10T22:00:34.663553+0000","last_peered":"2024-09-10T22:00:34.663553+0000","last_clean":"2024-09-10T22:00:34.663553+0000","last_became_active":"2024-09-10T22:00:33.182115+0000","last_became_peered":"2024-09-10T22:00:33.182115+0000","last_unstale":"2024-09-10T22:00:34.663553+0000","last_undegraded":"2024-09-10T22:00:34.663553+0000","last_fullsized":"2024-09-10T22:00:34.663553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:34.220172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185630+0000","last_change":"2024-09-10T22:00:33.215454+0000","last_active":"2024-09-10T22:00:34.185630+0000","last_peered":"2024-09-10T22:00:34.185630+0000","last_clean":"2024-09-10T22:00:34.185630+0000","last_became_active":"2024-09-10T22:00:33.214713+0000","last_became_peered":"2024-09-10T22:00:33.214713+0000","last_unstale":"2024-09-10T22:00:34.185630+0000","last_undegraded":"2024-09-10T22:00:34.185630+0000","last_fullsized":"2024-09-10T22:00:34.185630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:37.559387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665058+0000","last_change":"2024-09-10T22:00:33.201925+0000","last_active":"2024-09-10T22:00:34.665058+0000","last_peered":"2024-09-10T22:00:34.665058+0000","last_clean":"2024-09-10T22:00:34.665058+0000","last_became_active":"2024-09-10T22:00:33.197799+0000","last_became_peered":"2024-09-10T22:00:33.197799+0000","last_unstale":"2024-09-10T22:00:34.665058+0000","last_undegraded":"2024-09-10T22:00:34.665058+0000","last_fullsized":"2024-09-10T22:00:34.665058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:37.217143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185144+0000","last_change":"2024-09-10T22:00:33.211402+0000","last_active":"2024-09-10T22:00:34.185144+0000","last_peered":"2024-09-10T22:00:34.185144+0000","last_clean":"2024-09-10T22:00:34.185144+0000","last_became_active":"2024-09-10T22:00:33.210608+0000","last_became_peered":"2024-09-10T22:00:33.210608+0000","last_unstale":"2024-09-10T22:00:34.185144+0000","last_undegraded":"2024-09-10T22:00:34.185144+0000","last_fullsized":"2024-09-10T22:00:34.185144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:14.992964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664313+0000","last_change":"2024-09-10T22:00:33.197611+0000","last_active":"2024-09-10T22:00:34.664313+0000","last_peered":"2024-09-10T22:00:34.664313+0000","last_clean":"2024-09-10T22:00:34.664313+0000","last_became_active":"2024-09-10T22:00:33.192560+0000","last_became_peered":"2024-09-10T22:00:33.192560+0000","last_unstale":"2024-09-10T22:00:34.664313+0000","last_undegraded":"2024-09-10T22:00:34.664313+0000","last_fullsized":"2024-09-10T22:00:34.664313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:25.562048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664300+0000","last_change":"2024-09-10T22:00:33.213208+0000","last_active":"2024-09-10T22:00:34.664300+0000","last_peered":"2024-09-10T22:00:34.664300+0000","last_clean":"2024-09-10T22:00:34.664300+0000","last_became_active":"2024-09-10T22:00:33.212880+0000","last_became_peered":"2024-09-10T22:00:33.212880+0000","last_unstale":"2024-09-10T22:00:34.664300+0000","last_undegraded":"2024-09-10T22:00:34.664300+0000","last_fullsized":"2024-09-10T22:00:34.664300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:55.992463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665753+0000","last_change":"2024-09-10T22:00:33.215336+0000","last_active":"2024-09-10T22:00:34.665753+0000","last_peered":"2024-09-10T22:00:34.665753+0000","last_clean":"2024-09-10T22:00:34.665753+0000","last_became_active":"2024-09-10T22:00:33.214306+0000","last_became_peered":"2024-09-10T22:00:33.214306+0000","last_unstale":"2024-09-10T22:00:34.665753+0000","last_undegraded":"2024-09-10T22:00:34.665753+0000","last_fullsized":"2024-09-10T22:00:34.665753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:45.979508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187047+0000","last_change":"2024-09-10T22:00:33.208856+0000","last_active":"2024-09-10T22:00:34.187047+0000","last_peered":"2024-09-10T22:00:34.187047+0000","last_clean":"2024-09-10T22:00:34.187047+0000","last_became_active":"2024-09-10T22:00:33.208575+0000","last_became_peered":"2024-09-10T22:00:33.208575+0000","last_unstale":"2024-09-10T22:00:34.187047+0000","last_undegraded":"2024-09-10T22:00:34.187047+0000","last_fullsized":"2024-09-10T22:00:34.187047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.744906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186250+0000","last_change":"2024-09-10T22:00:33.214153+0000","last_active":"2024-09-10T22:00:34.186250+0000","last_peered":"2024-09-10T22:00:34.186250+0000","last_clean":"2024-09-10T22:00:34.186250+0000","last_became_active":"2024-09-10T22:00:33.212422+0000","last_became_peered":"2024-09-10T22:00:33.212422+0000","last_unstale":"2024-09-10T22:00:34.186250+0000","last_undegraded":"2024-09-10T22:00:34.186250+0000","last_fullsized":"2024-09-10T22:00:34.186250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:04.855711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665304+0000","last_change":"2024-09-10T22:00:33.184796+0000","last_active":"2024-09-10T22:00:34.665304+0000","last_peered":"2024-09-10T22:00:34.665304+0000","last_clean":"2024-09-10T22:00:34.665304+0000","last_became_active":"2024-09-10T22:00:33.180143+0000","last_became_peered":"2024-09-10T22:00:33.180143+0000","last_unstale":"2024-09-10T22:00:34.665304+0000","last_undegraded":"2024-09-10T22:00:34.665304+0000","last_fullsized":"2024-09-10T22:00:34.665304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:35.363109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663813+0000","last_change":"2024-09-10T22:00:33.192437+0000","last_active":"2024-09-10T22:00:34.663813+0000","last_peered":"2024-09-10T22:00:34.663813+0000","last_clean":"2024-09-10T22:00:34.663813+0000","last_became_active":"2024-09-10T22:00:33.192220+0000","last_became_peered":"2024-09-10T22:00:33.192220+0000","last_unstale":"2024-09-10T22:00:34.663813+0000","last_undegraded":"2024-09-10T22:00:34.663813+0000","last_fullsized":"2024-09-10T22:00:34.663813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:29.825070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663467+0000","last_change":"2024-09-10T22:00:33.184124+0000","last_active":"2024-09-10T22:00:34.663467+0000","last_peered":"2024-09-10T22:00:34.663467+0000","last_clean":"2024-09-10T22:00:34.663467+0000","last_became_active":"2024-09-10T22:00:33.182588+0000","last_became_peered":"2024-09-10T22:00:33.182588+0000","last_unstale":"2024-09-10T22:00:34.663467+0000","last_undegraded":"2024-09-10T22:00:34.663467+0000","last_fullsized":"2024-09-10T22:00:34.663467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:39.577554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183252+0000","last_change":"2024-09-10T22:00:33.178023+0000","last_active":"2024-09-10T22:00:34.183252+0000","last_peered":"2024-09-10T22:00:34.183252+0000","last_clean":"2024-09-10T22:00:34.183252+0000","last_became_active":"2024-09-10T22:00:33.176108+0000","last_became_peered":"2024-09-10T22:00:33.176108+0000","last_unstale":"2024-09-10T22:00:34.183252+0000","last_undegraded":"2024-09-10T22:00:34.183252+0000","last_fullsized":"2024-09-10T22:00:34.183252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:06.332661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184712+0000","last_change":"2024-09-10T22:00:33.208492+0000","last_active":"2024-09-10T22:00:34.184712+0000","last_peered":"2024-09-10T22:00:34.184712+0000","last_clean":"2024-09-10T22:00:34.184712+0000","last_became_active":"2024-09-10T22:00:33.208232+0000","last_became_peered":"2024-09-10T22:00:33.208232+0000","last_unstale":"2024-09-10T22:00:34.184712+0000","last_undegraded":"2024-09-10T22:00:34.184712+0000","last_fullsized":"2024-09-10T22:00:34.184712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.744906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185766+0000","last_change":"2024-09-10T22:00:33.212565+0000","last_active":"2024-09-10T22:00:34.185766+0000","last_peered":"2024-09-10T22:00:34.185766+0000","last_clean":"2024-09-10T22:00:34.185766+0000","last_became_active":"2024-09-10T22:00:33.210654+0000","last_became_peered":"2024-09-10T22:00:33.210654+0000","last_unstale":"2024-09-10T22:00:34.185766+0000","last_undegraded":"2024-09-10T22:00:34.185766+0000","last_fullsized":"2024-09-10T22:00:34.185766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:11.723619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663811+0000","last_change":"2024-09-10T22:00:33.192188+0000","last_active":"2024-09-10T22:00:34.663811+0000","last_peered":"2024-09-10T22:00:34.663811+0000","last_clean":"2024-09-10T22:00:34.663811+0000","last_became_active":"2024-09-10T22:00:33.191616+0000","last_became_peered":"2024-09-10T22:00:33.191616+0000","last_unstale":"2024-09-10T22:00:34.663811+0000","last_undegraded":"2024-09-10T22:00:34.663811+0000","last_fullsized":"2024-09-10T22:00:34.663811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:03.715049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665186+0000","last_change":"2024-09-10T22:00:33.203524+0000","last_active":"2024-09-10T22:00:34.665186+0000","last_peered":"2024-09-10T22:00:34.665186+0000","last_clean":"2024-09-10T22:00:34.665186+0000","last_became_active":"2024-09-10T22:00:33.202541+0000","last_became_peered":"2024-09-10T22:00:33.202541+0000","last_unstale":"2024-09-10T22:00:34.665186+0000","last_undegraded":"2024-09-10T22:00:34.665186+0000","last_fullsized":"2024-09-10T22:00:34.665186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:55.365445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185716+0000","last_change":"2024-09-10T22:00:33.192918+0000","last_active":"2024-09-10T22:00:34.185716+0000","last_peered":"2024-09-10T22:00:34.185716+0000","last_clean":"2024-09-10T22:00:34.185716+0000","last_became_active":"2024-09-10T22:00:33.192320+0000","last_became_peered":"2024-09-10T22:00:33.192320+0000","last_unstale":"2024-09-10T22:00:34.185716+0000","last_undegraded":"2024-09-10T22:00:34.185716+0000","last_fullsized":"2024-09-10T22:00:34.185716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:54.509264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664157+0000","last_change":"2024-09-10T22:00:33.186507+0000","last_active":"2024-09-10T22:00:34.664157+0000","last_peered":"2024-09-10T22:00:34.664157+0000","last_clean":"2024-09-10T22:00:34.664157+0000","last_became_active":"2024-09-10T22:00:33.186253+0000","last_became_peered":"2024-09-10T22:00:33.186253+0000","last_unstale":"2024-09-10T22:00:34.664157+0000","last_undegraded":"2024-09-10T22:00:34.664157+0000","last_fullsized":"2024-09-10T22:00:34.664157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:29.825070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185357+0000","last_change":"2024-09-10T22:00:33.213873+0000","last_active":"2024-09-10T22:00:34.185357+0000","last_peered":"2024-09-10T22:00:34.185357+0000","last_clean":"2024-09-10T22:00:34.185357+0000","last_became_active":"2024-09-10T22:00:33.211788+0000","last_became_peered":"2024-09-10T22:00:33.211788+0000","last_unstale":"2024-09-10T22:00:34.185357+0000","last_undegraded":"2024-09-10T22:00:34.185357+0000","last_fullsized":"2024-09-10T22:00:34.185357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:36.957181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663634+0000","last_change":"2024-09-10T22:00:33.182956+0000","last_active":"2024-09-10T22:00:34.663634+0000","last_peered":"2024-09-10T22:00:34.663634+0000","last_clean":"2024-09-10T22:00:34.663634+0000","last_became_active":"2024-09-10T22:00:33.182699+0000","last_became_peered":"2024-09-10T22:00:33.182699+0000","last_unstale":"2024-09-10T22:00:34.663634+0000","last_undegraded":"2024-09-10T22:00:34.663634+0000","last_fullsized":"2024-09-10T22:00:34.663634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:54.559113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184784+0000","last_change":"2024-09-10T22:00:33.208620+0000","last_active":"2024-09-10T22:00:34.184784+0000","last_peered":"2024-09-10T22:00:34.184784+0000","last_clean":"2024-09-10T22:00:34.184784+0000","last_became_active":"2024-09-10T22:00:33.208337+0000","last_became_peered":"2024-09-10T22:00:33.208337+0000","last_unstale":"2024-09-10T22:00:34.184784+0000","last_undegraded":"2024-09-10T22:00:34.184784+0000","last_fullsized":"2024-09-10T22:00:34.184784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:22.353564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664536+0000","last_change":"2024-09-10T22:00:33.185808+0000","last_active":"2024-09-10T22:00:34.664536+0000","last_peered":"2024-09-10T22:00:34.664536+0000","last_clean":"2024-09-10T22:00:34.664536+0000","last_became_active":"2024-09-10T22:00:33.185528+0000","last_became_peered":"2024-09-10T22:00:33.185528+0000","last_unstale":"2024-09-10T22:00:34.664536+0000","last_undegraded":"2024-09-10T22:00:34.664536+0000","last_fullsized":"2024-09-10T22:00:34.664536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:04.884964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187164+0000","last_change":"2024-09-10T22:00:33.191781+0000","last_active":"2024-09-10T22:00:34.187164+0000","last_peered":"2024-09-10T22:00:34.187164+0000","last_clean":"2024-09-10T22:00:34.187164+0000","last_became_active":"2024-09-10T22:00:33.191497+0000","last_became_peered":"2024-09-10T22:00:33.191497+0000","last_unstale":"2024-09-10T22:00:34.187164+0000","last_undegraded":"2024-09-10T22:00:34.187164+0000","last_fullsized":"2024-09-10T22:00:34.187164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:22.353564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183465+0000","last_change":"2024-09-10T22:00:33.183377+0000","last_active":"2024-09-10T22:00:34.183465+0000","last_peered":"2024-09-10T22:00:34.183465+0000","last_clean":"2024-09-10T22:00:34.183465+0000","last_became_active":"2024-09-10T22:00:33.181838+0000","last_became_peered":"2024-09-10T22:00:33.181838+0000","last_unstale":"2024-09-10T22:00:34.183465+0000","last_undegraded":"2024-09-10T22:00:34.183465+0000","last_fullsized":"2024-09-10T22:00:34.183465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:58.358390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663596+0000","last_change":"2024-09-10T22:00:33.185739+0000","last_active":"2024-09-10T22:00:34.663596+0000","last_peered":"2024-09-10T22:00:34.663596+0000","last_clean":"2024-09-10T22:00:34.663596+0000","last_became_active":"2024-09-10T22:00:33.185426+0000","last_became_peered":"2024-09-10T22:00:33.185426+0000","last_unstale":"2024-09-10T22:00:34.663596+0000","last_undegraded":"2024-09-10T22:00:34.663596+0000","last_fullsized":"2024-09-10T22:00:34.663596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:46.008480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664376+0000","last_change":"2024-09-10T22:00:33.183396+0000","last_active":"2024-09-10T22:00:34.664376+0000","last_peered":"2024-09-10T22:00:34.664376+0000","last_clean":"2024-09-10T22:00:34.664376+0000","last_became_active":"2024-09-10T22:00:33.181415+0000","last_became_peered":"2024-09-10T22:00:33.181415+0000","last_unstale":"2024-09-10T22:00:34.664376+0000","last_undegraded":"2024-09-10T22:00:34.664376+0000","last_fullsized":"2024-09-10T22:00:34.664376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:58.358390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185581+0000","last_change":"2024-09-10T22:00:33.202653+0000","last_active":"2024-09-10T22:00:34.185581+0000","last_peered":"2024-09-10T22:00:34.185581+0000","last_clean":"2024-09-10T22:00:34.185581+0000","last_became_active":"2024-09-10T22:00:33.202300+0000","last_became_peered":"2024-09-10T22:00:33.202300+0000","last_unstale":"2024-09-10T22:00:34.185581+0000","last_undegraded":"2024-09-10T22:00:34.185581+0000","last_fullsized":"2024-09-10T22:00:34.185581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:29.821720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663157+0000","last_change":"2024-09-10T22:00:33.213581+0000","last_active":"2024-09-10T22:00:34.663157+0000","last_peered":"2024-09-10T22:00:34.663157+0000","last_clean":"2024-09-10T22:00:34.663157+0000","last_became_active":"2024-09-10T22:00:33.213202+0000","last_became_peered":"2024-09-10T22:00:33.213202+0000","last_unstale":"2024-09-10T22:00:34.663157+0000","last_undegraded":"2024-09-10T22:00:34.663157+0000","last_fullsized":"2024-09-10T22:00:34.663157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:03.339805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184712+0000","last_change":"2024-09-10T22:00:33.179958+0000","last_active":"2024-09-10T22:00:34.184712+0000","last_peered":"2024-09-10T22:00:34.184712+0000","last_clean":"2024-09-10T22:00:34.184712+0000","last_became_active":"2024-09-10T22:00:33.179251+0000","last_became_peered":"2024-09-10T22:00:33.179251+0000","last_unstale":"2024-09-10T22:00:34.184712+0000","last_undegraded":"2024-09-10T22:00:34.184712+0000","last_fullsized":"2024-09-10T22:00:34.184712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:20.738322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664541+0000","last_change":"2024-09-10T22:00:33.181036+0000","last_active":"2024-09-10T22:00:34.664541+0000","last_peered":"2024-09-10T22:00:34.664541+0000","last_clean":"2024-09-10T22:00:34.664541+0000","last_became_active":"2024-09-10T22:00:33.179960+0000","last_became_peered":"2024-09-10T22:00:33.179960+0000","last_unstale":"2024-09-10T22:00:34.664541+0000","last_undegraded":"2024-09-10T22:00:34.664541+0000","last_fullsized":"2024-09-10T22:00:34.664541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:36.003731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663924+0000","last_change":"2024-09-10T22:00:32.187576+0000","last_active":"2024-09-10T22:00:34.663924+0000","last_peered":"2024-09-10T22:00:34.663924+0000","last_clean":"2024-09-10T22:00:34.663924+0000","last_became_active":"2024-09-10T22:00:32.187338+0000","last_became_peered":"2024-09-10T22:00:32.187338+0000","last_unstale":"2024-09-10T22:00:34.663924+0000","last_undegraded":"2024-09-10T22:00:34.663924+0000","last_fullsized":"2024-09-10T22:00:34.663924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:39.648880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664576+0000","last_change":"2024-09-10T22:00:33.184050+0000","last_active":"2024-09-10T22:00:34.664576+0000","last_peered":"2024-09-10T22:00:34.664576+0000","last_clean":"2024-09-10T22:00:34.664576+0000","last_became_active":"2024-09-10T22:00:33.183760+0000","last_became_peered":"2024-09-10T22:00:33.183760+0000","last_unstale":"2024-09-10T22:00:34.664576+0000","last_undegraded":"2024-09-10T22:00:34.664576+0000","last_fullsized":"2024-09-10T22:00:34.664576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:08.694915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186593+0000","last_change":"2024-09-10T22:00:32.181714+0000","last_active":"2024-09-10T22:00:34.186593+0000","last_peered":"2024-09-10T22:00:34.186593+0000","last_clean":"2024-09-10T22:00:34.186593+0000","last_became_active":"2024-09-10T22:00:32.180900+0000","last_became_peered":"2024-09-10T22:00:32.180900+0000","last_unstale":"2024-09-10T22:00:34.186593+0000","last_undegraded":"2024-09-10T22:00:34.186593+0000","last_fullsized":"2024-09-10T22:00:34.186593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:19.067912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663685+0000","last_change":"2024-09-10T22:00:32.194626+0000","last_active":"2024-09-10T22:00:34.663685+0000","last_peered":"2024-09-10T22:00:34.663685+0000","last_clean":"2024-09-10T22:00:34.663685+0000","last_became_active":"2024-09-10T22:00:32.194317+0000","last_became_peered":"2024-09-10T22:00:32.194317+0000","last_unstale":"2024-09-10T22:00:34.663685+0000","last_undegraded":"2024-09-10T22:00:34.663685+0000","last_fullsized":"2024-09-10T22:00:34.663685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:36.587242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663212+0000","last_change":"2024-09-10T22:00:33.180723+0000","last_active":"2024-09-10T22:00:34.663212+0000","last_peered":"2024-09-10T22:00:34.663212+0000","last_clean":"2024-09-10T22:00:34.663212+0000","last_became_active":"2024-09-10T22:00:33.180399+0000","last_became_peered":"2024-09-10T22:00:33.180399+0000","last_unstale":"2024-09-10T22:00:34.663212+0000","last_undegraded":"2024-09-10T22:00:34.663212+0000","last_fullsized":"2024-09-10T22:00:34.663212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:24.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184786+0000","last_change":"2024-09-10T22:00:32.184624+0000","last_active":"2024-09-10T22:00:34.184786+0000","last_peered":"2024-09-10T22:00:34.184786+0000","last_clean":"2024-09-10T22:00:34.184786+0000","last_became_active":"2024-09-10T22:00:32.184352+0000","last_became_peered":"2024-09-10T22:00:32.184352+0000","last_unstale":"2024-09-10T22:00:34.184786+0000","last_undegraded":"2024-09-10T22:00:34.184786+0000","last_fullsized":"2024-09-10T22:00:34.184786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:19.067912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183246+0000","last_change":"2024-09-10T22:00:33.178900+0000","last_active":"2024-09-10T22:00:34.183246+0000","last_peered":"2024-09-10T22:00:34.183246+0000","last_clean":"2024-09-10T22:00:34.183246+0000","last_became_active":"2024-09-10T22:00:33.178595+0000","last_became_peered":"2024-09-10T22:00:33.178595+0000","last_unstale":"2024-09-10T22:00:34.183246+0000","last_undegraded":"2024-09-10T22:00:34.183246+0000","last_fullsized":"2024-09-10T22:00:34.183246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:46.008480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187010+0000","last_change":"2024-09-10T22:00:32.193426+0000","last_active":"2024-09-10T22:00:34.187010+0000","last_peered":"2024-09-10T22:00:34.187010+0000","last_clean":"2024-09-10T22:00:34.187010+0000","last_became_active":"2024-09-10T22:00:32.189386+0000","last_became_peered":"2024-09-10T22:00:32.189386+0000","last_unstale":"2024-09-10T22:00:34.187010+0000","last_undegraded":"2024-09-10T22:00:34.187010+0000","last_fullsized":"2024-09-10T22:00:34.187010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:55.143270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185444+0000","last_change":"2024-09-10T22:00:33.213346+0000","last_active":"2024-09-10T22:00:34.185444+0000","last_peered":"2024-09-10T22:00:34.185444+0000","last_clean":"2024-09-10T22:00:34.185444+0000","last_became_active":"2024-09-10T22:00:33.213099+0000","last_became_peered":"2024-09-10T22:00:33.213099+0000","last_unstale":"2024-09-10T22:00:34.185444+0000","last_undegraded":"2024-09-10T22:00:34.185444+0000","last_fullsized":"2024-09-10T22:00:34.185444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:21.770779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185507+0000","last_change":"2024-09-10T22:00:32.195226+0000","last_active":"2024-09-10T22:00:34.185507+0000","last_peered":"2024-09-10T22:00:34.185507+0000","last_clean":"2024-09-10T22:00:34.185507+0000","last_became_active":"2024-09-10T22:00:32.194900+0000","last_became_peered":"2024-09-10T22:00:32.194900+0000","last_unstale":"2024-09-10T22:00:34.185507+0000","last_undegraded":"2024-09-10T22:00:34.185507+0000","last_fullsized":"2024-09-10T22:00:34.185507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:28.818521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663832+0000","last_change":"2024-09-10T22:00:33.213048+0000","last_active":"2024-09-10T22:00:34.663832+0000","last_peered":"2024-09-10T22:00:34.663832+0000","last_clean":"2024-09-10T22:00:34.663832+0000","last_became_active":"2024-09-10T22:00:33.212747+0000","last_became_peered":"2024-09-10T22:00:33.212747+0000","last_unstale":"2024-09-10T22:00:34.663832+0000","last_undegraded":"2024-09-10T22:00:34.663832+0000","last_fullsized":"2024-09-10T22:00:34.663832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:46.008480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183502+0000","last_change":"2024-09-10T22:00:32.194672+0000","last_active":"2024-09-10T22:00:34.183502+0000","last_peered":"2024-09-10T22:00:34.183502+0000","last_clean":"2024-09-10T22:00:34.183502+0000","last_became_active":"2024-09-10T22:00:32.194084+0000","last_became_peered":"2024-09-10T22:00:32.194084+0000","last_unstale":"2024-09-10T22:00:34.183502+0000","last_undegraded":"2024-09-10T22:00:34.183502+0000","last_fullsized":"2024-09-10T22:00:34.183502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:09.395633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665544+0000","last_change":"2024-09-10T22:00:33.176539+0000","last_active":"2024-09-10T22:00:34.665544+0000","last_peered":"2024-09-10T22:00:34.665544+0000","last_clean":"2024-09-10T22:00:34.665544+0000","last_became_active":"2024-09-10T22:00:33.175809+0000","last_became_peered":"2024-09-10T22:00:33.175809+0000","last_unstale":"2024-09-10T22:00:34.665544+0000","last_undegraded":"2024-09-10T22:00:34.665544+0000","last_fullsized":"2024-09-10T22:00:34.665544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:22.046625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185783+0000","last_change":"2024-09-10T22:00:32.187798+0000","last_active":"2024-09-10T22:00:34.185783+0000","last_peered":"2024-09-10T22:00:34.185783+0000","last_clean":"2024-09-10T22:00:34.185783+0000","last_became_active":"2024-09-10T22:00:32.184530+0000","last_became_peered":"2024-09-10T22:00:32.184530+0000","last_unstale":"2024-09-10T22:00:34.185783+0000","last_undegraded":"2024-09-10T22:00:34.185783+0000","last_fullsized":"2024-09-10T22:00:34.185783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:41.444694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664461+0000","last_change":"2024-09-10T22:00:33.189951+0000","last_active":"2024-09-10T22:00:34.664461+0000","last_peered":"2024-09-10T22:00:34.664461+0000","last_clean":"2024-09-10T22:00:34.664461+0000","last_became_active":"2024-09-10T22:00:33.189183+0000","last_became_peered":"2024-09-10T22:00:33.189183+0000","last_unstale":"2024-09-10T22:00:34.664461+0000","last_undegraded":"2024-09-10T22:00:34.664461+0000","last_fullsized":"2024-09-10T22:00:34.664461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:04.884964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664660+0000","last_change":"2024-09-10T22:00:32.171323+0000","last_active":"2024-09-10T22:00:34.664660+0000","last_peered":"2024-09-10T22:00:34.664660+0000","last_clean":"2024-09-10T22:00:34.664660+0000","last_became_active":"2024-09-10T22:00:32.171059+0000","last_became_peered":"2024-09-10T22:00:32.171059+0000","last_unstale":"2024-09-10T22:00:34.664660+0000","last_undegraded":"2024-09-10T22:00:34.664660+0000","last_fullsized":"2024-09-10T22:00:34.664660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:44.404901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663672+0000","last_change":"2024-09-10T22:00:33.196013+0000","last_active":"2024-09-10T22:00:34.663672+0000","last_peered":"2024-09-10T22:00:34.663672+0000","last_clean":"2024-09-10T22:00:34.663672+0000","last_became_active":"2024-09-10T22:00:33.195744+0000","last_became_peered":"2024-09-10T22:00:33.195744+0000","last_unstale":"2024-09-10T22:00:34.663672+0000","last_undegraded":"2024-09-10T22:00:34.663672+0000","last_fullsized":"2024-09-10T22:00:34.663672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:40.305055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185639+0000","last_change":"2024-09-10T22:00:32.196550+0000","last_active":"2024-09-10T22:00:34.185639+0000","last_peered":"2024-09-10T22:00:34.185639+0000","last_clean":"2024-09-10T22:00:34.185639+0000","last_became_active":"2024-09-10T22:00:32.196255+0000","last_became_peered":"2024-09-10T22:00:32.196255+0000","last_unstale":"2024-09-10T22:00:34.185639+0000","last_undegraded":"2024-09-10T22:00:34.185639+0000","last_fullsized":"2024-09-10T22:00:34.185639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:27.356827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665144+0000","last_change":"2024-09-10T22:00:33.197537+0000","last_active":"2024-09-10T22:00:34.665144+0000","last_peered":"2024-09-10T22:00:34.665144+0000","last_clean":"2024-09-10T22:00:34.665144+0000","last_became_active":"2024-09-10T22:00:33.197230+0000","last_became_peered":"2024-09-10T22:00:33.197230+0000","last_unstale":"2024-09-10T22:00:34.665144+0000","last_undegraded":"2024-09-10T22:00:34.665144+0000","last_fullsized":"2024-09-10T22:00:34.665144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:57.404481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184926+0000","last_change":"2024-09-10T22:00:32.187773+0000","last_active":"2024-09-10T22:00:34.184926+0000","last_peered":"2024-09-10T22:00:34.184926+0000","last_clean":"2024-09-10T22:00:34.184926+0000","last_became_active":"2024-09-10T22:00:32.185748+0000","last_became_peered":"2024-09-10T22:00:32.185748+0000","last_unstale":"2024-09-10T22:00:34.184926+0000","last_undegraded":"2024-09-10T22:00:34.184926+0000","last_fullsized":"2024-09-10T22:00:34.184926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:39.648880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184928+0000","last_change":"2024-09-10T22:00:33.180297+0000","last_active":"2024-09-10T22:00:34.184928+0000","last_peered":"2024-09-10T22:00:34.184928+0000","last_clean":"2024-09-10T22:00:34.184928+0000","last_became_active":"2024-09-10T22:00:33.179643+0000","last_became_peered":"2024-09-10T22:00:33.179643+0000","last_unstale":"2024-09-10T22:00:34.184928+0000","last_undegraded":"2024-09-10T22:00:34.184928+0000","last_fullsized":"2024-09-10T22:00:34.184928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:06.332661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.662989+0000","last_change":"2024-09-10T22:00:32.182584+0000","last_active":"2024-09-10T22:00:34.662989+0000","last_peered":"2024-09-10T22:00:34.662989+0000","last_clean":"2024-09-10T22:00:34.662989+0000","last_became_active":"2024-09-10T22:00:32.182289+0000","last_became_peered":"2024-09-10T22:00:32.182289+0000","last_unstale":"2024-09-10T22:00:34.662989+0000","last_undegraded":"2024-09-10T22:00:34.662989+0000","last_fullsized":"2024-09-10T22:00:34.662989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:41.773634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664524+0000","last_change":"2024-09-10T22:00:33.189852+0000","last_active":"2024-09-10T22:00:34.664524+0000","last_peered":"2024-09-10T22:00:34.664524+0000","last_clean":"2024-09-10T22:00:34.664524+0000","last_became_active":"2024-09-10T22:00:33.189492+0000","last_became_peered":"2024-09-10T22:00:33.189492+0000","last_unstale":"2024-09-10T22:00:34.664524+0000","last_undegraded":"2024-09-10T22:00:34.664524+0000","last_fullsized":"2024-09-10T22:00:34.664524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:45.010893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665636+0000","last_change":"2024-09-10T22:00:32.179367+0000","last_active":"2024-09-10T22:00:34.665636+0000","last_peered":"2024-09-10T22:00:34.665636+0000","last_clean":"2024-09-10T22:00:34.665636+0000","last_became_active":"2024-09-10T22:00:32.177804+0000","last_became_peered":"2024-09-10T22:00:32.177804+0000","last_unstale":"2024-09-10T22:00:34.665636+0000","last_undegraded":"2024-09-10T22:00:34.665636+0000","last_fullsized":"2024-09-10T22:00:34.665636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:13.368508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665546+0000","last_change":"2024-09-10T22:00:33.194213+0000","last_active":"2024-09-10T22:00:34.665546+0000","last_peered":"2024-09-10T22:00:34.665546+0000","last_clean":"2024-09-10T22:00:34.665546+0000","last_became_active":"2024-09-10T22:00:33.193708+0000","last_became_peered":"2024-09-10T22:00:33.193708+0000","last_unstale":"2024-09-10T22:00:34.665546+0000","last_undegraded":"2024-09-10T22:00:34.665546+0000","last_fullsized":"2024-09-10T22:00:34.665546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:39.533698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663916+0000","last_change":"2024-09-10T22:00:32.184130+0000","last_active":"2024-09-10T22:00:34.663916+0000","last_peered":"2024-09-10T22:00:34.663916+0000","last_clean":"2024-09-10T22:00:34.663916+0000","last_became_active":"2024-09-10T22:00:32.182524+0000","last_became_peered":"2024-09-10T22:00:32.182524+0000","last_unstale":"2024-09-10T22:00:34.663916+0000","last_undegraded":"2024-09-10T22:00:34.663916+0000","last_fullsized":"2024-09-10T22:00:34.663916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:26.056799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665889+0000","last_change":"2024-09-10T22:00:33.198846+0000","last_active":"2024-09-10T22:00:34.665889+0000","last_peered":"2024-09-10T22:00:34.665889+0000","last_clean":"2024-09-10T22:00:34.665889+0000","last_became_active":"2024-09-10T22:00:33.198166+0000","last_became_peered":"2024-09-10T22:00:33.198166+0000","last_unstale":"2024-09-10T22:00:34.665889+0000","last_undegraded":"2024-09-10T22:00:34.665889+0000","last_fullsized":"2024-09-10T22:00:34.665889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:21.862163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183552+0000","last_change":"2024-09-10T22:00:32.181657+0000","last_active":"2024-09-10T22:00:34.183552+0000","last_peered":"2024-09-10T22:00:34.183552+0000","last_clean":"2024-09-10T22:00:34.183552+0000","last_became_active":"2024-09-10T22:00:32.181365+0000","last_became_peered":"2024-09-10T22:00:32.181365+0000","last_unstale":"2024-09-10T22:00:34.183552+0000","last_undegraded":"2024-09-10T22:00:34.183552+0000","last_fullsized":"2024-09-10T22:00:34.183552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:34.356560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185996+0000","last_change":"2024-09-10T22:00:33.215101+0000","last_active":"2024-09-10T22:00:34.185996+0000","last_peered":"2024-09-10T22:00:34.185996+0000","last_clean":"2024-09-10T22:00:34.185996+0000","last_became_active":"2024-09-10T22:00:33.214294+0000","last_became_peered":"2024-09-10T22:00:33.214294+0000","last_unstale":"2024-09-10T22:00:34.185996+0000","last_undegraded":"2024-09-10T22:00:34.185996+0000","last_fullsized":"2024-09-10T22:00:34.185996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:59.470008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663655+0000","last_change":"2024-09-10T22:00:32.195097+0000","last_active":"2024-09-10T22:00:34.663655+0000","last_peered":"2024-09-10T22:00:34.663655+0000","last_clean":"2024-09-10T22:00:34.663655+0000","last_became_active":"2024-09-10T22:00:32.194103+0000","last_became_peered":"2024-09-10T22:00:32.194103+0000","last_unstale":"2024-09-10T22:00:34.663655+0000","last_undegraded":"2024-09-10T22:00:34.663655+0000","last_fullsized":"2024-09-10T22:00:34.663655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:57.181450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664931+0000","last_change":"2024-09-10T22:00:33.212536+0000","last_active":"2024-09-10T22:00:34.664931+0000","last_peered":"2024-09-10T22:00:34.664931+0000","last_clean":"2024-09-10T22:00:34.664931+0000","last_became_active":"2024-09-10T22:00:33.212204+0000","last_became_peered":"2024-09-10T22:00:33.212204+0000","last_unstale":"2024-09-10T22:00:34.664931+0000","last_undegraded":"2024-09-10T22:00:34.664931+0000","last_fullsized":"2024-09-10T22:00:34.664931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:41.803895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664227+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:00:34.664227+0000","last_peered":"2024-09-10T22:00:34.664227+0000","last_clean":"2024-09-10T22:00:34.664227+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:00:34.664227+0000","last_undegraded":"2024-09-10T22:00:34.664227+0000","last_fullsized":"2024-09-10T22:00:34.664227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185132+0000","last_change":"2024-09-10T22:00:32.195314+0000","last_active":"2024-09-10T22:00:34.185132+0000","last_peered":"2024-09-10T22:00:34.185132+0000","last_clean":"2024-09-10T22:00:34.185132+0000","last_became_active":"2024-09-10T22:00:32.194960+0000","last_became_peered":"2024-09-10T22:00:32.194960+0000","last_unstale":"2024-09-10T22:00:34.185132+0000","last_undegraded":"2024-09-10T22:00:34.185132+0000","last_fullsized":"2024-09-10T22:00:34.185132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:01.067800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663524+0000","last_change":"2024-09-10T22:00:33.208452+0000","last_active":"2024-09-10T22:00:34.663524+0000","last_peered":"2024-09-10T22:00:34.663524+0000","last_clean":"2024-09-10T22:00:34.663524+0000","last_became_active":"2024-09-10T22:00:33.208146+0000","last_became_peered":"2024-09-10T22:00:33.208146+0000","last_unstale":"2024-09-10T22:00:34.663524+0000","last_undegraded":"2024-09-10T22:00:34.663524+0000","last_fullsized":"2024-09-10T22:00:34.663524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:40.305055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664710+0000","last_change":"2024-09-10T22:00:32.192173+0000","last_active":"2024-09-10T22:00:34.664710+0000","last_peered":"2024-09-10T22:00:34.664710+0000","last_clean":"2024-09-10T22:00:34.664710+0000","last_became_active":"2024-09-10T22:00:32.191792+0000","last_became_peered":"2024-09-10T22:00:32.191792+0000","last_unstale":"2024-09-10T22:00:34.664710+0000","last_undegraded":"2024-09-10T22:00:34.664710+0000","last_fullsized":"2024-09-10T22:00:34.664710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:34.356560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665272+0000","last_change":"2024-09-10T22:00:33.174576+0000","last_active":"2024-09-10T22:00:34.665272+0000","last_peered":"2024-09-10T22:00:34.665272+0000","last_clean":"2024-09-10T22:00:34.665272+0000","last_became_active":"2024-09-10T22:00:33.174232+0000","last_became_peered":"2024-09-10T22:00:33.174232+0000","last_unstale":"2024-09-10T22:00:34.665272+0000","last_undegraded":"2024-09-10T22:00:34.665272+0000","last_fullsized":"2024-09-10T22:00:34.665272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:55.992463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663264+0000","last_change":"2024-09-10T22:00:32.178337+0000","last_active":"2024-09-10T22:00:34.663264+0000","last_peered":"2024-09-10T22:00:34.663264+0000","last_clean":"2024-09-10T22:00:34.663264+0000","last_became_active":"2024-09-10T22:00:32.178053+0000","last_became_peered":"2024-09-10T22:00:32.178053+0000","last_unstale":"2024-09-10T22:00:34.663264+0000","last_undegraded":"2024-09-10T22:00:34.663264+0000","last_fullsized":"2024-09-10T22:00:34.663264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:26.949229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183185+0000","last_change":"2024-09-10T22:00:33.184308+0000","last_active":"2024-09-10T22:00:34.183185+0000","last_peered":"2024-09-10T22:00:34.183185+0000","last_clean":"2024-09-10T22:00:34.183185+0000","last_became_active":"2024-09-10T22:00:33.184050+0000","last_became_peered":"2024-09-10T22:00:33.184050+0000","last_unstale":"2024-09-10T22:00:34.183185+0000","last_undegraded":"2024-09-10T22:00:34.183185+0000","last_fullsized":"2024-09-10T22:00:34.183185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:40.305055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185742+0000","last_change":"2024-09-10T22:00:32.195663+0000","last_active":"2024-09-10T22:00:34.185742+0000","last_peered":"2024-09-10T22:00:34.185742+0000","last_clean":"2024-09-10T22:00:34.185742+0000","last_became_active":"2024-09-10T22:00:32.195368+0000","last_became_peered":"2024-09-10T22:00:32.195368+0000","last_unstale":"2024-09-10T22:00:34.185742+0000","last_undegraded":"2024-09-10T22:00:34.185742+0000","last_fullsized":"2024-09-10T22:00:34.185742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:56.084833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664779+0000","last_change":"2024-09-10T22:00:33.186193+0000","last_active":"2024-09-10T22:00:34.664779+0000","last_peered":"2024-09-10T22:00:34.664779+0000","last_clean":"2024-09-10T22:00:34.664779+0000","last_became_active":"2024-09-10T22:00:33.180505+0000","last_became_peered":"2024-09-10T22:00:33.180505+0000","last_unstale":"2024-09-10T22:00:34.664779+0000","last_undegraded":"2024-09-10T22:00:34.664779+0000","last_fullsized":"2024-09-10T22:00:34.664779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:10.402182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183297+0000","last_change":"2024-09-10T22:00:32.184540+0000","last_active":"2024-09-10T22:00:34.183297+0000","last_peered":"2024-09-10T22:00:34.183297+0000","last_clean":"2024-09-10T22:00:34.183297+0000","last_became_active":"2024-09-10T22:00:32.184213+0000","last_became_peered":"2024-09-10T22:00:32.184213+0000","last_unstale":"2024-09-10T22:00:34.183297+0000","last_undegraded":"2024-09-10T22:00:34.183297+0000","last_fullsized":"2024-09-10T22:00:34.183297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:21.040076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186341+0000","last_change":"2024-09-10T22:00:33.212897+0000","last_active":"2024-09-10T22:00:34.186341+0000","last_peered":"2024-09-10T22:00:34.186341+0000","last_clean":"2024-09-10T22:00:34.186341+0000","last_became_active":"2024-09-10T22:00:33.212668+0000","last_became_peered":"2024-09-10T22:00:33.212668+0000","last_unstale":"2024-09-10T22:00:34.186341+0000","last_undegraded":"2024-09-10T22:00:34.186341+0000","last_fullsized":"2024-09-10T22:00:34.186341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:47.684127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665052+0000","last_change":"2024-09-10T22:00:32.193491+0000","last_active":"2024-09-10T22:00:34.665052+0000","last_peered":"2024-09-10T22:00:34.665052+0000","last_clean":"2024-09-10T22:00:34.665052+0000","last_became_active":"2024-09-10T22:00:32.193147+0000","last_became_peered":"2024-09-10T22:00:32.193147+0000","last_unstale":"2024-09-10T22:00:34.665052+0000","last_undegraded":"2024-09-10T22:00:34.665052+0000","last_fullsized":"2024-09-10T22:00:34.665052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:51.181211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183356+0000","last_change":"2024-09-10T22:00:33.189064+0000","last_active":"2024-09-10T22:00:34.183356+0000","last_peered":"2024-09-10T22:00:34.183356+0000","last_clean":"2024-09-10T22:00:34.183356+0000","last_became_active":"2024-09-10T22:00:33.188733+0000","last_became_peered":"2024-09-10T22:00:33.188733+0000","last_unstale":"2024-09-10T22:00:34.183356+0000","last_undegraded":"2024-09-10T22:00:34.183356+0000","last_fullsized":"2024-09-10T22:00:34.183356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:29.825070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183468+0000","last_change":"2024-09-10T22:00:32.186782+0000","last_active":"2024-09-10T22:00:34.183468+0000","last_peered":"2024-09-10T22:00:34.183468+0000","last_clean":"2024-09-10T22:00:34.183468+0000","last_became_active":"2024-09-10T22:00:32.186028+0000","last_became_peered":"2024-09-10T22:00:32.186028+0000","last_unstale":"2024-09-10T22:00:34.183468+0000","last_undegraded":"2024-09-10T22:00:34.183468+0000","last_fullsized":"2024-09-10T22:00:34.183468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:34.997182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664559+0000","last_change":"2024-09-10T22:00:33.201547+0000","last_active":"2024-09-10T22:00:34.664559+0000","last_peered":"2024-09-10T22:00:34.664559+0000","last_clean":"2024-09-10T22:00:34.664559+0000","last_became_active":"2024-09-10T22:00:33.201302+0000","last_became_peered":"2024-09-10T22:00:33.201302+0000","last_unstale":"2024-09-10T22:00:34.664559+0000","last_undegraded":"2024-09-10T22:00:34.664559+0000","last_fullsized":"2024-09-10T22:00:34.664559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:45.010893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187107+0000","last_change":"2024-09-10T22:00:32.196354+0000","last_active":"2024-09-10T22:00:34.187107+0000","last_peered":"2024-09-10T22:00:34.187107+0000","last_clean":"2024-09-10T22:00:34.187107+0000","last_became_active":"2024-09-10T22:00:32.196053+0000","last_became_peered":"2024-09-10T22:00:32.196053+0000","last_unstale":"2024-09-10T22:00:34.187107+0000","last_undegraded":"2024-09-10T22:00:34.187107+0000","last_fullsized":"2024-09-10T22:00:34.187107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:37.446124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663935+0000","last_change":"2024-09-10T22:00:33.176478+0000","last_active":"2024-09-10T22:00:34.663935+0000","last_peered":"2024-09-10T22:00:34.663935+0000","last_clean":"2024-09-10T22:00:34.663935+0000","last_became_active":"2024-09-10T22:00:33.176211+0000","last_became_peered":"2024-09-10T22:00:33.176211+0000","last_unstale":"2024-09-10T22:00:34.663935+0000","last_undegraded":"2024-09-10T22:00:34.663935+0000","last_fullsized":"2024-09-10T22:00:34.663935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:54.559113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664088+0000","last_change":"2024-09-10T22:00:32.189994+0000","last_active":"2024-09-10T22:00:34.664088+0000","last_peered":"2024-09-10T22:00:34.664088+0000","last_clean":"2024-09-10T22:00:34.664088+0000","last_became_active":"2024-09-10T22:00:32.189654+0000","last_became_peered":"2024-09-10T22:00:32.189654+0000","last_unstale":"2024-09-10T22:00:34.664088+0000","last_undegraded":"2024-09-10T22:00:34.664088+0000","last_fullsized":"2024-09-10T22:00:34.664088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:34.987281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664635+0000","last_change":"2024-09-10T22:00:33.183489+0000","last_active":"2024-09-10T22:00:34.664635+0000","last_peered":"2024-09-10T22:00:34.664635+0000","last_clean":"2024-09-10T22:00:34.664635+0000","last_became_active":"2024-09-10T22:00:33.183217+0000","last_became_peered":"2024-09-10T22:00:33.183217+0000","last_unstale":"2024-09-10T22:00:34.664635+0000","last_undegraded":"2024-09-10T22:00:34.664635+0000","last_fullsized":"2024-09-10T22:00:34.664635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:35.993830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183297+0000","last_change":"2024-09-10T22:00:32.178130+0000","last_active":"2024-09-10T22:00:34.183297+0000","last_peered":"2024-09-10T22:00:34.183297+0000","last_clean":"2024-09-10T22:00:34.183297+0000","last_became_active":"2024-09-10T22:00:32.177879+0000","last_became_peered":"2024-09-10T22:00:32.177879+0000","last_unstale":"2024-09-10T22:00:34.183297+0000","last_undegraded":"2024-09-10T22:00:34.183297+0000","last_fullsized":"2024-09-10T22:00:34.183297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:34.987281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664090+0000","last_change":"2024-09-10T22:00:33.179961+0000","last_active":"2024-09-10T22:00:34.664090+0000","last_peered":"2024-09-10T22:00:34.664090+0000","last_clean":"2024-09-10T22:00:34.664090+0000","last_became_active":"2024-09-10T22:00:33.179700+0000","last_became_peered":"2024-09-10T22:00:33.179700+0000","last_unstale":"2024-09-10T22:00:34.664090+0000","last_undegraded":"2024-09-10T22:00:34.664090+0000","last_fullsized":"2024-09-10T22:00:34.664090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:08.694915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183241+0000","last_change":"2024-09-10T22:00:32.180382+0000","last_active":"2024-09-10T22:00:34.183241+0000","last_peered":"2024-09-10T22:00:34.183241+0000","last_clean":"2024-09-10T22:00:34.183241+0000","last_became_active":"2024-09-10T22:00:32.180075+0000","last_became_peered":"2024-09-10T22:00:32.180075+0000","last_unstale":"2024-09-10T22:00:34.183241+0000","last_undegraded":"2024-09-10T22:00:34.183241+0000","last_fullsized":"2024-09-10T22:00:34.183241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:54.640505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664041+0000","last_change":"2024-09-10T22:00:33.180424+0000","last_active":"2024-09-10T22:00:34.664041+0000","last_peered":"2024-09-10T22:00:34.664041+0000","last_clean":"2024-09-10T22:00:34.664041+0000","last_became_active":"2024-09-10T22:00:33.180181+0000","last_became_peered":"2024-09-10T22:00:33.180181+0000","last_unstale":"2024-09-10T22:00:34.664041+0000","last_undegraded":"2024-09-10T22:00:34.664041+0000","last_fullsized":"2024-09-10T22:00:34.664041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:58.358390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665152+0000","last_change":"2024-09-10T22:00:32.191652+0000","last_active":"2024-09-10T22:00:34.665152+0000","last_peered":"2024-09-10T22:00:34.665152+0000","last_clean":"2024-09-10T22:00:34.665152+0000","last_became_active":"2024-09-10T22:00:32.191353+0000","last_became_peered":"2024-09-10T22:00:32.191353+0000","last_unstale":"2024-09-10T22:00:34.665152+0000","last_undegraded":"2024-09-10T22:00:34.665152+0000","last_fullsized":"2024-09-10T22:00:34.665152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:21.040076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184746+0000","last_change":"2024-09-10T22:00:33.184285+0000","last_active":"2024-09-10T22:00:34.184746+0000","last_peered":"2024-09-10T22:00:34.184746+0000","last_clean":"2024-09-10T22:00:34.184746+0000","last_became_active":"2024-09-10T22:00:33.183977+0000","last_became_peered":"2024-09-10T22:00:33.183977+0000","last_unstale":"2024-09-10T22:00:34.184746+0000","last_undegraded":"2024-09-10T22:00:34.184746+0000","last_fullsized":"2024-09-10T22:00:34.184746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:30.181250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186284+0000","last_change":"2024-09-10T22:00:33.207897+0000","last_active":"2024-09-10T22:00:34.186284+0000","last_peered":"2024-09-10T22:00:34.186284+0000","last_clean":"2024-09-10T22:00:34.186284+0000","last_became_active":"2024-09-10T22:00:33.207639+0000","last_became_peered":"2024-09-10T22:00:33.207639+0000","last_unstale":"2024-09-10T22:00:34.186284+0000","last_undegraded":"2024-09-10T22:00:34.186284+0000","last_fullsized":"2024-09-10T22:00:34.186284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:47.952904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664351+0000","last_change":"2024-09-10T22:00:32.199875+0000","last_active":"2024-09-10T22:00:34.664351+0000","last_peered":"2024-09-10T22:00:34.664351+0000","last_clean":"2024-09-10T22:00:34.664351+0000","last_became_active":"2024-09-10T22:00:32.199625+0000","last_became_peered":"2024-09-10T22:00:32.199625+0000","last_unstale":"2024-09-10T22:00:34.664351+0000","last_undegraded":"2024-09-10T22:00:34.664351+0000","last_fullsized":"2024-09-10T22:00:34.664351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:09.395633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185863+0000","last_change":"2024-09-10T22:00:33.207063+0000","last_active":"2024-09-10T22:00:34.185863+0000","last_peered":"2024-09-10T22:00:34.185863+0000","last_clean":"2024-09-10T22:00:34.185863+0000","last_became_active":"2024-09-10T22:00:33.206867+0000","last_became_peered":"2024-09-10T22:00:33.206867+0000","last_unstale":"2024-09-10T22:00:34.185863+0000","last_undegraded":"2024-09-10T22:00:34.185863+0000","last_fullsized":"2024-09-10T22:00:34.185863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:44.640052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664917+0000","last_change":"2024-09-10T22:00:32.196714+0000","last_active":"2024-09-10T22:00:34.664917+0000","last_peered":"2024-09-10T22:00:34.664917+0000","last_clean":"2024-09-10T22:00:34.664917+0000","last_became_active":"2024-09-10T22:00:32.196446+0000","last_became_peered":"2024-09-10T22:00:32.196446+0000","last_unstale":"2024-09-10T22:00:34.664917+0000","last_undegraded":"2024-09-10T22:00:34.664917+0000","last_fullsized":"2024-09-10T22:00:34.664917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:39.201788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663624+0000","last_change":"2024-09-10T22:00:33.215295+0000","last_active":"2024-09-10T22:00:34.663624+0000","last_peered":"2024-09-10T22:00:34.663624+0000","last_clean":"2024-09-10T22:00:34.663624+0000","last_became_active":"2024-09-10T22:00:33.215041+0000","last_became_peered":"2024-09-10T22:00:33.215041+0000","last_unstale":"2024-09-10T22:00:34.663624+0000","last_undegraded":"2024-09-10T22:00:34.663624+0000","last_fullsized":"2024-09-10T22:00:34.663624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:24.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664333+0000","last_change":"2024-09-10T22:00:32.180489+0000","last_active":"2024-09-10T22:00:34.664333+0000","last_peered":"2024-09-10T22:00:34.664333+0000","last_clean":"2024-09-10T22:00:34.664333+0000","last_became_active":"2024-09-10T22:00:32.180177+0000","last_became_peered":"2024-09-10T22:00:32.180177+0000","last_unstale":"2024-09-10T22:00:34.664333+0000","last_undegraded":"2024-09-10T22:00:34.664333+0000","last_fullsized":"2024-09-10T22:00:34.664333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:57.175682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664468+0000","last_change":"2024-09-10T22:00:33.192548+0000","last_active":"2024-09-10T22:00:34.664468+0000","last_peered":"2024-09-10T22:00:34.664468+0000","last_clean":"2024-09-10T22:00:34.664468+0000","last_became_active":"2024-09-10T22:00:33.192272+0000","last_became_peered":"2024-09-10T22:00:33.192272+0000","last_unstale":"2024-09-10T22:00:34.664468+0000","last_undegraded":"2024-09-10T22:00:34.664468+0000","last_fullsized":"2024-09-10T22:00:34.664468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:25.562048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663908+0000","last_change":"2024-09-10T22:00:32.182534+0000","last_active":"2024-09-10T22:00:34.663908+0000","last_peered":"2024-09-10T22:00:34.663908+0000","last_clean":"2024-09-10T22:00:34.663908+0000","last_became_active":"2024-09-10T22:00:32.182286+0000","last_became_peered":"2024-09-10T22:00:32.182286+0000","last_unstale":"2024-09-10T22:00:34.663908+0000","last_undegraded":"2024-09-10T22:00:34.663908+0000","last_fullsized":"2024-09-10T22:00:34.663908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:56.418214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":667234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":225280,"data_stored":76374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":76410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":827392,"data_stored":667290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:00:41.947 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:00:42.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.097+0000 7f8171800640 1 -- 172.21.15.66:0/2386032566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f816c15a310 msgr2=0x7f816c0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.097+0000 7f8171800640 1 --2- 172.21.15.66:0/2386032566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f816c15a310 0x7f816c0c48b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f815c00aa10 tx=0x7f815c01dba0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.097+0000 7f8171800640 1 -- 172.21.15.66:0/2386032566 shutdown_connections 2024-09-10T22:00:42.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.097+0000 7f8171800640 1 --2- 172.21.15.66:0/2386032566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f816c0aeec0 0x7f816c0c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.097+0000 7f8171800640 1 --2- 172.21.15.66:0/2386032566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f816c15a310 0x7f816c0c48b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.097+0000 7f8171800640 1 -- 172.21.15.66:0/2386032566 >> 172.21.15.66:0/2386032566 conn(0x7f816c0cfa30 msgr2=0x7f816c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:42.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.097+0000 7f8171800640 1 -- 172.21.15.66:0/2386032566 shutdown_connections 2024-09-10T22:00:42.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8171800640 1 -- 172.21.15.66:0/2386032566 wait complete. 2024-09-10T22:00:42.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8171800640 1 Processor -- start 2024-09-10T22:00:42.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8171800640 1 -- start start 2024-09-10T22:00:42.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8171800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c0aeec0 0x7f816c14d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:42.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8171800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f816c15a310 0x7f816c13bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:42.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8171800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f816c0c8cd0 0x7f816c0c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:42.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8171800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f816c167300 con 0x7f816c15a310 2024-09-10T22:00:42.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8171800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f816c15ec90 con 0x7f816c0aeec0 2024-09-10T22:00:42.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8171800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f816c167480 con 0x7f816c0c8cd0 2024-09-10T22:00:42.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8163e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f816c15a310 0x7f816c13bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:42.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8163e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f816c15a310 0x7f816c13bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60226/0 (socket says 172.21.15.66:60226) 2024-09-10T22:00:42.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8163e00640 1 -- 172.21.15.66:0/1145587869 learned_addr learned my addr 172.21.15.66:0/1145587869 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:42.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f816b400640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f816c0c8cd0 0x7f816c0c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:42.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f816aa00640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c0aeec0 0x7f816c14d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:42.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8163e00640 1 -- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f816c0c8cd0 msgr2=0x7f816c0c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8163e00640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f816c0c8cd0 0x7f816c0c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8163e00640 1 -- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c0aeec0 msgr2=0x7f816c14d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8163e00640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c0aeec0 0x7f816c14d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8163e00640 1 -- 172.21.15.66:0/1145587869 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c14e290 con 0x7f816c15a310 2024-09-10T22:00:42.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8163e00640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f816c15a310 0x7f816c13bfc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f815400c4b0 tx=0x7f815400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:42.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8162a00640 1 -- 172.21.15.66:0/1145587869 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81540052d0 con 0x7f816c15a310 2024-09-10T22:00:42.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8162a00640 1 -- 172.21.15.66:0/1145587869 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8154005470 con 0x7f816c15a310 2024-09-10T22:00:42.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8162a00640 1 -- 172.21.15.66:0/1145587869 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154012870 con 0x7f816c15a310 2024-09-10T22:00:42.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.101+0000 7f8171800640 1 -- 172.21.15.66:0/1145587869 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c139cb0 con 0x7f816c15a310 2024-09-10T22:00:42.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.105+0000 7f8171800640 1 -- 172.21.15.66:0/1145587869 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c14e570 con 0x7f816c15a310 2024-09-10T22:00:42.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.105+0000 7f8171800640 1 -- 172.21.15.66:0/1145587869 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816c15df60 con 0x7f816c15a310 2024-09-10T22:00:42.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.105+0000 7f8162a00640 1 -- 172.21.15.66:0/1145587869 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8154003600 con 0x7f816c15a310 2024-09-10T22:00:42.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.105+0000 7f8162a00640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8148072610 0x7f8148074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:42.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.105+0000 7f8162a00640 1 -- 172.21.15.66:0/1145587869 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8154092c20 con 0x7f816c15a310 2024-09-10T22:00:42.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.105+0000 7f816aa00640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8148072610 0x7f8148074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:42.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.109+0000 7f816aa00640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8148072610 0x7f8148074ad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f815c0b0000 tx=0x7f815c01c3e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.109+0000 7f8162a00640 1 -- 172.21.15.66:0/1145587869 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f815405dca0 con 0x7f816c15a310 2024-09-10T22:00:42.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.281+0000 7f8171800640 1 -- 172.21.15.66:0/1145587869 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f816c14d1a0 con 0x7f8148072610 2024-09-10T22:00:42.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.285+0000 7f8162a00640 1 -- 172.21.15.66:0/1145587869 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7f815c0fa830 con 0x7f8148072610 2024-09-10T22:00:42.295 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:42.297 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:00:42.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.289+0000 7f8171800640 1 -- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8148072610 msgr2=0x7f8148074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.289+0000 7f8171800640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8148072610 0x7f8148074ad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f815c0b0000 tx=0x7f815c01c3e0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.289+0000 7f8171800640 1 -- 172.21.15.66:0/1145587869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f816c15a310 msgr2=0x7f816c13bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.289+0000 7f8171800640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f816c15a310 0x7f816c13bfc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f815400c4b0 tx=0x7f815400c970 comp rx=0 tx=0).stop 2024-09-10T22:00:42.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.289+0000 7f8171800640 1 -- 172.21.15.66:0/1145587869 shutdown_connections 2024-09-10T22:00:42.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.289+0000 7f8171800640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8148072610 0x7f8148074ad0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.289+0000 7f8171800640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f816c0c8cd0 0x7f816c0c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.289+0000 7f8171800640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f816c15a310 0x7f816c13bfc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.289+0000 7f8171800640 1 --2- 172.21.15.66:0/1145587869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c0aeec0 0x7f816c14d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.289+0000 7f8171800640 1 -- 172.21.15.66:0/1145587869 >> 172.21.15.66:0/1145587869 conn(0x7f816c0cfa30 msgr2=0x7f816c0ca110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:42.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.289+0000 7f8171800640 1 -- 172.21.15.66:0/1145587869 shutdown_connections 2024-09-10T22:00:42.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.289+0000 7f8171800640 1 -- 172.21.15.66:0/1145587869 wait complete. 2024-09-10T22:00:42.317 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-10T22:00:40.311794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235848,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738872,"statfs":{"total":773094113280,"available":772852604928,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001675"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665723+0000","last_change":"2024-09-10T22:00:33.198521+0000","last_active":"2024-09-10T22:00:34.665723+0000","last_peered":"2024-09-10T22:00:34.665723+0000","last_clean":"2024-09-10T22:00:34.665723+0000","last_became_active":"2024-09-10T22:00:33.197869+0000","last_became_peered":"2024-09-10T22:00:33.197869+0000","last_unstale":"2024-09-10T22:00:34.665723+0000","last_undegraded":"2024-09-10T22:00:34.665723+0000","last_fullsized":"2024-09-10T22:00:34.665723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:48.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183587+0000","last_change":"2024-09-10T22:00:33.182512+0000","last_active":"2024-09-10T22:00:34.183587+0000","last_peered":"2024-09-10T22:00:34.183587+0000","last_clean":"2024-09-10T22:00:34.183587+0000","last_became_active":"2024-09-10T22:00:33.182230+0000","last_became_peered":"2024-09-10T22:00:33.182230+0000","last_unstale":"2024-09-10T22:00:34.183587+0000","last_undegraded":"2024-09-10T22:00:34.183587+0000","last_fullsized":"2024-09-10T22:00:34.183587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:08.694915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663553+0000","last_change":"2024-09-10T22:00:33.182382+0000","last_active":"2024-09-10T22:00:34.663553+0000","last_peered":"2024-09-10T22:00:34.663553+0000","last_clean":"2024-09-10T22:00:34.663553+0000","last_became_active":"2024-09-10T22:00:33.182115+0000","last_became_peered":"2024-09-10T22:00:33.182115+0000","last_unstale":"2024-09-10T22:00:34.663553+0000","last_undegraded":"2024-09-10T22:00:34.663553+0000","last_fullsized":"2024-09-10T22:00:34.663553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:34.220172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185630+0000","last_change":"2024-09-10T22:00:33.215454+0000","last_active":"2024-09-10T22:00:34.185630+0000","last_peered":"2024-09-10T22:00:34.185630+0000","last_clean":"2024-09-10T22:00:34.185630+0000","last_became_active":"2024-09-10T22:00:33.214713+0000","last_became_peered":"2024-09-10T22:00:33.214713+0000","last_unstale":"2024-09-10T22:00:34.185630+0000","last_undegraded":"2024-09-10T22:00:34.185630+0000","last_fullsized":"2024-09-10T22:00:34.185630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:37.559387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665058+0000","last_change":"2024-09-10T22:00:33.201925+0000","last_active":"2024-09-10T22:00:34.665058+0000","last_peered":"2024-09-10T22:00:34.665058+0000","last_clean":"2024-09-10T22:00:34.665058+0000","last_became_active":"2024-09-10T22:00:33.197799+0000","last_became_peered":"2024-09-10T22:00:33.197799+0000","last_unstale":"2024-09-10T22:00:34.665058+0000","last_undegraded":"2024-09-10T22:00:34.665058+0000","last_fullsized":"2024-09-10T22:00:34.665058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:37.217143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185144+0000","last_change":"2024-09-10T22:00:33.211402+0000","last_active":"2024-09-10T22:00:34.185144+0000","last_peered":"2024-09-10T22:00:34.185144+0000","last_clean":"2024-09-10T22:00:34.185144+0000","last_became_active":"2024-09-10T22:00:33.210608+0000","last_became_peered":"2024-09-10T22:00:33.210608+0000","last_unstale":"2024-09-10T22:00:34.185144+0000","last_undegraded":"2024-09-10T22:00:34.185144+0000","last_fullsized":"2024-09-10T22:00:34.185144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:14.992964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664313+0000","last_change":"2024-09-10T22:00:33.197611+0000","last_active":"2024-09-10T22:00:34.664313+0000","last_peered":"2024-09-10T22:00:34.664313+0000","last_clean":"2024-09-10T22:00:34.664313+0000","last_became_active":"2024-09-10T22:00:33.192560+0000","last_became_peered":"2024-09-10T22:00:33.192560+0000","last_unstale":"2024-09-10T22:00:34.664313+0000","last_undegraded":"2024-09-10T22:00:34.664313+0000","last_fullsized":"2024-09-10T22:00:34.664313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:25.562048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664300+0000","last_change":"2024-09-10T22:00:33.213208+0000","last_active":"2024-09-10T22:00:34.664300+0000","last_peered":"2024-09-10T22:00:34.664300+0000","last_clean":"2024-09-10T22:00:34.664300+0000","last_became_active":"2024-09-10T22:00:33.212880+0000","last_became_peered":"2024-09-10T22:00:33.212880+0000","last_unstale":"2024-09-10T22:00:34.664300+0000","last_undegraded":"2024-09-10T22:00:34.664300+0000","last_fullsized":"2024-09-10T22:00:34.664300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:55.992463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665753+0000","last_change":"2024-09-10T22:00:33.215336+0000","last_active":"2024-09-10T22:00:34.665753+0000","last_peered":"2024-09-10T22:00:34.665753+0000","last_clean":"2024-09-10T22:00:34.665753+0000","last_became_active":"2024-09-10T22:00:33.214306+0000","last_became_peered":"2024-09-10T22:00:33.214306+0000","last_unstale":"2024-09-10T22:00:34.665753+0000","last_undegraded":"2024-09-10T22:00:34.665753+0000","last_fullsized":"2024-09-10T22:00:34.665753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:45.979508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187047+0000","last_change":"2024-09-10T22:00:33.208856+0000","last_active":"2024-09-10T22:00:34.187047+0000","last_peered":"2024-09-10T22:00:34.187047+0000","last_clean":"2024-09-10T22:00:34.187047+0000","last_became_active":"2024-09-10T22:00:33.208575+0000","last_became_peered":"2024-09-10T22:00:33.208575+0000","last_unstale":"2024-09-10T22:00:34.187047+0000","last_undegraded":"2024-09-10T22:00:34.187047+0000","last_fullsized":"2024-09-10T22:00:34.187047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.744906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186250+0000","last_change":"2024-09-10T22:00:33.214153+0000","last_active":"2024-09-10T22:00:34.186250+0000","last_peered":"2024-09-10T22:00:34.186250+0000","last_clean":"2024-09-10T22:00:34.186250+0000","last_became_active":"2024-09-10T22:00:33.212422+0000","last_became_peered":"2024-09-10T22:00:33.212422+0000","last_unstale":"2024-09-10T22:00:34.186250+0000","last_undegraded":"2024-09-10T22:00:34.186250+0000","last_fullsized":"2024-09-10T22:00:34.186250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:04.855711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665304+0000","last_change":"2024-09-10T22:00:33.184796+0000","last_active":"2024-09-10T22:00:34.665304+0000","last_peered":"2024-09-10T22:00:34.665304+0000","last_clean":"2024-09-10T22:00:34.665304+0000","last_became_active":"2024-09-10T22:00:33.180143+0000","last_became_peered":"2024-09-10T22:00:33.180143+0000","last_unstale":"2024-09-10T22:00:34.665304+0000","last_undegraded":"2024-09-10T22:00:34.665304+0000","last_fullsized":"2024-09-10T22:00:34.665304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:35.363109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663813+0000","last_change":"2024-09-10T22:00:33.192437+0000","last_active":"2024-09-10T22:00:34.663813+0000","last_peered":"2024-09-10T22:00:34.663813+0000","last_clean":"2024-09-10T22:00:34.663813+0000","last_became_active":"2024-09-10T22:00:33.192220+0000","last_became_peered":"2024-09-10T22:00:33.192220+0000","last_unstale":"2024-09-10T22:00:34.663813+0000","last_undegraded":"2024-09-10T22:00:34.663813+0000","last_fullsized":"2024-09-10T22:00:34.663813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:29.825070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663467+0000","last_change":"2024-09-10T22:00:33.184124+0000","last_active":"2024-09-10T22:00:34.663467+0000","last_peered":"2024-09-10T22:00:34.663467+0000","last_clean":"2024-09-10T22:00:34.663467+0000","last_became_active":"2024-09-10T22:00:33.182588+0000","last_became_peered":"2024-09-10T22:00:33.182588+0000","last_unstale":"2024-09-10T22:00:34.663467+0000","last_undegraded":"2024-09-10T22:00:34.663467+0000","last_fullsized":"2024-09-10T22:00:34.663467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:39.577554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183252+0000","last_change":"2024-09-10T22:00:33.178023+0000","last_active":"2024-09-10T22:00:34.183252+0000","last_peered":"2024-09-10T22:00:34.183252+0000","last_clean":"2024-09-10T22:00:34.183252+0000","last_became_active":"2024-09-10T22:00:33.176108+0000","last_became_peered":"2024-09-10T22:00:33.176108+0000","last_unstale":"2024-09-10T22:00:34.183252+0000","last_undegraded":"2024-09-10T22:00:34.183252+0000","last_fullsized":"2024-09-10T22:00:34.183252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:06.332661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184712+0000","last_change":"2024-09-10T22:00:33.208492+0000","last_active":"2024-09-10T22:00:34.184712+0000","last_peered":"2024-09-10T22:00:34.184712+0000","last_clean":"2024-09-10T22:00:34.184712+0000","last_became_active":"2024-09-10T22:00:33.208232+0000","last_became_peered":"2024-09-10T22:00:33.208232+0000","last_unstale":"2024-09-10T22:00:34.184712+0000","last_undegraded":"2024-09-10T22:00:34.184712+0000","last_fullsized":"2024-09-10T22:00:34.184712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.744906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185766+0000","last_change":"2024-09-10T22:00:33.212565+0000","last_active":"2024-09-10T22:00:34.185766+0000","last_peered":"2024-09-10T22:00:34.185766+0000","last_clean":"2024-09-10T22:00:34.185766+0000","last_became_active":"2024-09-10T22:00:33.210654+0000","last_became_peered":"2024-09-10T22:00:33.210654+0000","last_unstale":"2024-09-10T22:00:34.185766+0000","last_undegraded":"2024-09-10T22:00:34.185766+0000","last_fullsized":"2024-09-10T22:00:34.185766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:11.723619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663811+0000","last_change":"2024-09-10T22:00:33.192188+0000","last_active":"2024-09-10T22:00:34.663811+0000","last_peered":"2024-09-10T22:00:34.663811+0000","last_clean":"2024-09-10T22:00:34.663811+0000","last_became_active":"2024-09-10T22:00:33.191616+0000","last_became_peered":"2024-09-10T22:00:33.191616+0000","last_unstale":"2024-09-10T22:00:34.663811+0000","last_undegraded":"2024-09-10T22:00:34.663811+0000","last_fullsized":"2024-09-10T22:00:34.663811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:03.715049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665186+0000","last_change":"2024-09-10T22:00:33.203524+0000","last_active":"2024-09-10T22:00:34.665186+0000","last_peered":"2024-09-10T22:00:34.665186+0000","last_clean":"2024-09-10T22:00:34.665186+0000","last_became_active":"2024-09-10T22:00:33.202541+0000","last_became_peered":"2024-09-10T22:00:33.202541+0000","last_unstale":"2024-09-10T22:00:34.665186+0000","last_undegraded":"2024-09-10T22:00:34.665186+0000","last_fullsized":"2024-09-10T22:00:34.665186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:55.365445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185716+0000","last_change":"2024-09-10T22:00:33.192918+0000","last_active":"2024-09-10T22:00:34.185716+0000","last_peered":"2024-09-10T22:00:34.185716+0000","last_clean":"2024-09-10T22:00:34.185716+0000","last_became_active":"2024-09-10T22:00:33.192320+0000","last_became_peered":"2024-09-10T22:00:33.192320+0000","last_unstale":"2024-09-10T22:00:34.185716+0000","last_undegraded":"2024-09-10T22:00:34.185716+0000","last_fullsized":"2024-09-10T22:00:34.185716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:54.509264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664157+0000","last_change":"2024-09-10T22:00:33.186507+0000","last_active":"2024-09-10T22:00:34.664157+0000","last_peered":"2024-09-10T22:00:34.664157+0000","last_clean":"2024-09-10T22:00:34.664157+0000","last_became_active":"2024-09-10T22:00:33.186253+0000","last_became_peered":"2024-09-10T22:00:33.186253+0000","last_unstale":"2024-09-10T22:00:34.664157+0000","last_undegraded":"2024-09-10T22:00:34.664157+0000","last_fullsized":"2024-09-10T22:00:34.664157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:29.825070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185357+0000","last_change":"2024-09-10T22:00:33.213873+0000","last_active":"2024-09-10T22:00:34.185357+0000","last_peered":"2024-09-10T22:00:34.185357+0000","last_clean":"2024-09-10T22:00:34.185357+0000","last_became_active":"2024-09-10T22:00:33.211788+0000","last_became_peered":"2024-09-10T22:00:33.211788+0000","last_unstale":"2024-09-10T22:00:34.185357+0000","last_undegraded":"2024-09-10T22:00:34.185357+0000","last_fullsized":"2024-09-10T22:00:34.185357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:36.957181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663634+0000","last_change":"2024-09-10T22:00:33.182956+0000","last_active":"2024-09-10T22:00:34.663634+0000","last_peered":"2024-09-10T22:00:34.663634+0000","last_clean":"2024-09-10T22:00:34.663634+0000","last_became_active":"2024-09-10T22:00:33.182699+0000","last_became_peered":"2024-09-10T22:00:33.182699+0000","last_unstale":"2024-09-10T22:00:34.663634+0000","last_undegraded":"2024-09-10T22:00:34.663634+0000","last_fullsized":"2024-09-10T22:00:34.663634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:54.559113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184784+0000","last_change":"2024-09-10T22:00:33.208620+0000","last_active":"2024-09-10T22:00:34.184784+0000","last_peered":"2024-09-10T22:00:34.184784+0000","last_clean":"2024-09-10T22:00:34.184784+0000","last_became_active":"2024-09-10T22:00:33.208337+0000","last_became_peered":"2024-09-10T22:00:33.208337+0000","last_unstale":"2024-09-10T22:00:34.184784+0000","last_undegraded":"2024-09-10T22:00:34.184784+0000","last_fullsized":"2024-09-10T22:00:34.184784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:22.353564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664536+0000","last_change":"2024-09-10T22:00:33.185808+0000","last_active":"2024-09-10T22:00:34.664536+0000","last_peered":"2024-09-10T22:00:34.664536+0000","last_clean":"2024-09-10T22:00:34.664536+0000","last_became_active":"2024-09-10T22:00:33.185528+0000","last_became_peered":"2024-09-10T22:00:33.185528+0000","last_unstale":"2024-09-10T22:00:34.664536+0000","last_undegraded":"2024-09-10T22:00:34.664536+0000","last_fullsized":"2024-09-10T22:00:34.664536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:04.884964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187164+0000","last_change":"2024-09-10T22:00:33.191781+0000","last_active":"2024-09-10T22:00:34.187164+0000","last_peered":"2024-09-10T22:00:34.187164+0000","last_clean":"2024-09-10T22:00:34.187164+0000","last_became_active":"2024-09-10T22:00:33.191497+0000","last_became_peered":"2024-09-10T22:00:33.191497+0000","last_unstale":"2024-09-10T22:00:34.187164+0000","last_undegraded":"2024-09-10T22:00:34.187164+0000","last_fullsized":"2024-09-10T22:00:34.187164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:22.353564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183465+0000","last_change":"2024-09-10T22:00:33.183377+0000","last_active":"2024-09-10T22:00:34.183465+0000","last_peered":"2024-09-10T22:00:34.183465+0000","last_clean":"2024-09-10T22:00:34.183465+0000","last_became_active":"2024-09-10T22:00:33.181838+0000","last_became_peered":"2024-09-10T22:00:33.181838+0000","last_unstale":"2024-09-10T22:00:34.183465+0000","last_undegraded":"2024-09-10T22:00:34.183465+0000","last_fullsized":"2024-09-10T22:00:34.183465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:58.358390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663596+0000","last_change":"2024-09-10T22:00:33.185739+0000","last_active":"2024-09-10T22:00:34.663596+0000","last_peered":"2024-09-10T22:00:34.663596+0000","last_clean":"2024-09-10T22:00:34.663596+0000","last_became_active":"2024-09-10T22:00:33.185426+0000","last_became_peered":"2024-09-10T22:00:33.185426+0000","last_unstale":"2024-09-10T22:00:34.663596+0000","last_undegraded":"2024-09-10T22:00:34.663596+0000","last_fullsized":"2024-09-10T22:00:34.663596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:46.008480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664376+0000","last_change":"2024-09-10T22:00:33.183396+0000","last_active":"2024-09-10T22:00:34.664376+0000","last_peered":"2024-09-10T22:00:34.664376+0000","last_clean":"2024-09-10T22:00:34.664376+0000","last_became_active":"2024-09-10T22:00:33.181415+0000","last_became_peered":"2024-09-10T22:00:33.181415+0000","last_unstale":"2024-09-10T22:00:34.664376+0000","last_undegraded":"2024-09-10T22:00:34.664376+0000","last_fullsized":"2024-09-10T22:00:34.664376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:58.358390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185581+0000","last_change":"2024-09-10T22:00:33.202653+0000","last_active":"2024-09-10T22:00:34.185581+0000","last_peered":"2024-09-10T22:00:34.185581+0000","last_clean":"2024-09-10T22:00:34.185581+0000","last_became_active":"2024-09-10T22:00:33.202300+0000","last_became_peered":"2024-09-10T22:00:33.202300+0000","last_unstale":"2024-09-10T22:00:34.185581+0000","last_undegraded":"2024-09-10T22:00:34.185581+0000","last_fullsized":"2024-09-10T22:00:34.185581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:29.821720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663157+0000","last_change":"2024-09-10T22:00:33.213581+0000","last_active":"2024-09-10T22:00:34.663157+0000","last_peered":"2024-09-10T22:00:34.663157+0000","last_clean":"2024-09-10T22:00:34.663157+0000","last_became_active":"2024-09-10T22:00:33.213202+0000","last_became_peered":"2024-09-10T22:00:33.213202+0000","last_unstale":"2024-09-10T22:00:34.663157+0000","last_undegraded":"2024-09-10T22:00:34.663157+0000","last_fullsized":"2024-09-10T22:00:34.663157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:03.339805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184712+0000","last_change":"2024-09-10T22:00:33.179958+0000","last_active":"2024-09-10T22:00:34.184712+0000","last_peered":"2024-09-10T22:00:34.184712+0000","last_clean":"2024-09-10T22:00:34.184712+0000","last_became_active":"2024-09-10T22:00:33.179251+0000","last_became_peered":"2024-09-10T22:00:33.179251+0000","last_unstale":"2024-09-10T22:00:34.184712+0000","last_undegraded":"2024-09-10T22:00:34.184712+0000","last_fullsized":"2024-09-10T22:00:34.184712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:20.738322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664541+0000","last_change":"2024-09-10T22:00:33.181036+0000","last_active":"2024-09-10T22:00:34.664541+0000","last_peered":"2024-09-10T22:00:34.664541+0000","last_clean":"2024-09-10T22:00:34.664541+0000","last_became_active":"2024-09-10T22:00:33.179960+0000","last_became_peered":"2024-09-10T22:00:33.179960+0000","last_unstale":"2024-09-10T22:00:34.664541+0000","last_undegraded":"2024-09-10T22:00:34.664541+0000","last_fullsized":"2024-09-10T22:00:34.664541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:36.003731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663924+0000","last_change":"2024-09-10T22:00:32.187576+0000","last_active":"2024-09-10T22:00:34.663924+0000","last_peered":"2024-09-10T22:00:34.663924+0000","last_clean":"2024-09-10T22:00:34.663924+0000","last_became_active":"2024-09-10T22:00:32.187338+0000","last_became_peered":"2024-09-10T22:00:32.187338+0000","last_unstale":"2024-09-10T22:00:34.663924+0000","last_undegraded":"2024-09-10T22:00:34.663924+0000","last_fullsized":"2024-09-10T22:00:34.663924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:39.648880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664576+0000","last_change":"2024-09-10T22:00:33.184050+0000","last_active":"2024-09-10T22:00:34.664576+0000","last_peered":"2024-09-10T22:00:34.664576+0000","last_clean":"2024-09-10T22:00:34.664576+0000","last_became_active":"2024-09-10T22:00:33.183760+0000","last_became_peered":"2024-09-10T22:00:33.183760+0000","last_unstale":"2024-09-10T22:00:34.664576+0000","last_undegraded":"2024-09-10T22:00:34.664576+0000","last_fullsized":"2024-09-10T22:00:34.664576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:08.694915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186593+0000","last_change":"2024-09-10T22:00:32.181714+0000","last_active":"2024-09-10T22:00:34.186593+0000","last_peered":"2024-09-10T22:00:34.186593+0000","last_clean":"2024-09-10T22:00:34.186593+0000","last_became_active":"2024-09-10T22:00:32.180900+0000","last_became_peered":"2024-09-10T22:00:32.180900+0000","last_unstale":"2024-09-10T22:00:34.186593+0000","last_undegraded":"2024-09-10T22:00:34.186593+0000","last_fullsized":"2024-09-10T22:00:34.186593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:19.067912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663685+0000","last_change":"2024-09-10T22:00:32.194626+0000","last_active":"2024-09-10T22:00:34.663685+0000","last_peered":"2024-09-10T22:00:34.663685+0000","last_clean":"2024-09-10T22:00:34.663685+0000","last_became_active":"2024-09-10T22:00:32.194317+0000","last_became_peered":"2024-09-10T22:00:32.194317+0000","last_unstale":"2024-09-10T22:00:34.663685+0000","last_undegraded":"2024-09-10T22:00:34.663685+0000","last_fullsized":"2024-09-10T22:00:34.663685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:36.587242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663212+0000","last_change":"2024-09-10T22:00:33.180723+0000","last_active":"2024-09-10T22:00:34.663212+0000","last_peered":"2024-09-10T22:00:34.663212+0000","last_clean":"2024-09-10T22:00:34.663212+0000","last_became_active":"2024-09-10T22:00:33.180399+0000","last_became_peered":"2024-09-10T22:00:33.180399+0000","last_unstale":"2024-09-10T22:00:34.663212+0000","last_undegraded":"2024-09-10T22:00:34.663212+0000","last_fullsized":"2024-09-10T22:00:34.663212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:24.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184786+0000","last_change":"2024-09-10T22:00:32.184624+0000","last_active":"2024-09-10T22:00:34.184786+0000","last_peered":"2024-09-10T22:00:34.184786+0000","last_clean":"2024-09-10T22:00:34.184786+0000","last_became_active":"2024-09-10T22:00:32.184352+0000","last_became_peered":"2024-09-10T22:00:32.184352+0000","last_unstale":"2024-09-10T22:00:34.184786+0000","last_undegraded":"2024-09-10T22:00:34.184786+0000","last_fullsized":"2024-09-10T22:00:34.184786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:19.067912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183246+0000","last_change":"2024-09-10T22:00:33.178900+0000","last_active":"2024-09-10T22:00:34.183246+0000","last_peered":"2024-09-10T22:00:34.183246+0000","last_clean":"2024-09-10T22:00:34.183246+0000","last_became_active":"2024-09-10T22:00:33.178595+0000","last_became_peered":"2024-09-10T22:00:33.178595+0000","last_unstale":"2024-09-10T22:00:34.183246+0000","last_undegraded":"2024-09-10T22:00:34.183246+0000","last_fullsized":"2024-09-10T22:00:34.183246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:46.008480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187010+0000","last_change":"2024-09-10T22:00:32.193426+0000","last_active":"2024-09-10T22:00:34.187010+0000","last_peered":"2024-09-10T22:00:34.187010+0000","last_clean":"2024-09-10T22:00:34.187010+0000","last_became_active":"2024-09-10T22:00:32.189386+0000","last_became_peered":"2024-09-10T22:00:32.189386+0000","last_unstale":"2024-09-10T22:00:34.187010+0000","last_undegraded":"2024-09-10T22:00:34.187010+0000","last_fullsized":"2024-09-10T22:00:34.187010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:55.143270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185444+0000","last_change":"2024-09-10T22:00:33.213346+0000","last_active":"2024-09-10T22:00:34.185444+0000","last_peered":"2024-09-10T22:00:34.185444+0000","last_clean":"2024-09-10T22:00:34.185444+0000","last_became_active":"2024-09-10T22:00:33.213099+0000","last_became_peered":"2024-09-10T22:00:33.213099+0000","last_unstale":"2024-09-10T22:00:34.185444+0000","last_undegraded":"2024-09-10T22:00:34.185444+0000","last_fullsized":"2024-09-10T22:00:34.185444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:21.770779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185507+0000","last_change":"2024-09-10T22:00:32.195226+0000","last_active":"2024-09-10T22:00:34.185507+0000","last_peered":"2024-09-10T22:00:34.185507+0000","last_clean":"2024-09-10T22:00:34.185507+0000","last_became_active":"2024-09-10T22:00:32.194900+0000","last_became_peered":"2024-09-10T22:00:32.194900+0000","last_unstale":"2024-09-10T22:00:34.185507+0000","last_undegraded":"2024-09-10T22:00:34.185507+0000","last_fullsized":"2024-09-10T22:00:34.185507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:28.818521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663832+0000","last_change":"2024-09-10T22:00:33.213048+0000","last_active":"2024-09-10T22:00:34.663832+0000","last_peered":"2024-09-10T22:00:34.663832+0000","last_clean":"2024-09-10T22:00:34.663832+0000","last_became_active":"2024-09-10T22:00:33.212747+0000","last_became_peered":"2024-09-10T22:00:33.212747+0000","last_unstale":"2024-09-10T22:00:34.663832+0000","last_undegraded":"2024-09-10T22:00:34.663832+0000","last_fullsized":"2024-09-10T22:00:34.663832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:46.008480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183502+0000","last_change":"2024-09-10T22:00:32.194672+0000","last_active":"2024-09-10T22:00:34.183502+0000","last_peered":"2024-09-10T22:00:34.183502+0000","last_clean":"2024-09-10T22:00:34.183502+0000","last_became_active":"2024-09-10T22:00:32.194084+0000","last_became_peered":"2024-09-10T22:00:32.194084+0000","last_unstale":"2024-09-10T22:00:34.183502+0000","last_undegraded":"2024-09-10T22:00:34.183502+0000","last_fullsized":"2024-09-10T22:00:34.183502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:09.395633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665544+0000","last_change":"2024-09-10T22:00:33.176539+0000","last_active":"2024-09-10T22:00:34.665544+0000","last_peered":"2024-09-10T22:00:34.665544+0000","last_clean":"2024-09-10T22:00:34.665544+0000","last_became_active":"2024-09-10T22:00:33.175809+0000","last_became_peered":"2024-09-10T22:00:33.175809+0000","last_unstale":"2024-09-10T22:00:34.665544+0000","last_undegraded":"2024-09-10T22:00:34.665544+0000","last_fullsized":"2024-09-10T22:00:34.665544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:22.046625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185783+0000","last_change":"2024-09-10T22:00:32.187798+0000","last_active":"2024-09-10T22:00:34.185783+0000","last_peered":"2024-09-10T22:00:34.185783+0000","last_clean":"2024-09-10T22:00:34.185783+0000","last_became_active":"2024-09-10T22:00:32.184530+0000","last_became_peered":"2024-09-10T22:00:32.184530+0000","last_unstale":"2024-09-10T22:00:34.185783+0000","last_undegraded":"2024-09-10T22:00:34.185783+0000","last_fullsized":"2024-09-10T22:00:34.185783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:41.444694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664461+0000","last_change":"2024-09-10T22:00:33.189951+0000","last_active":"2024-09-10T22:00:34.664461+0000","last_peered":"2024-09-10T22:00:34.664461+0000","last_clean":"2024-09-10T22:00:34.664461+0000","last_became_active":"2024-09-10T22:00:33.189183+0000","last_became_peered":"2024-09-10T22:00:33.189183+0000","last_unstale":"2024-09-10T22:00:34.664461+0000","last_undegraded":"2024-09-10T22:00:34.664461+0000","last_fullsized":"2024-09-10T22:00:34.664461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:04.884964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664660+0000","last_change":"2024-09-10T22:00:32.171323+0000","last_active":"2024-09-10T22:00:34.664660+0000","last_peered":"2024-09-10T22:00:34.664660+0000","last_clean":"2024-09-10T22:00:34.664660+0000","last_became_active":"2024-09-10T22:00:32.171059+0000","last_became_peered":"2024-09-10T22:00:32.171059+0000","last_unstale":"2024-09-10T22:00:34.664660+0000","last_undegraded":"2024-09-10T22:00:34.664660+0000","last_fullsized":"2024-09-10T22:00:34.664660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:44.404901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663672+0000","last_change":"2024-09-10T22:00:33.196013+0000","last_active":"2024-09-10T22:00:34.663672+0000","last_peered":"2024-09-10T22:00:34.663672+0000","last_clean":"2024-09-10T22:00:34.663672+0000","last_became_active":"2024-09-10T22:00:33.195744+0000","last_became_peered":"2024-09-10T22:00:33.195744+0000","last_unstale":"2024-09-10T22:00:34.663672+0000","last_undegraded":"2024-09-10T22:00:34.663672+0000","last_fullsized":"2024-09-10T22:00:34.663672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:40.305055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185639+0000","last_change":"2024-09-10T22:00:32.196550+0000","last_active":"2024-09-10T22:00:34.185639+0000","last_peered":"2024-09-10T22:00:34.185639+0000","last_clean":"2024-09-10T22:00:34.185639+0000","last_became_active":"2024-09-10T22:00:32.196255+0000","last_became_peered":"2024-09-10T22:00:32.196255+0000","last_unstale":"2024-09-10T22:00:34.185639+0000","last_undegraded":"2024-09-10T22:00:34.185639+0000","last_fullsized":"2024-09-10T22:00:34.185639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:27.356827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665144+0000","last_change":"2024-09-10T22:00:33.197537+0000","last_active":"2024-09-10T22:00:34.665144+0000","last_peered":"2024-09-10T22:00:34.665144+0000","last_clean":"2024-09-10T22:00:34.665144+0000","last_became_active":"2024-09-10T22:00:33.197230+0000","last_became_peered":"2024-09-10T22:00:33.197230+0000","last_unstale":"2024-09-10T22:00:34.665144+0000","last_undegraded":"2024-09-10T22:00:34.665144+0000","last_fullsized":"2024-09-10T22:00:34.665144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:57.404481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184926+0000","last_change":"2024-09-10T22:00:32.187773+0000","last_active":"2024-09-10T22:00:34.184926+0000","last_peered":"2024-09-10T22:00:34.184926+0000","last_clean":"2024-09-10T22:00:34.184926+0000","last_became_active":"2024-09-10T22:00:32.185748+0000","last_became_peered":"2024-09-10T22:00:32.185748+0000","last_unstale":"2024-09-10T22:00:34.184926+0000","last_undegraded":"2024-09-10T22:00:34.184926+0000","last_fullsized":"2024-09-10T22:00:34.184926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:39.648880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184928+0000","last_change":"2024-09-10T22:00:33.180297+0000","last_active":"2024-09-10T22:00:34.184928+0000","last_peered":"2024-09-10T22:00:34.184928+0000","last_clean":"2024-09-10T22:00:34.184928+0000","last_became_active":"2024-09-10T22:00:33.179643+0000","last_became_peered":"2024-09-10T22:00:33.179643+0000","last_unstale":"2024-09-10T22:00:34.184928+0000","last_undegraded":"2024-09-10T22:00:34.184928+0000","last_fullsized":"2024-09-10T22:00:34.184928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:06.332661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.662989+0000","last_change":"2024-09-10T22:00:32.182584+0000","last_active":"2024-09-10T22:00:34.662989+0000","last_peered":"2024-09-10T22:00:34.662989+0000","last_clean":"2024-09-10T22:00:34.662989+0000","last_became_active":"2024-09-10T22:00:32.182289+0000","last_became_peered":"2024-09-10T22:00:32.182289+0000","last_unstale":"2024-09-10T22:00:34.662989+0000","last_undegraded":"2024-09-10T22:00:34.662989+0000","last_fullsized":"2024-09-10T22:00:34.662989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:41.773634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664524+0000","last_change":"2024-09-10T22:00:33.189852+0000","last_active":"2024-09-10T22:00:34.664524+0000","last_peered":"2024-09-10T22:00:34.664524+0000","last_clean":"2024-09-10T22:00:34.664524+0000","last_became_active":"2024-09-10T22:00:33.189492+0000","last_became_peered":"2024-09-10T22:00:33.189492+0000","last_unstale":"2024-09-10T22:00:34.664524+0000","last_undegraded":"2024-09-10T22:00:34.664524+0000","last_fullsized":"2024-09-10T22:00:34.664524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:45.010893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665636+0000","last_change":"2024-09-10T22:00:32.179367+0000","last_active":"2024-09-10T22:00:34.665636+0000","last_peered":"2024-09-10T22:00:34.665636+0000","last_clean":"2024-09-10T22:00:34.665636+0000","last_became_active":"2024-09-10T22:00:32.177804+0000","last_became_peered":"2024-09-10T22:00:32.177804+0000","last_unstale":"2024-09-10T22:00:34.665636+0000","last_undegraded":"2024-09-10T22:00:34.665636+0000","last_fullsized":"2024-09-10T22:00:34.665636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:13.368508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665546+0000","last_change":"2024-09-10T22:00:33.194213+0000","last_active":"2024-09-10T22:00:34.665546+0000","last_peered":"2024-09-10T22:00:34.665546+0000","last_clean":"2024-09-10T22:00:34.665546+0000","last_became_active":"2024-09-10T22:00:33.193708+0000","last_became_peered":"2024-09-10T22:00:33.193708+0000","last_unstale":"2024-09-10T22:00:34.665546+0000","last_undegraded":"2024-09-10T22:00:34.665546+0000","last_fullsized":"2024-09-10T22:00:34.665546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:39.533698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663916+0000","last_change":"2024-09-10T22:00:32.184130+0000","last_active":"2024-09-10T22:00:34.663916+0000","last_peered":"2024-09-10T22:00:34.663916+0000","last_clean":"2024-09-10T22:00:34.663916+0000","last_became_active":"2024-09-10T22:00:32.182524+0000","last_became_peered":"2024-09-10T22:00:32.182524+0000","last_unstale":"2024-09-10T22:00:34.663916+0000","last_undegraded":"2024-09-10T22:00:34.663916+0000","last_fullsized":"2024-09-10T22:00:34.663916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:26.056799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665889+0000","last_change":"2024-09-10T22:00:33.198846+0000","last_active":"2024-09-10T22:00:34.665889+0000","last_peered":"2024-09-10T22:00:34.665889+0000","last_clean":"2024-09-10T22:00:34.665889+0000","last_became_active":"2024-09-10T22:00:33.198166+0000","last_became_peered":"2024-09-10T22:00:33.198166+0000","last_unstale":"2024-09-10T22:00:34.665889+0000","last_undegraded":"2024-09-10T22:00:34.665889+0000","last_fullsized":"2024-09-10T22:00:34.665889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:21.862163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183552+0000","last_change":"2024-09-10T22:00:32.181657+0000","last_active":"2024-09-10T22:00:34.183552+0000","last_peered":"2024-09-10T22:00:34.183552+0000","last_clean":"2024-09-10T22:00:34.183552+0000","last_became_active":"2024-09-10T22:00:32.181365+0000","last_became_peered":"2024-09-10T22:00:32.181365+0000","last_unstale":"2024-09-10T22:00:34.183552+0000","last_undegraded":"2024-09-10T22:00:34.183552+0000","last_fullsized":"2024-09-10T22:00:34.183552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:34.356560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185996+0000","last_change":"2024-09-10T22:00:33.215101+0000","last_active":"2024-09-10T22:00:34.185996+0000","last_peered":"2024-09-10T22:00:34.185996+0000","last_clean":"2024-09-10T22:00:34.185996+0000","last_became_active":"2024-09-10T22:00:33.214294+0000","last_became_peered":"2024-09-10T22:00:33.214294+0000","last_unstale":"2024-09-10T22:00:34.185996+0000","last_undegraded":"2024-09-10T22:00:34.185996+0000","last_fullsized":"2024-09-10T22:00:34.185996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:59.470008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663655+0000","last_change":"2024-09-10T22:00:32.195097+0000","last_active":"2024-09-10T22:00:34.663655+0000","last_peered":"2024-09-10T22:00:34.663655+0000","last_clean":"2024-09-10T22:00:34.663655+0000","last_became_active":"2024-09-10T22:00:32.194103+0000","last_became_peered":"2024-09-10T22:00:32.194103+0000","last_unstale":"2024-09-10T22:00:34.663655+0000","last_undegraded":"2024-09-10T22:00:34.663655+0000","last_fullsized":"2024-09-10T22:00:34.663655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:57.181450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664931+0000","last_change":"2024-09-10T22:00:33.212536+0000","last_active":"2024-09-10T22:00:34.664931+0000","last_peered":"2024-09-10T22:00:34.664931+0000","last_clean":"2024-09-10T22:00:34.664931+0000","last_became_active":"2024-09-10T22:00:33.212204+0000","last_became_peered":"2024-09-10T22:00:33.212204+0000","last_unstale":"2024-09-10T22:00:34.664931+0000","last_undegraded":"2024-09-10T22:00:34.664931+0000","last_fullsized":"2024-09-10T22:00:34.664931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:41.803895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664227+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:00:34.664227+0000","last_peered":"2024-09-10T22:00:34.664227+0000","last_clean":"2024-09-10T22:00:34.664227+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:00:34.664227+0000","last_undegraded":"2024-09-10T22:00:34.664227+0000","last_fullsized":"2024-09-10T22:00:34.664227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185132+0000","last_change":"2024-09-10T22:00:32.195314+0000","last_active":"2024-09-10T22:00:34.185132+0000","last_peered":"2024-09-10T22:00:34.185132+0000","last_clean":"2024-09-10T22:00:34.185132+0000","last_became_active":"2024-09-10T22:00:32.194960+0000","last_became_peered":"2024-09-10T22:00:32.194960+0000","last_unstale":"2024-09-10T22:00:34.185132+0000","last_undegraded":"2024-09-10T22:00:34.185132+0000","last_fullsized":"2024-09-10T22:00:34.185132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:01.067800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663524+0000","last_change":"2024-09-10T22:00:33.208452+0000","last_active":"2024-09-10T22:00:34.663524+0000","last_peered":"2024-09-10T22:00:34.663524+0000","last_clean":"2024-09-10T22:00:34.663524+0000","last_became_active":"2024-09-10T22:00:33.208146+0000","last_became_peered":"2024-09-10T22:00:33.208146+0000","last_unstale":"2024-09-10T22:00:34.663524+0000","last_undegraded":"2024-09-10T22:00:34.663524+0000","last_fullsized":"2024-09-10T22:00:34.663524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:40.305055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664710+0000","last_change":"2024-09-10T22:00:32.192173+0000","last_active":"2024-09-10T22:00:34.664710+0000","last_peered":"2024-09-10T22:00:34.664710+0000","last_clean":"2024-09-10T22:00:34.664710+0000","last_became_active":"2024-09-10T22:00:32.191792+0000","last_became_peered":"2024-09-10T22:00:32.191792+0000","last_unstale":"2024-09-10T22:00:34.664710+0000","last_undegraded":"2024-09-10T22:00:34.664710+0000","last_fullsized":"2024-09-10T22:00:34.664710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:34.356560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665272+0000","last_change":"2024-09-10T22:00:33.174576+0000","last_active":"2024-09-10T22:00:34.665272+0000","last_peered":"2024-09-10T22:00:34.665272+0000","last_clean":"2024-09-10T22:00:34.665272+0000","last_became_active":"2024-09-10T22:00:33.174232+0000","last_became_peered":"2024-09-10T22:00:33.174232+0000","last_unstale":"2024-09-10T22:00:34.665272+0000","last_undegraded":"2024-09-10T22:00:34.665272+0000","last_fullsized":"2024-09-10T22:00:34.665272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:55.992463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663264+0000","last_change":"2024-09-10T22:00:32.178337+0000","last_active":"2024-09-10T22:00:34.663264+0000","last_peered":"2024-09-10T22:00:34.663264+0000","last_clean":"2024-09-10T22:00:34.663264+0000","last_became_active":"2024-09-10T22:00:32.178053+0000","last_became_peered":"2024-09-10T22:00:32.178053+0000","last_unstale":"2024-09-10T22:00:34.663264+0000","last_undegraded":"2024-09-10T22:00:34.663264+0000","last_fullsized":"2024-09-10T22:00:34.663264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:26.949229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183185+0000","last_change":"2024-09-10T22:00:33.184308+0000","last_active":"2024-09-10T22:00:34.183185+0000","last_peered":"2024-09-10T22:00:34.183185+0000","last_clean":"2024-09-10T22:00:34.183185+0000","last_became_active":"2024-09-10T22:00:33.184050+0000","last_became_peered":"2024-09-10T22:00:33.184050+0000","last_unstale":"2024-09-10T22:00:34.183185+0000","last_undegraded":"2024-09-10T22:00:34.183185+0000","last_fullsized":"2024-09-10T22:00:34.183185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:40.305055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185742+0000","last_change":"2024-09-10T22:00:32.195663+0000","last_active":"2024-09-10T22:00:34.185742+0000","last_peered":"2024-09-10T22:00:34.185742+0000","last_clean":"2024-09-10T22:00:34.185742+0000","last_became_active":"2024-09-10T22:00:32.195368+0000","last_became_peered":"2024-09-10T22:00:32.195368+0000","last_unstale":"2024-09-10T22:00:34.185742+0000","last_undegraded":"2024-09-10T22:00:34.185742+0000","last_fullsized":"2024-09-10T22:00:34.185742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:56.084833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664779+0000","last_change":"2024-09-10T22:00:33.186193+0000","last_active":"2024-09-10T22:00:34.664779+0000","last_peered":"2024-09-10T22:00:34.664779+0000","last_clean":"2024-09-10T22:00:34.664779+0000","last_became_active":"2024-09-10T22:00:33.180505+0000","last_became_peered":"2024-09-10T22:00:33.180505+0000","last_unstale":"2024-09-10T22:00:34.664779+0000","last_undegraded":"2024-09-10T22:00:34.664779+0000","last_fullsized":"2024-09-10T22:00:34.664779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:10.402182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183297+0000","last_change":"2024-09-10T22:00:32.184540+0000","last_active":"2024-09-10T22:00:34.183297+0000","last_peered":"2024-09-10T22:00:34.183297+0000","last_clean":"2024-09-10T22:00:34.183297+0000","last_became_active":"2024-09-10T22:00:32.184213+0000","last_became_peered":"2024-09-10T22:00:32.184213+0000","last_unstale":"2024-09-10T22:00:34.183297+0000","last_undegraded":"2024-09-10T22:00:34.183297+0000","last_fullsized":"2024-09-10T22:00:34.183297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:21.040076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186341+0000","last_change":"2024-09-10T22:00:33.212897+0000","last_active":"2024-09-10T22:00:34.186341+0000","last_peered":"2024-09-10T22:00:34.186341+0000","last_clean":"2024-09-10T22:00:34.186341+0000","last_became_active":"2024-09-10T22:00:33.212668+0000","last_became_peered":"2024-09-10T22:00:33.212668+0000","last_unstale":"2024-09-10T22:00:34.186341+0000","last_undegraded":"2024-09-10T22:00:34.186341+0000","last_fullsized":"2024-09-10T22:00:34.186341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:47.684127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665052+0000","last_change":"2024-09-10T22:00:32.193491+0000","last_active":"2024-09-10T22:00:34.665052+0000","last_peered":"2024-09-10T22:00:34.665052+0000","last_clean":"2024-09-10T22:00:34.665052+0000","last_became_active":"2024-09-10T22:00:32.193147+0000","last_became_peered":"2024-09-10T22:00:32.193147+0000","last_unstale":"2024-09-10T22:00:34.665052+0000","last_undegraded":"2024-09-10T22:00:34.665052+0000","last_fullsized":"2024-09-10T22:00:34.665052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:51.181211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183356+0000","last_change":"2024-09-10T22:00:33.189064+0000","last_active":"2024-09-10T22:00:34.183356+0000","last_peered":"2024-09-10T22:00:34.183356+0000","last_clean":"2024-09-10T22:00:34.183356+0000","last_became_active":"2024-09-10T22:00:33.188733+0000","last_became_peered":"2024-09-10T22:00:33.188733+0000","last_unstale":"2024-09-10T22:00:34.183356+0000","last_undegraded":"2024-09-10T22:00:34.183356+0000","last_fullsized":"2024-09-10T22:00:34.183356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:29.825070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183468+0000","last_change":"2024-09-10T22:00:32.186782+0000","last_active":"2024-09-10T22:00:34.183468+0000","last_peered":"2024-09-10T22:00:34.183468+0000","last_clean":"2024-09-10T22:00:34.183468+0000","last_became_active":"2024-09-10T22:00:32.186028+0000","last_became_peered":"2024-09-10T22:00:32.186028+0000","last_unstale":"2024-09-10T22:00:34.183468+0000","last_undegraded":"2024-09-10T22:00:34.183468+0000","last_fullsized":"2024-09-10T22:00:34.183468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:34.997182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664559+0000","last_change":"2024-09-10T22:00:33.201547+0000","last_active":"2024-09-10T22:00:34.664559+0000","last_peered":"2024-09-10T22:00:34.664559+0000","last_clean":"2024-09-10T22:00:34.664559+0000","last_became_active":"2024-09-10T22:00:33.201302+0000","last_became_peered":"2024-09-10T22:00:33.201302+0000","last_unstale":"2024-09-10T22:00:34.664559+0000","last_undegraded":"2024-09-10T22:00:34.664559+0000","last_fullsized":"2024-09-10T22:00:34.664559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:45.010893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187107+0000","last_change":"2024-09-10T22:00:32.196354+0000","last_active":"2024-09-10T22:00:34.187107+0000","last_peered":"2024-09-10T22:00:34.187107+0000","last_clean":"2024-09-10T22:00:34.187107+0000","last_became_active":"2024-09-10T22:00:32.196053+0000","last_became_peered":"2024-09-10T22:00:32.196053+0000","last_unstale":"2024-09-10T22:00:34.187107+0000","last_undegraded":"2024-09-10T22:00:34.187107+0000","last_fullsized":"2024-09-10T22:00:34.187107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:37.446124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663935+0000","last_change":"2024-09-10T22:00:33.176478+0000","last_active":"2024-09-10T22:00:34.663935+0000","last_peered":"2024-09-10T22:00:34.663935+0000","last_clean":"2024-09-10T22:00:34.663935+0000","last_became_active":"2024-09-10T22:00:33.176211+0000","last_became_peered":"2024-09-10T22:00:33.176211+0000","last_unstale":"2024-09-10T22:00:34.663935+0000","last_undegraded":"2024-09-10T22:00:34.663935+0000","last_fullsized":"2024-09-10T22:00:34.663935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:54.559113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664088+0000","last_change":"2024-09-10T22:00:32.189994+0000","last_active":"2024-09-10T22:00:34.664088+0000","last_peered":"2024-09-10T22:00:34.664088+0000","last_clean":"2024-09-10T22:00:34.664088+0000","last_became_active":"2024-09-10T22:00:32.189654+0000","last_became_peered":"2024-09-10T22:00:32.189654+0000","last_unstale":"2024-09-10T22:00:34.664088+0000","last_undegraded":"2024-09-10T22:00:34.664088+0000","last_fullsized":"2024-09-10T22:00:34.664088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:34.987281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664635+0000","last_change":"2024-09-10T22:00:33.183489+0000","last_active":"2024-09-10T22:00:34.664635+0000","last_peered":"2024-09-10T22:00:34.664635+0000","last_clean":"2024-09-10T22:00:34.664635+0000","last_became_active":"2024-09-10T22:00:33.183217+0000","last_became_peered":"2024-09-10T22:00:33.183217+0000","last_unstale":"2024-09-10T22:00:34.664635+0000","last_undegraded":"2024-09-10T22:00:34.664635+0000","last_fullsized":"2024-09-10T22:00:34.664635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:35.993830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183297+0000","last_change":"2024-09-10T22:00:32.178130+0000","last_active":"2024-09-10T22:00:34.183297+0000","last_peered":"2024-09-10T22:00:34.183297+0000","last_clean":"2024-09-10T22:00:34.183297+0000","last_became_active":"2024-09-10T22:00:32.177879+0000","last_became_peered":"2024-09-10T22:00:32.177879+0000","last_unstale":"2024-09-10T22:00:34.183297+0000","last_undegraded":"2024-09-10T22:00:34.183297+0000","last_fullsized":"2024-09-10T22:00:34.183297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:34.987281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664090+0000","last_change":"2024-09-10T22:00:33.179961+0000","last_active":"2024-09-10T22:00:34.664090+0000","last_peered":"2024-09-10T22:00:34.664090+0000","last_clean":"2024-09-10T22:00:34.664090+0000","last_became_active":"2024-09-10T22:00:33.179700+0000","last_became_peered":"2024-09-10T22:00:33.179700+0000","last_unstale":"2024-09-10T22:00:34.664090+0000","last_undegraded":"2024-09-10T22:00:34.664090+0000","last_fullsized":"2024-09-10T22:00:34.664090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:08.694915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183241+0000","last_change":"2024-09-10T22:00:32.180382+0000","last_active":"2024-09-10T22:00:34.183241+0000","last_peered":"2024-09-10T22:00:34.183241+0000","last_clean":"2024-09-10T22:00:34.183241+0000","last_became_active":"2024-09-10T22:00:32.180075+0000","last_became_peered":"2024-09-10T22:00:32.180075+0000","last_unstale":"2024-09-10T22:00:34.183241+0000","last_undegraded":"2024-09-10T22:00:34.183241+0000","last_fullsized":"2024-09-10T22:00:34.183241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:54.640505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664041+0000","last_change":"2024-09-10T22:00:33.180424+0000","last_active":"2024-09-10T22:00:34.664041+0000","last_peered":"2024-09-10T22:00:34.664041+0000","last_clean":"2024-09-10T22:00:34.664041+0000","last_became_active":"2024-09-10T22:00:33.180181+0000","last_became_peered":"2024-09-10T22:00:33.180181+0000","last_unstale":"2024-09-10T22:00:34.664041+0000","last_undegraded":"2024-09-10T22:00:34.664041+0000","last_fullsized":"2024-09-10T22:00:34.664041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:58.358390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665152+0000","last_change":"2024-09-10T22:00:32.191652+0000","last_active":"2024-09-10T22:00:34.665152+0000","last_peered":"2024-09-10T22:00:34.665152+0000","last_clean":"2024-09-10T22:00:34.665152+0000","last_became_active":"2024-09-10T22:00:32.191353+0000","last_became_peered":"2024-09-10T22:00:32.191353+0000","last_unstale":"2024-09-10T22:00:34.665152+0000","last_undegraded":"2024-09-10T22:00:34.665152+0000","last_fullsized":"2024-09-10T22:00:34.665152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:21.040076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184746+0000","last_change":"2024-09-10T22:00:33.184285+0000","last_active":"2024-09-10T22:00:34.184746+0000","last_peered":"2024-09-10T22:00:34.184746+0000","last_clean":"2024-09-10T22:00:34.184746+0000","last_became_active":"2024-09-10T22:00:33.183977+0000","last_became_peered":"2024-09-10T22:00:33.183977+0000","last_unstale":"2024-09-10T22:00:34.184746+0000","last_undegraded":"2024-09-10T22:00:34.184746+0000","last_fullsized":"2024-09-10T22:00:34.184746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:30.181250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186284+0000","last_change":"2024-09-10T22:00:33.207897+0000","last_active":"2024-09-10T22:00:34.186284+0000","last_peered":"2024-09-10T22:00:34.186284+0000","last_clean":"2024-09-10T22:00:34.186284+0000","last_became_active":"2024-09-10T22:00:33.207639+0000","last_became_peered":"2024-09-10T22:00:33.207639+0000","last_unstale":"2024-09-10T22:00:34.186284+0000","last_undegraded":"2024-09-10T22:00:34.186284+0000","last_fullsized":"2024-09-10T22:00:34.186284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:47.952904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664351+0000","last_change":"2024-09-10T22:00:32.199875+0000","last_active":"2024-09-10T22:00:34.664351+0000","last_peered":"2024-09-10T22:00:34.664351+0000","last_clean":"2024-09-10T22:00:34.664351+0000","last_became_active":"2024-09-10T22:00:32.199625+0000","last_became_peered":"2024-09-10T22:00:32.199625+0000","last_unstale":"2024-09-10T22:00:34.664351+0000","last_undegraded":"2024-09-10T22:00:34.664351+0000","last_fullsized":"2024-09-10T22:00:34.664351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:09.395633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185863+0000","last_change":"2024-09-10T22:00:33.207063+0000","last_active":"2024-09-10T22:00:34.185863+0000","last_peered":"2024-09-10T22:00:34.185863+0000","last_clean":"2024-09-10T22:00:34.185863+0000","last_became_active":"2024-09-10T22:00:33.206867+0000","last_became_peered":"2024-09-10T22:00:33.206867+0000","last_unstale":"2024-09-10T22:00:34.185863+0000","last_undegraded":"2024-09-10T22:00:34.185863+0000","last_fullsized":"2024-09-10T22:00:34.185863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:44.640052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664917+0000","last_change":"2024-09-10T22:00:32.196714+0000","last_active":"2024-09-10T22:00:34.664917+0000","last_peered":"2024-09-10T22:00:34.664917+0000","last_clean":"2024-09-10T22:00:34.664917+0000","last_became_active":"2024-09-10T22:00:32.196446+0000","last_became_peered":"2024-09-10T22:00:32.196446+0000","last_unstale":"2024-09-10T22:00:34.664917+0000","last_undegraded":"2024-09-10T22:00:34.664917+0000","last_fullsized":"2024-09-10T22:00:34.664917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:39.201788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663624+0000","last_change":"2024-09-10T22:00:33.215295+0000","last_active":"2024-09-10T22:00:34.663624+0000","last_peered":"2024-09-10T22:00:34.663624+0000","last_clean":"2024-09-10T22:00:34.663624+0000","last_became_active":"2024-09-10T22:00:33.215041+0000","last_became_peered":"2024-09-10T22:00:33.215041+0000","last_unstale":"2024-09-10T22:00:34.663624+0000","last_undegraded":"2024-09-10T22:00:34.663624+0000","last_fullsized":"2024-09-10T22:00:34.663624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:24.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664333+0000","last_change":"2024-09-10T22:00:32.180489+0000","last_active":"2024-09-10T22:00:34.664333+0000","last_peered":"2024-09-10T22:00:34.664333+0000","last_clean":"2024-09-10T22:00:34.664333+0000","last_became_active":"2024-09-10T22:00:32.180177+0000","last_became_peered":"2024-09-10T22:00:32.180177+0000","last_unstale":"2024-09-10T22:00:34.664333+0000","last_undegraded":"2024-09-10T22:00:34.664333+0000","last_fullsized":"2024-09-10T22:00:34.664333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:57.175682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664468+0000","last_change":"2024-09-10T22:00:33.192548+0000","last_active":"2024-09-10T22:00:34.664468+0000","last_peered":"2024-09-10T22:00:34.664468+0000","last_clean":"2024-09-10T22:00:34.664468+0000","last_became_active":"2024-09-10T22:00:33.192272+0000","last_became_peered":"2024-09-10T22:00:33.192272+0000","last_unstale":"2024-09-10T22:00:34.664468+0000","last_undegraded":"2024-09-10T22:00:34.664468+0000","last_fullsized":"2024-09-10T22:00:34.664468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:25.562048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663908+0000","last_change":"2024-09-10T22:00:32.182534+0000","last_active":"2024-09-10T22:00:34.663908+0000","last_peered":"2024-09-10T22:00:34.663908+0000","last_clean":"2024-09-10T22:00:34.663908+0000","last_became_active":"2024-09-10T22:00:32.182286+0000","last_became_peered":"2024-09-10T22:00:32.182286+0000","last_unstale":"2024-09-10T22:00:34.663908+0000","last_undegraded":"2024-09-10T22:00:34.663908+0000","last_fullsized":"2024-09-10T22:00:34.663908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:56.418214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":667234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":225280,"data_stored":76374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":76410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":827392,"data_stored":667290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:00:42.321 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:00:42.321 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-10T22:00:42.322 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T22:00:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.501+0000 7f0fc5600640 1 -- 172.21.15.66:0/3105187099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa4003c90 msgr2=0x7f0fa4004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.501+0000 7f0fc5600640 1 --2- 172.21.15.66:0/3105187099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa4003c90 0x7f0fa4004090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0001860 tx=0x7f0fb001efb0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 -- 172.21.15.66:0/3105187099 shutdown_connections 2024-09-10T22:00:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 --2- 172.21.15.66:0/3105187099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa4003c90 0x7f0fa4004090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 -- 172.21.15.66:0/3105187099 >> 172.21.15.66:0/3105187099 conn(0x7f0fc00d0120 msgr2=0x7f0fc00d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 -- 172.21.15.66:0/3105187099 shutdown_connections 2024-09-10T22:00:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 -- 172.21.15.66:0/3105187099 wait complete. 2024-09-10T22:00:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 Processor -- start 2024-09-10T22:00:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 -- start start 2024-09-10T22:00:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa4003c90 0x7f0fc00c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fc00ca6f0 0x7f0fc00c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0fc00cab70 0x7f0fc00c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc006ef60 con 0x7f0fa4003c90 2024-09-10T22:00:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc006ede0 con 0x7f0fc00ca6f0 2024-09-10T22:00:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc5600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc006f0e0 con 0x7f0fc00cab70 2024-09-10T22:00:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa4003c90 0x7f0fc00c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa4003c90 0x7f0fc00c71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45430/0 (socket says 172.21.15.66:45430) 2024-09-10T22:00:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fbfe00640 1 -- 172.21.15.66:0/175397245 learned_addr learned my addr 172.21.15.66:0/175397245 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fc4c00640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0fc00cab70 0x7f0fc00c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fb7400640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fc00ca6f0 0x7f0fc00c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:42.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fbfe00640 1 -- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0fc00cab70 msgr2=0x7f0fc00c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fbfe00640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0fc00cab70 0x7f0fc00c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fbfe00640 1 -- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fc00ca6f0 msgr2=0x7f0fc00c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fbfe00640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fc00ca6f0 0x7f0fc00c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fbfe00640 1 -- 172.21.15.66:0/175397245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc00c7730 con 0x7f0fa4003c90 2024-09-10T22:00:42.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fb7400640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fc00ca6f0 0x7f0fc00c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:42.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.505+0000 7f0fbfe00640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa4003c90 0x7f0fc00c71f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0005040 tx=0x7f0fb0006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:42.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.509+0000 7f0fbcc00640 1 -- 172.21.15.66:0/175397245 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0003800 con 0x7f0fa4003c90 2024-09-10T22:00:42.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.509+0000 7f0fbcc00640 1 -- 172.21.15.66:0/175397245 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fb00b7070 con 0x7f0fa4003c90 2024-09-10T22:00:42.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.509+0000 7f0fbcc00640 1 -- 172.21.15.66:0/175397245 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0003d10 con 0x7f0fa4003c90 2024-09-10T22:00:42.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.509+0000 7f0fc5600640 1 -- 172.21.15.66:0/175397245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc0154a40 con 0x7f0fa4003c90 2024-09-10T22:00:42.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.509+0000 7f0fc5600640 1 -- 172.21.15.66:0/175397245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc00c5160 con 0x7f0fa4003c90 2024-09-10T22:00:42.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.509+0000 7f0fc5600640 1 -- 172.21.15.66:0/175397245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc00c55a0 con 0x7f0fa4003c90 2024-09-10T22:00:42.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.509+0000 7f0fbcc00640 1 -- 172.21.15.66:0/175397245 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0fb00bc470 con 0x7f0fa4003c90 2024-09-10T22:00:42.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.509+0000 7f0fbcc00640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0f9c0725e0 0x7f0f9c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:42.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.509+0000 7f0fbcc00640 1 -- 172.21.15.66:0/175397245 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0fb0130c10 con 0x7f0fa4003c90 2024-09-10T22:00:42.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.509+0000 7f0fb7400640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0f9c0725e0 0x7f0f9c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:42.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.513+0000 7f0fb7400640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0f9c0725e0 0x7f0f9c074aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8001870 tx=0x7f0fa8001ce0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:42.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.513+0000 7f0fbcc00640 1 -- 172.21.15.66:0/175397245 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0fb0018d60 con 0x7f0fa4003c90 2024-09-10T22:00:42.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.689+0000 7f0fc5600640 1 -- 172.21.15.66:0/175397245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f0fc006f640 con 0x7f0fa4003c90 2024-09-10T22:00:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.689+0000 7f0fbcc00640 1 -- 172.21.15.66:0/175397245 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7f0fb00fc3f0 con 0x7f0fa4003c90 2024-09-10T22:00:42.696 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:42.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.693+0000 7f0fc5600640 1 -- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0f9c0725e0 msgr2=0x7f0f9c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.693+0000 7f0fc5600640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0f9c0725e0 0x7f0f9c074aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8001870 tx=0x7f0fa8001ce0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.693+0000 7f0fc5600640 1 -- 172.21.15.66:0/175397245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa4003c90 msgr2=0x7f0fc00c71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.693+0000 7f0fc5600640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa4003c90 0x7f0fc00c71f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0005040 tx=0x7f0fb0006950 comp rx=0 tx=0).stop 2024-09-10T22:00:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.693+0000 7f0fc5600640 1 -- 172.21.15.66:0/175397245 shutdown_connections 2024-09-10T22:00:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.693+0000 7f0fc5600640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0f9c0725e0 0x7f0f9c074aa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.693+0000 7f0fc5600640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0fc00cab70 0x7f0fc00c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.693+0000 7f0fc5600640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fc00ca6f0 0x7f0fc00c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.693+0000 7f0fc5600640 1 --2- 172.21.15.66:0/175397245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa4003c90 0x7f0fc00c71f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.693+0000 7f0fc5600640 1 -- 172.21.15.66:0/175397245 >> 172.21.15.66:0/175397245 conn(0x7f0fc00d0120 msgr2=0x7f0fc00b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.693+0000 7f0fc5600640 1 -- 172.21.15.66:0/175397245 shutdown_connections 2024-09-10T22:00:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.693+0000 7f0fc5600640 1 -- 172.21.15.66:0/175397245 wait complete. 2024-09-10T22:00:42.718 INFO:teuthology.orchestra.run.smithi066.stdout:[{"version":27,"timestamp":"2024-09-10T22:00:20.891622+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"4537.831230"}]}] 2024-09-10T22:00:42.718 INFO:tasks.ceph_manager:config epoch is 27 2024-09-10T22:00:42.718 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T22:00:42.718 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T22:00:42.719 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T22:00:42.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.857+0000 7f06db200640 1 -- 172.21.15.66:0/365847350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06dc0cd7a0 msgr2=0x7f06dc0cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.857+0000 7f06db200640 1 --2- 172.21.15.66:0/365847350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06dc0cd7a0 0x7f06dc0cdc30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009bc0 tx=0x7f06d401cba0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 -- 172.21.15.66:0/365847350 shutdown_connections 2024-09-10T22:00:42.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 --2- 172.21.15.66:0/365847350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06dc0cd7a0 0x7f06dc0cdc30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 --2- 172.21.15.66:0/365847350 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f06dc15ef20 0x7f06dc0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 -- 172.21.15.66:0/365847350 >> 172.21.15.66:0/365847350 conn(0x7f06dc0d1b10 msgr2=0x7f06dc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:42.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 -- 172.21.15.66:0/365847350 shutdown_connections 2024-09-10T22:00:42.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 -- 172.21.15.66:0/365847350 wait complete. 2024-09-10T22:00:42.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 Processor -- start 2024-09-10T22:00:42.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 -- start start 2024-09-10T22:00:42.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06dc0cd7a0 0x7f06dc152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:42.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06dc15ef20 0x7f06dc1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f06dc15f300 0x7f06dc1576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc168ee0 con 0x7f06dc15ef20 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc168d60 con 0x7f06dc0cd7a0 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f06dc169060 con 0x7f06dc15f300 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06dc15ef20 0x7f06dc1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06dc15ef20 0x7f06dc1571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45436/0 (socket says 172.21.15.66:45436) 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d3e00640 1 -- 172.21.15.66:0/2608969023 learned_addr learned my addr 172.21.15.66:0/2608969023 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06da800640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f06dc15f300 0x7f06dc1576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d9e00640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06dc0cd7a0 0x7f06dc152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d3e00640 1 -- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f06dc15f300 msgr2=0x7f06dc1576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d3e00640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f06dc15f300 0x7f06dc1576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d3e00640 1 -- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06dc0cd7a0 msgr2=0x7f06dc152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d3e00640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06dc0cd7a0 0x7f06dc152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d3e00640 1 -- 172.21.15.66:0/2608969023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06dc0b6540 con 0x7f06dc15ef20 2024-09-10T22:00:42.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d3e00640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06dc15ef20 0x7f06dc1571b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f06c400c430 tx=0x7f06c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:42.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d2000640 1 -- 172.21.15.66:0/2608969023 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4011790 con 0x7f06dc15ef20 2024-09-10T22:00:42.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d2000640 1 -- 172.21.15.66:0/2608969023 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06c4017070 con 0x7f06dc15ef20 2024-09-10T22:00:42.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06db200640 1 -- 172.21.15.66:0/2608969023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06dc0b4b50 con 0x7f06dc15ef20 2024-09-10T22:00:42.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.861+0000 7f06d2000640 1 -- 172.21.15.66:0/2608969023 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4011ce0 con 0x7f06dc15ef20 2024-09-10T22:00:42.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.865+0000 7f06db200640 1 -- 172.21.15.66:0/2608969023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06dc153230 con 0x7f06dc15ef20 2024-09-10T22:00:42.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.865+0000 7f06db200640 1 -- 172.21.15.66:0/2608969023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06dc0cdc30 con 0x7f06dc15ef20 2024-09-10T22:00:42.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.865+0000 7f06d2000640 1 -- 172.21.15.66:0/2608969023 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f06c401c460 con 0x7f06dc15ef20 2024-09-10T22:00:42.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.865+0000 7f06d2000640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f06b80725e0 0x7f06b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:42.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.865+0000 7f06d2000640 1 -- 172.21.15.66:0/2608969023 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f06c4092f10 con 0x7f06dc15ef20 2024-09-10T22:00:42.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.865+0000 7f06d9e00640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f06b80725e0 0x7f06b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:42.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.865+0000 7f06d9e00640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f06b80725e0 0x7f06b8074aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009be0 tx=0x7f06cc0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:42.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:42.869+0000 7f06d2000640 1 -- 172.21.15.66:0/2608969023 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f06dc0b6540 con 0x7f06dc15ef20 2024-09-10T22:00:43.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.089+0000 7f06db200640 1 -- 172.21.15.66:0/2608969023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f06dc072e00 con 0x7f06dc15ef20 2024-09-10T22:00:43.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.089+0000 7f06d2000640 1 -- 172.21.15.66:0/2608969023 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+184985 (secure 0 0 0) 0x7f06c405dfc0 con 0x7f06dc15ef20 2024-09-10T22:00:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.093+0000 7f06db200640 1 -- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f06b80725e0 msgr2=0x7f06b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.093+0000 7f06db200640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f06b80725e0 0x7f06b8074aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009be0 tx=0x7f06cc0091e0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.093+0000 7f06db200640 1 -- 172.21.15.66:0/2608969023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06dc15ef20 msgr2=0x7f06dc1571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.093+0000 7f06db200640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06dc15ef20 0x7f06dc1571b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f06c400c430 tx=0x7f06c400c8f0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.093+0000 7f06db200640 1 -- 172.21.15.66:0/2608969023 shutdown_connections 2024-09-10T22:00:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.093+0000 7f06db200640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f06b80725e0 0x7f06b8074aa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.093+0000 7f06db200640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f06dc15f300 0x7f06dc1576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.093+0000 7f06db200640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06dc15ef20 0x7f06dc1571b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.093+0000 7f06db200640 1 --2- 172.21.15.66:0/2608969023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06dc0cd7a0 0x7f06dc152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.093+0000 7f06db200640 1 -- 172.21.15.66:0/2608969023 >> 172.21.15.66:0/2608969023 conn(0x7f06dc0d1b10 msgr2=0x7f06dc0aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:43.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.093+0000 7f06db200640 1 -- 172.21.15.66:0/2608969023 shutdown_connections 2024-09-10T22:00:43.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.093+0000 7f06db200640 1 -- 172.21.15.66:0/2608969023 wait complete. 2024-09-10T22:00:43.119 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":5,"flags":0,"active_gid":4123,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2679227974},{"type":"v1","addr":"172.21.15.94:6801","nonce":2679227974}]},"active_addr":"172.21.15.94:6801/2679227974","active_change":"2024-09-10T22:00:12.296805+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4126,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3435357317}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3280663360}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":2716784544}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1866628993}]}]} 2024-09-10T22:00:43.123 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T22:00:43.123 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T22:00:43.123 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:00:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.269+0000 7f60d5800640 1 -- 172.21.15.66:0/850990993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60d00d28c0 msgr2=0x7f60d00b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.269+0000 7f60d5800640 1 --2- 172.21.15.66:0/850990993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60d00d28c0 0x7f60d00b1120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60d00b6970 tx=0x7f60c801cd80 comp rx=0 tx=0).stop 2024-09-10T22:00:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.269+0000 7f60d5800640 1 -- 172.21.15.66:0/850990993 shutdown_connections 2024-09-10T22:00:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.269+0000 7f60d5800640 1 --2- 172.21.15.66:0/850990993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60d00d28c0 0x7f60d00b1120 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.269+0000 7f60d5800640 1 --2- 172.21.15.66:0/850990993 >> v2:172.21.15.94:3300/0 conn(0x7f60d0162630 0x7f60d00d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.269+0000 7f60d5800640 1 -- 172.21.15.66:0/850990993 >> 172.21.15.66:0/850990993 conn(0x7f60d00d3da0 msgr2=0x7f60d00d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.269+0000 7f60d5800640 1 -- 172.21.15.66:0/850990993 shutdown_connections 2024-09-10T22:00:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.269+0000 7f60d5800640 1 -- 172.21.15.66:0/850990993 wait complete. 2024-09-10T22:00:43.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60d5800640 1 Processor -- start 2024-09-10T22:00:43.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60d5800640 1 -- start start 2024-09-10T22:00:43.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60d5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60d00d28c0 0x7f60d0161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:43.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60d5800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60d0162630 0x7f60d0154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:43.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60d5800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d0153bb0 0x7f60d01604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:43.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60d5800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f60d016b9c0 con 0x7f60d00d28c0 2024-09-10T22:00:43.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60d5800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60d016b840 con 0x7f60d0153bb0 2024-09-10T22:00:43.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60d5800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f60d016bb40 con 0x7f60d0162630 2024-09-10T22:00:43.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60d00d28c0 0x7f60d0161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:43.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60d00d28c0 0x7f60d0161ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45456/0 (socket says 172.21.15.66:45456) 2024-09-10T22:00:43.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60cfe00640 1 -- 172.21.15.66:0/1873382605 learned_addr learned my addr 172.21.15.66:0/1873382605 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:43.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60d4e00640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d0153bb0 0x7f60d01604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:43.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60cfe00640 1 -- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60d0162630 msgr2=0x7f60d0154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:43.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60c7e00640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60d0162630 0x7f60d0154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:43.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60cfe00640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60d0162630 0x7f60d0154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60cfe00640 1 -- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d0153bb0 msgr2=0x7f60d01604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:43.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60cfe00640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d0153bb0 0x7f60d01604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60cfe00640 1 -- 172.21.15.66:0/1873382605 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d015fc30 con 0x7f60d00d28c0 2024-09-10T22:00:43.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60d4e00640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d0153bb0 0x7f60d01604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:00:43.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60c7e00640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60d0162630 0x7f60d0154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:00:43.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60cfe00640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60d00d28c0 0x7f60d0161ab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60c000d750 tx=0x7f60c000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:43.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60ccc00640 1 -- 172.21.15.66:0/1873382605 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0017070 con 0x7f60d00d28c0 2024-09-10T22:00:43.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60ccc00640 1 -- 172.21.15.66:0/1873382605 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60c0009d20 con 0x7f60d00d28c0 2024-09-10T22:00:43.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60ccc00640 1 -- 172.21.15.66:0/1873382605 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c000de60 con 0x7f60d00d28c0 2024-09-10T22:00:43.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60d5800640 1 -- 172.21.15.66:0/1873382605 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d015ff10 con 0x7f60d00d28c0 2024-09-10T22:00:43.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.273+0000 7f60d5800640 1 -- 172.21.15.66:0/1873382605 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d015a280 con 0x7f60d00d28c0 2024-09-10T22:00:43.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.277+0000 7f60d5800640 1 -- 172.21.15.66:0/1873382605 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d0160280 con 0x7f60d00d28c0 2024-09-10T22:00:43.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.277+0000 7f60ccc00640 1 -- 172.21.15.66:0/1873382605 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f60c0004280 con 0x7f60d00d28c0 2024-09-10T22:00:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.277+0000 7f60ccc00640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f60ac0725e0 0x7f60ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.277+0000 7f60ccc00640 1 -- 172.21.15.66:0/1873382605 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f60c0093640 con 0x7f60d00d28c0 2024-09-10T22:00:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.277+0000 7f60c7e00640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f60ac0725e0 0x7f60ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.277+0000 7f60c7e00640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f60ac0725e0 0x7f60ac074aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60d0161ff0 tx=0x7f60b8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:43.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.281+0000 7f60ccc00640 1 -- 172.21.15.66:0/1873382605 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f60c0020020 con 0x7f60d00d28c0 2024-09-10T22:00:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.457+0000 7f60d5800640 1 -- 172.21.15.66:0/1873382605 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60d008dd40 con 0x7f60d00d28c0 2024-09-10T22:00:43.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.457+0000 7f60ccc00640 1 -- 172.21.15.66:0/1873382605 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16593 (secure 0 0 0) 0x7f60c005ed10 con 0x7f60d00d28c0 2024-09-10T22:00:43.463 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:43.463 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":22,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:34.173529+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:30.545203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:31.538729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.461+0000 7f60d5800640 1 -- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f60ac0725e0 msgr2=0x7f60ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.461+0000 7f60d5800640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f60ac0725e0 0x7f60ac074aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60d0161ff0 tx=0x7f60b8009040 comp rx=0 tx=0).stop 2024-09-10T22:00:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.461+0000 7f60d5800640 1 -- 172.21.15.66:0/1873382605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60d00d28c0 msgr2=0x7f60d0161ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.461+0000 7f60d5800640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60d00d28c0 0x7f60d0161ab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60c000d750 tx=0x7f60c000dc10 comp rx=0 tx=0).stop 2024-09-10T22:00:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.461+0000 7f60d5800640 1 -- 172.21.15.66:0/1873382605 shutdown_connections 2024-09-10T22:00:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.461+0000 7f60d5800640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f60ac0725e0 0x7f60ac074aa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.461+0000 7f60d5800640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d0153bb0 0x7f60d01604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.461+0000 7f60d5800640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60d0162630 0x7f60d0154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.461+0000 7f60d5800640 1 --2- 172.21.15.66:0/1873382605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60d00d28c0 0x7f60d0161ab0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.461+0000 7f60d5800640 1 -- 172.21.15.66:0/1873382605 >> 172.21.15.66:0/1873382605 conn(0x7f60d00d3da0 msgr2=0x7f60d00d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:43.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.461+0000 7f60d5800640 1 -- 172.21.15.66:0/1873382605 shutdown_connections 2024-09-10T22:00:43.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.461+0000 7f60d5800640 1 -- 172.21.15.66:0/1873382605 wait complete. 2024-09-10T22:00:43.486 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T22:00:43.486 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:00:43.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.625+0000 7f8715400640 1 -- 172.21.15.66:0/2852818871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871015f340 msgr2=0x7f87100cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:43.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.625+0000 7f8715400640 1 --2- 172.21.15.66:0/2852818871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871015f340 0x7f87100cfb50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8700009900 tx=0x7f870001c8a0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.625+0000 7f8715400640 1 -- 172.21.15.66:0/2852818871 shutdown_connections 2024-09-10T22:00:43.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.625+0000 7f8715400640 1 --2- 172.21.15.66:0/2852818871 >> v2:172.21.15.94:3300/0 conn(0x7f87100af970 0x7f87100d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.625+0000 7f8715400640 1 --2- 172.21.15.66:0/2852818871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871015f340 0x7f87100cfb50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.625+0000 7f8715400640 1 -- 172.21.15.66:0/2852818871 >> 172.21.15.66:0/2852818871 conn(0x7f87100d1c50 msgr2=0x7f87100d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:43.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.625+0000 7f8715400640 1 -- 172.21.15.66:0/2852818871 shutdown_connections 2024-09-10T22:00:43.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.625+0000 7f8715400640 1 -- 172.21.15.66:0/2852818871 wait complete. 2024-09-10T22:00:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8715400640 1 Processor -- start 2024-09-10T22:00:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8715400640 1 -- start start 2024-09-10T22:00:43.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8715400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87100af970 0x7f871006e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8715400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87100b2bf0 0x7f8710165220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8715400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f871015f340 0x7f87100cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8715400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8710169190 con 0x7f87100af970 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8715400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8710169010 con 0x7f87100b2bf0 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8715400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8710169310 con 0x7f871015f340 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f870e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87100af970 0x7f871006e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f870e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87100af970 0x7f871006e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45468/0 (socket says 172.21.15.66:45468) 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f870e800640 1 -- 172.21.15.66:0/3977023973 learned_addr learned my addr 172.21.15.66:0/3977023973 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8707e00640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87100b2bf0 0x7f8710165220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f870f200640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f871015f340 0x7f87100cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f870e800640 1 -- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f871015f340 msgr2=0x7f87100cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f870e800640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f871015f340 0x7f87100cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f870e800640 1 -- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87100b2bf0 msgr2=0x7f8710165220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:43.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f870e800640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87100b2bf0 0x7f8710165220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f870e800640 1 -- 172.21.15.66:0/3977023973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87100cf370 con 0x7f87100af970 2024-09-10T22:00:43.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f870e800640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87100af970 0x7f871006e740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8700009850 tx=0x7f87000a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:43.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8706a00640 1 -- 172.21.15.66:0/3977023973 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87000b2040 con 0x7f87100af970 2024-09-10T22:00:43.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8706a00640 1 -- 172.21.15.66:0/3977023973 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87000043b0 con 0x7f87100af970 2024-09-10T22:00:43.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8715400640 1 -- 172.21.15.66:0/3977023973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87100b6140 con 0x7f87100af970 2024-09-10T22:00:43.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8706a00640 1 -- 172.21.15.66:0/3977023973 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700005280 con 0x7f87100af970 2024-09-10T22:00:43.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.629+0000 7f8715400640 1 -- 172.21.15.66:0/3977023973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87100b6700 con 0x7f87100af970 2024-09-10T22:00:43.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.633+0000 7f8715400640 1 -- 172.21.15.66:0/3977023973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871015fac0 con 0x7f87100af970 2024-09-10T22:00:43.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.633+0000 7f8706a00640 1 -- 172.21.15.66:0/3977023973 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8700005420 con 0x7f87100af970 2024-09-10T22:00:43.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.633+0000 7f8706a00640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86ec0725e0 0x7f86ec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:43.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.633+0000 7f8706a00640 1 -- 172.21.15.66:0/3977023973 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f870012cdc0 con 0x7f87100af970 2024-09-10T22:00:43.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.633+0000 7f8707e00640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86ec0725e0 0x7f86ec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:43.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.633+0000 7f8707e00640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86ec0725e0 0x7f86ec074aa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f87100cc0f0 tx=0x7f86f8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:43.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.637+0000 7f8706a00640 1 -- 172.21.15.66:0/3977023973 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8700016490 con 0x7f87100af970 2024-09-10T22:00:43.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.813+0000 7f8715400640 1 -- 172.21.15.66:0/3977023973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f871015e820 con 0x7f87100af970 2024-09-10T22:00:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8706a00640 1 -- 172.21.15.66:0/3977023973 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16593 (secure 0 0 0) 0x7f87000f8490 con 0x7f87100af970 2024-09-10T22:00:43.821 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:43.822 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":22,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:34.173529+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:30.545203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:31.538729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:43.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8715400640 1 -- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86ec0725e0 msgr2=0x7f86ec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8715400640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86ec0725e0 0x7f86ec074aa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f87100cc0f0 tx=0x7f86f8009040 comp rx=0 tx=0).stop 2024-09-10T22:00:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8715400640 1 -- 172.21.15.66:0/3977023973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87100af970 msgr2=0x7f871006e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8715400640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87100af970 0x7f871006e740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8700009850 tx=0x7f87000a5670 comp rx=0 tx=0).stop 2024-09-10T22:00:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8715400640 1 -- 172.21.15.66:0/3977023973 shutdown_connections 2024-09-10T22:00:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8715400640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86ec0725e0 0x7f86ec074aa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8715400640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f871015f340 0x7f87100cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8715400640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87100b2bf0 0x7f8710165220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8715400640 1 --2- 172.21.15.66:0/3977023973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87100af970 0x7f871006e740 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8715400640 1 -- 172.21.15.66:0/3977023973 >> 172.21.15.66:0/3977023973 conn(0x7f87100d1c50 msgr2=0x7f8710070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8715400640 1 -- 172.21.15.66:0/3977023973 shutdown_connections 2024-09-10T22:00:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:43.817+0000 7f8715400640 1 -- 172.21.15.66:0/3977023973 wait complete. 2024-09-10T22:00:43.844 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T22:00:43.845 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T22:00:43.845 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T22:00:43.846 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T22:00:43.846 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T22:00:43.846 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T22:00:43.847 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T22:00:43.847 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T22:00:44.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.089+0000 7f0159600640 1 -- 172.21.15.66:0/1174980985 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c003840 con 0x7f0154036250 2024-09-10T22:00:44.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.093+0000 7f015be00640 1 -- 172.21.15.66:0/1174980985 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0154036250 msgr2=0x7f01540e70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.093+0000 7f015be00640 1 --2- 172.21.15.66:0/1174980985 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0154036250 0x7f01540e70f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f014c005fd0 tx=0x7f014c01c720 comp rx=0 tx=0).stop 2024-09-10T22:00:44.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.093+0000 7f015be00640 1 -- 172.21.15.66:0/1174980985 shutdown_connections 2024-09-10T22:00:44.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.093+0000 7f015be00640 1 --2- 172.21.15.66:0/1174980985 >> v2:172.21.15.94:3300/0 conn(0x7f0154036630 0x7f0154033090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.093+0000 7f015be00640 1 --2- 172.21.15.66:0/1174980985 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0154036250 0x7f01540e70f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.093+0000 7f015be00640 1 -- 172.21.15.66:0/1174980985 >> 172.21.15.66:0/1174980985 conn(0x7f015401d780 msgr2=0x7f0154054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.097+0000 7f015be00640 1 -- 172.21.15.66:0/1174980985 shutdown_connections 2024-09-10T22:00:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.097+0000 7f015be00640 1 -- 172.21.15.66:0/1174980985 wait complete. 2024-09-10T22:00:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015be00640 1 Processor -- start 2024-09-10T22:00:44.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015be00640 1 -- start start 2024-09-10T22:00:44.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f8ba7200640 1 -- 172.21.15.66:0/3335803245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ba80734c0 msgr2=0x7f8ba80b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f8ba7200640 1 --2- 172.21.15.66:0/3335803245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ba80734c0 0x7f8ba80b42a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80af4b0 tx=0x7f8ba001c690 comp rx=0 tx=0).stop 2024-09-10T22:00:44.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015be00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0154036250 0x7f01540dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0154036630 0x7f015401ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0154055070 0x7f015401f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f01540e11e0 con 0x7f0154055070 2024-09-10T22:00:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01540e1360 con 0x7f0154036630 2024-09-10T22:00:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015be00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f01540e14e0 con 0x7f0154036250 2024-09-10T22:00:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7fe941e00640 1 -- 172.21.15.66:0/1839079013 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe93c06ed40 msgr2=0x7fe93c164c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7fe941e00640 1 --2- 172.21.15.66:0/1839079013 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe93c06ed40 0x7fe93c164c40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe934009930 tx=0x7fe93401d350 comp rx=0 tx=0).stop 2024-09-10T22:00:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f8ba7200640 1 -- 172.21.15.66:0/3335803245 shutdown_connections 2024-09-10T22:00:44.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f8ba7200640 1 --2- 172.21.15.66:0/3335803245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ba80734c0 0x7f8ba80b42a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f8ba7200640 1 --2- 172.21.15.66:0/3335803245 >> v2:172.21.15.66:3300/0 conn(0x7f8ba815b880 0x7f8ba806f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f8ba7200640 1 -- 172.21.15.66:0/3335803245 >> 172.21.15.66:0/3335803245 conn(0x7f8ba806cc20 msgr2=0x7f8ba806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0154036630 0x7f015401ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0154036630 0x7f015401ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:47164/0 (socket says 172.21.15.66:47164) 2024-09-10T22:00:44.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015a000640 1 -- 172.21.15.66:0/2281931308 learned_addr learned my addr 172.21.15.66:0/2281931308 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015aa00640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0154036250 0x7f01540dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7fe941e00640 1 -- 172.21.15.66:0/1839079013 shutdown_connections 2024-09-10T22:00:44.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7fe941e00640 1 --2- 172.21.15.66:0/1839079013 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe93c06ed40 0x7fe93c164c40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7fe941e00640 1 -- 172.21.15.66:0/1839079013 >> 172.21.15.66:0/1839079013 conn(0x7fe93c0cfa30 msgr2=0x7fe93c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015a000640 1 -- 172.21.15.66:0/2281931308 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0154036250 msgr2=0x7f01540dcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015a000640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0154036250 0x7f01540dcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015a000640 1 -- 172.21.15.66:0/2281931308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0154055070 msgr2=0x7f015401f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015a000640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0154055070 0x7f015401f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f015a000640 1 -- 172.21.15.66:0/2281931308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01540f2b80 con 0x7f0154036630 2024-09-10T22:00:44.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7fe941e00640 1 -- 172.21.15.66:0/1839079013 shutdown_connections 2024-09-10T22:00:44.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f8ba7200640 1 -- 172.21.15.66:0/3335803245 shutdown_connections 2024-09-10T22:00:44.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7f8ba7200640 1 -- 172.21.15.66:0/3335803245 wait complete. 2024-09-10T22:00:44.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f015a000640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0154036630 0x7f015401ed10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f014800e9f0 tx=0x7f014800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.101+0000 7fe941e00640 1 -- 172.21.15.66:0/1839079013 wait complete. 2024-09-10T22:00:44.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f0152a00640 1 -- 172.21.15.66:0/2281931308 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014800cd80 con 0x7f0154036630 2024-09-10T22:00:44.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f0152a00640 1 -- 172.21.15.66:0/2281931308 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0148010430 con 0x7f0154036630 2024-09-10T22:00:44.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f0152a00640 1 -- 172.21.15.66:0/2281931308 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148010860 con 0x7f0154036630 2024-09-10T22:00:44.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f37cea00640 1 -- 172.21.15.66:0/50518840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ac004810 msgr2=0x7f37ac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f37cea00640 1 --2- 172.21.15.66:0/50518840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ac004810 0x7f37ac004c90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009eb0 tx=0x7f37b801d5b0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f37cea00640 1 -- 172.21.15.66:0/50518840 shutdown_connections 2024-09-10T22:00:44.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f37cea00640 1 --2- 172.21.15.66:0/50518840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ac004810 0x7f37ac004c90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f37cea00640 1 -- 172.21.15.66:0/50518840 >> 172.21.15.66:0/50518840 conn(0x7f37c80d3e90 msgr2=0x7f37c80d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7fe941e00640 1 Processor -- start 2024-09-10T22:00:44.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7fe941e00640 1 -- start start 2024-09-10T22:00:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f8ba7200640 1 Processor -- start 2024-09-10T22:00:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f37cea00640 1 -- 172.21.15.66:0/50518840 shutdown_connections 2024-09-10T22:00:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f015be00640 1 -- 172.21.15.66:0/2281931308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01540f2e60 con 0x7f0154036630 2024-09-10T22:00:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f37cea00640 1 -- 172.21.15.66:0/50518840 wait complete. 2024-09-10T22:00:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f8ba7200640 1 -- start start 2024-09-10T22:00:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7f015be00640 1 -- 172.21.15.66:0/2281931308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01540f3420 con 0x7f0154036630 2024-09-10T22:00:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7fe941e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe93c06ed40 0x7fe93c143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7fe941e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe93c0706c0 0x7fe93c1ef290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7fe941e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe93c0c4fc0 0x7fe93c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7fe941e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c167c20 con 0x7fe93c06ed40 2024-09-10T22:00:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7fe941e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c167aa0 con 0x7fe93c0706c0 2024-09-10T22:00:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.105+0000 7fe941e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe93c167da0 con 0x7fe93c0c4fc0 2024-09-10T22:00:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f7d63200640 1 -- 172.21.15.66:0/761020334 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5c151140 msgr2=0x7f7d5c0cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f7d63200640 1 --2- 172.21.15.66:0/761020334 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5c151140 0x7f7d5c0cd510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009bf0 tx=0x7f7d5801cbe0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f0147e00640 1 -- 172.21.15.66:0/2281931308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0130001040 con 0x7f0154036630 2024-09-10T22:00:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe932000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe93c0706c0 0x7fe93c1ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe93b400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe93c0c4fc0 0x7fe93c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe93b400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe93c0c4fc0 0x7fe93c1536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:59566/0 (socket says 172.21.15.66:59566) 2024-09-10T22:00:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe932000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe93c0706c0 0x7fe93c1ef290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:47174/0 (socket says 172.21.15.66:47174) 2024-09-10T22:00:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe93b400640 1 -- 172.21.15.66:0/3540723925 learned_addr learned my addr 172.21.15.66:0/3540723925 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe93aa00640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe93c06ed40 0x7fe93c143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe93b400640 1 -- 172.21.15.66:0/3540723925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe93c0706c0 msgr2=0x7fe93c1ef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe93b400640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe93c0706c0 0x7fe93c1ef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe93b400640 1 -- 172.21.15.66:0/3540723925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe93c06ed40 msgr2=0x7fe93c143700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ba806fc20 0x7f8ba814a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ba806fc20 0x7f8ba814a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ba806fc20 0x7f8ba814a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45528/0 (socket says 172.21.15.66:45528) 2024-09-10T22:00:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba7200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ba80734c0 0x7f8ba814e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba7200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ba815b880 0x7f8ba81503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f7d63200640 1 -- 172.21.15.66:0/761020334 shutdown_connections 2024-09-10T22:00:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f7d63200640 1 --2- 172.21.15.66:0/761020334 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5c151140 0x7f7d5c0cd510 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f7d63200640 1 -- 172.21.15.66:0/761020334 >> 172.21.15.66:0/761020334 conn(0x7f7d5c0cc3c0 msgr2=0x7f7d5c0cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f37cea00640 1 Processor -- start 2024-09-10T22:00:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f37cea00640 1 -- start start 2024-09-10T22:00:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe93b400640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe93c06ed40 0x7fe93c143700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe93b400640 1 -- 172.21.15.66:0/3540723925 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c1540b0 con 0x7fe93c0c4fc0 2024-09-10T22:00:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f7d63200640 1 -- 172.21.15.66:0/761020334 shutdown_connections 2024-09-10T22:00:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe93aa00640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe93c06ed40 0x7fe93c143700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba5e00640 1 -- 172.21.15.66:0/3282986839 learned_addr learned my addr 172.21.15.66:0/3282986839 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba815bde0 con 0x7f8ba806fc20 2024-09-10T22:00:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba815bc60 con 0x7f8ba80734c0 2024-09-10T22:00:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba8158650 con 0x7f8ba815b880 2024-09-10T22:00:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba6800640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ba815b880 0x7f8ba81503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f7d63200640 1 -- 172.21.15.66:0/761020334 wait complete. 2024-09-10T22:00:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f37cea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ac004810 0x7f37c815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7fe93b400640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe93c0c4fc0 0x7fe93c1536c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe93401a520 tx=0x7fe9340041a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f37cea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c80cfe10 0x7f37c81fb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f37cea00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c80b6ce0 0x7f37c80b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f37cea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f37c80d2120 con 0x7f37ac004810 2024-09-10T22:00:44.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f37cea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f37c80d1fa0 con 0x7f37c80cfe10 2024-09-10T22:00:44.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f37cea00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f37c80d22a0 con 0x7f37c80b6ce0 2024-09-10T22:00:44.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba6800640 1 -- 172.21.15.66:0/3282986839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ba80734c0 msgr2=0x7f8ba814e780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:00:44.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba6800640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ba80734c0 0x7f8ba814e780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba6800640 1 -- 172.21.15.66:0/3282986839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ba806fc20 msgr2=0x7f8ba814a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f4f41800640 1 -- 172.21.15.66:0/1694606694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f3c161be0 msgr2=0x7f4f3c161fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f4f41800640 1 --2- 172.21.15.66:0/1694606694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f3c161be0 0x7f4f3c161fc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c08c820 tx=0x7f4f2c01c8c0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba6800640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ba806fc20 0x7f8ba814a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f8ba6800640 1 -- 172.21.15.66:0/3282986839 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba814dec0 con 0x7f8ba815b880 2024-09-10T22:00:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f4f41800640 1 -- 172.21.15.66:0/1694606694 shutdown_connections 2024-09-10T22:00:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f4f41800640 1 --2- 172.21.15.66:0/1694606694 >> v2:172.21.15.94:3300/0 conn(0x7f4f3c15e2c0 0x7f4f3c162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f4f41800640 1 --2- 172.21.15.66:0/1694606694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f3c161be0 0x7f4f3c161fc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f4f41800640 1 -- 172.21.15.66:0/1694606694 >> 172.21.15.66:0/1694606694 conn(0x7f4f3c0d3f80 msgr2=0x7f4f3c0d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f37cca00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c80b6ce0 0x7f37c80b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f37cca00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c80b6ce0 0x7f37c80b7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:59590/0 (socket says 172.21.15.66:59590) 2024-09-10T22:00:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f37cca00640 1 -- 172.21.15.66:0/3381581159 learned_addr learned my addr 172.21.15.66:0/3381581159 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f0152a00640 1 -- 172.21.15.66:0/2281931308 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0148010a00 con 0x7f0154036630 2024-09-10T22:00:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f37c7e00640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ac004810 0x7f37c815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f7d63200640 1 Processor -- start 2024-09-10T22:00:44.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7fe933e00640 1 -- 172.21.15.66:0/3540723925 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9340b2040 con 0x7fe93c0c4fc0 2024-09-10T22:00:44.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7fe933e00640 1 -- 172.21.15.66:0/3540723925 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe934003600 con 0x7fe93c0c4fc0 2024-09-10T22:00:44.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7fe933e00640 1 -- 172.21.15.66:0/3540723925 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934003a50 con 0x7fe93c0c4fc0 2024-09-10T22:00:44.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f8ba6800640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ba815b880 0x7f8ba81503b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00ac000 tx=0x7f8ba0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f37cca00640 1 -- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c80cfe10 msgr2=0x7f37c81fb500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:00:44.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f37cca00640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c80cfe10 0x7f37c81fb500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f37cca00640 1 -- 172.21.15.66:0/3381581159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ac004810 msgr2=0x7f37c815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f37cca00640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ac004810 0x7f37c815c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f37cca00640 1 -- 172.21.15.66:0/3381581159 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c80ca180 con 0x7f37c80b6ce0 2024-09-10T22:00:44.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f7d63200640 1 -- start start 2024-09-10T22:00:44.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f37cca00640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c80b6ce0 0x7f37c80b7160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f37c80c76a0 tx=0x7f37c0001640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.109+0000 7f4f41800640 1 -- 172.21.15.66:0/1694606694 shutdown_connections 2024-09-10T22:00:44.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f4f41800640 1 -- 172.21.15.66:0/1694606694 wait complete. 2024-09-10T22:00:44.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f0152a00640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f013c0725e0 0x7f013c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f0152a00640 1 -- 172.21.15.66:0/2281931308 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0148093e40 con 0x7f0154036630 2024-09-10T22:00:44.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f0152a00640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] conn(0x7f013c07a860 0x7f013c07cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f0152a00640 1 -- 172.21.15.66:0/2281931308 --> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f014801ee50 con 0x7f013c07a860 2024-09-10T22:00:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f37bfe00640 1 -- 172.21.15.66:0/3381581159 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c00107d0 con 0x7f37c80b6ce0 2024-09-10T22:00:44.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f0152a00640 1 -- 172.21.15.66:0/2281931308 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f01480964a0 con 0x7f0154036630 2024-09-10T22:00:44.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7fe941e00640 1 -- 172.21.15.66:0/3540723925 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c0b2ca0 con 0x7fe93c0c4fc0 2024-09-10T22:00:44.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7fe941e00640 1 -- 172.21.15.66:0/3540723925 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93c0b3260 con 0x7fe93c0c4fc0 2024-09-10T22:00:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f37bfe00640 1 -- 172.21.15.66:0/3381581159 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37c0016070 con 0x7f37c80b6ce0 2024-09-10T22:00:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f37bfe00640 1 -- 172.21.15.66:0/3381581159 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0010ce0 con 0x7f37c80b6ce0 2024-09-10T22:00:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f015aa00640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f013c0725e0 0x7f013c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f015b400640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] conn(0x7f013c07a860 0x7f013c07cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f8b9e000640 1 -- 172.21.15.66:0/3282986839 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0005230 con 0x7f8ba815b880 2024-09-10T22:00:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7fe941e00640 1 -- 172.21.15.66:0/3540723925 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe93c0c48b0 con 0x7fe93c0c4fc0 2024-09-10T22:00:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba814d620 con 0x7f8ba815b880 2024-09-10T22:00:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba814dbe0 con 0x7f8ba815b880 2024-09-10T22:00:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f8b9e000640 1 -- 172.21.15.66:0/3282986839 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ba00be070 con 0x7f8ba815b880 2024-09-10T22:00:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f4f41800640 1 Processor -- start 2024-09-10T22:00:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f4f41800640 1 -- start start 2024-09-10T22:00:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f37cea00640 1 -- 172.21.15.66:0/3381581159 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c80ca460 con 0x7f37c80b6ce0 2024-09-10T22:00:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f37cea00640 1 -- 172.21.15.66:0/3381581159 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c815bfb0 con 0x7f37c80b6ce0 2024-09-10T22:00:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f7d63200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c06fbd0 0x7f7d5c145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f7d63200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5c0ae9a0 0x7f7d5c15cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f7d63200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c151140 0x7f7d5c14b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f7d63200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c170b20 con 0x7f7d5c06fbd0 2024-09-10T22:00:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f7d63200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c1709a0 con 0x7f7d5c151140 2024-09-10T22:00:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f7d63200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5c170ca0 con 0x7f7d5c0ae9a0 2024-09-10T22:00:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.113+0000 7f8b9e000640 1 -- 172.21.15.66:0/3282986839 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00046d0 con 0x7f8ba815b880 2024-09-10T22:00:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d61400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5c0ae9a0 0x7f7d5c15cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f37cea00640 1 -- 172.21.15.66:0/3381581159 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37c80d2800 con 0x7f37c80b6ce0 2024-09-10T22:00:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d62800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c151140 0x7f7d5c14b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d62800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c151140 0x7f7d5c14b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:47212/0 (socket says 172.21.15.66:47212) 2024-09-10T22:00:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d62800640 1 -- 172.21.15.66:0/4109073556 learned_addr learned my addr 172.21.15.66:0/4109073556 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f015b400640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] conn(0x7f013c07a860 0x7f013c07cc80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f015aa00640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f013c0725e0 0x7f013c074aa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f014c0ae000 tx=0x7f014c01af60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b80001040 con 0x7f8ba815b880 2024-09-10T22:00:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d61e00640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c06fbd0 0x7f7d5c145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f41800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4f3c15e2c0 0x7f4f3c1494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d62800640 1 -- 172.21.15.66:0/4109073556 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5c0ae9a0 msgr2=0x7f7d5c15cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d62800640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5c0ae9a0 0x7f7d5c15cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d62800640 1 -- 172.21.15.66:0/4109073556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c06fbd0 msgr2=0x7f7d5c145f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d62800640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c06fbd0 0x7f7d5c145f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d62800640 1 -- 172.21.15.66:0/4109073556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c152900 con 0x7f7d5c151140 2024-09-10T22:00:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f41800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c161be0 0x7f4f3c0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f41800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f3c0b96e0 0x7f4f3c1ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3aa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4f3c15e2c0 0x7f4f3c1494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3aa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4f3c15e2c0 0x7f4f3c1494f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:59614/0 (socket says 172.21.15.66:59614) 2024-09-10T22:00:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d62800640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c151140 0x7f7d5c14b3f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00c3c0 tx=0x7f7d4c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c161be0 0x7f4f3c0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f41800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c16f920 con 0x7f4f3c0b96e0 2024-09-10T22:00:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f41800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c0b19e0 con 0x7f4f3c161be0 2024-09-10T22:00:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f41800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c16faa0 con 0x7f4f3c15e2c0 2024-09-10T22:00:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c161be0 0x7f4f3c0ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:47222/0 (socket says 172.21.15.66:47222) 2024-09-10T22:00:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3a000640 1 -- 172.21.15.66:0/3226635933 learned_addr learned my addr 172.21.15.66:0/3226635933 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3b400640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f3c0b96e0 0x7f4f3c1ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f0152a00640 1 -- 172.21.15.66:0/2281931308 <== osd.0 v2:172.21.15.66:6816/261900486 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f014801ee50 con 0x7f013c07a860 2024-09-10T22:00:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3a000640 1 -- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4f3c15e2c0 msgr2=0x7f4f3c1494f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3a000640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4f3c15e2c0 0x7f4f3c1494f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3a000640 1 -- 172.21.15.66:0/3226635933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f3c0b96e0 msgr2=0x7f4f3c1ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3a000640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f3c0b96e0 0x7f4f3c1ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3a000640 1 -- 172.21.15.66:0/3226635933 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c15ebd0 con 0x7f4f3c161be0 2024-09-10T22:00:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3a000640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c161be0 0x7f4f3c0ceaf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800b4a0 tx=0x7f4f2800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7fe933e00640 1 -- 172.21.15.66:0/3540723925 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe9340b5070 con 0x7fe93c0c4fc0 2024-09-10T22:00:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d56000640 1 -- 172.21.15.66:0/4109073556 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c011790 con 0x7f7d5c151140 2024-09-10T22:00:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f37bfe00640 1 -- 172.21.15.66:0/3381581159 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f37c0037b30 con 0x7f37c80b6ce0 2024-09-10T22:00:44.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f3b400640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f3c0b96e0 0x7f4f3c1ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:00:44.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f7d56000640 1 -- 172.21.15.66:0/4109073556 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d4c017070 con 0x7f7d5c151140 2024-09-10T22:00:44.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7fe933e00640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe9180726b0 0x7fe918074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f32a00640 1 -- 172.21.15.66:0/3226635933 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28014070 con 0x7f4f3c161be0 2024-09-10T22:00:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f32a00640 1 -- 172.21.15.66:0/3226635933 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f28010720 con 0x7f4f3c161be0 2024-09-10T22:00:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f32a00640 1 -- 172.21.15.66:0/3226635933 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28010b70 con 0x7f4f3c161be0 2024-09-10T22:00:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f8b9e000640 1 -- 172.21.15.66:0/3282986839 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8ba00dfb30 con 0x7f8ba815b880 2024-09-10T22:00:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7fe933e00640 1 -- 172.21.15.66:0/3540723925 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe93412cdc0 con 0x7fe93c0c4fc0 2024-09-10T22:00:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.117+0000 7f4f41800640 1 -- 172.21.15.66:0/3226635933 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3c15e8f0 con 0x7f4f3c161be0 2024-09-10T22:00:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7fe933e00640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] conn(0x7fe91807a8d0 0x7fe91807ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7fe933e00640 1 -- 172.21.15.66:0/3540723925 --> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9340a9ec0 con 0x7fe91807a8d0 2024-09-10T22:00:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7fe933e00640 1 -- 172.21.15.66:0/3540723925 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe93412d1b0 con 0x7fe93c0c4fc0 2024-09-10T22:00:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f7d56000640 1 -- 172.21.15.66:0/4109073556 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c011ce0 con 0x7f7d5c151140 2024-09-10T22:00:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f7d63200640 1 -- 172.21.15.66:0/4109073556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5c152be0 con 0x7f7d5c151140 2024-09-10T22:00:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f7d63200640 1 -- 172.21.15.66:0/4109073556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5c1531a0 con 0x7f7d5c151140 2024-09-10T22:00:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f4f41800640 1 -- 172.21.15.66:0/3226635933 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3c0cf030 con 0x7f4f3c161be0 2024-09-10T22:00:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f8b9e000640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8b840727e0 0x7f8b84074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f8b9e000640 1 -- 172.21.15.66:0/3282986839 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8ba00054c0 con 0x7f8ba815b880 2024-09-10T22:00:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7fe93aa00640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe9180726b0 0x7fe918074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f8b9e000640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] conn(0x7f8b8407a980 0x7f8b8407cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f8b9e000640 1 -- 172.21.15.66:0/3282986839 --> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ba000b0b0 con 0x7f8b8407a980 2024-09-10T22:00:44.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f8b9e000640 1 -- 172.21.15.66:0/3282986839 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8ba012e0c0 con 0x7f8ba815b880 2024-09-10T22:00:44.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f8ba5e00640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8b840727e0 0x7f8b84074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f37bfe00640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f37a40726e0 0x7f37a4074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f8ba5400640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] conn(0x7f8b8407a980 0x7f8b8407cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f8ba5e00640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8b840727e0 0x7f8b84074ca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b98005c50 tx=0x7f8b98009780 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7fe932000640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] conn(0x7fe91807a8d0 0x7fe91807ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7fe932000640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] conn(0x7fe91807a8d0 0x7fe91807ccf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f8ba5400640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] conn(0x7f8b8407a980 0x7f8b8407cda0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f7d63200640 1 -- 172.21.15.66:0/4109073556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d38001040 con 0x7f7d5c151140 2024-09-10T22:00:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f37bfe00640 1 -- 172.21.15.66:0/3381581159 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f37c0093640 con 0x7f37c80b6ce0 2024-09-10T22:00:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7f37bfe00640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] conn(0x7f37a407a960 0x7f37a407cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f37bfe00640 1 -- 172.21.15.66:0/3381581159 --> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37a407d430 con 0x7f37a407a960 2024-09-10T22:00:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f37bfe00640 1 -- 172.21.15.66:0/3381581159 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f37c0093a30 con 0x7f37c80b6ce0 2024-09-10T22:00:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7ffa2ec00640 1 -- 172.21.15.66:0/1201120341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa280cb8a0 msgr2=0x7ffa280cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.121+0000 7ffa2ec00640 1 --2- 172.21.15.66:0/1201120341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa280cb8a0 0x7ffa280cbcd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa20009bc0 tx=0x7ffa2001cba0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7ffa2ec00640 1 -- 172.21.15.66:0/1201120341 shutdown_connections 2024-09-10T22:00:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7ffa2ec00640 1 --2- 172.21.15.66:0/1201120341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa280cb8a0 0x7ffa280cbcd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7ffa2ec00640 1 -- 172.21.15.66:0/1201120341 >> 172.21.15.66:0/1201120341 conn(0x7ffa280cfa30 msgr2=0x7ffa280cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7ffa2ec00640 1 -- 172.21.15.66:0/1201120341 shutdown_connections 2024-09-10T22:00:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7ffa2ec00640 1 -- 172.21.15.66:0/1201120341 wait complete. 2024-09-10T22:00:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7fe93aa00640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe9180726b0 0x7fe918074b70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0012f0 tx=0x7fe92c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f37c7e00640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f37a40726e0 0x7f37a4074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f37c7400640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] conn(0x7f37a407a960 0x7f37a407cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f37c7400640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] conn(0x7f37a407a960 0x7f37a407cd80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f7d56000640 1 -- 172.21.15.66:0/4109073556 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7d4c01c460 con 0x7f7d5c151140 2024-09-10T22:00:44.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7fe933e00640 1 -- 172.21.15.66:0/3540723925 <== osd.1 v2:172.21.15.66:6808/1441775173 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9340a9ec0 con 0x7fe91807a8d0 2024-09-10T22:00:44.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f4f32a00640 1 -- 172.21.15.66:0/3226635933 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4f28017360 con 0x7f4f3c161be0 2024-09-10T22:00:44.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f4f41800640 1 -- 172.21.15.66:0/3226635933 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f3c15e6a0 con 0x7f4f3c161be0 2024-09-10T22:00:44.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f8b9e000640 1 -- 172.21.15.66:0/3282986839 <== osd.2 v2:172.21.15.66:6800/3422283261 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ba000b0b0 con 0x7f8b8407a980 2024-09-10T22:00:44.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7ffa2ec00640 1 Processor -- start 2024-09-10T22:00:44.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f7d56000640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7d400726b0 0x7f7d40074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f7d56000640 1 -- 172.21.15.66:0/4109073556 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7d4c092ef0 con 0x7f7d5c151140 2024-09-10T22:00:44.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f7d56000640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] conn(0x7f7d4007a8d0 0x7f7d4007ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f7d56000640 1 -- 172.21.15.66:0/4109073556 --> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d4c009e50 con 0x7f7d4007a8d0 2024-09-10T22:00:44.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f7d56000640 1 -- 172.21.15.66:0/4109073556 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7d4c095550 con 0x7f7d5c151140 2024-09-10T22:00:44.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7ffa2ec00640 1 -- start start 2024-09-10T22:00:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f4f32a00640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4f1c072630 0x7f4f1c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f4f32a00640 1 -- 172.21.15.66:0/3226635933 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4f28092ea0 con 0x7f4f3c161be0 2024-09-10T22:00:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f4f32a00640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] conn(0x7f4f1c07a850 0x7f4f1c07cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f4f32a00640 1 -- 172.21.15.66:0/3226635933 --> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f28018ec0 con 0x7f4f1c07a850 2024-09-10T22:00:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f4f32a00640 1 -- 172.21.15.66:0/3226635933 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4f280955f0 con 0x7f4f3c161be0 2024-09-10T22:00:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.125+0000 7f37bfe00640 1 -- 172.21.15.66:0/3381581159 <== osd.5 v2:172.21.15.94:6802/1528499896 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37a407d430 con 0x7f37a407a960 2024-09-10T22:00:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7f37c7e00640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f37a40726e0 0x7f37a4074ba0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f37b0000fc0 tx=0x7f37b0006d50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7f7d61e00640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7d400726b0 0x7f7d40074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7f4f3aa00640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4f1c072630 0x7f4f1c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa2ec00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa280ae620 0x7ffa28157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa2ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa280c4b10 0x7ffa2814ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa2ec00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa280cb8a0 0x7ffa28157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa2ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2806f500 con 0x7ffa280c4b10 2024-09-10T22:00:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa2ec00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2806f380 con 0x7ffa280cb8a0 2024-09-10T22:00:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa2ec00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2806f680 con 0x7ffa280ae620 2024-09-10T22:00:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7f7d61400640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] conn(0x7f7d4007a8d0 0x7f7d4007ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7f4f3b400640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] conn(0x7f4f1c07a850 0x7f4f1c07cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa280ae620 0x7ffa28157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa1f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa280c4b10 0x7ffa2814ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa1f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa280c4b10 0x7ffa2814ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45598/0 (socket says 172.21.15.66:45598) 2024-09-10T22:00:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa280ae620 0x7ffa28157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:59638/0 (socket says 172.21.15.66:59638) 2024-09-10T22:00:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa27e00640 1 -- 172.21.15.66:0/3747579156 learned_addr learned my addr 172.21.15.66:0/3747579156 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7f4f3aa00640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4f1c072630 0x7f4f1c074af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0af000 tx=0x7f4f2c01b100 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7f7d61e00640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7d400726b0 0x7f7d40074b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d580af000 tx=0x7f7d5801b420 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa27e00640 1 -- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa280cb8a0 msgr2=0x7ffa28157620 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:00:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa27e00640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa280cb8a0 0x7ffa28157620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7f7d61400640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] conn(0x7f7d4007a8d0 0x7f7d4007ccf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7f4f3b400640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] conn(0x7f4f1c07a850 0x7f4f1c07cc70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa27e00640 1 -- 172.21.15.66:0/3747579156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa280c4b10 msgr2=0x7ffa2814ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa27e00640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa280c4b10 0x7ffa2814ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa27e00640 1 -- 172.21.15.66:0/3747579156 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa28155c00 con 0x7ffa280ae620 2024-09-10T22:00:44.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa1f400640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa280c4b10 0x7ffa2814ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:44.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa27e00640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa280ae620 0x7ffa28157040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800c490 tx=0x7ffa18001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7f4f32a00640 1 -- 172.21.15.66:0/3226635933 <== osd.4 v2:172.21.15.94:6818/496135754 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f28018ec0 con 0x7f4f1c07a850 2024-09-10T22:00:44.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa24c00640 1 -- 172.21.15.66:0/3747579156 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18017070 con 0x7ffa280ae620 2024-09-10T22:00:44.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.129+0000 7ffa24c00640 1 -- 172.21.15.66:0/3747579156 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa18003600 con 0x7ffa280ae620 2024-09-10T22:00:44.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.133+0000 7f7d56000640 1 -- 172.21.15.66:0/4109073556 <== osd.3 v2:172.21.15.66:6824/4172584223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d4c009e50 con 0x7f7d4007a8d0 2024-09-10T22:00:44.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.133+0000 7ffa24c00640 1 -- 172.21.15.66:0/3747579156 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18003a90 con 0x7ffa280ae620 2024-09-10T22:00:44.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.133+0000 7ffa2ec00640 1 -- 172.21.15.66:0/3747579156 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa28155640 con 0x7ffa280ae620 2024-09-10T22:00:44.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.133+0000 7ffa2ec00640 1 -- 172.21.15.66:0/3747579156 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa281ff370 con 0x7ffa280ae620 2024-09-10T22:00:44.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.133+0000 7fa0e8200640 1 -- 172.21.15.66:0/3905536042 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0c8004810 msgr2=0x7fa0c8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.133+0000 7fa0e8200640 1 --2- 172.21.15.66:0/3905536042 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0c8004810 0x7fa0c8004c90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400a540 tx=0x7fa0d401ead0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.133+0000 7fa0e8200640 1 -- 172.21.15.66:0/3905536042 shutdown_connections 2024-09-10T22:00:44.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.133+0000 7fa0e8200640 1 --2- 172.21.15.66:0/3905536042 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0c8004810 0x7fa0c8004c90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.133+0000 7fa0e8200640 1 -- 172.21.15.66:0/3905536042 >> 172.21.15.66:0/3905536042 conn(0x7fa0e00cfa30 msgr2=0x7fa0e00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.133+0000 7fa0e8200640 1 -- 172.21.15.66:0/3905536042 shutdown_connections 2024-09-10T22:00:44.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.133+0000 7fa0e8200640 1 -- 172.21.15.66:0/3905536042 wait complete. 2024-09-10T22:00:44.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.133+0000 7ffa2ec00640 1 -- 172.21.15.66:0/3747579156 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa2806fb80 con 0x7ffa280ae620 2024-09-10T22:00:44.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7ffa24c00640 1 -- 172.21.15.66:0/3747579156 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ffa18033ce0 con 0x7ffa280ae620 2024-09-10T22:00:44.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7fa0e8200640 1 Processor -- start 2024-09-10T22:00:44.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7fa0e8200640 1 -- start start 2024-09-10T22:00:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7ffa24c00640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa040726e0 0x7ffa04074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7ffa24c00640 1 -- 172.21.15.66:0/3747579156 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ffa180927c0 con 0x7ffa280ae620 2024-09-10T22:00:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7ffa24c00640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] conn(0x7ffa0407a880 0x7ffa0407cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7ffa24c00640 1 -- 172.21.15.66:0/3747579156 --> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa0407d350 con 0x7ffa0407a880 2024-09-10T22:00:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7ffa1f400640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa040726e0 0x7ffa04074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7fa0e8200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0c8004810 0x7fa0e00b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7ffa2cc00640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] conn(0x7ffa0407a880 0x7ffa0407cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7fa0e8200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0e015c3f0 0x7fa0e01fac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7fa0e8200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0e015c870 0x7fa0e01fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7fa0e8200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e015a890 con 0x7fa0e015c870 2024-09-10T22:00:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7fa0e8200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e015a710 con 0x7fa0e015c3f0 2024-09-10T22:00:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.137+0000 7fa0e8200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e015aa10 con 0x7fa0c8004810 2024-09-10T22:00:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e5800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0c8004810 0x7fa0e00b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e5800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0c8004810 0x7fa0e00b53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:59640/0 (socket says 172.21.15.66:59640) 2024-09-10T22:00:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e5800640 1 -- 172.21.15.66:0/2775538774 learned_addr learned my addr 172.21.15.66:0/2775538774 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e6200640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0e015c870 0x7fa0e01fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e4e00640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0e015c3f0 0x7fa0e01fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e5800640 1 -- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0e015c3f0 msgr2=0x7fa0e01fac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e5800640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0e015c3f0 0x7fa0e01fac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e5800640 1 -- 172.21.15.66:0/2775538774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0e015c870 msgr2=0x7fa0e01fb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e5800640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0e015c870 0x7fa0e01fb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e5800640 1 -- 172.21.15.66:0/2775538774 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e0160e90 con 0x7fa0c8004810 2024-09-10T22:00:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7ffa1f400640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa040726e0 0x7ffa04074ba0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffa10007a60 tx=0x7ffa10009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7ffa2cc00640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] conn(0x7ffa0407a880 0x7ffa0407cca0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e5800640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0c8004810 0x7fa0e00b53c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc001410 tx=0x7fa0cc00ea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7ffa24c00640 1 -- 172.21.15.66:0/3747579156 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ffa18099050 con 0x7ffa280ae620 2024-09-10T22:00:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0dd600640 1 -- 172.21.15.66:0/2775538774 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc015070 con 0x7fa0c8004810 2024-09-10T22:00:44.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0dd600640 1 -- 172.21.15.66:0/2775538774 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0cc00c9c0 con 0x7fa0c8004810 2024-09-10T22:00:44.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0dd600640 1 -- 172.21.15.66:0/2775538774 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00ce10 con 0x7fa0c8004810 2024-09-10T22:00:44.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7ffa24c00640 1 -- 172.21.15.66:0/3747579156 <== osd.7 v2:172.21.15.94:6826/1972820213 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa0407d350 con 0x7ffa0407a880 2024-09-10T22:00:44.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e8200640 1 -- 172.21.15.66:0/2775538774 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e01544f0 con 0x7fa0c8004810 2024-09-10T22:00:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.141+0000 7fa0e8200640 1 -- 172.21.15.66:0/2775538774 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e0204f20 con 0x7fa0c8004810 2024-09-10T22:00:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.145+0000 7fa0e8200640 1 -- 172.21.15.66:0/2775538774 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0e015af70 con 0x7fa0c8004810 2024-09-10T22:00:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.145+0000 7fa0dd600640 1 -- 172.21.15.66:0/2775538774 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa0cc011460 con 0x7fa0c8004810 2024-09-10T22:00:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.149+0000 7fa0dd600640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0c00725e0 0x7fa0c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.149+0000 7fa0dd600640 1 -- 172.21.15.66:0/2775538774 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa0cc092570 con 0x7fa0c8004810 2024-09-10T22:00:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.149+0000 7fa0e4e00640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0c00725e0 0x7fa0c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.149+0000 7fa0dd600640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] conn(0x7fa0c007b460 0x7fa0c007d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.149+0000 7fa0dd600640 1 -- 172.21.15.66:0/2775538774 --> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0c007dfd0 con 0x7fa0c007b460 2024-09-10T22:00:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.149+0000 7fa0dd600640 1 -- 172.21.15.66:0/2775538774 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa0cc05b7b0 con 0x7fa0c8004810 2024-09-10T22:00:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.149+0000 7fa0e4e00640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0c00725e0 0x7fa0c0074aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40b0000 tx=0x7fa0d401b870 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.149+0000 7fa0e6200640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] conn(0x7fa0c007b460 0x7fa0c007d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.149+0000 7fa0e6200640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] conn(0x7fa0c007b460 0x7fa0c007d900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7f0147e00640 1 -- 172.21.15.66:0/2281931308 --> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0130002de0 con 0x7f013c07a860 2024-09-10T22:00:44.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7fa0dd600640 1 -- 172.21.15.66:0/2775538774 <== osd.6 v2:172.21.15.94:6810/47260656 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa0c007dfd0 con 0x7fa0c007b460 2024-09-10T22:00:44.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7fe941e00640 1 -- 172.21.15.66:0/3540723925 --> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe93c0cc010 con 0x7fe91807a8d0 2024-09-10T22:00:44.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7fe933e00640 1 -- 172.21.15.66:0/3540723925 <== osd.1 v2:172.21.15.66:6808/1441775173 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe93c0cc010 con 0x7fe91807a8d0 2024-09-10T22:00:44.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7f0152a00640 1 -- 172.21.15.66:0/2281931308 <== osd.0 v2:172.21.15.66:6816/261900486 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0130002de0 con 0x7f013c07a860 2024-09-10T22:00:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7fe941e00640 1 -- 172.21.15.66:0/3540723925 >> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] conn(0x7fe91807a8d0 msgr2=0x7fe91807ccf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7fe941e00640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] conn(0x7fe91807a8d0 0x7fe91807ccf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7fe941e00640 1 -- 172.21.15.66:0/3540723925 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe9180726b0 msgr2=0x7fe918074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7f0147e00640 1 -- 172.21.15.66:0/2281931308 >> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] conn(0x7f013c07a860 msgr2=0x7f013c07cc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7f0147e00640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] conn(0x7f013c07a860 0x7f013c07cc80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f0147e00640 1 -- 172.21.15.66:0/2281931308 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f013c0725e0 msgr2=0x7f013c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f0147e00640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f013c0725e0 0x7f013c074aa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f014c0ae000 tx=0x7f014c01af60 comp rx=0 tx=0).stop 2024-09-10T22:00:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f0147e00640 1 -- 172.21.15.66:0/2281931308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0154036630 msgr2=0x7f015401ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f0147e00640 1 --2- 172.21.15.66:0/2281931308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0154036630 0x7f015401ed10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f014800e9f0 tx=0x7f014800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f015b400640 1 -- 172.21.15.66:0/2281931308 reap_dead start 2024-09-10T22:00:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7fe941e00640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe9180726b0 0x7fe918074b70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0012f0 tx=0x7fe92c009040 comp rx=0 tx=0).stop 2024-09-10T22:00:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7fe941e00640 1 -- 172.21.15.66:0/3540723925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe93c0c4fc0 msgr2=0x7fe93c1536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.153+0000 7fe941e00640 1 --2- 172.21.15.66:0/3540723925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe93c0c4fc0 0x7fe93c1536c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe93401a520 tx=0x7fe9340041a0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f41800640 1 -- 172.21.15.66:0/3226635933 --> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f3c161fc0 con 0x7f4f1c07a850 2024-09-10T22:00:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f0147e00640 1 -- 172.21.15.66:0/2281931308 shutdown_connections 2024-09-10T22:00:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f0147e00640 1 -- 172.21.15.66:0/2281931308 >> 172.21.15.66:0/2281931308 conn(0x7f015401d780 msgr2=0x7f015400abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7fe93b400640 1 -- 172.21.15.66:0/3540723925 reap_dead start 2024-09-10T22:00:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7fe941e00640 1 -- 172.21.15.66:0/3540723925 shutdown_connections 2024-09-10T22:00:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7fe941e00640 1 -- 172.21.15.66:0/3540723925 >> 172.21.15.66:0/3540723925 conn(0x7fe93c0cfa30 msgr2=0x7fe93c15d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f32a00640 1 -- 172.21.15.66:0/3226635933 <== osd.4 v2:172.21.15.94:6818/496135754 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4f3c161fc0 con 0x7f4f1c07a850 2024-09-10T22:00:44.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7fe941e00640 1 -- 172.21.15.66:0/3540723925 shutdown_connections 2024-09-10T22:00:44.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7fe941e00640 1 -- 172.21.15.66:0/3540723925 wait complete. 2024-09-10T22:00:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f41800640 1 -- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] conn(0x7f4f1c07a850 msgr2=0x7f4f1c07cc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f41800640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] conn(0x7f4f1c07a850 0x7f4f1c07cc70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f41800640 1 -- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4f1c072630 msgr2=0x7f4f1c074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f41800640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4f1c072630 0x7f4f1c074af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0af000 tx=0x7f4f2c01b100 comp rx=0 tx=0).stop 2024-09-10T22:00:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f41800640 1 -- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c161be0 msgr2=0x7f4f3c0ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f41800640 1 --2- 172.21.15.66:0/3226635933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c161be0 0x7f4f3c0ceaf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800b4a0 tx=0x7f4f2800b960 comp rx=0 tx=0).stop 2024-09-10T22:00:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f3b400640 1 -- 172.21.15.66:0/3226635933 reap_dead start 2024-09-10T22:00:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f41800640 1 -- 172.21.15.66:0/3226635933 shutdown_connections 2024-09-10T22:00:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f41800640 1 -- 172.21.15.66:0/3226635933 >> 172.21.15.66:0/3226635933 conn(0x7f4f3c0d3f80 msgr2=0x7f4f3c06e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f41800640 1 -- 172.21.15.66:0/3226635933 shutdown_connections 2024-09-10T22:00:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f4f41800640 1 -- 172.21.15.66:0/3226635933 wait complete. 2024-09-10T22:00:44.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f0147e00640 1 -- 172.21.15.66:0/2281931308 shutdown_connections 2024-09-10T22:00:44.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f0147e00640 1 -- 172.21.15.66:0/2281931308 wait complete. 2024-09-10T22:00:44.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.157+0000 7f37cea00640 1 -- 172.21.15.66:0/3381581159 --> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37c80c73a0 con 0x7f37a407a960 2024-09-10T22:00:44.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.161+0000 7f37bfe00640 1 -- 172.21.15.66:0/3381581159 <== osd.5 v2:172.21.15.94:6802/1528499896 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f37c80c73a0 con 0x7f37a407a960 2024-09-10T22:00:44.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.161+0000 7f7d63200640 1 -- 172.21.15.66:0/4109073556 --> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7d38002de0 con 0x7f7d4007a8d0 2024-09-10T22:00:44.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.161+0000 7f7d56000640 1 -- 172.21.15.66:0/4109073556 <== osd.3 v2:172.21.15.66:6824/4172584223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7d38002de0 con 0x7f7d4007a8d0 2024-09-10T22:00:44.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.161+0000 7f37cea00640 1 -- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] conn(0x7f37a407a960 msgr2=0x7f37a407cd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.161+0000 7f37cea00640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] conn(0x7f37a407a960 0x7f37a407cd80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.161+0000 7f37cea00640 1 -- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f37a40726e0 msgr2=0x7f37a4074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.161+0000 7f37cea00640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f37a40726e0 0x7f37a4074ba0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f37b0000fc0 tx=0x7f37b0006d50 comp rx=0 tx=0).stop 2024-09-10T22:00:44.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.161+0000 7f37cea00640 1 -- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c80b6ce0 msgr2=0x7f37c80b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.161+0000 7f37cea00640 1 --2- 172.21.15.66:0/3381581159 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c80b6ce0 0x7f37c80b7160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f37c80c76a0 tx=0x7f37c0001640 comp rx=0 tx=0).stop 2024-09-10T22:00:44.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.161+0000 7f37cca00640 1 -- 172.21.15.66:0/3381581159 reap_dead start 2024-09-10T22:00:44.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f7d63200640 1 -- 172.21.15.66:0/4109073556 >> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] conn(0x7f7d4007a8d0 msgr2=0x7f7d4007ccf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f7d63200640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] conn(0x7f7d4007a8d0 0x7f7d4007ccf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f37cea00640 1 -- 172.21.15.66:0/3381581159 shutdown_connections 2024-09-10T22:00:44.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f37cea00640 1 -- 172.21.15.66:0/3381581159 >> 172.21.15.66:0/3381581159 conn(0x7f37c80d3e90 msgr2=0x7f37c806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f7d63200640 1 -- 172.21.15.66:0/4109073556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7d400726b0 msgr2=0x7f7d40074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f7d63200640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7d400726b0 0x7f7d40074b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d580af000 tx=0x7f7d5801b420 comp rx=0 tx=0).stop 2024-09-10T22:00:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f7d63200640 1 -- 172.21.15.66:0/4109073556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c151140 msgr2=0x7f7d5c14b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f7d63200640 1 --2- 172.21.15.66:0/4109073556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c151140 0x7f7d5c14b3f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00c3c0 tx=0x7f7d4c00c880 comp rx=0 tx=0).stop 2024-09-10T22:00:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f37cea00640 1 -- 172.21.15.66:0/3381581159 shutdown_connections 2024-09-10T22:00:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f37cea00640 1 -- 172.21.15.66:0/3381581159 wait complete. 2024-09-10T22:00:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 --> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b80002de0 con 0x7f8b8407a980 2024-09-10T22:00:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f7d62800640 1 -- 172.21.15.66:0/4109073556 reap_dead start 2024-09-10T22:00:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f7d63200640 1 -- 172.21.15.66:0/4109073556 shutdown_connections 2024-09-10T22:00:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f7d63200640 1 -- 172.21.15.66:0/4109073556 >> 172.21.15.66:0/4109073556 conn(0x7f7d5c0cc3c0 msgr2=0x7f7d5c0cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f7d63200640 1 -- 172.21.15.66:0/4109073556 shutdown_connections 2024-09-10T22:00:44.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f7d63200640 1 -- 172.21.15.66:0/4109073556 wait complete. 2024-09-10T22:00:44.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.165+0000 7f8b9e000640 1 -- 172.21.15.66:0/3282986839 <== osd.2 v2:172.21.15.66:6800/3422283261 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8b80002de0 con 0x7f8b8407a980 2024-09-10T22:00:44.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.173+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 >> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] conn(0x7f8b8407a980 msgr2=0x7f8b8407cda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.173+0000 7f8ba7200640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] conn(0x7f8b8407a980 0x7f8b8407cda0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.173+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8b840727e0 msgr2=0x7f8b84074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.173+0000 7f8ba7200640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8b840727e0 0x7f8b84074ca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b98005c50 tx=0x7f8b98009780 comp rx=0 tx=0).stop 2024-09-10T22:00:44.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.173+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ba815b880 msgr2=0x7f8ba81503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.173+0000 7f8ba7200640 1 --2- 172.21.15.66:0/3282986839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ba815b880 0x7f8ba81503b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00ac000 tx=0x7f8ba0004270 comp rx=0 tx=0).stop 2024-09-10T22:00:44.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.173+0000 7f8ba6800640 1 -- 172.21.15.66:0/3282986839 reap_dead start 2024-09-10T22:00:44.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.173+0000 7ffa2ec00640 1 -- 172.21.15.66:0/3747579156 --> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa28158fc0 con 0x7ffa0407a880 2024-09-10T22:00:44.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 shutdown_connections 2024-09-10T22:00:44.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 >> 172.21.15.66:0/3282986839 conn(0x7f8ba806cc20 msgr2=0x7f8ba815d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7ffa24c00640 1 -- 172.21.15.66:0/3747579156 <== osd.7 v2:172.21.15.94:6826/1972820213 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffa28158fc0 con 0x7ffa0407a880 2024-09-10T22:00:44.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 shutdown_connections 2024-09-10T22:00:44.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7f8ba7200640 1 -- 172.21.15.66:0/3282986839 wait complete. 2024-09-10T22:00:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7ffa2ec00640 1 -- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] conn(0x7ffa0407a880 msgr2=0x7ffa0407cca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7ffa2ec00640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] conn(0x7ffa0407a880 0x7ffa0407cca0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7ffa2ec00640 1 -- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa040726e0 msgr2=0x7ffa04074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7ffa2ec00640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa040726e0 0x7ffa04074ba0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffa10007a60 tx=0x7ffa10009040 comp rx=0 tx=0).stop 2024-09-10T22:00:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7ffa2ec00640 1 -- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa280ae620 msgr2=0x7ffa28157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7ffa2ec00640 1 --2- 172.21.15.66:0/3747579156 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa280ae620 0x7ffa28157040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800c490 tx=0x7ffa18001e80 comp rx=0 tx=0).stop 2024-09-10T22:00:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7ffa2cc00640 1 -- 172.21.15.66:0/3747579156 reap_dead start 2024-09-10T22:00:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7ffa2ec00640 1 -- 172.21.15.66:0/3747579156 shutdown_connections 2024-09-10T22:00:44.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7ffa2ec00640 1 -- 172.21.15.66:0/3747579156 >> 172.21.15.66:0/3747579156 conn(0x7ffa280cfa30 msgr2=0x7ffa280ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7ffa2ec00640 1 -- 172.21.15.66:0/3747579156 shutdown_connections 2024-09-10T22:00:44.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.177+0000 7ffa2ec00640 1 -- 172.21.15.66:0/3747579156 wait complete. 2024-09-10T22:00:44.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.181+0000 7fa0e8200640 1 -- 172.21.15.66:0/2775538774 --> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0e0162c50 con 0x7fa0c007b460 2024-09-10T22:00:44.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.181+0000 7fa0dd600640 1 -- 172.21.15.66:0/2775538774 <== osd.6 v2:172.21.15.94:6810/47260656 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa0e0162c50 con 0x7fa0c007b460 2024-09-10T22:00:44.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.185+0000 7fa0e8200640 1 -- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] conn(0x7fa0c007b460 msgr2=0x7fa0c007d900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.185+0000 7fa0e8200640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] conn(0x7fa0c007b460 0x7fa0c007d900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.189+0000 7fa0e8200640 1 -- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0c00725e0 msgr2=0x7fa0c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.189+0000 7fa0e8200640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0c00725e0 0x7fa0c0074aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40b0000 tx=0x7fa0d401b870 comp rx=0 tx=0).stop 2024-09-10T22:00:44.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.189+0000 7fa0e8200640 1 -- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0c8004810 msgr2=0x7fa0e00b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.189+0000 7fa0e8200640 1 --2- 172.21.15.66:0/2775538774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0c8004810 0x7fa0e00b53c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc001410 tx=0x7fa0cc00ea90 comp rx=0 tx=0).stop 2024-09-10T22:00:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.189+0000 7fa0e6200640 1 -- 172.21.15.66:0/2775538774 reap_dead start 2024-09-10T22:00:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.189+0000 7fa0e8200640 1 -- 172.21.15.66:0/2775538774 shutdown_connections 2024-09-10T22:00:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.189+0000 7fa0e8200640 1 -- 172.21.15.66:0/2775538774 >> 172.21.15.66:0/2775538774 conn(0x7fa0e00cfa30 msgr2=0x7fa0e01633a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.189+0000 7fa0e8200640 1 -- 172.21.15.66:0/2775538774 shutdown_connections 2024-09-10T22:00:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.189+0000 7fa0e8200640 1 -- 172.21.15.66:0/2775538774 wait complete. 2024-09-10T22:00:44.204 INFO:teuthology.orchestra.run.smithi066.stdout:64424509448 2024-09-10T22:00:44.204 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T22:00:44.205 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:00:44.206 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T22:00:44.207 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:00:44.207 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T22:00:44.214 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:00:44.214 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T22:00:44.216 INFO:teuthology.orchestra.run.smithi066.stdout:64424509446 2024-09-10T22:00:44.216 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T22:00:44.224 INFO:teuthology.orchestra.run.smithi066.stdout:60129542152 2024-09-10T22:00:44.224 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T22:00:44.230 INFO:teuthology.orchestra.run.smithi066.stdout:64424509448 2024-09-10T22:00:44.230 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:00:44.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T22:00:44.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T22:00:44.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.446+0000 7f7a22e00640 1 -- 172.21.15.66:0/1315125205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a040052f0 msgr2=0x7f7a0400fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.446+0000 7f7a1be00640 1 -- 172.21.15.66:0/1315125205 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c124700 con 0x7f7a040052f0 2024-09-10T22:00:44.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.446+0000 7f7a22e00640 1 --2- 172.21.15.66:0/1315125205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a040052f0 0x7f7a0400fde0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c052200 tx=0x7f7a1c0988d0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.446+0000 7f7a22e00640 1 -- 172.21.15.66:0/1315125205 shutdown_connections 2024-09-10T22:00:44.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.446+0000 7f7a22e00640 1 --2- 172.21.15.66:0/1315125205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a040052f0 0x7f7a0400fde0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.446+0000 7f7a22e00640 1 -- 172.21.15.66:0/1315125205 >> 172.21.15.66:0/1315125205 conn(0x7f7a14056720 msgr2=0x7f7a14056b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.446+0000 7f7a22e00640 1 -- 172.21.15.66:0/1315125205 shutdown_connections 2024-09-10T22:00:44.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.446+0000 7f7a22e00640 1 -- 172.21.15.66:0/1315125205 wait complete. 2024-09-10T22:00:44.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.446+0000 7efc9f200640 1 -- 172.21.15.66:0/974876444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc980aefd0 msgr2=0x7efc980c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.446+0000 7efc9f200640 1 --2- 172.21.15.66:0/974876444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc980aefd0 0x7efc980c4e60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efc8400b330 tx=0x7efc8401d0b0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.450+0000 7f7a22e00640 1 Processor -- start 2024-09-10T22:00:44.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.450+0000 7f7a22e00640 1 -- start start 2024-09-10T22:00:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7efc9f200640 1 -- 172.21.15.66:0/974876444 shutdown_connections 2024-09-10T22:00:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7efc9f200640 1 --2- 172.21.15.66:0/974876444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc980aefd0 0x7efc980c4e60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7efc9f200640 1 --2- 172.21.15.66:0/974876444 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc9816b0a0 0x7efc980c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7efc9f200640 1 -- 172.21.15.66:0/974876444 >> 172.21.15.66:0/974876444 conn(0x7efc980cfa30 msgr2=0x7efc980cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7efc9f200640 1 -- 172.21.15.66:0/974876444 shutdown_connections 2024-09-10T22:00:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7fcd2d000640 1 -- 172.21.15.66:0/1668252861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd280ce040 msgr2=0x7fcd2806fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7fcd2d000640 1 --2- 172.21.15.66:0/1668252861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd280ce040 0x7fcd2806fae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcd280c6710 tx=0x7fcd1801c520 comp rx=0 tx=0).stop 2024-09-10T22:00:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a22e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a040052f0 0x7f7a1417bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a22e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a1404f9f0 0x7f7a140d9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7fcd2d000640 1 -- 172.21.15.66:0/1668252861 shutdown_connections 2024-09-10T22:00:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7fcd2d000640 1 --2- 172.21.15.66:0/1668252861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd280ce040 0x7fcd2806fae0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7fcd2d000640 1 --2- 172.21.15.66:0/1668252861 >> v2:172.21.15.66:3300/0 conn(0x7fcd280c5ee0 0x7fcd2815f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7fcd2d000640 1 -- 172.21.15.66:0/1668252861 >> 172.21.15.66:0/1668252861 conn(0x7fcd280cfa30 msgr2=0x7fcd280cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7fcd2d000640 1 -- 172.21.15.66:0/1668252861 shutdown_connections 2024-09-10T22:00:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7fcd2d000640 1 -- 172.21.15.66:0/1668252861 wait complete. 2024-09-10T22:00:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a21a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a040052f0 0x7f7a1417bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a21a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a040052f0 0x7f7a1417bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:47322/0 (socket says 172.21.15.66:47322) 2024-09-10T22:00:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a21a00640 1 -- 172.21.15.66:0/1015346406 learned_addr learned my addr 172.21.15.66:0/1015346406 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7efc9f200640 1 -- 172.21.15.66:0/974876444 wait complete. 2024-09-10T22:00:44.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a21000640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a1404f9f0 0x7f7a140d9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a22e00640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a140dd120 0x7f7a1404d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a22e00640 1 -- 172.21.15.66:0/1015346406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7a140575a0 con 0x7f7a140dd120 2024-09-10T22:00:44.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a22e00640 1 -- 172.21.15.66:0/1015346406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a14057420 con 0x7f7a040052f0 2024-09-10T22:00:44.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a22e00640 1 -- 172.21.15.66:0/1015346406 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7a14035f20 con 0x7f7a1404f9f0 2024-09-10T22:00:44.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7efc9f200640 1 Processor -- start 2024-09-10T22:00:44.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a22400640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a140dd120 0x7f7a1404d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a21a00640 1 -- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a1404f9f0 msgr2=0x7f7a140d9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7fcd2d000640 1 Processor -- start 2024-09-10T22:00:44.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a21a00640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a1404f9f0 0x7f7a140d9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.454+0000 7f7a21a00640 1 -- 172.21.15.66:0/1015346406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a140dd120 msgr2=0x7f7a1404d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd2d000640 1 -- start start 2024-09-10T22:00:44.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7f7a21000640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a1404f9f0 0x7f7a140d9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:00:44.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7f7a21a00640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a140dd120 0x7f7a1404d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7f7a21a00640 1 -- 172.21.15.66:0/1015346406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a140dd6e0 con 0x7f7a040052f0 2024-09-10T22:00:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7f7a22400640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a140dd120 0x7f7a1404d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:00:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7f7a21a00640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a040052f0 0x7f7a1417bc10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7a10006770 tx=0x7f7a10003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7f7a19600640 1 -- 172.21.15.66:0/1015346406 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10015070 con 0x7f7a040052f0 2024-09-10T22:00:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7f7a19600640 1 -- 172.21.15.66:0/1015346406 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a1000f9e0 con 0x7f7a040052f0 2024-09-10T22:00:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7f7a19600640 1 -- 172.21.15.66:0/1015346406 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000fe30 con 0x7f7a040052f0 2024-09-10T22:00:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7f7a22e00640 1 -- 172.21.15.66:0/1015346406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1404ef10 con 0x7f7a040052f0 2024-09-10T22:00:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc9f200640 1 -- start start 2024-09-10T22:00:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc9f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc980aefd0 0x7efc9808ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd2d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd280c5ee0 0x7fcd28152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd2d000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd280ce040 0x7fcd280b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc9f200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc980ce1a0 0x7efc9817e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc97e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc980aefd0 0x7efc9808ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc97e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc980aefd0 0x7efc9808ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:47324/0 (socket says 172.21.15.66:47324) 2024-09-10T22:00:44.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc9f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9816b0a0 0x7efc9820ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd25e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd280c5ee0 0x7fcd28152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd25e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd280c5ee0 0x7fcd28152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45674/0 (socket says 172.21.15.66:45674) 2024-09-10T22:00:44.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd25e00640 1 -- 172.21.15.66:0/2158347873 learned_addr learned my addr 172.21.15.66:0/2158347873 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd2d000640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd280d0550 0x7fcd2815d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2816ae40 con 0x7fcd280c5ee0 2024-09-10T22:00:44.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2816acc0 con 0x7fcd280d0550 2024-09-10T22:00:44.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcd2816afc0 con 0x7fcd280ce040 2024-09-10T22:00:44.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7f7a22e00640 1 -- 172.21.15.66:0/1015346406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1404f4d0 con 0x7f7a040052f0 2024-09-10T22:00:44.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc97400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc980ce1a0 0x7efc9817e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc97400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc980ce1a0 0x7efc9817e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:59672/0 (socket says 172.21.15.66:59672) 2024-09-10T22:00:44.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc9f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efc9806fbb0 con 0x7efc9816b0a0 2024-09-10T22:00:44.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc9f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc9806fa30 con 0x7efc980aefd0 2024-09-10T22:00:44.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc9f200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7efc98177bd0 con 0x7efc980ce1a0 2024-09-10T22:00:44.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc97e00640 1 -- 172.21.15.66:0/4213588855 learned_addr learned my addr 172.21.15.66:0/4213588855 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd25400640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd280ce040 0x7fcd280b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd26800640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd280d0550 0x7fcd2815d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd25e00640 1 -- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd280ce040 msgr2=0x7fcd280b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd25e00640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd280ce040 0x7fcd280b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd25e00640 1 -- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd280d0550 msgr2=0x7fcd2815d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd25e00640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd280d0550 0x7fcd2815d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd25e00640 1 -- 172.21.15.66:0/2158347873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2815c290 con 0x7fcd280c5ee0 2024-09-10T22:00:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7fcd26800640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd280d0550 0x7fcd2815d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7fcd25e00640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd280c5ee0 0x7fcd28152680 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009d90 tx=0x7fcd2000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7fcd1e000640 1 -- 172.21.15.66:0/2158347873 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20003b80 con 0x7fcd280c5ee0 2024-09-10T22:00:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7fcd1e000640 1 -- 172.21.15.66:0/2158347873 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd20003d20 con 0x7fcd280c5ee0 2024-09-10T22:00:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7fcd1e000640 1 -- 172.21.15.66:0/2158347873 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20004280 con 0x7fcd280c5ee0 2024-09-10T22:00:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7f10c3600640 1 -- 172.21.15.66:0/310542324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10bc09c380 msgr2=0x7f10bc09c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7f10c3600640 1 --2- 172.21.15.66:0/310542324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10bc09c380 0x7f10bc09c760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0d3be0 tx=0x7f10b401cc30 comp rx=0 tx=0).stop 2024-09-10T22:00:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc9d200640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9816b0a0 0x7efc9820ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc97e00640 1 -- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc980ce1a0 msgr2=0x7efc9817e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc97e00640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc980ce1a0 0x7efc9817e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc97e00640 1 -- 172.21.15.66:0/4213588855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9816b0a0 msgr2=0x7efc9820ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc97e00640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9816b0a0 0x7efc9820ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.458+0000 7efc97e00640 1 -- 172.21.15.66:0/4213588855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc982104f0 con 0x7efc980aefd0 2024-09-10T22:00:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7efc97e00640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc980aefd0 0x7efc9808ade0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efc9000eda0 tx=0x7efc9000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2815c530 con 0x7fcd280c5ee0 2024-09-10T22:00:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2815caf0 con 0x7fcd280c5ee0 2024-09-10T22:00:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7efc8fe00640 1 -- 172.21.15.66:0/4213588855 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90004d30 con 0x7efc980aefd0 2024-09-10T22:00:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7efc9f200640 1 -- 172.21.15.66:0/4213588855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9817bc70 con 0x7efc980aefd0 2024-09-10T22:00:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7efc9f200640 1 -- 172.21.15.66:0/4213588855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9817d930 con 0x7efc980aefd0 2024-09-10T22:00:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7efc8fe00640 1 -- 172.21.15.66:0/4213588855 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc90003980 con 0x7efc980aefd0 2024-09-10T22:00:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7efc8fe00640 1 -- 172.21.15.66:0/4213588855 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90004d30 con 0x7efc980aefd0 2024-09-10T22:00:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2815f130 con 0x7fcd280c5ee0 2024-09-10T22:00:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7efc9f200640 1 -- 172.21.15.66:0/4213588855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc700052a0 con 0x7efc980aefd0 2024-09-10T22:00:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7f7a19600640 1 -- 172.21.15.66:0/1015346406 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7a10011430 con 0x7f7a040052f0 2024-09-10T22:00:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.462+0000 7f7a22e00640 1 -- 172.21.15.66:0/1015346406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1404f6d0 con 0x7f7a040052f0 2024-09-10T22:00:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7f10c3600640 1 -- 172.21.15.66:0/310542324 shutdown_connections 2024-09-10T22:00:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7f10c3600640 1 --2- 172.21.15.66:0/310542324 >> v2:172.21.15.94:3300/0 conn(0x7f10bc06dba0 0x7f10bc0b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7f10c3600640 1 --2- 172.21.15.66:0/310542324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10bc09c380 0x7f10bc09c760 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7f10c3600640 1 -- 172.21.15.66:0/310542324 >> 172.21.15.66:0/310542324 conn(0x7f10bc0d0e00 msgr2=0x7f10bc0d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7f10c3600640 1 -- 172.21.15.66:0/310542324 shutdown_connections 2024-09-10T22:00:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7f10c3600640 1 -- 172.21.15.66:0/310542324 wait complete. 2024-09-10T22:00:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7f7a19600640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f79fc0726e0 0x7f79fc074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7f7a19600640 1 -- 172.21.15.66:0/1015346406 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7a10094080 con 0x7f7a040052f0 2024-09-10T22:00:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7f7a21000640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f79fc0726e0 0x7f79fc074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7efc8fe00640 1 -- 172.21.15.66:0/4213588855 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efc9000b030 con 0x7efc980aefd0 2024-09-10T22:00:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7efc8fe00640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efc78072810 0x7efc78074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7efc8fe00640 1 -- 172.21.15.66:0/4213588855 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efc90093430 con 0x7efc980aefd0 2024-09-10T22:00:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7fcd1e000640 1 -- 172.21.15.66:0/2158347873 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcd20004420 con 0x7fcd280c5ee0 2024-09-10T22:00:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7fcd1e000640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd080725e0 0x7fcd08074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7fcd1e000640 1 -- 172.21.15.66:0/2158347873 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcd200940c0 con 0x7fcd280c5ee0 2024-09-10T22:00:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7fcd25400640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd080725e0 0x7fcd08074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7fcd25400640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd080725e0 0x7fcd08074aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcd14001ec0 tx=0x7fcd14009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.466+0000 7f10c3600640 1 Processor -- start 2024-09-10T22:00:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10c3600640 1 -- start start 2024-09-10T22:00:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10c3600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10bc06dba0 0x7f10bc0ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10c3600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc06e010 0x7f10bc1f9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10c3600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10bc09c380 0x7f10bc0b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10c3600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc0d3880 con 0x7f10bc06dba0 2024-09-10T22:00:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10c3600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc0d3700 con 0x7f10bc06e010 2024-09-10T22:00:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10c3600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f10bc166650 con 0x7f10bc09c380 2024-09-10T22:00:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7efc97400640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efc78072810 0x7efc78074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10bbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc06e010 0x7f10bc1f9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10c0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10bc06dba0 0x7f10bc0ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10bbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc06e010 0x7f10bc1f9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:47350/0 (socket says 172.21.15.66:47350) 2024-09-10T22:00:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10bbe00640 1 -- 172.21.15.66:0/844387549 learned_addr learned my addr 172.21.15.66:0/844387549 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f7a21000640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f79fc0726e0 0x7f79fc074ba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400ce00 tx=0x7f7a08006a40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10c1600640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10bc09c380 0x7f10bc0b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7efc97400640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efc78072810 0x7efc78074cd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efc9816a450 tx=0x7efc88009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f7a19600640 1 -- 172.21.15.66:0/1015346406 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7a1404f6d0 con 0x7f7a040052f0 2024-09-10T22:00:44.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10bbe00640 1 -- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10bc09c380 msgr2=0x7f10bc0b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.470+0000 7f10bbe00640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10bc09c380 0x7f10bc0b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.474+0000 7f10bbe00640 1 -- 172.21.15.66:0/844387549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10bc06dba0 msgr2=0x7f10bc0ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.474+0000 7f10bbe00640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10bc06dba0 0x7f10bc0ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.474+0000 7f10bbe00640 1 -- 172.21.15.66:0/844387549 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc1f9dd0 con 0x7f10bc06e010 2024-09-10T22:00:44.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.474+0000 7f10bbe00640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc06e010 0x7f10bc1f9890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c970 tx=0x7f10a800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f10b8c00640 1 -- 172.21.15.66:0/844387549 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8007bb0 con 0x7f10bc06e010 2024-09-10T22:00:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f10b8c00640 1 -- 172.21.15.66:0/844387549 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10a8007e40 con 0x7f10bc06e010 2024-09-10T22:00:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f10c3600640 1 -- 172.21.15.66:0/844387549 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10bc15aa20 con 0x7f10bc06e010 2024-09-10T22:00:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7efc8fe00640 1 -- 172.21.15.66:0/4213588855 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efc9005eb10 con 0x7efc980aefd0 2024-09-10T22:00:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f10b8c00640 1 -- 172.21.15.66:0/844387549 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8005260 con 0x7f10bc06e010 2024-09-10T22:00:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f10c3600640 1 -- 172.21.15.66:0/844387549 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10bc1fc940 con 0x7f10bc06e010 2024-09-10T22:00:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7fcd1e000640 1 -- 172.21.15.66:0/2158347873 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcd2005f700 con 0x7fcd280c5ee0 2024-09-10T22:00:44.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f10c3600640 1 -- 172.21.15.66:0/844387549 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10940052a0 con 0x7f10bc06e010 2024-09-10T22:00:44.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f8c62200640 1 -- 172.21.15.66:0/3730670930 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c5c0c5ee0 msgr2=0x7f8c5c15f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f8c62200640 1 --2- 172.21.15.66:0/3730670930 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c5c0c5ee0 0x7f8c5c15f130 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009900 tx=0x7f8c4c01c7c0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f8c62200640 1 -- 172.21.15.66:0/3730670930 shutdown_connections 2024-09-10T22:00:44.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f8c62200640 1 --2- 172.21.15.66:0/3730670930 >> v2:172.21.15.94:3300/0 conn(0x7f8c5c0d0550 0x7f8c5c06fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f8c62200640 1 --2- 172.21.15.66:0/3730670930 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c5c0c5ee0 0x7f8c5c15f130 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f8c62200640 1 -- 172.21.15.66:0/3730670930 >> 172.21.15.66:0/3730670930 conn(0x7f8c5c0cfa30 msgr2=0x7f8c5c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f8c62200640 1 -- 172.21.15.66:0/3730670930 shutdown_connections 2024-09-10T22:00:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.478+0000 7f8c62200640 1 -- 172.21.15.66:0/3730670930 wait complete. 2024-09-10T22:00:44.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.482+0000 7f8c62200640 1 Processor -- start 2024-09-10T22:00:44.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.482+0000 7f10b8c00640 1 -- 172.21.15.66:0/844387549 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f10a80054c0 con 0x7f10bc06e010 2024-09-10T22:00:44.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.482+0000 7f10b8c00640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f109c0725e0 0x7f109c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.482+0000 7f8c62200640 1 -- start start 2024-09-10T22:00:44.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.482+0000 7f10b8c00640 1 -- 172.21.15.66:0/844387549 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f10a80936c0 con 0x7f10bc06e010 2024-09-10T22:00:44.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.482+0000 7f10c0c00640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f109c0725e0 0x7f109c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.482+0000 7f10c0c00640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f109c0725e0 0x7f109c074aa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10b40af000 tx=0x7f10b401b470 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c62200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c5c0c5ee0 0x7f8c5c14f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c62200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c5c0cd930 0x7f8c5c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f10b8c00640 1 -- 172.21.15.66:0/844387549 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f10a8014030 con 0x7f10bc06e010 2024-09-10T22:00:44.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c62200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c5c0d0550 0x7f8c5c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c5aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c5c0cd930 0x7f8c5c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c5b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c5c0c5ee0 0x7f8c5c14f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c5b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c5c0c5ee0 0x7f8c5c14f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:47360/0 (socket says 172.21.15.66:47360) 2024-09-10T22:00:44.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c5aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c5c0cd930 0x7f8c5c06ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45710/0 (socket says 172.21.15.66:45710) 2024-09-10T22:00:44.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c5b400640 1 -- 172.21.15.66:0/3160422547 learned_addr learned my addr 172.21.15.66:0/3160422547 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c62200640 1 -- 172.21.15.66:0/3160422547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c16ae40 con 0x7f8c5c0cd930 2024-09-10T22:00:44.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c62200640 1 -- 172.21.15.66:0/3160422547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c16acc0 con 0x7f8c5c0c5ee0 2024-09-10T22:00:44.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c62200640 1 -- 172.21.15.66:0/3160422547 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5c16afc0 con 0x7f8c5c0d0550 2024-09-10T22:00:44.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c5be00640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c5c0d0550 0x7f8c5c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c5aa00640 1 -- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c5c0d0550 msgr2=0x7f8c5c072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c5aa00640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c5c0d0550 0x7f8c5c072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c5aa00640 1 -- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c5c0c5ee0 msgr2=0x7f8c5c14f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c5aa00640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c5c0c5ee0 0x7f8c5c14f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.486+0000 7f8c5aa00640 1 -- 172.21.15.66:0/3160422547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5c156aa0 con 0x7f8c5c0cd930 2024-09-10T22:00:44.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.490+0000 7f8c5aa00640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c5c0cd930 0x7f8c5c06ef60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0b5c00 tx=0x7f8c4800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.490+0000 7f8c53400640 1 -- 172.21.15.66:0/3160422547 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48004220 con 0x7f8c5c0cd930 2024-09-10T22:00:44.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.490+0000 7f8c53400640 1 -- 172.21.15.66:0/3160422547 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c480043c0 con 0x7f8c5c0cd930 2024-09-10T22:00:44.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.490+0000 7f8c53400640 1 -- 172.21.15.66:0/3160422547 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4800cb10 con 0x7f8c5c0cd930 2024-09-10T22:00:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.490+0000 7f8c62200640 1 -- 172.21.15.66:0/3160422547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5c156d80 con 0x7f8c5c0cd930 2024-09-10T22:00:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.490+0000 7f8c62200640 1 -- 172.21.15.66:0/3160422547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5c157340 con 0x7f8c5c0cd930 2024-09-10T22:00:44.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.494+0000 7f8c50c00640 1 -- 172.21.15.66:0/3160422547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c5c15f110 con 0x7f8c5c0cd930 2024-09-10T22:00:44.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.494+0000 7f8c53400640 1 -- 172.21.15.66:0/3160422547 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8c4800ccb0 con 0x7f8c5c0cd930 2024-09-10T22:00:44.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.498+0000 7f8c53400640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8c3c0725e0 0x7f8c3c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.498+0000 7f8c53400640 1 -- 172.21.15.66:0/3160422547 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8c48093a70 con 0x7f8c5c0cd930 2024-09-10T22:00:44.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.498+0000 7f8c5b400640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8c3c0725e0 0x7f8c3c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.506+0000 7f8c53400640 1 -- 172.21.15.66:0/3160422547 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8c4805ea60 con 0x7f8c5c0cd930 2024-09-10T22:00:44.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.510+0000 7f8c5b400640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8c3c0725e0 0x7f8c3c074aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c01ae70 tx=0x7f8c4c0a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f2240800640 1 -- 172.21.15.66:0/2252194805 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2238070720 msgr2=0x7f223806f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f2240800640 1 --2- 172.21.15.66:0/2252194805 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2238070720 0x7f223806f1e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2230009c30 tx=0x7f223001cca0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f44e6200640 1 -- 172.21.15.66:0/4168891114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44d8036250 msgr2=0x7f44d80e70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f44e6200640 1 --2- 172.21.15.66:0/4168891114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44d8036250 0x7f44d80e70f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0098f0 tx=0x7f44cc01d2b0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f2240800640 1 -- 172.21.15.66:0/2252194805 shutdown_connections 2024-09-10T22:00:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f2240800640 1 --2- 172.21.15.66:0/2252194805 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2238070720 0x7f223806f1e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f2240800640 1 -- 172.21.15.66:0/2252194805 >> 172.21.15.66:0/2252194805 conn(0x7f22380cf9b0 msgr2=0x7f22380cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f44e6200640 1 -- 172.21.15.66:0/4168891114 shutdown_connections 2024-09-10T22:00:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f44e6200640 1 --2- 172.21.15.66:0/4168891114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44d8036250 0x7f44d80e70f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f44e6200640 1 -- 172.21.15.66:0/4168891114 >> 172.21.15.66:0/4168891114 conn(0x7f44d801d780 msgr2=0x7f44d8054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f44e6200640 1 -- 172.21.15.66:0/4168891114 shutdown_connections 2024-09-10T22:00:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f44e6200640 1 -- 172.21.15.66:0/4168891114 wait complete. 2024-09-10T22:00:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f2240800640 1 -- 172.21.15.66:0/2252194805 shutdown_connections 2024-09-10T22:00:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.530+0000 7f2240800640 1 -- 172.21.15.66:0/2252194805 wait complete. 2024-09-10T22:00:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f44e6200640 1 Processor -- start 2024-09-10T22:00:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f44e6200640 1 -- start start 2024-09-10T22:00:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f2240800640 1 Processor -- start 2024-09-10T22:00:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f2240800640 1 -- start start 2024-09-10T22:00:44.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f44e6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44d8036250 0x7f44d80db770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f58f4000640 1 -- 172.21.15.66:0/3227703202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58ec06e010 msgr2=0x7f58ec0b7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f58f4000640 1 --2- 172.21.15.66:0/3227703202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58ec06e010 0x7f58ec0b7ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009bc0 tx=0x7f58e401cbc0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f44df400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44d8036250 0x7f44d80db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f44df400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44d8036250 0x7f44d80db770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45730/0 (socket says 172.21.15.66:45730) 2024-09-10T22:00:44.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f58f4000640 1 -- 172.21.15.66:0/3227703202 shutdown_connections 2024-09-10T22:00:44.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f58f4000640 1 --2- 172.21.15.66:0/3227703202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58ec06e010 0x7f58ec0b7ff0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f58f4000640 1 --2- 172.21.15.66:0/3227703202 >> v2:172.21.15.94:3300/0 conn(0x7f58ec09c380 0x7f58ec09c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f58f4000640 1 -- 172.21.15.66:0/3227703202 >> 172.21.15.66:0/3227703202 conn(0x7f58ec0d0e00 msgr2=0x7f58ec0d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f58f4000640 1 -- 172.21.15.66:0/3227703202 shutdown_connections 2024-09-10T22:00:44.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f58f4000640 1 -- 172.21.15.66:0/3227703202 wait complete. 2024-09-10T22:00:44.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f44df400640 1 -- 172.21.15.66:0/2267530483 learned_addr learned my addr 172.21.15.66:0/2267530483 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f44e6200640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44d8036630 0x7f44d80dabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f44e6200640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44d8055070 0x7f44d80db120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f44d80e28e0 con 0x7f44d8036250 2024-09-10T22:00:44.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f44dea00640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44d8036630 0x7f44d80dabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f44dfe00640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44d8055070 0x7f44d80db120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.534+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f44d80e2760 con 0x7f44d8036630 2024-09-10T22:00:44.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f44d80e2a60 con 0x7f44d8055070 2024-09-10T22:00:44.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f2240800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22380702b0 0x7f22381fc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f2240800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2238070720 0x7f22380ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f2240800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f223815b5f0 0x7f22380aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f2240800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f223816c6e0 con 0x7f223815b5f0 2024-09-10T22:00:44.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f2240800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f223816c560 con 0x7f22380702b0 2024-09-10T22:00:44.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f2240800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f223816c860 con 0x7f2238070720 2024-09-10T22:00:44.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f44dea00640 1 -- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44d8055070 msgr2=0x7f44d80db120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f44dea00640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44d8055070 0x7f44d80db120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f44dea00640 1 -- 172.21.15.66:0/2267530483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44d8036250 msgr2=0x7f44d80db770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f44dea00640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44d8036250 0x7f44d80db770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f223815b5f0 0x7f22380aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f44dea00640 1 -- 172.21.15.66:0/2267530483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d80dbfa0 con 0x7f44d8036630 2024-09-10T22:00:44.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f44df400640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44d8036250 0x7f44d80db770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:44.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f44dea00640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44d8036630 0x7f44d80dabe0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f44e006b2d0 tx=0x7f44e006ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f44d3400640 1 -- 172.21.15.66:0/2267530483 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e008f070 con 0x7f44d8036630 2024-09-10T22:00:44.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f44d3400640 1 -- 172.21.15.66:0/2267530483 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44e0074700 con 0x7f44d8036630 2024-09-10T22:00:44.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f223815b5f0 0x7f22380aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45746/0 (socket says 172.21.15.66:45746) 2024-09-10T22:00:44.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223e800640 1 -- 172.21.15.66:0/4112748680 learned_addr learned my addr 172.21.15.66:0/4112748680 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223de00640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22380702b0 0x7f22381fc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223d400640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2238070720 0x7f22380ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223de00640 1 -- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2238070720 msgr2=0x7f22380ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223de00640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2238070720 0x7f22380ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223de00640 1 -- 172.21.15.66:0/4112748680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f223815b5f0 msgr2=0x7f22380aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223de00640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f223815b5f0 0x7f22380aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223de00640 1 -- 172.21.15.66:0/4112748680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22380ce5b0 con 0x7f22380702b0 2024-09-10T22:00:44.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223e800640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f223815b5f0 0x7f22380aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:44.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223d400640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2238070720 0x7f22380ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:44.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f44d3400640 1 -- 172.21.15.66:0/2267530483 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0074b90 con 0x7f44d8036630 2024-09-10T22:00:44.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f223de00640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22380702b0 0x7f22381fc5f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f222800c3c0 tx=0x7f222800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f222e000640 1 -- 172.21.15.66:0/4112748680 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228017070 con 0x7f22380702b0 2024-09-10T22:00:44.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.538+0000 7f222e000640 1 -- 172.21.15.66:0/4112748680 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22280053c0 con 0x7f22380702b0 2024-09-10T22:00:44.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.542+0000 7f2240800640 1 -- 172.21.15.66:0/4112748680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22381fcc40 con 0x7f22380702b0 2024-09-10T22:00:44.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.542+0000 7f2240800640 1 -- 172.21.15.66:0/4112748680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22381fd200 con 0x7f22380702b0 2024-09-10T22:00:44.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.542+0000 7f222e000640 1 -- 172.21.15.66:0/4112748680 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228017070 con 0x7f22380702b0 2024-09-10T22:00:44.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.542+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d80dc280 con 0x7f44d8036630 2024-09-10T22:00:44.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.542+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d80dc840 con 0x7f44d8036630 2024-09-10T22:00:44.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.542+0000 7f2240800640 1 -- 172.21.15.66:0/4112748680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f223814dc90 con 0x7f22380702b0 2024-09-10T22:00:44.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.542+0000 7f58f4000640 1 Processor -- start 2024-09-10T22:00:44.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.542+0000 7f58f4000640 1 -- start start 2024-09-10T22:00:44.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d80e2f50 con 0x7f44d8036630 2024-09-10T22:00:44.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f222e000640 1 -- 172.21.15.66:0/4112748680 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2228007020 con 0x7f22380702b0 2024-09-10T22:00:44.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f4000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ec06dba0 0x7f58ec165ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ec06dba0 0x7f58ec165ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ec06dba0 0x7f58ec165ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45748/0 (socket says 172.21.15.66:45748) 2024-09-10T22:00:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f1600640 1 -- 172.21.15.66:0/2115021941 learned_addr learned my addr 172.21.15.66:0/2115021941 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f4000640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ec06e010 0x7f58ec152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f4000640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58ec09c380 0x7f58ec0b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1672a0 con 0x7f58ec06dba0 2024-09-10T22:00:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec167120 con 0x7f58ec06e010 2024-09-10T22:00:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f58ec167420 con 0x7f58ec09c380 2024-09-10T22:00:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f1600640 1 -- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58ec09c380 msgr2=0x7f58ec0b0bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:00:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f1600640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58ec09c380 0x7f58ec0b0bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f1600640 1 -- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ec06e010 msgr2=0x7f58ec152360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:00:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f1600640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ec06e010 0x7f58ec152360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f1600640 1 -- 172.21.15.66:0/2115021941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec15c580 con 0x7f58ec06dba0 2024-09-10T22:00:44.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f222e000640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f221c0729e0 0x7f221c074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f1600640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ec06dba0 0x7f58ec165ba0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f58d800e7d0 tx=0x7f58d800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f44d3400640 1 -- 172.21.15.66:0/2267530483 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f44e009c350 con 0x7f44d8036630 2024-09-10T22:00:44.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f223d400640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f221c0729e0 0x7f221c074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58e9600640 1 -- 172.21.15.66:0/2115021941 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d800ccc0 con 0x7f58ec06dba0 2024-09-10T22:00:44.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58e9600640 1 -- 172.21.15.66:0/2115021941 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58d8010430 con 0x7f58ec06dba0 2024-09-10T22:00:44.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58e9600640 1 -- 172.21.15.66:0/2115021941 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8010860 con 0x7f58ec06dba0 2024-09-10T22:00:44.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ec15c2a0 con 0x7f58ec06dba0 2024-09-10T22:00:44.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.546+0000 7f222e000640 1 -- 172.21.15.66:0/4112748680 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f22280050d0 con 0x7f22380702b0 2024-09-10T22:00:44.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.550+0000 7f222e000640 1 -- 172.21.15.66:0/4112748680 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f222805d650 con 0x7f22380702b0 2024-09-10T22:00:44.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.550+0000 7f223d400640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f221c0729e0 0x7f221c074ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22380b3b30 tx=0x7f2234009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.550+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ec151b60 con 0x7f58ec06dba0 2024-09-10T22:00:44.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.554+0000 7f44d3400640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f44c00726b0 0x7f44c0074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.554+0000 7f44d3400640 1 -- 172.21.15.66:0/2267530483 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f44e010e0c0 con 0x7f44d8036630 2024-09-10T22:00:44.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.554+0000 7f44df400640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f44c00726b0 0x7f44c0074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.554+0000 7f44d3400640 1 -- 172.21.15.66:0/2267530483 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f44e00d91c0 con 0x7f44d8036630 2024-09-10T22:00:44.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.554+0000 7f58e9600640 1 -- 172.21.15.66:0/2115021941 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f58d8010a00 con 0x7f58ec06dba0 2024-09-10T22:00:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.554+0000 7f58e9600640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f58cc072630 0x7f58cc074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.558+0000 7f44df400640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f44c00726b0 0x7f44c0074b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44cc01af00 tx=0x7f44cc01b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.554+0000 7f58e9600640 1 -- 172.21.15.66:0/2115021941 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f58d8093b00 con 0x7f58ec06dba0 2024-09-10T22:00:44.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.558+0000 7f58f0c00640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f58cc072630 0x7f58cc074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.558+0000 7f58f0c00640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f58cc072630 0x7f58cc074af0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f58ec1fee30 tx=0x7f58e00078b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.558+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ec151e90 con 0x7f58ec06dba0 2024-09-10T22:00:44.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.566+0000 7f58e9600640 1 -- 172.21.15.66:0/2115021941 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f58d805f350 con 0x7f58ec06dba0 2024-09-10T22:00:44.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.846+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcd2806ec10 con 0x7fcd280c5ee0 2024-09-10T22:00:44.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.846+0000 7fcd1e000640 1 -- 172.21.15.66:0/2158347873 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcd2005f000 con 0x7fcd280c5ee0 2024-09-10T22:00:44.852 INFO:teuthology.orchestra.run.smithi066.stdout:64424509446 2024-09-10T22:00:44.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.850+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd080725e0 msgr2=0x7fcd08074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.850+0000 7fcd2d000640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd080725e0 0x7fcd08074aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcd14001ec0 tx=0x7fcd14009040 comp rx=0 tx=0).stop 2024-09-10T22:00:44.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.850+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd280c5ee0 msgr2=0x7fcd28152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.850+0000 7fcd2d000640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd280c5ee0 0x7fcd28152680 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009d90 tx=0x7fcd2000c6a0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.850+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 shutdown_connections 2024-09-10T22:00:44.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.850+0000 7fcd2d000640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd080725e0 0x7fcd08074aa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.850+0000 7fcd2d000640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd280d0550 0x7fcd2815d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.850+0000 7fcd2d000640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd280ce040 0x7fcd280b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.850+0000 7fcd2d000640 1 --2- 172.21.15.66:0/2158347873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd280c5ee0 0x7fcd28152680 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.850+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 >> 172.21.15.66:0/2158347873 conn(0x7fcd280cfa30 msgr2=0x7fcd28161ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.850+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 shutdown_connections 2024-09-10T22:00:44.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.850+0000 7fcd2d000640 1 -- 172.21.15.66:0/2158347873 wait complete. 2024-09-10T22:00:44.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.854+0000 7f7a22e00640 1 -- 172.21.15.66:0/1015346406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7a14006940 con 0x7f7a040052f0 2024-09-10T22:00:44.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.854+0000 7f7a19600640 1 -- 172.21.15.66:0/1015346406 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7a10066e20 con 0x7f7a040052f0 2024-09-10T22:00:44.868 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:00:44.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.862+0000 7efc9f200640 1 -- 172.21.15.66:0/4213588855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7efc70005860 con 0x7efc980aefd0 2024-09-10T22:00:44.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.862+0000 7efc8fe00640 1 -- 172.21.15.66:0/4213588855 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc9005e410 con 0x7efc980aefd0 2024-09-10T22:00:44.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f7a0ea00640 1 -- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f79fc0726e0 msgr2=0x7f79fc074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f7a0ea00640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f79fc0726e0 0x7f79fc074ba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400ce00 tx=0x7f7a08006a40 comp rx=0 tx=0).stop 2024-09-10T22:00:44.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f7a0ea00640 1 -- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a040052f0 msgr2=0x7f7a1417bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f7a0ea00640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a040052f0 0x7f7a1417bc10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7a10006770 tx=0x7f7a10003a90 comp rx=0 tx=0).stop 2024-09-10T22:00:44.870 INFO:teuthology.orchestra.run.smithi066.stdout:64424509448 2024-09-10T22:00:44.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f10c3600640 1 -- 172.21.15.66:0/844387549 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1094005590 con 0x7f10bc06e010 2024-09-10T22:00:44.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f10b8c00640 1 -- 172.21.15.66:0/844387549 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f10a805e740 con 0x7f10bc06e010 2024-09-10T22:00:44.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f7a0ea00640 1 -- 172.21.15.66:0/1015346406 shutdown_connections 2024-09-10T22:00:44.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f7a0ea00640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f79fc0726e0 0x7f79fc074ba0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f7a0ea00640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a140dd120 0x7f7a1404d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f7a0ea00640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a1404f9f0 0x7f7a140d9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f7a0ea00640 1 --2- 172.21.15.66:0/1015346406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a040052f0 0x7f7a1417bc10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f7a0ea00640 1 -- 172.21.15.66:0/1015346406 >> 172.21.15.66:0/1015346406 conn(0x7f7a14056720 msgr2=0x7f7a140e1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.872 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:00:44.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f7a0ea00640 1 -- 172.21.15.66:0/1015346406 shutdown_connections 2024-09-10T22:00:44.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7efc9f200640 1 -- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efc78072810 msgr2=0x7efc78074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7f7a0ea00640 1 -- 172.21.15.66:0/1015346406 wait complete. 2024-09-10T22:00:44.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7efc9f200640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efc78072810 0x7efc78074cd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efc9816a450 tx=0x7efc88009040 comp rx=0 tx=0).stop 2024-09-10T22:00:44.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7efc9f200640 1 -- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc980aefd0 msgr2=0x7efc9808ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.866+0000 7efc9f200640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc980aefd0 0x7efc9808ade0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efc9000eda0 tx=0x7efc9000c6e0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7efc9f200640 1 -- 172.21.15.66:0/4213588855 shutdown_connections 2024-09-10T22:00:44.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7efc9f200640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efc78072810 0x7efc78074cd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7efc9f200640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9816b0a0 0x7efc9820ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7efc9f200640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc980ce1a0 0x7efc9817e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7efc9f200640 1 --2- 172.21.15.66:0/4213588855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc980aefd0 0x7efc9808ade0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7efc9f200640 1 -- 172.21.15.66:0/4213588855 >> 172.21.15.66:0/4213588855 conn(0x7efc980cfa30 msgr2=0x7efc980af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7efc9f200640 1 -- 172.21.15.66:0/4213588855 shutdown_connections 2024-09-10T22:00:44.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7f10c3600640 1 -- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f109c0725e0 msgr2=0x7f109c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7f10c3600640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f109c0725e0 0x7f109c074aa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10b40af000 tx=0x7f10b401b470 comp rx=0 tx=0).stop 2024-09-10T22:00:44.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7f10c3600640 1 -- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc06e010 msgr2=0x7f10bc1f9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7f10c3600640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc06e010 0x7f10bc1f9890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c970 tx=0x7f10a800ce30 comp rx=0 tx=0).stop 2024-09-10T22:00:44.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7efc9f200640 1 -- 172.21.15.66:0/4213588855 wait complete. 2024-09-10T22:00:44.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7f10c3600640 1 -- 172.21.15.66:0/844387549 shutdown_connections 2024-09-10T22:00:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7f10c3600640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f109c0725e0 0x7f109c074aa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7f10c3600640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10bc09c380 0x7f10bc0b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7f10c3600640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc06e010 0x7f10bc1f9890 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7f10c3600640 1 --2- 172.21.15.66:0/844387549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10bc06dba0 0x7f10bc0ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7f10c3600640 1 -- 172.21.15.66:0/844387549 >> 172.21.15.66:0/844387549 conn(0x7f10bc0d0e00 msgr2=0x7f10bc1607b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7f10c3600640 1 -- 172.21.15.66:0/844387549 shutdown_connections 2024-09-10T22:00:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.870+0000 7f10c3600640 1 -- 172.21.15.66:0/844387549 wait complete. 2024-09-10T22:00:44.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.878+0000 7f8c50c00640 1 -- 172.21.15.66:0/3160422547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8c5c050f60 con 0x7f8c5c0cd930 2024-09-10T22:00:44.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.886+0000 7f8c53400640 1 -- 172.21.15.66:0/3160422547 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8c48063490 con 0x7f8c5c0cd930 2024-09-10T22:00:44.890 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:00:44.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-09-10T22:00:44.893 DEBUG:teuthology.parallel:result is None 2024-09-10T22:00:44.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.890+0000 7f8c50c00640 1 -- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8c3c0725e0 msgr2=0x7f8c3c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.890+0000 7f8c50c00640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8c3c0725e0 0x7f8c3c074aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c01ae70 tx=0x7f8c4c0a7040 comp rx=0 tx=0).stop 2024-09-10T22:00:44.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.890+0000 7f8c50c00640 1 -- 172.21.15.66:0/3160422547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c5c0cd930 msgr2=0x7f8c5c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.890+0000 7f8c50c00640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c5c0cd930 0x7f8c5c06ef60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0b5c00 tx=0x7f8c4800eb50 comp rx=0 tx=0).stop 2024-09-10T22:00:44.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.890+0000 7f8c50c00640 1 -- 172.21.15.66:0/3160422547 shutdown_connections 2024-09-10T22:00:44.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.890+0000 7f8c50c00640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8c3c0725e0 0x7f8c3c074aa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.890+0000 7f8c50c00640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c5c0d0550 0x7f8c5c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.890+0000 7f8c50c00640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c5c0cd930 0x7f8c5c06ef60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.890+0000 7f8c50c00640 1 --2- 172.21.15.66:0/3160422547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c5c0c5ee0 0x7f8c5c14f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.890+0000 7f8c50c00640 1 -- 172.21.15.66:0/3160422547 >> 172.21.15.66:0/3160422547 conn(0x7f8c5c0cfa30 msgr2=0x7f8c5c161d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.890+0000 7f8c50c00640 1 -- 172.21.15.66:0/3160422547 shutdown_connections 2024-09-10T22:00:44.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.890+0000 7f8c50c00640 1 -- 172.21.15.66:0/3160422547 wait complete. 2024-09-10T22:00:44.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-09-10T22:00:44.905 DEBUG:teuthology.parallel:result is None 2024-09-10T22:00:44.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-09-10T22:00:44.913 DEBUG:teuthology.parallel:result is None 2024-09-10T22:00:44.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.4 2024-09-10T22:00:44.915 DEBUG:teuthology.parallel:result is None 2024-09-10T22:00:44.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.910+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f44d800dc00 con 0x7f44d8036630 2024-09-10T22:00:44.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.910+0000 7f44d3400640 1 -- 172.21.15.66:0/2267530483 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f44e00ddbf0 con 0x7f44d8036630 2024-09-10T22:00:44.917 INFO:teuthology.orchestra.run.smithi066.stdout:60129542152 2024-09-10T22:00:44.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f44c00726b0 msgr2=0x7f44c0074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f44e6200640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f44c00726b0 0x7f44c0074b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44cc01af00 tx=0x7f44cc01b090 comp rx=0 tx=0).stop 2024-09-10T22:00:44.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44d8036630 msgr2=0x7f44d80dabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f44e6200640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44d8036630 0x7f44d80dabe0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f44e006b2d0 tx=0x7f44e006ca10 comp rx=0 tx=0).stop 2024-09-10T22:00:44.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 shutdown_connections 2024-09-10T22:00:44.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f44e6200640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f44c00726b0 0x7f44c0074b70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f44e6200640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44d8055070 0x7f44d80db120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f44e6200640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44d8036630 0x7f44d80dabe0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f44e6200640 1 --2- 172.21.15.66:0/2267530483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44d8036250 0x7f44d80db770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 >> 172.21.15.66:0/2267530483 conn(0x7f44d801d780 msgr2=0x7f44d80e6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 shutdown_connections 2024-09-10T22:00:44.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f44e6200640 1 -- 172.21.15.66:0/2267530483 wait complete. 2024-09-10T22:00:44.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f2240800640 1 -- 172.21.15.66:0/4112748680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f22380b09d0 con 0x7f22380702b0 2024-09-10T22:00:44.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.914+0000 7f222e000640 1 -- 172.21.15.66:0/4112748680 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f22280663c0 con 0x7f22380702b0 2024-09-10T22:00:44.921 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:00:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f2240800640 1 -- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f221c0729e0 msgr2=0x7f221c074ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f2240800640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f221c0729e0 0x7f221c074ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22380b3b30 tx=0x7f2234009040 comp rx=0 tx=0).stop 2024-09-10T22:00:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f2240800640 1 -- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22380702b0 msgr2=0x7f22381fc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f2240800640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22380702b0 0x7f22381fc5f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f222800c3c0 tx=0x7f222800c880 comp rx=0 tx=0).stop 2024-09-10T22:00:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f2240800640 1 -- 172.21.15.66:0/4112748680 shutdown_connections 2024-09-10T22:00:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f2240800640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f221c0729e0 0x7f221c074ea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f2240800640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f223815b5f0 0x7f22380aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f2240800640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2238070720 0x7f22380ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f2240800640 1 --2- 172.21.15.66:0/4112748680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22380702b0 0x7f22381fc5f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f2240800640 1 -- 172.21.15.66:0/4112748680 >> 172.21.15.66:0/4112748680 conn(0x7f22380cf9b0 msgr2=0x7f22380b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f2240800640 1 -- 172.21.15.66:0/4112748680 shutdown_connections 2024-09-10T22:00:44.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f2240800640 1 -- 172.21.15.66:0/4112748680 wait complete. 2024-09-10T22:00:44.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f58ec09c760 con 0x7f58ec06dba0 2024-09-10T22:00:44.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.918+0000 7f58e9600640 1 -- 172.21.15.66:0/2115021941 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58d805ec50 con 0x7f58ec06dba0 2024-09-10T22:00:44.925 INFO:teuthology.orchestra.run.smithi066.stdout:64424509448 2024-09-10T22:00:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.922+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f58cc072630 msgr2=0x7f58cc074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.922+0000 7f58f4000640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f58cc072630 0x7f58cc074af0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f58ec1fee30 tx=0x7f58e00078b0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.922+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ec06dba0 msgr2=0x7f58ec165ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.922+0000 7f58f4000640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ec06dba0 0x7f58ec165ba0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f58d800e7d0 tx=0x7f58d800ec90 comp rx=0 tx=0).stop 2024-09-10T22:00:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.922+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 shutdown_connections 2024-09-10T22:00:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.922+0000 7f58f4000640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f58cc072630 0x7f58cc074af0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.922+0000 7f58f4000640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58ec09c380 0x7f58ec0b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.922+0000 7f58f4000640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ec06e010 0x7f58ec152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.922+0000 7f58f4000640 1 --2- 172.21.15.66:0/2115021941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ec06dba0 0x7f58ec165ba0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.922+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 >> 172.21.15.66:0/2115021941 conn(0x7f58ec0d0e00 msgr2=0x7f58ec1625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.922+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 shutdown_connections 2024-09-10T22:00:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:44.922+0000 7f58f4000640 1 -- 172.21.15.66:0/2115021941 wait complete. 2024-09-10T22:00:44.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.5 2024-09-10T22:00:44.929 DEBUG:teuthology.parallel:result is None 2024-09-10T22:00:44.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.6 2024-09-10T22:00:44.954 DEBUG:teuthology.parallel:result is None 2024-09-10T22:00:44.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.2 2024-09-10T22:00:44.954 DEBUG:teuthology.parallel:result is None 2024-09-10T22:00:44.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-09-10T22:00:44.957 DEBUG:teuthology.parallel:result is None 2024-09-10T22:00:44.957 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:00:44.957 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:00:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 -- 172.21.15.66:0/1385664784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5015a310 msgr2=0x7faa500c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 --2- 172.21.15.66:0/1385664784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5015a310 0x7faa500c48b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faa500af550 tx=0x7faa4001c830 comp rx=0 tx=0).stop 2024-09-10T22:00:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 -- 172.21.15.66:0/1385664784 shutdown_connections 2024-09-10T22:00:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 --2- 172.21.15.66:0/1385664784 >> v2:172.21.15.94:3300/0 conn(0x7faa500aeec0 0x7faa500c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 --2- 172.21.15.66:0/1385664784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5015a310 0x7faa500c48b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 -- 172.21.15.66:0/1385664784 >> 172.21.15.66:0/1385664784 conn(0x7faa500cfa30 msgr2=0x7faa500cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 -- 172.21.15.66:0/1385664784 shutdown_connections 2024-09-10T22:00:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 -- 172.21.15.66:0/1385664784 wait complete. 2024-09-10T22:00:45.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 Processor -- start 2024-09-10T22:00:45.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 -- start start 2024-09-10T22:00:45.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa500aeec0 0x7faa500c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5015a310 0x7faa5014f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa5014f800 0x7faa500af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa5015edb0 con 0x7faa5015a310 2024-09-10T22:00:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa5015ec30 con 0x7faa500aeec0 2024-09-10T22:00:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.146+0000 7faa55000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7faa50167300 con 0x7faa5014f800 2024-09-10T22:00:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa45400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5015a310 0x7faa5014f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa45400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5015a310 0x7faa5014f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45766/0 (socket says 172.21.15.66:45766) 2024-09-10T22:00:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa45400640 1 -- 172.21.15.66:0/27010846 learned_addr learned my addr 172.21.15.66:0/27010846 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa4de00640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa500aeec0 0x7faa500c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa4e800640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa5014f800 0x7faa500af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa45400640 1 -- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa5014f800 msgr2=0x7faa500af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa45400640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa5014f800 0x7faa500af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa45400640 1 -- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa500aeec0 msgr2=0x7faa500c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa45400640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa500aeec0 0x7faa500c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa45400640 1 -- 172.21.15.66:0/27010846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa500c99c0 con 0x7faa5015a310 2024-09-10T22:00:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa4e800640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa5014f800 0x7faa500af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa45400640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5015a310 0x7faa5014f2c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faa3800c500 tx=0x7faa3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa46a00640 1 -- 172.21.15.66:0/27010846 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38017070 con 0x7faa5015a310 2024-09-10T22:00:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa46a00640 1 -- 172.21.15.66:0/27010846 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa380053c0 con 0x7faa5015a310 2024-09-10T22:00:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa55000640 1 -- 172.21.15.66:0/27010846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa501f7d30 con 0x7faa5015a310 2024-09-10T22:00:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa46a00640 1 -- 172.21.15.66:0/27010846 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38012890 con 0x7faa5015a310 2024-09-10T22:00:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa55000640 1 -- 172.21.15.66:0/27010846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa501f8590 con 0x7faa5015a310 2024-09-10T22:00:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa55000640 1 -- 172.21.15.66:0/27010846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa50070420 con 0x7faa5015a310 2024-09-10T22:00:45.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.150+0000 7faa46a00640 1 -- 172.21.15.66:0/27010846 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faa38003600 con 0x7faa5015a310 2024-09-10T22:00:45.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.154+0000 7faa46a00640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa2c0725e0 0x7faa2c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:45.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.154+0000 7faa46a00640 1 -- 172.21.15.66:0/27010846 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7faa38092890 con 0x7faa5015a310 2024-09-10T22:00:45.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.154+0000 7faa4de00640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa2c0725e0 0x7faa2c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.154+0000 7faa4de00640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa2c0725e0 0x7faa2c074aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa400af000 tx=0x7faa4001b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.158+0000 7faa46a00640 1 -- 172.21.15.66:0/27010846 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faa3805d890 con 0x7faa5015a310 2024-09-10T22:00:45.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.326+0000 7faa55000640 1 -- 172.21.15.66:0/27010846 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa50069850 con 0x7faa2c0725e0 2024-09-10T22:00:45.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.330+0000 7faa46a00640 1 -- 172.21.15.66:0/27010846 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7faa50069850 con 0x7faa2c0725e0 2024-09-10T22:00:45.339 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:45.340 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:00:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.334+0000 7faa55000640 1 -- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa2c0725e0 msgr2=0x7faa2c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.334+0000 7faa55000640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa2c0725e0 0x7faa2c074aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa400af000 tx=0x7faa4001b070 comp rx=0 tx=0).stop 2024-09-10T22:00:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.334+0000 7faa55000640 1 -- 172.21.15.66:0/27010846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5015a310 msgr2=0x7faa5014f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.334+0000 7faa55000640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5015a310 0x7faa5014f2c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faa3800c500 tx=0x7faa3800c9c0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.334+0000 7faa55000640 1 -- 172.21.15.66:0/27010846 shutdown_connections 2024-09-10T22:00:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.334+0000 7faa55000640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa2c0725e0 0x7faa2c074aa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.334+0000 7faa55000640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa5014f800 0x7faa500af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.334+0000 7faa55000640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5015a310 0x7faa5014f2c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.334+0000 7faa55000640 1 --2- 172.21.15.66:0/27010846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa500aeec0 0x7faa500c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.334+0000 7faa55000640 1 -- 172.21.15.66:0/27010846 >> 172.21.15.66:0/27010846 conn(0x7faa500cfa30 msgr2=0x7faa501503a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:45.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.334+0000 7faa55000640 1 -- 172.21.15.66:0/27010846 shutdown_connections 2024-09-10T22:00:45.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.334+0000 7faa55000640 1 -- 172.21.15.66:0/27010846 wait complete. 2024-09-10T22:00:45.360 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-10T22:00:44.312680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235848,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738872,"statfs":{"total":773094113280,"available":772852604928,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.129861"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665723+0000","last_change":"2024-09-10T22:00:33.198521+0000","last_active":"2024-09-10T22:00:34.665723+0000","last_peered":"2024-09-10T22:00:34.665723+0000","last_clean":"2024-09-10T22:00:34.665723+0000","last_became_active":"2024-09-10T22:00:33.197869+0000","last_became_peered":"2024-09-10T22:00:33.197869+0000","last_unstale":"2024-09-10T22:00:34.665723+0000","last_undegraded":"2024-09-10T22:00:34.665723+0000","last_fullsized":"2024-09-10T22:00:34.665723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:48.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183587+0000","last_change":"2024-09-10T22:00:33.182512+0000","last_active":"2024-09-10T22:00:34.183587+0000","last_peered":"2024-09-10T22:00:34.183587+0000","last_clean":"2024-09-10T22:00:34.183587+0000","last_became_active":"2024-09-10T22:00:33.182230+0000","last_became_peered":"2024-09-10T22:00:33.182230+0000","last_unstale":"2024-09-10T22:00:34.183587+0000","last_undegraded":"2024-09-10T22:00:34.183587+0000","last_fullsized":"2024-09-10T22:00:34.183587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:08.694915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663553+0000","last_change":"2024-09-10T22:00:33.182382+0000","last_active":"2024-09-10T22:00:34.663553+0000","last_peered":"2024-09-10T22:00:34.663553+0000","last_clean":"2024-09-10T22:00:34.663553+0000","last_became_active":"2024-09-10T22:00:33.182115+0000","last_became_peered":"2024-09-10T22:00:33.182115+0000","last_unstale":"2024-09-10T22:00:34.663553+0000","last_undegraded":"2024-09-10T22:00:34.663553+0000","last_fullsized":"2024-09-10T22:00:34.663553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:34.220172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185630+0000","last_change":"2024-09-10T22:00:33.215454+0000","last_active":"2024-09-10T22:00:34.185630+0000","last_peered":"2024-09-10T22:00:34.185630+0000","last_clean":"2024-09-10T22:00:34.185630+0000","last_became_active":"2024-09-10T22:00:33.214713+0000","last_became_peered":"2024-09-10T22:00:33.214713+0000","last_unstale":"2024-09-10T22:00:34.185630+0000","last_undegraded":"2024-09-10T22:00:34.185630+0000","last_fullsized":"2024-09-10T22:00:34.185630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:37.559387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665058+0000","last_change":"2024-09-10T22:00:33.201925+0000","last_active":"2024-09-10T22:00:34.665058+0000","last_peered":"2024-09-10T22:00:34.665058+0000","last_clean":"2024-09-10T22:00:34.665058+0000","last_became_active":"2024-09-10T22:00:33.197799+0000","last_became_peered":"2024-09-10T22:00:33.197799+0000","last_unstale":"2024-09-10T22:00:34.665058+0000","last_undegraded":"2024-09-10T22:00:34.665058+0000","last_fullsized":"2024-09-10T22:00:34.665058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:37.217143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185144+0000","last_change":"2024-09-10T22:00:33.211402+0000","last_active":"2024-09-10T22:00:34.185144+0000","last_peered":"2024-09-10T22:00:34.185144+0000","last_clean":"2024-09-10T22:00:34.185144+0000","last_became_active":"2024-09-10T22:00:33.210608+0000","last_became_peered":"2024-09-10T22:00:33.210608+0000","last_unstale":"2024-09-10T22:00:34.185144+0000","last_undegraded":"2024-09-10T22:00:34.185144+0000","last_fullsized":"2024-09-10T22:00:34.185144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:14.992964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664313+0000","last_change":"2024-09-10T22:00:33.197611+0000","last_active":"2024-09-10T22:00:34.664313+0000","last_peered":"2024-09-10T22:00:34.664313+0000","last_clean":"2024-09-10T22:00:34.664313+0000","last_became_active":"2024-09-10T22:00:33.192560+0000","last_became_peered":"2024-09-10T22:00:33.192560+0000","last_unstale":"2024-09-10T22:00:34.664313+0000","last_undegraded":"2024-09-10T22:00:34.664313+0000","last_fullsized":"2024-09-10T22:00:34.664313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:25.562048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664300+0000","last_change":"2024-09-10T22:00:33.213208+0000","last_active":"2024-09-10T22:00:34.664300+0000","last_peered":"2024-09-10T22:00:34.664300+0000","last_clean":"2024-09-10T22:00:34.664300+0000","last_became_active":"2024-09-10T22:00:33.212880+0000","last_became_peered":"2024-09-10T22:00:33.212880+0000","last_unstale":"2024-09-10T22:00:34.664300+0000","last_undegraded":"2024-09-10T22:00:34.664300+0000","last_fullsized":"2024-09-10T22:00:34.664300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:55.992463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665753+0000","last_change":"2024-09-10T22:00:33.215336+0000","last_active":"2024-09-10T22:00:34.665753+0000","last_peered":"2024-09-10T22:00:34.665753+0000","last_clean":"2024-09-10T22:00:34.665753+0000","last_became_active":"2024-09-10T22:00:33.214306+0000","last_became_peered":"2024-09-10T22:00:33.214306+0000","last_unstale":"2024-09-10T22:00:34.665753+0000","last_undegraded":"2024-09-10T22:00:34.665753+0000","last_fullsized":"2024-09-10T22:00:34.665753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:45.979508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187047+0000","last_change":"2024-09-10T22:00:33.208856+0000","last_active":"2024-09-10T22:00:34.187047+0000","last_peered":"2024-09-10T22:00:34.187047+0000","last_clean":"2024-09-10T22:00:34.187047+0000","last_became_active":"2024-09-10T22:00:33.208575+0000","last_became_peered":"2024-09-10T22:00:33.208575+0000","last_unstale":"2024-09-10T22:00:34.187047+0000","last_undegraded":"2024-09-10T22:00:34.187047+0000","last_fullsized":"2024-09-10T22:00:34.187047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.744906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186250+0000","last_change":"2024-09-10T22:00:33.214153+0000","last_active":"2024-09-10T22:00:34.186250+0000","last_peered":"2024-09-10T22:00:34.186250+0000","last_clean":"2024-09-10T22:00:34.186250+0000","last_became_active":"2024-09-10T22:00:33.212422+0000","last_became_peered":"2024-09-10T22:00:33.212422+0000","last_unstale":"2024-09-10T22:00:34.186250+0000","last_undegraded":"2024-09-10T22:00:34.186250+0000","last_fullsized":"2024-09-10T22:00:34.186250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:04.855711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665304+0000","last_change":"2024-09-10T22:00:33.184796+0000","last_active":"2024-09-10T22:00:34.665304+0000","last_peered":"2024-09-10T22:00:34.665304+0000","last_clean":"2024-09-10T22:00:34.665304+0000","last_became_active":"2024-09-10T22:00:33.180143+0000","last_became_peered":"2024-09-10T22:00:33.180143+0000","last_unstale":"2024-09-10T22:00:34.665304+0000","last_undegraded":"2024-09-10T22:00:34.665304+0000","last_fullsized":"2024-09-10T22:00:34.665304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:35.363109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663813+0000","last_change":"2024-09-10T22:00:33.192437+0000","last_active":"2024-09-10T22:00:34.663813+0000","last_peered":"2024-09-10T22:00:34.663813+0000","last_clean":"2024-09-10T22:00:34.663813+0000","last_became_active":"2024-09-10T22:00:33.192220+0000","last_became_peered":"2024-09-10T22:00:33.192220+0000","last_unstale":"2024-09-10T22:00:34.663813+0000","last_undegraded":"2024-09-10T22:00:34.663813+0000","last_fullsized":"2024-09-10T22:00:34.663813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:29.825070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663467+0000","last_change":"2024-09-10T22:00:33.184124+0000","last_active":"2024-09-10T22:00:34.663467+0000","last_peered":"2024-09-10T22:00:34.663467+0000","last_clean":"2024-09-10T22:00:34.663467+0000","last_became_active":"2024-09-10T22:00:33.182588+0000","last_became_peered":"2024-09-10T22:00:33.182588+0000","last_unstale":"2024-09-10T22:00:34.663467+0000","last_undegraded":"2024-09-10T22:00:34.663467+0000","last_fullsized":"2024-09-10T22:00:34.663467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:39.577554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183252+0000","last_change":"2024-09-10T22:00:33.178023+0000","last_active":"2024-09-10T22:00:34.183252+0000","last_peered":"2024-09-10T22:00:34.183252+0000","last_clean":"2024-09-10T22:00:34.183252+0000","last_became_active":"2024-09-10T22:00:33.176108+0000","last_became_peered":"2024-09-10T22:00:33.176108+0000","last_unstale":"2024-09-10T22:00:34.183252+0000","last_undegraded":"2024-09-10T22:00:34.183252+0000","last_fullsized":"2024-09-10T22:00:34.183252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:06.332661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184712+0000","last_change":"2024-09-10T22:00:33.208492+0000","last_active":"2024-09-10T22:00:34.184712+0000","last_peered":"2024-09-10T22:00:34.184712+0000","last_clean":"2024-09-10T22:00:34.184712+0000","last_became_active":"2024-09-10T22:00:33.208232+0000","last_became_peered":"2024-09-10T22:00:33.208232+0000","last_unstale":"2024-09-10T22:00:34.184712+0000","last_undegraded":"2024-09-10T22:00:34.184712+0000","last_fullsized":"2024-09-10T22:00:34.184712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.744906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185766+0000","last_change":"2024-09-10T22:00:33.212565+0000","last_active":"2024-09-10T22:00:34.185766+0000","last_peered":"2024-09-10T22:00:34.185766+0000","last_clean":"2024-09-10T22:00:34.185766+0000","last_became_active":"2024-09-10T22:00:33.210654+0000","last_became_peered":"2024-09-10T22:00:33.210654+0000","last_unstale":"2024-09-10T22:00:34.185766+0000","last_undegraded":"2024-09-10T22:00:34.185766+0000","last_fullsized":"2024-09-10T22:00:34.185766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:11.723619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663811+0000","last_change":"2024-09-10T22:00:33.192188+0000","last_active":"2024-09-10T22:00:34.663811+0000","last_peered":"2024-09-10T22:00:34.663811+0000","last_clean":"2024-09-10T22:00:34.663811+0000","last_became_active":"2024-09-10T22:00:33.191616+0000","last_became_peered":"2024-09-10T22:00:33.191616+0000","last_unstale":"2024-09-10T22:00:34.663811+0000","last_undegraded":"2024-09-10T22:00:34.663811+0000","last_fullsized":"2024-09-10T22:00:34.663811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:03.715049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665186+0000","last_change":"2024-09-10T22:00:33.203524+0000","last_active":"2024-09-10T22:00:34.665186+0000","last_peered":"2024-09-10T22:00:34.665186+0000","last_clean":"2024-09-10T22:00:34.665186+0000","last_became_active":"2024-09-10T22:00:33.202541+0000","last_became_peered":"2024-09-10T22:00:33.202541+0000","last_unstale":"2024-09-10T22:00:34.665186+0000","last_undegraded":"2024-09-10T22:00:34.665186+0000","last_fullsized":"2024-09-10T22:00:34.665186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:55.365445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185716+0000","last_change":"2024-09-10T22:00:33.192918+0000","last_active":"2024-09-10T22:00:34.185716+0000","last_peered":"2024-09-10T22:00:34.185716+0000","last_clean":"2024-09-10T22:00:34.185716+0000","last_became_active":"2024-09-10T22:00:33.192320+0000","last_became_peered":"2024-09-10T22:00:33.192320+0000","last_unstale":"2024-09-10T22:00:34.185716+0000","last_undegraded":"2024-09-10T22:00:34.185716+0000","last_fullsized":"2024-09-10T22:00:34.185716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:54.509264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664157+0000","last_change":"2024-09-10T22:00:33.186507+0000","last_active":"2024-09-10T22:00:34.664157+0000","last_peered":"2024-09-10T22:00:34.664157+0000","last_clean":"2024-09-10T22:00:34.664157+0000","last_became_active":"2024-09-10T22:00:33.186253+0000","last_became_peered":"2024-09-10T22:00:33.186253+0000","last_unstale":"2024-09-10T22:00:34.664157+0000","last_undegraded":"2024-09-10T22:00:34.664157+0000","last_fullsized":"2024-09-10T22:00:34.664157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:29.825070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185357+0000","last_change":"2024-09-10T22:00:33.213873+0000","last_active":"2024-09-10T22:00:34.185357+0000","last_peered":"2024-09-10T22:00:34.185357+0000","last_clean":"2024-09-10T22:00:34.185357+0000","last_became_active":"2024-09-10T22:00:33.211788+0000","last_became_peered":"2024-09-10T22:00:33.211788+0000","last_unstale":"2024-09-10T22:00:34.185357+0000","last_undegraded":"2024-09-10T22:00:34.185357+0000","last_fullsized":"2024-09-10T22:00:34.185357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:36.957181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663634+0000","last_change":"2024-09-10T22:00:33.182956+0000","last_active":"2024-09-10T22:00:34.663634+0000","last_peered":"2024-09-10T22:00:34.663634+0000","last_clean":"2024-09-10T22:00:34.663634+0000","last_became_active":"2024-09-10T22:00:33.182699+0000","last_became_peered":"2024-09-10T22:00:33.182699+0000","last_unstale":"2024-09-10T22:00:34.663634+0000","last_undegraded":"2024-09-10T22:00:34.663634+0000","last_fullsized":"2024-09-10T22:00:34.663634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:54.559113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184784+0000","last_change":"2024-09-10T22:00:33.208620+0000","last_active":"2024-09-10T22:00:34.184784+0000","last_peered":"2024-09-10T22:00:34.184784+0000","last_clean":"2024-09-10T22:00:34.184784+0000","last_became_active":"2024-09-10T22:00:33.208337+0000","last_became_peered":"2024-09-10T22:00:33.208337+0000","last_unstale":"2024-09-10T22:00:34.184784+0000","last_undegraded":"2024-09-10T22:00:34.184784+0000","last_fullsized":"2024-09-10T22:00:34.184784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:22.353564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664536+0000","last_change":"2024-09-10T22:00:33.185808+0000","last_active":"2024-09-10T22:00:34.664536+0000","last_peered":"2024-09-10T22:00:34.664536+0000","last_clean":"2024-09-10T22:00:34.664536+0000","last_became_active":"2024-09-10T22:00:33.185528+0000","last_became_peered":"2024-09-10T22:00:33.185528+0000","last_unstale":"2024-09-10T22:00:34.664536+0000","last_undegraded":"2024-09-10T22:00:34.664536+0000","last_fullsized":"2024-09-10T22:00:34.664536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:04.884964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187164+0000","last_change":"2024-09-10T22:00:33.191781+0000","last_active":"2024-09-10T22:00:34.187164+0000","last_peered":"2024-09-10T22:00:34.187164+0000","last_clean":"2024-09-10T22:00:34.187164+0000","last_became_active":"2024-09-10T22:00:33.191497+0000","last_became_peered":"2024-09-10T22:00:33.191497+0000","last_unstale":"2024-09-10T22:00:34.187164+0000","last_undegraded":"2024-09-10T22:00:34.187164+0000","last_fullsized":"2024-09-10T22:00:34.187164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:22.353564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183465+0000","last_change":"2024-09-10T22:00:33.183377+0000","last_active":"2024-09-10T22:00:34.183465+0000","last_peered":"2024-09-10T22:00:34.183465+0000","last_clean":"2024-09-10T22:00:34.183465+0000","last_became_active":"2024-09-10T22:00:33.181838+0000","last_became_peered":"2024-09-10T22:00:33.181838+0000","last_unstale":"2024-09-10T22:00:34.183465+0000","last_undegraded":"2024-09-10T22:00:34.183465+0000","last_fullsized":"2024-09-10T22:00:34.183465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:58.358390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663596+0000","last_change":"2024-09-10T22:00:33.185739+0000","last_active":"2024-09-10T22:00:34.663596+0000","last_peered":"2024-09-10T22:00:34.663596+0000","last_clean":"2024-09-10T22:00:34.663596+0000","last_became_active":"2024-09-10T22:00:33.185426+0000","last_became_peered":"2024-09-10T22:00:33.185426+0000","last_unstale":"2024-09-10T22:00:34.663596+0000","last_undegraded":"2024-09-10T22:00:34.663596+0000","last_fullsized":"2024-09-10T22:00:34.663596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:46.008480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664376+0000","last_change":"2024-09-10T22:00:33.183396+0000","last_active":"2024-09-10T22:00:34.664376+0000","last_peered":"2024-09-10T22:00:34.664376+0000","last_clean":"2024-09-10T22:00:34.664376+0000","last_became_active":"2024-09-10T22:00:33.181415+0000","last_became_peered":"2024-09-10T22:00:33.181415+0000","last_unstale":"2024-09-10T22:00:34.664376+0000","last_undegraded":"2024-09-10T22:00:34.664376+0000","last_fullsized":"2024-09-10T22:00:34.664376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:58.358390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185581+0000","last_change":"2024-09-10T22:00:33.202653+0000","last_active":"2024-09-10T22:00:34.185581+0000","last_peered":"2024-09-10T22:00:34.185581+0000","last_clean":"2024-09-10T22:00:34.185581+0000","last_became_active":"2024-09-10T22:00:33.202300+0000","last_became_peered":"2024-09-10T22:00:33.202300+0000","last_unstale":"2024-09-10T22:00:34.185581+0000","last_undegraded":"2024-09-10T22:00:34.185581+0000","last_fullsized":"2024-09-10T22:00:34.185581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:29.821720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663157+0000","last_change":"2024-09-10T22:00:33.213581+0000","last_active":"2024-09-10T22:00:34.663157+0000","last_peered":"2024-09-10T22:00:34.663157+0000","last_clean":"2024-09-10T22:00:34.663157+0000","last_became_active":"2024-09-10T22:00:33.213202+0000","last_became_peered":"2024-09-10T22:00:33.213202+0000","last_unstale":"2024-09-10T22:00:34.663157+0000","last_undegraded":"2024-09-10T22:00:34.663157+0000","last_fullsized":"2024-09-10T22:00:34.663157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:03.339805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184712+0000","last_change":"2024-09-10T22:00:33.179958+0000","last_active":"2024-09-10T22:00:34.184712+0000","last_peered":"2024-09-10T22:00:34.184712+0000","last_clean":"2024-09-10T22:00:34.184712+0000","last_became_active":"2024-09-10T22:00:33.179251+0000","last_became_peered":"2024-09-10T22:00:33.179251+0000","last_unstale":"2024-09-10T22:00:34.184712+0000","last_undegraded":"2024-09-10T22:00:34.184712+0000","last_fullsized":"2024-09-10T22:00:34.184712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:20.738322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664541+0000","last_change":"2024-09-10T22:00:33.181036+0000","last_active":"2024-09-10T22:00:34.664541+0000","last_peered":"2024-09-10T22:00:34.664541+0000","last_clean":"2024-09-10T22:00:34.664541+0000","last_became_active":"2024-09-10T22:00:33.179960+0000","last_became_peered":"2024-09-10T22:00:33.179960+0000","last_unstale":"2024-09-10T22:00:34.664541+0000","last_undegraded":"2024-09-10T22:00:34.664541+0000","last_fullsized":"2024-09-10T22:00:34.664541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:36.003731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663924+0000","last_change":"2024-09-10T22:00:32.187576+0000","last_active":"2024-09-10T22:00:34.663924+0000","last_peered":"2024-09-10T22:00:34.663924+0000","last_clean":"2024-09-10T22:00:34.663924+0000","last_became_active":"2024-09-10T22:00:32.187338+0000","last_became_peered":"2024-09-10T22:00:32.187338+0000","last_unstale":"2024-09-10T22:00:34.663924+0000","last_undegraded":"2024-09-10T22:00:34.663924+0000","last_fullsized":"2024-09-10T22:00:34.663924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:39.648880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664576+0000","last_change":"2024-09-10T22:00:33.184050+0000","last_active":"2024-09-10T22:00:34.664576+0000","last_peered":"2024-09-10T22:00:34.664576+0000","last_clean":"2024-09-10T22:00:34.664576+0000","last_became_active":"2024-09-10T22:00:33.183760+0000","last_became_peered":"2024-09-10T22:00:33.183760+0000","last_unstale":"2024-09-10T22:00:34.664576+0000","last_undegraded":"2024-09-10T22:00:34.664576+0000","last_fullsized":"2024-09-10T22:00:34.664576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:08.694915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186593+0000","last_change":"2024-09-10T22:00:32.181714+0000","last_active":"2024-09-10T22:00:34.186593+0000","last_peered":"2024-09-10T22:00:34.186593+0000","last_clean":"2024-09-10T22:00:34.186593+0000","last_became_active":"2024-09-10T22:00:32.180900+0000","last_became_peered":"2024-09-10T22:00:32.180900+0000","last_unstale":"2024-09-10T22:00:34.186593+0000","last_undegraded":"2024-09-10T22:00:34.186593+0000","last_fullsized":"2024-09-10T22:00:34.186593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:19.067912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663685+0000","last_change":"2024-09-10T22:00:32.194626+0000","last_active":"2024-09-10T22:00:34.663685+0000","last_peered":"2024-09-10T22:00:34.663685+0000","last_clean":"2024-09-10T22:00:34.663685+0000","last_became_active":"2024-09-10T22:00:32.194317+0000","last_became_peered":"2024-09-10T22:00:32.194317+0000","last_unstale":"2024-09-10T22:00:34.663685+0000","last_undegraded":"2024-09-10T22:00:34.663685+0000","last_fullsized":"2024-09-10T22:00:34.663685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:36.587242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663212+0000","last_change":"2024-09-10T22:00:33.180723+0000","last_active":"2024-09-10T22:00:34.663212+0000","last_peered":"2024-09-10T22:00:34.663212+0000","last_clean":"2024-09-10T22:00:34.663212+0000","last_became_active":"2024-09-10T22:00:33.180399+0000","last_became_peered":"2024-09-10T22:00:33.180399+0000","last_unstale":"2024-09-10T22:00:34.663212+0000","last_undegraded":"2024-09-10T22:00:34.663212+0000","last_fullsized":"2024-09-10T22:00:34.663212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:24.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184786+0000","last_change":"2024-09-10T22:00:32.184624+0000","last_active":"2024-09-10T22:00:34.184786+0000","last_peered":"2024-09-10T22:00:34.184786+0000","last_clean":"2024-09-10T22:00:34.184786+0000","last_became_active":"2024-09-10T22:00:32.184352+0000","last_became_peered":"2024-09-10T22:00:32.184352+0000","last_unstale":"2024-09-10T22:00:34.184786+0000","last_undegraded":"2024-09-10T22:00:34.184786+0000","last_fullsized":"2024-09-10T22:00:34.184786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:19.067912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183246+0000","last_change":"2024-09-10T22:00:33.178900+0000","last_active":"2024-09-10T22:00:34.183246+0000","last_peered":"2024-09-10T22:00:34.183246+0000","last_clean":"2024-09-10T22:00:34.183246+0000","last_became_active":"2024-09-10T22:00:33.178595+0000","last_became_peered":"2024-09-10T22:00:33.178595+0000","last_unstale":"2024-09-10T22:00:34.183246+0000","last_undegraded":"2024-09-10T22:00:34.183246+0000","last_fullsized":"2024-09-10T22:00:34.183246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:46.008480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187010+0000","last_change":"2024-09-10T22:00:32.193426+0000","last_active":"2024-09-10T22:00:34.187010+0000","last_peered":"2024-09-10T22:00:34.187010+0000","last_clean":"2024-09-10T22:00:34.187010+0000","last_became_active":"2024-09-10T22:00:32.189386+0000","last_became_peered":"2024-09-10T22:00:32.189386+0000","last_unstale":"2024-09-10T22:00:34.187010+0000","last_undegraded":"2024-09-10T22:00:34.187010+0000","last_fullsized":"2024-09-10T22:00:34.187010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:55.143270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185444+0000","last_change":"2024-09-10T22:00:33.213346+0000","last_active":"2024-09-10T22:00:34.185444+0000","last_peered":"2024-09-10T22:00:34.185444+0000","last_clean":"2024-09-10T22:00:34.185444+0000","last_became_active":"2024-09-10T22:00:33.213099+0000","last_became_peered":"2024-09-10T22:00:33.213099+0000","last_unstale":"2024-09-10T22:00:34.185444+0000","last_undegraded":"2024-09-10T22:00:34.185444+0000","last_fullsized":"2024-09-10T22:00:34.185444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:21.770779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185507+0000","last_change":"2024-09-10T22:00:32.195226+0000","last_active":"2024-09-10T22:00:34.185507+0000","last_peered":"2024-09-10T22:00:34.185507+0000","last_clean":"2024-09-10T22:00:34.185507+0000","last_became_active":"2024-09-10T22:00:32.194900+0000","last_became_peered":"2024-09-10T22:00:32.194900+0000","last_unstale":"2024-09-10T22:00:34.185507+0000","last_undegraded":"2024-09-10T22:00:34.185507+0000","last_fullsized":"2024-09-10T22:00:34.185507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:28.818521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663832+0000","last_change":"2024-09-10T22:00:33.213048+0000","last_active":"2024-09-10T22:00:34.663832+0000","last_peered":"2024-09-10T22:00:34.663832+0000","last_clean":"2024-09-10T22:00:34.663832+0000","last_became_active":"2024-09-10T22:00:33.212747+0000","last_became_peered":"2024-09-10T22:00:33.212747+0000","last_unstale":"2024-09-10T22:00:34.663832+0000","last_undegraded":"2024-09-10T22:00:34.663832+0000","last_fullsized":"2024-09-10T22:00:34.663832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:46.008480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183502+0000","last_change":"2024-09-10T22:00:32.194672+0000","last_active":"2024-09-10T22:00:34.183502+0000","last_peered":"2024-09-10T22:00:34.183502+0000","last_clean":"2024-09-10T22:00:34.183502+0000","last_became_active":"2024-09-10T22:00:32.194084+0000","last_became_peered":"2024-09-10T22:00:32.194084+0000","last_unstale":"2024-09-10T22:00:34.183502+0000","last_undegraded":"2024-09-10T22:00:34.183502+0000","last_fullsized":"2024-09-10T22:00:34.183502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:09.395633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665544+0000","last_change":"2024-09-10T22:00:33.176539+0000","last_active":"2024-09-10T22:00:34.665544+0000","last_peered":"2024-09-10T22:00:34.665544+0000","last_clean":"2024-09-10T22:00:34.665544+0000","last_became_active":"2024-09-10T22:00:33.175809+0000","last_became_peered":"2024-09-10T22:00:33.175809+0000","last_unstale":"2024-09-10T22:00:34.665544+0000","last_undegraded":"2024-09-10T22:00:34.665544+0000","last_fullsized":"2024-09-10T22:00:34.665544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:22.046625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185783+0000","last_change":"2024-09-10T22:00:32.187798+0000","last_active":"2024-09-10T22:00:34.185783+0000","last_peered":"2024-09-10T22:00:34.185783+0000","last_clean":"2024-09-10T22:00:34.185783+0000","last_became_active":"2024-09-10T22:00:32.184530+0000","last_became_peered":"2024-09-10T22:00:32.184530+0000","last_unstale":"2024-09-10T22:00:34.185783+0000","last_undegraded":"2024-09-10T22:00:34.185783+0000","last_fullsized":"2024-09-10T22:00:34.185783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:41.444694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664461+0000","last_change":"2024-09-10T22:00:33.189951+0000","last_active":"2024-09-10T22:00:34.664461+0000","last_peered":"2024-09-10T22:00:34.664461+0000","last_clean":"2024-09-10T22:00:34.664461+0000","last_became_active":"2024-09-10T22:00:33.189183+0000","last_became_peered":"2024-09-10T22:00:33.189183+0000","last_unstale":"2024-09-10T22:00:34.664461+0000","last_undegraded":"2024-09-10T22:00:34.664461+0000","last_fullsized":"2024-09-10T22:00:34.664461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:04.884964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664660+0000","last_change":"2024-09-10T22:00:32.171323+0000","last_active":"2024-09-10T22:00:34.664660+0000","last_peered":"2024-09-10T22:00:34.664660+0000","last_clean":"2024-09-10T22:00:34.664660+0000","last_became_active":"2024-09-10T22:00:32.171059+0000","last_became_peered":"2024-09-10T22:00:32.171059+0000","last_unstale":"2024-09-10T22:00:34.664660+0000","last_undegraded":"2024-09-10T22:00:34.664660+0000","last_fullsized":"2024-09-10T22:00:34.664660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:44.404901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663672+0000","last_change":"2024-09-10T22:00:33.196013+0000","last_active":"2024-09-10T22:00:34.663672+0000","last_peered":"2024-09-10T22:00:34.663672+0000","last_clean":"2024-09-10T22:00:34.663672+0000","last_became_active":"2024-09-10T22:00:33.195744+0000","last_became_peered":"2024-09-10T22:00:33.195744+0000","last_unstale":"2024-09-10T22:00:34.663672+0000","last_undegraded":"2024-09-10T22:00:34.663672+0000","last_fullsized":"2024-09-10T22:00:34.663672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:40.305055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185639+0000","last_change":"2024-09-10T22:00:32.196550+0000","last_active":"2024-09-10T22:00:34.185639+0000","last_peered":"2024-09-10T22:00:34.185639+0000","last_clean":"2024-09-10T22:00:34.185639+0000","last_became_active":"2024-09-10T22:00:32.196255+0000","last_became_peered":"2024-09-10T22:00:32.196255+0000","last_unstale":"2024-09-10T22:00:34.185639+0000","last_undegraded":"2024-09-10T22:00:34.185639+0000","last_fullsized":"2024-09-10T22:00:34.185639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:27.356827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665144+0000","last_change":"2024-09-10T22:00:33.197537+0000","last_active":"2024-09-10T22:00:34.665144+0000","last_peered":"2024-09-10T22:00:34.665144+0000","last_clean":"2024-09-10T22:00:34.665144+0000","last_became_active":"2024-09-10T22:00:33.197230+0000","last_became_peered":"2024-09-10T22:00:33.197230+0000","last_unstale":"2024-09-10T22:00:34.665144+0000","last_undegraded":"2024-09-10T22:00:34.665144+0000","last_fullsized":"2024-09-10T22:00:34.665144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:57.404481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184926+0000","last_change":"2024-09-10T22:00:32.187773+0000","last_active":"2024-09-10T22:00:34.184926+0000","last_peered":"2024-09-10T22:00:34.184926+0000","last_clean":"2024-09-10T22:00:34.184926+0000","last_became_active":"2024-09-10T22:00:32.185748+0000","last_became_peered":"2024-09-10T22:00:32.185748+0000","last_unstale":"2024-09-10T22:00:34.184926+0000","last_undegraded":"2024-09-10T22:00:34.184926+0000","last_fullsized":"2024-09-10T22:00:34.184926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:39.648880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184928+0000","last_change":"2024-09-10T22:00:33.180297+0000","last_active":"2024-09-10T22:00:34.184928+0000","last_peered":"2024-09-10T22:00:34.184928+0000","last_clean":"2024-09-10T22:00:34.184928+0000","last_became_active":"2024-09-10T22:00:33.179643+0000","last_became_peered":"2024-09-10T22:00:33.179643+0000","last_unstale":"2024-09-10T22:00:34.184928+0000","last_undegraded":"2024-09-10T22:00:34.184928+0000","last_fullsized":"2024-09-10T22:00:34.184928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:06.332661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.662989+0000","last_change":"2024-09-10T22:00:32.182584+0000","last_active":"2024-09-10T22:00:34.662989+0000","last_peered":"2024-09-10T22:00:34.662989+0000","last_clean":"2024-09-10T22:00:34.662989+0000","last_became_active":"2024-09-10T22:00:32.182289+0000","last_became_peered":"2024-09-10T22:00:32.182289+0000","last_unstale":"2024-09-10T22:00:34.662989+0000","last_undegraded":"2024-09-10T22:00:34.662989+0000","last_fullsized":"2024-09-10T22:00:34.662989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:41.773634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664524+0000","last_change":"2024-09-10T22:00:33.189852+0000","last_active":"2024-09-10T22:00:34.664524+0000","last_peered":"2024-09-10T22:00:34.664524+0000","last_clean":"2024-09-10T22:00:34.664524+0000","last_became_active":"2024-09-10T22:00:33.189492+0000","last_became_peered":"2024-09-10T22:00:33.189492+0000","last_unstale":"2024-09-10T22:00:34.664524+0000","last_undegraded":"2024-09-10T22:00:34.664524+0000","last_fullsized":"2024-09-10T22:00:34.664524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:45.010893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665636+0000","last_change":"2024-09-10T22:00:32.179367+0000","last_active":"2024-09-10T22:00:34.665636+0000","last_peered":"2024-09-10T22:00:34.665636+0000","last_clean":"2024-09-10T22:00:34.665636+0000","last_became_active":"2024-09-10T22:00:32.177804+0000","last_became_peered":"2024-09-10T22:00:32.177804+0000","last_unstale":"2024-09-10T22:00:34.665636+0000","last_undegraded":"2024-09-10T22:00:34.665636+0000","last_fullsized":"2024-09-10T22:00:34.665636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:13.368508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665546+0000","last_change":"2024-09-10T22:00:33.194213+0000","last_active":"2024-09-10T22:00:34.665546+0000","last_peered":"2024-09-10T22:00:34.665546+0000","last_clean":"2024-09-10T22:00:34.665546+0000","last_became_active":"2024-09-10T22:00:33.193708+0000","last_became_peered":"2024-09-10T22:00:33.193708+0000","last_unstale":"2024-09-10T22:00:34.665546+0000","last_undegraded":"2024-09-10T22:00:34.665546+0000","last_fullsized":"2024-09-10T22:00:34.665546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:39.533698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663916+0000","last_change":"2024-09-10T22:00:32.184130+0000","last_active":"2024-09-10T22:00:34.663916+0000","last_peered":"2024-09-10T22:00:34.663916+0000","last_clean":"2024-09-10T22:00:34.663916+0000","last_became_active":"2024-09-10T22:00:32.182524+0000","last_became_peered":"2024-09-10T22:00:32.182524+0000","last_unstale":"2024-09-10T22:00:34.663916+0000","last_undegraded":"2024-09-10T22:00:34.663916+0000","last_fullsized":"2024-09-10T22:00:34.663916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:26.056799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665889+0000","last_change":"2024-09-10T22:00:33.198846+0000","last_active":"2024-09-10T22:00:34.665889+0000","last_peered":"2024-09-10T22:00:34.665889+0000","last_clean":"2024-09-10T22:00:34.665889+0000","last_became_active":"2024-09-10T22:00:33.198166+0000","last_became_peered":"2024-09-10T22:00:33.198166+0000","last_unstale":"2024-09-10T22:00:34.665889+0000","last_undegraded":"2024-09-10T22:00:34.665889+0000","last_fullsized":"2024-09-10T22:00:34.665889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:21.862163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183552+0000","last_change":"2024-09-10T22:00:32.181657+0000","last_active":"2024-09-10T22:00:34.183552+0000","last_peered":"2024-09-10T22:00:34.183552+0000","last_clean":"2024-09-10T22:00:34.183552+0000","last_became_active":"2024-09-10T22:00:32.181365+0000","last_became_peered":"2024-09-10T22:00:32.181365+0000","last_unstale":"2024-09-10T22:00:34.183552+0000","last_undegraded":"2024-09-10T22:00:34.183552+0000","last_fullsized":"2024-09-10T22:00:34.183552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:34.356560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185996+0000","last_change":"2024-09-10T22:00:33.215101+0000","last_active":"2024-09-10T22:00:34.185996+0000","last_peered":"2024-09-10T22:00:34.185996+0000","last_clean":"2024-09-10T22:00:34.185996+0000","last_became_active":"2024-09-10T22:00:33.214294+0000","last_became_peered":"2024-09-10T22:00:33.214294+0000","last_unstale":"2024-09-10T22:00:34.185996+0000","last_undegraded":"2024-09-10T22:00:34.185996+0000","last_fullsized":"2024-09-10T22:00:34.185996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:59.470008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663655+0000","last_change":"2024-09-10T22:00:32.195097+0000","last_active":"2024-09-10T22:00:34.663655+0000","last_peered":"2024-09-10T22:00:34.663655+0000","last_clean":"2024-09-10T22:00:34.663655+0000","last_became_active":"2024-09-10T22:00:32.194103+0000","last_became_peered":"2024-09-10T22:00:32.194103+0000","last_unstale":"2024-09-10T22:00:34.663655+0000","last_undegraded":"2024-09-10T22:00:34.663655+0000","last_fullsized":"2024-09-10T22:00:34.663655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:57.181450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664931+0000","last_change":"2024-09-10T22:00:33.212536+0000","last_active":"2024-09-10T22:00:34.664931+0000","last_peered":"2024-09-10T22:00:34.664931+0000","last_clean":"2024-09-10T22:00:34.664931+0000","last_became_active":"2024-09-10T22:00:33.212204+0000","last_became_peered":"2024-09-10T22:00:33.212204+0000","last_unstale":"2024-09-10T22:00:34.664931+0000","last_undegraded":"2024-09-10T22:00:34.664931+0000","last_fullsized":"2024-09-10T22:00:34.664931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:41.803895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664227+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:00:34.664227+0000","last_peered":"2024-09-10T22:00:34.664227+0000","last_clean":"2024-09-10T22:00:34.664227+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:00:34.664227+0000","last_undegraded":"2024-09-10T22:00:34.664227+0000","last_fullsized":"2024-09-10T22:00:34.664227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185132+0000","last_change":"2024-09-10T22:00:32.195314+0000","last_active":"2024-09-10T22:00:34.185132+0000","last_peered":"2024-09-10T22:00:34.185132+0000","last_clean":"2024-09-10T22:00:34.185132+0000","last_became_active":"2024-09-10T22:00:32.194960+0000","last_became_peered":"2024-09-10T22:00:32.194960+0000","last_unstale":"2024-09-10T22:00:34.185132+0000","last_undegraded":"2024-09-10T22:00:34.185132+0000","last_fullsized":"2024-09-10T22:00:34.185132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:01.067800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663524+0000","last_change":"2024-09-10T22:00:33.208452+0000","last_active":"2024-09-10T22:00:34.663524+0000","last_peered":"2024-09-10T22:00:34.663524+0000","last_clean":"2024-09-10T22:00:34.663524+0000","last_became_active":"2024-09-10T22:00:33.208146+0000","last_became_peered":"2024-09-10T22:00:33.208146+0000","last_unstale":"2024-09-10T22:00:34.663524+0000","last_undegraded":"2024-09-10T22:00:34.663524+0000","last_fullsized":"2024-09-10T22:00:34.663524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:40.305055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664710+0000","last_change":"2024-09-10T22:00:32.192173+0000","last_active":"2024-09-10T22:00:34.664710+0000","last_peered":"2024-09-10T22:00:34.664710+0000","last_clean":"2024-09-10T22:00:34.664710+0000","last_became_active":"2024-09-10T22:00:32.191792+0000","last_became_peered":"2024-09-10T22:00:32.191792+0000","last_unstale":"2024-09-10T22:00:34.664710+0000","last_undegraded":"2024-09-10T22:00:34.664710+0000","last_fullsized":"2024-09-10T22:00:34.664710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:34.356560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665272+0000","last_change":"2024-09-10T22:00:33.174576+0000","last_active":"2024-09-10T22:00:34.665272+0000","last_peered":"2024-09-10T22:00:34.665272+0000","last_clean":"2024-09-10T22:00:34.665272+0000","last_became_active":"2024-09-10T22:00:33.174232+0000","last_became_peered":"2024-09-10T22:00:33.174232+0000","last_unstale":"2024-09-10T22:00:34.665272+0000","last_undegraded":"2024-09-10T22:00:34.665272+0000","last_fullsized":"2024-09-10T22:00:34.665272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:55.992463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663264+0000","last_change":"2024-09-10T22:00:32.178337+0000","last_active":"2024-09-10T22:00:34.663264+0000","last_peered":"2024-09-10T22:00:34.663264+0000","last_clean":"2024-09-10T22:00:34.663264+0000","last_became_active":"2024-09-10T22:00:32.178053+0000","last_became_peered":"2024-09-10T22:00:32.178053+0000","last_unstale":"2024-09-10T22:00:34.663264+0000","last_undegraded":"2024-09-10T22:00:34.663264+0000","last_fullsized":"2024-09-10T22:00:34.663264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:26.949229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183185+0000","last_change":"2024-09-10T22:00:33.184308+0000","last_active":"2024-09-10T22:00:34.183185+0000","last_peered":"2024-09-10T22:00:34.183185+0000","last_clean":"2024-09-10T22:00:34.183185+0000","last_became_active":"2024-09-10T22:00:33.184050+0000","last_became_peered":"2024-09-10T22:00:33.184050+0000","last_unstale":"2024-09-10T22:00:34.183185+0000","last_undegraded":"2024-09-10T22:00:34.183185+0000","last_fullsized":"2024-09-10T22:00:34.183185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:40.305055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185742+0000","last_change":"2024-09-10T22:00:32.195663+0000","last_active":"2024-09-10T22:00:34.185742+0000","last_peered":"2024-09-10T22:00:34.185742+0000","last_clean":"2024-09-10T22:00:34.185742+0000","last_became_active":"2024-09-10T22:00:32.195368+0000","last_became_peered":"2024-09-10T22:00:32.195368+0000","last_unstale":"2024-09-10T22:00:34.185742+0000","last_undegraded":"2024-09-10T22:00:34.185742+0000","last_fullsized":"2024-09-10T22:00:34.185742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:56.084833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664779+0000","last_change":"2024-09-10T22:00:33.186193+0000","last_active":"2024-09-10T22:00:34.664779+0000","last_peered":"2024-09-10T22:00:34.664779+0000","last_clean":"2024-09-10T22:00:34.664779+0000","last_became_active":"2024-09-10T22:00:33.180505+0000","last_became_peered":"2024-09-10T22:00:33.180505+0000","last_unstale":"2024-09-10T22:00:34.664779+0000","last_undegraded":"2024-09-10T22:00:34.664779+0000","last_fullsized":"2024-09-10T22:00:34.664779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:10.402182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183297+0000","last_change":"2024-09-10T22:00:32.184540+0000","last_active":"2024-09-10T22:00:34.183297+0000","last_peered":"2024-09-10T22:00:34.183297+0000","last_clean":"2024-09-10T22:00:34.183297+0000","last_became_active":"2024-09-10T22:00:32.184213+0000","last_became_peered":"2024-09-10T22:00:32.184213+0000","last_unstale":"2024-09-10T22:00:34.183297+0000","last_undegraded":"2024-09-10T22:00:34.183297+0000","last_fullsized":"2024-09-10T22:00:34.183297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:21.040076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186341+0000","last_change":"2024-09-10T22:00:33.212897+0000","last_active":"2024-09-10T22:00:34.186341+0000","last_peered":"2024-09-10T22:00:34.186341+0000","last_clean":"2024-09-10T22:00:34.186341+0000","last_became_active":"2024-09-10T22:00:33.212668+0000","last_became_peered":"2024-09-10T22:00:33.212668+0000","last_unstale":"2024-09-10T22:00:34.186341+0000","last_undegraded":"2024-09-10T22:00:34.186341+0000","last_fullsized":"2024-09-10T22:00:34.186341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:47.684127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665052+0000","last_change":"2024-09-10T22:00:32.193491+0000","last_active":"2024-09-10T22:00:34.665052+0000","last_peered":"2024-09-10T22:00:34.665052+0000","last_clean":"2024-09-10T22:00:34.665052+0000","last_became_active":"2024-09-10T22:00:32.193147+0000","last_became_peered":"2024-09-10T22:00:32.193147+0000","last_unstale":"2024-09-10T22:00:34.665052+0000","last_undegraded":"2024-09-10T22:00:34.665052+0000","last_fullsized":"2024-09-10T22:00:34.665052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:51.181211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183356+0000","last_change":"2024-09-10T22:00:33.189064+0000","last_active":"2024-09-10T22:00:34.183356+0000","last_peered":"2024-09-10T22:00:34.183356+0000","last_clean":"2024-09-10T22:00:34.183356+0000","last_became_active":"2024-09-10T22:00:33.188733+0000","last_became_peered":"2024-09-10T22:00:33.188733+0000","last_unstale":"2024-09-10T22:00:34.183356+0000","last_undegraded":"2024-09-10T22:00:34.183356+0000","last_fullsized":"2024-09-10T22:00:34.183356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:29.825070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183468+0000","last_change":"2024-09-10T22:00:32.186782+0000","last_active":"2024-09-10T22:00:34.183468+0000","last_peered":"2024-09-10T22:00:34.183468+0000","last_clean":"2024-09-10T22:00:34.183468+0000","last_became_active":"2024-09-10T22:00:32.186028+0000","last_became_peered":"2024-09-10T22:00:32.186028+0000","last_unstale":"2024-09-10T22:00:34.183468+0000","last_undegraded":"2024-09-10T22:00:34.183468+0000","last_fullsized":"2024-09-10T22:00:34.183468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:34.997182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664559+0000","last_change":"2024-09-10T22:00:33.201547+0000","last_active":"2024-09-10T22:00:34.664559+0000","last_peered":"2024-09-10T22:00:34.664559+0000","last_clean":"2024-09-10T22:00:34.664559+0000","last_became_active":"2024-09-10T22:00:33.201302+0000","last_became_peered":"2024-09-10T22:00:33.201302+0000","last_unstale":"2024-09-10T22:00:34.664559+0000","last_undegraded":"2024-09-10T22:00:34.664559+0000","last_fullsized":"2024-09-10T22:00:34.664559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:45.010893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187107+0000","last_change":"2024-09-10T22:00:32.196354+0000","last_active":"2024-09-10T22:00:34.187107+0000","last_peered":"2024-09-10T22:00:34.187107+0000","last_clean":"2024-09-10T22:00:34.187107+0000","last_became_active":"2024-09-10T22:00:32.196053+0000","last_became_peered":"2024-09-10T22:00:32.196053+0000","last_unstale":"2024-09-10T22:00:34.187107+0000","last_undegraded":"2024-09-10T22:00:34.187107+0000","last_fullsized":"2024-09-10T22:00:34.187107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:37.446124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663935+0000","last_change":"2024-09-10T22:00:33.176478+0000","last_active":"2024-09-10T22:00:34.663935+0000","last_peered":"2024-09-10T22:00:34.663935+0000","last_clean":"2024-09-10T22:00:34.663935+0000","last_became_active":"2024-09-10T22:00:33.176211+0000","last_became_peered":"2024-09-10T22:00:33.176211+0000","last_unstale":"2024-09-10T22:00:34.663935+0000","last_undegraded":"2024-09-10T22:00:34.663935+0000","last_fullsized":"2024-09-10T22:00:34.663935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:54.559113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664088+0000","last_change":"2024-09-10T22:00:32.189994+0000","last_active":"2024-09-10T22:00:34.664088+0000","last_peered":"2024-09-10T22:00:34.664088+0000","last_clean":"2024-09-10T22:00:34.664088+0000","last_became_active":"2024-09-10T22:00:32.189654+0000","last_became_peered":"2024-09-10T22:00:32.189654+0000","last_unstale":"2024-09-10T22:00:34.664088+0000","last_undegraded":"2024-09-10T22:00:34.664088+0000","last_fullsized":"2024-09-10T22:00:34.664088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:34.987281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664635+0000","last_change":"2024-09-10T22:00:33.183489+0000","last_active":"2024-09-10T22:00:34.664635+0000","last_peered":"2024-09-10T22:00:34.664635+0000","last_clean":"2024-09-10T22:00:34.664635+0000","last_became_active":"2024-09-10T22:00:33.183217+0000","last_became_peered":"2024-09-10T22:00:33.183217+0000","last_unstale":"2024-09-10T22:00:34.664635+0000","last_undegraded":"2024-09-10T22:00:34.664635+0000","last_fullsized":"2024-09-10T22:00:34.664635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:35.993830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183297+0000","last_change":"2024-09-10T22:00:32.178130+0000","last_active":"2024-09-10T22:00:34.183297+0000","last_peered":"2024-09-10T22:00:34.183297+0000","last_clean":"2024-09-10T22:00:34.183297+0000","last_became_active":"2024-09-10T22:00:32.177879+0000","last_became_peered":"2024-09-10T22:00:32.177879+0000","last_unstale":"2024-09-10T22:00:34.183297+0000","last_undegraded":"2024-09-10T22:00:34.183297+0000","last_fullsized":"2024-09-10T22:00:34.183297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:34.987281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664090+0000","last_change":"2024-09-10T22:00:33.179961+0000","last_active":"2024-09-10T22:00:34.664090+0000","last_peered":"2024-09-10T22:00:34.664090+0000","last_clean":"2024-09-10T22:00:34.664090+0000","last_became_active":"2024-09-10T22:00:33.179700+0000","last_became_peered":"2024-09-10T22:00:33.179700+0000","last_unstale":"2024-09-10T22:00:34.664090+0000","last_undegraded":"2024-09-10T22:00:34.664090+0000","last_fullsized":"2024-09-10T22:00:34.664090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:08.694915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183241+0000","last_change":"2024-09-10T22:00:32.180382+0000","last_active":"2024-09-10T22:00:34.183241+0000","last_peered":"2024-09-10T22:00:34.183241+0000","last_clean":"2024-09-10T22:00:34.183241+0000","last_became_active":"2024-09-10T22:00:32.180075+0000","last_became_peered":"2024-09-10T22:00:32.180075+0000","last_unstale":"2024-09-10T22:00:34.183241+0000","last_undegraded":"2024-09-10T22:00:34.183241+0000","last_fullsized":"2024-09-10T22:00:34.183241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:54.640505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664041+0000","last_change":"2024-09-10T22:00:33.180424+0000","last_active":"2024-09-10T22:00:34.664041+0000","last_peered":"2024-09-10T22:00:34.664041+0000","last_clean":"2024-09-10T22:00:34.664041+0000","last_became_active":"2024-09-10T22:00:33.180181+0000","last_became_peered":"2024-09-10T22:00:33.180181+0000","last_unstale":"2024-09-10T22:00:34.664041+0000","last_undegraded":"2024-09-10T22:00:34.664041+0000","last_fullsized":"2024-09-10T22:00:34.664041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:58.358390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665152+0000","last_change":"2024-09-10T22:00:32.191652+0000","last_active":"2024-09-10T22:00:34.665152+0000","last_peered":"2024-09-10T22:00:34.665152+0000","last_clean":"2024-09-10T22:00:34.665152+0000","last_became_active":"2024-09-10T22:00:32.191353+0000","last_became_peered":"2024-09-10T22:00:32.191353+0000","last_unstale":"2024-09-10T22:00:34.665152+0000","last_undegraded":"2024-09-10T22:00:34.665152+0000","last_fullsized":"2024-09-10T22:00:34.665152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:21.040076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184746+0000","last_change":"2024-09-10T22:00:33.184285+0000","last_active":"2024-09-10T22:00:34.184746+0000","last_peered":"2024-09-10T22:00:34.184746+0000","last_clean":"2024-09-10T22:00:34.184746+0000","last_became_active":"2024-09-10T22:00:33.183977+0000","last_became_peered":"2024-09-10T22:00:33.183977+0000","last_unstale":"2024-09-10T22:00:34.184746+0000","last_undegraded":"2024-09-10T22:00:34.184746+0000","last_fullsized":"2024-09-10T22:00:34.184746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:30.181250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186284+0000","last_change":"2024-09-10T22:00:33.207897+0000","last_active":"2024-09-10T22:00:34.186284+0000","last_peered":"2024-09-10T22:00:34.186284+0000","last_clean":"2024-09-10T22:00:34.186284+0000","last_became_active":"2024-09-10T22:00:33.207639+0000","last_became_peered":"2024-09-10T22:00:33.207639+0000","last_unstale":"2024-09-10T22:00:34.186284+0000","last_undegraded":"2024-09-10T22:00:34.186284+0000","last_fullsized":"2024-09-10T22:00:34.186284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:47.952904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664351+0000","last_change":"2024-09-10T22:00:32.199875+0000","last_active":"2024-09-10T22:00:34.664351+0000","last_peered":"2024-09-10T22:00:34.664351+0000","last_clean":"2024-09-10T22:00:34.664351+0000","last_became_active":"2024-09-10T22:00:32.199625+0000","last_became_peered":"2024-09-10T22:00:32.199625+0000","last_unstale":"2024-09-10T22:00:34.664351+0000","last_undegraded":"2024-09-10T22:00:34.664351+0000","last_fullsized":"2024-09-10T22:00:34.664351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:09.395633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185863+0000","last_change":"2024-09-10T22:00:33.207063+0000","last_active":"2024-09-10T22:00:34.185863+0000","last_peered":"2024-09-10T22:00:34.185863+0000","last_clean":"2024-09-10T22:00:34.185863+0000","last_became_active":"2024-09-10T22:00:33.206867+0000","last_became_peered":"2024-09-10T22:00:33.206867+0000","last_unstale":"2024-09-10T22:00:34.185863+0000","last_undegraded":"2024-09-10T22:00:34.185863+0000","last_fullsized":"2024-09-10T22:00:34.185863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:44.640052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664917+0000","last_change":"2024-09-10T22:00:32.196714+0000","last_active":"2024-09-10T22:00:34.664917+0000","last_peered":"2024-09-10T22:00:34.664917+0000","last_clean":"2024-09-10T22:00:34.664917+0000","last_became_active":"2024-09-10T22:00:32.196446+0000","last_became_peered":"2024-09-10T22:00:32.196446+0000","last_unstale":"2024-09-10T22:00:34.664917+0000","last_undegraded":"2024-09-10T22:00:34.664917+0000","last_fullsized":"2024-09-10T22:00:34.664917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:39.201788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663624+0000","last_change":"2024-09-10T22:00:33.215295+0000","last_active":"2024-09-10T22:00:34.663624+0000","last_peered":"2024-09-10T22:00:34.663624+0000","last_clean":"2024-09-10T22:00:34.663624+0000","last_became_active":"2024-09-10T22:00:33.215041+0000","last_became_peered":"2024-09-10T22:00:33.215041+0000","last_unstale":"2024-09-10T22:00:34.663624+0000","last_undegraded":"2024-09-10T22:00:34.663624+0000","last_fullsized":"2024-09-10T22:00:34.663624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:24.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664333+0000","last_change":"2024-09-10T22:00:32.180489+0000","last_active":"2024-09-10T22:00:34.664333+0000","last_peered":"2024-09-10T22:00:34.664333+0000","last_clean":"2024-09-10T22:00:34.664333+0000","last_became_active":"2024-09-10T22:00:32.180177+0000","last_became_peered":"2024-09-10T22:00:32.180177+0000","last_unstale":"2024-09-10T22:00:34.664333+0000","last_undegraded":"2024-09-10T22:00:34.664333+0000","last_fullsized":"2024-09-10T22:00:34.664333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:57.175682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664468+0000","last_change":"2024-09-10T22:00:33.192548+0000","last_active":"2024-09-10T22:00:34.664468+0000","last_peered":"2024-09-10T22:00:34.664468+0000","last_clean":"2024-09-10T22:00:34.664468+0000","last_became_active":"2024-09-10T22:00:33.192272+0000","last_became_peered":"2024-09-10T22:00:33.192272+0000","last_unstale":"2024-09-10T22:00:34.664468+0000","last_undegraded":"2024-09-10T22:00:34.664468+0000","last_fullsized":"2024-09-10T22:00:34.664468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:25.562048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663908+0000","last_change":"2024-09-10T22:00:32.182534+0000","last_active":"2024-09-10T22:00:34.663908+0000","last_peered":"2024-09-10T22:00:34.663908+0000","last_clean":"2024-09-10T22:00:34.663908+0000","last_became_active":"2024-09-10T22:00:32.182286+0000","last_became_peered":"2024-09-10T22:00:32.182286+0000","last_unstale":"2024-09-10T22:00:34.663908+0000","last_undegraded":"2024-09-10T22:00:34.663908+0000","last_fullsized":"2024-09-10T22:00:34.663908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:56.418214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":667234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":225280,"data_stored":76374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":76410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":827392,"data_stored":667290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:00:45.364 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:00:45.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.542+0000 7fa616600640 1 -- 172.21.15.66:0/1836006999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa61015ef20 msgr2=0x7fa6100cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.542+0000 7fa616600640 1 --2- 172.21.15.66:0/1836006999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa61015ef20 0x7fa6100cecc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa604009900 tx=0x7fa60401c8a0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.542+0000 7fa616600640 1 -- 172.21.15.66:0/1836006999 shutdown_connections 2024-09-10T22:00:45.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.542+0000 7fa616600640 1 --2- 172.21.15.66:0/1836006999 >> v2:172.21.15.94:3300/0 conn(0x7fa6100cd7a0 0x7fa6100cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.542+0000 7fa616600640 1 --2- 172.21.15.66:0/1836006999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa61015ef20 0x7fa6100cecc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.542+0000 7fa616600640 1 -- 172.21.15.66:0/1836006999 >> 172.21.15.66:0/1836006999 conn(0x7fa6100d1b10 msgr2=0x7fa6100d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:45.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.542+0000 7fa616600640 1 -- 172.21.15.66:0/1836006999 shutdown_connections 2024-09-10T22:00:45.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.542+0000 7fa616600640 1 -- 172.21.15.66:0/1836006999 wait complete. 2024-09-10T22:00:45.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.542+0000 7fa616600640 1 Processor -- start 2024-09-10T22:00:45.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.542+0000 7fa616600640 1 -- start start 2024-09-10T22:00:45.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa616600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6100cd7a0 0x7fa6101f8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:45.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa616600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa61015ef20 0x7fa61014f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa616600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa61015f300 0x7fa61006e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa616600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa610168ee0 con 0x7fa61015f300 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa616600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa610168d60 con 0x7fa6100cd7a0 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa616600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa610169060 con 0x7fa61015ef20 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa615c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa61015f300 0x7fa61006e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa615c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa61015f300 0x7fa61006e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45790/0 (socket says 172.21.15.66:45790) 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa615c00640 1 -- 172.21.15.66:0/1021510363 learned_addr learned my addr 172.21.15.66:0/1021510363 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa615200640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6100cd7a0 0x7fa6101f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa60fe00640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa61015ef20 0x7fa61014f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa615c00640 1 -- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa61015ef20 msgr2=0x7fa61014f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa615c00640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa61015ef20 0x7fa61014f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa615c00640 1 -- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6100cd7a0 msgr2=0x7fa6101f8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa615c00640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6100cd7a0 0x7fa6101f8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa615c00640 1 -- 172.21.15.66:0/1021510363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6101f89b0 con 0x7fa61015f300 2024-09-10T22:00:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa60fe00640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa61015ef20 0x7fa61014f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa615c00640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa61015f300 0x7fa61006e4e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa60800cce0 tx=0x7fa60800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa60cc00640 1 -- 172.21.15.66:0/1021510363 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60800f450 con 0x7fa61015f300 2024-09-10T22:00:45.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa60cc00640 1 -- 172.21.15.66:0/1021510363 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa60800f700 con 0x7fa61015f300 2024-09-10T22:00:45.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa60cc00640 1 -- 172.21.15.66:0/1021510363 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60800fb50 con 0x7fa61015f300 2024-09-10T22:00:45.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa616600640 1 -- 172.21.15.66:0/1021510363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6100c8f60 con 0x7fa61015f300 2024-09-10T22:00:45.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa616600640 1 -- 172.21.15.66:0/1021510363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa610071c00 con 0x7fa61015f300 2024-09-10T22:00:45.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.546+0000 7fa616600640 1 -- 172.21.15.66:0/1021510363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6100cdc10 con 0x7fa61015f300 2024-09-10T22:00:45.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.550+0000 7fa60cc00640 1 -- 172.21.15.66:0/1021510363 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa60800e3b0 con 0x7fa61015f300 2024-09-10T22:00:45.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.550+0000 7fa60cc00640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa5f00725e0 0x7fa5f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:45.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.550+0000 7fa60cc00640 1 -- 172.21.15.66:0/1021510363 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa608093dc0 con 0x7fa61015f300 2024-09-10T22:00:45.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.550+0000 7fa615200640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa5f00725e0 0x7fa5f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:45.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.550+0000 7fa615200640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa5f00725e0 0x7fa5f0074aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa6040af000 tx=0x7fa60401b0e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:45.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.554+0000 7fa60cc00640 1 -- 172.21.15.66:0/1021510363 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa608014030 con 0x7fa61015f300 2024-09-10T22:00:45.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.722+0000 7fa616600640 1 -- 172.21.15.66:0/1021510363 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6100d2680 con 0x7fa5f00725e0 2024-09-10T22:00:45.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.730+0000 7fa60cc00640 1 -- 172.21.15.66:0/1021510363 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7fa6100d2680 con 0x7fa5f00725e0 2024-09-10T22:00:45.737 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:45.738 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:00:45.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.730+0000 7fa616600640 1 -- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa5f00725e0 msgr2=0x7fa5f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.730+0000 7fa616600640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa5f00725e0 0x7fa5f0074aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa6040af000 tx=0x7fa60401b0e0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.730+0000 7fa616600640 1 -- 172.21.15.66:0/1021510363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa61015f300 msgr2=0x7fa61006e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.730+0000 7fa616600640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa61015f300 0x7fa61006e4e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa60800cce0 tx=0x7fa60800a620 comp rx=0 tx=0).stop 2024-09-10T22:00:45.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.734+0000 7fa616600640 1 -- 172.21.15.66:0/1021510363 shutdown_connections 2024-09-10T22:00:45.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.734+0000 7fa616600640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa5f00725e0 0x7fa5f0074aa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.734+0000 7fa616600640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa61015f300 0x7fa61006e4e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.734+0000 7fa616600640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa61015ef20 0x7fa61014f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.734+0000 7fa616600640 1 --2- 172.21.15.66:0/1021510363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6100cd7a0 0x7fa6101f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.734+0000 7fa616600640 1 -- 172.21.15.66:0/1021510363 >> 172.21.15.66:0/1021510363 conn(0x7fa6100d1b10 msgr2=0x7fa61006f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:45.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.734+0000 7fa616600640 1 -- 172.21.15.66:0/1021510363 shutdown_connections 2024-09-10T22:00:45.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.734+0000 7fa616600640 1 -- 172.21.15.66:0/1021510363 wait complete. 2024-09-10T22:00:45.757 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-10T22:00:44.312680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235848,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738872,"statfs":{"total":773094113280,"available":772852604928,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.129861"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665723+0000","last_change":"2024-09-10T22:00:33.198521+0000","last_active":"2024-09-10T22:00:34.665723+0000","last_peered":"2024-09-10T22:00:34.665723+0000","last_clean":"2024-09-10T22:00:34.665723+0000","last_became_active":"2024-09-10T22:00:33.197869+0000","last_became_peered":"2024-09-10T22:00:33.197869+0000","last_unstale":"2024-09-10T22:00:34.665723+0000","last_undegraded":"2024-09-10T22:00:34.665723+0000","last_fullsized":"2024-09-10T22:00:34.665723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:48.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183587+0000","last_change":"2024-09-10T22:00:33.182512+0000","last_active":"2024-09-10T22:00:34.183587+0000","last_peered":"2024-09-10T22:00:34.183587+0000","last_clean":"2024-09-10T22:00:34.183587+0000","last_became_active":"2024-09-10T22:00:33.182230+0000","last_became_peered":"2024-09-10T22:00:33.182230+0000","last_unstale":"2024-09-10T22:00:34.183587+0000","last_undegraded":"2024-09-10T22:00:34.183587+0000","last_fullsized":"2024-09-10T22:00:34.183587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:08.694915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663553+0000","last_change":"2024-09-10T22:00:33.182382+0000","last_active":"2024-09-10T22:00:34.663553+0000","last_peered":"2024-09-10T22:00:34.663553+0000","last_clean":"2024-09-10T22:00:34.663553+0000","last_became_active":"2024-09-10T22:00:33.182115+0000","last_became_peered":"2024-09-10T22:00:33.182115+0000","last_unstale":"2024-09-10T22:00:34.663553+0000","last_undegraded":"2024-09-10T22:00:34.663553+0000","last_fullsized":"2024-09-10T22:00:34.663553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:34.220172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185630+0000","last_change":"2024-09-10T22:00:33.215454+0000","last_active":"2024-09-10T22:00:34.185630+0000","last_peered":"2024-09-10T22:00:34.185630+0000","last_clean":"2024-09-10T22:00:34.185630+0000","last_became_active":"2024-09-10T22:00:33.214713+0000","last_became_peered":"2024-09-10T22:00:33.214713+0000","last_unstale":"2024-09-10T22:00:34.185630+0000","last_undegraded":"2024-09-10T22:00:34.185630+0000","last_fullsized":"2024-09-10T22:00:34.185630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:37.559387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665058+0000","last_change":"2024-09-10T22:00:33.201925+0000","last_active":"2024-09-10T22:00:34.665058+0000","last_peered":"2024-09-10T22:00:34.665058+0000","last_clean":"2024-09-10T22:00:34.665058+0000","last_became_active":"2024-09-10T22:00:33.197799+0000","last_became_peered":"2024-09-10T22:00:33.197799+0000","last_unstale":"2024-09-10T22:00:34.665058+0000","last_undegraded":"2024-09-10T22:00:34.665058+0000","last_fullsized":"2024-09-10T22:00:34.665058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:37.217143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185144+0000","last_change":"2024-09-10T22:00:33.211402+0000","last_active":"2024-09-10T22:00:34.185144+0000","last_peered":"2024-09-10T22:00:34.185144+0000","last_clean":"2024-09-10T22:00:34.185144+0000","last_became_active":"2024-09-10T22:00:33.210608+0000","last_became_peered":"2024-09-10T22:00:33.210608+0000","last_unstale":"2024-09-10T22:00:34.185144+0000","last_undegraded":"2024-09-10T22:00:34.185144+0000","last_fullsized":"2024-09-10T22:00:34.185144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:14.992964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664313+0000","last_change":"2024-09-10T22:00:33.197611+0000","last_active":"2024-09-10T22:00:34.664313+0000","last_peered":"2024-09-10T22:00:34.664313+0000","last_clean":"2024-09-10T22:00:34.664313+0000","last_became_active":"2024-09-10T22:00:33.192560+0000","last_became_peered":"2024-09-10T22:00:33.192560+0000","last_unstale":"2024-09-10T22:00:34.664313+0000","last_undegraded":"2024-09-10T22:00:34.664313+0000","last_fullsized":"2024-09-10T22:00:34.664313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:25.562048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664300+0000","last_change":"2024-09-10T22:00:33.213208+0000","last_active":"2024-09-10T22:00:34.664300+0000","last_peered":"2024-09-10T22:00:34.664300+0000","last_clean":"2024-09-10T22:00:34.664300+0000","last_became_active":"2024-09-10T22:00:33.212880+0000","last_became_peered":"2024-09-10T22:00:33.212880+0000","last_unstale":"2024-09-10T22:00:34.664300+0000","last_undegraded":"2024-09-10T22:00:34.664300+0000","last_fullsized":"2024-09-10T22:00:34.664300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:55.992463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665753+0000","last_change":"2024-09-10T22:00:33.215336+0000","last_active":"2024-09-10T22:00:34.665753+0000","last_peered":"2024-09-10T22:00:34.665753+0000","last_clean":"2024-09-10T22:00:34.665753+0000","last_became_active":"2024-09-10T22:00:33.214306+0000","last_became_peered":"2024-09-10T22:00:33.214306+0000","last_unstale":"2024-09-10T22:00:34.665753+0000","last_undegraded":"2024-09-10T22:00:34.665753+0000","last_fullsized":"2024-09-10T22:00:34.665753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:45.979508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187047+0000","last_change":"2024-09-10T22:00:33.208856+0000","last_active":"2024-09-10T22:00:34.187047+0000","last_peered":"2024-09-10T22:00:34.187047+0000","last_clean":"2024-09-10T22:00:34.187047+0000","last_became_active":"2024-09-10T22:00:33.208575+0000","last_became_peered":"2024-09-10T22:00:33.208575+0000","last_unstale":"2024-09-10T22:00:34.187047+0000","last_undegraded":"2024-09-10T22:00:34.187047+0000","last_fullsized":"2024-09-10T22:00:34.187047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.744906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186250+0000","last_change":"2024-09-10T22:00:33.214153+0000","last_active":"2024-09-10T22:00:34.186250+0000","last_peered":"2024-09-10T22:00:34.186250+0000","last_clean":"2024-09-10T22:00:34.186250+0000","last_became_active":"2024-09-10T22:00:33.212422+0000","last_became_peered":"2024-09-10T22:00:33.212422+0000","last_unstale":"2024-09-10T22:00:34.186250+0000","last_undegraded":"2024-09-10T22:00:34.186250+0000","last_fullsized":"2024-09-10T22:00:34.186250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:04.855711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665304+0000","last_change":"2024-09-10T22:00:33.184796+0000","last_active":"2024-09-10T22:00:34.665304+0000","last_peered":"2024-09-10T22:00:34.665304+0000","last_clean":"2024-09-10T22:00:34.665304+0000","last_became_active":"2024-09-10T22:00:33.180143+0000","last_became_peered":"2024-09-10T22:00:33.180143+0000","last_unstale":"2024-09-10T22:00:34.665304+0000","last_undegraded":"2024-09-10T22:00:34.665304+0000","last_fullsized":"2024-09-10T22:00:34.665304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:35.363109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663813+0000","last_change":"2024-09-10T22:00:33.192437+0000","last_active":"2024-09-10T22:00:34.663813+0000","last_peered":"2024-09-10T22:00:34.663813+0000","last_clean":"2024-09-10T22:00:34.663813+0000","last_became_active":"2024-09-10T22:00:33.192220+0000","last_became_peered":"2024-09-10T22:00:33.192220+0000","last_unstale":"2024-09-10T22:00:34.663813+0000","last_undegraded":"2024-09-10T22:00:34.663813+0000","last_fullsized":"2024-09-10T22:00:34.663813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:29.825070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663467+0000","last_change":"2024-09-10T22:00:33.184124+0000","last_active":"2024-09-10T22:00:34.663467+0000","last_peered":"2024-09-10T22:00:34.663467+0000","last_clean":"2024-09-10T22:00:34.663467+0000","last_became_active":"2024-09-10T22:00:33.182588+0000","last_became_peered":"2024-09-10T22:00:33.182588+0000","last_unstale":"2024-09-10T22:00:34.663467+0000","last_undegraded":"2024-09-10T22:00:34.663467+0000","last_fullsized":"2024-09-10T22:00:34.663467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:39.577554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183252+0000","last_change":"2024-09-10T22:00:33.178023+0000","last_active":"2024-09-10T22:00:34.183252+0000","last_peered":"2024-09-10T22:00:34.183252+0000","last_clean":"2024-09-10T22:00:34.183252+0000","last_became_active":"2024-09-10T22:00:33.176108+0000","last_became_peered":"2024-09-10T22:00:33.176108+0000","last_unstale":"2024-09-10T22:00:34.183252+0000","last_undegraded":"2024-09-10T22:00:34.183252+0000","last_fullsized":"2024-09-10T22:00:34.183252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:06.332661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184712+0000","last_change":"2024-09-10T22:00:33.208492+0000","last_active":"2024-09-10T22:00:34.184712+0000","last_peered":"2024-09-10T22:00:34.184712+0000","last_clean":"2024-09-10T22:00:34.184712+0000","last_became_active":"2024-09-10T22:00:33.208232+0000","last_became_peered":"2024-09-10T22:00:33.208232+0000","last_unstale":"2024-09-10T22:00:34.184712+0000","last_undegraded":"2024-09-10T22:00:34.184712+0000","last_fullsized":"2024-09-10T22:00:34.184712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.744906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185766+0000","last_change":"2024-09-10T22:00:33.212565+0000","last_active":"2024-09-10T22:00:34.185766+0000","last_peered":"2024-09-10T22:00:34.185766+0000","last_clean":"2024-09-10T22:00:34.185766+0000","last_became_active":"2024-09-10T22:00:33.210654+0000","last_became_peered":"2024-09-10T22:00:33.210654+0000","last_unstale":"2024-09-10T22:00:34.185766+0000","last_undegraded":"2024-09-10T22:00:34.185766+0000","last_fullsized":"2024-09-10T22:00:34.185766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:11.723619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663811+0000","last_change":"2024-09-10T22:00:33.192188+0000","last_active":"2024-09-10T22:00:34.663811+0000","last_peered":"2024-09-10T22:00:34.663811+0000","last_clean":"2024-09-10T22:00:34.663811+0000","last_became_active":"2024-09-10T22:00:33.191616+0000","last_became_peered":"2024-09-10T22:00:33.191616+0000","last_unstale":"2024-09-10T22:00:34.663811+0000","last_undegraded":"2024-09-10T22:00:34.663811+0000","last_fullsized":"2024-09-10T22:00:34.663811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:03.715049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665186+0000","last_change":"2024-09-10T22:00:33.203524+0000","last_active":"2024-09-10T22:00:34.665186+0000","last_peered":"2024-09-10T22:00:34.665186+0000","last_clean":"2024-09-10T22:00:34.665186+0000","last_became_active":"2024-09-10T22:00:33.202541+0000","last_became_peered":"2024-09-10T22:00:33.202541+0000","last_unstale":"2024-09-10T22:00:34.665186+0000","last_undegraded":"2024-09-10T22:00:34.665186+0000","last_fullsized":"2024-09-10T22:00:34.665186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:55.365445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185716+0000","last_change":"2024-09-10T22:00:33.192918+0000","last_active":"2024-09-10T22:00:34.185716+0000","last_peered":"2024-09-10T22:00:34.185716+0000","last_clean":"2024-09-10T22:00:34.185716+0000","last_became_active":"2024-09-10T22:00:33.192320+0000","last_became_peered":"2024-09-10T22:00:33.192320+0000","last_unstale":"2024-09-10T22:00:34.185716+0000","last_undegraded":"2024-09-10T22:00:34.185716+0000","last_fullsized":"2024-09-10T22:00:34.185716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:54.509264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664157+0000","last_change":"2024-09-10T22:00:33.186507+0000","last_active":"2024-09-10T22:00:34.664157+0000","last_peered":"2024-09-10T22:00:34.664157+0000","last_clean":"2024-09-10T22:00:34.664157+0000","last_became_active":"2024-09-10T22:00:33.186253+0000","last_became_peered":"2024-09-10T22:00:33.186253+0000","last_unstale":"2024-09-10T22:00:34.664157+0000","last_undegraded":"2024-09-10T22:00:34.664157+0000","last_fullsized":"2024-09-10T22:00:34.664157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:29.825070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185357+0000","last_change":"2024-09-10T22:00:33.213873+0000","last_active":"2024-09-10T22:00:34.185357+0000","last_peered":"2024-09-10T22:00:34.185357+0000","last_clean":"2024-09-10T22:00:34.185357+0000","last_became_active":"2024-09-10T22:00:33.211788+0000","last_became_peered":"2024-09-10T22:00:33.211788+0000","last_unstale":"2024-09-10T22:00:34.185357+0000","last_undegraded":"2024-09-10T22:00:34.185357+0000","last_fullsized":"2024-09-10T22:00:34.185357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:36.957181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663634+0000","last_change":"2024-09-10T22:00:33.182956+0000","last_active":"2024-09-10T22:00:34.663634+0000","last_peered":"2024-09-10T22:00:34.663634+0000","last_clean":"2024-09-10T22:00:34.663634+0000","last_became_active":"2024-09-10T22:00:33.182699+0000","last_became_peered":"2024-09-10T22:00:33.182699+0000","last_unstale":"2024-09-10T22:00:34.663634+0000","last_undegraded":"2024-09-10T22:00:34.663634+0000","last_fullsized":"2024-09-10T22:00:34.663634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:54.559113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184784+0000","last_change":"2024-09-10T22:00:33.208620+0000","last_active":"2024-09-10T22:00:34.184784+0000","last_peered":"2024-09-10T22:00:34.184784+0000","last_clean":"2024-09-10T22:00:34.184784+0000","last_became_active":"2024-09-10T22:00:33.208337+0000","last_became_peered":"2024-09-10T22:00:33.208337+0000","last_unstale":"2024-09-10T22:00:34.184784+0000","last_undegraded":"2024-09-10T22:00:34.184784+0000","last_fullsized":"2024-09-10T22:00:34.184784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:22.353564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664536+0000","last_change":"2024-09-10T22:00:33.185808+0000","last_active":"2024-09-10T22:00:34.664536+0000","last_peered":"2024-09-10T22:00:34.664536+0000","last_clean":"2024-09-10T22:00:34.664536+0000","last_became_active":"2024-09-10T22:00:33.185528+0000","last_became_peered":"2024-09-10T22:00:33.185528+0000","last_unstale":"2024-09-10T22:00:34.664536+0000","last_undegraded":"2024-09-10T22:00:34.664536+0000","last_fullsized":"2024-09-10T22:00:34.664536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:04.884964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187164+0000","last_change":"2024-09-10T22:00:33.191781+0000","last_active":"2024-09-10T22:00:34.187164+0000","last_peered":"2024-09-10T22:00:34.187164+0000","last_clean":"2024-09-10T22:00:34.187164+0000","last_became_active":"2024-09-10T22:00:33.191497+0000","last_became_peered":"2024-09-10T22:00:33.191497+0000","last_unstale":"2024-09-10T22:00:34.187164+0000","last_undegraded":"2024-09-10T22:00:34.187164+0000","last_fullsized":"2024-09-10T22:00:34.187164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:22.353564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183465+0000","last_change":"2024-09-10T22:00:33.183377+0000","last_active":"2024-09-10T22:00:34.183465+0000","last_peered":"2024-09-10T22:00:34.183465+0000","last_clean":"2024-09-10T22:00:34.183465+0000","last_became_active":"2024-09-10T22:00:33.181838+0000","last_became_peered":"2024-09-10T22:00:33.181838+0000","last_unstale":"2024-09-10T22:00:34.183465+0000","last_undegraded":"2024-09-10T22:00:34.183465+0000","last_fullsized":"2024-09-10T22:00:34.183465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:58.358390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663596+0000","last_change":"2024-09-10T22:00:33.185739+0000","last_active":"2024-09-10T22:00:34.663596+0000","last_peered":"2024-09-10T22:00:34.663596+0000","last_clean":"2024-09-10T22:00:34.663596+0000","last_became_active":"2024-09-10T22:00:33.185426+0000","last_became_peered":"2024-09-10T22:00:33.185426+0000","last_unstale":"2024-09-10T22:00:34.663596+0000","last_undegraded":"2024-09-10T22:00:34.663596+0000","last_fullsized":"2024-09-10T22:00:34.663596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:46.008480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664376+0000","last_change":"2024-09-10T22:00:33.183396+0000","last_active":"2024-09-10T22:00:34.664376+0000","last_peered":"2024-09-10T22:00:34.664376+0000","last_clean":"2024-09-10T22:00:34.664376+0000","last_became_active":"2024-09-10T22:00:33.181415+0000","last_became_peered":"2024-09-10T22:00:33.181415+0000","last_unstale":"2024-09-10T22:00:34.664376+0000","last_undegraded":"2024-09-10T22:00:34.664376+0000","last_fullsized":"2024-09-10T22:00:34.664376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:58.358390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185581+0000","last_change":"2024-09-10T22:00:33.202653+0000","last_active":"2024-09-10T22:00:34.185581+0000","last_peered":"2024-09-10T22:00:34.185581+0000","last_clean":"2024-09-10T22:00:34.185581+0000","last_became_active":"2024-09-10T22:00:33.202300+0000","last_became_peered":"2024-09-10T22:00:33.202300+0000","last_unstale":"2024-09-10T22:00:34.185581+0000","last_undegraded":"2024-09-10T22:00:34.185581+0000","last_fullsized":"2024-09-10T22:00:34.185581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:29.821720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663157+0000","last_change":"2024-09-10T22:00:33.213581+0000","last_active":"2024-09-10T22:00:34.663157+0000","last_peered":"2024-09-10T22:00:34.663157+0000","last_clean":"2024-09-10T22:00:34.663157+0000","last_became_active":"2024-09-10T22:00:33.213202+0000","last_became_peered":"2024-09-10T22:00:33.213202+0000","last_unstale":"2024-09-10T22:00:34.663157+0000","last_undegraded":"2024-09-10T22:00:34.663157+0000","last_fullsized":"2024-09-10T22:00:34.663157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:03.339805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184712+0000","last_change":"2024-09-10T22:00:33.179958+0000","last_active":"2024-09-10T22:00:34.184712+0000","last_peered":"2024-09-10T22:00:34.184712+0000","last_clean":"2024-09-10T22:00:34.184712+0000","last_became_active":"2024-09-10T22:00:33.179251+0000","last_became_peered":"2024-09-10T22:00:33.179251+0000","last_unstale":"2024-09-10T22:00:34.184712+0000","last_undegraded":"2024-09-10T22:00:34.184712+0000","last_fullsized":"2024-09-10T22:00:34.184712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:20.738322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664541+0000","last_change":"2024-09-10T22:00:33.181036+0000","last_active":"2024-09-10T22:00:34.664541+0000","last_peered":"2024-09-10T22:00:34.664541+0000","last_clean":"2024-09-10T22:00:34.664541+0000","last_became_active":"2024-09-10T22:00:33.179960+0000","last_became_peered":"2024-09-10T22:00:33.179960+0000","last_unstale":"2024-09-10T22:00:34.664541+0000","last_undegraded":"2024-09-10T22:00:34.664541+0000","last_fullsized":"2024-09-10T22:00:34.664541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:36.003731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663924+0000","last_change":"2024-09-10T22:00:32.187576+0000","last_active":"2024-09-10T22:00:34.663924+0000","last_peered":"2024-09-10T22:00:34.663924+0000","last_clean":"2024-09-10T22:00:34.663924+0000","last_became_active":"2024-09-10T22:00:32.187338+0000","last_became_peered":"2024-09-10T22:00:32.187338+0000","last_unstale":"2024-09-10T22:00:34.663924+0000","last_undegraded":"2024-09-10T22:00:34.663924+0000","last_fullsized":"2024-09-10T22:00:34.663924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:39.648880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664576+0000","last_change":"2024-09-10T22:00:33.184050+0000","last_active":"2024-09-10T22:00:34.664576+0000","last_peered":"2024-09-10T22:00:34.664576+0000","last_clean":"2024-09-10T22:00:34.664576+0000","last_became_active":"2024-09-10T22:00:33.183760+0000","last_became_peered":"2024-09-10T22:00:33.183760+0000","last_unstale":"2024-09-10T22:00:34.664576+0000","last_undegraded":"2024-09-10T22:00:34.664576+0000","last_fullsized":"2024-09-10T22:00:34.664576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:08.694915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186593+0000","last_change":"2024-09-10T22:00:32.181714+0000","last_active":"2024-09-10T22:00:34.186593+0000","last_peered":"2024-09-10T22:00:34.186593+0000","last_clean":"2024-09-10T22:00:34.186593+0000","last_became_active":"2024-09-10T22:00:32.180900+0000","last_became_peered":"2024-09-10T22:00:32.180900+0000","last_unstale":"2024-09-10T22:00:34.186593+0000","last_undegraded":"2024-09-10T22:00:34.186593+0000","last_fullsized":"2024-09-10T22:00:34.186593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:19.067912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663685+0000","last_change":"2024-09-10T22:00:32.194626+0000","last_active":"2024-09-10T22:00:34.663685+0000","last_peered":"2024-09-10T22:00:34.663685+0000","last_clean":"2024-09-10T22:00:34.663685+0000","last_became_active":"2024-09-10T22:00:32.194317+0000","last_became_peered":"2024-09-10T22:00:32.194317+0000","last_unstale":"2024-09-10T22:00:34.663685+0000","last_undegraded":"2024-09-10T22:00:34.663685+0000","last_fullsized":"2024-09-10T22:00:34.663685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:36.587242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663212+0000","last_change":"2024-09-10T22:00:33.180723+0000","last_active":"2024-09-10T22:00:34.663212+0000","last_peered":"2024-09-10T22:00:34.663212+0000","last_clean":"2024-09-10T22:00:34.663212+0000","last_became_active":"2024-09-10T22:00:33.180399+0000","last_became_peered":"2024-09-10T22:00:33.180399+0000","last_unstale":"2024-09-10T22:00:34.663212+0000","last_undegraded":"2024-09-10T22:00:34.663212+0000","last_fullsized":"2024-09-10T22:00:34.663212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:24.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184786+0000","last_change":"2024-09-10T22:00:32.184624+0000","last_active":"2024-09-10T22:00:34.184786+0000","last_peered":"2024-09-10T22:00:34.184786+0000","last_clean":"2024-09-10T22:00:34.184786+0000","last_became_active":"2024-09-10T22:00:32.184352+0000","last_became_peered":"2024-09-10T22:00:32.184352+0000","last_unstale":"2024-09-10T22:00:34.184786+0000","last_undegraded":"2024-09-10T22:00:34.184786+0000","last_fullsized":"2024-09-10T22:00:34.184786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:19.067912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183246+0000","last_change":"2024-09-10T22:00:33.178900+0000","last_active":"2024-09-10T22:00:34.183246+0000","last_peered":"2024-09-10T22:00:34.183246+0000","last_clean":"2024-09-10T22:00:34.183246+0000","last_became_active":"2024-09-10T22:00:33.178595+0000","last_became_peered":"2024-09-10T22:00:33.178595+0000","last_unstale":"2024-09-10T22:00:34.183246+0000","last_undegraded":"2024-09-10T22:00:34.183246+0000","last_fullsized":"2024-09-10T22:00:34.183246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:46.008480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187010+0000","last_change":"2024-09-10T22:00:32.193426+0000","last_active":"2024-09-10T22:00:34.187010+0000","last_peered":"2024-09-10T22:00:34.187010+0000","last_clean":"2024-09-10T22:00:34.187010+0000","last_became_active":"2024-09-10T22:00:32.189386+0000","last_became_peered":"2024-09-10T22:00:32.189386+0000","last_unstale":"2024-09-10T22:00:34.187010+0000","last_undegraded":"2024-09-10T22:00:34.187010+0000","last_fullsized":"2024-09-10T22:00:34.187010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:55.143270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185444+0000","last_change":"2024-09-10T22:00:33.213346+0000","last_active":"2024-09-10T22:00:34.185444+0000","last_peered":"2024-09-10T22:00:34.185444+0000","last_clean":"2024-09-10T22:00:34.185444+0000","last_became_active":"2024-09-10T22:00:33.213099+0000","last_became_peered":"2024-09-10T22:00:33.213099+0000","last_unstale":"2024-09-10T22:00:34.185444+0000","last_undegraded":"2024-09-10T22:00:34.185444+0000","last_fullsized":"2024-09-10T22:00:34.185444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:21.770779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185507+0000","last_change":"2024-09-10T22:00:32.195226+0000","last_active":"2024-09-10T22:00:34.185507+0000","last_peered":"2024-09-10T22:00:34.185507+0000","last_clean":"2024-09-10T22:00:34.185507+0000","last_became_active":"2024-09-10T22:00:32.194900+0000","last_became_peered":"2024-09-10T22:00:32.194900+0000","last_unstale":"2024-09-10T22:00:34.185507+0000","last_undegraded":"2024-09-10T22:00:34.185507+0000","last_fullsized":"2024-09-10T22:00:34.185507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:28.818521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663832+0000","last_change":"2024-09-10T22:00:33.213048+0000","last_active":"2024-09-10T22:00:34.663832+0000","last_peered":"2024-09-10T22:00:34.663832+0000","last_clean":"2024-09-10T22:00:34.663832+0000","last_became_active":"2024-09-10T22:00:33.212747+0000","last_became_peered":"2024-09-10T22:00:33.212747+0000","last_unstale":"2024-09-10T22:00:34.663832+0000","last_undegraded":"2024-09-10T22:00:34.663832+0000","last_fullsized":"2024-09-10T22:00:34.663832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:46.008480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183502+0000","last_change":"2024-09-10T22:00:32.194672+0000","last_active":"2024-09-10T22:00:34.183502+0000","last_peered":"2024-09-10T22:00:34.183502+0000","last_clean":"2024-09-10T22:00:34.183502+0000","last_became_active":"2024-09-10T22:00:32.194084+0000","last_became_peered":"2024-09-10T22:00:32.194084+0000","last_unstale":"2024-09-10T22:00:34.183502+0000","last_undegraded":"2024-09-10T22:00:34.183502+0000","last_fullsized":"2024-09-10T22:00:34.183502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:09.395633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665544+0000","last_change":"2024-09-10T22:00:33.176539+0000","last_active":"2024-09-10T22:00:34.665544+0000","last_peered":"2024-09-10T22:00:34.665544+0000","last_clean":"2024-09-10T22:00:34.665544+0000","last_became_active":"2024-09-10T22:00:33.175809+0000","last_became_peered":"2024-09-10T22:00:33.175809+0000","last_unstale":"2024-09-10T22:00:34.665544+0000","last_undegraded":"2024-09-10T22:00:34.665544+0000","last_fullsized":"2024-09-10T22:00:34.665544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:22.046625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185783+0000","last_change":"2024-09-10T22:00:32.187798+0000","last_active":"2024-09-10T22:00:34.185783+0000","last_peered":"2024-09-10T22:00:34.185783+0000","last_clean":"2024-09-10T22:00:34.185783+0000","last_became_active":"2024-09-10T22:00:32.184530+0000","last_became_peered":"2024-09-10T22:00:32.184530+0000","last_unstale":"2024-09-10T22:00:34.185783+0000","last_undegraded":"2024-09-10T22:00:34.185783+0000","last_fullsized":"2024-09-10T22:00:34.185783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:41.444694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664461+0000","last_change":"2024-09-10T22:00:33.189951+0000","last_active":"2024-09-10T22:00:34.664461+0000","last_peered":"2024-09-10T22:00:34.664461+0000","last_clean":"2024-09-10T22:00:34.664461+0000","last_became_active":"2024-09-10T22:00:33.189183+0000","last_became_peered":"2024-09-10T22:00:33.189183+0000","last_unstale":"2024-09-10T22:00:34.664461+0000","last_undegraded":"2024-09-10T22:00:34.664461+0000","last_fullsized":"2024-09-10T22:00:34.664461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:04.884964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664660+0000","last_change":"2024-09-10T22:00:32.171323+0000","last_active":"2024-09-10T22:00:34.664660+0000","last_peered":"2024-09-10T22:00:34.664660+0000","last_clean":"2024-09-10T22:00:34.664660+0000","last_became_active":"2024-09-10T22:00:32.171059+0000","last_became_peered":"2024-09-10T22:00:32.171059+0000","last_unstale":"2024-09-10T22:00:34.664660+0000","last_undegraded":"2024-09-10T22:00:34.664660+0000","last_fullsized":"2024-09-10T22:00:34.664660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:44.404901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663672+0000","last_change":"2024-09-10T22:00:33.196013+0000","last_active":"2024-09-10T22:00:34.663672+0000","last_peered":"2024-09-10T22:00:34.663672+0000","last_clean":"2024-09-10T22:00:34.663672+0000","last_became_active":"2024-09-10T22:00:33.195744+0000","last_became_peered":"2024-09-10T22:00:33.195744+0000","last_unstale":"2024-09-10T22:00:34.663672+0000","last_undegraded":"2024-09-10T22:00:34.663672+0000","last_fullsized":"2024-09-10T22:00:34.663672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:40.305055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185639+0000","last_change":"2024-09-10T22:00:32.196550+0000","last_active":"2024-09-10T22:00:34.185639+0000","last_peered":"2024-09-10T22:00:34.185639+0000","last_clean":"2024-09-10T22:00:34.185639+0000","last_became_active":"2024-09-10T22:00:32.196255+0000","last_became_peered":"2024-09-10T22:00:32.196255+0000","last_unstale":"2024-09-10T22:00:34.185639+0000","last_undegraded":"2024-09-10T22:00:34.185639+0000","last_fullsized":"2024-09-10T22:00:34.185639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:27.356827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665144+0000","last_change":"2024-09-10T22:00:33.197537+0000","last_active":"2024-09-10T22:00:34.665144+0000","last_peered":"2024-09-10T22:00:34.665144+0000","last_clean":"2024-09-10T22:00:34.665144+0000","last_became_active":"2024-09-10T22:00:33.197230+0000","last_became_peered":"2024-09-10T22:00:33.197230+0000","last_unstale":"2024-09-10T22:00:34.665144+0000","last_undegraded":"2024-09-10T22:00:34.665144+0000","last_fullsized":"2024-09-10T22:00:34.665144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:57.404481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184926+0000","last_change":"2024-09-10T22:00:32.187773+0000","last_active":"2024-09-10T22:00:34.184926+0000","last_peered":"2024-09-10T22:00:34.184926+0000","last_clean":"2024-09-10T22:00:34.184926+0000","last_became_active":"2024-09-10T22:00:32.185748+0000","last_became_peered":"2024-09-10T22:00:32.185748+0000","last_unstale":"2024-09-10T22:00:34.184926+0000","last_undegraded":"2024-09-10T22:00:34.184926+0000","last_fullsized":"2024-09-10T22:00:34.184926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:39.648880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184928+0000","last_change":"2024-09-10T22:00:33.180297+0000","last_active":"2024-09-10T22:00:34.184928+0000","last_peered":"2024-09-10T22:00:34.184928+0000","last_clean":"2024-09-10T22:00:34.184928+0000","last_became_active":"2024-09-10T22:00:33.179643+0000","last_became_peered":"2024-09-10T22:00:33.179643+0000","last_unstale":"2024-09-10T22:00:34.184928+0000","last_undegraded":"2024-09-10T22:00:34.184928+0000","last_fullsized":"2024-09-10T22:00:34.184928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:06.332661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.662989+0000","last_change":"2024-09-10T22:00:32.182584+0000","last_active":"2024-09-10T22:00:34.662989+0000","last_peered":"2024-09-10T22:00:34.662989+0000","last_clean":"2024-09-10T22:00:34.662989+0000","last_became_active":"2024-09-10T22:00:32.182289+0000","last_became_peered":"2024-09-10T22:00:32.182289+0000","last_unstale":"2024-09-10T22:00:34.662989+0000","last_undegraded":"2024-09-10T22:00:34.662989+0000","last_fullsized":"2024-09-10T22:00:34.662989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:41.773634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664524+0000","last_change":"2024-09-10T22:00:33.189852+0000","last_active":"2024-09-10T22:00:34.664524+0000","last_peered":"2024-09-10T22:00:34.664524+0000","last_clean":"2024-09-10T22:00:34.664524+0000","last_became_active":"2024-09-10T22:00:33.189492+0000","last_became_peered":"2024-09-10T22:00:33.189492+0000","last_unstale":"2024-09-10T22:00:34.664524+0000","last_undegraded":"2024-09-10T22:00:34.664524+0000","last_fullsized":"2024-09-10T22:00:34.664524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:45.010893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665636+0000","last_change":"2024-09-10T22:00:32.179367+0000","last_active":"2024-09-10T22:00:34.665636+0000","last_peered":"2024-09-10T22:00:34.665636+0000","last_clean":"2024-09-10T22:00:34.665636+0000","last_became_active":"2024-09-10T22:00:32.177804+0000","last_became_peered":"2024-09-10T22:00:32.177804+0000","last_unstale":"2024-09-10T22:00:34.665636+0000","last_undegraded":"2024-09-10T22:00:34.665636+0000","last_fullsized":"2024-09-10T22:00:34.665636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:13.368508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665546+0000","last_change":"2024-09-10T22:00:33.194213+0000","last_active":"2024-09-10T22:00:34.665546+0000","last_peered":"2024-09-10T22:00:34.665546+0000","last_clean":"2024-09-10T22:00:34.665546+0000","last_became_active":"2024-09-10T22:00:33.193708+0000","last_became_peered":"2024-09-10T22:00:33.193708+0000","last_unstale":"2024-09-10T22:00:34.665546+0000","last_undegraded":"2024-09-10T22:00:34.665546+0000","last_fullsized":"2024-09-10T22:00:34.665546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:39.533698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663916+0000","last_change":"2024-09-10T22:00:32.184130+0000","last_active":"2024-09-10T22:00:34.663916+0000","last_peered":"2024-09-10T22:00:34.663916+0000","last_clean":"2024-09-10T22:00:34.663916+0000","last_became_active":"2024-09-10T22:00:32.182524+0000","last_became_peered":"2024-09-10T22:00:32.182524+0000","last_unstale":"2024-09-10T22:00:34.663916+0000","last_undegraded":"2024-09-10T22:00:34.663916+0000","last_fullsized":"2024-09-10T22:00:34.663916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:26.056799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665889+0000","last_change":"2024-09-10T22:00:33.198846+0000","last_active":"2024-09-10T22:00:34.665889+0000","last_peered":"2024-09-10T22:00:34.665889+0000","last_clean":"2024-09-10T22:00:34.665889+0000","last_became_active":"2024-09-10T22:00:33.198166+0000","last_became_peered":"2024-09-10T22:00:33.198166+0000","last_unstale":"2024-09-10T22:00:34.665889+0000","last_undegraded":"2024-09-10T22:00:34.665889+0000","last_fullsized":"2024-09-10T22:00:34.665889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:21.862163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183552+0000","last_change":"2024-09-10T22:00:32.181657+0000","last_active":"2024-09-10T22:00:34.183552+0000","last_peered":"2024-09-10T22:00:34.183552+0000","last_clean":"2024-09-10T22:00:34.183552+0000","last_became_active":"2024-09-10T22:00:32.181365+0000","last_became_peered":"2024-09-10T22:00:32.181365+0000","last_unstale":"2024-09-10T22:00:34.183552+0000","last_undegraded":"2024-09-10T22:00:34.183552+0000","last_fullsized":"2024-09-10T22:00:34.183552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:34.356560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185996+0000","last_change":"2024-09-10T22:00:33.215101+0000","last_active":"2024-09-10T22:00:34.185996+0000","last_peered":"2024-09-10T22:00:34.185996+0000","last_clean":"2024-09-10T22:00:34.185996+0000","last_became_active":"2024-09-10T22:00:33.214294+0000","last_became_peered":"2024-09-10T22:00:33.214294+0000","last_unstale":"2024-09-10T22:00:34.185996+0000","last_undegraded":"2024-09-10T22:00:34.185996+0000","last_fullsized":"2024-09-10T22:00:34.185996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:59.470008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663655+0000","last_change":"2024-09-10T22:00:32.195097+0000","last_active":"2024-09-10T22:00:34.663655+0000","last_peered":"2024-09-10T22:00:34.663655+0000","last_clean":"2024-09-10T22:00:34.663655+0000","last_became_active":"2024-09-10T22:00:32.194103+0000","last_became_peered":"2024-09-10T22:00:32.194103+0000","last_unstale":"2024-09-10T22:00:34.663655+0000","last_undegraded":"2024-09-10T22:00:34.663655+0000","last_fullsized":"2024-09-10T22:00:34.663655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:57.181450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664931+0000","last_change":"2024-09-10T22:00:33.212536+0000","last_active":"2024-09-10T22:00:34.664931+0000","last_peered":"2024-09-10T22:00:34.664931+0000","last_clean":"2024-09-10T22:00:34.664931+0000","last_became_active":"2024-09-10T22:00:33.212204+0000","last_became_peered":"2024-09-10T22:00:33.212204+0000","last_unstale":"2024-09-10T22:00:34.664931+0000","last_undegraded":"2024-09-10T22:00:34.664931+0000","last_fullsized":"2024-09-10T22:00:34.664931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:41.803895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664227+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:00:34.664227+0000","last_peered":"2024-09-10T22:00:34.664227+0000","last_clean":"2024-09-10T22:00:34.664227+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:00:34.664227+0000","last_undegraded":"2024-09-10T22:00:34.664227+0000","last_fullsized":"2024-09-10T22:00:34.664227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185132+0000","last_change":"2024-09-10T22:00:32.195314+0000","last_active":"2024-09-10T22:00:34.185132+0000","last_peered":"2024-09-10T22:00:34.185132+0000","last_clean":"2024-09-10T22:00:34.185132+0000","last_became_active":"2024-09-10T22:00:32.194960+0000","last_became_peered":"2024-09-10T22:00:32.194960+0000","last_unstale":"2024-09-10T22:00:34.185132+0000","last_undegraded":"2024-09-10T22:00:34.185132+0000","last_fullsized":"2024-09-10T22:00:34.185132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:01.067800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663524+0000","last_change":"2024-09-10T22:00:33.208452+0000","last_active":"2024-09-10T22:00:34.663524+0000","last_peered":"2024-09-10T22:00:34.663524+0000","last_clean":"2024-09-10T22:00:34.663524+0000","last_became_active":"2024-09-10T22:00:33.208146+0000","last_became_peered":"2024-09-10T22:00:33.208146+0000","last_unstale":"2024-09-10T22:00:34.663524+0000","last_undegraded":"2024-09-10T22:00:34.663524+0000","last_fullsized":"2024-09-10T22:00:34.663524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:40.305055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664710+0000","last_change":"2024-09-10T22:00:32.192173+0000","last_active":"2024-09-10T22:00:34.664710+0000","last_peered":"2024-09-10T22:00:34.664710+0000","last_clean":"2024-09-10T22:00:34.664710+0000","last_became_active":"2024-09-10T22:00:32.191792+0000","last_became_peered":"2024-09-10T22:00:32.191792+0000","last_unstale":"2024-09-10T22:00:34.664710+0000","last_undegraded":"2024-09-10T22:00:34.664710+0000","last_fullsized":"2024-09-10T22:00:34.664710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:34.356560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665272+0000","last_change":"2024-09-10T22:00:33.174576+0000","last_active":"2024-09-10T22:00:34.665272+0000","last_peered":"2024-09-10T22:00:34.665272+0000","last_clean":"2024-09-10T22:00:34.665272+0000","last_became_active":"2024-09-10T22:00:33.174232+0000","last_became_peered":"2024-09-10T22:00:33.174232+0000","last_unstale":"2024-09-10T22:00:34.665272+0000","last_undegraded":"2024-09-10T22:00:34.665272+0000","last_fullsized":"2024-09-10T22:00:34.665272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:55.992463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663264+0000","last_change":"2024-09-10T22:00:32.178337+0000","last_active":"2024-09-10T22:00:34.663264+0000","last_peered":"2024-09-10T22:00:34.663264+0000","last_clean":"2024-09-10T22:00:34.663264+0000","last_became_active":"2024-09-10T22:00:32.178053+0000","last_became_peered":"2024-09-10T22:00:32.178053+0000","last_unstale":"2024-09-10T22:00:34.663264+0000","last_undegraded":"2024-09-10T22:00:34.663264+0000","last_fullsized":"2024-09-10T22:00:34.663264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:26.949229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183185+0000","last_change":"2024-09-10T22:00:33.184308+0000","last_active":"2024-09-10T22:00:34.183185+0000","last_peered":"2024-09-10T22:00:34.183185+0000","last_clean":"2024-09-10T22:00:34.183185+0000","last_became_active":"2024-09-10T22:00:33.184050+0000","last_became_peered":"2024-09-10T22:00:33.184050+0000","last_unstale":"2024-09-10T22:00:34.183185+0000","last_undegraded":"2024-09-10T22:00:34.183185+0000","last_fullsized":"2024-09-10T22:00:34.183185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:40.305055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185742+0000","last_change":"2024-09-10T22:00:32.195663+0000","last_active":"2024-09-10T22:00:34.185742+0000","last_peered":"2024-09-10T22:00:34.185742+0000","last_clean":"2024-09-10T22:00:34.185742+0000","last_became_active":"2024-09-10T22:00:32.195368+0000","last_became_peered":"2024-09-10T22:00:32.195368+0000","last_unstale":"2024-09-10T22:00:34.185742+0000","last_undegraded":"2024-09-10T22:00:34.185742+0000","last_fullsized":"2024-09-10T22:00:34.185742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:56.084833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664779+0000","last_change":"2024-09-10T22:00:33.186193+0000","last_active":"2024-09-10T22:00:34.664779+0000","last_peered":"2024-09-10T22:00:34.664779+0000","last_clean":"2024-09-10T22:00:34.664779+0000","last_became_active":"2024-09-10T22:00:33.180505+0000","last_became_peered":"2024-09-10T22:00:33.180505+0000","last_unstale":"2024-09-10T22:00:34.664779+0000","last_undegraded":"2024-09-10T22:00:34.664779+0000","last_fullsized":"2024-09-10T22:00:34.664779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:10.402182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183297+0000","last_change":"2024-09-10T22:00:32.184540+0000","last_active":"2024-09-10T22:00:34.183297+0000","last_peered":"2024-09-10T22:00:34.183297+0000","last_clean":"2024-09-10T22:00:34.183297+0000","last_became_active":"2024-09-10T22:00:32.184213+0000","last_became_peered":"2024-09-10T22:00:32.184213+0000","last_unstale":"2024-09-10T22:00:34.183297+0000","last_undegraded":"2024-09-10T22:00:34.183297+0000","last_fullsized":"2024-09-10T22:00:34.183297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:21.040076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186341+0000","last_change":"2024-09-10T22:00:33.212897+0000","last_active":"2024-09-10T22:00:34.186341+0000","last_peered":"2024-09-10T22:00:34.186341+0000","last_clean":"2024-09-10T22:00:34.186341+0000","last_became_active":"2024-09-10T22:00:33.212668+0000","last_became_peered":"2024-09-10T22:00:33.212668+0000","last_unstale":"2024-09-10T22:00:34.186341+0000","last_undegraded":"2024-09-10T22:00:34.186341+0000","last_fullsized":"2024-09-10T22:00:34.186341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:47.684127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665052+0000","last_change":"2024-09-10T22:00:32.193491+0000","last_active":"2024-09-10T22:00:34.665052+0000","last_peered":"2024-09-10T22:00:34.665052+0000","last_clean":"2024-09-10T22:00:34.665052+0000","last_became_active":"2024-09-10T22:00:32.193147+0000","last_became_peered":"2024-09-10T22:00:32.193147+0000","last_unstale":"2024-09-10T22:00:34.665052+0000","last_undegraded":"2024-09-10T22:00:34.665052+0000","last_fullsized":"2024-09-10T22:00:34.665052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:51.181211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183356+0000","last_change":"2024-09-10T22:00:33.189064+0000","last_active":"2024-09-10T22:00:34.183356+0000","last_peered":"2024-09-10T22:00:34.183356+0000","last_clean":"2024-09-10T22:00:34.183356+0000","last_became_active":"2024-09-10T22:00:33.188733+0000","last_became_peered":"2024-09-10T22:00:33.188733+0000","last_unstale":"2024-09-10T22:00:34.183356+0000","last_undegraded":"2024-09-10T22:00:34.183356+0000","last_fullsized":"2024-09-10T22:00:34.183356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:29.825070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183468+0000","last_change":"2024-09-10T22:00:32.186782+0000","last_active":"2024-09-10T22:00:34.183468+0000","last_peered":"2024-09-10T22:00:34.183468+0000","last_clean":"2024-09-10T22:00:34.183468+0000","last_became_active":"2024-09-10T22:00:32.186028+0000","last_became_peered":"2024-09-10T22:00:32.186028+0000","last_unstale":"2024-09-10T22:00:34.183468+0000","last_undegraded":"2024-09-10T22:00:34.183468+0000","last_fullsized":"2024-09-10T22:00:34.183468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:34.997182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664559+0000","last_change":"2024-09-10T22:00:33.201547+0000","last_active":"2024-09-10T22:00:34.664559+0000","last_peered":"2024-09-10T22:00:34.664559+0000","last_clean":"2024-09-10T22:00:34.664559+0000","last_became_active":"2024-09-10T22:00:33.201302+0000","last_became_peered":"2024-09-10T22:00:33.201302+0000","last_unstale":"2024-09-10T22:00:34.664559+0000","last_undegraded":"2024-09-10T22:00:34.664559+0000","last_fullsized":"2024-09-10T22:00:34.664559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:45.010893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.187107+0000","last_change":"2024-09-10T22:00:32.196354+0000","last_active":"2024-09-10T22:00:34.187107+0000","last_peered":"2024-09-10T22:00:34.187107+0000","last_clean":"2024-09-10T22:00:34.187107+0000","last_became_active":"2024-09-10T22:00:32.196053+0000","last_became_peered":"2024-09-10T22:00:32.196053+0000","last_unstale":"2024-09-10T22:00:34.187107+0000","last_undegraded":"2024-09-10T22:00:34.187107+0000","last_fullsized":"2024-09-10T22:00:34.187107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:37.446124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663935+0000","last_change":"2024-09-10T22:00:33.176478+0000","last_active":"2024-09-10T22:00:34.663935+0000","last_peered":"2024-09-10T22:00:34.663935+0000","last_clean":"2024-09-10T22:00:34.663935+0000","last_became_active":"2024-09-10T22:00:33.176211+0000","last_became_peered":"2024-09-10T22:00:33.176211+0000","last_unstale":"2024-09-10T22:00:34.663935+0000","last_undegraded":"2024-09-10T22:00:34.663935+0000","last_fullsized":"2024-09-10T22:00:34.663935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:54.559113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664088+0000","last_change":"2024-09-10T22:00:32.189994+0000","last_active":"2024-09-10T22:00:34.664088+0000","last_peered":"2024-09-10T22:00:34.664088+0000","last_clean":"2024-09-10T22:00:34.664088+0000","last_became_active":"2024-09-10T22:00:32.189654+0000","last_became_peered":"2024-09-10T22:00:32.189654+0000","last_unstale":"2024-09-10T22:00:34.664088+0000","last_undegraded":"2024-09-10T22:00:34.664088+0000","last_fullsized":"2024-09-10T22:00:34.664088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:34.987281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664635+0000","last_change":"2024-09-10T22:00:33.183489+0000","last_active":"2024-09-10T22:00:34.664635+0000","last_peered":"2024-09-10T22:00:34.664635+0000","last_clean":"2024-09-10T22:00:34.664635+0000","last_became_active":"2024-09-10T22:00:33.183217+0000","last_became_peered":"2024-09-10T22:00:33.183217+0000","last_unstale":"2024-09-10T22:00:34.664635+0000","last_undegraded":"2024-09-10T22:00:34.664635+0000","last_fullsized":"2024-09-10T22:00:34.664635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:35.993830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183297+0000","last_change":"2024-09-10T22:00:32.178130+0000","last_active":"2024-09-10T22:00:34.183297+0000","last_peered":"2024-09-10T22:00:34.183297+0000","last_clean":"2024-09-10T22:00:34.183297+0000","last_became_active":"2024-09-10T22:00:32.177879+0000","last_became_peered":"2024-09-10T22:00:32.177879+0000","last_unstale":"2024-09-10T22:00:34.183297+0000","last_undegraded":"2024-09-10T22:00:34.183297+0000","last_fullsized":"2024-09-10T22:00:34.183297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:34.987281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664090+0000","last_change":"2024-09-10T22:00:33.179961+0000","last_active":"2024-09-10T22:00:34.664090+0000","last_peered":"2024-09-10T22:00:34.664090+0000","last_clean":"2024-09-10T22:00:34.664090+0000","last_became_active":"2024-09-10T22:00:33.179700+0000","last_became_peered":"2024-09-10T22:00:33.179700+0000","last_unstale":"2024-09-10T22:00:34.664090+0000","last_undegraded":"2024-09-10T22:00:34.664090+0000","last_fullsized":"2024-09-10T22:00:34.664090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:08.694915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.183241+0000","last_change":"2024-09-10T22:00:32.180382+0000","last_active":"2024-09-10T22:00:34.183241+0000","last_peered":"2024-09-10T22:00:34.183241+0000","last_clean":"2024-09-10T22:00:34.183241+0000","last_became_active":"2024-09-10T22:00:32.180075+0000","last_became_peered":"2024-09-10T22:00:32.180075+0000","last_unstale":"2024-09-10T22:00:34.183241+0000","last_undegraded":"2024-09-10T22:00:34.183241+0000","last_fullsized":"2024-09-10T22:00:34.183241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:54.640505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664041+0000","last_change":"2024-09-10T22:00:33.180424+0000","last_active":"2024-09-10T22:00:34.664041+0000","last_peered":"2024-09-10T22:00:34.664041+0000","last_clean":"2024-09-10T22:00:34.664041+0000","last_became_active":"2024-09-10T22:00:33.180181+0000","last_became_peered":"2024-09-10T22:00:33.180181+0000","last_unstale":"2024-09-10T22:00:34.664041+0000","last_undegraded":"2024-09-10T22:00:34.664041+0000","last_fullsized":"2024-09-10T22:00:34.664041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:58.358390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.665152+0000","last_change":"2024-09-10T22:00:32.191652+0000","last_active":"2024-09-10T22:00:34.665152+0000","last_peered":"2024-09-10T22:00:34.665152+0000","last_clean":"2024-09-10T22:00:34.665152+0000","last_became_active":"2024-09-10T22:00:32.191353+0000","last_became_peered":"2024-09-10T22:00:32.191353+0000","last_unstale":"2024-09-10T22:00:34.665152+0000","last_undegraded":"2024-09-10T22:00:34.665152+0000","last_fullsized":"2024-09-10T22:00:34.665152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:21.040076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.184746+0000","last_change":"2024-09-10T22:00:33.184285+0000","last_active":"2024-09-10T22:00:34.184746+0000","last_peered":"2024-09-10T22:00:34.184746+0000","last_clean":"2024-09-10T22:00:34.184746+0000","last_became_active":"2024-09-10T22:00:33.183977+0000","last_became_peered":"2024-09-10T22:00:33.183977+0000","last_unstale":"2024-09-10T22:00:34.184746+0000","last_undegraded":"2024-09-10T22:00:34.184746+0000","last_fullsized":"2024-09-10T22:00:34.184746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:30.181250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.186284+0000","last_change":"2024-09-10T22:00:33.207897+0000","last_active":"2024-09-10T22:00:34.186284+0000","last_peered":"2024-09-10T22:00:34.186284+0000","last_clean":"2024-09-10T22:00:34.186284+0000","last_became_active":"2024-09-10T22:00:33.207639+0000","last_became_peered":"2024-09-10T22:00:33.207639+0000","last_unstale":"2024-09-10T22:00:34.186284+0000","last_undegraded":"2024-09-10T22:00:34.186284+0000","last_fullsized":"2024-09-10T22:00:34.186284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:47.952904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664351+0000","last_change":"2024-09-10T22:00:32.199875+0000","last_active":"2024-09-10T22:00:34.664351+0000","last_peered":"2024-09-10T22:00:34.664351+0000","last_clean":"2024-09-10T22:00:34.664351+0000","last_became_active":"2024-09-10T22:00:32.199625+0000","last_became_peered":"2024-09-10T22:00:32.199625+0000","last_unstale":"2024-09-10T22:00:34.664351+0000","last_undegraded":"2024-09-10T22:00:34.664351+0000","last_fullsized":"2024-09-10T22:00:34.664351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:09.395633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.185863+0000","last_change":"2024-09-10T22:00:33.207063+0000","last_active":"2024-09-10T22:00:34.185863+0000","last_peered":"2024-09-10T22:00:34.185863+0000","last_clean":"2024-09-10T22:00:34.185863+0000","last_became_active":"2024-09-10T22:00:33.206867+0000","last_became_peered":"2024-09-10T22:00:33.206867+0000","last_unstale":"2024-09-10T22:00:34.185863+0000","last_undegraded":"2024-09-10T22:00:34.185863+0000","last_fullsized":"2024-09-10T22:00:34.185863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:44.640052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664917+0000","last_change":"2024-09-10T22:00:32.196714+0000","last_active":"2024-09-10T22:00:34.664917+0000","last_peered":"2024-09-10T22:00:34.664917+0000","last_clean":"2024-09-10T22:00:34.664917+0000","last_became_active":"2024-09-10T22:00:32.196446+0000","last_became_peered":"2024-09-10T22:00:32.196446+0000","last_unstale":"2024-09-10T22:00:34.664917+0000","last_undegraded":"2024-09-10T22:00:34.664917+0000","last_fullsized":"2024-09-10T22:00:34.664917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:39.201788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663624+0000","last_change":"2024-09-10T22:00:33.215295+0000","last_active":"2024-09-10T22:00:34.663624+0000","last_peered":"2024-09-10T22:00:34.663624+0000","last_clean":"2024-09-10T22:00:34.663624+0000","last_became_active":"2024-09-10T22:00:33.215041+0000","last_became_peered":"2024-09-10T22:00:33.215041+0000","last_unstale":"2024-09-10T22:00:34.663624+0000","last_undegraded":"2024-09-10T22:00:34.663624+0000","last_fullsized":"2024-09-10T22:00:34.663624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:24.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664333+0000","last_change":"2024-09-10T22:00:32.180489+0000","last_active":"2024-09-10T22:00:34.664333+0000","last_peered":"2024-09-10T22:00:34.664333+0000","last_clean":"2024-09-10T22:00:34.664333+0000","last_became_active":"2024-09-10T22:00:32.180177+0000","last_became_peered":"2024-09-10T22:00:32.180177+0000","last_unstale":"2024-09-10T22:00:34.664333+0000","last_undegraded":"2024-09-10T22:00:34.664333+0000","last_fullsized":"2024-09-10T22:00:34.664333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:57.175682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.664468+0000","last_change":"2024-09-10T22:00:33.192548+0000","last_active":"2024-09-10T22:00:34.664468+0000","last_peered":"2024-09-10T22:00:34.664468+0000","last_clean":"2024-09-10T22:00:34.664468+0000","last_became_active":"2024-09-10T22:00:33.192272+0000","last_became_peered":"2024-09-10T22:00:33.192272+0000","last_unstale":"2024-09-10T22:00:34.664468+0000","last_undegraded":"2024-09-10T22:00:34.664468+0000","last_fullsized":"2024-09-10T22:00:34.664468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:32.154625+0000","last_clean_scrub_stamp":"2024-09-10T22:00:32.154625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:25.562048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T22:00:34.663908+0000","last_change":"2024-09-10T22:00:32.182534+0000","last_active":"2024-09-10T22:00:34.663908+0000","last_peered":"2024-09-10T22:00:34.663908+0000","last_clean":"2024-09-10T22:00:34.663908+0000","last_became_active":"2024-09-10T22:00:32.182286+0000","last_became_peered":"2024-09-10T22:00:32.182286+0000","last_unstale":"2024-09-10T22:00:34.663908+0000","last_undegraded":"2024-09-10T22:00:34.663908+0000","last_fullsized":"2024-09-10T22:00:34.663908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:31.148076+0000","last_clean_scrub_stamp":"2024-09-10T22:00:31.148076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:56.418214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":667234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":225280,"data_stored":76374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":76410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":827392,"data_stored":667290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:00:45.761 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:00:45.761 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T22:00:45.761 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T22:00:45.761 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T22:00:45.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.942+0000 7ff3c5000640 1 -- 172.21.15.66:0/1240532875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c00cff10 msgr2=0x7ff3c00d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.942+0000 7ff3c5000640 1 --2- 172.21.15.66:0/1240532875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c00cff10 0x7ff3c00d03a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009c80 tx=0x7ff3b801cce0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 -- 172.21.15.66:0/1240532875 shutdown_connections 2024-09-10T22:00:45.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 --2- 172.21.15.66:0/1240532875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c00cff10 0x7ff3c00d03a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 --2- 172.21.15.66:0/1240532875 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3c015bfb0 0x7ff3c00ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 -- 172.21.15.66:0/1240532875 >> 172.21.15.66:0/1240532875 conn(0x7ff3c0074240 msgr2=0x7ff3c0074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:45.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 -- 172.21.15.66:0/1240532875 shutdown_connections 2024-09-10T22:00:45.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 -- 172.21.15.66:0/1240532875 wait complete. 2024-09-10T22:00:45.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 Processor -- start 2024-09-10T22:00:45.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 -- start start 2024-09-10T22:00:45.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3c00b0590 0x7ff3c00c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:45.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3c00cff10 0x7ff3c01562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:45.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c015bfb0 0x7ff3c00c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:45.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c00cd990 con 0x7ff3c015bfb0 2024-09-10T22:00:45.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c00cdb10 con 0x7ff3c00b0590 2024-09-10T22:00:45.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c01692a0 con 0x7ff3c00cff10 2024-09-10T22:00:45.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3be800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c015bfb0 0x7ff3c00c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3be800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c015bfb0 0x7ff3c00c3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45798/0 (socket says 172.21.15.66:45798) 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3be800640 1 -- 172.21.15.66:0/2950268978 learned_addr learned my addr 172.21.15.66:0/2950268978 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3bde00640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3c00b0590 0x7ff3c00c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3b5400640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3c00cff10 0x7ff3c01562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3be800640 1 -- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3c00cff10 msgr2=0x7ff3c01562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3be800640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3c00cff10 0x7ff3c01562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3be800640 1 -- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3c00b0590 msgr2=0x7ff3c00c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3be800640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3c00b0590 0x7ff3c00c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3be800640 1 -- 172.21.15.66:0/2950268978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c01567e0 con 0x7ff3c015bfb0 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3b5400640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3c00cff10 0x7ff3c01562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3be800640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c015bfb0 0x7ff3c00c3e60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009bd0 tx=0x7ff3b8009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3b6a00640 1 -- 172.21.15.66:0/2950268978 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80bb070 con 0x7ff3c015bfb0 2024-09-10T22:00:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3b6a00640 1 -- 172.21.15.66:0/2950268978 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3b8004e00 con 0x7ff3c015bfb0 2024-09-10T22:00:45.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.946+0000 7ff3c5000640 1 -- 172.21.15.66:0/2950268978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c01f80e0 con 0x7ff3c015bfb0 2024-09-10T22:00:45.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.950+0000 7ff3b6a00640 1 -- 172.21.15.66:0/2950268978 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80b2040 con 0x7ff3c015bfb0 2024-09-10T22:00:45.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.950+0000 7ff3c5000640 1 -- 172.21.15.66:0/2950268978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c01f86a0 con 0x7ff3c015bfb0 2024-09-10T22:00:45.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.950+0000 7ff3c5000640 1 -- 172.21.15.66:0/2950268978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c00d0380 con 0x7ff3c015bfb0 2024-09-10T22:00:45.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.950+0000 7ff3b6a00640 1 -- 172.21.15.66:0/2950268978 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff3b80b5030 con 0x7ff3c015bfb0 2024-09-10T22:00:45.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.950+0000 7ff3b6a00640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff39c0725e0 0x7ff39c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:45.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.950+0000 7ff3b6a00640 1 -- 172.21.15.66:0/2950268978 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff3b812cfe0 con 0x7ff3c015bfb0 2024-09-10T22:00:45.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.950+0000 7ff3bde00640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff39c0725e0 0x7ff39c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:45.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.954+0000 7ff3bde00640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff39c0725e0 0x7ff39c074aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0004630 tx=0x7ff3b000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:45.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:45.954+0000 7ff3b6a00640 1 -- 172.21.15.66:0/2950268978 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff3b8016990 con 0x7ff3c015bfb0 2024-09-10T22:00:46.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 -- 172.21.15.66:0/2950268978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff3c0099e20 con 0x7ff3c015bfb0 2024-09-10T22:00:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3b6a00640 1 -- 172.21.15.66:0/2950268978 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff3b80f8030 con 0x7ff3c015bfb0 2024-09-10T22:00:46.170 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:46.170 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T22:00:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 -- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff39c0725e0 msgr2=0x7ff39c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff39c0725e0 0x7ff39c074aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0004630 tx=0x7ff3b000b2e0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 -- 172.21.15.66:0/2950268978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c015bfb0 msgr2=0x7ff3c00c3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c015bfb0 0x7ff3c00c3e60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009bd0 tx=0x7ff3b8009390 comp rx=0 tx=0).stop 2024-09-10T22:00:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 -- 172.21.15.66:0/2950268978 shutdown_connections 2024-09-10T22:00:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff39c0725e0 0x7ff39c074aa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c015bfb0 0x7ff3c00c3e60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3c00cff10 0x7ff3c01562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 --2- 172.21.15.66:0/2950268978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3c00b0590 0x7ff3c00c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 -- 172.21.15.66:0/2950268978 >> 172.21.15.66:0/2950268978 conn(0x7ff3c0074240 msgr2=0x7ff3c006f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 -- 172.21.15.66:0/2950268978 shutdown_connections 2024-09-10T22:00:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.166+0000 7ff3c5000640 1 -- 172.21.15.66:0/2950268978 wait complete. 2024-09-10T22:00:46.192 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T22:00:46.192 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:00:46.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.334+0000 7f9e75c00640 1 -- 172.21.15.66:0/1303994841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e700b15d0 msgr2=0x7f9e700cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:46.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.334+0000 7f9e75c00640 1 --2- 172.21.15.66:0/1303994841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e700b15d0 0x7f9e700cd7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009bc0 tx=0x7f9e6801cb60 comp rx=0 tx=0).stop 2024-09-10T22:00:46.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.334+0000 7f9e75c00640 1 -- 172.21.15.66:0/1303994841 shutdown_connections 2024-09-10T22:00:46.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.334+0000 7f9e75c00640 1 --2- 172.21.15.66:0/1303994841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e700b15d0 0x7f9e700cd7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.334+0000 7f9e75c00640 1 --2- 172.21.15.66:0/1303994841 >> v2:172.21.15.94:3300/0 conn(0x7f9e7015f300 0x7f9e700b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.334+0000 7f9e75c00640 1 -- 172.21.15.66:0/1303994841 >> 172.21.15.66:0/1303994841 conn(0x7f9e700d1b10 msgr2=0x7f9e700d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:46.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.334+0000 7f9e75c00640 1 -- 172.21.15.66:0/1303994841 shutdown_connections 2024-09-10T22:00:46.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.334+0000 7f9e75c00640 1 -- 172.21.15.66:0/1303994841 wait complete. 2024-09-10T22:00:46.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.334+0000 7f9e75c00640 1 Processor -- start 2024-09-10T22:00:46.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.334+0000 7f9e75c00640 1 -- start start 2024-09-10T22:00:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e75c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e700b15d0 0x7f9e700b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e75c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e7015ef20 0x7f9e70166160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e75c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e7015f300 0x7f9e700b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e75c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9e70169310 con 0x7f9e7015ef20 2024-09-10T22:00:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e75c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9e70169190 con 0x7f9e700b15d0 2024-09-10T22:00:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e75c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9e70169490 con 0x7f9e7015f300 2024-09-10T22:00:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e6f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e7015ef20 0x7f9e70166160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e6f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e7015ef20 0x7f9e70166160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45822/0 (socket says 172.21.15.66:45822) 2024-09-10T22:00:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e6f400640 1 -- 172.21.15.66:0/850385516 learned_addr learned my addr 172.21.15.66:0/850385516 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e6fe00640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e700b15d0 0x7f9e700b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e75200640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e7015f300 0x7f9e700b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e6f400640 1 -- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e7015f300 msgr2=0x7f9e700b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e6f400640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e7015f300 0x7f9e700b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e6f400640 1 -- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e700b15d0 msgr2=0x7f9e700b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e6f400640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e700b15d0 0x7f9e700b6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e6f400640 1 -- 172.21.15.66:0/850385516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e700b6ad0 con 0x7f9e7015ef20 2024-09-10T22:00:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e6f400640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e7015ef20 0x7f9e70166160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00d750 tx=0x7f9e5c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e67e00640 1 -- 172.21.15.66:0/850385516 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c017070 con 0x7f9e7015ef20 2024-09-10T22:00:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e67e00640 1 -- 172.21.15.66:0/850385516 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e5c009cc0 con 0x7f9e7015ef20 2024-09-10T22:00:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e67e00640 1 -- 172.21.15.66:0/850385516 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00de60 con 0x7f9e7015ef20 2024-09-10T22:00:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e75c00640 1 -- 172.21.15.66:0/850385516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e701fb560 con 0x7f9e7015ef20 2024-09-10T22:00:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e75c00640 1 -- 172.21.15.66:0/850385516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e701fbb20 con 0x7f9e7015ef20 2024-09-10T22:00:46.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.338+0000 7f9e75c00640 1 -- 172.21.15.66:0/850385516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e700cd7a0 con 0x7f9e7015ef20 2024-09-10T22:00:46.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.342+0000 7f9e67e00640 1 -- 172.21.15.66:0/850385516 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9e5c0106c0 con 0x7f9e7015ef20 2024-09-10T22:00:46.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.342+0000 7f9e67e00640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9e500725e0 0x7f9e50074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:46.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.342+0000 7f9e67e00640 1 -- 172.21.15.66:0/850385516 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9e5c093760 con 0x7f9e7015ef20 2024-09-10T22:00:46.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.342+0000 7f9e6fe00640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9e500725e0 0x7f9e50074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:46.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.342+0000 7f9e6fe00640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9e500725e0 0x7f9e50074aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9e60000f70 tx=0x7f9e60009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:46.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.346+0000 7f9e67e00640 1 -- 172.21.15.66:0/850385516 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9e5c05eda0 con 0x7f9e7015ef20 2024-09-10T22:00:46.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.554+0000 7f9e75c00640 1 -- 172.21.15.66:0/850385516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e70072e00 con 0x7f9e7015ef20 2024-09-10T22:00:46.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.554+0000 7f9e67e00640 1 -- 172.21.15.66:0/850385516 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f9e5c05e6a0 con 0x7f9e7015ef20 2024-09-10T22:00:46.560 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:46.560 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:00:36:190941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2728893456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2728893456},{"type":"v1","addr":"172.21.15.94:6835","nonce":2728893456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2533443317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2533443317},{"type":"v1","addr":"172.21.15.66:6834","nonce":2533443317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/317149387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":317149387},{"type":"v1","addr":"172.21.15.66:6835","nonce":317149387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:00:32.573702+0000","modified":"2024-09-10T22:00:35.587262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/4271184466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":4271184466},{"type":"v1","addr":"172.21.15.94:6837","nonce":4271184466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-10T22:00:46.561 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:00:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.558+0000 7f9e75c00640 1 -- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9e500725e0 msgr2=0x7f9e50074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.558+0000 7f9e75c00640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9e500725e0 0x7f9e50074aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9e60000f70 tx=0x7f9e60009040 comp rx=0 tx=0).stop 2024-09-10T22:00:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.558+0000 7f9e75c00640 1 -- 172.21.15.66:0/850385516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e7015ef20 msgr2=0x7f9e70166160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.558+0000 7f9e75c00640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e7015ef20 0x7f9e70166160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00d750 tx=0x7f9e5c00dc10 comp rx=0 tx=0).stop 2024-09-10T22:00:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.558+0000 7f9e75c00640 1 -- 172.21.15.66:0/850385516 shutdown_connections 2024-09-10T22:00:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.558+0000 7f9e75c00640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9e500725e0 0x7f9e50074aa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.558+0000 7f9e75c00640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e7015f300 0x7f9e700b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.558+0000 7f9e75c00640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e7015ef20 0x7f9e70166160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.558+0000 7f9e75c00640 1 --2- 172.21.15.66:0/850385516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e700b15d0 0x7f9e700b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.558+0000 7f9e75c00640 1 -- 172.21.15.66:0/850385516 >> 172.21.15.66:0/850385516 conn(0x7f9e700d1b10 msgr2=0x7f9e700d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.558+0000 7f9e75c00640 1 -- 172.21.15.66:0/850385516 shutdown_connections 2024-09-10T22:00:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.558+0000 7f9e75c00640 1 -- 172.21.15.66:0/850385516 wait complete. 2024-09-10T22:00:46.583 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:00:46.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.722+0000 7fb848600640 1 -- 172.21.15.66:0/4128259165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8400cf180 msgr2=0x7fb8400d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:46.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.722+0000 7fb848600640 1 --2- 172.21.15.66:0/4128259165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8400cf180 0x7fb8400d04e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb838009bc0 tx=0x7fb83801cb10 comp rx=0 tx=0).stop 2024-09-10T22:00:46.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 -- 172.21.15.66:0/4128259165 shutdown_connections 2024-09-10T22:00:46.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 --2- 172.21.15.66:0/4128259165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8400cf180 0x7fb8400d04e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 --2- 172.21.15.66:0/4128259165 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb84015bd40 0x7fb8400cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 -- 172.21.15.66:0/4128259165 >> 172.21.15.66:0/4128259165 conn(0x7fb8400d1c60 msgr2=0x7fb8400d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:46.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 -- 172.21.15.66:0/4128259165 shutdown_connections 2024-09-10T22:00:46.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 -- 172.21.15.66:0/4128259165 wait complete. 2024-09-10T22:00:46.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 Processor -- start 2024-09-10T22:00:46.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 -- start start 2024-09-10T22:00:46.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8400cf180 0x7fb84014e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:46.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8400cf830 0x7fb8400b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:46.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb84015bd40 0x7fb8400b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:46.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb840168ba0 con 0x7fb8400cf830 2024-09-10T22:00:46.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb840168a20 con 0x7fb8400cf180 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb848600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb840168d20 con 0x7fb84015bd40 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb845200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8400cf830 0x7fb8400b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb845200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8400cf830 0x7fb8400b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45832/0 (socket says 172.21.15.66:45832) 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb845200640 1 -- 172.21.15.66:0/2497019468 learned_addr learned my addr 172.21.15.66:0/2497019468 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb845c00640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8400cf180 0x7fb84014e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb846600640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb84015bd40 0x7fb8400b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb845200640 1 -- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb84015bd40 msgr2=0x7fb8400b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb845200640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb84015bd40 0x7fb8400b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb845200640 1 -- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8400cf180 msgr2=0x7fb84014e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb845200640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8400cf180 0x7fb84014e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.726+0000 7fb845200640 1 -- 172.21.15.66:0/2497019468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84015b910 con 0x7fb8400cf830 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.730+0000 7fb845200640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8400cf830 0x7fb8400b5d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00c490 tx=0x7fb82c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:46.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.730+0000 7fb83d600640 1 -- 172.21.15.66:0/2497019468 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c016020 con 0x7fb8400cf830 2024-09-10T22:00:46.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.730+0000 7fb83d600640 1 -- 172.21.15.66:0/2497019468 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb82c012940 con 0x7fb8400cf830 2024-09-10T22:00:46.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.730+0000 7fb848600640 1 -- 172.21.15.66:0/2497019468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8401527e0 con 0x7fb8400cf830 2024-09-10T22:00:46.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.730+0000 7fb83d600640 1 -- 172.21.15.66:0/2497019468 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c012d90 con 0x7fb8400cf830 2024-09-10T22:00:46.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.730+0000 7fb848600640 1 -- 172.21.15.66:0/2497019468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8400b5540 con 0x7fb8400cf830 2024-09-10T22:00:46.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.730+0000 7fb848600640 1 -- 172.21.15.66:0/2497019468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8400aec10 con 0x7fb8400cf830 2024-09-10T22:00:46.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.730+0000 7fb83d600640 1 -- 172.21.15.66:0/2497019468 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb82c01b400 con 0x7fb8400cf830 2024-09-10T22:00:46.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.730+0000 7fb83d600640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8200725e0 0x7fb820074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:46.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.730+0000 7fb83d600640 1 -- 172.21.15.66:0/2497019468 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb82c093540 con 0x7fb8400cf830 2024-09-10T22:00:46.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.730+0000 7fb845c00640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8200725e0 0x7fb820074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:46.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.734+0000 7fb845c00640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8200725e0 0x7fb820074aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb834009600 tx=0x7fb8340091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:46.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.734+0000 7fb83d600640 1 -- 172.21.15.66:0/2497019468 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb82c020020 con 0x7fb8400cf830 2024-09-10T22:00:46.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.942+0000 7fb848600640 1 -- 172.21.15.66:0/2497019468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb840071050 con 0x7fb8400cf830 2024-09-10T22:00:46.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.942+0000 7fb83d600640 1 -- 172.21.15.66:0/2497019468 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7fb82c05e560 con 0x7fb8400cf830 2024-09-10T22:00:46.948 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:46.948 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:00:36:190941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2728893456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2728893456},{"type":"v1","addr":"172.21.15.94:6835","nonce":2728893456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2533443317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2533443317},{"type":"v1","addr":"172.21.15.66:6834","nonce":2533443317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/317149387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":317149387},{"type":"v1","addr":"172.21.15.66:6835","nonce":317149387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:00:32.573702+0000","modified":"2024-09-10T22:00:35.587262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/4271184466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":4271184466},{"type":"v1","addr":"172.21.15.94:6837","nonce":4271184466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-10T22:00:46.948 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:00:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.946+0000 7fb848600640 1 -- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8200725e0 msgr2=0x7fb820074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.946+0000 7fb848600640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8200725e0 0x7fb820074aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb834009600 tx=0x7fb8340091e0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.946+0000 7fb848600640 1 -- 172.21.15.66:0/2497019468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8400cf830 msgr2=0x7fb8400b5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.946+0000 7fb848600640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8400cf830 0x7fb8400b5d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00c490 tx=0x7fb82c00c950 comp rx=0 tx=0).stop 2024-09-10T22:00:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.946+0000 7fb848600640 1 -- 172.21.15.66:0/2497019468 shutdown_connections 2024-09-10T22:00:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.946+0000 7fb848600640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8200725e0 0x7fb820074aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.946+0000 7fb848600640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb84015bd40 0x7fb8400b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.946+0000 7fb848600640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8400cf830 0x7fb8400b5d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.946+0000 7fb848600640 1 --2- 172.21.15.66:0/2497019468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8400cf180 0x7fb84014e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.946+0000 7fb848600640 1 -- 172.21.15.66:0/2497019468 >> 172.21.15.66:0/2497019468 conn(0x7fb8400d1c60 msgr2=0x7fb84006f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.946+0000 7fb848600640 1 -- 172.21.15.66:0/2497019468 shutdown_connections 2024-09-10T22:00:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:46.946+0000 7fb848600640 1 -- 172.21.15.66:0/2497019468 wait complete. 2024-09-10T22:00:46.970 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:00:47.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.110+0000 7f9933e00640 1 -- 172.21.15.66:0/1670825301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f992c0b0f60 msgr2=0x7f992c0d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.110+0000 7f9933e00640 1 --2- 172.21.15.66:0/1670825301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f992c0b0f60 0x7f992c0d0140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9918009840 tx=0x7f991801c6d0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.110+0000 7f9933e00640 1 -- 172.21.15.66:0/1670825301 shutdown_connections 2024-09-10T22:00:47.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.110+0000 7f9933e00640 1 --2- 172.21.15.66:0/1670825301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f992c0b0f60 0x7f992c0d0140 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.110+0000 7f9933e00640 1 --2- 172.21.15.66:0/1670825301 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f992c15f460 0x7f992c0cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.110+0000 7f9933e00640 1 -- 172.21.15.66:0/1670825301 >> 172.21.15.66:0/1670825301 conn(0x7f992c0d1d00 msgr2=0x7f992c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:47.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.110+0000 7f9933e00640 1 -- 172.21.15.66:0/1670825301 shutdown_connections 2024-09-10T22:00:47.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.110+0000 7f9933e00640 1 -- 172.21.15.66:0/1670825301 wait complete. 2024-09-10T22:00:47.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9933e00640 1 Processor -- start 2024-09-10T22:00:47.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9933e00640 1 -- start start 2024-09-10T22:00:47.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9933e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f992c0b0f60 0x7f992c148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:47.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9933e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f992c0b32a0 0x7f992c06f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:47.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9933e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f992c15f460 0x7f992c152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:47.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9933e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f992c168ff0 con 0x7f992c15f460 2024-09-10T22:00:47.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9933e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f992c168e70 con 0x7f992c0b0f60 2024-09-10T22:00:47.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9933e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f992c169170 con 0x7f992c0b32a0 2024-09-10T22:00:47.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9931e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f992c15f460 0x7f992c152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:47.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9931e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f992c15f460 0x7f992c152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45856/0 (socket says 172.21.15.66:45856) 2024-09-10T22:00:47.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9931e00640 1 -- 172.21.15.66:0/2824464691 learned_addr learned my addr 172.21.15.66:0/2824464691 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:47.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9930a00640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f992c0b32a0 0x7f992c06f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:47.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9931400640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f992c0b0f60 0x7f992c148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:47.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9931e00640 1 -- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f992c0b32a0 msgr2=0x7f992c06f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9931e00640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f992c0b32a0 0x7f992c06f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9931e00640 1 -- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f992c0b0f60 msgr2=0x7f992c148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9931e00640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f992c0b0f60 0x7f992c148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9931e00640 1 -- 172.21.15.66:0/2824464691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992c15a8b0 con 0x7f992c15f460 2024-09-10T22:00:47.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9931e00640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f992c15f460 0x7f992c152c90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f992400cbc0 tx=0x7f992400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:47.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9929600640 1 -- 172.21.15.66:0/2824464691 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924016020 con 0x7f992c15f460 2024-09-10T22:00:47.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9929600640 1 -- 172.21.15.66:0/2824464691 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f992400ab00 con 0x7f992c15f460 2024-09-10T22:00:47.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9929600640 1 -- 172.21.15.66:0/2824464691 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924012420 con 0x7f992c15f460 2024-09-10T22:00:47.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.114+0000 7f9933e00640 1 -- 172.21.15.66:0/2824464691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992c15ab90 con 0x7f992c15f460 2024-09-10T22:00:47.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.118+0000 7f9933e00640 1 -- 172.21.15.66:0/2824464691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992c15b150 con 0x7f992c15f460 2024-09-10T22:00:47.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.118+0000 7f9933e00640 1 -- 172.21.15.66:0/2824464691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f992c0b14b0 con 0x7f992c15f460 2024-09-10T22:00:47.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.118+0000 7f9929600640 1 -- 172.21.15.66:0/2824464691 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f99240125c0 con 0x7f992c15f460 2024-09-10T22:00:47.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.118+0000 7f9929600640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f990c0725e0 0x7f990c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:47.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.118+0000 7f9929600640 1 -- 172.21.15.66:0/2824464691 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9924093750 con 0x7f992c15f460 2024-09-10T22:00:47.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.118+0000 7f9931400640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f990c0725e0 0x7f990c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:47.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.122+0000 7f9931400640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f990c0725e0 0x7f990c074aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f992000bb50 tx=0x7f992000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:47.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.122+0000 7f9929600640 1 -- 172.21.15.66:0/2824464691 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9924020020 con 0x7f992c15f460 2024-09-10T22:00:47.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.298+0000 7f9933e00640 1 -- 172.21.15.66:0/2824464691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f992c0b19a0 con 0x7f992c15f460 2024-09-10T22:00:47.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.298+0000 7f9929600640 1 -- 172.21.15.66:0/2824464691 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16593 (secure 0 0 0) 0x7f992405e740 con 0x7f992c15f460 2024-09-10T22:00:47.304 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:47.304 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":22,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:34.173529+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:30.545203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:31.538729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.302+0000 7f9933e00640 1 -- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f990c0725e0 msgr2=0x7f990c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.302+0000 7f9933e00640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f990c0725e0 0x7f990c074aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f992000bb50 tx=0x7f992000b2e0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.302+0000 7f9933e00640 1 -- 172.21.15.66:0/2824464691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f992c15f460 msgr2=0x7f992c152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.302+0000 7f9933e00640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f992c15f460 0x7f992c152c90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f992400cbc0 tx=0x7f992400ccb0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.302+0000 7f9933e00640 1 -- 172.21.15.66:0/2824464691 shutdown_connections 2024-09-10T22:00:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.302+0000 7f9933e00640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f990c0725e0 0x7f990c074aa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.302+0000 7f9933e00640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f992c15f460 0x7f992c152c90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.302+0000 7f9933e00640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f992c0b32a0 0x7f992c06f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.302+0000 7f9933e00640 1 --2- 172.21.15.66:0/2824464691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f992c0b0f60 0x7f992c148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.302+0000 7f9933e00640 1 -- 172.21.15.66:0/2824464691 >> 172.21.15.66:0/2824464691 conn(0x7f992c0d1d00 msgr2=0x7f992c15e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:47.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.302+0000 7f9933e00640 1 -- 172.21.15.66:0/2824464691 shutdown_connections 2024-09-10T22:00:47.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.302+0000 7f9933e00640 1 -- 172.21.15.66:0/2824464691 wait complete. 2024-09-10T22:00:47.326 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:00:47.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.470+0000 7fb8da400640 1 -- 172.21.15.66:0/3842028443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8b80052f0 msgr2=0x7fb8b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.470+0000 7fb8da400640 1 --2- 172.21.15.66:0/3842028443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8b80052f0 0x7fb8b800b9d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009eb0 tx=0x7fb8cc01d5f0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.470+0000 7fb8da400640 1 -- 172.21.15.66:0/3842028443 shutdown_connections 2024-09-10T22:00:47.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.470+0000 7fb8da400640 1 --2- 172.21.15.66:0/3842028443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8b80052f0 0x7fb8b800b9d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.470+0000 7fb8da400640 1 -- 172.21.15.66:0/3842028443 >> 172.21.15.66:0/3842028443 conn(0x7fb8d40cfa30 msgr2=0x7fb8d40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:47.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.470+0000 7fb8da400640 1 -- 172.21.15.66:0/3842028443 shutdown_connections 2024-09-10T22:00:47.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.470+0000 7fb8da400640 1 -- 172.21.15.66:0/3842028443 wait complete. 2024-09-10T22:00:47.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.470+0000 7fb8da400640 1 Processor -- start 2024-09-10T22:00:47.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.470+0000 7fb8da400640 1 -- start start 2024-09-10T22:00:47.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8da400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8b80052f0 0x7fb8d40b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:47.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8da400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8d40c5e40 0x7fb8d40c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:47.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8da400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8d4153300 0x7fb8d4153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:47.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8da400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d415ce00 con 0x7fb8d4153300 2024-09-10T22:00:47.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8da400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d415cc80 con 0x7fb8b80052f0 2024-09-10T22:00:47.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8da400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d415cf80 con 0x7fb8d40c5e40 2024-09-10T22:00:47.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8d3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8d4153300 0x7fb8d4153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:47.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8d3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8d4153300 0x7fb8d4153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45860/0 (socket says 172.21.15.66:45860) 2024-09-10T22:00:47.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8d3e00640 1 -- 172.21.15.66:0/3292254714 learned_addr learned my addr 172.21.15.66:0/3292254714 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:47.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8d3400640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8b80052f0 0x7fb8d40b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:47.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8caa00640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8d40c5e40 0x7fb8d40c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:47.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8d3e00640 1 -- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8d40c5e40 msgr2=0x7fb8d40c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8d3e00640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8d40c5e40 0x7fb8d40c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8d3e00640 1 -- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8b80052f0 msgr2=0x7fb8d40b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8d3e00640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8b80052f0 0x7fb8d40b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8d3e00640 1 -- 172.21.15.66:0/3292254714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d40b2c50 con 0x7fb8d4153300 2024-09-10T22:00:47.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8d3e00640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8d4153300 0x7fb8d4153790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009e00 tx=0x7fb8cc004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:47.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8cbe00640 1 -- 172.21.15.66:0/3292254714 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc009ab0 con 0x7fb8d4153300 2024-09-10T22:00:47.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8cbe00640 1 -- 172.21.15.66:0/3292254714 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8cc0b1070 con 0x7fb8d4153300 2024-09-10T22:00:47.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8da400640 1 -- 172.21.15.66:0/3292254714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d4072670 con 0x7fb8d4153300 2024-09-10T22:00:47.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8cbe00640 1 -- 172.21.15.66:0/3292254714 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc004870 con 0x7fb8d4153300 2024-09-10T22:00:47.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8da400640 1 -- 172.21.15.66:0/3292254714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d4072c30 con 0x7fb8d4153300 2024-09-10T22:00:47.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8da400640 1 -- 172.21.15.66:0/3292254714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d40ae8d0 con 0x7fb8d4153300 2024-09-10T22:00:47.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.474+0000 7fb8cbe00640 1 -- 172.21.15.66:0/3292254714 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb8cc004a50 con 0x7fb8d4153300 2024-09-10T22:00:47.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.478+0000 7fb8cbe00640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8b00725e0 0x7fb8b0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:47.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.478+0000 7fb8cbe00640 1 -- 172.21.15.66:0/3292254714 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb8cc12cf10 con 0x7fb8d4153300 2024-09-10T22:00:47.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.478+0000 7fb8d3400640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8b00725e0 0x7fb8b0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:47.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.478+0000 7fb8d3400640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8b00725e0 0x7fb8b0074aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb8d415ec00 tx=0x7fb8c40067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:47.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.482+0000 7fb8cbe00640 1 -- 172.21.15.66:0/3292254714 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb8cc017300 con 0x7fb8d4153300 2024-09-10T22:00:47.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.690+0000 7fb8da400640 1 -- 172.21.15.66:0/3292254714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8d4070470 con 0x7fb8d4153300 2024-09-10T22:00:47.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.690+0000 7fb8cbe00640 1 -- 172.21.15.66:0/3292254714 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7fb8cc0f8050 con 0x7fb8d4153300 2024-09-10T22:00:47.695 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:47.696 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:00:36:190941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2728893456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2728893456},{"type":"v1","addr":"172.21.15.94:6835","nonce":2728893456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2533443317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2533443317},{"type":"v1","addr":"172.21.15.66:6834","nonce":2533443317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/317149387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":317149387},{"type":"v1","addr":"172.21.15.66:6835","nonce":317149387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:00:32.573702+0000","modified":"2024-09-10T22:00:35.587262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/4271184466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":4271184466},{"type":"v1","addr":"172.21.15.94:6837","nonce":4271184466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-10T22:00:47.696 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:00:47.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.694+0000 7fb8da400640 1 -- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8b00725e0 msgr2=0x7fb8b0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.694+0000 7fb8da400640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8b00725e0 0x7fb8b0074aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb8d415ec00 tx=0x7fb8c40067b0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.694+0000 7fb8da400640 1 -- 172.21.15.66:0/3292254714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8d4153300 msgr2=0x7fb8d4153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.694+0000 7fb8da400640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8d4153300 0x7fb8d4153790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009e00 tx=0x7fb8cc004680 comp rx=0 tx=0).stop 2024-09-10T22:00:47.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.694+0000 7fb8da400640 1 -- 172.21.15.66:0/3292254714 shutdown_connections 2024-09-10T22:00:47.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.694+0000 7fb8da400640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8b00725e0 0x7fb8b0074aa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.694+0000 7fb8da400640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8d4153300 0x7fb8d4153790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.694+0000 7fb8da400640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8d40c5e40 0x7fb8d40c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.694+0000 7fb8da400640 1 --2- 172.21.15.66:0/3292254714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8b80052f0 0x7fb8d40b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.694+0000 7fb8da400640 1 -- 172.21.15.66:0/3292254714 >> 172.21.15.66:0/3292254714 conn(0x7fb8d40cfa30 msgr2=0x7fb8d406e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:47.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.694+0000 7fb8da400640 1 -- 172.21.15.66:0/3292254714 shutdown_connections 2024-09-10T22:00:47.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:47.694+0000 7fb8da400640 1 -- 172.21.15.66:0/3292254714 wait complete. 2024-09-10T22:00:48.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:00:32.573702+0000', 'modified': '2024-09-10T22:00:35.587262+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4378}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4378': {'gid': 4378, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6837/4271184466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6836', 'nonce': 4271184466}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 4271184466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4378, 'qdb_cluster': [4378]} 2024-09-10T22:00:48.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:00:48.119 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T22:00:48.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 -- 172.21.15.66:0/174871565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3780737c0 msgr2=0x7fb37806cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 --2- 172.21.15.66:0/174871565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3780737c0 0x7fb37806cf70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb37806d8e0 tx=0x7fb36001c890 comp rx=0 tx=0).stop 2024-09-10T22:00:48.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 -- 172.21.15.66:0/174871565 shutdown_connections 2024-09-10T22:00:48.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 --2- 172.21.15.66:0/174871565 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37806d4b0 0x7fb378073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 --2- 172.21.15.66:0/174871565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3780737c0 0x7fb37806cf70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 --2- 172.21.15.66:0/174871565 >> v2:172.21.15.94:3300/0 conn(0x7fb378152b90 0x7fb37806f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 -- 172.21.15.66:0/174871565 >> 172.21.15.66:0/174871565 conn(0x7fb3780b26c0 msgr2=0x7fb3780b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:48.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 -- 172.21.15.66:0/174871565 shutdown_connections 2024-09-10T22:00:48.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 -- 172.21.15.66:0/174871565 wait complete. 2024-09-10T22:00:48.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 Processor -- start 2024-09-10T22:00:48.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 -- start start 2024-09-10T22:00:48.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb37806d4b0 0x7fb3780c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3780737c0 0x7fb37814de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb378152b90 0x7fb3780cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb37815f9d0 con 0x7fb3780737c0 2024-09-10T22:00:48.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb37815f850 con 0x7fb37806d4b0 2024-09-10T22:00:48.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37fc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb37815fb50 con 0x7fb378152b90 2024-09-10T22:00:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb36fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3780737c0 0x7fb37814de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb37806d4b0 0x7fb3780c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb36fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3780737c0 0x7fb37814de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45882/0 (socket says 172.21.15.66:45882) 2024-09-10T22:00:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb36fe00640 1 -- 172.21.15.66:0/1890848333 learned_addr learned my addr 172.21.15.66:0/1890848333 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.266+0000 7fb37dc00640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb378152b90 0x7fb3780cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.270+0000 7fb36fe00640 1 -- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb378152b90 msgr2=0x7fb3780cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.270+0000 7fb36fe00640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb378152b90 0x7fb3780cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.270+0000 7fb36fe00640 1 -- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb37806d4b0 msgr2=0x7fb3780c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.270+0000 7fb36fe00640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb37806d4b0 0x7fb3780c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.270+0000 7fb36fe00640 1 -- 172.21.15.66:0/1890848333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3781531e0 con 0x7fb3780737c0 2024-09-10T22:00:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.270+0000 7fb36fe00640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3780737c0 0x7fb37814de90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb3600b0040 tx=0x7fb360004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.270+0000 7fb375600640 1 -- 172.21.15.66:0/1890848333 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600b2040 con 0x7fb3780737c0 2024-09-10T22:00:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.270+0000 7fb375600640 1 -- 172.21.15.66:0/1890848333 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb360004450 con 0x7fb3780737c0 2024-09-10T22:00:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.270+0000 7fb375600640 1 -- 172.21.15.66:0/1890848333 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360005290 con 0x7fb3780737c0 2024-09-10T22:00:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.270+0000 7fb37fc00640 1 -- 172.21.15.66:0/1890848333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3781534c0 con 0x7fb3780737c0 2024-09-10T22:00:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.270+0000 7fb37fc00640 1 -- 172.21.15.66:0/1890848333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3780c7540 con 0x7fb3780737c0 2024-09-10T22:00:48.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.270+0000 7fb375600640 1 -- 172.21.15.66:0/1890848333 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb360005430 con 0x7fb3780737c0 2024-09-10T22:00:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb375600640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3540725e0 0x7fb354074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb375600640 1 -- 172.21.15.66:0/1890848333 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb36012cd40 con 0x7fb3780737c0 2024-09-10T22:00:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 Processor -- start 2024-09-10T22:00:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 -- start start 2024-09-10T22:00:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37d200640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3540725e0 0x7fb354074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37d200640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3540725e0 0x7fb354074aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb36800aac0 tx=0x7fb36800a250 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:48.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 0x561ad919cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad919d2f0 0x561ad91508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad9150e10 0x561ad91511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561ad90713f0 con 0x561ad919c9d0 2024-09-10T22:00:48.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561ad8faeb20 con 0x561ad919d2f0 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x561ad8f9df50 con 0x561ad9150e10 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 0x561ad919cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 0x561ad919cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45888/0 (socket says 172.21.15.66:45888) 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37d200640 1 -- 172.21.15.66:0/571156878 learned_addr learned my addr 172.21.15.66:0/571156878 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb36fe00640 1 --2- 172.21.15.66:0/571156878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad919d2f0 0x561ad91508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37dc00640 1 --2- 172.21.15.66:0/571156878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad9150e10 0x561ad91511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37d200640 1 -- 172.21.15.66:0/571156878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad9150e10 msgr2=0x561ad91511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37d200640 1 --2- 172.21.15.66:0/571156878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad9150e10 0x561ad91511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37d200640 1 -- 172.21.15.66:0/571156878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad919d2f0 msgr2=0x561ad91508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37d200640 1 --2- 172.21.15.66:0/571156878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad919d2f0 0x561ad91508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37d200640 1 -- 172.21.15.66:0/571156878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ad8f65590 con 0x561ad919c9d0 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37dc00640 1 --2- 172.21.15.66:0/571156878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad9150e10 0x561ad91511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb37d200640 1 --2- 172.21.15.66:0/571156878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 0x561ad919cdb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb36800fec0 tx=0x7fb368012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37b5f02634b380d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb36d600640 1 -- 172.21.15.66:0/571156878 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368016070 con 0x561ad919c9d0 2024-09-10T22:00:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb36d600640 1 -- 172.21.15.66:0/571156878 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb368025630 con 0x561ad919c9d0 2024-09-10T22:00:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb36d600640 1 -- 172.21.15.66:0/571156878 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3680259e0 con 0x561ad919c9d0 2024-09-10T22:00:48.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 -- 172.21.15.66:0/571156878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 msgr2=0x561ad919cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 --2- 172.21.15.66:0/571156878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 0x561ad919cdb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb36800fec0 tx=0x7fb368012000 comp rx=0 tx=0).stop 2024-09-10T22:00:48.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 -- 172.21.15.66:0/571156878 shutdown_connections 2024-09-10T22:00:48.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 --2- 172.21.15.66:0/571156878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad9150e10 0x561ad91511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 --2- 172.21.15.66:0/571156878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad919d2f0 0x561ad91508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 --2- 172.21.15.66:0/571156878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 0x561ad919cdb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 -- 172.21.15.66:0/571156878 >> 172.21.15.66:0/571156878 conn(0x561ad903a130 msgr2=0x561ad917fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:48.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 -- 172.21.15.66:0/571156878 shutdown_connections 2024-09-10T22:00:48.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 -- 172.21.15.66:0/571156878 wait complete. 2024-09-10T22:00:48.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.274+0000 7fb382f98000 1 -- start start 2024-09-10T22:00:48.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 1 Processor -- start 2024-09-10T22:00:48.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad9150e10 0x561ad917d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919c9d0 0x561ad917de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919d2f0 0x561ad917e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561ad8f385d0 con 0x561ad919d2f0 2024-09-10T22:00:48.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561ad8fa1d60 con 0x561ad9150e10 2024-09-10T22:00:48.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x561ad8f44500 con 0x561ad919c9d0 2024-09-10T22:00:48.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 10 client.0 _ll_register_callbacks cb 0x561ad8f48810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:00:48.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb37dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919d2f0 0x561ad917e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb37dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919d2f0 0x561ad917e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45902/0 (socket says 172.21.15.66:45902) 2024-09-10T22:00:48.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb37dc00640 1 -- 172.21.15.66:0/2773405022 learned_addr learned my addr 172.21.15.66:0/2773405022 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:48.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb37d200640 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad9150e10 0x561ad917d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb36fe00640 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919c9d0 0x561ad917de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb37dc00640 1 -- 172.21.15.66:0/2773405022 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919c9d0 msgr2=0x561ad917de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:00:48.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb37dc00640 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919c9d0 0x561ad917de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb37dc00640 1 -- 172.21.15.66:0/2773405022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad9150e10 msgr2=0x561ad917d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:00:48.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb37dc00640 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad9150e10 0x561ad917d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb37dc00640 1 -- 172.21.15.66:0/2773405022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ad8f3c170 con 0x561ad919d2f0 2024-09-10T22:00:48.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb37dc00640 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919d2f0 0x561ad917e3b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb370011a90 tx=0x7fb370011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76a3d8ec2386f9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:48.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb366000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:00:48.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb366000640 1 -- 172.21.15.66:0/2773405022 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37001e060 con 0x561ad919d2f0 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb366000640 1 -- 172.21.15.66:0/2773405022 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb370003980 con 0x561ad919d2f0 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 1 -- 172.21.15.66:0/2773405022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561ad8f360f0 con 0x561ad919d2f0 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb366000640 1 -- 172.21.15.66:0/2773405022 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fb370024050 con 0x561ad919d2f0 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 10 client.4659 fetch_fsmap learned FSMap version 8 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 1 -- 172.21.15.66:0/2773405022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x561ad9022ef0 con 0x561ad919d2f0 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb366000640 1 -- 172.21.15.66:0/2773405022 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x561ad8f360f0 con 0x561ad919d2f0 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb366000640 1 -- 172.21.15.66:0/2773405022 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fb370027020 con 0x561ad919d2f0 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 10 client.4659 fetch_fsmap finished waiting for FSMap version 8 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 10 client.4659 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 20 client.4659 populate_metadata read hostname 'smithi066' 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.94:6836/4271184466,v1:172.21.15.94:6837/4271184466] conn(0x561ad917f0f0 0x561ad91aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 4 client.4659 mds_command: new command op to 4378 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:00:48.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb382f98000 1 -- 172.21.15.66:0/2773405022 --> [v2:172.21.15.94:6836/4271184466,v1:172.21.15.94:6837/4271184466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ad8fb0a70 con 0x561ad917f0f0 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.278+0000 7fb37d200640 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.94:6836/4271184466,v1:172.21.15.94:6837/4271184466] conn(0x561ad917f0f0 0x561ad91aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb366000640 10 client.4659 ms_handle_connect on v2:172.21.15.94:6836/4271184466 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb37d200640 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.94:6836/4271184466,v1:172.21.15.94:6837/4271184466] conn(0x561ad917f0f0 0x561ad91aec90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=854acc3bce23cff7 server_cookie=61d3e1b233bf000b in_seq=0 out_seq=0 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb366000640 1 -- 172.21.15.66:0/2773405022 <== mds.0 v2:172.21.15.94:6836/4271184466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561ad8fb0a70 con 0x561ad917f0f0 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb366000640 10 client.4659 handle_command_reply: tid=0 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb382f98000 1 client.4659 shutdown 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb382f98000 1 -- 172.21.15.66:0/2773405022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919d2f0 msgr2=0x561ad917e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb382f98000 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919d2f0 0x561ad917e3b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb370011a90 tx=0x7fb370011e90 comp rx=0 tx=0).stop 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb382f98000 1 -- 172.21.15.66:0/2773405022 shutdown_connections 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb382f98000 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.94:6836/4271184466,v1:172.21.15.94:6837/4271184466] conn(0x561ad917f0f0 0x561ad91aec90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb382f98000 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919d2f0 0x561ad917e3b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb382f98000 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919c9d0 0x561ad917de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb382f98000 1 --2- 172.21.15.66:0/2773405022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad9150e10 0x561ad917d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb382f98000 1 -- 172.21.15.66:0/2773405022 >> 172.21.15.66:0/2773405022 conn(0x561ad903a130 msgr2=0x561ad914ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:48.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb366000640 0 client.4659 ms_handle_reset on v2:172.21.15.94:6836/4271184466 2024-09-10T22:00:48.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb382f98000 1 -- 172.21.15.66:0/2773405022 shutdown_connections 2024-09-10T22:00:48.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb382f98000 1 -- 172.21.15.66:0/2773405022 wait complete. 2024-09-10T22:00:48.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.282+0000 7fb382f98000 20 client.4659 trim_cache size 0 max 16384 2024-09-10T22:00:48.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 Processor -- start 2024-09-10T22:00:48.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 -- start start 2024-09-10T22:00:48.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad9150e10 0x561ad919bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 0x561ad919c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919d2f0 0x561ad91aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb378203ec0 con 0x561ad919c9d0 2024-09-10T22:00:48.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb378202ed0 con 0x561ad9150e10 2024-09-10T22:00:48.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb3781f79e0 con 0x561ad919d2f0 2024-09-10T22:00:48.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb36fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 0x561ad919c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb36fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 0x561ad919c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45916/0 (socket says 172.21.15.66:45916) 2024-09-10T22:00:48.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb36fe00640 1 -- 172.21.15.66:0/2059044370 learned_addr learned my addr 172.21.15.66:0/2059044370 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:48.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb37dc00640 1 --2- 172.21.15.66:0/2059044370 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919d2f0 0x561ad91aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb37d200640 1 --2- 172.21.15.66:0/2059044370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad9150e10 0x561ad919bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb36fe00640 1 -- 172.21.15.66:0/2059044370 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919d2f0 msgr2=0x561ad91aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:00:48.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb36fe00640 1 --2- 172.21.15.66:0/2059044370 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919d2f0 0x561ad91aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb36fe00640 1 -- 172.21.15.66:0/2059044370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad9150e10 msgr2=0x561ad919bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:00:48.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb36fe00640 1 --2- 172.21.15.66:0/2059044370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad9150e10 0x561ad919bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb36fe00640 1 -- 172.21.15.66:0/2059044370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ad8ff9d90 con 0x561ad919c9d0 2024-09-10T22:00:48.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb36fe00640 1 --2- 172.21.15.66:0/2059044370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 0x561ad919c200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb3600f7ac0 tx=0x7fb3600095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c305a46e42dd2af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:48.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb36cc00640 1 -- 172.21.15.66:0/2059044370 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600f7dc0 con 0x561ad919c9d0 2024-09-10T22:00:48.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb36cc00640 1 -- 172.21.15.66:0/2059044370 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb360101210 con 0x561ad919c9d0 2024-09-10T22:00:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb36cc00640 1 -- 172.21.15.66:0/2059044370 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600f4030 con 0x561ad919c9d0 2024-09-10T22:00:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 -- 172.21.15.66:0/2059044370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 msgr2=0x561ad919c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 --2- 172.21.15.66:0/2059044370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 0x561ad919c200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb3600f7ac0 tx=0x7fb3600095e0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 -- 172.21.15.66:0/2059044370 shutdown_connections 2024-09-10T22:00:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 --2- 172.21.15.66:0/2059044370 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919d2f0 0x561ad91aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 --2- 172.21.15.66:0/2059044370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad919c9d0 0x561ad919c200 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 --2- 172.21.15.66:0/2059044370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad9150e10 0x561ad919bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 -- 172.21.15.66:0/2059044370 >> 172.21.15.66:0/2059044370 conn(0x561ad903a130 msgr2=0x561ad919b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 -- 172.21.15.66:0/2059044370 shutdown_connections 2024-09-10T22:00:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.302+0000 7fb382f98000 1 -- 172.21.15.66:0/2059044370 wait complete. 2024-09-10T22:00:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 1 -- start start 2024-09-10T22:00:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 1 Processor -- start 2024-09-10T22:00:48.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad9150e10 0x561ad9213230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919c9d0 0x561ad9213770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad919d2f0 0x561ad9213cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561ad914ad80 con 0x561ad9150e10 2024-09-10T22:00:48.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561ad8fa1070 con 0x561ad919d2f0 2024-09-10T22:00:48.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x561ad8fad8f0 con 0x561ad919c9d0 2024-09-10T22:00:48.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 10 client.0 _ll_register_callbacks cb 0x7fb370007980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:00:48.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb37d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad9150e10 0x561ad9213230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb37d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad9150e10 0x561ad9213230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45920/0 (socket says 172.21.15.66:45920) 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb36fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919c9d0 0x561ad9213770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb37d200640 1 -- 172.21.15.66:0/683934788 learned_addr learned my addr 172.21.15.66:0/683934788 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb37dc00640 1 --2- 172.21.15.66:0/683934788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad919d2f0 0x561ad9213cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb37d200640 1 -- 172.21.15.66:0/683934788 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919c9d0 msgr2=0x561ad9213770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb37d200640 1 --2- 172.21.15.66:0/683934788 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919c9d0 0x561ad9213770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb37d200640 1 -- 172.21.15.66:0/683934788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad919d2f0 msgr2=0x561ad9213cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb37d200640 1 --2- 172.21.15.66:0/683934788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad919d2f0 0x561ad9213cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb37d200640 1 -- 172.21.15.66:0/683934788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ad8f64a90 con 0x561ad9150e10 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb37d200640 1 --2- 172.21.15.66:0/683934788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad9150e10 0x561ad9213230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb36800f790 tx=0x7fb368004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f06b73a67b447e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb366000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb366000640 1 -- 172.21.15.66:0/683934788 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368025940 con 0x561ad9150e10 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 1 -- 172.21.15.66:0/683934788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb360101210 con 0x561ad9150e10 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb366000640 1 -- 172.21.15.66:0/683934788 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb368025ae0 con 0x561ad9150e10 2024-09-10T22:00:48.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb366000640 1 -- 172.21.15.66:0/683934788 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fb368044050 con 0x561ad9150e10 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 10 client.4665 fetch_fsmap learned FSMap version 8 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 1 -- 172.21.15.66:0/683934788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x561ad8f43050 con 0x561ad9150e10 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb366000640 1 -- 172.21.15.66:0/683934788 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb360101210 con 0x561ad9150e10 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb366000640 1 -- 172.21.15.66:0/683934788 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fb368047020 con 0x561ad9150e10 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 10 client.4665 fetch_fsmap finished waiting for FSMap version 8 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 10 client.4665 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 20 client.4665 populate_metadata read hostname 'smithi066' 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 1 --2- 172.21.15.66:0/683934788 >> [v2:172.21.15.94:6836/4271184466,v1:172.21.15.94:6837/4271184466] conn(0x561ad91b2660 0x561ad91b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 4 client.4665 mds_command: new command op to 4378 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb382f98000 1 -- 172.21.15.66:0/683934788 --> [v2:172.21.15.94:6836/4271184466,v1:172.21.15.94:6837/4271184466] -- command(tid 0: {"prefix": "status"}) -- 0x561ad8fac520 con 0x561ad91b2660 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.306+0000 7fb36fe00640 1 --2- 172.21.15.66:0/683934788 >> [v2:172.21.15.94:6836/4271184466,v1:172.21.15.94:6837/4271184466] conn(0x561ad91b2660 0x561ad91b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb36fe00640 1 --2- 172.21.15.66:0/683934788 >> [v2:172.21.15.94:6836/4271184466,v1:172.21.15.94:6837/4271184466] conn(0x561ad91b2660 0x561ad91b1780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=730897c82628f60d server_cookie=9a32624adadf3464 in_seq=0 out_seq=0 2024-09-10T22:00:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb366000640 10 client.4665 ms_handle_connect on v2:172.21.15.94:6836/4271184466 2024-09-10T22:00:48.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb366000640 1 -- 172.21.15.66:0/683934788 <== mds.0 v2:172.21.15.94:6836/4271184466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x561ad8fac520 con 0x561ad91b2660 2024-09-10T22:00:48.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb366000640 10 client.4665 handle_command_reply: tid=0 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb382f98000 1 client.4665 shutdown 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb382f98000 1 -- 172.21.15.66:0/683934788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad9150e10 msgr2=0x561ad9213230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb382f98000 1 --2- 172.21.15.66:0/683934788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad9150e10 0x561ad9213230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb36800f790 tx=0x7fb368004740 comp rx=0 tx=0).stop 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb382f98000 1 -- 172.21.15.66:0/683934788 shutdown_connections 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb382f98000 1 --2- 172.21.15.66:0/683934788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad919d2f0 0x561ad9213cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb382f98000 1 --2- 172.21.15.66:0/683934788 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x561ad919c9d0 0x561ad9213770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb382f98000 1 --2- 172.21.15.66:0/683934788 >> [v2:172.21.15.94:6836/4271184466,v1:172.21.15.94:6837/4271184466] conn(0x561ad91b2660 0x561ad91b1780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb382f98000 1 --2- 172.21.15.66:0/683934788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ad9150e10 0x561ad9213230 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb382f98000 1 -- 172.21.15.66:0/683934788 >> 172.21.15.66:0/683934788 conn(0x561ad903a130 msgr2=0x561ad9158a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb366000640 0 client.4665 ms_handle_reset on v2:172.21.15.94:6836/4271184466 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb382f98000 1 -- 172.21.15.66:0/683934788 shutdown_connections 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb382f98000 1 -- 172.21.15.66:0/683934788 wait complete. 2024-09-10T22:00:48.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb382f98000 20 client.4665 trim_cache size 0 max 16384 2024-09-10T22:00:48.318 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:00:48.318 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:00:48.318 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:00:48.318 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4378, 2024-09-10T22:00:48.318 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:00:48.318 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:00:48.318 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:00:48.318 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 15.135482484000001, 2024-09-10T22:00:48.318 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 8, 2024-09-10T22:00:48.318 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 22, 2024-09-10T22:00:48.318 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 22, 2024-09-10T22:00:48.319 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 21.167415852000001 2024-09-10T22:00:48.319 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:00:48.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb37fc00640 1 -- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3540725e0 msgr2=0x7fb354074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb37fc00640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3540725e0 0x7fb354074aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb36800aac0 tx=0x7fb36800a250 comp rx=0 tx=0).stop 2024-09-10T22:00:48.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb37fc00640 1 -- 172.21.15.66:0/1890848333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3780737c0 msgr2=0x7fb37814de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb37fc00640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3780737c0 0x7fb37814de90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb3600b0040 tx=0x7fb360004040 comp rx=0 tx=0).stop 2024-09-10T22:00:48.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb37fc00640 1 -- 172.21.15.66:0/1890848333 shutdown_connections 2024-09-10T22:00:48.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb37fc00640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3540725e0 0x7fb354074aa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb37fc00640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb378152b90 0x7fb3780cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb37fc00640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3780737c0 0x7fb37814de90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb37fc00640 1 --2- 172.21.15.66:0/1890848333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb37806d4b0 0x7fb3780c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb37fc00640 1 -- 172.21.15.66:0/1890848333 >> 172.21.15.66:0/1890848333 conn(0x7fb3780b26c0 msgr2=0x7fb37815d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:48.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb37fc00640 1 -- 172.21.15.66:0/1890848333 shutdown_connections 2024-09-10T22:00:48.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.310+0000 7fb37fc00640 1 -- 172.21.15.66:0/1890848333 wait complete. 2024-09-10T22:00:48.333 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-10T22:00:48.360 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-10T22:00:48.360 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-10T22:00:48.361 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-10T22:00:48.361 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:00:48.361 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:00:48.361 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-10T22:00:48.361 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:00:48.361 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T22:00:48.361 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-10T22:00:48.361 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:00:48.361 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T22:00:48.361 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-10T22:00:48.362 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:00:48.362 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T22:00:48.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:48.362 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:00:48.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:48.372 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-09-10T22:00:48.440 INFO:teuthology.orchestra.run.smithi066.stderr:Cannot find device "ceph-brx" 2024-09-10T22:00:48.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:00:48.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:48.441 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T22:00:48.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:48.452 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-09-10T22:00:48.516 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-09-10T22:00:48.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:00:48.518 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-10T22:00:48.518 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T22:00:48.518 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:00:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 -- 172.21.15.66:0/3144520189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2a806dcd0 msgr2=0x7fb2a806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 --2- 172.21.15.66:0/3144520189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2a806dcd0 0x7fb2a806e1f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80d2cf0 tx=0x7fb29401c8c0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 -- 172.21.15.66:0/3144520189 shutdown_connections 2024-09-10T22:00:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 --2- 172.21.15.66:0/3144520189 >> v2:172.21.15.94:3300/0 conn(0x7fb2a806e730 0x7fb2a80ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 --2- 172.21.15.66:0/3144520189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2a806dcd0 0x7fb2a806e1f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 --2- 172.21.15.66:0/3144520189 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2a8161570 0x7fb2a80b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 -- 172.21.15.66:0/3144520189 >> 172.21.15.66:0/3144520189 conn(0x7fb2a80d1c60 msgr2=0x7fb2a80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 -- 172.21.15.66:0/3144520189 shutdown_connections 2024-09-10T22:00:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 -- 172.21.15.66:0/3144520189 wait complete. 2024-09-10T22:00:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 Processor -- start 2024-09-10T22:00:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 -- start start 2024-09-10T22:00:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2a806dcd0 0x7fb2a80b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2a806e730 0x7fb2a80c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a8161570 0x7fb2a8158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a80d00b0 con 0x7fb2a806dcd0 2024-09-10T22:00:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a80cff30 con 0x7fb2a8161570 2024-09-10T22:00:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2ade00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a80d0230 con 0x7fb2a806e730 2024-09-10T22:00:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2a6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2a806dcd0 0x7fb2a80b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2a6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2a806dcd0 0x7fb2a80b7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45950/0 (socket says 172.21.15.66:45950) 2024-09-10T22:00:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2a6a00640 1 -- 172.21.15.66:0/2676780901 learned_addr learned my addr 172.21.15.66:0/2676780901 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2a6000640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2a806e730 0x7fb2a80c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2a7400640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a8161570 0x7fb2a8158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2a6a00640 1 -- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2a806e730 msgr2=0x7fb2a80c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2a6a00640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2a806e730 0x7fb2a80c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2a6a00640 1 -- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a8161570 msgr2=0x7fb2a8158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2a6a00640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a8161570 0x7fb2a8158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.662+0000 7fb2a6a00640 1 -- 172.21.15.66:0/2676780901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a80b7970 con 0x7fb2a806dcd0 2024-09-10T22:00:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.666+0000 7fb2a6a00640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2a806dcd0 0x7fb2a80b7430 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb29800e7c0 tx=0x7fb29800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.666+0000 7fb29f400640 1 -- 172.21.15.66:0/2676780901 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29800ccc0 con 0x7fb2a806dcd0 2024-09-10T22:00:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.666+0000 7fb29f400640 1 -- 172.21.15.66:0/2676780901 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb29800ce80 con 0x7fb2a806dcd0 2024-09-10T22:00:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.666+0000 7fb29f400640 1 -- 172.21.15.66:0/2676780901 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298005280 con 0x7fb2a806dcd0 2024-09-10T22:00:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.666+0000 7fb2ade00640 1 -- 172.21.15.66:0/2676780901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a8159f80 con 0x7fb2a806dcd0 2024-09-10T22:00:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.666+0000 7fb2ade00640 1 -- 172.21.15.66:0/2676780901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a815a540 con 0x7fb2a806dcd0 2024-09-10T22:00:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.666+0000 7fb2ade00640 1 -- 172.21.15.66:0/2676780901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a80ae9d0 con 0x7fb2a806dcd0 2024-09-10T22:00:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.666+0000 7fb29f400640 1 -- 172.21.15.66:0/2676780901 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb298005420 con 0x7fb2a806dcd0 2024-09-10T22:00:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.666+0000 7fb29f400640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb2880725e0 0x7fb288074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.666+0000 7fb29f400640 1 -- 172.21.15.66:0/2676780901 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb298093a30 con 0x7fb2a806dcd0 2024-09-10T22:00:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.670+0000 7fb2a6000640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb2880725e0 0x7fb288074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.670+0000 7fb2a6000640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb2880725e0 0x7fb288074aa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb2940af000 tx=0x7fb29401b100 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:48.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.670+0000 7fb29f400640 1 -- 172.21.15.66:0/2676780901 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb29805eab0 con 0x7fb2a806dcd0 2024-09-10T22:00:48.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 -- 172.21.15.66:0/2676780901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb2a80b3c80 con 0x7fb2a806dcd0 2024-09-10T22:00:48.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb29f400640 1 -- 172.21.15.66:0/2676780901 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fb2980634e0 con 0x7fb2a806dcd0 2024-09-10T22:00:48.891 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:00:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 -- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb2880725e0 msgr2=0x7fb288074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb2880725e0 0x7fb288074aa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb2940af000 tx=0x7fb29401b100 comp rx=0 tx=0).stop 2024-09-10T22:00:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 -- 172.21.15.66:0/2676780901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2a806dcd0 msgr2=0x7fb2a80b7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2a806dcd0 0x7fb2a80b7430 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb29800e7c0 tx=0x7fb29800ec80 comp rx=0 tx=0).stop 2024-09-10T22:00:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 -- 172.21.15.66:0/2676780901 shutdown_connections 2024-09-10T22:00:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb2880725e0 0x7fb288074aa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a8161570 0x7fb2a8158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2a806e730 0x7fb2a80c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 --2- 172.21.15.66:0/2676780901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2a806dcd0 0x7fb2a80b7430 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 -- 172.21.15.66:0/2676780901 >> 172.21.15.66:0/2676780901 conn(0x7fb2a80d1c60 msgr2=0x7fb2a815fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 -- 172.21.15.66:0/2676780901 shutdown_connections 2024-09-10T22:00:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:48.886+0000 7fb2ade00640 1 -- 172.21.15.66:0/2676780901 wait complete. 2024-09-10T22:00:48.912 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:00:48.912 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:00:48.912 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:00:48.912 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:00:48.912 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:00:48.912 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:00:48.912 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:00:48.912 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:00:48.912 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:00:48.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:48.913 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:00:48.922 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:00:48.922 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:00:48.922 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:00:48.922 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:48.922 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:00:48.922 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:48.922 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:00:48.922 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42426sec preferred_lft 42426sec 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:48.923 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:00:48.924 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:81:fc:4c:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:48.924 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:00:48.924 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:48.924 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T22:00:48.924 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:48.924 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:00:48.924 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add name ceph-brx type bridge 2024-09-10T22:00:48.924 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr flush dev ceph-brx 2024-09-10T22:00:48.924 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set ceph-brx up 2024-09-10T22:00:48.925 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T22:00:48.925 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:00:49.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:49.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:49.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:49.078 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:00:49.097 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:00:49.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:49.099 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:00:49.172 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:00:49.172 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:00:49.172 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:00:49.172 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:00:49.172 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:00:49.172 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:00:49.172 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:00:49.172 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:00:49.173 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:00:49.173 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:00:49.173 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:00:49.173 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:49.173 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:00:49.173 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:00:49.173 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:00:49.173 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:00:49.174 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:00:49.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:49.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:49.308 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:00:49.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:49.317 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-09-10T22:00:49.373 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:49.374 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:00:49.374 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:00:49.374 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T22:00:49.374 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:00:49.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:49.501 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T22:00:49.501 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:49.502 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:00:49.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T22:00:49.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T22:00:49.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T22:00:49.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T22:00:49.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T22:00:49.502 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:00:49.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:49.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:49.781 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:49.781 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:00:49.781 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:00:49.781 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T22:00:49.781 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:00:49.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:49.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:49.914 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T22:00:49.914 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:00:49.914 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:00:49.923 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:00:49.923 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:00:49.923 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:00:49.976 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:00:50.041 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:00:50.094 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:00:50.094 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:00:50.094 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:00:50.094 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:00:50.094 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:00:50.094 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:00:50.094 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:00:50.094 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:00:50.094 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:00:50.095 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:00:50.096 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:00:50.096 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:00:50.096 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:00:50.096 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:00:50.096 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:00:50.096 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:00:50.096 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:00:50.096 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:00:50.096 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:00:50.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:50.097 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:00:50.148 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:00:50.148 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T22:00:50.195 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:00:50.215 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:00:50.266 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:00:50.266 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:00:50.266 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:00:50.266 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:00:50.266 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:00:50.267 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:00:50.268 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:00:50.268 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:00:50.268 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:00:50.268 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:00:50.268 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:00:50.268 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:00:50.268 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:00:50.268 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:00:50.268 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:00:50.268 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:00:50.268 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:00:50.268 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:00:50.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:50.269 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:00:50.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-09-10T22:00:50.286+0000 7f1cb9766580 -1 init, newargv = 0x55c7b7867c90 newargc=15 2024-09-10T22:00:50.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[35551]: starting ceph client 2024-09-10T22:00:50.301 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[35551]: starting fuse 2024-09-10T22:00:51.401 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:00:51.426 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:00:51.479 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:00:51.479 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:00:51.479 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:00:51.479 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:00:51.479 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:00:51.479 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:00:51.480 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:00:51.481 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:00:51.481 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:00:51.481 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:00:51.481 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:00:51.481 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:00:51.481 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:00:51.481 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:00:51.481 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:00:51.481 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:00:51.481 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:00:51.481 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:00:51.481 INFO:teuthology.orchestra.run.smithi066.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T22:00:51.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:51.482 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:00:51.533 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-09-10T22:00:51.533 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T22:00:51.533 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-09-10T22:00:51.534 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:00:51.535 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-09-10T22:00:51.535 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:00:51.535 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:00:51.535 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:00:51.535 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-09-10T22:00:51.535 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-09-10T22:00:51.535 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-09-10T22:00:51.535 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:00:51.535 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:00:51.535 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-09-10T22:00:51.535 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:00:51.669 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.35551.asok 2024-09-10T22:00:51.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:51.681 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35551.asok 2024-09-10T22:00:51.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:51.681 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35551.asok status 2024-09-10T22:00:51.857 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:00:51.857 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-09-10T22:00:51.857 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "35551", 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4639, 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-09-10T22:00:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 4639 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3993583449 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3993583449 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:00:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.4639 172.21.15.66:0/3993583449", 2024-09-10T22:00:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "172.21.15.66:0/3993583449", 2024-09-10T22:00:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-09-10T22:00:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 8, 2024-09-10T22:00:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 22, 2024-09-10T22:00:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:00:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-09-10T22:00:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-09-10T22:00:51.860 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:00:51.872 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T22:00:51.872 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:00:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.014+0000 7f1c1ec00640 1 -- 172.21.15.66:0/2413394557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c180cf830 msgr2=0x7f1c180d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.014+0000 7f1c1ec00640 1 --2- 172.21.15.66:0/2413394557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c180cf830 0x7f1c180d04e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1c18073840 tx=0x7f1c0001c770 comp rx=0 tx=0).stop 2024-09-10T22:00:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.014+0000 7f1c1ec00640 1 -- 172.21.15.66:0/2413394557 shutdown_connections 2024-09-10T22:00:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.014+0000 7f1c1ec00640 1 --2- 172.21.15.66:0/2413394557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c180cf830 0x7f1c180d04e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.014+0000 7f1c1ec00640 1 --2- 172.21.15.66:0/2413394557 >> v2:172.21.15.94:3300/0 conn(0x7f1c1815bd40 0x7f1c180cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.014+0000 7f1c1ec00640 1 -- 172.21.15.66:0/2413394557 >> 172.21.15.66:0/2413394557 conn(0x7f1c180d1c60 msgr2=0x7f1c180d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.014+0000 7f1c1ec00640 1 -- 172.21.15.66:0/2413394557 shutdown_connections 2024-09-10T22:00:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.014+0000 7f1c1ec00640 1 -- 172.21.15.66:0/2413394557 wait complete. 2024-09-10T22:00:52.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1ec00640 1 Processor -- start 2024-09-10T22:00:52.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1ec00640 1 -- start start 2024-09-10T22:00:52.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1ec00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c180aec10 0x7f1c181f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:52.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1ec00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c180cf830 0x7f1c180b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:52.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c1815bd40 0x7f1c180b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:52.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18168fd0 con 0x7f1c1815bd40 2024-09-10T22:00:52.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1ec00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18168e50 con 0x7f1c180cf830 2024-09-10T22:00:52.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1ec00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1c18169150 con 0x7f1c180aec10 2024-09-10T22:00:52.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c1815bd40 0x7f1c180b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:52.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c1815bd40 0x7f1c180b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45970/0 (socket says 172.21.15.66:45970) 2024-09-10T22:00:52.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1cc00640 1 -- 172.21.15.66:0/3299581202 learned_addr learned my addr 172.21.15.66:0/3299581202 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:52.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c17e00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c180aec10 0x7f1c181f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:52.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c0fe00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c180cf830 0x7f1c180b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:52.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1cc00640 1 -- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c180aec10 msgr2=0x7f1c181f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:52.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1cc00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c180aec10 0x7f1c181f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1cc00640 1 -- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c180cf830 msgr2=0x7f1c180b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:52.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1cc00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c180cf830 0x7f1c180b7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1cc00640 1 -- 172.21.15.66:0/3299581202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c180b1d90 con 0x7f1c1815bd40 2024-09-10T22:00:52.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1cc00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c1815bd40 0x7f1c180b5190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000cbf0 tx=0x7f1c1000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:52.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c14c00640 1 -- 172.21.15.66:0/3299581202 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10016020 con 0x7f1c1815bd40 2024-09-10T22:00:52.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c14c00640 1 -- 172.21.15.66:0/3299581202 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c1000ab40 con 0x7f1c1815bd40 2024-09-10T22:00:52.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1ec00640 1 -- 172.21.15.66:0/3299581202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c18156b70 con 0x7f1c1815bd40 2024-09-10T22:00:52.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c14c00640 1 -- 172.21.15.66:0/3299581202 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10012420 con 0x7f1c1815bd40 2024-09-10T22:00:52.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1ec00640 1 -- 172.21.15.66:0/3299581202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c18165cb0 con 0x7f1c1815bd40 2024-09-10T22:00:52.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.018+0000 7f1c1ec00640 1 -- 172.21.15.66:0/3299581202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c180af0a0 con 0x7f1c1815bd40 2024-09-10T22:00:52.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.022+0000 7f1c14c00640 1 -- 172.21.15.66:0/3299581202 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1c1001a3b0 con 0x7f1c1815bd40 2024-09-10T22:00:52.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.022+0000 7f1c14c00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1bf40725e0 0x7f1bf4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:52.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.022+0000 7f1c14c00640 1 -- 172.21.15.66:0/3299581202 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1c10093440 con 0x7f1c1815bd40 2024-09-10T22:00:52.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.022+0000 7f1c17e00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1bf40725e0 0x7f1bf4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:52.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.022+0000 7f1c17e00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1bf40725e0 0x7f1bf4074aa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800bb50 tx=0x7f1c0800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:52.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.026+0000 7f1c14c00640 1 -- 172.21.15.66:0/3299581202 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1c10020030 con 0x7f1c1815bd40 2024-09-10T22:00:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.238+0000 7f1c1ec00640 1 -- 172.21.15.66:0/3299581202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1c180b0df0 con 0x7f1c1815bd40 2024-09-10T22:00:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.238+0000 7f1c14c00640 1 -- 172.21.15.66:0/3299581202 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f1c10067230 con 0x7f1c1815bd40 2024-09-10T22:00:52.244 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:00:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.242+0000 7f1c1ec00640 1 -- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1bf40725e0 msgr2=0x7f1bf4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.242+0000 7f1c1ec00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1bf40725e0 0x7f1bf4074aa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800bb50 tx=0x7f1c0800b2e0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.242+0000 7f1c1ec00640 1 -- 172.21.15.66:0/3299581202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c1815bd40 msgr2=0x7f1c180b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.242+0000 7f1c1ec00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c1815bd40 0x7f1c180b5190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000cbf0 tx=0x7f1c1000cce0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.242+0000 7f1c1ec00640 1 -- 172.21.15.66:0/3299581202 shutdown_connections 2024-09-10T22:00:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.242+0000 7f1c1ec00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1bf40725e0 0x7f1bf4074aa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.242+0000 7f1c1ec00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c1815bd40 0x7f1c180b5190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.242+0000 7f1c1ec00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c180cf830 0x7f1c180b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.242+0000 7f1c1ec00640 1 --2- 172.21.15.66:0/3299581202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c180aec10 0x7f1c181f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.242+0000 7f1c1ec00640 1 -- 172.21.15.66:0/3299581202 >> 172.21.15.66:0/3299581202 conn(0x7f1c180d1c60 msgr2=0x7f1c180b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.242+0000 7f1c1ec00640 1 -- 172.21.15.66:0/3299581202 shutdown_connections 2024-09-10T22:00:52.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52.242+0000 7f1c1ec00640 1 -- 172.21.15.66:0/3299581202 wait complete. 2024-09-10T22:00:52.265 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:00:52.265 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:00:52.265 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:00:52.265 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-10T22:00:52.266 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T22:00:52.266 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:00:52.266 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:00:52.266 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:00:52.266 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-10T22:00:52.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:52.266 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:00:52.275 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:00:52.275 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:00:52.275 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:00:52.275 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.276 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:00:52.276 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.276 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:00:52.276 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:52.276 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:00:52.276 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:00:52.276 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:52.276 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:00:52.276 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:00:52.276 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:52.276 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42422sec preferred_lft 42422sec 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:81:fc:4c:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:00:52.277 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.278 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T22:00:52.278 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.278 INFO:teuthology.orchestra.run.smithi066.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:00:52.278 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:00:52.278 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T22:00:52.278 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:52.279 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:00:52.342 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:00:52.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:52.344 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:00:52.400 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:00:52.401 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:00:52.401 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:00:52.401 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:00:52.401 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:00:52.401 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:00:52.401 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:00:52.401 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:00:52.401 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:00:52.401 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:00:52.401 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:00:52.402 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:52.402 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:00:52.402 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:00:52.402 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:00:52.402 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:00:52.402 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:00:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:52.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:52.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:52.544 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:00:52.552 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:00:52.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:52.554 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-09-10T22:00:52.608 INFO:teuthology.orchestra.run.smithi066.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-10T22:00:52.609 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:52.610 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:00:52.610 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:00:52.610 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-10T22:00:52.610 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:00:52.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:52.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:52.738 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T22:00:52.759 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:00:52.760 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:00:52.760 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:00:52.760 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.760 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:00:52.760 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.760 INFO:teuthology.orchestra.run.smithi066.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:00:52.760 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether be:ad:91:a2:c7:ff brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T22:00:52.760 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T22:00:52.760 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.760 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::bcad:91ff:fea2:c7ff/64 scope link 2024-09-10T22:00:52.761 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:52.762 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T22:00:52.825 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:00:52.825 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:00:52.825 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:00:52.825 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.825 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:00:52.825 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.825 INFO:teuthology.orchestra.run.smithi066.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:00:52.825 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether be:ad:91:a2:c7:ff brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T22:00:52.825 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T22:00:52.825 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.826 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::bcad:91ff:fea2:c7ff/64 scope link 2024-09-10T22:00:52.826 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:52.827 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-10T22:00:52.827 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:52.827 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:00:52.827 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-10T22:00:52.827 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T22:00:52.827 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-10T22:00:52.827 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-10T22:00:52.827 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-10T22:00:52.828 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:00:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:53.014 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:53.014 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:00:53.014 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.1 up 2024-09-10T22:00:53.014 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T22:00:53.014 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:00:53.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:53.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:53.100 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-10T22:00:53.101 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:00:53.101 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-10T22:00:53.110 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-10T22:00:53.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:00:53.110 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-10T22:00:53.168 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:00:53.238 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:00:53.292 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:00:53.292 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:00:53.292 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:00:53.292 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:00:53.292 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:00:53.292 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:00:53.292 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:00:53.293 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:00:53.293 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:00:53.293 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:00:53.293 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:00:53.293 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:00:53.293 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:00:53.294 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:00:53.294 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:00:53.294 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:00:53.294 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:00:53.294 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:00:53.294 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:00:53.294 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:00:53.294 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:00:53.294 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:00:53.294 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:00:53.294 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:00:53.295 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:00:53.295 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:00:53.295 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:00:53.295 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:00:53.295 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:00:53.295 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:00:53.295 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:00:53.295 INFO:teuthology.orchestra.run.smithi066.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T22:00:53.295 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:00:53.295 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:00:53.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:53.296 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:00:53.347 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-09-10T22:00:53.347 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-09-10T22:00:53.347 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-10T22:00:53.392 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:00:53.413 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:00:53.471 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:00:53.471 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:00:53.471 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:00:53.471 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:00:53.471 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:00:53.471 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:00:53.471 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:00:53.471 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:00:53.472 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:00:53.472 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:00:53.472 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:00:53.472 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:00:53.472 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:00:53.472 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:00:53.472 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:00:53.472 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:00:53.472 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:00:53.472 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:00:53.472 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T22:00:53.473 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:00:53.474 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:00:53.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:53.474 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:00:53.489 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi066.stderr:2024-09-10T22:00:53.486+0000 7fb381649580 -1 init, newargv = 0x56356c884d60 newargc=15 2024-09-10T22:00:53.489 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi066.stderr:ceph-fuse[35696]: starting ceph client 2024-09-10T22:00:53.497 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi066.stderr:ceph-fuse[35696]: starting fuse 2024-09-10T22:00:53.497 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-09-10T22:00:53.498 INFO:teuthology.orchestra.run.smithi066.stdout:45 2024-09-10T22:00:53.498 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2024-09-10T22:00:53.498 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:00:53.498 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-09-10T22:00:53.498 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-09-10T22:00:53.498 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:00:53.498 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-09-10T22:00:53.498 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:00:53.498 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-09-10T22:00:53.498 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-10T22:00:53.498 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-09-10T22:00:53.498 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-10T22:00:53.499 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:00:53.499 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-09-10T22:00:53.499 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-09-10T22:00:53.499 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-09-10T22:00:53.499 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-09-10T22:00:53.499 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:00:53.499 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-09-10T22:00:53.499 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:00:53.499 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:00:53.499 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:00:53.499 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-09-10T22:00:53.499 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-09-10T22:00:53.500 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-09-10T22:00:53.500 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:00:53.500 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:00:53.500 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.2")) 2024-09-10T22:00:53.500 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:00:53.627 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.2.35696.asok 2024-09-10T22:00:53.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:53.639 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35696.asok 2024-09-10T22:00:53.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:53.639 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35696.asok status 2024-09-10T22:00:53.812 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:00:53.812 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "2", 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "35696", 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4689, 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-09-10T22:00:53.813 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 4689 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.2:0", 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3797945890 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.2:0", 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3797945890 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:00:53.814 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.4689 192.168.0.2:0/3797945890", 2024-09-10T22:00:53.815 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.2:0/3797945890", 2024-09-10T22:00:53.815 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-09-10T22:00:53.815 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 8, 2024-09-10T22:00:53.815 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 22, 2024-09-10T22:00:53.815 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:00:53.815 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-09-10T22:00:53.815 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-09-10T22:00:53.815 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:00:53.827 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T22:00:53.827 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:00:53.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 -- 172.21.15.66:0/941937742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c0aec10 msgr2=0x7f261c0af010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:53.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 --2- 172.21.15.66:0/941937742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c0aec10 0x7f261c0af010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f261c073840 tx=0x7f261401ccf0 comp rx=0 tx=0).stop 2024-09-10T22:00:53.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 -- 172.21.15.66:0/941937742 shutdown_connections 2024-09-10T22:00:53.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 --2- 172.21.15.66:0/941937742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c0aec10 0x7f261c0af010 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:53.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 --2- 172.21.15.66:0/941937742 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f261c0cf830 0x7f261c0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:53.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 --2- 172.21.15.66:0/941937742 >> v2:172.21.15.94:3300/0 conn(0x7f261c15bd40 0x7f261c0cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:53.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 -- 172.21.15.66:0/941937742 >> 172.21.15.66:0/941937742 conn(0x7f261c0d1c60 msgr2=0x7f261c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:53.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 -- 172.21.15.66:0/941937742 shutdown_connections 2024-09-10T22:00:53.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 -- 172.21.15.66:0/941937742 wait complete. 2024-09-10T22:00:53.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 Processor -- start 2024-09-10T22:00:53.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 -- start start 2024-09-10T22:00:53.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c0aec10 0x7f261c0c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f261c0cf830 0x7f261c0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.970+0000 7f2624000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f261c15bd40 0x7f261c0caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2624000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f261c0cea60 con 0x7f261c0aec10 2024-09-10T22:00:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2624000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f261c0ce8e0 con 0x7f261c15bd40 2024-09-10T22:00:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2624000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f261c0cebe0 con 0x7f261c0cf830 2024-09-10T22:00:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2621600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c0aec10 0x7f261c0c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2621600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c0aec10 0x7f261c0c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57578/0 (socket says 172.21.15.66:57578) 2024-09-10T22:00:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2621600640 1 -- 172.21.15.66:0/3645336860 learned_addr learned my addr 172.21.15.66:0/3645336860 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2622000640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f261c15bd40 0x7f261c0caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2620c00640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f261c0cf830 0x7f261c0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2621600640 1 -- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f261c0cf830 msgr2=0x7f261c0ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2621600640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f261c0cf830 0x7f261c0ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2621600640 1 -- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f261c15bd40 msgr2=0x7f261c0caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2621600640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f261c15bd40 0x7f261c0caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2621600640 1 -- 172.21.15.66:0/3645336860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c14c2a0 con 0x7f261c0aec10 2024-09-10T22:00:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2621600640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c0aec10 0x7f261c0c8d30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f261000d950 tx=0x7f261000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2619600640 1 -- 172.21.15.66:0/3645336860 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610017070 con 0x7f261c0aec10 2024-09-10T22:00:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2619600640 1 -- 172.21.15.66:0/3645336860 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2610009d20 con 0x7f261c0aec10 2024-09-10T22:00:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2619600640 1 -- 172.21.15.66:0/3645336860 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100106c0 con 0x7f261c0aec10 2024-09-10T22:00:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2624000640 1 -- 172.21.15.66:0/3645336860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261c0cb670 con 0x7f261c0aec10 2024-09-10T22:00:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2624000640 1 -- 172.21.15.66:0/3645336860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261c0cbc30 con 0x7f261c0aec10 2024-09-10T22:00:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2624000640 1 -- 172.21.15.66:0/3645336860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261c0c7d10 con 0x7f261c0aec10 2024-09-10T22:00:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.974+0000 7f2619600640 1 -- 172.21.15.66:0/3645336860 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2610010860 con 0x7f261c0aec10 2024-09-10T22:00:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.978+0000 7f2619600640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f25fc0725e0 0x7f25fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.978+0000 7f2619600640 1 -- 172.21.15.66:0/3645336860 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2610093690 con 0x7f261c0aec10 2024-09-10T22:00:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.978+0000 7f2620c00640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f25fc0725e0 0x7f25fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.978+0000 7f2620c00640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f25fc0725e0 0x7f25fc074aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f261c070a60 tx=0x7f260800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:53.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:53.982+0000 7f2619600640 1 -- 172.21.15.66:0/3645336860 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f261005e680 con 0x7f261c0aec10 2024-09-10T22:00:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 -- 172.21.15.66:0/3645336860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f261c0af010 con 0x7f261c0aec10 2024-09-10T22:00:54.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2619600640 1 -- 172.21.15.66:0/3645336860 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f26100630b0 con 0x7f261c0aec10 2024-09-10T22:00:54.194 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:00:54.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 -- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f25fc0725e0 msgr2=0x7f25fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:54.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f25fc0725e0 0x7f25fc074aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f261c070a60 tx=0x7f260800b2e0 comp rx=0 tx=0).stop 2024-09-10T22:00:54.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 -- 172.21.15.66:0/3645336860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c0aec10 msgr2=0x7f261c0c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:54.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c0aec10 0x7f261c0c8d30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f261000d950 tx=0x7f261000de10 comp rx=0 tx=0).stop 2024-09-10T22:00:54.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 -- 172.21.15.66:0/3645336860 shutdown_connections 2024-09-10T22:00:54.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f25fc0725e0 0x7f25fc074aa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:54.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f261c15bd40 0x7f261c0caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:54.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f261c0cf830 0x7f261c0ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:54.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 --2- 172.21.15.66:0/3645336860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c0aec10 0x7f261c0c8d30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:54.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 -- 172.21.15.66:0/3645336860 >> 172.21.15.66:0/3645336860 conn(0x7f261c0d1c60 msgr2=0x7f261c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:54.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 -- 172.21.15.66:0/3645336860 shutdown_connections 2024-09-10T22:00:54.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:54.190+0000 7f2624000640 1 -- 172.21.15.66:0/3645336860 wait complete. 2024-09-10T22:00:54.216 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:00:54.216 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:00:54.216 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T22:00:54.216 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T22:00:54.216 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T22:00:54.216 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:00:54.216 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:00:54.216 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:00:54.216 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T22:00:54.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:54.217 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T22:00:54.226 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:00:54.226 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:00:54.226 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:54.227 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:00:54.228 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42393sec preferred_lft 42393sec 2024-09-10T22:00:54.228 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T22:00:54.228 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:54.228 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:00:54.228 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:36:e5:45:f6 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:54.228 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:00:54.228 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:54.229 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T22:00:54.229 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:54.229 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:00:54.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-09-10T22:00:54.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-09-10T22:00:54.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-09-10T22:00:54.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T22:00:54.229 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:00:54.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:54.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:54.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:54.394 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:00:54.412 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T22:00:54.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:54.414 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T22:00:54.502 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T22:00:54.502 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:00:54.502 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:00:54.502 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:00:54.502 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:00:54.502 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:00:54.502 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:00:54.502 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:00:54.502 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:00:54.502 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:00:54.503 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:00:54.503 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:54.503 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:00:54.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:00:54.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:00:54.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:00:54.503 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:00:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:54.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:54.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:54.596 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T22:00:54.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:54.605 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-09-10T22:00:54.661 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:54.661 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:00:54.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:00:54.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T22:00:54.661 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:00:54.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:54.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:54.787 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T22:00:54.787 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:54.787 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:00:54.787 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T22:00:54.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T22:00:54.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T22:00:54.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T22:00:54.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T22:00:54.788 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:00:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:54.851 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:00:54.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:00:54.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:54.951 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:54.951 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:00:54.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T22:00:54.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T22:00:54.951 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:00:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:55.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:55.074 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T22:00:55.074 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:00:55.075 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T22:00:55.125 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T22:00:55.125 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:00:55.126 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T22:00:55.176 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T22:00:55.236 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:00:55.289 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T22:00:55.290 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T22:00:55.290 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T22:00:55.290 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T22:00:55.290 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T22:00:55.290 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T22:00:55.290 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T22:00:55.290 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T22:00:55.290 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T22:00:55.290 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T22:00:55.290 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T22:00:55.290 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T22:00:55.291 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T22:00:55.292 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T22:00:55.292 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T22:00:55.292 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T22:00:55.292 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:00:55.292 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:00:55.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:55.293 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T22:00:55.348 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:00:55.348 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T22:00:55.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T22:00:55.411 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:00:55.466 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T22:00:55.466 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T22:00:55.466 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T22:00:55.466 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T22:00:55.466 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T22:00:55.466 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T22:00:55.466 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T22:00:55.466 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T22:00:55.466 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T22:00:55.466 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T22:00:55.467 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T22:00:55.468 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T22:00:55.468 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T22:00:55.468 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T22:00:55.468 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T22:00:55.468 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T22:00:55.468 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T22:00:55.468 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:00:55.468 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:00:55.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:55.469 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T22:00:55.485 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi094.stderr:2024-09-10T22:00:55.485+0000 7f7743d9e580 -1 init, newargv = 0x561468f0dca0 newargc=15 2024-09-10T22:00:55.485 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi094.stderr:ceph-fuse[33554]: starting ceph client 2024-09-10T22:00:55.494 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi094.stderr:ceph-fuse[33554]: starting fuse 2024-09-10T22:00:56.595 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T22:00:56.618 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:00:56.671 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T22:00:56.671 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T22:00:56.671 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T22:00:56.671 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T22:00:56.671 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T22:00:56.671 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T22:00:56.671 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T22:00:56.671 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T22:00:56.671 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T22:00:56.671 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T22:00:56.671 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T22:00:56.672 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T22:00:56.673 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T22:00:56.673 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T22:00:56.673 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T22:00:56.673 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T22:00:56.673 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:00:56.673 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:00:56.673 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-10T22:00:56.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:56.674 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T22:00:56.725 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T22:00:56.725 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T22:00:56.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:00:56.725 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T22:00:56.726 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T22:00:56.726 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T22:00:56.726 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T22:00:56.726 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T22:00:56.726 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T22:00:56.726 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T22:00:56.726 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T22:00:56.726 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T22:00:56.727 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T22:00:56.727 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T22:00:56.727 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T22:00:56.727 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T22:00:56.727 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T22:00:56.727 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T22:00:56.727 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T22:00:56.728 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:00:56.728 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:00:56.728 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:00:56.728 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T22:00:56.728 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T22:00:56.728 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T22:00:56.728 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:00:56.729 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T22:00:56.729 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.1")) 2024-09-10T22:00:56.729 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T22:00:56.857 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.1.33554.asok 2024-09-10T22:00:56.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:56.870 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33554.asok 2024-09-10T22:00:56.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:56.871 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33554.asok status 2024-09-10T22:00:57.043 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T22:00:57.044 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T22:00:57.044 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:00:57.044 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:00:57.044 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "1", 2024-09-10T22:00:57.044 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T22:00:57.044 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T22:00:57.044 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "33554", 2024-09-10T22:00:57.044 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T22:00:57.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T22:00:57.044 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T22:00:57.045 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T22:00:57.045 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4547, 2024-09-10T22:00:57.045 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T22:00:57.045 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T22:00:57.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T22:00:57.045 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 4547 2024-09-10T22:00:57.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T22:00:57.045 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T22:00:57.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T22:00:57.046 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:00:57.046 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2314477633 2024-09-10T22:00:57.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T22:00:57.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T22:00:57.046 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T22:00:57.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T22:00:57.046 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:00:57.046 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2314477633 2024-09-10T22:00:57.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T22:00:57.047 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.4547 192.168.0.1:0/2314477633", 2024-09-10T22:00:57.047 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/2314477633", 2024-09-10T22:00:57.047 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T22:00:57.047 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 8, 2024-09-10T22:00:57.047 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 22, 2024-09-10T22:00:57.047 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:00:57.047 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T22:00:57.047 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T22:00:57.047 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T22:00:57.058 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T22:00:57.058 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:00:57.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.214+0000 7f1eb6c00640 1 -- 172.21.15.66:0/1290218391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb015f300 msgr2=0x7f1eb00b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:57.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.214+0000 7f1eb6c00640 1 --2- 172.21.15.66:0/1290218391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb015f300 0x7f1eb00b1090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009870 tx=0x7f1e9c01c820 comp rx=0 tx=0).stop 2024-09-10T22:00:57.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.214+0000 7f1eb6c00640 1 -- 172.21.15.66:0/1290218391 shutdown_connections 2024-09-10T22:00:57.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.214+0000 7f1eb6c00640 1 --2- 172.21.15.66:0/1290218391 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1eb00b15d0 0x7f1eb00cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.214+0000 7f1eb6c00640 1 --2- 172.21.15.66:0/1290218391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb015f300 0x7f1eb00b1090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.214+0000 7f1eb6c00640 1 -- 172.21.15.66:0/1290218391 >> 172.21.15.66:0/1290218391 conn(0x7f1eb00d1b10 msgr2=0x7f1eb00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:57.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.214+0000 7f1eb6c00640 1 -- 172.21.15.66:0/1290218391 shutdown_connections 2024-09-10T22:00:57.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.214+0000 7f1eb6c00640 1 -- 172.21.15.66:0/1290218391 wait complete. 2024-09-10T22:00:57.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6c00640 1 Processor -- start 2024-09-10T22:00:57.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6c00640 1 -- start start 2024-09-10T22:00:57.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb00b15d0 0x7f1eb0163f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:57.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1eb015ef20 0x7f1eb015dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:57.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb015f300 0x7f1eb006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:57.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01692c0 con 0x7f1eb00b15d0 2024-09-10T22:00:57.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0169140 con 0x7f1eb015f300 2024-09-10T22:00:57.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb0169440 con 0x7f1eb015ef20 2024-09-10T22:00:57.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb00b15d0 0x7f1eb0163f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:57.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb00b15d0 0x7f1eb0163f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57602/0 (socket says 172.21.15.66:57602) 2024-09-10T22:00:57.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb5800640 1 -- 172.21.15.66:0/3697652911 learned_addr learned my addr 172.21.15.66:0/3697652911 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:57.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6200640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb015f300 0x7f1eb006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:57.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb4e00640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1eb015ef20 0x7f1eb015dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:57.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb5800640 1 -- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1eb015ef20 msgr2=0x7f1eb015dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:57.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb5800640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1eb015ef20 0x7f1eb015dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb5800640 1 -- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb015f300 msgr2=0x7f1eb006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb5800640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb015f300 0x7f1eb006dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb5800640 1 -- 172.21.15.66:0/3697652911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb01666d0 con 0x7f1eb00b15d0 2024-09-10T22:00:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6200640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb015f300 0x7f1eb006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:00:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb4e00640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1eb015ef20 0x7f1eb015dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:00:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb5800640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb00b15d0 0x7f1eb0163f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c500 tx=0x7f1ea400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1ead600640 1 -- 172.21.15.66:0/3697652911 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4017070 con 0x7f1eb00b15d0 2024-09-10T22:00:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1ead600640 1 -- 172.21.15.66:0/3697652911 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ea4003600 con 0x7f1eb00b15d0 2024-09-10T22:00:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1ead600640 1 -- 172.21.15.66:0/3697652911 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4003a90 con 0x7f1eb00b15d0 2024-09-10T22:00:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6c00640 1 -- 172.21.15.66:0/3697652911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb0165e30 con 0x7f1eb00b15d0 2024-09-10T22:00:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6c00640 1 -- 172.21.15.66:0/3697652911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb01663f0 con 0x7f1eb00b15d0 2024-09-10T22:00:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.218+0000 7f1eb6c00640 1 -- 172.21.15.66:0/3697652911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb00cd7a0 con 0x7f1eb00b15d0 2024-09-10T22:00:57.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.222+0000 7f1ead600640 1 -- 172.21.15.66:0/3697652911 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1ea4003c30 con 0x7f1eb00b15d0 2024-09-10T22:00:57.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.222+0000 7f1ead600640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1e900725e0 0x7f1e90074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:57.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.222+0000 7f1ead600640 1 -- 172.21.15.66:0/3697652911 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1ea4092a10 con 0x7f1eb00b15d0 2024-09-10T22:00:57.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.222+0000 7f1eb4e00640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1e900725e0 0x7f1e90074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:57.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.222+0000 7f1eb4e00640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1e900725e0 0x7f1e90074aa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0af000 tx=0x7f1e9c01b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:57.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.226+0000 7f1ead600640 1 -- 172.21.15.66:0/3697652911 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ea405e1a0 con 0x7f1eb00b15d0 2024-09-10T22:00:57.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.438+0000 7f1eb6c00640 1 -- 172.21.15.66:0/3697652911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1eb0072e00 con 0x7f1eb00b15d0 2024-09-10T22:00:57.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.438+0000 7f1ead600640 1 -- 172.21.15.66:0/3697652911 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f1ea405daa0 con 0x7f1eb00b15d0 2024-09-10T22:00:57.445 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:00:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.442+0000 7f1eb6c00640 1 -- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1e900725e0 msgr2=0x7f1e90074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.442+0000 7f1eb6c00640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1e900725e0 0x7f1e90074aa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0af000 tx=0x7f1e9c01b060 comp rx=0 tx=0).stop 2024-09-10T22:00:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.442+0000 7f1eb6c00640 1 -- 172.21.15.66:0/3697652911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb00b15d0 msgr2=0x7f1eb0163f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.442+0000 7f1eb6c00640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb00b15d0 0x7f1eb0163f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c500 tx=0x7f1ea400c9c0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.442+0000 7f1eb6c00640 1 -- 172.21.15.66:0/3697652911 shutdown_connections 2024-09-10T22:00:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.442+0000 7f1eb6c00640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1e900725e0 0x7f1e90074aa0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.442+0000 7f1eb6c00640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb015f300 0x7f1eb006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.442+0000 7f1eb6c00640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1eb015ef20 0x7f1eb015dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.442+0000 7f1eb6c00640 1 --2- 172.21.15.66:0/3697652911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb00b15d0 0x7f1eb0163f00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.442+0000 7f1eb6c00640 1 -- 172.21.15.66:0/3697652911 >> 172.21.15.66:0/3697652911 conn(0x7f1eb00d1b10 msgr2=0x7f1eb00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.442+0000 7f1eb6c00640 1 -- 172.21.15.66:0/3697652911 shutdown_connections 2024-09-10T22:00:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:57.442+0000 7f1eb6c00640 1 -- 172.21.15.66:0/3697652911 wait complete. 2024-09-10T22:00:57.467 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:00:57.467 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:00:57.467 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T22:00:57.467 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-10T22:00:57.467 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T22:00:57.467 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:00:57.467 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:00:57.467 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:00:57.468 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-10T22:00:57.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:57.468 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T22:00:57.477 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:00:57.477 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:00:57.477 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:00:57.477 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:57.477 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T22:00:57.477 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:57.477 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:00:57.477 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:57.477 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T22:00:57.477 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:00:57.477 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42390sec preferred_lft 42390sec 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:36:e5:45:f6 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:00:57.478 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T22:00:57.479 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:00:57.479 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:57.479 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T22:00:57.479 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:57.479 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:00:57.479 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:00:57.479 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T22:00:57.479 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:57.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:57.480 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:00:57.538 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T22:00:57.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:57.541 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T22:00:57.600 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T22:00:57.600 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:00:57.600 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:00:57.600 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:00:57.600 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:00:57.600 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:00:57.601 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:00:57.601 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:00:57.601 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:00:57.601 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:00:57.601 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:00:57.601 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:57.601 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:00:57.601 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:00:57.602 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:00:57.602 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:00:57.602 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:00:57.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:57.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:57.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:57.734 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T22:00:57.743 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T22:00:57.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:57.744 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-09-10T22:00:57.794 INFO:teuthology.orchestra.run.smithi094.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-10T22:00:57.795 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:57.795 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:00:57.796 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T22:00:57.796 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-10T22:00:57.796 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:00:57.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:57.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:57.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:57.922 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T22:00:57.942 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:00:57.943 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:00:57.943 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:00:57.943 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:57.943 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T22:00:57.943 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:57.943 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:00:57.943 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 4e:6b:ef:29:c7:56 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T22:00:57.943 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T22:00:57.943 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:57.943 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::4c6b:efff:fe29:c756/64 scope link 2024-09-10T22:00:57.943 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:57.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:57.945 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T22:00:58.012 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:00:58.012 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:00:58.012 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:00:58.012 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:58.012 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T22:00:58.012 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:58.012 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:00:58.012 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 4e:6b:ef:29:c7:56 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T22:00:58.012 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T22:00:58.013 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:58.013 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::4c6b:efff:fe29:c756/64 scope link 2024-09-10T22:00:58.013 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:00:58.014 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-10T22:00:58.014 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:58.014 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:00:58.014 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-10T22:00:58.014 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T22:00:58.014 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-10T22:00:58.014 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-10T22:00:58.015 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-10T22:00:58.015 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:00:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:58.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:58.195 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:00:58.195 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:00:58.196 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.1 up 2024-09-10T22:00:58.196 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T22:00:58.196 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:00:58.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:00:58.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:58.279 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-10T22:00:58.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:00:58.279 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-10T22:00:58.339 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-10T22:00:58.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:00:58.340 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-10T22:00:58.394 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T22:00:58.464 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:00:58.521 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T22:00:58.521 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T22:00:58.521 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T22:00:58.521 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T22:00:58.521 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T22:00:58.521 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T22:00:58.521 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T22:00:58.521 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:00:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T22:00:58.523 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T22:00:58.523 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T22:00:58.523 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T22:00:58.523 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T22:00:58.523 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T22:00:58.523 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T22:00:58.523 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T22:00:58.523 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:00:58.523 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:00:58.523 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-10T22:00:58.523 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T22:00:58.523 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T22:00:58.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:58.524 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T22:00:58.574 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T22:00:58.574 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-09-10T22:00:58.574 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-10T22:00:58.624 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T22:00:58.645 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:00:58.700 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T22:00:58.700 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T22:00:58.700 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T22:00:58.700 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T22:00:58.700 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T22:00:58.700 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T22:00:58.700 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T22:00:58.700 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T22:00:58.701 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T22:00:58.701 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T22:00:58.701 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T22:00:58.701 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T22:00:58.701 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:00:58.701 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T22:00:58.701 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T22:00:58.701 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T22:00:58.701 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T22:00:58.701 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T22:00:58.701 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T22:00:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T22:00:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:00:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T22:00:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T22:00:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T22:00:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T22:00:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T22:00:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T22:00:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T22:00:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T22:00:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:00:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:00:58.703 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-10T22:00:58.703 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T22:00:58.703 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T22:00:58.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:58.704 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T22:00:58.717 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi094.stderr:2024-09-10T22:00:58.713+0000 7f52f8648580 -1 init, newargv = 0x55e591164ef0 newargc=15 2024-09-10T22:00:58.717 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi094.stderr:ceph-fuse[33673]: starting ceph client 2024-09-10T22:00:58.724 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi094.stderr:ceph-fuse[33673]: starting fuse 2024-09-10T22:00:58.727 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T22:00:58.727 INFO:teuthology.orchestra.run.smithi094.stdout:45 2024-09-10T22:00:58.727 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2024-09-10T22:00:58.727 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:00:58.727 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T22:00:58.727 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T22:00:58.728 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T22:00:58.728 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T22:00:58.728 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T22:00:58.728 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T22:00:58.728 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-10T22:00:58.728 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T22:00:58.728 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-10T22:00:58.728 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T22:00:58.728 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T22:00:58.728 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T22:00:58.729 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T22:00:58.729 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T22:00:58.729 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T22:00:58.729 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T22:00:58.729 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:00:58.729 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:00:58.729 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:00:58.729 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T22:00:58.729 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T22:00:58.729 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T22:00:58.729 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:00:58.730 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T22:00:58.730 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.3")) 2024-09-10T22:00:58.730 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T22:00:58.849 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.3.33673.asok 2024-09-10T22:00:58.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:00:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:00:58.861 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33673.asok 2024-09-10T22:00:58.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:58.861 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33673.asok status 2024-09-10T22:00:59.036 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "3", 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "33673", 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4556, 2024-09-10T22:00:59.037 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 4556 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.2:0", 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 109906750 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.2:0", 2024-09-10T22:00:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 109906750 2024-09-10T22:00:59.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T22:00:59.039 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.4556 192.168.0.2:0/109906750", 2024-09-10T22:00:59.039 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.2:0/109906750", 2024-09-10T22:00:59.039 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T22:00:59.039 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 8, 2024-09-10T22:00:59.039 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 22, 2024-09-10T22:00:59.039 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:00:59.039 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T22:00:59.039 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T22:00:59.039 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T22:00:59.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:59.052 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-09-10T22:00:59.052 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:00:59.063 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-09-10T22:00:59.063 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T22:00:59.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:59.063 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:00:59.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:59.133 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-09-10T22:00:59.133 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-10T22:00:59.186 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-09-10T22:00:59.187 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-10T22:00:59.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:59.187 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-10T22:00:59.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:59.254 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T22:00:59.254 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T22:00:59.263 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T22:00:59.263 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T22:00:59.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:59.264 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T22:00:59.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:59.324 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T22:00:59.324 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-10T22:00:59.377 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T22:00:59.377 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-10T22:00:59.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:00:59.377 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-10T22:00:59.444 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-10T22:00:59.547 INFO:tasks.cephfs_test_runner:Starting test: test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Initiate cloning where clone subvolume and source subvolume are located 2024-09-10T22:00:59.548 INFO:tasks.cephfs_test_runner:test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-10T22:00:59.548 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:00:59.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.702+0000 7f3ddae00640 1 -- 172.21.15.66:0/530958915 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3dd40c4ef0 msgr2=0x7f3dd40c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:59.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.702+0000 7f3ddae00640 1 --2- 172.21.15.66:0/530958915 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3dd40c4ef0 0x7f3dd40c5360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009bc0 tx=0x7f3dcc01cba0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.702+0000 7f3ddae00640 1 -- 172.21.15.66:0/530958915 shutdown_connections 2024-09-10T22:00:59.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.702+0000 7f3ddae00640 1 --2- 172.21.15.66:0/530958915 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3dd40c4ef0 0x7f3dd40c5360 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.702+0000 7f3ddae00640 1 --2- 172.21.15.66:0/530958915 >> v2:172.21.15.94:3300/0 conn(0x7f3dd40c4b10 0x7f3dd406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.702+0000 7f3ddae00640 1 -- 172.21.15.66:0/530958915 >> 172.21.15.66:0/530958915 conn(0x7f3dd40cfa30 msgr2=0x7f3dd40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:59.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.702+0000 7f3ddae00640 1 -- 172.21.15.66:0/530958915 shutdown_connections 2024-09-10T22:00:59.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.702+0000 7f3ddae00640 1 -- 172.21.15.66:0/530958915 wait complete. 2024-09-10T22:00:59.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.702+0000 7f3ddae00640 1 Processor -- start 2024-09-10T22:00:59.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.702+0000 7f3ddae00640 1 -- start start 2024-09-10T22:00:59.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.702+0000 7f3ddae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dd40ae620 0x7f3dd41549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:59.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.702+0000 7f3ddae00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3dd40c4b10 0x7f3dd41572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:59.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3ddae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dd40c4ef0 0x7f3dd41521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:59.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3ddae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4167bd0 con 0x7f3dd40ae620 2024-09-10T22:00:59.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3ddae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4167a50 con 0x7f3dd40c4ef0 2024-09-10T22:00:59.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3ddae00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd4167d50 con 0x7f3dd40c4b10 2024-09-10T22:00:59.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dd3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dd40ae620 0x7f3dd41549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dcbe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3dd40c4b10 0x7f3dd41572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dd8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dd40c4ef0 0x7f3dd41521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dd3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dd40ae620 0x7f3dd41549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57604/0 (socket says 172.21.15.66:57604) 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dd3e00640 1 -- 172.21.15.66:0/2712000522 learned_addr learned my addr 172.21.15.66:0/2712000522 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dcbe00640 1 -- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dd40c4ef0 msgr2=0x7f3dd41521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dcbe00640 1 --2- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dd40c4ef0 0x7f3dd41521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dcbe00640 1 -- 172.21.15.66:0/2712000522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dd40ae620 msgr2=0x7f3dd41549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dcbe00640 1 --2- 172.21.15.66:0/2712000522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dd40ae620 0x7f3dd41549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dcbe00640 1 -- 172.21.15.66:0/2712000522 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd4155060 con 0x7f3dd40c4b10 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dcbe00640 1 --2- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3dd40c4b10 0x7f3dd41572d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c440 tx=0x7f3dbc00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dd0c00640 1 -- 172.21.15.66:0/2712000522 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc017070 con 0x7f3dd40c4b10 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dd0c00640 1 -- 172.21.15.66:0/2712000522 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dbc003600 con 0x7f3dd40c4b10 2024-09-10T22:00:59.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dd0c00640 1 -- 172.21.15.66:0/2712000522 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc003a50 con 0x7f3dd40c4b10 2024-09-10T22:00:59.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3ddae00640 1 -- 172.21.15.66:0/2712000522 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd41567f0 con 0x7f3dd40c4b10 2024-09-10T22:00:59.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3ddae00640 1 -- 172.21.15.66:0/2712000522 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd4156db0 con 0x7f3dd40c4b10 2024-09-10T22:00:59.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3ddae00640 1 -- 172.21.15.66:0/2712000522 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd41565e0 con 0x7f3dd40c4b10 2024-09-10T22:00:59.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.706+0000 7f3dd0c00640 1 -- 172.21.15.66:0/2712000522 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3dbc003bf0 con 0x7f3dd40c4b10 2024-09-10T22:00:59.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.710+0000 7f3dd0c00640 1 --2- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3db00725e0 0x7f3db0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:59.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.710+0000 7f3dd0c00640 1 -- 172.21.15.66:0/2712000522 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3dbc092770 con 0x7f3dd40c4b10 2024-09-10T22:00:59.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.710+0000 7f3dd3e00640 1 --2- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3db00725e0 0x7f3db0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:59.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.710+0000 7f3dd3e00640 1 --2- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3db00725e0 0x7f3db0074aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009ca0 tx=0x7f3dc40091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:59.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.714+0000 7f3dd0c00640 1 -- 172.21.15.66:0/2712000522 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3dbc007e20 con 0x7f3dd40c4b10 2024-09-10T22:00:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.890+0000 7f3ddae00640 1 -- 172.21.15.66:0/2712000522 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3dd4159690 con 0x7f3dd40c4b10 2024-09-10T22:00:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.890+0000 7f3dd0c00640 1 -- 172.21.15.66:0/2712000522 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16593 (secure 0 0 0) 0x7f3dbc089e20 con 0x7f3dd40c4b10 2024-09-10T22:00:59.895 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:00:59.895 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":22,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:34.173529+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:30.545203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:31.538729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.894+0000 7f3ddae00640 1 -- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3db00725e0 msgr2=0x7f3db0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.894+0000 7f3ddae00640 1 --2- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3db00725e0 0x7f3db0074aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009ca0 tx=0x7f3dc40091e0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.894+0000 7f3ddae00640 1 -- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3dd40c4b10 msgr2=0x7f3dd41572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.894+0000 7f3ddae00640 1 --2- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3dd40c4b10 0x7f3dd41572d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c440 tx=0x7f3dbc00c900 comp rx=0 tx=0).stop 2024-09-10T22:00:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.894+0000 7f3ddae00640 1 -- 172.21.15.66:0/2712000522 shutdown_connections 2024-09-10T22:00:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.894+0000 7f3ddae00640 1 --2- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3db00725e0 0x7f3db0074aa0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.894+0000 7f3ddae00640 1 --2- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dd40c4ef0 0x7f3dd41521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.894+0000 7f3ddae00640 1 --2- 172.21.15.66:0/2712000522 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3dd40c4b10 0x7f3dd41572d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.894+0000 7f3ddae00640 1 --2- 172.21.15.66:0/2712000522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dd40ae620 0x7f3dd41549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.894+0000 7f3ddae00640 1 -- 172.21.15.66:0/2712000522 >> 172.21.15.66:0/2712000522 conn(0x7f3dd40cfa30 msgr2=0x7f3dd40af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.894+0000 7f3ddae00640 1 -- 172.21.15.66:0/2712000522 shutdown_connections 2024-09-10T22:00:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:00:59.894+0000 7f3ddae00640 1 -- 172.21.15.66:0/2712000522 wait complete. 2024-09-10T22:00:59.917 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:00:22.340318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T22:00:30.545203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T22:00:31.538729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:00:59.917 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:01:00.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.062+0000 7f9566a00640 1 -- 172.21.15.66:0/434751861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f956015bd40 msgr2=0x7f95600cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.062+0000 7f9566a00640 1 --2- 172.21.15.66:0/434751861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f956015bd40 0x7f95600cffa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9550009840 tx=0x7f955001c790 comp rx=0 tx=0).stop 2024-09-10T22:01:00.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.062+0000 7f9566a00640 1 -- 172.21.15.66:0/434751861 shutdown_connections 2024-09-10T22:01:00.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.062+0000 7f9566a00640 1 --2- 172.21.15.66:0/434751861 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95600cf830 0x7f95600d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.062+0000 7f9566a00640 1 --2- 172.21.15.66:0/434751861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f956015bd40 0x7f95600cffa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.062+0000 7f9566a00640 1 -- 172.21.15.66:0/434751861 >> 172.21.15.66:0/434751861 conn(0x7f95600d1c60 msgr2=0x7f95600d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:00.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.062+0000 7f9566a00640 1 -- 172.21.15.66:0/434751861 shutdown_connections 2024-09-10T22:01:00.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.062+0000 7f9566a00640 1 -- 172.21.15.66:0/434751861 wait complete. 2024-09-10T22:01:00.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9566a00640 1 Processor -- start 2024-09-10T22:01:00.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9566a00640 1 -- start start 2024-09-10T22:01:00.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9566a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95600aec10 0x7f9560164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:00.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9566a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95600cf830 0x7f9560165210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:00.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9566a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f956015bd40 0x7f95601fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:00.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9566a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9560169070 con 0x7f95600cf830 2024-09-10T22:01:00.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9566a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9560168ef0 con 0x7f956015bd40 2024-09-10T22:01:00.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9566a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f95601691f0 con 0x7f95600aec10 2024-09-10T22:01:00.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9557e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95600cf830 0x7f9560165210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:00.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f955fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95600aec10 0x7f9560164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:00.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9557e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95600cf830 0x7f9560165210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57624/0 (socket says 172.21.15.66:57624) 2024-09-10T22:01:00.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9557e00640 1 -- 172.21.15.66:0/440980196 learned_addr learned my addr 172.21.15.66:0/440980196 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:00.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f955fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95600aec10 0x7f9560164cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:46442/0 (socket says 172.21.15.66:46442) 2024-09-10T22:01:00.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9564a00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f956015bd40 0x7f95601fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:00.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f955fe00640 1 -- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f956015bd40 msgr2=0x7f95601fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f955fe00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f956015bd40 0x7f95601fb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f955fe00640 1 -- 172.21.15.66:0/440980196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95600cf830 msgr2=0x7f9560165210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f955fe00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95600cf830 0x7f9560165210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f955fe00640 1 -- 172.21.15.66:0/440980196 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9560163c70 con 0x7f95600aec10 2024-09-10T22:01:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f955fe00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95600aec10 0x7f9560164cd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9560073b60 tx=0x7f9550005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f955cc00640 1 -- 172.21.15.66:0/440980196 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500bb070 con 0x7f95600aec10 2024-09-10T22:01:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f955cc00640 1 -- 172.21.15.66:0/440980196 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95500053e0 con 0x7f95600aec10 2024-09-10T22:01:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9566a00640 1 -- 172.21.15.66:0/440980196 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9560163f50 con 0x7f95600aec10 2024-09-10T22:01:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f955cc00640 1 -- 172.21.15.66:0/440980196 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500b2040 con 0x7f95600aec10 2024-09-10T22:01:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.066+0000 7f9566a00640 1 -- 172.21.15.66:0/440980196 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9560164510 con 0x7f95600aec10 2024-09-10T22:01:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.070+0000 7f9566a00640 1 -- 172.21.15.66:0/440980196 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95600af010 con 0x7f95600aec10 2024-09-10T22:01:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.070+0000 7f955cc00640 1 -- 172.21.15.66:0/440980196 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f95500b5030 con 0x7f95600aec10 2024-09-10T22:01:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.070+0000 7f955cc00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f953c072610 0x7f953c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.070+0000 7f955cc00640 1 -- 172.21.15.66:0/440980196 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f955012c700 con 0x7f95600aec10 2024-09-10T22:01:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.070+0000 7f9557e00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f953c072610 0x7f953c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.070+0000 7f9557e00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f953c072610 0x7f953c074ad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95600b44e0 tx=0x7f95480091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:00.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.074+0000 7f955cc00640 1 -- 172.21.15.66:0/440980196 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9550016410 con 0x7f95600aec10 2024-09-10T22:01:00.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.250+0000 7f9566a00640 1 -- 172.21.15.66:0/440980196 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f956015f220 con 0x7f95600aec10 2024-09-10T22:01:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.250+0000 7f955cc00640 1 -- 172.21.15.66:0/440980196 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f95500f7780 con 0x7f95600aec10 2024-09-10T22:01:00.255 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-09-10T22:01:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.250+0000 7f9566a00640 1 -- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f953c072610 msgr2=0x7f953c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.250+0000 7f9566a00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f953c072610 0x7f953c074ad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95600b44e0 tx=0x7f95480091e0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.250+0000 7f9566a00640 1 -- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95600aec10 msgr2=0x7f9560164cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.250+0000 7f9566a00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95600aec10 0x7f9560164cd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9560073b60 tx=0x7f9550005db0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.250+0000 7f9566a00640 1 -- 172.21.15.66:0/440980196 shutdown_connections 2024-09-10T22:01:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.250+0000 7f9566a00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f953c072610 0x7f953c074ad0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.250+0000 7f9566a00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f956015bd40 0x7f95601fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.250+0000 7f9566a00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95600cf830 0x7f9560165210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.250+0000 7f9566a00640 1 --2- 172.21.15.66:0/440980196 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95600aec10 0x7f9560164cd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.250+0000 7f9566a00640 1 -- 172.21.15.66:0/440980196 >> 172.21.15.66:0/440980196 conn(0x7f95600d1c60 msgr2=0x7f95600b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:00.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.254+0000 7f9566a00640 1 -- 172.21.15.66:0/440980196 shutdown_connections 2024-09-10T22:01:00.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.254+0000 7f9566a00640 1 -- 172.21.15.66:0/440980196 wait complete. 2024-09-10T22:01:00.276 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T22:01:00.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.418+0000 7f8556a00640 1 -- 172.21.15.66:0/287269052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f855015bd40 msgr2=0x7f85500cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.418+0000 7f8556a00640 1 --2- 172.21.15.66:0/287269052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f855015bd40 0x7f85500cffa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f85500cc2e0 tx=0x7f854001c830 comp rx=0 tx=0).stop 2024-09-10T22:01:00.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.418+0000 7f8556a00640 1 -- 172.21.15.66:0/287269052 shutdown_connections 2024-09-10T22:01:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.418+0000 7f8556a00640 1 --2- 172.21.15.66:0/287269052 >> v2:172.21.15.94:3300/0 conn(0x7f85500cf180 0x7f85500d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.418+0000 7f8556a00640 1 --2- 172.21.15.66:0/287269052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f855015bd40 0x7f85500cffa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.418+0000 7f8556a00640 1 -- 172.21.15.66:0/287269052 >> 172.21.15.66:0/287269052 conn(0x7f85500d1c60 msgr2=0x7f85500d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8556a00640 1 -- 172.21.15.66:0/287269052 shutdown_connections 2024-09-10T22:01:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8556a00640 1 -- 172.21.15.66:0/287269052 wait complete. 2024-09-10T22:01:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8556a00640 1 Processor -- start 2024-09-10T22:01:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8556a00640 1 -- start start 2024-09-10T22:01:00.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8556a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500cf180 0x7f855015def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:00.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8556a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85500cf830 0x7f85501f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:00.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8556a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f855015bd40 0x7f85500cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:00.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8556a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8550168ba0 con 0x7f85500cf830 2024-09-10T22:01:00.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8556a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8550168a20 con 0x7f85500cf180 2024-09-10T22:01:00.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8556a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8550168d20 con 0x7f855015bd40 2024-09-10T22:01:00.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8547400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85500cf830 0x7f85501f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:00.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8547400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85500cf830 0x7f85501f6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57638/0 (socket says 172.21.15.66:57638) 2024-09-10T22:01:00.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8547400640 1 -- 172.21.15.66:0/3359129576 learned_addr learned my addr 172.21.15.66:0/3359129576 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:00.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f854fe00640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500cf180 0x7f855015def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:00.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8554a00640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f855015bd40 0x7f85500cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:00.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8547400640 1 -- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f855015bd40 msgr2=0x7f85500cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8547400640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f855015bd40 0x7f85500cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8547400640 1 -- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500cf180 msgr2=0x7f855015def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8547400640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500cf180 0x7f855015def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8547400640 1 -- 172.21.15.66:0/3359129576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855015d2d0 con 0x7f85500cf830 2024-09-10T22:01:00.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8554a00640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f855015bd40 0x7f85500cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:00.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.422+0000 7f8547400640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85500cf830 0x7f85501f6290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f853800c500 tx=0x7f853800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:00.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.426+0000 7f854cc00640 1 -- 172.21.15.66:0/3359129576 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538017070 con 0x7f85500cf830 2024-09-10T22:01:00.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.426+0000 7f854cc00640 1 -- 172.21.15.66:0/3359129576 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85380053c0 con 0x7f85500cf830 2024-09-10T22:01:00.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.426+0000 7f854cc00640 1 -- 172.21.15.66:0/3359129576 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538012890 con 0x7f85500cf830 2024-09-10T22:01:00.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.426+0000 7f8556a00640 1 -- 172.21.15.66:0/3359129576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855015c510 con 0x7f85500cf830 2024-09-10T22:01:00.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.426+0000 7f8556a00640 1 -- 172.21.15.66:0/3359129576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855015cad0 con 0x7f85500cf830 2024-09-10T22:01:00.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.426+0000 7f8556a00640 1 -- 172.21.15.66:0/3359129576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85500aec10 con 0x7f85500cf830 2024-09-10T22:01:00.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.426+0000 7f854cc00640 1 -- 172.21.15.66:0/3359129576 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8538003600 con 0x7f85500cf830 2024-09-10T22:01:00.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.426+0000 7f854cc00640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f852c0725e0 0x7f852c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:00.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.426+0000 7f854cc00640 1 -- 172.21.15.66:0/3359129576 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8538092800 con 0x7f85500cf830 2024-09-10T22:01:00.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.426+0000 7f854fe00640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f852c0725e0 0x7f852c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:00.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.430+0000 7f854fe00640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f852c0725e0 0x7f852c074aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f85400af000 tx=0x7f85400a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.430+0000 7f854cc00640 1 -- 172.21.15.66:0/3359129576 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f853805d800 con 0x7f85500cf830 2024-09-10T22:01:00.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.602+0000 7f8556a00640 1 -- 172.21.15.66:0/3359129576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f85501558d0 con 0x7f85500cf830 2024-09-10T22:01:00.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.602+0000 7f854cc00640 1 -- 172.21.15.66:0/3359129576 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f8538062230 con 0x7f85500cf830 2024-09-10T22:01:00.608 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-09-10T22:01:00.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.606+0000 7f8556a00640 1 -- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f852c0725e0 msgr2=0x7f852c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.606+0000 7f8556a00640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f852c0725e0 0x7f852c074aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f85400af000 tx=0x7f85400a7040 comp rx=0 tx=0).stop 2024-09-10T22:01:00.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.606+0000 7f8556a00640 1 -- 172.21.15.66:0/3359129576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85500cf830 msgr2=0x7f85501f6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.606+0000 7f8556a00640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85500cf830 0x7f85501f6290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f853800c500 tx=0x7f853800c9c0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.606+0000 7f8556a00640 1 -- 172.21.15.66:0/3359129576 shutdown_connections 2024-09-10T22:01:00.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.606+0000 7f8556a00640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f852c0725e0 0x7f852c074aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.606+0000 7f8556a00640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f855015bd40 0x7f85500cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.606+0000 7f8556a00640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85500cf830 0x7f85501f6290 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.606+0000 7f8556a00640 1 --2- 172.21.15.66:0/3359129576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500cf180 0x7f855015def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.606+0000 7f8556a00640 1 -- 172.21.15.66:0/3359129576 >> 172.21.15.66:0/3359129576 conn(0x7f85500d1c60 msgr2=0x7f85500b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:00.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.606+0000 7f8556a00640 1 -- 172.21.15.66:0/3359129576 shutdown_connections 2024-09-10T22:01:00.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.606+0000 7f8556a00640 1 -- 172.21.15.66:0/3359129576 wait complete. 2024-09-10T22:01:00.630 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T22:01:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.774+0000 7f4e7a400640 1 -- 172.21.15.66:0/2638052230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e740d2770 msgr2=0x7f4e7415fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.774+0000 7f4e7a400640 1 --2- 172.21.15.66:0/2638052230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e740d2770 0x7f4e7415fac0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4e740d1c20 tx=0x7f4e6c01ca30 comp rx=0 tx=0).stop 2024-09-10T22:01:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.774+0000 7f4e7a400640 1 -- 172.21.15.66:0/2638052230 shutdown_connections 2024-09-10T22:01:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.774+0000 7f4e7a400640 1 --2- 172.21.15.66:0/2638052230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e740d2770 0x7f4e7415fac0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.774+0000 7f4e7a400640 1 --2- 172.21.15.66:0/2638052230 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e740b2bf0 0x7f4e740d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.774+0000 7f4e7a400640 1 --2- 172.21.15.66:0/2638052230 >> v2:172.21.15.94:3300/0 conn(0x7f4e7415f340 0x7f4e740cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.774+0000 7f4e7a400640 1 -- 172.21.15.66:0/2638052230 >> 172.21.15.66:0/2638052230 conn(0x7f4e740d1c50 msgr2=0x7f4e740d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.774+0000 7f4e7a400640 1 -- 172.21.15.66:0/2638052230 shutdown_connections 2024-09-10T22:01:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.774+0000 7f4e7a400640 1 -- 172.21.15.66:0/2638052230 wait complete. 2024-09-10T22:01:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e7a400640 1 Processor -- start 2024-09-10T22:01:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e7a400640 1 -- start start 2024-09-10T22:01:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e7a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e740b2bf0 0x7f4e7414d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e7a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e740d2770 0x7f4e7414c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e7a400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e7415f340 0x7f4e74156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e7a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4e740aecf0 con 0x7f4e740b2bf0 2024-09-10T22:01:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e7a400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4e740aeb70 con 0x7f4e740d2770 2024-09-10T22:01:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e7a400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4e740aee70 con 0x7f4e7415f340 2024-09-10T22:01:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e73400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e740b2bf0 0x7f4e7414d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e73400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e740b2bf0 0x7f4e7414d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57660/0 (socket says 172.21.15.66:57660) 2024-09-10T22:01:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e73400640 1 -- 172.21.15.66:0/1370097560 learned_addr learned my addr 172.21.15.66:0/1370097560 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e73400640 1 -- 172.21.15.66:0/1370097560 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e7415f340 msgr2=0x7f4e74156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e73400640 1 --2- 172.21.15.66:0/1370097560 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e7415f340 0x7f4e74156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e73400640 1 -- 172.21.15.66:0/1370097560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e740d2770 msgr2=0x7f4e7414c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e73400640 1 --2- 172.21.15.66:0/1370097560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e740d2770 0x7f4e7414c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e73400640 1 -- 172.21.15.66:0/1370097560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e74165b70 con 0x7f4e740b2bf0 2024-09-10T22:01:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e73400640 1 --2- 172.21.15.66:0/1370097560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e740b2bf0 0x7f4e7414d230 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400e7d0 tx=0x7f4e6400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e6b400640 1 -- 172.21.15.66:0/1370097560 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6400ccc0 con 0x7f4e740b2bf0 2024-09-10T22:01:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e6b400640 1 -- 172.21.15.66:0/1370097560 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e64010430 con 0x7f4e740b2bf0 2024-09-10T22:01:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e7a400640 1 -- 172.21.15.66:0/1370097560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e74165e50 con 0x7f4e740b2bf0 2024-09-10T22:01:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e6b400640 1 -- 172.21.15.66:0/1370097560 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64010860 con 0x7f4e740b2bf0 2024-09-10T22:01:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e7a400640 1 -- 172.21.15.66:0/1370097560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7406e010 con 0x7f4e740b2bf0 2024-09-10T22:01:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.778+0000 7f4e7a400640 1 -- 172.21.15.66:0/1370097560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7415fac0 con 0x7f4e740b2bf0 2024-09-10T22:01:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.782+0000 7f4e6b400640 1 -- 172.21.15.66:0/1370097560 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4e64010a00 con 0x7f4e740b2bf0 2024-09-10T22:01:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.782+0000 7f4e6b400640 1 --2- 172.21.15.66:0/1370097560 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4e500725e0 0x7f4e50074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.782+0000 7f4e6b400640 1 -- 172.21.15.66:0/1370097560 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4e64093ca0 con 0x7f4e740b2bf0 2024-09-10T22:01:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.782+0000 7f4e6be00640 1 --2- 172.21.15.66:0/1370097560 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4e500725e0 0x7f4e50074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.782+0000 7f4e6be00640 1 --2- 172.21.15.66:0/1370097560 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4e500725e0 0x7f4e50074aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0058d0 tx=0x7f4e5c005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.786+0000 7f4e6b400640 1 -- 172.21.15.66:0/1370097560 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4e6405f2e0 con 0x7f4e740b2bf0 2024-09-10T22:01:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.958+0000 7f4e7a400640 1 -- 172.21.15.66:0/1370097560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4e740cc060 con 0x7f4e740b2bf0 2024-09-10T22:01:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.958+0000 7f4e6b400640 1 -- 172.21.15.66:0/1370097560 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f4e6405ebe0 con 0x7f4e740b2bf0 2024-09-10T22:01:00.964 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-09-10T22:01:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.962+0000 7f4e7a400640 1 -- 172.21.15.66:0/1370097560 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4e500725e0 msgr2=0x7f4e50074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.962+0000 7f4e7a400640 1 --2- 172.21.15.66:0/1370097560 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4e500725e0 0x7f4e50074aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0058d0 tx=0x7f4e5c005860 comp rx=0 tx=0).stop 2024-09-10T22:01:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.962+0000 7f4e7a400640 1 -- 172.21.15.66:0/1370097560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e740b2bf0 msgr2=0x7f4e7414d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.962+0000 7f4e7a400640 1 --2- 172.21.15.66:0/1370097560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e740b2bf0 0x7f4e7414d230 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400e7d0 tx=0x7f4e6400ec90 comp rx=0 tx=0).stop 2024-09-10T22:01:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.962+0000 7f4e7a400640 1 -- 172.21.15.66:0/1370097560 shutdown_connections 2024-09-10T22:01:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.962+0000 7f4e7a400640 1 --2- 172.21.15.66:0/1370097560 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4e500725e0 0x7f4e50074aa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.962+0000 7f4e7a400640 1 --2- 172.21.15.66:0/1370097560 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e7415f340 0x7f4e74156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.962+0000 7f4e7a400640 1 --2- 172.21.15.66:0/1370097560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e740d2770 0x7f4e7414c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.962+0000 7f4e7a400640 1 --2- 172.21.15.66:0/1370097560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e740b2bf0 0x7f4e7414d230 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.962+0000 7f4e7a400640 1 -- 172.21.15.66:0/1370097560 >> 172.21.15.66:0/1370097560 conn(0x7f4e740d1c50 msgr2=0x7f4e7415c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.962+0000 7f4e7a400640 1 -- 172.21.15.66:0/1370097560 shutdown_connections 2024-09-10T22:01:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:00.962+0000 7f4e7a400640 1 -- 172.21.15.66:0/1370097560 wait complete. 2024-09-10T22:01:00.984 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-10T22:01:01.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.126+0000 7f5465e00640 1 -- 172.21.15.66:0/3788763614 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54600cea30 msgr2=0x7f54600ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.126+0000 7f5465e00640 1 --2- 172.21.15.66:0/3788763614 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54600cea30 0x7f54600ceff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5458009bf0 tx=0x7f545801cbe0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.126+0000 7f5465e00640 1 -- 172.21.15.66:0/3788763614 shutdown_connections 2024-09-10T22:01:01.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.126+0000 7f5465e00640 1 --2- 172.21.15.66:0/3788763614 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54600cea30 0x7f54600ceff0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.126+0000 7f5465e00640 1 -- 172.21.15.66:0/3788763614 >> 172.21.15.66:0/3788763614 conn(0x7f54600b26c0 msgr2=0x7f54600b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:01.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.126+0000 7f5465e00640 1 -- 172.21.15.66:0/3788763614 shutdown_connections 2024-09-10T22:01:01.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.126+0000 7f5465e00640 1 -- 172.21.15.66:0/3788763614 wait complete. 2024-09-10T22:01:01.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5465e00640 1 Processor -- start 2024-09-10T22:01:01.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5465e00640 1 -- start start 2024-09-10T22:01:01.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5465e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54600cea30 0x7f5460152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5465e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f546015a8a0 0x7f5460164db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5465e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f546015ac80 0x7f546015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5465e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5460168e40 con 0x7f546015ac80 2024-09-10T22:01:01.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5465e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5460168cc0 con 0x7f54600cea30 2024-09-10T22:01:01.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5465e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5460168fc0 con 0x7f546015a8a0 2024-09-10T22:01:01.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f545f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f546015ac80 0x7f546015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f545f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f546015ac80 0x7f546015cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57672/0 (socket says 172.21.15.66:57672) 2024-09-10T22:01:01.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f545f400640 1 -- 172.21.15.66:0/1596990323 learned_addr learned my addr 172.21.15.66:0/1596990323 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:01.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f545ea00640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54600cea30 0x7f5460152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f545e000640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f546015a8a0 0x7f5460164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f545f400640 1 -- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f546015a8a0 msgr2=0x7f5460164db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f545f400640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f546015a8a0 0x7f5460164db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f545f400640 1 -- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54600cea30 msgr2=0x7f5460152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f545f400640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54600cea30 0x7f5460152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f545f400640 1 -- 172.21.15.66:0/1596990323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54601528a0 con 0x7f546015ac80 2024-09-10T22:01:01.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f545f400640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f546015ac80 0x7f546015cf90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f545801a810 tx=0x7f5458009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:01.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5457400640 1 -- 172.21.15.66:0/1596990323 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54580bb070 con 0x7f546015ac80 2024-09-10T22:01:01.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5457400640 1 -- 172.21.15.66:0/1596990323 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5458004d40 con 0x7f546015ac80 2024-09-10T22:01:01.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5457400640 1 -- 172.21.15.66:0/1596990323 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54580b2040 con 0x7f546015ac80 2024-09-10T22:01:01.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5465e00640 1 -- 172.21.15.66:0/1596990323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5460163d10 con 0x7f546015ac80 2024-09-10T22:01:01.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5465e00640 1 -- 172.21.15.66:0/1596990323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54601642d0 con 0x7f546015ac80 2024-09-10T22:01:01.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.130+0000 7f5465e00640 1 -- 172.21.15.66:0/1596990323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54600ceff0 con 0x7f546015ac80 2024-09-10T22:01:01.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.134+0000 7f5457400640 1 -- 172.21.15.66:0/1596990323 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f54580b5030 con 0x7f546015ac80 2024-09-10T22:01:01.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.134+0000 7f5457400640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f54400725e0 0x7f5440074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.134+0000 7f5457400640 1 -- 172.21.15.66:0/1596990323 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f545812caf0 con 0x7f546015ac80 2024-09-10T22:01:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.134+0000 7f545ea00640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f54400725e0 0x7f5440074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.134+0000 7f545ea00640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f54400725e0 0x7f5440074aa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5450007a80 tx=0x7f5450009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:01.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.138+0000 7f5457400640 1 -- 172.21.15.66:0/1596990323 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f54580167d0 con 0x7f546015ac80 2024-09-10T22:01:01.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.374+0000 7f5465e00640 1 -- 172.21.15.66:0/1596990323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7f5460075bb0 con 0x7f546015ac80 2024-09-10T22:01:01.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5457400640 1 -- 172.21.15.66:0/1596990323 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v49) ==== 187+0+0 (secure 0 0 0) 0x7f54580f81c0 con 0x7f546015ac80 2024-09-10T22:01:01.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5465e00640 1 -- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f54400725e0 msgr2=0x7f5440074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5465e00640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f54400725e0 0x7f5440074aa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5450007a80 tx=0x7f5450009040 comp rx=0 tx=0).stop 2024-09-10T22:01:01.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5465e00640 1 -- 172.21.15.66:0/1596990323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f546015ac80 msgr2=0x7f546015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5465e00640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f546015ac80 0x7f546015cf90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f545801a810 tx=0x7f5458009200 comp rx=0 tx=0).stop 2024-09-10T22:01:01.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5465e00640 1 -- 172.21.15.66:0/1596990323 shutdown_connections 2024-09-10T22:01:01.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5465e00640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f54400725e0 0x7f5440074aa0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5465e00640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f546015ac80 0x7f546015cf90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5465e00640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f546015a8a0 0x7f5460164db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5465e00640 1 --2- 172.21.15.66:0/1596990323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54600cea30 0x7f5460152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5465e00640 1 -- 172.21.15.66:0/1596990323 >> 172.21.15.66:0/1596990323 conn(0x7f54600b26c0 msgr2=0x7f54601539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:01.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5465e00640 1 -- 172.21.15.66:0/1596990323 shutdown_connections 2024-09-10T22:01:01.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.446+0000 7f5465e00640 1 -- 172.21.15.66:0/1596990323 wait complete. 2024-09-10T22:01:01.471 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T22:01:01.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 -- 172.21.15.66:0/3232066077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d4415f300 msgr2=0x7f3d440b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 --2- 172.21.15.66:0/3232066077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d4415f300 0x7f3d440b1090 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009900 tx=0x7f3d2c01c8b0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 -- 172.21.15.66:0/3232066077 shutdown_connections 2024-09-10T22:01:01.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 --2- 172.21.15.66:0/3232066077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d4415f300 0x7f3d440b1090 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 --2- 172.21.15.66:0/3232066077 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d4415ef20 0x7f3d440cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 -- 172.21.15.66:0/3232066077 >> 172.21.15.66:0/3232066077 conn(0x7f3d440d1b10 msgr2=0x7f3d440d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:01.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 -- 172.21.15.66:0/3232066077 shutdown_connections 2024-09-10T22:01:01.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 -- 172.21.15.66:0/3232066077 wait complete. 2024-09-10T22:01:01.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 Processor -- start 2024-09-10T22:01:01.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 -- start start 2024-09-10T22:01:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d440b15d0 0x7f3d440b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d4415ef20 0x7f3d44071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d4415f300 0x7f3d4414dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3d44168ee0 con 0x7f3d4415f300 2024-09-10T22:01:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d44168d60 con 0x7f3d4415ef20 2024-09-10T22:01:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d49800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3d44169060 con 0x7f3d440b15d0 2024-09-10T22:01:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d48e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d4415f300 0x7f3d4414dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d48e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d4415f300 0x7f3d4414dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57698/0 (socket says 172.21.15.66:57698) 2024-09-10T22:01:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d48e00640 1 -- 172.21.15.66:0/2577852491 learned_addr learned my addr 172.21.15.66:0/2577852491 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d43e00640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d440b15d0 0x7f3d440b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.614+0000 7f3d3be00640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d4415ef20 0x7f3d44071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d48e00640 1 -- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d440b15d0 msgr2=0x7f3d440b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d48e00640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d440b15d0 0x7f3d440b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d48e00640 1 -- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d4415ef20 msgr2=0x7f3d44071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d48e00640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d4415ef20 0x7f3d44071990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d48e00640 1 -- 172.21.15.66:0/2577852491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d440b5ff0 con 0x7f3d4415f300 2024-09-10T22:01:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d48e00640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d4415f300 0x7f3d4414dbd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00c700 tx=0x7f3d3c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d40c00640 1 -- 172.21.15.66:0/2577852491 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c003db0 con 0x7f3d4415f300 2024-09-10T22:01:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d40c00640 1 -- 172.21.15.66:0/2577852491 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d3c00ce10 con 0x7f3d4415f300 2024-09-10T22:01:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d40c00640 1 -- 172.21.15.66:0/2577852491 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0052b0 con 0x7f3d4415f300 2024-09-10T22:01:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d49800640 1 -- 172.21.15.66:0/2577852491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d440b4cc0 con 0x7f3d4415f300 2024-09-10T22:01:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d49800640 1 -- 172.21.15.66:0/2577852491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d440b5280 con 0x7f3d4415f300 2024-09-10T22:01:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d49800640 1 -- 172.21.15.66:0/2577852491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d440cecc0 con 0x7f3d4415f300 2024-09-10T22:01:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.618+0000 7f3d40c00640 1 -- 172.21.15.66:0/2577852491 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3d3c005450 con 0x7f3d4415f300 2024-09-10T22:01:01.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.622+0000 7f3d40c00640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3d200725e0 0x7f3d20074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.622+0000 7f3d40c00640 1 -- 172.21.15.66:0/2577852491 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3d3c092e60 con 0x7f3d4415f300 2024-09-10T22:01:01.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.622+0000 7f3d43e00640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3d200725e0 0x7f3d20074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.622+0000 7f3d43e00640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3d200725e0 0x7f3d20074aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3d4415df60 tx=0x7f3d3400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.626+0000 7f3d40c00640 1 -- 172.21.15.66:0/2577852491 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3d3c05dee0 con 0x7f3d4415f300 2024-09-10T22:01:01.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.798+0000 7f3d49800640 1 -- 172.21.15.66:0/2577852491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f3d440b1150 con 0x7f3d4415f300 2024-09-10T22:01:01.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.798+0000 7f3d40c00640 1 -- 172.21.15.66:0/2577852491 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7f3d3c062910 con 0x7f3d4415f300 2024-09-10T22:01:01.805 INFO:teuthology.orchestra.run.smithi066.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T22:01:01.805 INFO:teuthology.orchestra.run.smithi066.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T22:01:01.805 INFO:teuthology.orchestra.run.smithi066.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T22:01:01.805 INFO:teuthology.orchestra.run.smithi066.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T22:01:01.805 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4673.907365 2024-09-10T22:01:01.805 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4537.831230 2024-09-10T22:01:01.805 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 5706.341617 2024-09-10T22:01:01.805 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4484.013907 2024-09-10T22:01:01.805 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3862.285837 2024-09-10T22:01:01.805 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3776.439661 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3820.270368 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3863.602758 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds 20/20 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_ms 2/2 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_frag true 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_scatterstat true 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_verify_scatter true 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced client_mount_timeout 600 2024-09-10T22:01:01.806 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_client 20/20 2024-09-10T22:01:01.807 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_ms 1/1 2024-09-10T22:01:01.807 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T22:01:01.807 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T22:01:01.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.802+0000 7f3d49800640 1 -- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3d200725e0 msgr2=0x7f3d20074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.802+0000 7f3d49800640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3d200725e0 0x7f3d20074aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3d4415df60 tx=0x7f3d3400b390 comp rx=0 tx=0).stop 2024-09-10T22:01:01.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.802+0000 7f3d49800640 1 -- 172.21.15.66:0/2577852491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d4415f300 msgr2=0x7f3d4414dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.802+0000 7f3d49800640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d4415f300 0x7f3d4414dbd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00c700 tx=0x7f3d3c00cbc0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.802+0000 7f3d49800640 1 -- 172.21.15.66:0/2577852491 shutdown_connections 2024-09-10T22:01:01.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.802+0000 7f3d49800640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3d200725e0 0x7f3d20074aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.802+0000 7f3d49800640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d4415f300 0x7f3d4414dbd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.802+0000 7f3d49800640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d4415ef20 0x7f3d44071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.802+0000 7f3d49800640 1 --2- 172.21.15.66:0/2577852491 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d440b15d0 0x7f3d440b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.802+0000 7f3d49800640 1 -- 172.21.15.66:0/2577852491 >> 172.21.15.66:0/2577852491 conn(0x7f3d440d1b10 msgr2=0x7f3d4415e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:01.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.802+0000 7f3d49800640 1 -- 172.21.15.66:0/2577852491 shutdown_connections 2024-09-10T22:01:01.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.802+0000 7f3d49800640 1 -- 172.21.15.66:0/2577852491 wait complete. 2024-09-10T22:01:01.826 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:01:01.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.970+0000 7f1ee0c00640 1 -- 172.21.15.66:0/632771879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1edc0b32a0 msgr2=0x7f1edc0af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.970+0000 7f1ee0c00640 1 --2- 172.21.15.66:0/632771879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1edc0b32a0 0x7f1edc0af390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0b1340 tx=0x7f1ed401cc30 comp rx=0 tx=0).stop 2024-09-10T22:01:01.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.970+0000 7f1ee0c00640 1 -- 172.21.15.66:0/632771879 shutdown_connections 2024-09-10T22:01:01.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.970+0000 7f1ee0c00640 1 --2- 172.21.15.66:0/632771879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1edc0b32a0 0x7f1edc0af390 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.970+0000 7f1ee0c00640 1 --2- 172.21.15.66:0/632771879 >> v2:172.21.15.94:3300/0 conn(0x7f1edc0b0f60 0x7f1edc0d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.970+0000 7f1ee0c00640 1 -- 172.21.15.66:0/632771879 >> 172.21.15.66:0/632771879 conn(0x7f1edc0d1d00 msgr2=0x7f1edc0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:01.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.970+0000 7f1ee0c00640 1 -- 172.21.15.66:0/632771879 shutdown_connections 2024-09-10T22:01:01.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.970+0000 7f1ee0c00640 1 -- 172.21.15.66:0/632771879 wait complete. 2024-09-10T22:01:01.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.970+0000 7f1ee0c00640 1 Processor -- start 2024-09-10T22:01:01.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.970+0000 7f1ee0c00640 1 -- start start 2024-09-10T22:01:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ee0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1edc0b0f60 0x7f1edc15e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ee0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1edc0b32a0 0x7f1edc15e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ee0c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1edc15f460 0x7f1edc1f7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ee0c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc0b0550 con 0x7f1edc0b0f60 2024-09-10T22:01:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ee0c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc0b03d0 con 0x7f1edc0b32a0 2024-09-10T22:01:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ee0c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1edc0b6050 con 0x7f1edc15f460 2024-09-10T22:01:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1edc0b0f60 0x7f1edc15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1edc0b0f60 0x7f1edc15e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57720/0 (socket says 172.21.15.66:57720) 2024-09-10T22:01:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed9e00640 1 -- 172.21.15.66:0/2908058315 learned_addr learned my addr 172.21.15.66:0/2908058315 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed9e00640 1 -- 172.21.15.66:0/2908058315 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1edc15f460 msgr2=0x7f1edc1f7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed9e00640 1 --2- 172.21.15.66:0/2908058315 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1edc15f460 0x7f1edc1f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed9e00640 1 -- 172.21.15.66:0/2908058315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1edc0b32a0 msgr2=0x7f1edc15e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed9e00640 1 --2- 172.21.15.66:0/2908058315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1edc0b32a0 0x7f1edc15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed9e00640 1 -- 172.21.15.66:0/2908058315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1edc1f82c0 con 0x7f1edc0b0f60 2024-09-10T22:01:01.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed9e00640 1 --2- 172.21.15.66:0/2908058315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1edc0b0f60 0x7f1edc15e130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00c3c0 tx=0x7f1ecc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:01.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed2000640 1 -- 172.21.15.66:0/2908058315 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc017070 con 0x7f1edc0b0f60 2024-09-10T22:01:01.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed2000640 1 -- 172.21.15.66:0/2908058315 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ecc0053c0 con 0x7f1edc0b0f60 2024-09-10T22:01:01.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed2000640 1 -- 172.21.15.66:0/2908058315 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc003860 con 0x7f1edc0b0f60 2024-09-10T22:01:01.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ee0c00640 1 -- 172.21.15.66:0/2908058315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1edc1f85a0 con 0x7f1edc0b0f60 2024-09-10T22:01:01.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ee0c00640 1 -- 172.21.15.66:0/2908058315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1edc0b62d0 con 0x7f1edc0b0f60 2024-09-10T22:01:01.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ee0c00640 1 -- 172.21.15.66:0/2908058315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1edc0af390 con 0x7f1edc0b0f60 2024-09-10T22:01:01.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.974+0000 7f1ed2000640 1 -- 172.21.15.66:0/2908058315 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1ecc003a00 con 0x7f1edc0b0f60 2024-09-10T22:01:01.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.978+0000 7f1ed2000640 1 --2- 172.21.15.66:0/2908058315 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1ebc0725e0 0x7f1ebc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.978+0000 7f1ed2000640 1 -- 172.21.15.66:0/2908058315 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1ecc092870 con 0x7f1edc0b0f60 2024-09-10T22:01:01.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.978+0000 7f1ed9400640 1 --2- 172.21.15.66:0/2908058315 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1ebc0725e0 0x7f1ebc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.978+0000 7f1ed9400640 1 --2- 172.21.15.66:0/2908058315 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1ebc0725e0 0x7f1ebc074aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1edc1fc180 tx=0x7f1ec8006ca0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:01.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:01.982+0000 7f1ed2000640 1 -- 172.21.15.66:0/2908058315 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ecc05df50 con 0x7f1edc0b0f60 2024-09-10T22:01:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.154+0000 7f1ee0c00640 1 -- 172.21.15.66:0/2908058315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1edc074aa0 con 0x7f1edc0b0f60 2024-09-10T22:01:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.158+0000 7f1ed2000640 1 -- 172.21.15.66:0/2908058315 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f1ecc05d850 con 0x7f1edc0b0f60 2024-09-10T22:01:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.162+0000 7f1ee0c00640 1 -- 172.21.15.66:0/2908058315 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1ebc0725e0 msgr2=0x7f1ebc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.162+0000 7f1ee0c00640 1 --2- 172.21.15.66:0/2908058315 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1ebc0725e0 0x7f1ebc074aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1edc1fc180 tx=0x7f1ec8006ca0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.162+0000 7f1ee0c00640 1 -- 172.21.15.66:0/2908058315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1edc0b0f60 msgr2=0x7f1edc15e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.162+0000 7f1ee0c00640 1 --2- 172.21.15.66:0/2908058315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1edc0b0f60 0x7f1edc15e130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00c3c0 tx=0x7f1ecc00c880 comp rx=0 tx=0).stop 2024-09-10T22:01:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.162+0000 7f1ee0c00640 1 -- 172.21.15.66:0/2908058315 shutdown_connections 2024-09-10T22:01:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.162+0000 7f1ee0c00640 1 --2- 172.21.15.66:0/2908058315 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1ebc0725e0 0x7f1ebc074aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.162+0000 7f1ee0c00640 1 --2- 172.21.15.66:0/2908058315 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1edc15f460 0x7f1edc1f7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.162+0000 7f1ee0c00640 1 --2- 172.21.15.66:0/2908058315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1edc0b32a0 0x7f1edc15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.162+0000 7f1ee0c00640 1 --2- 172.21.15.66:0/2908058315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1edc0b0f60 0x7f1edc15e130 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.162+0000 7f1ee0c00640 1 -- 172.21.15.66:0/2908058315 >> 172.21.15.66:0/2908058315 conn(0x7f1edc0d1d00 msgr2=0x7f1edc0b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.162+0000 7f1ee0c00640 1 -- 172.21.15.66:0/2908058315 shutdown_connections 2024-09-10T22:01:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.162+0000 7f1ee0c00640 1 -- 172.21.15.66:0/2908058315 wait complete. 2024-09-10T22:01:02.185 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:01:02.185 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:01:02.259 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:01:02.356 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:01:02.356 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:01:02.356 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:02.366 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:01:02.366 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:02.421 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:01:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.606+0000 7f5231600640 1 -- 172.21.15.66:0/1888263497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c0ae620 msgr2=0x7f522c06e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.606+0000 7f5231600640 1 --2- 172.21.15.66:0/1888263497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c0ae620 0x7f522c06e960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5218009900 tx=0x7f521801c8a0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.606+0000 7f5231600640 1 -- 172.21.15.66:0/1888263497 shutdown_connections 2024-09-10T22:01:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.606+0000 7f5231600640 1 --2- 172.21.15.66:0/1888263497 >> v2:172.21.15.94:3300/0 conn(0x7f522c0c4ef0 0x7f522c0c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.606+0000 7f5231600640 1 --2- 172.21.15.66:0/1888263497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c0ae620 0x7f522c06e960 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.606+0000 7f5231600640 1 -- 172.21.15.66:0/1888263497 >> 172.21.15.66:0/1888263497 conn(0x7f522c0cfa30 msgr2=0x7f522c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.606+0000 7f5231600640 1 -- 172.21.15.66:0/1888263497 shutdown_connections 2024-09-10T22:01:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.606+0000 7f5231600640 1 -- 172.21.15.66:0/1888263497 wait complete. 2024-09-10T22:01:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5231600640 1 Processor -- start 2024-09-10T22:01:02.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5231600640 1 -- start start 2024-09-10T22:01:02.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5231600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c0ae620 0x7f522c14e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5231600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f522c0c4b10 0x7f522c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5231600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f522c0c4ef0 0x7f522c0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5231600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f522c167b80 con 0x7f522c0ae620 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5231600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f522c167a00 con 0x7f522c0c4ef0 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5231600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f522c167d00 con 0x7f522c0c4b10 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f522aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c0ae620 0x7f522c14e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f522aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c0ae620 0x7f522c14e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43182/0 (socket says 172.21.15.66:43182) 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f522aa00640 1 -- 172.21.15.66:0/1563682960 learned_addr learned my addr 172.21.15.66:0/1563682960 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f522b400640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f522c0c4ef0 0x7f522c0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f522a000640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f522c0c4b10 0x7f522c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f522aa00640 1 -- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f522c0c4b10 msgr2=0x7f522c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f522aa00640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f522c0c4b10 0x7f522c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f522aa00640 1 -- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f522c0c4ef0 msgr2=0x7f522c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:02.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f522aa00640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f522c0c4ef0 0x7f522c0ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f522aa00640 1 -- 172.21.15.66:0/1563682960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c14e690 con 0x7f522c0ae620 2024-09-10T22:01:02.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f522a000640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f522c0c4b10 0x7f522c0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:02.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f522aa00640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c0ae620 0x7f522c14e150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f521c00c500 tx=0x7f521c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5222a00640 1 -- 172.21.15.66:0/1563682960 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c017070 con 0x7f522c0ae620 2024-09-10T22:01:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5222a00640 1 -- 172.21.15.66:0/1563682960 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f521c0053c0 con 0x7f522c0ae620 2024-09-10T22:01:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5222a00640 1 -- 172.21.15.66:0/1563682960 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c012890 con 0x7f522c0ae620 2024-09-10T22:01:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5231600640 1 -- 172.21.15.66:0/1563682960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522c14eba0 con 0x7f522c0ae620 2024-09-10T22:01:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.610+0000 7f5231600640 1 -- 172.21.15.66:0/1563682960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522c0ce9a0 con 0x7f522c0ae620 2024-09-10T22:01:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.614+0000 7f5231600640 1 -- 172.21.15.66:0/1563682960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522c06e960 con 0x7f522c0ae620 2024-09-10T22:01:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.614+0000 7f5222a00640 1 -- 172.21.15.66:0/1563682960 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f521c003600 con 0x7f522c0ae620 2024-09-10T22:01:02.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.614+0000 7f5222a00640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f520c0725e0 0x7f520c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:02.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.614+0000 7f5222a00640 1 -- 172.21.15.66:0/1563682960 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f521c092890 con 0x7f522c0ae620 2024-09-10T22:01:02.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.614+0000 7f522a000640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f520c0725e0 0x7f520c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:02.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.614+0000 7f522a000640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f520c0725e0 0x7f520c074aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52180af000 tx=0x7f521801b0e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.618+0000 7f5222a00640 1 -- 172.21.15.66:0/1563682960 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f521c05d890 con 0x7f522c0ae620 2024-09-10T22:01:02.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.806+0000 7f5231600640 1 -- 172.21.15.66:0/1563682960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f522c0b09b0 con 0x7f522c0ae620 2024-09-10T22:01:02.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.806+0000 7f5222a00640 1 -- 172.21.15.66:0/1563682960 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f521c0622c0 con 0x7f522c0ae620 2024-09-10T22:01:02.812 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:01:02.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.810+0000 7f5231600640 1 -- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f520c0725e0 msgr2=0x7f520c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:02.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.810+0000 7f5231600640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f520c0725e0 0x7f520c074aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52180af000 tx=0x7f521801b0e0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.810+0000 7f5231600640 1 -- 172.21.15.66:0/1563682960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c0ae620 msgr2=0x7f522c14e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:02.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.810+0000 7f5231600640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c0ae620 0x7f522c14e150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f521c00c500 tx=0x7f521c00c9c0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.810+0000 7f5231600640 1 -- 172.21.15.66:0/1563682960 shutdown_connections 2024-09-10T22:01:02.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.810+0000 7f5231600640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f520c0725e0 0x7f520c074aa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.810+0000 7f5231600640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f522c0c4ef0 0x7f522c0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.810+0000 7f5231600640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f522c0c4b10 0x7f522c0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.810+0000 7f5231600640 1 --2- 172.21.15.66:0/1563682960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f522c0ae620 0x7f522c14e150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:02.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.810+0000 7f5231600640 1 -- 172.21.15.66:0/1563682960 >> 172.21.15.66:0/1563682960 conn(0x7f522c0cfa30 msgr2=0x7f522c0afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:02.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.810+0000 7f5231600640 1 -- 172.21.15.66:0/1563682960 shutdown_connections 2024-09-10T22:01:02.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:02.810+0000 7f5231600640 1 -- 172.21.15.66:0/1563682960 wait complete. 2024-09-10T22:01:02.834 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:01:02.834 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:02.843 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:01:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.030+0000 7fcde6200640 1 -- 172.21.15.66:0/1919247848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0160370 msgr2=0x7fcde00cfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.030+0000 7fcde6200640 1 --2- 172.21.15.66:0/1919247848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0160370 0x7fcde00cfe80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcde00b6c80 tx=0x7fcdd801ccc0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.030+0000 7fcde6200640 1 -- 172.21.15.66:0/1919247848 shutdown_connections 2024-09-10T22:01:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.030+0000 7fcde6200640 1 --2- 172.21.15.66:0/1919247848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0160370 0x7fcde00cfe80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.030+0000 7fcde6200640 1 --2- 172.21.15.66:0/1919247848 >> v2:172.21.15.94:3300/0 conn(0x7fcde0071230 0x7fcde006d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.030+0000 7fcde6200640 1 -- 172.21.15.66:0/1919247848 >> 172.21.15.66:0/1919247848 conn(0x7fcde00d0b30 msgr2=0x7fcde00d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.030+0000 7fcde6200640 1 -- 172.21.15.66:0/1919247848 shutdown_connections 2024-09-10T22:01:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.030+0000 7fcde6200640 1 -- 172.21.15.66:0/1919247848 wait complete. 2024-09-10T22:01:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcde6200640 1 Processor -- start 2024-09-10T22:01:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcde6200640 1 -- start start 2024-09-10T22:01:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcde6200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde0071230 0x7fcde0155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcde6200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcde0156430 0x7fcde0153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcde6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0160370 0x7fcde00b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcde6200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0168b90 con 0x7fcde0160370 2024-09-10T22:01:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcde6200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcde015d010 con 0x7fcde0071230 2024-09-10T22:01:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcde6200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcde0168d10 con 0x7fcde0156430 2024-09-10T22:01:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0160370 0x7fcde00b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcddf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde0071230 0x7fcde0155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcddf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde0071230 0x7fcde0155690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:55306/0 (socket says 172.21.15.66:55306) 2024-09-10T22:01:03.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcdd7e00640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcde0156430 0x7fcde0153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcddf400640 1 -- 172.21.15.66:0/2204461167 learned_addr learned my addr 172.21.15.66:0/2204461167 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:03.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcddfe00640 1 -- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcde0156430 msgr2=0x7fcde0153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcddfe00640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcde0156430 0x7fcde0153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcddfe00640 1 -- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde0071230 msgr2=0x7fcde0155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcddfe00640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde0071230 0x7fcde0155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcddfe00640 1 -- 172.21.15.66:0/2204461167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde01fb890 con 0x7fcde0160370 2024-09-10T22:01:03.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcddfe00640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0160370 0x7fcde00b5620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80ac000 tx=0x7fcdd8009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:03.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcdd7400640 1 -- 172.21.15.66:0/2204461167 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80ba020 con 0x7fcde0160370 2024-09-10T22:01:03.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcdd7400640 1 -- 172.21.15.66:0/2204461167 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdd8004d60 con 0x7fcde0160370 2024-09-10T22:01:03.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcdd7400640 1 -- 172.21.15.66:0/2204461167 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80b2040 con 0x7fcde0160370 2024-09-10T22:01:03.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcde6200640 1 -- 172.21.15.66:0/2204461167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde01fad10 con 0x7fcde0160370 2024-09-10T22:01:03.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcde6200640 1 -- 172.21.15.66:0/2204461167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde01fb2d0 con 0x7fcde0160370 2024-09-10T22:01:03.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.034+0000 7fcde6200640 1 -- 172.21.15.66:0/2204461167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde0153f70 con 0x7fcde0160370 2024-09-10T22:01:03.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.038+0000 7fcdd7400640 1 -- 172.21.15.66:0/2204461167 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcdd80b5070 con 0x7fcde0160370 2024-09-10T22:01:03.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.038+0000 7fcdd7400640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcdbc0725e0 0x7fcdbc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.038+0000 7fcdd7400640 1 -- 172.21.15.66:0/2204461167 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcdd812d270 con 0x7fcde0160370 2024-09-10T22:01:03.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.038+0000 7fcddf400640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcdbc0725e0 0x7fcdbc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.038+0000 7fcddf400640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcdbc0725e0 0x7fcdbc074aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0001910 tx=0x7fcdd0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:03.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.042+0000 7fcdd7400640 1 -- 172.21.15.66:0/2204461167 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcdd80c4020 con 0x7fcde0160370 2024-09-10T22:01:03.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.226+0000 7fcde6200640 1 -- 172.21.15.66:0/2204461167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcde0073960 con 0x7fcde0160370 2024-09-10T22:01:03.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.226+0000 7fcdd7400640 1 -- 172.21.15.66:0/2204461167 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fcdd8016970 con 0x7fcde0160370 2024-09-10T22:01:03.232 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:01:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.230+0000 7fcde6200640 1 -- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcdbc0725e0 msgr2=0x7fcdbc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.230+0000 7fcde6200640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcdbc0725e0 0x7fcdbc074aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0001910 tx=0x7fcdd0009040 comp rx=0 tx=0).stop 2024-09-10T22:01:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.230+0000 7fcde6200640 1 -- 172.21.15.66:0/2204461167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0160370 msgr2=0x7fcde00b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.230+0000 7fcde6200640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0160370 0x7fcde00b5620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80ac000 tx=0x7fcdd8009350 comp rx=0 tx=0).stop 2024-09-10T22:01:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.230+0000 7fcde6200640 1 -- 172.21.15.66:0/2204461167 shutdown_connections 2024-09-10T22:01:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.230+0000 7fcde6200640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcdbc0725e0 0x7fcdbc074aa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.230+0000 7fcde6200640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0160370 0x7fcde00b5620 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.230+0000 7fcde6200640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcde0156430 0x7fcde0153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.230+0000 7fcde6200640 1 --2- 172.21.15.66:0/2204461167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde0071230 0x7fcde0155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.230+0000 7fcde6200640 1 -- 172.21.15.66:0/2204461167 >> 172.21.15.66:0/2204461167 conn(0x7fcde00d0b30 msgr2=0x7fcde00743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.230+0000 7fcde6200640 1 -- 172.21.15.66:0/2204461167 shutdown_connections 2024-09-10T22:01:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.230+0000 7fcde6200640 1 -- 172.21.15.66:0/2204461167 wait complete. 2024-09-10T22:01:03.253 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-09-10T22:01:03.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:03.254 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T22:01:03.285 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:01:03.285 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:01:03.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:03.285 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:01:03.345 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:01:03.345 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:03.355 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:01:03.355 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:03.405 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:01:03.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.546+0000 7f2508400640 1 -- 172.21.15.66:0/3165923838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000c4fc0 msgr2=0x7f25000ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.546+0000 7f2508400640 1 --2- 172.21.15.66:0/3165923838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000c4fc0 0x7f25000ca6e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009bc0 tx=0x7f24f801cb10 comp rx=0 tx=0).stop 2024-09-10T22:01:03.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 -- 172.21.15.66:0/3165923838 shutdown_connections 2024-09-10T22:01:03.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 --2- 172.21.15.66:0/3165923838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000c4fc0 0x7f25000ca6e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 --2- 172.21.15.66:0/3165923838 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f25000d0550 0x7f25000ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 -- 172.21.15.66:0/3165923838 >> 172.21.15.66:0/3165923838 conn(0x7f25000cfa30 msgr2=0x7f25000cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:03.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 -- 172.21.15.66:0/3165923838 shutdown_connections 2024-09-10T22:01:03.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 -- 172.21.15.66:0/3165923838 wait complete. 2024-09-10T22:01:03.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 Processor -- start 2024-09-10T22:01:03.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 -- start start 2024-09-10T22:01:03.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25000c4fc0 0x7f2500155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f250015a310 0x7f25001559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000c9850 0x7f25000c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2500167480 con 0x7f25000c9850 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2500167300 con 0x7f25000c4fc0 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2508400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2500167600 con 0x7f250015a310 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2506400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000c9850 0x7f25000c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2506400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000c9850 0x7f25000c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43196/0 (socket says 172.21.15.66:43196) 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2506400640 1 -- 172.21.15.66:0/693307255 learned_addr learned my addr 172.21.15.66:0/693307255 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2506400640 1 -- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f250015a310 msgr2=0x7f25001559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2506400640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f250015a310 0x7f25001559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2506400640 1 -- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25000c4fc0 msgr2=0x7f2500155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2505a00640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25000c4fc0 0x7f2500155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2506400640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25000c4fc0 0x7f2500155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2506400640 1 -- 172.21.15.66:0/693307255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2500156310 con 0x7f25000c9850 2024-09-10T22:01:03.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2505a00640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25000c4fc0 0x7f2500155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:03.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.550+0000 7f2506400640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000c9850 0x7f25000c6b40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f24f801a740 tx=0x7f24f8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:03.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.554+0000 7f24fd600640 1 -- 172.21.15.66:0/693307255 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f80ba020 con 0x7f25000c9850 2024-09-10T22:01:03.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.554+0000 7f24fd600640 1 -- 172.21.15.66:0/693307255 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24f8004d40 con 0x7f25000c9850 2024-09-10T22:01:03.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.554+0000 7f24fd600640 1 -- 172.21.15.66:0/693307255 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f80b2040 con 0x7f25000c9850 2024-09-10T22:01:03.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.554+0000 7f2508400640 1 -- 172.21.15.66:0/693307255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25001565f0 con 0x7f25000c9850 2024-09-10T22:01:03.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.554+0000 7f2508400640 1 -- 172.21.15.66:0/693307255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250014ef40 con 0x7f25000c9850 2024-09-10T22:01:03.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.554+0000 7f2508400640 1 -- 172.21.15.66:0/693307255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25000c48b0 con 0x7f25000c9850 2024-09-10T22:01:03.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.554+0000 7f24fd600640 1 -- 172.21.15.66:0/693307255 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f24f80b5070 con 0x7f25000c9850 2024-09-10T22:01:03.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.554+0000 7f24fd600640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24e00725e0 0x7f24e0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.554+0000 7f24fd600640 1 -- 172.21.15.66:0/693307255 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f24f812d4a0 con 0x7f25000c9850 2024-09-10T22:01:03.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.554+0000 7f2505a00640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24e00725e0 0x7f24e0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.558+0000 7f2505a00640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24e00725e0 0x7f24e0074aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f25000c96f0 tx=0x7f24f4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:03.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.558+0000 7f24fd600640 1 -- 172.21.15.66:0/693307255 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f24f80c4020 con 0x7f25000c9850 2024-09-10T22:01:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 -- 172.21.15.66:0/693307255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2500070420 con 0x7f25000c9850 2024-09-10T22:01:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f24fd600640 1 -- 172.21.15.66:0/693307255 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f24f8016790 con 0x7f25000c9850 2024-09-10T22:01:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:01:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 -- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24e00725e0 msgr2=0x7f24e0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24e00725e0 0x7f24e0074aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f25000c96f0 tx=0x7f24f4009040 comp rx=0 tx=0).stop 2024-09-10T22:01:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 -- 172.21.15.66:0/693307255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000c9850 msgr2=0x7f25000c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000c9850 0x7f25000c6b40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f24f801a740 tx=0x7f24f8003f60 comp rx=0 tx=0).stop 2024-09-10T22:01:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 -- 172.21.15.66:0/693307255 shutdown_connections 2024-09-10T22:01:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24e00725e0 0x7f24e0074aa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000c9850 0x7f25000c6b40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f250015a310 0x7f25001559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 --2- 172.21.15.66:0/693307255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25000c4fc0 0x7f2500155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 -- 172.21.15.66:0/693307255 >> 172.21.15.66:0/693307255 conn(0x7f25000cfa30 msgr2=0x7f2500150030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 -- 172.21.15.66:0/693307255 shutdown_connections 2024-09-10T22:01:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.746+0000 7f2508400640 1 -- 172.21.15.66:0/693307255 wait complete. 2024-09-10T22:01:03.771 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:01:03.771 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:03.781 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:01:03.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.926+0000 7f2f88a00640 1 -- 172.21.15.66:0/977289369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f68003a10 msgr2=0x7f2f68003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.926+0000 7f2f88a00640 1 --2- 172.21.15.66:0/977289369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f68003a10 0x7f2f68003e30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009b60 tx=0x7f2f7401d2b0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.926+0000 7f2f82800640 1 -- 172.21.15.66:0/977289369 reap_dead start 2024-09-10T22:01:03.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.926+0000 7f2f88a00640 1 -- 172.21.15.66:0/977289369 shutdown_connections 2024-09-10T22:01:03.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.926+0000 7f2f88a00640 1 -- 172.21.15.66:0/977289369 >> 172.21.15.66:0/977289369 conn(0x7f2f840cfa30 msgr2=0x7f2f840cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:03.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.926+0000 7f2f88a00640 1 -- 172.21.15.66:0/977289369 shutdown_connections 2024-09-10T22:01:03.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.926+0000 7f2f88a00640 1 -- 172.21.15.66:0/977289369 wait complete. 2024-09-10T22:01:03.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.926+0000 7f2f88a00640 1 Processor -- start 2024-09-10T22:01:03.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.926+0000 7f2f88a00640 1 -- start start 2024-09-10T22:01:03.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f88a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f68003a10 0x7f2f841fc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f88a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f840c61c0 0x7f2f840afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f88a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f84153eb0 0x7f2f84154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f88a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2f84166bd0 con 0x7f2f68003a10 2024-09-10T22:01:03.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f88a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f840ce6b0 con 0x7f2f84153eb0 2024-09-10T22:01:03.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f88a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2f84166d50 con 0x7f2f840c61c0 2024-09-10T22:01:03.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f81e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f68003a10 0x7f2f841fc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f81e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f68003a10 0x7f2f841fc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43224/0 (socket says 172.21.15.66:43224) 2024-09-10T22:01:03.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f81e00640 1 -- 172.21.15.66:0/1542229404 learned_addr learned my addr 172.21.15.66:0/1542229404 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:03.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f79400640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f840c61c0 0x7f2f840afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f82800640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f84153eb0 0x7f2f84154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f81e00640 1 -- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f840c61c0 msgr2=0x7f2f840afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f81e00640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f840c61c0 0x7f2f840afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f81e00640 1 -- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f84153eb0 msgr2=0x7f2f84154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f81e00640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f84153eb0 0x7f2f84154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f81e00640 1 -- 172.21.15.66:0/1542229404 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f841f7f00 con 0x7f2f68003a10 2024-09-10T22:01:03.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f81e00640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f68003a10 0x7f2f841fc000 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f740af040 tx=0x7f2f740055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:03.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f7aa00640 1 -- 172.21.15.66:0/1542229404 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f740b1040 con 0x7f2f68003a10 2024-09-10T22:01:03.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f7aa00640 1 -- 172.21.15.66:0/1542229404 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f740bb070 con 0x7f2f68003a10 2024-09-10T22:01:03.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f88a00640 1 -- 172.21.15.66:0/1542229404 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f840b02f0 con 0x7f2f68003a10 2024-09-10T22:01:03.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f7aa00640 1 -- 172.21.15.66:0/1542229404 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74004290 con 0x7f2f68003a10 2024-09-10T22:01:03.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f88a00640 1 -- 172.21.15.66:0/1542229404 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f841f84a0 con 0x7f2f68003a10 2024-09-10T22:01:03.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f88a00640 1 -- 172.21.15.66:0/1542229404 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f840ca080 con 0x7f2f68003a10 2024-09-10T22:01:03.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.930+0000 7f2f7aa00640 1 -- 172.21.15.66:0/1542229404 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2f740046d0 con 0x7f2f68003a10 2024-09-10T22:01:03.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.934+0000 7f2f7aa00640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f600725e0 0x7f2f60074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.934+0000 7f2f7aa00640 1 -- 172.21.15.66:0/1542229404 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7412df60 con 0x7f2f68003a10 2024-09-10T22:01:03.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.934+0000 7f2f79400640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f600725e0 0x7f2f60074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.934+0000 7f2f79400640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f600725e0 0x7f2f60074aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c000c10 tx=0x7f2f6c001ba0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:03.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:03.938+0000 7f2f7aa00640 1 -- 172.21.15.66:0/1542229404 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2f74016fc0 con 0x7f2f68003a10 2024-09-10T22:01:04.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.126+0000 7f2f88a00640 1 -- 172.21.15.66:0/1542229404 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2f840c65a0 con 0x7f2f68003a10 2024-09-10T22:01:04.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.126+0000 7f2f7aa00640 1 -- 172.21.15.66:0/1542229404 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f2f740f9060 con 0x7f2f68003a10 2024-09-10T22:01:04.133 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:01:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.130+0000 7f2f88a00640 1 -- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f600725e0 msgr2=0x7f2f60074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.130+0000 7f2f88a00640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f600725e0 0x7f2f60074aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c000c10 tx=0x7f2f6c001ba0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.130+0000 7f2f88a00640 1 -- 172.21.15.66:0/1542229404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f68003a10 msgr2=0x7f2f841fc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.130+0000 7f2f88a00640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f68003a10 0x7f2f841fc000 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f740af040 tx=0x7f2f740055d0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.130+0000 7f2f88a00640 1 -- 172.21.15.66:0/1542229404 shutdown_connections 2024-09-10T22:01:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.130+0000 7f2f88a00640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f600725e0 0x7f2f60074aa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.130+0000 7f2f88a00640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f84153eb0 0x7f2f84154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.130+0000 7f2f88a00640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f840c61c0 0x7f2f840afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.130+0000 7f2f88a00640 1 --2- 172.21.15.66:0/1542229404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f68003a10 0x7f2f841fc000 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.130+0000 7f2f88a00640 1 -- 172.21.15.66:0/1542229404 >> 172.21.15.66:0/1542229404 conn(0x7f2f840cfa30 msgr2=0x7f2f8406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:04.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.130+0000 7f2f88a00640 1 -- 172.21.15.66:0/1542229404 shutdown_connections 2024-09-10T22:01:04.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.130+0000 7f2f88a00640 1 -- 172.21.15.66:0/1542229404 wait complete. 2024-09-10T22:01:04.153 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T22:01:04.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:04.154 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T22:01:04.191 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:01:04.191 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:01:04.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:04.191 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T22:01:04.248 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:01:04.248 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:04.257 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:01:04.257 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:04.308 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:01:04.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.490+0000 7f35b3000640 1 -- 172.21.15.66:0/4117500272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ac15bd40 msgr2=0x7f35ac0cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:04.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.490+0000 7f35b3000640 1 --2- 172.21.15.66:0/4117500272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ac15bd40 0x7f35ac0cffa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0747e0 tx=0x7f359c01c770 comp rx=0 tx=0).stop 2024-09-10T22:01:04.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.490+0000 7f35b3000640 1 -- 172.21.15.66:0/4117500272 shutdown_connections 2024-09-10T22:01:04.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.490+0000 7f35b3000640 1 --2- 172.21.15.66:0/4117500272 >> v2:172.21.15.94:3300/0 conn(0x7f35ac0cf830 0x7f35ac0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.490+0000 7f35b3000640 1 --2- 172.21.15.66:0/4117500272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ac15bd40 0x7f35ac0cffa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.490+0000 7f35b3000640 1 -- 172.21.15.66:0/4117500272 >> 172.21.15.66:0/4117500272 conn(0x7f35ac0d1c60 msgr2=0x7f35ac0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:04.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.490+0000 7f35b3000640 1 -- 172.21.15.66:0/4117500272 shutdown_connections 2024-09-10T22:01:04.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.490+0000 7f35b3000640 1 -- 172.21.15.66:0/4117500272 wait complete. 2024-09-10T22:01:04.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35b3000640 1 Processor -- start 2024-09-10T22:01:04.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35b3000640 1 -- start start 2024-09-10T22:01:04.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35b3000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ac0aec10 0x7f35ac158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:04.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35b3000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35ac0cf830 0x7f35ac14aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:04.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35b3000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac15bd40 0x7f35ac164800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:04.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35b3000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac168fd0 con 0x7f35ac0aec10 2024-09-10T22:01:04.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35b3000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac168e50 con 0x7f35ac15bd40 2024-09-10T22:01:04.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35b3000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f35ac169150 con 0x7f35ac0cf830 2024-09-10T22:01:04.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35abe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ac0aec10 0x7f35ac158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:04.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35abe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ac0aec10 0x7f35ac158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43238/0 (socket says 172.21.15.66:43238) 2024-09-10T22:01:04.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35abe00640 1 -- 172.21.15.66:0/2355479750 learned_addr learned my addr 172.21.15.66:0/2355479750 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:04.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35a3e00640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35ac0cf830 0x7f35ac14aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:04.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35abe00640 1 -- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35ac0cf830 msgr2=0x7f35ac14aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:04.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35abe00640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35ac0cf830 0x7f35ac14aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35abe00640 1 -- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac15bd40 msgr2=0x7f35ac164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:04.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35abe00640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac15bd40 0x7f35ac164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35abe00640 1 -- 172.21.15.66:0/2355479750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac0b7020 con 0x7f35ac0aec10 2024-09-10T22:01:04.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35abe00640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ac0aec10 0x7f35ac158420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f359c000c10 tx=0x7f359c0a55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:04.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35a8c00640 1 -- 172.21.15.66:0/2355479750 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c0bb070 con 0x7f35ac0aec10 2024-09-10T22:01:04.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35a8c00640 1 -- 172.21.15.66:0/2355479750 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f359c004470 con 0x7f35ac0aec10 2024-09-10T22:01:04.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35a8c00640 1 -- 172.21.15.66:0/2355479750 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c0b2040 con 0x7f35ac0aec10 2024-09-10T22:01:04.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35b3000640 1 -- 172.21.15.66:0/2355479750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac0b7300 con 0x7f35ac0aec10 2024-09-10T22:01:04.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.494+0000 7f35b3000640 1 -- 172.21.15.66:0/2355479750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac14b580 con 0x7f35ac0aec10 2024-09-10T22:01:04.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.498+0000 7f35b3000640 1 -- 172.21.15.66:0/2355479750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac0af0a0 con 0x7f35ac0aec10 2024-09-10T22:01:04.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.498+0000 7f35a8c00640 1 -- 172.21.15.66:0/2355479750 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f359c0b5030 con 0x7f35ac0aec10 2024-09-10T22:01:04.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.498+0000 7f35a8c00640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35880725e0 0x7f3588074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:04.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.498+0000 7f35a8c00640 1 -- 172.21.15.66:0/2355479750 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f359c12ca60 con 0x7f35ac0aec10 2024-09-10T22:01:04.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.498+0000 7f35a3e00640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35880725e0 0x7f3588074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:04.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.498+0000 7f35a3e00640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35880725e0 0x7f3588074aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3594005d90 tx=0x7f3594005c60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:04.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.502+0000 7f35a8c00640 1 -- 172.21.15.66:0/2355479750 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f359c0f2de0 con 0x7f35ac0aec10 2024-09-10T22:01:04.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.686+0000 7f35b3000640 1 -- 172.21.15.66:0/2355479750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f35ac0b0df0 con 0x7f35ac0aec10 2024-09-10T22:01:04.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35a8c00640 1 -- 172.21.15.66:0/2355479750 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f359c0163f0 con 0x7f35ac0aec10 2024-09-10T22:01:04.693 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:01:04.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35b3000640 1 -- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35880725e0 msgr2=0x7f3588074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:04.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35b3000640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35880725e0 0x7f3588074aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3594005d90 tx=0x7f3594005c60 comp rx=0 tx=0).stop 2024-09-10T22:01:04.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35b3000640 1 -- 172.21.15.66:0/2355479750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ac0aec10 msgr2=0x7f35ac158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:04.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35b3000640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ac0aec10 0x7f35ac158420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f359c000c10 tx=0x7f359c0a55c0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35b3000640 1 -- 172.21.15.66:0/2355479750 shutdown_connections 2024-09-10T22:01:04.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35b3000640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35880725e0 0x7f3588074aa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35b3000640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35ac15bd40 0x7f35ac164800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35b3000640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35ac0cf830 0x7f35ac14aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35b3000640 1 --2- 172.21.15.66:0/2355479750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ac0aec10 0x7f35ac158420 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35b3000640 1 -- 172.21.15.66:0/2355479750 >> 172.21.15.66:0/2355479750 conn(0x7f35ac0d1c60 msgr2=0x7f35ac06e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:04.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35b3000640 1 -- 172.21.15.66:0/2355479750 shutdown_connections 2024-09-10T22:01:04.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.690+0000 7f35b3000640 1 -- 172.21.15.66:0/2355479750 wait complete. 2024-09-10T22:01:04.714 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:01:04.714 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:04.723 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:01:04.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.906+0000 7fd1d7000640 1 -- 172.21.15.66:0/2358038609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1d00b0f60 msgr2=0x7fd1d00d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:04.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.906+0000 7fd1d7000640 1 --2- 172.21.15.66:0/2358038609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1d00b0f60 0x7fd1d00d0310 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0073ef0 tx=0x7fd1b801c770 comp rx=0 tx=0).stop 2024-09-10T22:01:04.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.906+0000 7fd1d7000640 1 -- 172.21.15.66:0/2358038609 shutdown_connections 2024-09-10T22:01:04.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.906+0000 7fd1d7000640 1 --2- 172.21.15.66:0/2358038609 >> v2:172.21.15.94:3300/0 conn(0x7fd1d00b32a0 0x7fd1d00af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.906+0000 7fd1d7000640 1 --2- 172.21.15.66:0/2358038609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1d00b0f60 0x7fd1d00d0310 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.906+0000 7fd1d7000640 1 -- 172.21.15.66:0/2358038609 >> 172.21.15.66:0/2358038609 conn(0x7fd1d00d1d00 msgr2=0x7fd1d00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:04.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.906+0000 7fd1d7000640 1 -- 172.21.15.66:0/2358038609 shutdown_connections 2024-09-10T22:01:04.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.906+0000 7fd1d7000640 1 -- 172.21.15.66:0/2358038609 wait complete. 2024-09-10T22:01:04.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.906+0000 7fd1d7000640 1 Processor -- start 2024-09-10T22:01:04.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.906+0000 7fd1d7000640 1 -- start start 2024-09-10T22:01:04.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d7000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1d00b0f60 0x7fd1d0164540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:04.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d7000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1d00b32a0 0x7fd1d015dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:04.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d7000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1d015f460 0x7fd1d01f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:04.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d7000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d00b0520 con 0x7fd1d015f460 2024-09-10T22:01:04.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d7000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d00b03a0 con 0x7fd1d00b32a0 2024-09-10T22:01:04.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d7000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d00b06a0 con 0x7fd1d00b0f60 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1d015f460 0x7fd1d01f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1d015f460 0x7fd1d01f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43262/0 (socket says 172.21.15.66:43262) 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1c7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1d00b32a0 0x7fd1d015dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d5000640 1 -- 172.21.15.66:0/1315410449 learned_addr learned my addr 172.21.15.66:0/1315410449 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1cfe00640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1d00b0f60 0x7fd1d0164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d5000640 1 -- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1d00b0f60 msgr2=0x7fd1d0164540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d5000640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1d00b0f60 0x7fd1d0164540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d5000640 1 -- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1d00b32a0 msgr2=0x7fd1d015dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d5000640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1d00b32a0 0x7fd1d015dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d5000640 1 -- 172.21.15.66:0/1315410449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d00b1bb0 con 0x7fd1d015f460 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d5000640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1d015f460 0x7fd1d01f7ba0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800eb20 tx=0x7fd1c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1ccc00640 1 -- 172.21.15.66:0/1315410449 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8004b30 con 0x7fd1d015f460 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1ccc00640 1 -- 172.21.15.66:0/1315410449 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1c8004cf0 con 0x7fd1d015f460 2024-09-10T22:01:04.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1ccc00640 1 -- 172.21.15.66:0/1315410449 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8005280 con 0x7fd1d015f460 2024-09-10T22:01:04.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d7000640 1 -- 172.21.15.66:0/1315410449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d01662d0 con 0x7fd1d015f460 2024-09-10T22:01:04.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d7000640 1 -- 172.21.15.66:0/1315410449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d0166890 con 0x7fd1d015f460 2024-09-10T22:01:04.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1d7000640 1 -- 172.21.15.66:0/1315410449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d00af390 con 0x7fd1d015f460 2024-09-10T22:01:04.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.910+0000 7fd1ccc00640 1 -- 172.21.15.66:0/1315410449 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd1c8005420 con 0x7fd1d015f460 2024-09-10T22:01:04.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.914+0000 7fd1ccc00640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd1ac0725e0 0x7fd1ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:04.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.914+0000 7fd1ccc00640 1 -- 172.21.15.66:0/1315410449 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd1c8093e30 con 0x7fd1d015f460 2024-09-10T22:01:04.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.914+0000 7fd1cfe00640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd1ac0725e0 0x7fd1ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:04.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.914+0000 7fd1cfe00640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd1ac0725e0 0x7fd1ac074aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0007fd0 tx=0x7fd1c0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:04.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:04.918+0000 7fd1ccc00640 1 -- 172.21.15.66:0/1315410449 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd1c805eec0 con 0x7fd1d015f460 2024-09-10T22:01:05.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.102+0000 7fd1d7000640 1 -- 172.21.15.66:0/1315410449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd1d00b46e0 con 0x7fd1d015f460 2024-09-10T22:01:05.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.102+0000 7fd1ccc00640 1 -- 172.21.15.66:0/1315410449 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fd1c80638f0 con 0x7fd1d015f460 2024-09-10T22:01:05.108 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:01:05.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.106+0000 7fd1d7000640 1 -- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd1ac0725e0 msgr2=0x7fd1ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:05.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.106+0000 7fd1d7000640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd1ac0725e0 0x7fd1ac074aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0007fd0 tx=0x7fd1c0009040 comp rx=0 tx=0).stop 2024-09-10T22:01:05.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.106+0000 7fd1d7000640 1 -- 172.21.15.66:0/1315410449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1d015f460 msgr2=0x7fd1d01f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:05.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.106+0000 7fd1d7000640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1d015f460 0x7fd1d01f7ba0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800eb20 tx=0x7fd1c800c2d0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.106+0000 7fd1d7000640 1 -- 172.21.15.66:0/1315410449 shutdown_connections 2024-09-10T22:01:05.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.106+0000 7fd1d7000640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd1ac0725e0 0x7fd1ac074aa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.106+0000 7fd1d7000640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1d015f460 0x7fd1d01f7ba0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.106+0000 7fd1d7000640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1d00b32a0 0x7fd1d015dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.106+0000 7fd1d7000640 1 --2- 172.21.15.66:0/1315410449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1d00b0f60 0x7fd1d0164540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.106+0000 7fd1d7000640 1 -- 172.21.15.66:0/1315410449 >> 172.21.15.66:0/1315410449 conn(0x7fd1d00d1d00 msgr2=0x7fd1d00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:05.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.106+0000 7fd1d7000640 1 -- 172.21.15.66:0/1315410449 shutdown_connections 2024-09-10T22:01:05.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.106+0000 7fd1d7000640 1 -- 172.21.15.66:0/1315410449 wait complete. 2024-09-10T22:01:05.130 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-09-10T22:01:05.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:05.130 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-10T22:01:05.151 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:01:05.151 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:01:05.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:05.151 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T22:01:05.205 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:01:05.205 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:05.216 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:01:05.216 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:05.265 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:01:05.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.406+0000 7f8fa1200640 1 -- 172.21.15.66:0/2389238433 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f9c15f300 msgr2=0x7f8f9c0b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:05.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.406+0000 7f8fa1200640 1 --2- 172.21.15.66:0/2389238433 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f9c15f300 0x7f8f9c0b1090 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009840 tx=0x7f8f8401c820 comp rx=0 tx=0).stop 2024-09-10T22:01:05.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.406+0000 7f8fa1200640 1 -- 172.21.15.66:0/2389238433 shutdown_connections 2024-09-10T22:01:05.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.406+0000 7f8fa1200640 1 --2- 172.21.15.66:0/2389238433 >> v2:172.21.15.94:3300/0 conn(0x7f8f9c0b15d0 0x7f8f9c0cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.406+0000 7f8fa1200640 1 --2- 172.21.15.66:0/2389238433 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f9c15f300 0x7f8f9c0b1090 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.406+0000 7f8fa1200640 1 -- 172.21.15.66:0/2389238433 >> 172.21.15.66:0/2389238433 conn(0x7f8f9c0d1b10 msgr2=0x7f8f9c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:05.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.406+0000 7f8fa1200640 1 -- 172.21.15.66:0/2389238433 shutdown_connections 2024-09-10T22:01:05.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.406+0000 7f8fa1200640 1 -- 172.21.15.66:0/2389238433 wait complete. 2024-09-10T22:01:05.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8fa1200640 1 Processor -- start 2024-09-10T22:01:05.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8fa1200640 1 -- start start 2024-09-10T22:01:05.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8fa1200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f9c0b15d0 0x7f8f9c0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:05.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8fa1200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f9c15ef20 0x7f8f9c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:05.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8fa1200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f9c15f300 0x7f8f9c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:05.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8fa1200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c169360 con 0x7f8f9c15ef20 2024-09-10T22:01:05.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8fa1200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1691e0 con 0x7f8f9c15f300 2024-09-10T22:01:05.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8fa1200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9c1694e0 con 0x7f8f9c0b15d0 2024-09-10T22:01:05.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f9c15ef20 0x7f8f9c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:05.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f9c15ef20 0x7f8f9c15c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43272/0 (socket says 172.21.15.66:43272) 2024-09-10T22:01:05.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f93e00640 1 -- 172.21.15.66:0/4129350636 learned_addr learned my addr 172.21.15.66:0/4129350636 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:05.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f9b400640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f9c0b15d0 0x7f8f9c0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:05.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f9be00640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f9c15f300 0x7f8f9c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:05.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f93e00640 1 -- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f9c0b15d0 msgr2=0x7f8f9c0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:05.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f93e00640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f9c0b15d0 0x7f8f9c0cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f93e00640 1 -- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f9c15f300 msgr2=0x7f8f9c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:05.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f93e00640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f9c15f300 0x7f8f9c15dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f93e00640 1 -- 172.21.15.66:0/4129350636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c0b2f10 con 0x7f8f9c15ef20 2024-09-10T22:01:05.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f9b400640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f9c0b15d0 0x7f8f9c0cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:01:05.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f93e00640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f9c15ef20 0x7f8f9c15c9a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009790 tx=0x7f8f840a6600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:05.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.410+0000 7f8f93400640 1 -- 172.21.15.66:0/4129350636 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84004040 con 0x7f8f9c15ef20 2024-09-10T22:01:05.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.414+0000 7f8fa1200640 1 -- 172.21.15.66:0/4129350636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9c0c5640 con 0x7f8f9c15ef20 2024-09-10T22:01:05.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.414+0000 7f8f93400640 1 -- 172.21.15.66:0/4129350636 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f840042d0 con 0x7f8f9c15ef20 2024-09-10T22:01:05.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.414+0000 7f8f93400640 1 -- 172.21.15.66:0/4129350636 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84003600 con 0x7f8f9c15ef20 2024-09-10T22:01:05.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.414+0000 7f8fa1200640 1 -- 172.21.15.66:0/4129350636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9c0c5ea0 con 0x7f8f9c15ef20 2024-09-10T22:01:05.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.414+0000 7f8fa1200640 1 -- 172.21.15.66:0/4129350636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f9c0c59f0 con 0x7f8f9c15ef20 2024-09-10T22:01:05.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.414+0000 7f8f93400640 1 -- 172.21.15.66:0/4129350636 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8f9c0d0100 con 0x7f8f9c15ef20 2024-09-10T22:01:05.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.414+0000 7f8f93400640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f780725e0 0x7f8f78074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:05.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.414+0000 7f8f93400640 1 -- 172.21.15.66:0/4129350636 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8f8412db50 con 0x7f8f9c15ef20 2024-09-10T22:01:05.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.414+0000 7f8f9b400640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f780725e0 0x7f8f78074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:05.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.418+0000 7f8f9b400640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f780725e0 0x7f8f78074aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c000f70 tx=0x7f8f8c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:05.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.418+0000 7f8f93400640 1 -- 172.21.15.66:0/4129350636 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8f84016410 con 0x7f8f9c15ef20 2024-09-10T22:01:05.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.602+0000 7f8fa1200640 1 -- 172.21.15.66:0/4129350636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8f9c072e00 con 0x7f8f9c15ef20 2024-09-10T22:01:05.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.602+0000 7f8f93400640 1 -- 172.21.15.66:0/4129350636 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f8f840f8ac0 con 0x7f8f9c15ef20 2024-09-10T22:01:05.608 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:01:05.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.606+0000 7f8fa1200640 1 -- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f780725e0 msgr2=0x7f8f78074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:05.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.606+0000 7f8fa1200640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f780725e0 0x7f8f78074aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c000f70 tx=0x7f8f8c009040 comp rx=0 tx=0).stop 2024-09-10T22:01:05.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.606+0000 7f8fa1200640 1 -- 172.21.15.66:0/4129350636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f9c15ef20 msgr2=0x7f8f9c15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:05.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.606+0000 7f8fa1200640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f9c15ef20 0x7f8f9c15c9a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009790 tx=0x7f8f840a6600 comp rx=0 tx=0).stop 2024-09-10T22:01:05.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.606+0000 7f8fa1200640 1 -- 172.21.15.66:0/4129350636 shutdown_connections 2024-09-10T22:01:05.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.606+0000 7f8fa1200640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f780725e0 0x7f8f78074aa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.606+0000 7f8fa1200640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f9c15f300 0x7f8f9c15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.606+0000 7f8fa1200640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f9c15ef20 0x7f8f9c15c9a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.606+0000 7f8fa1200640 1 --2- 172.21.15.66:0/4129350636 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f9c0b15d0 0x7f8f9c0cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.606+0000 7f8fa1200640 1 -- 172.21.15.66:0/4129350636 >> 172.21.15.66:0/4129350636 conn(0x7f8f9c0d1b10 msgr2=0x7f8f9c06f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:05.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.606+0000 7f8fa1200640 1 -- 172.21.15.66:0/4129350636 shutdown_connections 2024-09-10T22:01:05.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.606+0000 7f8fa1200640 1 -- 172.21.15.66:0/4129350636 wait complete. 2024-09-10T22:01:05.629 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:01:05.629 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:05.642 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:01:05.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 -- 172.21.15.66:0/1027822070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40c0b2bf0 msgr2=0x7fd40c0aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:05.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 --2- 172.21.15.66:0/1027822070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40c0b2bf0 0x7fd40c0aeea0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd40c073790 tx=0x7fd40401cb80 comp rx=0 tx=0).stop 2024-09-10T22:01:05.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 -- 172.21.15.66:0/1027822070 shutdown_connections 2024-09-10T22:01:05.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 --2- 172.21.15.66:0/1027822070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40c0b2bf0 0x7fd40c0aeea0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 --2- 172.21.15.66:0/1027822070 >> v2:172.21.15.94:3300/0 conn(0x7fd40c0d0260 0x7fd40c0ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 -- 172.21.15.66:0/1027822070 >> 172.21.15.66:0/1027822070 conn(0x7fd40c0d1c50 msgr2=0x7fd40c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:05.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 -- 172.21.15.66:0/1027822070 shutdown_connections 2024-09-10T22:01:05.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 -- 172.21.15.66:0/1027822070 wait complete. 2024-09-10T22:01:05.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 Processor -- start 2024-09-10T22:01:05.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 -- start start 2024-09-10T22:01:05.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40c0b2bf0 0x7fd40c14f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:05.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd40c0d0260 0x7fd40c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:05.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd40c15f340 0x7fd40c151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:05.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c1691e0 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c169060 con 0x7fd40c0d0260 2024-09-10T22:01:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd411c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd40c169360 con 0x7fd40c15f340 2024-09-10T22:01:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd40aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40c0b2bf0 0x7fd40c14f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd40aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40c0b2bf0 0x7fd40c14f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43300/0 (socket says 172.21.15.66:43300) 2024-09-10T22:01:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd40aa00640 1 -- 172.21.15.66:0/1156735223 learned_addr learned my addr 172.21.15.66:0/1156735223 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd40a000640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd40c0d0260 0x7fd40c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd40b400640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd40c15f340 0x7fd40c151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd40aa00640 1 -- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd40c15f340 msgr2=0x7fd40c151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd40aa00640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd40c15f340 0x7fd40c151900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd40aa00640 1 -- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd40c0d0260 msgr2=0x7fd40c154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd40aa00640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd40c0d0260 0x7fd40c154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd40aa00640 1 -- 172.21.15.66:0/1156735223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c14fe50 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.786+0000 7fd40b400640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd40c15f340 0x7fd40c151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:05.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.790+0000 7fd40aa00640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40c0b2bf0 0x7fd40c14f800 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc001ec0 tx=0x7fd3fc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:05.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.790+0000 7fd402a00640 1 -- 172.21.15.66:0/1156735223 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc017070 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.790+0000 7fd402a00640 1 -- 172.21.15.66:0/1156735223 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3fc003600 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.790+0000 7fd402a00640 1 -- 172.21.15.66:0/1156735223 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc003a50 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.790+0000 7fd411c00640 1 -- 172.21.15.66:0/1156735223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c155380 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.790+0000 7fd411c00640 1 -- 172.21.15.66:0/1156735223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c153ff0 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.790+0000 7fd411c00640 1 -- 172.21.15.66:0/1156735223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c0cfb50 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.790+0000 7fd402a00640 1 -- 172.21.15.66:0/1156735223 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd3fc003bf0 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.790+0000 7fd402a00640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3ec0725e0 0x7fd3ec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:05.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.790+0000 7fd402a00640 1 -- 172.21.15.66:0/1156735223 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd3fc0928c0 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.790+0000 7fd40a000640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3ec0725e0 0x7fd3ec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:05.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.794+0000 7fd40a000640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3ec0725e0 0x7fd3ec074aa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0c8c30 tx=0x7fd3f800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:05.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.794+0000 7fd402a00640 1 -- 172.21.15.66:0/1156735223 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd3fc05d8b0 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.982+0000 7fd411c00640 1 -- 172.21.15.66:0/1156735223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd40c0aff30 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.982+0000 7fd402a00640 1 -- 172.21.15.66:0/1156735223 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fd3fc0622e0 con 0x7fd40c0b2bf0 2024-09-10T22:01:05.988 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:01:05.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.986+0000 7fd411c00640 1 -- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3ec0725e0 msgr2=0x7fd3ec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:05.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.986+0000 7fd411c00640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3ec0725e0 0x7fd3ec074aa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0c8c30 tx=0x7fd3f800b390 comp rx=0 tx=0).stop 2024-09-10T22:01:05.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.986+0000 7fd411c00640 1 -- 172.21.15.66:0/1156735223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40c0b2bf0 msgr2=0x7fd40c14f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:05.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.986+0000 7fd411c00640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40c0b2bf0 0x7fd40c14f800 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc001ec0 tx=0x7fd3fc00c890 comp rx=0 tx=0).stop 2024-09-10T22:01:05.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.986+0000 7fd411c00640 1 -- 172.21.15.66:0/1156735223 shutdown_connections 2024-09-10T22:01:05.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.986+0000 7fd411c00640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3ec0725e0 0x7fd3ec074aa0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.986+0000 7fd411c00640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd40c15f340 0x7fd40c151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.986+0000 7fd411c00640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd40c0d0260 0x7fd40c154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.986+0000 7fd411c00640 1 --2- 172.21.15.66:0/1156735223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40c0b2bf0 0x7fd40c14f800 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:05.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.986+0000 7fd411c00640 1 -- 172.21.15.66:0/1156735223 >> 172.21.15.66:0/1156735223 conn(0x7fd40c0d1c50 msgr2=0x7fd40c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:05.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.986+0000 7fd411c00640 1 -- 172.21.15.66:0/1156735223 shutdown_connections 2024-09-10T22:01:05.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:05.986+0000 7fd411c00640 1 -- 172.21.15.66:0/1156735223 wait complete. 2024-09-10T22:01:06.009 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T22:01:06.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:06.009 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-10T22:01:06.032 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:01:06.032 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:01:06.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:06.032 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T22:01:06.082 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T22:01:06.082 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:01:06.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.222+0000 7fcc56800640 1 -- 172.21.15.66:0/395448662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc50074930 msgr2=0x7fcc50073be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:06.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.222+0000 7fcc56800640 1 --2- 172.21.15.66:0/395448662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc50074930 0x7fcc50073be0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009840 tx=0x7fcc3801c730 comp rx=0 tx=0).stop 2024-09-10T22:01:06.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.222+0000 7fcc56800640 1 -- 172.21.15.66:0/395448662 shutdown_connections 2024-09-10T22:01:06.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.222+0000 7fcc56800640 1 --2- 172.21.15.66:0/395448662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc50074930 0x7fcc50073be0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.222+0000 7fcc56800640 1 --2- 172.21.15.66:0/395448662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcc50070e30 0x7fcc500acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.222+0000 7fcc56800640 1 -- 172.21.15.66:0/395448662 >> 172.21.15.66:0/395448662 conn(0x7fcc500cc900 msgr2=0x7fcc500ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:06.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc56800640 1 -- 172.21.15.66:0/395448662 shutdown_connections 2024-09-10T22:01:06.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc56800640 1 -- 172.21.15.66:0/395448662 wait complete. 2024-09-10T22:01:06.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc56800640 1 Processor -- start 2024-09-10T22:01:06.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc56800640 1 -- start start 2024-09-10T22:01:06.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc56800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc50070e30 0x7fcc500c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:06.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc56800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcc50074120 0x7fcc50150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:06.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc56800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc50074930 0x7fcc501fbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:06.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc56800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcc500af5e0 con 0x7fcc50070e30 2024-09-10T22:01:06.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc56800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc500af460 con 0x7fcc50074930 2024-09-10T22:01:06.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc56800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcc5015c8b0 con 0x7fcc50074120 2024-09-10T22:01:06.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc4f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc50070e30 0x7fcc500c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:06.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc4f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc50070e30 0x7fcc500c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43322/0 (socket says 172.21.15.66:43322) 2024-09-10T22:01:06.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc4f400640 1 -- 172.21.15.66:0/3722313600 learned_addr learned my addr 172.21.15.66:0/3722313600 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc47e00640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcc50074120 0x7fcc50150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc4f400640 1 -- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcc50074120 msgr2=0x7fcc50150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc4fe00640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc50074930 0x7fcc501fbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc4f400640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcc50074120 0x7fcc50150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc4f400640 1 -- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc50074930 msgr2=0x7fcc501fbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc4f400640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc50074930 0x7fcc501fbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc4f400640 1 -- 172.21.15.66:0/3722313600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5015dac0 con 0x7fcc50070e30 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc4fe00640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc50074930 0x7fcc501fbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc4f400640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc50070e30 0x7fcc500c6110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000d950 tx=0x7fcc4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.226+0000 7fcc47400640 1 -- 172.21.15.66:0/3722313600 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40017070 con 0x7fcc50070e30 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.230+0000 7fcc47400640 1 -- 172.21.15.66:0/3722313600 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc40009d20 con 0x7fcc50070e30 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.230+0000 7fcc47400640 1 -- 172.21.15.66:0/3722313600 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc400106c0 con 0x7fcc50070e30 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.230+0000 7fcc56800640 1 -- 172.21.15.66:0/3722313600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc500b17d0 con 0x7fcc50070e30 2024-09-10T22:01:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.230+0000 7fcc56800640 1 -- 172.21.15.66:0/3722313600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc50050ff0 con 0x7fcc50070e30 2024-09-10T22:01:06.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.230+0000 7fcc56800640 1 -- 172.21.15.66:0/3722313600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc501fc360 con 0x7fcc50070e30 2024-09-10T22:01:06.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.230+0000 7fcc47400640 1 -- 172.21.15.66:0/3722313600 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcc40010860 con 0x7fcc50070e30 2024-09-10T22:01:06.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.230+0000 7fcc47400640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcc2c0725e0 0x7fcc2c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:06.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.230+0000 7fcc47400640 1 -- 172.21.15.66:0/3722313600 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcc400936e0 con 0x7fcc50070e30 2024-09-10T22:01:06.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.230+0000 7fcc47e00640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcc2c0725e0 0x7fcc2c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:06.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.234+0000 7fcc47e00640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcc2c0725e0 0x7fcc2c074aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcc380af000 tx=0x7fcc3801afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:06.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.234+0000 7fcc47400640 1 -- 172.21.15.66:0/3722313600 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcc4005edb0 con 0x7fcc50070e30 2024-09-10T22:01:06.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 -- 172.21.15.66:0/3722313600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc5015cd80 con 0x7fcc50070e30 2024-09-10T22:01:06.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc47400640 1 -- 172.21.15.66:0/3722313600 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7fcc4005e6b0 con 0x7fcc50070e30 2024-09-10T22:01:06.450 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:06.450 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:00:36:190941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2728893456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2728893456},{"type":"v1","addr":"172.21.15.94:6835","nonce":2728893456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2533443317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2533443317},{"type":"v1","addr":"172.21.15.66:6834","nonce":2533443317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/317149387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":317149387},{"type":"v1","addr":"172.21.15.66:6835","nonce":317149387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:00:32.573702+0000","modified":"2024-09-10T22:00:35.587262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/4271184466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":4271184466},{"type":"v1","addr":"172.21.15.94:6837","nonce":4271184466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-10T22:01:06.451 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:01:06.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 -- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcc2c0725e0 msgr2=0x7fcc2c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:06.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcc2c0725e0 0x7fcc2c074aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcc380af000 tx=0x7fcc3801afd0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 -- 172.21.15.66:0/3722313600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc50070e30 msgr2=0x7fcc500c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc50070e30 0x7fcc500c6110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000d950 tx=0x7fcc4000de10 comp rx=0 tx=0).stop 2024-09-10T22:01:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 -- 172.21.15.66:0/3722313600 shutdown_connections 2024-09-10T22:01:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcc2c0725e0 0x7fcc2c074aa0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc50074930 0x7fcc501fbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcc50074120 0x7fcc50150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 --2- 172.21.15.66:0/3722313600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc50070e30 0x7fcc500c6110 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 -- 172.21.15.66:0/3722313600 >> 172.21.15.66:0/3722313600 conn(0x7fcc500cc900 msgr2=0x7fcc5015d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 -- 172.21.15.66:0/3722313600 shutdown_connections 2024-09-10T22:01:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.446+0000 7fcc56800640 1 -- 172.21.15.66:0/3722313600 wait complete. 2024-09-10T22:01:06.472 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:01:06.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 -- 172.21.15.66:0/1889970280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0c4b10 msgr2=0x7fd46c06e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:06.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 --2- 172.21.15.66:0/1889970280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0c4b10 0x7fd46c06e960 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0ae5b0 tx=0x7fd46001c830 comp rx=0 tx=0).stop 2024-09-10T22:01:06.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 -- 172.21.15.66:0/1889970280 shutdown_connections 2024-09-10T22:01:06.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 --2- 172.21.15.66:0/1889970280 >> v2:172.21.15.94:3300/0 conn(0x7fd46c0c4ef0 0x7fd46c0c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 --2- 172.21.15.66:0/1889970280 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd46c0ae620 0x7fd46c0725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 --2- 172.21.15.66:0/1889970280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0c4b10 0x7fd46c06e960 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 -- 172.21.15.66:0/1889970280 >> 172.21.15.66:0/1889970280 conn(0x7fd46c0cfa30 msgr2=0x7fd46c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:06.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 -- 172.21.15.66:0/1889970280 shutdown_connections 2024-09-10T22:01:06.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 -- 172.21.15.66:0/1889970280 wait complete. 2024-09-10T22:01:06.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 Processor -- start 2024-09-10T22:01:06.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 -- start start 2024-09-10T22:01:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0ae620 0x7fd46c14d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd46c0c4b10 0x7fd46c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd46c0c4ef0 0x7fd46c1404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c166d80 con 0x7fd46c0ae620 2024-09-10T22:01:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c166c00 con 0x7fd46c0c4ef0 2024-09-10T22:01:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.614+0000 7fd473e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd46c166f00 con 0x7fd46c0c4b10 2024-09-10T22:01:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd471400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0ae620 0x7fd46c14d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd471400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0ae620 0x7fd46c14d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43348/0 (socket says 172.21.15.66:43348) 2024-09-10T22:01:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd471400640 1 -- 172.21.15.66:0/3870590049 learned_addr learned my addr 172.21.15.66:0/3870590049 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd470a00640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd46c0c4b10 0x7fd46c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd471e00640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd46c0c4ef0 0x7fd46c1404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd471400640 1 -- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd46c0c4b10 msgr2=0x7fd46c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd471400640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd46c0c4b10 0x7fd46c15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd471400640 1 -- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd46c0c4ef0 msgr2=0x7fd46c1404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd471400640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd46c0c4ef0 0x7fd46c1404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd471400640 1 -- 172.21.15.66:0/3870590049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46c0b3c20 con 0x7fd46c0ae620 2024-09-10T22:01:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd471e00640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd46c0c4ef0 0x7fd46c1404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd471400640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0ae620 0x7fd46c14d540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd4600b0040 tx=0x7fd4600a6730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd469600640 1 -- 172.21.15.66:0/3870590049 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460004340 con 0x7fd46c0ae620 2024-09-10T22:01:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd469600640 1 -- 172.21.15.66:0/3870590049 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd460005030 con 0x7fd46c0ae620 2024-09-10T22:01:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd469600640 1 -- 172.21.15.66:0/3870590049 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460005480 con 0x7fd46c0ae620 2024-09-10T22:01:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd473e00640 1 -- 172.21.15.66:0/3870590049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46c0b3f00 con 0x7fd46c0ae620 2024-09-10T22:01:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd473e00640 1 -- 172.21.15.66:0/3870590049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd46c0b44c0 con 0x7fd46c0ae620 2024-09-10T22:01:06.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd473e00640 1 -- 172.21.15.66:0/3870590049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd46c06e960 con 0x7fd46c0ae620 2024-09-10T22:01:06.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.618+0000 7fd469600640 1 -- 172.21.15.66:0/3870590049 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd4600046d0 con 0x7fd46c0ae620 2024-09-10T22:01:06.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.622+0000 7fd469600640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd44c0725e0 0x7fd44c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:06.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.622+0000 7fd469600640 1 -- 172.21.15.66:0/3870590049 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd46012dad0 con 0x7fd46c0ae620 2024-09-10T22:01:06.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.622+0000 7fd470a00640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd44c0725e0 0x7fd44c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:06.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.622+0000 7fd470a00640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd44c0725e0 0x7fd44c074aa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd46c156510 tx=0x7fd45800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:06.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.626+0000 7fd469600640 1 -- 172.21.15.66:0/3870590049 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd4600164b0 con 0x7fd46c0ae620 2024-09-10T22:01:06.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.834+0000 7fd473e00640 1 -- 172.21.15.66:0/3870590049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd46c072fc0 con 0x7fd46c0ae620 2024-09-10T22:01:06.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.834+0000 7fd469600640 1 -- 172.21.15.66:0/3870590049 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7fd4600f91a0 con 0x7fd46c0ae620 2024-09-10T22:01:06.839 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:06.839 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:00:36:190941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2728893456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2728893456},{"type":"v1","addr":"172.21.15.94:6835","nonce":2728893456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2533443317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2533443317},{"type":"v1","addr":"172.21.15.66:6834","nonce":2533443317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/317149387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":317149387},{"type":"v1","addr":"172.21.15.66:6835","nonce":317149387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:00:32.573702+0000","modified":"2024-09-10T22:00:35.587262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/4271184466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":4271184466},{"type":"v1","addr":"172.21.15.94:6837","nonce":4271184466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-10T22:01:06.839 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:01:06.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.838+0000 7fd473e00640 1 -- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd44c0725e0 msgr2=0x7fd44c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:06.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.838+0000 7fd473e00640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd44c0725e0 0x7fd44c074aa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd46c156510 tx=0x7fd45800b2e0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.838+0000 7fd473e00640 1 -- 172.21.15.66:0/3870590049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0ae620 msgr2=0x7fd46c14d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.838+0000 7fd473e00640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0ae620 0x7fd46c14d540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd4600b0040 tx=0x7fd4600a6730 comp rx=0 tx=0).stop 2024-09-10T22:01:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.838+0000 7fd473e00640 1 -- 172.21.15.66:0/3870590049 shutdown_connections 2024-09-10T22:01:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.838+0000 7fd473e00640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd44c0725e0 0x7fd44c074aa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.838+0000 7fd473e00640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd46c0c4ef0 0x7fd46c1404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.838+0000 7fd473e00640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd46c0c4b10 0x7fd46c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.838+0000 7fd473e00640 1 --2- 172.21.15.66:0/3870590049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0ae620 0x7fd46c14d540 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.838+0000 7fd473e00640 1 -- 172.21.15.66:0/3870590049 >> 172.21.15.66:0/3870590049 conn(0x7fd46c0cfa30 msgr2=0x7fd46c1567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.838+0000 7fd473e00640 1 -- 172.21.15.66:0/3870590049 shutdown_connections 2024-09-10T22:01:06.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:06.838+0000 7fd473e00640 1 -- 172.21.15.66:0/3870590049 wait complete. 2024-09-10T22:01:06.860 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:01:07.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.006+0000 7f6309e00640 1 -- 172.21.15.66:0/1214360026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e8003a60 msgr2=0x7f62e8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.006+0000 7f6309e00640 1 --2- 172.21.15.66:0/1214360026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e8003a60 0x7f62e8003e80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f62f40016c0 tx=0x7f62f401c730 comp rx=0 tx=0).stop 2024-09-10T22:01:07.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.006+0000 7f6309e00640 1 -- 172.21.15.66:0/1214360026 shutdown_connections 2024-09-10T22:01:07.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.006+0000 7f6309e00640 1 --2- 172.21.15.66:0/1214360026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e8003a60 0x7f62e8003e80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.006+0000 7f6309e00640 1 -- 172.21.15.66:0/1214360026 >> 172.21.15.66:0/1214360026 conn(0x7f63040d1c60 msgr2=0x7f63040d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:07.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.006+0000 7f6309e00640 1 -- 172.21.15.66:0/1214360026 shutdown_connections 2024-09-10T22:01:07.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.006+0000 7f6309e00640 1 -- 172.21.15.66:0/1214360026 wait complete. 2024-09-10T22:01:07.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.006+0000 7f6309e00640 1 Processor -- start 2024-09-10T22:01:07.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.006+0000 7f6309e00640 1 -- start start 2024-09-10T22:01:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6309e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e8003a60 0x7f630406ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6309e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6304159970 0x7f630406f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6309e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63041fb400 0x7f63041fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6309e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f63040ce170 con 0x7f62e8003a60 2024-09-10T22:01:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6309e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f63040af930 con 0x7f63041fb400 2024-09-10T22:01:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6309e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f63040ce2f0 con 0x7f6304159970 2024-09-10T22:01:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6302a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e8003a60 0x7f630406ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6302a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e8003a60 0x7f630406ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43358/0 (socket says 172.21.15.66:43358) 2024-09-10T22:01:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6302a00640 1 -- 172.21.15.66:0/274930350 learned_addr learned my addr 172.21.15.66:0/274930350 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:07.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6303400640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63041fb400 0x7f63041fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:07.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f62fa000640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6304159970 0x7f630406f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:07.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6302a00640 1 -- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6304159970 msgr2=0x7f630406f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6302a00640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6304159970 0x7f630406f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6302a00640 1 -- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63041fb400 msgr2=0x7f63041fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6302a00640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63041fb400 0x7f63041fb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6302a00640 1 -- 172.21.15.66:0/274930350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6304159ea0 con 0x7f62e8003a60 2024-09-10T22:01:07.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6303400640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63041fb400 0x7f63041fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:07.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6302a00640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e8003a60 0x7f630406ed70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f62f4006040 tx=0x7f62f4003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:07.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f62fbe00640 1 -- 172.21.15.66:0/274930350 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f40ad070 con 0x7f62e8003a60 2024-09-10T22:01:07.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f62fbe00640 1 -- 172.21.15.66:0/274930350 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62f4006500 con 0x7f62e8003a60 2024-09-10T22:01:07.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6309e00640 1 -- 172.21.15.66:0/274930350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63041f80d0 con 0x7f62e8003a60 2024-09-10T22:01:07.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f62fbe00640 1 -- 172.21.15.66:0/274930350 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4001310 con 0x7f62e8003a60 2024-09-10T22:01:07.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6309e00640 1 -- 172.21.15.66:0/274930350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63041f8690 con 0x7f62e8003a60 2024-09-10T22:01:07.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.010+0000 7f6309e00640 1 -- 172.21.15.66:0/274930350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f630406f7f0 con 0x7f62e8003a60 2024-09-10T22:01:07.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.014+0000 7f62fbe00640 1 -- 172.21.15.66:0/274930350 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f62f40b3070 con 0x7f62e8003a60 2024-09-10T22:01:07.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.014+0000 7f62fbe00640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f62e0072610 0x7f62e0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:07.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.014+0000 7f62fbe00640 1 -- 172.21.15.66:0/274930350 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f62f412cd40 con 0x7f62e8003a60 2024-09-10T22:01:07.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.014+0000 7f62fa000640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f62e0072610 0x7f62e0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:07.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.014+0000 7f62fa000640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f62e0072610 0x7f62e0074ad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f62ec000fc0 tx=0x7f62ec0067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:07.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.018+0000 7f62fbe00640 1 -- 172.21.15.66:0/274930350 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f62f4016490 con 0x7f62e8003a60 2024-09-10T22:01:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 -- 172.21.15.66:0/274930350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63040b0df0 con 0x7f62e8003a60 2024-09-10T22:01:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f62fbe00640 1 -- 172.21.15.66:0/274930350 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16593 (secure 0 0 0) 0x7f62f40f8410 con 0x7f62e8003a60 2024-09-10T22:01:07.198 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:07.198 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":22,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:34.173529+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:30.545203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:31.538729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:01:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 -- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f62e0072610 msgr2=0x7f62e0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f62e0072610 0x7f62e0074ad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f62ec000fc0 tx=0x7f62ec0067b0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 -- 172.21.15.66:0/274930350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e8003a60 msgr2=0x7f630406ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e8003a60 0x7f630406ed70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f62f4006040 tx=0x7f62f4003be0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 -- 172.21.15.66:0/274930350 shutdown_connections 2024-09-10T22:01:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f62e0072610 0x7f62e0074ad0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63041fb400 0x7f63041fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6304159970 0x7f630406f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 --2- 172.21.15.66:0/274930350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e8003a60 0x7f630406ed70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 -- 172.21.15.66:0/274930350 >> 172.21.15.66:0/274930350 conn(0x7f63040d1c60 msgr2=0x7f63040b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 -- 172.21.15.66:0/274930350 shutdown_connections 2024-09-10T22:01:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.194+0000 7f6309e00640 1 -- 172.21.15.66:0/274930350 wait complete. 2024-09-10T22:01:07.271 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:01:07.272 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:01:07.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 -- 172.21.15.66:0/1058358303 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd3015f300 msgr2=0x7fcd300b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 --2- 172.21.15.66:0/1058358303 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd3015f300 0x7fcd300b1090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009870 tx=0x7fcd1c01c780 comp rx=0 tx=0).stop 2024-09-10T22:01:07.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 -- 172.21.15.66:0/1058358303 shutdown_connections 2024-09-10T22:01:07.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 --2- 172.21.15.66:0/1058358303 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd3015f300 0x7fcd300b1090 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 -- 172.21.15.66:0/1058358303 >> 172.21.15.66:0/1058358303 conn(0x7fcd300d1b10 msgr2=0x7fcd300d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:07.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 -- 172.21.15.66:0/1058358303 shutdown_connections 2024-09-10T22:01:07.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 -- 172.21.15.66:0/1058358303 wait complete. 2024-09-10T22:01:07.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 Processor -- start 2024-09-10T22:01:07.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 -- start start 2024-09-10T22:01:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd300b15d0 0x7fcd3015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd3015ef20 0x7fcd300cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3015f300 0x7fcd3006f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3016a100 con 0x7fcd3015f300 2024-09-10T22:01:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd30169f80 con 0x7fcd3015ef20 2024-09-10T22:01:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.414+0000 7fcd2f200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcd3016a280 con 0x7fcd300b15d0 2024-09-10T22:01:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3015f300 0x7fcd3006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3015f300 0x7fcd3006f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43360/0 (socket says 172.21.15.66:43360) 2024-09-10T22:01:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2de00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd300b15d0 0x7fcd3015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2e800640 1 -- 172.21.15.66:0/2611355272 learned_addr learned my addr 172.21.15.66:0/2611355272 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2d400640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd3015ef20 0x7fcd300cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2e800640 1 -- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd300b15d0 msgr2=0x7fcd3015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2e800640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd300b15d0 0x7fcd3015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2e800640 1 -- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd3015ef20 msgr2=0x7fcd300cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2e800640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd3015ef20 0x7fcd300cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2e800640 1 -- 172.21.15.66:0/2611355272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd300cb330 con 0x7fcd3015f300 2024-09-10T22:01:07.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2de00640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd300b15d0 0x7fcd3015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:01:07.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2e800640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3015f300 0x7fcd3006f840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800c7a0 tx=0x7fcd2800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:07.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd26000640 1 -- 172.21.15.66:0/2611355272 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28003b80 con 0x7fcd3015f300 2024-09-10T22:01:07.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2f200640 1 -- 172.21.15.66:0/2611355272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd300cb610 con 0x7fcd3015f300 2024-09-10T22:01:07.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd26000640 1 -- 172.21.15.66:0/2611355272 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd28003d40 con 0x7fcd3015f300 2024-09-10T22:01:07.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd26000640 1 -- 172.21.15.66:0/2611355272 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2800aab0 con 0x7fcd3015f300 2024-09-10T22:01:07.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2f200640 1 -- 172.21.15.66:0/2611355272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3006e260 con 0x7fcd3015f300 2024-09-10T22:01:07.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd2f200640 1 -- 172.21.15.66:0/2611355272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd300b1090 con 0x7fcd3015f300 2024-09-10T22:01:07.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.418+0000 7fcd26000640 1 -- 172.21.15.66:0/2611355272 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcd2800ac50 con 0x7fcd3015f300 2024-09-10T22:01:07.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.422+0000 7fcd26000640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd100725e0 0x7fcd10074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:07.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.422+0000 7fcd26000640 1 -- 172.21.15.66:0/2611355272 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcd28092ff0 con 0x7fcd3015f300 2024-09-10T22:01:07.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.422+0000 7fcd2de00640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd100725e0 0x7fcd10074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:07.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.422+0000 7fcd2de00640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd100725e0 0x7fcd10074aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcd20001230 tx=0x7fcd20001760 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.426+0000 7fcd26000640 1 -- 172.21.15.66:0/2611355272 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcd2805e030 con 0x7fcd3015f300 2024-09-10T22:01:07.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 -- 172.21.15.66:0/2611355272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd3006e430 con 0x7fcd3015f300 2024-09-10T22:01:07.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd26000640 1 -- 172.21.15.66:0/2611355272 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7fcd28062a60 con 0x7fcd3015f300 2024-09-10T22:01:07.638 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:07.638 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:00:36:190941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2728893456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2728893456},{"type":"v1","addr":"172.21.15.94:6835","nonce":2728893456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2533443317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2533443317},{"type":"v1","addr":"172.21.15.66:6834","nonce":2533443317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/317149387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":317149387},{"type":"v1","addr":"172.21.15.66:6835","nonce":317149387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:00:32.573702+0000","modified":"2024-09-10T22:00:35.587262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/4271184466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":4271184466},{"type":"v1","addr":"172.21.15.94:6837","nonce":4271184466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-10T22:01:07.638 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:01:07.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 -- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd100725e0 msgr2=0x7fcd10074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd100725e0 0x7fcd10074aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcd20001230 tx=0x7fcd20001760 comp rx=0 tx=0).stop 2024-09-10T22:01:07.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 -- 172.21.15.66:0/2611355272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3015f300 msgr2=0x7fcd3006f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3015f300 0x7fcd3006f840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800c7a0 tx=0x7fcd2800cc60 comp rx=0 tx=0).stop 2024-09-10T22:01:07.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 -- 172.21.15.66:0/2611355272 shutdown_connections 2024-09-10T22:01:07.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd100725e0 0x7fcd10074aa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3015f300 0x7fcd3006f840 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd3015ef20 0x7fcd300cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 --2- 172.21.15.66:0/2611355272 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd300b15d0 0x7fcd3015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 -- 172.21.15.66:0/2611355272 >> 172.21.15.66:0/2611355272 conn(0x7fcd300d1b10 msgr2=0x7fcd3015ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:07.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 -- 172.21.15.66:0/2611355272 shutdown_connections 2024-09-10T22:01:07.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.634+0000 7fcd2f200640 1 -- 172.21.15.66:0/2611355272 wait complete. 2024-09-10T22:01:07.660 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:01:07.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.802+0000 7f666d400640 1 -- 172.21.15.66:0/1570419010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f664c003a90 msgr2=0x7f664c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.802+0000 7f666d400640 1 --2- 172.21.15.66:0/1570419010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f664c003a90 0x7f664c003eb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6658006f90 tx=0x7f665801cee0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.802+0000 7f6667200640 1 -- 172.21.15.66:0/1570419010 reap_dead start 2024-09-10T22:01:07.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.802+0000 7f666d400640 1 -- 172.21.15.66:0/1570419010 shutdown_connections 2024-09-10T22:01:07.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.802+0000 7f666d400640 1 -- 172.21.15.66:0/1570419010 >> 172.21.15.66:0/1570419010 conn(0x7f66680cfa30 msgr2=0x7f66680cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:07.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.802+0000 7f666d400640 1 -- 172.21.15.66:0/1570419010 shutdown_connections 2024-09-10T22:01:07.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.802+0000 7f666d400640 1 -- 172.21.15.66:0/1570419010 wait complete. 2024-09-10T22:01:07.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f666d400640 1 Processor -- start 2024-09-10T22:01:07.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f666d400640 1 -- start start 2024-09-10T22:01:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f666d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f664c003a90 0x7f6668150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f666d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6668158020 0x7f66680ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f666d400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66681584a0 0x7f66680ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f666d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f666815a520 con 0x7f664c003a90 2024-09-10T22:01:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f666d400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f666806ed40 con 0x7f6668158020 2024-09-10T22:01:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f666d400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f666815a6a0 con 0x7f66681584a0 2024-09-10T22:01:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f6666800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f664c003a90 0x7f6668150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f6666800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f664c003a90 0x7f6668150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43374/0 (socket says 172.21.15.66:43374) 2024-09-10T22:01:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f6666800640 1 -- 172.21.15.66:0/3218268768 learned_addr learned my addr 172.21.15.66:0/3218268768 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f6667200640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66681584a0 0x7f66680ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f665de00640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6668158020 0x7f66680ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f6666800640 1 -- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66681584a0 msgr2=0x7f66680ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f6666800640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66681584a0 0x7f66680ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f6666800640 1 -- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6668158020 msgr2=0x7f66680ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f6666800640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6668158020 0x7f66680ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f6666800640 1 -- 172.21.15.66:0/3218268768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6668150b70 con 0x7f664c003a90 2024-09-10T22:01:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f665de00640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6668158020 0x7f66680ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f6666800640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f664c003a90 0x7f6668150630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6658006040 tx=0x7f6658006c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f665f400640 1 -- 172.21.15.66:0/3218268768 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580a5b70 con 0x7f664c003a90 2024-09-10T22:01:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f665f400640 1 -- 172.21.15.66:0/3218268768 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66580014c0 con 0x7f664c003a90 2024-09-10T22:01:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f665f400640 1 -- 172.21.15.66:0/3218268768 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580060d0 con 0x7f664c003a90 2024-09-10T22:01:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f666d400640 1 -- 172.21.15.66:0/3218268768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66681fbea0 con 0x7f664c003a90 2024-09-10T22:01:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f666d400640 1 -- 172.21.15.66:0/3218268768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66680736f0 con 0x7f664c003a90 2024-09-10T22:01:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.806+0000 7f666d400640 1 -- 172.21.15.66:0/3218268768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66680c48b0 con 0x7f664c003a90 2024-09-10T22:01:07.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.810+0000 7f665f400640 1 -- 172.21.15.66:0/3218268768 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6658006270 con 0x7f664c003a90 2024-09-10T22:01:07.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.810+0000 7f665f400640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6644072610 0x7f6644074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:07.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.810+0000 7f665f400640 1 -- 172.21.15.66:0/3218268768 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f665812c7d0 con 0x7f664c003a90 2024-09-10T22:01:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.810+0000 7f665de00640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6644072610 0x7f6644074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.810+0000 7f665de00640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6644072610 0x7f6644074ad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f66680b1340 tx=0x7f66500091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:07.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:07.814+0000 7f665f400640 1 -- 172.21.15.66:0/3218268768 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6658016c90 con 0x7f664c003a90 2024-09-10T22:01:08.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.022+0000 7f666d400640 1 -- 172.21.15.66:0/3218268768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66680cb000 con 0x7f664c003a90 2024-09-10T22:01:08.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f665f400640 1 -- 172.21.15.66:0/3218268768 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f66580f7f40 con 0x7f664c003a90 2024-09-10T22:01:08.029 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:08.029 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:00:36:190941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2728893456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2728893456},{"type":"v1","addr":"172.21.15.94:6835","nonce":2728893456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2533443317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2533443317},{"type":"v1","addr":"172.21.15.66:6834","nonce":2533443317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/317149387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":317149387},{"type":"v1","addr":"172.21.15.66:6835","nonce":317149387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:00:32.573702+0000","modified":"2024-09-10T22:00:35.587262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/4271184466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":4271184466},{"type":"v1","addr":"172.21.15.94:6837","nonce":4271184466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-10T22:01:08.029 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:01:08.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f666d400640 1 -- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6644072610 msgr2=0x7f6644074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f666d400640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6644072610 0x7f6644074ad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f66680b1340 tx=0x7f66500091e0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f666d400640 1 -- 172.21.15.66:0/3218268768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f664c003a90 msgr2=0x7f6668150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f666d400640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f664c003a90 0x7f6668150630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6658006040 tx=0x7f6658006c50 comp rx=0 tx=0).stop 2024-09-10T22:01:08.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f666d400640 1 -- 172.21.15.66:0/3218268768 shutdown_connections 2024-09-10T22:01:08.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f666d400640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6644072610 0x7f6644074ad0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f666d400640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66681584a0 0x7f66680ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f666d400640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6668158020 0x7f66680ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f666d400640 1 --2- 172.21.15.66:0/3218268768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f664c003a90 0x7f6668150630 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f666d400640 1 -- 172.21.15.66:0/3218268768 >> 172.21.15.66:0/3218268768 conn(0x7f66680cfa30 msgr2=0x7f6668156db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:08.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f666d400640 1 -- 172.21.15.66:0/3218268768 shutdown_connections 2024-09-10T22:01:08.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.026+0000 7f666d400640 1 -- 172.21.15.66:0/3218268768 wait complete. 2024-09-10T22:01:08.052 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:01:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.194+0000 7f333e400640 1 -- 172.21.15.66:0/2520173188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333815f2c0 msgr2=0x7f333815ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.194+0000 7f333e400640 1 --2- 172.21.15.66:0/2520173188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333815f2c0 0x7f333815ea90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3324009900 tx=0x7f332401c850 comp rx=0 tx=0).stop 2024-09-10T22:01:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.194+0000 7f333e400640 1 -- 172.21.15.66:0/2520173188 shutdown_connections 2024-09-10T22:01:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.194+0000 7f333e400640 1 --2- 172.21.15.66:0/2520173188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333815f2c0 0x7f333815ea90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.194+0000 7f333e400640 1 --2- 172.21.15.66:0/2520173188 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f333815e6b0 0x7f3338068990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.194+0000 7f333e400640 1 -- 172.21.15.66:0/2520173188 >> 172.21.15.66:0/2520173188 conn(0x7f3338073440 msgr2=0x7f33380b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.194+0000 7f333e400640 1 -- 172.21.15.66:0/2520173188 shutdown_connections 2024-09-10T22:01:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.194+0000 7f333e400640 1 -- 172.21.15.66:0/2520173188 wait complete. 2024-09-10T22:01:08.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f333e400640 1 Processor -- start 2024-09-10T22:01:08.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f333e400640 1 -- start start 2024-09-10T22:01:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f333e400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f333815b0c0 0x7f333809de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f333e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f333815e6b0 0x7f333809c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f333e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333815f2c0 0x7f33381f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f333e400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3338168640 con 0x7f333815e6b0 2024-09-10T22:01:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f333e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33381684c0 con 0x7f333815f2c0 2024-09-10T22:01:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f333e400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f33381687c0 con 0x7f333815b0c0 2024-09-10T22:01:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f3336a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f333815e6b0 0x7f333809c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f3336a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f333815e6b0 0x7f333809c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43390/0 (socket says 172.21.15.66:43390) 2024-09-10T22:01:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f3336a00640 1 -- 172.21.15.66:0/3090675821 learned_addr learned my addr 172.21.15.66:0/3090675821 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f3337e00640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333815f2c0 0x7f33381f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f3337400640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f333815b0c0 0x7f333809de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f3336a00640 1 -- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f333815b0c0 msgr2=0x7f333809de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f3336a00640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f333815b0c0 0x7f333809de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f3336a00640 1 -- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333815f2c0 msgr2=0x7f33381f7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f3336a00640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333815f2c0 0x7f33381f7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f3336a00640 1 -- 172.21.15.66:0/3090675821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333815a700 con 0x7f333815e6b0 2024-09-10T22:01:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f3336a00640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f333815e6b0 0x7f333809c050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3324009850 tx=0x7f3324005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f332f400640 1 -- 172.21.15.66:0/3090675821 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33240b2040 con 0x7f333815e6b0 2024-09-10T22:01:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f332f400640 1 -- 172.21.15.66:0/3090675821 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3324004040 con 0x7f333815e6b0 2024-09-10T22:01:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f332f400640 1 -- 172.21.15.66:0/3090675821 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33240044b0 con 0x7f333815e6b0 2024-09-10T22:01:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f333e400640 1 -- 172.21.15.66:0/3090675821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333815a9a0 con 0x7f333815e6b0 2024-09-10T22:01:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f333e400640 1 -- 172.21.15.66:0/3090675821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3338150bc0 con 0x7f333815e6b0 2024-09-10T22:01:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.198+0000 7f333e400640 1 -- 172.21.15.66:0/3090675821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3338068990 con 0x7f333815e6b0 2024-09-10T22:01:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.202+0000 7f332f400640 1 -- 172.21.15.66:0/3090675821 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3324003600 con 0x7f333815e6b0 2024-09-10T22:01:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.202+0000 7f332f400640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3318072610 0x7f3318074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.202+0000 7f332f400640 1 -- 172.21.15.66:0/3090675821 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f332412d380 con 0x7f333815e6b0 2024-09-10T22:01:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.202+0000 7f3337400640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3318072610 0x7f3318074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.202+0000 7f3337400640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3318072610 0x7f3318074ad0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3328009600 tx=0x7f33280091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.206+0000 7f332f400640 1 -- 172.21.15.66:0/3090675821 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f33240c6020 con 0x7f333815e6b0 2024-09-10T22:01:08.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 -- 172.21.15.66:0/3090675821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3338069000 con 0x7f333815e6b0 2024-09-10T22:01:08.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f332f400640 1 -- 172.21.15.66:0/3090675821 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16593 (secure 0 0 0) 0x7f33240164d0 con 0x7f333815e6b0 2024-09-10T22:01:08.402 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:08.402 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":22,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:00:34.173529+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:30.545203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:31.538729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:01:08.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 -- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3318072610 msgr2=0x7f3318074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3318072610 0x7f3318074ad0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3328009600 tx=0x7f33280091e0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 -- 172.21.15.66:0/3090675821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f333815e6b0 msgr2=0x7f333809c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f333815e6b0 0x7f333809c050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3324009850 tx=0x7f3324005760 comp rx=0 tx=0).stop 2024-09-10T22:01:08.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 -- 172.21.15.66:0/3090675821 shutdown_connections 2024-09-10T22:01:08.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3318072610 0x7f3318074ad0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333815f2c0 0x7f33381f7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f333815e6b0 0x7f333809c050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 --2- 172.21.15.66:0/3090675821 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f333815b0c0 0x7f333809de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 -- 172.21.15.66:0/3090675821 >> 172.21.15.66:0/3090675821 conn(0x7f3338073440 msgr2=0x7f333815b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:08.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 -- 172.21.15.66:0/3090675821 shutdown_connections 2024-09-10T22:01:08.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.398+0000 7f333e400640 1 -- 172.21.15.66:0/3090675821 wait complete. 2024-09-10T22:01:08.425 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:01:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.566+0000 7fbf87800640 1 -- 172.21.15.66:0/2069399527 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf8015f460 msgr2=0x7fbf800cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.566+0000 7fbf87800640 1 --2- 172.21.15.66:0/2069399527 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf8015f460 0x7fbf800cfc00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009840 tx=0x7fbf7401c6d0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.566+0000 7fbf87800640 1 -- 172.21.15.66:0/2069399527 shutdown_connections 2024-09-10T22:01:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.566+0000 7fbf87800640 1 --2- 172.21.15.66:0/2069399527 >> v2:172.21.15.94:3300/0 conn(0x7fbf800b32a0 0x7fbf800af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.566+0000 7fbf87800640 1 --2- 172.21.15.66:0/2069399527 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf8015f460 0x7fbf800cfc00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.566+0000 7fbf87800640 1 -- 172.21.15.66:0/2069399527 >> 172.21.15.66:0/2069399527 conn(0x7fbf800d1d00 msgr2=0x7fbf800d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.566+0000 7fbf87800640 1 -- 172.21.15.66:0/2069399527 shutdown_connections 2024-09-10T22:01:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.566+0000 7fbf87800640 1 -- 172.21.15.66:0/2069399527 wait complete. 2024-09-10T22:01:08.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf87800640 1 Processor -- start 2024-09-10T22:01:08.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf87800640 1 -- start start 2024-09-10T22:01:08.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf87800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf800b0f60 0x7fbf801496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:08.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf87800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf800b32a0 0x7fbf801519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:08.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf87800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf8015f460 0x7fbf80151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:08.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf87800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf800b05b0 con 0x7fbf8015f460 2024-09-10T22:01:08.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf87800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbf800b0430 con 0x7fbf800b0f60 2024-09-10T22:01:08.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf87800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbf80168c60 con 0x7fbf800b32a0 2024-09-10T22:01:08.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf85800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf8015f460 0x7fbf80151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:08.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf85800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf8015f460 0x7fbf80151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43402/0 (socket says 172.21.15.66:43402) 2024-09-10T22:01:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf85800640 1 -- 172.21.15.66:0/2227854986 learned_addr learned my addr 172.21.15.66:0/2227854986 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf84e00640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf800b0f60 0x7fbf801496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf7fe00640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf800b32a0 0x7fbf801519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf85800640 1 -- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf800b32a0 msgr2=0x7fbf801519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf85800640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf800b32a0 0x7fbf801519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf85800640 1 -- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf800b0f60 msgr2=0x7fbf801496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf85800640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf800b0f60 0x7fbf801496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf85800640 1 -- 172.21.15.66:0/2227854986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf800cd710 con 0x7fbf8015f460 2024-09-10T22:01:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf84e00640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf800b0f60 0x7fbf801496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf85800640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf8015f460 0x7fbf80151f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800ef30 tx=0x7fbf7800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:08.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf7cc00640 1 -- 172.21.15.66:0/2227854986 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78004b30 con 0x7fbf8015f460 2024-09-10T22:01:08.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf7cc00640 1 -- 172.21.15.66:0/2227854986 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf78004dc0 con 0x7fbf8015f460 2024-09-10T22:01:08.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf87800640 1 -- 172.21.15.66:0/2227854986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8015c940 con 0x7fbf8015f460 2024-09-10T22:01:08.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.570+0000 7fbf7cc00640 1 -- 172.21.15.66:0/2227854986 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7800b7d0 con 0x7fbf8015f460 2024-09-10T22:01:08.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.574+0000 7fbf87800640 1 -- 172.21.15.66:0/2227854986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf8015cf00 con 0x7fbf8015f460 2024-09-10T22:01:08.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.574+0000 7fbf87800640 1 -- 172.21.15.66:0/2227854986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf800af390 con 0x7fbf8015f460 2024-09-10T22:01:08.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.574+0000 7fbf7cc00640 1 -- 172.21.15.66:0/2227854986 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbf7800b040 con 0x7fbf8015f460 2024-09-10T22:01:08.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.574+0000 7fbf7cc00640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbf600725e0 0x7fbf60074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:08.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.574+0000 7fbf7cc00640 1 -- 172.21.15.66:0/2227854986 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbf780939c0 con 0x7fbf8015f460 2024-09-10T22:01:08.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.574+0000 7fbf84e00640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbf600725e0 0x7fbf60074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:08.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.578+0000 7fbf84e00640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbf600725e0 0x7fbf60074aa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbf740af000 tx=0x7fbf7401af10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:08.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.578+0000 7fbf7cc00640 1 -- 172.21.15.66:0/2227854986 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbf7805ef80 con 0x7fbf8015f460 2024-09-10T22:01:08.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.786+0000 7fbf87800640 1 -- 172.21.15.66:0/2227854986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbf800b25b0 con 0x7fbf8015f460 2024-09-10T22:01:08.791 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:01:08.786+0000 7fcc93a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:01:08.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf7cc00640 1 -- 172.21.15.66:0/2227854986 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fbf7805e880 con 0x7fbf8015f460 2024-09-10T22:01:08.793 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:01:08.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf87800640 1 -- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbf600725e0 msgr2=0x7fbf60074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf87800640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbf600725e0 0x7fbf60074aa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbf740af000 tx=0x7fbf7401af10 comp rx=0 tx=0).stop 2024-09-10T22:01:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf87800640 1 -- 172.21.15.66:0/2227854986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf8015f460 msgr2=0x7fbf80151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf87800640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf8015f460 0x7fbf80151f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800ef30 tx=0x7fbf7800c800 comp rx=0 tx=0).stop 2024-09-10T22:01:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf87800640 1 -- 172.21.15.66:0/2227854986 shutdown_connections 2024-09-10T22:01:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf87800640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbf600725e0 0x7fbf60074aa0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf87800640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf8015f460 0x7fbf80151f10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf87800640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf800b32a0 0x7fbf801519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf87800640 1 --2- 172.21.15.66:0/2227854986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf800b0f60 0x7fbf801496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf87800640 1 -- 172.21.15.66:0/2227854986 >> 172.21.15.66:0/2227854986 conn(0x7fbf800d1d00 msgr2=0x7fbf8006e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf87800640 1 -- 172.21.15.66:0/2227854986 shutdown_connections 2024-09-10T22:01:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.790+0000 7fbf87800640 1 -- 172.21.15.66:0/2227854986 wait complete. 2024-09-10T22:01:08.804 INFO:tasks.ceph.mds.b.smithi094.stderr: -2298> 2024-09-10T22:00:38.145+0000 7fe5bfe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:01:08.816 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:01:08.851 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:01:08.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.958+0000 7eff01000640 1 -- 172.21.15.66:0/1094361320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc15bfb0 msgr2=0x7efefc0ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.958+0000 7eff01000640 1 --2- 172.21.15.66:0/1094361320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc15bfb0 0x7efefc0ccd40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efefc15c510 tx=0x7efeec01c760 comp rx=0 tx=0).stop 2024-09-10T22:01:08.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.958+0000 7eff01000640 1 -- 172.21.15.66:0/1094361320 shutdown_connections 2024-09-10T22:01:08.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.958+0000 7eff01000640 1 --2- 172.21.15.66:0/1094361320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc15bfb0 0x7efefc0ccd40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.958+0000 7eff01000640 1 -- 172.21.15.66:0/1094361320 >> 172.21.15.66:0/1094361320 conn(0x7efefc074240 msgr2=0x7efefc074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:08.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.958+0000 7eff01000640 1 -- 172.21.15.66:0/1094361320 shutdown_connections 2024-09-10T22:01:08.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.958+0000 7eff01000640 1 -- 172.21.15.66:0/1094361320 wait complete. 2024-09-10T22:01:08.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7eff01000640 1 Processor -- start 2024-09-10T22:01:08.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7eff01000640 1 -- start start 2024-09-10T22:01:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7eff01000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efefc0b0590 0x7efefc1fb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7eff01000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efefc0cff10 0x7efefc0af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7eff01000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc15bfb0 0x7efefc0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7eff01000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efefc16a6c0 con 0x7efefc0b0590 2024-09-10T22:01:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7eff01000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efefc0cdba0 con 0x7efefc15bfb0 2024-09-10T22:01:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7eff01000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7efefc16a840 con 0x7efefc0cff10 2024-09-10T22:01:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efefc0b0590 0x7efefc1fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efefc0b0590 0x7efefc1fb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43408/0 (socket says 172.21.15.66:43408) 2024-09-10T22:01:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef9e00640 1 -- 172.21.15.66:0/516101826 learned_addr learned my addr 172.21.15.66:0/516101826 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef9400640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efefc0cff10 0x7efefc0af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efefa800640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc15bfb0 0x7efefc0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:08.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef9e00640 1 -- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efefc0cff10 msgr2=0x7efefc0af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef9e00640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efefc0cff10 0x7efefc0af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef9e00640 1 -- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc15bfb0 msgr2=0x7efefc0ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:08.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef9e00640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc15bfb0 0x7efefc0ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:08.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef9e00640 1 -- 172.21.15.66:0/516101826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc1fbc30 con 0x7efefc0b0590 2024-09-10T22:01:08.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef9400640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efefc0cff10 0x7efefc0af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:08.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef9e00640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efefc0b0590 0x7efefc1fb360 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efeec019370 tx=0x7efeec0a5f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:08.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef2000640 1 -- 172.21.15.66:0/516101826 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec0b2040 con 0x7efefc0b0590 2024-09-10T22:01:08.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7eff01000640 1 -- 172.21.15.66:0/516101826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efefc1fbf10 con 0x7efefc0b0590 2024-09-10T22:01:08.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.962+0000 7efef2000640 1 -- 172.21.15.66:0/516101826 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efeec0bb070 con 0x7efefc0b0590 2024-09-10T22:01:08.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.966+0000 7efef2000640 1 -- 172.21.15.66:0/516101826 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec003600 con 0x7efefc0b0590 2024-09-10T22:01:08.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.966+0000 7eff01000640 1 -- 172.21.15.66:0/516101826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efefc1fc4d0 con 0x7efefc0b0590 2024-09-10T22:01:08.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.966+0000 7eff01000640 1 -- 172.21.15.66:0/516101826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efefc0d0340 con 0x7efefc0b0590 2024-09-10T22:01:08.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.966+0000 7efef2000640 1 -- 172.21.15.66:0/516101826 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efeec0b5070 con 0x7efefc0b0590 2024-09-10T22:01:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.966+0000 7efef2000640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efedc0725e0 0x7efedc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.966+0000 7efef2000640 1 -- 172.21.15.66:0/516101826 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7efeec12d200 con 0x7efefc0b0590 2024-09-10T22:01:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.966+0000 7efef9400640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efedc0725e0 0x7efedc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.970+0000 7efef9400640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efedc0725e0 0x7efedc074aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efefc0926d0 tx=0x7efee8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:08.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:08.970+0000 7efef2000640 1 -- 172.21.15.66:0/516101826 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efeec016410 con 0x7efefc0b0590 2024-09-10T22:01:09.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.182+0000 7eff01000640 1 -- 172.21.15.66:0/516101826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7efefc092ca0 con 0x7efefc0b0590 2024-09-10T22:01:09.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.790+0000 7efef2000640 1 -- 172.21.15.66:0/516101826 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7efeec0f8760 con 0x7efefc0b0590 2024-09-10T22:01:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.794+0000 7eff01000640 1 -- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efedc0725e0 msgr2=0x7efedc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.794+0000 7eff01000640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efedc0725e0 0x7efedc074aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efefc0926d0 tx=0x7efee8009040 comp rx=0 tx=0).stop 2024-09-10T22:01:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.794+0000 7eff01000640 1 -- 172.21.15.66:0/516101826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efefc0b0590 msgr2=0x7efefc1fb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.794+0000 7eff01000640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efefc0b0590 0x7efefc1fb360 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efeec019370 tx=0x7efeec0a5f90 comp rx=0 tx=0).stop 2024-09-10T22:01:09.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.794+0000 7eff01000640 1 -- 172.21.15.66:0/516101826 shutdown_connections 2024-09-10T22:01:09.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.794+0000 7eff01000640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efedc0725e0 0x7efedc074aa0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.794+0000 7eff01000640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc15bfb0 0x7efefc0ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.794+0000 7eff01000640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efefc0cff10 0x7efefc0af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.794+0000 7eff01000640 1 --2- 172.21.15.66:0/516101826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efefc0b0590 0x7efefc1fb360 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.794+0000 7eff01000640 1 -- 172.21.15.66:0/516101826 >> 172.21.15.66:0/516101826 conn(0x7efefc074240 msgr2=0x7efefc0cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:09.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.794+0000 7eff01000640 1 -- 172.21.15.66:0/516101826 shutdown_connections 2024-09-10T22:01:09.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.794+0000 7eff01000640 1 -- 172.21.15.66:0/516101826 wait complete. 2024-09-10T22:01:09.821 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:01:09.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.970+0000 7ff5cca00640 1 -- 172.21.15.66:0/2677328921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c80b0e20 msgr2=0x7ff5c8160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:09.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.970+0000 7ff5cca00640 1 --2- 172.21.15.66:0/2677328921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c80b0e20 0x7ff5c8160460 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009c80 tx=0x7ff5c001cda0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.970+0000 7ff5cca00640 1 -- 172.21.15.66:0/2677328921 shutdown_connections 2024-09-10T22:01:09.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.970+0000 7ff5cca00640 1 --2- 172.21.15.66:0/2677328921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c80b0e20 0x7ff5c8160460 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.970+0000 7ff5cca00640 1 -- 172.21.15.66:0/2677328921 >> 172.21.15.66:0/2677328921 conn(0x7ff5c80d1c60 msgr2=0x7ff5c80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:09.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.970+0000 7ff5cca00640 1 -- 172.21.15.66:0/2677328921 shutdown_connections 2024-09-10T22:01:09.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.970+0000 7ff5cca00640 1 -- 172.21.15.66:0/2677328921 wait complete. 2024-09-10T22:01:09.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5cca00640 1 Processor -- start 2024-09-10T22:01:09.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5cca00640 1 -- start start 2024-09-10T22:01:09.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5cca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b0e20 0x7ff5c8149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:09.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5cca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c80cdf90 0x7ff5c80b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:09.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5cca00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c80ce8d0 0x7ff5c80b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:09.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5cca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8169e90 con 0x7ff5c80cdf90 2024-09-10T22:01:09.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5cca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8169d10 con 0x7ff5c80b0e20 2024-09-10T22:01:09.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5cca00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c816a010 con 0x7ff5c80ce8d0 2024-09-10T22:01:09.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5bfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c80cdf90 0x7ff5c80b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:09.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5bfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c80cdf90 0x7ff5c80b5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43414/0 (socket says 172.21.15.66:43414) 2024-09-10T22:01:09.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5bfe00640 1 -- 172.21.15.66:0/3570464549 learned_addr learned my addr 172.21.15.66:0/3570464549 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:09.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5c6800640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c80ce8d0 0x7ff5c80b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:09.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5bfe00640 1 -- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c80ce8d0 msgr2=0x7ff5c80b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:09.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5c5e00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b0e20 0x7ff5c8149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:09.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5bfe00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c80ce8d0 0x7ff5c80b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5bfe00640 1 -- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b0e20 msgr2=0x7ff5c8149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:09.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5bfe00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b0e20 0x7ff5c8149f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5bfe00640 1 -- 172.21.15.66:0/3570464549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c80b6370 con 0x7ff5c80cdf90 2024-09-10T22:01:09.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5c6800640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c80ce8d0 0x7ff5c80b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:09.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5c5e00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b0e20 0x7ff5c8149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:09.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5bfe00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c80cdf90 0x7ff5c80b5b20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000c410 tx=0x7ff5b000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:09.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5be000640 1 -- 172.21.15.66:0/3570464549 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0017070 con 0x7ff5c80cdf90 2024-09-10T22:01:09.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5be000640 1 -- 172.21.15.66:0/3570464549 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5b0003600 con 0x7ff5c80cdf90 2024-09-10T22:01:09.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5be000640 1 -- 172.21.15.66:0/3570464549 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0003a90 con 0x7ff5c80cdf90 2024-09-10T22:01:09.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5cca00640 1 -- 172.21.15.66:0/3570464549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c80b6650 con 0x7ff5c80cdf90 2024-09-10T22:01:09.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5cca00640 1 -- 172.21.15.66:0/3570464549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c80b6c10 con 0x7ff5c80cdf90 2024-09-10T22:01:09.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.974+0000 7ff5cca00640 1 -- 172.21.15.66:0/3570464549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c80cecb0 con 0x7ff5c80cdf90 2024-09-10T22:01:09.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.978+0000 7ff5be000640 1 -- 172.21.15.66:0/3570464549 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff5b0003c30 con 0x7ff5c80cdf90 2024-09-10T22:01:09.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.978+0000 7ff5be000640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff5a40725e0 0x7ff5a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:09.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.978+0000 7ff5be000640 1 -- 172.21.15.66:0/3570464549 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7ff5b0092890 con 0x7ff5c80cdf90 2024-09-10T22:01:09.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.978+0000 7ff5c5e00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff5a40725e0 0x7ff5a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:09.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.978+0000 7ff5c5e00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff5a40725e0 0x7ff5a4074aa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff5c806ff10 tx=0x7ff5b8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:09.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:09.982+0000 7ff5be000640 1 -- 172.21.15.66:0/3570464549 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff5b005df00 con 0x7ff5c80cdf90 2024-09-10T22:01:10.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.158+0000 7ff5cca00640 1 -- 172.21.15.66:0/3570464549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff5c8070550 con 0x7ff5c80cdf90 2024-09-10T22:01:10.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.798+0000 7ff5be000640 1 -- 172.21.15.66:0/3570464549 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7ff5b005d800 con 0x7ff5c80cdf90 2024-09-10T22:01:10.802 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:01:10.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.802+0000 7ff5cca00640 1 -- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff5a40725e0 msgr2=0x7ff5a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:10.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.802+0000 7ff5cca00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff5a40725e0 0x7ff5a4074aa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff5c806ff10 tx=0x7ff5b8009040 comp rx=0 tx=0).stop 2024-09-10T22:01:10.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.802+0000 7ff5cca00640 1 -- 172.21.15.66:0/3570464549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c80cdf90 msgr2=0x7ff5c80b5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:10.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.802+0000 7ff5cca00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c80cdf90 0x7ff5c80b5b20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000c410 tx=0x7ff5b000c8d0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.802+0000 7ff5cca00640 1 -- 172.21.15.66:0/3570464549 shutdown_connections 2024-09-10T22:01:10.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.802+0000 7ff5cca00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff5a40725e0 0x7ff5a4074aa0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.802+0000 7ff5cca00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c80ce8d0 0x7ff5c80b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.802+0000 7ff5cca00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c80cdf90 0x7ff5c80b5b20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.802+0000 7ff5cca00640 1 --2- 172.21.15.66:0/3570464549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c80b0e20 0x7ff5c8149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.802+0000 7ff5cca00640 1 -- 172.21.15.66:0/3570464549 >> 172.21.15.66:0/3570464549 conn(0x7ff5c80d1c60 msgr2=0x7ff5c80cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:10.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.802+0000 7ff5cca00640 1 -- 172.21.15.66:0/3570464549 shutdown_connections 2024-09-10T22:01:10.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.802+0000 7ff5cca00640 1 -- 172.21.15.66:0/3570464549 wait complete. 2024-09-10T22:01:10.831 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:01:10.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.982+0000 7f34c9a00640 1 -- 172.21.15.66:0/3814295227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c415bd40 msgr2=0x7f34c40742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:10.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.982+0000 7f34c9a00640 1 --2- 172.21.15.66:0/3814295227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c415bd40 0x7f34c40742d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009900 tx=0x7f34b401c8a0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.982+0000 7f34c9a00640 1 -- 172.21.15.66:0/3814295227 shutdown_connections 2024-09-10T22:01:10.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.982+0000 7f34c9a00640 1 --2- 172.21.15.66:0/3814295227 >> v2:172.21.15.94:3300/0 conn(0x7f34c4074c50 0x7f34c40cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.982+0000 7f34c9a00640 1 --2- 172.21.15.66:0/3814295227 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34c4074810 0x7f34c40cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.982+0000 7f34c9a00640 1 --2- 172.21.15.66:0/3814295227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c415bd40 0x7f34c40742d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.982+0000 7f34c9a00640 1 -- 172.21.15.66:0/3814295227 >> 172.21.15.66:0/3814295227 conn(0x7f34c40d0350 msgr2=0x7f34c40d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:10.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.982+0000 7f34c9a00640 1 -- 172.21.15.66:0/3814295227 shutdown_connections 2024-09-10T22:01:10.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.982+0000 7f34c9a00640 1 -- 172.21.15.66:0/3814295227 wait complete. 2024-09-10T22:01:10.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.982+0000 7f34c9a00640 1 Processor -- start 2024-09-10T22:01:10.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.982+0000 7f34c9a00640 1 -- start start 2024-09-10T22:01:10.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34c9a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34c4074810 0x7f34c40c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:10.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34c9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c4074c50 0x7f34c415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:10.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34c9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c415bd40 0x7f34c415eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:10.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34c9a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34c40aeee0 con 0x7f34c4074c50 2024-09-10T22:01:10.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34c9a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34c40aed60 con 0x7f34c415bd40 2024-09-10T22:01:10.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34c9a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f34c40af060 con 0x7f34c4074810 2024-09-10T22:01:10.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c4074c50 0x7f34c415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:10.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c4074c50 0x7f34c415e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43426/0 (socket says 172.21.15.66:43426) 2024-09-10T22:01:10.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34bbe00640 1 -- 172.21.15.66:0/1931596748 learned_addr learned my addr 172.21.15.66:0/1931596748 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:10.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34c2a00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34c4074810 0x7f34c40c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:10.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34c3400640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c415bd40 0x7f34c415eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:10.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34bbe00640 1 -- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34c4074810 msgr2=0x7f34c40c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:10.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34bbe00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34c4074810 0x7f34c40c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34bbe00640 1 -- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c415bd40 msgr2=0x7f34c415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:10.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34bbe00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c415bd40 0x7f34c415eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34bbe00640 1 -- 172.21.15.66:0/1931596748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c41fb820 con 0x7f34c4074c50 2024-09-10T22:01:10.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34bbe00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c4074c50 0x7f34c415e5a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00d750 tx=0x7f34ac00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:10.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34baa00640 1 -- 172.21.15.66:0/1931596748 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac017020 con 0x7f34c4074c50 2024-09-10T22:01:10.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34baa00640 1 -- 172.21.15.66:0/1931596748 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34ac0097c0 con 0x7f34c4074c50 2024-09-10T22:01:10.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34baa00640 1 -- 172.21.15.66:0/1931596748 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00de60 con 0x7f34c4074c50 2024-09-10T22:01:10.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34c9a00640 1 -- 172.21.15.66:0/1931596748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c41fbb00 con 0x7f34c4074c50 2024-09-10T22:01:10.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34c9a00640 1 -- 172.21.15.66:0/1931596748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c40afbe0 con 0x7f34c4074c50 2024-09-10T22:01:10.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.986+0000 7f34c9a00640 1 -- 172.21.15.66:0/1931596748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c40af570 con 0x7f34c4074c50 2024-09-10T22:01:10.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.990+0000 7f34baa00640 1 -- 172.21.15.66:0/1931596748 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f34ac009c10 con 0x7f34c4074c50 2024-09-10T22:01:10.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.990+0000 7f34baa00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f34a00725e0 0x7f34a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:10.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.990+0000 7f34baa00640 1 -- 172.21.15.66:0/1931596748 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5357+0+0 (secure 0 0 0) 0x7f34ac093760 con 0x7f34c4074c50 2024-09-10T22:01:10.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.990+0000 7f34c2a00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f34a00725e0 0x7f34a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:10.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.990+0000 7f34c2a00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f34a00725e0 0x7f34a0074aa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f34b40af000 tx=0x7f34b401b0e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:10.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:10.994+0000 7f34baa00640 1 -- 172.21.15.66:0/1931596748 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f34ac021020 con 0x7f34c4074c50 2024-09-10T22:01:11.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.166+0000 7f34c9a00640 1 -- 172.21.15.66:0/1931596748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f34c40ad620 con 0x7f34c4074c50 2024-09-10T22:01:11.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.798+0000 7f34baa00640 1 -- 172.21.15.66:0/1931596748 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f34ac093020 con 0x7f34c4074c50 2024-09-10T22:01:11.804 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:01:11.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.802+0000 7f34c9a00640 1 -- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f34a00725e0 msgr2=0x7f34a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:11.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.802+0000 7f34c9a00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f34a00725e0 0x7f34a0074aa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f34b40af000 tx=0x7f34b401b0e0 comp rx=0 tx=0).stop 2024-09-10T22:01:11.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.802+0000 7f34c9a00640 1 -- 172.21.15.66:0/1931596748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c4074c50 msgr2=0x7f34c415e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:11.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.802+0000 7f34c9a00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c4074c50 0x7f34c415e5a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00d750 tx=0x7f34ac00dc10 comp rx=0 tx=0).stop 2024-09-10T22:01:11.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.802+0000 7f34c9a00640 1 -- 172.21.15.66:0/1931596748 shutdown_connections 2024-09-10T22:01:11.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.802+0000 7f34c9a00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f34a00725e0 0x7f34a0074aa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:11.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.802+0000 7f34c9a00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c415bd40 0x7f34c415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:11.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.802+0000 7f34c9a00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c4074c50 0x7f34c415e5a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:11.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.802+0000 7f34c9a00640 1 --2- 172.21.15.66:0/1931596748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34c4074810 0x7f34c40c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:11.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.802+0000 7f34c9a00640 1 -- 172.21.15.66:0/1931596748 >> 172.21.15.66:0/1931596748 conn(0x7f34c40d0350 msgr2=0x7f34c41592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:11.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.802+0000 7f34c9a00640 1 -- 172.21.15.66:0/1931596748 shutdown_connections 2024-09-10T22:01:11.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:11.806+0000 7f34c9a00640 1 -- 172.21.15.66:0/1931596748 wait complete. 2024-09-10T22:01:11.833 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:01:11.833 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:01:11.833 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:01:11.833 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:01:11.834 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:01:11.834 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:01:11.834 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:01:11.834 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:01:11.834 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:01:11.834 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:01:11.834 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:01:11.834 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:01:11.835 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:01:11.835 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:01:11.835 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:01:11.835 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:01:11.848 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:01:11.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:01:11.849 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:01:11.850 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:01:11.850 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:01:11.851 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:01:11.851 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:01:11.852 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:01:11.896 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:01:11.896 DEBUG:teuthology.parallel:result is None 2024-09-10T22:01:11.897 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:01:11.898 DEBUG:teuthology.parallel:result is None 2024-09-10T22:01:11.899 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:01:11.899 DEBUG:teuthology.parallel:result is None 2024-09-10T22:01:11.901 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:01:11.901 DEBUG:teuthology.parallel:result is None 2024-09-10T22:01:11.901 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T22:01:11.983 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:01:11.983 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:01:11.987 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:01:11.988 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:01:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.046+0000 7fe803600640 1 -- 172.21.15.66:0/920978090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0cf830 msgr2=0x7fe7fc0d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.046+0000 7fe803600640 1 --2- 172.21.15.66:0/920978090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0cf830 0x7fe7fc0d04e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009900 tx=0x7fe7e801c850 comp rx=0 tx=0).stop 2024-09-10T22:01:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 -- 172.21.15.66:0/920978090 shutdown_connections 2024-09-10T22:01:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 --2- 172.21.15.66:0/920978090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0cf830 0x7fe7fc0d04e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 --2- 172.21.15.66:0/920978090 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc15bd40 0x7fe7fc0cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 -- 172.21.15.66:0/920978090 >> 172.21.15.66:0/920978090 conn(0x7fe7fc0d1c60 msgr2=0x7fe7fc0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 -- 172.21.15.66:0/920978090 shutdown_connections 2024-09-10T22:01:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 -- 172.21.15.66:0/920978090 wait complete. 2024-09-10T22:01:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 Processor -- start 2024-09-10T22:01:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 -- start start 2024-09-10T22:01:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7fc0aec10 0x7fe7fc1f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc0cf830 0x7fe7fc159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc15bd40 0x7fe7fc15ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1690c0 con 0x7fe7fc15bd40 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc168f40 con 0x7fe7fc0aec10 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe7fc169240 con 0x7fe7fc0cf830 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe801600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc15bd40 0x7fe7fc15ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe7fbe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc0cf830 0x7fe7fc159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe800c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7fc0aec10 0x7fe7fc1f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe7fbe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc0cf830 0x7fe7fc159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:58224/0 (socket says 172.21.15.66:58224) 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe7fbe00640 1 -- 172.21.15.66:0/3974008253 learned_addr learned my addr 172.21.15.66:0/3974008253 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe7fbe00640 1 -- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7fc0aec10 msgr2=0x7fe7fc1f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe7fbe00640 1 --2- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7fc0aec10 0x7fe7fc1f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe7fbe00640 1 -- 172.21.15.66:0/3974008253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc15bd40 msgr2=0x7fe7fc15ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe7fbe00640 1 --2- 172.21.15.66:0/3974008253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc15bd40 0x7fe7fc15ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe7fbe00640 1 -- 172.21.15.66:0/3974008253 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7fc1f8580 con 0x7fe7fc0cf830 2024-09-10T22:01:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe7fbe00640 1 --2- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc0cf830 0x7fe7fc159fb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009850 tx=0x7fe7e80a56a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe7f8c00640 1 -- 172.21.15.66:0/3974008253 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80b2040 con 0x7fe7fc0cf830 2024-09-10T22:01:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe7f8c00640 1 -- 172.21.15.66:0/3974008253 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7e8004380 con 0x7fe7fc0cf830 2024-09-10T22:01:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe7f8c00640 1 -- 172.21.15.66:0/3974008253 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8005290 con 0x7fe7fc0cf830 2024-09-10T22:01:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.050+0000 7fe803600640 1 -- 172.21.15.66:0/3974008253 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7fc1f8860 con 0x7fe7fc0cf830 2024-09-10T22:01:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.054+0000 7fe803600640 1 -- 172.21.15.66:0/3974008253 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7fc1f8e20 con 0x7fe7fc0cf830 2024-09-10T22:01:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.054+0000 7fe803600640 1 -- 172.21.15.66:0/3974008253 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7fc0af040 con 0x7fe7fc0cf830 2024-09-10T22:01:12.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.054+0000 7fe7f8c00640 1 -- 172.21.15.66:0/3974008253 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe7e8005430 con 0x7fe7fc0cf830 2024-09-10T22:01:12.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.054+0000 7fe7f8c00640 1 --2- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7dc072610 0x7fe7dc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:12.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.054+0000 7fe7f8c00640 1 -- 172.21.15.66:0/3974008253 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7fe7e812c770 con 0x7fe7fc0cf830 2024-09-10T22:01:12.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.054+0000 7fe800c00640 1 --2- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7dc072610 0x7fe7dc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.058+0000 7fe800c00640 1 --2- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7dc072610 0x7fe7dc074ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009b40 tx=0x7fe7f00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.058+0000 7fe7f8c00640 1 -- 172.21.15.66:0/3974008253 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe7e80164d0 con 0x7fe7fc0cf830 2024-09-10T22:01:12.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.242+0000 7fe803600640 1 -- 172.21.15.66:0/3974008253 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe7fc0b0df0 con 0x7fe7fc0cf830 2024-09-10T22:01:12.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe7f8c00640 1 -- 172.21.15.66:0/3974008253 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7fe7e80f7af0 con 0x7fe7fc0cf830 2024-09-10T22:01:12.826 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-09-10T22:01:12.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe803600640 1 -- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7dc072610 msgr2=0x7fe7dc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:12.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe803600640 1 --2- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7dc072610 0x7fe7dc074ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009b40 tx=0x7fe7f00091e0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe803600640 1 -- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc0cf830 msgr2=0x7fe7fc159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:12.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe803600640 1 --2- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc0cf830 0x7fe7fc159fb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009850 tx=0x7fe7e80a56a0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe803600640 1 -- 172.21.15.66:0/3974008253 shutdown_connections 2024-09-10T22:01:12.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe803600640 1 --2- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7dc072610 0x7fe7dc074ad0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe803600640 1 --2- 172.21.15.66:0/3974008253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc15bd40 0x7fe7fc15ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe803600640 1 --2- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc0cf830 0x7fe7fc159fb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe803600640 1 --2- 172.21.15.66:0/3974008253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7fc0aec10 0x7fe7fc1f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe803600640 1 -- 172.21.15.66:0/3974008253 >> 172.21.15.66:0/3974008253 conn(0x7fe7fc0d1c60 msgr2=0x7fe7fc165b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:12.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe803600640 1 -- 172.21.15.66:0/3974008253 shutdown_connections 2024-09-10T22:01:12.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.822+0000 7fe803600640 1 -- 172.21.15.66:0/3974008253 wait complete. 2024-09-10T22:01:12.850 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T22:01:12.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 -- 172.21.15.66:0/1543252754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd6815dd50 msgr2=0x7fdd680c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 --2- 172.21.15.66:0/1543252754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd6815dd50 0x7fdd680c48b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdd6815b510 tx=0x7fdd5801d230 comp rx=0 tx=0).stop 2024-09-10T22:01:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 -- 172.21.15.66:0/1543252754 shutdown_connections 2024-09-10T22:01:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 --2- 172.21.15.66:0/1543252754 >> v2:172.21.15.94:3300/0 conn(0x7fdd6815e130 0x7fdd680c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 --2- 172.21.15.66:0/1543252754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd6815dd50 0x7fdd680c48b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 -- 172.21.15.66:0/1543252754 >> 172.21.15.66:0/1543252754 conn(0x7fdd680cfa30 msgr2=0x7fdd680cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 -- 172.21.15.66:0/1543252754 shutdown_connections 2024-09-10T22:01:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 -- 172.21.15.66:0/1543252754 wait complete. 2024-09-10T22:01:12.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 Processor -- start 2024-09-10T22:01:12.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 -- start start 2024-09-10T22:01:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd680706c0 0x7fdd680728e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6815dd50 0x7fdd681411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd6815e130 0x7fdd68150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdd68167fe0 con 0x7fdd680706c0 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd68167e60 con 0x7fdd6815dd50 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd6f200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdd68168160 con 0x7fdd6815e130 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd67e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd680706c0 0x7fdd680728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd67e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd680706c0 0x7fdd680728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41614/0 (socket says 172.21.15.66:41614) 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.994+0000 7fdd67e00640 1 -- 172.21.15.66:0/2320275597 learned_addr learned my addr 172.21.15.66:0/2320275597 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd5fe00640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6815dd50 0x7fdd681411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd6d200640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd6815e130 0x7fdd68150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd67e00640 1 -- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd6815e130 msgr2=0x7fdd68150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd67e00640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd6815e130 0x7fdd68150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd67e00640 1 -- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6815dd50 msgr2=0x7fdd681411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd67e00640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6815dd50 0x7fdd681411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd67e00640 1 -- 172.21.15.66:0/2320275597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd680738c0 con 0x7fdd680706c0 2024-09-10T22:01:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd5fe00640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6815dd50 0x7fdd681411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd6d200640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd6815e130 0x7fdd68150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd67e00640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd680706c0 0x7fdd680728e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd58000c10 tx=0x7fdd58004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd64c00640 1 -- 172.21.15.66:0/2320275597 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd580bc070 con 0x7fdd680706c0 2024-09-10T22:01:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd64c00640 1 -- 172.21.15.66:0/2320275597 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd58003600 con 0x7fdd680706c0 2024-09-10T22:01:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd64c00640 1 -- 172.21.15.66:0/2320275597 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd580b2040 con 0x7fdd680706c0 2024-09-10T22:01:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd6f200640 1 -- 172.21.15.66:0/2320275597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd680c88c0 con 0x7fdd680706c0 2024-09-10T22:01:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd6f200640 1 -- 172.21.15.66:0/2320275597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd680c8e20 con 0x7fdd680706c0 2024-09-10T22:01:13.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd6f200640 1 -- 172.21.15.66:0/2320275597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd6815c3b0 con 0x7fdd680706c0 2024-09-10T22:01:13.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd64c00640 1 -- 172.21.15.66:0/2320275597 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdd580b5070 con 0x7fdd680706c0 2024-09-10T22:01:13.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:12.998+0000 7fdd64c00640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdd440725e0 0x7fdd44074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:13.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.002+0000 7fdd64c00640 1 -- 172.21.15.66:0/2320275597 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fdd5812c820 con 0x7fdd680706c0 2024-09-10T22:01:13.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.002+0000 7fdd5fe00640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdd440725e0 0x7fdd44074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:13.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.002+0000 7fdd5fe00640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdd440725e0 0x7fdd44074aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdd680cb3e0 tx=0x7fdd50009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.002+0000 7fdd64c00640 1 -- 172.21.15.66:0/2320275597 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdd680c88c0 con 0x7fdd680706c0 2024-09-10T22:01:13.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 -- 172.21.15.66:0/2320275597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fdd68076650 con 0x7fdd680706c0 2024-09-10T22:01:13.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd64c00640 1 -- 172.21.15.66:0/2320275597 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+6645 (secure 0 0 0) 0x7fdd680738c0 con 0x7fdd680706c0 2024-09-10T22:01:13.250 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:13.250 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:01:13.250 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmxQD7CBAAttZP0KtdjTI7QWbD7CVinQ==", 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-09-10T22:01:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmI7ufFRAANelhvqylKyfGPVfBuY40kg==", 2024-09-10T22:01:13.252 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.252 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:01:13.252 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:01:13.252 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:01:13.252 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.252 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.252 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.252 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.252 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-09-10T22:01:13.252 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQnVoEBAAvsZ/TSOOP/JgJBr/aHwI/g==", 2024-09-10T22:01:13.252 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.252 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmRQkTHRAAHPIdwqtkkyh31D7IO2vxPA==", 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.253 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBCweBmDJkxJBAAVSQNiMAQl5bR3RUwVOuGYw==", 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBFweBmwcdsOxAAPPQyzktcfiF8pJNP8uAjwg==", 2024-09-10T22:01:13.254 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.255 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:01:13.255 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:01:13.255 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.255 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.255 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.255 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-09-10T22:01:13.255 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBJweBmXQhZExAAnEf8mjGX6nnYwWygKjo/2A==", 2024-09-10T22:01:13.255 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.255 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:01:13.255 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:01:13.255 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBNweBmlauHAxAA/k9xRqbHZsqJq9OYbJKfNQ==", 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.256 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBRweBmHnFDJRAAgF1ZAPvmoNzeUftZQyL0eg==", 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBUweBm/hXpORAA6jpOI/I+QyB2mZsEoAWF0w==", 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:01:13.257 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBYweBmYu2MExAAPD9WcxrBLzwwJQm19z7YcQ==", 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.258 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbweBmXfJ9KRAA/AbWlli+Qfg7gy0SblJ7fg==", 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw==", 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:01:13.259 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmbS6IKhAAaEmK9irxMipALgkAct/+OQ==", 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.260 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQ47KJhAAI50ZaSQIhH6FxRM48JZICA==", 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmUcLCMBAAdf4tfcpVaZoaKjS1UzrxhA==", 2024-09-10T22:01:13.261 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA+weBmPIAzKBAAkyVV875PNhBmEoKsrAkg8w==", 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:01:13.262 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-mds", 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBmweBmGszyNxAAJYzo/q2pGiiuVjjA2PpfWQ==", 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-mds" 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-mgr", 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBmweBmYTrzNxAAi0GCy0TrdKgRbEP1hyNuaQ==", 2024-09-10T22:01:13.263 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.264 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-10T22:01:13.264 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.264 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.264 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-osd", 2024-09-10T22:01:13.264 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBmweBmZqjzNxAAdlrDwDRkpH7/XoTd1wRnNw==", 2024-09-10T22:01:13.264 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.264 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-osd" 2024-09-10T22:01:13.264 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.264 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.264 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-rbd", 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBmweBmpcPzNxAASa4/wncDfjzpWLqnSoQ/Ug==", 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBmweBmjNHzNxAAhzdcZdqqrM9NHWmCyHDcig==", 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.265 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-rgw", 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBmweBmreDzNxAAJkGStXUFMQS0YzPBrm/qwA==", 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmLiy1BBAADnraML8IUet7Sw0j83TUVQ==", 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.266 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.267 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:13.267 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:13.267 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-09-10T22:01:13.267 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmItWpABAAlJ+6dFjvtBV3g+jYmUh0tQ==", 2024-09-10T22:01:13.267 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:01:13.267 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:01:13.267 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:01:13.267 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:01:13.267 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.267 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:13.267 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T22:01:13.267 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:01:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 -- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdd440725e0 msgr2=0x7fdd44074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdd440725e0 0x7fdd44074aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdd680cb3e0 tx=0x7fdd50009040 comp rx=0 tx=0).stop 2024-09-10T22:01:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 -- 172.21.15.66:0/2320275597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd680706c0 msgr2=0x7fdd680728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd680706c0 0x7fdd680728e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd58000c10 tx=0x7fdd58004040 comp rx=0 tx=0).stop 2024-09-10T22:01:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 -- 172.21.15.66:0/2320275597 shutdown_connections 2024-09-10T22:01:13.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdd440725e0 0x7fdd44074aa0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd6815e130 0x7fdd68150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6815dd50 0x7fdd681411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 --2- 172.21.15.66:0/2320275597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd680706c0 0x7fdd680728e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 -- 172.21.15.66:0/2320275597 >> 172.21.15.66:0/2320275597 conn(0x7fdd680cfa30 msgr2=0x7fdd6815c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:13.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 -- 172.21.15.66:0/2320275597 shutdown_connections 2024-09-10T22:01:13.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.246+0000 7fdd6f200640 1 -- 172.21.15.66:0/2320275597 wait complete. 2024-09-10T22:01:13.272 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-10T22:01:13.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.414+0000 7f488ca00640 1 -- 172.21.15.66:0/3154420071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f488815ef20 msgr2=0x7f48880cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:13.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.414+0000 7f488ca00640 1 --2- 172.21.15.66:0/3154420071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f488815ef20 0x7f48880cecc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4878009870 tx=0x7f487801c820 comp rx=0 tx=0).stop 2024-09-10T22:01:13.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.414+0000 7f488ca00640 1 -- 172.21.15.66:0/3154420071 shutdown_connections 2024-09-10T22:01:13.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.414+0000 7f488ca00640 1 --2- 172.21.15.66:0/3154420071 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48880cd7a0 0x7f48880cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.414+0000 7f488ca00640 1 --2- 172.21.15.66:0/3154420071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f488815ef20 0x7f48880cecc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.414+0000 7f488ca00640 1 -- 172.21.15.66:0/3154420071 >> 172.21.15.66:0/3154420071 conn(0x7f48880d1b10 msgr2=0x7f48880d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:13.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.414+0000 7f488ca00640 1 -- 172.21.15.66:0/3154420071 shutdown_connections 2024-09-10T22:01:13.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.414+0000 7f488ca00640 1 -- 172.21.15.66:0/3154420071 wait complete. 2024-09-10T22:01:13.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f488ca00640 1 Processor -- start 2024-09-10T22:01:13.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f488ca00640 1 -- start start 2024-09-10T22:01:13.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f488ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48880cd7a0 0x7f4888159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:13.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f488ca00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f488815ef20 0x7f48880c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:13.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f488ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f488815f300 0x7f488806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:13.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f488ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4888168d60 con 0x7f488815f300 2024-09-10T22:01:13.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f488ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48880cf390 con 0x7f48880cd7a0 2024-09-10T22:01:13.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f488ca00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4888168ee0 con 0x7f488815ef20 2024-09-10T22:01:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f4887200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f488815f300 0x7f488806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f4886800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48880cd7a0 0x7f4888159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f4887200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f488815f300 0x7f488806e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41634/0 (socket says 172.21.15.66:41634) 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f487fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f488815ef20 0x7f48880c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f4887200640 1 -- 172.21.15.66:0/1760117602 learned_addr learned my addr 172.21.15.66:0/1760117602 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f4886800640 1 -- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f488815ef20 msgr2=0x7f48880c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f4886800640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f488815ef20 0x7f48880c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f4886800640 1 -- 172.21.15.66:0/1760117602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f488815f300 msgr2=0x7f488806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f4886800640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f488815f300 0x7f488806e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f4886800640 1 -- 172.21.15.66:0/1760117602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4888164090 con 0x7f48880cd7a0 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f4887200640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f488815f300 0x7f488806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f4886800640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48880cd7a0 0x7f4888159480 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f48780ac000 tx=0x7f4878005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f487ea00640 1 -- 172.21.15.66:0/1760117602 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48780b2040 con 0x7f48880cd7a0 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f487ea00640 1 -- 172.21.15.66:0/1760117602 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4878004330 con 0x7f48880cd7a0 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f487ea00640 1 -- 172.21.15.66:0/1760117602 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878003820 con 0x7f48880cd7a0 2024-09-10T22:01:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f488ca00640 1 -- 172.21.15.66:0/1760117602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488806ddd0 con 0x7f48880cd7a0 2024-09-10T22:01:13.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f488ca00640 1 -- 172.21.15.66:0/1760117602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f488806e390 con 0x7f48880cd7a0 2024-09-10T22:01:13.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.418+0000 7f488ca00640 1 -- 172.21.15.66:0/1760117602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48880cdc30 con 0x7f48880cd7a0 2024-09-10T22:01:13.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.422+0000 7f487ea00640 1 -- 172.21.15.66:0/1760117602 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f48780b5070 con 0x7f48880cd7a0 2024-09-10T22:01:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.422+0000 7f487ea00640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f48640725e0 0x7f4864074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.422+0000 7f487ea00640 1 -- 172.21.15.66:0/1760117602 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f487812c4a0 con 0x7f48880cd7a0 2024-09-10T22:01:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.422+0000 7f487fe00640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f48640725e0 0x7f4864074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.422+0000 7f487fe00640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f48640725e0 0x7f4864074aa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f48880cbea0 tx=0x7f4870001380 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:13.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.426+0000 7f487ea00640 1 -- 172.21.15.66:0/1760117602 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4878016410 con 0x7f48880cd7a0 2024-09-10T22:01:13.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.662+0000 7f488ca00640 1 -- 172.21.15.66:0/1760117602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f4888072e00 con 0x7f48880cd7a0 2024-09-10T22:01:13.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f487ea00640 1 -- 172.21.15.66:0/1760117602 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v19) ==== 90+0+0 (secure 0 0 0) 0x7f48780f7860 con 0x7f48880cd7a0 2024-09-10T22:01:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f488ca00640 1 -- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f48640725e0 msgr2=0x7f4864074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f488ca00640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f48640725e0 0x7f4864074aa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f48880cbea0 tx=0x7f4870001380 comp rx=0 tx=0).stop 2024-09-10T22:01:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f488ca00640 1 -- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48880cd7a0 msgr2=0x7f4888159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f488ca00640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48880cd7a0 0x7f4888159480 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f48780ac000 tx=0x7f4878005db0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f488ca00640 1 -- 172.21.15.66:0/1760117602 shutdown_connections 2024-09-10T22:01:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f488ca00640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f48640725e0 0x7f4864074aa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f488ca00640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f488815f300 0x7f488806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f488ca00640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f488815ef20 0x7f48880c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f488ca00640 1 --2- 172.21.15.66:0/1760117602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48880cd7a0 0x7f4888159480 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f488ca00640 1 -- 172.21.15.66:0/1760117602 >> 172.21.15.66:0/1760117602 conn(0x7f48880d1b10 msgr2=0x7f48880b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:13.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f488ca00640 1 -- 172.21.15.66:0/1760117602 shutdown_connections 2024-09-10T22:01:13.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.670+0000 7f488ca00640 1 -- 172.21.15.66:0/1760117602 wait complete. 2024-09-10T22:01:13.695 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-10T22:01:13.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.842+0000 7fbd98c00640 1 -- 172.21.15.66:0/2882543605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd94160280 msgr2=0x7fbd940b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:13.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.842+0000 7fbd98c00640 1 --2- 172.21.15.66:0/2882543605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd94160280 0x7fbd940b3170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009840 tx=0x7fbd7c01d1c0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.842+0000 7fbd98c00640 1 -- 172.21.15.66:0/2882543605 shutdown_connections 2024-09-10T22:01:13.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.842+0000 7fbd98c00640 1 --2- 172.21.15.66:0/2882543605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd94160280 0x7fbd940b3170 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.842+0000 7fbd98c00640 1 -- 172.21.15.66:0/2882543605 >> 172.21.15.66:0/2882543605 conn(0x7fbd940d1c60 msgr2=0x7fbd940d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:13.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.842+0000 7fbd98c00640 1 -- 172.21.15.66:0/2882543605 shutdown_connections 2024-09-10T22:01:13.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.842+0000 7fbd98c00640 1 -- 172.21.15.66:0/2882543605 wait complete. 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.842+0000 7fbd98c00640 1 Processor -- start 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.842+0000 7fbd98c00640 1 -- start start 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd98c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd940b1e10 0x7fbd9414da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd98c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd9415f450 0x7fbd941512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd98c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd94160280 0x7fbd94153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd98c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9416a040 con 0x7fbd940b1e10 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd98c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94169ec0 con 0x7fbd94160280 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd98c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9416a1c0 con 0x7fbd9415f450 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd91e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd940b1e10 0x7fbd9414da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd91e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd940b1e10 0x7fbd9414da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41644/0 (socket says 172.21.15.66:41644) 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd91e00640 1 -- 172.21.15.66:0/2379455563 learned_addr learned my addr 172.21.15.66:0/2379455563 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd92800640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd94160280 0x7fbd94153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd89400640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd9415f450 0x7fbd941512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd91e00640 1 -- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd9415f450 msgr2=0x7fbd941512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd91e00640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd9415f450 0x7fbd941512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd91e00640 1 -- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd94160280 msgr2=0x7fbd94153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd91e00640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd94160280 0x7fbd94153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd91e00640 1 -- 172.21.15.66:0/2379455563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd94150230 con 0x7fbd940b1e10 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd89400640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd9415f450 0x7fbd941512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd91e00640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd940b1e10 0x7fbd9414da50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400c500 tx=0x7fbd8400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd8aa00640 1 -- 172.21.15.66:0/2379455563 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84017070 con 0x7fbd940b1e10 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd8aa00640 1 -- 172.21.15.66:0/2379455563 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd840053c0 con 0x7fbd940b1e10 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd98c00640 1 -- 172.21.15.66:0/2379455563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd94150510 con 0x7fbd940b1e10 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd8aa00640 1 -- 172.21.15.66:0/2379455563 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84012890 con 0x7fbd940b1e10 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd98c00640 1 -- 172.21.15.66:0/2379455563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd94150ad0 con 0x7fbd940b1e10 2024-09-10T22:01:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd98c00640 1 -- 172.21.15.66:0/2379455563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd940b3170 con 0x7fbd940b1e10 2024-09-10T22:01:13.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.846+0000 7fbd8aa00640 1 -- 172.21.15.66:0/2379455563 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbd84003600 con 0x7fbd940b1e10 2024-09-10T22:01:13.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.850+0000 7fbd8aa00640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd700725e0 0x7fbd70074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:13.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.850+0000 7fbd8aa00640 1 -- 172.21.15.66:0/2379455563 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fbd84092480 con 0x7fbd940b1e10 2024-09-10T22:01:13.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.850+0000 7fbd89400640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd700725e0 0x7fbd70074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:13.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.850+0000 7fbd89400640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd700725e0 0x7fbd70074aa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0af000 tx=0x7fbd7c01afa0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:13.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:13.854+0000 7fbd8aa00640 1 -- 172.21.15.66:0/2379455563 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbd8405d870 con 0x7fbd940b1e10 2024-09-10T22:01:14.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.086+0000 7fbd98c00640 1 -- 172.21.15.66:0/2379455563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fbd94074a80 con 0x7fbd940b1e10 2024-09-10T22:01:14.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.086+0000 7fbd8aa00640 1 -- 172.21.15.66:0/2379455563 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v20) ==== 90+0+0 (secure 0 0 0) 0x7fbd840622a0 con 0x7fbd940b1e10 2024-09-10T22:01:14.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.090+0000 7fbd98c00640 1 -- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd700725e0 msgr2=0x7fbd70074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:14.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.090+0000 7fbd98c00640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd700725e0 0x7fbd70074aa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0af000 tx=0x7fbd7c01afa0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.090+0000 7fbd98c00640 1 -- 172.21.15.66:0/2379455563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd940b1e10 msgr2=0x7fbd9414da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:14.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.090+0000 7fbd98c00640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd940b1e10 0x7fbd9414da50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400c500 tx=0x7fbd8400c9c0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.090+0000 7fbd98c00640 1 -- 172.21.15.66:0/2379455563 shutdown_connections 2024-09-10T22:01:14.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.090+0000 7fbd98c00640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd700725e0 0x7fbd70074aa0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.090+0000 7fbd98c00640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd94160280 0x7fbd94153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.090+0000 7fbd98c00640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd9415f450 0x7fbd941512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.090+0000 7fbd98c00640 1 --2- 172.21.15.66:0/2379455563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd940b1e10 0x7fbd9414da50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.090+0000 7fbd98c00640 1 -- 172.21.15.66:0/2379455563 >> 172.21.15.66:0/2379455563 conn(0x7fbd940d1c60 msgr2=0x7fbd940b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:14.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.090+0000 7fbd98c00640 1 -- 172.21.15.66:0/2379455563 shutdown_connections 2024-09-10T22:01:14.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.090+0000 7fbd98c00640 1 -- 172.21.15.66:0/2379455563 wait complete. 2024-09-10T22:01:14.116 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-10T22:01:14.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.258+0000 7fdac1200640 1 -- 172.21.15.66:0/3679927285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc15ae20 msgr2=0x7fdabc0aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:14.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.258+0000 7fdac1200640 1 --2- 172.21.15.66:0/3679927285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc15ae20 0x7fdabc0aee10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdab400acd0 tx=0x7fdab401dd80 comp rx=0 tx=0).stop 2024-09-10T22:01:14.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.258+0000 7fdac1200640 1 -- 172.21.15.66:0/3679927285 shutdown_connections 2024-09-10T22:01:14.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.258+0000 7fdac1200640 1 --2- 172.21.15.66:0/3679927285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc15ae20 0x7fdabc0aee10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.258+0000 7fdac1200640 1 -- 172.21.15.66:0/3679927285 >> 172.21.15.66:0/3679927285 conn(0x7fdabc0cc3c0 msgr2=0x7fdabc0cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:14.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.258+0000 7fdac1200640 1 -- 172.21.15.66:0/3679927285 shutdown_connections 2024-09-10T22:01:14.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.258+0000 7fdac1200640 1 -- 172.21.15.66:0/3679927285 wait complete. 2024-09-10T22:01:14.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdac1200640 1 Processor -- start 2024-09-10T22:01:14.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdac1200640 1 -- start start 2024-09-10T22:01:14.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdac1200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc0ae9a0 0x7fdabc0733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:14.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdac1200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc151020 0x7fdabc0725f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:14.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdac1200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdabc15ae20 0x7fdabc1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:14.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdac1200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc06fdd0 con 0x7fdabc15ae20 2024-09-10T22:01:14.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdac1200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc06fc50 con 0x7fdabc0ae9a0 2024-09-10T22:01:14.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdac1200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdabc167d50 con 0x7fdabc151020 2024-09-10T22:01:14.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdabbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdabc15ae20 0x7fdabc1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:14.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdabbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdabc15ae20 0x7fdabc1539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41650/0 (socket says 172.21.15.66:41650) 2024-09-10T22:01:14.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdabbe00640 1 -- 172.21.15.66:0/3323191463 learned_addr learned my addr 172.21.15.66:0/3323191463 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:14.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdab2a00640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc151020 0x7fdabc0725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:14.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdabb400640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc0ae9a0 0x7fdabc0733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdabbe00640 1 -- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc151020 msgr2=0x7fdabc0725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdabbe00640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc151020 0x7fdabc0725f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdabbe00640 1 -- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc0ae9a0 msgr2=0x7fdabc0733b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdabbe00640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc0ae9a0 0x7fdabc0733b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdabbe00640 1 -- 172.21.15.66:0/3323191463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdabc1f74a0 con 0x7fdabc15ae20 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdabb400640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc0ae9a0 0x7fdabc0733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdab2a00640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc151020 0x7fdabc0725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdabbe00640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdabc15ae20 0x7fdabc1539c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdab401a990 tx=0x7fdab4003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdab3e00640 1 -- 172.21.15.66:0/3323191463 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab40b3040 con 0x7fdabc15ae20 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdab3e00640 1 -- 172.21.15.66:0/3323191463 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdab4004040 con 0x7fdabc15ae20 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdab3e00640 1 -- 172.21.15.66:0/3323191463 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab40043f0 con 0x7fdabc15ae20 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdac1200640 1 -- 172.21.15.66:0/3323191463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdabc1fb4c0 con 0x7fdabc15ae20 2024-09-10T22:01:14.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.262+0000 7fdac1200640 1 -- 172.21.15.66:0/3323191463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdabc1fb7a0 con 0x7fdabc15ae20 2024-09-10T22:01:14.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.266+0000 7fdac1200640 1 -- 172.21.15.66:0/3323191463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdabc0aedf0 con 0x7fdabc15ae20 2024-09-10T22:01:14.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.266+0000 7fdab3e00640 1 -- 172.21.15.66:0/3323191463 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdab400b040 con 0x7fdabc15ae20 2024-09-10T22:01:14.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.266+0000 7fdab3e00640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda980725e0 0x7fda98074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:14.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.266+0000 7fdab3e00640 1 -- 172.21.15.66:0/3323191463 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fdab412d0f0 con 0x7fdabc15ae20 2024-09-10T22:01:14.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.266+0000 7fdabb400640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda980725e0 0x7fda98074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:14.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.266+0000 7fdabb400640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda980725e0 0x7fda98074aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdaac000f70 tx=0x7fdaac009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:14.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.270+0000 7fdab3e00640 1 -- 172.21.15.66:0/3323191463 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdab4017a00 con 0x7fdabc15ae20 2024-09-10T22:01:14.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.510+0000 7fdac1200640 1 -- 172.21.15.66:0/3323191463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fdabc06da10 con 0x7fdabc15ae20 2024-09-10T22:01:14.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.510+0000 7fdab3e00640 1 -- 172.21.15.66:0/3323191463 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v21) ==== 90+0+0 (secure 0 0 0) 0x7fdab40f8b10 con 0x7fdabc15ae20 2024-09-10T22:01:14.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.514+0000 7fdac1200640 1 -- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda980725e0 msgr2=0x7fda98074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:14.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.514+0000 7fdac1200640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda980725e0 0x7fda98074aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdaac000f70 tx=0x7fdaac009040 comp rx=0 tx=0).stop 2024-09-10T22:01:14.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.514+0000 7fdac1200640 1 -- 172.21.15.66:0/3323191463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdabc15ae20 msgr2=0x7fdabc1539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:14.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.514+0000 7fdac1200640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdabc15ae20 0x7fdabc1539c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdab401a990 tx=0x7fdab4003a00 comp rx=0 tx=0).stop 2024-09-10T22:01:14.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.514+0000 7fdac1200640 1 -- 172.21.15.66:0/3323191463 shutdown_connections 2024-09-10T22:01:14.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.514+0000 7fdac1200640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda980725e0 0x7fda98074aa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.514+0000 7fdac1200640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdabc15ae20 0x7fdabc1539c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.514+0000 7fdac1200640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc151020 0x7fdabc0725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.514+0000 7fdac1200640 1 --2- 172.21.15.66:0/3323191463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc0ae9a0 0x7fdabc0733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.514+0000 7fdac1200640 1 -- 172.21.15.66:0/3323191463 >> 172.21.15.66:0/3323191463 conn(0x7fdabc0cc3c0 msgr2=0x7fdabc1503f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:14.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.514+0000 7fdac1200640 1 -- 172.21.15.66:0/3323191463 shutdown_connections 2024-09-10T22:01:14.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.514+0000 7fdac1200640 1 -- 172.21.15.66:0/3323191463 wait complete. 2024-09-10T22:01:14.537 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-10T22:01:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 -- 172.21.15.66:0/1976809692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab0c15f460 msgr2=0x7fab0c0cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 --2- 172.21.15.66:0/1976809692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab0c15f460 0x7fab0c0cfc00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faafc009840 tx=0x7faafc01c670 comp rx=0 tx=0).stop 2024-09-10T22:01:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 -- 172.21.15.66:0/1976809692 shutdown_connections 2024-09-10T22:01:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 --2- 172.21.15.66:0/1976809692 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fab0c0b32a0 0x7fab0c0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 --2- 172.21.15.66:0/1976809692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab0c15f460 0x7fab0c0cfc00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 -- 172.21.15.66:0/1976809692 >> 172.21.15.66:0/1976809692 conn(0x7fab0c0d1d00 msgr2=0x7fab0c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 -- 172.21.15.66:0/1976809692 shutdown_connections 2024-09-10T22:01:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 -- 172.21.15.66:0/1976809692 wait complete. 2024-09-10T22:01:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 Processor -- start 2024-09-10T22:01:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 -- start start 2024-09-10T22:01:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fab0c0b0f60 0x7fab0c1fa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab0c0b32a0 0x7fab0c0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab0c15f460 0x7fab0c0c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0b0600 con 0x7fab0c0b32a0 2024-09-10T22:01:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0b0480 con 0x7fab0c15f460 2024-09-10T22:01:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab12c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c168c60 con 0x7fab0c0b0f60 2024-09-10T22:01:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab03e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab0c0b32a0 0x7fab0c0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab03e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab0c0b32a0 0x7fab0c0cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41666/0 (socket says 172.21.15.66:41666) 2024-09-10T22:01:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab03e00640 1 -- 172.21.15.66:0/1521046495 learned_addr learned my addr 172.21.15.66:0/1521046495 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab10c00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab0c15f460 0x7fab0c0c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab0be00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fab0c0b0f60 0x7fab0c1fa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab03e00640 1 -- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fab0c0b0f60 msgr2=0x7fab0c1fa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab03e00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fab0c0b0f60 0x7fab0c1fa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab03e00640 1 -- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab0c15f460 msgr2=0x7fab0c0c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab03e00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab0c15f460 0x7fab0c0c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.682+0000 7fab03e00640 1 -- 172.21.15.66:0/1521046495 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c1f7f70 con 0x7fab0c0b32a0 2024-09-10T22:01:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.686+0000 7fab03e00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab0c0b32a0 0x7fab0c0cb900 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faaf400c4d0 tx=0x7faaf400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.686+0000 7fab08c00640 1 -- 172.21.15.66:0/1521046495 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4016020 con 0x7fab0c0b32a0 2024-09-10T22:01:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.686+0000 7fab08c00640 1 -- 172.21.15.66:0/1521046495 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaf4012920 con 0x7fab0c0b32a0 2024-09-10T22:01:14.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.686+0000 7fab12c00640 1 -- 172.21.15.66:0/1521046495 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c1f8250 con 0x7fab0c0b32a0 2024-09-10T22:01:14.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.686+0000 7fab08c00640 1 -- 172.21.15.66:0/1521046495 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4012d30 con 0x7fab0c0b32a0 2024-09-10T22:01:14.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.686+0000 7fab12c00640 1 -- 172.21.15.66:0/1521046495 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c1f8830 con 0x7fab0c0b32a0 2024-09-10T22:01:14.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.686+0000 7fab12c00640 1 -- 172.21.15.66:0/1521046495 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c0af390 con 0x7fab0c0b32a0 2024-09-10T22:01:14.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.686+0000 7fab08c00640 1 -- 172.21.15.66:0/1521046495 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faaf401b400 con 0x7fab0c0b32a0 2024-09-10T22:01:14.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.686+0000 7fab08c00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faae8072610 0x7faae8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:14.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.690+0000 7fab08c00640 1 -- 172.21.15.66:0/1521046495 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7faaf4092fe0 con 0x7fab0c0b32a0 2024-09-10T22:01:14.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.690+0000 7fab0be00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faae8072610 0x7faae8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:14.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.690+0000 7fab0be00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faae8072610 0x7faae8074ad0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faafc0af000 tx=0x7faafc01af10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:14.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.694+0000 7fab08c00640 1 -- 172.21.15.66:0/1521046495 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faaf4020020 con 0x7fab0c0b32a0 2024-09-10T22:01:14.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.930+0000 7fab12c00640 1 -- 172.21.15.66:0/1521046495 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fab0c074aa0 con 0x7fab0c0b32a0 2024-09-10T22:01:14.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.930+0000 7fab08c00640 1 -- 172.21.15.66:0/1521046495 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v22) ==== 90+0+0 (secure 0 0 0) 0x7faaf405e3d0 con 0x7fab0c0b32a0 2024-09-10T22:01:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.934+0000 7fab12c00640 1 -- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faae8072610 msgr2=0x7faae8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.934+0000 7fab12c00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faae8072610 0x7faae8074ad0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faafc0af000 tx=0x7faafc01af10 comp rx=0 tx=0).stop 2024-09-10T22:01:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.934+0000 7fab12c00640 1 -- 172.21.15.66:0/1521046495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab0c0b32a0 msgr2=0x7fab0c0cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.934+0000 7fab12c00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab0c0b32a0 0x7fab0c0cb900 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faaf400c4d0 tx=0x7faaf400c990 comp rx=0 tx=0).stop 2024-09-10T22:01:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.934+0000 7fab12c00640 1 -- 172.21.15.66:0/1521046495 shutdown_connections 2024-09-10T22:01:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.934+0000 7fab12c00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faae8072610 0x7faae8074ad0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.934+0000 7fab12c00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab0c15f460 0x7fab0c0c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.934+0000 7fab12c00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab0c0b32a0 0x7fab0c0cb900 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.934+0000 7fab12c00640 1 --2- 172.21.15.66:0/1521046495 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fab0c0b0f60 0x7fab0c1fa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.934+0000 7fab12c00640 1 -- 172.21.15.66:0/1521046495 >> 172.21.15.66:0/1521046495 conn(0x7fab0c0d1d00 msgr2=0x7fab0c0d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.934+0000 7fab12c00640 1 -- 172.21.15.66:0/1521046495 shutdown_connections 2024-09-10T22:01:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:14.934+0000 7fab12c00640 1 -- 172.21.15.66:0/1521046495 wait complete. 2024-09-10T22:01:14.958 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-10T22:01:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.102+0000 7f61a0c00640 1 -- 172.21.15.66:0/2709389891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f619c15a310 msgr2=0x7f619c0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.102+0000 7f61a0c00640 1 --2- 172.21.15.66:0/2709389891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f619c15a310 0x7f619c0c48b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f618c00a950 tx=0x7f618c01dae0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.102+0000 7f61a0c00640 1 -- 172.21.15.66:0/2709389891 shutdown_connections 2024-09-10T22:01:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.102+0000 7f61a0c00640 1 --2- 172.21.15.66:0/2709389891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f619c15a310 0x7f619c0c48b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.102+0000 7f61a0c00640 1 -- 172.21.15.66:0/2709389891 >> 172.21.15.66:0/2709389891 conn(0x7f619c0cfa30 msgr2=0x7f619c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.102+0000 7f61a0c00640 1 -- 172.21.15.66:0/2709389891 shutdown_connections 2024-09-10T22:01:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.102+0000 7f61a0c00640 1 -- 172.21.15.66:0/2709389891 wait complete. 2024-09-10T22:01:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.102+0000 7f61a0c00640 1 Processor -- start 2024-09-10T22:01:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.102+0000 7f61a0c00640 1 -- start start 2024-09-10T22:01:15.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f61a0c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f619c0aeec0 0x7f619c137210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:15.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f61a0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c15a310 0x7f619c14f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:15.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f61a0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f619c139c60 0x7f619c138890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:15.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f61a0c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f619c168750 con 0x7f619c15a310 2024-09-10T22:01:15.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f61a0c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f619c1685d0 con 0x7f619c139c60 2024-09-10T22:01:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f61a0c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f619c1688d0 con 0x7f619c0aeec0 2024-09-10T22:01:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6193e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c15a310 0x7f619c14f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6193e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c15a310 0x7f619c14f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41676/0 (socket says 172.21.15.66:41676) 2024-09-10T22:01:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6193e00640 1 -- 172.21.15.66:0/2163351456 learned_addr learned my addr 172.21.15.66:0/2163351456 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f619a800640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f619c139c60 0x7f619c138890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6199e00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f619c0aeec0 0x7f619c137210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6193e00640 1 -- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f619c0aeec0 msgr2=0x7f619c137210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6193e00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f619c0aeec0 0x7f619c137210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6193e00640 1 -- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f619c139c60 msgr2=0x7f619c138890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6193e00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f619c139c60 0x7f619c138890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6193e00640 1 -- 172.21.15.66:0/2163351456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c1377f0 con 0x7f619c15a310 2024-09-10T22:01:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6199e00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f619c0aeec0 0x7f619c137210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6193e00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c15a310 0x7f619c14f640 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f618400c4b0 tx=0x7f618400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6192000640 1 -- 172.21.15.66:0/2163351456 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61840052d0 con 0x7f619c15a310 2024-09-10T22:01:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6192000640 1 -- 172.21.15.66:0/2163351456 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6184005470 con 0x7f619c15a310 2024-09-10T22:01:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6192000640 1 -- 172.21.15.66:0/2163351456 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184012870 con 0x7f619c15a310 2024-09-10T22:01:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f61a0c00640 1 -- 172.21.15.66:0/2163351456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619c137ad0 con 0x7f619c15a310 2024-09-10T22:01:15.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f61a0c00640 1 -- 172.21.15.66:0/2163351456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619c138090 con 0x7f619c15a310 2024-09-10T22:01:15.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f61a0c00640 1 -- 172.21.15.66:0/2163351456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f619c0c48b0 con 0x7f619c15a310 2024-09-10T22:01:15.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.106+0000 7f6192000640 1 -- 172.21.15.66:0/2163351456 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6184003600 con 0x7f619c15a310 2024-09-10T22:01:15.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.110+0000 7f6192000640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6178072610 0x7f6178074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:15.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.110+0000 7f6192000640 1 -- 172.21.15.66:0/2163351456 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f6184092a50 con 0x7f619c15a310 2024-09-10T22:01:15.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.110+0000 7f6199e00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6178072610 0x7f6178074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:15.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.110+0000 7f6199e00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6178072610 0x7f6178074ad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f618c0b0000 tx=0x7f618c01c320 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:15.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.114+0000 7f6192000640 1 -- 172.21.15.66:0/2163351456 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f618405e3e0 con 0x7f619c15a310 2024-09-10T22:01:15.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.346+0000 7f61a0c00640 1 -- 172.21.15.66:0/2163351456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f619c0cb8b0 con 0x7f619c15a310 2024-09-10T22:01:15.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.346+0000 7f6192000640 1 -- 172.21.15.66:0/2163351456 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v23) ==== 97+0+0 (secure 0 0 0) 0x7f618405dce0 con 0x7f619c15a310 2024-09-10T22:01:15.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.350+0000 7f61a0c00640 1 -- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6178072610 msgr2=0x7f6178074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.350+0000 7f61a0c00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6178072610 0x7f6178074ad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f618c0b0000 tx=0x7f618c01c320 comp rx=0 tx=0).stop 2024-09-10T22:01:15.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.350+0000 7f61a0c00640 1 -- 172.21.15.66:0/2163351456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c15a310 msgr2=0x7f619c14f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.350+0000 7f61a0c00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c15a310 0x7f619c14f640 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f618400c4b0 tx=0x7f618400c970 comp rx=0 tx=0).stop 2024-09-10T22:01:15.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.350+0000 7f61a0c00640 1 -- 172.21.15.66:0/2163351456 shutdown_connections 2024-09-10T22:01:15.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.350+0000 7f61a0c00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6178072610 0x7f6178074ad0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.350+0000 7f61a0c00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f619c139c60 0x7f619c138890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.350+0000 7f61a0c00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c15a310 0x7f619c14f640 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.350+0000 7f61a0c00640 1 --2- 172.21.15.66:0/2163351456 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f619c0aeec0 0x7f619c137210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.350+0000 7f61a0c00640 1 -- 172.21.15.66:0/2163351456 >> 172.21.15.66:0/2163351456 conn(0x7f619c0cfa30 msgr2=0x7f619c0b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:15.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.350+0000 7f61a0c00640 1 -- 172.21.15.66:0/2163351456 shutdown_connections 2024-09-10T22:01:15.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.350+0000 7f61a0c00640 1 -- 172.21.15.66:0/2163351456 wait complete. 2024-09-10T22:01:15.375 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-10T22:01:15.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.518+0000 7f4b05000640 1 -- 172.21.15.66:0/187869287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b0015f460 msgr2=0x7f4b000cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.518+0000 7f4b05000640 1 --2- 172.21.15.66:0/187869287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b0015f460 0x7f4b000cfc00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4b000b1340 tx=0x7f4af001c4b0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.518+0000 7f4b05000640 1 -- 172.21.15.66:0/187869287 shutdown_connections 2024-09-10T22:01:15.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.518+0000 7f4b05000640 1 --2- 172.21.15.66:0/187869287 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b000b32a0 0x7f4b000b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.518+0000 7f4b05000640 1 --2- 172.21.15.66:0/187869287 >> v2:172.21.15.94:3300/0 conn(0x7f4b000b0f60 0x7f4b000d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.518+0000 7f4b05000640 1 --2- 172.21.15.66:0/187869287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b0015f460 0x7f4b000cfc00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.518+0000 7f4b05000640 1 -- 172.21.15.66:0/187869287 >> 172.21.15.66:0/187869287 conn(0x7f4b000d1d00 msgr2=0x7f4b000d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:15.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.518+0000 7f4b05000640 1 -- 172.21.15.66:0/187869287 shutdown_connections 2024-09-10T22:01:15.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.518+0000 7f4b05000640 1 -- 172.21.15.66:0/187869287 wait complete. 2024-09-10T22:01:15.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.518+0000 7f4b05000640 1 Processor -- start 2024-09-10T22:01:15.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.518+0000 7f4b05000640 1 -- start start 2024-09-10T22:01:15.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4b05000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b000b0f60 0x7f4b0014c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:15.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4b05000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b000b32a0 0x7f4b000cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4b05000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b0015f460 0x7f4b00154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4b05000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4b000af720 con 0x7f4b000b32a0 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4b05000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b000af5a0 con 0x7f4b000b0f60 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4b05000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4b000af8a0 con 0x7f4b0015f460 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b000b32a0 0x7f4b000cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b000b32a0 0x7f4b000cef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41704/0 (socket says 172.21.15.66:41704) 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af7e00640 1 -- 172.21.15.66:0/2864181126 learned_addr learned my addr 172.21.15.66:0/2864181126 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4afde00640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b000b0f60 0x7f4b0014c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4afe800640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b0015f460 0x7f4b00154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af7e00640 1 -- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b0015f460 msgr2=0x7f4b00154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af7e00640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b0015f460 0x7f4b00154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af7e00640 1 -- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b000b0f60 msgr2=0x7f4b0014c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af7e00640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b000b0f60 0x7f4b0014c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af7e00640 1 -- 172.21.15.66:0/2864181126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b000b6d80 con 0x7f4b000b32a0 2024-09-10T22:01:15.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4afe800640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b0015f460 0x7f4b00154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:15.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af7e00640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b000b32a0 0x7f4b000cef40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800d720 tx=0x7f4ae800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:15.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af6000640 1 -- 172.21.15.66:0/2864181126 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8017070 con 0x7f4b000b32a0 2024-09-10T22:01:15.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af6000640 1 -- 172.21.15.66:0/2864181126 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ae8009d20 con 0x7f4b000b32a0 2024-09-10T22:01:15.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af6000640 1 -- 172.21.15.66:0/2864181126 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8004280 con 0x7f4b000b32a0 2024-09-10T22:01:15.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4b05000640 1 -- 172.21.15.66:0/2864181126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0014c890 con 0x7f4b000b32a0 2024-09-10T22:01:15.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4b05000640 1 -- 172.21.15.66:0/2864181126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b000b62a0 con 0x7f4b000b32a0 2024-09-10T22:01:15.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4b05000640 1 -- 172.21.15.66:0/2864181126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b000b5d20 con 0x7f4b000b32a0 2024-09-10T22:01:15.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.522+0000 7f4af6000640 1 -- 172.21.15.66:0/2864181126 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4ae8005020 con 0x7f4b000b32a0 2024-09-10T22:01:15.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.526+0000 7f4af6000640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4adc072610 0x7f4adc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:15.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.526+0000 7f4af6000640 1 -- 172.21.15.66:0/2864181126 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f4ae8093390 con 0x7f4b000b32a0 2024-09-10T22:01:15.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.526+0000 7f4afde00640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4adc072610 0x7f4adc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:15.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.526+0000 7f4afde00640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4adc072610 0x7f4adc074ad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4af00af000 tx=0x7f4af00b4040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:15.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.530+0000 7f4af6000640 1 -- 172.21.15.66:0/2864181126 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4ae8020020 con 0x7f4b000b32a0 2024-09-10T22:01:15.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.766+0000 7f4b05000640 1 -- 172.21.15.66:0/2864181126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f4b00074aa0 con 0x7f4b000b32a0 2024-09-10T22:01:15.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.766+0000 7f4af6000640 1 -- 172.21.15.66:0/2864181126 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v24) ==== 90+0+0 (secure 0 0 0) 0x7f4ae805ed20 con 0x7f4b000b32a0 2024-09-10T22:01:15.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.770+0000 7f4b05000640 1 -- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4adc072610 msgr2=0x7f4adc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.770+0000 7f4b05000640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4adc072610 0x7f4adc074ad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4af00af000 tx=0x7f4af00b4040 comp rx=0 tx=0).stop 2024-09-10T22:01:15.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.770+0000 7f4b05000640 1 -- 172.21.15.66:0/2864181126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b000b32a0 msgr2=0x7f4b000cef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.770+0000 7f4b05000640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b000b32a0 0x7f4b000cef40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800d720 tx=0x7f4ae800dbe0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.770+0000 7f4b05000640 1 -- 172.21.15.66:0/2864181126 shutdown_connections 2024-09-10T22:01:15.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.770+0000 7f4b05000640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4adc072610 0x7f4adc074ad0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.770+0000 7f4b05000640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b0015f460 0x7f4b00154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.770+0000 7f4b05000640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b000b32a0 0x7f4b000cef40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.770+0000 7f4b05000640 1 --2- 172.21.15.66:0/2864181126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b000b0f60 0x7f4b0014c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.770+0000 7f4b05000640 1 -- 172.21.15.66:0/2864181126 >> 172.21.15.66:0/2864181126 conn(0x7f4b000d1d00 msgr2=0x7f4b0006e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:15.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.770+0000 7f4b05000640 1 -- 172.21.15.66:0/2864181126 shutdown_connections 2024-09-10T22:01:15.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.770+0000 7f4b05000640 1 -- 172.21.15.66:0/2864181126 wait complete. 2024-09-10T22:01:15.793 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:01:15.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.934+0000 7ff811800640 1 -- 172.21.15.66:0/1888600485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c15f340 msgr2=0x7ff80c0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.934+0000 7ff811800640 1 --2- 172.21.15.66:0/1888600485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c15f340 0x7ff80c0cfb50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009bf0 tx=0x7ff7fc01cba0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.934+0000 7ff811800640 1 -- 172.21.15.66:0/1888600485 shutdown_connections 2024-09-10T22:01:15.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.934+0000 7ff811800640 1 --2- 172.21.15.66:0/1888600485 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff80c0af970 0x7ff80c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.934+0000 7ff811800640 1 --2- 172.21.15.66:0/1888600485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c15f340 0x7ff80c0cfb50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.934+0000 7ff811800640 1 -- 172.21.15.66:0/1888600485 >> 172.21.15.66:0/1888600485 conn(0x7ff80c0d1c50 msgr2=0x7ff80c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:15.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.934+0000 7ff811800640 1 -- 172.21.15.66:0/1888600485 shutdown_connections 2024-09-10T22:01:15.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.934+0000 7ff811800640 1 -- 172.21.15.66:0/1888600485 wait complete. 2024-09-10T22:01:15.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff811800640 1 Processor -- start 2024-09-10T22:01:15.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff811800640 1 -- start start 2024-09-10T22:01:15.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff811800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff80c0af970 0x7ff80c0c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:15.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff811800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c0b2bf0 0x7ff80c071a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:15.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff811800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c15f340 0x7ff80c1658f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:15.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff811800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c169190 con 0x7ff80c15f340 2024-09-10T22:01:15.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff811800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c169010 con 0x7ff80c0b2bf0 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff811800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff80c169310 con 0x7ff80c0af970 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff80b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c15f340 0x7ff80c1658f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff80b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c15f340 0x7ff80c1658f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41730/0 (socket says 172.21.15.66:41730) 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff80b400640 1 -- 172.21.15.66:0/4246057013 learned_addr learned my addr 172.21.15.66:0/4246057013 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff80aa00640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff80c0af970 0x7ff80c0c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff802000640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c0b2bf0 0x7ff80c071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff80b400640 1 -- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff80c0af970 msgr2=0x7ff80c0c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff80b400640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff80c0af970 0x7ff80c0c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff80b400640 1 -- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c0b2bf0 msgr2=0x7ff80c071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff80b400640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c0b2bf0 0x7ff80c071a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff80b400640 1 -- 172.21.15.66:0/4246057013 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80c06f770 con 0x7ff80c15f340 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff802000640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c0b2bf0 0x7ff80c071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:15.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff80b400640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c15f340 0x7ff80c1658f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff80400d740 tx=0x7ff80400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:15.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff803400640 1 -- 172.21.15.66:0/4246057013 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804017070 con 0x7ff80c15f340 2024-09-10T22:01:15.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff803400640 1 -- 172.21.15.66:0/4246057013 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff80400de50 con 0x7ff80c15f340 2024-09-10T22:01:15.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff811800640 1 -- 172.21.15.66:0/4246057013 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80c06fa50 con 0x7ff80c15f340 2024-09-10T22:01:15.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff803400640 1 -- 172.21.15.66:0/4246057013 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804010460 con 0x7ff80c15f340 2024-09-10T22:01:15.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.938+0000 7ff811800640 1 -- 172.21.15.66:0/4246057013 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80c1653d0 con 0x7ff80c15f340 2024-09-10T22:01:15.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.942+0000 7ff811800640 1 -- 172.21.15.66:0/4246057013 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff80c1655d0 con 0x7ff80c15f340 2024-09-10T22:01:15.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.942+0000 7ff803400640 1 -- 172.21.15.66:0/4246057013 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff804010600 con 0x7ff80c15f340 2024-09-10T22:01:15.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.942+0000 7ff803400640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff7e80725e0 0x7ff7e8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:15.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.942+0000 7ff803400640 1 -- 172.21.15.66:0/4246057013 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7ff8040933f0 con 0x7ff80c15f340 2024-09-10T22:01:15.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.942+0000 7ff80aa00640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff7e80725e0 0x7ff7e8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:15.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.942+0000 7ff80aa00640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff7e80725e0 0x7ff7e8074aa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0af000 tx=0x7ff7fc01b3e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:15.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:15.946+0000 7ff803400640 1 -- 172.21.15.66:0/4246057013 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff80405ed90 con 0x7ff80c15f340 2024-09-10T22:01:16.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.126+0000 7ff811800640 1 -- 172.21.15.66:0/4246057013 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff80c0713c0 con 0x7ff80c15f340 2024-09-10T22:01:16.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.126+0000 7ff803400640 1 -- 172.21.15.66:0/4246057013 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7ff80405e690 con 0x7ff80c15f340 2024-09-10T22:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:01:16.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.130+0000 7ff811800640 1 -- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff7e80725e0 msgr2=0x7ff7e8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:16.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.130+0000 7ff811800640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff7e80725e0 0x7ff7e8074aa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0af000 tx=0x7ff7fc01b3e0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.130+0000 7ff811800640 1 -- 172.21.15.66:0/4246057013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c15f340 msgr2=0x7ff80c1658f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:16.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.130+0000 7ff811800640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c15f340 0x7ff80c1658f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff80400d740 tx=0x7ff80400dc00 comp rx=0 tx=0).stop 2024-09-10T22:01:16.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.130+0000 7ff811800640 1 -- 172.21.15.66:0/4246057013 shutdown_connections 2024-09-10T22:01:16.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.130+0000 7ff811800640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff7e80725e0 0x7ff7e8074aa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.130+0000 7ff811800640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c15f340 0x7ff80c1658f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.130+0000 7ff811800640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c0b2bf0 0x7ff80c071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.130+0000 7ff811800640 1 --2- 172.21.15.66:0/4246057013 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff80c0af970 0x7ff80c0c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.130+0000 7ff811800640 1 -- 172.21.15.66:0/4246057013 >> 172.21.15.66:0/4246057013 conn(0x7ff80c0d1c50 msgr2=0x7ff80c0b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:16.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.130+0000 7ff811800640 1 -- 172.21.15.66:0/4246057013 shutdown_connections 2024-09-10T22:01:16.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.130+0000 7ff811800640 1 -- 172.21.15.66:0/4246057013 wait complete. 2024-09-10T22:01:16.153 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:01:16.153 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:01:16.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.298+0000 7f0d03600640 1 -- 172.21.15.66:0/3134848264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cfc0cee30 msgr2=0x7f0cfc0cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:16.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.298+0000 7f0d03600640 1 --2- 172.21.15.66:0/3134848264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cfc0cee30 0x7f0cfc0cf210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000a950 tx=0x7f0cf0020360 comp rx=0 tx=0).stop 2024-09-10T22:01:16.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.298+0000 7f0d03600640 1 -- 172.21.15.66:0/3134848264 shutdown_connections 2024-09-10T22:01:16.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.298+0000 7f0d03600640 1 --2- 172.21.15.66:0/3134848264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cfc15ddd0 0x7f0cfc15e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.298+0000 7f0d03600640 1 --2- 172.21.15.66:0/3134848264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cfc0cee30 0x7f0cfc0cf210 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.298+0000 7f0d03600640 1 -- 172.21.15.66:0/3134848264 >> 172.21.15.66:0/3134848264 conn(0x7f0cfc0cfaa0 msgr2=0x7f0cfc0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:16.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.298+0000 7f0d03600640 1 -- 172.21.15.66:0/3134848264 shutdown_connections 2024-09-10T22:01:16.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.298+0000 7f0d03600640 1 -- 172.21.15.66:0/3134848264 wait complete. 2024-09-10T22:01:16.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d03600640 1 Processor -- start 2024-09-10T22:01:16.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d03600640 1 -- start start 2024-09-10T22:01:16.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d03600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cfc06ef30 0x7f0cfc0aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:16.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d03600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cfc0cee30 0x7f0cfc15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:16.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d03600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cfc15ddd0 0x7f0cfc15bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:16.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d03600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc15c740 con 0x7f0cfc06ef30 2024-09-10T22:01:16.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d03600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc15c8c0 con 0x7f0cfc15ddd0 2024-09-10T22:01:16.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d03600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0cfc06fad0 con 0x7f0cfc0cee30 2024-09-10T22:01:16.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d00c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cfc06ef30 0x7f0cfc0aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d00c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cfc06ef30 0x7f0cfc0aef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41754/0 (socket says 172.21.15.66:41754) 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d00c00640 1 -- 172.21.15.66:0/3851165615 learned_addr learned my addr 172.21.15.66:0/3851165615 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d00c00640 1 -- 172.21.15.66:0/3851165615 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cfc0cee30 msgr2=0x7f0cfc15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d00c00640 1 --2- 172.21.15.66:0/3851165615 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cfc0cee30 0x7f0cfc15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d00c00640 1 -- 172.21.15.66:0/3851165615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cfc15ddd0 msgr2=0x7f0cfc15bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d00c00640 1 --2- 172.21.15.66:0/3851165615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cfc15ddd0 0x7f0cfc15bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d00c00640 1 -- 172.21.15.66:0/3851165615 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc1f47d0 con 0x7f0cfc06ef30 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d00c00640 1 --2- 172.21.15.66:0/3851165615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cfc06ef30 0x7f0cfc0aef10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0cf001ea10 tx=0x7f0cf00a9ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0cf8c00640 1 -- 172.21.15.66:0/3851165615 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00b5070 con 0x7f0cfc06ef30 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0cf8c00640 1 -- 172.21.15.66:0/3851165615 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cf00b6040 con 0x7f0cfc06ef30 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d03600640 1 -- 172.21.15.66:0/3851165615 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc146da0 con 0x7f0cfc06ef30 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0cf8c00640 1 -- 172.21.15.66:0/3851165615 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00b0e40 con 0x7f0cfc06ef30 2024-09-10T22:01:16.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.302+0000 7f0d03600640 1 -- 172.21.15.66:0/3851165615 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc0a9550 con 0x7f0cfc06ef30 2024-09-10T22:01:16.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.306+0000 7f0d03600640 1 -- 172.21.15.66:0/3851165615 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cfc0d07c0 con 0x7f0cfc06ef30 2024-09-10T22:01:16.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.306+0000 7f0cf8c00640 1 -- 172.21.15.66:0/3851165615 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0cf000b040 con 0x7f0cfc06ef30 2024-09-10T22:01:16.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.306+0000 7f0cf8c00640 1 --2- 172.21.15.66:0/3851165615 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0cdc0725e0 0x7f0cdc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:16.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.306+0000 7f0cf8c00640 1 -- 172.21.15.66:0/3851165615 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f0cf0130a70 con 0x7f0cfc06ef30 2024-09-10T22:01:16.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.306+0000 7f0cfbe00640 1 --2- 172.21.15.66:0/3851165615 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0cdc0725e0 0x7f0cdc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:16.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.306+0000 7f0cfbe00640 1 --2- 172.21.15.66:0/3851165615 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0cdc0725e0 0x7f0cdc074aa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0af450 tx=0x7f0ce8005f50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:16.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.310+0000 7f0cf8c00640 1 -- 172.21.15.66:0/3851165615 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0cf00c8020 con 0x7f0cfc06ef30 2024-09-10T22:01:16.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.486+0000 7f0d03600640 1 -- 172.21.15.66:0/3851165615 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0cfc15e1f0 con 0x7f0cfc06ef30 2024-09-10T22:01:16.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.826+0000 7f0cf8c00640 1 -- 172.21.15.66:0/3851165615 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f0cfc1f47d0 con 0x7f0cfc06ef30 2024-09-10T22:01:16.831 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:01:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.830+0000 7f0d03600640 1 -- 172.21.15.66:0/3851165615 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0cdc0725e0 msgr2=0x7f0cdc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.830+0000 7f0d03600640 1 --2- 172.21.15.66:0/3851165615 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0cdc0725e0 0x7f0cdc074aa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0af450 tx=0x7f0ce8005f50 comp rx=0 tx=0).stop 2024-09-10T22:01:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.830+0000 7f0d03600640 1 -- 172.21.15.66:0/3851165615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cfc06ef30 msgr2=0x7f0cfc0aef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.830+0000 7f0d03600640 1 --2- 172.21.15.66:0/3851165615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cfc06ef30 0x7f0cfc0aef10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0cf001ea10 tx=0x7f0cf00a9ee0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.830+0000 7f0d03600640 1 -- 172.21.15.66:0/3851165615 shutdown_connections 2024-09-10T22:01:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.830+0000 7f0d03600640 1 --2- 172.21.15.66:0/3851165615 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0cdc0725e0 0x7f0cdc074aa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.830+0000 7f0d03600640 1 --2- 172.21.15.66:0/3851165615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cfc15ddd0 0x7f0cfc15bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.830+0000 7f0d03600640 1 --2- 172.21.15.66:0/3851165615 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cfc0cee30 0x7f0cfc15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.830+0000 7f0d03600640 1 --2- 172.21.15.66:0/3851165615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cfc06ef30 0x7f0cfc0aef10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:16.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.830+0000 7f0d03600640 1 -- 172.21.15.66:0/3851165615 >> 172.21.15.66:0/3851165615 conn(0x7f0cfc0cfaa0 msgr2=0x7f0cfc072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:16.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.830+0000 7f0d03600640 1 -- 172.21.15.66:0/3851165615 shutdown_connections 2024-09-10T22:01:16.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:16.830+0000 7f0d03600640 1 -- 172.21.15.66:0/3851165615 wait complete. 2024-09-10T22:01:16.856 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:01:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 -- 172.21.15.66:0/3738700787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f300d1b80 msgr2=0x7f7f300b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 --2- 172.21.15.66:0/3738700787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f300d1b80 0x7f7f300b36d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009840 tx=0x7f7f1801c820 comp rx=0 tx=0).stop 2024-09-10T22:01:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 -- 172.21.15.66:0/3738700787 shutdown_connections 2024-09-10T22:01:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 --2- 172.21.15.66:0/3738700787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f300d1b80 0x7f7f300b36d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 -- 172.21.15.66:0/3738700787 >> 172.21.15.66:0/3738700787 conn(0x7f7f300d3da0 msgr2=0x7f7f300d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 -- 172.21.15.66:0/3738700787 shutdown_connections 2024-09-10T22:01:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 -- 172.21.15.66:0/3738700787 wait complete. 2024-09-10T22:01:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 Processor -- start 2024-09-10T22:01:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 -- start start 2024-09-10T22:01:17.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f300b4bd0 0x7f7f300ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:17.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f30162630 0x7f7f301fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:17.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f300c8a10 0x7f7f30151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:17.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3016c380 con 0x7f7f300b4bd0 2024-09-10T22:01:17.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3016c200 con 0x7f7f300c8a10 2024-09-10T22:01:17.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.006+0000 7f7f35a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3016c500 con 0x7f7f30162630 2024-09-10T22:01:17.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f300b4bd0 0x7f7f300ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:17.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f300b4bd0 0x7f7f300ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41774/0 (socket says 172.21.15.66:41774) 2024-09-10T22:01:17.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2fe00640 1 -- 172.21.15.66:0/897389318 learned_addr learned my addr 172.21.15.66:0/897389318 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f27e00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f30162630 0x7f7f301fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f35000640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f300c8a10 0x7f7f30151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2fe00640 1 -- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f30162630 msgr2=0x7f7f301fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2fe00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f30162630 0x7f7f301fbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2fe00640 1 -- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f300c8a10 msgr2=0x7f7f30151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2fe00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f300c8a10 0x7f7f30151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2fe00640 1 -- 172.21.15.66:0/897389318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f301520d0 con 0x7f7f300b4bd0 2024-09-10T22:01:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f35000640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f300c8a10 0x7f7f30151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2fe00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f300b4bd0 0x7f7f300ca260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c3c0 tx=0x7f7f2000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:17.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2cc00640 1 -- 172.21.15.66:0/897389318 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20017070 con 0x7f7f300b4bd0 2024-09-10T22:01:17.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2cc00640 1 -- 172.21.15.66:0/897389318 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f20003600 con 0x7f7f300b4bd0 2024-09-10T22:01:17.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f35a00640 1 -- 172.21.15.66:0/897389318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f300c7970 con 0x7f7f300b4bd0 2024-09-10T22:01:17.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2cc00640 1 -- 172.21.15.66:0/897389318 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20003a50 con 0x7f7f300b4bd0 2024-09-10T22:01:17.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f35a00640 1 -- 172.21.15.66:0/897389318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f300c7f30 con 0x7f7f300b4bd0 2024-09-10T22:01:17.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f35a00640 1 -- 172.21.15.66:0/897389318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f3008dd40 con 0x7f7f300b4bd0 2024-09-10T22:01:17.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.010+0000 7f7f2cc00640 1 -- 172.21.15.66:0/897389318 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7f20003bf0 con 0x7f7f300b4bd0 2024-09-10T22:01:17.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.014+0000 7f7f2cc00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7f0c072610 0x7f7f0c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:17.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.014+0000 7f7f2cc00640 1 -- 172.21.15.66:0/897389318 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f7f200924e0 con 0x7f7f300b4bd0 2024-09-10T22:01:17.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.014+0000 7f7f27e00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7f0c072610 0x7f7f0c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.014+0000 7f7f27e00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7f0c072610 0x7f7f0c074ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7f180af000 tx=0x7f7f1801b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:17.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.018+0000 7f7f2cc00640 1 -- 172.21.15.66:0/897389318 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7f301520d0 con 0x7f7f300b4bd0 2024-09-10T22:01:17.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.194+0000 7f7f35a00640 1 -- 172.21.15.66:0/897389318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7f300d27d0 con 0x7f7f300b4bd0 2024-09-10T22:01:17.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.830+0000 7f7f2cc00640 1 -- 172.21.15.66:0/897389318 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f7f2005d740 con 0x7f7f300b4bd0 2024-09-10T22:01:17.834 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:01:17.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.830+0000 7f7f35a00640 1 -- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7f0c072610 msgr2=0x7f7f0c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:17.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.830+0000 7f7f35a00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7f0c072610 0x7f7f0c074ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7f180af000 tx=0x7f7f1801b060 comp rx=0 tx=0).stop 2024-09-10T22:01:17.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.830+0000 7f7f35a00640 1 -- 172.21.15.66:0/897389318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f300b4bd0 msgr2=0x7f7f300ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:17.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.830+0000 7f7f35a00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f300b4bd0 0x7f7f300ca260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c3c0 tx=0x7f7f2000c880 comp rx=0 tx=0).stop 2024-09-10T22:01:17.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.830+0000 7f7f35a00640 1 -- 172.21.15.66:0/897389318 shutdown_connections 2024-09-10T22:01:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.830+0000 7f7f35a00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7f0c072610 0x7f7f0c074ad0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.830+0000 7f7f35a00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f300c8a10 0x7f7f30151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.830+0000 7f7f35a00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f30162630 0x7f7f301fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.830+0000 7f7f35a00640 1 --2- 172.21.15.66:0/897389318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f300b4bd0 0x7f7f300ca260 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c3c0 tx=0x7f7f2000c880 comp rx=0 tx=0).stop 2024-09-10T22:01:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.830+0000 7f7f35a00640 1 -- 172.21.15.66:0/897389318 >> 172.21.15.66:0/897389318 conn(0x7f7f300d3da0 msgr2=0x7f7f300b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.834+0000 7f7f35a00640 1 -- 172.21.15.66:0/897389318 shutdown_connections 2024-09-10T22:01:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:17.834+0000 7f7f35a00640 1 -- 172.21.15.66:0/897389318 wait complete. 2024-09-10T22:01:17.876 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:01:18.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 -- 172.21.15.66:0/2393715317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f096c15f450 msgr2=0x7f096c0b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:18.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 --2- 172.21.15.66:0/2393715317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f096c15f450 0x7f096c0b3170 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0960009840 tx=0x7f096001d1f0 comp rx=0 tx=0).stop 2024-09-10T22:01:18.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 -- 172.21.15.66:0/2393715317 shutdown_connections 2024-09-10T22:01:18.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 --2- 172.21.15.66:0/2393715317 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f096c0b1e10 0x7f096c15fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:18.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 --2- 172.21.15.66:0/2393715317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f096c15f450 0x7f096c0b3170 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:18.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 -- 172.21.15.66:0/2393715317 >> 172.21.15.66:0/2393715317 conn(0x7f096c0d1c60 msgr2=0x7f096c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:18.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 -- 172.21.15.66:0/2393715317 shutdown_connections 2024-09-10T22:01:18.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 -- 172.21.15.66:0/2393715317 wait complete. 2024-09-10T22:01:18.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 Processor -- start 2024-09-10T22:01:18.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 -- start start 2024-09-10T22:01:18.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f096c0b1e10 0x7f096c159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:18.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f096c15f450 0x7f096c0b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:18.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c1600b0 0x7f096c0b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:18.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f096c169380 con 0x7f096c0b1e10 2024-09-10T22:01:18.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f096c169200 con 0x7f096c1600b0 2024-09-10T22:01:18.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0973a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f096c169500 con 0x7f096c15f450 2024-09-10T22:01:18.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0971000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f096c0b1e10 0x7f096c159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:18.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0971000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f096c0b1e10 0x7f096c159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41784/0 (socket says 172.21.15.66:41784) 2024-09-10T22:01:18.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0971000640 1 -- 172.21.15.66:0/2170872474 learned_addr learned my addr 172.21.15.66:0/2170872474 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:18.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f0971a00640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c1600b0 0x7f096c0b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:18.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.022+0000 7f096be00640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f096c15f450 0x7f096c0b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:18.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0971000640 1 -- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f096c15f450 msgr2=0x7f096c0b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:18.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0971000640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f096c15f450 0x7f096c0b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:18.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0971000640 1 -- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c1600b0 msgr2=0x7f096c0b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:18.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0971000640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c1600b0 0x7f096c0b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:18.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0971000640 1 -- 172.21.15.66:0/2170872474 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c15d2f0 con 0x7f096c0b1e10 2024-09-10T22:01:18.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0971000640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f096c0b1e10 0x7f096c159c00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f096001ae40 tx=0x7f0960005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:18.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0968c00640 1 -- 172.21.15.66:0/2170872474 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09600bb070 con 0x7f096c0b1e10 2024-09-10T22:01:18.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0968c00640 1 -- 172.21.15.66:0/2170872474 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09600a5cc0 con 0x7f096c0b1e10 2024-09-10T22:01:18.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0968c00640 1 -- 172.21.15.66:0/2170872474 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09600b2040 con 0x7f096c0b1e10 2024-09-10T22:01:18.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0973a00640 1 -- 172.21.15.66:0/2170872474 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096c15d5d0 con 0x7f096c0b1e10 2024-09-10T22:01:18.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0973a00640 1 -- 172.21.15.66:0/2170872474 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096c15db90 con 0x7f096c0b1e10 2024-09-10T22:01:18.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0973a00640 1 -- 172.21.15.66:0/2170872474 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096c15fb70 con 0x7f096c0b1e10 2024-09-10T22:01:18.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.026+0000 7f0968c00640 1 -- 172.21.15.66:0/2170872474 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f09600b5030 con 0x7f096c0b1e10 2024-09-10T22:01:18.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.030+0000 7f0968c00640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f094c0725e0 0x7f094c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:18.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.030+0000 7f0968c00640 1 -- 172.21.15.66:0/2170872474 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f096012cbd0 con 0x7f096c0b1e10 2024-09-10T22:01:18.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.030+0000 7f096be00640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f094c0725e0 0x7f094c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:18.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.030+0000 7f096be00640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f094c0725e0 0x7f094c074aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f096c15ca50 tx=0x7f095800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:18.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.034+0000 7f0968c00640 1 -- 172.21.15.66:0/2170872474 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0960016410 con 0x7f096c0b1e10 2024-09-10T22:01:18.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.238+0000 7f0973a00640 1 -- 172.21.15.66:0/2170872474 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f096c0d2a40 con 0x7f096c0b1e10 2024-09-10T22:01:18.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.858+0000 7f0968c00640 1 -- 172.21.15.66:0/2170872474 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f09600f7ca0 con 0x7f096c0b1e10 2024-09-10T22:01:18.864 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 4 and data pool 5 2024-09-10T22:01:18.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.866+0000 7f0973a00640 1 -- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f094c0725e0 msgr2=0x7f094c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:18.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.866+0000 7f0973a00640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f094c0725e0 0x7f094c074aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f096c15ca50 tx=0x7f095800b390 comp rx=0 tx=0).stop 2024-09-10T22:01:18.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.866+0000 7f0973a00640 1 -- 172.21.15.66:0/2170872474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f096c0b1e10 msgr2=0x7f096c159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:18.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.866+0000 7f0973a00640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f096c0b1e10 0x7f096c159c00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f096001ae40 tx=0x7f0960005dd0 comp rx=0 tx=0).stop 2024-09-10T22:01:18.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.866+0000 7f0973a00640 1 -- 172.21.15.66:0/2170872474 shutdown_connections 2024-09-10T22:01:18.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.866+0000 7f0973a00640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f094c0725e0 0x7f094c074aa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:18.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.866+0000 7f0973a00640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c1600b0 0x7f096c0b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:18.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.866+0000 7f0973a00640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f096c15f450 0x7f096c0b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:18.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.866+0000 7f0973a00640 1 --2- 172.21.15.66:0/2170872474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f096c0b1e10 0x7f096c159c00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:18.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.866+0000 7f0973a00640 1 -- 172.21.15.66:0/2170872474 >> 172.21.15.66:0/2170872474 conn(0x7f096c0d1c60 msgr2=0x7f096c0708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:18.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.870+0000 7f0973a00640 1 -- 172.21.15.66:0/2170872474 shutdown_connections 2024-09-10T22:01:18.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:18.870+0000 7f0973a00640 1 -- 172.21.15.66:0/2170872474 wait complete. 2024-09-10T22:01:18.907 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:01:19.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 -- 172.21.15.66:0/671904708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd43015f460 msgr2=0x7fd4300cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 --2- 172.21.15.66:0/671904708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd43015f460 0x7fd4300cfc00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd424009900 tx=0x7fd42401c790 comp rx=0 tx=0).stop 2024-09-10T22:01:19.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 -- 172.21.15.66:0/671904708 shutdown_connections 2024-09-10T22:01:19.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 --2- 172.21.15.66:0/671904708 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd4300b0f60 0x7fd4300d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 --2- 172.21.15.66:0/671904708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd43015f460 0x7fd4300cfc00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 -- 172.21.15.66:0/671904708 >> 172.21.15.66:0/671904708 conn(0x7fd4300d1d00 msgr2=0x7fd4300d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:19.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 -- 172.21.15.66:0/671904708 shutdown_connections 2024-09-10T22:01:19.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 -- 172.21.15.66:0/671904708 wait complete. 2024-09-10T22:01:19.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 Processor -- start 2024-09-10T22:01:19.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 -- start start 2024-09-10T22:01:19.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4300b0f60 0x7fd4300b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:19.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4300b32a0 0x7fd43015e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:19.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd43015f460 0x7fd430148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:19.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd430168ff0 con 0x7fd4300b32a0 2024-09-10T22:01:19.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd430168e70 con 0x7fd4300b0f60 2024-09-10T22:01:19.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd437c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd430169170 con 0x7fd43015f460 2024-09-10T22:01:19.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd42fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4300b32a0 0x7fd43015e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:19.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd42fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4300b32a0 0x7fd43015e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41798/0 (socket says 172.21.15.66:41798) 2024-09-10T22:01:19.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd42fe00640 1 -- 172.21.15.66:0/42639931 learned_addr learned my addr 172.21.15.66:0/42639931 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:19.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd435200640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4300b0f60 0x7fd4300b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:19.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.050+0000 7fd435c00640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd43015f460 0x7fd430148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:19.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd42fe00640 1 -- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd43015f460 msgr2=0x7fd430148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd42fe00640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd43015f460 0x7fd430148eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd42fe00640 1 -- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4300b0f60 msgr2=0x7fd4300b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd42fe00640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4300b0f60 0x7fd4300b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd42fe00640 1 -- 172.21.15.66:0/42639931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4301fc240 con 0x7fd4300b32a0 2024-09-10T22:01:19.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd435200640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4300b0f60 0x7fd4300b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:01:19.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd42fe00640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4300b32a0 0x7fd43015e6e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00e9c0 tx=0x7fd41c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:19.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd42cc00640 1 -- 172.21.15.66:0/42639931 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c00cd20 con 0x7fd4300b32a0 2024-09-10T22:01:19.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd42cc00640 1 -- 172.21.15.66:0/42639931 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd41c005020 con 0x7fd4300b32a0 2024-09-10T22:01:19.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd42cc00640 1 -- 172.21.15.66:0/42639931 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c005470 con 0x7fd4300b32a0 2024-09-10T22:01:19.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd437c00640 1 -- 172.21.15.66:0/42639931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4301fb3e0 con 0x7fd4300b32a0 2024-09-10T22:01:19.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd437c00640 1 -- 172.21.15.66:0/42639931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4301fb9a0 con 0x7fd4300b32a0 2024-09-10T22:01:19.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd437c00640 1 -- 172.21.15.66:0/42639931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4300b14b0 con 0x7fd4300b32a0 2024-09-10T22:01:19.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd42cc00640 1 -- 172.21.15.66:0/42639931 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd41c0047b0 con 0x7fd4300b32a0 2024-09-10T22:01:19.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd42cc00640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd410072610 0x7fd410074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:19.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.054+0000 7fd42cc00640 1 -- 172.21.15.66:0/42639931 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd41c093b00 con 0x7fd4300b32a0 2024-09-10T22:01:19.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.058+0000 7fd435200640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd410072610 0x7fd410074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:19.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.058+0000 7fd435200640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd410072610 0x7fd410074ad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd4240af000 tx=0x7fd42401afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:19.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.058+0000 7fd42cc00640 1 -- 172.21.15.66:0/42639931 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd41c05eb80 con 0x7fd4300b32a0 2024-09-10T22:01:19.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.234+0000 7fd437c00640 1 -- 172.21.15.66:0/42639931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd430072640 con 0x7fd4300b32a0 2024-09-10T22:01:19.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.234+0000 7fd42cc00640 1 -- 172.21.15.66:0/42639931 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16670 (secure 0 0 0) 0x7fd41c093e20 con 0x7fd4300b32a0 2024-09-10T22:01:19.239 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:19.239 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":32,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:01:18.244113+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:01:16.499954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T22:01:17.208498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:01:19.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.238+0000 7fd437c00640 1 -- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd410072610 msgr2=0x7fd410074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.238+0000 7fd437c00640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd410072610 0x7fd410074ad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd4240af000 tx=0x7fd42401afd0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.238+0000 7fd437c00640 1 -- 172.21.15.66:0/42639931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4300b32a0 msgr2=0x7fd43015e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.238+0000 7fd437c00640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4300b32a0 0x7fd43015e6e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00e9c0 tx=0x7fd41c00ee80 comp rx=0 tx=0).stop 2024-09-10T22:01:19.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.238+0000 7fd437c00640 1 -- 172.21.15.66:0/42639931 shutdown_connections 2024-09-10T22:01:19.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.238+0000 7fd437c00640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd410072610 0x7fd410074ad0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.238+0000 7fd437c00640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd43015f460 0x7fd430148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.238+0000 7fd437c00640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4300b32a0 0x7fd43015e6e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.238+0000 7fd437c00640 1 --2- 172.21.15.66:0/42639931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4300b0f60 0x7fd4300b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.238+0000 7fd437c00640 1 -- 172.21.15.66:0/42639931 >> 172.21.15.66:0/42639931 conn(0x7fd4300d1d00 msgr2=0x7fd4300cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:19.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.238+0000 7fd437c00640 1 -- 172.21.15.66:0/42639931 shutdown_connections 2024-09-10T22:01:19.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.238+0000 7fd437c00640 1 -- 172.21.15.66:0/42639931 wait complete. 2024-09-10T22:01:19.261 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:01:19.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.402+0000 7fbf03c00640 1 -- 172.21.15.66:0/431205960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbefc15a310 msgr2=0x7fbefc0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.402+0000 7fbf03c00640 1 --2- 172.21.15.66:0/431205960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbefc15a310 0x7fbefc0c48b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009900 tx=0x7fbef001c850 comp rx=0 tx=0).stop 2024-09-10T22:01:19.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.402+0000 7fbf03c00640 1 -- 172.21.15.66:0/431205960 shutdown_connections 2024-09-10T22:01:19.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.402+0000 7fbf03c00640 1 --2- 172.21.15.66:0/431205960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbefc0aeec0 0x7fbefc0c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.402+0000 7fbf03c00640 1 --2- 172.21.15.66:0/431205960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbefc15a310 0x7fbefc0c48b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.402+0000 7fbf03c00640 1 -- 172.21.15.66:0/431205960 >> 172.21.15.66:0/431205960 conn(0x7fbefc0cfa30 msgr2=0x7fbefc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:19.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.402+0000 7fbf03c00640 1 -- 172.21.15.66:0/431205960 shutdown_connections 2024-09-10T22:01:19.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.402+0000 7fbf03c00640 1 -- 172.21.15.66:0/431205960 wait complete. 2024-09-10T22:01:19.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf03c00640 1 Processor -- start 2024-09-10T22:01:19.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf03c00640 1 -- start start 2024-09-10T22:01:19.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf03c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbefc0aeec0 0x7fbefc1485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:19.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf03c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbefc15a310 0x7fbefc0c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:19.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf03c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbefc0b2940 0x7fbefc0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:19.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf03c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc15edb0 con 0x7fbefc0b2940 2024-09-10T22:01:19.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf03c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc15ec30 con 0x7fbefc15a310 2024-09-10T22:01:19.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf03c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbefc167300 con 0x7fbefc0aeec0 2024-09-10T22:01:19.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf01c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbefc0b2940 0x7fbefc0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:19.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf01c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbefc0b2940 0x7fbefc0c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41808/0 (socket says 172.21.15.66:41808) 2024-09-10T22:01:19.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf01c00640 1 -- 172.21.15.66:0/1001931342 learned_addr learned my addr 172.21.15.66:0/1001931342 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:19.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf01200640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbefc0aeec0 0x7fbefc1485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:19.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbefbe00640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbefc15a310 0x7fbefc0c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf01c00640 1 -- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbefc0aeec0 msgr2=0x7fbefc1485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf01c00640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbefc0aeec0 0x7fbefc1485d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf01c00640 1 -- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbefc15a310 msgr2=0x7fbefc0c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf01c00640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbefc15a310 0x7fbefc0c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf01c00640 1 -- 172.21.15.66:0/1001931342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc14e570 con 0x7fbefc0b2940 2024-09-10T22:01:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf01c00640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbefc0b2940 0x7fbefc0c6fb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbef400ec90 tx=0x7fbef400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbef8c00640 1 -- 172.21.15.66:0/1001931342 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4004d30 con 0x7fbefc0b2940 2024-09-10T22:01:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbef8c00640 1 -- 172.21.15.66:0/1001931342 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbef400b570 con 0x7fbefc0b2940 2024-09-10T22:01:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbef8c00640 1 -- 172.21.15.66:0/1001931342 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef400b980 con 0x7fbefc0b2940 2024-09-10T22:01:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf03c00640 1 -- 172.21.15.66:0/1001931342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbefc0c96b0 con 0x7fbefc0b2940 2024-09-10T22:01:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.406+0000 7fbf03c00640 1 -- 172.21.15.66:0/1001931342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbefc14c3a0 con 0x7fbefc0b2940 2024-09-10T22:01:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.410+0000 7fbf03c00640 1 -- 172.21.15.66:0/1001931342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbefc15df60 con 0x7fbefc0b2940 2024-09-10T22:01:19.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.410+0000 7fbef8c00640 1 -- 172.21.15.66:0/1001931342 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbef4015070 con 0x7fbefc0b2940 2024-09-10T22:01:19.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.410+0000 7fbef8c00640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbedc072610 0x7fbedc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:19.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.410+0000 7fbef8c00640 1 -- 172.21.15.66:0/1001931342 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fbef4093a60 con 0x7fbefc0b2940 2024-09-10T22:01:19.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.410+0000 7fbf01200640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbedc072610 0x7fbedc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:19.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.410+0000 7fbf01200640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbedc072610 0x7fbedc074ad0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbef00ae000 tx=0x7fbef001b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:19.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.414+0000 7fbef8c00640 1 -- 172.21.15.66:0/1001931342 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbef405e9c0 con 0x7fbefc0b2940 2024-09-10T22:01:19.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.590+0000 7fbf03c00640 1 -- 172.21.15.66:0/1001931342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbefc0b12b0 con 0x7fbefc0b2940 2024-09-10T22:01:19.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.590+0000 7fbef8c00640 1 -- 172.21.15.66:0/1001931342 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16670 (secure 0 0 0) 0x7fbef40633f0 con 0x7fbefc0b2940 2024-09-10T22:01:19.596 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:19.596 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":32,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:01:18.244113+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:01:16.499954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T22:01:17.208498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:01:19.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.594+0000 7fbf03c00640 1 -- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbedc072610 msgr2=0x7fbedc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.594+0000 7fbf03c00640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbedc072610 0x7fbedc074ad0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbef00ae000 tx=0x7fbef001b090 comp rx=0 tx=0).stop 2024-09-10T22:01:19.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.594+0000 7fbf03c00640 1 -- 172.21.15.66:0/1001931342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbefc0b2940 msgr2=0x7fbefc0c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.594+0000 7fbf03c00640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbefc0b2940 0x7fbefc0c6fb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbef400ec90 tx=0x7fbef400c7a0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.594+0000 7fbf03c00640 1 -- 172.21.15.66:0/1001931342 shutdown_connections 2024-09-10T22:01:19.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.594+0000 7fbf03c00640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbedc072610 0x7fbedc074ad0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.594+0000 7fbf03c00640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbefc0b2940 0x7fbefc0c6fb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.594+0000 7fbf03c00640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbefc15a310 0x7fbefc0c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.594+0000 7fbf03c00640 1 --2- 172.21.15.66:0/1001931342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbefc0aeec0 0x7fbefc1485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.594+0000 7fbf03c00640 1 -- 172.21.15.66:0/1001931342 >> 172.21.15.66:0/1001931342 conn(0x7fbefc0cfa30 msgr2=0x7fbefc0af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:19.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.594+0000 7fbf03c00640 1 -- 172.21.15.66:0/1001931342 shutdown_connections 2024-09-10T22:01:19.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.594+0000 7fbf03c00640 1 -- 172.21.15.66:0/1001931342 wait complete. 2024-09-10T22:01:19.619 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:01:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 -- 172.21.15.66:0/3424700544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b4c15f340 msgr2=0x7f3b4c0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 --2- 172.21.15.66:0/3424700544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b4c15f340 0x7f3b4c0cfb50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009900 tx=0x7f3b3c01c850 comp rx=0 tx=0).stop 2024-09-10T22:01:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 -- 172.21.15.66:0/3424700544 shutdown_connections 2024-09-10T22:01:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 --2- 172.21.15.66:0/3424700544 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b4c0b2bf0 0x7f3b4c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 --2- 172.21.15.66:0/3424700544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b4c15f340 0x7f3b4c0cfb50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 -- 172.21.15.66:0/3424700544 >> 172.21.15.66:0/3424700544 conn(0x7f3b4c0d1c50 msgr2=0x7f3b4c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 -- 172.21.15.66:0/3424700544 shutdown_connections 2024-09-10T22:01:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 -- 172.21.15.66:0/3424700544 wait complete. 2024-09-10T22:01:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 Processor -- start 2024-09-10T22:01:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 -- start start 2024-09-10T22:01:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b4c0b2bf0 0x7f3b4c1fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b4c0d2770 0x7f3b4c165320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b4c15f340 0x7f3b4c157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c169190 con 0x7f3b4c15f340 2024-09-10T22:01:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c169010 con 0x7f3b4c0d2770 2024-09-10T22:01:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b52800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c169310 con 0x7f3b4c0b2bf0 2024-09-10T22:01:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b4c15f340 0x7f3b4c157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:19.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b4b400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b4c0b2bf0 0x7f3b4c1fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:19.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b4c15f340 0x7f3b4c157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41826/0 (socket says 172.21.15.66:41826) 2024-09-10T22:01:19.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b4be00640 1 -- 172.21.15.66:0/2989621635 learned_addr learned my addr 172.21.15.66:0/2989621635 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:19.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.766+0000 7f3b42a00640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b4c0d2770 0x7f3b4c165320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:19.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b4b400640 1 -- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b4c0d2770 msgr2=0x7f3b4c165320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b4b400640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b4c0d2770 0x7f3b4c165320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b4b400640 1 -- 172.21.15.66:0/2989621635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b4c15f340 msgr2=0x7f3b4c157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:19.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b4b400640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b4c15f340 0x7f3b4c157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:19.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b4b400640 1 -- 172.21.15.66:0/2989621635 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c1f9090 con 0x7f3b4c0b2bf0 2024-09-10T22:01:19.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b4be00640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b4c15f340 0x7f3b4c157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:01:19.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b4b400640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b4c0b2bf0 0x7f3b4c1fc0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0ac000 tx=0x7f3b3c005d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:19.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b43e00640 1 -- 172.21.15.66:0/2989621635 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0b2040 con 0x7f3b4c0b2bf0 2024-09-10T22:01:19.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b52800640 1 -- 172.21.15.66:0/2989621635 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c1f8510 con 0x7f3b4c0b2bf0 2024-09-10T22:01:19.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b43e00640 1 -- 172.21.15.66:0/2989621635 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b3c004130 con 0x7f3b4c0b2bf0 2024-09-10T22:01:19.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b43e00640 1 -- 172.21.15.66:0/2989621635 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c003600 con 0x7f3b4c0b2bf0 2024-09-10T22:01:19.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b52800640 1 -- 172.21.15.66:0/2989621635 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c1f8ad0 con 0x7f3b4c0b2bf0 2024-09-10T22:01:19.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b52800640 1 -- 172.21.15.66:0/2989621635 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c0ce380 con 0x7f3b4c0b2bf0 2024-09-10T22:01:19.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.770+0000 7f3b43e00640 1 -- 172.21.15.66:0/2989621635 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3b3c0037a0 con 0x7f3b4c0b2bf0 2024-09-10T22:01:19.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.774+0000 7f3b43e00640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3b280725e0 0x7f3b28074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:19.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.774+0000 7f3b43e00640 1 -- 172.21.15.66:0/2989621635 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3b3c12dfc0 con 0x7f3b4c0b2bf0 2024-09-10T22:01:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.774+0000 7f3b42a00640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3b280725e0 0x7f3b28074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.774+0000 7f3b42a00640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3b280725e0 0x7f3b28074aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0ce0f0 tx=0x7f3b3400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:19.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.774+0000 7f3b43e00640 1 -- 172.21.15.66:0/2989621635 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3b4c0ce380 con 0x7f3b4c0b2bf0 2024-09-10T22:01:19.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:19.990+0000 7f3b52800640 1 -- 172.21.15.66:0/2989621635 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3b4c15faa0 con 0x7f3b4c0b2bf0 2024-09-10T22:01:20.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.870+0000 7f3b43e00640 1 -- 172.21.15.66:0/2989621635 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f3b3c0164d0 con 0x7f3b4c0b2bf0 2024-09-10T22:01:20.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.874+0000 7f3b52800640 1 -- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3b280725e0 msgr2=0x7f3b28074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:20.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.874+0000 7f3b52800640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3b280725e0 0x7f3b28074aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0ce0f0 tx=0x7f3b3400b390 comp rx=0 tx=0).stop 2024-09-10T22:01:20.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.874+0000 7f3b52800640 1 -- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b4c0b2bf0 msgr2=0x7f3b4c1fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:20.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.874+0000 7f3b52800640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b4c0b2bf0 0x7f3b4c1fc0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0ac000 tx=0x7f3b3c005d60 comp rx=0 tx=0).stop 2024-09-10T22:01:20.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.874+0000 7f3b52800640 1 -- 172.21.15.66:0/2989621635 shutdown_connections 2024-09-10T22:01:20.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.874+0000 7f3b52800640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3b280725e0 0x7f3b28074aa0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.874+0000 7f3b52800640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b4c15f340 0x7f3b4c157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.874+0000 7f3b52800640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b4c0d2770 0x7f3b4c165320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.874+0000 7f3b52800640 1 --2- 172.21.15.66:0/2989621635 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b4c0b2bf0 0x7f3b4c1fc0c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.874+0000 7f3b52800640 1 -- 172.21.15.66:0/2989621635 >> 172.21.15.66:0/2989621635 conn(0x7f3b4c0d1c50 msgr2=0x7f3b4c15b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:20.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.874+0000 7f3b52800640 1 -- 172.21.15.66:0/2989621635 shutdown_connections 2024-09-10T22:01:20.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:20.874+0000 7f3b52800640 1 -- 172.21.15.66:0/2989621635 wait complete. 2024-09-10T22:01:20.898 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:01:20.899 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:01:21.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.046+0000 7f19e9600640 1 -- 172.21.15.66:0/2392939472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e40aeec0 msgr2=0x7f19e415a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:21.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.046+0000 7f19e9600640 1 --2- 172.21.15.66:0/2392939472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e40aeec0 0x7f19e415a310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009900 tx=0x7f19cc01c7f0 comp rx=0 tx=0).stop 2024-09-10T22:01:21.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.046+0000 7f19e9600640 1 -- 172.21.15.66:0/2392939472 shutdown_connections 2024-09-10T22:01:21.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.046+0000 7f19e9600640 1 --2- 172.21.15.66:0/2392939472 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e40ae8e0 0x7f19e40c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:21.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.046+0000 7f19e9600640 1 --2- 172.21.15.66:0/2392939472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e40aeec0 0x7f19e415a310 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:21.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.046+0000 7f19e9600640 1 -- 172.21.15.66:0/2392939472 >> 172.21.15.66:0/2392939472 conn(0x7f19e40cfa30 msgr2=0x7f19e40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:21.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.046+0000 7f19e9600640 1 -- 172.21.15.66:0/2392939472 shutdown_connections 2024-09-10T22:01:21.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.046+0000 7f19e9600640 1 -- 172.21.15.66:0/2392939472 wait complete. 2024-09-10T22:01:21.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e9600640 1 Processor -- start 2024-09-10T22:01:21.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e9600640 1 -- start start 2024-09-10T22:01:21.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e9600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ae8e0 0x7f19e40b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:21.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e9600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e40aeec0 0x7f19e40b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:21.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e9600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e40ae020 0x7f19e40c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:21.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e9600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4167f50 con 0x7f19e40ae8e0 2024-09-10T22:01:21.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e9600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4167dd0 con 0x7f19e40aeec0 2024-09-10T22:01:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e9600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f19e41680d0 con 0x7f19e40ae020 2024-09-10T22:01:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ae8e0 0x7f19e40b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ae8e0 0x7f19e40b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41834/0 (socket says 172.21.15.66:41834) 2024-09-10T22:01:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e2a00640 1 -- 172.21.15.66:0/2625432011 learned_addr learned my addr 172.21.15.66:0/2625432011 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19da000640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e40aeec0 0x7f19e40b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e3400640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e40ae020 0x7f19e40c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:21.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e2a00640 1 -- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e40ae020 msgr2=0x7f19e40c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:21.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e2a00640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e40ae020 0x7f19e40c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:21.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e2a00640 1 -- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e40aeec0 msgr2=0x7f19e40b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:21.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e2a00640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e40aeec0 0x7f19e40b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:21.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e2a00640 1 -- 172.21.15.66:0/2625432011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e4157a40 con 0x7f19e40ae8e0 2024-09-10T22:01:21.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e2a00640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ae8e0 0x7f19e40b05c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f19d400c3c0 tx=0x7f19d400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:21.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19db400640 1 -- 172.21.15.66:0/2625432011 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4017070 con 0x7f19e40ae8e0 2024-09-10T22:01:21.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19db400640 1 -- 172.21.15.66:0/2625432011 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d40053c0 con 0x7f19e40ae8e0 2024-09-10T22:01:21.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e9600640 1 -- 172.21.15.66:0/2625432011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e4157d20 con 0x7f19e40ae8e0 2024-09-10T22:01:21.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19db400640 1 -- 172.21.15.66:0/2625432011 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4003860 con 0x7f19e40ae8e0 2024-09-10T22:01:21.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e9600640 1 -- 172.21.15.66:0/2625432011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e41582e0 con 0x7f19e40ae8e0 2024-09-10T22:01:21.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.050+0000 7f19e9600640 1 -- 172.21.15.66:0/2625432011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e40c48b0 con 0x7f19e40ae8e0 2024-09-10T22:01:21.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.054+0000 7f19db400640 1 -- 172.21.15.66:0/2625432011 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f19d4003a00 con 0x7f19e40ae8e0 2024-09-10T22:01:21.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.054+0000 7f19db400640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19c00725e0 0x7f19c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:21.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.054+0000 7f19db400640 1 -- 172.21.15.66:0/2625432011 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f19d4092860 con 0x7f19e40ae8e0 2024-09-10T22:01:21.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.054+0000 7f19da000640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19c00725e0 0x7f19c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:21.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.054+0000 7f19da000640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19c00725e0 0x7f19c0074aa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0af000 tx=0x7f19cc01b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:21.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.058+0000 7f19db400640 1 -- 172.21.15.66:0/2625432011 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f19d405d8e0 con 0x7f19e40ae8e0 2024-09-10T22:01:21.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.266+0000 7f19e9600640 1 -- 172.21.15.66:0/2625432011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f19e40ce1e0 con 0x7f19e40ae8e0 2024-09-10T22:01:21.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.874+0000 7f19db400640 1 -- 172.21.15.66:0/2625432011 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f19d4062310 con 0x7f19e40ae8e0 2024-09-10T22:01:21.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.878+0000 7f19e9600640 1 -- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19c00725e0 msgr2=0x7f19c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:21.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.878+0000 7f19e9600640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19c00725e0 0x7f19c0074aa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0af000 tx=0x7f19cc01b090 comp rx=0 tx=0).stop 2024-09-10T22:01:21.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.878+0000 7f19e9600640 1 -- 172.21.15.66:0/2625432011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ae8e0 msgr2=0x7f19e40b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:21.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.878+0000 7f19e9600640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ae8e0 0x7f19e40b05c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f19d400c3c0 tx=0x7f19d400c7c0 comp rx=0 tx=0).stop 2024-09-10T22:01:21.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.878+0000 7f19e9600640 1 -- 172.21.15.66:0/2625432011 shutdown_connections 2024-09-10T22:01:21.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.878+0000 7f19e9600640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19c00725e0 0x7f19c0074aa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:21.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.878+0000 7f19e9600640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e40ae020 0x7f19e40c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:21.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.878+0000 7f19e9600640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e40aeec0 0x7f19e40b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:21.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.878+0000 7f19e9600640 1 --2- 172.21.15.66:0/2625432011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ae8e0 0x7f19e40b05c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:21.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.878+0000 7f19e9600640 1 -- 172.21.15.66:0/2625432011 >> 172.21.15.66:0/2625432011 conn(0x7f19e40cfa30 msgr2=0x7f19e40ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:21.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.878+0000 7f19e9600640 1 -- 172.21.15.66:0/2625432011 shutdown_connections 2024-09-10T22:01:21.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:21.878+0000 7f19e9600640 1 -- 172.21.15.66:0/2625432011 wait complete. 2024-09-10T22:01:21.904 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:01:21.905 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:01:22.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.050+0000 7f0a99e00640 1 -- 172.21.15.66:0/853082838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a78004d50 msgr2=0x7f0a780051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.050+0000 7f0a99e00640 1 --2- 172.21.15.66:0/853082838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a78004d50 0x7f0a780051d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c001dc0 tx=0x7f0a8c01cb30 comp rx=0 tx=0).stop 2024-09-10T22:01:22.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.050+0000 7f0a93400640 1 -- 172.21.15.66:0/853082838 reap_dead start 2024-09-10T22:01:22.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.050+0000 7f0a99e00640 1 -- 172.21.15.66:0/853082838 shutdown_connections 2024-09-10T22:01:22.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.050+0000 7f0a99e00640 1 -- 172.21.15.66:0/853082838 >> 172.21.15.66:0/853082838 conn(0x7f0a940d1c60 msgr2=0x7f0a940d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:22.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.050+0000 7f0a99e00640 1 -- 172.21.15.66:0/853082838 shutdown_connections 2024-09-10T22:01:22.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.050+0000 7f0a99e00640 1 -- 172.21.15.66:0/853082838 wait complete. 2024-09-10T22:01:22.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a99e00640 1 Processor -- start 2024-09-10T22:01:22.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a99e00640 1 -- start start 2024-09-10T22:01:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a99e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a78004d50 0x7f0a940c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a99e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a94071ba0 0x7f0a940c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a99e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a94072020 0x7f0a94151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a99e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0a940ce170 con 0x7f0a94072020 2024-09-10T22:01:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a99e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a940cfb80 con 0x7f0a94071ba0 2024-09-10T22:01:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a99e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0a940ce2f0 con 0x7f0a78004d50 2024-09-10T22:01:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a93400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a94072020 0x7f0a94151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a93400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a94072020 0x7f0a94151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41850/0 (socket says 172.21.15.66:41850) 2024-09-10T22:01:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a93400640 1 -- 172.21.15.66:0/4006335705 learned_addr learned my addr 172.21.15.66:0/4006335705 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a92a00640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a78004d50 0x7f0a940c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a92000640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a94071ba0 0x7f0a940c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a93400640 1 -- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a78004d50 msgr2=0x7f0a940c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a93400640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a78004d50 0x7f0a940c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a93400640 1 -- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a94071ba0 msgr2=0x7f0a940c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a93400640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a94071ba0 0x7f0a940c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a93400640 1 -- 172.21.15.66:0/4006335705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a940b6d10 con 0x7f0a94072020 2024-09-10T22:01:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a93400640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a94072020 0x7f0a94151cd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c001dc0 tx=0x7f0a8c01cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a8b400640 1 -- 172.21.15.66:0/4006335705 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c006cb0 con 0x7f0a94072020 2024-09-10T22:01:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a8b400640 1 -- 172.21.15.66:0/4006335705 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a8c003c70 con 0x7f0a94072020 2024-09-10T22:01:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a8b400640 1 -- 172.21.15.66:0/4006335705 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c003e60 con 0x7f0a94072020 2024-09-10T22:01:22.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a99e00640 1 -- 172.21.15.66:0/4006335705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a940b6ff0 con 0x7f0a94072020 2024-09-10T22:01:22.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.054+0000 7f0a99e00640 1 -- 172.21.15.66:0/4006335705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a940b4bc0 con 0x7f0a94072020 2024-09-10T22:01:22.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.058+0000 7f0a99e00640 1 -- 172.21.15.66:0/4006335705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a940cffa0 con 0x7f0a94072020 2024-09-10T22:01:22.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.058+0000 7f0a8b400640 1 -- 172.21.15.66:0/4006335705 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0a8c006330 con 0x7f0a94072020 2024-09-10T22:01:22.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.058+0000 7f0a8b400640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0a740725e0 0x7f0a74074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.058+0000 7f0a8b400640 1 -- 172.21.15.66:0/4006335705 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0a8c12c790 con 0x7f0a94072020 2024-09-10T22:01:22.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.058+0000 7f0a92a00640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0a740725e0 0x7f0a74074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.058+0000 7f0a92a00640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0a740725e0 0x7f0a74074aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0a84003fb0 tx=0x7f0a840067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:22.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.062+0000 7f0a8b400640 1 -- 172.21.15.66:0/4006335705 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0a8c0b4070 con 0x7f0a94072020 2024-09-10T22:01:22.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.270+0000 7f0a99e00640 1 -- 172.21.15.66:0/4006335705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a940b0df0 con 0x7f0a94072020 2024-09-10T22:01:22.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.270+0000 7f0a8b400640 1 -- 172.21.15.66:0/4006335705 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5217 (secure 0 0 0) 0x7f0a8c016890 con 0x7f0a94072020 2024-09-10T22:01:22.275 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:22.275 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"btime":"2024-09-10T22:01:21:872011+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1019192175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1019192175},{"type":"v1","addr":"172.21.15.94:6837","nonce":1019192175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/554383263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":554383263},{"type":"v1","addr":"172.21.15.94:6836","nonce":554383263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2368842390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2368842390},{"type":"v1","addr":"172.21.15.66:6834","nonce":2368842390}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:01:18.244077+0000","modified":"2024-09-10T22:01:21.271158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/297228676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":297228676},{"type":"v1","addr":"172.21.15.66:6835","nonce":297228676}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-09-10T22:01:22.275 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-09-10T22:01:22.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.274+0000 7f0a99e00640 1 -- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0a740725e0 msgr2=0x7f0a74074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.274+0000 7f0a99e00640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0a740725e0 0x7f0a74074aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0a84003fb0 tx=0x7f0a840067b0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.274+0000 7f0a99e00640 1 -- 172.21.15.66:0/4006335705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a94072020 msgr2=0x7f0a94151cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.274+0000 7f0a99e00640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a94072020 0x7f0a94151cd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c001dc0 tx=0x7f0a8c01cd80 comp rx=0 tx=0).stop 2024-09-10T22:01:22.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.274+0000 7f0a99e00640 1 -- 172.21.15.66:0/4006335705 shutdown_connections 2024-09-10T22:01:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.274+0000 7f0a99e00640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0a740725e0 0x7f0a74074aa0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.274+0000 7f0a99e00640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a94072020 0x7f0a94151cd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.274+0000 7f0a99e00640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a94071ba0 0x7f0a940c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.274+0000 7f0a99e00640 1 --2- 172.21.15.66:0/4006335705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a78004d50 0x7f0a940c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.274+0000 7f0a99e00640 1 -- 172.21.15.66:0/4006335705 >> 172.21.15.66:0/4006335705 conn(0x7f0a940d1c60 msgr2=0x7f0a940b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.274+0000 7f0a99e00640 1 -- 172.21.15.66:0/4006335705 shutdown_connections 2024-09-10T22:01:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.274+0000 7f0a99e00640 1 -- 172.21.15.66:0/4006335705 wait complete. 2024-09-10T22:01:22.296 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:01:22.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.438+0000 7fa3d2c00640 1 -- 172.21.15.66:0/4064906847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc0d06b0 msgr2=0x7fa3cc0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.438+0000 7fa3d2c00640 1 --2- 172.21.15.66:0/4064906847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc0d06b0 0x7fa3cc0ce170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009840 tx=0x7fa3b801c6b0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.438+0000 7fa3d2c00640 1 -- 172.21.15.66:0/4064906847 shutdown_connections 2024-09-10T22:01:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.438+0000 7fa3d2c00640 1 --2- 172.21.15.66:0/4064906847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc0d06b0 0x7fa3cc0ce170 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.438+0000 7fa3d2c00640 1 -- 172.21.15.66:0/4064906847 >> 172.21.15.66:0/4064906847 conn(0x7fa3cc0d1c60 msgr2=0x7fa3cc0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.438+0000 7fa3d2c00640 1 -- 172.21.15.66:0/4064906847 shutdown_connections 2024-09-10T22:01:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.438+0000 7fa3d2c00640 1 -- 172.21.15.66:0/4064906847 wait complete. 2024-09-10T22:01:22.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d2c00640 1 Processor -- start 2024-09-10T22:01:22.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d2c00640 1 -- start start 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d2c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3cc0afc20 0x7fa3cc15b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d2c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc0d06b0 0x7fa3cc1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d2c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3cc15bd40 0x7fa3cc159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d2c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc16a0c0 con 0x7fa3cc15bd40 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d2c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc169f40 con 0x7fa3cc0d06b0 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d2c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa3cc16a240 con 0x7fa3cc0afc20 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3cc15bd40 0x7fa3cc159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3cc15bd40 0x7fa3cc159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40356/0 (socket says 172.21.15.66:40356) 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d0c00640 1 -- 172.21.15.66:0/1044003050 learned_addr learned my addr 172.21.15.66:0/1044003050 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3cbe00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3cc0afc20 0x7fa3cc15b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3cb400640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc0d06b0 0x7fa3cc1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d0c00640 1 -- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3cc0afc20 msgr2=0x7fa3cc15b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d0c00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3cc0afc20 0x7fa3cc15b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d0c00640 1 -- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc0d06b0 msgr2=0x7fa3cc1566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d0c00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc0d06b0 0x7fa3cc1566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d0c00640 1 -- 172.21.15.66:0/1044003050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3cc15b840 con 0x7fa3cc15bd40 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3cb400640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc0d06b0 0x7fa3cc1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3cbe00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3cc0afc20 0x7fa3cc15b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:22.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d0c00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3cc15bd40 0x7fa3cc159fe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400c780 tx=0x7fa3c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:22.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1044003050 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4003b80 con 0x7fa3cc15bd40 2024-09-10T22:01:22.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1044003050 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3c4003d20 con 0x7fa3cc15bd40 2024-09-10T22:01:22.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1044003050 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c400aab0 con 0x7fa3cc15bd40 2024-09-10T22:01:22.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d2c00640 1 -- 172.21.15.66:0/1044003050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3cc1f92e0 con 0x7fa3cc15bd40 2024-09-10T22:01:22.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.442+0000 7fa3d2c00640 1 -- 172.21.15.66:0/1044003050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3cc1f9b80 con 0x7fa3cc15bd40 2024-09-10T22:01:22.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.446+0000 7fa3d2c00640 1 -- 172.21.15.66:0/1044003050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3cc0cffa0 con 0x7fa3cc15bd40 2024-09-10T22:01:22.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.446+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1044003050 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa3c400ac50 con 0x7fa3cc15bd40 2024-09-10T22:01:22.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.446+0000 7fa3c3e00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa3ac0725e0 0x7fa3ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.446+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1044003050 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa3c40930e0 con 0x7fa3cc15bd40 2024-09-10T22:01:22.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.446+0000 7fa3cbe00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa3ac0725e0 0x7fa3ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.446+0000 7fa3cbe00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa3ac0725e0 0x7fa3ac074aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc001230 tx=0x7fa3bc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:22.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.450+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1044003050 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa3c405e7b0 con 0x7fa3cc15bd40 2024-09-10T22:01:22.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.622+0000 7fa3d2c00640 1 -- 172.21.15.66:0/1044003050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3cc0b0df0 con 0x7fa3cc15bd40 2024-09-10T22:01:22.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1044003050 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16657 (secure 0 0 0) 0x7fa3c405e0b0 con 0x7fa3cc15bd40 2024-09-10T22:01:22.629 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:22.629 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":33,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:01:19.860933+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:01:16.499954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T22:01:17.208498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:01:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3d2c00640 1 -- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa3ac0725e0 msgr2=0x7fa3ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3d2c00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa3ac0725e0 0x7fa3ac074aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc001230 tx=0x7fa3bc009040 comp rx=0 tx=0).stop 2024-09-10T22:01:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3d2c00640 1 -- 172.21.15.66:0/1044003050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3cc15bd40 msgr2=0x7fa3cc159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3d2c00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3cc15bd40 0x7fa3cc159fe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400c780 tx=0x7fa3c400cc40 comp rx=0 tx=0).stop 2024-09-10T22:01:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3d2c00640 1 -- 172.21.15.66:0/1044003050 shutdown_connections 2024-09-10T22:01:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3d2c00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa3ac0725e0 0x7fa3ac074aa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3d2c00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3cc15bd40 0x7fa3cc159fe0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3d2c00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc0d06b0 0x7fa3cc1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3d2c00640 1 --2- 172.21.15.66:0/1044003050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3cc0afc20 0x7fa3cc15b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3d2c00640 1 -- 172.21.15.66:0/1044003050 >> 172.21.15.66:0/1044003050 conn(0x7fa3cc0d1c60 msgr2=0x7fa3cc157c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3d2c00640 1 -- 172.21.15.66:0/1044003050 shutdown_connections 2024-09-10T22:01:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.626+0000 7fa3d2c00640 1 -- 172.21.15.66:0/1044003050 wait complete. 2024-09-10T22:01:22.651 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:01:22.651 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:01:22.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 -- 172.21.15.66:0/2080992805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f246415cc80 msgr2=0x7f24640ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 --2- 172.21.15.66:0/2080992805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f246415cc80 0x7f24640ae710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f244c009900 tx=0x7f244c01c850 comp rx=0 tx=0).stop 2024-09-10T22:01:22.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 -- 172.21.15.66:0/2080992805 shutdown_connections 2024-09-10T22:01:22.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 --2- 172.21.15.66:0/2080992805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f246415cc80 0x7f24640ae710 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 --2- 172.21.15.66:0/2080992805 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2464159900 0x7f24640ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 -- 172.21.15.66:0/2080992805 >> 172.21.15.66:0/2080992805 conn(0x7f24640cfa30 msgr2=0x7f24640cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:22.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 -- 172.21.15.66:0/2080992805 shutdown_connections 2024-09-10T22:01:22.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 -- 172.21.15.66:0/2080992805 wait complete. 2024-09-10T22:01:22.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 Processor -- start 2024-09-10T22:01:22.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 -- start start 2024-09-10T22:01:22.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24640aec50 0x7f24641fb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2464159900 0x7f2464147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f246415cc80 0x7f246415ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2464168f70 con 0x7f246415cc80 2024-09-10T22:01:22.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2464168df0 con 0x7f24640aec50 2024-09-10T22:01:22.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2469a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f24641690f0 con 0x7f2464159900 2024-09-10T22:01:22.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2463400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f246415cc80 0x7f246415ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2463400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f246415cc80 0x7f246415ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40378/0 (socket says 172.21.15.66:40378) 2024-09-10T22:01:22.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2463400640 1 -- 172.21.15.66:0/851177294 learned_addr learned my addr 172.21.15.66:0/851177294 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:22.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.794+0000 7f2462a00640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24640aec50 0x7f24641fb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f245be00640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2464159900 0x7f2464147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f2463400640 1 -- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2464159900 msgr2=0x7f2464147390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f2463400640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2464159900 0x7f2464147390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f2463400640 1 -- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24640aec50 msgr2=0x7f24641fb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f2463400640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24640aec50 0x7f24641fb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f2463400640 1 -- 172.21.15.66:0/851177294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246415edf0 con 0x7f246415cc80 2024-09-10T22:01:22.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f2463400640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f246415cc80 0x7f246415ba70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f245c00c7c0 tx=0x7f245c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:22.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f245aa00640 1 -- 172.21.15.66:0/851177294 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c003b80 con 0x7f246415cc80 2024-09-10T22:01:22.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f245aa00640 1 -- 172.21.15.66:0/851177294 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f245c003d20 con 0x7f246415cc80 2024-09-10T22:01:22.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f245aa00640 1 -- 172.21.15.66:0/851177294 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c00aab0 con 0x7f246415cc80 2024-09-10T22:01:22.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f2469a00640 1 -- 172.21.15.66:0/851177294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2464159e70 con 0x7f246415cc80 2024-09-10T22:01:22.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f2469a00640 1 -- 172.21.15.66:0/851177294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246415f090 con 0x7f246415cc80 2024-09-10T22:01:22.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f2469a00640 1 -- 172.21.15.66:0/851177294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24640ca080 con 0x7f246415cc80 2024-09-10T22:01:22.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.798+0000 7f245aa00640 1 -- 172.21.15.66:0/851177294 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f245c00ac50 con 0x7f246415cc80 2024-09-10T22:01:22.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.802+0000 7f245aa00640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24400725e0 0x7f2440074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.802+0000 7f245aa00640 1 -- 172.21.15.66:0/851177294 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f245c093070 con 0x7f246415cc80 2024-09-10T22:01:22.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.802+0000 7f2462a00640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24400725e0 0x7f2440074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.802+0000 7f2462a00640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24400725e0 0x7f2440074aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2464158be0 tx=0x7f2454009110 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:22.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.802+0000 7f245aa00640 1 -- 172.21.15.66:0/851177294 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f246415d0e0 con 0x7f246415cc80 2024-09-10T22:01:22.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.978+0000 7f2469a00640 1 -- 172.21.15.66:0/851177294 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f246415a200 con 0x7f24400725e0 2024-09-10T22:01:22.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.982+0000 7f245aa00640 1 -- 172.21.15.66:0/851177294 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7f246415a200 con 0x7f24400725e0 2024-09-10T22:01:22.989 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:22.990 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:01:22.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.986+0000 7f2469a00640 1 -- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24400725e0 msgr2=0x7f2440074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.986+0000 7f2469a00640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24400725e0 0x7f2440074aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2464158be0 tx=0x7f2454009110 comp rx=0 tx=0).stop 2024-09-10T22:01:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.986+0000 7f2469a00640 1 -- 172.21.15.66:0/851177294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f246415cc80 msgr2=0x7f246415ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.986+0000 7f2469a00640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f246415cc80 0x7f246415ba70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f245c00c7c0 tx=0x7f245c00cc80 comp rx=0 tx=0).stop 2024-09-10T22:01:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.986+0000 7f2469a00640 1 -- 172.21.15.66:0/851177294 shutdown_connections 2024-09-10T22:01:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.986+0000 7f2469a00640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24400725e0 0x7f2440074aa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.986+0000 7f2469a00640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f246415cc80 0x7f246415ba70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.986+0000 7f2469a00640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2464159900 0x7f2464147390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.986+0000 7f2469a00640 1 --2- 172.21.15.66:0/851177294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24640aec50 0x7f24641fb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.986+0000 7f2469a00640 1 -- 172.21.15.66:0/851177294 >> 172.21.15.66:0/851177294 conn(0x7f24640cfa30 msgr2=0x7f246406ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:22.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.986+0000 7f2469a00640 1 -- 172.21.15.66:0/851177294 shutdown_connections 2024-09-10T22:01:22.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:22.986+0000 7f2469a00640 1 -- 172.21.15.66:0/851177294 wait complete. 2024-09-10T22:01:23.009 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-10T22:01:22.321715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236976,"kb_used_data":4072,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737744,"statfs":{"total":773094113280,"available":772851449856,"internally_reserved":0,"allocated":4169728,"data_stored":2301364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.489981"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869646+0000","last_change":"2024-09-10T22:01:18.873322+0000","last_active":"2024-09-10T22:01:19.869646+0000","last_peered":"2024-09-10T22:01:19.869646+0000","last_clean":"2024-09-10T22:01:19.869646+0000","last_became_active":"2024-09-10T22:01:18.873061+0000","last_became_peered":"2024-09-10T22:01:18.873061+0000","last_unstale":"2024-09-10T22:01:19.869646+0000","last_undegraded":"2024-09-10T22:01:19.869646+0000","last_fullsized":"2024-09-10T22:01:19.869646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:02.950345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869402+0000","last_change":"2024-09-10T22:01:18.872583+0000","last_active":"2024-09-10T22:01:19.869402+0000","last_peered":"2024-09-10T22:01:19.869402+0000","last_clean":"2024-09-10T22:01:19.869402+0000","last_became_active":"2024-09-10T22:01:18.872345+0000","last_became_peered":"2024-09-10T22:01:18.872345+0000","last_unstale":"2024-09-10T22:01:19.869402+0000","last_undegraded":"2024-09-10T22:01:19.869402+0000","last_fullsized":"2024-09-10T22:01:19.869402+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:41.792870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971394+0000","last_change":"2024-09-10T22:01:18.870777+0000","last_active":"2024-09-10T22:01:19.971394+0000","last_peered":"2024-09-10T22:01:19.971394+0000","last_clean":"2024-09-10T22:01:19.971394+0000","last_became_active":"2024-09-10T22:01:18.870107+0000","last_became_peered":"2024-09-10T22:01:18.870107+0000","last_unstale":"2024-09-10T22:01:19.971394+0000","last_undegraded":"2024-09-10T22:01:19.971394+0000","last_fullsized":"2024-09-10T22:01:19.971394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:10.233265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877138+0000","last_change":"2024-09-10T22:01:18.864672+0000","last_active":"2024-09-10T22:01:19.877138+0000","last_peered":"2024-09-10T22:01:19.877138+0000","last_clean":"2024-09-10T22:01:19.877138+0000","last_became_active":"2024-09-10T22:01:18.864345+0000","last_became_peered":"2024-09-10T22:01:18.864345+0000","last_unstale":"2024-09-10T22:01:19.877138+0000","last_undegraded":"2024-09-10T22:01:19.877138+0000","last_fullsized":"2024-09-10T22:01:19.877138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:08.025235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131283+0000","last_change":"2024-09-10T22:01:18.878087+0000","last_active":"2024-09-10T22:01:20.131283+0000","last_peered":"2024-09-10T22:01:20.131283+0000","last_clean":"2024-09-10T22:01:20.131283+0000","last_became_active":"2024-09-10T22:01:18.877799+0000","last_became_peered":"2024-09-10T22:01:18.877799+0000","last_unstale":"2024-09-10T22:01:20.131283+0000","last_undegraded":"2024-09-10T22:01:20.131283+0000","last_fullsized":"2024-09-10T22:01:20.131283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:47.396364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868281+0000","last_change":"2024-09-10T22:01:18.869808+0000","last_active":"2024-09-10T22:01:19.868281+0000","last_peered":"2024-09-10T22:01:19.868281+0000","last_clean":"2024-09-10T22:01:19.868281+0000","last_became_active":"2024-09-10T22:01:18.869477+0000","last_became_peered":"2024-09-10T22:01:18.869477+0000","last_unstale":"2024-09-10T22:01:19.868281+0000","last_undegraded":"2024-09-10T22:01:19.868281+0000","last_fullsized":"2024-09-10T22:01:19.868281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:53.871376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878053+0000","last_change":"2024-09-10T22:01:18.882447+0000","last_active":"2024-09-10T22:01:19.878053+0000","last_peered":"2024-09-10T22:01:19.878053+0000","last_clean":"2024-09-10T22:01:19.878053+0000","last_became_active":"2024-09-10T22:01:18.882166+0000","last_became_peered":"2024-09-10T22:01:18.882166+0000","last_unstale":"2024-09-10T22:01:19.878053+0000","last_undegraded":"2024-09-10T22:01:19.878053+0000","last_fullsized":"2024-09-10T22:01:19.878053+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:53.871376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877771+0000","last_change":"2024-09-10T22:01:18.875580+0000","last_active":"2024-09-10T22:01:19.877771+0000","last_peered":"2024-09-10T22:01:19.877771+0000","last_clean":"2024-09-10T22:01:19.877771+0000","last_became_active":"2024-09-10T22:01:18.875277+0000","last_became_peered":"2024-09-10T22:01:18.875277+0000","last_unstale":"2024-09-10T22:01:19.877771+0000","last_undegraded":"2024-09-10T22:01:19.877771+0000","last_fullsized":"2024-09-10T22:01:19.877771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:03.853727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.871154+0000","last_change":"2024-09-10T22:01:18.863796+0000","last_active":"2024-09-10T22:01:19.871154+0000","last_peered":"2024-09-10T22:01:19.871154+0000","last_clean":"2024-09-10T22:01:19.871154+0000","last_became_active":"2024-09-10T22:01:18.863418+0000","last_became_peered":"2024-09-10T22:01:18.863418+0000","last_unstale":"2024-09-10T22:01:19.871154+0000","last_undegraded":"2024-09-10T22:01:19.871154+0000","last_fullsized":"2024-09-10T22:01:19.871154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:22.047393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877440+0000","last_change":"2024-09-10T22:01:18.883097+0000","last_active":"2024-09-10T22:01:19.877440+0000","last_peered":"2024-09-10T22:01:19.877440+0000","last_clean":"2024-09-10T22:01:19.877440+0000","last_became_active":"2024-09-10T22:01:18.881934+0000","last_became_peered":"2024-09-10T22:01:18.881934+0000","last_unstale":"2024-09-10T22:01:19.877440+0000","last_undegraded":"2024-09-10T22:01:19.877440+0000","last_fullsized":"2024-09-10T22:01:19.877440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:34.846790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878055+0000","last_change":"2024-09-10T22:01:18.879296+0000","last_active":"2024-09-10T22:01:19.878055+0000","last_peered":"2024-09-10T22:01:19.878055+0000","last_clean":"2024-09-10T22:01:19.878055+0000","last_became_active":"2024-09-10T22:01:18.876671+0000","last_became_peered":"2024-09-10T22:01:18.876671+0000","last_unstale":"2024-09-10T22:01:19.878055+0000","last_undegraded":"2024-09-10T22:01:19.878055+0000","last_fullsized":"2024-09-10T22:01:19.878055+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:39.796957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971435+0000","last_change":"2024-09-10T22:01:18.873275+0000","last_active":"2024-09-10T22:01:19.971435+0000","last_peered":"2024-09-10T22:01:19.971435+0000","last_clean":"2024-09-10T22:01:19.971435+0000","last_became_active":"2024-09-10T22:01:18.873019+0000","last_became_peered":"2024-09-10T22:01:18.873019+0000","last_unstale":"2024-09-10T22:01:19.971435+0000","last_undegraded":"2024-09-10T22:01:19.971435+0000","last_fullsized":"2024-09-10T22:01:19.971435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:10.125819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131439+0000","last_change":"2024-09-10T22:01:18.868693+0000","last_active":"2024-09-10T22:01:20.131439+0000","last_peered":"2024-09-10T22:01:20.131439+0000","last_clean":"2024-09-10T22:01:20.131439+0000","last_became_active":"2024-09-10T22:01:18.867908+0000","last_became_peered":"2024-09-10T22:01:18.867908+0000","last_unstale":"2024-09-10T22:01:20.131439+0000","last_undegraded":"2024-09-10T22:01:20.131439+0000","last_fullsized":"2024-09-10T22:01:20.131439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:11.839759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868619+0000","last_change":"2024-09-10T22:01:18.869785+0000","last_active":"2024-09-10T22:01:19.868619+0000","last_peered":"2024-09-10T22:01:19.868619+0000","last_clean":"2024-09-10T22:01:19.868619+0000","last_became_active":"2024-09-10T22:01:18.869501+0000","last_became_peered":"2024-09-10T22:01:18.869501+0000","last_unstale":"2024-09-10T22:01:19.868619+0000","last_undegraded":"2024-09-10T22:01:19.868619+0000","last_fullsized":"2024-09-10T22:01:19.868619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:24.916481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971324+0000","last_change":"2024-09-10T22:01:18.879837+0000","last_active":"2024-09-10T22:01:19.971324+0000","last_peered":"2024-09-10T22:01:19.971324+0000","last_clean":"2024-09-10T22:01:19.971324+0000","last_became_active":"2024-09-10T22:01:18.878722+0000","last_became_peered":"2024-09-10T22:01:18.878722+0000","last_unstale":"2024-09-10T22:01:19.971324+0000","last_undegraded":"2024-09-10T22:01:19.971324+0000","last_fullsized":"2024-09-10T22:01:19.971324+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:39.796957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878457+0000","last_change":"2024-09-10T22:01:18.877337+0000","last_active":"2024-09-10T22:01:19.878457+0000","last_peered":"2024-09-10T22:01:19.878457+0000","last_clean":"2024-09-10T22:01:19.878457+0000","last_became_active":"2024-09-10T22:01:18.876530+0000","last_became_peered":"2024-09-10T22:01:18.876530+0000","last_unstale":"2024-09-10T22:01:19.878457+0000","last_undegraded":"2024-09-10T22:01:19.878457+0000","last_fullsized":"2024-09-10T22:01:19.878457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:05.685293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878011+0000","last_change":"2024-09-10T22:01:18.878864+0000","last_active":"2024-09-10T22:01:19.878011+0000","last_peered":"2024-09-10T22:01:19.878011+0000","last_clean":"2024-09-10T22:01:19.878011+0000","last_became_active":"2024-09-10T22:01:18.878447+0000","last_became_peered":"2024-09-10T22:01:18.878447+0000","last_unstale":"2024-09-10T22:01:19.878011+0000","last_undegraded":"2024-09-10T22:01:19.878011+0000","last_fullsized":"2024-09-10T22:01:19.878011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:19.927290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869506+0000","last_change":"2024-09-10T22:01:18.870331+0000","last_active":"2024-09-10T22:01:19.869506+0000","last_peered":"2024-09-10T22:01:19.869506+0000","last_clean":"2024-09-10T22:01:19.869506+0000","last_became_active":"2024-09-10T22:01:18.870089+0000","last_became_peered":"2024-09-10T22:01:18.870089+0000","last_unstale":"2024-09-10T22:01:19.869506+0000","last_undegraded":"2024-09-10T22:01:19.869506+0000","last_fullsized":"2024-09-10T22:01:19.869506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:27:53.966547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869242+0000","last_change":"2024-09-10T22:01:18.872946+0000","last_active":"2024-09-10T22:01:19.869242+0000","last_peered":"2024-09-10T22:01:19.869242+0000","last_clean":"2024-09-10T22:01:19.869242+0000","last_became_active":"2024-09-10T22:01:18.872681+0000","last_became_peered":"2024-09-10T22:01:18.872681+0000","last_unstale":"2024-09-10T22:01:19.869242+0000","last_undegraded":"2024-09-10T22:01:19.869242+0000","last_fullsized":"2024-09-10T22:01:19.869242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:33.759315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131545+0000","last_change":"2024-09-10T22:01:18.870556+0000","last_active":"2024-09-10T22:01:20.131545+0000","last_peered":"2024-09-10T22:01:20.131545+0000","last_clean":"2024-09-10T22:01:20.131545+0000","last_became_active":"2024-09-10T22:01:18.870287+0000","last_became_peered":"2024-09-10T22:01:18.870287+0000","last_unstale":"2024-09-10T22:01:20.131545+0000","last_undegraded":"2024-09-10T22:01:20.131545+0000","last_fullsized":"2024-09-10T22:01:20.131545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:41.445213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870655+0000","last_change":"2024-09-10T22:01:18.864075+0000","last_active":"2024-09-10T22:01:19.870655+0000","last_peered":"2024-09-10T22:01:19.870655+0000","last_clean":"2024-09-10T22:01:19.870655+0000","last_became_active":"2024-09-10T22:01:18.863799+0000","last_became_peered":"2024-09-10T22:01:18.863799+0000","last_unstale":"2024-09-10T22:01:19.870655+0000","last_undegraded":"2024-09-10T22:01:19.870655+0000","last_fullsized":"2024-09-10T22:01:19.870655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:46.701358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870583+0000","last_change":"2024-09-10T22:01:18.875872+0000","last_active":"2024-09-10T22:01:19.870583+0000","last_peered":"2024-09-10T22:01:19.870583+0000","last_clean":"2024-09-10T22:01:19.870583+0000","last_became_active":"2024-09-10T22:01:18.875590+0000","last_became_peered":"2024-09-10T22:01:18.875590+0000","last_unstale":"2024-09-10T22:01:19.870583+0000","last_undegraded":"2024-09-10T22:01:19.870583+0000","last_fullsized":"2024-09-10T22:01:19.870583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:56.993820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874280+0000","last_change":"2024-09-10T22:01:18.876960+0000","last_active":"2024-09-10T22:01:19.874280+0000","last_peered":"2024-09-10T22:01:19.874280+0000","last_clean":"2024-09-10T22:01:19.874280+0000","last_became_active":"2024-09-10T22:01:18.876738+0000","last_became_peered":"2024-09-10T22:01:18.876738+0000","last_unstale":"2024-09-10T22:01:19.874280+0000","last_undegraded":"2024-09-10T22:01:19.874280+0000","last_fullsized":"2024-09-10T22:01:19.874280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:41.464337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971187+0000","last_change":"2024-09-10T22:01:18.871967+0000","last_active":"2024-09-10T22:01:19.971187+0000","last_peered":"2024-09-10T22:01:19.971187+0000","last_clean":"2024-09-10T22:01:19.971187+0000","last_became_active":"2024-09-10T22:01:18.871602+0000","last_became_peered":"2024-09-10T22:01:18.871602+0000","last_unstale":"2024-09-10T22:01:19.971187+0000","last_undegraded":"2024-09-10T22:01:19.971187+0000","last_fullsized":"2024-09-10T22:01:19.971187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:05.462364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877505+0000","last_change":"2024-09-10T22:01:18.882253+0000","last_active":"2024-09-10T22:01:19.877505+0000","last_peered":"2024-09-10T22:01:19.877505+0000","last_clean":"2024-09-10T22:01:19.877505+0000","last_became_active":"2024-09-10T22:01:18.877282+0000","last_became_peered":"2024-09-10T22:01:18.877282+0000","last_unstale":"2024-09-10T22:01:19.877505+0000","last_undegraded":"2024-09-10T22:01:19.877505+0000","last_fullsized":"2024-09-10T22:01:19.877505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:05.462364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869606+0000","last_change":"2024-09-10T22:01:18.876226+0000","last_active":"2024-09-10T22:01:19.869606+0000","last_peered":"2024-09-10T22:01:19.869606+0000","last_clean":"2024-09-10T22:01:19.869606+0000","last_became_active":"2024-09-10T22:01:18.875980+0000","last_became_peered":"2024-09-10T22:01:18.875980+0000","last_unstale":"2024-09-10T22:01:19.869606+0000","last_undegraded":"2024-09-10T22:01:19.869606+0000","last_fullsized":"2024-09-10T22:01:19.869606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:36.104388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877382+0000","last_change":"2024-09-10T22:01:18.878125+0000","last_active":"2024-09-10T22:01:19.877382+0000","last_peered":"2024-09-10T22:01:19.877382+0000","last_clean":"2024-09-10T22:01:19.877382+0000","last_became_active":"2024-09-10T22:01:18.877821+0000","last_became_peered":"2024-09-10T22:01:18.877821+0000","last_unstale":"2024-09-10T22:01:19.877382+0000","last_undegraded":"2024-09-10T22:01:19.877382+0000","last_fullsized":"2024-09-10T22:01:19.877382+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:07.047625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971587+0000","last_change":"2024-09-10T22:01:18.866074+0000","last_active":"2024-09-10T22:01:19.971587+0000","last_peered":"2024-09-10T22:01:19.971587+0000","last_clean":"2024-09-10T22:01:19.971587+0000","last_became_active":"2024-09-10T22:01:18.865776+0000","last_became_peered":"2024-09-10T22:01:18.865776+0000","last_unstale":"2024-09-10T22:01:19.971587+0000","last_undegraded":"2024-09-10T22:01:19.971587+0000","last_fullsized":"2024-09-10T22:01:19.971587+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:37.351347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870648+0000","last_change":"2024-09-10T22:01:18.875141+0000","last_active":"2024-09-10T22:01:19.870648+0000","last_peered":"2024-09-10T22:01:19.870648+0000","last_clean":"2024-09-10T22:01:19.870648+0000","last_became_active":"2024-09-10T22:01:18.874858+0000","last_became_peered":"2024-09-10T22:01:18.874858+0000","last_unstale":"2024-09-10T22:01:19.870648+0000","last_undegraded":"2024-09-10T22:01:19.870648+0000","last_fullsized":"2024-09-10T22:01:19.870648+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:15.917682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877979+0000","last_change":"2024-09-10T22:01:18.875750+0000","last_active":"2024-09-10T22:01:19.877979+0000","last_peered":"2024-09-10T22:01:19.877979+0000","last_clean":"2024-09-10T22:01:19.877979+0000","last_became_active":"2024-09-10T22:01:18.875425+0000","last_became_peered":"2024-09-10T22:01:18.875425+0000","last_unstale":"2024-09-10T22:01:19.877979+0000","last_undegraded":"2024-09-10T22:01:19.877979+0000","last_fullsized":"2024-09-10T22:01:19.877979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:51.395881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869280+0000","last_change":"2024-09-10T22:01:18.873494+0000","last_active":"2024-09-10T22:01:19.869280+0000","last_peered":"2024-09-10T22:01:19.869280+0000","last_clean":"2024-09-10T22:01:19.869280+0000","last_became_active":"2024-09-10T22:01:18.872889+0000","last_became_peered":"2024-09-10T22:01:18.872889+0000","last_unstale":"2024-09-10T22:01:19.869280+0000","last_undegraded":"2024-09-10T22:01:19.869280+0000","last_fullsized":"2024-09-10T22:01:19.869280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:19.013885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877764+0000","last_change":"2024-09-10T22:01:18.877627+0000","last_active":"2024-09-10T22:01:19.877764+0000","last_peered":"2024-09-10T22:01:19.877764+0000","last_clean":"2024-09-10T22:01:19.877764+0000","last_became_active":"2024-09-10T22:01:18.876020+0000","last_became_peered":"2024-09-10T22:01:18.876020+0000","last_unstale":"2024-09-10T22:01:19.877764+0000","last_undegraded":"2024-09-10T22:01:19.877764+0000","last_fullsized":"2024-09-10T22:01:19.877764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:34.846790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874358+0000","last_change":"2024-09-10T22:01:18.875822+0000","last_active":"2024-09-10T22:01:19.874358+0000","last_peered":"2024-09-10T22:01:19.874358+0000","last_clean":"2024-09-10T22:01:19.874358+0000","last_became_active":"2024-09-10T22:01:18.875540+0000","last_became_peered":"2024-09-10T22:01:18.875540+0000","last_unstale":"2024-09-10T22:01:19.874358+0000","last_undegraded":"2024-09-10T22:01:19.874358+0000","last_fullsized":"2024-09-10T22:01:19.874358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:46.164767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869711+0000","last_change":"2024-09-10T22:01:17.858193+0000","last_active":"2024-09-10T22:01:19.869711+0000","last_peered":"2024-09-10T22:01:19.869711+0000","last_clean":"2024-09-10T22:01:19.869711+0000","last_became_active":"2024-09-10T22:01:17.857928+0000","last_became_peered":"2024-09-10T22:01:17.857928+0000","last_unstale":"2024-09-10T22:01:19.869711+0000","last_undegraded":"2024-09-10T22:01:19.869711+0000","last_fullsized":"2024-09-10T22:01:19.869711+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:04.203230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869683+0000","last_change":"2024-09-10T22:01:18.870101+0000","last_active":"2024-09-10T22:01:19.869683+0000","last_peered":"2024-09-10T22:01:19.869683+0000","last_clean":"2024-09-10T22:01:19.869683+0000","last_became_active":"2024-09-10T22:01:18.863218+0000","last_became_peered":"2024-09-10T22:01:18.863218+0000","last_unstale":"2024-09-10T22:01:19.869683+0000","last_undegraded":"2024-09-10T22:01:19.869683+0000","last_fullsized":"2024-09-10T22:01:19.869683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:21.793658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874344+0000","last_change":"2024-09-10T22:01:17.861902+0000","last_active":"2024-09-10T22:01:19.874344+0000","last_peered":"2024-09-10T22:01:19.874344+0000","last_clean":"2024-09-10T22:01:19.874344+0000","last_became_active":"2024-09-10T22:01:17.861630+0000","last_became_peered":"2024-09-10T22:01:17.861630+0000","last_unstale":"2024-09-10T22:01:19.874344+0000","last_undegraded":"2024-09-10T22:01:19.874344+0000","last_fullsized":"2024-09-10T22:01:19.874344+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:02.295993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.871102+0000","last_change":"2024-09-10T22:01:18.868619+0000","last_active":"2024-09-10T22:01:19.871102+0000","last_peered":"2024-09-10T22:01:19.871102+0000","last_clean":"2024-09-10T22:01:19.871102+0000","last_became_active":"2024-09-10T22:01:18.868364+0000","last_became_peered":"2024-09-10T22:01:18.868364+0000","last_unstale":"2024-09-10T22:01:19.871102+0000","last_undegraded":"2024-09-10T22:01:19.871102+0000","last_fullsized":"2024-09-10T22:01:19.871102+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:26.336553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869085+0000","last_change":"2024-09-10T22:01:17.866264+0000","last_active":"2024-09-10T22:01:19.869085+0000","last_peered":"2024-09-10T22:01:19.869085+0000","last_clean":"2024-09-10T22:01:19.869085+0000","last_became_active":"2024-09-10T22:01:17.865988+0000","last_became_peered":"2024-09-10T22:01:17.865988+0000","last_unstale":"2024-09-10T22:01:19.869085+0000","last_undegraded":"2024-09-10T22:01:19.869085+0000","last_fullsized":"2024-09-10T22:01:19.869085+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:33.079690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877502+0000","last_change":"2024-09-10T22:01:18.882546+0000","last_active":"2024-09-10T22:01:19.877502+0000","last_peered":"2024-09-10T22:01:19.877502+0000","last_clean":"2024-09-10T22:01:19.877502+0000","last_became_active":"2024-09-10T22:01:18.882111+0000","last_became_peered":"2024-09-10T22:01:18.882111+0000","last_unstale":"2024-09-10T22:01:19.877502+0000","last_undegraded":"2024-09-10T22:01:19.877502+0000","last_fullsized":"2024-09-10T22:01:19.877502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:50.356457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131459+0000","last_change":"2024-09-10T22:01:17.857345+0000","last_active":"2024-09-10T22:01:20.131459+0000","last_peered":"2024-09-10T22:01:20.131459+0000","last_clean":"2024-09-10T22:01:20.131459+0000","last_became_active":"2024-09-10T22:01:17.857048+0000","last_became_peered":"2024-09-10T22:01:17.857048+0000","last_unstale":"2024-09-10T22:01:20.131459+0000","last_undegraded":"2024-09-10T22:01:20.131459+0000","last_fullsized":"2024-09-10T22:01:20.131459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:31.530052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878066+0000","last_change":"2024-09-10T22:01:18.868505+0000","last_active":"2024-09-10T22:01:19.878066+0000","last_peered":"2024-09-10T22:01:19.878066+0000","last_clean":"2024-09-10T22:01:19.878066+0000","last_became_active":"2024-09-10T22:01:18.868179+0000","last_became_peered":"2024-09-10T22:01:18.868179+0000","last_unstale":"2024-09-10T22:01:19.878066+0000","last_undegraded":"2024-09-10T22:01:19.878066+0000","last_fullsized":"2024-09-10T22:01:19.878066+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:29.481248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870759+0000","last_change":"2024-09-10T22:01:17.865813+0000","last_active":"2024-09-10T22:01:19.870759+0000","last_peered":"2024-09-10T22:01:19.870759+0000","last_clean":"2024-09-10T22:01:19.870759+0000","last_became_active":"2024-09-10T22:01:17.865564+0000","last_became_peered":"2024-09-10T22:01:17.865564+0000","last_unstale":"2024-09-10T22:01:19.870759+0000","last_undegraded":"2024-09-10T22:01:19.870759+0000","last_fullsized":"2024-09-10T22:01:19.870759+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:46.578161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971390+0000","last_change":"2024-09-10T22:01:18.865856+0000","last_active":"2024-09-10T22:01:19.971390+0000","last_peered":"2024-09-10T22:01:19.971390+0000","last_clean":"2024-09-10T22:01:19.971390+0000","last_became_active":"2024-09-10T22:01:18.865340+0000","last_became_peered":"2024-09-10T22:01:18.865340+0000","last_unstale":"2024-09-10T22:01:19.971390+0000","last_undegraded":"2024-09-10T22:01:19.971390+0000","last_fullsized":"2024-09-10T22:01:19.971390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:28.298204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870426+0000","last_change":"2024-09-10T22:01:17.859555+0000","last_active":"2024-09-10T22:01:19.870426+0000","last_peered":"2024-09-10T22:01:19.870426+0000","last_clean":"2024-09-10T22:01:19.870426+0000","last_became_active":"2024-09-10T22:01:17.859204+0000","last_became_peered":"2024-09-10T22:01:17.859204+0000","last_unstale":"2024-09-10T22:01:19.870426+0000","last_undegraded":"2024-09-10T22:01:19.870426+0000","last_fullsized":"2024-09-10T22:01:19.870426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:25.755703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869555+0000","last_change":"2024-09-10T22:01:18.868796+0000","last_active":"2024-09-10T22:01:19.869555+0000","last_peered":"2024-09-10T22:01:19.869555+0000","last_clean":"2024-09-10T22:01:19.869555+0000","last_became_active":"2024-09-10T22:01:18.868064+0000","last_became_peered":"2024-09-10T22:01:18.868064+0000","last_unstale":"2024-09-10T22:01:19.869555+0000","last_undegraded":"2024-09-10T22:01:19.869555+0000","last_fullsized":"2024-09-10T22:01:19.869555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:30.265708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868127+0000","last_change":"2024-09-10T22:01:17.858789+0000","last_active":"2024-09-10T22:01:19.868127+0000","last_peered":"2024-09-10T22:01:19.868127+0000","last_clean":"2024-09-10T22:01:19.868127+0000","last_became_active":"2024-09-10T22:01:17.858503+0000","last_became_peered":"2024-09-10T22:01:17.858503+0000","last_unstale":"2024-09-10T22:01:19.868127+0000","last_undegraded":"2024-09-10T22:01:19.868127+0000","last_fullsized":"2024-09-10T22:01:19.868127+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:36.694934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.876894+0000","last_change":"2024-09-10T22:01:18.868425+0000","last_active":"2024-09-10T22:01:19.876894+0000","last_peered":"2024-09-10T22:01:19.876894+0000","last_clean":"2024-09-10T22:01:19.876894+0000","last_became_active":"2024-09-10T22:01:18.868119+0000","last_became_peered":"2024-09-10T22:01:18.868119+0000","last_unstale":"2024-09-10T22:01:19.876894+0000","last_undegraded":"2024-09-10T22:01:19.876894+0000","last_fullsized":"2024-09-10T22:01:19.876894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:51.416819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869616+0000","last_change":"2024-09-10T22:01:17.853234+0000","last_active":"2024-09-10T22:01:19.869616+0000","last_peered":"2024-09-10T22:01:19.869616+0000","last_clean":"2024-09-10T22:01:19.869616+0000","last_became_active":"2024-09-10T22:01:17.852912+0000","last_became_peered":"2024-09-10T22:01:17.852912+0000","last_unstale":"2024-09-10T22:01:19.869616+0000","last_undegraded":"2024-09-10T22:01:19.869616+0000","last_fullsized":"2024-09-10T22:01:19.869616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:40.036066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971618+0000","last_change":"2024-09-10T22:01:18.873210+0000","last_active":"2024-09-10T22:01:19.971618+0000","last_peered":"2024-09-10T22:01:19.971618+0000","last_clean":"2024-09-10T22:01:19.971618+0000","last_became_active":"2024-09-10T22:01:18.872945+0000","last_became_peered":"2024-09-10T22:01:18.872945+0000","last_unstale":"2024-09-10T22:01:19.971618+0000","last_undegraded":"2024-09-10T22:01:19.971618+0000","last_fullsized":"2024-09-10T22:01:19.971618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:33.957217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869412+0000","last_change":"2024-09-10T22:01:17.865798+0000","last_active":"2024-09-10T22:01:19.869412+0000","last_peered":"2024-09-10T22:01:19.869412+0000","last_clean":"2024-09-10T22:01:19.869412+0000","last_became_active":"2024-09-10T22:01:17.865521+0000","last_became_peered":"2024-09-10T22:01:17.865521+0000","last_unstale":"2024-09-10T22:01:19.869412+0000","last_undegraded":"2024-09-10T22:01:19.869412+0000","last_fullsized":"2024-09-10T22:01:19.869412+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:36.586342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869378+0000","last_change":"2024-09-10T22:01:18.874307+0000","last_active":"2024-09-10T22:01:19.869378+0000","last_peered":"2024-09-10T22:01:19.869378+0000","last_clean":"2024-09-10T22:01:19.869378+0000","last_became_active":"2024-09-10T22:01:18.874045+0000","last_became_peered":"2024-09-10T22:01:18.874045+0000","last_unstale":"2024-09-10T22:01:19.869378+0000","last_undegraded":"2024-09-10T22:01:19.869378+0000","last_fullsized":"2024-09-10T22:01:19.869378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:07.543195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874460+0000","last_change":"2024-09-10T22:01:17.869964+0000","last_active":"2024-09-10T22:01:19.874460+0000","last_peered":"2024-09-10T22:01:19.874460+0000","last_clean":"2024-09-10T22:01:19.874460+0000","last_became_active":"2024-09-10T22:01:17.869541+0000","last_became_peered":"2024-09-10T22:01:17.869541+0000","last_unstale":"2024-09-10T22:01:19.874460+0000","last_undegraded":"2024-09-10T22:01:19.874460+0000","last_fullsized":"2024-09-10T22:01:19.874460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:17.864650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869289+0000","last_change":"2024-09-10T22:01:18.878517+0000","last_active":"2024-09-10T22:01:19.869289+0000","last_peered":"2024-09-10T22:01:19.869289+0000","last_clean":"2024-09-10T22:01:19.869289+0000","last_became_active":"2024-09-10T22:01:18.878351+0000","last_became_peered":"2024-09-10T22:01:18.878351+0000","last_unstale":"2024-09-10T22:01:19.869289+0000","last_undegraded":"2024-09-10T22:01:19.869289+0000","last_fullsized":"2024-09-10T22:01:19.869289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:05.685293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971352+0000","last_change":"2024-09-10T22:01:17.862765+0000","last_active":"2024-09-10T22:01:19.971352+0000","last_peered":"2024-09-10T22:01:19.971352+0000","last_clean":"2024-09-10T22:01:19.971352+0000","last_became_active":"2024-09-10T22:01:17.861335+0000","last_became_peered":"2024-09-10T22:01:17.861335+0000","last_unstale":"2024-09-10T22:01:19.971352+0000","last_undegraded":"2024-09-10T22:01:19.971352+0000","last_fullsized":"2024-09-10T22:01:19.971352+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:36.694934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971274+0000","last_change":"2024-09-10T22:01:18.871412+0000","last_active":"2024-09-10T22:01:19.971274+0000","last_peered":"2024-09-10T22:01:19.971274+0000","last_clean":"2024-09-10T22:01:19.971274+0000","last_became_active":"2024-09-10T22:01:18.871042+0000","last_became_peered":"2024-09-10T22:01:18.871042+0000","last_unstale":"2024-09-10T22:01:19.971274+0000","last_undegraded":"2024-09-10T22:01:19.971274+0000","last_fullsized":"2024-09-10T22:01:19.971274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:19.927290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869177+0000","last_change":"2024-09-10T22:01:17.866217+0000","last_active":"2024-09-10T22:01:19.869177+0000","last_peered":"2024-09-10T22:01:19.869177+0000","last_clean":"2024-09-10T22:01:19.869177+0000","last_became_active":"2024-09-10T22:01:17.865942+0000","last_became_peered":"2024-09-10T22:01:17.865942+0000","last_unstale":"2024-09-10T22:01:19.869177+0000","last_undegraded":"2024-09-10T22:01:19.869177+0000","last_fullsized":"2024-09-10T22:01:19.869177+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:29.260133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.876922+0000","last_change":"2024-09-10T22:01:18.867134+0000","last_active":"2024-09-10T22:01:19.876922+0000","last_peered":"2024-09-10T22:01:19.876922+0000","last_clean":"2024-09-10T22:01:19.876922+0000","last_became_active":"2024-09-10T22:01:18.866884+0000","last_became_peered":"2024-09-10T22:01:18.866884+0000","last_unstale":"2024-09-10T22:01:19.876922+0000","last_undegraded":"2024-09-10T22:01:19.876922+0000","last_fullsized":"2024-09-10T22:01:19.876922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:46.701358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870775+0000","last_change":"2024-09-10T22:01:17.855694+0000","last_active":"2024-09-10T22:01:19.870775+0000","last_peered":"2024-09-10T22:01:19.870775+0000","last_clean":"2024-09-10T22:01:19.870775+0000","last_became_active":"2024-09-10T22:01:17.855323+0000","last_became_peered":"2024-09-10T22:01:17.855323+0000","last_unstale":"2024-09-10T22:01:19.870775+0000","last_undegraded":"2024-09-10T22:01:19.870775+0000","last_fullsized":"2024-09-10T22:01:19.870775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:08.932820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869100+0000","last_change":"2024-09-10T22:01:18.869148+0000","last_active":"2024-09-10T22:01:19.869100+0000","last_peered":"2024-09-10T22:01:19.869100+0000","last_clean":"2024-09-10T22:01:19.869100+0000","last_became_active":"2024-09-10T22:01:18.868772+0000","last_became_peered":"2024-09-10T22:01:18.868772+0000","last_unstale":"2024-09-10T22:01:19.869100+0000","last_undegraded":"2024-09-10T22:01:19.869100+0000","last_fullsized":"2024-09-10T22:01:19.869100+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:14.196122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.876970+0000","last_change":"2024-09-10T22:01:17.852954+0000","last_active":"2024-09-10T22:01:19.876970+0000","last_peered":"2024-09-10T22:01:19.876970+0000","last_clean":"2024-09-10T22:01:19.876970+0000","last_became_active":"2024-09-10T22:01:17.852697+0000","last_became_peered":"2024-09-10T22:01:17.852697+0000","last_unstale":"2024-09-10T22:01:19.876970+0000","last_undegraded":"2024-09-10T22:01:19.876970+0000","last_fullsized":"2024-09-10T22:01:19.876970+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:02.295993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878260+0000","last_change":"2024-09-10T22:01:18.876788+0000","last_active":"2024-09-10T22:01:19.878260+0000","last_peered":"2024-09-10T22:01:19.878260+0000","last_clean":"2024-09-10T22:01:19.878260+0000","last_became_active":"2024-09-10T22:01:18.876238+0000","last_became_peered":"2024-09-10T22:01:18.876238+0000","last_unstale":"2024-09-10T22:01:19.878260+0000","last_undegraded":"2024-09-10T22:01:19.878260+0000","last_fullsized":"2024-09-10T22:01:19.878260+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:33.759315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868335+0000","last_change":"2024-09-10T22:01:17.862292+0000","last_active":"2024-09-10T22:01:19.868335+0000","last_peered":"2024-09-10T22:01:19.868335+0000","last_clean":"2024-09-10T22:01:19.868335+0000","last_became_active":"2024-09-10T22:01:17.861498+0000","last_became_peered":"2024-09-10T22:01:17.861498+0000","last_unstale":"2024-09-10T22:01:19.868335+0000","last_undegraded":"2024-09-10T22:01:19.868335+0000","last_fullsized":"2024-09-10T22:01:19.868335+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:08.359041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869496+0000","last_change":"2024-09-10T22:01:18.873210+0000","last_active":"2024-09-10T22:01:19.869496+0000","last_peered":"2024-09-10T22:01:19.869496+0000","last_clean":"2024-09-10T22:01:19.869496+0000","last_became_active":"2024-09-10T22:01:18.872610+0000","last_became_peered":"2024-09-10T22:01:18.872610+0000","last_unstale":"2024-09-10T22:01:19.869496+0000","last_undegraded":"2024-09-10T22:01:19.869496+0000","last_fullsized":"2024-09-10T22:01:19.869496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:21.599818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868186+0000","last_change":"2024-09-10T22:01:17.865774+0000","last_active":"2024-09-10T22:01:19.868186+0000","last_peered":"2024-09-10T22:01:19.868186+0000","last_clean":"2024-09-10T22:01:19.868186+0000","last_became_active":"2024-09-10T22:01:17.865488+0000","last_became_peered":"2024-09-10T22:01:17.865488+0000","last_unstale":"2024-09-10T22:01:19.868186+0000","last_undegraded":"2024-09-10T22:01:19.868186+0000","last_fullsized":"2024-09-10T22:01:19.868186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:57.577284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.871070+0000","last_change":"2024-09-10T22:01:18.866748+0000","last_active":"2024-09-10T22:01:19.871070+0000","last_peered":"2024-09-10T22:01:19.871070+0000","last_clean":"2024-09-10T22:01:19.871070+0000","last_became_active":"2024-09-10T22:01:18.866386+0000","last_became_peered":"2024-09-10T22:01:18.866386+0000","last_unstale":"2024-09-10T22:01:19.871070+0000","last_undegraded":"2024-09-10T22:01:19.871070+0000","last_fullsized":"2024-09-10T22:01:19.871070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:34.079361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877515+0000","last_change":"2024-09-10T22:01:17.867707+0000","last_active":"2024-09-10T22:01:19.877515+0000","last_peered":"2024-09-10T22:01:19.877515+0000","last_clean":"2024-09-10T22:01:19.877515+0000","last_became_active":"2024-09-10T22:01:17.867395+0000","last_became_peered":"2024-09-10T22:01:17.867395+0000","last_unstale":"2024-09-10T22:01:19.877515+0000","last_undegraded":"2024-09-10T22:01:19.877515+0000","last_fullsized":"2024-09-10T22:01:19.877515+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:35.136565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971752+0000","last_change":"2024-09-10T22:01:18.869383+0000","last_active":"2024-09-10T22:01:19.971752+0000","last_peered":"2024-09-10T22:01:19.971752+0000","last_clean":"2024-09-10T22:01:19.971752+0000","last_became_active":"2024-09-10T22:01:18.869119+0000","last_became_peered":"2024-09-10T22:01:18.869119+0000","last_unstale":"2024-09-10T22:01:19.971752+0000","last_undegraded":"2024-09-10T22:01:19.971752+0000","last_fullsized":"2024-09-10T22:01:19.971752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:03.772153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.871118+0000","last_change":"2024-09-10T22:01:17.850829+0000","last_active":"2024-09-10T22:01:19.871118+0000","last_peered":"2024-09-10T22:01:19.871118+0000","last_clean":"2024-09-10T22:01:19.871118+0000","last_became_active":"2024-09-10T22:01:17.850332+0000","last_became_peered":"2024-09-10T22:01:17.850332+0000","last_unstale":"2024-09-10T22:01:19.871118+0000","last_undegraded":"2024-09-10T22:01:19.871118+0000","last_fullsized":"2024-09-10T22:01:19.871118+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:32.724306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877182+0000","last_change":"2024-09-10T22:01:18.876334+0000","last_active":"2024-09-10T22:01:19.877182+0000","last_peered":"2024-09-10T22:01:19.877182+0000","last_clean":"2024-09-10T22:01:19.877182+0000","last_became_active":"2024-09-10T22:01:18.876026+0000","last_became_peered":"2024-09-10T22:01:18.876026+0000","last_unstale":"2024-09-10T22:01:19.877182+0000","last_undegraded":"2024-09-10T22:01:19.877182+0000","last_fullsized":"2024-09-10T22:01:19.877182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:36.576646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870944+0000","last_change":"2024-09-10T22:01:17.861200+0000","last_active":"2024-09-10T22:01:19.870944+0000","last_peered":"2024-09-10T22:01:19.870944+0000","last_clean":"2024-09-10T22:01:19.870944+0000","last_became_active":"2024-09-10T22:01:17.860796+0000","last_became_peered":"2024-09-10T22:01:17.860796+0000","last_unstale":"2024-09-10T22:01:19.870944+0000","last_undegraded":"2024-09-10T22:01:19.870944+0000","last_fullsized":"2024-09-10T22:01:19.870944+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:18.433905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131599+0000","last_change":"2024-09-10T22:01:18.879012+0000","last_active":"2024-09-10T22:01:20.131599+0000","last_peered":"2024-09-10T22:01:20.131599+0000","last_clean":"2024-09-10T22:01:20.131599+0000","last_became_active":"2024-09-10T22:01:18.878175+0000","last_became_peered":"2024-09-10T22:01:18.878175+0000","last_unstale":"2024-09-10T22:01:20.131599+0000","last_undegraded":"2024-09-10T22:01:20.131599+0000","last_fullsized":"2024-09-10T22:01:20.131599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:15.918564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870816+0000","last_change":"2024-09-10T22:01:17.861102+0000","last_active":"2024-09-10T22:01:19.870816+0000","last_peered":"2024-09-10T22:01:19.870816+0000","last_clean":"2024-09-10T22:01:19.870816+0000","last_became_active":"2024-09-10T22:01:17.860779+0000","last_became_peered":"2024-09-10T22:01:17.860779+0000","last_unstale":"2024-09-10T22:01:19.870816+0000","last_undegraded":"2024-09-10T22:01:19.870816+0000","last_fullsized":"2024-09-10T22:01:19.870816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:30.650130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131319+0000","last_change":"2024-09-10T22:01:18.871029+0000","last_active":"2024-09-10T22:01:20.131319+0000","last_peered":"2024-09-10T22:01:20.131319+0000","last_clean":"2024-09-10T22:01:20.131319+0000","last_became_active":"2024-09-10T22:01:18.866768+0000","last_became_peered":"2024-09-10T22:01:18.866768+0000","last_unstale":"2024-09-10T22:01:20.131319+0000","last_undegraded":"2024-09-10T22:01:20.131319+0000","last_fullsized":"2024-09-10T22:01:20.131319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:54.676713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874530+0000","last_change":"2024-09-10T22:01:17.851819+0000","last_active":"2024-09-10T22:01:19.874530+0000","last_peered":"2024-09-10T22:01:19.874530+0000","last_clean":"2024-09-10T22:01:19.874530+0000","last_became_active":"2024-09-10T22:01:17.850945+0000","last_became_peered":"2024-09-10T22:01:17.850945+0000","last_unstale":"2024-09-10T22:01:19.874530+0000","last_undegraded":"2024-09-10T22:01:19.874530+0000","last_fullsized":"2024-09-10T22:01:19.874530+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:35.136565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131482+0000","last_change":"2024-09-10T22:01:18.868998+0000","last_active":"2024-09-10T22:01:20.131482+0000","last_peered":"2024-09-10T22:01:20.131482+0000","last_clean":"2024-09-10T22:01:20.131482+0000","last_became_active":"2024-09-10T22:01:18.868508+0000","last_became_peered":"2024-09-10T22:01:18.868508+0000","last_unstale":"2024-09-10T22:01:20.131482+0000","last_undegraded":"2024-09-10T22:01:20.131482+0000","last_fullsized":"2024-09-10T22:01:20.131482+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:08.433283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868998+0000","last_change":"2024-09-10T22:01:17.849861+0000","last_active":"2024-09-10T22:01:19.868998+0000","last_peered":"2024-09-10T22:01:19.868998+0000","last_clean":"2024-09-10T22:01:19.868998+0000","last_became_active":"2024-09-10T22:01:17.849568+0000","last_became_peered":"2024-09-10T22:01:17.849568+0000","last_unstale":"2024-09-10T22:01:19.868998+0000","last_undegraded":"2024-09-10T22:01:19.868998+0000","last_fullsized":"2024-09-10T22:01:19.868998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:53.054618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131241+0000","last_change":"2024-09-10T22:01:18.865130+0000","last_active":"2024-09-10T22:01:20.131241+0000","last_peered":"2024-09-10T22:01:20.131241+0000","last_clean":"2024-09-10T22:01:20.131241+0000","last_became_active":"2024-09-10T22:01:18.864863+0000","last_became_peered":"2024-09-10T22:01:18.864863+0000","last_unstale":"2024-09-10T22:01:20.131241+0000","last_undegraded":"2024-09-10T22:01:20.131241+0000","last_fullsized":"2024-09-10T22:01:20.131241+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:54.059872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870174+0000","last_change":"2024-09-10T22:01:17.850707+0000","last_active":"2024-09-10T22:01:19.870174+0000","last_peered":"2024-09-10T22:01:19.870174+0000","last_clean":"2024-09-10T22:01:19.870174+0000","last_became_active":"2024-09-10T22:01:17.850396+0000","last_became_peered":"2024-09-10T22:01:17.850396+0000","last_unstale":"2024-09-10T22:01:19.870174+0000","last_undegraded":"2024-09-10T22:01:19.870174+0000","last_fullsized":"2024-09-10T22:01:19.870174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:42.075103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877753+0000","last_change":"2024-09-10T22:01:18.883013+0000","last_active":"2024-09-10T22:01:19.877753+0000","last_peered":"2024-09-10T22:01:19.877753+0000","last_clean":"2024-09-10T22:01:19.877753+0000","last_became_active":"2024-09-10T22:01:18.882792+0000","last_became_peered":"2024-09-10T22:01:18.882792+0000","last_unstale":"2024-09-10T22:01:19.877753+0000","last_undegraded":"2024-09-10T22:01:19.877753+0000","last_fullsized":"2024-09-10T22:01:19.877753+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:22.047393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.970933+0000","last_change":"2024-09-10T22:01:17.861920+0000","last_active":"2024-09-10T22:01:19.970933+0000","last_peered":"2024-09-10T22:01:19.970933+0000","last_clean":"2024-09-10T22:01:19.970933+0000","last_became_active":"2024-09-10T22:01:17.861634+0000","last_became_peered":"2024-09-10T22:01:17.861634+0000","last_unstale":"2024-09-10T22:01:19.970933+0000","last_undegraded":"2024-09-10T22:01:19.970933+0000","last_fullsized":"2024-09-10T22:01:19.970933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:57.577284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131607+0000","last_change":"2024-09-10T22:01:18.870633+0000","last_active":"2024-09-10T22:01:20.131607+0000","last_peered":"2024-09-10T22:01:20.131607+0000","last_clean":"2024-09-10T22:01:20.131607+0000","last_became_active":"2024-09-10T22:01:18.870291+0000","last_became_peered":"2024-09-10T22:01:18.870291+0000","last_unstale":"2024-09-10T22:01:20.131607+0000","last_undegraded":"2024-09-10T22:01:20.131607+0000","last_fullsized":"2024-09-10T22:01:20.131607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:01.983686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877542+0000","last_change":"2024-09-10T22:01:17.858498+0000","last_active":"2024-09-10T22:01:19.877542+0000","last_peered":"2024-09-10T22:01:19.877542+0000","last_clean":"2024-09-10T22:01:19.877542+0000","last_became_active":"2024-09-10T22:01:17.858227+0000","last_became_peered":"2024-09-10T22:01:17.858227+0000","last_unstale":"2024-09-10T22:01:19.877542+0000","last_undegraded":"2024-09-10T22:01:19.877542+0000","last_fullsized":"2024-09-10T22:01:19.877542+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:04.203230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971729+0000","last_change":"2024-09-10T22:01:18.870760+0000","last_active":"2024-09-10T22:01:19.971729+0000","last_peered":"2024-09-10T22:01:19.971729+0000","last_clean":"2024-09-10T22:01:19.971729+0000","last_became_active":"2024-09-10T22:01:18.870510+0000","last_became_peered":"2024-09-10T22:01:18.870510+0000","last_unstale":"2024-09-10T22:01:19.971729+0000","last_undegraded":"2024-09-10T22:01:19.971729+0000","last_fullsized":"2024-09-10T22:01:19.971729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:12.922313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.876139+0000","last_change":"2024-09-10T22:01:17.851892+0000","last_active":"2024-09-10T22:01:19.876139+0000","last_peered":"2024-09-10T22:01:19.876139+0000","last_clean":"2024-09-10T22:01:19.876139+0000","last_became_active":"2024-09-10T22:01:17.851244+0000","last_became_peered":"2024-09-10T22:01:17.851244+0000","last_unstale":"2024-09-10T22:01:19.876139+0000","last_undegraded":"2024-09-10T22:01:19.876139+0000","last_fullsized":"2024-09-10T22:01:19.876139+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:18.742907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874252+0000","last_change":"2024-09-10T22:01:18.862523+0000","last_active":"2024-09-10T22:01:19.874252+0000","last_peered":"2024-09-10T22:01:19.874252+0000","last_clean":"2024-09-10T22:01:19.874252+0000","last_became_active":"2024-09-10T22:01:18.861607+0000","last_became_peered":"2024-09-10T22:01:18.861607+0000","last_unstale":"2024-09-10T22:01:19.874252+0000","last_undegraded":"2024-09-10T22:01:19.874252+0000","last_fullsized":"2024-09-10T22:01:19.874252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:21.793658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877213+0000","last_change":"2024-09-10T22:01:17.858275+0000","last_active":"2024-09-10T22:01:19.877213+0000","last_peered":"2024-09-10T22:01:19.877213+0000","last_clean":"2024-09-10T22:01:19.877213+0000","last_became_active":"2024-09-10T22:01:17.857981+0000","last_became_peered":"2024-09-10T22:01:17.857981+0000","last_unstale":"2024-09-10T22:01:19.877213+0000","last_undegraded":"2024-09-10T22:01:19.877213+0000","last_fullsized":"2024-09-10T22:01:19.877213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:13.631349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869168+0000","last_change":"2024-09-10T22:01:18.866301+0000","last_active":"2024-09-10T22:01:19.869168+0000","last_peered":"2024-09-10T22:01:19.869168+0000","last_clean":"2024-09-10T22:01:19.869168+0000","last_became_active":"2024-09-10T22:01:18.865947+0000","last_became_peered":"2024-09-10T22:01:18.865947+0000","last_unstale":"2024-09-10T22:01:19.869168+0000","last_undegraded":"2024-09-10T22:01:19.869168+0000","last_fullsized":"2024-09-10T22:01:19.869168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:32.295747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874612+0000","last_change":"2024-09-10T22:01:17.853418+0000","last_active":"2024-09-10T22:01:19.874612+0000","last_peered":"2024-09-10T22:01:19.874612+0000","last_clean":"2024-09-10T22:01:19.874612+0000","last_became_active":"2024-09-10T22:01:17.853148+0000","last_became_peered":"2024-09-10T22:01:17.853148+0000","last_unstale":"2024-09-10T22:01:19.874612+0000","last_undegraded":"2024-09-10T22:01:19.874612+0000","last_fullsized":"2024-09-10T22:01:19.874612+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:13.631349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874425+0000","last_change":"2024-09-10T22:01:18.871427+0000","last_active":"2024-09-10T22:01:19.874425+0000","last_peered":"2024-09-10T22:01:19.874425+0000","last_clean":"2024-09-10T22:01:19.874425+0000","last_became_active":"2024-09-10T22:01:18.869064+0000","last_became_peered":"2024-09-10T22:01:18.869064+0000","last_unstale":"2024-09-10T22:01:19.874425+0000","last_undegraded":"2024-09-10T22:01:19.874425+0000","last_fullsized":"2024-09-10T22:01:19.874425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:51.416819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877072+0000","last_change":"2024-09-10T22:01:17.853258+0000","last_active":"2024-09-10T22:01:19.877072+0000","last_peered":"2024-09-10T22:01:19.877072+0000","last_clean":"2024-09-10T22:01:19.877072+0000","last_became_active":"2024-09-10T22:01:17.852933+0000","last_became_peered":"2024-09-10T22:01:17.852933+0000","last_unstale":"2024-09-10T22:01:19.877072+0000","last_undegraded":"2024-09-10T22:01:19.877072+0000","last_fullsized":"2024-09-10T22:01:19.877072+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:18.742907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868182+0000","last_change":"2024-09-10T22:01:18.869936+0000","last_active":"2024-09-10T22:01:19.868182+0000","last_peered":"2024-09-10T22:01:19.868182+0000","last_clean":"2024-09-10T22:01:19.868182+0000","last_became_active":"2024-09-10T22:01:18.869680+0000","last_became_peered":"2024-09-10T22:01:18.869680+0000","last_unstale":"2024-09-10T22:01:19.868182+0000","last_undegraded":"2024-09-10T22:01:19.868182+0000","last_fullsized":"2024-09-10T22:01:19.868182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:29.481248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870252+0000","last_change":"2024-09-10T22:01:17.854341+0000","last_active":"2024-09-10T22:01:19.870252+0000","last_peered":"2024-09-10T22:01:19.870252+0000","last_clean":"2024-09-10T22:01:19.870252+0000","last_became_active":"2024-09-10T22:01:17.854067+0000","last_became_peered":"2024-09-10T22:01:17.854067+0000","last_unstale":"2024-09-10T22:01:19.870252+0000","last_undegraded":"2024-09-10T22:01:19.870252+0000","last_fullsized":"2024-09-10T22:01:19.870252+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:24.204319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870325+0000","last_change":"2024-09-10T22:01:18.875620+0000","last_active":"2024-09-10T22:01:19.870325+0000","last_peered":"2024-09-10T22:01:19.870325+0000","last_clean":"2024-09-10T22:01:19.870325+0000","last_became_active":"2024-09-10T22:01:18.875242+0000","last_became_peered":"2024-09-10T22:01:18.875242+0000","last_unstale":"2024-09-10T22:01:19.870325+0000","last_undegraded":"2024-09-10T22:01:19.870325+0000","last_fullsized":"2024-09-10T22:01:19.870325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:37.591916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868080+0000","last_change":"2024-09-10T22:01:17.861574+0000","last_active":"2024-09-10T22:01:19.868080+0000","last_peered":"2024-09-10T22:01:19.868080+0000","last_clean":"2024-09-10T22:01:19.868080+0000","last_became_active":"2024-09-10T22:01:17.860999+0000","last_became_peered":"2024-09-10T22:01:17.860999+0000","last_unstale":"2024-09-10T22:01:19.868080+0000","last_undegraded":"2024-09-10T22:01:19.868080+0000","last_fullsized":"2024-09-10T22:01:19.868080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:15.531745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878112+0000","last_change":"2024-09-10T22:01:18.878139+0000","last_active":"2024-09-10T22:01:19.878112+0000","last_peered":"2024-09-10T22:01:19.878112+0000","last_clean":"2024-09-10T22:01:19.878112+0000","last_became_active":"2024-09-10T22:01:18.877966+0000","last_became_peered":"2024-09-10T22:01:18.877966+0000","last_unstale":"2024-09-10T22:01:19.878112+0000","last_undegraded":"2024-09-10T22:01:19.878112+0000","last_fullsized":"2024-09-10T22:01:19.878112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:14.196122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869052+0000","last_change":"2024-09-10T22:01:17.857059+0000","last_active":"2024-09-10T22:01:19.869052+0000","last_peered":"2024-09-10T22:01:19.869052+0000","last_clean":"2024-09-10T22:01:19.869052+0000","last_became_active":"2024-09-10T22:01:17.856184+0000","last_became_peered":"2024-09-10T22:01:17.856184+0000","last_unstale":"2024-09-10T22:01:19.869052+0000","last_undegraded":"2024-09-10T22:01:19.869052+0000","last_fullsized":"2024-09-10T22:01:19.869052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:46.578161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":130,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878510+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:01:19.878510+0000","last_peered":"2024-09-10T22:01:19.878510+0000","last_clean":"2024-09-10T22:01:19.878510+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:01:19.878510+0000","last_undegraded":"2024-09-10T22:01:19.878510+0000","last_fullsized":"2024-09-10T22:01:19.878510+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":962560,"data_stored":729934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":372736,"data_stored":140092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":372736,"data_stored":140148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":372736,"data_stored":139580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":141110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":380928,"data_stored":141110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":958464,"data_stored":729844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":139546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:01:23.013 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:01:23.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.194+0000 7fa7d7e00640 1 -- 172.21.15.66:0/1797518130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7d815f100 msgr2=0x7fa7d80d1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:23.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.194+0000 7fa7d7e00640 1 --2- 172.21.15.66:0/1797518130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7d815f100 0x7fa7d80d1960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009870 tx=0x7fa7c801c820 comp rx=0 tx=0).stop 2024-09-10T22:01:23.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.194+0000 7fa7d7e00640 1 -- 172.21.15.66:0/1797518130 shutdown_connections 2024-09-10T22:01:23.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.194+0000 7fa7d7e00640 1 --2- 172.21.15.66:0/1797518130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7d815f100 0x7fa7d80d1960 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.194+0000 7fa7d7e00640 1 -- 172.21.15.66:0/1797518130 >> 172.21.15.66:0/1797518130 conn(0x7fa7d809aa50 msgr2=0x7fa7d809ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:23.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.194+0000 7fa7d7e00640 1 -- 172.21.15.66:0/1797518130 shutdown_connections 2024-09-10T22:01:23.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.194+0000 7fa7d7e00640 1 -- 172.21.15.66:0/1797518130 wait complete. 2024-09-10T22:01:23.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.194+0000 7fa7d7e00640 1 Processor -- start 2024-09-10T22:01:23.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.194+0000 7fa7d7e00640 1 -- start start 2024-09-10T22:01:23.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7d80b3110 0x7fa7d814d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:23.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d80d0110 0x7fa7d80cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7d815f100 0x7fa7d80b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d816a820 con 0x7fa7d815f100 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d816a6a0 con 0x7fa7d80d0110 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d816a9a0 con 0x7fa7d80b3110 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7d815f100 0x7fa7d80b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7d815f100 0x7fa7d80b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40390/0 (socket says 172.21.15.66:40390) 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7400640 1 -- 172.21.15.66:0/936268070 learned_addr learned my addr 172.21.15.66:0/936268070 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d6a00640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7d80b3110 0x7fa7d814d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d6000640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d80d0110 0x7fa7d80cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7400640 1 -- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7d80b3110 msgr2=0x7fa7d814d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7400640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7d80b3110 0x7fa7d814d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7400640 1 -- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d80d0110 msgr2=0x7fa7d80cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7400640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d80d0110 0x7fa7d80cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7400640 1 -- 172.21.15.66:0/936268070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d81f6180 con 0x7fa7d815f100 2024-09-10T22:01:23.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d6a00640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7d80b3110 0x7fa7d814d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:23.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d6000640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d80d0110 0x7fa7d80cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:23.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7400640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7d815f100 0x7fa7d80b3620 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000c7c0 tx=0x7fa7d000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:23.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7cea00640 1 -- 172.21.15.66:0/936268070 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0003b80 con 0x7fa7d815f100 2024-09-10T22:01:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7cea00640 1 -- 172.21.15.66:0/936268070 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7d0003d20 con 0x7fa7d815f100 2024-09-10T22:01:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7e00640 1 -- 172.21.15.66:0/936268070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d81f6460 con 0x7fa7d815f100 2024-09-10T22:01:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7cea00640 1 -- 172.21.15.66:0/936268070 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d000aa50 con 0x7fa7d815f100 2024-09-10T22:01:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7e00640 1 -- 172.21.15.66:0/936268070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d81f6a20 con 0x7fa7d815f100 2024-09-10T22:01:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7d7e00640 1 -- 172.21.15.66:0/936268070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d80d1960 con 0x7fa7d815f100 2024-09-10T22:01:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.198+0000 7fa7cea00640 1 -- 172.21.15.66:0/936268070 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa7d000abf0 con 0x7fa7d815f100 2024-09-10T22:01:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.202+0000 7fa7cea00640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa7b80725e0 0x7fa7b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.202+0000 7fa7cea00640 1 -- 172.21.15.66:0/936268070 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa7d00931c0 con 0x7fa7d815f100 2024-09-10T22:01:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.202+0000 7fa7d6a00640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa7b80725e0 0x7fa7b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.202+0000 7fa7d6a00640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa7b80725e0 0x7fa7b8074aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80af000 tx=0x7fa7c801b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:23.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.206+0000 7fa7cea00640 1 -- 172.21.15.66:0/936268070 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa7d005e780 con 0x7fa7d815f100 2024-09-10T22:01:23.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.374+0000 7fa7d7e00640 1 -- 172.21.15.66:0/936268070 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7d80d2970 con 0x7fa7b80725e0 2024-09-10T22:01:23.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.378+0000 7fa7cea00640 1 -- 172.21.15.66:0/936268070 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7fa7d80d2970 con 0x7fa7b80725e0 2024-09-10T22:01:23.386 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:23.388 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:01:23.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.382+0000 7fa7d7e00640 1 -- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa7b80725e0 msgr2=0x7fa7b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:23.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.382+0000 7fa7d7e00640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa7b80725e0 0x7fa7b8074aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80af000 tx=0x7fa7c801b060 comp rx=0 tx=0).stop 2024-09-10T22:01:23.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.382+0000 7fa7d7e00640 1 -- 172.21.15.66:0/936268070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7d815f100 msgr2=0x7fa7d80b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:23.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.382+0000 7fa7d7e00640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7d815f100 0x7fa7d80b3620 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000c7c0 tx=0x7fa7d000cc80 comp rx=0 tx=0).stop 2024-09-10T22:01:23.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.382+0000 7fa7d7e00640 1 -- 172.21.15.66:0/936268070 shutdown_connections 2024-09-10T22:01:23.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.382+0000 7fa7d7e00640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa7b80725e0 0x7fa7b8074aa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.382+0000 7fa7d7e00640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7d815f100 0x7fa7d80b3620 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.382+0000 7fa7d7e00640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d80d0110 0x7fa7d80cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.382+0000 7fa7d7e00640 1 --2- 172.21.15.66:0/936268070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7d80b3110 0x7fa7d814d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.382+0000 7fa7d7e00640 1 -- 172.21.15.66:0/936268070 >> 172.21.15.66:0/936268070 conn(0x7fa7d809aa50 msgr2=0x7fa7d80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:23.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.382+0000 7fa7d7e00640 1 -- 172.21.15.66:0/936268070 shutdown_connections 2024-09-10T22:01:23.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.382+0000 7fa7d7e00640 1 -- 172.21.15.66:0/936268070 wait complete. 2024-09-10T22:01:23.408 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-10T22:01:22.321715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236976,"kb_used_data":4072,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737744,"statfs":{"total":773094113280,"available":772851449856,"internally_reserved":0,"allocated":4169728,"data_stored":2301364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.489981"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869646+0000","last_change":"2024-09-10T22:01:18.873322+0000","last_active":"2024-09-10T22:01:19.869646+0000","last_peered":"2024-09-10T22:01:19.869646+0000","last_clean":"2024-09-10T22:01:19.869646+0000","last_became_active":"2024-09-10T22:01:18.873061+0000","last_became_peered":"2024-09-10T22:01:18.873061+0000","last_unstale":"2024-09-10T22:01:19.869646+0000","last_undegraded":"2024-09-10T22:01:19.869646+0000","last_fullsized":"2024-09-10T22:01:19.869646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:02.950345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869402+0000","last_change":"2024-09-10T22:01:18.872583+0000","last_active":"2024-09-10T22:01:19.869402+0000","last_peered":"2024-09-10T22:01:19.869402+0000","last_clean":"2024-09-10T22:01:19.869402+0000","last_became_active":"2024-09-10T22:01:18.872345+0000","last_became_peered":"2024-09-10T22:01:18.872345+0000","last_unstale":"2024-09-10T22:01:19.869402+0000","last_undegraded":"2024-09-10T22:01:19.869402+0000","last_fullsized":"2024-09-10T22:01:19.869402+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:41.792870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971394+0000","last_change":"2024-09-10T22:01:18.870777+0000","last_active":"2024-09-10T22:01:19.971394+0000","last_peered":"2024-09-10T22:01:19.971394+0000","last_clean":"2024-09-10T22:01:19.971394+0000","last_became_active":"2024-09-10T22:01:18.870107+0000","last_became_peered":"2024-09-10T22:01:18.870107+0000","last_unstale":"2024-09-10T22:01:19.971394+0000","last_undegraded":"2024-09-10T22:01:19.971394+0000","last_fullsized":"2024-09-10T22:01:19.971394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:10.233265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877138+0000","last_change":"2024-09-10T22:01:18.864672+0000","last_active":"2024-09-10T22:01:19.877138+0000","last_peered":"2024-09-10T22:01:19.877138+0000","last_clean":"2024-09-10T22:01:19.877138+0000","last_became_active":"2024-09-10T22:01:18.864345+0000","last_became_peered":"2024-09-10T22:01:18.864345+0000","last_unstale":"2024-09-10T22:01:19.877138+0000","last_undegraded":"2024-09-10T22:01:19.877138+0000","last_fullsized":"2024-09-10T22:01:19.877138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:08.025235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131283+0000","last_change":"2024-09-10T22:01:18.878087+0000","last_active":"2024-09-10T22:01:20.131283+0000","last_peered":"2024-09-10T22:01:20.131283+0000","last_clean":"2024-09-10T22:01:20.131283+0000","last_became_active":"2024-09-10T22:01:18.877799+0000","last_became_peered":"2024-09-10T22:01:18.877799+0000","last_unstale":"2024-09-10T22:01:20.131283+0000","last_undegraded":"2024-09-10T22:01:20.131283+0000","last_fullsized":"2024-09-10T22:01:20.131283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:47.396364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868281+0000","last_change":"2024-09-10T22:01:18.869808+0000","last_active":"2024-09-10T22:01:19.868281+0000","last_peered":"2024-09-10T22:01:19.868281+0000","last_clean":"2024-09-10T22:01:19.868281+0000","last_became_active":"2024-09-10T22:01:18.869477+0000","last_became_peered":"2024-09-10T22:01:18.869477+0000","last_unstale":"2024-09-10T22:01:19.868281+0000","last_undegraded":"2024-09-10T22:01:19.868281+0000","last_fullsized":"2024-09-10T22:01:19.868281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:53.871376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878053+0000","last_change":"2024-09-10T22:01:18.882447+0000","last_active":"2024-09-10T22:01:19.878053+0000","last_peered":"2024-09-10T22:01:19.878053+0000","last_clean":"2024-09-10T22:01:19.878053+0000","last_became_active":"2024-09-10T22:01:18.882166+0000","last_became_peered":"2024-09-10T22:01:18.882166+0000","last_unstale":"2024-09-10T22:01:19.878053+0000","last_undegraded":"2024-09-10T22:01:19.878053+0000","last_fullsized":"2024-09-10T22:01:19.878053+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:53.871376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877771+0000","last_change":"2024-09-10T22:01:18.875580+0000","last_active":"2024-09-10T22:01:19.877771+0000","last_peered":"2024-09-10T22:01:19.877771+0000","last_clean":"2024-09-10T22:01:19.877771+0000","last_became_active":"2024-09-10T22:01:18.875277+0000","last_became_peered":"2024-09-10T22:01:18.875277+0000","last_unstale":"2024-09-10T22:01:19.877771+0000","last_undegraded":"2024-09-10T22:01:19.877771+0000","last_fullsized":"2024-09-10T22:01:19.877771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:03.853727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.871154+0000","last_change":"2024-09-10T22:01:18.863796+0000","last_active":"2024-09-10T22:01:19.871154+0000","last_peered":"2024-09-10T22:01:19.871154+0000","last_clean":"2024-09-10T22:01:19.871154+0000","last_became_active":"2024-09-10T22:01:18.863418+0000","last_became_peered":"2024-09-10T22:01:18.863418+0000","last_unstale":"2024-09-10T22:01:19.871154+0000","last_undegraded":"2024-09-10T22:01:19.871154+0000","last_fullsized":"2024-09-10T22:01:19.871154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:22.047393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877440+0000","last_change":"2024-09-10T22:01:18.883097+0000","last_active":"2024-09-10T22:01:19.877440+0000","last_peered":"2024-09-10T22:01:19.877440+0000","last_clean":"2024-09-10T22:01:19.877440+0000","last_became_active":"2024-09-10T22:01:18.881934+0000","last_became_peered":"2024-09-10T22:01:18.881934+0000","last_unstale":"2024-09-10T22:01:19.877440+0000","last_undegraded":"2024-09-10T22:01:19.877440+0000","last_fullsized":"2024-09-10T22:01:19.877440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:34.846790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878055+0000","last_change":"2024-09-10T22:01:18.879296+0000","last_active":"2024-09-10T22:01:19.878055+0000","last_peered":"2024-09-10T22:01:19.878055+0000","last_clean":"2024-09-10T22:01:19.878055+0000","last_became_active":"2024-09-10T22:01:18.876671+0000","last_became_peered":"2024-09-10T22:01:18.876671+0000","last_unstale":"2024-09-10T22:01:19.878055+0000","last_undegraded":"2024-09-10T22:01:19.878055+0000","last_fullsized":"2024-09-10T22:01:19.878055+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:39.796957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971435+0000","last_change":"2024-09-10T22:01:18.873275+0000","last_active":"2024-09-10T22:01:19.971435+0000","last_peered":"2024-09-10T22:01:19.971435+0000","last_clean":"2024-09-10T22:01:19.971435+0000","last_became_active":"2024-09-10T22:01:18.873019+0000","last_became_peered":"2024-09-10T22:01:18.873019+0000","last_unstale":"2024-09-10T22:01:19.971435+0000","last_undegraded":"2024-09-10T22:01:19.971435+0000","last_fullsized":"2024-09-10T22:01:19.971435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:10.125819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131439+0000","last_change":"2024-09-10T22:01:18.868693+0000","last_active":"2024-09-10T22:01:20.131439+0000","last_peered":"2024-09-10T22:01:20.131439+0000","last_clean":"2024-09-10T22:01:20.131439+0000","last_became_active":"2024-09-10T22:01:18.867908+0000","last_became_peered":"2024-09-10T22:01:18.867908+0000","last_unstale":"2024-09-10T22:01:20.131439+0000","last_undegraded":"2024-09-10T22:01:20.131439+0000","last_fullsized":"2024-09-10T22:01:20.131439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:11.839759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868619+0000","last_change":"2024-09-10T22:01:18.869785+0000","last_active":"2024-09-10T22:01:19.868619+0000","last_peered":"2024-09-10T22:01:19.868619+0000","last_clean":"2024-09-10T22:01:19.868619+0000","last_became_active":"2024-09-10T22:01:18.869501+0000","last_became_peered":"2024-09-10T22:01:18.869501+0000","last_unstale":"2024-09-10T22:01:19.868619+0000","last_undegraded":"2024-09-10T22:01:19.868619+0000","last_fullsized":"2024-09-10T22:01:19.868619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:24.916481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971324+0000","last_change":"2024-09-10T22:01:18.879837+0000","last_active":"2024-09-10T22:01:19.971324+0000","last_peered":"2024-09-10T22:01:19.971324+0000","last_clean":"2024-09-10T22:01:19.971324+0000","last_became_active":"2024-09-10T22:01:18.878722+0000","last_became_peered":"2024-09-10T22:01:18.878722+0000","last_unstale":"2024-09-10T22:01:19.971324+0000","last_undegraded":"2024-09-10T22:01:19.971324+0000","last_fullsized":"2024-09-10T22:01:19.971324+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:39.796957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878457+0000","last_change":"2024-09-10T22:01:18.877337+0000","last_active":"2024-09-10T22:01:19.878457+0000","last_peered":"2024-09-10T22:01:19.878457+0000","last_clean":"2024-09-10T22:01:19.878457+0000","last_became_active":"2024-09-10T22:01:18.876530+0000","last_became_peered":"2024-09-10T22:01:18.876530+0000","last_unstale":"2024-09-10T22:01:19.878457+0000","last_undegraded":"2024-09-10T22:01:19.878457+0000","last_fullsized":"2024-09-10T22:01:19.878457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:05.685293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878011+0000","last_change":"2024-09-10T22:01:18.878864+0000","last_active":"2024-09-10T22:01:19.878011+0000","last_peered":"2024-09-10T22:01:19.878011+0000","last_clean":"2024-09-10T22:01:19.878011+0000","last_became_active":"2024-09-10T22:01:18.878447+0000","last_became_peered":"2024-09-10T22:01:18.878447+0000","last_unstale":"2024-09-10T22:01:19.878011+0000","last_undegraded":"2024-09-10T22:01:19.878011+0000","last_fullsized":"2024-09-10T22:01:19.878011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:19.927290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869506+0000","last_change":"2024-09-10T22:01:18.870331+0000","last_active":"2024-09-10T22:01:19.869506+0000","last_peered":"2024-09-10T22:01:19.869506+0000","last_clean":"2024-09-10T22:01:19.869506+0000","last_became_active":"2024-09-10T22:01:18.870089+0000","last_became_peered":"2024-09-10T22:01:18.870089+0000","last_unstale":"2024-09-10T22:01:19.869506+0000","last_undegraded":"2024-09-10T22:01:19.869506+0000","last_fullsized":"2024-09-10T22:01:19.869506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:27:53.966547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869242+0000","last_change":"2024-09-10T22:01:18.872946+0000","last_active":"2024-09-10T22:01:19.869242+0000","last_peered":"2024-09-10T22:01:19.869242+0000","last_clean":"2024-09-10T22:01:19.869242+0000","last_became_active":"2024-09-10T22:01:18.872681+0000","last_became_peered":"2024-09-10T22:01:18.872681+0000","last_unstale":"2024-09-10T22:01:19.869242+0000","last_undegraded":"2024-09-10T22:01:19.869242+0000","last_fullsized":"2024-09-10T22:01:19.869242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:33.759315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131545+0000","last_change":"2024-09-10T22:01:18.870556+0000","last_active":"2024-09-10T22:01:20.131545+0000","last_peered":"2024-09-10T22:01:20.131545+0000","last_clean":"2024-09-10T22:01:20.131545+0000","last_became_active":"2024-09-10T22:01:18.870287+0000","last_became_peered":"2024-09-10T22:01:18.870287+0000","last_unstale":"2024-09-10T22:01:20.131545+0000","last_undegraded":"2024-09-10T22:01:20.131545+0000","last_fullsized":"2024-09-10T22:01:20.131545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:41.445213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870655+0000","last_change":"2024-09-10T22:01:18.864075+0000","last_active":"2024-09-10T22:01:19.870655+0000","last_peered":"2024-09-10T22:01:19.870655+0000","last_clean":"2024-09-10T22:01:19.870655+0000","last_became_active":"2024-09-10T22:01:18.863799+0000","last_became_peered":"2024-09-10T22:01:18.863799+0000","last_unstale":"2024-09-10T22:01:19.870655+0000","last_undegraded":"2024-09-10T22:01:19.870655+0000","last_fullsized":"2024-09-10T22:01:19.870655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:46.701358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870583+0000","last_change":"2024-09-10T22:01:18.875872+0000","last_active":"2024-09-10T22:01:19.870583+0000","last_peered":"2024-09-10T22:01:19.870583+0000","last_clean":"2024-09-10T22:01:19.870583+0000","last_became_active":"2024-09-10T22:01:18.875590+0000","last_became_peered":"2024-09-10T22:01:18.875590+0000","last_unstale":"2024-09-10T22:01:19.870583+0000","last_undegraded":"2024-09-10T22:01:19.870583+0000","last_fullsized":"2024-09-10T22:01:19.870583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:56.993820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874280+0000","last_change":"2024-09-10T22:01:18.876960+0000","last_active":"2024-09-10T22:01:19.874280+0000","last_peered":"2024-09-10T22:01:19.874280+0000","last_clean":"2024-09-10T22:01:19.874280+0000","last_became_active":"2024-09-10T22:01:18.876738+0000","last_became_peered":"2024-09-10T22:01:18.876738+0000","last_unstale":"2024-09-10T22:01:19.874280+0000","last_undegraded":"2024-09-10T22:01:19.874280+0000","last_fullsized":"2024-09-10T22:01:19.874280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:41.464337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971187+0000","last_change":"2024-09-10T22:01:18.871967+0000","last_active":"2024-09-10T22:01:19.971187+0000","last_peered":"2024-09-10T22:01:19.971187+0000","last_clean":"2024-09-10T22:01:19.971187+0000","last_became_active":"2024-09-10T22:01:18.871602+0000","last_became_peered":"2024-09-10T22:01:18.871602+0000","last_unstale":"2024-09-10T22:01:19.971187+0000","last_undegraded":"2024-09-10T22:01:19.971187+0000","last_fullsized":"2024-09-10T22:01:19.971187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:05.462364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877505+0000","last_change":"2024-09-10T22:01:18.882253+0000","last_active":"2024-09-10T22:01:19.877505+0000","last_peered":"2024-09-10T22:01:19.877505+0000","last_clean":"2024-09-10T22:01:19.877505+0000","last_became_active":"2024-09-10T22:01:18.877282+0000","last_became_peered":"2024-09-10T22:01:18.877282+0000","last_unstale":"2024-09-10T22:01:19.877505+0000","last_undegraded":"2024-09-10T22:01:19.877505+0000","last_fullsized":"2024-09-10T22:01:19.877505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:05.462364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869606+0000","last_change":"2024-09-10T22:01:18.876226+0000","last_active":"2024-09-10T22:01:19.869606+0000","last_peered":"2024-09-10T22:01:19.869606+0000","last_clean":"2024-09-10T22:01:19.869606+0000","last_became_active":"2024-09-10T22:01:18.875980+0000","last_became_peered":"2024-09-10T22:01:18.875980+0000","last_unstale":"2024-09-10T22:01:19.869606+0000","last_undegraded":"2024-09-10T22:01:19.869606+0000","last_fullsized":"2024-09-10T22:01:19.869606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:36.104388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877382+0000","last_change":"2024-09-10T22:01:18.878125+0000","last_active":"2024-09-10T22:01:19.877382+0000","last_peered":"2024-09-10T22:01:19.877382+0000","last_clean":"2024-09-10T22:01:19.877382+0000","last_became_active":"2024-09-10T22:01:18.877821+0000","last_became_peered":"2024-09-10T22:01:18.877821+0000","last_unstale":"2024-09-10T22:01:19.877382+0000","last_undegraded":"2024-09-10T22:01:19.877382+0000","last_fullsized":"2024-09-10T22:01:19.877382+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:07.047625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971587+0000","last_change":"2024-09-10T22:01:18.866074+0000","last_active":"2024-09-10T22:01:19.971587+0000","last_peered":"2024-09-10T22:01:19.971587+0000","last_clean":"2024-09-10T22:01:19.971587+0000","last_became_active":"2024-09-10T22:01:18.865776+0000","last_became_peered":"2024-09-10T22:01:18.865776+0000","last_unstale":"2024-09-10T22:01:19.971587+0000","last_undegraded":"2024-09-10T22:01:19.971587+0000","last_fullsized":"2024-09-10T22:01:19.971587+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:37.351347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870648+0000","last_change":"2024-09-10T22:01:18.875141+0000","last_active":"2024-09-10T22:01:19.870648+0000","last_peered":"2024-09-10T22:01:19.870648+0000","last_clean":"2024-09-10T22:01:19.870648+0000","last_became_active":"2024-09-10T22:01:18.874858+0000","last_became_peered":"2024-09-10T22:01:18.874858+0000","last_unstale":"2024-09-10T22:01:19.870648+0000","last_undegraded":"2024-09-10T22:01:19.870648+0000","last_fullsized":"2024-09-10T22:01:19.870648+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:15.917682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877979+0000","last_change":"2024-09-10T22:01:18.875750+0000","last_active":"2024-09-10T22:01:19.877979+0000","last_peered":"2024-09-10T22:01:19.877979+0000","last_clean":"2024-09-10T22:01:19.877979+0000","last_became_active":"2024-09-10T22:01:18.875425+0000","last_became_peered":"2024-09-10T22:01:18.875425+0000","last_unstale":"2024-09-10T22:01:19.877979+0000","last_undegraded":"2024-09-10T22:01:19.877979+0000","last_fullsized":"2024-09-10T22:01:19.877979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:51.395881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869280+0000","last_change":"2024-09-10T22:01:18.873494+0000","last_active":"2024-09-10T22:01:19.869280+0000","last_peered":"2024-09-10T22:01:19.869280+0000","last_clean":"2024-09-10T22:01:19.869280+0000","last_became_active":"2024-09-10T22:01:18.872889+0000","last_became_peered":"2024-09-10T22:01:18.872889+0000","last_unstale":"2024-09-10T22:01:19.869280+0000","last_undegraded":"2024-09-10T22:01:19.869280+0000","last_fullsized":"2024-09-10T22:01:19.869280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:19.013885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877764+0000","last_change":"2024-09-10T22:01:18.877627+0000","last_active":"2024-09-10T22:01:19.877764+0000","last_peered":"2024-09-10T22:01:19.877764+0000","last_clean":"2024-09-10T22:01:19.877764+0000","last_became_active":"2024-09-10T22:01:18.876020+0000","last_became_peered":"2024-09-10T22:01:18.876020+0000","last_unstale":"2024-09-10T22:01:19.877764+0000","last_undegraded":"2024-09-10T22:01:19.877764+0000","last_fullsized":"2024-09-10T22:01:19.877764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:34.846790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874358+0000","last_change":"2024-09-10T22:01:18.875822+0000","last_active":"2024-09-10T22:01:19.874358+0000","last_peered":"2024-09-10T22:01:19.874358+0000","last_clean":"2024-09-10T22:01:19.874358+0000","last_became_active":"2024-09-10T22:01:18.875540+0000","last_became_peered":"2024-09-10T22:01:18.875540+0000","last_unstale":"2024-09-10T22:01:19.874358+0000","last_undegraded":"2024-09-10T22:01:19.874358+0000","last_fullsized":"2024-09-10T22:01:19.874358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:46.164767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869711+0000","last_change":"2024-09-10T22:01:17.858193+0000","last_active":"2024-09-10T22:01:19.869711+0000","last_peered":"2024-09-10T22:01:19.869711+0000","last_clean":"2024-09-10T22:01:19.869711+0000","last_became_active":"2024-09-10T22:01:17.857928+0000","last_became_peered":"2024-09-10T22:01:17.857928+0000","last_unstale":"2024-09-10T22:01:19.869711+0000","last_undegraded":"2024-09-10T22:01:19.869711+0000","last_fullsized":"2024-09-10T22:01:19.869711+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:04.203230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869683+0000","last_change":"2024-09-10T22:01:18.870101+0000","last_active":"2024-09-10T22:01:19.869683+0000","last_peered":"2024-09-10T22:01:19.869683+0000","last_clean":"2024-09-10T22:01:19.869683+0000","last_became_active":"2024-09-10T22:01:18.863218+0000","last_became_peered":"2024-09-10T22:01:18.863218+0000","last_unstale":"2024-09-10T22:01:19.869683+0000","last_undegraded":"2024-09-10T22:01:19.869683+0000","last_fullsized":"2024-09-10T22:01:19.869683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:21.793658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874344+0000","last_change":"2024-09-10T22:01:17.861902+0000","last_active":"2024-09-10T22:01:19.874344+0000","last_peered":"2024-09-10T22:01:19.874344+0000","last_clean":"2024-09-10T22:01:19.874344+0000","last_became_active":"2024-09-10T22:01:17.861630+0000","last_became_peered":"2024-09-10T22:01:17.861630+0000","last_unstale":"2024-09-10T22:01:19.874344+0000","last_undegraded":"2024-09-10T22:01:19.874344+0000","last_fullsized":"2024-09-10T22:01:19.874344+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:02.295993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.871102+0000","last_change":"2024-09-10T22:01:18.868619+0000","last_active":"2024-09-10T22:01:19.871102+0000","last_peered":"2024-09-10T22:01:19.871102+0000","last_clean":"2024-09-10T22:01:19.871102+0000","last_became_active":"2024-09-10T22:01:18.868364+0000","last_became_peered":"2024-09-10T22:01:18.868364+0000","last_unstale":"2024-09-10T22:01:19.871102+0000","last_undegraded":"2024-09-10T22:01:19.871102+0000","last_fullsized":"2024-09-10T22:01:19.871102+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:26.336553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869085+0000","last_change":"2024-09-10T22:01:17.866264+0000","last_active":"2024-09-10T22:01:19.869085+0000","last_peered":"2024-09-10T22:01:19.869085+0000","last_clean":"2024-09-10T22:01:19.869085+0000","last_became_active":"2024-09-10T22:01:17.865988+0000","last_became_peered":"2024-09-10T22:01:17.865988+0000","last_unstale":"2024-09-10T22:01:19.869085+0000","last_undegraded":"2024-09-10T22:01:19.869085+0000","last_fullsized":"2024-09-10T22:01:19.869085+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:33.079690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877502+0000","last_change":"2024-09-10T22:01:18.882546+0000","last_active":"2024-09-10T22:01:19.877502+0000","last_peered":"2024-09-10T22:01:19.877502+0000","last_clean":"2024-09-10T22:01:19.877502+0000","last_became_active":"2024-09-10T22:01:18.882111+0000","last_became_peered":"2024-09-10T22:01:18.882111+0000","last_unstale":"2024-09-10T22:01:19.877502+0000","last_undegraded":"2024-09-10T22:01:19.877502+0000","last_fullsized":"2024-09-10T22:01:19.877502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:50.356457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131459+0000","last_change":"2024-09-10T22:01:17.857345+0000","last_active":"2024-09-10T22:01:20.131459+0000","last_peered":"2024-09-10T22:01:20.131459+0000","last_clean":"2024-09-10T22:01:20.131459+0000","last_became_active":"2024-09-10T22:01:17.857048+0000","last_became_peered":"2024-09-10T22:01:17.857048+0000","last_unstale":"2024-09-10T22:01:20.131459+0000","last_undegraded":"2024-09-10T22:01:20.131459+0000","last_fullsized":"2024-09-10T22:01:20.131459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:31.530052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878066+0000","last_change":"2024-09-10T22:01:18.868505+0000","last_active":"2024-09-10T22:01:19.878066+0000","last_peered":"2024-09-10T22:01:19.878066+0000","last_clean":"2024-09-10T22:01:19.878066+0000","last_became_active":"2024-09-10T22:01:18.868179+0000","last_became_peered":"2024-09-10T22:01:18.868179+0000","last_unstale":"2024-09-10T22:01:19.878066+0000","last_undegraded":"2024-09-10T22:01:19.878066+0000","last_fullsized":"2024-09-10T22:01:19.878066+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:29.481248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870759+0000","last_change":"2024-09-10T22:01:17.865813+0000","last_active":"2024-09-10T22:01:19.870759+0000","last_peered":"2024-09-10T22:01:19.870759+0000","last_clean":"2024-09-10T22:01:19.870759+0000","last_became_active":"2024-09-10T22:01:17.865564+0000","last_became_peered":"2024-09-10T22:01:17.865564+0000","last_unstale":"2024-09-10T22:01:19.870759+0000","last_undegraded":"2024-09-10T22:01:19.870759+0000","last_fullsized":"2024-09-10T22:01:19.870759+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:46.578161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971390+0000","last_change":"2024-09-10T22:01:18.865856+0000","last_active":"2024-09-10T22:01:19.971390+0000","last_peered":"2024-09-10T22:01:19.971390+0000","last_clean":"2024-09-10T22:01:19.971390+0000","last_became_active":"2024-09-10T22:01:18.865340+0000","last_became_peered":"2024-09-10T22:01:18.865340+0000","last_unstale":"2024-09-10T22:01:19.971390+0000","last_undegraded":"2024-09-10T22:01:19.971390+0000","last_fullsized":"2024-09-10T22:01:19.971390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:28.298204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870426+0000","last_change":"2024-09-10T22:01:17.859555+0000","last_active":"2024-09-10T22:01:19.870426+0000","last_peered":"2024-09-10T22:01:19.870426+0000","last_clean":"2024-09-10T22:01:19.870426+0000","last_became_active":"2024-09-10T22:01:17.859204+0000","last_became_peered":"2024-09-10T22:01:17.859204+0000","last_unstale":"2024-09-10T22:01:19.870426+0000","last_undegraded":"2024-09-10T22:01:19.870426+0000","last_fullsized":"2024-09-10T22:01:19.870426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:25.755703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869555+0000","last_change":"2024-09-10T22:01:18.868796+0000","last_active":"2024-09-10T22:01:19.869555+0000","last_peered":"2024-09-10T22:01:19.869555+0000","last_clean":"2024-09-10T22:01:19.869555+0000","last_became_active":"2024-09-10T22:01:18.868064+0000","last_became_peered":"2024-09-10T22:01:18.868064+0000","last_unstale":"2024-09-10T22:01:19.869555+0000","last_undegraded":"2024-09-10T22:01:19.869555+0000","last_fullsized":"2024-09-10T22:01:19.869555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:30.265708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868127+0000","last_change":"2024-09-10T22:01:17.858789+0000","last_active":"2024-09-10T22:01:19.868127+0000","last_peered":"2024-09-10T22:01:19.868127+0000","last_clean":"2024-09-10T22:01:19.868127+0000","last_became_active":"2024-09-10T22:01:17.858503+0000","last_became_peered":"2024-09-10T22:01:17.858503+0000","last_unstale":"2024-09-10T22:01:19.868127+0000","last_undegraded":"2024-09-10T22:01:19.868127+0000","last_fullsized":"2024-09-10T22:01:19.868127+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:36.694934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.876894+0000","last_change":"2024-09-10T22:01:18.868425+0000","last_active":"2024-09-10T22:01:19.876894+0000","last_peered":"2024-09-10T22:01:19.876894+0000","last_clean":"2024-09-10T22:01:19.876894+0000","last_became_active":"2024-09-10T22:01:18.868119+0000","last_became_peered":"2024-09-10T22:01:18.868119+0000","last_unstale":"2024-09-10T22:01:19.876894+0000","last_undegraded":"2024-09-10T22:01:19.876894+0000","last_fullsized":"2024-09-10T22:01:19.876894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:51.416819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869616+0000","last_change":"2024-09-10T22:01:17.853234+0000","last_active":"2024-09-10T22:01:19.869616+0000","last_peered":"2024-09-10T22:01:19.869616+0000","last_clean":"2024-09-10T22:01:19.869616+0000","last_became_active":"2024-09-10T22:01:17.852912+0000","last_became_peered":"2024-09-10T22:01:17.852912+0000","last_unstale":"2024-09-10T22:01:19.869616+0000","last_undegraded":"2024-09-10T22:01:19.869616+0000","last_fullsized":"2024-09-10T22:01:19.869616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:40.036066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971618+0000","last_change":"2024-09-10T22:01:18.873210+0000","last_active":"2024-09-10T22:01:19.971618+0000","last_peered":"2024-09-10T22:01:19.971618+0000","last_clean":"2024-09-10T22:01:19.971618+0000","last_became_active":"2024-09-10T22:01:18.872945+0000","last_became_peered":"2024-09-10T22:01:18.872945+0000","last_unstale":"2024-09-10T22:01:19.971618+0000","last_undegraded":"2024-09-10T22:01:19.971618+0000","last_fullsized":"2024-09-10T22:01:19.971618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:33.957217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869412+0000","last_change":"2024-09-10T22:01:17.865798+0000","last_active":"2024-09-10T22:01:19.869412+0000","last_peered":"2024-09-10T22:01:19.869412+0000","last_clean":"2024-09-10T22:01:19.869412+0000","last_became_active":"2024-09-10T22:01:17.865521+0000","last_became_peered":"2024-09-10T22:01:17.865521+0000","last_unstale":"2024-09-10T22:01:19.869412+0000","last_undegraded":"2024-09-10T22:01:19.869412+0000","last_fullsized":"2024-09-10T22:01:19.869412+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:36.586342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869378+0000","last_change":"2024-09-10T22:01:18.874307+0000","last_active":"2024-09-10T22:01:19.869378+0000","last_peered":"2024-09-10T22:01:19.869378+0000","last_clean":"2024-09-10T22:01:19.869378+0000","last_became_active":"2024-09-10T22:01:18.874045+0000","last_became_peered":"2024-09-10T22:01:18.874045+0000","last_unstale":"2024-09-10T22:01:19.869378+0000","last_undegraded":"2024-09-10T22:01:19.869378+0000","last_fullsized":"2024-09-10T22:01:19.869378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:07.543195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874460+0000","last_change":"2024-09-10T22:01:17.869964+0000","last_active":"2024-09-10T22:01:19.874460+0000","last_peered":"2024-09-10T22:01:19.874460+0000","last_clean":"2024-09-10T22:01:19.874460+0000","last_became_active":"2024-09-10T22:01:17.869541+0000","last_became_peered":"2024-09-10T22:01:17.869541+0000","last_unstale":"2024-09-10T22:01:19.874460+0000","last_undegraded":"2024-09-10T22:01:19.874460+0000","last_fullsized":"2024-09-10T22:01:19.874460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:17.864650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869289+0000","last_change":"2024-09-10T22:01:18.878517+0000","last_active":"2024-09-10T22:01:19.869289+0000","last_peered":"2024-09-10T22:01:19.869289+0000","last_clean":"2024-09-10T22:01:19.869289+0000","last_became_active":"2024-09-10T22:01:18.878351+0000","last_became_peered":"2024-09-10T22:01:18.878351+0000","last_unstale":"2024-09-10T22:01:19.869289+0000","last_undegraded":"2024-09-10T22:01:19.869289+0000","last_fullsized":"2024-09-10T22:01:19.869289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:05.685293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971352+0000","last_change":"2024-09-10T22:01:17.862765+0000","last_active":"2024-09-10T22:01:19.971352+0000","last_peered":"2024-09-10T22:01:19.971352+0000","last_clean":"2024-09-10T22:01:19.971352+0000","last_became_active":"2024-09-10T22:01:17.861335+0000","last_became_peered":"2024-09-10T22:01:17.861335+0000","last_unstale":"2024-09-10T22:01:19.971352+0000","last_undegraded":"2024-09-10T22:01:19.971352+0000","last_fullsized":"2024-09-10T22:01:19.971352+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:36.694934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971274+0000","last_change":"2024-09-10T22:01:18.871412+0000","last_active":"2024-09-10T22:01:19.971274+0000","last_peered":"2024-09-10T22:01:19.971274+0000","last_clean":"2024-09-10T22:01:19.971274+0000","last_became_active":"2024-09-10T22:01:18.871042+0000","last_became_peered":"2024-09-10T22:01:18.871042+0000","last_unstale":"2024-09-10T22:01:19.971274+0000","last_undegraded":"2024-09-10T22:01:19.971274+0000","last_fullsized":"2024-09-10T22:01:19.971274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:19.927290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869177+0000","last_change":"2024-09-10T22:01:17.866217+0000","last_active":"2024-09-10T22:01:19.869177+0000","last_peered":"2024-09-10T22:01:19.869177+0000","last_clean":"2024-09-10T22:01:19.869177+0000","last_became_active":"2024-09-10T22:01:17.865942+0000","last_became_peered":"2024-09-10T22:01:17.865942+0000","last_unstale":"2024-09-10T22:01:19.869177+0000","last_undegraded":"2024-09-10T22:01:19.869177+0000","last_fullsized":"2024-09-10T22:01:19.869177+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:29.260133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.876922+0000","last_change":"2024-09-10T22:01:18.867134+0000","last_active":"2024-09-10T22:01:19.876922+0000","last_peered":"2024-09-10T22:01:19.876922+0000","last_clean":"2024-09-10T22:01:19.876922+0000","last_became_active":"2024-09-10T22:01:18.866884+0000","last_became_peered":"2024-09-10T22:01:18.866884+0000","last_unstale":"2024-09-10T22:01:19.876922+0000","last_undegraded":"2024-09-10T22:01:19.876922+0000","last_fullsized":"2024-09-10T22:01:19.876922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:46.701358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870775+0000","last_change":"2024-09-10T22:01:17.855694+0000","last_active":"2024-09-10T22:01:19.870775+0000","last_peered":"2024-09-10T22:01:19.870775+0000","last_clean":"2024-09-10T22:01:19.870775+0000","last_became_active":"2024-09-10T22:01:17.855323+0000","last_became_peered":"2024-09-10T22:01:17.855323+0000","last_unstale":"2024-09-10T22:01:19.870775+0000","last_undegraded":"2024-09-10T22:01:19.870775+0000","last_fullsized":"2024-09-10T22:01:19.870775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:08.932820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869100+0000","last_change":"2024-09-10T22:01:18.869148+0000","last_active":"2024-09-10T22:01:19.869100+0000","last_peered":"2024-09-10T22:01:19.869100+0000","last_clean":"2024-09-10T22:01:19.869100+0000","last_became_active":"2024-09-10T22:01:18.868772+0000","last_became_peered":"2024-09-10T22:01:18.868772+0000","last_unstale":"2024-09-10T22:01:19.869100+0000","last_undegraded":"2024-09-10T22:01:19.869100+0000","last_fullsized":"2024-09-10T22:01:19.869100+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:14.196122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.876970+0000","last_change":"2024-09-10T22:01:17.852954+0000","last_active":"2024-09-10T22:01:19.876970+0000","last_peered":"2024-09-10T22:01:19.876970+0000","last_clean":"2024-09-10T22:01:19.876970+0000","last_became_active":"2024-09-10T22:01:17.852697+0000","last_became_peered":"2024-09-10T22:01:17.852697+0000","last_unstale":"2024-09-10T22:01:19.876970+0000","last_undegraded":"2024-09-10T22:01:19.876970+0000","last_fullsized":"2024-09-10T22:01:19.876970+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:02.295993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878260+0000","last_change":"2024-09-10T22:01:18.876788+0000","last_active":"2024-09-10T22:01:19.878260+0000","last_peered":"2024-09-10T22:01:19.878260+0000","last_clean":"2024-09-10T22:01:19.878260+0000","last_became_active":"2024-09-10T22:01:18.876238+0000","last_became_peered":"2024-09-10T22:01:18.876238+0000","last_unstale":"2024-09-10T22:01:19.878260+0000","last_undegraded":"2024-09-10T22:01:19.878260+0000","last_fullsized":"2024-09-10T22:01:19.878260+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:33.759315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868335+0000","last_change":"2024-09-10T22:01:17.862292+0000","last_active":"2024-09-10T22:01:19.868335+0000","last_peered":"2024-09-10T22:01:19.868335+0000","last_clean":"2024-09-10T22:01:19.868335+0000","last_became_active":"2024-09-10T22:01:17.861498+0000","last_became_peered":"2024-09-10T22:01:17.861498+0000","last_unstale":"2024-09-10T22:01:19.868335+0000","last_undegraded":"2024-09-10T22:01:19.868335+0000","last_fullsized":"2024-09-10T22:01:19.868335+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:08.359041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869496+0000","last_change":"2024-09-10T22:01:18.873210+0000","last_active":"2024-09-10T22:01:19.869496+0000","last_peered":"2024-09-10T22:01:19.869496+0000","last_clean":"2024-09-10T22:01:19.869496+0000","last_became_active":"2024-09-10T22:01:18.872610+0000","last_became_peered":"2024-09-10T22:01:18.872610+0000","last_unstale":"2024-09-10T22:01:19.869496+0000","last_undegraded":"2024-09-10T22:01:19.869496+0000","last_fullsized":"2024-09-10T22:01:19.869496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:21.599818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868186+0000","last_change":"2024-09-10T22:01:17.865774+0000","last_active":"2024-09-10T22:01:19.868186+0000","last_peered":"2024-09-10T22:01:19.868186+0000","last_clean":"2024-09-10T22:01:19.868186+0000","last_became_active":"2024-09-10T22:01:17.865488+0000","last_became_peered":"2024-09-10T22:01:17.865488+0000","last_unstale":"2024-09-10T22:01:19.868186+0000","last_undegraded":"2024-09-10T22:01:19.868186+0000","last_fullsized":"2024-09-10T22:01:19.868186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:57.577284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.871070+0000","last_change":"2024-09-10T22:01:18.866748+0000","last_active":"2024-09-10T22:01:19.871070+0000","last_peered":"2024-09-10T22:01:19.871070+0000","last_clean":"2024-09-10T22:01:19.871070+0000","last_became_active":"2024-09-10T22:01:18.866386+0000","last_became_peered":"2024-09-10T22:01:18.866386+0000","last_unstale":"2024-09-10T22:01:19.871070+0000","last_undegraded":"2024-09-10T22:01:19.871070+0000","last_fullsized":"2024-09-10T22:01:19.871070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:34.079361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877515+0000","last_change":"2024-09-10T22:01:17.867707+0000","last_active":"2024-09-10T22:01:19.877515+0000","last_peered":"2024-09-10T22:01:19.877515+0000","last_clean":"2024-09-10T22:01:19.877515+0000","last_became_active":"2024-09-10T22:01:17.867395+0000","last_became_peered":"2024-09-10T22:01:17.867395+0000","last_unstale":"2024-09-10T22:01:19.877515+0000","last_undegraded":"2024-09-10T22:01:19.877515+0000","last_fullsized":"2024-09-10T22:01:19.877515+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:35.136565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971752+0000","last_change":"2024-09-10T22:01:18.869383+0000","last_active":"2024-09-10T22:01:19.971752+0000","last_peered":"2024-09-10T22:01:19.971752+0000","last_clean":"2024-09-10T22:01:19.971752+0000","last_became_active":"2024-09-10T22:01:18.869119+0000","last_became_peered":"2024-09-10T22:01:18.869119+0000","last_unstale":"2024-09-10T22:01:19.971752+0000","last_undegraded":"2024-09-10T22:01:19.971752+0000","last_fullsized":"2024-09-10T22:01:19.971752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:03.772153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.871118+0000","last_change":"2024-09-10T22:01:17.850829+0000","last_active":"2024-09-10T22:01:19.871118+0000","last_peered":"2024-09-10T22:01:19.871118+0000","last_clean":"2024-09-10T22:01:19.871118+0000","last_became_active":"2024-09-10T22:01:17.850332+0000","last_became_peered":"2024-09-10T22:01:17.850332+0000","last_unstale":"2024-09-10T22:01:19.871118+0000","last_undegraded":"2024-09-10T22:01:19.871118+0000","last_fullsized":"2024-09-10T22:01:19.871118+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:32.724306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877182+0000","last_change":"2024-09-10T22:01:18.876334+0000","last_active":"2024-09-10T22:01:19.877182+0000","last_peered":"2024-09-10T22:01:19.877182+0000","last_clean":"2024-09-10T22:01:19.877182+0000","last_became_active":"2024-09-10T22:01:18.876026+0000","last_became_peered":"2024-09-10T22:01:18.876026+0000","last_unstale":"2024-09-10T22:01:19.877182+0000","last_undegraded":"2024-09-10T22:01:19.877182+0000","last_fullsized":"2024-09-10T22:01:19.877182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:36.576646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870944+0000","last_change":"2024-09-10T22:01:17.861200+0000","last_active":"2024-09-10T22:01:19.870944+0000","last_peered":"2024-09-10T22:01:19.870944+0000","last_clean":"2024-09-10T22:01:19.870944+0000","last_became_active":"2024-09-10T22:01:17.860796+0000","last_became_peered":"2024-09-10T22:01:17.860796+0000","last_unstale":"2024-09-10T22:01:19.870944+0000","last_undegraded":"2024-09-10T22:01:19.870944+0000","last_fullsized":"2024-09-10T22:01:19.870944+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:18.433905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131599+0000","last_change":"2024-09-10T22:01:18.879012+0000","last_active":"2024-09-10T22:01:20.131599+0000","last_peered":"2024-09-10T22:01:20.131599+0000","last_clean":"2024-09-10T22:01:20.131599+0000","last_became_active":"2024-09-10T22:01:18.878175+0000","last_became_peered":"2024-09-10T22:01:18.878175+0000","last_unstale":"2024-09-10T22:01:20.131599+0000","last_undegraded":"2024-09-10T22:01:20.131599+0000","last_fullsized":"2024-09-10T22:01:20.131599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:15.918564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870816+0000","last_change":"2024-09-10T22:01:17.861102+0000","last_active":"2024-09-10T22:01:19.870816+0000","last_peered":"2024-09-10T22:01:19.870816+0000","last_clean":"2024-09-10T22:01:19.870816+0000","last_became_active":"2024-09-10T22:01:17.860779+0000","last_became_peered":"2024-09-10T22:01:17.860779+0000","last_unstale":"2024-09-10T22:01:19.870816+0000","last_undegraded":"2024-09-10T22:01:19.870816+0000","last_fullsized":"2024-09-10T22:01:19.870816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:30.650130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131319+0000","last_change":"2024-09-10T22:01:18.871029+0000","last_active":"2024-09-10T22:01:20.131319+0000","last_peered":"2024-09-10T22:01:20.131319+0000","last_clean":"2024-09-10T22:01:20.131319+0000","last_became_active":"2024-09-10T22:01:18.866768+0000","last_became_peered":"2024-09-10T22:01:18.866768+0000","last_unstale":"2024-09-10T22:01:20.131319+0000","last_undegraded":"2024-09-10T22:01:20.131319+0000","last_fullsized":"2024-09-10T22:01:20.131319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:54.676713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874530+0000","last_change":"2024-09-10T22:01:17.851819+0000","last_active":"2024-09-10T22:01:19.874530+0000","last_peered":"2024-09-10T22:01:19.874530+0000","last_clean":"2024-09-10T22:01:19.874530+0000","last_became_active":"2024-09-10T22:01:17.850945+0000","last_became_peered":"2024-09-10T22:01:17.850945+0000","last_unstale":"2024-09-10T22:01:19.874530+0000","last_undegraded":"2024-09-10T22:01:19.874530+0000","last_fullsized":"2024-09-10T22:01:19.874530+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:35.136565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131482+0000","last_change":"2024-09-10T22:01:18.868998+0000","last_active":"2024-09-10T22:01:20.131482+0000","last_peered":"2024-09-10T22:01:20.131482+0000","last_clean":"2024-09-10T22:01:20.131482+0000","last_became_active":"2024-09-10T22:01:18.868508+0000","last_became_peered":"2024-09-10T22:01:18.868508+0000","last_unstale":"2024-09-10T22:01:20.131482+0000","last_undegraded":"2024-09-10T22:01:20.131482+0000","last_fullsized":"2024-09-10T22:01:20.131482+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:08.433283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868998+0000","last_change":"2024-09-10T22:01:17.849861+0000","last_active":"2024-09-10T22:01:19.868998+0000","last_peered":"2024-09-10T22:01:19.868998+0000","last_clean":"2024-09-10T22:01:19.868998+0000","last_became_active":"2024-09-10T22:01:17.849568+0000","last_became_peered":"2024-09-10T22:01:17.849568+0000","last_unstale":"2024-09-10T22:01:19.868998+0000","last_undegraded":"2024-09-10T22:01:19.868998+0000","last_fullsized":"2024-09-10T22:01:19.868998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:53.054618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131241+0000","last_change":"2024-09-10T22:01:18.865130+0000","last_active":"2024-09-10T22:01:20.131241+0000","last_peered":"2024-09-10T22:01:20.131241+0000","last_clean":"2024-09-10T22:01:20.131241+0000","last_became_active":"2024-09-10T22:01:18.864863+0000","last_became_peered":"2024-09-10T22:01:18.864863+0000","last_unstale":"2024-09-10T22:01:20.131241+0000","last_undegraded":"2024-09-10T22:01:20.131241+0000","last_fullsized":"2024-09-10T22:01:20.131241+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:54.059872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870174+0000","last_change":"2024-09-10T22:01:17.850707+0000","last_active":"2024-09-10T22:01:19.870174+0000","last_peered":"2024-09-10T22:01:19.870174+0000","last_clean":"2024-09-10T22:01:19.870174+0000","last_became_active":"2024-09-10T22:01:17.850396+0000","last_became_peered":"2024-09-10T22:01:17.850396+0000","last_unstale":"2024-09-10T22:01:19.870174+0000","last_undegraded":"2024-09-10T22:01:19.870174+0000","last_fullsized":"2024-09-10T22:01:19.870174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:42.075103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877753+0000","last_change":"2024-09-10T22:01:18.883013+0000","last_active":"2024-09-10T22:01:19.877753+0000","last_peered":"2024-09-10T22:01:19.877753+0000","last_clean":"2024-09-10T22:01:19.877753+0000","last_became_active":"2024-09-10T22:01:18.882792+0000","last_became_peered":"2024-09-10T22:01:18.882792+0000","last_unstale":"2024-09-10T22:01:19.877753+0000","last_undegraded":"2024-09-10T22:01:19.877753+0000","last_fullsized":"2024-09-10T22:01:19.877753+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:22.047393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.970933+0000","last_change":"2024-09-10T22:01:17.861920+0000","last_active":"2024-09-10T22:01:19.970933+0000","last_peered":"2024-09-10T22:01:19.970933+0000","last_clean":"2024-09-10T22:01:19.970933+0000","last_became_active":"2024-09-10T22:01:17.861634+0000","last_became_peered":"2024-09-10T22:01:17.861634+0000","last_unstale":"2024-09-10T22:01:19.970933+0000","last_undegraded":"2024-09-10T22:01:19.970933+0000","last_fullsized":"2024-09-10T22:01:19.970933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:57.577284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:20.131607+0000","last_change":"2024-09-10T22:01:18.870633+0000","last_active":"2024-09-10T22:01:20.131607+0000","last_peered":"2024-09-10T22:01:20.131607+0000","last_clean":"2024-09-10T22:01:20.131607+0000","last_became_active":"2024-09-10T22:01:18.870291+0000","last_became_peered":"2024-09-10T22:01:18.870291+0000","last_unstale":"2024-09-10T22:01:20.131607+0000","last_undegraded":"2024-09-10T22:01:20.131607+0000","last_fullsized":"2024-09-10T22:01:20.131607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:01.983686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877542+0000","last_change":"2024-09-10T22:01:17.858498+0000","last_active":"2024-09-10T22:01:19.877542+0000","last_peered":"2024-09-10T22:01:19.877542+0000","last_clean":"2024-09-10T22:01:19.877542+0000","last_became_active":"2024-09-10T22:01:17.858227+0000","last_became_peered":"2024-09-10T22:01:17.858227+0000","last_unstale":"2024-09-10T22:01:19.877542+0000","last_undegraded":"2024-09-10T22:01:19.877542+0000","last_fullsized":"2024-09-10T22:01:19.877542+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:04.203230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.971729+0000","last_change":"2024-09-10T22:01:18.870760+0000","last_active":"2024-09-10T22:01:19.971729+0000","last_peered":"2024-09-10T22:01:19.971729+0000","last_clean":"2024-09-10T22:01:19.971729+0000","last_became_active":"2024-09-10T22:01:18.870510+0000","last_became_peered":"2024-09-10T22:01:18.870510+0000","last_unstale":"2024-09-10T22:01:19.971729+0000","last_undegraded":"2024-09-10T22:01:19.971729+0000","last_fullsized":"2024-09-10T22:01:19.971729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:12.922313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.876139+0000","last_change":"2024-09-10T22:01:17.851892+0000","last_active":"2024-09-10T22:01:19.876139+0000","last_peered":"2024-09-10T22:01:19.876139+0000","last_clean":"2024-09-10T22:01:19.876139+0000","last_became_active":"2024-09-10T22:01:17.851244+0000","last_became_peered":"2024-09-10T22:01:17.851244+0000","last_unstale":"2024-09-10T22:01:19.876139+0000","last_undegraded":"2024-09-10T22:01:19.876139+0000","last_fullsized":"2024-09-10T22:01:19.876139+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:18.742907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874252+0000","last_change":"2024-09-10T22:01:18.862523+0000","last_active":"2024-09-10T22:01:19.874252+0000","last_peered":"2024-09-10T22:01:19.874252+0000","last_clean":"2024-09-10T22:01:19.874252+0000","last_became_active":"2024-09-10T22:01:18.861607+0000","last_became_peered":"2024-09-10T22:01:18.861607+0000","last_unstale":"2024-09-10T22:01:19.874252+0000","last_undegraded":"2024-09-10T22:01:19.874252+0000","last_fullsized":"2024-09-10T22:01:19.874252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:21.793658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877213+0000","last_change":"2024-09-10T22:01:17.858275+0000","last_active":"2024-09-10T22:01:19.877213+0000","last_peered":"2024-09-10T22:01:19.877213+0000","last_clean":"2024-09-10T22:01:19.877213+0000","last_became_active":"2024-09-10T22:01:17.857981+0000","last_became_peered":"2024-09-10T22:01:17.857981+0000","last_unstale":"2024-09-10T22:01:19.877213+0000","last_undegraded":"2024-09-10T22:01:19.877213+0000","last_fullsized":"2024-09-10T22:01:19.877213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:13.631349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869168+0000","last_change":"2024-09-10T22:01:18.866301+0000","last_active":"2024-09-10T22:01:19.869168+0000","last_peered":"2024-09-10T22:01:19.869168+0000","last_clean":"2024-09-10T22:01:19.869168+0000","last_became_active":"2024-09-10T22:01:18.865947+0000","last_became_peered":"2024-09-10T22:01:18.865947+0000","last_unstale":"2024-09-10T22:01:19.869168+0000","last_undegraded":"2024-09-10T22:01:19.869168+0000","last_fullsized":"2024-09-10T22:01:19.869168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:32.295747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874612+0000","last_change":"2024-09-10T22:01:17.853418+0000","last_active":"2024-09-10T22:01:19.874612+0000","last_peered":"2024-09-10T22:01:19.874612+0000","last_clean":"2024-09-10T22:01:19.874612+0000","last_became_active":"2024-09-10T22:01:17.853148+0000","last_became_peered":"2024-09-10T22:01:17.853148+0000","last_unstale":"2024-09-10T22:01:19.874612+0000","last_undegraded":"2024-09-10T22:01:19.874612+0000","last_fullsized":"2024-09-10T22:01:19.874612+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:13.631349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.874425+0000","last_change":"2024-09-10T22:01:18.871427+0000","last_active":"2024-09-10T22:01:19.874425+0000","last_peered":"2024-09-10T22:01:19.874425+0000","last_clean":"2024-09-10T22:01:19.874425+0000","last_became_active":"2024-09-10T22:01:18.869064+0000","last_became_peered":"2024-09-10T22:01:18.869064+0000","last_unstale":"2024-09-10T22:01:19.874425+0000","last_undegraded":"2024-09-10T22:01:19.874425+0000","last_fullsized":"2024-09-10T22:01:19.874425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:51.416819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.877072+0000","last_change":"2024-09-10T22:01:17.853258+0000","last_active":"2024-09-10T22:01:19.877072+0000","last_peered":"2024-09-10T22:01:19.877072+0000","last_clean":"2024-09-10T22:01:19.877072+0000","last_became_active":"2024-09-10T22:01:17.852933+0000","last_became_peered":"2024-09-10T22:01:17.852933+0000","last_unstale":"2024-09-10T22:01:19.877072+0000","last_undegraded":"2024-09-10T22:01:19.877072+0000","last_fullsized":"2024-09-10T22:01:19.877072+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:18.742907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868182+0000","last_change":"2024-09-10T22:01:18.869936+0000","last_active":"2024-09-10T22:01:19.868182+0000","last_peered":"2024-09-10T22:01:19.868182+0000","last_clean":"2024-09-10T22:01:19.868182+0000","last_became_active":"2024-09-10T22:01:18.869680+0000","last_became_peered":"2024-09-10T22:01:18.869680+0000","last_unstale":"2024-09-10T22:01:19.868182+0000","last_undegraded":"2024-09-10T22:01:19.868182+0000","last_fullsized":"2024-09-10T22:01:19.868182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:29.481248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870252+0000","last_change":"2024-09-10T22:01:17.854341+0000","last_active":"2024-09-10T22:01:19.870252+0000","last_peered":"2024-09-10T22:01:19.870252+0000","last_clean":"2024-09-10T22:01:19.870252+0000","last_became_active":"2024-09-10T22:01:17.854067+0000","last_became_peered":"2024-09-10T22:01:17.854067+0000","last_unstale":"2024-09-10T22:01:19.870252+0000","last_undegraded":"2024-09-10T22:01:19.870252+0000","last_fullsized":"2024-09-10T22:01:19.870252+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:24.204319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.870325+0000","last_change":"2024-09-10T22:01:18.875620+0000","last_active":"2024-09-10T22:01:19.870325+0000","last_peered":"2024-09-10T22:01:19.870325+0000","last_clean":"2024-09-10T22:01:19.870325+0000","last_became_active":"2024-09-10T22:01:18.875242+0000","last_became_peered":"2024-09-10T22:01:18.875242+0000","last_unstale":"2024-09-10T22:01:19.870325+0000","last_undegraded":"2024-09-10T22:01:19.870325+0000","last_fullsized":"2024-09-10T22:01:19.870325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:37.591916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.868080+0000","last_change":"2024-09-10T22:01:17.861574+0000","last_active":"2024-09-10T22:01:19.868080+0000","last_peered":"2024-09-10T22:01:19.868080+0000","last_clean":"2024-09-10T22:01:19.868080+0000","last_became_active":"2024-09-10T22:01:17.860999+0000","last_became_peered":"2024-09-10T22:01:17.860999+0000","last_unstale":"2024-09-10T22:01:19.868080+0000","last_undegraded":"2024-09-10T22:01:19.868080+0000","last_fullsized":"2024-09-10T22:01:19.868080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:15.531745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878112+0000","last_change":"2024-09-10T22:01:18.878139+0000","last_active":"2024-09-10T22:01:19.878112+0000","last_peered":"2024-09-10T22:01:19.878112+0000","last_clean":"2024-09-10T22:01:19.878112+0000","last_became_active":"2024-09-10T22:01:18.877966+0000","last_became_peered":"2024-09-10T22:01:18.877966+0000","last_unstale":"2024-09-10T22:01:19.878112+0000","last_undegraded":"2024-09-10T22:01:19.878112+0000","last_fullsized":"2024-09-10T22:01:19.878112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:17.830821+0000","last_clean_scrub_stamp":"2024-09-10T22:01:17.830821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:14.196122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.869052+0000","last_change":"2024-09-10T22:01:17.857059+0000","last_active":"2024-09-10T22:01:19.869052+0000","last_peered":"2024-09-10T22:01:19.869052+0000","last_clean":"2024-09-10T22:01:19.869052+0000","last_became_active":"2024-09-10T22:01:17.856184+0000","last_became_peered":"2024-09-10T22:01:17.856184+0000","last_unstale":"2024-09-10T22:01:19.869052+0000","last_undegraded":"2024-09-10T22:01:19.869052+0000","last_fullsized":"2024-09-10T22:01:19.869052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:01:16.825247+0000","last_clean_scrub_stamp":"2024-09-10T22:01:16.825247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:46.578161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":130,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T22:01:19.878510+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:01:19.878510+0000","last_peered":"2024-09-10T22:01:19.878510+0000","last_clean":"2024-09-10T22:01:19.878510+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:01:19.878510+0000","last_undegraded":"2024-09-10T22:01:19.878510+0000","last_fullsized":"2024-09-10T22:01:19.878510+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":962560,"data_stored":729934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":372736,"data_stored":140092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":372736,"data_stored":140148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":372736,"data_stored":139580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":141110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":380928,"data_stored":141110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":958464,"data_stored":729844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":139546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:01:23.415 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:01:23.415 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T22:01:23.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.598+0000 7fe3cce00640 1 -- 172.21.15.66:0/349669420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c80b32a0 msgr2=0x7fe3c80af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:23.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.598+0000 7fe3cce00640 1 --2- 172.21.15.66:0/349669420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c80b32a0 0x7fe3c80af390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009bc0 tx=0x7fe3c001c9f0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.598+0000 7fe3cce00640 1 -- 172.21.15.66:0/349669420 shutdown_connections 2024-09-10T22:01:23.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.598+0000 7fe3cce00640 1 --2- 172.21.15.66:0/349669420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c80b32a0 0x7fe3c80af390 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.598+0000 7fe3cce00640 1 --2- 172.21.15.66:0/349669420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3c80b0f60 0x7fe3c80d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.598+0000 7fe3cce00640 1 -- 172.21.15.66:0/349669420 >> 172.21.15.66:0/349669420 conn(0x7fe3c80d1d00 msgr2=0x7fe3c80d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:23.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.598+0000 7fe3cce00640 1 -- 172.21.15.66:0/349669420 shutdown_connections 2024-09-10T22:01:23.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.598+0000 7fe3cce00640 1 -- 172.21.15.66:0/349669420 wait complete. 2024-09-10T22:01:23.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.598+0000 7fe3cce00640 1 Processor -- start 2024-09-10T22:01:23.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.598+0000 7fe3cce00640 1 -- start start 2024-09-10T22:01:23.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.598+0000 7fe3cce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c80b0f60 0x7fe3c80b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:23.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.598+0000 7fe3cce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c80b32a0 0x7fe3c8147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:23.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3cce00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3c815f460 0x7fe3c81fae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:23.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3cce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c80b0600 con 0x7fe3c80b0f60 2024-09-10T22:01:23.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3cce00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c80b0480 con 0x7fe3c80b32a0 2024-09-10T22:01:23.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3cce00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c8168c60 con 0x7fe3c815f460 2024-09-10T22:01:23.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3c5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c80b0f60 0x7fe3c80b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:23.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3c5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c80b0f60 0x7fe3c80b72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40422/0 (socket says 172.21.15.66:40422) 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3c5e00640 1 -- 172.21.15.66:0/2062448656 learned_addr learned my addr 172.21.15.66:0/2062448656 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3c6800640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3c815f460 0x7fe3c81fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3bfe00640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c80b32a0 0x7fe3c8147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3c5e00640 1 -- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3c815f460 msgr2=0x7fe3c81fae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3c5e00640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3c815f460 0x7fe3c81fae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3c5e00640 1 -- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c80b32a0 msgr2=0x7fe3c8147390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3c5e00640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c80b32a0 0x7fe3c8147390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3c5e00640 1 -- 172.21.15.66:0/2062448656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c80c54d0 con 0x7fe3c80b0f60 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3c5e00640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c80b0f60 0x7fe3c80b72b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800c400 tx=0x7fe3b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3be000640 1 -- 172.21.15.66:0/2062448656 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8011790 con 0x7fe3c80b0f60 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3be000640 1 -- 172.21.15.66:0/2062448656 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3b8017070 con 0x7fe3c80b0f60 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3be000640 1 -- 172.21.15.66:0/2062448656 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8011cc0 con 0x7fe3c80b0f60 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3cce00640 1 -- 172.21.15.66:0/2062448656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c80b6710 con 0x7fe3c80b0f60 2024-09-10T22:01:23.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3cce00640 1 -- 172.21.15.66:0/2062448656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c80b6cd0 con 0x7fe3c80b0f60 2024-09-10T22:01:23.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3cce00640 1 -- 172.21.15.66:0/2062448656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c80af390 con 0x7fe3c80b0f60 2024-09-10T22:01:23.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.602+0000 7fe3be000640 1 -- 172.21.15.66:0/2062448656 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe3b801c460 con 0x7fe3c80b0f60 2024-09-10T22:01:23.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.606+0000 7fe3be000640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe3a40725e0 0x7fe3a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:23.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.606+0000 7fe3be000640 1 -- 172.21.15.66:0/2062448656 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe3b8092ea0 con 0x7fe3c80b0f60 2024-09-10T22:01:23.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.606+0000 7fe3bfe00640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe3a40725e0 0x7fe3a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:23.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.606+0000 7fe3bfe00640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe3a40725e0 0x7fe3a4074aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe3c814ce10 tx=0x7fe3b00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:23.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.610+0000 7fe3be000640 1 -- 172.21.15.66:0/2062448656 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe3b805df20 con 0x7fe3c80b0f60 2024-09-10T22:01:23.829 INFO:tasks.ceph.mds.a.smithi066.stderr:2024-09-10T22:01:23.826+0000 7f2246a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:01:23.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.850+0000 7fe3cce00640 1 -- 172.21.15.66:0/2062448656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe3c8074aa0 con 0x7fe3c80b0f60 2024-09-10T22:01:23.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.850+0000 7fe3be000640 1 -- 172.21.15.66:0/2062448656 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) ==== 190+0+147 (secure 0 0 0) 0x7fe3b8062950 con 0x7fe3c80b0f60 2024-09-10T22:01:23.856 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T22:01:23.856 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw== 2024-09-10T22:01:23.856 INFO:teuthology.orchestra.run.smithi066.stdout: caps mds = "allow" 2024-09-10T22:01:23.856 INFO:teuthology.orchestra.run.smithi066.stdout: caps mon = "allow r" 2024-09-10T22:01:23.856 INFO:teuthology.orchestra.run.smithi066.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T22:01:23.856 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-09-10T22:01:23.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.854+0000 7fe3cce00640 1 -- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe3a40725e0 msgr2=0x7fe3a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:23.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.854+0000 7fe3cce00640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe3a40725e0 0x7fe3a4074aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe3c814ce10 tx=0x7fe3b00091e0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.854+0000 7fe3cce00640 1 -- 172.21.15.66:0/2062448656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c80b0f60 msgr2=0x7fe3c80b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:23.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.854+0000 7fe3cce00640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c80b0f60 0x7fe3c80b72b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800c400 tx=0x7fe3b800c8c0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.854+0000 7fe3cce00640 1 -- 172.21.15.66:0/2062448656 shutdown_connections 2024-09-10T22:01:23.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.854+0000 7fe3cce00640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe3a40725e0 0x7fe3a4074aa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.854+0000 7fe3cce00640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3c815f460 0x7fe3c81fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.854+0000 7fe3cce00640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c80b32a0 0x7fe3c8147390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.854+0000 7fe3cce00640 1 --2- 172.21.15.66:0/2062448656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c80b0f60 0x7fe3c80b72b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:23.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.854+0000 7fe3cce00640 1 -- 172.21.15.66:0/2062448656 >> 172.21.15.66:0/2062448656 conn(0x7fe3c80d1d00 msgr2=0x7fe3c815e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:23.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.854+0000 7fe3cce00640 1 -- 172.21.15.66:0/2062448656 shutdown_connections 2024-09-10T22:01:23.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:23.854+0000 7fe3cce00640 1 -- 172.21.15.66:0/2062448656 wait complete. 2024-09-10T22:01:23.877 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:01:24.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 -- 172.21.15.66:0/1999530437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec0c4fc0 msgr2=0x7fc6ec0ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 --2- 172.21.15.66:0/1999530437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec0c4fc0 0x7fc6ec0ca6e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0cf9e0 tx=0x7fc6e401ccf0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 -- 172.21.15.66:0/1999530437 shutdown_connections 2024-09-10T22:01:24.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 --2- 172.21.15.66:0/1999530437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec0c4fc0 0x7fc6ec0ca6e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 --2- 172.21.15.66:0/1999530437 >> v2:172.21.15.94:3300/0 conn(0x7fc6ec0d0550 0x7fc6ec0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 -- 172.21.15.66:0/1999530437 >> 172.21.15.66:0/1999530437 conn(0x7fc6ec0cfa30 msgr2=0x7fc6ec0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:24.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 -- 172.21.15.66:0/1999530437 shutdown_connections 2024-09-10T22:01:24.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 -- 172.21.15.66:0/1999530437 wait complete. 2024-09-10T22:01:24.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 Processor -- start 2024-09-10T22:01:24.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 -- start start 2024-09-10T22:01:24.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ec0c4fc0 0x7fc6ec1547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc6ec15a310 0x7fc6ec149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec149cf0 0x7fc6ec156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec165470 con 0x7fc6ec149cf0 2024-09-10T22:01:24.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec1652f0 con 0x7fc6ec0c4fc0 2024-09-10T22:01:24.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f3400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ec1655f0 con 0x7fc6ec15a310 2024-09-10T22:01:24.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec149cf0 0x7fc6ec156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec149cf0 0x7fc6ec156cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40452/0 (socket says 172.21.15.66:40452) 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f1400640 1 -- 172.21.15.66:0/671446872 learned_addr learned my addr 172.21.15.66:0/671446872 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6f0a00640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ec0c4fc0 0x7fc6ec1547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.022+0000 7fc6e3e00640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc6ec15a310 0x7fc6ec149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6f1400640 1 -- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc6ec15a310 msgr2=0x7fc6ec149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6f1400640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc6ec15a310 0x7fc6ec149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6f1400640 1 -- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ec0c4fc0 msgr2=0x7fc6ec1547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6f1400640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ec0c4fc0 0x7fc6ec1547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6f1400640 1 -- 172.21.15.66:0/671446872 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec154d00 con 0x7fc6ec149cf0 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6f0a00640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ec0c4fc0 0x7fc6ec1547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6f1400640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec149cf0 0x7fc6ec156cb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4019900 tx=0x7fc6e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6e9600640 1 -- 172.21.15.66:0/671446872 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40bb070 con 0x7fc6ec149cf0 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6e9600640 1 -- 172.21.15.66:0/671446872 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6e4004b40 con 0x7fc6ec149cf0 2024-09-10T22:01:24.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6e9600640 1 -- 172.21.15.66:0/671446872 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40b2040 con 0x7fc6ec149cf0 2024-09-10T22:01:24.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6f3400640 1 -- 172.21.15.66:0/671446872 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ec149ab0 con 0x7fc6ec149cf0 2024-09-10T22:01:24.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6f3400640 1 -- 172.21.15.66:0/671446872 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ec155260 con 0x7fc6ec149cf0 2024-09-10T22:01:24.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6f3400640 1 -- 172.21.15.66:0/671446872 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ec0cb970 con 0x7fc6ec149cf0 2024-09-10T22:01:24.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6e9600640 1 -- 172.21.15.66:0/671446872 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc6e40b5030 con 0x7fc6ec149cf0 2024-09-10T22:01:24.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6e9600640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc6c80725e0 0x7fc6c8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.026+0000 7fc6e9600640 1 -- 172.21.15.66:0/671446872 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc6e412ced0 con 0x7fc6ec149cf0 2024-09-10T22:01:24.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.030+0000 7fc6f0a00640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc6c80725e0 0x7fc6c8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.030+0000 7fc6f0a00640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc6c80725e0 0x7fc6c8074aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0016b0 tx=0x7fc6dc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:24.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.030+0000 7fc6e9600640 1 -- 172.21.15.66:0/671446872 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc6ec154d00 con 0x7fc6ec149cf0 2024-09-10T22:01:24.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 -- 172.21.15.66:0/671446872 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6ec1507b0 con 0x7fc6ec149cf0 2024-09-10T22:01:24.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6e9600640 1 -- 172.21.15.66:0/671446872 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5217 (secure 0 0 0) 0x7fc6e4016970 con 0x7fc6ec149cf0 2024-09-10T22:01:24.246 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:01:24.246 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"btime":"2024-09-10T22:01:21:872011+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1019192175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1019192175},{"type":"v1","addr":"172.21.15.94:6837","nonce":1019192175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/554383263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":554383263},{"type":"v1","addr":"172.21.15.94:6836","nonce":554383263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2368842390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2368842390},{"type":"v1","addr":"172.21.15.66:6834","nonce":2368842390}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:01:18.244077+0000","modified":"2024-09-10T22:01:21.271158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/297228676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":297228676},{"type":"v1","addr":"172.21.15.66:6835","nonce":297228676}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-09-10T22:01:24.246 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-09-10T22:01:24.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 -- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc6c80725e0 msgr2=0x7fc6c8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc6c80725e0 0x7fc6c8074aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0016b0 tx=0x7fc6dc009040 comp rx=0 tx=0).stop 2024-09-10T22:01:24.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 -- 172.21.15.66:0/671446872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec149cf0 msgr2=0x7fc6ec156cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec149cf0 0x7fc6ec156cb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4019900 tx=0x7fc6e4004270 comp rx=0 tx=0).stop 2024-09-10T22:01:24.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 -- 172.21.15.66:0/671446872 shutdown_connections 2024-09-10T22:01:24.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc6c80725e0 0x7fc6c8074aa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec149cf0 0x7fc6ec156cb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc6ec15a310 0x7fc6ec149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 --2- 172.21.15.66:0/671446872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ec0c4fc0 0x7fc6ec1547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 -- 172.21.15.66:0/671446872 >> 172.21.15.66:0/671446872 conn(0x7fc6ec0cfa30 msgr2=0x7fc6ec0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:24.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 -- 172.21.15.66:0/671446872 shutdown_connections 2024-09-10T22:01:24.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.242+0000 7fc6f3400640 1 -- 172.21.15.66:0/671446872 wait complete. 2024-09-10T22:01:24.670 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:01:18.244077+0000', 'modified': '2024-09-10T22:01:21.271158+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4908}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4908': {'gid': 4908, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/297228676', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6833', 'nonce': 297228676}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 297228676}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4908, 'qdb_cluster': [4908]} 2024-09-10T22:01:24.670 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:01:24.671 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T22:01:24.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.814+0000 7fdfed200640 1 -- 172.21.15.66:0/3697709946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfcc0061a0 msgr2=0x7fdfcc00cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.814+0000 7fdfed200640 1 --2- 172.21.15.66:0/3697709946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfcc0061a0 0x7fdfcc00cae0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80cfa00 tx=0x7fdfe001cb80 comp rx=0 tx=0).stop 2024-09-10T22:01:24.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.814+0000 7fdfe6800640 1 -- 172.21.15.66:0/3697709946 reap_dead start 2024-09-10T22:01:24.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.814+0000 7fdfed200640 1 -- 172.21.15.66:0/3697709946 shutdown_connections 2024-09-10T22:01:24.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.814+0000 7fdfed200640 1 -- 172.21.15.66:0/3697709946 >> 172.21.15.66:0/3697709946 conn(0x7fdfe80cfa30 msgr2=0x7fdfe80cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:24.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.814+0000 7fdfed200640 1 -- 172.21.15.66:0/3697709946 shutdown_connections 2024-09-10T22:01:24.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.814+0000 7fdfed200640 1 -- 172.21.15.66:0/3697709946 wait complete. 2024-09-10T22:01:24.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfed200640 1 Processor -- start 2024-09-10T22:01:24.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfed200640 1 -- start start 2024-09-10T22:01:24.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfed200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdfcc0061a0 0x7fdfe80b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfed200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfe813d220 0x7fdfe81635a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfed200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfe813d6a0 0x7fdfe8164460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfed200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe80cb230 con 0x7fdfe813d220 2024-09-10T22:01:24.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfed200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe80cb0b0 con 0x7fdfe813d6a0 2024-09-10T22:01:24.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfed200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe8166300 con 0x7fdfcc0061a0 2024-09-10T22:01:24.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfdd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfe813d220 0x7fdfe81635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfdd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfe813d220 0x7fdfe81635a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40464/0 (socket says 172.21.15.66:40464) 2024-09-10T22:01:24.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfdd400640 1 -- 172.21.15.66:0/3382406361 learned_addr learned my addr 172.21.15.66:0/3382406361 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:24.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfe6800640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfe813d6a0 0x7fdfe8164460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfe5e00640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdfcc0061a0 0x7fdfe80b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfdd400640 1 -- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdfcc0061a0 msgr2=0x7fdfe80b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfdd400640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdfcc0061a0 0x7fdfe80b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfdd400640 1 -- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfe813d6a0 msgr2=0x7fdfe8164460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfdd400640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfe813d6a0 0x7fdfe8164460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfdd400640 1 -- 172.21.15.66:0/3382406361 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe80b53a0 con 0x7fdfe813d220 2024-09-10T22:01:24.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfe6800640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfe813d6a0 0x7fdfe8164460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:24.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfe5e00640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdfcc0061a0 0x7fdfe80b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:24.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfdd400640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfe813d220 0x7fdfe81635a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0011b10 tx=0x7fdfd0009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:24.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfdea00640 1 -- 172.21.15.66:0/3382406361 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd001e070 con 0x7fdfe813d220 2024-09-10T22:01:24.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfdea00640 1 -- 172.21.15.66:0/3382406361 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfd000fe20 con 0x7fdfe813d220 2024-09-10T22:01:24.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfdea00640 1 -- 172.21.15.66:0/3382406361 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd00042b0 con 0x7fdfe813d220 2024-09-10T22:01:24.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfed200640 1 -- 172.21.15.66:0/3382406361 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe80b5680 con 0x7fdfe813d220 2024-09-10T22:01:24.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.818+0000 7fdfed200640 1 -- 172.21.15.66:0/3382406361 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe81505c0 con 0x7fdfe813d220 2024-09-10T22:01:24.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.822+0000 7fdfdea00640 1 -- 172.21.15.66:0/3382406361 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdfd0003600 con 0x7fdfe813d220 2024-09-10T22:01:24.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.822+0000 7fdfdea00640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdfc40725e0 0x7fdfc4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.822+0000 7fdfdea00640 1 -- 172.21.15.66:0/3382406361 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdfd0093ff0 con 0x7fdfe813d220 2024-09-10T22:01:24.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.822+0000 7fdff066a000 1 Processor -- start 2024-09-10T22:01:24.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.822+0000 7fdfe5e00640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdfc40725e0 0x7fdfc4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.822+0000 7fdff066a000 1 -- start start 2024-09-10T22:01:24.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b709d0 0x55a7e8b70db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b712f0 0x55a7e8b248d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b24e10 0x55a7e8b251f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a7e8a453f0 con 0x55a7e8b712f0 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a7e8982b20 con 0x55a7e8b709d0 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55a7e8971f50 con 0x55a7e8b24e10 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfe5e00640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdfc40725e0 0x7fdfc4074aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8001230 tx=0x7fdfd8001e80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfdd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b712f0 0x55a7e8b248d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfdd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b712f0 0x55a7e8b248d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40470/0 (socket says 172.21.15.66:40470) 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfdd400640 1 -- 172.21.15.66:0/3347307202 learned_addr learned my addr 172.21.15.66:0/3347307202 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfe5e00640 1 --2- 172.21.15.66:0/3347307202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b709d0 0x55a7e8b70db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfe6800640 1 --2- 172.21.15.66:0/3347307202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b24e10 0x55a7e8b251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfdd400640 1 -- 172.21.15.66:0/3347307202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b24e10 msgr2=0x55a7e8b251f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfdd400640 1 --2- 172.21.15.66:0/3347307202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b24e10 0x55a7e8b251f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfdd400640 1 -- 172.21.15.66:0/3347307202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b709d0 msgr2=0x55a7e8b70db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfdd400640 1 --2- 172.21.15.66:0/3347307202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b709d0 0x55a7e8b70db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfdd400640 1 -- 172.21.15.66:0/3347307202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e8939590 con 0x55a7e8b712f0 2024-09-10T22:01:24.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfdd400640 1 --2- 172.21.15.66:0/3347307202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b712f0 0x55a7e8b248d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdfd005ed40 tx=0x7fdfd005ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167315dae173bafc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:24.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfd6a00640 1 -- 172.21.15.66:0/3347307202 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0068490 con 0x55a7e8b712f0 2024-09-10T22:01:24.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfd6a00640 1 -- 172.21.15.66:0/3347307202 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfd005b3a0 con 0x55a7e8b712f0 2024-09-10T22:01:24.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdfd6a00640 1 -- 172.21.15.66:0/3347307202 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd005e500 con 0x55a7e8b712f0 2024-09-10T22:01:24.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 -- 172.21.15.66:0/3347307202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b712f0 msgr2=0x55a7e8b248d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 --2- 172.21.15.66:0/3347307202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b712f0 0x55a7e8b248d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdfd005ed40 tx=0x7fdfd005ec90 comp rx=0 tx=0).stop 2024-09-10T22:01:24.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 -- 172.21.15.66:0/3347307202 shutdown_connections 2024-09-10T22:01:24.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 --2- 172.21.15.66:0/3347307202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b24e10 0x55a7e8b251f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 --2- 172.21.15.66:0/3347307202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b712f0 0x55a7e8b248d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 --2- 172.21.15.66:0/3347307202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b709d0 0x55a7e8b70db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 -- 172.21.15.66:0/3347307202 >> 172.21.15.66:0/3347307202 conn(0x55a7e8a0e130 msgr2=0x55a7e8b53e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:24.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 -- 172.21.15.66:0/3347307202 shutdown_connections 2024-09-10T22:01:24.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 -- 172.21.15.66:0/3347307202 wait complete. 2024-09-10T22:01:24.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 -- start start 2024-09-10T22:01:24.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.826+0000 7fdff066a000 1 Processor -- start 2024-09-10T22:01:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b24e10 0x55a7e8b516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 0x55a7e8b51e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b712f0 0x55a7e8b523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a7e890c5d0 con 0x55a7e8b709d0 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a7e8975d60 con 0x55a7e8b24e10 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55a7e8918500 con 0x55a7e8b712f0 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 10 client.0 _ll_register_callbacks cb 0x55a7e891c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfdd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 0x55a7e8b51e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfdd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 0x55a7e8b51e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40474/0 (socket says 172.21.15.66:40474) 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfdd400640 1 -- 172.21.15.66:0/1735430063 learned_addr learned my addr 172.21.15.66:0/1735430063 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfe5e00640 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b24e10 0x55a7e8b516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfe6800640 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b712f0 0x55a7e8b523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfdd400640 1 -- 172.21.15.66:0/1735430063 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b712f0 msgr2=0x55a7e8b523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfdd400640 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b712f0 0x55a7e8b523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfdd400640 1 -- 172.21.15.66:0/1735430063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b24e10 msgr2=0x55a7e8b516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfdd400640 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b24e10 0x55a7e8b516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfdd400640 1 -- 172.21.15.66:0/1735430063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e8910170 con 0x55a7e8b709d0 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfe6800640 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b712f0 0x55a7e8b523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfdd400640 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 0x55a7e8b51e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0094310 tx=0x7fdfd005b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8b6598eb4b6661 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfcb400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfcb400640 1 -- 172.21.15.66:0/1735430063 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd005a990 con 0x55a7e8b709d0 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfcb400640 1 -- 172.21.15.66:0/1735430063 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdfd008cd30 con 0x55a7e8b709d0 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 -- 172.21.15.66:0/1735430063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a7e890a0f0 con 0x55a7e8b709d0 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfcb400640 1 -- 172.21.15.66:0/1735430063 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fdfd0092170 con 0x55a7e8b709d0 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 10 client.5079 fetch_fsmap learned FSMap version 17 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 -- 172.21.15.66:0/1735430063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55a7e89f6ef0 con 0x55a7e8b709d0 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfcb400640 1 -- 172.21.15.66:0/1735430063 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a7e890a0f0 con 0x55a7e8b709d0 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfcb400640 1 -- 172.21.15.66:0/1735430063 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fdfd005ab30 con 0x55a7e8b709d0 2024-09-10T22:01:24.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 10 client.5079 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T22:01:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 10 client.5079 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T22:01:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 20 client.5079 populate_metadata read hostname 'smithi066' 2024-09-10T22:01:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.66:6833/297228676,v1:172.21.15.66:6835/297228676] conn(0x55a7e8b530f0 0x55a7e8b82c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 4 client.5079 mds_command: new command op to 4908 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:01:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 -- 172.21.15.66:0/1735430063 --> [v2:172.21.15.66:6833/297228676,v1:172.21.15.66:6835/297228676] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7e8984a70 con 0x55a7e8b530f0 2024-09-10T22:01:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfe5e00640 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.66:6833/297228676,v1:172.21.15.66:6835/297228676] conn(0x55a7e8b530f0 0x55a7e8b82c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfe5e00640 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.66:6833/297228676,v1:172.21.15.66:6835/297228676] conn(0x55a7e8b530f0 0x55a7e8b82c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541a1739a970cee server_cookie=36872ef9701dfbda in_seq=0 out_seq=0 2024-09-10T22:01:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfcb400640 10 client.5079 ms_handle_connect on v2:172.21.15.66:6833/297228676 2024-09-10T22:01:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfcb400640 1 -- 172.21.15.66:0/1735430063 <== mds.0 v2:172.21.15.66:6833/297228676 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a7e8984a70 con 0x55a7e8b530f0 2024-09-10T22:01:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfcb400640 10 client.5079 handle_command_reply: tid=0 2024-09-10T22:01:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 client.5079 shutdown 2024-09-10T22:01:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 -- 172.21.15.66:0/1735430063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 msgr2=0x55a7e8b51e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 0x55a7e8b51e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0094310 tx=0x7fdfd005b690 comp rx=0 tx=0).stop 2024-09-10T22:01:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 -- 172.21.15.66:0/1735430063 shutdown_connections 2024-09-10T22:01:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.66:6833/297228676,v1:172.21.15.66:6835/297228676] conn(0x55a7e8b530f0 0x55a7e8b82c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b712f0 0x55a7e8b523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 0x55a7e8b51e70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 --2- 172.21.15.66:0/1735430063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b24e10 0x55a7e8b516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdfcb400640 0 client.5079 ms_handle_reset on v2:172.21.15.66:6833/297228676 2024-09-10T22:01:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.830+0000 7fdff066a000 1 -- 172.21.15.66:0/1735430063 >> 172.21.15.66:0/1735430063 conn(0x55a7e8a0e130 msgr2=0x55a7e8b23fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.834+0000 7fdff066a000 1 -- 172.21.15.66:0/1735430063 shutdown_connections 2024-09-10T22:01:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.834+0000 7fdff066a000 1 -- 172.21.15.66:0/1735430063 wait complete. 2024-09-10T22:01:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.834+0000 7fdff066a000 20 client.5079 trim_cache size 0 max 16384 2024-09-10T22:01:24.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.850+0000 7fdff066a000 1 Processor -- start 2024-09-10T22:01:24.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.850+0000 7fdff066a000 1 -- start start 2024-09-10T22:01:24.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.850+0000 7fdff066a000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b24e10 0x55a7e8be2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.850+0000 7fdff066a000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 0x55a7e8be3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.850+0000 7fdff066a000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b712f0 0x55a7e8be3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.850+0000 7fdff066a000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8203630 con 0x55a7e8b709d0 2024-09-10T22:01:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.850+0000 7fdff066a000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81fb700 con 0x55a7e8b24e10 2024-09-10T22:01:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.850+0000 7fdff066a000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe8203040 con 0x55a7e8b712f0 2024-09-10T22:01:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.850+0000 7fdfdd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 0x55a7e8be3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.850+0000 7fdfdd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 0x55a7e8be3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40488/0 (socket says 172.21.15.66:40488) 2024-09-10T22:01:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.850+0000 7fdfdd400640 1 -- 172.21.15.66:0/3411070050 learned_addr learned my addr 172.21.15.66:0/3411070050 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfe6800640 1 --2- 172.21.15.66:0/3411070050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b712f0 0x55a7e8be3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfe5e00640 1 --2- 172.21.15.66:0/3411070050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b24e10 0x55a7e8be2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfdd400640 1 -- 172.21.15.66:0/3411070050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b712f0 msgr2=0x55a7e8be3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfdd400640 1 --2- 172.21.15.66:0/3411070050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b712f0 0x55a7e8be3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfdd400640 1 -- 172.21.15.66:0/3411070050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b24e10 msgr2=0x55a7e8be2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:24.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfdd400640 1 --2- 172.21.15.66:0/3411070050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b24e10 0x55a7e8be2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfdd400640 1 -- 172.21.15.66:0/3411070050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e89cdd90 con 0x55a7e8b709d0 2024-09-10T22:01:24.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfdd400640 1 --2- 172.21.15.66:0/3411070050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 0x55a7e8be3520 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdfd005ed40 tx=0x7fdfd005e1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cacc29f518891bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:24.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfd6000640 1 -- 172.21.15.66:0/3411070050 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd005dd10 con 0x55a7e8b709d0 2024-09-10T22:01:24.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfd6000640 1 -- 172.21.15.66:0/3411070050 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfd008b170 con 0x55a7e8b709d0 2024-09-10T22:01:24.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfd6000640 1 -- 172.21.15.66:0/3411070050 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd005a990 con 0x55a7e8b709d0 2024-09-10T22:01:24.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 -- 172.21.15.66:0/3411070050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 msgr2=0x55a7e8be3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 --2- 172.21.15.66:0/3411070050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 0x55a7e8be3520 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdfd005ed40 tx=0x7fdfd005e1f0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 -- 172.21.15.66:0/3411070050 shutdown_connections 2024-09-10T22:01:24.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 --2- 172.21.15.66:0/3411070050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b712f0 0x55a7e8be3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 --2- 172.21.15.66:0/3411070050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b709d0 0x55a7e8be3520 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 --2- 172.21.15.66:0/3411070050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b24e10 0x55a7e8be2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 -- 172.21.15.66:0/3411070050 >> 172.21.15.66:0/3411070050 conn(0x55a7e8a0e130 msgr2=0x55a7e8b2ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:24.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 -- 172.21.15.66:0/3411070050 shutdown_connections 2024-09-10T22:01:24.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 -- 172.21.15.66:0/3411070050 wait complete. 2024-09-10T22:01:24.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 -- start start 2024-09-10T22:01:24.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 Processor -- start 2024-09-10T22:01:24.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b24e10 0x55a7e8b84fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b709d0 0x55a7e8b85510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b712f0 0x55a7e8b85a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a7e890c5d0 con 0x55a7e8b24e10 2024-09-10T22:01:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a7e893dec0 con 0x55a7e8b712f0 2024-09-10T22:01:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55a7e8975d60 con 0x55a7e8b709d0 2024-09-10T22:01:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdff066a000 10 client.0 _ll_register_callbacks cb 0x7fdfd005e840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:01:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfe5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b24e10 0x55a7e8b84fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfe5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b24e10 0x55a7e8b84fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40502/0 (socket says 172.21.15.66:40502) 2024-09-10T22:01:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfe5e00640 1 -- 172.21.15.66:0/247466405 learned_addr learned my addr 172.21.15.66:0/247466405 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfe6800640 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b712f0 0x55a7e8b85a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.854+0000 7fdfdd400640 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b709d0 0x55a7e8b85510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfe5e00640 1 -- 172.21.15.66:0/247466405 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b709d0 msgr2=0x55a7e8b85510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfe5e00640 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b709d0 0x55a7e8b85510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfe5e00640 1 -- 172.21.15.66:0/247466405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b712f0 msgr2=0x55a7e8b85a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:24.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfe5e00640 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b712f0 0x55a7e8b85a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfe5e00640 1 -- 172.21.15.66:0/247466405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e8938a90 con 0x55a7e8b24e10 2024-09-10T22:01:24.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfe6800640 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b712f0 0x55a7e8b85a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:24.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfe5e00640 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b24e10 0x55a7e8b84fd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8006700 tx=0x7fdfe806ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97fa2584efbb715 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:24.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfcb400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:01:24.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfcb400640 1 -- 172.21.15.66:0/247466405 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd802d040 con 0x55a7e8b24e10 2024-09-10T22:01:24.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfcb400640 1 -- 172.21.15.66:0/247466405 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdfd8034b30 con 0x55a7e8b24e10 2024-09-10T22:01:24.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 -- 172.21.15.66:0/247466405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd008b170 con 0x55a7e8b24e10 2024-09-10T22:01:24.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfcb400640 1 -- 172.21.15.66:0/247466405 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fdfd8016070 con 0x55a7e8b24e10 2024-09-10T22:01:24.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 10 client.5085 fetch_fsmap learned FSMap version 17 2024-09-10T22:01:24.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 -- 172.21.15.66:0/247466405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55a7e8917050 con 0x55a7e8b24e10 2024-09-10T22:01:24.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfcb400640 1 -- 172.21.15.66:0/247466405 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfd008b170 con 0x55a7e8b24e10 2024-09-10T22:01:24.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfcb400640 1 -- 172.21.15.66:0/247466405 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fdfd80054a0 con 0x55a7e8b24e10 2024-09-10T22:01:24.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 10 client.5085 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T22:01:24.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 10 client.5085 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T22:01:24.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 20 client.5085 populate_metadata read hostname 'smithi066' 2024-09-10T22:01:24.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.66:6833/297228676,v1:172.21.15.66:6835/297228676] conn(0x55a7e8bed050 0x55a7e8bef440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 4 client.5085 mds_command: new command op to 4908 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:01:24.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 -- 172.21.15.66:0/247466405 --> [v2:172.21.15.66:6833/297228676,v1:172.21.15.66:6835/297228676] -- command(tid 0: {"prefix": "status"}) -- 0x55a7e8980520 con 0x55a7e8bed050 2024-09-10T22:01:24.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfdd400640 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.66:6833/297228676,v1:172.21.15.66:6835/297228676] conn(0x55a7e8bed050 0x55a7e8bef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfcb400640 10 client.5085 ms_handle_connect on v2:172.21.15.66:6833/297228676 2024-09-10T22:01:24.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfdd400640 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.66:6833/297228676,v1:172.21.15.66:6835/297228676] conn(0x55a7e8bed050 0x55a7e8bef440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b1fd506616e4d1 server_cookie=fa6c4272494f9f9b in_seq=0 out_seq=0 2024-09-10T22:01:24.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfcb400640 1 -- 172.21.15.66:0/247466405 <== mds.0 v2:172.21.15.66:6833/297228676 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a7e8980520 con 0x55a7e8bed050 2024-09-10T22:01:24.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfcb400640 10 client.5085 handle_command_reply: tid=0 2024-09-10T22:01:24.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 client.5085 shutdown 2024-09-10T22:01:24.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 -- 172.21.15.66:0/247466405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b24e10 msgr2=0x55a7e8b84fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b24e10 0x55a7e8b84fd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8006700 tx=0x7fdfe806ff20 comp rx=0 tx=0).stop 2024-09-10T22:01:24.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 -- 172.21.15.66:0/247466405 shutdown_connections 2024-09-10T22:01:24.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.66:6833/297228676,v1:172.21.15.66:6835/297228676] conn(0x55a7e8bed050 0x55a7e8bef440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e8b712f0 0x55a7e8b85a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a7e8b709d0 0x55a7e8b85510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 --2- 172.21.15.66:0/247466405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7e8b24e10 0x55a7e8b84fd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfcb400640 0 client.5085 ms_handle_reset on v2:172.21.15.66:6833/297228676 2024-09-10T22:01:24.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 -- 172.21.15.66:0/247466405 >> 172.21.15.66:0/247466405 conn(0x55a7e8a0e130 msgr2=0x55a7e8b6f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:24.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 -- 172.21.15.66:0/247466405 shutdown_connections 2024-09-10T22:01:24.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 1 -- 172.21.15.66:0/247466405 wait complete. 2024-09-10T22:01:24.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdff066a000 20 client.5085 trim_cache size 0 max 16384 2024-09-10T22:01:24.870 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:01:24.870 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:01:24.870 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:01:24.870 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4908, 2024-09-10T22:01:24.870 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:01:24.870 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:01:24.870 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:01:24.870 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 6.0005793809999997, 2024-09-10T22:01:24.871 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 17, 2024-09-10T22:01:24.871 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 33, 2024-09-10T22:01:24.871 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 33, 2024-09-10T22:01:24.871 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 12.870734304000001 2024-09-10T22:01:24.871 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:01:24.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfed200640 1 -- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdfc40725e0 msgr2=0x7fdfc4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfed200640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdfc40725e0 0x7fdfc4074aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8001230 tx=0x7fdfd8001e80 comp rx=0 tx=0).stop 2024-09-10T22:01:24.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfed200640 1 -- 172.21.15.66:0/3382406361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfe813d220 msgr2=0x7fdfe81635a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfed200640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfe813d220 0x7fdfe81635a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0011b10 tx=0x7fdfd0009e50 comp rx=0 tx=0).stop 2024-09-10T22:01:24.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfed200640 1 -- 172.21.15.66:0/3382406361 shutdown_connections 2024-09-10T22:01:24.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfed200640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdfc40725e0 0x7fdfc4074aa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfed200640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfe813d6a0 0x7fdfe8164460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfed200640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfe813d220 0x7fdfe81635a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfed200640 1 --2- 172.21.15.66:0/3382406361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdfcc0061a0 0x7fdfe80b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfed200640 1 -- 172.21.15.66:0/3382406361 >> 172.21.15.66:0/3382406361 conn(0x7fdfe80cfa30 msgr2=0x7fdfe80af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:24.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfed200640 1 -- 172.21.15.66:0/3382406361 shutdown_connections 2024-09-10T22:01:24.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:24.858+0000 7fdfed200640 1 -- 172.21.15.66:0/3382406361 wait complete. 2024-09-10T22:01:24.883 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:01:25.000 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:01:25.003 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:01:25.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.026+0000 7f6e41200640 1 -- 172.21.15.66:0/1095377835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e3c0cf830 msgr2=0x7f6e3c0d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:25.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.026+0000 7f6e41200640 1 --2- 172.21.15.66:0/1095377835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e3c0cf830 0x7f6e3c0d04e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009900 tx=0x7f6e2401c850 comp rx=0 tx=0).stop 2024-09-10T22:01:25.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.026+0000 7f6e41200640 1 -- 172.21.15.66:0/1095377835 shutdown_connections 2024-09-10T22:01:25.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.026+0000 7f6e41200640 1 --2- 172.21.15.66:0/1095377835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e3c0cf830 0x7f6e3c0d04e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.026+0000 7f6e41200640 1 --2- 172.21.15.66:0/1095377835 >> v2:172.21.15.94:3300/0 conn(0x7f6e3c15bd40 0x7f6e3c0cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.026+0000 7f6e41200640 1 -- 172.21.15.66:0/1095377835 >> 172.21.15.66:0/1095377835 conn(0x7f6e3c0d1c60 msgr2=0x7f6e3c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:25.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.026+0000 7f6e41200640 1 -- 172.21.15.66:0/1095377835 shutdown_connections 2024-09-10T22:01:25.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.026+0000 7f6e41200640 1 -- 172.21.15.66:0/1095377835 wait complete. 2024-09-10T22:01:25.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e41200640 1 Processor -- start 2024-09-10T22:01:25.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e41200640 1 -- start start 2024-09-10T22:01:25.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e41200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c0aec10 0x7f6e3c14f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:25.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e41200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e3c0cf830 0x7f6e3c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:25.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e41200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e3c15bd40 0x7f6e3c071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:25.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e41200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c1690c0 con 0x7f6e3c15bd40 2024-09-10T22:01:25.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e41200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c168f40 con 0x7f6e3c0aec10 2024-09-10T22:01:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e41200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3c169240 con 0x7f6e3c0cf830 2024-09-10T22:01:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e3a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e3c15bd40 0x7f6e3c071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e3a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e3c15bd40 0x7f6e3c071e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40504/0 (socket says 172.21.15.66:40504) 2024-09-10T22:01:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e3a800640 1 -- 172.21.15.66:0/2400035804 learned_addr learned my addr 172.21.15.66:0/2400035804 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e39e00640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c0aec10 0x7f6e3c14f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e31400640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e3c0cf830 0x7f6e3c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e3a800640 1 -- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e3c0cf830 msgr2=0x7f6e3c071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e3a800640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e3c0cf830 0x7f6e3c071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e3a800640 1 -- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c0aec10 msgr2=0x7f6e3c14f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e3a800640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c0aec10 0x7f6e3c14f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e3a800640 1 -- 172.21.15.66:0/2400035804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c1f6050 con 0x7f6e3c15bd40 2024-09-10T22:01:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e39e00640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c0aec10 0x7f6e3c14f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:01:25.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e3a800640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e3c15bd40 0x7f6e3c071e80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400c7c0 tx=0x7f6e3400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:25.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e32a00640 1 -- 172.21.15.66:0/2400035804 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34003b80 con 0x7f6e3c15bd40 2024-09-10T22:01:25.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e32a00640 1 -- 172.21.15.66:0/2400035804 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e34003d20 con 0x7f6e3c15bd40 2024-09-10T22:01:25.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e32a00640 1 -- 172.21.15.66:0/2400035804 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3400aab0 con 0x7f6e3c15bd40 2024-09-10T22:01:25.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e41200640 1 -- 172.21.15.66:0/2400035804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c1f6330 con 0x7f6e3c15bd40 2024-09-10T22:01:25.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.030+0000 7f6e41200640 1 -- 172.21.15.66:0/2400035804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c1f68f0 con 0x7f6e3c15bd40 2024-09-10T22:01:25.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.034+0000 7f6e41200640 1 -- 172.21.15.66:0/2400035804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e3c0af010 con 0x7f6e3c15bd40 2024-09-10T22:01:25.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.034+0000 7f6e32a00640 1 -- 172.21.15.66:0/2400035804 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6e3400ac50 con 0x7f6e3c15bd40 2024-09-10T22:01:25.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.034+0000 7f6e32a00640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6e180725e0 0x7f6e18074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:25.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.034+0000 7f6e32a00640 1 -- 172.21.15.66:0/2400035804 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6e34092fe0 con 0x7f6e3c15bd40 2024-09-10T22:01:25.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.034+0000 7f6e39e00640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6e180725e0 0x7f6e18074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:25.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.038+0000 7f6e39e00640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6e180725e0 0x7f6e18074aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c050fa0 tx=0x7f6e2c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:25.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.038+0000 7f6e32a00640 1 -- 172.21.15.66:0/2400035804 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6e3405e060 con 0x7f6e3c15bd40 2024-09-10T22:01:25.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.250+0000 7f6e41200640 1 -- 172.21.15.66:0/2400035804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6e3c0b0df0 con 0x7f6e3c15bd40 2024-09-10T22:01:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.250+0000 7f6e32a00640 1 -- 172.21.15.66:0/2400035804 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f6e34062a90 con 0x7f6e3c15bd40 2024-09-10T22:01:25.255 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:01:25.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.254+0000 7f6e41200640 1 -- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6e180725e0 msgr2=0x7f6e18074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:25.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.254+0000 7f6e41200640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6e180725e0 0x7f6e18074aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c050fa0 tx=0x7f6e2c0091e0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.254+0000 7f6e41200640 1 -- 172.21.15.66:0/2400035804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e3c15bd40 msgr2=0x7f6e3c071e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:25.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.254+0000 7f6e41200640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e3c15bd40 0x7f6e3c071e80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400c7c0 tx=0x7f6e3400cc80 comp rx=0 tx=0).stop 2024-09-10T22:01:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.254+0000 7f6e41200640 1 -- 172.21.15.66:0/2400035804 shutdown_connections 2024-09-10T22:01:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.254+0000 7f6e41200640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6e180725e0 0x7f6e18074aa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.254+0000 7f6e41200640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e3c15bd40 0x7f6e3c071e80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.254+0000 7f6e41200640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e3c0cf830 0x7f6e3c071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.254+0000 7f6e41200640 1 --2- 172.21.15.66:0/2400035804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c0aec10 0x7f6e3c14f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.254+0000 7f6e41200640 1 -- 172.21.15.66:0/2400035804 >> 172.21.15.66:0/2400035804 conn(0x7f6e3c0d1c60 msgr2=0x7f6e3c1554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.254+0000 7f6e41200640 1 -- 172.21.15.66:0/2400035804 shutdown_connections 2024-09-10T22:01:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25.254+0000 7f6e41200640 1 -- 172.21.15.66:0/2400035804 wait complete. 2024-09-10T22:01:25.276 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:01:25.276 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:01:25.276 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:01:25.276 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:01:25.276 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:01:25.276 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:01:25.277 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:01:25.277 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:01:25.277 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:01:25.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:25.277 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:01:25.286 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:01:25.286 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:01:25.286 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:01:25.286 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:01:25.286 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:01:25.286 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:01:25.286 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:01:25.286 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:01:25.286 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42389sec preferred_lft 42389sec 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:01:25.287 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:81:fc:4c:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:01:25.288 INFO:teuthology.orchestra.run.smithi066.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:01:25.289 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:01:25.289 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T22:01:25.289 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:01:25.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:25.290 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:01:25.350 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:01:25.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:25.352 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:01:25.407 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:01:25.407 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:01:25.407 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:01:25.408 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:01:25.408 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:01:25.408 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:01:25.408 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:01:25.408 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:01:25.408 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:01:25.408 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:01:25.408 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:01:25.408 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:01:25.409 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:01:25.409 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:01:25.409 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:01:25.409 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:01:25.409 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:01:25.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:01:25.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:01:25.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:25.543 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:01:25.552 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:01:25.552 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:01:25.553 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:01:25.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:25.553 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:01:25.620 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T22:01:25.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:01:25.620 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:01:25.672 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:01:25.672 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:01:25.672 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:01:25.723 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:01:25.788 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:01:25.842 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:01:25.842 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:01:25.842 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:01:25.842 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:01:25.842 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:01:25.842 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:01:25.842 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:01:25.842 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:01:25.842 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:01:25.842 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:01:25.842 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:01:25.843 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:01:25.844 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:01:25.844 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:01:25.844 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:01:25.844 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:01:25.844 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:01:25.844 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:01:25.844 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:01:25.844 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:01:25.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:25.845 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:01:25.900 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:01:25.901 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T22:01:25.951 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:01:25.971 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:01:26.026 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:01:26.027 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:01:26.027 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:01:26.027 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:01:26.027 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:01:26.027 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:01:26.027 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:01:26.027 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:01:26.027 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:01:26.027 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:01:26.027 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:01:26.027 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:01:26.028 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:01:26.029 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:01:26.029 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:01:26.029 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:01:26.029 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:01:26.029 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:01:26.029 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:01:26.029 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:01:26.029 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:01:26.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:26.030 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:01:26.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-09-10T22:01:26.046+0000 7faeb819e580 -1 init, newargv = 0x55ef8732e340 newargc=15 2024-09-10T22:01:26.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[37214]: starting ceph client 2024-09-10T22:01:26.056 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[37214]: starting fuse 2024-09-10T22:01:26.057 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-09-10T22:01:26.058 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T22:01:26.058 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:01:26.058 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-09-10T22:01:26.058 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-09-10T22:01:26.058 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:01:26.058 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-09-10T22:01:26.058 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:01:26.058 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-09-10T22:01:26.058 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T22:01:26.058 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-09-10T22:01:26.058 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T22:01:26.058 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-09-10T22:01:26.059 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:01:26.182 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.37214.asok 2024-09-10T22:01:26.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:01:26.194 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37214.asok 2024-09-10T22:01:26.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:26.194 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37214.asok status 2024-09-10T22:01:26.368 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:01:26.369 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-09-10T22:01:26.369 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:01:26.369 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:01:26.369 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-09-10T22:01:26.369 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-09-10T22:01:26.369 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T22:01:26.369 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "37214", 2024-09-10T22:01:26.369 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-09-10T22:01:26.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:26.369 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-09-10T22:01:26.369 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-09-10T22:01:26.369 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5094, 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 5094 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 982849647 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:01:26.370 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:01:26.371 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 982849647 2024-09-10T22:01:26.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:01:26.371 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.5094 192.168.0.1:0/982849647", 2024-09-10T22:01:26.371 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/982849647", 2024-09-10T22:01:26.371 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-09-10T22:01:26.371 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 17, 2024-09-10T22:01:26.371 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 33, 2024-09-10T22:01:26.371 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:01:26.371 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-09-10T22:01:26.371 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-09-10T22:01:26.371 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:01:26.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:26.384 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-09-10T22:01:26.384 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:01:26.394 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-09-10T22:01:26.394 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T22:01:26.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:26.394 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:01:26.460 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T22:01:26.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.650+0000 7fcea1200640 1 -- 172.21.15.66:0/2353858684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c073cd0 msgr2=0x7fce9c0b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:26.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.650+0000 7fcea1200640 1 --2- 172.21.15.66:0/2353858684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c073cd0 0x7fce9c0b3210 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fce94009c80 tx=0x7fce9401cce0 comp rx=0 tx=0).stop 2024-09-10T22:01:26.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.650+0000 7fcea1200640 1 -- 172.21.15.66:0/2353858684 shutdown_connections 2024-09-10T22:01:26.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.650+0000 7fcea1200640 1 --2- 172.21.15.66:0/2353858684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c073cd0 0x7fce9c0b3210 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:26.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.650+0000 7fcea1200640 1 --2- 172.21.15.66:0/2353858684 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce9c154b60 0x7fce9c15be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:26.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.650+0000 7fcea1200640 1 -- 172.21.15.66:0/2353858684 >> 172.21.15.66:0/2353858684 conn(0x7fce9c06cc20 msgr2=0x7fce9c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:26.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.650+0000 7fcea1200640 1 -- 172.21.15.66:0/2353858684 shutdown_connections 2024-09-10T22:01:26.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.650+0000 7fcea1200640 1 -- 172.21.15.66:0/2353858684 wait complete. 2024-09-10T22:01:26.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.650+0000 7fcea1200640 1 Processor -- start 2024-09-10T22:01:26.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.650+0000 7fcea1200640 1 -- start start 2024-09-10T22:01:26.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fcea1200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c073cd0 0x7fce9c0af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:26.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fcea1200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce9c154b60 0x7fce9c14b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:26.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fcea1200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce9c154f40 0x7fce9c0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:26.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fcea1200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c06fc10 con 0x7fce9c073cd0 2024-09-10T22:01:26.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fcea1200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c06fa90 con 0x7fce9c154b60 2024-09-10T22:01:26.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fcea1200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fce9c06fd90 con 0x7fce9c154f40 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce9b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c073cd0 0x7fce9c0af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce9b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c073cd0 0x7fce9c0af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40532/0 (socket says 172.21.15.66:40532) 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce9b400640 1 -- 172.21.15.66:0/4025636102 learned_addr learned my addr 172.21.15.66:0/4025636102 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce9be00640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce9c154f40 0x7fce9c0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce9aa00640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce9c154b60 0x7fce9c14b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce9b400640 1 -- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce9c154f40 msgr2=0x7fce9c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce9b400640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce9c154f40 0x7fce9c0ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce9b400640 1 -- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce9c154b60 msgr2=0x7fce9c14b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce9b400640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce9c154b60 0x7fce9c14b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce9b400640 1 -- 172.21.15.66:0/4025636102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c0b12e0 con 0x7fce9c073cd0 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce9aa00640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce9c154b60 0x7fce9c14b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce9b400640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c073cd0 0x7fce9c0af7f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00e9f0 tx=0x7fce8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce93400640 1 -- 172.21.15.66:0/4025636102 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00cd80 con 0x7fce9c073cd0 2024-09-10T22:01:26.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce93400640 1 -- 172.21.15.66:0/4025636102 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce8c010430 con 0x7fce9c073cd0 2024-09-10T22:01:26.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fce93400640 1 -- 172.21.15.66:0/4025636102 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c010860 con 0x7fce9c073cd0 2024-09-10T22:01:26.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fcea1200640 1 -- 172.21.15.66:0/4025636102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9c0b0240 con 0x7fce9c073cd0 2024-09-10T22:01:26.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fcea1200640 1 -- 172.21.15.66:0/4025636102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9c0b0800 con 0x7fce9c073cd0 2024-09-10T22:01:26.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.654+0000 7fcea1200640 1 -- 172.21.15.66:0/4025636102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce9c070260 con 0x7fce9c073cd0 2024-09-10T22:01:26.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.658+0000 7fce93400640 1 -- 172.21.15.66:0/4025636102 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fce8c010a00 con 0x7fce9c073cd0 2024-09-10T22:01:26.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.658+0000 7fce93400640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce7c0725e0 0x7fce7c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:26.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.658+0000 7fce93400640 1 -- 172.21.15.66:0/4025636102 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fce8c093d80 con 0x7fce9c073cd0 2024-09-10T22:01:26.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.658+0000 7fce9aa00640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce7c0725e0 0x7fce7c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:26.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.658+0000 7fce9aa00640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce7c0725e0 0x7fce7c074aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce9c155380 tx=0x7fce88009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:26.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.662+0000 7fce93400640 1 -- 172.21.15.66:0/4025636102 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fce8c05eec0 con 0x7fce9c073cd0 2024-09-10T22:01:26.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.874+0000 7fcea1200640 1 -- 172.21.15.66:0/4025636102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fce9c15c150 con 0x7fce9c073cd0 2024-09-10T22:01:26.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.894+0000 7fce93400640 1 -- 172.21.15.66:0/4025636102 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7fce8c0638f0 con 0x7fce9c073cd0 2024-09-10T22:01:26.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.898+0000 7fcea1200640 1 -- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce7c0725e0 msgr2=0x7fce7c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:26.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.898+0000 7fcea1200640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce7c0725e0 0x7fce7c074aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce9c155380 tx=0x7fce88009040 comp rx=0 tx=0).stop 2024-09-10T22:01:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.898+0000 7fcea1200640 1 -- 172.21.15.66:0/4025636102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c073cd0 msgr2=0x7fce9c0af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.898+0000 7fcea1200640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c073cd0 0x7fce9c0af7f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00e9f0 tx=0x7fce8c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:01:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.898+0000 7fcea1200640 1 -- 172.21.15.66:0/4025636102 shutdown_connections 2024-09-10T22:01:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.898+0000 7fcea1200640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce7c0725e0 0x7fce7c074aa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.898+0000 7fcea1200640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce9c154f40 0x7fce9c0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.898+0000 7fcea1200640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce9c154b60 0x7fce9c14b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.898+0000 7fcea1200640 1 --2- 172.21.15.66:0/4025636102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c073cd0 0x7fce9c0af7f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.898+0000 7fcea1200640 1 -- 172.21.15.66:0/4025636102 >> 172.21.15.66:0/4025636102 conn(0x7fce9c06cc20 msgr2=0x7fce9c14bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.898+0000 7fcea1200640 1 -- 172.21.15.66:0/4025636102 shutdown_connections 2024-09-10T22:01:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:26.898+0000 7fcea1200640 1 -- 172.21.15.66:0/4025636102 wait complete. 2024-09-10T22:01:26.921 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:01:27.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 -- 172.21.15.66:0/3429258796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f304815a310 msgr2=0x7f30480c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 --2- 172.21.15.66:0/3429258796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f304815a310 0x7f30480c48b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f30480cf9e0 tx=0x7f303801db60 comp rx=0 tx=0).stop 2024-09-10T22:01:27.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 -- 172.21.15.66:0/3429258796 shutdown_connections 2024-09-10T22:01:27.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 --2- 172.21.15.66:0/3429258796 >> v2:172.21.15.94:3300/0 conn(0x7f30480d0550 0x7f30480c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 --2- 172.21.15.66:0/3429258796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f304815a310 0x7f30480c48b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 -- 172.21.15.66:0/3429258796 >> 172.21.15.66:0/3429258796 conn(0x7f30480cfa30 msgr2=0x7f30480cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:27.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 -- 172.21.15.66:0/3429258796 shutdown_connections 2024-09-10T22:01:27.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 -- 172.21.15.66:0/3429258796 wait complete. 2024-09-10T22:01:27.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 Processor -- start 2024-09-10T22:01:27.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 -- start start 2024-09-10T22:01:27.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30480aeec0 0x7f304813e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304815a310 0x7f304813ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f304815dac0 0x7f30481f9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f304815edb0 con 0x7f30480aeec0 2024-09-10T22:01:27.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f304815ec30 con 0x7f304815a310 2024-09-10T22:01:27.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.066+0000 7f304e600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3048167300 con 0x7f304815dac0 2024-09-10T22:01:27.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f3047400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30480aeec0 0x7f304813e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f3047400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30480aeec0 0x7f304813e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40556/0 (socket says 172.21.15.66:40556) 2024-09-10T22:01:27.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f3047400640 1 -- 172.21.15.66:0/327479387 learned_addr learned my addr 172.21.15.66:0/327479387 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:27.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f303fe00640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304815a310 0x7f304813ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f3047e00640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f304815dac0 0x7f30481f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f3047400640 1 -- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f304815dac0 msgr2=0x7f30481f9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f3047400640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f304815dac0 0x7f30481f9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f3047400640 1 -- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304815a310 msgr2=0x7f304813ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f3047400640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304815a310 0x7f304813ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f3047400640 1 -- 172.21.15.66:0/327479387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304815c7a0 con 0x7f30480aeec0 2024-09-10T22:01:27.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f3047e00640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f304815dac0 0x7f30481f9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:27.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f3047400640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30480aeec0 0x7f304813e6f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f30380b1040 tx=0x7f3038004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:27.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f303f400640 1 -- 172.21.15.66:0/327479387 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30380b3070 con 0x7f30480aeec0 2024-09-10T22:01:27.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f303f400640 1 -- 172.21.15.66:0/327479387 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3038005530 con 0x7f30480aeec0 2024-09-10T22:01:27.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f303f400640 1 -- 172.21.15.66:0/327479387 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3038004930 con 0x7f30480aeec0 2024-09-10T22:01:27.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f304e600640 1 -- 172.21.15.66:0/327479387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304815ca40 con 0x7f30480aeec0 2024-09-10T22:01:27.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f304e600640 1 -- 172.21.15.66:0/327479387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304815d000 con 0x7f30480aeec0 2024-09-10T22:01:27.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f304e600640 1 -- 172.21.15.66:0/327479387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304815df60 con 0x7f30480aeec0 2024-09-10T22:01:27.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.070+0000 7f303f400640 1 -- 172.21.15.66:0/327479387 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f303800b040 con 0x7f30480aeec0 2024-09-10T22:01:27.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.074+0000 7f303f400640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3024072610 0x7f3024074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.074+0000 7f303f400640 1 -- 172.21.15.66:0/327479387 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f303812d340 con 0x7f30480aeec0 2024-09-10T22:01:27.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.074+0000 7f303fe00640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3024072610 0x7f3024074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.074+0000 7f303fe00640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3024072610 0x7f3024074ad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f304814d0f0 tx=0x7f303000b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:27.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.078+0000 7f303f400640 1 -- 172.21.15.66:0/327479387 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f30380177e0 con 0x7f30480aeec0 2024-09-10T22:01:27.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.270+0000 7f304e600640 1 -- 172.21.15.66:0/327479387 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f304814d690 con 0x7f3024072610 2024-09-10T22:01:27.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f303f400640 1 -- 172.21.15.66:0/327479387 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f304814d690 con 0x7f3024072610 2024-09-10T22:01:27.278 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:01:27.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:01:27.278 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:01:27.278 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:01:27.278 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:01:27.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f304e600640 1 -- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3024072610 msgr2=0x7f3024074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f304e600640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3024072610 0x7f3024074ad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f304814d0f0 tx=0x7f303000b390 comp rx=0 tx=0).stop 2024-09-10T22:01:27.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f304e600640 1 -- 172.21.15.66:0/327479387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30480aeec0 msgr2=0x7f304813e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f304e600640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30480aeec0 0x7f304813e6f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f30380b1040 tx=0x7f3038004600 comp rx=0 tx=0).stop 2024-09-10T22:01:27.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f304e600640 1 -- 172.21.15.66:0/327479387 shutdown_connections 2024-09-10T22:01:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f304e600640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3024072610 0x7f3024074ad0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f304e600640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f304815dac0 0x7f30481f9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f304e600640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304815a310 0x7f304813ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f304e600640 1 --2- 172.21.15.66:0/327479387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30480aeec0 0x7f304813e6f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f304e600640 1 -- 172.21.15.66:0/327479387 >> 172.21.15.66:0/327479387 conn(0x7f30480cfa30 msgr2=0x7f30480b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f304e600640 1 -- 172.21.15.66:0/327479387 shutdown_connections 2024-09-10T22:01:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.274+0000 7f304e600640 1 -- 172.21.15.66:0/327479387 wait complete. 2024-09-10T22:01:27.300 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:01:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 -- 172.21.15.66:0/1606041097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0460160280 msgr2=0x7f04600b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 --2- 172.21.15.66:0/1606041097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0460160280 0x7f04600b3170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f04600b3080 tx=0x7f044c01d240 comp rx=0 tx=0).stop 2024-09-10T22:01:27.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 -- 172.21.15.66:0/1606041097 shutdown_connections 2024-09-10T22:01:27.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 --2- 172.21.15.66:0/1606041097 >> v2:172.21.15.94:3300/0 conn(0x7f04600b1e10 0x7f04600aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 --2- 172.21.15.66:0/1606041097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0460160280 0x7f04600b3170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 -- 172.21.15.66:0/1606041097 >> 172.21.15.66:0/1606041097 conn(0x7f04600d1c60 msgr2=0x7f04600d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:27.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 -- 172.21.15.66:0/1606041097 shutdown_connections 2024-09-10T22:01:27.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 -- 172.21.15.66:0/1606041097 wait complete. 2024-09-10T22:01:27.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 Processor -- start 2024-09-10T22:01:27.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 -- start start 2024-09-10T22:01:27.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04600b1e10 0x7f0460150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f046015f450 0x7f04600c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0460160280 0x7f04601545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04600aff60 con 0x7f046015f450 2024-09-10T22:01:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04600afde0 con 0x7f04600b1e10 2024-09-10T22:01:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0468000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0460168eb0 con 0x7f0460160280 2024-09-10T22:01:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0464c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f046015f450 0x7f04600c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0464c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f046015f450 0x7f04600c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40576/0 (socket says 172.21.15.66:40576) 2024-09-10T22:01:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0464c00640 1 -- 172.21.15.66:0/2101688790 learned_addr learned my addr 172.21.15.66:0/2101688790 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0465600640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04600b1e10 0x7f0460150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.446+0000 7f0466000640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0460160280 0x7f04601545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f0464c00640 1 -- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0460160280 msgr2=0x7f04601545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f0464c00640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0460160280 0x7f04601545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f0464c00640 1 -- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04600b1e10 msgr2=0x7f0460150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f0464c00640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04600b1e10 0x7f0460150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f0464c00640 1 -- 172.21.15.66:0/2101688790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04600c5f60 con 0x7f046015f450 2024-09-10T22:01:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f0465600640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04600b1e10 0x7f0460150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f0464c00640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f046015f450 0x7f04600c61a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f044c0b0040 tx=0x7f044c0a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f045d600640 1 -- 172.21.15.66:0/2101688790 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c0b2040 con 0x7f046015f450 2024-09-10T22:01:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f045d600640 1 -- 172.21.15.66:0/2101688790 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f044c004430 con 0x7f046015f450 2024-09-10T22:01:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f045d600640 1 -- 172.21.15.66:0/2101688790 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c005280 con 0x7f046015f450 2024-09-10T22:01:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f0468000640 1 -- 172.21.15.66:0/2101688790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04600c59a0 con 0x7f046015f450 2024-09-10T22:01:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f0468000640 1 -- 172.21.15.66:0/2101688790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0460153dc0 con 0x7f046015f450 2024-09-10T22:01:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f0468000640 1 -- 172.21.15.66:0/2101688790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04600aec10 con 0x7f046015f450 2024-09-10T22:01:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f045d600640 1 -- 172.21.15.66:0/2101688790 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f044c0c0060 con 0x7f046015f450 2024-09-10T22:01:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f045d600640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04400725e0 0x7f0440074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f045d600640 1 -- 172.21.15.66:0/2101688790 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f044c12cbf0 con 0x7f046015f450 2024-09-10T22:01:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.450+0000 7f0465600640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04400725e0 0x7f0440074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.454+0000 7f0465600640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04400725e0 0x7f0440074aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0454000f70 tx=0x7f0454009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:27.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.454+0000 7f045d600640 1 -- 172.21.15.66:0/2101688790 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f044c016490 con 0x7f046015f450 2024-09-10T22:01:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.626+0000 7f0468000640 1 -- 172.21.15.66:0/2101688790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f04600d2a40 con 0x7f046015f450 2024-09-10T22:01:27.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f045d600640 1 -- 172.21.15.66:0/2101688790 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f044c0f8230 con 0x7f046015f450 2024-09-10T22:01:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f0468000640 1 -- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04400725e0 msgr2=0x7f0440074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f0468000640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04400725e0 0x7f0440074aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0454000f70 tx=0x7f0454009040 comp rx=0 tx=0).stop 2024-09-10T22:01:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f0468000640 1 -- 172.21.15.66:0/2101688790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f046015f450 msgr2=0x7f04600c61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f0468000640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f046015f450 0x7f04600c61a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f044c0b0040 tx=0x7f044c0a57d0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f0468000640 1 -- 172.21.15.66:0/2101688790 shutdown_connections 2024-09-10T22:01:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f0468000640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04400725e0 0x7f0440074aa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f0468000640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0460160280 0x7f04601545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f0468000640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f046015f450 0x7f04600c61a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f0468000640 1 --2- 172.21.15.66:0/2101688790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04600b1e10 0x7f0460150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f0468000640 1 -- 172.21.15.66:0/2101688790 >> 172.21.15.66:0/2101688790 conn(0x7f04600d1c60 msgr2=0x7f04600ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f0468000640 1 -- 172.21.15.66:0/2101688790 shutdown_connections 2024-09-10T22:01:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.630+0000 7f0468000640 1 -- 172.21.15.66:0/2101688790 wait complete. 2024-09-10T22:01:27.654 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-10T22:01:27.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.798+0000 7feeef200640 1 -- 172.21.15.66:0/2204162934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee80c4fc0 msgr2=0x7feee80ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.798+0000 7feeef200640 1 --2- 172.21.15.66:0/2204162934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee80c4fc0 0x7feee80ca8b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feee0009c80 tx=0x7feee001cd10 comp rx=0 tx=0).stop 2024-09-10T22:01:27.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.798+0000 7feeef200640 1 -- 172.21.15.66:0/2204162934 shutdown_connections 2024-09-10T22:01:27.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.798+0000 7feeef200640 1 --2- 172.21.15.66:0/2204162934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee80c4fc0 0x7feee80ca8b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.798+0000 7feeef200640 1 -- 172.21.15.66:0/2204162934 >> 172.21.15.66:0/2204162934 conn(0x7feee80cfa30 msgr2=0x7feee80cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:27.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.798+0000 7feeef200640 1 -- 172.21.15.66:0/2204162934 shutdown_connections 2024-09-10T22:01:27.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.798+0000 7feeef200640 1 -- 172.21.15.66:0/2204162934 wait complete. 2024-09-10T22:01:27.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.798+0000 7feeef200640 1 Processor -- start 2024-09-10T22:01:27.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.798+0000 7feeef200640 1 -- start start 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feeef200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feee80c4fc0 0x7feee80b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feeef200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee815a310 0x7feee80adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feeef200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee80ceec0 0x7feee81567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feeef200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feee8168eb0 con 0x7feee815a310 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feeef200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feee8168d30 con 0x7feee80ceec0 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feeef200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7feee8169030 con 0x7feee80c4fc0 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feedfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee815a310 0x7feee80adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feedfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee815a310 0x7feee80adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40596/0 (socket says 172.21.15.66:40596) 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feedfe00640 1 -- 172.21.15.66:0/1192239472 learned_addr learned my addr 172.21.15.66:0/1192239472 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feee7e00640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feee80c4fc0 0x7feee80b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feeed200640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee80ceec0 0x7feee81567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feedfe00640 1 -- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feee80c4fc0 msgr2=0x7feee80b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feedfe00640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feee80c4fc0 0x7feee80b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feedfe00640 1 -- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee80ceec0 msgr2=0x7feee81567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feedfe00640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee80ceec0 0x7feee81567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feedfe00640 1 -- 172.21.15.66:0/1192239472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee80cd930 con 0x7feee815a310 2024-09-10T22:01:27.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feedfe00640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee815a310 0x7feee80adb20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feed000c450 tx=0x7feed000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:27.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feee4c00640 1 -- 172.21.15.66:0/1192239472 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0005480 con 0x7feee815a310 2024-09-10T22:01:27.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feee4c00640 1 -- 172.21.15.66:0/1192239472 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feed0003600 con 0x7feee815a310 2024-09-10T22:01:27.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feee4c00640 1 -- 172.21.15.66:0/1192239472 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0003a30 con 0x7feee815a310 2024-09-10T22:01:27.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feeef200640 1 -- 172.21.15.66:0/1192239472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee80b1220 con 0x7feee815a310 2024-09-10T22:01:27.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feeef200640 1 -- 172.21.15.66:0/1192239472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee80cded0 con 0x7feee815a310 2024-09-10T22:01:27.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.802+0000 7feeef200640 1 -- 172.21.15.66:0/1192239472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee80c48b0 con 0x7feee815a310 2024-09-10T22:01:27.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.806+0000 7feee4c00640 1 -- 172.21.15.66:0/1192239472 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feed0003bd0 con 0x7feee815a310 2024-09-10T22:01:27.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.806+0000 7feee4c00640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feec40725e0 0x7feec4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.806+0000 7feee4c00640 1 -- 172.21.15.66:0/1192239472 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7feed0092c40 con 0x7feee815a310 2024-09-10T22:01:27.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.806+0000 7feee7e00640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feec40725e0 0x7feec4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.806+0000 7feee7e00640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feec40725e0 0x7feec4074aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feed8007fd0 tx=0x7feed8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:27.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.810+0000 7feee4c00640 1 -- 172.21.15.66:0/1192239472 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feed005dcc0 con 0x7feee815a310 2024-09-10T22:01:27.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 -- 172.21.15.66:0/1192239472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7feee8158530 con 0x7feee815a310 2024-09-10T22:01:27.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feee4c00640 1 -- 172.21.15.66:0/1192239472 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v28) ==== 116+0+2 (secure 0 0 0) 0x7feed00626f0 con 0x7feee815a310 2024-09-10T22:01:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:4 2024-09-10T22:01:27.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 -- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feec40725e0 msgr2=0x7feec4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feec40725e0 0x7feec4074aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feed8007fd0 tx=0x7feed8009040 comp rx=0 tx=0).stop 2024-09-10T22:01:27.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 -- 172.21.15.66:0/1192239472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee815a310 msgr2=0x7feee80adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee815a310 0x7feee80adb20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feed000c450 tx=0x7feed000c910 comp rx=0 tx=0).stop 2024-09-10T22:01:27.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 -- 172.21.15.66:0/1192239472 shutdown_connections 2024-09-10T22:01:27.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feec40725e0 0x7feec4074aa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee80ceec0 0x7feee81567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee815a310 0x7feee80adb20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 --2- 172.21.15.66:0/1192239472 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feee80c4fc0 0x7feee80b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 -- 172.21.15.66:0/1192239472 >> 172.21.15.66:0/1192239472 conn(0x7feee80cfa30 msgr2=0x7feee80cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:27.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 -- 172.21.15.66:0/1192239472 shutdown_connections 2024-09-10T22:01:27.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:27.986+0000 7feeef200640 1 -- 172.21.15.66:0/1192239472 wait complete. 2024-09-10T22:01:28.010 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:01:28.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.154+0000 7f891a600640 1 -- 172.21.15.66:0/1052036483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89140d0260 msgr2=0x7f89140ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.154+0000 7f891a600640 1 --2- 172.21.15.66:0/1052036483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89140d0260 0x7f89140ae960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009840 tx=0x7f88fc01c820 comp rx=0 tx=0).stop 2024-09-10T22:01:28.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.154+0000 7f891a600640 1 -- 172.21.15.66:0/1052036483 shutdown_connections 2024-09-10T22:01:28.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.154+0000 7f891a600640 1 --2- 172.21.15.66:0/1052036483 >> v2:172.21.15.94:3300/0 conn(0x7f89140b2bf0 0x7f89140aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.154+0000 7f891a600640 1 --2- 172.21.15.66:0/1052036483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89140d0260 0x7f89140ae960 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.154+0000 7f891a600640 1 -- 172.21.15.66:0/1052036483 >> 172.21.15.66:0/1052036483 conn(0x7f89140d1c50 msgr2=0x7f89140d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:28.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.154+0000 7f891a600640 1 -- 172.21.15.66:0/1052036483 shutdown_connections 2024-09-10T22:01:28.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.154+0000 7f891a600640 1 -- 172.21.15.66:0/1052036483 wait complete. 2024-09-10T22:01:28.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f891a600640 1 Processor -- start 2024-09-10T22:01:28.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f891a600640 1 -- start start 2024-09-10T22:01:28.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f891a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89140b2bf0 0x7f891406e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:28.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f891a600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89140d0260 0x7f8914153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:28.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f891a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f891415f340 0x7f89141519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:28.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f891a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8914169240 con 0x7f891415f340 2024-09-10T22:01:28.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f891a600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89141690c0 con 0x7f89140b2bf0 2024-09-10T22:01:28.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f891a600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f89141693c0 con 0x7f89140d0260 2024-09-10T22:01:28.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f8913e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f891415f340 0x7f89141519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:28.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f8913e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f891415f340 0x7f89141519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40604/0 (socket says 172.21.15.66:40604) 2024-09-10T22:01:28.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f8913e00640 1 -- 172.21.15.66:0/256989566 learned_addr learned my addr 172.21.15.66:0/256989566 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:28.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f890be00640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89140d0260 0x7f8914153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:28.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f8913400640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89140b2bf0 0x7f891406e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:28.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f8913e00640 1 -- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89140d0260 msgr2=0x7f8914153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f8913e00640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89140d0260 0x7f8914153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f8913e00640 1 -- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89140b2bf0 msgr2=0x7f891406e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f8913e00640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89140b2bf0 0x7f891406e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f8913e00640 1 -- 172.21.15.66:0/256989566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891406e7e0 con 0x7f891415f340 2024-09-10T22:01:28.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f890be00640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89140d0260 0x7f8914153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:01:28.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f8913e00640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f891415f340 0x7f89141519b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f890c00eb20 tx=0x7f890c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:28.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f890b400640 1 -- 172.21.15.66:0/256989566 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c004b30 con 0x7f891415f340 2024-09-10T22:01:28.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f890b400640 1 -- 172.21.15.66:0/256989566 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f890c004cf0 con 0x7f891415f340 2024-09-10T22:01:28.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f891a600640 1 -- 172.21.15.66:0/256989566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891414fe50 con 0x7f891415f340 2024-09-10T22:01:28.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f890b400640 1 -- 172.21.15.66:0/256989566 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c005280 con 0x7f891415f340 2024-09-10T22:01:28.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f891a600640 1 -- 172.21.15.66:0/256989566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891406ea80 con 0x7f891415f340 2024-09-10T22:01:28.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.158+0000 7f891a600640 1 -- 172.21.15.66:0/256989566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89140cfb50 con 0x7f891415f340 2024-09-10T22:01:28.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.162+0000 7f890b400640 1 -- 172.21.15.66:0/256989566 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f890c005420 con 0x7f891415f340 2024-09-10T22:01:28.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.162+0000 7f890b400640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f88f0072610 0x7f88f0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:28.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.162+0000 7f890b400640 1 -- 172.21.15.66:0/256989566 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f890c093e90 con 0x7f891415f340 2024-09-10T22:01:28.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.162+0000 7f8913400640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f88f0072610 0x7f88f0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:28.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.162+0000 7f8913400640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f88f0072610 0x7f88f0074ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8904007fd0 tx=0x7f8904007ea0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.166+0000 7f890b400640 1 -- 172.21.15.66:0/256989566 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f890c05ee00 con 0x7f891415f340 2024-09-10T22:01:28.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.342+0000 7f891a600640 1 -- 172.21.15.66:0/256989566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f89140afe00 con 0x7f891415f340 2024-09-10T22:01:28.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.342+0000 7f890b400640 1 -- 172.21.15.66:0/256989566 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v29)=0 v29) ==== 131+0+0 (secure 0 0 0) 0x7f890c063830 con 0x7f891415f340 2024-09-10T22:01:28.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.346+0000 7f891a600640 1 -- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f88f0072610 msgr2=0x7f88f0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.346+0000 7f891a600640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f88f0072610 0x7f88f0074ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8904007fd0 tx=0x7f8904007ea0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.346+0000 7f891a600640 1 -- 172.21.15.66:0/256989566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f891415f340 msgr2=0x7f89141519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.346+0000 7f891a600640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f891415f340 0x7f89141519b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f890c00eb20 tx=0x7f890c00c2d0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.346+0000 7f891a600640 1 -- 172.21.15.66:0/256989566 shutdown_connections 2024-09-10T22:01:28.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.346+0000 7f891a600640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f88f0072610 0x7f88f0074ad0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.346+0000 7f891a600640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f891415f340 0x7f89141519b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.346+0000 7f891a600640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89140d0260 0x7f8914153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.346+0000 7f891a600640 1 --2- 172.21.15.66:0/256989566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89140b2bf0 0x7f891406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.346+0000 7f891a600640 1 -- 172.21.15.66:0/256989566 >> 172.21.15.66:0/256989566 conn(0x7f89140d1c50 msgr2=0x7f89141533d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:28.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.346+0000 7f891a600640 1 -- 172.21.15.66:0/256989566 shutdown_connections 2024-09-10T22:01:28.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.346+0000 7f891a600640 1 -- 172.21.15.66:0/256989566 wait complete. 2024-09-10T22:01:28.371 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group1 2024-09-10T22:01:28.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.514+0000 7f0dde600640 1 -- 172.21.15.66:0/2276437608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd815f460 msgr2=0x7f0dd80cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.514+0000 7f0dde600640 1 --2- 172.21.15.66:0/2276437608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd815f460 0x7f0dd80cfc00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8073ef0 tx=0x7f0dc801c6b0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.514+0000 7f0dde600640 1 -- 172.21.15.66:0/2276437608 shutdown_connections 2024-09-10T22:01:28.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.514+0000 7f0dde600640 1 --2- 172.21.15.66:0/2276437608 >> v2:172.21.15.94:3300/0 conn(0x7f0dd80b32a0 0x7f0dd80af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.514+0000 7f0dde600640 1 --2- 172.21.15.66:0/2276437608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd815f460 0x7f0dd80cfc00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.514+0000 7f0dde600640 1 -- 172.21.15.66:0/2276437608 >> 172.21.15.66:0/2276437608 conn(0x7f0dd80d1d00 msgr2=0x7f0dd80d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:28.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.514+0000 7f0dde600640 1 -- 172.21.15.66:0/2276437608 shutdown_connections 2024-09-10T22:01:28.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.514+0000 7f0dde600640 1 -- 172.21.15.66:0/2276437608 wait complete. 2024-09-10T22:01:28.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dde600640 1 Processor -- start 2024-09-10T22:01:28.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dde600640 1 -- start start 2024-09-10T22:01:28.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dde600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0dd80b0f60 0x7f0dd80c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:28.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dde600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd80b32a0 0x7f0dd8149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:28.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dde600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dd815f460 0x7f0dd815dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:28.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dde600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd80b0520 con 0x7f0dd80b32a0 2024-09-10T22:01:28.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dde600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd80b03a0 con 0x7f0dd815f460 2024-09-10T22:01:28.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dde600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd80b06a0 con 0x7f0dd80b0f60 2024-09-10T22:01:28.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dcea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd80b32a0 0x7f0dd8149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:28.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dcea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd80b32a0 0x7f0dd8149000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40624/0 (socket says 172.21.15.66:40624) 2024-09-10T22:01:28.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dcea00640 1 -- 172.21.15.66:0/4258277073 learned_addr learned my addr 172.21.15.66:0/4258277073 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:28.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dd7400640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0dd80b0f60 0x7f0dd80c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:28.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dd7e00640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dd815f460 0x7f0dd815dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:28.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dcea00640 1 -- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0dd80b0f60 msgr2=0x7f0dd80c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dcea00640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0dd80b0f60 0x7f0dd80c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dcea00640 1 -- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dd815f460 msgr2=0x7f0dd815dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dcea00640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dd815f460 0x7f0dd815dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dcea00640 1 -- 172.21.15.66:0/4258277073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd80cc100 con 0x7f0dd80b32a0 2024-09-10T22:01:28.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dcea00640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd80b32a0 0x7f0dd8149000 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000c4e0 tx=0x7f0dc000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:28.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dcfe00640 1 -- 172.21.15.66:0/4258277073 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0016020 con 0x7f0dd80b32a0 2024-09-10T22:01:28.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dcfe00640 1 -- 172.21.15.66:0/4258277073 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dc0012920 con 0x7f0dd80b32a0 2024-09-10T22:01:28.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dcfe00640 1 -- 172.21.15.66:0/4258277073 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0012d50 con 0x7f0dd80b32a0 2024-09-10T22:01:28.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dde600640 1 -- 172.21.15.66:0/4258277073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd81fb010 con 0x7f0dd80b32a0 2024-09-10T22:01:28.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.518+0000 7f0dde600640 1 -- 172.21.15.66:0/4258277073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd81fb5d0 con 0x7f0dd80b32a0 2024-09-10T22:01:28.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.522+0000 7f0dde600640 1 -- 172.21.15.66:0/4258277073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd80af390 con 0x7f0dd80b32a0 2024-09-10T22:01:28.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.522+0000 7f0dcfe00640 1 -- 172.21.15.66:0/4258277073 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0dc001a3b0 con 0x7f0dd80b32a0 2024-09-10T22:01:28.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.522+0000 7f0dcfe00640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0db40725e0 0x7f0db4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:28.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.522+0000 7f0dcfe00640 1 -- 172.21.15.66:0/4258277073 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0dc00930f0 con 0x7f0dd80b32a0 2024-09-10T22:01:28.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.522+0000 7f0dd7400640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0db40725e0 0x7f0db4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.526+0000 7f0dd7400640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0db40725e0 0x7f0db4074aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80af000 tx=0x7f0dc801aef0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:28.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.526+0000 7f0dcfe00640 1 -- 172.21.15.66:0/4258277073 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0dc0020030 con 0x7f0dd80b32a0 2024-09-10T22:01:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.726+0000 7f0dde600640 1 -- 172.21.15.66:0/4258277073 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f0dd81fbc20 con 0x7f0db40725e0 2024-09-10T22:01:28.737 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:01:28.736+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:01:28.737 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:01:28.736+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:01:28.738 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:01:28.736+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:01:28.738 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:01:28.736+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:01:28.738 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:01:28.736+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:01:28.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.754+0000 7f0dcfe00640 1 -- 172.21.15.66:0/4258277073 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0dd81fbc20 con 0x7f0db40725e0 2024-09-10T22:01:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.754+0000 7f0dde600640 1 -- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0db40725e0 msgr2=0x7f0db4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.754+0000 7f0dde600640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0db40725e0 0x7f0db4074aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80af000 tx=0x7f0dc801aef0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.754+0000 7f0dde600640 1 -- 172.21.15.66:0/4258277073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd80b32a0 msgr2=0x7f0dd8149000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.754+0000 7f0dde600640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd80b32a0 0x7f0dd8149000 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000c4e0 tx=0x7f0dc000c9a0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.754+0000 7f0dde600640 1 -- 172.21.15.66:0/4258277073 shutdown_connections 2024-09-10T22:01:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.754+0000 7f0dde600640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0db40725e0 0x7f0db4074aa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.754+0000 7f0dde600640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dd815f460 0x7f0dd815dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.754+0000 7f0dde600640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dd80b32a0 0x7f0dd8149000 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.758+0000 7f0dde600640 1 --2- 172.21.15.66:0/4258277073 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0dd80b0f60 0x7f0dd80c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.758+0000 7f0dde600640 1 -- 172.21.15.66:0/4258277073 >> 172.21.15.66:0/4258277073 conn(0x7f0dd80d1d00 msgr2=0x7f0dd806e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.758+0000 7f0dde600640 1 -- 172.21.15.66:0/4258277073 shutdown_connections 2024-09-10T22:01:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.758+0000 7f0dde600640 1 -- 172.21.15.66:0/4258277073 wait complete. 2024-09-10T22:01:28.781 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 group1 --mode=777 2024-09-10T22:01:28.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.930+0000 7f65a8400640 1 -- 172.21.15.66:0/2751826899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a015be70 msgr2=0x7f65a00d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.930+0000 7f65a8400640 1 --2- 172.21.15.66:0/2751826899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a015be70 0x7f65a00d02e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6594009840 tx=0x7f659401c820 comp rx=0 tx=0).stop 2024-09-10T22:01:28.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.930+0000 7f65a8400640 1 -- 172.21.15.66:0/2751826899 shutdown_connections 2024-09-10T22:01:28.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.930+0000 7f65a8400640 1 --2- 172.21.15.66:0/2751826899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a015be70 0x7f65a00d02e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.930+0000 7f65a8400640 1 -- 172.21.15.66:0/2751826899 >> 172.21.15.66:0/2751826899 conn(0x7f65a009aa50 msgr2=0x7f65a009ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:28.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.930+0000 7f65a8400640 1 -- 172.21.15.66:0/2751826899 shutdown_connections 2024-09-10T22:01:28.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.930+0000 7f65a8400640 1 -- 172.21.15.66:0/2751826899 wait complete. 2024-09-10T22:01:28.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a8400640 1 Processor -- start 2024-09-10T22:01:28.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a8400640 1 -- start start 2024-09-10T22:01:28.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a8400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65a0072220 0x7f65a00c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:28.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a8400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a00d1c90 0x7f65a00c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:28.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a8400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a015be70 0x7f65a0143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:28.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a8400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f65a016a660 con 0x7f65a00d1c90 2024-09-10T22:01:28.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a8400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65a016a4e0 con 0x7f65a015be70 2024-09-10T22:01:28.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a8400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f65a016a7e0 con 0x7f65a0072220 2024-09-10T22:01:28.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a00d1c90 0x7f65a00c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:28.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a00d1c90 0x7f65a00c7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40634/0 (socket says 172.21.15.66:40634) 2024-09-10T22:01:28.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a5000640 1 -- 172.21.15.66:0/469063470 learned_addr learned my addr 172.21.15.66:0/469063470 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:28.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a5a00640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65a0072220 0x7f65a00c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:28.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a6400640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a015be70 0x7f65a0143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:28.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a5000640 1 -- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65a0072220 msgr2=0x7f65a00c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a5000640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65a0072220 0x7f65a00c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a5000640 1 -- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a015be70 msgr2=0x7f65a0143b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a5000640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a015be70 0x7f65a0143b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a5000640 1 -- 172.21.15.66:0/469063470 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a00c8140 con 0x7f65a00d1c90 2024-09-10T22:01:28.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a5000640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a00d1c90 0x7f65a00c7ac0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f658c00c500 tx=0x7f658c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:28.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f659d600640 1 -- 172.21.15.66:0/469063470 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c017070 con 0x7f65a00d1c90 2024-09-10T22:01:28.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f659d600640 1 -- 172.21.15.66:0/469063470 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f658c0053c0 con 0x7f65a00d1c90 2024-09-10T22:01:28.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a8400640 1 -- 172.21.15.66:0/469063470 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a00c8420 con 0x7f65a00d1c90 2024-09-10T22:01:28.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f659d600640 1 -- 172.21.15.66:0/469063470 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c012890 con 0x7f65a00d1c90 2024-09-10T22:01:28.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.934+0000 7f65a8400640 1 -- 172.21.15.66:0/469063470 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a00c89e0 con 0x7f65a00d1c90 2024-09-10T22:01:28.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.938+0000 7f65a8400640 1 -- 172.21.15.66:0/469063470 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a00b28b0 con 0x7f65a00d1c90 2024-09-10T22:01:28.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.938+0000 7f659d600640 1 -- 172.21.15.66:0/469063470 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f658c003600 con 0x7f65a00d1c90 2024-09-10T22:01:28.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.938+0000 7f659d600640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65800725e0 0x7f6580074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:28.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.938+0000 7f659d600640 1 -- 172.21.15.66:0/469063470 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f658c092820 con 0x7f65a00d1c90 2024-09-10T22:01:28.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.938+0000 7f65a5a00640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65800725e0 0x7f6580074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:28.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.942+0000 7f65a5a00640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65800725e0 0x7f6580074aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f65940af000 tx=0x7f659401b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:28.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:28.942+0000 7f659d600640 1 -- 172.21.15.66:0/469063470 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f658c05d820 con 0x7f65a00d1c90 2024-09-10T22:01:29.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.142+0000 7f65a8400640 1 -- 172.21.15.66:0/469063470 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f65a015a3a0 con 0x7f65800725e0 2024-09-10T22:01:29.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.274+0000 7f659d600640 1 -- 172.21.15.66:0/469063470 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f65a015a3a0 con 0x7f65800725e0 2024-09-10T22:01:29.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.278+0000 7f65a8400640 1 -- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65800725e0 msgr2=0x7f6580074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:29.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.278+0000 7f65a8400640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65800725e0 0x7f6580074aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f65940af000 tx=0x7f659401b060 comp rx=0 tx=0).stop 2024-09-10T22:01:29.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.278+0000 7f65a8400640 1 -- 172.21.15.66:0/469063470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a00d1c90 msgr2=0x7f65a00c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:29.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.278+0000 7f65a8400640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a00d1c90 0x7f65a00c7ac0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f658c00c500 tx=0x7f658c00c9c0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.278+0000 7f65a8400640 1 -- 172.21.15.66:0/469063470 shutdown_connections 2024-09-10T22:01:29.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.278+0000 7f65a8400640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65800725e0 0x7f6580074aa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.278+0000 7f65a8400640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a015be70 0x7f65a0143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.278+0000 7f65a8400640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a00d1c90 0x7f65a00c7ac0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.278+0000 7f65a8400640 1 --2- 172.21.15.66:0/469063470 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65a0072220 0x7f65a00c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.278+0000 7f65a8400640 1 -- 172.21.15.66:0/469063470 >> 172.21.15.66:0/469063470 conn(0x7f65a009aa50 msgr2=0x7f65a00af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:29.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.278+0000 7f65a8400640 1 -- 172.21.15.66:0/469063470 shutdown_connections 2024-09-10T22:01:29.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.278+0000 7f65a8400640 1 -- 172.21.15.66:0/469063470 wait complete. 2024-09-10T22:01:29.301 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-10T22:01:29.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.446+0000 7f6a93a00640 1 -- 172.21.15.66:0/3774036548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c0b0e20 msgr2=0x7f6a8c160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:29.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.446+0000 7f6a93a00640 1 --2- 172.21.15.66:0/3774036548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c0b0e20 0x7f6a8c160290 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009c80 tx=0x7f6a8401cda0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.446+0000 7f6a93a00640 1 -- 172.21.15.66:0/3774036548 shutdown_connections 2024-09-10T22:01:29.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.446+0000 7f6a93a00640 1 --2- 172.21.15.66:0/3774036548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c0b0e20 0x7f6a8c160290 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.446+0000 7f6a93a00640 1 --2- 172.21.15.66:0/3774036548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a8c0cdf90 0x7f6a8c15fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.446+0000 7f6a93a00640 1 -- 172.21.15.66:0/3774036548 >> 172.21.15.66:0/3774036548 conn(0x7f6a8c0d1c60 msgr2=0x7f6a8c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:29.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.446+0000 7f6a93a00640 1 -- 172.21.15.66:0/3774036548 shutdown_connections 2024-09-10T22:01:29.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.446+0000 7f6a93a00640 1 -- 172.21.15.66:0/3774036548 wait complete. 2024-09-10T22:01:29.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a93a00640 1 Processor -- start 2024-09-10T22:01:29.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a93a00640 1 -- start start 2024-09-10T22:01:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a93a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c0b0e20 0x7f6a8c1f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a93a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a8c0cdf90 0x7f6a8c06f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a93a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8c0ce8d0 0x7f6a8c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a93a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c1691a0 con 0x7f6a8c0b0e20 2024-09-10T22:01:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a93a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c169020 con 0x7f6a8c0ce8d0 2024-09-10T22:01:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a93a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6a8c169320 con 0x7f6a8c0cdf90 2024-09-10T22:01:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a91000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c0b0e20 0x7f6a8c1f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a91000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c0b0e20 0x7f6a8c1f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40638/0 (socket says 172.21.15.66:40638) 2024-09-10T22:01:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a91000640 1 -- 172.21.15.66:0/3254967683 learned_addr learned my addr 172.21.15.66:0/3254967683 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:01:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a8be00640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a8c0cdf90 0x7f6a8c06f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a91a00640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8c0ce8d0 0x7f6a8c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a91000640 1 -- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a8c0cdf90 msgr2=0x7f6a8c06f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a91000640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a8c0cdf90 0x7f6a8c06f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a91000640 1 -- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8c0ce8d0 msgr2=0x7f6a8c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a91000640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8c0ce8d0 0x7f6a8c158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a91000640 1 -- 172.21.15.66:0/3254967683 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c06fd00 con 0x7f6a8c0b0e20 2024-09-10T22:01:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a91000640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c0b0e20 0x7f6a8c1f7a20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000c3c0 tx=0x7f6a8000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a88c00640 1 -- 172.21.15.66:0/3254967683 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80017070 con 0x7f6a8c0b0e20 2024-09-10T22:01:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a88c00640 1 -- 172.21.15.66:0/3254967683 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a800053c0 con 0x7f6a8c0b0e20 2024-09-10T22:01:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a88c00640 1 -- 172.21.15.66:0/3254967683 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80003860 con 0x7f6a8c0b0e20 2024-09-10T22:01:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a93a00640 1 -- 172.21.15.66:0/3254967683 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8c152100 con 0x7f6a8c0b0e20 2024-09-10T22:01:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a93a00640 1 -- 172.21.15.66:0/3254967683 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a8c1526c0 con 0x7f6a8c0b0e20 2024-09-10T22:01:29.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.450+0000 7f6a93a00640 1 -- 172.21.15.66:0/3254967683 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a8c0cecb0 con 0x7f6a8c0b0e20 2024-09-10T22:01:29.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.454+0000 7f6a88c00640 1 -- 172.21.15.66:0/3254967683 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6a80003a00 con 0x7f6a8c0b0e20 2024-09-10T22:01:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.454+0000 7f6a88c00640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a6c0725e0 0x7f6a6c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.454+0000 7f6a88c00640 1 -- 172.21.15.66:0/3254967683 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6a800928e0 con 0x7f6a8c0b0e20 2024-09-10T22:01:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.454+0000 7f6a8be00640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a6c0725e0 0x7f6a6c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:29.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.454+0000 7f6a8be00640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a6c0725e0 0x7f6a6c074aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6a78005fd0 tx=0x7f6a78005db0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:29.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.458+0000 7f6a88c00640 1 -- 172.21.15.66:0/3254967683 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6a8005df20 con 0x7f6a8c0b0e20 2024-09-10T22:01:29.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.654+0000 7f6a93a00640 1 -- 172.21.15.66:0/3254967683 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f6a8c06d0d0 con 0x7f6a6c0725e0 2024-09-10T22:01:29.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a88c00640 1 -- 172.21.15.66:0/3254967683 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7f6a8c06d0d0 con 0x7f6a6c0725e0 2024-09-10T22:01:29.666 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc 2024-09-10T22:01:29.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a93a00640 1 -- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a6c0725e0 msgr2=0x7f6a6c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a93a00640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a6c0725e0 0x7f6a6c074aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6a78005fd0 tx=0x7f6a78005db0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a93a00640 1 -- 172.21.15.66:0/3254967683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c0b0e20 msgr2=0x7f6a8c1f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a93a00640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c0b0e20 0x7f6a8c1f7a20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000c3c0 tx=0x7f6a8000c880 comp rx=0 tx=0).stop 2024-09-10T22:01:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a93a00640 1 -- 172.21.15.66:0/3254967683 shutdown_connections 2024-09-10T22:01:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a93a00640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a6c0725e0 0x7f6a6c074aa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a93a00640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8c0ce8d0 0x7f6a8c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a93a00640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a8c0cdf90 0x7f6a8c06f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a93a00640 1 --2- 172.21.15.66:0/3254967683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c0b0e20 0x7f6a8c1f7a20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a93a00640 1 -- 172.21.15.66:0/3254967683 >> 172.21.15.66:0/3254967683 conn(0x7f6a8c0d1c60 msgr2=0x7f6a8c0b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a93a00640 1 -- 172.21.15.66:0/3254967683 shutdown_connections 2024-09-10T22:01:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:01:29.662+0000 7f6a93a00640 1 -- 172.21.15.66:0/3254967683 wait complete. 2024-09-10T22:01:29.687 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc 2024-09-10T22:01:29.687 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:01:29.688 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:29.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:29.697 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:01:36.852 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:01:36.853 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:01:36.853 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.08678 s, 152 MB/s 2024-09-10T22:01:36.854 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:01:36.854 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:36.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:36.863 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:01:43.738 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:01:43.739 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:01:43.739 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.81945 s, 157 MB/s 2024-09-10T22:01:43.740 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:01:43.740 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:43.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:43.749 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:01:50.646 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:01:50.647 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:01:50.647 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.84222 s, 157 MB/s 2024-09-10T22:01:50.648 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:01:50.648 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:50.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:50.657 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:01:55.068 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:01:55.073 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:01:57.672 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:01:57.672 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:01:57.672 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.95946 s, 154 MB/s 2024-09-10T22:01:57.673 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:01:57.673 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:57.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:57.682 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:02:04.632 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:02:04.632 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:02:04.632 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.89609 s, 156 MB/s 2024-09-10T22:02:04.633 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:02:04.633 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:02:04.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:02:04.642 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:02:11.610 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:02:11.610 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:02:11.610 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.9092 s, 155 MB/s 2024-09-10T22:02:11.611 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:02:11.611 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:02:11.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:02:11.619 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:02:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:02:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:02:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.88532 s, 156 MB/s 2024-09-10T22:02:18.558 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:02:18.559 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:02:18.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:02:18.567 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:02:25.125 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:02:25.129 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:02:25.486 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:02:25.486 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:02:25.487 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.86633 s, 156 MB/s 2024-09-10T22:02:25.488 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:02:25.488 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:02:25.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:02:25.496 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:02:32.412 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:02:32.412 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:02:32.412 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.85979 s, 157 MB/s 2024-09-10T22:02:32.413 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:02:32.414 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:02:32.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:02:32.422 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:02:39.362 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:02:39.362 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:02:39.362 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.88194 s, 156 MB/s 2024-09-10T22:02:39.363 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 group1 2024-09-10T22:02:39.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 -- 172.21.15.66:0/3991664190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ee006eea0 msgr2=0x7f3ee006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 --2- 172.21.15.66:0/3991664190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ee006eea0 0x7f3ee006e4a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009cb0 tx=0x7f3ed801f5a0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 -- 172.21.15.66:0/3991664190 shutdown_connections 2024-09-10T22:02:39.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 --2- 172.21.15.66:0/3991664190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ee006eea0 0x7f3ee006e4a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 --2- 172.21.15.66:0/3991664190 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ee00713b0 0x7f3ee00b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 -- 172.21.15.66:0/3991664190 >> 172.21.15.66:0/3991664190 conn(0x7f3ee00d1d00 msgr2=0x7f3ee00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:39.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 -- 172.21.15.66:0/3991664190 shutdown_connections 2024-09-10T22:02:39.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 -- 172.21.15.66:0/3991664190 wait complete. 2024-09-10T22:02:39.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 Processor -- start 2024-09-10T22:02:39.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 -- start start 2024-09-10T22:02:39.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee006eea0 0x7f3ee00af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ee0070fd0 0x7f3ee009eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ee00713b0 0x7f3ee009f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee00d03b0 con 0x7f3ee00713b0 2024-09-10T22:02:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee00d0230 con 0x7f3ee006eea0 2024-09-10T22:02:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee6e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee00d0530 con 0x7f3ee0070fd0 2024-09-10T22:02:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ee00713b0 0x7f3ee009f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ee00713b0 0x7f3ee009f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35138/0 (socket says 172.21.15.66:35138) 2024-09-10T22:02:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ed7e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ee0070fd0 0x7f3ee009eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3ee4e00640 1 -- 172.21.15.66:0/2244649760 learned_addr learned my addr 172.21.15.66:0/2244649760 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.513+0000 7f3edfe00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee006eea0 0x7f3ee00af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3ed7e00640 1 -- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee006eea0 msgr2=0x7f3ee00af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3ed7e00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee006eea0 0x7f3ee00af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3ed7e00640 1 -- 172.21.15.66:0/2244649760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ee00713b0 msgr2=0x7f3ee009f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3ed7e00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ee00713b0 0x7f3ee009f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3ed7e00640 1 -- 172.21.15.66:0/2244649760 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee01f54f0 con 0x7f3ee0070fd0 2024-09-10T22:02:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3ed7e00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ee0070fd0 0x7f3ee009eed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800e9f0 tx=0x7f3ec800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3edcc00640 1 -- 172.21.15.66:0/2244649760 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800ccc0 con 0x7f3ee0070fd0 2024-09-10T22:02:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3edcc00640 1 -- 172.21.15.66:0/2244649760 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ec8004020 con 0x7f3ee0070fd0 2024-09-10T22:02:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3edcc00640 1 -- 172.21.15.66:0/2244649760 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800ce60 con 0x7f3ee0070fd0 2024-09-10T22:02:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3ee6e00640 1 -- 172.21.15.66:0/2244649760 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee015ef40 con 0x7f3ee0070fd0 2024-09-10T22:02:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3ee6e00640 1 -- 172.21.15.66:0/2244649760 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee015f500 con 0x7f3ee0070fd0 2024-09-10T22:02:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3ee6e00640 1 -- 172.21.15.66:0/2244649760 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee00b1180 con 0x7f3ee0070fd0 2024-09-10T22:02:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3edcc00640 1 -- 172.21.15.66:0/2244649760 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 6) ==== 96445+0+0 (secure 0 0 0) 0x7f3ec8003600 con 0x7f3ee0070fd0 2024-09-10T22:02:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3edcc00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3ebc072670 0x7f3ebc074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.517+0000 7f3edcc00640 1 -- 172.21.15.66:0/2244649760 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3ec8093b80 con 0x7f3ee0070fd0 2024-09-10T22:02:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.521+0000 7f3edfe00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3ebc072670 0x7f3ebc074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:39.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.521+0000 7f3edfe00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3ebc072670 0x7f3ebc074b30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0001820 tx=0x7f3ed0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:39.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.525+0000 7f3edcc00640 1 -- 172.21.15.66:0/2244649760 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3ec805eaa0 con 0x7f3ee0070fd0 2024-09-10T22:02:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.725+0000 7f3ee6e00640 1 -- 172.21.15.66:0/2244649760 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f3ee00d2b80 con 0x7f3ebc072670 2024-09-10T22:02:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.737+0000 7f3edcc00640 1 -- 172.21.15.66:0/2244649760 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3ee00d2b80 con 0x7f3ebc072670 2024-09-10T22:02:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.741+0000 7f3ee6e00640 1 -- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3ebc072670 msgr2=0x7f3ebc074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.741+0000 7f3ee6e00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3ebc072670 0x7f3ebc074b30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0001820 tx=0x7f3ed0009040 comp rx=0 tx=0).stop 2024-09-10T22:02:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.741+0000 7f3ee6e00640 1 -- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ee0070fd0 msgr2=0x7f3ee009eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.741+0000 7f3ee6e00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ee0070fd0 0x7f3ee009eed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800e9f0 tx=0x7f3ec800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.741+0000 7f3ee6e00640 1 -- 172.21.15.66:0/2244649760 shutdown_connections 2024-09-10T22:02:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.741+0000 7f3ee6e00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3ebc072670 0x7f3ebc074b30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.741+0000 7f3ee6e00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ee00713b0 0x7f3ee009f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.741+0000 7f3ee6e00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ee0070fd0 0x7f3ee009eed0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.741+0000 7f3ee6e00640 1 --2- 172.21.15.66:0/2244649760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee006eea0 0x7f3ee00af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.741+0000 7f3ee6e00640 1 -- 172.21.15.66:0/2244649760 >> 172.21.15.66:0/2244649760 conn(0x7f3ee00d1d00 msgr2=0x7f3ee0161650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.741+0000 7f3ee6e00640 1 -- 172.21.15.66:0/2244649760 shutdown_connections 2024-09-10T22:02:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.741+0000 7f3ee6e00640 1 -- 172.21.15.66:0/2244649760 wait complete. 2024-09-10T22:02:39.765 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-10T22:02:39.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.909+0000 7fd677400640 1 -- 172.21.15.66:0/2839656593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6700cd8d0 msgr2=0x7fd6700d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.909+0000 7fd677400640 1 --2- 172.21.15.66:0/2839656593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6700cd8d0 0x7fd6700d0220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd6700d1860 tx=0x7fd66801cb50 comp rx=0 tx=0).stop 2024-09-10T22:02:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.909+0000 7fd677400640 1 -- 172.21.15.66:0/2839656593 shutdown_connections 2024-09-10T22:02:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.909+0000 7fd677400640 1 --2- 172.21.15.66:0/2839656593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6700cd8d0 0x7fd6700d0220 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.909+0000 7fd677400640 1 --2- 172.21.15.66:0/2839656593 >> v2:172.21.15.94:3300/0 conn(0x7fd6700d08a0 0x7fd6700cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.909+0000 7fd677400640 1 -- 172.21.15.66:0/2839656593 >> 172.21.15.66:0/2839656593 conn(0x7fd6700d1c60 msgr2=0x7fd6700d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.909+0000 7fd677400640 1 -- 172.21.15.66:0/2839656593 shutdown_connections 2024-09-10T22:02:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.909+0000 7fd677400640 1 -- 172.21.15.66:0/2839656593 wait complete. 2024-09-10T22:02:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd677400640 1 Processor -- start 2024-09-10T22:02:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd677400640 1 -- start start 2024-09-10T22:02:39.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd677400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd6700cd8d0 0x7fd6700c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd677400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6700d08a0 0x7fd67014bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd677400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67015df80 0x7fd67015bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd677400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd67006df10 con 0x7fd67015df80 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd677400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd67006dd90 con 0x7fd6700d08a0 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd677400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd67006e090 con 0x7fd6700cd8d0 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd675400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67015df80 0x7fd67015bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd675400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67015df80 0x7fd67015bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35164/0 (socket says 172.21.15.66:35164) 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd675400640 1 -- 172.21.15.66:0/1262243830 learned_addr learned my addr 172.21.15.66:0/1262243830 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd66fe00640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6700d08a0 0x7fd67014bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd674a00640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd6700cd8d0 0x7fd6700c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd675400640 1 -- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd6700cd8d0 msgr2=0x7fd6700c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd675400640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd6700cd8d0 0x7fd6700c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd675400640 1 -- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6700d08a0 msgr2=0x7fd67014bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd675400640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6700d08a0 0x7fd67014bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd675400640 1 -- 172.21.15.66:0/1262243830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd670161240 con 0x7fd67015df80 2024-09-10T22:02:39.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd675400640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67015df80 0x7fd67015bda0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd66801a780 tx=0x7fd668009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:39.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd66cc00640 1 -- 172.21.15.66:0/1262243830 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6680bb070 con 0x7fd67015df80 2024-09-10T22:02:39.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd66cc00640 1 -- 172.21.15.66:0/1262243830 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd668004d40 con 0x7fd67015df80 2024-09-10T22:02:39.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd66cc00640 1 -- 172.21.15.66:0/1262243830 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6680b2040 con 0x7fd67015df80 2024-09-10T22:02:39.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd677400640 1 -- 172.21.15.66:0/1262243830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6700ca7a0 con 0x7fd67015df80 2024-09-10T22:02:39.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd677400640 1 -- 172.21.15.66:0/1262243830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67015c2e0 con 0x7fd67015df80 2024-09-10T22:02:39.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.913+0000 7fd677400640 1 -- 172.21.15.66:0/1262243830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67006e5b0 con 0x7fd67015df80 2024-09-10T22:02:39.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.917+0000 7fd66cc00640 1 -- 172.21.15.66:0/1262243830 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 6) ==== 96445+0+0 (secure 0 0 0) 0x7fd6680b5030 con 0x7fd67015df80 2024-09-10T22:02:39.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.917+0000 7fd66cc00640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd650072670 0x7fd650074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:39.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.917+0000 7fd66cc00640 1 -- 172.21.15.66:0/1262243830 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd66812cff0 con 0x7fd67015df80 2024-09-10T22:02:39.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.917+0000 7fd674a00640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd650072670 0x7fd650074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:39.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.917+0000 7fd674a00640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd650072670 0x7fd650074b30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd664007fd0 tx=0x7fd664009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:39.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:39.921+0000 7fd66cc00640 1 -- 172.21.15.66:0/1262243830 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd668016770 con 0x7fd67015df80 2024-09-10T22:02:40.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.125+0000 7fd677400640 1 -- 172.21.15.66:0/1262243830 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7fd670154510 con 0x7fd650072670 2024-09-10T22:02:40.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.129+0000 7fd66cc00640 1 -- 172.21.15.66:0/1262243830 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7fd670154510 con 0x7fd650072670 2024-09-10T22:02:40.137 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc 2024-09-10T22:02:40.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.133+0000 7fd677400640 1 -- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd650072670 msgr2=0x7fd650074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:40.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.133+0000 7fd677400640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd650072670 0x7fd650074b30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd664007fd0 tx=0x7fd664009040 comp rx=0 tx=0).stop 2024-09-10T22:02:40.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.133+0000 7fd677400640 1 -- 172.21.15.66:0/1262243830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67015df80 msgr2=0x7fd67015bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:40.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.133+0000 7fd677400640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67015df80 0x7fd67015bda0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd66801a780 tx=0x7fd668009300 comp rx=0 tx=0).stop 2024-09-10T22:02:40.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.133+0000 7fd677400640 1 -- 172.21.15.66:0/1262243830 shutdown_connections 2024-09-10T22:02:40.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.133+0000 7fd677400640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd650072670 0x7fd650074b30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:40.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.133+0000 7fd677400640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67015df80 0x7fd67015bda0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:40.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.133+0000 7fd677400640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6700d08a0 0x7fd67014bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:40.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.133+0000 7fd677400640 1 --2- 172.21.15.66:0/1262243830 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd6700cd8d0 0x7fd6700c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:40.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.133+0000 7fd677400640 1 -- 172.21.15.66:0/1262243830 >> 172.21.15.66:0/1262243830 conn(0x7fd6700d1c60 msgr2=0x7fd6700b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:40.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.133+0000 7fd677400640 1 -- 172.21.15.66:0/1262243830 shutdown_connections 2024-09-10T22:02:40.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:40.133+0000 7fd677400640 1 -- 172.21.15.66:0/1262243830 wait complete. 2024-09-10T22:02:40.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:02:40.159 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc 2024-09-10T22:02:40.159 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:02:40.189 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc 2024-09-10T22:02:40.190 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="10654449664" 2024-09-10T22:02:40.190 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:02:40.190 INFO:tasks.cephfs.test_volumes:r_size = 10654449664 exp_size = 10737418240 2024-09-10T22:02:42.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:02:42.191 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc 2024-09-10T22:02:42.191 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:02:42.205 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/group1/sv1/c63794aa-b899-4e91-92bc-107119bddadc 2024-09-10T22:02:42.206 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="10737418240" 2024-09-10T22:02:42.206 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:02:42.207 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-10T22:02:42.207 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 --group-name group1 2024-09-10T22:02:42.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 -- 172.21.15.66:0/2384230576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f18004d50 msgr2=0x7f6f180051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:42.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 --2- 172.21.15.66:0/2384230576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f18004d50 0x7f6f180051d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009ee0 tx=0x7f6f2c01d5f0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f33400640 1 -- 172.21.15.66:0/2384230576 reap_dead start 2024-09-10T22:02:42.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 -- 172.21.15.66:0/2384230576 shutdown_connections 2024-09-10T22:02:42.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 -- 172.21.15.66:0/2384230576 >> 172.21.15.66:0/2384230576 conn(0x7f6f3406cdf0 msgr2=0x7f6f3406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:42.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 -- 172.21.15.66:0/2384230576 shutdown_connections 2024-09-10T22:02:42.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 -- 172.21.15.66:0/2384230576 wait complete. 2024-09-10T22:02:42.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 Processor -- start 2024-09-10T22:02:42.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 -- start start 2024-09-10T22:02:42.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f18004d50 0x7f6f340af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:42.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f340b21a0 0x7f6f34143ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:42.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f341f4f30 0x7f6f341f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:42.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3415fc00 con 0x7f6f18004d50 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3415fa80 con 0x7f6f341f4f30 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.413+0000 7f6f39600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6f3415fd80 con 0x7f6f340b21a0 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f32a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f18004d50 0x7f6f340af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f32a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f18004d50 0x7f6f340af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33988/0 (socket says 172.21.15.66:33988) 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f32a00640 1 -- 172.21.15.66:0/2168263644 learned_addr learned my addr 172.21.15.66:0/2168263644 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f2be00640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f340b21a0 0x7f6f34143ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f33400640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f341f4f30 0x7f6f341f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f32a00640 1 -- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f340b21a0 msgr2=0x7f6f34143ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f32a00640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f340b21a0 0x7f6f34143ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f32a00640 1 -- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f341f4f30 msgr2=0x7f6f341f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f32a00640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f341f4f30 0x7f6f341f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f32a00640 1 -- 172.21.15.66:0/2168263644 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f340b1120 con 0x7f6f18004d50 2024-09-10T22:02:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f2be00640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f340b21a0 0x7f6f34143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:42.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f32a00640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f18004d50 0x7f6f340af690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6f24006690 tx=0x7f6f240066c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:42.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f2aa00640 1 -- 172.21.15.66:0/2168263644 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400f760 con 0x7f6f18004d50 2024-09-10T22:02:42.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f2aa00640 1 -- 172.21.15.66:0/2168263644 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f24016020 con 0x7f6f18004d50 2024-09-10T22:02:42.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f2aa00640 1 -- 172.21.15.66:0/2168263644 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400fba0 con 0x7f6f18004d50 2024-09-10T22:02:42.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f39600640 1 -- 172.21.15.66:0/2168263644 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f34144010 con 0x7f6f18004d50 2024-09-10T22:02:42.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f39600640 1 -- 172.21.15.66:0/2168263644 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f340b16c0 con 0x7f6f18004d50 2024-09-10T22:02:42.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f39600640 1 -- 172.21.15.66:0/2168263644 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f340ced20 con 0x7f6f18004d50 2024-09-10T22:02:42.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.417+0000 7f6f2aa00640 1 -- 172.21.15.66:0/2168263644 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 6) ==== 96445+0+0 (secure 0 0 0) 0x7f6f24011430 con 0x7f6f18004d50 2024-09-10T22:02:42.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.421+0000 7f6f2aa00640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f10072670 0x7f6f10074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:42.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.421+0000 7f6f2aa00640 1 -- 172.21.15.66:0/2168263644 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6f24093ff0 con 0x7f6f18004d50 2024-09-10T22:02:42.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.421+0000 7f6f2be00640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f10072670 0x7f6f10074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:42.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.421+0000 7f6f2be00640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f10072670 0x7f6f10074b30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009c40 tx=0x7f6f1c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:42.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.425+0000 7f6f2aa00640 1 -- 172.21.15.66:0/2168263644 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6f24020020 con 0x7f6f18004d50 2024-09-10T22:02:42.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.633+0000 7f6f39600640 1 -- 172.21.15.66:0/2168263644 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f6f34073010 con 0x7f6f10072670 2024-09-10T22:02:42.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.697+0000 7f6f2aa00640 1 -- 172.21.15.66:0/2168263644 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6f34073010 con 0x7f6f10072670 2024-09-10T22:02:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.697+0000 7f6f39600640 1 -- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f10072670 msgr2=0x7f6f10074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.697+0000 7f6f39600640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f10072670 0x7f6f10074b30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009c40 tx=0x7f6f1c0091e0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.697+0000 7f6f39600640 1 -- 172.21.15.66:0/2168263644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f18004d50 msgr2=0x7f6f340af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.697+0000 7f6f39600640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f18004d50 0x7f6f340af690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6f24006690 tx=0x7f6f240066c0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.697+0000 7f6f39600640 1 -- 172.21.15.66:0/2168263644 shutdown_connections 2024-09-10T22:02:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.697+0000 7f6f39600640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f10072670 0x7f6f10074b30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.697+0000 7f6f39600640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f341f4f30 0x7f6f341f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.697+0000 7f6f39600640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f340b21a0 0x7f6f34143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.697+0000 7f6f39600640 1 --2- 172.21.15.66:0/2168263644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f18004d50 0x7f6f340af690 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.697+0000 7f6f39600640 1 -- 172.21.15.66:0/2168263644 >> 172.21.15.66:0/2168263644 conn(0x7f6f3406cdf0 msgr2=0x7f6f3415e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.697+0000 7f6f39600640 1 -- 172.21.15.66:0/2168263644 shutdown_connections 2024-09-10T22:02:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.701+0000 7f6f39600640 1 -- 172.21.15.66:0/2168263644 wait complete. 2024-09-10T22:02:42.706 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:42.703+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:42.706 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:42.703+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:42.706 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:42.703+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:42.706 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:42.703+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:42.706 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:42.703+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:42.723 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:42.719+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:42.723 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:42.719+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:42.723 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:42.719+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:42.723 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:42.719+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:42.723 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:42.719+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:42.725 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:02:42.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.865+0000 7f2f8f200640 1 -- 172.21.15.66:0/114266138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f90073d90 msgr2=0x7f2f9006f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:42.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 --2- 172.21.15.66:0/114266138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f90073d90 0x7f2f9006f600 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009900 tx=0x7f2f7801c8b0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 -- 172.21.15.66:0/114266138 shutdown_connections 2024-09-10T22:02:42.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 --2- 172.21.15.66:0/114266138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f90073d90 0x7f2f9006f600 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 --2- 172.21.15.66:0/114266138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f90158370 0x7f2f900cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 -- 172.21.15.66:0/114266138 >> 172.21.15.66:0/114266138 conn(0x7f2f900cc3c0 msgr2=0x7f2f900cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:42.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 -- 172.21.15.66:0/114266138 shutdown_connections 2024-09-10T22:02:42.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 -- 172.21.15.66:0/114266138 wait complete. 2024-09-10T22:02:42.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 Processor -- start 2024-09-10T22:02:42.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 -- start start 2024-09-10T22:02:42.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f90073d90 0x7f2f90157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:42.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f900741b0 0x7f2f9015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:42.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f90158370 0x7f2f9015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:42.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90167f10 con 0x7f2f900741b0 2024-09-10T22:02:42.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f900caa60 con 0x7f2f90158370 2024-09-10T22:02:42.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8f200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2f90168090 con 0x7f2f90073d90 2024-09-10T22:02:42.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f87e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f900741b0 0x7f2f9015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:42.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8de00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f90073d90 0x7f2f90157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:42.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8de00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f90073d90 0x7f2f90157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:49262/0 (socket says 172.21.15.66:49262) 2024-09-10T22:02:42.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8de00640 1 -- 172.21.15.66:0/3353246104 learned_addr learned my addr 172.21.15.66:0/3353246104 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:42.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f8e800640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f90158370 0x7f2f9015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:42.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f87e00640 1 -- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f90073d90 msgr2=0x7f2f90157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:42.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f87e00640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f90073d90 0x7f2f90157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f87e00640 1 -- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f90158370 msgr2=0x7f2f9015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:42.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f87e00640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f90158370 0x7f2f9015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:42.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f87e00640 1 -- 172.21.15.66:0/3353246104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f90157b50 con 0x7f2f900741b0 2024-09-10T22:02:42.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.869+0000 7f2f87e00640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f900741b0 0x7f2f9015d1a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2f780ac000 tx=0x7f2f78004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:42.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.873+0000 7f2f86000640 1 -- 172.21.15.66:0/3353246104 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f780b2040 con 0x7f2f900741b0 2024-09-10T22:02:42.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.873+0000 7f2f86000640 1 -- 172.21.15.66:0/3353246104 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f78004430 con 0x7f2f900741b0 2024-09-10T22:02:42.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.873+0000 7f2f86000640 1 -- 172.21.15.66:0/3353246104 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78005290 con 0x7f2f900741b0 2024-09-10T22:02:42.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.873+0000 7f2f8f200640 1 -- 172.21.15.66:0/3353246104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f901fb4d0 con 0x7f2f900741b0 2024-09-10T22:02:42.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.873+0000 7f2f8f200640 1 -- 172.21.15.66:0/3353246104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f901fba90 con 0x7f2f900741b0 2024-09-10T22:02:42.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.873+0000 7f2f8f200640 1 -- 172.21.15.66:0/3353246104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f9006f600 con 0x7f2f900741b0 2024-09-10T22:02:42.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.873+0000 7f2f86000640 1 -- 172.21.15.66:0/3353246104 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 6) ==== 96445+0+0 (secure 0 0 0) 0x7f2f78005430 con 0x7f2f900741b0 2024-09-10T22:02:42.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.873+0000 7f2f86000640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f6c072670 0x7f2f6c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:42.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.873+0000 7f2f86000640 1 -- 172.21.15.66:0/3353246104 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7812cdd0 con 0x7f2f900741b0 2024-09-10T22:02:42.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.873+0000 7f2f8de00640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f6c072670 0x7f2f6c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:42.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.877+0000 7f2f8de00640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f6c072670 0x7f2f6c074b30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2f90152410 tx=0x7f2f8000b410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:42.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:42.881+0000 7f2f86000640 1 -- 172.21.15.66:0/3353246104 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2f780164d0 con 0x7f2f900741b0 2024-09-10T22:02:43.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.113+0000 7f2f8f200640 1 -- 172.21.15.66:0/3353246104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2f9015b360 con 0x7f2f900741b0 2024-09-10T22:02:43.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.113+0000 7f2f86000640 1 -- 172.21.15.66:0/3353246104 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1882 (secure 0 0 0) 0x7f2f780f7cf0 con 0x7f2f900741b0 2024-09-10T22:02:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:02:43.121 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:02:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:02:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:02:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:02:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:02:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:02:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 156, 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:02:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 33, 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:02:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:02:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:02:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:02:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:02:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:02:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:02:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:02:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 2585, 2024-09-10T22:02:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 10738562327, 2024-09-10T22:02:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 21735567360, 2024-09-10T22:02:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 751358545920, 2024-09-10T22:02:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:02:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 162156507, 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 0, 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 41 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 18, 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:01:26:894150+0000", 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 2, 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 2, 2024-09-10T22:02:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "a", 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 4908 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:02:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:02:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:02:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:02:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 5, 2024-09-10T22:02:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:02:24.335428+0000", 2024-09-10T22:02:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:02:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": {} 2024-09-10T22:02:43.127 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:02:43.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.117+0000 7f2f8f200640 1 -- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f6c072670 msgr2=0x7f2f6c074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:43.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.117+0000 7f2f8f200640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f6c072670 0x7f2f6c074b30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2f90152410 tx=0x7f2f8000b410 comp rx=0 tx=0).stop 2024-09-10T22:02:43.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.117+0000 7f2f8f200640 1 -- 172.21.15.66:0/3353246104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f900741b0 msgr2=0x7f2f9015d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:43.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.117+0000 7f2f8f200640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f900741b0 0x7f2f9015d1a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2f780ac000 tx=0x7f2f78004040 comp rx=0 tx=0).stop 2024-09-10T22:02:43.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.117+0000 7f2f8f200640 1 -- 172.21.15.66:0/3353246104 shutdown_connections 2024-09-10T22:02:43.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.117+0000 7f2f8f200640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f6c072670 0x7f2f6c074b30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.117+0000 7f2f8f200640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f90158370 0x7f2f9015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.117+0000 7f2f8f200640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f900741b0 0x7f2f9015d1a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.117+0000 7f2f8f200640 1 --2- 172.21.15.66:0/3353246104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f90073d90 0x7f2f90157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.117+0000 7f2f8f200640 1 -- 172.21.15.66:0/3353246104 >> 172.21.15.66:0/3353246104 conn(0x7f2f900cc3c0 msgr2=0x7f2f9015b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.117+0000 7f2f8f200640 1 -- 172.21.15.66:0/3353246104 shutdown_connections 2024-09-10T22:02:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:43.117+0000 7f2f8f200640 1 -- 172.21.15.66:0/3353246104 wait complete. 2024-09-10T22:02:43.711 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:43.707+0000 7f5fdc000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:43.711 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:43.707+0000 7f5fdc000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:43.711 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:43.707+0000 7f5fdc000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:43.711 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:43.707+0000 7f5fdc000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:43.711 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:43.707+0000 7f5fdc000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:44.142 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:02:44.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.353+0000 7f8342e00640 1 -- 172.21.15.66:0/3114764482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833c0d04b0 msgr2=0x7f833c15e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:44.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.353+0000 7f8342e00640 1 --2- 172.21.15.66:0/3114764482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833c0d04b0 0x7f833c15e100 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f833c0b4c20 tx=0x7f832801c810 comp rx=0 tx=0).stop 2024-09-10T22:02:44.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.353+0000 7f8342e00640 1 -- 172.21.15.66:0/3114764482 shutdown_connections 2024-09-10T22:02:44.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.353+0000 7f8342e00640 1 --2- 172.21.15.66:0/3114764482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833c0d04b0 0x7f833c15e100 secure :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f833c0b4c20 tx=0x7f832801c810 comp rx=0 tx=0).stop 2024-09-10T22:02:44.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.353+0000 7f8342e00640 1 -- 172.21.15.66:0/3114764482 >> 172.21.15.66:0/3114764482 conn(0x7f833c0d0a30 msgr2=0x7f833c0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:44.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.357+0000 7f8342e00640 1 -- 172.21.15.66:0/3114764482 shutdown_connections 2024-09-10T22:02:44.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.357+0000 7f8342e00640 1 -- 172.21.15.66:0/3114764482 wait complete. 2024-09-10T22:02:44.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.361+0000 7f8342e00640 1 Processor -- start 2024-09-10T22:02:44.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.361+0000 7f8342e00640 1 -- start start 2024-09-10T22:02:44.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.365+0000 7f8342e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833c06d890 0x7f833c13f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:44.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.365+0000 7f8342e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f833c0d1550 0x7f833c158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:44.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.365+0000 7f8342e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f833c071860 0x7f833c15ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:44.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.365+0000 7f8342e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f833c16c7b0 con 0x7f833c06d890 2024-09-10T22:02:44.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.365+0000 7f8342e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f833c16c630 con 0x7f833c0d1550 2024-09-10T22:02:44.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.365+0000 7f8342e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f833c16c930 con 0x7f833c071860 2024-09-10T22:02:44.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.365+0000 7f8341a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833c06d890 0x7f833c13f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:44.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8341a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833c06d890 0x7f833c13f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34016/0 (socket says 172.21.15.66:34016) 2024-09-10T22:02:44.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8341a00640 1 -- 172.21.15.66:0/2434424699 learned_addr learned my addr 172.21.15.66:0/2434424699 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:44.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8341000640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f833c0d1550 0x7f833c158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:44.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8341a00640 1 -- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f833c071860 msgr2=0x7f833c15ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:44.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8341a00640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f833c071860 0x7f833c15ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8341a00640 1 -- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f833c0d1550 msgr2=0x7f833c158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:44.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8341a00640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f833c0d1550 0x7f833c158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8341a00640 1 -- 172.21.15.66:0/2434424699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c13f640 con 0x7f833c06d890 2024-09-10T22:02:44.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8341a00640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833c06d890 0x7f833c13f100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f833000c450 tx=0x7f833000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:44.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8339600640 1 -- 172.21.15.66:0/2434424699 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330012830 con 0x7f833c06d890 2024-09-10T22:02:44.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8339600640 1 -- 172.21.15.66:0/2434424699 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8330017070 con 0x7f833c06d890 2024-09-10T22:02:44.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8339600640 1 -- 172.21.15.66:0/2434424699 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330012d60 con 0x7f833c06d890 2024-09-10T22:02:44.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8342e00640 1 -- 172.21.15.66:0/2434424699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833c158f60 con 0x7f833c06d890 2024-09-10T22:02:44.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8342e00640 1 -- 172.21.15.66:0/2434424699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833c159520 con 0x7f833c06d890 2024-09-10T22:02:44.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.373+0000 7f8339600640 1 -- 172.21.15.66:0/2434424699 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 6) ==== 96445+0+0 (secure 0 0 0) 0x7f833001c460 con 0x7f833c06d890 2024-09-10T22:02:44.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.369+0000 7f8342e00640 1 -- 172.21.15.66:0/2434424699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f833c159be0 con 0x7f833c06d890 2024-09-10T22:02:44.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.373+0000 7f8339600640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f831c0726c0 0x7f831c074b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:44.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.377+0000 7f8339600640 1 -- 172.21.15.66:0/2434424699 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f83300930b0 con 0x7f833c06d890 2024-09-10T22:02:44.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.377+0000 7f8341000640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f831c0726c0 0x7f831c074b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:44.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.377+0000 7f8341000640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f831c0726c0 0x7f831c074b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f83280af000 tx=0x7f832801b050 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.385+0000 7f8339600640 1 -- 172.21.15.66:0/2434424699 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f833c159be0 con 0x7f833c06d890 2024-09-10T22:02:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.465+0000 7f8339600640 1 -- 172.21.15.66:0/2434424699 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 7) ==== 96598+0+0 (secure 0 0 0) 0x7f833005a340 con 0x7f833c06d890 2024-09-10T22:02:44.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.805+0000 7f8342e00640 1 -- 172.21.15.66:0/2434424699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f833c0cce40 con 0x7f833c06d890 2024-09-10T22:02:44.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.805+0000 7f8339600640 1 -- 172.21.15.66:0/2434424699 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2107 (secure 0 0 0) 0x7f83300865f0 con 0x7f833c06d890 2024-09-10T22:02:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:02:44.813 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:02:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:02:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:02:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:02:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:02:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:02:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:02:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 157, 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:02:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 33, 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:02:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 2585, 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 10738562327, 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 21735567360, 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 751358545920, 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 95057231, 2024-09-10T22:02:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 0, 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 24 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 18, 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:01:26:894150+0000", 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 2, 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 2, 2024-09-10T22:02:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:02:44.818 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "a", 2024-09-10T22:02:44.818 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:02:44.818 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 4908 2024-09-10T22:02:44.818 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:02:44.818 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:02:44.818 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:02:44.818 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 5, 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:02:24.335428+0000", 2024-09-10T22:02:44.819 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:02:44.820 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:02:44.820 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": { 2024-09-10T22:02:44.820 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr-vol-ongoing-clones": { 2024-09-10T22:02:44.820 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "1 ongoing clones - average progress is 0.0% (0s)\n [............................] ", 2024-09-10T22:02:44.820 INFO:teuthology.orchestra.run.smithi066.stdout: "progress": 0, 2024-09-10T22:02:44.820 INFO:teuthology.orchestra.run.smithi066.stdout: "add_to_ceph_s": true 2024-09-10T22:02:44.820 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:02:44.820 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:02:44.820 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:02:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.809+0000 7f8342e00640 1 -- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f831c0726c0 msgr2=0x7f831c074b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.809+0000 7f8342e00640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f831c0726c0 0x7f831c074b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f83280af000 tx=0x7f832801b050 comp rx=0 tx=0).stop 2024-09-10T22:02:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.809+0000 7f8342e00640 1 -- 172.21.15.66:0/2434424699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833c06d890 msgr2=0x7f833c13f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:44.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.809+0000 7f8342e00640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833c06d890 0x7f833c13f100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f833000c450 tx=0x7f833000c910 comp rx=0 tx=0).stop 2024-09-10T22:02:44.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.809+0000 7f8342e00640 1 -- 172.21.15.66:0/2434424699 shutdown_connections 2024-09-10T22:02:44.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.809+0000 7f8342e00640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f831c0726c0 0x7f831c074b80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.809+0000 7f8342e00640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f833c071860 0x7f833c15ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.809+0000 7f8342e00640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f833c0d1550 0x7f833c158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.809+0000 7f8342e00640 1 --2- 172.21.15.66:0/2434424699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833c06d890 0x7f833c13f100 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.809+0000 7f8342e00640 1 -- 172.21.15.66:0/2434424699 >> 172.21.15.66:0/2434424699 conn(0x7f833c0d0a30 msgr2=0x7f833c160320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:44.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.809+0000 7f8342e00640 1 -- 172.21.15.66:0/2434424699 shutdown_connections 2024-09-10T22:02:44.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:44.809+0000 7f8342e00640 1 -- 172.21.15.66:0/2434424699 wait complete. 2024-09-10T22:02:44.844 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-10T22:02:45.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.081+0000 7fbfa1800640 1 -- 172.21.15.66:0/3608959024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf80004f50 msgr2=0x7fbf8000dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:45.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.081+0000 7fbfa1800640 1 --2- 172.21.15.66:0/3608959024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf80004f50 0x7fbf8000dbe0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009f90 tx=0x7fbf9401c990 comp rx=0 tx=0).stop 2024-09-10T22:02:45.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.081+0000 7fbf9b400640 1 -- 172.21.15.66:0/3608959024 reap_dead start 2024-09-10T22:02:45.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.081+0000 7fbfa1800640 1 -- 172.21.15.66:0/3608959024 shutdown_connections 2024-09-10T22:02:45.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.081+0000 7fbfa1800640 1 -- 172.21.15.66:0/3608959024 >> 172.21.15.66:0/3608959024 conn(0x7fbf9c0d0310 msgr2=0x7fbf9c0d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:45.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.081+0000 7fbfa1800640 1 -- 172.21.15.66:0/3608959024 shutdown_connections 2024-09-10T22:02:45.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.085+0000 7fbfa1800640 1 -- 172.21.15.66:0/3608959024 wait complete. 2024-09-10T22:02:45.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.085+0000 7fbfa1800640 1 Processor -- start 2024-09-10T22:02:45.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.085+0000 7fbfa1800640 1 -- start start 2024-09-10T22:02:45.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbfa1800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf80004f50 0x7fbf9c099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:45.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbfa1800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c154460 0x7fbf9c0b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbfa1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf9c0ca0a0 0x7fbf9c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbfa1800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c0ca7a0 con 0x7fbf9c0ca0a0 2024-09-10T22:02:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbfa1800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c0ca620 con 0x7fbf9c154460 2024-09-10T22:02:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbfa1800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbf9c0d0e30 con 0x7fbf80004f50 2024-09-10T22:02:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf9b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf9c0ca0a0 0x7fbf9c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf9b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf9c0ca0a0 0x7fbf9c0b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34036/0 (socket says 172.21.15.66:34036) 2024-09-10T22:02:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf9b400640 1 -- 172.21.15.66:0/2208703036 learned_addr learned my addr 172.21.15.66:0/2208703036 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf9aa00640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf80004f50 0x7fbf9c099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf9b400640 1 -- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf80004f50 msgr2=0x7fbf9c099860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf9b400640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf80004f50 0x7fbf9c099860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf9b400640 1 -- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c154460 msgr2=0x7fbf9c0b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf9b400640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c154460 0x7fbf9c0b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf9b400640 1 -- 172.21.15.66:0/2208703036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9c0c3c50 con 0x7fbf9c0ca0a0 2024-09-10T22:02:45.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf9b400640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf9c0ca0a0 0x7fbf9c0b5c90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbf94002240 tx=0x7fbf9400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:45.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf92a00640 1 -- 172.21.15.66:0/2208703036 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf940b2040 con 0x7fbf9c0ca0a0 2024-09-10T22:02:45.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf92a00640 1 -- 172.21.15.66:0/2208703036 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf940a5790 con 0x7fbf9c0ca0a0 2024-09-10T22:02:45.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbfa1800640 1 -- 172.21.15.66:0/2208703036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9c154040 con 0x7fbf9c0ca0a0 2024-09-10T22:02:45.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbf92a00640 1 -- 172.21.15.66:0/2208703036 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94003be0 con 0x7fbf9c0ca0a0 2024-09-10T22:02:45.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbfa1800640 1 -- 172.21.15.66:0/2208703036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9c157350 con 0x7fbf9c0ca0a0 2024-09-10T22:02:45.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.089+0000 7fbfa1800640 1 -- 172.21.15.66:0/2208703036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf9c155f40 con 0x7fbf9c0ca0a0 2024-09-10T22:02:45.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.093+0000 7fbf92a00640 1 -- 172.21.15.66:0/2208703036 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 7) ==== 96598+0+0 (secure 0 0 0) 0x7fbf940b5070 con 0x7fbf9c0ca0a0 2024-09-10T22:02:45.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.093+0000 7fbf92a00640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbf7c072820 0x7fbf7c074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:45.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.093+0000 7fbf92a00640 1 -- 172.21.15.66:0/2208703036 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbf9412ce30 con 0x7fbf9c0ca0a0 2024-09-10T22:02:45.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.097+0000 7fbf9aa00640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbf7c072820 0x7fbf7c074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.101+0000 7fbf92a00640 1 -- 172.21.15.66:0/2208703036 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbf940166a0 con 0x7fbf9c0ca0a0 2024-09-10T22:02:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.101+0000 7fbf9aa00640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbf7c072820 0x7fbf7c074ce0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0036b0 tx=0x7fbf8c00c040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:45.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.417+0000 7fbfa1800640 1 -- 172.21.15.66:0/2208703036 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fbf9c0b61d0 con 0x7fbf7c072820 2024-09-10T22:02:45.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.605+0000 7fbf92a00640 1 -- 172.21.15.66:0/2208703036 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbf8c00abe0 con 0x7fbf7c072820 2024-09-10T22:02:45.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.609+0000 7fbfa1800640 1 -- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbf7c072820 msgr2=0x7fbf7c074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:45.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.609+0000 7fbfa1800640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbf7c072820 0x7fbf7c074ce0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0036b0 tx=0x7fbf8c00c040 comp rx=0 tx=0).stop 2024-09-10T22:02:45.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.609+0000 7fbfa1800640 1 -- 172.21.15.66:0/2208703036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf9c0ca0a0 msgr2=0x7fbf9c0b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.609+0000 7fbfa1800640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf9c0ca0a0 0x7fbf9c0b5c90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbf94002240 tx=0x7fbf9400c000 comp rx=0 tx=0).stop 2024-09-10T22:02:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.609+0000 7fbfa1800640 1 -- 172.21.15.66:0/2208703036 shutdown_connections 2024-09-10T22:02:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.609+0000 7fbfa1800640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbf7c072820 0x7fbf7c074ce0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.609+0000 7fbfa1800640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf9c0ca0a0 0x7fbf9c0b5c90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.609+0000 7fbfa1800640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c154460 0x7fbf9c0b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.609+0000 7fbfa1800640 1 --2- 172.21.15.66:0/2208703036 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf80004f50 0x7fbf9c099860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.609+0000 7fbfa1800640 1 -- 172.21.15.66:0/2208703036 >> 172.21.15.66:0/2208703036 conn(0x7fbf9c0d0310 msgr2=0x7fbf9c157730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.609+0000 7fbfa1800640 1 -- 172.21.15.66:0/2208703036 shutdown_connections 2024-09-10T22:02:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.609+0000 7fbfa1800640 1 -- 172.21.15.66:0/2208703036 wait complete. 2024-09-10T22:02:45.634 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:02:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.777+0000 7f082ac00640 1 -- 172.21.15.66:0/2597284650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f082415f340 msgr2=0x7f08240cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.777+0000 7f082ac00640 1 --2- 172.21.15.66:0/2597284650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f082415f340 0x7f08240cfb50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0814009900 tx=0x7f081401c550 comp rx=0 tx=0).stop 2024-09-10T22:02:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.777+0000 7f082ac00640 1 -- 172.21.15.66:0/2597284650 shutdown_connections 2024-09-10T22:02:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.777+0000 7f082ac00640 1 --2- 172.21.15.66:0/2597284650 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08240d2770 0x7f082415fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.777+0000 7f082ac00640 1 --2- 172.21.15.66:0/2597284650 >> v2:172.21.15.94:3300/0 conn(0x7f08240b2bf0 0x7f08240d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.777+0000 7f082ac00640 1 --2- 172.21.15.66:0/2597284650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f082415f340 0x7f08240cfb50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.777+0000 7f082ac00640 1 -- 172.21.15.66:0/2597284650 >> 172.21.15.66:0/2597284650 conn(0x7f08240d1c50 msgr2=0x7f08240d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:45.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.777+0000 7f082ac00640 1 -- 172.21.15.66:0/2597284650 shutdown_connections 2024-09-10T22:02:45.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.777+0000 7f082ac00640 1 -- 172.21.15.66:0/2597284650 wait complete. 2024-09-10T22:02:45.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.777+0000 7f082ac00640 1 Processor -- start 2024-09-10T22:02:45.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.777+0000 7f082ac00640 1 -- start start 2024-09-10T22:02:45.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f082ac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240b2bf0 0x7f0824150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:45.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f082ac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08240d2770 0x7f0824159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:45.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f082ac00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f082415f340 0x7f082415a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:45.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f082ac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f08240aecf0 con 0x7f08240d2770 2024-09-10T22:02:45.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f082ac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08240aeb70 con 0x7f08240b2bf0 2024-09-10T22:02:45.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f082ac00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f08240aee70 con 0x7f082415f340 2024-09-10T22:02:45.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f081be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08240d2770 0x7f0824159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:45.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f081be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08240d2770 0x7f0824159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34050/0 (socket says 172.21.15.66:34050) 2024-09-10T22:02:45.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f081be00640 1 -- 172.21.15.66:0/610429673 learned_addr learned my addr 172.21.15.66:0/610429673 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:45.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f0823e00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240b2bf0 0x7f0824150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:45.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f0828c00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f082415f340 0x7f082415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:45.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f081be00640 1 -- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f082415f340 msgr2=0x7f082415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:45.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f081be00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f082415f340 0x7f082415a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f081be00640 1 -- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240b2bf0 msgr2=0x7f0824150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:45.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f081be00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240b2bf0 0x7f0824150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f081be00640 1 -- 172.21.15.66:0/610429673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08241fc0c0 con 0x7f08240d2770 2024-09-10T22:02:45.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f081be00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08240d2770 0x7f0824159e20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f080c00d980 tx=0x7f080c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:45.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f0820c00640 1 -- 172.21.15.66:0/610429673 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c017070 con 0x7f08240d2770 2024-09-10T22:02:45.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f0820c00640 1 -- 172.21.15.66:0/610429673 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f080c009d20 con 0x7f08240d2770 2024-09-10T22:02:45.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f0820c00640 1 -- 172.21.15.66:0/610429673 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c0106c0 con 0x7f08240d2770 2024-09-10T22:02:45.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f082ac00640 1 -- 172.21.15.66:0/610429673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082406dd30 con 0x7f08240d2770 2024-09-10T22:02:45.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f082ac00640 1 -- 172.21.15.66:0/610429673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082406e2f0 con 0x7f08240d2770 2024-09-10T22:02:45.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f082ac00640 1 -- 172.21.15.66:0/610429673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f082415fac0 con 0x7f08240d2770 2024-09-10T22:02:45.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.781+0000 7f0820c00640 1 -- 172.21.15.66:0/610429673 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 7) ==== 96598+0+0 (secure 0 0 0) 0x7f080c010860 con 0x7f08240d2770 2024-09-10T22:02:45.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.785+0000 7f0820c00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0800072850 0x7f0800074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:45.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.785+0000 7f0820c00640 1 -- 172.21.15.66:0/610429673 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f080c093b50 con 0x7f08240d2770 2024-09-10T22:02:45.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.785+0000 7f0823e00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0800072850 0x7f0800074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:45.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.785+0000 7f0823e00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0800072850 0x7f0800074d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f08140af000 tx=0x7f081401ad90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:45.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.789+0000 7f0820c00640 1 -- 172.21.15.66:0/610429673 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f080c05e6d0 con 0x7f08240d2770 2024-09-10T22:02:45.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.989+0000 7f082ac00640 1 -- 172.21.15.66:0/610429673 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f08241516e0 con 0x7f0800072850 2024-09-10T22:02:45.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.989+0000 7f0820c00640 1 -- 172.21.15.66:0/610429673 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f08241516e0 con 0x7f0800072850 2024-09-10T22:02:45.996 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:02:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:02:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:02:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:02:45.997 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:02:45.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.993+0000 7f082ac00640 1 -- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0800072850 msgr2=0x7f0800074d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:45.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.993+0000 7f082ac00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0800072850 0x7f0800074d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f08140af000 tx=0x7f081401ad90 comp rx=0 tx=0).stop 2024-09-10T22:02:45.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.993+0000 7f082ac00640 1 -- 172.21.15.66:0/610429673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08240d2770 msgr2=0x7f0824159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:45.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.993+0000 7f082ac00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08240d2770 0x7f0824159e20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f080c00d980 tx=0x7f080c00de40 comp rx=0 tx=0).stop 2024-09-10T22:02:45.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.993+0000 7f082ac00640 1 -- 172.21.15.66:0/610429673 shutdown_connections 2024-09-10T22:02:45.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.993+0000 7f082ac00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0800072850 0x7f0800074d10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.993+0000 7f082ac00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f082415f340 0x7f082415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.993+0000 7f082ac00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08240d2770 0x7f0824159e20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.993+0000 7f082ac00640 1 --2- 172.21.15.66:0/610429673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240b2bf0 0x7f0824150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:45.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.993+0000 7f082ac00640 1 -- 172.21.15.66:0/610429673 >> 172.21.15.66:0/610429673 conn(0x7f08240d1c50 msgr2=0x7f0824151a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:45.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.993+0000 7f082ac00640 1 -- 172.21.15.66:0/610429673 shutdown_connections 2024-09-10T22:02:45.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:45.993+0000 7f082ac00640 1 -- 172.21.15.66:0/610429673 wait complete. 2024-09-10T22:02:46.017 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-10T22:02:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.161+0000 7f2fb5e00640 1 -- 172.21.15.66:0/376887998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fb00ce770 msgr2=0x7f2fb00ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.161+0000 7f2fb5e00640 1 --2- 172.21.15.66:0/376887998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fb00ce770 0x7f2fb00ccd40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0073c50 tx=0x7f2fa801f580 comp rx=0 tx=0).stop 2024-09-10T22:02:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.161+0000 7f2fb5e00640 1 -- 172.21.15.66:0/376887998 shutdown_connections 2024-09-10T22:02:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.161+0000 7f2fb5e00640 1 --2- 172.21.15.66:0/376887998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fb00ce770 0x7f2fb00ccd40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.161+0000 7f2fb5e00640 1 --2- 172.21.15.66:0/376887998 >> v2:172.21.15.94:3300/0 conn(0x7f2fb00cbb40 0x7f2fb00ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.161+0000 7f2fb5e00640 1 --2- 172.21.15.66:0/376887998 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2fb015ada0 0x7f2fb00caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.161+0000 7f2fb5e00640 1 -- 172.21.15.66:0/376887998 >> 172.21.15.66:0/376887998 conn(0x7f2fb0074240 msgr2=0x7f2fb0074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.161+0000 7f2fb5e00640 1 -- 172.21.15.66:0/376887998 shutdown_connections 2024-09-10T22:02:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.161+0000 7f2fb5e00640 1 -- 172.21.15.66:0/376887998 wait complete. 2024-09-10T22:02:46.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.161+0000 7f2fb5e00640 1 Processor -- start 2024-09-10T22:02:46.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.161+0000 7f2fb5e00640 1 -- start start 2024-09-10T22:02:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fb5e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2fb00cbb40 0x7f2fb01f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fb5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fb00ce770 0x7f2fb015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fb5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb015ada0 0x7f2fb015c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fb5e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb00cd610 con 0x7f2fb00ce770 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fb5e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb00cd490 con 0x7f2fb015ada0 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fb5e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb00cd790 con 0x7f2fb00cbb40 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fb00ce770 0x7f2fb015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fb00ce770 0x7f2fb015c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34074/0 (socket says 172.21.15.66:34074) 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7e00640 1 -- 172.21.15.66:0/1297461990 learned_addr learned my addr 172.21.15.66:0/1297461990 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2faea00640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2fb00cbb40 0x7f2fb01f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2faf400640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb015ada0 0x7f2fb015c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7e00640 1 -- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2fb00cbb40 msgr2=0x7f2fb01f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7e00640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2fb00cbb40 0x7f2fb01f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7e00640 1 -- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb015ada0 msgr2=0x7f2fb015c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7e00640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb015ada0 0x7f2fb015c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7e00640 1 -- 172.21.15.66:0/1297461990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb0068b20 con 0x7f2fb00ce770 2024-09-10T22:02:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2faf400640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb015ada0 0x7f2fb015c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7e00640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fb00ce770 0x7f2fb015c210 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800e9c0 tx=0x7f2f9800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7400640 1 -- 172.21.15.66:0/1297461990 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800ccc0 con 0x7f2fb00ce770 2024-09-10T22:02:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7400640 1 -- 172.21.15.66:0/1297461990 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f98004020 con 0x7f2fb00ce770 2024-09-10T22:02:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7400640 1 -- 172.21.15.66:0/1297461990 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98004470 con 0x7f2fb00ce770 2024-09-10T22:02:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fb5e00640 1 -- 172.21.15.66:0/1297461990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb015b450 con 0x7f2fb00ce770 2024-09-10T22:02:46.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fb5e00640 1 -- 172.21.15.66:0/1297461990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb015ba10 con 0x7f2fb00ce770 2024-09-10T22:02:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fb5e00640 1 -- 172.21.15.66:0/1297461990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb00ccd40 con 0x7f2fb00ce770 2024-09-10T22:02:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.165+0000 7f2fa7400640 1 -- 172.21.15.66:0/1297461990 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 7) ==== 96598+0+0 (secure 0 0 0) 0x7f2f98003600 con 0x7f2fb00ce770 2024-09-10T22:02:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.169+0000 7f2fa7400640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f8c072820 0x7f2f8c074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.169+0000 7f2fa7400640 1 -- 172.21.15.66:0/1297461990 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2f980941d0 con 0x7f2fb00ce770 2024-09-10T22:02:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.169+0000 7f2faea00640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f8c072820 0x7f2f8c074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.169+0000 7f2faea00640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f8c072820 0x7f2f8c074ce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009b40 tx=0x7f2fa00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:46.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.173+0000 7f2fa7400640 1 -- 172.21.15.66:0/1297461990 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2f9805f3a0 con 0x7f2fb00ce770 2024-09-10T22:02:46.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.373+0000 7f2fb5e00640 1 -- 172.21.15.66:0/1297461990 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2fb00af3b0 con 0x7f2f8c072820 2024-09-10T22:02:46.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fa7400640 1 -- 172.21.15.66:0/1297461990 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+44 (secure 0 0 0) 0x7f2fb00af3b0 con 0x7f2f8c072820 2024-09-10T22:02:46.381 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:02:46.381 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:02:46.381 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:02:46.381 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ss1clone1" 2024-09-10T22:02:46.381 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:02:46.382 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:02:46.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fb5e00640 1 -- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f8c072820 msgr2=0x7f2f8c074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fb5e00640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f8c072820 0x7f2f8c074ce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009b40 tx=0x7f2fa00091e0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fb5e00640 1 -- 172.21.15.66:0/1297461990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fb00ce770 msgr2=0x7f2fb015c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fb5e00640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fb00ce770 0x7f2fb015c210 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800e9c0 tx=0x7f2f9800ee80 comp rx=0 tx=0).stop 2024-09-10T22:02:46.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fb5e00640 1 -- 172.21.15.66:0/1297461990 shutdown_connections 2024-09-10T22:02:46.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fb5e00640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f8c072820 0x7f2f8c074ce0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fb5e00640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb015ada0 0x7f2fb015c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fb5e00640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fb00ce770 0x7f2fb015c210 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fb5e00640 1 --2- 172.21.15.66:0/1297461990 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2fb00cbb40 0x7f2fb01f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fb5e00640 1 -- 172.21.15.66:0/1297461990 >> 172.21.15.66:0/1297461990 conn(0x7f2fb0074240 msgr2=0x7f2fb006eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:46.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fb5e00640 1 -- 172.21.15.66:0/1297461990 shutdown_connections 2024-09-10T22:02:46.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.377+0000 7f2fb5e00640 1 -- 172.21.15.66:0/1297461990 wait complete. 2024-09-10T22:02:46.402 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-10T22:02:46.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.545+0000 7f5a21400640 1 -- 172.21.15.66:0/1965886124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a00005240 msgr2=0x7f5a0000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.545+0000 7f5a21400640 1 --2- 172.21.15.66:0/1965886124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a00005240 0x7f5a0000b9d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400c640 tx=0x7f5a1401d1e0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.545+0000 7f5a21400640 1 -- 172.21.15.66:0/1965886124 shutdown_connections 2024-09-10T22:02:46.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.545+0000 7f5a21400640 1 --2- 172.21.15.66:0/1965886124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a00005240 0x7f5a0000b9d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a21400640 1 -- 172.21.15.66:0/1965886124 >> 172.21.15.66:0/1965886124 conn(0x7f5a1c0d1c60 msgr2=0x7f5a1c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:46.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a21400640 1 -- 172.21.15.66:0/1965886124 shutdown_connections 2024-09-10T22:02:46.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a21400640 1 -- 172.21.15.66:0/1965886124 wait complete. 2024-09-10T22:02:46.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a21400640 1 Processor -- start 2024-09-10T22:02:46.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a21400640 1 -- start start 2024-09-10T22:02:46.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a21400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a00005240 0x7f5a1c15d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:46.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a21400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a1c071cd0 0x7f5a1c150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:46.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a21400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a1c072110 0x7f5a1c151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:46.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a21400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c0cfbc0 con 0x7f5a1c071cd0 2024-09-10T22:02:46.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a21400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c0cfa40 con 0x7f5a00005240 2024-09-10T22:02:46.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a21400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1c0cfd40 con 0x7f5a1c072110 2024-09-10T22:02:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a13e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a1c071cd0 0x7f5a1c150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a13e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a1c071cd0 0x7f5a1c150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34096/0 (socket says 172.21.15.66:34096) 2024-09-10T22:02:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a13e00640 1 -- 172.21.15.66:0/3987767778 learned_addr learned my addr 172.21.15.66:0/3987767778 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a1b200640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a1c072110 0x7f5a1c151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a1a800640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a00005240 0x7f5a1c15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a13e00640 1 -- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a1c072110 msgr2=0x7f5a1c151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a13e00640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a1c072110 0x7f5a1c151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a13e00640 1 -- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a00005240 msgr2=0x7f5a1c15d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a13e00640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a00005240 0x7f5a1c15d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a13e00640 1 -- 172.21.15.66:0/3987767778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c15c8c0 con 0x7f5a1c071cd0 2024-09-10T22:02:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a1a800640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a00005240 0x7f5a1c15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:02:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a1b200640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a1c072110 0x7f5a1c151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:02:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.549+0000 7f5a13e00640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a1c071cd0 0x7f5a1c150ce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5a04003ed0 tx=0x7f5a04001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:46.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.553+0000 7f5a12a00640 1 -- 172.21.15.66:0/3987767778 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04015070 con 0x7f5a1c071cd0 2024-09-10T22:02:46.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.553+0000 7f5a12a00640 1 -- 172.21.15.66:0/3987767778 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a04011a00 con 0x7f5a1c071cd0 2024-09-10T22:02:46.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.553+0000 7f5a12a00640 1 -- 172.21.15.66:0/3987767778 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04011e50 con 0x7f5a1c071cd0 2024-09-10T22:02:46.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.553+0000 7f5a21400640 1 -- 172.21.15.66:0/3987767778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c15cba0 con 0x7f5a1c071cd0 2024-09-10T22:02:46.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.553+0000 7f5a21400640 1 -- 172.21.15.66:0/3987767778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c15d160 con 0x7f5a1c071cd0 2024-09-10T22:02:46.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.553+0000 7f5a21400640 1 -- 172.21.15.66:0/3987767778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1c0d02a0 con 0x7f5a1c071cd0 2024-09-10T22:02:46.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.553+0000 7f5a12a00640 1 -- 172.21.15.66:0/3987767778 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 7) ==== 96598+0+0 (secure 0 0 0) 0x7f5a040103c0 con 0x7f5a1c071cd0 2024-09-10T22:02:46.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.553+0000 7f5a12a00640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f59f80729c0 0x7f59f8074e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:46.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.553+0000 7f5a12a00640 1 -- 172.21.15.66:0/3987767778 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5a04094830 con 0x7f5a1c071cd0 2024-09-10T22:02:46.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.553+0000 7f5a1a800640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f59f80729c0 0x7f59f8074e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:46.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.557+0000 7f5a1a800640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f59f80729c0 0x7f59f8074e80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c005950 tx=0x7f5a0c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:46.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.557+0000 7f5a12a00640 1 -- 172.21.15.66:0/3987767778 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5a0405fa00 con 0x7f5a1c071cd0 2024-09-10T22:02:46.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:46.757+0000 7f5a21400640 1 -- 172.21.15.66:0/3987767778 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f5a1c070550 con 0x7f59f80729c0 2024-09-10T22:02:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.013+0000 7f5a12a00640 1 -- 172.21.15.66:0/3987767778 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5a1c070550 con 0x7f59f80729c0 2024-09-10T22:02:47.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.017+0000 7f5a21400640 1 -- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f59f80729c0 msgr2=0x7f59f8074e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:47.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.017+0000 7f5a21400640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f59f80729c0 0x7f59f8074e80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c005950 tx=0x7f5a0c0091e0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.017+0000 7f5a21400640 1 -- 172.21.15.66:0/3987767778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a1c071cd0 msgr2=0x7f5a1c150ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:47.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.017+0000 7f5a21400640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a1c071cd0 0x7f5a1c150ce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5a04003ed0 tx=0x7f5a04001810 comp rx=0 tx=0).stop 2024-09-10T22:02:47.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.017+0000 7f5a21400640 1 -- 172.21.15.66:0/3987767778 shutdown_connections 2024-09-10T22:02:47.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.017+0000 7f5a21400640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f59f80729c0 0x7f59f8074e80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.017+0000 7f5a21400640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a1c072110 0x7f5a1c151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.017+0000 7f5a21400640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a1c071cd0 0x7f5a1c150ce0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.017+0000 7f5a21400640 1 --2- 172.21.15.66:0/3987767778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a00005240 0x7f5a1c15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.017+0000 7f5a21400640 1 -- 172.21.15.66:0/3987767778 >> 172.21.15.66:0/3987767778 conn(0x7f5a1c0d1c60 msgr2=0x7f5a1c06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.017+0000 7f5a21400640 1 -- 172.21.15.66:0/3987767778 shutdown_connections 2024-09-10T22:02:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.017+0000 7f5a21400640 1 -- 172.21.15.66:0/3987767778 wait complete. 2024-09-10T22:02:47.023 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:47.019+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:47.023 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:47.019+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:47.023 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:47.019+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:47.023 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:47.019+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:47.023 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:47.019+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:47.039 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:47.035+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:47.039 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:47.035+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:47.039 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:47.035+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:47.040 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:47.035+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:47.040 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:02:47.035+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:02:47.044 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T22:02:47.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:02:47.104 INFO:teuthology.orchestra.run.smithi066.stdout:2 2024-09-10T22:02:47.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:02:47.111 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:02:47.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.257+0000 7fe87e200640 1 -- 172.21.15.66:0/1741972793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85c004810 msgr2=0x7fe85c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:47.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.257+0000 7fe87e200640 1 --2- 172.21.15.66:0/1741972793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85c004810 0x7fe85c004c90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe864009b40 tx=0x7fe86401d270 comp rx=0 tx=0).stop 2024-09-10T22:02:47.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.257+0000 7fe877e00640 1 -- 172.21.15.66:0/1741972793 reap_dead start 2024-09-10T22:02:47.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.257+0000 7fe87e200640 1 -- 172.21.15.66:0/1741972793 shutdown_connections 2024-09-10T22:02:47.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.257+0000 7fe87e200640 1 -- 172.21.15.66:0/1741972793 >> 172.21.15.66:0/1741972793 conn(0x7fe8780d1c50 msgr2=0x7fe8780d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:47.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.257+0000 7fe87e200640 1 -- 172.21.15.66:0/1741972793 shutdown_connections 2024-09-10T22:02:47.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.257+0000 7fe87e200640 1 -- 172.21.15.66:0/1741972793 wait complete. 2024-09-10T22:02:47.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe87e200640 1 Processor -- start 2024-09-10T22:02:47.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe87e200640 1 -- start start 2024-09-10T22:02:47.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe87e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe85c004810 0x7fe878158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:47.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe87e200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe878155fd0 0x7fe8780c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:47.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe87e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe87806dd30 0x7fe87806e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:47.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe87e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe878168190 con 0x7fe85c004810 2024-09-10T22:02:47.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe87e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe878168010 con 0x7fe87806dd30 2024-09-10T22:02:47.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe87e200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe878168310 con 0x7fe878155fd0 2024-09-10T22:02:47.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe877400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe85c004810 0x7fe878158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:47.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe877400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe85c004810 0x7fe878158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34108/0 (socket says 172.21.15.66:34108) 2024-09-10T22:02:47.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe877400640 1 -- 172.21.15.66:0/2096190986 learned_addr learned my addr 172.21.15.66:0/2096190986 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:47.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe876a00640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe878155fd0 0x7fe8780c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:47.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe877e00640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe87806dd30 0x7fe87806e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:47.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe877400640 1 -- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe878155fd0 msgr2=0x7fe8780c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:47.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe877400640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe878155fd0 0x7fe8780c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe877400640 1 -- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe87806dd30 msgr2=0x7fe87806e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:47.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe877400640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe87806dd30 0x7fe87806e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe877400640 1 -- 172.21.15.66:0/2096190986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8781670e0 con 0x7fe85c004810 2024-09-10T22:02:47.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe877400640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe85c004810 0x7fe878158ae0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe868003e40 tx=0x7fe868001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:47.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe86f400640 1 -- 172.21.15.66:0/2096190986 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868015070 con 0x7fe85c004810 2024-09-10T22:02:47.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe86f400640 1 -- 172.21.15.66:0/2096190986 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8680119c0 con 0x7fe85c004810 2024-09-10T22:02:47.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe86f400640 1 -- 172.21.15.66:0/2096190986 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868011d70 con 0x7fe85c004810 2024-09-10T22:02:47.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe87e200640 1 -- 172.21.15.66:0/2096190986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe878155d30 con 0x7fe85c004810 2024-09-10T22:02:47.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.261+0000 7fe87e200640 1 -- 172.21.15.66:0/2096190986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe878167640 con 0x7fe85c004810 2024-09-10T22:02:47.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.265+0000 7fe87e200640 1 -- 172.21.15.66:0/2096190986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8780aeea0 con 0x7fe85c004810 2024-09-10T22:02:47.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.265+0000 7fe86f400640 1 -- 172.21.15.66:0/2096190986 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 7) ==== 96598+0+0 (secure 0 0 0) 0x7fe8680103c0 con 0x7fe85c004810 2024-09-10T22:02:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.265+0000 7fe86f400640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe858072850 0x7fe858074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.265+0000 7fe86f400640 1 -- 172.21.15.66:0/2096190986 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe8680947c0 con 0x7fe85c004810 2024-09-10T22:02:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.265+0000 7fe876a00640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe858072850 0x7fe858074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:47.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.265+0000 7fe876a00640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe858072850 0x7fe858074d10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe8640ad000 tx=0x7fe86401a010 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:47.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.269+0000 7fe86f400640 1 -- 172.21.15.66:0/2096190986 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe8781670e0 con 0x7fe85c004810 2024-09-10T22:02:47.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.441+0000 7fe87e200640 1 -- 172.21.15.66:0/2096190986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fe878050f90 con 0x7fe85c004810 2024-09-10T22:02:47.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.441+0000 7fe86f400640 1 -- 172.21.15.66:0/2096190986 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v29)=0 v29) ==== 131+0+0 (secure 0 0 0) 0x7fe86805f3d0 con 0x7fe85c004810 2024-09-10T22:02:47.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.445+0000 7fe87e200640 1 -- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe858072850 msgr2=0x7fe858074d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:47.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.445+0000 7fe87e200640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe858072850 0x7fe858074d10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe8640ad000 tx=0x7fe86401a010 comp rx=0 tx=0).stop 2024-09-10T22:02:47.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.445+0000 7fe87e200640 1 -- 172.21.15.66:0/2096190986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe85c004810 msgr2=0x7fe878158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.445+0000 7fe87e200640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe85c004810 0x7fe878158ae0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe868003e40 tx=0x7fe868001820 comp rx=0 tx=0).stop 2024-09-10T22:02:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.445+0000 7fe87e200640 1 -- 172.21.15.66:0/2096190986 shutdown_connections 2024-09-10T22:02:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.445+0000 7fe87e200640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe858072850 0x7fe858074d10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.445+0000 7fe87e200640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe87806dd30 0x7fe87806e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.445+0000 7fe87e200640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe878155fd0 0x7fe8780c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.445+0000 7fe87e200640 1 --2- 172.21.15.66:0/2096190986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe85c004810 0x7fe878158ae0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.445+0000 7fe87e200640 1 -- 172.21.15.66:0/2096190986 >> 172.21.15.66:0/2096190986 conn(0x7fe8780d1c50 msgr2=0x7fe8780b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.445+0000 7fe87e200640 1 -- 172.21.15.66:0/2096190986 shutdown_connections 2024-09-10T22:02:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.445+0000 7fe87e200640 1 -- 172.21.15.66:0/2096190986 wait complete. 2024-09-10T22:02:47.469 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:02:47.470 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:02:47.547 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:02:47.635 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:02:47.636 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:02:47.636 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:02:47.636 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:02:47.644 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:02:47.645 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:02:47.696 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:02:47.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.881+0000 7fbf16200640 1 -- 172.21.15.66:0/29624119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf100afc20 msgr2=0x7fbf100ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:47.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.881+0000 7fbf16200640 1 --2- 172.21.15.66:0/29624119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf100afc20 0x7fbf100ce6b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009bc0 tx=0x7fbf0801cb10 comp rx=0 tx=0).stop 2024-09-10T22:02:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.881+0000 7fbf16200640 1 -- 172.21.15.66:0/29624119 shutdown_connections 2024-09-10T22:02:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.881+0000 7fbf16200640 1 --2- 172.21.15.66:0/29624119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf100afc20 0x7fbf100ce6b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.881+0000 7fbf16200640 1 --2- 172.21.15.66:0/29624119 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf100d06b0 0x7fbf100ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.881+0000 7fbf16200640 1 -- 172.21.15.66:0/29624119 >> 172.21.15.66:0/29624119 conn(0x7fbf100d1c60 msgr2=0x7fbf100d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.881+0000 7fbf16200640 1 -- 172.21.15.66:0/29624119 shutdown_connections 2024-09-10T22:02:47.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.881+0000 7fbf16200640 1 -- 172.21.15.66:0/29624119 wait complete. 2024-09-10T22:02:47.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.881+0000 7fbf16200640 1 Processor -- start 2024-09-10T22:02:47.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf16200640 1 -- start start 2024-09-10T22:02:47.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf16200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf100afc20 0x7fbf10150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:47.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf16200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf100d06b0 0x7fbf10159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:47.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf16200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf1015bd40 0x7fbf100b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:47.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf16200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10169250 con 0x7fbf100d06b0 2024-09-10T22:02:47.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf16200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbf101690d0 con 0x7fbf100afc20 2024-09-10T22:02:47.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf16200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbf101693d0 con 0x7fbf1015bd40 2024-09-10T22:02:47.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf0ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf100d06b0 0x7fbf10159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:47.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf0ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf100d06b0 0x7fbf10159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34120/0 (socket says 172.21.15.66:34120) 2024-09-10T22:02:47.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf0ea00640 1 -- 172.21.15.66:0/3109786267 learned_addr learned my addr 172.21.15.66:0/3109786267 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:47.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf0f400640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf100afc20 0x7fbf10150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:47.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf0fe00640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf1015bd40 0x7fbf100b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:47.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf0ea00640 1 -- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf1015bd40 msgr2=0x7fbf100b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:47.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf0ea00640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf1015bd40 0x7fbf100b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf0ea00640 1 -- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf100afc20 msgr2=0x7fbf10150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:47.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf0ea00640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf100afc20 0x7fbf10150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:47.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf0ea00640 1 -- 172.21.15.66:0/3109786267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf100b4a90 con 0x7fbf100d06b0 2024-09-10T22:02:47.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf0ea00640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf100d06b0 0x7fbf10159800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00e9c0 tx=0x7fbefc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:47.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf07400640 1 -- 172.21.15.66:0/3109786267 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc00cd80 con 0x7fbf100d06b0 2024-09-10T22:02:47.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf07400640 1 -- 172.21.15.66:0/3109786267 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbefc004020 con 0x7fbf100d06b0 2024-09-10T22:02:47.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf07400640 1 -- 172.21.15.66:0/3109786267 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc004450 con 0x7fbf100d06b0 2024-09-10T22:02:47.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf16200640 1 -- 172.21.15.66:0/3109786267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf101fb490 con 0x7fbf100d06b0 2024-09-10T22:02:47.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf16200640 1 -- 172.21.15.66:0/3109786267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf101fb730 con 0x7fbf100d06b0 2024-09-10T22:02:47.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.885+0000 7fbf16200640 1 -- 172.21.15.66:0/3109786267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf100cffa0 con 0x7fbf100d06b0 2024-09-10T22:02:47.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.889+0000 7fbf07400640 1 -- 172.21.15.66:0/3109786267 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 7) ==== 96598+0+0 (secure 0 0 0) 0x7fbefc003600 con 0x7fbf100d06b0 2024-09-10T22:02:47.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.889+0000 7fbf07400640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbef0072820 0x7fbef0074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:47.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.889+0000 7fbf07400640 1 -- 172.21.15.66:0/3109786267 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbefc0941f0 con 0x7fbf100d06b0 2024-09-10T22:02:47.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.889+0000 7fbf0f400640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbef0072820 0x7fbef0074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:47.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.889+0000 7fbf0f400640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbef0072820 0x7fbef0074ce0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf00001870 tx=0x7fbf00001380 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:47.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:47.893+0000 7fbf07400640 1 -- 172.21.15.66:0/3109786267 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbefc05ed70 con 0x7fbf100d06b0 2024-09-10T22:02:48.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.077+0000 7fbf16200640 1 -- 172.21.15.66:0/3109786267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbf10150f70 con 0x7fbf100d06b0 2024-09-10T22:02:48.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.077+0000 7fbf07400640 1 -- 172.21.15.66:0/3109786267 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7fbefc0637a0 con 0x7fbf100d06b0 2024-09-10T22:02:48.084 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:02:48.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.081+0000 7fbf16200640 1 -- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbef0072820 msgr2=0x7fbef0074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:48.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.081+0000 7fbf16200640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbef0072820 0x7fbef0074ce0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf00001870 tx=0x7fbf00001380 comp rx=0 tx=0).stop 2024-09-10T22:02:48.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.081+0000 7fbf16200640 1 -- 172.21.15.66:0/3109786267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf100d06b0 msgr2=0x7fbf10159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:48.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.081+0000 7fbf16200640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf100d06b0 0x7fbf10159800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00e9c0 tx=0x7fbefc00ee80 comp rx=0 tx=0).stop 2024-09-10T22:02:48.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.081+0000 7fbf16200640 1 -- 172.21.15.66:0/3109786267 shutdown_connections 2024-09-10T22:02:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.081+0000 7fbf16200640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbef0072820 0x7fbef0074ce0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.081+0000 7fbf16200640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbf1015bd40 0x7fbf100b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.081+0000 7fbf16200640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf100d06b0 0x7fbf10159800 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.081+0000 7fbf16200640 1 --2- 172.21.15.66:0/3109786267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf100afc20 0x7fbf10150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.081+0000 7fbf16200640 1 -- 172.21.15.66:0/3109786267 >> 172.21.15.66:0/3109786267 conn(0x7fbf100d1c60 msgr2=0x7fbf1006e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.081+0000 7fbf16200640 1 -- 172.21.15.66:0/3109786267 shutdown_connections 2024-09-10T22:02:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:48.081+0000 7fbf16200640 1 -- 172.21.15.66:0/3109786267 wait complete. 2024-09-10T22:02:48.105 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-09-10T22:02:48.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:02:48.105 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T22:02:48.124 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[37214]: fuse finished with error 0 and tester_r 0 2024-09-10T22:02:50.171 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:02:50.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:02:50.171 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:02:50.180 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:02:50.180 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:02:50.180 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:02:50.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:02:50.189 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:02:50.190 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:02:50.190 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:02:50.228 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:02:50.228 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:02:50.229 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:02:50.229 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:02:50.240 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:02:50.241 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:02:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.413+0000 7fa347c00640 1 -- 172.21.15.66:0/2143341298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa34015e570 msgr2=0x7fa3400cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.413+0000 7fa347c00640 1 --2- 172.21.15.66:0/2143341298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa34015e570 0x7fa3400cdf00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa338009c80 tx=0x7fa33801ccb0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.413+0000 7fa347c00640 1 -- 172.21.15.66:0/2143341298 shutdown_connections 2024-09-10T22:02:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.413+0000 7fa347c00640 1 --2- 172.21.15.66:0/2143341298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa34015e570 0x7fa3400cdf00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.413+0000 7fa347c00640 1 --2- 172.21.15.66:0/2143341298 >> v2:172.21.15.94:3300/0 conn(0x7fa3400ced10 0x7fa3400cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.413+0000 7fa347c00640 1 -- 172.21.15.66:0/2143341298 >> 172.21.15.66:0/2143341298 conn(0x7fa3400d0330 msgr2=0x7fa3400d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.413+0000 7fa347c00640 1 -- 172.21.15.66:0/2143341298 shutdown_connections 2024-09-10T22:02:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.413+0000 7fa347c00640 1 -- 172.21.15.66:0/2143341298 wait complete. 2024-09-10T22:02:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.413+0000 7fa347c00640 1 Processor -- start 2024-09-10T22:02:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.413+0000 7fa347c00640 1 -- start start 2024-09-10T22:02:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa347c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3400ced10 0x7fa340157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa347c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa34015e190 0x7fa3400aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa347c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34015e570 0x7fa340154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa347c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3400ccbc0 con 0x7fa34015e570 2024-09-10T22:02:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa347c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3400cca40 con 0x7fa34015e190 2024-09-10T22:02:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa347c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa3400cc4f0 con 0x7fa3400ced10 2024-09-10T22:02:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa345c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34015e570 0x7fa340154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa33fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa34015e190 0x7fa3400aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa345c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34015e570 0x7fa340154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34124/0 (socket says 172.21.15.66:34124) 2024-09-10T22:02:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa345c00640 1 -- 172.21.15.66:0/1680179958 learned_addr learned my addr 172.21.15.66:0/1680179958 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa345200640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3400ced10 0x7fa340157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa345c00640 1 -- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3400ced10 msgr2=0x7fa340157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa345c00640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3400ced10 0x7fa340157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa345c00640 1 -- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa34015e190 msgr2=0x7fa3400aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa345c00640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa34015e190 0x7fa3400aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa345c00640 1 -- 172.21.15.66:0/1680179958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa340165670 con 0x7fa34015e570 2024-09-10T22:02:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa33fe00640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa34015e190 0x7fa3400aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:02:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa345c00640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34015e570 0x7fa340154530 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa33801c720 tx=0x7fa338005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa33cc00640 1 -- 172.21.15.66:0/1680179958 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3380b2040 con 0x7fa34015e570 2024-09-10T22:02:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa33cc00640 1 -- 172.21.15.66:0/1680179958 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa338004040 con 0x7fa34015e570 2024-09-10T22:02:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa33cc00640 1 -- 172.21.15.66:0/1680179958 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3380043f0 con 0x7fa34015e570 2024-09-10T22:02:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa347c00640 1 -- 172.21.15.66:0/1680179958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3400ae300 con 0x7fa34015e570 2024-09-10T22:02:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa347c00640 1 -- 172.21.15.66:0/1680179958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3400ae8c0 con 0x7fa34015e570 2024-09-10T22:02:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.417+0000 7fa347c00640 1 -- 172.21.15.66:0/1680179958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3400cdf00 con 0x7fa34015e570 2024-09-10T22:02:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.421+0000 7fa33cc00640 1 -- 172.21.15.66:0/1680179958 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fa3380b5070 con 0x7fa34015e570 2024-09-10T22:02:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.421+0000 7fa33cc00640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa320072970 0x7fa320074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:50.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.421+0000 7fa33cc00640 1 -- 172.21.15.66:0/1680179958 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa33812d770 con 0x7fa34015e570 2024-09-10T22:02:50.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.421+0000 7fa345200640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa320072970 0x7fa320074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:50.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.421+0000 7fa345200640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa320072970 0x7fa320074e30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa334000f70 tx=0x7fa334009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:50.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.425+0000 7fa33cc00640 1 -- 172.21.15.66:0/1680179958 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa3380c4020 con 0x7fa34015e570 2024-09-10T22:02:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.633+0000 7fa347c00640 1 -- 172.21.15.66:0/1680179958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3400751c0 con 0x7fa34015e570 2024-09-10T22:02:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa33cc00640 1 -- 172.21.15.66:0/1680179958 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5217 (secure 0 0 0) 0x7fa338016990 con 0x7fa34015e570 2024-09-10T22:02:50.642 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:02:50.642 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":18,"btime":"2024-09-10T22:01:26:894150+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1019192175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1019192175},{"type":"v1","addr":"172.21.15.94:6837","nonce":1019192175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/554383263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":554383263},{"type":"v1","addr":"172.21.15.94:6836","nonce":554383263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2368842390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2368842390},{"type":"v1","addr":"172.21.15.66:6834","nonce":2368842390}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:01:18.244077+0000","modified":"2024-09-10T22:01:21.271158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/297228676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":297228676},{"type":"v1","addr":"172.21.15.66:6835","nonce":297228676}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-09-10T22:02:50.642 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 18 2024-09-10T22:02:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa347c00640 1 -- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa320072970 msgr2=0x7fa320074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa347c00640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa320072970 0x7fa320074e30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa334000f70 tx=0x7fa334009040 comp rx=0 tx=0).stop 2024-09-10T22:02:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa347c00640 1 -- 172.21.15.66:0/1680179958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34015e570 msgr2=0x7fa340154530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa347c00640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34015e570 0x7fa340154530 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa33801c720 tx=0x7fa338005c20 comp rx=0 tx=0).stop 2024-09-10T22:02:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa347c00640 1 -- 172.21.15.66:0/1680179958 shutdown_connections 2024-09-10T22:02:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa347c00640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa320072970 0x7fa320074e30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa347c00640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34015e570 0x7fa340154530 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa347c00640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa34015e190 0x7fa3400aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa347c00640 1 --2- 172.21.15.66:0/1680179958 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3400ced10 0x7fa340157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa347c00640 1 -- 172.21.15.66:0/1680179958 >> 172.21.15.66:0/1680179958 conn(0x7fa3400d0330 msgr2=0x7fa3401551e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa347c00640 1 -- 172.21.15.66:0/1680179958 shutdown_connections 2024-09-10T22:02:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.637+0000 7fa347c00640 1 -- 172.21.15.66:0/1680179958 wait complete. 2024-09-10T22:02:50.665 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:02:50.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 -- 172.21.15.66:0/293960127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0074004810 msgr2=0x7f0074004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 --2- 172.21.15.66:0/293960127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0074004810 0x7f0074004c90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0078001860 tx=0x7f007801c8e0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f008f400640 1 -- 172.21.15.66:0/293960127 reap_dead start 2024-09-10T22:02:50.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 -- 172.21.15.66:0/293960127 shutdown_connections 2024-09-10T22:02:50.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 -- 172.21.15.66:0/293960127 >> 172.21.15.66:0/293960127 conn(0x7f00900cfa30 msgr2=0x7f00900cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:50.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 -- 172.21.15.66:0/293960127 shutdown_connections 2024-09-10T22:02:50.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 -- 172.21.15.66:0/293960127 wait complete. 2024-09-10T22:02:50.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 Processor -- start 2024-09-10T22:02:50.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 -- start start 2024-09-10T22:02:50.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0074004810 0x7f00901f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00900c6b00 0x7f00901f91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00901f9710 0x7f00901f9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0090168250 con 0x7f0074004810 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00901680d0 con 0x7f00900c6b00 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0095800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f00901683d0 con 0x7f00901f9710 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f008ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0074004810 0x7f00901f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f008ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0074004810 0x7f00901f8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34144/0 (socket says 172.21.15.66:34144) 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f008ea00640 1 -- 172.21.15.66:0/670828996 learned_addr learned my addr 172.21.15.66:0/670828996 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0087e00640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00900c6b00 0x7f00901f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f008f400640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00901f9710 0x7f00901f9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f008ea00640 1 -- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00901f9710 msgr2=0x7f00901f9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f008ea00640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00901f9710 0x7f00901f9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f008ea00640 1 -- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00900c6b00 msgr2=0x7f00901f91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f008ea00640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00900c6b00 0x7f00901f91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f008ea00640 1 -- 172.21.15.66:0/670828996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00900c7070 con 0x7f0074004810 2024-09-10T22:02:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f0087e00640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00900c6b00 0x7f00901f91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.809+0000 7f008f400640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00901f9710 0x7f00901f9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:02:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.813+0000 7f008ea00640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0074004810 0x7f00901f8900 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f00800119b0 tx=0x7f0080011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.813+0000 7f0086a00640 1 -- 172.21.15.66:0/670828996 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008000fdb0 con 0x7f0074004810 2024-09-10T22:02:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.813+0000 7f0086a00640 1 -- 172.21.15.66:0/670828996 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0080004030 con 0x7f0074004810 2024-09-10T22:02:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.813+0000 7f0095800640 1 -- 172.21.15.66:0/670828996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00900c9460 con 0x7f0074004810 2024-09-10T22:02:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.813+0000 7f0086a00640 1 -- 172.21.15.66:0/670828996 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080004480 con 0x7f0074004810 2024-09-10T22:02:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.813+0000 7f0095800640 1 -- 172.21.15.66:0/670828996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00900c9a20 con 0x7f0074004810 2024-09-10T22:02:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.813+0000 7f0095800640 1 -- 172.21.15.66:0/670828996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00900cd930 con 0x7f0074004810 2024-09-10T22:02:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.813+0000 7f0086a00640 1 -- 172.21.15.66:0/670828996 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f0080003600 con 0x7f0074004810 2024-09-10T22:02:50.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.813+0000 7f0086a00640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f006c072970 0x7f006c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:50.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.813+0000 7f0086a00640 1 -- 172.21.15.66:0/670828996 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f00800945c0 con 0x7f0074004810 2024-09-10T22:02:50.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.813+0000 7f0087e00640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f006c072970 0x7f006c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.817+0000 7f0087e00640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f006c072970 0x7f006c074e30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0078001830 tx=0x7f0078019720 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:50.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:50.817+0000 7f0086a00640 1 -- 172.21.15.66:0/670828996 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f008005f7d0 con 0x7f0074004810 2024-09-10T22:02:51.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0095800640 1 -- 172.21.15.66:0/670828996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00900b0490 con 0x7f0074004810 2024-09-10T22:02:51.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0086a00640 1 -- 172.21.15.66:0/670828996 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5217 (secure 0 0 0) 0x7f008005f0d0 con 0x7f0074004810 2024-09-10T22:02:51.047 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:02:51.047 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":18,"btime":"2024-09-10T22:01:26:894150+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1019192175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1019192175},{"type":"v1","addr":"172.21.15.94:6837","nonce":1019192175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/554383263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":554383263},{"type":"v1","addr":"172.21.15.94:6836","nonce":554383263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2368842390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2368842390},{"type":"v1","addr":"172.21.15.66:6834","nonce":2368842390}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:01:18.244077+0000","modified":"2024-09-10T22:01:21.271158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/297228676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":297228676},{"type":"v1","addr":"172.21.15.66:6835","nonce":297228676}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-09-10T22:02:51.047 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 18 2024-09-10T22:02:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0095800640 1 -- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f006c072970 msgr2=0x7f006c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0095800640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f006c072970 0x7f006c074e30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0078001830 tx=0x7f0078019720 comp rx=0 tx=0).stop 2024-09-10T22:02:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0095800640 1 -- 172.21.15.66:0/670828996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0074004810 msgr2=0x7f00901f8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0095800640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0074004810 0x7f00901f8900 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f00800119b0 tx=0x7f0080011e70 comp rx=0 tx=0).stop 2024-09-10T22:02:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0095800640 1 -- 172.21.15.66:0/670828996 shutdown_connections 2024-09-10T22:02:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0095800640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f006c072970 0x7f006c074e30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0095800640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00901f9710 0x7f00901f9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0095800640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00900c6b00 0x7f00901f91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0095800640 1 --2- 172.21.15.66:0/670828996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0074004810 0x7f00901f8900 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0095800640 1 -- 172.21.15.66:0/670828996 >> 172.21.15.66:0/670828996 conn(0x7f00900cfa30 msgr2=0x7f00900b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.041+0000 7f0095800640 1 -- 172.21.15.66:0/670828996 shutdown_connections 2024-09-10T22:02:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.045+0000 7f0095800640 1 -- 172.21.15.66:0/670828996 wait complete. 2024-09-10T22:02:51.070 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:02:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 -- 172.21.15.66:0/4123735227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8815cc80 msgr2=0x7f3e880cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 --2- 172.21.15.66:0/4123735227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8815cc80 0x7f3e880cd930 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3e88072560 tx=0x7f3e8001ccf0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 -- 172.21.15.66:0/4123735227 shutdown_connections 2024-09-10T22:02:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 --2- 172.21.15.66:0/4123735227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8815cc80 0x7f3e880cd930 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 --2- 172.21.15.66:0/4123735227 >> v2:172.21.15.94:3300/0 conn(0x7f3e880aee20 0x7f3e880ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 -- 172.21.15.66:0/4123735227 >> 172.21.15.66:0/4123735227 conn(0x7f3e880cfa30 msgr2=0x7f3e880cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 -- 172.21.15.66:0/4123735227 shutdown_connections 2024-09-10T22:02:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 -- 172.21.15.66:0/4123735227 wait complete. 2024-09-10T22:02:51.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 Processor -- start 2024-09-10T22:02:51.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 -- start start 2024-09-10T22:02:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e880aee20 0x7f3e881fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e88159900 0x7f3e881f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e8815cc80 0x7f3e88073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e88168f70 con 0x7f3e880aee20 2024-09-10T22:02:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e88168df0 con 0x7f3e88159900 2024-09-10T22:02:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e8dc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3e881690f0 con 0x7f3e8815cc80 2024-09-10T22:02:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e86a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e880aee20 0x7f3e881fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e86a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e880aee20 0x7f3e881fbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34166/0 (socket says 172.21.15.66:34166) 2024-09-10T22:02:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e86a00640 1 -- 172.21.15.66:0/2101664671 learned_addr learned my addr 172.21.15.66:0/2101664671 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e7fe00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e88159900 0x7f3e881f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.213+0000 7f3e87400640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e8815cc80 0x7f3e88073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e86a00640 1 -- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e8815cc80 msgr2=0x7f3e88073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e86a00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e8815cc80 0x7f3e88073ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e86a00640 1 -- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e88159900 msgr2=0x7f3e881f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e86a00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e88159900 0x7f3e881f89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e86a00640 1 -- 172.21.15.66:0/2101664671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e88073160 con 0x7f3e880aee20 2024-09-10T22:02:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e86a00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e880aee20 0x7f3e881fbf40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800c530 tx=0x7f3e7800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e7ea00640 1 -- 172.21.15.66:0/2101664671 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78017070 con 0x7f3e880aee20 2024-09-10T22:02:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e7ea00640 1 -- 172.21.15.66:0/2101664671 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e780053c0 con 0x7f3e880aee20 2024-09-10T22:02:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e7ea00640 1 -- 172.21.15.66:0/2101664671 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78012890 con 0x7f3e880aee20 2024-09-10T22:02:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e8dc00640 1 -- 172.21.15.66:0/2101664671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e88073440 con 0x7f3e880aee20 2024-09-10T22:02:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e8dc00640 1 -- 172.21.15.66:0/2101664671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e881f8480 con 0x7f3e880aee20 2024-09-10T22:02:51.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e8dc00640 1 -- 172.21.15.66:0/2101664671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e881f8680 con 0x7f3e880aee20 2024-09-10T22:02:51.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e7ea00640 1 -- 172.21.15.66:0/2101664671 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f3e78003600 con 0x7f3e880aee20 2024-09-10T22:02:51.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e7ea00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3e64072940 0x7f3e64074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:51.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.217+0000 7f3e7ea00640 1 -- 172.21.15.66:0/2101664671 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3e78092d70 con 0x7f3e880aee20 2024-09-10T22:02:51.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.221+0000 7f3e7fe00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3e64072940 0x7f3e64074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:51.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.221+0000 7f3e7fe00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3e64072940 0x7f3e64074e00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e8815f0f0 tx=0x7f3e700091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:51.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.221+0000 7f3e7ea00640 1 -- 172.21.15.66:0/2101664671 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3e881f8680 con 0x7f3e880aee20 2024-09-10T22:02:51.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.397+0000 7f3e8dc00640 1 -- 172.21.15.66:0/2101664671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e880c9ce0 con 0x7f3e880aee20 2024-09-10T22:02:51.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.397+0000 7f3e7ea00640 1 -- 172.21.15.66:0/2101664671 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16657 (secure 0 0 0) 0x7f3e7805d930 con 0x7f3e880aee20 2024-09-10T22:02:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:02:51.404 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":33,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:01:19.860933+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:01:16.499954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T22:01:17.208498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:02:51.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.401+0000 7f3e8dc00640 1 -- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3e64072940 msgr2=0x7f3e64074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.401+0000 7f3e8dc00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3e64072940 0x7f3e64074e00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e8815f0f0 tx=0x7f3e700091e0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.401+0000 7f3e8dc00640 1 -- 172.21.15.66:0/2101664671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e880aee20 msgr2=0x7f3e881fbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.401+0000 7f3e8dc00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e880aee20 0x7f3e881fbf40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800c530 tx=0x7f3e7800c9f0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.401+0000 7f3e8dc00640 1 -- 172.21.15.66:0/2101664671 shutdown_connections 2024-09-10T22:02:51.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.401+0000 7f3e8dc00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3e64072940 0x7f3e64074e00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.401+0000 7f3e8dc00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e8815cc80 0x7f3e88073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.401+0000 7f3e8dc00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e88159900 0x7f3e881f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.401+0000 7f3e8dc00640 1 --2- 172.21.15.66:0/2101664671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e880aee20 0x7f3e881fbf40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.401+0000 7f3e8dc00640 1 -- 172.21.15.66:0/2101664671 >> 172.21.15.66:0/2101664671 conn(0x7f3e880cfa30 msgr2=0x7f3e88151790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:51.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.401+0000 7f3e8dc00640 1 -- 172.21.15.66:0/2101664671 shutdown_connections 2024-09-10T22:02:51.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.401+0000 7f3e8dc00640 1 -- 172.21.15.66:0/2101664671 wait complete. 2024-09-10T22:02:51.427 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:02:51.427 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:02:51.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.569+0000 7fa028800640 1 -- 172.21.15.66:0/1558375261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0200c4fc0 msgr2=0x7fa0200ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.569+0000 7fa028800640 1 --2- 172.21.15.66:0/1558375261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0200c4fc0 0x7fa0200ca8b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009bf0 tx=0x7fa01c01ca70 comp rx=0 tx=0).stop 2024-09-10T22:02:51.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.569+0000 7fa028800640 1 -- 172.21.15.66:0/1558375261 shutdown_connections 2024-09-10T22:02:51.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.569+0000 7fa028800640 1 --2- 172.21.15.66:0/1558375261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0200c4fc0 0x7fa0200ca8b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.569+0000 7fa028800640 1 -- 172.21.15.66:0/1558375261 >> 172.21.15.66:0/1558375261 conn(0x7fa0200cfa30 msgr2=0x7fa0200cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:51.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.569+0000 7fa028800640 1 -- 172.21.15.66:0/1558375261 shutdown_connections 2024-09-10T22:02:51.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.569+0000 7fa028800640 1 -- 172.21.15.66:0/1558375261 wait complete. 2024-09-10T22:02:51.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.569+0000 7fa028800640 1 Processor -- start 2024-09-10T22:02:51.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.569+0000 7fa028800640 1 -- start start 2024-09-10T22:02:51.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa028800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0200c4fc0 0x7fa02014eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:51.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa028800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa02015a310 0x7fa0200ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:51.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa028800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa02014fbc0 0x7fa0200ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:51.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa028800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0201686a0 con 0x7fa0200c4fc0 2024-09-10T22:02:51.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa028800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa020168520 con 0x7fa02014fbc0 2024-09-10T22:02:51.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa028800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa020168820 con 0x7fa02015a310 2024-09-10T22:02:51.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa025e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0200c4fc0 0x7fa02014eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:51.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa025e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0200c4fc0 0x7fa02014eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34170/0 (socket says 172.21.15.66:34170) 2024-09-10T22:02:51.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa025e00640 1 -- 172.21.15.66:0/1358542787 learned_addr learned my addr 172.21.15.66:0/1358542787 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:51.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa025400640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa02015a310 0x7fa0200ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:51.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa026800640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa02014fbc0 0x7fa0200ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:51.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa025e00640 1 -- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa02015a310 msgr2=0x7fa0200ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa025e00640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa02015a310 0x7fa0200ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa025e00640 1 -- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa02014fbc0 msgr2=0x7fa0200ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa025e00640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa02014fbc0 0x7fa0200ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa025e00640 1 -- 172.21.15.66:0/1358542787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02014f970 con 0x7fa0200c4fc0 2024-09-10T22:02:51.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa026800640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa02014fbc0 0x7fa0200ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:02:51.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa025e00640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0200c4fc0 0x7fa02014eec0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa01400c420 tx=0x7fa01400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:51.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa01a000640 1 -- 172.21.15.66:0/1358542787 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014017070 con 0x7fa0200c4fc0 2024-09-10T22:02:51.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa01a000640 1 -- 172.21.15.66:0/1358542787 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0140053c0 con 0x7fa0200c4fc0 2024-09-10T22:02:51.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa01a000640 1 -- 172.21.15.66:0/1358542787 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014003860 con 0x7fa0200c4fc0 2024-09-10T22:02:51.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa028800640 1 -- 172.21.15.66:0/1358542787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa02014f690 con 0x7fa0200c4fc0 2024-09-10T22:02:51.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa028800640 1 -- 172.21.15.66:0/1358542787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0200b0940 con 0x7fa0200c4fc0 2024-09-10T22:02:51.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa028800640 1 -- 172.21.15.66:0/1358542787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0200c48b0 con 0x7fa0200c4fc0 2024-09-10T22:02:51.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.573+0000 7fa01a000640 1 -- 172.21.15.66:0/1358542787 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fa014003a00 con 0x7fa0200c4fc0 2024-09-10T22:02:51.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.577+0000 7fa01a000640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa004072970 0x7fa004074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:51.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.577+0000 7fa01a000640 1 -- 172.21.15.66:0/1358542787 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa014092f90 con 0x7fa0200c4fc0 2024-09-10T22:02:51.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.577+0000 7fa025400640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa004072970 0x7fa004074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:51.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.577+0000 7fa025400640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa004072970 0x7fa004074e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa020071010 tx=0x7fa010009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:51.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.581+0000 7fa01a000640 1 -- 172.21.15.66:0/1358542787 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa01405e110 con 0x7fa0200c4fc0 2024-09-10T22:02:51.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.789+0000 7fa028800640 1 -- 172.21.15.66:0/1358542787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa020070420 con 0x7fa0200c4fc0 2024-09-10T22:02:51.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.789+0000 7fa01a000640 1 -- 172.21.15.66:0/1358542787 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5217 (secure 0 0 0) 0x7fa01405da10 con 0x7fa0200c4fc0 2024-09-10T22:02:51.795 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:02:51.795 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":18,"btime":"2024-09-10T22:01:26:894150+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1019192175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1019192175},{"type":"v1","addr":"172.21.15.94:6837","nonce":1019192175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/554383263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":554383263},{"type":"v1","addr":"172.21.15.94:6836","nonce":554383263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2368842390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2368842390},{"type":"v1","addr":"172.21.15.66:6834","nonce":2368842390}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:01:18.244077+0000","modified":"2024-09-10T22:01:21.271158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/297228676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":297228676},{"type":"v1","addr":"172.21.15.66:6835","nonce":297228676}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-09-10T22:02:51.795 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 18 2024-09-10T22:02:51.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.793+0000 7fa028800640 1 -- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa004072970 msgr2=0x7fa004074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.793+0000 7fa028800640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa004072970 0x7fa004074e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa020071010 tx=0x7fa010009040 comp rx=0 tx=0).stop 2024-09-10T22:02:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.793+0000 7fa028800640 1 -- 172.21.15.66:0/1358542787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0200c4fc0 msgr2=0x7fa02014eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.793+0000 7fa028800640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0200c4fc0 0x7fa02014eec0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa01400c420 tx=0x7fa01400c8e0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.793+0000 7fa028800640 1 -- 172.21.15.66:0/1358542787 shutdown_connections 2024-09-10T22:02:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.793+0000 7fa028800640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa004072970 0x7fa004074e30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.793+0000 7fa028800640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa02014fbc0 0x7fa0200ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.793+0000 7fa028800640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa02015a310 0x7fa0200ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.793+0000 7fa028800640 1 --2- 172.21.15.66:0/1358542787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0200c4fc0 0x7fa02014eec0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.793+0000 7fa028800640 1 -- 172.21.15.66:0/1358542787 >> 172.21.15.66:0/1358542787 conn(0x7fa0200cfa30 msgr2=0x7fa0200cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.793+0000 7fa028800640 1 -- 172.21.15.66:0/1358542787 shutdown_connections 2024-09-10T22:02:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.793+0000 7fa028800640 1 -- 172.21.15.66:0/1358542787 wait complete. 2024-09-10T22:02:51.817 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:02:51.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.957+0000 7f6c5de00640 1 -- 172.21.15.66:0/1599852346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c58070860 msgr2=0x7f6c580cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.957+0000 7f6c5de00640 1 --2- 172.21.15.66:0/1599852346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c58070860 0x7f6c580cce40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6c58072d00 tx=0x7f6c5001cb80 comp rx=0 tx=0).stop 2024-09-10T22:02:51.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.957+0000 7f6c5de00640 1 -- 172.21.15.66:0/1599852346 shutdown_connections 2024-09-10T22:02:51.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.957+0000 7f6c5de00640 1 --2- 172.21.15.66:0/1599852346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c58070860 0x7f6c580cce40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.957+0000 7f6c5de00640 1 --2- 172.21.15.66:0/1599852346 >> v2:172.21.15.94:3300/0 conn(0x7f6c5806d330 0x7f6c5815f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.957+0000 7f6c5de00640 1 -- 172.21.15.66:0/1599852346 >> 172.21.15.66:0/1599852346 conn(0x7f6c580d0b30 msgr2=0x7f6c580d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:51.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.957+0000 7f6c5de00640 1 -- 172.21.15.66:0/1599852346 shutdown_connections 2024-09-10T22:02:51.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.957+0000 7f6c5de00640 1 -- 172.21.15.66:0/1599852346 wait complete. 2024-09-10T22:02:51.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.957+0000 7f6c5de00640 1 Processor -- start 2024-09-10T22:02:51.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.957+0000 7f6c5de00640 1 -- start start 2024-09-10T22:02:51.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c5de00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6c5806d330 0x7f6c580c4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:51.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c5de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c58070860 0x7f6c5814d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:51.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c5de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c58071a50 0x7f6c580c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:51.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c5de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58168dc0 con 0x7f6c58071a50 2024-09-10T22:02:51.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c5de00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58168c40 con 0x7f6c58070860 2024-09-10T22:02:51.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c5de00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6c58168f40 con 0x7f6c5806d330 2024-09-10T22:02:51.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c57400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c58071a50 0x7f6c580c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c57400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c58071a50 0x7f6c580c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34192/0 (socket says 172.21.15.66:34192) 2024-09-10T22:02:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c57400640 1 -- 172.21.15.66:0/3978426348 learned_addr learned my addr 172.21.15.66:0/3978426348 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c56a00640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6c5806d330 0x7f6c580c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c56000640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c58070860 0x7f6c5814d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c57400640 1 -- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6c5806d330 msgr2=0x7f6c580c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c57400640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6c5806d330 0x7f6c580c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c57400640 1 -- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c58070860 msgr2=0x7f6c5814d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c57400640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c58070860 0x7f6c5814d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c57400640 1 -- 172.21.15.66:0/3978426348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c580c6180 con 0x7f6c58071a50 2024-09-10T22:02:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c57400640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c58071a50 0x7f6c580c7c00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6c5001a7b0 tx=0x7f6c500a59f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c4f400640 1 -- 172.21.15.66:0/3978426348 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c500bb070 con 0x7f6c58071a50 2024-09-10T22:02:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c4f400640 1 -- 172.21.15.66:0/3978426348 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c50004e00 con 0x7f6c58071a50 2024-09-10T22:02:51.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c5de00640 1 -- 172.21.15.66:0/3978426348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c580ca6e0 con 0x7f6c58071a50 2024-09-10T22:02:51.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c4f400640 1 -- 172.21.15.66:0/3978426348 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c500b2040 con 0x7f6c58071a50 2024-09-10T22:02:51.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c5de00640 1 -- 172.21.15.66:0/3978426348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c580c6420 con 0x7f6c58071a50 2024-09-10T22:02:51.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c5de00640 1 -- 172.21.15.66:0/3978426348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c580cac00 con 0x7f6c58071a50 2024-09-10T22:02:51.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.961+0000 7f6c4f400640 1 -- 172.21.15.66:0/3978426348 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f6c500b5030 con 0x7f6c58071a50 2024-09-10T22:02:51.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.965+0000 7f6c4f400640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6c38072940 0x7f6c38074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:51.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.965+0000 7f6c4f400640 1 -- 172.21.15.66:0/3978426348 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6c5012d7a0 con 0x7f6c58071a50 2024-09-10T22:02:51.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.965+0000 7f6c56a00640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6c38072940 0x7f6c38074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:51.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.965+0000 7f6c56a00640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6c38072940 0x7f6c38074e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6c48000f70 tx=0x7f6c48009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:51.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:51.969+0000 7f6c4f400640 1 -- 172.21.15.66:0/3978426348 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6c580cac00 con 0x7f6c58071a50 2024-09-10T22:02:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.185+0000 7f6c5de00640 1 -- 172.21.15.66:0/3978426348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c5815f690 con 0x7f6c58071a50 2024-09-10T22:02:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.185+0000 7f6c4f400640 1 -- 172.21.15.66:0/3978426348 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5217 (secure 0 0 0) 0x7f6c50016770 con 0x7f6c58071a50 2024-09-10T22:02:52.191 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:02:52.191 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":18,"btime":"2024-09-10T22:01:26:894150+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1019192175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1019192175},{"type":"v1","addr":"172.21.15.94:6837","nonce":1019192175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/554383263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":554383263},{"type":"v1","addr":"172.21.15.94:6836","nonce":554383263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2368842390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2368842390},{"type":"v1","addr":"172.21.15.66:6834","nonce":2368842390}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:01:18.244077+0000","modified":"2024-09-10T22:01:21.271158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/297228676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":297228676},{"type":"v1","addr":"172.21.15.66:6835","nonce":297228676}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-09-10T22:02:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 18 2024-09-10T22:02:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.189+0000 7f6c5de00640 1 -- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6c38072940 msgr2=0x7f6c38074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.189+0000 7f6c5de00640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6c38072940 0x7f6c38074e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6c48000f70 tx=0x7f6c48009040 comp rx=0 tx=0).stop 2024-09-10T22:02:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.189+0000 7f6c5de00640 1 -- 172.21.15.66:0/3978426348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c58071a50 msgr2=0x7f6c580c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.189+0000 7f6c5de00640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c58071a50 0x7f6c580c7c00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6c5001a7b0 tx=0x7f6c500a59f0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.189+0000 7f6c5de00640 1 -- 172.21.15.66:0/3978426348 shutdown_connections 2024-09-10T22:02:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.189+0000 7f6c5de00640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6c38072940 0x7f6c38074e00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.189+0000 7f6c5de00640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c58071a50 0x7f6c580c7c00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.189+0000 7f6c5de00640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c58070860 0x7f6c5814d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.189+0000 7f6c5de00640 1 --2- 172.21.15.66:0/3978426348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6c5806d330 0x7f6c580c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.189+0000 7f6c5de00640 1 -- 172.21.15.66:0/3978426348 >> 172.21.15.66:0/3978426348 conn(0x7f6c580d0b30 msgr2=0x7f6c580d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.189+0000 7f6c5de00640 1 -- 172.21.15.66:0/3978426348 shutdown_connections 2024-09-10T22:02:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.189+0000 7f6c5de00640 1 -- 172.21.15.66:0/3978426348 wait complete. 2024-09-10T22:02:52.212 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:02:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.353+0000 7f8de0400640 1 -- 172.21.15.66:0/1109531071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd815bd40 msgr2=0x7f8dd80cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.353+0000 7f8de0400640 1 --2- 172.21.15.66:0/1109531071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd815bd40 0x7f8dd80cffa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009900 tx=0x7f8dcc01c850 comp rx=0 tx=0).stop 2024-09-10T22:02:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.353+0000 7f8de0400640 1 -- 172.21.15.66:0/1109531071 shutdown_connections 2024-09-10T22:02:52.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.353+0000 7f8de0400640 1 --2- 172.21.15.66:0/1109531071 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd80cf180 0x7f8dd80d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.353+0000 7f8de0400640 1 --2- 172.21.15.66:0/1109531071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd815bd40 0x7f8dd80cffa0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.353+0000 7f8de0400640 1 -- 172.21.15.66:0/1109531071 >> 172.21.15.66:0/1109531071 conn(0x7f8dd80d1c60 msgr2=0x7f8dd80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:52.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.353+0000 7f8de0400640 1 -- 172.21.15.66:0/1109531071 shutdown_connections 2024-09-10T22:02:52.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.353+0000 7f8de0400640 1 -- 172.21.15.66:0/1109531071 wait complete. 2024-09-10T22:02:52.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.353+0000 7f8de0400640 1 Processor -- start 2024-09-10T22:02:52.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.353+0000 7f8de0400640 1 -- start start 2024-09-10T22:02:52.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8de0400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd80cf180 0x7f8dd815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:52.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8de0400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd80cf830 0x7f8dd81572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:52.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8de0400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd815bd40 0x7f8dd81fab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:52.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8de0400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd8168ba0 con 0x7f8dd80cf830 2024-09-10T22:02:52.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8de0400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd8168a20 con 0x7f8dd815bd40 2024-09-10T22:02:52.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8de0400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd8168d20 con 0x7f8dd80cf180 2024-09-10T22:02:52.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8ddd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd80cf830 0x7f8dd81572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:52.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8ddd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd80cf830 0x7f8dd81572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57946/0 (socket says 172.21.15.66:57946) 2024-09-10T22:02:52.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8ddd000640 1 -- 172.21.15.66:0/3361855800 learned_addr learned my addr 172.21.15.66:0/3361855800 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:52.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8ddd000640 1 -- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd80cf180 msgr2=0x7f8dd815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8ddda00640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd80cf180 0x7f8dd815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:52.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8dde400640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd815bd40 0x7f8dd81fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:52.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8ddd000640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd80cf180 0x7f8dd815ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8ddd000640 1 -- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd815bd40 msgr2=0x7f8dd81fab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8ddd000640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd815bd40 0x7f8dd81fab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8ddd000640 1 -- 172.21.15.66:0/3361855800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd80b4c30 con 0x7f8dd80cf830 2024-09-10T22:02:52.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8ddda00640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd80cf180 0x7f8dd815ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:02:52.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8dde400640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd815bd40 0x7f8dd81fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:02:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8ddd000640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd80cf830 0x7f8dd81572d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c530 tx=0x7f8dc400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8dd5600640 1 -- 172.21.15.66:0/3361855800 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4017070 con 0x7f8dd80cf830 2024-09-10T22:02:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8dd5600640 1 -- 172.21.15.66:0/3361855800 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dc4003600 con 0x7f8dd80cf830 2024-09-10T22:02:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8dd5600640 1 -- 172.21.15.66:0/3361855800 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4003a50 con 0x7f8dd80cf830 2024-09-10T22:02:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8de0400640 1 -- 172.21.15.66:0/3361855800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd80b4f10 con 0x7f8dd80cf830 2024-09-10T22:02:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8de0400640 1 -- 172.21.15.66:0/3361855800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd80b54d0 con 0x7f8dd80cf830 2024-09-10T22:02:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.357+0000 7f8de0400640 1 -- 172.21.15.66:0/3361855800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd80aec10 con 0x7f8dd80cf830 2024-09-10T22:02:52.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.361+0000 7f8dd5600640 1 -- 172.21.15.66:0/3361855800 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f8dc4003bf0 con 0x7f8dd80cf830 2024-09-10T22:02:52.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.361+0000 7f8dd5600640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8db8072940 0x7f8db8074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:52.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.361+0000 7f8dd5600640 1 -- 172.21.15.66:0/3361855800 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8dc4092f30 con 0x7f8dd80cf830 2024-09-10T22:02:52.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.361+0000 7f8ddda00640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8db8072940 0x7f8db8074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:52.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.361+0000 7f8ddda00640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8db8072940 0x7f8db8074e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0af000 tx=0x7f8dcc01b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:52.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.365+0000 7f8dd5600640 1 -- 172.21.15.66:0/3361855800 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8dc405e200 con 0x7f8dd80cf830 2024-09-10T22:02:52.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.541+0000 7f8de0400640 1 -- 172.21.15.66:0/3361855800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8dd80b0df0 con 0x7f8dd80cf830 2024-09-10T22:02:52.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.541+0000 7f8dd5600640 1 -- 172.21.15.66:0/3361855800 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16657 (secure 0 0 0) 0x7f8dc405db00 con 0x7f8dd80cf830 2024-09-10T22:02:52.548 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:02:52.548 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":33,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:01:19.860933+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:01:16.499954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T22:01:17.208498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:02:52.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.545+0000 7f8de0400640 1 -- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8db8072940 msgr2=0x7f8db8074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.545+0000 7f8de0400640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8db8072940 0x7f8db8074e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0af000 tx=0x7f8dcc01b090 comp rx=0 tx=0).stop 2024-09-10T22:02:52.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.545+0000 7f8de0400640 1 -- 172.21.15.66:0/3361855800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd80cf830 msgr2=0x7f8dd81572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.545+0000 7f8de0400640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd80cf830 0x7f8dd81572d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c530 tx=0x7f8dc400c9f0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.545+0000 7f8de0400640 1 -- 172.21.15.66:0/3361855800 shutdown_connections 2024-09-10T22:02:52.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.545+0000 7f8de0400640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8db8072940 0x7f8db8074e00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.545+0000 7f8de0400640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd815bd40 0x7f8dd81fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.545+0000 7f8de0400640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd80cf830 0x7f8dd81572d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.545+0000 7f8de0400640 1 --2- 172.21.15.66:0/3361855800 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd80cf180 0x7f8dd815ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.545+0000 7f8de0400640 1 -- 172.21.15.66:0/3361855800 >> 172.21.15.66:0/3361855800 conn(0x7f8dd80d1c60 msgr2=0x7f8dd806dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:52.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.545+0000 7f8de0400640 1 -- 172.21.15.66:0/3361855800 shutdown_connections 2024-09-10T22:02:52.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.545+0000 7f8de0400640 1 -- 172.21.15.66:0/3361855800 wait complete. 2024-09-10T22:02:52.570 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:02:52.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.709+0000 7f3317e00640 1 -- 172.21.15.66:0/3622283671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100d0550 msgr2=0x7f33100c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.709+0000 7f3317e00640 1 --2- 172.21.15.66:0/3622283671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100d0550 0x7f33100c4df0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f331006c4f0 tx=0x7f32fc01c830 comp rx=0 tx=0).stop 2024-09-10T22:02:52.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.709+0000 7f3317e00640 1 -- 172.21.15.66:0/3622283671 shutdown_connections 2024-09-10T22:02:52.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.709+0000 7f3317e00640 1 --2- 172.21.15.66:0/3622283671 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33100aeec0 0x7f331015df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.709+0000 7f3317e00640 1 --2- 172.21.15.66:0/3622283671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100d0550 0x7f33100c4df0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.709+0000 7f3317e00640 1 --2- 172.21.15.66:0/3622283671 >> v2:172.21.15.94:3300/0 conn(0x7f331015a310 0x7f33100c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.709+0000 7f3317e00640 1 -- 172.21.15.66:0/3622283671 >> 172.21.15.66:0/3622283671 conn(0x7f33100cfa30 msgr2=0x7f33100cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.709+0000 7f3317e00640 1 -- 172.21.15.66:0/3622283671 shutdown_connections 2024-09-10T22:02:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.709+0000 7f3317e00640 1 -- 172.21.15.66:0/3622283671 wait complete. 2024-09-10T22:02:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3317e00640 1 Processor -- start 2024-09-10T22:02:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3317e00640 1 -- start start 2024-09-10T22:02:52.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3317e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100aeec0 0x7f331015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:52.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3317e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f331015a310 0x7f33100b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:52.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3317e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33100b55a0 0x7f33101f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:52.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3317e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f331015ec10 con 0x7f33100aeec0 2024-09-10T22:02:52.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3317e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f331015ea90 con 0x7f331015a310 2024-09-10T22:02:52.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3317e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f331015ed90 con 0x7f33100b55a0 2024-09-10T22:02:52.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3315400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100aeec0 0x7f331015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:52.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3315400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100aeec0 0x7f331015a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57962/0 (socket says 172.21.15.66:57962) 2024-09-10T22:02:52.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3315400640 1 -- 172.21.15.66:0/87932532 learned_addr learned my addr 172.21.15.66:0/87932532 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:52.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3314a00640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f331015a310 0x7f33100b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:52.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3315e00640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33100b55a0 0x7f33101f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3315400640 1 -- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33100b55a0 msgr2=0x7f33101f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3315400640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33100b55a0 0x7f33101f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3315400640 1 -- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f331015a310 msgr2=0x7f33100b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3315400640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f331015a310 0x7f33100b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3315400640 1 -- 172.21.15.66:0/87932532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331015aee0 con 0x7f33100aeec0 2024-09-10T22:02:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3315400640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100aeec0 0x7f331015a9a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f330400e7a0 tx=0x7f330400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f330d600640 1 -- 172.21.15.66:0/87932532 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330400cc60 con 0x7f33100aeec0 2024-09-10T22:02:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f330d600640 1 -- 172.21.15.66:0/87932532 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3304010430 con 0x7f33100aeec0 2024-09-10T22:02:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f330d600640 1 -- 172.21.15.66:0/87932532 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304010860 con 0x7f33100aeec0 2024-09-10T22:02:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.713+0000 7f3317e00640 1 -- 172.21.15.66:0/87932532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33100b2fe0 con 0x7f33100aeec0 2024-09-10T22:02:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.717+0000 7f3317e00640 1 -- 172.21.15.66:0/87932532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3310074210 con 0x7f33100aeec0 2024-09-10T22:02:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.717+0000 7f3317e00640 1 -- 172.21.15.66:0/87932532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f331015df60 con 0x7f33100aeec0 2024-09-10T22:02:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.717+0000 7f330d600640 1 -- 172.21.15.66:0/87932532 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f3304010a00 con 0x7f33100aeec0 2024-09-10T22:02:52.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.717+0000 7f330d600640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32f0072940 0x7f32f0074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:52.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.717+0000 7f330d600640 1 -- 172.21.15.66:0/87932532 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3304094290 con 0x7f33100aeec0 2024-09-10T22:02:52.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.717+0000 7f3314a00640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32f0072940 0x7f32f0074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.721+0000 7f3314a00640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32f0072940 0x7f32f0074e00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0ae000 tx=0x7f32fc01b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:52.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.721+0000 7f330d600640 1 -- 172.21.15.66:0/87932532 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f330405edc0 con 0x7f33100aeec0 2024-09-10T22:02:52.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.937+0000 7f3317e00640 1 -- 172.21.15.66:0/87932532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f33100b12e0 con 0x7f33100aeec0 2024-09-10T22:02:52.942 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:02:52.937+0000 7fcc93a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:02:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.937+0000 7f330d600640 1 -- 172.21.15.66:0/87932532 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f33040637f0 con 0x7f33100aeec0 2024-09-10T22:02:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:02:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.941+0000 7f3317e00640 1 -- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32f0072940 msgr2=0x7f32f0074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.941+0000 7f3317e00640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32f0072940 0x7f32f0074e00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0ae000 tx=0x7f32fc01b070 comp rx=0 tx=0).stop 2024-09-10T22:02:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.941+0000 7f3317e00640 1 -- 172.21.15.66:0/87932532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100aeec0 msgr2=0x7f331015a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.941+0000 7f3317e00640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100aeec0 0x7f331015a9a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f330400e7a0 tx=0x7f330400ec60 comp rx=0 tx=0).stop 2024-09-10T22:02:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.941+0000 7f3317e00640 1 -- 172.21.15.66:0/87932532 shutdown_connections 2024-09-10T22:02:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.941+0000 7f3317e00640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32f0072940 0x7f32f0074e00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.941+0000 7f3317e00640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33100b55a0 0x7f33101f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.941+0000 7f3317e00640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f331015a310 0x7f33100b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.941+0000 7f3317e00640 1 --2- 172.21.15.66:0/87932532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100aeec0 0x7f331015a9a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.941+0000 7f3317e00640 1 -- 172.21.15.66:0/87932532 >> 172.21.15.66:0/87932532 conn(0x7f33100cfa30 msgr2=0x7f33100afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:52.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.941+0000 7f3317e00640 1 -- 172.21.15.66:0/87932532 shutdown_connections 2024-09-10T22:02:52.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:52.941+0000 7f3317e00640 1 -- 172.21.15.66:0/87932532 wait complete. 2024-09-10T22:02:52.970 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:02:53.016 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:02:53.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.113+0000 7f9ae3200640 1 -- 172.21.15.66:0/4273979318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ae415ef20 msgr2=0x7f9ae40cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:53.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.113+0000 7f9ae3200640 1 --2- 172.21.15.66:0/4273979318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ae415ef20 0x7f9ae40cecc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009930 tx=0x7f9ad401c8e0 comp rx=0 tx=0).stop 2024-09-10T22:02:53.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 -- 172.21.15.66:0/4273979318 shutdown_connections 2024-09-10T22:02:53.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 --2- 172.21.15.66:0/4273979318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae40cd7a0 0x7f9ae40cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 --2- 172.21.15.66:0/4273979318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ae415ef20 0x7f9ae40cecc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 -- 172.21.15.66:0/4273979318 >> 172.21.15.66:0/4273979318 conn(0x7f9ae40d1b10 msgr2=0x7f9ae40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 -- 172.21.15.66:0/4273979318 shutdown_connections 2024-09-10T22:02:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 -- 172.21.15.66:0/4273979318 wait complete. 2024-09-10T22:02:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 Processor -- start 2024-09-10T22:02:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 -- start start 2024-09-10T22:02:53.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ae40cd7a0 0x7f9ae409b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:53.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ae415ef20 0x7f9ae40c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:53.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae415f300 0x7f9ae40ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:53.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae4168ee0 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae4168d60 con 0x7f9ae415ef20 2024-09-10T22:02:53.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae3200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae4169060 con 0x7f9ae415f300 2024-09-10T22:02:53.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ae40cd7a0 0x7f9ae409b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:53.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ae40cd7a0 0x7f9ae409b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58000/0 (socket says 172.21.15.66:58000) 2024-09-10T22:02:53.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae1e00640 1 -- 172.21.15.66:0/3748366555 learned_addr learned my addr 172.21.15.66:0/3748366555 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:53.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9adbe00640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ae415ef20 0x7f9ae40c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:53.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae2800640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae415f300 0x7f9ae40ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:53.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae1e00640 1 -- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae415f300 msgr2=0x7f9ae40ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:53.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae1e00640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae415f300 0x7f9ae40ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:53.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae1e00640 1 -- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ae415ef20 msgr2=0x7f9ae40c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae1e00640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ae415ef20 0x7f9ae40c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae1e00640 1 -- 172.21.15.66:0/3748366555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae40cc000 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae2800640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae415f300 0x7f9ae40ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.117+0000 7f9ae1e00640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ae40cd7a0 0x7f9ae409b980 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009850 tx=0x7f9ad40a5760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.121+0000 7f9ada000640 1 -- 172.21.15.66:0/3748366555 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40b2040 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.121+0000 7f9ada000640 1 -- 172.21.15.66:0/3748366555 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ad40053e0 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.121+0000 7f9ada000640 1 -- 172.21.15.66:0/3748366555 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40046d0 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.121+0000 7f9ae3200640 1 -- 172.21.15.66:0/3748366555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae409e540 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.121+0000 7f9ae3200640 1 -- 172.21.15.66:0/3748366555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae40c9330 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.121+0000 7f9ae3200640 1 -- 172.21.15.66:0/3748366555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae40cdc30 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.121+0000 7f9ada000640 1 -- 172.21.15.66:0/3748366555 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f9ae40d0100 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.121+0000 7f9ada000640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ac0072940 0x7f9ac0074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.121+0000 7f9ada000640 1 -- 172.21.15.66:0/3748366555 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f9ad412d520 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.121+0000 7f9adbe00640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ac0072940 0x7f9ac0074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.125+0000 7f9adbe00640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ac0072940 0x7f9ac0074e00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ae409d1a0 tx=0x7f9acc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.125+0000 7f9ada000640 1 -- 172.21.15.66:0/3748366555 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9ad40164d0 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.337+0000 7f9ae3200640 1 -- 172.21.15.66:0/3748366555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9ae4072e00 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.941+0000 7f9ada000640 1 -- 172.21.15.66:0/3748366555 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f9ad40f8710 con 0x7f9ae40cd7a0 2024-09-10T22:02:53.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.945+0000 7f9ae3200640 1 -- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ac0072940 msgr2=0x7f9ac0074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:53.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.945+0000 7f9ae3200640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ac0072940 0x7f9ac0074e00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ae409d1a0 tx=0x7f9acc009040 comp rx=0 tx=0).stop 2024-09-10T22:02:53.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.945+0000 7f9ae3200640 1 -- 172.21.15.66:0/3748366555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ae40cd7a0 msgr2=0x7f9ae409b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:53.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.945+0000 7f9ae3200640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ae40cd7a0 0x7f9ae409b980 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009850 tx=0x7f9ad40a5760 comp rx=0 tx=0).stop 2024-09-10T22:02:53.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.945+0000 7f9ae3200640 1 -- 172.21.15.66:0/3748366555 shutdown_connections 2024-09-10T22:02:53.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.945+0000 7f9ae3200640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ac0072940 0x7f9ac0074e00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:53.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.945+0000 7f9ae3200640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae415f300 0x7f9ae40ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:53.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.945+0000 7f9ae3200640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ae415ef20 0x7f9ae40c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:53.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.945+0000 7f9ae3200640 1 --2- 172.21.15.66:0/3748366555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ae40cd7a0 0x7f9ae409b980 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:53.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.945+0000 7f9ae3200640 1 -- 172.21.15.66:0/3748366555 >> 172.21.15.66:0/3748366555 conn(0x7f9ae40d1b10 msgr2=0x7f9ae4071e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:53.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.945+0000 7f9ae3200640 1 -- 172.21.15.66:0/3748366555 shutdown_connections 2024-09-10T22:02:53.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:53.945+0000 7f9ae3200640 1 -- 172.21.15.66:0/3748366555 wait complete. 2024-09-10T22:02:53.971 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:02:54.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.157+0000 7fe3afe00640 1 -- 172.21.15.66:0/277659680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a80cf250 msgr2=0x7fe3a815f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:54.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.157+0000 7fe3afe00640 1 --2- 172.21.15.66:0/277659680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a80cf250 0x7fe3a815f620 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80cd630 tx=0x7fe3a001ccc0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.157+0000 7fe3afe00640 1 -- 172.21.15.66:0/277659680 shutdown_connections 2024-09-10T22:02:54.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.157+0000 7fe3afe00640 1 --2- 172.21.15.66:0/277659680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a80cf250 0x7fe3a815f620 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.157+0000 7fe3afe00640 1 --2- 172.21.15.66:0/277659680 >> v2:172.21.15.94:3300/0 conn(0x7fe3a8145050 0x7fe3a806e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.157+0000 7fe3afe00640 1 -- 172.21.15.66:0/277659680 >> 172.21.15.66:0/277659680 conn(0x7fe3a80cfaa0 msgr2=0x7fe3a80cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:54.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.157+0000 7fe3afe00640 1 -- 172.21.15.66:0/277659680 shutdown_connections 2024-09-10T22:02:54.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.157+0000 7fe3afe00640 1 -- 172.21.15.66:0/277659680 wait complete. 2024-09-10T22:02:54.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3afe00640 1 Processor -- start 2024-09-10T22:02:54.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3afe00640 1 -- start start 2024-09-10T22:02:54.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3afe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a80cf250 0x7fe3a813e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:54.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3afe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a8144ac0 0x7fe3a8159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:54.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3afe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a8145050 0x7fe3a806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:54.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3afe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8168cb0 con 0x7fe3a8144ac0 2024-09-10T22:02:54.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3afe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8168b30 con 0x7fe3a80cf250 2024-09-10T22:02:54.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3afe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a8168e30 con 0x7fe3a8145050 2024-09-10T22:02:54.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3aca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a8144ac0 0x7fe3a8159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:54.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3aca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a8144ac0 0x7fe3a8159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58026/0 (socket says 172.21.15.66:58026) 2024-09-10T22:02:54.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3aca00640 1 -- 172.21.15.66:0/2505086770 learned_addr learned my addr 172.21.15.66:0/2505086770 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:54.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3ad400640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a80cf250 0x7fe3a813e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:54.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3ade00640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a8145050 0x7fe3a806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:54.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3aca00640 1 -- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a8145050 msgr2=0x7fe3a806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:54.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3aca00640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a8145050 0x7fe3a806eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3aca00640 1 -- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a80cf250 msgr2=0x7fe3a813e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:54.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3aca00640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a80cf250 0x7fe3a813e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3aca00640 1 -- 172.21.15.66:0/2505086770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a813eb70 con 0x7fe3a8144ac0 2024-09-10T22:02:54.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3aca00640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a8144ac0 0x7fe3a8159a80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe39400c9f0 tx=0x7fe39400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3a5600640 1 -- 172.21.15.66:0/2505086770 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394007bb0 con 0x7fe3a8144ac0 2024-09-10T22:02:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3a5600640 1 -- 172.21.15.66:0/2505086770 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe394007e40 con 0x7fe3a8144ac0 2024-09-10T22:02:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3afe00640 1 -- 172.21.15.66:0/2505086770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a815ccc0 con 0x7fe3a8144ac0 2024-09-10T22:02:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3a5600640 1 -- 172.21.15.66:0/2505086770 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39400f690 con 0x7fe3a8144ac0 2024-09-10T22:02:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3afe00640 1 -- 172.21.15.66:0/2505086770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a806d350 con 0x7fe3a8144ac0 2024-09-10T22:02:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.161+0000 7fe3afe00640 1 -- 172.21.15.66:0/2505086770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a815f620 con 0x7fe3a8144ac0 2024-09-10T22:02:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.165+0000 7fe3a5600640 1 -- 172.21.15.66:0/2505086770 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fe394020030 con 0x7fe3a8144ac0 2024-09-10T22:02:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.165+0000 7fe3a5600640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe388072940 0x7fe388074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:54.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.165+0000 7fe3a5600640 1 -- 172.21.15.66:0/2505086770 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fe394093e20 con 0x7fe3a8144ac0 2024-09-10T22:02:54.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.165+0000 7fe3ad400640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe388072940 0x7fe388074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:54.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.165+0000 7fe3ad400640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe388072940 0x7fe388074e00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe39c007fd0 tx=0x7fe39c007e40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:54.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.169+0000 7fe3a5600640 1 -- 172.21.15.66:0/2505086770 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe3a815ccc0 con 0x7fe3a8144ac0 2024-09-10T22:02:54.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.345+0000 7fe3afe00640 1 -- 172.21.15.66:0/2505086770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe3a8072b70 con 0x7fe3a8144ac0 2024-09-10T22:02:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.941+0000 7fe3a5600640 1 -- 172.21.15.66:0/2505086770 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7fe394014030 con 0x7fe3a8144ac0 2024-09-10T22:02:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:02:54.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.945+0000 7fe3afe00640 1 -- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe388072940 msgr2=0x7fe388074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:54.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.945+0000 7fe3afe00640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe388072940 0x7fe388074e00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe39c007fd0 tx=0x7fe39c007e40 comp rx=0 tx=0).stop 2024-09-10T22:02:54.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.945+0000 7fe3afe00640 1 -- 172.21.15.66:0/2505086770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a8144ac0 msgr2=0x7fe3a8159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:54.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.945+0000 7fe3afe00640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a8144ac0 0x7fe3a8159a80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe39400c9f0 tx=0x7fe39400ceb0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.945+0000 7fe3afe00640 1 -- 172.21.15.66:0/2505086770 shutdown_connections 2024-09-10T22:02:54.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.945+0000 7fe3afe00640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe388072940 0x7fe388074e00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.945+0000 7fe3afe00640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a8145050 0x7fe3a806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.945+0000 7fe3afe00640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a8144ac0 0x7fe3a8159a80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.945+0000 7fe3afe00640 1 --2- 172.21.15.66:0/2505086770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a80cf250 0x7fe3a813e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.945+0000 7fe3afe00640 1 -- 172.21.15.66:0/2505086770 >> 172.21.15.66:0/2505086770 conn(0x7fe3a80cfaa0 msgr2=0x7fe3a80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:54.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.945+0000 7fe3afe00640 1 -- 172.21.15.66:0/2505086770 shutdown_connections 2024-09-10T22:02:54.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:54.945+0000 7fe3afe00640 1 -- 172.21.15.66:0/2505086770 wait complete. 2024-09-10T22:02:54.976 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:02:55.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.121+0000 7f8af3e00640 1 -- 172.21.15.66:0/2170021292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aec15f450 msgr2=0x7f8aec0b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:55.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.121+0000 7f8af3e00640 1 --2- 172.21.15.66:0/2170021292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aec15f450 0x7f8aec0b3170 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009840 tx=0x7f8ae001d1c0 comp rx=0 tx=0).stop 2024-09-10T22:02:55.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.121+0000 7f8af3e00640 1 -- 172.21.15.66:0/2170021292 shutdown_connections 2024-09-10T22:02:55.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.121+0000 7f8af3e00640 1 --2- 172.21.15.66:0/2170021292 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aec160280 0x7f8aec0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:55.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.121+0000 7f8af3e00640 1 --2- 172.21.15.66:0/2170021292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aec15f450 0x7f8aec0b3170 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:55.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.121+0000 7f8af3e00640 1 -- 172.21.15.66:0/2170021292 >> 172.21.15.66:0/2170021292 conn(0x7f8aec0d1c60 msgr2=0x7f8aec0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:55.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.121+0000 7f8af3e00640 1 -- 172.21.15.66:0/2170021292 shutdown_connections 2024-09-10T22:02:55.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.121+0000 7f8af3e00640 1 -- 172.21.15.66:0/2170021292 wait complete. 2024-09-10T22:02:55.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af3e00640 1 Processor -- start 2024-09-10T22:02:55.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af3e00640 1 -- start start 2024-09-10T22:02:55.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aec0b1e10 0x7f8aec156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:55.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aec15f450 0x7f8aec159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aec160280 0x7f8aec15a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec0aff30 con 0x7f8aec0b1e10 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec0afdb0 con 0x7f8aec160280 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af3e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8aec168eb0 con 0x7f8aec15f450 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aec0b1e10 0x7f8aec156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aec0b1e10 0x7f8aec156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58046/0 (socket says 172.21.15.66:58046) 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af1400640 1 -- 172.21.15.66:0/2625309558 learned_addr learned my addr 172.21.15.66:0/2625309558 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af0a00640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aec15f450 0x7f8aec159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af1e00640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aec160280 0x7f8aec15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af1400640 1 -- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aec15f450 msgr2=0x7f8aec159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af1400640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aec15f450 0x7f8aec159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af1400640 1 -- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aec160280 msgr2=0x7f8aec15a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:55.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af1400640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aec160280 0x7f8aec15a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:55.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af1400640 1 -- 172.21.15.66:0/2625309558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec1574c0 con 0x7f8aec0b1e10 2024-09-10T22:02:55.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af0a00640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aec15f450 0x7f8aec159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:55.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af1e00640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aec160280 0x7f8aec15a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:55.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af1400640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aec0b1e10 0x7f8aec156f80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009790 tx=0x7f8ae00a56e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:55.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8ae9600640 1 -- 172.21.15.66:0/2625309558 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00b2040 con 0x7f8aec0b1e10 2024-09-10T22:02:55.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8ae9600640 1 -- 172.21.15.66:0/2625309558 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ae0005020 con 0x7f8aec0b1e10 2024-09-10T22:02:55.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8ae9600640 1 -- 172.21.15.66:0/2625309558 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00044f0 con 0x7f8aec0b1e10 2024-09-10T22:02:55.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af3e00640 1 -- 172.21.15.66:0/2625309558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aec0ce1f0 con 0x7f8aec0b1e10 2024-09-10T22:02:55.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.125+0000 7f8af3e00640 1 -- 172.21.15.66:0/2625309558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aec1fb380 con 0x7f8aec0b1e10 2024-09-10T22:02:55.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.129+0000 7f8af3e00640 1 -- 172.21.15.66:0/2625309558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aec0b3170 con 0x7f8aec0b1e10 2024-09-10T22:02:55.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.129+0000 7f8ae9600640 1 -- 172.21.15.66:0/2625309558 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f8ae00b5070 con 0x7f8aec0b1e10 2024-09-10T22:02:55.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.129+0000 7f8ae9600640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8acc072940 0x7f8acc074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:55.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.129+0000 7f8ae9600640 1 -- 172.21.15.66:0/2625309558 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5357+0+0 (secure 0 0 0) 0x7f8ae012cca0 con 0x7f8aec0b1e10 2024-09-10T22:02:55.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.129+0000 7f8af0a00640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8acc072940 0x7f8acc074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:55.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.133+0000 7f8af0a00640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8acc072940 0x7f8acc074e00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8aec15ec70 tx=0x7f8ad8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:55.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.133+0000 7f8ae9600640 1 -- 172.21.15.66:0/2625309558 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ae0016410 con 0x7f8aec0b1e10 2024-09-10T22:02:55.189 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:02:55.192 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:02:55.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.309+0000 7f8af3e00640 1 -- 172.21.15.66:0/2625309558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8aec0d2a40 con 0x7f8aec0b1e10 2024-09-10T22:02:55.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.953+0000 7f8ae9600640 1 -- 172.21.15.66:0/2625309558 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7f8ae00f7e80 con 0x7f8aec0b1e10 2024-09-10T22:02:55.957 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:02:55.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.953+0000 7f8af3e00640 1 -- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8acc072940 msgr2=0x7f8acc074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:55.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.953+0000 7f8af3e00640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8acc072940 0x7f8acc074e00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8aec15ec70 tx=0x7f8ad8009040 comp rx=0 tx=0).stop 2024-09-10T22:02:55.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.953+0000 7f8af3e00640 1 -- 172.21.15.66:0/2625309558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aec0b1e10 msgr2=0x7f8aec156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:55.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.953+0000 7f8af3e00640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aec0b1e10 0x7f8aec156f80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009790 tx=0x7f8ae00a56e0 comp rx=0 tx=0).stop 2024-09-10T22:02:55.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.957+0000 7f8af3e00640 1 -- 172.21.15.66:0/2625309558 shutdown_connections 2024-09-10T22:02:55.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.957+0000 7f8af3e00640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8acc072940 0x7f8acc074e00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:55.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.957+0000 7f8af3e00640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aec160280 0x7f8aec15a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:55.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.957+0000 7f8af3e00640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aec15f450 0x7f8aec159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:55.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.957+0000 7f8af3e00640 1 --2- 172.21.15.66:0/2625309558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aec0b1e10 0x7f8aec156f80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:55.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.957+0000 7f8af3e00640 1 -- 172.21.15.66:0/2625309558 >> 172.21.15.66:0/2625309558 conn(0x7f8aec0d1c60 msgr2=0x7f8aec1591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:55.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.957+0000 7f8af3e00640 1 -- 172.21.15.66:0/2625309558 shutdown_connections 2024-09-10T22:02:55.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:55.957+0000 7f8af3e00640 1 -- 172.21.15.66:0/2625309558 wait complete. 2024-09-10T22:02:56.000 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T22:02:56.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.253+0000 7fa69be00640 1 -- 172.21.15.66:0/3974810334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800052f0 msgr2=0x7fa68000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.253+0000 7fa69be00640 1 --2- 172.21.15.66:0/3974810334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800052f0 0x7fa68000b9d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa694001dc0 tx=0x7fa69401cb30 comp rx=0 tx=0).stop 2024-09-10T22:02:56.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.253+0000 7fa69be00640 1 -- 172.21.15.66:0/3974810334 shutdown_connections 2024-09-10T22:02:56.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.253+0000 7fa69be00640 1 --2- 172.21.15.66:0/3974810334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800052f0 0x7fa68000b9d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.253+0000 7fa69be00640 1 -- 172.21.15.66:0/3974810334 >> 172.21.15.66:0/3974810334 conn(0x7fa69c0d1b10 msgr2=0x7fa69c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:56.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.253+0000 7fa69be00640 1 -- 172.21.15.66:0/3974810334 shutdown_connections 2024-09-10T22:02:56.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.253+0000 7fa69be00640 1 -- 172.21.15.66:0/3974810334 wait complete. 2024-09-10T22:02:56.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69be00640 1 Processor -- start 2024-09-10T22:02:56.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69be00640 1 -- start start 2024-09-10T22:02:56.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800052f0 0x7fa69c0b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:56.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69be00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa69c1f78e0 0x7fa69c1f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:56.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa69c1f82a0 0x7fa69c0b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:56.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c06e120 con 0x7fa6800052f0 2024-09-10T22:02:56.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c06dfa0 con 0x7fa69c1f82a0 2024-09-10T22:02:56.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69be00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa69c06e2a0 con 0x7fa69c1f78e0 2024-09-10T22:02:56.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800052f0 0x7fa69c0b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:56.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800052f0 0x7fa69c0b73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58070/0 (socket says 172.21.15.66:58070) 2024-09-10T22:02:56.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69aa00640 1 -- 172.21.15.66:0/1663352988 learned_addr learned my addr 172.21.15.66:0/1663352988 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:56.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa693e00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa69c1f78e0 0x7fa69c1f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:56.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69aa00640 1 -- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa69c1f78e0 msgr2=0x7fa69c1f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69aa00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa69c1f78e0 0x7fa69c1f7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69aa00640 1 -- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa69c1f82a0 msgr2=0x7fa69c0b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69aa00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa69c1f82a0 0x7fa69c0b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69aa00640 1 -- 172.21.15.66:0/1663352988 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69c0cc8e0 con 0x7fa6800052f0 2024-09-10T22:02:56.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69aa00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800052f0 0x7fa69c0b73b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00fa00 tx=0x7fa68c00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:56.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa692a00640 1 -- 172.21.15.66:0/1663352988 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c00bcb0 con 0x7fa6800052f0 2024-09-10T22:02:56.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa692a00640 1 -- 172.21.15.66:0/1663352988 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa68c011440 con 0x7fa6800052f0 2024-09-10T22:02:56.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69be00640 1 -- 172.21.15.66:0/1663352988 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69c0ccbc0 con 0x7fa6800052f0 2024-09-10T22:02:56.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa692a00640 1 -- 172.21.15.66:0/1663352988 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c011870 con 0x7fa6800052f0 2024-09-10T22:02:56.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.257+0000 7fa69be00640 1 -- 172.21.15.66:0/1663352988 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa69c0cd180 con 0x7fa6800052f0 2024-09-10T22:02:56.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.261+0000 7fa69be00640 1 -- 172.21.15.66:0/1663352988 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69c0d16a0 con 0x7fa6800052f0 2024-09-10T22:02:56.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.261+0000 7fa692a00640 1 -- 172.21.15.66:0/1663352988 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fa68c023030 con 0x7fa6800052f0 2024-09-10T22:02:56.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.261+0000 7fa692a00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa678072940 0x7fa678074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:56.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.261+0000 7fa692a00640 1 -- 172.21.15.66:0/1663352988 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fa68c093f40 con 0x7fa6800052f0 2024-09-10T22:02:56.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.261+0000 7fa693e00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa678072940 0x7fa678074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:56.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.265+0000 7fa693e00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa678072940 0x7fa678074e00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0ce900 tx=0x7fa684001140 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:56.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.265+0000 7fa692a00640 1 -- 172.21.15.66:0/1663352988 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa68c05f3c0 con 0x7fa6800052f0 2024-09-10T22:02:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.441+0000 7fa69be00640 1 -- 172.21.15.66:0/1663352988 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fa69c072e00 con 0x7fa6800052f0 2024-09-10T22:02:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.445+0000 7fa692a00640 1 -- 172.21.15.66:0/1663352988 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v30) ==== 71+0+0 (secure 0 0 0) 0x7fa68c05ecc0 con 0x7fa6800052f0 2024-09-10T22:02:56.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.445+0000 7fa69be00640 1 -- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa678072940 msgr2=0x7fa678074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.445+0000 7fa69be00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa678072940 0x7fa678074e00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0ce900 tx=0x7fa684001140 comp rx=0 tx=0).stop 2024-09-10T22:02:56.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.445+0000 7fa69be00640 1 -- 172.21.15.66:0/1663352988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800052f0 msgr2=0x7fa69c0b73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.445+0000 7fa69be00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800052f0 0x7fa69c0b73b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00fa00 tx=0x7fa68c00fec0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.449+0000 7fa69be00640 1 -- 172.21.15.66:0/1663352988 shutdown_connections 2024-09-10T22:02:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.449+0000 7fa69be00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa678072940 0x7fa678074e00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.449+0000 7fa69be00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa69c1f82a0 0x7fa69c0b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.449+0000 7fa69be00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa69c1f78e0 0x7fa69c1f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.449+0000 7fa69be00640 1 --2- 172.21.15.66:0/1663352988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800052f0 0x7fa69c0b73b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.449+0000 7fa69be00640 1 -- 172.21.15.66:0/1663352988 >> 172.21.15.66:0/1663352988 conn(0x7fa69c0d1b10 msgr2=0x7fa69c0b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.449+0000 7fa69be00640 1 -- 172.21.15.66:0/1663352988 shutdown_connections 2024-09-10T22:02:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.449+0000 7fa69be00640 1 -- 172.21.15.66:0/1663352988 wait complete. 2024-09-10T22:02:56.472 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-10T22:02:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.733+0000 7feecb800640 1 -- 172.21.15.66:0/2557356851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec40c4fc0 msgr2=0x7feec406e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.733+0000 7feecb800640 1 --2- 172.21.15.66:0/2557356851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec40c4fc0 0x7feec406e630 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feeb000afd0 tx=0x7feeb001dd90 comp rx=0 tx=0).stop 2024-09-10T22:02:56.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.733+0000 7feecb800640 1 -- 172.21.15.66:0/2557356851 shutdown_connections 2024-09-10T22:02:56.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.733+0000 7feecb800640 1 --2- 172.21.15.66:0/2557356851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec40c4fc0 0x7feec406e630 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feeb000afd0 tx=0x7feeb001dd90 comp rx=0 tx=0).stop 2024-09-10T22:02:56.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.733+0000 7feecb800640 1 -- 172.21.15.66:0/2557356851 >> 172.21.15.66:0/2557356851 conn(0x7feec40cfa30 msgr2=0x7feec40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:56.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.733+0000 7feecb800640 1 -- 172.21.15.66:0/2557356851 shutdown_connections 2024-09-10T22:02:56.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.733+0000 7feecb800640 1 -- 172.21.15.66:0/2557356851 wait complete. 2024-09-10T22:02:56.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.737+0000 7feecb800640 1 Processor -- start 2024-09-10T22:02:56.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.737+0000 7feecb800640 1 -- start start 2024-09-10T22:02:56.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feecb800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feec415dd50 0x7feec41fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:56.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feecb800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec415e130 0x7feec40a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:56.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feecb800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feec41ff600 0x7feec40b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec8e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feec415dd50 0x7feec41fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec8e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feec415dd50 0x7feec41fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:38084/0 (socket says 172.21.15.66:38084) 2024-09-10T22:02:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec415e130 0x7feec40a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec415e130 0x7feec40a94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:56052/0 (socket says 172.21.15.66:56052) 2024-09-10T22:02:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feec41ff600 0x7feec40b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feec41ff600 0x7feec40b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58082/0 (socket says 172.21.15.66:58082) 2024-09-10T22:02:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feec4168370 con 0x7feec41ff600 2024-09-10T22:02:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feec41681f0 con 0x7feec415e130 2024-09-10T22:02:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7feec41684f0 con 0x7feec415dd50 2024-09-10T22:02:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec8e00640 1 -- 172.21.15.66:0/2315617296 learned_addr learned my addr 172.21.15.66:0/2315617296 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec8e00640 1 -- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec415e130 msgr2=0x7feec40a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec8e00640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec415e130 0x7feec40a94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec8e00640 1 -- 172.21.15.66:0/2315617296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feec41ff600 msgr2=0x7feec40b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec8e00640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feec41ff600 0x7feec40b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec8e00640 1 -- 172.21.15.66:0/2315617296 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec40b5700 con 0x7feec415dd50 2024-09-10T22:02:56.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.741+0000 7feec3e00640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec415e130 0x7feec40a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:02:56.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.745+0000 7feec8e00640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feec415dd50 0x7feec41fff80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c760 tx=0x7feeb800cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:56.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.745+0000 7feec0c00640 1 -- 172.21.15.66:0/2315617296 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8017070 con 0x7feec415dd50 2024-09-10T22:02:56.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.745+0000 7feec0c00640 1 -- 172.21.15.66:0/2315617296 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeb8005360 con 0x7feec415dd50 2024-09-10T22:02:56.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.745+0000 7feec0c00640 1 -- 172.21.15.66:0/2315617296 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb800a8a0 con 0x7feec415dd50 2024-09-10T22:02:56.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.745+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec40b2c00 con 0x7feec415dd50 2024-09-10T22:02:56.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.745+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feec41ffa90 con 0x7feec415dd50 2024-09-10T22:02:56.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.745+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec406e630 con 0x7feec415dd50 2024-09-10T22:02:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.749+0000 7feec0c00640 1 -- 172.21.15.66:0/2315617296 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7feeb8037b80 con 0x7feec415dd50 2024-09-10T22:02:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.749+0000 7feec0c00640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feea4072a40 0x7feea4074f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:56.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.749+0000 7feec0c00640 1 -- 172.21.15.66:0/2315617296 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7feeb8092d00 con 0x7feec415dd50 2024-09-10T22:02:56.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.749+0000 7feec3e00640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feea4072a40 0x7feea4074f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:56.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.753+0000 7feec3e00640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feea4072a40 0x7feea4074f00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feeb00af000 tx=0x7feeb001bb70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:56.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:56.757+0000 7feec0c00640 1 -- 172.21.15.66:0/2315617296 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feeb805dc40 con 0x7feec415dd50 2024-09-10T22:02:57.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.037+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7feec4159f80 con 0x7feec415dd50 2024-09-10T22:02:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.965+0000 7feec0c00640 1 -- 172.21.15.66:0/2315617296 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v128) ==== 184+0+0 (secure 0 0 0) 0x7feeb8062670 con 0x7feec415dd50 2024-09-10T22:02:57.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.969+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feea4072a40 msgr2=0x7feea4074f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:57.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.969+0000 7feecb800640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feea4072a40 0x7feea4074f00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feeb00af000 tx=0x7feeb001bb70 comp rx=0 tx=0).stop 2024-09-10T22:02:57.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.969+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feec415dd50 msgr2=0x7feec41fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:57.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.969+0000 7feecb800640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feec415dd50 0x7feec41fff80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c760 tx=0x7feeb800cc20 comp rx=0 tx=0).stop 2024-09-10T22:02:57.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.969+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 shutdown_connections 2024-09-10T22:02:57.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.969+0000 7feecb800640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feea4072a40 0x7feea4074f00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:57.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.969+0000 7feecb800640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feec41ff600 0x7feec40b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:57.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.969+0000 7feecb800640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec415e130 0x7feec40a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:57.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.969+0000 7feecb800640 1 --2- 172.21.15.66:0/2315617296 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feec415dd50 0x7feec41fff80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:57.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.969+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 >> 172.21.15.66:0/2315617296 conn(0x7feec40cfa30 msgr2=0x7feec4073da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:57.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.969+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 shutdown_connections 2024-09-10T22:02:57.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:57.969+0000 7feecb800640 1 -- 172.21.15.66:0/2315617296 wait complete. 2024-09-10T22:02:57.996 INFO:tasks.cephfs_test_runner:Initiate cloning where clone subvolume and source subvolume are located ... ok 2024-09-10T22:02:57.996 INFO:tasks.cephfs_test_runner:Starting test: test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-10T22:02:57.996 INFO:tasks.cephfs_test_runner:test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-10T22:02:57.997 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:02:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 -- 172.21.15.66:0/1975530761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd028003a60 msgr2=0x7fd028003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 --2- 172.21.15.66:0/1975530761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd028003a60 0x7fd028003e80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd034009b60 tx=0x7fd03401d270 comp rx=0 tx=0).stop 2024-09-10T22:02:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 -- 172.21.15.66:0/1975530761 shutdown_connections 2024-09-10T22:02:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 --2- 172.21.15.66:0/1975530761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd028003a60 0x7fd028003e80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 -- 172.21.15.66:0/1975530761 >> 172.21.15.66:0/1975530761 conn(0x7fd0440cfa30 msgr2=0x7fd0440cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 -- 172.21.15.66:0/1975530761 shutdown_connections 2024-09-10T22:02:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 -- 172.21.15.66:0/1975530761 wait complete. 2024-09-10T22:02:58.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 Processor -- start 2024-09-10T22:02:58.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 -- start start 2024-09-10T22:02:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd028003a60 0x7fd04415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0440b3670 0x7fd04415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0440b3af0 0x7fd04415b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd0440cbdd0 con 0x7fd028003a60 2024-09-10T22:02:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0440cbc50 con 0x7fd0440b3af0 2024-09-10T22:02:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.141+0000 7fd04a600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd0440cbf50 con 0x7fd0440b3670 2024-09-10T22:02:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd043400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd028003a60 0x7fd04415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd043400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd028003a60 0x7fd04415d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58114/0 (socket says 172.21.15.66:58114) 2024-09-10T22:02:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd043400640 1 -- 172.21.15.66:0/4090501547 learned_addr learned my addr 172.21.15.66:0/4090501547 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd03be00640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0440b3670 0x7fd04415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd043e00640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0440b3af0 0x7fd04415b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd043400640 1 -- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0440b3670 msgr2=0x7fd04415b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd043400640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0440b3670 0x7fd04415b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd043400640 1 -- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0440b3af0 msgr2=0x7fd04415b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd043400640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0440b3af0 0x7fd04415b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd043400640 1 -- 172.21.15.66:0/4090501547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04415ddf0 con 0x7fd028003a60 2024-09-10T22:02:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd043400640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd028003a60 0x7fd04415d8b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd0340af040 tx=0x7fd0340056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd03b400640 1 -- 172.21.15.66:0/4090501547 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0340bd070 con 0x7fd028003a60 2024-09-10T22:02:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd03b400640 1 -- 172.21.15.66:0/4090501547 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0340ac8a0 con 0x7fd028003a60 2024-09-10T22:02:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd03b400640 1 -- 172.21.15.66:0/4090501547 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0340b1040 con 0x7fd028003a60 2024-09-10T22:02:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd04a600640 1 -- 172.21.15.66:0/4090501547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04415e080 con 0x7fd028003a60 2024-09-10T22:02:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd04a600640 1 -- 172.21.15.66:0/4090501547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0440b2e70 con 0x7fd028003a60 2024-09-10T22:02:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd04a600640 1 -- 172.21.15.66:0/4090501547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04415b050 con 0x7fd028003a60 2024-09-10T22:02:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.145+0000 7fd03b400640 1 -- 172.21.15.66:0/4090501547 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fd0340046d0 con 0x7fd028003a60 2024-09-10T22:02:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.149+0000 7fd03b400640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd020072970 0x7fd020074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.149+0000 7fd03b400640 1 -- 172.21.15.66:0/4090501547 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fd03412db40 con 0x7fd028003a60 2024-09-10T22:02:58.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.149+0000 7fd03be00640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd020072970 0x7fd020074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.149+0000 7fd03be00640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd020072970 0x7fd020074e30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd0440b5b80 tx=0x7fd02c0067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.153+0000 7fd03b400640 1 -- 172.21.15.66:0/4090501547 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd034016f80 con 0x7fd028003a60 2024-09-10T22:02:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.325+0000 7fd04a600640 1 -- 172.21.15.66:0/4090501547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0440cea00 con 0x7fd028003a60 2024-09-10T22:02:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.325+0000 7fd03b400640 1 -- 172.21.15.66:0/4090501547 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+12958 (secure 0 0 0) 0x7fd0340f8870 con 0x7fd028003a60 2024-09-10T22:02:58.333 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:02:58.333 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":36,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:02:55.955699+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/297228676":"2024-09-11T22:02:52.942269+0000","172.21.15.66:6833/297228676":"2024-09-11T22:02:52.942269+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:02:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.329+0000 7fd04a600640 1 -- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd020072970 msgr2=0x7fd020074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.329+0000 7fd04a600640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd020072970 0x7fd020074e30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd0440b5b80 tx=0x7fd02c0067b0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.329+0000 7fd04a600640 1 -- 172.21.15.66:0/4090501547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd028003a60 msgr2=0x7fd04415d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.329+0000 7fd04a600640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd028003a60 0x7fd04415d8b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd0340af040 tx=0x7fd0340056a0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.329+0000 7fd04a600640 1 -- 172.21.15.66:0/4090501547 shutdown_connections 2024-09-10T22:02:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.329+0000 7fd04a600640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd020072970 0x7fd020074e30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.329+0000 7fd04a600640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0440b3af0 0x7fd04415b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.329+0000 7fd04a600640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0440b3670 0x7fd04415b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.329+0000 7fd04a600640 1 --2- 172.21.15.66:0/4090501547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd028003a60 0x7fd04415d8b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.329+0000 7fd04a600640 1 -- 172.21.15.66:0/4090501547 >> 172.21.15.66:0/4090501547 conn(0x7fd0440cfa30 msgr2=0x7fd0440cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:58.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.329+0000 7fd04a600640 1 -- 172.21.15.66:0/4090501547 shutdown_connections 2024-09-10T22:02:58.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.329+0000 7fd04a600640 1 -- 172.21.15.66:0/4090501547 wait complete. 2024-09-10T22:02:58.356 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:00:22.340318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:02:58.356 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:02:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 -- 172.21.15.66:0/361921425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0161100 msgr2=0x7f39a00d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 --2- 172.21.15.66:0/361921425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0161100 0x7f39a00d12c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3990009840 tx=0x7f399001c6e0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 -- 172.21.15.66:0/361921425 shutdown_connections 2024-09-10T22:02:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 --2- 172.21.15.66:0/361921425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0161100 0x7f39a00d12c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 -- 172.21.15.66:0/361921425 >> 172.21.15.66:0/361921425 conn(0x7f39a00d1b10 msgr2=0x7f39a00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 -- 172.21.15.66:0/361921425 shutdown_connections 2024-09-10T22:02:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 -- 172.21.15.66:0/361921425 wait complete. 2024-09-10T22:02:58.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 Processor -- start 2024-09-10T22:02:58.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 -- start start 2024-09-10T22:02:58.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39a006e440 0x7f39a00b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0161100 0x7f39a0152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39a01614e0 0x7f39a0157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f39a016c080 con 0x7f39a01614e0 2024-09-10T22:02:58.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39a016bf00 con 0x7f39a0161100 2024-09-10T22:02:58.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.501+0000 7f39a5000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f39a016c200 con 0x7f39a006e440 2024-09-10T22:02:58.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f399fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39a01614e0 0x7f39a0157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f399fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39a01614e0 0x7f39a0157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58128/0 (socket says 172.21.15.66:58128) 2024-09-10T22:02:58.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f399fe00640 1 -- 172.21.15.66:0/934584703 learned_addr learned my addr 172.21.15.66:0/934584703 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:58.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f399f400640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39a006e440 0x7f39a00b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f3997e00640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0161100 0x7f39a0152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f399fe00640 1 -- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39a006e440 msgr2=0x7f39a00b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f399fe00640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39a006e440 0x7f39a00b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f399fe00640 1 -- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0161100 msgr2=0x7f39a0152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f399fe00640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0161100 0x7f39a0152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f399fe00640 1 -- 172.21.15.66:0/934584703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a00cbb40 con 0x7f39a01614e0 2024-09-10T22:02:58.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f399fe00640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39a01614e0 0x7f39a0157b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f399800c7c0 tx=0x7f399800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:58.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f3997400640 1 -- 172.21.15.66:0/934584703 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998003b80 con 0x7f39a01614e0 2024-09-10T22:02:58.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f3997400640 1 -- 172.21.15.66:0/934584703 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3998003d20 con 0x7f39a01614e0 2024-09-10T22:02:58.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f3997400640 1 -- 172.21.15.66:0/934584703 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399800aab0 con 0x7f39a01614e0 2024-09-10T22:02:58.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f39a5000640 1 -- 172.21.15.66:0/934584703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a00b6dd0 con 0x7f39a01614e0 2024-09-10T22:02:58.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f39a5000640 1 -- 172.21.15.66:0/934584703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a01f7d40 con 0x7f39a01614e0 2024-09-10T22:02:58.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f39a5000640 1 -- 172.21.15.66:0/934584703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a00d12c0 con 0x7f39a01614e0 2024-09-10T22:02:58.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.505+0000 7f3997400640 1 -- 172.21.15.66:0/934584703 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f399800ac50 con 0x7f39a01614e0 2024-09-10T22:02:58.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.509+0000 7f3997400640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f397c0728b0 0x7f397c074d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.509+0000 7f3997400640 1 -- 172.21.15.66:0/934584703 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f3998093440 con 0x7f39a01614e0 2024-09-10T22:02:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.509+0000 7f399f400640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f397c0728b0 0x7f397c074d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.509+0000 7f399f400640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f397c0728b0 0x7f397c074d70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f39900af000 tx=0x7f39900b2040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.513+0000 7f3997400640 1 -- 172.21.15.66:0/934584703 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f399805e190 con 0x7f39a01614e0 2024-09-10T22:02:58.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 -- 172.21.15.66:0/934584703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f39a0072e00 con 0x7f39a01614e0 2024-09-10T22:02:58.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f3997400640 1 -- 172.21.15.66:0/934584703 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f3998062bc0 con 0x7f39a01614e0 2024-09-10T22:02:58.694 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-09-10T22:02:58.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 -- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f397c0728b0 msgr2=0x7f397c074d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f397c0728b0 0x7f397c074d70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f39900af000 tx=0x7f39900b2040 comp rx=0 tx=0).stop 2024-09-10T22:02:58.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 -- 172.21.15.66:0/934584703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39a01614e0 msgr2=0x7f39a0157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39a01614e0 0x7f39a0157b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f399800c7c0 tx=0x7f399800cc80 comp rx=0 tx=0).stop 2024-09-10T22:02:58.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 -- 172.21.15.66:0/934584703 shutdown_connections 2024-09-10T22:02:58.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f397c0728b0 0x7f397c074d70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39a01614e0 0x7f39a0157b50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0161100 0x7f39a0152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 --2- 172.21.15.66:0/934584703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39a006e440 0x7f39a00b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 -- 172.21.15.66:0/934584703 >> 172.21.15.66:0/934584703 conn(0x7f39a00d1b10 msgr2=0x7f39a006dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:58.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 -- 172.21.15.66:0/934584703 shutdown_connections 2024-09-10T22:02:58.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.689+0000 7f39a5000640 1 -- 172.21.15.66:0/934584703 wait complete. 2024-09-10T22:02:58.715 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-10T22:02:58.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.857+0000 7f44c9600640 1 -- 172.21.15.66:0/1875293587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c415f340 msgr2=0x7f44c40cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.857+0000 7f44c9600640 1 --2- 172.21.15.66:0/1875293587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c415f340 0x7f44c40cfb50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009840 tx=0x7f44b401c6b0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.857+0000 7f44c9600640 1 -- 172.21.15.66:0/1875293587 shutdown_connections 2024-09-10T22:02:58.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.857+0000 7f44c9600640 1 --2- 172.21.15.66:0/1875293587 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44c40b2bf0 0x7f44c40d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.857+0000 7f44c9600640 1 --2- 172.21.15.66:0/1875293587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c415f340 0x7f44c40cfb50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.857+0000 7f44c9600640 1 -- 172.21.15.66:0/1875293587 >> 172.21.15.66:0/1875293587 conn(0x7f44c40d1c50 msgr2=0x7f44c40d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:58.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.857+0000 7f44c9600640 1 -- 172.21.15.66:0/1875293587 shutdown_connections 2024-09-10T22:02:58.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.857+0000 7f44c9600640 1 -- 172.21.15.66:0/1875293587 wait complete. 2024-09-10T22:02:58.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.857+0000 7f44c9600640 1 Processor -- start 2024-09-10T22:02:58.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.857+0000 7f44c9600640 1 -- start start 2024-09-10T22:02:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c9600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c40b2bf0 0x7f44c4148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c9600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44c40d2770 0x7f44c40b7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c9600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c415f340 0x7f44c41f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c9600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4169190 con 0x7f44c40b2bf0 2024-09-10T22:02:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c9600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4169010 con 0x7f44c415f340 2024-09-10T22:02:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c9600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f44c4169310 con 0x7f44c40d2770 2024-09-10T22:02:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c40b2bf0 0x7f44c4148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c40b2bf0 0x7f44c4148220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58134/0 (socket says 172.21.15.66:58134) 2024-09-10T22:02:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c2a00640 1 -- 172.21.15.66:0/3230253344 learned_addr learned my addr 172.21.15.66:0/3230253344 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:02:58.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44bbe00640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44c40d2770 0x7f44c40b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c3400640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c415f340 0x7f44c41f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c2a00640 1 -- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44c40d2770 msgr2=0x7f44c40b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c2a00640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44c40d2770 0x7f44c40b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c2a00640 1 -- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c415f340 msgr2=0x7f44c41f8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c2a00640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c415f340 0x7f44c41f8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c2a00640 1 -- 172.21.15.66:0/3230253344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c41f8e50 con 0x7f44c40b2bf0 2024-09-10T22:02:58.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44bbe00640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44c40d2770 0x7f44c40b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c3400640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c415f340 0x7f44c41f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c2a00640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c40b2bf0 0x7f44c4148220 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f44b4000c10 tx=0x7f44b40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44baa00640 1 -- 172.21.15.66:0/3230253344 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4004750 con 0x7f44c40b2bf0 2024-09-10T22:02:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44baa00640 1 -- 172.21.15.66:0/3230253344 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44b40048f0 con 0x7f44c40b2bf0 2024-09-10T22:02:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c9600640 1 -- 172.21.15.66:0/3230253344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c41f9130 con 0x7f44c40b2bf0 2024-09-10T22:02:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44baa00640 1 -- 172.21.15.66:0/3230253344 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4004ce0 con 0x7f44c40b2bf0 2024-09-10T22:02:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c9600640 1 -- 172.21.15.66:0/3230253344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41fbba0 con 0x7f44c40b2bf0 2024-09-10T22:02:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.861+0000 7f44c9600640 1 -- 172.21.15.66:0/3230253344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c415fac0 con 0x7f44c40b2bf0 2024-09-10T22:02:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.865+0000 7f44baa00640 1 -- 172.21.15.66:0/3230253344 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f44b40a8430 con 0x7f44c40b2bf0 2024-09-10T22:02:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.865+0000 7f44baa00640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f44a0072940 0x7f44a0074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.865+0000 7f44baa00640 1 -- 172.21.15.66:0/3230253344 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f44b412d690 con 0x7f44c40b2bf0 2024-09-10T22:02:58.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.865+0000 7f44bbe00640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f44a0072940 0x7f44a0074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.865+0000 7f44bbe00640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f44a0072940 0x7f44a0074e00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44c4072000 tx=0x7f44ac0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:58.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:58.869+0000 7f44baa00640 1 -- 172.21.15.66:0/3230253344 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f44b40c1030 con 0x7f44c40b2bf0 2024-09-10T22:02:59.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.113+0000 7f44c9600640 1 -- 172.21.15.66:0/3230253344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7f44c40c8a40 con 0x7f44c40b2bf0 2024-09-10T22:02:59.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.969+0000 7f44baa00640 1 -- 172.21.15.66:0/3230253344 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v130) ==== 169+0+0 (secure 0 0 0) 0x7f44b40ac030 con 0x7f44c40b2bf0 2024-09-10T22:02:59.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.969+0000 7f44c9600640 1 -- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f44a0072940 msgr2=0x7f44a0074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:59.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.969+0000 7f44c9600640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f44a0072940 0x7f44a0074e00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44c4072000 tx=0x7f44ac0091e0 comp rx=0 tx=0).stop 2024-09-10T22:02:59.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.969+0000 7f44c9600640 1 -- 172.21.15.66:0/3230253344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c40b2bf0 msgr2=0x7f44c4148220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:59.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.969+0000 7f44c9600640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c40b2bf0 0x7f44c4148220 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f44b4000c10 tx=0x7f44b40056d0 comp rx=0 tx=0).stop 2024-09-10T22:02:59.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.973+0000 7f44c9600640 1 -- 172.21.15.66:0/3230253344 shutdown_connections 2024-09-10T22:02:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.973+0000 7f44c9600640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f44a0072940 0x7f44a0074e00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.973+0000 7f44c9600640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c415f340 0x7f44c41f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.973+0000 7f44c9600640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44c40d2770 0x7f44c40b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.973+0000 7f44c9600640 1 --2- 172.21.15.66:0/3230253344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c40b2bf0 0x7f44c4148220 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.973+0000 7f44c9600640 1 -- 172.21.15.66:0/3230253344 >> 172.21.15.66:0/3230253344 conn(0x7f44c40d1c50 msgr2=0x7f44c4070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.973+0000 7f44c9600640 1 -- 172.21.15.66:0/3230253344 shutdown_connections 2024-09-10T22:02:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:02:59.973+0000 7f44c9600640 1 -- 172.21.15.66:0/3230253344 wait complete. 2024-09-10T22:03:00.002 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T22:03:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.145+0000 7f9dc1000640 1 -- 172.21.15.66:0/4255539276 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dbc0b0f60 msgr2=0x7f9dbc0d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.145+0000 7f9dc1000640 1 --2- 172.21.15.66:0/4255539276 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dbc0b0f60 0x7f9dbc0d0140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009840 tx=0x7f9da401c6d0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 -- 172.21.15.66:0/4255539276 shutdown_connections 2024-09-10T22:03:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 --2- 172.21.15.66:0/4255539276 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dbc0b0f60 0x7f9dbc0d0140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 --2- 172.21.15.66:0/4255539276 >> v2:172.21.15.94:3300/0 conn(0x7f9dbc15f460 0x7f9dbc0cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 -- 172.21.15.66:0/4255539276 >> 172.21.15.66:0/4255539276 conn(0x7f9dbc0d1d00 msgr2=0x7f9dbc0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 -- 172.21.15.66:0/4255539276 shutdown_connections 2024-09-10T22:03:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 -- 172.21.15.66:0/4255539276 wait complete. 2024-09-10T22:03:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 Processor -- start 2024-09-10T22:03:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 -- start start 2024-09-10T22:03:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dbc0b0f60 0x7f9dbc155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dbc0b32a0 0x7f9dbc06f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc15f460 0x7f9dbc1fb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc168ff0 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc168e70 con 0x7f9dbc15f460 2024-09-10T22:03:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dc1000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9dbc169170 con 0x7f9dbc0b32a0 2024-09-10T22:03:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9db9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dbc0b0f60 0x7f9dbc155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9db9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dbc0b0f60 0x7f9dbc155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58150/0 (socket says 172.21.15.66:58150) 2024-09-10T22:03:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9db9e00640 1 -- 172.21.15.66:0/3100837296 learned_addr learned my addr 172.21.15.66:0/3100837296 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9db3e00640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dbc0b32a0 0x7f9dbc06f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9dba800640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc15f460 0x7f9dbc1fb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9db9e00640 1 -- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dbc0b32a0 msgr2=0x7f9dbc06f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9db9e00640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dbc0b32a0 0x7f9dbc06f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9db9e00640 1 -- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc15f460 msgr2=0x7f9dbc1fb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9db9e00640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc15f460 0x7f9dbc1fb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9db9e00640 1 -- 172.21.15.66:0/3100837296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc1fc3a0 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9db3e00640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dbc0b32a0 0x7f9dbc06f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:03:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.149+0000 7f9db9e00640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dbc0b0f60 0x7f9dbc155030 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00e970 tx=0x7f9dac00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.153+0000 7f9db2000640 1 -- 172.21.15.66:0/3100837296 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac00cd80 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.153+0000 7f9db2000640 1 -- 172.21.15.66:0/3100837296 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dac004020 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.153+0000 7f9db2000640 1 -- 172.21.15.66:0/3100837296 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac004470 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.153+0000 7f9dc1000640 1 -- 172.21.15.66:0/3100837296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dbc155570 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.153+0000 7f9dc1000640 1 -- 172.21.15.66:0/3100837296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dbc06e8e0 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.153+0000 7f9dc1000640 1 -- 172.21.15.66:0/3100837296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dbc0b14b0 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.153+0000 7f9db2000640 1 -- 172.21.15.66:0/3100837296 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f9dac003600 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.153+0000 7f9db2000640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d98072940 0x7f9d98074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:00.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.153+0000 7f9db2000640 1 -- 172.21.15.66:0/3100837296 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f9dac094060 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.153+0000 7f9db3e00640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d98072940 0x7f9d98074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:00.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.157+0000 7f9db3e00640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d98072940 0x7f9d98074e00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9da40af000 tx=0x7f9da401af10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:00.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.157+0000 7f9db2000640 1 -- 172.21.15.66:0/3100837296 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9dac05ed00 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.333+0000 7f9dc1000640 1 -- 172.21.15.66:0/3100837296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f9dbc0cc0b0 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.333+0000 7f9db2000640 1 -- 172.21.15.66:0/3100837296 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v30) ==== 59+0+1998 (secure 0 0 0) 0x7f9dac063730 con 0x7f9dbc0b0f60 2024-09-10T22:03:00.340 INFO:teuthology.orchestra.run.smithi066.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T22:03:00.340 INFO:teuthology.orchestra.run.smithi066.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T22:03:00.340 INFO:teuthology.orchestra.run.smithi066.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T22:03:00.340 INFO:teuthology.orchestra.run.smithi066.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T22:03:00.340 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4673.907365 2024-09-10T22:03:00.340 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4537.831230 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 5706.341617 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4484.013907 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3862.285837 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3776.439661 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3820.270368 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3863.602758 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds 20/20 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_ms 2/2 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_frag true 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_scatterstat true 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T22:03:00.341 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_verify_scatter true 2024-09-10T22:03:00.342 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T22:03:00.342 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T22:03:00.342 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T22:03:00.342 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced client_mount_timeout 600 2024-09-10T22:03:00.342 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_client 20/20 2024-09-10T22:03:00.342 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_ms 1/1 2024-09-10T22:03:00.342 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T22:03:00.342 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T22:03:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.337+0000 7f9dc1000640 1 -- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d98072940 msgr2=0x7f9d98074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.337+0000 7f9dc1000640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d98072940 0x7f9d98074e00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9da40af000 tx=0x7f9da401af10 comp rx=0 tx=0).stop 2024-09-10T22:03:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.337+0000 7f9dc1000640 1 -- 172.21.15.66:0/3100837296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dbc0b0f60 msgr2=0x7f9dbc155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.337+0000 7f9dc1000640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dbc0b0f60 0x7f9dbc155030 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00e970 tx=0x7f9dac00ee30 comp rx=0 tx=0).stop 2024-09-10T22:03:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.337+0000 7f9dc1000640 1 -- 172.21.15.66:0/3100837296 shutdown_connections 2024-09-10T22:03:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.337+0000 7f9dc1000640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d98072940 0x7f9d98074e00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.337+0000 7f9dc1000640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc15f460 0x7f9dbc1fb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.337+0000 7f9dc1000640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dbc0b32a0 0x7f9dbc06f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.337+0000 7f9dc1000640 1 --2- 172.21.15.66:0/3100837296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dbc0b0f60 0x7f9dbc155030 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.337+0000 7f9dc1000640 1 -- 172.21.15.66:0/3100837296 >> 172.21.15.66:0/3100837296 conn(0x7f9dbc0d1d00 msgr2=0x7f9dbc15c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.337+0000 7f9dc1000640 1 -- 172.21.15.66:0/3100837296 shutdown_connections 2024-09-10T22:03:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.337+0000 7f9dc1000640 1 -- 172.21.15.66:0/3100837296 wait complete. 2024-09-10T22:03:00.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:03:00.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.505+0000 7fc409e00640 1 -- 172.21.15.66:0/3998646580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc40415f340 msgr2=0x7fc4040cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:00.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.505+0000 7fc409e00640 1 --2- 172.21.15.66:0/3998646580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc40415f340 0x7fc4040cfb50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009930 tx=0x7fc3f401c8e0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.505+0000 7fc409e00640 1 -- 172.21.15.66:0/3998646580 shutdown_connections 2024-09-10T22:03:00.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.505+0000 7fc409e00640 1 --2- 172.21.15.66:0/3998646580 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4040af970 0x7fc4040d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.505+0000 7fc409e00640 1 --2- 172.21.15.66:0/3998646580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc40415f340 0x7fc4040cfb50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.505+0000 7fc409e00640 1 -- 172.21.15.66:0/3998646580 >> 172.21.15.66:0/3998646580 conn(0x7fc4040d1c50 msgr2=0x7fc4040d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:00.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.505+0000 7fc409e00640 1 -- 172.21.15.66:0/3998646580 shutdown_connections 2024-09-10T22:03:00.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.505+0000 7fc409e00640 1 -- 172.21.15.66:0/3998646580 wait complete. 2024-09-10T22:03:00.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc409e00640 1 Processor -- start 2024-09-10T22:03:00.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc409e00640 1 -- start start 2024-09-10T22:03:00.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc409e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4040af970 0x7fc4040cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:00.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc409e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040b2bf0 0x7fc404155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:00.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc409e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc40415f340 0x7fc40415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:00.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc409e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc404169190 con 0x7fc4040b2bf0 2024-09-10T22:03:00.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc409e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc404169010 con 0x7fc4040af970 2024-09-10T22:03:00.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc409e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc404169310 con 0x7fc40415f340 2024-09-10T22:03:00.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc3fbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040b2bf0 0x7fc404155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc3fbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040b2bf0 0x7fc404155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58172/0 (socket says 172.21.15.66:58172) 2024-09-10T22:03:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc3fbe00640 1 -- 172.21.15.66:0/243329580 learned_addr learned my addr 172.21.15.66:0/243329580 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc403400640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc40415f340 0x7fc40415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc402a00640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4040af970 0x7fc4040cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc3fbe00640 1 -- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc40415f340 msgr2=0x7fc40415a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc3fbe00640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc40415f340 0x7fc40415a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc3fbe00640 1 -- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4040af970 msgr2=0x7fc4040cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc3fbe00640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4040af970 0x7fc4040cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc3fbe00640 1 -- 172.21.15.66:0/243329580 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4040c9ef0 con 0x7fc4040b2bf0 2024-09-10T22:03:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc3fbe00640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040b2bf0 0x7fc404155f60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00c530 tx=0x7fc3ec00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:00.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc3fb400640 1 -- 172.21.15.66:0/243329580 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec017070 con 0x7fc4040b2bf0 2024-09-10T22:03:00.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc3fb400640 1 -- 172.21.15.66:0/243329580 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3ec0053c0 con 0x7fc4040b2bf0 2024-09-10T22:03:00.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc3fb400640 1 -- 172.21.15.66:0/243329580 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec012890 con 0x7fc4040b2bf0 2024-09-10T22:03:00.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc409e00640 1 -- 172.21.15.66:0/243329580 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4040b7460 con 0x7fc4040b2bf0 2024-09-10T22:03:00.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.509+0000 7fc409e00640 1 -- 172.21.15.66:0/243329580 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4041564a0 con 0x7fc4040b2bf0 2024-09-10T22:03:00.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.513+0000 7fc409e00640 1 -- 172.21.15.66:0/243329580 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40415fac0 con 0x7fc4040b2bf0 2024-09-10T22:03:00.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.513+0000 7fc3fb400640 1 -- 172.21.15.66:0/243329580 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fc3ec003600 con 0x7fc4040b2bf0 2024-09-10T22:03:00.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.513+0000 7fc3fb400640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3e0072940 0x7fc3e0074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:00.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.513+0000 7fc3fb400640 1 -- 172.21.15.66:0/243329580 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fc3ec092a50 con 0x7fc4040b2bf0 2024-09-10T22:03:00.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.513+0000 7fc402a00640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3e0072940 0x7fc3e0074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:00.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.513+0000 7fc402a00640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3e0072940 0x7fc3e0074e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40af000 tx=0x7fc3f401b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.517+0000 7fc3fb400640 1 -- 172.21.15.66:0/243329580 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc3ec05d950 con 0x7fc4040b2bf0 2024-09-10T22:03:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.693+0000 7fc409e00640 1 -- 172.21.15.66:0/243329580 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc40415c130 con 0x7fc4040b2bf0 2024-09-10T22:03:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.693+0000 7fc3fb400640 1 -- 172.21.15.66:0/243329580 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7fc3ec062380 con 0x7fc4040b2bf0 2024-09-10T22:03:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.697+0000 7fc409e00640 1 -- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3e0072940 msgr2=0x7fc3e0074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.697+0000 7fc409e00640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3e0072940 0x7fc3e0074e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40af000 tx=0x7fc3f401b120 comp rx=0 tx=0).stop 2024-09-10T22:03:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.697+0000 7fc409e00640 1 -- 172.21.15.66:0/243329580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040b2bf0 msgr2=0x7fc404155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.697+0000 7fc409e00640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040b2bf0 0x7fc404155f60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00c530 tx=0x7fc3ec00c9f0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.697+0000 7fc409e00640 1 -- 172.21.15.66:0/243329580 shutdown_connections 2024-09-10T22:03:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.697+0000 7fc409e00640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3e0072940 0x7fc3e0074e00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.697+0000 7fc409e00640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc40415f340 0x7fc40415a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.697+0000 7fc409e00640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040b2bf0 0x7fc404155f60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.697+0000 7fc409e00640 1 --2- 172.21.15.66:0/243329580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4040af970 0x7fc4040cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.697+0000 7fc409e00640 1 -- 172.21.15.66:0/243329580 >> 172.21.15.66:0/243329580 conn(0x7fc4040d1c50 msgr2=0x7fc4040717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:00.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.697+0000 7fc409e00640 1 -- 172.21.15.66:0/243329580 shutdown_connections 2024-09-10T22:03:00.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:00.697+0000 7fc409e00640 1 -- 172.21.15.66:0/243329580 wait complete. 2024-09-10T22:03:00.724 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:03:00.724 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:03:00.827 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:03:00.911 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:03:00.912 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:03:00.912 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:03:00.920 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:03:00.921 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:03:00.921 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:03:00.929 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:03:00.929 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:03:00.929 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:03:00.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:03:00.973 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:03:00.973 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:03:00.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:03:00.982 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T22:03:00.983 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:03:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 -- 172.21.15.66:0/2657609234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c15bd40 msgr2=0x7f4b6c0cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 --2- 172.21.15.66:0/2657609234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c15bd40 0x7f4b6c0cffa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0cc2e0 tx=0x7f4b6001c830 comp rx=0 tx=0).stop 2024-09-10T22:03:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 -- 172.21.15.66:0/2657609234 shutdown_connections 2024-09-10T22:03:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 --2- 172.21.15.66:0/2657609234 >> v2:172.21.15.94:3300/0 conn(0x7f4b6c0cf180 0x7f4b6c0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 --2- 172.21.15.66:0/2657609234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c15bd40 0x7f4b6c0cffa0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 -- 172.21.15.66:0/2657609234 >> 172.21.15.66:0/2657609234 conn(0x7f4b6c0d1c60 msgr2=0x7f4b6c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 -- 172.21.15.66:0/2657609234 shutdown_connections 2024-09-10T22:03:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 -- 172.21.15.66:0/2657609234 wait complete. 2024-09-10T22:03:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 Processor -- start 2024-09-10T22:03:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 -- start start 2024-09-10T22:03:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c0cf180 0x7f4b6c15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b6c0cf830 0x7f4b6c15d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b6c15bd40 0x7f4b6c1533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c168ba0 con 0x7f4b6c0cf180 2024-09-10T22:03:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c168a20 con 0x7f4b6c15bd40 2024-09-10T22:03:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b73a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6c168d20 con 0x7f4b6c0cf830 2024-09-10T22:03:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b71000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c0cf180 0x7f4b6c15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b71000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c0cf180 0x7f4b6c15d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58188/0 (socket says 172.21.15.66:58188) 2024-09-10T22:03:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b71000640 1 -- 172.21.15.66:0/2177236625 learned_addr learned my addr 172.21.15.66:0/2177236625 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b6be00640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b6c0cf830 0x7f4b6c15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b71a00640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b6c15bd40 0x7f4b6c1533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b71000640 1 -- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b6c0cf830 msgr2=0x7f4b6c15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b71000640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b6c0cf830 0x7f4b6c15d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b71000640 1 -- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b6c15bd40 msgr2=0x7f4b6c1533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b71000640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b6c15bd40 0x7f4b6c1533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.161+0000 7f4b71000640 1 -- 172.21.15.66:0/2177236625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c14f640 con 0x7f4b6c0cf180 2024-09-10T22:03:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.165+0000 7f4b71000640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c0cf180 0x7f4b6c15d310 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4b600b0040 tx=0x7f4b600a5760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:01.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.165+0000 7f4b68c00640 1 -- 172.21.15.66:0/2177236625 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b600b2040 con 0x7f4b6c0cf180 2024-09-10T22:03:01.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.165+0000 7f4b68c00640 1 -- 172.21.15.66:0/2177236625 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b600043c0 con 0x7f4b6c0cf180 2024-09-10T22:03:01.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.165+0000 7f4b68c00640 1 -- 172.21.15.66:0/2177236625 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60003860 con 0x7f4b6c0cf180 2024-09-10T22:03:01.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.165+0000 7f4b73a00640 1 -- 172.21.15.66:0/2177236625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6c1f8850 con 0x7f4b6c0cf180 2024-09-10T22:03:01.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.165+0000 7f4b73a00640 1 -- 172.21.15.66:0/2177236625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6c1f8e10 con 0x7f4b6c0cf180 2024-09-10T22:03:01.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.165+0000 7f4b73a00640 1 -- 172.21.15.66:0/2177236625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6c0aec10 con 0x7f4b6c0cf180 2024-09-10T22:03:01.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.165+0000 7f4b68c00640 1 -- 172.21.15.66:0/2177236625 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f4b60003a00 con 0x7f4b6c0cf180 2024-09-10T22:03:01.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.165+0000 7f4b68c00640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b4c072940 0x7f4b4c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.169+0000 7f4b68c00640 1 -- 172.21.15.66:0/2177236625 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f4b6012d700 con 0x7f4b6c0cf180 2024-09-10T22:03:01.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.169+0000 7f4b6be00640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b4c072940 0x7f4b4c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.169+0000 7f4b6be00640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b4c072940 0x7f4b4c074e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c1f8710 tx=0x7f4b58009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:01.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.169+0000 7f4b68c00640 1 -- 172.21.15.66:0/2177236625 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4b600164b0 con 0x7f4b6c0cf180 2024-09-10T22:03:01.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.377+0000 7f4b73a00640 1 -- 172.21.15.66:0/2177236625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b6c0b0df0 con 0x7f4b6c0cf180 2024-09-10T22:03:01.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.377+0000 7f4b68c00640 1 -- 172.21.15.66:0/2177236625 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3908 (secure 0 0 0) 0x7f4b600f83f0 con 0x7f4b6c0cf180 2024-09-10T22:03:01.385 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:03:01.385 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":20,"btime":"2024-09-10T22:02:53:944582+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1019192175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1019192175},{"type":"v1","addr":"172.21.15.94:6837","nonce":1019192175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/554383263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":554383263},{"type":"v1","addr":"172.21.15.94:6836","nonce":554383263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2368842390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2368842390},{"type":"v1","addr":"172.21.15.66:6834","nonce":2368842390}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3201283656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3201283656},{"type":"v1","addr":"172.21.15.66:6835","nonce":3201283656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-10T22:03:01.385 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 20 2024-09-10T22:03:01.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.381+0000 7f4b73a00640 1 -- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b4c072940 msgr2=0x7f4b4c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.381+0000 7f4b73a00640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b4c072940 0x7f4b4c074e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c1f8710 tx=0x7f4b58009040 comp rx=0 tx=0).stop 2024-09-10T22:03:01.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.381+0000 7f4b73a00640 1 -- 172.21.15.66:0/2177236625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c0cf180 msgr2=0x7f4b6c15d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.381+0000 7f4b73a00640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c0cf180 0x7f4b6c15d310 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4b600b0040 tx=0x7f4b600a5760 comp rx=0 tx=0).stop 2024-09-10T22:03:01.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.381+0000 7f4b73a00640 1 -- 172.21.15.66:0/2177236625 shutdown_connections 2024-09-10T22:03:01.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.381+0000 7f4b73a00640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b4c072940 0x7f4b4c074e00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.381+0000 7f4b73a00640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b6c15bd40 0x7f4b6c1533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.381+0000 7f4b73a00640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b6c0cf830 0x7f4b6c15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.381+0000 7f4b73a00640 1 --2- 172.21.15.66:0/2177236625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c0cf180 0x7f4b6c15d310 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.381+0000 7f4b73a00640 1 -- 172.21.15.66:0/2177236625 >> 172.21.15.66:0/2177236625 conn(0x7f4b6c0d1c60 msgr2=0x7f4b6c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:01.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.381+0000 7f4b73a00640 1 -- 172.21.15.66:0/2177236625 shutdown_connections 2024-09-10T22:03:01.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.381+0000 7f4b73a00640 1 -- 172.21.15.66:0/2177236625 wait complete. 2024-09-10T22:03:01.407 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:03:01.407 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:03:01.407 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:03:01.407 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:03:01.407 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:03:01.408 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:03:01.408 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:03:01.408 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:03:01.408 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:03:01.408 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:03:01.408 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:03:01.408 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:03:01.409 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:03:01.409 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:03:01.409 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:03:01.409 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:03:01.419 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:03:01.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:03:01.420 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:03:01.420 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:03:01.422 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:03:01.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:03:01.423 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:03:01.423 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:03:01.466 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:03:01.467 DEBUG:teuthology.parallel:result is None 2024-09-10T22:03:01.467 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:03:01.468 DEBUG:teuthology.parallel:result is None 2024-09-10T22:03:01.469 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:03:01.469 DEBUG:teuthology.parallel:result is None 2024-09-10T22:03:01.469 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:03:01.470 DEBUG:teuthology.parallel:result is None 2024-09-10T22:03:01.470 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T22:03:01.553 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:03:01.557 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:03:01.558 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:03:01.558 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:03:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.617+0000 7f9914c00640 1 -- 172.21.15.66:0/1123606391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f991015bd40 msgr2=0x7f99100cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.617+0000 7f9914c00640 1 --2- 172.21.15.66:0/1123606391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f991015bd40 0x7f99100cffa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9910073290 tx=0x7f990001c830 comp rx=0 tx=0).stop 2024-09-10T22:03:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.617+0000 7f9914c00640 1 -- 172.21.15.66:0/1123606391 shutdown_connections 2024-09-10T22:03:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.617+0000 7f9914c00640 1 --2- 172.21.15.66:0/1123606391 >> v2:172.21.15.94:3300/0 conn(0x7f99100aec10 0x7f99100af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.617+0000 7f9914c00640 1 --2- 172.21.15.66:0/1123606391 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99100cf830 0x7f99100d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.617+0000 7f9914c00640 1 --2- 172.21.15.66:0/1123606391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f991015bd40 0x7f99100cffa0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.617+0000 7f9914c00640 1 -- 172.21.15.66:0/1123606391 >> 172.21.15.66:0/1123606391 conn(0x7f99100d1c60 msgr2=0x7f99100d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.617+0000 7f9914c00640 1 -- 172.21.15.66:0/1123606391 shutdown_connections 2024-09-10T22:03:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.617+0000 7f9914c00640 1 -- 172.21.15.66:0/1123606391 wait complete. 2024-09-10T22:03:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.617+0000 7f9914c00640 1 Processor -- start 2024-09-10T22:03:01.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9914c00640 1 -- start start 2024-09-10T22:03:01.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9914c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99100aec10 0x7f991015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9914c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99100cf830 0x7f9910140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9914c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f991015bd40 0x7f9910158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9914c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f99100ce9c0 con 0x7f991015bd40 2024-09-10T22:03:01.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9914c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99100ce840 con 0x7f99100cf830 2024-09-10T22:03:01.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9914c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f99100ceb40 con 0x7f99100aec10 2024-09-10T22:03:01.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f990de00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99100aec10 0x7f991015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9907e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99100cf830 0x7f9910140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f990de00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99100aec10 0x7f991015c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:38226/0 (socket says 172.21.15.66:38226) 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f990e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f991015bd40 0x7f9910158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f990de00640 1 -- 172.21.15.66:0/3298320696 learned_addr learned my addr 172.21.15.66:0/3298320696 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f990de00640 1 -- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99100cf830 msgr2=0x7f9910140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f990de00640 1 --2- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99100cf830 0x7f9910140ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f990de00640 1 -- 172.21.15.66:0/3298320696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f991015bd40 msgr2=0x7f9910158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f990de00640 1 --2- 172.21.15.66:0/3298320696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f991015bd40 0x7f9910158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f990de00640 1 -- 172.21.15.66:0/3298320696 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9910141410 con 0x7f99100aec10 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f990de00640 1 --2- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99100aec10 0x7f991015c7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f99000b0040 tx=0x7f99000a57c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9906000640 1 -- 172.21.15.66:0/3298320696 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99000b2040 con 0x7f99100aec10 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9906000640 1 -- 172.21.15.66:0/3298320696 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9900004340 con 0x7f99100aec10 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9914c00640 1 -- 172.21.15.66:0/3298320696 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9910141690 con 0x7f99100aec10 2024-09-10T22:03:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9906000640 1 -- 172.21.15.66:0/3298320696 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99000044f0 con 0x7f99100aec10 2024-09-10T22:03:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9914c00640 1 -- 172.21.15.66:0/3298320696 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9910141c50 con 0x7f99100aec10 2024-09-10T22:03:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.621+0000 7f9914c00640 1 -- 172.21.15.66:0/3298320696 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99100af010 con 0x7f99100aec10 2024-09-10T22:03:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.625+0000 7f9906000640 1 -- 172.21.15.66:0/3298320696 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f9900005290 con 0x7f99100aec10 2024-09-10T22:03:01.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.625+0000 7f9906000640 1 --2- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f98ec072940 0x7f98ec074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.625+0000 7f9906000640 1 -- 172.21.15.66:0/3298320696 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f990012d560 con 0x7f99100aec10 2024-09-10T22:03:01.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.625+0000 7f9907e00640 1 --2- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f98ec072940 0x7f98ec074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.625+0000 7f9907e00640 1 --2- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f98ec072940 0x7f98ec074e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f991015cd00 tx=0x7f98f800f7a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:01.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.629+0000 7f9906000640 1 -- 172.21.15.66:0/3298320696 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9910141410 con 0x7f99100aec10 2024-09-10T22:03:01.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.813+0000 7f9914c00640 1 -- 172.21.15.66:0/3298320696 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f991015a420 con 0x7f99100aec10 2024-09-10T22:03:01.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.985+0000 7f9906000640 1 -- 172.21.15.66:0/3298320696 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) ==== 97+0+0 (secure 0 0 0) 0x7f99000164b0 con 0x7f99100aec10 2024-09-10T22:03:01.991 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-09-10T22:03:01.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.989+0000 7f9914c00640 1 -- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f98ec072940 msgr2=0x7f98ec074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.989+0000 7f9914c00640 1 --2- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f98ec072940 0x7f98ec074e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f991015cd00 tx=0x7f98f800f7a0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.989+0000 7f9914c00640 1 -- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99100aec10 msgr2=0x7f991015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.989+0000 7f9914c00640 1 --2- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99100aec10 0x7f991015c7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f99000b0040 tx=0x7f99000a57c0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.989+0000 7f9914c00640 1 -- 172.21.15.66:0/3298320696 shutdown_connections 2024-09-10T22:03:01.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.989+0000 7f9914c00640 1 --2- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f98ec072940 0x7f98ec074e00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.989+0000 7f9914c00640 1 --2- 172.21.15.66:0/3298320696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f991015bd40 0x7f9910158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.989+0000 7f9914c00640 1 --2- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99100cf830 0x7f9910140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.989+0000 7f9914c00640 1 --2- 172.21.15.66:0/3298320696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99100aec10 0x7f991015c7c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.989+0000 7f9914c00640 1 -- 172.21.15.66:0/3298320696 >> 172.21.15.66:0/3298320696 conn(0x7f99100d1c60 msgr2=0x7f99100b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:01.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.989+0000 7f9914c00640 1 -- 172.21.15.66:0/3298320696 shutdown_connections 2024-09-10T22:03:01.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:01.989+0000 7f9914c00640 1 -- 172.21.15.66:0/3298320696 wait complete. 2024-09-10T22:03:02.016 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T22:03:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 -- 172.21.15.66:0/2661225289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd63c0c4fc0 msgr2=0x7fd63c0ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 --2- 172.21.15.66:0/2661225289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd63c0c4fc0 0x7fd63c0ca6e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd634009c80 tx=0x7fd63401cd10 comp rx=0 tx=0).stop 2024-09-10T22:03:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 -- 172.21.15.66:0/2661225289 shutdown_connections 2024-09-10T22:03:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 --2- 172.21.15.66:0/2661225289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd63c0c4fc0 0x7fd63c0ca6e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 --2- 172.21.15.66:0/2661225289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd63c0d0550 0x7fd63c0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 -- 172.21.15.66:0/2661225289 >> 172.21.15.66:0/2661225289 conn(0x7fd63c0cfa30 msgr2=0x7fd63c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 -- 172.21.15.66:0/2661225289 shutdown_connections 2024-09-10T22:03:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 -- 172.21.15.66:0/2661225289 wait complete. 2024-09-10T22:03:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 Processor -- start 2024-09-10T22:03:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 -- start start 2024-09-10T22:03:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd63c0c4fc0 0x7fd63c147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd63c15a310 0x7fd63c152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd63c1f6710 0x7fd63c14ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c167480 con 0x7fd63c1f6710 2024-09-10T22:03:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c167300 con 0x7fd63c15a310 2024-09-10T22:03:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd643600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd63c167600 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.157+0000 7fd640c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd63c0c4fc0 0x7fd63c147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd63be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd63c15a310 0x7fd63c152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd640c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd63c0c4fc0 0x7fd63c147a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:38248/0 (socket says 172.21.15.66:38248) 2024-09-10T22:03:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd641600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd63c1f6710 0x7fd63c14ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd640c00640 1 -- 172.21.15.66:0/1854994548 learned_addr learned my addr 172.21.15.66:0/1854994548 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd640c00640 1 -- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd63c15a310 msgr2=0x7fd63c152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd640c00640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd63c15a310 0x7fd63c152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd640c00640 1 -- 172.21.15.66:0/1854994548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd63c1f6710 msgr2=0x7fd63c14ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd640c00640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd63c1f6710 0x7fd63c14ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd640c00640 1 -- 172.21.15.66:0/1854994548 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c150e10 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd63be00640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd63c15a310 0x7fd63c152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:03:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd641600640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd63c1f6710 0x7fd63c14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:03:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd640c00640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd63c0c4fc0 0x7fd63c147a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd63000c3c0 tx=0x7fd63000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd638c00640 1 -- 172.21.15.66:0/1854994548 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd630017070 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd638c00640 1 -- 172.21.15.66:0/1854994548 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6300053c0 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd638c00640 1 -- 172.21.15.66:0/1854994548 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd630003860 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd643600640 1 -- 172.21.15.66:0/1854994548 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c0b2ac0 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd643600640 1 -- 172.21.15.66:0/1854994548 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c0b3080 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd643600640 1 -- 172.21.15.66:0/1854994548 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63c0b3860 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.161+0000 7fd638c00640 1 -- 172.21.15.66:0/1854994548 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fd630003a00 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.165+0000 7fd638c00640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd61c072940 0x7fd61c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.165+0000 7fd638c00640 1 -- 172.21.15.66:0/1854994548 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7fd630092b00 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.165+0000 7fd63be00640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd61c072940 0x7fd61c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.165+0000 7fd63be00640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd61c072940 0x7fd61c074e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0b4180 tx=0x7fd628010040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:02.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.169+0000 7fd638c00640 1 -- 172.21.15.66:0/1854994548 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd63005d980 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.401+0000 7fd643600640 1 -- 172.21.15.66:0/1854994548 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd63c0c48b0 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.401+0000 7fd638c00640 1 -- 172.21.15.66:0/1854994548 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) ==== 80+0+5304 (secure 0 0 0) 0x7fd6300623b0 con 0x7fd63c0c4fc0 2024-09-10T22:03:02.409 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:03:02.409 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:03:02.409 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-09-10T22:03:02.409 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.409 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-09-10T22:03:02.409 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmxQD7CBAAttZP0KtdjTI7QWbD7CVinQ==", 2024-09-10T22:03:02.409 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmI7ufFRAANelhvqylKyfGPVfBuY40kg==", 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:03:02.410 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQnVoEBAAvsZ/TSOOP/JgJBr/aHwI/g==", 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.411 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmRQkTHRAAHPIdwqtkkyh31D7IO2vxPA==", 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBCweBmDJkxJBAAVSQNiMAQl5bR3RUwVOuGYw==", 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:03:02.412 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBFweBmwcdsOxAAPPQyzktcfiF8pJNP8uAjwg==", 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.413 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBJweBmXQhZExAAnEf8mjGX6nnYwWygKjo/2A==", 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBNweBmlauHAxAA/k9xRqbHZsqJq9OYbJKfNQ==", 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:03:02.414 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBRweBmHnFDJRAAgF1ZAPvmoNzeUftZQyL0eg==", 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.415 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBUweBm/hXpORAA6jpOI/I+QyB2mZsEoAWF0w==", 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBYweBmYu2MExAAPD9WcxrBLzwwJQm19z7YcQ==", 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:03:02.416 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbweBmXfJ9KRAA/AbWlli+Qfg7gy0SblJ7fg==", 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-09-10T22:03:02.417 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw==", 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmbS6IKhAAaEmK9irxMipALgkAct/+OQ==", 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:03:02.418 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQ47KJhAAI50ZaSQIhH6FxRM48JZICA==", 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmUcLCMBAAdf4tfcpVaZoaKjS1UzrxhA==", 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA+weBmPIAzKBAAkyVV875PNhBmEoKsrAkg8w==", 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.420 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmLiy1BBAADnraML8IUet7Sw0j83TUVQ==", 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:02.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.422 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-09-10T22:03:02.422 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmItWpABAAlJ+6dFjvtBV3g+jYmUh0tQ==", 2024-09-10T22:03:02.422 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:03:02.422 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:03:02.422 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:03:02.422 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:03:02.422 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.422 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.422 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T22:03:02.422 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:03:02.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.405+0000 7fd643600640 1 -- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd61c072940 msgr2=0x7fd61c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.405+0000 7fd643600640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd61c072940 0x7fd61c074e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0b4180 tx=0x7fd628010040 comp rx=0 tx=0).stop 2024-09-10T22:03:02.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.405+0000 7fd643600640 1 -- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd63c0c4fc0 msgr2=0x7fd63c147a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.405+0000 7fd643600640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd63c0c4fc0 0x7fd63c147a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd63000c3c0 tx=0x7fd63000c880 comp rx=0 tx=0).stop 2024-09-10T22:03:02.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.405+0000 7fd643600640 1 -- 172.21.15.66:0/1854994548 shutdown_connections 2024-09-10T22:03:02.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.405+0000 7fd643600640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd61c072940 0x7fd61c074e00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.405+0000 7fd643600640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd63c1f6710 0x7fd63c14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.405+0000 7fd643600640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd63c15a310 0x7fd63c152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.405+0000 7fd643600640 1 --2- 172.21.15.66:0/1854994548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd63c0c4fc0 0x7fd63c147a30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.405+0000 7fd643600640 1 -- 172.21.15.66:0/1854994548 >> 172.21.15.66:0/1854994548 conn(0x7fd63c0cfa30 msgr2=0x7fd63c0724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:02.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.405+0000 7fd643600640 1 -- 172.21.15.66:0/1854994548 shutdown_connections 2024-09-10T22:03:02.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.405+0000 7fd643600640 1 -- 172.21.15.66:0/1854994548 wait complete. 2024-09-10T22:03:02.431 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:03:02.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.573+0000 7f2e74400640 1 -- 172.21.15.66:0/433317369 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6c0cec50 msgr2=0x7f2e6c0cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.573+0000 7f2e74400640 1 --2- 172.21.15.66:0/433317369 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6c0cec50 0x7f2e6c0cc8a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009870 tx=0x7f2e5801c860 comp rx=0 tx=0).stop 2024-09-10T22:03:02.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.573+0000 7f2e74400640 1 -- 172.21.15.66:0/433317369 shutdown_connections 2024-09-10T22:03:02.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.573+0000 7f2e74400640 1 --2- 172.21.15.66:0/433317369 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6c0cec50 0x7f2e6c0cc8a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.573+0000 7f2e74400640 1 -- 172.21.15.66:0/433317369 >> 172.21.15.66:0/433317369 conn(0x7f2e6c0d1d00 msgr2=0x7f2e6c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:02.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.573+0000 7f2e74400640 1 -- 172.21.15.66:0/433317369 shutdown_connections 2024-09-10T22:03:02.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.573+0000 7f2e74400640 1 -- 172.21.15.66:0/433317369 wait complete. 2024-09-10T22:03:02.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.573+0000 7f2e74400640 1 Processor -- start 2024-09-10T22:03:02.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.573+0000 7f2e74400640 1 -- start start 2024-09-10T22:03:02.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e74400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6c06f0c0 0x7f2e6c1593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e74400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6c0cec50 0x7f2e6c14eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e74400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e6c161610 0x7f2e6c0b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e74400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c16c1f0 con 0x7f2e6c0cec50 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e74400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c16c070 con 0x7f2e6c161610 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e74400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6c16c370 con 0x7f2e6c06f0c0 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e71000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6c0cec50 0x7f2e6c14eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e71000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6c0cec50 0x7f2e6c14eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43236/0 (socket says 172.21.15.66:43236) 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e71000640 1 -- 172.21.15.66:0/892282053 learned_addr learned my addr 172.21.15.66:0/892282053 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e71a00640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6c06f0c0 0x7f2e6c1593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e72400640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e6c161610 0x7f2e6c0b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e71000640 1 -- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6c06f0c0 msgr2=0x7f2e6c1593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e71000640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6c06f0c0 0x7f2e6c1593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e71000640 1 -- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e6c161610 msgr2=0x7f2e6c0b6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e71000640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e6c161610 0x7f2e6c0b6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e71000640 1 -- 172.21.15.66:0/892282053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c15ee30 con 0x7f2e6c0cec50 2024-09-10T22:03:02.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e71000640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6c0cec50 0x7f2e6c14eae0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2e5801af10 tx=0x7f2e580a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:02.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e69600640 1 -- 172.21.15.66:0/892282053 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e580b2040 con 0x7f2e6c0cec50 2024-09-10T22:03:02.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e69600640 1 -- 172.21.15.66:0/892282053 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e580043f0 con 0x7f2e6c0cec50 2024-09-10T22:03:02.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e69600640 1 -- 172.21.15.66:0/892282053 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58005220 con 0x7f2e6c0cec50 2024-09-10T22:03:02.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e74400640 1 -- 172.21.15.66:0/892282053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6c159a90 con 0x7f2e6c0cec50 2024-09-10T22:03:02.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e74400640 1 -- 172.21.15.66:0/892282053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6c158500 con 0x7f2e6c0cec50 2024-09-10T22:03:02.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.577+0000 7f2e74400640 1 -- 172.21.15.66:0/892282053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e6c0ce540 con 0x7f2e6c0cec50 2024-09-10T22:03:02.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.581+0000 7f2e69600640 1 -- 172.21.15.66:0/892282053 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f2e580b5070 con 0x7f2e6c0cec50 2024-09-10T22:03:02.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.581+0000 7f2e69600640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2e4c072940 0x7f2e4c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.581+0000 7f2e69600640 1 -- 172.21.15.66:0/892282053 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f2e5812cb70 con 0x7f2e6c0cec50 2024-09-10T22:03:02.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.581+0000 7f2e71a00640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2e4c072940 0x7f2e4c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.581+0000 7f2e71a00640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2e4c072940 0x7f2e4c074e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e60007fd0 tx=0x7f2e60009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:02.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.585+0000 7f2e69600640 1 -- 172.21.15.66:0/892282053 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2e58016450 con 0x7f2e6c0cec50 2024-09-10T22:03:02.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 -- 172.21.15.66:0/892282053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2e6c15aa50 con 0x7f2e6c0cec50 2024-09-10T22:03:02.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e69600640 1 -- 172.21.15.66:0/892282053 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) ==== 84+0+65 (secure 0 0 0) 0x7f2e580f7a80 con 0x7f2e6c0cec50 2024-09-10T22:03:02.766 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:03:02.766 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:03:02.767 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:02.767 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:03:02.767 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:03:02.767 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:02.767 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:03:02.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 -- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2e4c072940 msgr2=0x7f2e4c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2e4c072940 0x7f2e4c074e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e60007fd0 tx=0x7f2e60009040 comp rx=0 tx=0).stop 2024-09-10T22:03:02.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 -- 172.21.15.66:0/892282053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6c0cec50 msgr2=0x7f2e6c14eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6c0cec50 0x7f2e6c14eae0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2e5801af10 tx=0x7f2e580a5630 comp rx=0 tx=0).stop 2024-09-10T22:03:02.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 -- 172.21.15.66:0/892282053 shutdown_connections 2024-09-10T22:03:02.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2e4c072940 0x7f2e4c074e00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e6c161610 0x7f2e6c0b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6c0cec50 0x7f2e6c14eae0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 --2- 172.21.15.66:0/892282053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6c06f0c0 0x7f2e6c1593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 -- 172.21.15.66:0/892282053 >> 172.21.15.66:0/892282053 conn(0x7f2e6c0d1d00 msgr2=0x7f2e6c06f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:02.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 -- 172.21.15.66:0/892282053 shutdown_connections 2024-09-10T22:03:02.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.761+0000 7f2e74400640 1 -- 172.21.15.66:0/892282053 wait complete. 2024-09-10T22:03:02.788 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:03:02.788 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:03:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.933+0000 7f40f7e00640 1 -- 172.21.15.66:0/2988485417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40f815b9e0 msgr2=0x7f40f80d1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.933+0000 7f40f7e00640 1 --2- 172.21.15.66:0/2988485417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40f815b9e0 0x7f40f80d1940 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f40f80cc110 tx=0x7f40e801c800 comp rx=0 tx=0).stop 2024-09-10T22:03:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.933+0000 7f40f7e00640 1 -- 172.21.15.66:0/2988485417 shutdown_connections 2024-09-10T22:03:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.933+0000 7f40f7e00640 1 --2- 172.21.15.66:0/2988485417 >> v2:172.21.15.94:3300/0 conn(0x7f40f80b2a70 0x7f40f80d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.933+0000 7f40f7e00640 1 --2- 172.21.15.66:0/2988485417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40f815b9e0 0x7f40f80d1940 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.933+0000 7f40f7e00640 1 -- 172.21.15.66:0/2988485417 >> 172.21.15.66:0/2988485417 conn(0x7f40f809aa50 msgr2=0x7f40f809ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.933+0000 7f40f7e00640 1 -- 172.21.15.66:0/2988485417 shutdown_connections 2024-09-10T22:03:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.933+0000 7f40f7e00640 1 -- 172.21.15.66:0/2988485417 wait complete. 2024-09-10T22:03:02.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f7e00640 1 Processor -- start 2024-09-10T22:03:02.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f7e00640 1 -- start start 2024-09-10T22:03:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f80b2a70 0x7f40f80c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40f815b9e0 0x7f40f814dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f40f815bdc0 0x7f40f80b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8168d80 con 0x7f40f815b9e0 2024-09-10T22:03:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f7e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8168c00 con 0x7f40f80b2a70 2024-09-10T22:03:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f7e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f40f8168f00 con 0x7f40f815bdc0 2024-09-10T22:03:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40ee000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40f815b9e0 0x7f40f814dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40ee000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40f815b9e0 0x7f40f814dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43252/0 (socket says 172.21.15.66:43252) 2024-09-10T22:03:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40ee000640 1 -- 172.21.15.66:0/435731331 learned_addr learned my addr 172.21.15.66:0/435731331 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f6a00640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f80b2a70 0x7f40f80c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f7400640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f40f815bdc0 0x7f40f80b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40ee000640 1 -- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f40f815bdc0 msgr2=0x7f40f80b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40ee000640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f40f815bdc0 0x7f40f80b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40ee000640 1 -- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f80b2a70 msgr2=0x7f40f80c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40ee000640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f80b2a70 0x7f40f80c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40ee000640 1 -- 172.21.15.66:0/435731331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f8157da0 con 0x7f40f815b9e0 2024-09-10T22:03:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40ee000640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40f815b9e0 0x7f40f814dba0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c430 tx=0x7f40e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40ef400640 1 -- 172.21.15.66:0/435731331 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0011790 con 0x7f40f815b9e0 2024-09-10T22:03:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40ef400640 1 -- 172.21.15.66:0/435731331 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40e0017070 con 0x7f40f815b9e0 2024-09-10T22:03:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f7e00640 1 -- 172.21.15.66:0/435731331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f80b5510 con 0x7f40f815b9e0 2024-09-10T22:03:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40ef400640 1 -- 172.21.15.66:0/435731331 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0011cc0 con 0x7f40f815b9e0 2024-09-10T22:03:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f7e00640 1 -- 172.21.15.66:0/435731331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f814e0e0 con 0x7f40f815b9e0 2024-09-10T22:03:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.937+0000 7f40f7e00640 1 -- 172.21.15.66:0/435731331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f80ce540 con 0x7f40f815b9e0 2024-09-10T22:03:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.941+0000 7f40ef400640 1 -- 172.21.15.66:0/435731331 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f40e001c460 con 0x7f40f815b9e0 2024-09-10T22:03:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.941+0000 7f40ef400640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f40d4072940 0x7f40d4074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.941+0000 7f40ef400640 1 -- 172.21.15.66:0/435731331 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f40e0093240 con 0x7f40f815b9e0 2024-09-10T22:03:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.941+0000 7f40f6a00640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f40d4072940 0x7f40d4074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.941+0000 7f40f6a00640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f40d4072940 0x7f40d4074e00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f40e80af000 tx=0x7f40e801b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:02.945+0000 7f40ef400640 1 -- 172.21.15.66:0/435731331 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f40e005e7a0 con 0x7f40f815b9e0 2024-09-10T22:03:03.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:03.125+0000 7f40f7e00640 1 -- 172.21.15.66:0/435731331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f40f8155300 con 0x7f40f815b9e0 2024-09-10T22:03:04.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:03.997+0000 7f40ef400640 1 -- 172.21.15.66:0/435731331 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) ==== 138+0+0 (secure 0 0 0) 0x7f40e005e0a0 con 0x7f40f815b9e0 2024-09-10T22:03:04.001 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:03:04.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.001+0000 7f40f7e00640 1 -- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f40d4072940 msgr2=0x7f40d4074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.001+0000 7f40f7e00640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f40d4072940 0x7f40d4074e00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f40e80af000 tx=0x7f40e801b040 comp rx=0 tx=0).stop 2024-09-10T22:03:04.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.001+0000 7f40f7e00640 1 -- 172.21.15.66:0/435731331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40f815b9e0 msgr2=0x7f40f814dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.001+0000 7f40f7e00640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40f815b9e0 0x7f40f814dba0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c430 tx=0x7f40e000c8f0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.001+0000 7f40f7e00640 1 -- 172.21.15.66:0/435731331 shutdown_connections 2024-09-10T22:03:04.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.001+0000 7f40f7e00640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f40d4072940 0x7f40d4074e00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.001+0000 7f40f7e00640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f40f815bdc0 0x7f40f80b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.001+0000 7f40f7e00640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40f815b9e0 0x7f40f814dba0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.001+0000 7f40f7e00640 1 --2- 172.21.15.66:0/435731331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f80b2a70 0x7f40f80c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.001+0000 7f40f7e00640 1 -- 172.21.15.66:0/435731331 >> 172.21.15.66:0/435731331 conn(0x7f40f809aa50 msgr2=0x7f40f8155820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:04.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.005+0000 7f40f7e00640 1 -- 172.21.15.66:0/435731331 shutdown_connections 2024-09-10T22:03:04.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.005+0000 7f40f7e00640 1 -- 172.21.15.66:0/435731331 wait complete. 2024-09-10T22:03:04.035 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:03:04.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.189+0000 7fd01b200640 1 -- 172.21.15.66:0/2863029698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0140b32a0 msgr2=0x7fd0140d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.189+0000 7fd01b200640 1 --2- 172.21.15.66:0/2863029698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0140b32a0 0x7fd0140d0310 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009bc0 tx=0x7fd00c01c9a0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.189+0000 7fd01b200640 1 -- 172.21.15.66:0/2863029698 shutdown_connections 2024-09-10T22:03:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.189+0000 7fd01b200640 1 --2- 172.21.15.66:0/2863029698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0140b32a0 0x7fd0140d0310 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd01b200640 1 -- 172.21.15.66:0/2863029698 >> 172.21.15.66:0/2863029698 conn(0x7fd0140d1d00 msgr2=0x7fd0140d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd01b200640 1 -- 172.21.15.66:0/2863029698 shutdown_connections 2024-09-10T22:03:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd01b200640 1 -- 172.21.15.66:0/2863029698 wait complete. 2024-09-10T22:03:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd01b200640 1 Processor -- start 2024-09-10T22:03:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd01b200640 1 -- start start 2024-09-10T22:03:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd01b200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0140b0f60 0x7fd014149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd01b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0140b32a0 0x7fd0140b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd01b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd01415f460 0x7fd0140b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd01b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd014169df0 con 0x7fd01415f460 2024-09-10T22:03:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd01b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd014169c70 con 0x7fd0140b32a0 2024-09-10T22:03:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd01b200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd014169f70 con 0x7fd0140b0f60 2024-09-10T22:03:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd019200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd01415f460 0x7fd0140b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd019200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd01415f460 0x7fd0140b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43264/0 (socket says 172.21.15.66:43264) 2024-09-10T22:03:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd019200640 1 -- 172.21.15.66:0/165697368 learned_addr learned my addr 172.21.15.66:0/165697368 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd00b400640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0140b32a0 0x7fd0140b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd013e00640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0140b0f60 0x7fd014149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd019200640 1 -- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0140b0f60 msgr2=0x7fd014149170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd019200640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0140b0f60 0x7fd014149170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd019200640 1 -- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0140b32a0 msgr2=0x7fd0140b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd019200640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0140b32a0 0x7fd0140b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd019200640 1 -- 172.21.15.66:0/165697368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01414e3b0 con 0x7fd01415f460 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd00b400640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0140b32a0 0x7fd0140b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.193+0000 7fd019200640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd01415f460 0x7fd0140b3cc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0195b0 tx=0x7fd00c009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.197+0000 7fd010c00640 1 -- 172.21.15.66:0/165697368 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0bb070 con 0x7fd01415f460 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.197+0000 7fd010c00640 1 -- 172.21.15.66:0/165697368 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd00c004e00 con 0x7fd01415f460 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.197+0000 7fd010c00640 1 -- 172.21.15.66:0/165697368 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0b2040 con 0x7fd01415f460 2024-09-10T22:03:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.197+0000 7fd01b200640 1 -- 172.21.15.66:0/165697368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01414b7b0 con 0x7fd01415f460 2024-09-10T22:03:04.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.197+0000 7fd01b200640 1 -- 172.21.15.66:0/165697368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01414e950 con 0x7fd01415f460 2024-09-10T22:03:04.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.197+0000 7fd01b200640 1 -- 172.21.15.66:0/165697368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0140b71f0 con 0x7fd01415f460 2024-09-10T22:03:04.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.197+0000 7fd010c00640 1 -- 172.21.15.66:0/165697368 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fd00c0b5070 con 0x7fd01415f460 2024-09-10T22:03:04.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.197+0000 7fd010c00640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcff0072940 0x7fcff0074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.197+0000 7fd010c00640 1 -- 172.21.15.66:0/165697368 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5227+0+0 (secure 0 0 0) 0x7fd00c12cc30 con 0x7fd01415f460 2024-09-10T22:03:04.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.197+0000 7fd013e00640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcff0072940 0x7fcff0074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.201+0000 7fd013e00640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcff0072940 0x7fcff0074e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd004007fd0 tx=0x7fd004009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:04.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.201+0000 7fd010c00640 1 -- 172.21.15.66:0/165697368 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd0140b71f0 con 0x7fd01415f460 2024-09-10T22:03:04.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:04.377+0000 7fd01b200640 1 -- 172.21.15.66:0/165697368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd0140af390 con 0x7fd01415f460 2024-09-10T22:03:05.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.005+0000 7fd010c00640 1 -- 172.21.15.66:0/165697368 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) ==== 170+0+0 (secure 0 0 0) 0x7fd00c0166b0 con 0x7fd01415f460 2024-09-10T22:03:05.007 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:03:05.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.009+0000 7fd01b200640 1 -- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcff0072940 msgr2=0x7fcff0074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:05.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.009+0000 7fd01b200640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcff0072940 0x7fcff0074e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd004007fd0 tx=0x7fd004009040 comp rx=0 tx=0).stop 2024-09-10T22:03:05.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.009+0000 7fd01b200640 1 -- 172.21.15.66:0/165697368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd01415f460 msgr2=0x7fd0140b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:05.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.009+0000 7fd01b200640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd01415f460 0x7fd0140b3cc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0195b0 tx=0x7fd00c009200 comp rx=0 tx=0).stop 2024-09-10T22:03:05.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.009+0000 7fd01b200640 1 -- 172.21.15.66:0/165697368 shutdown_connections 2024-09-10T22:03:05.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.009+0000 7fd01b200640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcff0072940 0x7fcff0074e00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:05.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.009+0000 7fd01b200640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd01415f460 0x7fd0140b3cc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:05.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.009+0000 7fd01b200640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0140b32a0 0x7fd0140b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:05.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.009+0000 7fd01b200640 1 --2- 172.21.15.66:0/165697368 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0140b0f60 0x7fd014149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:05.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.009+0000 7fd01b200640 1 -- 172.21.15.66:0/165697368 >> 172.21.15.66:0/165697368 conn(0x7fd0140d1d00 msgr2=0x7fd0140d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:05.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.009+0000 7fd01b200640 1 -- 172.21.15.66:0/165697368 shutdown_connections 2024-09-10T22:03:05.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.009+0000 7fd01b200640 1 -- 172.21.15.66:0/165697368 wait complete. 2024-09-10T22:03:05.044 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:03:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.193+0000 7f4b38a00640 1 -- 172.21.15.66:0/314943894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b18005220 msgr2=0x7f4b1800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.193+0000 7f4b38a00640 1 --2- 172.21.15.66:0/314943894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b18005220 0x7f4b1800b9d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4b340b1a40 tx=0x7f4b2c01caf0 comp rx=0 tx=0).stop 2024-09-10T22:03:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.193+0000 7f4b38a00640 1 -- 172.21.15.66:0/314943894 shutdown_connections 2024-09-10T22:03:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.193+0000 7f4b38a00640 1 --2- 172.21.15.66:0/314943894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b18005220 0x7f4b1800b9d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.193+0000 7f4b38a00640 1 -- 172.21.15.66:0/314943894 >> 172.21.15.66:0/314943894 conn(0x7f4b340d1b10 msgr2=0x7f4b340d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.193+0000 7f4b38a00640 1 -- 172.21.15.66:0/314943894 shutdown_connections 2024-09-10T22:03:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.193+0000 7f4b38a00640 1 -- 172.21.15.66:0/314943894 wait complete. 2024-09-10T22:03:05.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b38a00640 1 Processor -- start 2024-09-10T22:03:05.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b38a00640 1 -- start start 2024-09-10T22:03:05.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b38a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b18005220 0x7f4b34157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:05.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b38a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b340af850 0x7f4b340afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:05.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b38a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b341f7d00 0x7f4b341f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:05.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b38a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4b340b1420 con 0x7f4b340af850 2024-09-10T22:03:05.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b38a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b340b12a0 con 0x7f4b341f7d00 2024-09-10T22:03:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b38a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4b340cecc0 con 0x7f4b18005220 2024-09-10T22:03:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b31e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b340af850 0x7f4b340afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b31e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b340af850 0x7f4b340afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43276/0 (socket says 172.21.15.66:43276) 2024-09-10T22:03:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b31e00640 1 -- 172.21.15.66:0/1500619694 learned_addr learned my addr 172.21.15.66:0/1500619694 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b32800640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b18005220 0x7f4b34157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b33200640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b341f7d00 0x7f4b341f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b31e00640 1 -- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b18005220 msgr2=0x7f4b34157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b31e00640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b18005220 0x7f4b34157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b31e00640 1 -- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b341f7d00 msgr2=0x7f4b341f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b31e00640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b341f7d00 0x7f4b341f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b31e00640 1 -- 172.21.15.66:0/1500619694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b341f89f0 con 0x7f4b340af850 2024-09-10T22:03:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b31e00640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b340af850 0x7f4b340afcb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4b200118d0 tx=0x7f4b20011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:05.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b2aa00640 1 -- 172.21.15.66:0/1500619694 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2001e070 con 0x7f4b340af850 2024-09-10T22:03:05.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b2aa00640 1 -- 172.21.15.66:0/1500619694 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b2000fb50 con 0x7f4b340af850 2024-09-10T22:03:05.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b2aa00640 1 -- 172.21.15.66:0/1500619694 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20019410 con 0x7f4b340af850 2024-09-10T22:03:05.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b38a00640 1 -- 172.21.15.66:0/1500619694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b340ca0d0 con 0x7f4b340af850 2024-09-10T22:03:05.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.197+0000 7f4b38a00640 1 -- 172.21.15.66:0/1500619694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b340ca690 con 0x7f4b340af850 2024-09-10T22:03:05.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.201+0000 7f4b38a00640 1 -- 172.21.15.66:0/1500619694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b340cc620 con 0x7f4b340af850 2024-09-10T22:03:05.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.201+0000 7f4b2aa00640 1 -- 172.21.15.66:0/1500619694 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f4b200195b0 con 0x7f4b340af850 2024-09-10T22:03:05.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.201+0000 7f4b2aa00640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b14072940 0x7f4b14074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:05.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.201+0000 7f4b2aa00640 1 -- 172.21.15.66:0/1500619694 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5609+0+0 (secure 0 0 0) 0x7f4b20094340 con 0x7f4b340af850 2024-09-10T22:03:05.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.201+0000 7f4b32800640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b14072940 0x7f4b14074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:05.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.201+0000 7f4b32800640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b14072940 0x7f4b14074e00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4b24006c20 tx=0x7f4b240067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:05.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.205+0000 7f4b2aa00640 1 -- 172.21.15.66:0/1500619694 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4b2005ee90 con 0x7f4b340af850 2024-09-10T22:03:05.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:05.413+0000 7f4b38a00640 1 -- 172.21.15.66:0/1500619694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4b34072e00 con 0x7f4b340af850 2024-09-10T22:03:06.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.021+0000 7f4b2aa00640 1 -- 172.21.15.66:0/1500619694 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) ==== 172+0+0 (secure 0 0 0) 0x7f4b200638c0 con 0x7f4b340af850 2024-09-10T22:03:06.029 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 6 and data pool 7 2024-09-10T22:03:06.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.033+0000 7f4b38a00640 1 -- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b14072940 msgr2=0x7f4b14074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.033+0000 7f4b38a00640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b14072940 0x7f4b14074e00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4b24006c20 tx=0x7f4b240067b0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.033+0000 7f4b38a00640 1 -- 172.21.15.66:0/1500619694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b340af850 msgr2=0x7f4b340afcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.033+0000 7f4b38a00640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b340af850 0x7f4b340afcb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4b200118d0 tx=0x7f4b20011d90 comp rx=0 tx=0).stop 2024-09-10T22:03:06.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.033+0000 7f4b38a00640 1 -- 172.21.15.66:0/1500619694 shutdown_connections 2024-09-10T22:03:06.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.033+0000 7f4b38a00640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b14072940 0x7f4b14074e00 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4b24006c20 tx=0x7f4b240067b0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.033+0000 7f4b38a00640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b341f7d00 0x7f4b341f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.033+0000 7f4b38a00640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b340af850 0x7f4b340afcb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.033+0000 7f4b38a00640 1 --2- 172.21.15.66:0/1500619694 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b18005220 0x7f4b34157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.033+0000 7f4b38a00640 1 -- 172.21.15.66:0/1500619694 >> 172.21.15.66:0/1500619694 conn(0x7f4b340d1b10 msgr2=0x7f4b3406dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:06.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.033+0000 7f4b38a00640 1 -- 172.21.15.66:0/1500619694 shutdown_connections 2024-09-10T22:03:06.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.033+0000 7f4b38a00640 1 -- 172.21.15.66:0/1500619694 wait complete. 2024-09-10T22:03:06.070 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:03:06.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.217+0000 7f2c81c00640 1 -- 172.21.15.66:0/3739637838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c0d1b80 msgr2=0x7f2c7c0b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.217+0000 7f2c81c00640 1 --2- 172.21.15.66:0/3739637838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c0d1b80 0x7f2c7c0b0a10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c08e200 tx=0x7f2c6401c8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.217+0000 7f2c81c00640 1 -- 172.21.15.66:0/3739637838 shutdown_connections 2024-09-10T22:03:06.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.217+0000 7f2c81c00640 1 --2- 172.21.15.66:0/3739637838 >> v2:172.21.15.94:3300/0 conn(0x7f2c7c0d28c0 0x7f2c7c0b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.217+0000 7f2c81c00640 1 --2- 172.21.15.66:0/3739637838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c0d1b80 0x7f2c7c0b0a10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.217+0000 7f2c81c00640 1 --2- 172.21.15.66:0/3739637838 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c162630 0x7f2c7c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.217+0000 7f2c81c00640 1 -- 172.21.15.66:0/3739637838 >> 172.21.15.66:0/3739637838 conn(0x7f2c7c0d3da0 msgr2=0x7f2c7c0d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:06.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.217+0000 7f2c81c00640 1 -- 172.21.15.66:0/3739637838 shutdown_connections 2024-09-10T22:03:06.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.217+0000 7f2c81c00640 1 -- 172.21.15.66:0/3739637838 wait complete. 2024-09-10T22:03:06.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.217+0000 7f2c81c00640 1 Processor -- start 2024-09-10T22:03:06.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.217+0000 7f2c81c00640 1 -- start start 2024-09-10T22:03:06.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c81c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c0d28c0 0x7f2c7c14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c81c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c162630 0x7f2c7c152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c81c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c14f3f0 0x7f2c7c160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c81c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c0b44c0 con 0x7f2c7c14f3f0 2024-09-10T22:03:06.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c81c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c0b4340 con 0x7f2c7c0d28c0 2024-09-10T22:03:06.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c81c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2c7c0b4640 con 0x7f2c7c162630 2024-09-10T22:03:06.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c81200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c14f3f0 0x7f2c7c160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c7be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c0d28c0 0x7f2c7c14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c73e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c162630 0x7f2c7c152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c73e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c162630 0x7f2c7c152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:48698/0 (socket says 172.21.15.66:48698) 2024-09-10T22:03:06.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c7be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c0d28c0 0x7f2c7c14ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:34400/0 (socket says 172.21.15.66:34400) 2024-09-10T22:03:06.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c73e00640 1 -- 172.21.15.66:0/2449522163 learned_addr learned my addr 172.21.15.66:0/2449522163 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:06.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c81200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c14f3f0 0x7f2c7c160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43304/0 (socket says 172.21.15.66:43304) 2024-09-10T22:03:06.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c73e00640 1 -- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c0d28c0 msgr2=0x7f2c7c14ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c73e00640 1 --2- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c0d28c0 0x7f2c7c14ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c73e00640 1 -- 172.21.15.66:0/2449522163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c14f3f0 msgr2=0x7f2c7c160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c73e00640 1 --2- 172.21.15.66:0/2449522163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c14f3f0 0x7f2c7c160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c73e00640 1 -- 172.21.15.66:0/2449522163 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c161720 con 0x7f2c7c162630 2024-09-10T22:03:06.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c73e00640 1 --2- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c162630 0x7f2c7c152bb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c640b0040 tx=0x7f2c640a55c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:06.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c78c00640 1 -- 172.21.15.66:0/2449522163 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c640b2040 con 0x7f2c7c162630 2024-09-10T22:03:06.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c78c00640 1 -- 172.21.15.66:0/2449522163 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c64004470 con 0x7f2c7c162630 2024-09-10T22:03:06.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c78c00640 1 -- 172.21.15.66:0/2449522163 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c640052d0 con 0x7f2c7c162630 2024-09-10T22:03:06.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c81c00640 1 -- 172.21.15.66:0/2449522163 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7c1520d0 con 0x7f2c7c162630 2024-09-10T22:03:06.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c81c00640 1 -- 172.21.15.66:0/2449522163 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7c152690 con 0x7f2c7c162630 2024-09-10T22:03:06.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.221+0000 7f2c81c00640 1 -- 172.21.15.66:0/2449522163 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7c152860 con 0x7f2c7c162630 2024-09-10T22:03:06.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.225+0000 7f2c78c00640 1 -- 172.21.15.66:0/2449522163 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f2c64005470 con 0x7f2c7c162630 2024-09-10T22:03:06.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.225+0000 7f2c78c00640 1 --2- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c58072940 0x7f2c58074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.225+0000 7f2c78c00640 1 -- 172.21.15.66:0/2449522163 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f2c6412d250 con 0x7f2c7c162630 2024-09-10T22:03:06.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.225+0000 7f2c7be00640 1 --2- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c58072940 0x7f2c58074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.225+0000 7f2c7be00640 1 --2- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c58072940 0x7f2c58074e00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00bb50 tx=0x7f2c6c00b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:06.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.229+0000 7f2c78c00640 1 -- 172.21.15.66:0/2449522163 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2c640164b0 con 0x7f2c7c162630 2024-09-10T22:03:06.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.405+0000 7f2c81c00640 1 -- 172.21.15.66:0/2449522163 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c7c08dd40 con 0x7f2c7c162630 2024-09-10T22:03:06.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.405+0000 7f2c78c00640 1 -- 172.21.15.66:0/2449522163 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16614 (secure 0 0 0) 0x7f2c640f7c90 con 0x7f2c7c162630 2024-09-10T22:03:06.407 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:03:06.408 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":43,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:03:05.416767+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:03:03.140557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T22:03:04.388164+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:03:06.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.409+0000 7f2c81c00640 1 -- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c58072940 msgr2=0x7f2c58074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.409+0000 7f2c81c00640 1 --2- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c58072940 0x7f2c58074e00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00bb50 tx=0x7f2c6c00b2e0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.409+0000 7f2c81c00640 1 -- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c162630 msgr2=0x7f2c7c152bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.409+0000 7f2c81c00640 1 --2- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c162630 0x7f2c7c152bb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c640b0040 tx=0x7f2c640a55c0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.409+0000 7f2c81c00640 1 -- 172.21.15.66:0/2449522163 shutdown_connections 2024-09-10T22:03:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.409+0000 7f2c81c00640 1 --2- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c58072940 0x7f2c58074e00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.409+0000 7f2c81c00640 1 --2- 172.21.15.66:0/2449522163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c7c14f3f0 0x7f2c7c160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.409+0000 7f2c81c00640 1 --2- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c162630 0x7f2c7c152bb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.409+0000 7f2c81c00640 1 --2- 172.21.15.66:0/2449522163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c0d28c0 0x7f2c7c14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.409+0000 7f2c81c00640 1 -- 172.21.15.66:0/2449522163 >> 172.21.15.66:0/2449522163 conn(0x7f2c7c0d3da0 msgr2=0x7f2c7c15e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.409+0000 7f2c81c00640 1 -- 172.21.15.66:0/2449522163 shutdown_connections 2024-09-10T22:03:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.409+0000 7f2c81c00640 1 -- 172.21.15.66:0/2449522163 wait complete. 2024-09-10T22:03:06.429 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:03:06.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.573+0000 7fd821a00640 1 -- 172.21.15.66:0/3025889374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd800003a10 msgr2=0x7fd800003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.573+0000 7fd821a00640 1 --2- 172.21.15.66:0/3025889374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd800003a10 0x7fd800003e30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00c310 tx=0x7fd80c01cf90 comp rx=0 tx=0).stop 2024-09-10T22:03:06.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.573+0000 7fd81b400640 1 -- 172.21.15.66:0/3025889374 reap_dead start 2024-09-10T22:03:06.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.573+0000 7fd821a00640 1 -- 172.21.15.66:0/3025889374 shutdown_connections 2024-09-10T22:03:06.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.573+0000 7fd821a00640 1 -- 172.21.15.66:0/3025889374 >> 172.21.15.66:0/3025889374 conn(0x7fd81c0cfa30 msgr2=0x7fd81c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:06.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.573+0000 7fd821a00640 1 -- 172.21.15.66:0/3025889374 shutdown_connections 2024-09-10T22:03:06.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.573+0000 7fd821a00640 1 -- 172.21.15.66:0/3025889374 wait complete. 2024-09-10T22:03:06.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd821a00640 1 Processor -- start 2024-09-10T22:03:06.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd821a00640 1 -- start start 2024-09-10T22:03:06.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd821a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd800003a10 0x7fd81c151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd821a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd81c0b44c0 0x7fd81c1505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd821a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c15fdd0 0x7fd81c160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd821a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c164b00 con 0x7fd800003a10 2024-09-10T22:03:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd821a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c164980 con 0x7fd81c15fdd0 2024-09-10T22:03:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd821a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd81c164c80 con 0x7fd81c0b44c0 2024-09-10T22:03:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd81aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd800003a10 0x7fd81c151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd81aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd800003a10 0x7fd81c151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43326/0 (socket says 172.21.15.66:43326) 2024-09-10T22:03:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd81aa00640 1 -- 172.21.15.66:0/216855981 learned_addr learned my addr 172.21.15.66:0/216855981 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd81b400640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c15fdd0 0x7fd81c160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd813e00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd81c0b44c0 0x7fd81c1505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd81aa00640 1 -- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd81c0b44c0 msgr2=0x7fd81c1505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd81aa00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd81c0b44c0 0x7fd81c1505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd81aa00640 1 -- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c15fdd0 msgr2=0x7fd81c160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd81aa00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c15fdd0 0x7fd81c160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd81aa00640 1 -- 172.21.15.66:0/216855981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c151ef0 con 0x7fd800003a10 2024-09-10T22:03:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd81aa00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd800003a10 0x7fd81c151910 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd80c01b690 tx=0x7fd80c006cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd812a00640 1 -- 172.21.15.66:0/216855981 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0060c0 con 0x7fd800003a10 2024-09-10T22:03:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd812a00640 1 -- 172.21.15.66:0/216855981 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd80c003600 con 0x7fd800003a10 2024-09-10T22:03:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd812a00640 1 -- 172.21.15.66:0/216855981 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c003a10 con 0x7fd800003a10 2024-09-10T22:03:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd821a00640 1 -- 172.21.15.66:0/216855981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81c1521d0 con 0x7fd800003a10 2024-09-10T22:03:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.577+0000 7fd821a00640 1 -- 172.21.15.66:0/216855981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81c152790 con 0x7fd800003a10 2024-09-10T22:03:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.581+0000 7fd821a00640 1 -- 172.21.15.66:0/216855981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd81c06e630 con 0x7fd800003a10 2024-09-10T22:03:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.581+0000 7fd812a00640 1 -- 172.21.15.66:0/216855981 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fd80c0a5ce0 con 0x7fd800003a10 2024-09-10T22:03:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.581+0000 7fd812a00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd7f8072940 0x7fd7f8074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.581+0000 7fd812a00640 1 -- 172.21.15.66:0/216855981 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fd80c12dc40 con 0x7fd800003a10 2024-09-10T22:03:06.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.581+0000 7fd813e00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd7f8072940 0x7fd7f8074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.581+0000 7fd813e00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd7f8072940 0x7fd7f8074e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0cf5f0 tx=0x7fd8040091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:06.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.585+0000 7fd812a00640 1 -- 172.21.15.66:0/216855981 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd80c016d70 con 0x7fd800003a10 2024-09-10T22:03:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.765+0000 7fd821a00640 1 -- 172.21.15.66:0/216855981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd81c150b00 con 0x7fd800003a10 2024-09-10T22:03:06.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.765+0000 7fd812a00640 1 -- 172.21.15.66:0/216855981 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16614 (secure 0 0 0) 0x7fd80c006260 con 0x7fd800003a10 2024-09-10T22:03:06.767 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:03:06.767 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":43,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:03:05.416767+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:03:03.140557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T22:03:04.388164+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:03:06.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.765+0000 7fd821a00640 1 -- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd7f8072940 msgr2=0x7fd7f8074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.765+0000 7fd821a00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd7f8072940 0x7fd7f8074e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0cf5f0 tx=0x7fd8040091e0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.765+0000 7fd821a00640 1 -- 172.21.15.66:0/216855981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd800003a10 msgr2=0x7fd81c151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.765+0000 7fd821a00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd800003a10 0x7fd81c151910 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd80c01b690 tx=0x7fd80c006cf0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.765+0000 7fd821a00640 1 -- 172.21.15.66:0/216855981 shutdown_connections 2024-09-10T22:03:06.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.765+0000 7fd821a00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd7f8072940 0x7fd7f8074e00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.765+0000 7fd821a00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c15fdd0 0x7fd81c160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.765+0000 7fd821a00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd81c0b44c0 0x7fd81c1505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.765+0000 7fd821a00640 1 --2- 172.21.15.66:0/216855981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd800003a10 0x7fd81c151910 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.765+0000 7fd821a00640 1 -- 172.21.15.66:0/216855981 >> 172.21.15.66:0/216855981 conn(0x7fd81c0cfa30 msgr2=0x7fd81c150da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.769+0000 7fd821a00640 1 -- 172.21.15.66:0/216855981 shutdown_connections 2024-09-10T22:03:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.769+0000 7fd821a00640 1 -- 172.21.15.66:0/216855981 wait complete. 2024-09-10T22:03:06.790 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:03:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.937+0000 7f197bc00640 1 -- 172.21.15.66:0/4188686013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740b2bf0 msgr2=0x7f19740d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.937+0000 7f197bc00640 1 --2- 172.21.15.66:0/4188686013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740b2bf0 0x7f19740d0090 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f197406d720 tx=0x7f196001c600 comp rx=0 tx=0).stop 2024-09-10T22:03:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.937+0000 7f197bc00640 1 -- 172.21.15.66:0/4188686013 shutdown_connections 2024-09-10T22:03:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.937+0000 7f197bc00640 1 --2- 172.21.15.66:0/4188686013 >> v2:172.21.15.94:3300/0 conn(0x7f19740d2770 0x7f197415fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.937+0000 7f197bc00640 1 --2- 172.21.15.66:0/4188686013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740b2bf0 0x7f19740d0090 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.937+0000 7f197bc00640 1 --2- 172.21.15.66:0/4188686013 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f197415f340 0x7f19740cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.937+0000 7f197bc00640 1 -- 172.21.15.66:0/4188686013 >> 172.21.15.66:0/4188686013 conn(0x7f19740d1c50 msgr2=0x7f19740d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.937+0000 7f197bc00640 1 -- 172.21.15.66:0/4188686013 shutdown_connections 2024-09-10T22:03:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.937+0000 7f197bc00640 1 -- 172.21.15.66:0/4188686013 wait complete. 2024-09-10T22:03:06.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f197bc00640 1 Processor -- start 2024-09-10T22:03:07.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f197bc00640 1 -- start start 2024-09-10T22:03:07.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f197bc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740b2bf0 0x7f197414f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:07.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f197bc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19740d2770 0x7f19740b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:07.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f197bc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f197415f340 0x7f19740b7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:07.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f197bc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19740aecf0 con 0x7f19740b2bf0 2024-09-10T22:03:07.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f197bc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19740aeb70 con 0x7f19740d2770 2024-09-10T22:03:07.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f197bc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f19740aee70 con 0x7f197415f340 2024-09-10T22:03:07.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1979200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740b2bf0 0x7f197414f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:07.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1979200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740b2bf0 0x7f197414f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43336/0 (socket says 172.21.15.66:43336) 2024-09-10T22:03:07.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1979200640 1 -- 172.21.15.66:0/2919354289 learned_addr learned my addr 172.21.15.66:0/2919354289 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:07.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1979c00640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f197415f340 0x7f19740b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:07.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1973e00640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19740d2770 0x7f19740b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:07.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1979200640 1 -- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f197415f340 msgr2=0x7f19740b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:07.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1979200640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f197415f340 0x7f19740b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:07.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1979200640 1 -- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19740d2770 msgr2=0x7f19740b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:07.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1979200640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19740d2770 0x7f19740b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:07.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1979200640 1 -- 172.21.15.66:0/2919354289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19740b76e0 con 0x7f19740b2bf0 2024-09-10T22:03:07.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1979200640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740b2bf0 0x7f197414f700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f19680058d0 tx=0x7f196800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:07.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1970c00640 1 -- 172.21.15.66:0/2919354289 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968017070 con 0x7f19740b2bf0 2024-09-10T22:03:07.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1970c00640 1 -- 172.21.15.66:0/2919354289 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1968009d20 con 0x7f19740b2bf0 2024-09-10T22:03:07.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f1970c00640 1 -- 172.21.15.66:0/2919354289 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19680106c0 con 0x7f19740b2bf0 2024-09-10T22:03:07.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f197bc00640 1 -- 172.21.15.66:0/2919354289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1974156940 con 0x7f19740b2bf0 2024-09-10T22:03:07.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f197bc00640 1 -- 172.21.15.66:0/2919354289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1974156f00 con 0x7f19740b2bf0 2024-09-10T22:03:07.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.941+0000 7f197bc00640 1 -- 172.21.15.66:0/2919354289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f197415fac0 con 0x7f19740b2bf0 2024-09-10T22:03:07.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.945+0000 7f1970c00640 1 -- 172.21.15.66:0/2919354289 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f1968010860 con 0x7f19740b2bf0 2024-09-10T22:03:07.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.945+0000 7f1970c00640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1954072940 0x7f1954074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:07.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.945+0000 7f1970c00640 1 -- 172.21.15.66:0/2919354289 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f1968093d80 con 0x7f19740b2bf0 2024-09-10T22:03:07.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.945+0000 7f1973e00640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1954072940 0x7f1954074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:07.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.945+0000 7f1973e00640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1954072940 0x7f1954074e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f19600af000 tx=0x7f19600b2040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:07.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:06.949+0000 7f1970c00640 1 -- 172.21.15.66:0/2919354289 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f196805e7b0 con 0x7f19740b2bf0 2024-09-10T22:03:07.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:07.165+0000 7f197bc00640 1 -- 172.21.15.66:0/2919354289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f19740d22d0 con 0x7f19740b2bf0 2024-09-10T22:03:07.996 INFO:tasks.ceph.mds.a.smithi066.stderr:2024-09-10T22:03:07.993+0000 7fd746600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:03:08.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.021+0000 7f1970c00640 1 -- 172.21.15.66:0/2919354289 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f19680631e0 con 0x7f19740b2bf0 2024-09-10T22:03:08.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.025+0000 7f197bc00640 1 -- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1954072940 msgr2=0x7f1954074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:08.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.025+0000 7f197bc00640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1954072940 0x7f1954074e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f19600af000 tx=0x7f19600b2040 comp rx=0 tx=0).stop 2024-09-10T22:03:08.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.025+0000 7f197bc00640 1 -- 172.21.15.66:0/2919354289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740b2bf0 msgr2=0x7f197414f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:08.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.025+0000 7f197bc00640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740b2bf0 0x7f197414f700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f19680058d0 tx=0x7f196800dde0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.025+0000 7f197bc00640 1 -- 172.21.15.66:0/2919354289 shutdown_connections 2024-09-10T22:03:08.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.025+0000 7f197bc00640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1954072940 0x7f1954074e00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.025+0000 7f197bc00640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f197415f340 0x7f19740b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.025+0000 7f197bc00640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19740d2770 0x7f19740b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.025+0000 7f197bc00640 1 --2- 172.21.15.66:0/2919354289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740b2bf0 0x7f197414f700 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.025+0000 7f197bc00640 1 -- 172.21.15.66:0/2919354289 >> 172.21.15.66:0/2919354289 conn(0x7f19740d1c50 msgr2=0x7f1974071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:08.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.025+0000 7f197bc00640 1 -- 172.21.15.66:0/2919354289 shutdown_connections 2024-09-10T22:03:08.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.025+0000 7f197bc00640 1 -- 172.21.15.66:0/2919354289 wait complete. 2024-09-10T22:03:08.049 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:03:08.049 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:03:08.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.201+0000 7fbe74000640 1 -- 172.21.15.66:0/1496382758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe540052f0 msgr2=0x7fbe5400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:08.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.201+0000 7fbe74000640 1 --2- 172.21.15.66:0/1496382758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe540052f0 0x7fbe5400b9d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400c610 tx=0x7fbe6401f9c0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.201+0000 7fbe74000640 1 -- 172.21.15.66:0/1496382758 shutdown_connections 2024-09-10T22:03:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.201+0000 7fbe74000640 1 --2- 172.21.15.66:0/1496382758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe540052f0 0x7fbe5400b9d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.201+0000 7fbe74000640 1 -- 172.21.15.66:0/1496382758 >> 172.21.15.66:0/1496382758 conn(0x7fbe6c0cfa30 msgr2=0x7fbe6c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.201+0000 7fbe74000640 1 -- 172.21.15.66:0/1496382758 shutdown_connections 2024-09-10T22:03:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.201+0000 7fbe74000640 1 -- 172.21.15.66:0/1496382758 wait complete. 2024-09-10T22:03:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.201+0000 7fbe74000640 1 Processor -- start 2024-09-10T22:03:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.201+0000 7fbe74000640 1 -- start start 2024-09-10T22:03:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.201+0000 7fbe74000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe540052f0 0x7fbe6c141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe74000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe6c141810 0x7fbe6c0b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe74000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe6c14f140 0x7fbe6c14f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe74000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c0ae8e0 con 0x7fbe6c14f140 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe74000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c0af0d0 con 0x7fbe6c141810 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe74000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6c0aea60 con 0x7fbe540052f0 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe72000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe6c14f140 0x7fbe6c14f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe72000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe6c14f140 0x7fbe6c14f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43352/0 (socket says 172.21.15.66:43352) 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe72000640 1 -- 172.21.15.66:0/494086837 learned_addr learned my addr 172.21.15.66:0/494086837 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe71600640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe540052f0 0x7fbe6c141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe70c00640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe6c141810 0x7fbe6c0b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe72000640 1 -- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe540052f0 msgr2=0x7fbe6c141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe72000640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe540052f0 0x7fbe6c141230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe72000640 1 -- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe6c141810 msgr2=0x7fbe6c0b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe72000640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe6c141810 0x7fbe6c0b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe72000640 1 -- 172.21.15.66:0/494086837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c0cf810 con 0x7fbe6c14f140 2024-09-10T22:03:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe71600640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe540052f0 0x7fbe6c141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe70c00640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe6c141810 0x7fbe6c0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe72000640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe6c14f140 0x7fbe6c14f5c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400c610 tx=0x7fbe64001fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe69600640 1 -- 172.21.15.66:0/494086837 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64003b80 con 0x7fbe6c14f140 2024-09-10T22:03:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe69600640 1 -- 172.21.15.66:0/494086837 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe64003e30 con 0x7fbe6c14f140 2024-09-10T22:03:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe69600640 1 -- 172.21.15.66:0/494086837 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64006330 con 0x7fbe6c14f140 2024-09-10T22:03:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe74000640 1 -- 172.21.15.66:0/494086837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6c0b3ac0 con 0x7fbe6c14f140 2024-09-10T22:03:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe74000640 1 -- 172.21.15.66:0/494086837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6c0b4080 con 0x7fbe6c14f140 2024-09-10T22:03:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe74000640 1 -- 172.21.15.66:0/494086837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe6c0c4a90 con 0x7fbe6c14f140 2024-09-10T22:03:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.205+0000 7fbe69600640 1 -- 172.21.15.66:0/494086837 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fbe640064d0 con 0x7fbe6c14f140 2024-09-10T22:03:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.209+0000 7fbe69600640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbe4c072940 0x7fbe4c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.209+0000 7fbe69600640 1 -- 172.21.15.66:0/494086837 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fbe64130ff0 con 0x7fbe6c14f140 2024-09-10T22:03:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.209+0000 7fbe71600640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbe4c072940 0x7fbe4c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.209+0000 7fbe71600640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbe4c072940 0x7fbe4c074e00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe60000c10 tx=0x7fbe60006b80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.213+0000 7fbe69600640 1 -- 172.21.15.66:0/494086837 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbe64019770 con 0x7fbe6c14f140 2024-09-10T22:03:08.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:08.417+0000 7fbe74000640 1 -- 172.21.15.66:0/494086837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbe6c070420 con 0x7fbe6c14f140 2024-09-10T22:03:09.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.025+0000 7fbe69600640 1 -- 172.21.15.66:0/494086837 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fbe640fc210 con 0x7fbe6c14f140 2024-09-10T22:03:09.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.025+0000 7fbe74000640 1 -- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbe4c072940 msgr2=0x7fbe4c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.025+0000 7fbe74000640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbe4c072940 0x7fbe4c074e00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe60000c10 tx=0x7fbe60006b80 comp rx=0 tx=0).stop 2024-09-10T22:03:09.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.029+0000 7fbe74000640 1 -- 172.21.15.66:0/494086837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe6c14f140 msgr2=0x7fbe6c14f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.029+0000 7fbe74000640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe6c14f140 0x7fbe6c14f5c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400c610 tx=0x7fbe64001fd0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.029+0000 7fbe74000640 1 -- 172.21.15.66:0/494086837 shutdown_connections 2024-09-10T22:03:09.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.029+0000 7fbe74000640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbe4c072940 0x7fbe4c074e00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.029+0000 7fbe74000640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe6c14f140 0x7fbe6c14f5c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.029+0000 7fbe74000640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe6c141810 0x7fbe6c0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.029+0000 7fbe74000640 1 --2- 172.21.15.66:0/494086837 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe540052f0 0x7fbe6c141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.029+0000 7fbe74000640 1 -- 172.21.15.66:0/494086837 >> 172.21.15.66:0/494086837 conn(0x7fbe6c0cfa30 msgr2=0x7fbe6c0b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:09.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.029+0000 7fbe74000640 1 -- 172.21.15.66:0/494086837 shutdown_connections 2024-09-10T22:03:09.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.029+0000 7fbe74000640 1 -- 172.21.15.66:0/494086837 wait complete. 2024-09-10T22:03:09.053 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:03:09.053 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:03:09.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.201+0000 7f9cf6200640 1 -- 172.21.15.66:0/1868145082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cf00d0330 msgr2=0x7f9cf015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.201+0000 7f9cf6200640 1 --2- 172.21.15.66:0/1868145082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cf00d0330 0x7f9cf015e680 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009840 tx=0x7f9ce001c640 comp rx=0 tx=0).stop 2024-09-10T22:03:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.201+0000 7f9cf6200640 1 -- 172.21.15.66:0/1868145082 shutdown_connections 2024-09-10T22:03:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.201+0000 7f9cf6200640 1 --2- 172.21.15.66:0/1868145082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cf00d0330 0x7f9cf015e680 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.201+0000 7f9cf6200640 1 -- 172.21.15.66:0/1868145082 >> 172.21.15.66:0/1868145082 conn(0x7f9cf00d0b90 msgr2=0x7f9cf00d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.201+0000 7f9cf6200640 1 -- 172.21.15.66:0/1868145082 shutdown_connections 2024-09-10T22:03:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.201+0000 7f9cf6200640 1 -- 172.21.15.66:0/1868145082 wait complete. 2024-09-10T22:03:09.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.201+0000 7f9cf6200640 1 Processor -- start 2024-09-10T22:03:09.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cf6200640 1 -- start start 2024-09-10T22:03:09.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cf6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cf00d0330 0x7f9cf013b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:09.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cf6200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cf00d0710 0x7f9cf014d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cf6200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf015d370 0x7f9cf00cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cf6200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0168d70 con 0x7f9cf00d0330 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cf6200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0168bf0 con 0x7f9cf015d370 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cf6200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf0168ef0 con 0x7f9cf00d0710 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cef400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cf00d0330 0x7f9cf013b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cef400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cf00d0330 0x7f9cf013b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43362/0 (socket says 172.21.15.66:43362) 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cef400640 1 -- 172.21.15.66:0/697633261 learned_addr learned my addr 172.21.15.66:0/697633261 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cefe00640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf015d370 0x7f9cf00cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9ce6a00640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cf00d0710 0x7f9cf014d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cef400640 1 -- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cf00d0710 msgr2=0x7f9cf014d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cef400640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cf00d0710 0x7f9cf014d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cef400640 1 -- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf015d370 msgr2=0x7f9cf00cf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cef400640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf015d370 0x7f9cf00cf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cef400640 1 -- 172.21.15.66:0/697633261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf00b47d0 con 0x7f9cf00d0330 2024-09-10T22:03:09.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cefe00640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf015d370 0x7f9cf00cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:09.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cef400640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cf00d0330 0x7f9cf013b420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009790 tx=0x7f9ce0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:09.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9ce7e00640 1 -- 172.21.15.66:0/697633261 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00046d0 con 0x7f9cf00d0330 2024-09-10T22:03:09.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9ce7e00640 1 -- 172.21.15.66:0/697633261 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ce0004870 con 0x7f9cf00d0330 2024-09-10T22:03:09.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cf6200640 1 -- 172.21.15.66:0/697633261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf014cb40 con 0x7f9cf00d0330 2024-09-10T22:03:09.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9ce7e00640 1 -- 172.21.15.66:0/697633261 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0004c60 con 0x7f9cf00d0330 2024-09-10T22:03:09.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cf6200640 1 -- 172.21.15.66:0/697633261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf014d100 con 0x7f9cf00d0330 2024-09-10T22:03:09.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.205+0000 7f9cf6200640 1 -- 172.21.15.66:0/697633261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf015e680 con 0x7f9cf00d0330 2024-09-10T22:03:09.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.209+0000 7f9ce7e00640 1 -- 172.21.15.66:0/697633261 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f9ce00c3060 con 0x7f9cf00d0330 2024-09-10T22:03:09.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.209+0000 7f9ce7e00640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ccc072940 0x7f9ccc074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:09.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.209+0000 7f9ce7e00640 1 -- 172.21.15.66:0/697633261 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9ce012daa0 con 0x7f9cf00d0330 2024-09-10T22:03:09.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.209+0000 7f9ce6a00640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ccc072940 0x7f9ccc074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:09.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.209+0000 7f9ce6a00640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ccc072940 0x7f9ccc074e00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cf014c950 tx=0x7f9cd8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:09.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.213+0000 7f9ce7e00640 1 -- 172.21.15.66:0/697633261 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9ce00ac030 con 0x7f9cf00d0330 2024-09-10T22:03:09.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9cf6200640 1 -- 172.21.15.66:0/697633261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cf0157850 con 0x7f9cf00d0330 2024-09-10T22:03:09.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9ce7e00640 1 -- 172.21.15.66:0/697633261 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7f9cf015efd0 con 0x7f9cf00d0330 2024-09-10T22:03:09.423 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:03:09.423 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"btime":"2024-09-10T22:03:09:024518+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5248,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2731842573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2731842573},{"type":"v1","addr":"172.21.15.94:6837","nonce":2731842573}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1324134239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1324134239},{"type":"v1","addr":"172.21.15.66:6833","nonce":1324134239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1694406832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1694406832},{"type":"v1","addr":"172.21.15.94:6836","nonce":1694406832}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:03:05.416734+0000","modified":"2024-09-10T22:03:08.421387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5412":{"gid":5412,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1215985114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1215985114},{"type":"v1","addr":"172.21.15.66:6835","nonce":1215985114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5412,"qdb_cluster":[5412]},"id":3}]} 2024-09-10T22:03:09.423 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 26 2024-09-10T22:03:09.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9cf6200640 1 -- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ccc072940 msgr2=0x7f9ccc074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9cf6200640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ccc072940 0x7f9ccc074e00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cf014c950 tx=0x7f9cd8009040 comp rx=0 tx=0).stop 2024-09-10T22:03:09.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9cf6200640 1 -- 172.21.15.66:0/697633261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cf00d0330 msgr2=0x7f9cf013b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9cf6200640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cf00d0330 0x7f9cf013b420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009790 tx=0x7f9ce0005270 comp rx=0 tx=0).stop 2024-09-10T22:03:09.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9cf6200640 1 -- 172.21.15.66:0/697633261 shutdown_connections 2024-09-10T22:03:09.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9cf6200640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ccc072940 0x7f9ccc074e00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9cf6200640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf015d370 0x7f9cf00cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9cf6200640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cf00d0710 0x7f9cf014d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9cf6200640 1 --2- 172.21.15.66:0/697633261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cf00d0330 0x7f9cf013b420 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9cf6200640 1 -- 172.21.15.66:0/697633261 >> 172.21.15.66:0/697633261 conn(0x7f9cf00d0b90 msgr2=0x7f9cf01659c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:09.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.421+0000 7f9cf6200640 1 -- 172.21.15.66:0/697633261 shutdown_connections 2024-09-10T22:03:09.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.425+0000 7f9cf6200640 1 -- 172.21.15.66:0/697633261 wait complete. 2024-09-10T22:03:09.446 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:03:09.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 -- 172.21.15.66:0/3347282683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09f80d1b80 msgr2=0x7f09f80b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 --2- 172.21.15.66:0/3347282683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09f80d1b80 0x7f09f80b36d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f09f80d3d40 tx=0x7f09e401c8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 -- 172.21.15.66:0/3347282683 shutdown_connections 2024-09-10T22:03:09.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 --2- 172.21.15.66:0/3347282683 >> v2:172.21.15.94:3300/0 conn(0x7f09f80b4bd0 0x7f09f80b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 --2- 172.21.15.66:0/3347282683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09f80d1b80 0x7f09f80b36d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 -- 172.21.15.66:0/3347282683 >> 172.21.15.66:0/3347282683 conn(0x7f09f80d3da0 msgr2=0x7f09f80d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:09.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 -- 172.21.15.66:0/3347282683 shutdown_connections 2024-09-10T22:03:09.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 -- 172.21.15.66:0/3347282683 wait complete. 2024-09-10T22:03:09.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 Processor -- start 2024-09-10T22:03:09.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 -- start start 2024-09-10T22:03:09.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09f80b4bd0 0x7f09f815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:09.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09f8162630 0x7f09f80c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:09.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09f80cfbc0 0x7f09f80d0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:09.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09f816b2a0 con 0x7f09f80b4bd0 2024-09-10T22:03:09.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f09f816b120 con 0x7f09f8162630 2024-09-10T22:03:09.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fe000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f09f816b420 con 0x7f09f80cfbc0 2024-09-10T22:03:09.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fcc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09f80b4bd0 0x7f09f815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:09.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fcc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09f80b4bd0 0x7f09f815b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43394/0 (socket says 172.21.15.66:43394) 2024-09-10T22:03:09.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fcc00640 1 -- 172.21.15.66:0/4078617570 learned_addr learned my addr 172.21.15.66:0/4078617570 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:09.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09f7e00640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09f8162630 0x7f09f80c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:09.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fd600640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09f80cfbc0 0x7f09f80d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:09.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fcc00640 1 -- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09f80cfbc0 msgr2=0x7f09f80d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fcc00640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09f80cfbc0 0x7f09f80d0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fcc00640 1 -- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09f8162630 msgr2=0x7f09f80c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fcc00640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09f8162630 0x7f09f80c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fcc00640 1 -- 172.21.15.66:0/4078617570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f815c490 con 0x7f09f80b4bd0 2024-09-10T22:03:09.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.593+0000 7f09fd600640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09f80cfbc0 0x7f09f80d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:09.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.597+0000 7f09fcc00640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09f80b4bd0 0x7f09f815b170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00c3f0 tx=0x7f09ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:09.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.597+0000 7f09f4c00640 1 -- 172.21.15.66:0/4078617570 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec017070 con 0x7f09f80b4bd0 2024-09-10T22:03:09.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.597+0000 7f09f4c00640 1 -- 172.21.15.66:0/4078617570 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09ec0053c0 con 0x7f09f80b4bd0 2024-09-10T22:03:09.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.597+0000 7f09fe000640 1 -- 172.21.15.66:0/4078617570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f815c770 con 0x7f09f80b4bd0 2024-09-10T22:03:09.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.597+0000 7f09f4c00640 1 -- 172.21.15.66:0/4078617570 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec003860 con 0x7f09f80b4bd0 2024-09-10T22:03:09.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.597+0000 7f09fe000640 1 -- 172.21.15.66:0/4078617570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f815c120 con 0x7f09f80b4bd0 2024-09-10T22:03:09.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.597+0000 7f09fe000640 1 -- 172.21.15.66:0/4078617570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f808dd40 con 0x7f09f80b4bd0 2024-09-10T22:03:09.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.597+0000 7f09f4c00640 1 -- 172.21.15.66:0/4078617570 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f09ec003a00 con 0x7f09f80b4bd0 2024-09-10T22:03:09.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.597+0000 7f09f4c00640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09d8072940 0x7f09d8074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:09.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.601+0000 7f09f4c00640 1 -- 172.21.15.66:0/4078617570 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f09ec092dc0 con 0x7f09f80b4bd0 2024-09-10T22:03:09.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.601+0000 7f09f7e00640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09d8072940 0x7f09d8074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:09.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.601+0000 7f09f7e00640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09d8072940 0x7f09d8074e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f09e40af000 tx=0x7f09e401b100 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:09.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.605+0000 7f09f4c00640 1 -- 172.21.15.66:0/4078617570 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f09ec05d980 con 0x7f09f80b4bd0 2024-09-10T22:03:09.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.781+0000 7f09fe000640 1 -- 172.21.15.66:0/4078617570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09f80b3ca0 con 0x7f09f80b4bd0 2024-09-10T22:03:09.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.781+0000 7f09f4c00640 1 -- 172.21.15.66:0/4078617570 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16601 (secure 0 0 0) 0x7f09ec0623b0 con 0x7f09f80b4bd0 2024-09-10T22:03:09.783 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:03:09.783 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":44,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:03:07.018625+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:03:03.140557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T22:03:04.388164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:03:09.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.781+0000 7f09fe000640 1 -- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09d8072940 msgr2=0x7f09d8074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.781+0000 7f09fe000640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09d8072940 0x7f09d8074e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f09e40af000 tx=0x7f09e401b100 comp rx=0 tx=0).stop 2024-09-10T22:03:09.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.781+0000 7f09fe000640 1 -- 172.21.15.66:0/4078617570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09f80b4bd0 msgr2=0x7f09f815b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.781+0000 7f09fe000640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09f80b4bd0 0x7f09f815b170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00c3f0 tx=0x7f09ec00c8b0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.781+0000 7f09fe000640 1 -- 172.21.15.66:0/4078617570 shutdown_connections 2024-09-10T22:03:09.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.781+0000 7f09fe000640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09d8072940 0x7f09d8074e00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.781+0000 7f09fe000640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09f80cfbc0 0x7f09f80d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.781+0000 7f09fe000640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09f8162630 0x7f09f80c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.781+0000 7f09fe000640 1 --2- 172.21.15.66:0/4078617570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09f80b4bd0 0x7f09f815b170 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.781+0000 7f09fe000640 1 -- 172.21.15.66:0/4078617570 >> 172.21.15.66:0/4078617570 conn(0x7f09f80d3da0 msgr2=0x7f09f815f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:09.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.785+0000 7f09fe000640 1 -- 172.21.15.66:0/4078617570 shutdown_connections 2024-09-10T22:03:09.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.785+0000 7f09fe000640 1 -- 172.21.15.66:0/4078617570 wait complete. 2024-09-10T22:03:09.806 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:03:09.806 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:03:09.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 -- 172.21.15.66:0/2479786921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f134015f450 msgr2=0x7f13400b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 --2- 172.21.15.66:0/2479786921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f134015f450 0x7f13400b3170 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1330009840 tx=0x7f133001d1c0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 -- 172.21.15.66:0/2479786921 shutdown_connections 2024-09-10T22:03:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 --2- 172.21.15.66:0/2479786921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f134015f450 0x7f13400b3170 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 -- 172.21.15.66:0/2479786921 >> 172.21.15.66:0/2479786921 conn(0x7f13400d1c60 msgr2=0x7f13400d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 -- 172.21.15.66:0/2479786921 shutdown_connections 2024-09-10T22:03:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 -- 172.21.15.66:0/2479786921 wait complete. 2024-09-10T22:03:09.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 Processor -- start 2024-09-10T22:03:09.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 -- start start 2024-09-10T22:03:09.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13400b1e10 0x7f134015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f134015f450 0x7f134015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1340160280 0x7f134015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f134016a040 con 0x7f13400b1e10 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1340169ec0 con 0x7f134015f450 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1346600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f134016a1c0 con 0x7f1340160280 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f133f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13400b1e10 0x7f134015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f133f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13400b1e10 0x7f134015b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43410/0 (socket says 172.21.15.66:43410) 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f133f400640 1 -- 172.21.15.66:0/4045005515 learned_addr learned my addr 172.21.15.66:0/4045005515 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f1337e00640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f134015f450 0x7f134015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f133fe00640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1340160280 0x7f134015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f133f400640 1 -- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1340160280 msgr2=0x7f134015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f133f400640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1340160280 0x7f134015d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f133f400640 1 -- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f134015f450 msgr2=0x7f134015be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:09.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f133f400640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f134015f450 0x7f134015be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:09.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.953+0000 7f133f400640 1 -- 172.21.15.66:0/4045005515 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134015a760 con 0x7f13400b1e10 2024-09-10T22:03:09.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.957+0000 7f133f400640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13400b1e10 0x7f134015b920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1330009790 tx=0x7f13300041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:09.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.957+0000 7f1337400640 1 -- 172.21.15.66:0/4045005515 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13300b2040 con 0x7f13400b1e10 2024-09-10T22:03:09.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.957+0000 7f1337400640 1 -- 172.21.15.66:0/4045005515 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1330005030 con 0x7f13400b1e10 2024-09-10T22:03:09.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.957+0000 7f1337400640 1 -- 172.21.15.66:0/4045005515 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330005460 con 0x7f13400b1e10 2024-09-10T22:03:09.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.957+0000 7f1346600640 1 -- 172.21.15.66:0/4045005515 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134015aa40 con 0x7f13400b1e10 2024-09-10T22:03:09.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.957+0000 7f1346600640 1 -- 172.21.15.66:0/4045005515 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1340153000 con 0x7f13400b1e10 2024-09-10T22:03:09.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.957+0000 7f1346600640 1 -- 172.21.15.66:0/4045005515 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13400b3170 con 0x7f13400b1e10 2024-09-10T22:03:09.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.957+0000 7f1337400640 1 -- 172.21.15.66:0/4045005515 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f13300b5070 con 0x7f13400b1e10 2024-09-10T22:03:09.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.957+0000 7f1337400640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f131c072940 0x7f131c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:09.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.957+0000 7f1337400640 1 -- 172.21.15.66:0/4045005515 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f133012d690 con 0x7f13400b1e10 2024-09-10T22:03:09.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.957+0000 7f1337e00640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f131c072940 0x7f131c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:09.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.961+0000 7f1337e00640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f131c072940 0x7f131c074e00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f134015d620 tx=0x7f1328009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:09.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:09.961+0000 7f1337400640 1 -- 172.21.15.66:0/4045005515 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f13300bd060 con 0x7f13400b1e10 2024-09-10T22:03:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.141+0000 7f1346600640 1 -- 172.21.15.66:0/4045005515 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f134015a510 con 0x7f131c072940 2024-09-10T22:03:10.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.145+0000 7f1337400640 1 -- 172.21.15.66:0/4045005515 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251928 (secure 0 0 0) 0x7f134015a510 con 0x7f131c072940 2024-09-10T22:03:10.151 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:03:10.154 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:03:10.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.149+0000 7f1346600640 1 -- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f131c072940 msgr2=0x7f131c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:10.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.149+0000 7f1346600640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f131c072940 0x7f131c074e00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f134015d620 tx=0x7f1328009040 comp rx=0 tx=0).stop 2024-09-10T22:03:10.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.149+0000 7f1346600640 1 -- 172.21.15.66:0/4045005515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13400b1e10 msgr2=0x7f134015b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:10.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.149+0000 7f1346600640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13400b1e10 0x7f134015b920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1330009790 tx=0x7f13300041a0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.149+0000 7f1346600640 1 -- 172.21.15.66:0/4045005515 shutdown_connections 2024-09-10T22:03:10.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.149+0000 7f1346600640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f131c072940 0x7f131c074e00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.149+0000 7f1346600640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1340160280 0x7f134015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.149+0000 7f1346600640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f134015f450 0x7f134015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.149+0000 7f1346600640 1 --2- 172.21.15.66:0/4045005515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13400b1e10 0x7f134015b920 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.149+0000 7f1346600640 1 -- 172.21.15.66:0/4045005515 >> 172.21.15.66:0/4045005515 conn(0x7f13400d1c60 msgr2=0x7f13400cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:10.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.149+0000 7f1346600640 1 -- 172.21.15.66:0/4045005515 shutdown_connections 2024-09-10T22:03:10.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.149+0000 7f1346600640 1 -- 172.21.15.66:0/4045005515 wait complete. 2024-09-10T22:03:10.171 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-10T22:03:08.345381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":139,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275396,"kb_used_data":5080,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754699324,"statfs":{"total":773094113280,"available":772812107776,"internally_reserved":0,"allocated":5201920,"data_stored":2754391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":276614708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.343454"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039719+0000","last_change":"2024-09-10T22:03:06.039627+0000","last_active":"2024-09-10T22:03:07.039719+0000","last_peered":"2024-09-10T22:03:07.039719+0000","last_clean":"2024-09-10T22:03:07.039719+0000","last_became_active":"2024-09-10T22:03:06.039339+0000","last_became_peered":"2024-09-10T22:03:06.039339+0000","last_unstale":"2024-09-10T22:03:07.039719+0000","last_undegraded":"2024-09-10T22:03:07.039719+0000","last_fullsized":"2024-09-10T22:03:07.039719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:22.417821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039978+0000","last_change":"2024-09-10T22:03:06.037554+0000","last_active":"2024-09-10T22:03:07.039978+0000","last_peered":"2024-09-10T22:03:07.039978+0000","last_clean":"2024-09-10T22:03:07.039978+0000","last_became_active":"2024-09-10T22:03:06.037227+0000","last_became_peered":"2024-09-10T22:03:06.037227+0000","last_unstale":"2024-09-10T22:03:07.039978+0000","last_undegraded":"2024-09-10T22:03:07.039978+0000","last_fullsized":"2024-09-10T22:03:07.039978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:03.034797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026256+0000","last_change":"2024-09-10T22:03:06.028445+0000","last_active":"2024-09-10T22:03:07.026256+0000","last_peered":"2024-09-10T22:03:07.026256+0000","last_clean":"2024-09-10T22:03:07.026256+0000","last_became_active":"2024-09-10T22:03:06.027823+0000","last_became_peered":"2024-09-10T22:03:06.027823+0000","last_unstale":"2024-09-10T22:03:07.026256+0000","last_undegraded":"2024-09-10T22:03:07.026256+0000","last_fullsized":"2024-09-10T22:03:07.026256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:47.056575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026221+0000","last_change":"2024-09-10T22:03:06.028899+0000","last_active":"2024-09-10T22:03:07.026221+0000","last_peered":"2024-09-10T22:03:07.026221+0000","last_clean":"2024-09-10T22:03:07.026221+0000","last_became_active":"2024-09-10T22:03:06.028621+0000","last_became_peered":"2024-09-10T22:03:06.028621+0000","last_unstale":"2024-09-10T22:03:07.026221+0000","last_undegraded":"2024-09-10T22:03:07.026221+0000","last_fullsized":"2024-09-10T22:03:07.026221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:46.795332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039569+0000","last_change":"2024-09-10T22:03:06.037232+0000","last_active":"2024-09-10T22:03:07.039569+0000","last_peered":"2024-09-10T22:03:07.039569+0000","last_clean":"2024-09-10T22:03:07.039569+0000","last_became_active":"2024-09-10T22:03:06.036887+0000","last_became_peered":"2024-09-10T22:03:06.036887+0000","last_unstale":"2024-09-10T22:03:07.039569+0000","last_undegraded":"2024-09-10T22:03:07.039569+0000","last_fullsized":"2024-09-10T22:03:07.039569+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:01:20.819903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038630+0000","last_change":"2024-09-10T22:03:06.035654+0000","last_active":"2024-09-10T22:03:07.038630+0000","last_peered":"2024-09-10T22:03:07.038630+0000","last_clean":"2024-09-10T22:03:07.038630+0000","last_became_active":"2024-09-10T22:03:06.035384+0000","last_became_peered":"2024-09-10T22:03:06.035384+0000","last_unstale":"2024-09-10T22:03:07.038630+0000","last_undegraded":"2024-09-10T22:03:07.038630+0000","last_fullsized":"2024-09-10T22:03:07.038630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:37.723354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039702+0000","last_change":"2024-09-10T22:03:06.038694+0000","last_active":"2024-09-10T22:03:07.039702+0000","last_peered":"2024-09-10T22:03:07.039702+0000","last_clean":"2024-09-10T22:03:07.039702+0000","last_became_active":"2024-09-10T22:03:06.038388+0000","last_became_peered":"2024-09-10T22:03:06.038388+0000","last_unstale":"2024-09-10T22:03:07.039702+0000","last_undegraded":"2024-09-10T22:03:07.039702+0000","last_fullsized":"2024-09-10T22:03:07.039702+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:01:20.819903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025648+0000","last_change":"2024-09-10T22:03:06.029337+0000","last_active":"2024-09-10T22:03:07.025648+0000","last_peered":"2024-09-10T22:03:07.025648+0000","last_clean":"2024-09-10T22:03:07.025648+0000","last_became_active":"2024-09-10T22:03:06.028302+0000","last_became_peered":"2024-09-10T22:03:06.028302+0000","last_unstale":"2024-09-10T22:03:07.025648+0000","last_undegraded":"2024-09-10T22:03:07.025648+0000","last_fullsized":"2024-09-10T22:03:07.025648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:47.056575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026770+0000","last_change":"2024-09-10T22:03:06.025932+0000","last_active":"2024-09-10T22:03:07.026770+0000","last_peered":"2024-09-10T22:03:07.026770+0000","last_clean":"2024-09-10T22:03:07.026770+0000","last_became_active":"2024-09-10T22:03:06.025550+0000","last_became_peered":"2024-09-10T22:03:06.025550+0000","last_unstale":"2024-09-10T22:03:07.026770+0000","last_undegraded":"2024-09-10T22:03:07.026770+0000","last_fullsized":"2024-09-10T22:03:07.026770+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:38.800945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026786+0000","last_change":"2024-09-10T22:03:06.027344+0000","last_active":"2024-09-10T22:03:07.026786+0000","last_peered":"2024-09-10T22:03:07.026786+0000","last_clean":"2024-09-10T22:03:07.026786+0000","last_became_active":"2024-09-10T22:03:06.027082+0000","last_became_peered":"2024-09-10T22:03:06.027082+0000","last_unstale":"2024-09-10T22:03:07.026786+0000","last_undegraded":"2024-09-10T22:03:07.026786+0000","last_fullsized":"2024-09-10T22:03:07.026786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:28.781866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026359+0000","last_change":"2024-09-10T22:03:06.026675+0000","last_active":"2024-09-10T22:03:07.026359+0000","last_peered":"2024-09-10T22:03:07.026359+0000","last_clean":"2024-09-10T22:03:07.026359+0000","last_became_active":"2024-09-10T22:03:06.024203+0000","last_became_peered":"2024-09-10T22:03:06.024203+0000","last_unstale":"2024-09-10T22:03:07.026359+0000","last_undegraded":"2024-09-10T22:03:07.026359+0000","last_fullsized":"2024-09-10T22:03:07.026359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:37.688947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039471+0000","last_change":"2024-09-10T22:03:06.034620+0000","last_active":"2024-09-10T22:03:07.039471+0000","last_peered":"2024-09-10T22:03:07.039471+0000","last_clean":"2024-09-10T22:03:07.039471+0000","last_became_active":"2024-09-10T22:03:06.034158+0000","last_became_peered":"2024-09-10T22:03:06.034158+0000","last_unstale":"2024-09-10T22:03:07.039471+0000","last_undegraded":"2024-09-10T22:03:07.039471+0000","last_fullsized":"2024-09-10T22:03:07.039471+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:32.805553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.042309+0000","last_change":"2024-09-10T22:03:06.037582+0000","last_active":"2024-09-10T22:03:06.042309+0000","last_peered":"2024-09-10T22:03:06.042309+0000","last_clean":"2024-09-10T22:03:06.042309+0000","last_became_active":"2024-09-10T22:03:06.037283+0000","last_became_peered":"2024-09-10T22:03:06.037283+0000","last_unstale":"2024-09-10T22:03:06.042309+0000","last_undegraded":"2024-09-10T22:03:06.042309+0000","last_fullsized":"2024-09-10T22:03:06.042309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:58.670828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025866+0000","last_change":"2024-09-10T22:03:06.030447+0000","last_active":"2024-09-10T22:03:07.025866+0000","last_peered":"2024-09-10T22:03:07.025866+0000","last_clean":"2024-09-10T22:03:07.025866+0000","last_became_active":"2024-09-10T22:03:06.030179+0000","last_became_peered":"2024-09-10T22:03:06.030179+0000","last_unstale":"2024-09-10T22:03:07.025866+0000","last_undegraded":"2024-09-10T22:03:07.025866+0000","last_fullsized":"2024-09-10T22:03:07.025866+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:37.643981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038289+0000","last_change":"2024-09-10T22:03:06.028413+0000","last_active":"2024-09-10T22:03:07.038289+0000","last_peered":"2024-09-10T22:03:07.038289+0000","last_clean":"2024-09-10T22:03:07.038289+0000","last_became_active":"2024-09-10T22:03:06.027971+0000","last_became_peered":"2024-09-10T22:03:06.027971+0000","last_unstale":"2024-09-10T22:03:07.038289+0000","last_undegraded":"2024-09-10T22:03:07.038289+0000","last_fullsized":"2024-09-10T22:03:07.038289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:30.382237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026361+0000","last_change":"2024-09-10T22:03:06.032259+0000","last_active":"2024-09-10T22:03:07.026361+0000","last_peered":"2024-09-10T22:03:07.026361+0000","last_clean":"2024-09-10T22:03:07.026361+0000","last_became_active":"2024-09-10T22:03:06.029852+0000","last_became_peered":"2024-09-10T22:03:06.029852+0000","last_unstale":"2024-09-10T22:03:07.026361+0000","last_undegraded":"2024-09-10T22:03:07.026361+0000","last_fullsized":"2024-09-10T22:03:07.026361+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:47.709655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026379+0000","last_change":"2024-09-10T22:03:06.028591+0000","last_active":"2024-09-10T22:03:07.026379+0000","last_peered":"2024-09-10T22:03:07.026379+0000","last_clean":"2024-09-10T22:03:07.026379+0000","last_became_active":"2024-09-10T22:03:06.027382+0000","last_became_peered":"2024-09-10T22:03:06.027382+0000","last_unstale":"2024-09-10T22:03:07.026379+0000","last_undegraded":"2024-09-10T22:03:07.026379+0000","last_fullsized":"2024-09-10T22:03:07.026379+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:09.663390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027324+0000","last_change":"2024-09-10T22:03:06.036419+0000","last_active":"2024-09-10T22:03:07.027324+0000","last_peered":"2024-09-10T22:03:07.027324+0000","last_clean":"2024-09-10T22:03:07.027324+0000","last_became_active":"2024-09-10T22:03:06.031292+0000","last_became_peered":"2024-09-10T22:03:06.031292+0000","last_unstale":"2024-09-10T22:03:07.027324+0000","last_undegraded":"2024-09-10T22:03:07.027324+0000","last_fullsized":"2024-09-10T22:03:07.027324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:18.461668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027495+0000","last_change":"2024-09-10T22:03:06.031396+0000","last_active":"2024-09-10T22:03:07.027495+0000","last_peered":"2024-09-10T22:03:07.027495+0000","last_clean":"2024-09-10T22:03:07.027495+0000","last_became_active":"2024-09-10T22:03:06.031109+0000","last_became_peered":"2024-09-10T22:03:06.031109+0000","last_unstale":"2024-09-10T22:03:07.027495+0000","last_undegraded":"2024-09-10T22:03:07.027495+0000","last_fullsized":"2024-09-10T22:03:07.027495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:00.117477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.031024+0000","last_change":"2024-09-10T22:03:06.028434+0000","last_active":"2024-09-10T22:03:06.031024+0000","last_peered":"2024-09-10T22:03:06.031024+0000","last_clean":"2024-09-10T22:03:06.031024+0000","last_became_active":"2024-09-10T22:03:06.028070+0000","last_became_peered":"2024-09-10T22:03:06.028070+0000","last_unstale":"2024-09-10T22:03:06.031024+0000","last_undegraded":"2024-09-10T22:03:06.031024+0000","last_fullsized":"2024-09-10T22:03:06.031024+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:46.063806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038646+0000","last_change":"2024-09-10T22:03:06.029313+0000","last_active":"2024-09-10T22:03:07.038646+0000","last_peered":"2024-09-10T22:03:07.038646+0000","last_clean":"2024-09-10T22:03:07.038646+0000","last_became_active":"2024-09-10T22:03:06.028936+0000","last_became_peered":"2024-09-10T22:03:06.028936+0000","last_unstale":"2024-09-10T22:03:07.038646+0000","last_undegraded":"2024-09-10T22:03:07.038646+0000","last_fullsized":"2024-09-10T22:03:07.038646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:57.653238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026125+0000","last_change":"2024-09-10T22:03:06.027468+0000","last_active":"2024-09-10T22:03:07.026125+0000","last_peered":"2024-09-10T22:03:07.026125+0000","last_clean":"2024-09-10T22:03:07.026125+0000","last_became_active":"2024-09-10T22:03:06.027176+0000","last_became_peered":"2024-09-10T22:03:06.027176+0000","last_unstale":"2024-09-10T22:03:07.026125+0000","last_undegraded":"2024-09-10T22:03:07.026125+0000","last_fullsized":"2024-09-10T22:03:07.026125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:18.461668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026919+0000","last_change":"2024-09-10T22:03:06.037106+0000","last_active":"2024-09-10T22:03:07.026919+0000","last_peered":"2024-09-10T22:03:07.026919+0000","last_clean":"2024-09-10T22:03:07.026919+0000","last_became_active":"2024-09-10T22:03:06.035908+0000","last_became_peered":"2024-09-10T22:03:06.035908+0000","last_unstale":"2024-09-10T22:03:07.026919+0000","last_undegraded":"2024-09-10T22:03:07.026919+0000","last_fullsized":"2024-09-10T22:03:07.026919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:50.117268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026578+0000","last_change":"2024-09-10T22:03:06.035594+0000","last_active":"2024-09-10T22:03:07.026578+0000","last_peered":"2024-09-10T22:03:07.026578+0000","last_clean":"2024-09-10T22:03:07.026578+0000","last_became_active":"2024-09-10T22:03:06.035337+0000","last_became_peered":"2024-09-10T22:03:06.035337+0000","last_unstale":"2024-09-10T22:03:07.026578+0000","last_undegraded":"2024-09-10T22:03:07.026578+0000","last_fullsized":"2024-09-10T22:03:07.026578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:41.893796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027486+0000","last_change":"2024-09-10T22:03:06.024010+0000","last_active":"2024-09-10T22:03:07.027486+0000","last_peered":"2024-09-10T22:03:07.027486+0000","last_clean":"2024-09-10T22:03:07.027486+0000","last_became_active":"2024-09-10T22:03:06.023686+0000","last_became_peered":"2024-09-10T22:03:06.023686+0000","last_unstale":"2024-09-10T22:03:07.027486+0000","last_undegraded":"2024-09-10T22:03:07.027486+0000","last_fullsized":"2024-09-10T22:03:07.027486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:08.682948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.036110+0000","last_change":"2024-09-10T22:03:06.029755+0000","last_active":"2024-09-10T22:03:06.036110+0000","last_peered":"2024-09-10T22:03:06.036110+0000","last_clean":"2024-09-10T22:03:06.036110+0000","last_became_active":"2024-09-10T22:03:06.029478+0000","last_became_peered":"2024-09-10T22:03:06.029478+0000","last_unstale":"2024-09-10T22:03:06.036110+0000","last_undegraded":"2024-09-10T22:03:06.036110+0000","last_fullsized":"2024-09-10T22:03:06.036110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:10.497569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026489+0000","last_change":"2024-09-10T22:03:06.029699+0000","last_active":"2024-09-10T22:03:07.026489+0000","last_peered":"2024-09-10T22:03:07.026489+0000","last_clean":"2024-09-10T22:03:07.026489+0000","last_became_active":"2024-09-10T22:03:06.028169+0000","last_became_peered":"2024-09-10T22:03:06.028169+0000","last_unstale":"2024-09-10T22:03:07.026489+0000","last_undegraded":"2024-09-10T22:03:07.026489+0000","last_fullsized":"2024-09-10T22:03:07.026489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:04.451178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039947+0000","last_change":"2024-09-10T22:03:06.034331+0000","last_active":"2024-09-10T22:03:07.039947+0000","last_peered":"2024-09-10T22:03:07.039947+0000","last_clean":"2024-09-10T22:03:07.039947+0000","last_became_active":"2024-09-10T22:03:06.033766+0000","last_became_peered":"2024-09-10T22:03:06.033766+0000","last_unstale":"2024-09-10T22:03:07.039947+0000","last_undegraded":"2024-09-10T22:03:07.039947+0000","last_fullsized":"2024-09-10T22:03:07.039947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:28.781866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026980+0000","last_change":"2024-09-10T22:03:06.025202+0000","last_active":"2024-09-10T22:03:07.026980+0000","last_peered":"2024-09-10T22:03:07.026980+0000","last_clean":"2024-09-10T22:03:07.026980+0000","last_became_active":"2024-09-10T22:03:06.024919+0000","last_became_peered":"2024-09-10T22:03:06.024919+0000","last_unstale":"2024-09-10T22:03:07.026980+0000","last_undegraded":"2024-09-10T22:03:07.026980+0000","last_fullsized":"2024-09-10T22:03:07.026980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:46.605911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026700+0000","last_change":"2024-09-10T22:03:06.036207+0000","last_active":"2024-09-10T22:03:07.026700+0000","last_peered":"2024-09-10T22:03:07.026700+0000","last_clean":"2024-09-10T22:03:07.026700+0000","last_became_active":"2024-09-10T22:03:06.035283+0000","last_became_peered":"2024-09-10T22:03:06.035283+0000","last_unstale":"2024-09-10T22:03:07.026700+0000","last_undegraded":"2024-09-10T22:03:07.026700+0000","last_fullsized":"2024-09-10T22:03:07.026700+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:53.137403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038323+0000","last_change":"2024-09-10T22:03:06.029630+0000","last_active":"2024-09-10T22:03:07.038323+0000","last_peered":"2024-09-10T22:03:07.038323+0000","last_clean":"2024-09-10T22:03:07.038323+0000","last_became_active":"2024-09-10T22:03:06.029361+0000","last_became_peered":"2024-09-10T22:03:06.029361+0000","last_unstale":"2024-09-10T22:03:07.038323+0000","last_undegraded":"2024-09-10T22:03:07.038323+0000","last_fullsized":"2024-09-10T22:03:07.038323+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:30.075016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.040197+0000","last_change":"2024-09-10T22:03:06.040090+0000","last_active":"2024-09-10T22:03:06.040197+0000","last_peered":"2024-09-10T22:03:06.040197+0000","last_clean":"2024-09-10T22:03:06.040197+0000","last_became_active":"2024-09-10T22:03:06.039810+0000","last_became_peered":"2024-09-10T22:03:06.039810+0000","last_unstale":"2024-09-10T22:03:06.040197+0000","last_undegraded":"2024-09-10T22:03:06.040197+0000","last_fullsized":"2024-09-10T22:03:06.040197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:32.918048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039754+0000","last_change":"2024-09-10T22:03:06.031324+0000","last_active":"2024-09-10T22:03:07.039754+0000","last_peered":"2024-09-10T22:03:07.039754+0000","last_clean":"2024-09-10T22:03:07.039754+0000","last_became_active":"2024-09-10T22:03:06.031015+0000","last_became_peered":"2024-09-10T22:03:06.031015+0000","last_unstale":"2024-09-10T22:03:07.039754+0000","last_undegraded":"2024-09-10T22:03:07.039754+0000","last_fullsized":"2024-09-10T22:03:07.039754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:18.808705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":19,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039179+0000","last_change":"2024-09-10T22:03:05.028045+0000","last_active":"2024-09-10T22:03:07.039179+0000","last_peered":"2024-09-10T22:03:07.039179+0000","last_clean":"2024-09-10T22:03:07.039179+0000","last_became_active":"2024-09-10T22:03:05.027746+0000","last_became_peered":"2024-09-10T22:03:05.027746+0000","last_unstale":"2024-09-10T22:03:07.039179+0000","last_undegraded":"2024-09-10T22:03:07.039179+0000","last_fullsized":"2024-09-10T22:03:07.039179+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:37.798038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027631+0000","last_change":"2024-09-10T22:03:06.021989+0000","last_active":"2024-09-10T22:03:07.027631+0000","last_peered":"2024-09-10T22:03:07.027631+0000","last_clean":"2024-09-10T22:03:07.027631+0000","last_became_active":"2024-09-10T22:03:06.021242+0000","last_became_peered":"2024-09-10T22:03:06.021242+0000","last_unstale":"2024-09-10T22:03:07.027631+0000","last_undegraded":"2024-09-10T22:03:07.027631+0000","last_fullsized":"2024-09-10T22:03:07.027631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:18.808705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027149+0000","last_change":"2024-09-10T22:03:05.031225+0000","last_active":"2024-09-10T22:03:07.027149+0000","last_peered":"2024-09-10T22:03:07.027149+0000","last_clean":"2024-09-10T22:03:07.027149+0000","last_became_active":"2024-09-10T22:03:05.030047+0000","last_became_peered":"2024-09-10T22:03:05.030047+0000","last_unstale":"2024-09-10T22:03:07.027149+0000","last_undegraded":"2024-09-10T22:03:07.027149+0000","last_fullsized":"2024-09-10T22:03:07.027149+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:45.603004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.040054+0000","last_change":"2024-09-10T22:03:06.033328+0000","last_active":"2024-09-10T22:03:07.040054+0000","last_peered":"2024-09-10T22:03:07.040054+0000","last_clean":"2024-09-10T22:03:07.040054+0000","last_became_active":"2024-09-10T22:03:06.033062+0000","last_became_peered":"2024-09-10T22:03:06.033062+0000","last_unstale":"2024-09-10T22:03:07.040054+0000","last_undegraded":"2024-09-10T22:03:07.040054+0000","last_fullsized":"2024-09-10T22:03:07.040054+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:22.417821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039526+0000","last_change":"2024-09-10T22:03:05.032199+0000","last_active":"2024-09-10T22:03:07.039526+0000","last_peered":"2024-09-10T22:03:07.039526+0000","last_clean":"2024-09-10T22:03:07.039526+0000","last_became_active":"2024-09-10T22:03:05.030195+0000","last_became_peered":"2024-09-10T22:03:05.030195+0000","last_unstale":"2024-09-10T22:03:07.039526+0000","last_undegraded":"2024-09-10T22:03:07.039526+0000","last_fullsized":"2024-09-10T22:03:07.039526+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:35.474493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039420+0000","last_change":"2024-09-10T22:03:06.023976+0000","last_active":"2024-09-10T22:03:07.039420+0000","last_peered":"2024-09-10T22:03:07.039420+0000","last_clean":"2024-09-10T22:03:07.039420+0000","last_became_active":"2024-09-10T22:03:06.023720+0000","last_became_peered":"2024-09-10T22:03:06.023720+0000","last_unstale":"2024-09-10T22:03:07.039420+0000","last_undegraded":"2024-09-10T22:03:07.039420+0000","last_fullsized":"2024-09-10T22:03:07.039420+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:00.117477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039092+0000","last_change":"2024-09-10T22:03:05.026215+0000","last_active":"2024-09-10T22:03:07.039092+0000","last_peered":"2024-09-10T22:03:07.039092+0000","last_clean":"2024-09-10T22:03:07.039092+0000","last_became_active":"2024-09-10T22:03:05.025925+0000","last_became_peered":"2024-09-10T22:03:05.025925+0000","last_unstale":"2024-09-10T22:03:07.039092+0000","last_undegraded":"2024-09-10T22:03:07.039092+0000","last_fullsized":"2024-09-10T22:03:07.039092+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:08.660483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026190+0000","last_change":"2024-09-10T22:03:06.024371+0000","last_active":"2024-09-10T22:03:07.026190+0000","last_peered":"2024-09-10T22:03:07.026190+0000","last_clean":"2024-09-10T22:03:07.026190+0000","last_became_active":"2024-09-10T22:03:06.024088+0000","last_became_peered":"2024-09-10T22:03:06.024088+0000","last_unstale":"2024-09-10T22:03:07.026190+0000","last_undegraded":"2024-09-10T22:03:07.026190+0000","last_fullsized":"2024-09-10T22:03:07.026190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:46.631165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038773+0000","last_change":"2024-09-10T22:03:05.016208+0000","last_active":"2024-09-10T22:03:07.038773+0000","last_peered":"2024-09-10T22:03:07.038773+0000","last_clean":"2024-09-10T22:03:07.038773+0000","last_became_active":"2024-09-10T22:03:05.015927+0000","last_became_peered":"2024-09-10T22:03:05.015927+0000","last_unstale":"2024-09-10T22:03:07.038773+0000","last_undegraded":"2024-09-10T22:03:07.038773+0000","last_fullsized":"2024-09-10T22:03:07.038773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:22.527704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025823+0000","last_change":"2024-09-10T22:03:06.043252+0000","last_active":"2024-09-10T22:03:07.025823+0000","last_peered":"2024-09-10T22:03:07.025823+0000","last_clean":"2024-09-10T22:03:07.025823+0000","last_became_active":"2024-09-10T22:03:06.042971+0000","last_became_peered":"2024-09-10T22:03:06.042971+0000","last_unstale":"2024-09-10T22:03:07.025823+0000","last_undegraded":"2024-09-10T22:03:07.025823+0000","last_fullsized":"2024-09-10T22:03:07.025823+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:53.137403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026334+0000","last_change":"2024-09-10T22:03:05.023976+0000","last_active":"2024-09-10T22:03:07.026334+0000","last_peered":"2024-09-10T22:03:07.026334+0000","last_clean":"2024-09-10T22:03:07.026334+0000","last_became_active":"2024-09-10T22:03:05.023659+0000","last_became_peered":"2024-09-10T22:03:05.023659+0000","last_unstale":"2024-09-10T22:03:07.026334+0000","last_undegraded":"2024-09-10T22:03:07.026334+0000","last_fullsized":"2024-09-10T22:03:07.026334+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:38.227987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025690+0000","last_change":"2024-09-10T22:03:06.028567+0000","last_active":"2024-09-10T22:03:07.025690+0000","last_peered":"2024-09-10T22:03:07.025690+0000","last_clean":"2024-09-10T22:03:07.025690+0000","last_became_active":"2024-09-10T22:03:06.027918+0000","last_became_peered":"2024-09-10T22:03:06.027918+0000","last_unstale":"2024-09-10T22:03:07.025690+0000","last_undegraded":"2024-09-10T22:03:07.025690+0000","last_fullsized":"2024-09-10T22:03:07.025690+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:09.663390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039774+0000","last_change":"2024-09-10T22:03:05.031904+0000","last_active":"2024-09-10T22:03:07.039774+0000","last_peered":"2024-09-10T22:03:07.039774+0000","last_clean":"2024-09-10T22:03:07.039774+0000","last_became_active":"2024-09-10T22:03:05.031358+0000","last_became_peered":"2024-09-10T22:03:05.031358+0000","last_unstale":"2024-09-10T22:03:07.039774+0000","last_undegraded":"2024-09-10T22:03:07.039774+0000","last_fullsized":"2024-09-10T22:03:07.039774+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:36.337853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026659+0000","last_change":"2024-09-10T22:03:06.039979+0000","last_active":"2024-09-10T22:03:07.026659+0000","last_peered":"2024-09-10T22:03:07.026659+0000","last_clean":"2024-09-10T22:03:07.026659+0000","last_became_active":"2024-09-10T22:03:06.038933+0000","last_became_peered":"2024-09-10T22:03:06.038933+0000","last_unstale":"2024-09-10T22:03:07.026659+0000","last_undegraded":"2024-09-10T22:03:07.026659+0000","last_fullsized":"2024-09-10T22:03:07.026659+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:39.230893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025766+0000","last_change":"2024-09-10T22:03:05.035816+0000","last_active":"2024-09-10T22:03:07.025766+0000","last_peered":"2024-09-10T22:03:07.025766+0000","last_clean":"2024-09-10T22:03:07.025766+0000","last_became_active":"2024-09-10T22:03:05.035398+0000","last_became_peered":"2024-09-10T22:03:05.035398+0000","last_unstale":"2024-09-10T22:03:07.025766+0000","last_undegraded":"2024-09-10T22:03:07.025766+0000","last_fullsized":"2024-09-10T22:03:07.025766+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:24.588960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026818+0000","last_change":"2024-09-10T22:03:06.031664+0000","last_active":"2024-09-10T22:03:07.026818+0000","last_peered":"2024-09-10T22:03:07.026818+0000","last_clean":"2024-09-10T22:03:07.026818+0000","last_became_active":"2024-09-10T22:03:06.030950+0000","last_became_peered":"2024-09-10T22:03:06.030950+0000","last_unstale":"2024-09-10T22:03:07.026818+0000","last_undegraded":"2024-09-10T22:03:07.026818+0000","last_fullsized":"2024-09-10T22:03:07.026818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:37.643981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026812+0000","last_change":"2024-09-10T22:03:05.024528+0000","last_active":"2024-09-10T22:03:07.026812+0000","last_peered":"2024-09-10T22:03:07.026812+0000","last_clean":"2024-09-10T22:03:07.026812+0000","last_became_active":"2024-09-10T22:03:05.024254+0000","last_became_peered":"2024-09-10T22:03:05.024254+0000","last_unstale":"2024-09-10T22:03:07.026812+0000","last_undegraded":"2024-09-10T22:03:07.026812+0000","last_fullsized":"2024-09-10T22:03:07.026812+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:40.890890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039632+0000","last_change":"2024-09-10T22:03:06.039734+0000","last_active":"2024-09-10T22:03:07.039632+0000","last_peered":"2024-09-10T22:03:07.039632+0000","last_clean":"2024-09-10T22:03:07.039632+0000","last_became_active":"2024-09-10T22:03:06.039429+0000","last_became_peered":"2024-09-10T22:03:06.039429+0000","last_unstale":"2024-09-10T22:03:07.039632+0000","last_undegraded":"2024-09-10T22:03:07.039632+0000","last_fullsized":"2024-09-10T22:03:07.039632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:03.034797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026882+0000","last_change":"2024-09-10T22:03:05.029224+0000","last_active":"2024-09-10T22:03:07.026882+0000","last_peered":"2024-09-10T22:03:07.026882+0000","last_clean":"2024-09-10T22:03:07.026882+0000","last_became_active":"2024-09-10T22:03:05.029043+0000","last_became_peered":"2024-09-10T22:03:05.029043+0000","last_unstale":"2024-09-10T22:03:07.026882+0000","last_undegraded":"2024-09-10T22:03:07.026882+0000","last_fullsized":"2024-09-10T22:03:07.026882+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:32.578778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039496+0000","last_change":"2024-09-10T22:03:06.031415+0000","last_active":"2024-09-10T22:03:07.039496+0000","last_peered":"2024-09-10T22:03:07.039496+0000","last_clean":"2024-09-10T22:03:07.039496+0000","last_became_active":"2024-09-10T22:03:06.031049+0000","last_became_peered":"2024-09-10T22:03:06.031049+0000","last_unstale":"2024-09-10T22:03:07.039496+0000","last_undegraded":"2024-09-10T22:03:07.039496+0000","last_fullsized":"2024-09-10T22:03:07.039496+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:14.013068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039727+0000","last_change":"2024-09-10T22:03:05.024883+0000","last_active":"2024-09-10T22:03:07.039727+0000","last_peered":"2024-09-10T22:03:07.039727+0000","last_clean":"2024-09-10T22:03:07.039727+0000","last_became_active":"2024-09-10T22:03:05.024620+0000","last_became_peered":"2024-09-10T22:03:05.024620+0000","last_unstale":"2024-09-10T22:03:07.039727+0000","last_undegraded":"2024-09-10T22:03:07.039727+0000","last_fullsized":"2024-09-10T22:03:07.039727+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:46.053668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039847+0000","last_change":"2024-09-10T22:03:06.038013+0000","last_active":"2024-09-10T22:03:07.039847+0000","last_peered":"2024-09-10T22:03:07.039847+0000","last_clean":"2024-09-10T22:03:07.039847+0000","last_became_active":"2024-09-10T22:03:06.037742+0000","last_became_peered":"2024-09-10T22:03:06.037742+0000","last_unstale":"2024-09-10T22:03:07.039847+0000","last_undegraded":"2024-09-10T22:03:07.039847+0000","last_fullsized":"2024-09-10T22:03:07.039847+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:20.984037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.027787+0000","last_change":"2024-09-10T22:03:05.025047+0000","last_active":"2024-09-10T22:03:06.027787+0000","last_peered":"2024-09-10T22:03:06.027787+0000","last_clean":"2024-09-10T22:03:06.027787+0000","last_became_active":"2024-09-10T22:03:05.024736+0000","last_became_peered":"2024-09-10T22:03:05.024736+0000","last_unstale":"2024-09-10T22:03:06.027787+0000","last_undegraded":"2024-09-10T22:03:06.027787+0000","last_fullsized":"2024-09-10T22:03:06.027787+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:40.706358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026890+0000","last_change":"2024-09-10T22:03:06.030932+0000","last_active":"2024-09-10T22:03:07.026890+0000","last_peered":"2024-09-10T22:03:07.026890+0000","last_clean":"2024-09-10T22:03:07.026890+0000","last_became_active":"2024-09-10T22:03:06.030509+0000","last_became_peered":"2024-09-10T22:03:06.030509+0000","last_unstale":"2024-09-10T22:03:07.026890+0000","last_undegraded":"2024-09-10T22:03:07.026890+0000","last_fullsized":"2024-09-10T22:03:07.026890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:36.477399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038782+0000","last_change":"2024-09-10T22:03:05.033734+0000","last_active":"2024-09-10T22:03:07.038782+0000","last_peered":"2024-09-10T22:03:07.038782+0000","last_clean":"2024-09-10T22:03:07.038782+0000","last_became_active":"2024-09-10T22:03:05.033447+0000","last_became_peered":"2024-09-10T22:03:05.033447+0000","last_unstale":"2024-09-10T22:03:07.038782+0000","last_undegraded":"2024-09-10T22:03:07.038782+0000","last_fullsized":"2024-09-10T22:03:07.038782+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:17.458762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025965+0000","last_change":"2024-09-10T22:03:06.036466+0000","last_active":"2024-09-10T22:03:07.025965+0000","last_peered":"2024-09-10T22:03:07.025965+0000","last_clean":"2024-09-10T22:03:07.025965+0000","last_became_active":"2024-09-10T22:03:06.036181+0000","last_became_peered":"2024-09-10T22:03:06.036181+0000","last_unstale":"2024-09-10T22:03:07.025965+0000","last_undegraded":"2024-09-10T22:03:07.025965+0000","last_fullsized":"2024-09-10T22:03:07.025965+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:23.530611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039062+0000","last_change":"2024-09-10T22:03:05.025872+0000","last_active":"2024-09-10T22:03:07.039062+0000","last_peered":"2024-09-10T22:03:07.039062+0000","last_clean":"2024-09-10T22:03:07.039062+0000","last_became_active":"2024-09-10T22:03:05.025562+0000","last_became_peered":"2024-09-10T22:03:05.025562+0000","last_unstale":"2024-09-10T22:03:07.039062+0000","last_undegraded":"2024-09-10T22:03:07.039062+0000","last_fullsized":"2024-09-10T22:03:07.039062+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:45.628259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038174+0000","last_change":"2024-09-10T22:03:06.033388+0000","last_active":"2024-09-10T22:03:07.038174+0000","last_peered":"2024-09-10T22:03:07.038174+0000","last_clean":"2024-09-10T22:03:07.038174+0000","last_became_active":"2024-09-10T22:03:06.031209+0000","last_became_peered":"2024-09-10T22:03:06.031209+0000","last_unstale":"2024-09-10T22:03:07.038174+0000","last_undegraded":"2024-09-10T22:03:07.038174+0000","last_fullsized":"2024-09-10T22:03:07.038174+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:06.631843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038419+0000","last_change":"2024-09-10T22:03:05.033647+0000","last_active":"2024-09-10T22:03:07.038419+0000","last_peered":"2024-09-10T22:03:07.038419+0000","last_clean":"2024-09-10T22:03:07.038419+0000","last_became_active":"2024-09-10T22:03:05.033309+0000","last_became_peered":"2024-09-10T22:03:05.033309+0000","last_unstale":"2024-09-10T22:03:07.038419+0000","last_undegraded":"2024-09-10T22:03:07.038419+0000","last_fullsized":"2024-09-10T22:03:07.038419+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:36.641074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039106+0000","last_change":"2024-09-10T22:03:06.040834+0000","last_active":"2024-09-10T22:03:07.039106+0000","last_peered":"2024-09-10T22:03:07.039106+0000","last_clean":"2024-09-10T22:03:07.039106+0000","last_became_active":"2024-09-10T22:03:06.040563+0000","last_became_peered":"2024-09-10T22:03:06.040563+0000","last_unstale":"2024-09-10T22:03:07.039106+0000","last_undegraded":"2024-09-10T22:03:07.039106+0000","last_fullsized":"2024-09-10T22:03:07.039106+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:32.805553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":23,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025807+0000","last_change":"2024-09-10T22:03:05.032004+0000","last_active":"2024-09-10T22:03:07.025807+0000","last_peered":"2024-09-10T22:03:07.025807+0000","last_clean":"2024-09-10T22:03:07.025807+0000","last_became_active":"2024-09-10T22:03:05.031714+0000","last_became_peered":"2024-09-10T22:03:05.031714+0000","last_unstale":"2024-09-10T22:03:07.025807+0000","last_undegraded":"2024-09-10T22:03:07.025807+0000","last_fullsized":"2024-09-10T22:03:07.025807+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:09.176212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.036084+0000","last_change":"2024-09-10T22:03:06.026449+0000","last_active":"2024-09-10T22:03:06.036084+0000","last_peered":"2024-09-10T22:03:06.036084+0000","last_clean":"2024-09-10T22:03:06.036084+0000","last_became_active":"2024-09-10T22:03:06.026192+0000","last_became_peered":"2024-09-10T22:03:06.026192+0000","last_unstale":"2024-09-10T22:03:06.036084+0000","last_undegraded":"2024-09-10T22:03:06.036084+0000","last_fullsized":"2024-09-10T22:03:06.036084+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:23.133689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.029132+0000","last_change":"2024-09-10T22:03:05.037720+0000","last_active":"2024-09-10T22:03:06.029132+0000","last_peered":"2024-09-10T22:03:06.029132+0000","last_clean":"2024-09-10T22:03:06.029132+0000","last_became_active":"2024-09-10T22:03:05.037347+0000","last_became_peered":"2024-09-10T22:03:05.037347+0000","last_unstale":"2024-09-10T22:03:06.029132+0000","last_undegraded":"2024-09-10T22:03:06.029132+0000","last_fullsized":"2024-09-10T22:03:06.029132+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:40.664783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026957+0000","last_change":"2024-09-10T22:03:06.034835+0000","last_active":"2024-09-10T22:03:07.026957+0000","last_peered":"2024-09-10T22:03:07.026957+0000","last_clean":"2024-09-10T22:03:07.026957+0000","last_became_active":"2024-09-10T22:03:06.034530+0000","last_became_peered":"2024-09-10T22:03:06.034530+0000","last_unstale":"2024-09-10T22:03:07.026957+0000","last_undegraded":"2024-09-10T22:03:07.026957+0000","last_fullsized":"2024-09-10T22:03:07.026957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.353762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027003+0000","last_change":"2024-09-10T22:03:05.031526+0000","last_active":"2024-09-10T22:03:07.027003+0000","last_peered":"2024-09-10T22:03:07.027003+0000","last_clean":"2024-09-10T22:03:07.027003+0000","last_became_active":"2024-09-10T22:03:05.030940+0000","last_became_peered":"2024-09-10T22:03:05.030940+0000","last_unstale":"2024-09-10T22:03:07.027003+0000","last_undegraded":"2024-09-10T22:03:07.027003+0000","last_fullsized":"2024-09-10T22:03:07.027003+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:49.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027209+0000","last_change":"2024-09-10T22:03:06.032416+0000","last_active":"2024-09-10T22:03:07.027209+0000","last_peered":"2024-09-10T22:03:07.027209+0000","last_clean":"2024-09-10T22:03:07.027209+0000","last_became_active":"2024-09-10T22:03:06.032138+0000","last_became_peered":"2024-09-10T22:03:06.032138+0000","last_unstale":"2024-09-10T22:03:07.027209+0000","last_undegraded":"2024-09-10T22:03:07.027209+0000","last_fullsized":"2024-09-10T22:03:07.027209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:30:42.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039692+0000","last_change":"2024-09-10T22:03:05.033438+0000","last_active":"2024-09-10T22:03:07.039692+0000","last_peered":"2024-09-10T22:03:07.039692+0000","last_clean":"2024-09-10T22:03:07.039692+0000","last_became_active":"2024-09-10T22:03:05.033163+0000","last_became_peered":"2024-09-10T22:03:05.033163+0000","last_unstale":"2024-09-10T22:03:07.039692+0000","last_undegraded":"2024-09-10T22:03:07.039692+0000","last_fullsized":"2024-09-10T22:03:07.039692+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:46.053668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027428+0000","last_change":"2024-09-10T22:03:06.028611+0000","last_active":"2024-09-10T22:03:07.027428+0000","last_peered":"2024-09-10T22:03:07.027428+0000","last_clean":"2024-09-10T22:03:07.027428+0000","last_became_active":"2024-09-10T22:03:06.027653+0000","last_became_peered":"2024-09-10T22:03:06.027653+0000","last_unstale":"2024-09-10T22:03:07.027428+0000","last_undegraded":"2024-09-10T22:03:07.027428+0000","last_fullsized":"2024-09-10T22:03:07.027428+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:09.170787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039596+0000","last_change":"2024-09-10T22:03:05.019638+0000","last_active":"2024-09-10T22:03:07.039596+0000","last_peered":"2024-09-10T22:03:07.039596+0000","last_clean":"2024-09-10T22:03:07.039596+0000","last_became_active":"2024-09-10T22:03:05.019366+0000","last_became_peered":"2024-09-10T22:03:05.019366+0000","last_unstale":"2024-09-10T22:03:07.039596+0000","last_undegraded":"2024-09-10T22:03:07.039596+0000","last_fullsized":"2024-09-10T22:03:07.039596+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:45.792426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038796+0000","last_change":"2024-09-10T22:03:06.030808+0000","last_active":"2024-09-10T22:03:07.038796+0000","last_peered":"2024-09-10T22:03:07.038796+0000","last_clean":"2024-09-10T22:03:07.038796+0000","last_became_active":"2024-09-10T22:03:06.030375+0000","last_became_peered":"2024-09-10T22:03:06.030375+0000","last_unstale":"2024-09-10T22:03:07.038796+0000","last_undegraded":"2024-09-10T22:03:07.038796+0000","last_fullsized":"2024-09-10T22:03:07.038796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:55.553544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038952+0000","last_change":"2024-09-10T22:03:05.014844+0000","last_active":"2024-09-10T22:03:07.038952+0000","last_peered":"2024-09-10T22:03:07.038952+0000","last_clean":"2024-09-10T22:03:07.038952+0000","last_became_active":"2024-09-10T22:03:05.014520+0000","last_became_peered":"2024-09-10T22:03:05.014520+0000","last_unstale":"2024-09-10T22:03:07.038952+0000","last_undegraded":"2024-09-10T22:03:07.038952+0000","last_fullsized":"2024-09-10T22:03:07.038952+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:21.628678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038966+0000","last_change":"2024-09-10T22:03:06.028489+0000","last_active":"2024-09-10T22:03:07.038966+0000","last_peered":"2024-09-10T22:03:07.038966+0000","last_clean":"2024-09-10T22:03:07.038966+0000","last_became_active":"2024-09-10T22:03:06.028229+0000","last_became_peered":"2024-09-10T22:03:06.028229+0000","last_unstale":"2024-09-10T22:03:07.038966+0000","last_undegraded":"2024-09-10T22:03:07.038966+0000","last_fullsized":"2024-09-10T22:03:07.038966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:08.682948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039394+0000","last_change":"2024-09-10T22:03:05.028405+0000","last_active":"2024-09-10T22:03:07.039394+0000","last_peered":"2024-09-10T22:03:07.039394+0000","last_clean":"2024-09-10T22:03:07.039394+0000","last_became_active":"2024-09-10T22:03:05.028024+0000","last_became_peered":"2024-09-10T22:03:05.028024+0000","last_unstale":"2024-09-10T22:03:07.039394+0000","last_undegraded":"2024-09-10T22:03:07.039394+0000","last_fullsized":"2024-09-10T22:03:07.039394+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:45.628259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.042157+0000","last_change":"2024-09-10T22:03:06.042102+0000","last_active":"2024-09-10T22:03:06.042157+0000","last_peered":"2024-09-10T22:03:06.042157+0000","last_clean":"2024-09-10T22:03:06.042157+0000","last_became_active":"2024-09-10T22:03:06.041909+0000","last_became_peered":"2024-09-10T22:03:06.041909+0000","last_unstale":"2024-09-10T22:03:06.042157+0000","last_undegraded":"2024-09-10T22:03:06.042157+0000","last_fullsized":"2024-09-10T22:03:06.042157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:24.358679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025938+0000","last_change":"2024-09-10T22:03:05.034465+0000","last_active":"2024-09-10T22:03:07.025938+0000","last_peered":"2024-09-10T22:03:07.025938+0000","last_clean":"2024-09-10T22:03:07.025938+0000","last_became_active":"2024-09-10T22:03:05.034213+0000","last_became_peered":"2024-09-10T22:03:05.034213+0000","last_unstale":"2024-09-10T22:03:07.025938+0000","last_undegraded":"2024-09-10T22:03:07.025938+0000","last_fullsized":"2024-09-10T22:03:07.025938+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:19.448220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025757+0000","last_change":"2024-09-10T22:03:06.025288+0000","last_active":"2024-09-10T22:03:07.025757+0000","last_peered":"2024-09-10T22:03:07.025757+0000","last_clean":"2024-09-10T22:03:07.025757+0000","last_became_active":"2024-09-10T22:03:06.024908+0000","last_became_peered":"2024-09-10T22:03:06.024908+0000","last_unstale":"2024-09-10T22:03:07.025757+0000","last_undegraded":"2024-09-10T22:03:07.025757+0000","last_fullsized":"2024-09-10T22:03:07.025757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:04.451178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038941+0000","last_change":"2024-09-10T22:03:05.025906+0000","last_active":"2024-09-10T22:03:07.038941+0000","last_peered":"2024-09-10T22:03:07.038941+0000","last_clean":"2024-09-10T22:03:07.038941+0000","last_became_active":"2024-09-10T22:03:05.025637+0000","last_became_peered":"2024-09-10T22:03:05.025637+0000","last_unstale":"2024-09-10T22:03:07.038941+0000","last_undegraded":"2024-09-10T22:03:07.038941+0000","last_fullsized":"2024-09-10T22:03:07.038941+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:16.620483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027626+0000","last_change":"2024-09-10T22:03:06.030444+0000","last_active":"2024-09-10T22:03:07.027626+0000","last_peered":"2024-09-10T22:03:07.027626+0000","last_clean":"2024-09-10T22:03:07.027626+0000","last_became_active":"2024-09-10T22:03:06.030015+0000","last_became_peered":"2024-09-10T22:03:06.030015+0000","last_unstale":"2024-09-10T22:03:07.027626+0000","last_undegraded":"2024-09-10T22:03:07.027626+0000","last_fullsized":"2024-09-10T22:03:07.027626+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:22.607616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.048554+0000","last_change":"2024-09-10T22:03:05.028365+0000","last_active":"2024-09-10T22:03:07.048554+0000","last_peered":"2024-09-10T22:03:07.048554+0000","last_clean":"2024-09-10T22:03:07.048554+0000","last_became_active":"2024-09-10T22:03:05.028085+0000","last_became_peered":"2024-09-10T22:03:05.028085+0000","last_unstale":"2024-09-10T22:03:07.048554+0000","last_undegraded":"2024-09-10T22:03:07.048554+0000","last_fullsized":"2024-09-10T22:03:07.048554+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:06.009142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026452+0000","last_change":"2024-09-10T22:03:06.026047+0000","last_active":"2024-09-10T22:03:07.026452+0000","last_peered":"2024-09-10T22:03:07.026452+0000","last_clean":"2024-09-10T22:03:07.026452+0000","last_became_active":"2024-09-10T22:03:06.025760+0000","last_became_peered":"2024-09-10T22:03:06.025760+0000","last_unstale":"2024-09-10T22:03:07.026452+0000","last_undegraded":"2024-09-10T22:03:07.026452+0000","last_fullsized":"2024-09-10T22:03:07.026452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:07.012049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039836+0000","last_change":"2024-09-10T22:03:05.024587+0000","last_active":"2024-09-10T22:03:07.039836+0000","last_peered":"2024-09-10T22:03:07.039836+0000","last_clean":"2024-09-10T22:03:07.039836+0000","last_became_active":"2024-09-10T22:03:05.024301+0000","last_became_peered":"2024-09-10T22:03:05.024301+0000","last_unstale":"2024-09-10T22:03:07.039836+0000","last_undegraded":"2024-09-10T22:03:07.039836+0000","last_fullsized":"2024-09-10T22:03:07.039836+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:03.448272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039390+0000","last_change":"2024-09-10T22:03:06.037303+0000","last_active":"2024-09-10T22:03:07.039390+0000","last_peered":"2024-09-10T22:03:07.039390+0000","last_clean":"2024-09-10T22:03:07.039390+0000","last_became_active":"2024-09-10T22:03:06.035954+0000","last_became_peered":"2024-09-10T22:03:06.035954+0000","last_unstale":"2024-09-10T22:03:07.039390+0000","last_undegraded":"2024-09-10T22:03:07.039390+0000","last_fullsized":"2024-09-10T22:03:07.039390+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:28.781866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038879+0000","last_change":"2024-09-10T22:03:05.030431+0000","last_active":"2024-09-10T22:03:07.038879+0000","last_peered":"2024-09-10T22:03:07.038879+0000","last_clean":"2024-09-10T22:03:07.038879+0000","last_became_active":"2024-09-10T22:03:05.030078+0000","last_became_peered":"2024-09-10T22:03:05.030078+0000","last_unstale":"2024-09-10T22:03:07.038879+0000","last_undegraded":"2024-09-10T22:03:07.038879+0000","last_fullsized":"2024-09-10T22:03:07.038879+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:38.227987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039004+0000","last_change":"2024-09-10T22:03:06.038238+0000","last_active":"2024-09-10T22:03:07.039004+0000","last_peered":"2024-09-10T22:03:07.039004+0000","last_clean":"2024-09-10T22:03:07.039004+0000","last_became_active":"2024-09-10T22:03:06.037913+0000","last_became_peered":"2024-09-10T22:03:06.037913+0000","last_unstale":"2024-09-10T22:03:07.039004+0000","last_undegraded":"2024-09-10T22:03:07.039004+0000","last_fullsized":"2024-09-10T22:03:07.039004+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:55.234307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039325+0000","last_change":"2024-09-10T22:03:05.023416+0000","last_active":"2024-09-10T22:03:07.039325+0000","last_peered":"2024-09-10T22:03:07.039325+0000","last_clean":"2024-09-10T22:03:07.039325+0000","last_became_active":"2024-09-10T22:03:05.023156+0000","last_became_peered":"2024-09-10T22:03:05.023156+0000","last_unstale":"2024-09-10T22:03:07.039325+0000","last_undegraded":"2024-09-10T22:03:07.039325+0000","last_fullsized":"2024-09-10T22:03:07.039325+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:36.337853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038655+0000","last_change":"2024-09-10T22:03:06.041670+0000","last_active":"2024-09-10T22:03:07.038655+0000","last_peered":"2024-09-10T22:03:07.038655+0000","last_clean":"2024-09-10T22:03:07.038655+0000","last_became_active":"2024-09-10T22:03:06.041375+0000","last_became_peered":"2024-09-10T22:03:06.041375+0000","last_unstale":"2024-09-10T22:03:07.038655+0000","last_undegraded":"2024-09-10T22:03:07.038655+0000","last_fullsized":"2024-09-10T22:03:07.038655+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:39.734532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026452+0000","last_change":"2024-09-10T22:03:05.022463+0000","last_active":"2024-09-10T22:03:07.026452+0000","last_peered":"2024-09-10T22:03:07.026452+0000","last_clean":"2024-09-10T22:03:07.026452+0000","last_became_active":"2024-09-10T22:03:05.022074+0000","last_became_peered":"2024-09-10T22:03:05.022074+0000","last_unstale":"2024-09-10T22:03:07.026452+0000","last_undegraded":"2024-09-10T22:03:07.026452+0000","last_fullsized":"2024-09-10T22:03:07.026452+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:35.474493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039118+0000","last_change":"2024-09-10T22:03:06.041241+0000","last_active":"2024-09-10T22:03:07.039118+0000","last_peered":"2024-09-10T22:03:07.039118+0000","last_clean":"2024-09-10T22:03:07.039118+0000","last_became_active":"2024-09-10T22:03:06.040931+0000","last_became_peered":"2024-09-10T22:03:06.040931+0000","last_unstale":"2024-09-10T22:03:07.039118+0000","last_undegraded":"2024-09-10T22:03:07.039118+0000","last_fullsized":"2024-09-10T22:03:07.039118+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:50.477410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038564+0000","last_change":"2024-09-10T22:03:05.030279+0000","last_active":"2024-09-10T22:03:07.038564+0000","last_peered":"2024-09-10T22:03:07.038564+0000","last_clean":"2024-09-10T22:03:07.038564+0000","last_became_active":"2024-09-10T22:03:05.029962+0000","last_became_peered":"2024-09-10T22:03:05.029962+0000","last_unstale":"2024-09-10T22:03:07.038564+0000","last_undegraded":"2024-09-10T22:03:07.038564+0000","last_fullsized":"2024-09-10T22:03:07.038564+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:40.890890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027062+0000","last_change":"2024-09-10T22:03:06.034821+0000","last_active":"2024-09-10T22:03:07.027062+0000","last_peered":"2024-09-10T22:03:07.027062+0000","last_clean":"2024-09-10T22:03:07.027062+0000","last_became_active":"2024-09-10T22:03:06.032302+0000","last_became_peered":"2024-09-10T22:03:06.032302+0000","last_unstale":"2024-09-10T22:03:07.027062+0000","last_undegraded":"2024-09-10T22:03:07.027062+0000","last_fullsized":"2024-09-10T22:03:07.027062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:23.530611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025862+0000","last_change":"2024-09-10T22:03:05.038660+0000","last_active":"2024-09-10T22:03:07.025862+0000","last_peered":"2024-09-10T22:03:07.025862+0000","last_clean":"2024-09-10T22:03:07.025862+0000","last_became_active":"2024-09-10T22:03:05.038474+0000","last_became_peered":"2024-09-10T22:03:05.038474+0000","last_unstale":"2024-09-10T22:03:07.025862+0000","last_undegraded":"2024-09-10T22:03:07.025862+0000","last_fullsized":"2024-09-10T22:03:07.025862+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:46.706749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026783+0000","last_change":"2024-09-10T22:03:06.034875+0000","last_active":"2024-09-10T22:03:07.026783+0000","last_peered":"2024-09-10T22:03:07.026783+0000","last_clean":"2024-09-10T22:03:07.026783+0000","last_became_active":"2024-09-10T22:03:06.034489+0000","last_became_peered":"2024-09-10T22:03:06.034489+0000","last_unstale":"2024-09-10T22:03:07.026783+0000","last_undegraded":"2024-09-10T22:03:07.026783+0000","last_fullsized":"2024-09-10T22:03:07.026783+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:50.412298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039846+0000","last_change":"2024-09-10T22:03:05.038671+0000","last_active":"2024-09-10T22:03:07.039846+0000","last_peered":"2024-09-10T22:03:07.039846+0000","last_clean":"2024-09-10T22:03:07.039846+0000","last_became_active":"2024-09-10T22:03:05.038365+0000","last_became_peered":"2024-09-10T22:03:05.038365+0000","last_unstale":"2024-09-10T22:03:07.039846+0000","last_undegraded":"2024-09-10T22:03:07.039846+0000","last_fullsized":"2024-09-10T22:03:07.039846+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:08.660483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":152,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038791+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:03:07.038791+0000","last_peered":"2024-09-10T22:03:07.038791+0000","last_clean":"2024-09-10T22:03:07.038791+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:03:07.038791+0000","last_undegraded":"2024-09-10T22:03:07.038791+0000","last_fullsized":"2024-09-10T22:03:07.038791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":1093632,"data_stored":786826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509477,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":197040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35836,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336004,"statfs":{"total":96636764160,"available":96600068096,"internally_reserved":0,"allocated":499712,"data_stored":196438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509477,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35844,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335996,"statfs":{"total":96636764160,"available":96600059904,"internally_reserved":0,"allocated":507904,"data_stored":197980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":507904,"data_stored":197006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35852,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335988,"statfs":{"total":96636764160,"available":96600051712,"internally_reserved":0,"allocated":512000,"data_stored":202738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1093632,"data_stored":786770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:03:10.176 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:03:10.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.361+0000 7f9ad7600640 1 -- 172.21.15.66:0/3719890227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad015bd40 msgr2=0x7f9ad00cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:10.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.361+0000 7f9ad7600640 1 --2- 172.21.15.66:0/3719890227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad015bd40 0x7f9ad00cffa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009900 tx=0x7f9ac401c850 comp rx=0 tx=0).stop 2024-09-10T22:03:10.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.361+0000 7f9ad7600640 1 -- 172.21.15.66:0/3719890227 shutdown_connections 2024-09-10T22:03:10.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.361+0000 7f9ad7600640 1 --2- 172.21.15.66:0/3719890227 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ad00cf180 0x7f9ad00d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.361+0000 7f9ad7600640 1 --2- 172.21.15.66:0/3719890227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad015bd40 0x7f9ad00cffa0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.361+0000 7f9ad7600640 1 -- 172.21.15.66:0/3719890227 >> 172.21.15.66:0/3719890227 conn(0x7f9ad00d1c60 msgr2=0x7f9ad00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:10.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.361+0000 7f9ad7600640 1 -- 172.21.15.66:0/3719890227 shutdown_connections 2024-09-10T22:03:10.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.361+0000 7f9ad7600640 1 -- 172.21.15.66:0/3719890227 wait complete. 2024-09-10T22:03:10.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad7600640 1 Processor -- start 2024-09-10T22:03:10.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad7600640 1 -- start start 2024-09-10T22:03:10.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad7600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ad00cf180 0x7f9ad00b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:10.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad7600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad00cf830 0x7f9ad00b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:10.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad7600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad015bd40 0x7f9ad01573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:10.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad7600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0168ba0 con 0x7f9ad00cf830 2024-09-10T22:03:10.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad7600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0168a20 con 0x7f9ad015bd40 2024-09-10T22:03:10.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad7600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad0168d20 con 0x7f9ad00cf180 2024-09-10T22:03:10.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9acfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad00cf830 0x7f9ad00b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:10.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9acfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad00cf830 0x7f9ad00b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43430/0 (socket says 172.21.15.66:43430) 2024-09-10T22:03:10.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9acfe00640 1 -- 172.21.15.66:0/506106764 learned_addr learned my addr 172.21.15.66:0/506106764 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:10.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad4c00640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ad00cf180 0x7f9ad00b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:10.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad5600640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad015bd40 0x7f9ad01573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:10.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9acfe00640 1 -- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ad00cf180 msgr2=0x7f9ad00b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:10.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9acfe00640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ad00cf180 0x7f9ad00b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9acfe00640 1 -- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad015bd40 msgr2=0x7f9ad01573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:10.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9acfe00640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad015bd40 0x7f9ad01573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9acfe00640 1 -- 172.21.15.66:0/506106764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad00b1840 con 0x7f9ad00cf830 2024-09-10T22:03:10.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad5600640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad015bd40 0x7f9ad01573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:10.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9acfe00640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad00cf830 0x7f9ad00b7560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00c4b0 tx=0x7f9abc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:10.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9accc00640 1 -- 172.21.15.66:0/506106764 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0052d0 con 0x7f9ad00cf830 2024-09-10T22:03:10.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9accc00640 1 -- 172.21.15.66:0/506106764 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9abc005470 con 0x7f9ad00cf830 2024-09-10T22:03:10.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad7600640 1 -- 172.21.15.66:0/506106764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad00b1b20 con 0x7f9ad00cf830 2024-09-10T22:03:10.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9accc00640 1 -- 172.21.15.66:0/506106764 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc012870 con 0x7f9ad00cf830 2024-09-10T22:03:10.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad7600640 1 -- 172.21.15.66:0/506106764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad00b20e0 con 0x7f9ad00cf830 2024-09-10T22:03:10.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.365+0000 7f9ad7600640 1 -- 172.21.15.66:0/506106764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad00aec10 con 0x7f9ad00cf830 2024-09-10T22:03:10.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.369+0000 7f9accc00640 1 -- 172.21.15.66:0/506106764 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f9abc003600 con 0x7f9ad00cf830 2024-09-10T22:03:10.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.369+0000 7f9accc00640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ab0072940 0x7f9ab0074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:10.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.369+0000 7f9accc00640 1 -- 172.21.15.66:0/506106764 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9abc093410 con 0x7f9ad00cf830 2024-09-10T22:03:10.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.369+0000 7f9ad4c00640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ab0072940 0x7f9ab0074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:10.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.369+0000 7f9ad4c00640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ab0072940 0x7f9ab0074e00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40af000 tx=0x7f9ac401b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:10.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.373+0000 7f9accc00640 1 -- 172.21.15.66:0/506106764 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9abc05e560 con 0x7f9ad00cf830 2024-09-10T22:03:10.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.545+0000 7f9ad7600640 1 -- 172.21.15.66:0/506106764 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ad0076150 con 0x7f9ab0072940 2024-09-10T22:03:10.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.549+0000 7f9accc00640 1 -- 172.21.15.66:0/506106764 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251928 (secure 0 0 0) 0x7f9ad0076150 con 0x7f9ab0072940 2024-09-10T22:03:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:03:10.557 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:03:10.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.553+0000 7f9ad7600640 1 -- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ab0072940 msgr2=0x7f9ab0074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:10.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.553+0000 7f9ad7600640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ab0072940 0x7f9ab0074e00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40af000 tx=0x7f9ac401b090 comp rx=0 tx=0).stop 2024-09-10T22:03:10.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.553+0000 7f9ad7600640 1 -- 172.21.15.66:0/506106764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad00cf830 msgr2=0x7f9ad00b7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:10.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.553+0000 7f9ad7600640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad00cf830 0x7f9ad00b7560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00c4b0 tx=0x7f9abc00c970 comp rx=0 tx=0).stop 2024-09-10T22:03:10.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.553+0000 7f9ad7600640 1 -- 172.21.15.66:0/506106764 shutdown_connections 2024-09-10T22:03:10.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.553+0000 7f9ad7600640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ab0072940 0x7f9ab0074e00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.553+0000 7f9ad7600640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad015bd40 0x7f9ad01573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.553+0000 7f9ad7600640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad00cf830 0x7f9ad00b7560 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.553+0000 7f9ad7600640 1 --2- 172.21.15.66:0/506106764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ad00cf180 0x7f9ad00b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.553+0000 7f9ad7600640 1 -- 172.21.15.66:0/506106764 >> 172.21.15.66:0/506106764 conn(0x7f9ad00d1c60 msgr2=0x7f9ad015a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:10.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.553+0000 7f9ad7600640 1 -- 172.21.15.66:0/506106764 shutdown_connections 2024-09-10T22:03:10.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.553+0000 7f9ad7600640 1 -- 172.21.15.66:0/506106764 wait complete. 2024-09-10T22:03:10.577 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-10T22:03:10.345972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":139,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275396,"kb_used_data":5080,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754699324,"statfs":{"total":773094113280,"available":772812107776,"internally_reserved":0,"allocated":5201920,"data_stored":2754391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":276614708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001449"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039719+0000","last_change":"2024-09-10T22:03:06.039627+0000","last_active":"2024-09-10T22:03:07.039719+0000","last_peered":"2024-09-10T22:03:07.039719+0000","last_clean":"2024-09-10T22:03:07.039719+0000","last_became_active":"2024-09-10T22:03:06.039339+0000","last_became_peered":"2024-09-10T22:03:06.039339+0000","last_unstale":"2024-09-10T22:03:07.039719+0000","last_undegraded":"2024-09-10T22:03:07.039719+0000","last_fullsized":"2024-09-10T22:03:07.039719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:22.417821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039978+0000","last_change":"2024-09-10T22:03:06.037554+0000","last_active":"2024-09-10T22:03:07.039978+0000","last_peered":"2024-09-10T22:03:07.039978+0000","last_clean":"2024-09-10T22:03:07.039978+0000","last_became_active":"2024-09-10T22:03:06.037227+0000","last_became_peered":"2024-09-10T22:03:06.037227+0000","last_unstale":"2024-09-10T22:03:07.039978+0000","last_undegraded":"2024-09-10T22:03:07.039978+0000","last_fullsized":"2024-09-10T22:03:07.039978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:03.034797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026256+0000","last_change":"2024-09-10T22:03:06.028445+0000","last_active":"2024-09-10T22:03:07.026256+0000","last_peered":"2024-09-10T22:03:07.026256+0000","last_clean":"2024-09-10T22:03:07.026256+0000","last_became_active":"2024-09-10T22:03:06.027823+0000","last_became_peered":"2024-09-10T22:03:06.027823+0000","last_unstale":"2024-09-10T22:03:07.026256+0000","last_undegraded":"2024-09-10T22:03:07.026256+0000","last_fullsized":"2024-09-10T22:03:07.026256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:47.056575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026221+0000","last_change":"2024-09-10T22:03:06.028899+0000","last_active":"2024-09-10T22:03:07.026221+0000","last_peered":"2024-09-10T22:03:07.026221+0000","last_clean":"2024-09-10T22:03:07.026221+0000","last_became_active":"2024-09-10T22:03:06.028621+0000","last_became_peered":"2024-09-10T22:03:06.028621+0000","last_unstale":"2024-09-10T22:03:07.026221+0000","last_undegraded":"2024-09-10T22:03:07.026221+0000","last_fullsized":"2024-09-10T22:03:07.026221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:46.795332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039569+0000","last_change":"2024-09-10T22:03:06.037232+0000","last_active":"2024-09-10T22:03:07.039569+0000","last_peered":"2024-09-10T22:03:07.039569+0000","last_clean":"2024-09-10T22:03:07.039569+0000","last_became_active":"2024-09-10T22:03:06.036887+0000","last_became_peered":"2024-09-10T22:03:06.036887+0000","last_unstale":"2024-09-10T22:03:07.039569+0000","last_undegraded":"2024-09-10T22:03:07.039569+0000","last_fullsized":"2024-09-10T22:03:07.039569+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:01:20.819903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038630+0000","last_change":"2024-09-10T22:03:06.035654+0000","last_active":"2024-09-10T22:03:07.038630+0000","last_peered":"2024-09-10T22:03:07.038630+0000","last_clean":"2024-09-10T22:03:07.038630+0000","last_became_active":"2024-09-10T22:03:06.035384+0000","last_became_peered":"2024-09-10T22:03:06.035384+0000","last_unstale":"2024-09-10T22:03:07.038630+0000","last_undegraded":"2024-09-10T22:03:07.038630+0000","last_fullsized":"2024-09-10T22:03:07.038630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:37.723354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039702+0000","last_change":"2024-09-10T22:03:06.038694+0000","last_active":"2024-09-10T22:03:07.039702+0000","last_peered":"2024-09-10T22:03:07.039702+0000","last_clean":"2024-09-10T22:03:07.039702+0000","last_became_active":"2024-09-10T22:03:06.038388+0000","last_became_peered":"2024-09-10T22:03:06.038388+0000","last_unstale":"2024-09-10T22:03:07.039702+0000","last_undegraded":"2024-09-10T22:03:07.039702+0000","last_fullsized":"2024-09-10T22:03:07.039702+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:01:20.819903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025648+0000","last_change":"2024-09-10T22:03:06.029337+0000","last_active":"2024-09-10T22:03:07.025648+0000","last_peered":"2024-09-10T22:03:07.025648+0000","last_clean":"2024-09-10T22:03:07.025648+0000","last_became_active":"2024-09-10T22:03:06.028302+0000","last_became_peered":"2024-09-10T22:03:06.028302+0000","last_unstale":"2024-09-10T22:03:07.025648+0000","last_undegraded":"2024-09-10T22:03:07.025648+0000","last_fullsized":"2024-09-10T22:03:07.025648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:47.056575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026770+0000","last_change":"2024-09-10T22:03:06.025932+0000","last_active":"2024-09-10T22:03:07.026770+0000","last_peered":"2024-09-10T22:03:07.026770+0000","last_clean":"2024-09-10T22:03:07.026770+0000","last_became_active":"2024-09-10T22:03:06.025550+0000","last_became_peered":"2024-09-10T22:03:06.025550+0000","last_unstale":"2024-09-10T22:03:07.026770+0000","last_undegraded":"2024-09-10T22:03:07.026770+0000","last_fullsized":"2024-09-10T22:03:07.026770+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:38.800945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026786+0000","last_change":"2024-09-10T22:03:06.027344+0000","last_active":"2024-09-10T22:03:07.026786+0000","last_peered":"2024-09-10T22:03:07.026786+0000","last_clean":"2024-09-10T22:03:07.026786+0000","last_became_active":"2024-09-10T22:03:06.027082+0000","last_became_peered":"2024-09-10T22:03:06.027082+0000","last_unstale":"2024-09-10T22:03:07.026786+0000","last_undegraded":"2024-09-10T22:03:07.026786+0000","last_fullsized":"2024-09-10T22:03:07.026786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:28.781866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026359+0000","last_change":"2024-09-10T22:03:06.026675+0000","last_active":"2024-09-10T22:03:07.026359+0000","last_peered":"2024-09-10T22:03:07.026359+0000","last_clean":"2024-09-10T22:03:07.026359+0000","last_became_active":"2024-09-10T22:03:06.024203+0000","last_became_peered":"2024-09-10T22:03:06.024203+0000","last_unstale":"2024-09-10T22:03:07.026359+0000","last_undegraded":"2024-09-10T22:03:07.026359+0000","last_fullsized":"2024-09-10T22:03:07.026359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:37.688947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039471+0000","last_change":"2024-09-10T22:03:06.034620+0000","last_active":"2024-09-10T22:03:07.039471+0000","last_peered":"2024-09-10T22:03:07.039471+0000","last_clean":"2024-09-10T22:03:07.039471+0000","last_became_active":"2024-09-10T22:03:06.034158+0000","last_became_peered":"2024-09-10T22:03:06.034158+0000","last_unstale":"2024-09-10T22:03:07.039471+0000","last_undegraded":"2024-09-10T22:03:07.039471+0000","last_fullsized":"2024-09-10T22:03:07.039471+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:32.805553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.042309+0000","last_change":"2024-09-10T22:03:06.037582+0000","last_active":"2024-09-10T22:03:06.042309+0000","last_peered":"2024-09-10T22:03:06.042309+0000","last_clean":"2024-09-10T22:03:06.042309+0000","last_became_active":"2024-09-10T22:03:06.037283+0000","last_became_peered":"2024-09-10T22:03:06.037283+0000","last_unstale":"2024-09-10T22:03:06.042309+0000","last_undegraded":"2024-09-10T22:03:06.042309+0000","last_fullsized":"2024-09-10T22:03:06.042309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:58.670828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025866+0000","last_change":"2024-09-10T22:03:06.030447+0000","last_active":"2024-09-10T22:03:07.025866+0000","last_peered":"2024-09-10T22:03:07.025866+0000","last_clean":"2024-09-10T22:03:07.025866+0000","last_became_active":"2024-09-10T22:03:06.030179+0000","last_became_peered":"2024-09-10T22:03:06.030179+0000","last_unstale":"2024-09-10T22:03:07.025866+0000","last_undegraded":"2024-09-10T22:03:07.025866+0000","last_fullsized":"2024-09-10T22:03:07.025866+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:37.643981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038289+0000","last_change":"2024-09-10T22:03:06.028413+0000","last_active":"2024-09-10T22:03:07.038289+0000","last_peered":"2024-09-10T22:03:07.038289+0000","last_clean":"2024-09-10T22:03:07.038289+0000","last_became_active":"2024-09-10T22:03:06.027971+0000","last_became_peered":"2024-09-10T22:03:06.027971+0000","last_unstale":"2024-09-10T22:03:07.038289+0000","last_undegraded":"2024-09-10T22:03:07.038289+0000","last_fullsized":"2024-09-10T22:03:07.038289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:30.382237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026361+0000","last_change":"2024-09-10T22:03:06.032259+0000","last_active":"2024-09-10T22:03:07.026361+0000","last_peered":"2024-09-10T22:03:07.026361+0000","last_clean":"2024-09-10T22:03:07.026361+0000","last_became_active":"2024-09-10T22:03:06.029852+0000","last_became_peered":"2024-09-10T22:03:06.029852+0000","last_unstale":"2024-09-10T22:03:07.026361+0000","last_undegraded":"2024-09-10T22:03:07.026361+0000","last_fullsized":"2024-09-10T22:03:07.026361+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:47.709655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026379+0000","last_change":"2024-09-10T22:03:06.028591+0000","last_active":"2024-09-10T22:03:07.026379+0000","last_peered":"2024-09-10T22:03:07.026379+0000","last_clean":"2024-09-10T22:03:07.026379+0000","last_became_active":"2024-09-10T22:03:06.027382+0000","last_became_peered":"2024-09-10T22:03:06.027382+0000","last_unstale":"2024-09-10T22:03:07.026379+0000","last_undegraded":"2024-09-10T22:03:07.026379+0000","last_fullsized":"2024-09-10T22:03:07.026379+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:09.663390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027324+0000","last_change":"2024-09-10T22:03:06.036419+0000","last_active":"2024-09-10T22:03:07.027324+0000","last_peered":"2024-09-10T22:03:07.027324+0000","last_clean":"2024-09-10T22:03:07.027324+0000","last_became_active":"2024-09-10T22:03:06.031292+0000","last_became_peered":"2024-09-10T22:03:06.031292+0000","last_unstale":"2024-09-10T22:03:07.027324+0000","last_undegraded":"2024-09-10T22:03:07.027324+0000","last_fullsized":"2024-09-10T22:03:07.027324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:18.461668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027495+0000","last_change":"2024-09-10T22:03:06.031396+0000","last_active":"2024-09-10T22:03:07.027495+0000","last_peered":"2024-09-10T22:03:07.027495+0000","last_clean":"2024-09-10T22:03:07.027495+0000","last_became_active":"2024-09-10T22:03:06.031109+0000","last_became_peered":"2024-09-10T22:03:06.031109+0000","last_unstale":"2024-09-10T22:03:07.027495+0000","last_undegraded":"2024-09-10T22:03:07.027495+0000","last_fullsized":"2024-09-10T22:03:07.027495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:00.117477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.031024+0000","last_change":"2024-09-10T22:03:06.028434+0000","last_active":"2024-09-10T22:03:06.031024+0000","last_peered":"2024-09-10T22:03:06.031024+0000","last_clean":"2024-09-10T22:03:06.031024+0000","last_became_active":"2024-09-10T22:03:06.028070+0000","last_became_peered":"2024-09-10T22:03:06.028070+0000","last_unstale":"2024-09-10T22:03:06.031024+0000","last_undegraded":"2024-09-10T22:03:06.031024+0000","last_fullsized":"2024-09-10T22:03:06.031024+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:46.063806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038646+0000","last_change":"2024-09-10T22:03:06.029313+0000","last_active":"2024-09-10T22:03:07.038646+0000","last_peered":"2024-09-10T22:03:07.038646+0000","last_clean":"2024-09-10T22:03:07.038646+0000","last_became_active":"2024-09-10T22:03:06.028936+0000","last_became_peered":"2024-09-10T22:03:06.028936+0000","last_unstale":"2024-09-10T22:03:07.038646+0000","last_undegraded":"2024-09-10T22:03:07.038646+0000","last_fullsized":"2024-09-10T22:03:07.038646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:57.653238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026125+0000","last_change":"2024-09-10T22:03:06.027468+0000","last_active":"2024-09-10T22:03:07.026125+0000","last_peered":"2024-09-10T22:03:07.026125+0000","last_clean":"2024-09-10T22:03:07.026125+0000","last_became_active":"2024-09-10T22:03:06.027176+0000","last_became_peered":"2024-09-10T22:03:06.027176+0000","last_unstale":"2024-09-10T22:03:07.026125+0000","last_undegraded":"2024-09-10T22:03:07.026125+0000","last_fullsized":"2024-09-10T22:03:07.026125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:18.461668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026919+0000","last_change":"2024-09-10T22:03:06.037106+0000","last_active":"2024-09-10T22:03:07.026919+0000","last_peered":"2024-09-10T22:03:07.026919+0000","last_clean":"2024-09-10T22:03:07.026919+0000","last_became_active":"2024-09-10T22:03:06.035908+0000","last_became_peered":"2024-09-10T22:03:06.035908+0000","last_unstale":"2024-09-10T22:03:07.026919+0000","last_undegraded":"2024-09-10T22:03:07.026919+0000","last_fullsized":"2024-09-10T22:03:07.026919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:50.117268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026578+0000","last_change":"2024-09-10T22:03:06.035594+0000","last_active":"2024-09-10T22:03:07.026578+0000","last_peered":"2024-09-10T22:03:07.026578+0000","last_clean":"2024-09-10T22:03:07.026578+0000","last_became_active":"2024-09-10T22:03:06.035337+0000","last_became_peered":"2024-09-10T22:03:06.035337+0000","last_unstale":"2024-09-10T22:03:07.026578+0000","last_undegraded":"2024-09-10T22:03:07.026578+0000","last_fullsized":"2024-09-10T22:03:07.026578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:41.893796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027486+0000","last_change":"2024-09-10T22:03:06.024010+0000","last_active":"2024-09-10T22:03:07.027486+0000","last_peered":"2024-09-10T22:03:07.027486+0000","last_clean":"2024-09-10T22:03:07.027486+0000","last_became_active":"2024-09-10T22:03:06.023686+0000","last_became_peered":"2024-09-10T22:03:06.023686+0000","last_unstale":"2024-09-10T22:03:07.027486+0000","last_undegraded":"2024-09-10T22:03:07.027486+0000","last_fullsized":"2024-09-10T22:03:07.027486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:08.682948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.036110+0000","last_change":"2024-09-10T22:03:06.029755+0000","last_active":"2024-09-10T22:03:06.036110+0000","last_peered":"2024-09-10T22:03:06.036110+0000","last_clean":"2024-09-10T22:03:06.036110+0000","last_became_active":"2024-09-10T22:03:06.029478+0000","last_became_peered":"2024-09-10T22:03:06.029478+0000","last_unstale":"2024-09-10T22:03:06.036110+0000","last_undegraded":"2024-09-10T22:03:06.036110+0000","last_fullsized":"2024-09-10T22:03:06.036110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:10.497569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026489+0000","last_change":"2024-09-10T22:03:06.029699+0000","last_active":"2024-09-10T22:03:07.026489+0000","last_peered":"2024-09-10T22:03:07.026489+0000","last_clean":"2024-09-10T22:03:07.026489+0000","last_became_active":"2024-09-10T22:03:06.028169+0000","last_became_peered":"2024-09-10T22:03:06.028169+0000","last_unstale":"2024-09-10T22:03:07.026489+0000","last_undegraded":"2024-09-10T22:03:07.026489+0000","last_fullsized":"2024-09-10T22:03:07.026489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:04.451178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039947+0000","last_change":"2024-09-10T22:03:06.034331+0000","last_active":"2024-09-10T22:03:07.039947+0000","last_peered":"2024-09-10T22:03:07.039947+0000","last_clean":"2024-09-10T22:03:07.039947+0000","last_became_active":"2024-09-10T22:03:06.033766+0000","last_became_peered":"2024-09-10T22:03:06.033766+0000","last_unstale":"2024-09-10T22:03:07.039947+0000","last_undegraded":"2024-09-10T22:03:07.039947+0000","last_fullsized":"2024-09-10T22:03:07.039947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:28.781866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026980+0000","last_change":"2024-09-10T22:03:06.025202+0000","last_active":"2024-09-10T22:03:07.026980+0000","last_peered":"2024-09-10T22:03:07.026980+0000","last_clean":"2024-09-10T22:03:07.026980+0000","last_became_active":"2024-09-10T22:03:06.024919+0000","last_became_peered":"2024-09-10T22:03:06.024919+0000","last_unstale":"2024-09-10T22:03:07.026980+0000","last_undegraded":"2024-09-10T22:03:07.026980+0000","last_fullsized":"2024-09-10T22:03:07.026980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:46.605911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026700+0000","last_change":"2024-09-10T22:03:06.036207+0000","last_active":"2024-09-10T22:03:07.026700+0000","last_peered":"2024-09-10T22:03:07.026700+0000","last_clean":"2024-09-10T22:03:07.026700+0000","last_became_active":"2024-09-10T22:03:06.035283+0000","last_became_peered":"2024-09-10T22:03:06.035283+0000","last_unstale":"2024-09-10T22:03:07.026700+0000","last_undegraded":"2024-09-10T22:03:07.026700+0000","last_fullsized":"2024-09-10T22:03:07.026700+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:53.137403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038323+0000","last_change":"2024-09-10T22:03:06.029630+0000","last_active":"2024-09-10T22:03:07.038323+0000","last_peered":"2024-09-10T22:03:07.038323+0000","last_clean":"2024-09-10T22:03:07.038323+0000","last_became_active":"2024-09-10T22:03:06.029361+0000","last_became_peered":"2024-09-10T22:03:06.029361+0000","last_unstale":"2024-09-10T22:03:07.038323+0000","last_undegraded":"2024-09-10T22:03:07.038323+0000","last_fullsized":"2024-09-10T22:03:07.038323+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:30.075016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.040197+0000","last_change":"2024-09-10T22:03:06.040090+0000","last_active":"2024-09-10T22:03:06.040197+0000","last_peered":"2024-09-10T22:03:06.040197+0000","last_clean":"2024-09-10T22:03:06.040197+0000","last_became_active":"2024-09-10T22:03:06.039810+0000","last_became_peered":"2024-09-10T22:03:06.039810+0000","last_unstale":"2024-09-10T22:03:06.040197+0000","last_undegraded":"2024-09-10T22:03:06.040197+0000","last_fullsized":"2024-09-10T22:03:06.040197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:32.918048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039754+0000","last_change":"2024-09-10T22:03:06.031324+0000","last_active":"2024-09-10T22:03:07.039754+0000","last_peered":"2024-09-10T22:03:07.039754+0000","last_clean":"2024-09-10T22:03:07.039754+0000","last_became_active":"2024-09-10T22:03:06.031015+0000","last_became_peered":"2024-09-10T22:03:06.031015+0000","last_unstale":"2024-09-10T22:03:07.039754+0000","last_undegraded":"2024-09-10T22:03:07.039754+0000","last_fullsized":"2024-09-10T22:03:07.039754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:18.808705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":19,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039179+0000","last_change":"2024-09-10T22:03:05.028045+0000","last_active":"2024-09-10T22:03:07.039179+0000","last_peered":"2024-09-10T22:03:07.039179+0000","last_clean":"2024-09-10T22:03:07.039179+0000","last_became_active":"2024-09-10T22:03:05.027746+0000","last_became_peered":"2024-09-10T22:03:05.027746+0000","last_unstale":"2024-09-10T22:03:07.039179+0000","last_undegraded":"2024-09-10T22:03:07.039179+0000","last_fullsized":"2024-09-10T22:03:07.039179+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:37.798038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027631+0000","last_change":"2024-09-10T22:03:06.021989+0000","last_active":"2024-09-10T22:03:07.027631+0000","last_peered":"2024-09-10T22:03:07.027631+0000","last_clean":"2024-09-10T22:03:07.027631+0000","last_became_active":"2024-09-10T22:03:06.021242+0000","last_became_peered":"2024-09-10T22:03:06.021242+0000","last_unstale":"2024-09-10T22:03:07.027631+0000","last_undegraded":"2024-09-10T22:03:07.027631+0000","last_fullsized":"2024-09-10T22:03:07.027631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:18.808705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027149+0000","last_change":"2024-09-10T22:03:05.031225+0000","last_active":"2024-09-10T22:03:07.027149+0000","last_peered":"2024-09-10T22:03:07.027149+0000","last_clean":"2024-09-10T22:03:07.027149+0000","last_became_active":"2024-09-10T22:03:05.030047+0000","last_became_peered":"2024-09-10T22:03:05.030047+0000","last_unstale":"2024-09-10T22:03:07.027149+0000","last_undegraded":"2024-09-10T22:03:07.027149+0000","last_fullsized":"2024-09-10T22:03:07.027149+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:45.603004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.040054+0000","last_change":"2024-09-10T22:03:06.033328+0000","last_active":"2024-09-10T22:03:07.040054+0000","last_peered":"2024-09-10T22:03:07.040054+0000","last_clean":"2024-09-10T22:03:07.040054+0000","last_became_active":"2024-09-10T22:03:06.033062+0000","last_became_peered":"2024-09-10T22:03:06.033062+0000","last_unstale":"2024-09-10T22:03:07.040054+0000","last_undegraded":"2024-09-10T22:03:07.040054+0000","last_fullsized":"2024-09-10T22:03:07.040054+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:22.417821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039526+0000","last_change":"2024-09-10T22:03:05.032199+0000","last_active":"2024-09-10T22:03:07.039526+0000","last_peered":"2024-09-10T22:03:07.039526+0000","last_clean":"2024-09-10T22:03:07.039526+0000","last_became_active":"2024-09-10T22:03:05.030195+0000","last_became_peered":"2024-09-10T22:03:05.030195+0000","last_unstale":"2024-09-10T22:03:07.039526+0000","last_undegraded":"2024-09-10T22:03:07.039526+0000","last_fullsized":"2024-09-10T22:03:07.039526+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:35.474493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039420+0000","last_change":"2024-09-10T22:03:06.023976+0000","last_active":"2024-09-10T22:03:07.039420+0000","last_peered":"2024-09-10T22:03:07.039420+0000","last_clean":"2024-09-10T22:03:07.039420+0000","last_became_active":"2024-09-10T22:03:06.023720+0000","last_became_peered":"2024-09-10T22:03:06.023720+0000","last_unstale":"2024-09-10T22:03:07.039420+0000","last_undegraded":"2024-09-10T22:03:07.039420+0000","last_fullsized":"2024-09-10T22:03:07.039420+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:00.117477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039092+0000","last_change":"2024-09-10T22:03:05.026215+0000","last_active":"2024-09-10T22:03:07.039092+0000","last_peered":"2024-09-10T22:03:07.039092+0000","last_clean":"2024-09-10T22:03:07.039092+0000","last_became_active":"2024-09-10T22:03:05.025925+0000","last_became_peered":"2024-09-10T22:03:05.025925+0000","last_unstale":"2024-09-10T22:03:07.039092+0000","last_undegraded":"2024-09-10T22:03:07.039092+0000","last_fullsized":"2024-09-10T22:03:07.039092+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:08.660483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026190+0000","last_change":"2024-09-10T22:03:06.024371+0000","last_active":"2024-09-10T22:03:07.026190+0000","last_peered":"2024-09-10T22:03:07.026190+0000","last_clean":"2024-09-10T22:03:07.026190+0000","last_became_active":"2024-09-10T22:03:06.024088+0000","last_became_peered":"2024-09-10T22:03:06.024088+0000","last_unstale":"2024-09-10T22:03:07.026190+0000","last_undegraded":"2024-09-10T22:03:07.026190+0000","last_fullsized":"2024-09-10T22:03:07.026190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:46.631165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038773+0000","last_change":"2024-09-10T22:03:05.016208+0000","last_active":"2024-09-10T22:03:07.038773+0000","last_peered":"2024-09-10T22:03:07.038773+0000","last_clean":"2024-09-10T22:03:07.038773+0000","last_became_active":"2024-09-10T22:03:05.015927+0000","last_became_peered":"2024-09-10T22:03:05.015927+0000","last_unstale":"2024-09-10T22:03:07.038773+0000","last_undegraded":"2024-09-10T22:03:07.038773+0000","last_fullsized":"2024-09-10T22:03:07.038773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:22.527704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025823+0000","last_change":"2024-09-10T22:03:06.043252+0000","last_active":"2024-09-10T22:03:07.025823+0000","last_peered":"2024-09-10T22:03:07.025823+0000","last_clean":"2024-09-10T22:03:07.025823+0000","last_became_active":"2024-09-10T22:03:06.042971+0000","last_became_peered":"2024-09-10T22:03:06.042971+0000","last_unstale":"2024-09-10T22:03:07.025823+0000","last_undegraded":"2024-09-10T22:03:07.025823+0000","last_fullsized":"2024-09-10T22:03:07.025823+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:53.137403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026334+0000","last_change":"2024-09-10T22:03:05.023976+0000","last_active":"2024-09-10T22:03:07.026334+0000","last_peered":"2024-09-10T22:03:07.026334+0000","last_clean":"2024-09-10T22:03:07.026334+0000","last_became_active":"2024-09-10T22:03:05.023659+0000","last_became_peered":"2024-09-10T22:03:05.023659+0000","last_unstale":"2024-09-10T22:03:07.026334+0000","last_undegraded":"2024-09-10T22:03:07.026334+0000","last_fullsized":"2024-09-10T22:03:07.026334+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:38.227987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025690+0000","last_change":"2024-09-10T22:03:06.028567+0000","last_active":"2024-09-10T22:03:07.025690+0000","last_peered":"2024-09-10T22:03:07.025690+0000","last_clean":"2024-09-10T22:03:07.025690+0000","last_became_active":"2024-09-10T22:03:06.027918+0000","last_became_peered":"2024-09-10T22:03:06.027918+0000","last_unstale":"2024-09-10T22:03:07.025690+0000","last_undegraded":"2024-09-10T22:03:07.025690+0000","last_fullsized":"2024-09-10T22:03:07.025690+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:09.663390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039774+0000","last_change":"2024-09-10T22:03:05.031904+0000","last_active":"2024-09-10T22:03:07.039774+0000","last_peered":"2024-09-10T22:03:07.039774+0000","last_clean":"2024-09-10T22:03:07.039774+0000","last_became_active":"2024-09-10T22:03:05.031358+0000","last_became_peered":"2024-09-10T22:03:05.031358+0000","last_unstale":"2024-09-10T22:03:07.039774+0000","last_undegraded":"2024-09-10T22:03:07.039774+0000","last_fullsized":"2024-09-10T22:03:07.039774+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:36.337853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026659+0000","last_change":"2024-09-10T22:03:06.039979+0000","last_active":"2024-09-10T22:03:07.026659+0000","last_peered":"2024-09-10T22:03:07.026659+0000","last_clean":"2024-09-10T22:03:07.026659+0000","last_became_active":"2024-09-10T22:03:06.038933+0000","last_became_peered":"2024-09-10T22:03:06.038933+0000","last_unstale":"2024-09-10T22:03:07.026659+0000","last_undegraded":"2024-09-10T22:03:07.026659+0000","last_fullsized":"2024-09-10T22:03:07.026659+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:39.230893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025766+0000","last_change":"2024-09-10T22:03:05.035816+0000","last_active":"2024-09-10T22:03:07.025766+0000","last_peered":"2024-09-10T22:03:07.025766+0000","last_clean":"2024-09-10T22:03:07.025766+0000","last_became_active":"2024-09-10T22:03:05.035398+0000","last_became_peered":"2024-09-10T22:03:05.035398+0000","last_unstale":"2024-09-10T22:03:07.025766+0000","last_undegraded":"2024-09-10T22:03:07.025766+0000","last_fullsized":"2024-09-10T22:03:07.025766+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:24.588960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026818+0000","last_change":"2024-09-10T22:03:06.031664+0000","last_active":"2024-09-10T22:03:07.026818+0000","last_peered":"2024-09-10T22:03:07.026818+0000","last_clean":"2024-09-10T22:03:07.026818+0000","last_became_active":"2024-09-10T22:03:06.030950+0000","last_became_peered":"2024-09-10T22:03:06.030950+0000","last_unstale":"2024-09-10T22:03:07.026818+0000","last_undegraded":"2024-09-10T22:03:07.026818+0000","last_fullsized":"2024-09-10T22:03:07.026818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:37.643981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026812+0000","last_change":"2024-09-10T22:03:05.024528+0000","last_active":"2024-09-10T22:03:07.026812+0000","last_peered":"2024-09-10T22:03:07.026812+0000","last_clean":"2024-09-10T22:03:07.026812+0000","last_became_active":"2024-09-10T22:03:05.024254+0000","last_became_peered":"2024-09-10T22:03:05.024254+0000","last_unstale":"2024-09-10T22:03:07.026812+0000","last_undegraded":"2024-09-10T22:03:07.026812+0000","last_fullsized":"2024-09-10T22:03:07.026812+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:40.890890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039632+0000","last_change":"2024-09-10T22:03:06.039734+0000","last_active":"2024-09-10T22:03:07.039632+0000","last_peered":"2024-09-10T22:03:07.039632+0000","last_clean":"2024-09-10T22:03:07.039632+0000","last_became_active":"2024-09-10T22:03:06.039429+0000","last_became_peered":"2024-09-10T22:03:06.039429+0000","last_unstale":"2024-09-10T22:03:07.039632+0000","last_undegraded":"2024-09-10T22:03:07.039632+0000","last_fullsized":"2024-09-10T22:03:07.039632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:03.034797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026882+0000","last_change":"2024-09-10T22:03:05.029224+0000","last_active":"2024-09-10T22:03:07.026882+0000","last_peered":"2024-09-10T22:03:07.026882+0000","last_clean":"2024-09-10T22:03:07.026882+0000","last_became_active":"2024-09-10T22:03:05.029043+0000","last_became_peered":"2024-09-10T22:03:05.029043+0000","last_unstale":"2024-09-10T22:03:07.026882+0000","last_undegraded":"2024-09-10T22:03:07.026882+0000","last_fullsized":"2024-09-10T22:03:07.026882+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:32.578778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039496+0000","last_change":"2024-09-10T22:03:06.031415+0000","last_active":"2024-09-10T22:03:07.039496+0000","last_peered":"2024-09-10T22:03:07.039496+0000","last_clean":"2024-09-10T22:03:07.039496+0000","last_became_active":"2024-09-10T22:03:06.031049+0000","last_became_peered":"2024-09-10T22:03:06.031049+0000","last_unstale":"2024-09-10T22:03:07.039496+0000","last_undegraded":"2024-09-10T22:03:07.039496+0000","last_fullsized":"2024-09-10T22:03:07.039496+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:14.013068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039727+0000","last_change":"2024-09-10T22:03:05.024883+0000","last_active":"2024-09-10T22:03:07.039727+0000","last_peered":"2024-09-10T22:03:07.039727+0000","last_clean":"2024-09-10T22:03:07.039727+0000","last_became_active":"2024-09-10T22:03:05.024620+0000","last_became_peered":"2024-09-10T22:03:05.024620+0000","last_unstale":"2024-09-10T22:03:07.039727+0000","last_undegraded":"2024-09-10T22:03:07.039727+0000","last_fullsized":"2024-09-10T22:03:07.039727+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:46.053668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039847+0000","last_change":"2024-09-10T22:03:06.038013+0000","last_active":"2024-09-10T22:03:07.039847+0000","last_peered":"2024-09-10T22:03:07.039847+0000","last_clean":"2024-09-10T22:03:07.039847+0000","last_became_active":"2024-09-10T22:03:06.037742+0000","last_became_peered":"2024-09-10T22:03:06.037742+0000","last_unstale":"2024-09-10T22:03:07.039847+0000","last_undegraded":"2024-09-10T22:03:07.039847+0000","last_fullsized":"2024-09-10T22:03:07.039847+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:20.984037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.027787+0000","last_change":"2024-09-10T22:03:05.025047+0000","last_active":"2024-09-10T22:03:06.027787+0000","last_peered":"2024-09-10T22:03:06.027787+0000","last_clean":"2024-09-10T22:03:06.027787+0000","last_became_active":"2024-09-10T22:03:05.024736+0000","last_became_peered":"2024-09-10T22:03:05.024736+0000","last_unstale":"2024-09-10T22:03:06.027787+0000","last_undegraded":"2024-09-10T22:03:06.027787+0000","last_fullsized":"2024-09-10T22:03:06.027787+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:40.706358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026890+0000","last_change":"2024-09-10T22:03:06.030932+0000","last_active":"2024-09-10T22:03:07.026890+0000","last_peered":"2024-09-10T22:03:07.026890+0000","last_clean":"2024-09-10T22:03:07.026890+0000","last_became_active":"2024-09-10T22:03:06.030509+0000","last_became_peered":"2024-09-10T22:03:06.030509+0000","last_unstale":"2024-09-10T22:03:07.026890+0000","last_undegraded":"2024-09-10T22:03:07.026890+0000","last_fullsized":"2024-09-10T22:03:07.026890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:36.477399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038782+0000","last_change":"2024-09-10T22:03:05.033734+0000","last_active":"2024-09-10T22:03:07.038782+0000","last_peered":"2024-09-10T22:03:07.038782+0000","last_clean":"2024-09-10T22:03:07.038782+0000","last_became_active":"2024-09-10T22:03:05.033447+0000","last_became_peered":"2024-09-10T22:03:05.033447+0000","last_unstale":"2024-09-10T22:03:07.038782+0000","last_undegraded":"2024-09-10T22:03:07.038782+0000","last_fullsized":"2024-09-10T22:03:07.038782+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:17.458762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025965+0000","last_change":"2024-09-10T22:03:06.036466+0000","last_active":"2024-09-10T22:03:07.025965+0000","last_peered":"2024-09-10T22:03:07.025965+0000","last_clean":"2024-09-10T22:03:07.025965+0000","last_became_active":"2024-09-10T22:03:06.036181+0000","last_became_peered":"2024-09-10T22:03:06.036181+0000","last_unstale":"2024-09-10T22:03:07.025965+0000","last_undegraded":"2024-09-10T22:03:07.025965+0000","last_fullsized":"2024-09-10T22:03:07.025965+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:23.530611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039062+0000","last_change":"2024-09-10T22:03:05.025872+0000","last_active":"2024-09-10T22:03:07.039062+0000","last_peered":"2024-09-10T22:03:07.039062+0000","last_clean":"2024-09-10T22:03:07.039062+0000","last_became_active":"2024-09-10T22:03:05.025562+0000","last_became_peered":"2024-09-10T22:03:05.025562+0000","last_unstale":"2024-09-10T22:03:07.039062+0000","last_undegraded":"2024-09-10T22:03:07.039062+0000","last_fullsized":"2024-09-10T22:03:07.039062+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:45.628259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038174+0000","last_change":"2024-09-10T22:03:06.033388+0000","last_active":"2024-09-10T22:03:07.038174+0000","last_peered":"2024-09-10T22:03:07.038174+0000","last_clean":"2024-09-10T22:03:07.038174+0000","last_became_active":"2024-09-10T22:03:06.031209+0000","last_became_peered":"2024-09-10T22:03:06.031209+0000","last_unstale":"2024-09-10T22:03:07.038174+0000","last_undegraded":"2024-09-10T22:03:07.038174+0000","last_fullsized":"2024-09-10T22:03:07.038174+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:06.631843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038419+0000","last_change":"2024-09-10T22:03:05.033647+0000","last_active":"2024-09-10T22:03:07.038419+0000","last_peered":"2024-09-10T22:03:07.038419+0000","last_clean":"2024-09-10T22:03:07.038419+0000","last_became_active":"2024-09-10T22:03:05.033309+0000","last_became_peered":"2024-09-10T22:03:05.033309+0000","last_unstale":"2024-09-10T22:03:07.038419+0000","last_undegraded":"2024-09-10T22:03:07.038419+0000","last_fullsized":"2024-09-10T22:03:07.038419+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:36.641074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039106+0000","last_change":"2024-09-10T22:03:06.040834+0000","last_active":"2024-09-10T22:03:07.039106+0000","last_peered":"2024-09-10T22:03:07.039106+0000","last_clean":"2024-09-10T22:03:07.039106+0000","last_became_active":"2024-09-10T22:03:06.040563+0000","last_became_peered":"2024-09-10T22:03:06.040563+0000","last_unstale":"2024-09-10T22:03:07.039106+0000","last_undegraded":"2024-09-10T22:03:07.039106+0000","last_fullsized":"2024-09-10T22:03:07.039106+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:32.805553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":23,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025807+0000","last_change":"2024-09-10T22:03:05.032004+0000","last_active":"2024-09-10T22:03:07.025807+0000","last_peered":"2024-09-10T22:03:07.025807+0000","last_clean":"2024-09-10T22:03:07.025807+0000","last_became_active":"2024-09-10T22:03:05.031714+0000","last_became_peered":"2024-09-10T22:03:05.031714+0000","last_unstale":"2024-09-10T22:03:07.025807+0000","last_undegraded":"2024-09-10T22:03:07.025807+0000","last_fullsized":"2024-09-10T22:03:07.025807+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:09.176212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.036084+0000","last_change":"2024-09-10T22:03:06.026449+0000","last_active":"2024-09-10T22:03:06.036084+0000","last_peered":"2024-09-10T22:03:06.036084+0000","last_clean":"2024-09-10T22:03:06.036084+0000","last_became_active":"2024-09-10T22:03:06.026192+0000","last_became_peered":"2024-09-10T22:03:06.026192+0000","last_unstale":"2024-09-10T22:03:06.036084+0000","last_undegraded":"2024-09-10T22:03:06.036084+0000","last_fullsized":"2024-09-10T22:03:06.036084+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:23.133689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.029132+0000","last_change":"2024-09-10T22:03:05.037720+0000","last_active":"2024-09-10T22:03:06.029132+0000","last_peered":"2024-09-10T22:03:06.029132+0000","last_clean":"2024-09-10T22:03:06.029132+0000","last_became_active":"2024-09-10T22:03:05.037347+0000","last_became_peered":"2024-09-10T22:03:05.037347+0000","last_unstale":"2024-09-10T22:03:06.029132+0000","last_undegraded":"2024-09-10T22:03:06.029132+0000","last_fullsized":"2024-09-10T22:03:06.029132+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:40.664783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026957+0000","last_change":"2024-09-10T22:03:06.034835+0000","last_active":"2024-09-10T22:03:07.026957+0000","last_peered":"2024-09-10T22:03:07.026957+0000","last_clean":"2024-09-10T22:03:07.026957+0000","last_became_active":"2024-09-10T22:03:06.034530+0000","last_became_peered":"2024-09-10T22:03:06.034530+0000","last_unstale":"2024-09-10T22:03:07.026957+0000","last_undegraded":"2024-09-10T22:03:07.026957+0000","last_fullsized":"2024-09-10T22:03:07.026957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.353762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027003+0000","last_change":"2024-09-10T22:03:05.031526+0000","last_active":"2024-09-10T22:03:07.027003+0000","last_peered":"2024-09-10T22:03:07.027003+0000","last_clean":"2024-09-10T22:03:07.027003+0000","last_became_active":"2024-09-10T22:03:05.030940+0000","last_became_peered":"2024-09-10T22:03:05.030940+0000","last_unstale":"2024-09-10T22:03:07.027003+0000","last_undegraded":"2024-09-10T22:03:07.027003+0000","last_fullsized":"2024-09-10T22:03:07.027003+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:49.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027209+0000","last_change":"2024-09-10T22:03:06.032416+0000","last_active":"2024-09-10T22:03:07.027209+0000","last_peered":"2024-09-10T22:03:07.027209+0000","last_clean":"2024-09-10T22:03:07.027209+0000","last_became_active":"2024-09-10T22:03:06.032138+0000","last_became_peered":"2024-09-10T22:03:06.032138+0000","last_unstale":"2024-09-10T22:03:07.027209+0000","last_undegraded":"2024-09-10T22:03:07.027209+0000","last_fullsized":"2024-09-10T22:03:07.027209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:30:42.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039692+0000","last_change":"2024-09-10T22:03:05.033438+0000","last_active":"2024-09-10T22:03:07.039692+0000","last_peered":"2024-09-10T22:03:07.039692+0000","last_clean":"2024-09-10T22:03:07.039692+0000","last_became_active":"2024-09-10T22:03:05.033163+0000","last_became_peered":"2024-09-10T22:03:05.033163+0000","last_unstale":"2024-09-10T22:03:07.039692+0000","last_undegraded":"2024-09-10T22:03:07.039692+0000","last_fullsized":"2024-09-10T22:03:07.039692+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:46.053668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027428+0000","last_change":"2024-09-10T22:03:06.028611+0000","last_active":"2024-09-10T22:03:07.027428+0000","last_peered":"2024-09-10T22:03:07.027428+0000","last_clean":"2024-09-10T22:03:07.027428+0000","last_became_active":"2024-09-10T22:03:06.027653+0000","last_became_peered":"2024-09-10T22:03:06.027653+0000","last_unstale":"2024-09-10T22:03:07.027428+0000","last_undegraded":"2024-09-10T22:03:07.027428+0000","last_fullsized":"2024-09-10T22:03:07.027428+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:09.170787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039596+0000","last_change":"2024-09-10T22:03:05.019638+0000","last_active":"2024-09-10T22:03:07.039596+0000","last_peered":"2024-09-10T22:03:07.039596+0000","last_clean":"2024-09-10T22:03:07.039596+0000","last_became_active":"2024-09-10T22:03:05.019366+0000","last_became_peered":"2024-09-10T22:03:05.019366+0000","last_unstale":"2024-09-10T22:03:07.039596+0000","last_undegraded":"2024-09-10T22:03:07.039596+0000","last_fullsized":"2024-09-10T22:03:07.039596+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:45.792426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038796+0000","last_change":"2024-09-10T22:03:06.030808+0000","last_active":"2024-09-10T22:03:07.038796+0000","last_peered":"2024-09-10T22:03:07.038796+0000","last_clean":"2024-09-10T22:03:07.038796+0000","last_became_active":"2024-09-10T22:03:06.030375+0000","last_became_peered":"2024-09-10T22:03:06.030375+0000","last_unstale":"2024-09-10T22:03:07.038796+0000","last_undegraded":"2024-09-10T22:03:07.038796+0000","last_fullsized":"2024-09-10T22:03:07.038796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:55.553544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038952+0000","last_change":"2024-09-10T22:03:05.014844+0000","last_active":"2024-09-10T22:03:07.038952+0000","last_peered":"2024-09-10T22:03:07.038952+0000","last_clean":"2024-09-10T22:03:07.038952+0000","last_became_active":"2024-09-10T22:03:05.014520+0000","last_became_peered":"2024-09-10T22:03:05.014520+0000","last_unstale":"2024-09-10T22:03:07.038952+0000","last_undegraded":"2024-09-10T22:03:07.038952+0000","last_fullsized":"2024-09-10T22:03:07.038952+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:21.628678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038966+0000","last_change":"2024-09-10T22:03:06.028489+0000","last_active":"2024-09-10T22:03:07.038966+0000","last_peered":"2024-09-10T22:03:07.038966+0000","last_clean":"2024-09-10T22:03:07.038966+0000","last_became_active":"2024-09-10T22:03:06.028229+0000","last_became_peered":"2024-09-10T22:03:06.028229+0000","last_unstale":"2024-09-10T22:03:07.038966+0000","last_undegraded":"2024-09-10T22:03:07.038966+0000","last_fullsized":"2024-09-10T22:03:07.038966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:08.682948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039394+0000","last_change":"2024-09-10T22:03:05.028405+0000","last_active":"2024-09-10T22:03:07.039394+0000","last_peered":"2024-09-10T22:03:07.039394+0000","last_clean":"2024-09-10T22:03:07.039394+0000","last_became_active":"2024-09-10T22:03:05.028024+0000","last_became_peered":"2024-09-10T22:03:05.028024+0000","last_unstale":"2024-09-10T22:03:07.039394+0000","last_undegraded":"2024-09-10T22:03:07.039394+0000","last_fullsized":"2024-09-10T22:03:07.039394+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:45.628259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T22:03:06.042157+0000","last_change":"2024-09-10T22:03:06.042102+0000","last_active":"2024-09-10T22:03:06.042157+0000","last_peered":"2024-09-10T22:03:06.042157+0000","last_clean":"2024-09-10T22:03:06.042157+0000","last_became_active":"2024-09-10T22:03:06.041909+0000","last_became_peered":"2024-09-10T22:03:06.041909+0000","last_unstale":"2024-09-10T22:03:06.042157+0000","last_undegraded":"2024-09-10T22:03:06.042157+0000","last_fullsized":"2024-09-10T22:03:06.042157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:24.358679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025938+0000","last_change":"2024-09-10T22:03:05.034465+0000","last_active":"2024-09-10T22:03:07.025938+0000","last_peered":"2024-09-10T22:03:07.025938+0000","last_clean":"2024-09-10T22:03:07.025938+0000","last_became_active":"2024-09-10T22:03:05.034213+0000","last_became_peered":"2024-09-10T22:03:05.034213+0000","last_unstale":"2024-09-10T22:03:07.025938+0000","last_undegraded":"2024-09-10T22:03:07.025938+0000","last_fullsized":"2024-09-10T22:03:07.025938+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:19.448220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025757+0000","last_change":"2024-09-10T22:03:06.025288+0000","last_active":"2024-09-10T22:03:07.025757+0000","last_peered":"2024-09-10T22:03:07.025757+0000","last_clean":"2024-09-10T22:03:07.025757+0000","last_became_active":"2024-09-10T22:03:06.024908+0000","last_became_peered":"2024-09-10T22:03:06.024908+0000","last_unstale":"2024-09-10T22:03:07.025757+0000","last_undegraded":"2024-09-10T22:03:07.025757+0000","last_fullsized":"2024-09-10T22:03:07.025757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:04.451178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038941+0000","last_change":"2024-09-10T22:03:05.025906+0000","last_active":"2024-09-10T22:03:07.038941+0000","last_peered":"2024-09-10T22:03:07.038941+0000","last_clean":"2024-09-10T22:03:07.038941+0000","last_became_active":"2024-09-10T22:03:05.025637+0000","last_became_peered":"2024-09-10T22:03:05.025637+0000","last_unstale":"2024-09-10T22:03:07.038941+0000","last_undegraded":"2024-09-10T22:03:07.038941+0000","last_fullsized":"2024-09-10T22:03:07.038941+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:16.620483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027626+0000","last_change":"2024-09-10T22:03:06.030444+0000","last_active":"2024-09-10T22:03:07.027626+0000","last_peered":"2024-09-10T22:03:07.027626+0000","last_clean":"2024-09-10T22:03:07.027626+0000","last_became_active":"2024-09-10T22:03:06.030015+0000","last_became_peered":"2024-09-10T22:03:06.030015+0000","last_unstale":"2024-09-10T22:03:07.027626+0000","last_undegraded":"2024-09-10T22:03:07.027626+0000","last_fullsized":"2024-09-10T22:03:07.027626+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:22.607616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.048554+0000","last_change":"2024-09-10T22:03:05.028365+0000","last_active":"2024-09-10T22:03:07.048554+0000","last_peered":"2024-09-10T22:03:07.048554+0000","last_clean":"2024-09-10T22:03:07.048554+0000","last_became_active":"2024-09-10T22:03:05.028085+0000","last_became_peered":"2024-09-10T22:03:05.028085+0000","last_unstale":"2024-09-10T22:03:07.048554+0000","last_undegraded":"2024-09-10T22:03:07.048554+0000","last_fullsized":"2024-09-10T22:03:07.048554+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:06.009142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026452+0000","last_change":"2024-09-10T22:03:06.026047+0000","last_active":"2024-09-10T22:03:07.026452+0000","last_peered":"2024-09-10T22:03:07.026452+0000","last_clean":"2024-09-10T22:03:07.026452+0000","last_became_active":"2024-09-10T22:03:06.025760+0000","last_became_peered":"2024-09-10T22:03:06.025760+0000","last_unstale":"2024-09-10T22:03:07.026452+0000","last_undegraded":"2024-09-10T22:03:07.026452+0000","last_fullsized":"2024-09-10T22:03:07.026452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:07.012049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039836+0000","last_change":"2024-09-10T22:03:05.024587+0000","last_active":"2024-09-10T22:03:07.039836+0000","last_peered":"2024-09-10T22:03:07.039836+0000","last_clean":"2024-09-10T22:03:07.039836+0000","last_became_active":"2024-09-10T22:03:05.024301+0000","last_became_peered":"2024-09-10T22:03:05.024301+0000","last_unstale":"2024-09-10T22:03:07.039836+0000","last_undegraded":"2024-09-10T22:03:07.039836+0000","last_fullsized":"2024-09-10T22:03:07.039836+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:03.448272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039390+0000","last_change":"2024-09-10T22:03:06.037303+0000","last_active":"2024-09-10T22:03:07.039390+0000","last_peered":"2024-09-10T22:03:07.039390+0000","last_clean":"2024-09-10T22:03:07.039390+0000","last_became_active":"2024-09-10T22:03:06.035954+0000","last_became_peered":"2024-09-10T22:03:06.035954+0000","last_unstale":"2024-09-10T22:03:07.039390+0000","last_undegraded":"2024-09-10T22:03:07.039390+0000","last_fullsized":"2024-09-10T22:03:07.039390+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:28.781866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038879+0000","last_change":"2024-09-10T22:03:05.030431+0000","last_active":"2024-09-10T22:03:07.038879+0000","last_peered":"2024-09-10T22:03:07.038879+0000","last_clean":"2024-09-10T22:03:07.038879+0000","last_became_active":"2024-09-10T22:03:05.030078+0000","last_became_peered":"2024-09-10T22:03:05.030078+0000","last_unstale":"2024-09-10T22:03:07.038879+0000","last_undegraded":"2024-09-10T22:03:07.038879+0000","last_fullsized":"2024-09-10T22:03:07.038879+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:38.227987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039004+0000","last_change":"2024-09-10T22:03:06.038238+0000","last_active":"2024-09-10T22:03:07.039004+0000","last_peered":"2024-09-10T22:03:07.039004+0000","last_clean":"2024-09-10T22:03:07.039004+0000","last_became_active":"2024-09-10T22:03:06.037913+0000","last_became_peered":"2024-09-10T22:03:06.037913+0000","last_unstale":"2024-09-10T22:03:07.039004+0000","last_undegraded":"2024-09-10T22:03:07.039004+0000","last_fullsized":"2024-09-10T22:03:07.039004+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:55.234307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039325+0000","last_change":"2024-09-10T22:03:05.023416+0000","last_active":"2024-09-10T22:03:07.039325+0000","last_peered":"2024-09-10T22:03:07.039325+0000","last_clean":"2024-09-10T22:03:07.039325+0000","last_became_active":"2024-09-10T22:03:05.023156+0000","last_became_peered":"2024-09-10T22:03:05.023156+0000","last_unstale":"2024-09-10T22:03:07.039325+0000","last_undegraded":"2024-09-10T22:03:07.039325+0000","last_fullsized":"2024-09-10T22:03:07.039325+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:36.337853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038655+0000","last_change":"2024-09-10T22:03:06.041670+0000","last_active":"2024-09-10T22:03:07.038655+0000","last_peered":"2024-09-10T22:03:07.038655+0000","last_clean":"2024-09-10T22:03:07.038655+0000","last_became_active":"2024-09-10T22:03:06.041375+0000","last_became_peered":"2024-09-10T22:03:06.041375+0000","last_unstale":"2024-09-10T22:03:07.038655+0000","last_undegraded":"2024-09-10T22:03:07.038655+0000","last_fullsized":"2024-09-10T22:03:07.038655+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:39.734532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026452+0000","last_change":"2024-09-10T22:03:05.022463+0000","last_active":"2024-09-10T22:03:07.026452+0000","last_peered":"2024-09-10T22:03:07.026452+0000","last_clean":"2024-09-10T22:03:07.026452+0000","last_became_active":"2024-09-10T22:03:05.022074+0000","last_became_peered":"2024-09-10T22:03:05.022074+0000","last_unstale":"2024-09-10T22:03:07.026452+0000","last_undegraded":"2024-09-10T22:03:07.026452+0000","last_fullsized":"2024-09-10T22:03:07.026452+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:35.474493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039118+0000","last_change":"2024-09-10T22:03:06.041241+0000","last_active":"2024-09-10T22:03:07.039118+0000","last_peered":"2024-09-10T22:03:07.039118+0000","last_clean":"2024-09-10T22:03:07.039118+0000","last_became_active":"2024-09-10T22:03:06.040931+0000","last_became_peered":"2024-09-10T22:03:06.040931+0000","last_unstale":"2024-09-10T22:03:07.039118+0000","last_undegraded":"2024-09-10T22:03:07.039118+0000","last_fullsized":"2024-09-10T22:03:07.039118+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:50.477410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038564+0000","last_change":"2024-09-10T22:03:05.030279+0000","last_active":"2024-09-10T22:03:07.038564+0000","last_peered":"2024-09-10T22:03:07.038564+0000","last_clean":"2024-09-10T22:03:07.038564+0000","last_became_active":"2024-09-10T22:03:05.029962+0000","last_became_peered":"2024-09-10T22:03:05.029962+0000","last_unstale":"2024-09-10T22:03:07.038564+0000","last_undegraded":"2024-09-10T22:03:07.038564+0000","last_fullsized":"2024-09-10T22:03:07.038564+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:40.890890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.027062+0000","last_change":"2024-09-10T22:03:06.034821+0000","last_active":"2024-09-10T22:03:07.027062+0000","last_peered":"2024-09-10T22:03:07.027062+0000","last_clean":"2024-09-10T22:03:07.027062+0000","last_became_active":"2024-09-10T22:03:06.032302+0000","last_became_peered":"2024-09-10T22:03:06.032302+0000","last_unstale":"2024-09-10T22:03:07.027062+0000","last_undegraded":"2024-09-10T22:03:07.027062+0000","last_fullsized":"2024-09-10T22:03:07.027062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:23.530611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.025862+0000","last_change":"2024-09-10T22:03:05.038660+0000","last_active":"2024-09-10T22:03:07.025862+0000","last_peered":"2024-09-10T22:03:07.025862+0000","last_clean":"2024-09-10T22:03:07.025862+0000","last_became_active":"2024-09-10T22:03:05.038474+0000","last_became_peered":"2024-09-10T22:03:05.038474+0000","last_unstale":"2024-09-10T22:03:07.025862+0000","last_undegraded":"2024-09-10T22:03:07.025862+0000","last_fullsized":"2024-09-10T22:03:07.025862+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:46.706749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.026783+0000","last_change":"2024-09-10T22:03:06.034875+0000","last_active":"2024-09-10T22:03:07.026783+0000","last_peered":"2024-09-10T22:03:07.026783+0000","last_clean":"2024-09-10T22:03:07.026783+0000","last_became_active":"2024-09-10T22:03:06.034489+0000","last_became_peered":"2024-09-10T22:03:06.034489+0000","last_unstale":"2024-09-10T22:03:07.026783+0000","last_undegraded":"2024-09-10T22:03:07.026783+0000","last_fullsized":"2024-09-10T22:03:07.026783+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:05.000871+0000","last_clean_scrub_stamp":"2024-09-10T22:03:05.000871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:50.412298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.039846+0000","last_change":"2024-09-10T22:03:05.038671+0000","last_active":"2024-09-10T22:03:07.039846+0000","last_peered":"2024-09-10T22:03:07.039846+0000","last_clean":"2024-09-10T22:03:07.039846+0000","last_became_active":"2024-09-10T22:03:05.038365+0000","last_became_peered":"2024-09-10T22:03:05.038365+0000","last_unstale":"2024-09-10T22:03:07.039846+0000","last_undegraded":"2024-09-10T22:03:07.039846+0000","last_fullsized":"2024-09-10T22:03:07.039846+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:03:03.997964+0000","last_clean_scrub_stamp":"2024-09-10T22:03:03.997964+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:08.660483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":152,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T22:03:07.038791+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:03:07.038791+0000","last_peered":"2024-09-10T22:03:07.038791+0000","last_clean":"2024-09-10T22:03:07.038791+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:03:07.038791+0000","last_undegraded":"2024-09-10T22:03:07.038791+0000","last_fullsized":"2024-09-10T22:03:07.038791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":1093632,"data_stored":786826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509477,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":197040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35836,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336004,"statfs":{"total":96636764160,"available":96600068096,"internally_reserved":0,"allocated":499712,"data_stored":196438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509477,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35844,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335996,"statfs":{"total":96636764160,"available":96600059904,"internally_reserved":0,"allocated":507904,"data_stored":197980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":507904,"data_stored":197006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35852,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335988,"statfs":{"total":96636764160,"available":96600051712,"internally_reserved":0,"allocated":512000,"data_stored":202738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1093632,"data_stored":786770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":483328,"data_stored":189593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:03:10.581 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:03:10.581 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T22:03:10.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.773+0000 7fa554800640 1 -- 172.21.15.66:0/582679557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa54c071700 msgr2=0x7fa54c0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:10.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.773+0000 7fa554800640 1 --2- 172.21.15.66:0/582679557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa54c071700 0x7fa54c0b0410 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa54c073480 tx=0x7fa53c01c6e0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.773+0000 7fa554800640 1 -- 172.21.15.66:0/582679557 shutdown_connections 2024-09-10T22:03:10.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.773+0000 7fa554800640 1 --2- 172.21.15.66:0/582679557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa54c071700 0x7fa54c0b0410 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.773+0000 7fa554800640 1 --2- 172.21.15.66:0/582679557 >> v2:172.21.15.94:3300/0 conn(0x7fa54c070ac0 0x7fa54c070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.773+0000 7fa554800640 1 -- 172.21.15.66:0/582679557 >> 172.21.15.66:0/582679557 conn(0x7fa54c0d1d00 msgr2=0x7fa54c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:10.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.773+0000 7fa554800640 1 -- 172.21.15.66:0/582679557 shutdown_connections 2024-09-10T22:03:10.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.773+0000 7fa554800640 1 -- 172.21.15.66:0/582679557 wait complete. 2024-09-10T22:03:10.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.773+0000 7fa554800640 1 Processor -- start 2024-09-10T22:03:10.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.773+0000 7fa554800640 1 -- start start 2024-09-10T22:03:10.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa554800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa54c070ac0 0x7fa54c14ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:10.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa554800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa54c071700 0x7fa54c14e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:10.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa554800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa54c0b0ef0 0x7fa54c15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:10.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa554800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c160b50 con 0x7fa54c070ac0 2024-09-10T22:03:10.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa554800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1609d0 con 0x7fa54c0b0ef0 2024-09-10T22:03:10.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa554800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa54c160cd0 con 0x7fa54c071700 2024-09-10T22:03:10.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa551e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa54c070ac0 0x7fa54c14ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:10.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa551e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa54c070ac0 0x7fa54c14ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43460/0 (socket says 172.21.15.66:43460) 2024-09-10T22:03:10.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa551e00640 1 -- 172.21.15.66:0/148958914 learned_addr learned my addr 172.21.15.66:0/148958914 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:10.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa551400640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa54c071700 0x7fa54c14e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:10.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa552800640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa54c0b0ef0 0x7fa54c15c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:10.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa551e00640 1 -- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa54c071700 msgr2=0x7fa54c14e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:10.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa551e00640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa54c071700 0x7fa54c14e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa551e00640 1 -- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa54c0b0ef0 msgr2=0x7fa54c15c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:10.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa551e00640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa54c0b0ef0 0x7fa54c15c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:10.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa551e00640 1 -- 172.21.15.66:0/148958914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54c14ff50 con 0x7fa54c070ac0 2024-09-10T22:03:10.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa551e00640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa54c070ac0 0x7fa54c14ec60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa54000e9f0 tx=0x7fa54000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:10.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa546000640 1 -- 172.21.15.66:0/148958914 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54000cd80 con 0x7fa54c070ac0 2024-09-10T22:03:10.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa546000640 1 -- 172.21.15.66:0/148958914 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa540010430 con 0x7fa54c070ac0 2024-09-10T22:03:10.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa554800640 1 -- 172.21.15.66:0/148958914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54c14ea20 con 0x7fa54c070ac0 2024-09-10T22:03:10.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa546000640 1 -- 172.21.15.66:0/148958914 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540010860 con 0x7fa54c070ac0 2024-09-10T22:03:10.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa554800640 1 -- 172.21.15.66:0/148958914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54c14f6b0 con 0x7fa54c070ac0 2024-09-10T22:03:10.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.777+0000 7fa554800640 1 -- 172.21.15.66:0/148958914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa54c070ea0 con 0x7fa54c070ac0 2024-09-10T22:03:10.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.781+0000 7fa546000640 1 -- 172.21.15.66:0/148958914 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fa540010a00 con 0x7fa54c070ac0 2024-09-10T22:03:10.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.781+0000 7fa546000640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa530072940 0x7fa530074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:10.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.781+0000 7fa546000640 1 -- 172.21.15.66:0/148958914 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fa540094410 con 0x7fa54c070ac0 2024-09-10T22:03:10.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.781+0000 7fa551400640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa530072940 0x7fa530074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:10.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.781+0000 7fa551400640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa530072940 0x7fa530074e00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0af000 tx=0x7fa53c01af20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:10.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:10.785+0000 7fa546000640 1 -- 172.21.15.66:0/148958914 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa54005f090 con 0x7fa54c070ac0 2024-09-10T22:03:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.021+0000 7fa554800640 1 -- 172.21.15.66:0/148958914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa54c15c5a0 con 0x7fa54c070ac0 2024-09-10T22:03:11.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.025+0000 7fa546000640 1 -- 172.21.15.66:0/148958914 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) ==== 190+0+147 (secure 0 0 0) 0x7fa540063ac0 con 0x7fa54c070ac0 2024-09-10T22:03:11.026 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T22:03:11.027 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw== 2024-09-10T22:03:11.027 INFO:teuthology.orchestra.run.smithi066.stdout: caps mds = "allow" 2024-09-10T22:03:11.027 INFO:teuthology.orchestra.run.smithi066.stdout: caps mon = "allow r" 2024-09-10T22:03:11.027 INFO:teuthology.orchestra.run.smithi066.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T22:03:11.027 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-09-10T22:03:11.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.025+0000 7fa554800640 1 -- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa530072940 msgr2=0x7fa530074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:11.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.025+0000 7fa554800640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa530072940 0x7fa530074e00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0af000 tx=0x7fa53c01af20 comp rx=0 tx=0).stop 2024-09-10T22:03:11.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.025+0000 7fa554800640 1 -- 172.21.15.66:0/148958914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa54c070ac0 msgr2=0x7fa54c14ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:11.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.025+0000 7fa554800640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa54c070ac0 0x7fa54c14ec60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa54000e9f0 tx=0x7fa54000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.025+0000 7fa554800640 1 -- 172.21.15.66:0/148958914 shutdown_connections 2024-09-10T22:03:11.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.025+0000 7fa554800640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa530072940 0x7fa530074e00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.025+0000 7fa554800640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa54c0b0ef0 0x7fa54c15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.025+0000 7fa554800640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa54c071700 0x7fa54c14e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.025+0000 7fa554800640 1 --2- 172.21.15.66:0/148958914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa54c070ac0 0x7fa54c14ec60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.025+0000 7fa554800640 1 -- 172.21.15.66:0/148958914 >> 172.21.15.66:0/148958914 conn(0x7fa54c0d1d00 msgr2=0x7fa54c0b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:11.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.029+0000 7fa554800640 1 -- 172.21.15.66:0/148958914 shutdown_connections 2024-09-10T22:03:11.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.029+0000 7fa554800640 1 -- 172.21.15.66:0/148958914 wait complete. 2024-09-10T22:03:11.049 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:03:11.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.193+0000 7f8663000640 1 -- 172.21.15.66:0/1790414222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8640003a40 msgr2=0x7f8640003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:11.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.193+0000 7f8663000640 1 --2- 172.21.15.66:0/1790414222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8640003a40 0x7f8640003e60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f864c005fd0 tx=0x7f864c01cd60 comp rx=0 tx=0).stop 2024-09-10T22:03:11.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.193+0000 7f8661000640 1 -- 172.21.15.66:0/1790414222 reap_dead start 2024-09-10T22:03:11.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.193+0000 7f8663000640 1 -- 172.21.15.66:0/1790414222 shutdown_connections 2024-09-10T22:03:11.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.193+0000 7f8663000640 1 -- 172.21.15.66:0/1790414222 >> 172.21.15.66:0/1790414222 conn(0x7f865c0d1c60 msgr2=0x7f865c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:11.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.193+0000 7f8663000640 1 -- 172.21.15.66:0/1790414222 shutdown_connections 2024-09-10T22:03:11.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.193+0000 7f8663000640 1 -- 172.21.15.66:0/1790414222 wait complete. 2024-09-10T22:03:11.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8663000640 1 Processor -- start 2024-09-10T22:03:11.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8663000640 1 -- start start 2024-09-10T22:03:11.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8663000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8640003a40 0x7f865c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:11.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8663000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0b67e0 0x7f865c0b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:11.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8663000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f865c06f1a0 0x7f865c06f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:11.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8663000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f865c167ed0 con 0x7f865c06f1a0 2024-09-10T22:03:11.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8663000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f865c167d50 con 0x7f865c0b67e0 2024-09-10T22:03:11.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8663000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f865c168050 con 0x7f8640003a40 2024-09-10T22:03:11.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8661000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f865c06f1a0 0x7f865c06f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:11.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8661000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f865c06f1a0 0x7f865c06f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43492/0 (socket says 172.21.15.66:43492) 2024-09-10T22:03:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8661000640 1 -- 172.21.15.66:0/3362248811 learned_addr learned my addr 172.21.15.66:0/3362248811 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f865be00640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8640003a40 0x7f865c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8653e00640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0b67e0 0x7f865c0b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8661000640 1 -- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8640003a40 msgr2=0x7f865c156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8661000640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8640003a40 0x7f865c156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8661000640 1 -- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0b67e0 msgr2=0x7f865c0b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8661000640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0b67e0 0x7f865c0b6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8661000640 1 -- 172.21.15.66:0/3362248811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c06c8b0 con 0x7f865c06f1a0 2024-09-10T22:03:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8661000640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f865c06f1a0 0x7f865c06f600 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f865c073840 tx=0x7f865400eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8658c00640 1 -- 172.21.15.66:0/3362248811 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865400c9b0 con 0x7f865c06f1a0 2024-09-10T22:03:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8658c00640 1 -- 172.21.15.66:0/3362248811 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8654018070 con 0x7f865c06f1a0 2024-09-10T22:03:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8663000640 1 -- 172.21.15.66:0/3362248811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c1f8310 con 0x7f865c06f1a0 2024-09-10T22:03:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8658c00640 1 -- 172.21.15.66:0/3362248811 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654011460 con 0x7f865c06f1a0 2024-09-10T22:03:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.197+0000 7f8663000640 1 -- 172.21.15.66:0/3362248811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c1f88d0 con 0x7f865c06f1a0 2024-09-10T22:03:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.201+0000 7f8663000640 1 -- 172.21.15.66:0/3362248811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865c0aec10 con 0x7f865c06f1a0 2024-09-10T22:03:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.201+0000 7f8658c00640 1 -- 172.21.15.66:0/3362248811 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f8654011600 con 0x7f865c06f1a0 2024-09-10T22:03:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.201+0000 7f8658c00640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8638072970 0x7f8638074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.201+0000 7f8658c00640 1 -- 172.21.15.66:0/3362248811 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f8654093b80 con 0x7f865c06f1a0 2024-09-10T22:03:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.201+0000 7f865be00640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8638072970 0x7f8638074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:11.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.201+0000 7f865be00640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8638072970 0x7f8638074e30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f864c0b0000 tx=0x7f864c0a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:11.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.205+0000 7f8658c00640 1 -- 172.21.15.66:0/3362248811 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f865405e6b0 con 0x7f865c06f1a0 2024-09-10T22:03:11.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.421+0000 7f8663000640 1 -- 172.21.15.66:0/3362248811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f865c0b06f0 con 0x7f865c06f1a0 2024-09-10T22:03:11.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.421+0000 7f8658c00640 1 -- 172.21.15.66:0/3362248811 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7f8654067420 con 0x7f865c06f1a0 2024-09-10T22:03:11.424 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:03:11.424 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"btime":"2024-09-10T22:03:09:024518+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5248,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2731842573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2731842573},{"type":"v1","addr":"172.21.15.94:6837","nonce":2731842573}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1324134239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1324134239},{"type":"v1","addr":"172.21.15.66:6833","nonce":1324134239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1694406832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1694406832},{"type":"v1","addr":"172.21.15.94:6836","nonce":1694406832}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:03:05.416734+0000","modified":"2024-09-10T22:03:08.421387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5412":{"gid":5412,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1215985114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1215985114},{"type":"v1","addr":"172.21.15.66:6835","nonce":1215985114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5412,"qdb_cluster":[5412]},"id":3}]} 2024-09-10T22:03:11.424 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 26 2024-09-10T22:03:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.425+0000 7f8663000640 1 -- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8638072970 msgr2=0x7f8638074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.425+0000 7f8663000640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8638072970 0x7f8638074e30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f864c0b0000 tx=0x7f864c0a7040 comp rx=0 tx=0).stop 2024-09-10T22:03:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.425+0000 7f8663000640 1 -- 172.21.15.66:0/3362248811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f865c06f1a0 msgr2=0x7f865c06f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.425+0000 7f8663000640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f865c06f1a0 0x7f865c06f600 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f865c073840 tx=0x7f865400eb60 comp rx=0 tx=0).stop 2024-09-10T22:03:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.425+0000 7f8663000640 1 -- 172.21.15.66:0/3362248811 shutdown_connections 2024-09-10T22:03:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.425+0000 7f8663000640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8638072970 0x7f8638074e30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.425+0000 7f8663000640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f865c06f1a0 0x7f865c06f600 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.425+0000 7f8663000640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0b67e0 0x7f865c0b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.425+0000 7f8663000640 1 --2- 172.21.15.66:0/3362248811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8640003a40 0x7f865c156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.425+0000 7f8663000640 1 -- 172.21.15.66:0/3362248811 >> 172.21.15.66:0/3362248811 conn(0x7f865c0d1c60 msgr2=0x7f865c1580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.425+0000 7f8663000640 1 -- 172.21.15.66:0/3362248811 shutdown_connections 2024-09-10T22:03:11.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.425+0000 7f8663000640 1 -- 172.21.15.66:0/3362248811 wait complete. 2024-09-10T22:03:11.848 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:03:05.416734+0000', 'modified': '2024-09-10T22:03:08.421387+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5412}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5412': {'gid': 5412, 'name': 'a', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/1215985114', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 1215985114}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 1215985114}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5412, 'qdb_cluster': [5412]} 2024-09-10T22:03:11.848 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:03:11.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T22:03:11.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.997+0000 7fe3a9200640 1 -- 172.21.15.66:0/1536861413 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a40ae4a0 msgr2=0x7fe3a40afc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.997+0000 7fe3a9200640 1 --2- 172.21.15.66:0/1536861413 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a40ae4a0 0x7fe3a40afc50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe390009840 tx=0x7fe39001c820 comp rx=0 tx=0).stop 2024-09-10T22:03:12.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.997+0000 7fe3a9200640 1 -- 172.21.15.66:0/1536861413 shutdown_connections 2024-09-10T22:03:12.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.997+0000 7fe3a9200640 1 --2- 172.21.15.66:0/1536861413 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a40ae4a0 0x7fe3a40afc50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.997+0000 7fe3a9200640 1 --2- 172.21.15.66:0/1536861413 >> v2:172.21.15.94:3300/0 conn(0x7fe3a415def0 0x7fe3a40adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.997+0000 7fe3a9200640 1 -- 172.21.15.66:0/1536861413 >> 172.21.15.66:0/1536861413 conn(0x7fe3a4074240 msgr2=0x7fe3a4074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:12.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.997+0000 7fe3a9200640 1 -- 172.21.15.66:0/1536861413 shutdown_connections 2024-09-10T22:03:12.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:11.997+0000 7fe3a9200640 1 -- 172.21.15.66:0/1536861413 wait complete. 2024-09-10T22:03:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a9200640 1 Processor -- start 2024-09-10T22:03:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a9200640 1 -- start start 2024-09-10T22:03:12.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a9200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a40ae4a0 0x7fe3a41550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a9200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a40b0190 0x7fe3a4157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a9200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a415def0 0x7fe3a415aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a9200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a416aeb0 con 0x7fe3a40ae4a0 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a9200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a416ad30 con 0x7fe3a415def0 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a9200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a416b030 con 0x7fe3a40b0190 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a40ae4a0 0x7fe3a41550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a40ae4a0 0x7fe3a41550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43506/0 (socket says 172.21.15.66:43506) 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a1e00640 1 -- 172.21.15.66:0/2922623073 learned_addr learned my addr 172.21.15.66:0/2922623073 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a1400640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a40b0190 0x7fe3a4157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a2800640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a415def0 0x7fe3a415aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a1e00640 1 -- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a40b0190 msgr2=0x7fe3a4157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a1e00640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a40b0190 0x7fe3a4157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a1e00640 1 -- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a415def0 msgr2=0x7fe3a415aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a1e00640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a415def0 0x7fe3a415aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a1e00640 1 -- 172.21.15.66:0/2922623073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a409d5d0 con 0x7fe3a40ae4a0 2024-09-10T22:03:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a1e00640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a40ae4a0 0x7fe3a41550c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe39400e7a0 tx=0x7fe39400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe39a000640 1 -- 172.21.15.66:0/2922623073 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39400ccc0 con 0x7fe3a40ae4a0 2024-09-10T22:03:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe39a000640 1 -- 172.21.15.66:0/2922623073 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe394010430 con 0x7fe3a40ae4a0 2024-09-10T22:03:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe39a000640 1 -- 172.21.15.66:0/2922623073 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394010860 con 0x7fe3a40ae4a0 2024-09-10T22:03:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a9200640 1 -- 172.21.15.66:0/2922623073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a409d8b0 con 0x7fe3a40ae4a0 2024-09-10T22:03:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.001+0000 7fe3a9200640 1 -- 172.21.15.66:0/2922623073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a409d230 con 0x7fe3a40ae4a0 2024-09-10T22:03:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe39a000640 1 -- 172.21.15.66:0/2922623073 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fe394010a00 con 0x7fe3a40ae4a0 2024-09-10T22:03:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe39a000640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe384072940 0x7fe384074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3ac54a000 1 Processor -- start 2024-09-10T22:03:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe39a000640 1 -- 172.21.15.66:0/2922623073 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fe3940942b0 con 0x7fe3a40ae4a0 2024-09-10T22:03:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3ac54a000 1 -- start start 2024-09-10T22:03:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3a1400640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe384072940 0x7fe384074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3ac54a000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297487f9d0 0x56297487fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3ac54a000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5629748802f0 0x5629748338d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3ac54a000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 0x5629748341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3ac54a000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5629747543f0 con 0x562974833e10 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3ac54a000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562974691b20 con 0x56297487f9d0 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3ac54a000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x562974680f50 con 0x5629748802f0 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 0x5629748341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 0x5629748341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43520/0 (socket says 172.21.15.66:43520) 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3a2800640 1 -- 172.21.15.66:0/1639895340 learned_addr learned my addr 172.21.15.66:0/1639895340 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3a1e00640 1 --2- 172.21.15.66:0/1639895340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297487f9d0 0x56297487fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3a1400640 1 --2- 172.21.15.66:0/1639895340 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5629748802f0 0x5629748338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3a1400640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe384072940 0x7fe384074e00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe3900af000 tx=0x7fe39001b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3a2800640 1 -- 172.21.15.66:0/1639895340 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5629748802f0 msgr2=0x5629748338d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3a2800640 1 --2- 172.21.15.66:0/1639895340 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5629748802f0 0x5629748338d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3a2800640 1 -- 172.21.15.66:0/1639895340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297487f9d0 msgr2=0x56297487fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3a2800640 1 --2- 172.21.15.66:0/1639895340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297487f9d0 0x56297487fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.005+0000 7fe3a2800640 1 -- 172.21.15.66:0/1639895340 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562974648590 con 0x562974833e10 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a2800640 1 --2- 172.21.15.66:0/1639895340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 0x5629748341f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe39c002050 tx=0x7fe39c002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d22e97249b7808 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe38ea00640 1 -- 172.21.15.66:0/1639895340 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00ea70 con 0x562974833e10 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe38ea00640 1 -- 172.21.15.66:0/1639895340 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe39c019070 con 0x562974833e10 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe38ea00640 1 -- 172.21.15.66:0/1639895340 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c014480 con 0x562974833e10 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 -- 172.21.15.66:0/1639895340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 msgr2=0x5629748341f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/1639895340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 0x5629748341f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe39c002050 tx=0x7fe39c002140 comp rx=0 tx=0).stop 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 -- 172.21.15.66:0/1639895340 shutdown_connections 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/1639895340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 0x5629748341f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/1639895340 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5629748802f0 0x5629748338d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/1639895340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297487f9d0 0x56297487fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 -- 172.21.15.66:0/1639895340 >> 172.21.15.66:0/1639895340 conn(0x56297471d130 msgr2=0x562974862e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 -- 172.21.15.66:0/1639895340 shutdown_connections 2024-09-10T22:03:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 -- 172.21.15.66:0/1639895340 wait complete. 2024-09-10T22:03:12.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 -- start start 2024-09-10T22:03:12.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 Processor -- start 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562974833e10 0x562974835240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297487f9d0 0x562974835780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629748802f0 0x562974835cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56297461b5d0 con 0x5629748802f0 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562974684d60 con 0x562974833e10 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x562974627500 con 0x56297487f9d0 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3ac54a000 10 client.0 _ll_register_callbacks cb 0x56297462b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629748802f0 0x562974835cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629748802f0 0x562974835cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43536/0 (socket says 172.21.15.66:43536) 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a2800640 1 -- 172.21.15.66:0/2653480022 learned_addr learned my addr 172.21.15.66:0/2653480022 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a1400640 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297487f9d0 0x562974835780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a1e00640 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562974833e10 0x562974835240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a2800640 1 -- 172.21.15.66:0/2653480022 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297487f9d0 msgr2=0x562974835780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a2800640 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297487f9d0 0x562974835780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a2800640 1 -- 172.21.15.66:0/2653480022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562974833e10 msgr2=0x562974835240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a2800640 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562974833e10 0x562974835240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a2800640 1 -- 172.21.15.66:0/2653480022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56297461f170 con 0x5629748802f0 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a1e00640 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562974833e10 0x562974835240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe3a2800640 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629748802f0 0x562974835cc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe39c016040 tx=0x7fe39c001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81dc75d35a3e7b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.009+0000 7fe383400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe383400640 1 -- 172.21.15.66:0/2653480022 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00ee40 con 0x5629748802f0 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 -- 172.21.15.66:0/2653480022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5629746190f0 con 0x5629748802f0 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe383400640 1 -- 172.21.15.66:0/2653480022 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fe39c004c20 con 0x5629748802f0 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe383400640 1 -- 172.21.15.66:0/2653480022 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe39c004e20 con 0x5629748802f0 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 10 client.5529 fetch_fsmap learned FSMap version 26 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 -- 172.21.15.66:0/2653480022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x562974705ef0 con 0x5629748802f0 2024-09-10T22:03:12.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe383400640 1 -- 172.21.15.66:0/2653480022 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5629746190f0 con 0x5629748802f0 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe383400640 1 -- 172.21.15.66:0/2653480022 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fe39c005130 con 0x5629748802f0 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 10 client.5529 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 10 client.5529 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 20 client.5529 populate_metadata read hostname 'smithi066' 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.66:6834/1215985114,v1:172.21.15.66:6835/1215985114] conn(0x56297488ec70 0x562974891060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 4 client.5529 mds_command: new command op to 5412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 -- 172.21.15.66:0/2653480022 --> [v2:172.21.15.66:6834/1215985114,v1:172.21.15.66:6835/1215985114] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562974693a70 con 0x56297488ec70 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3a1e00640 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.66:6834/1215985114,v1:172.21.15.66:6835/1215985114] conn(0x56297488ec70 0x562974891060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3a1e00640 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.66:6834/1215985114,v1:172.21.15.66:6835/1215985114] conn(0x56297488ec70 0x562974891060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e982c2416958c37 server_cookie=22e87a3a8dbfeb84 in_seq=0 out_seq=0 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe383400640 10 client.5529 ms_handle_connect on v2:172.21.15.66:6834/1215985114 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe383400640 1 -- 172.21.15.66:0/2653480022 <== mds.0 v2:172.21.15.66:6834/1215985114 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe39408af90 con 0x56297488ec70 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe383400640 10 client.5529 handle_command_reply: tid=0 2024-09-10T22:03:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 client.5529 shutdown 2024-09-10T22:03:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 -- 172.21.15.66:0/2653480022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629748802f0 msgr2=0x562974835cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629748802f0 0x562974835cc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe39c016040 tx=0x7fe39c001d70 comp rx=0 tx=0).stop 2024-09-10T22:03:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 -- 172.21.15.66:0/2653480022 shutdown_connections 2024-09-10T22:03:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.66:6834/1215985114,v1:172.21.15.66:6835/1215985114] conn(0x56297488ec70 0x562974891060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629748802f0 0x562974835cc0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297487f9d0 0x562974835780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/2653480022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562974833e10 0x562974835240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe383400640 0 client.5529 ms_handle_reset on v2:172.21.15.66:6834/1215985114 2024-09-10T22:03:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 -- 172.21.15.66:0/2653480022 >> 172.21.15.66:0/2653480022 conn(0x56297471d130 msgr2=0x562974832fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 -- 172.21.15.66:0/2653480022 shutdown_connections 2024-09-10T22:03:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 1 -- 172.21.15.66:0/2653480022 wait complete. 2024-09-10T22:03:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.013+0000 7fe3ac54a000 20 client.5529 trim_cache size 0 max 16384 2024-09-10T22:03:12.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3ac54a000 1 Processor -- start 2024-09-10T22:03:12.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3ac54a000 1 -- start start 2024-09-10T22:03:12.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3ac54a000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562974833e10 0x5629748f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3ac54a000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297487f9d0 0x5629748f1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3ac54a000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629748802f0 0x5629748f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3ac54a000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4202f30 con 0x56297487f9d0 2024-09-10T22:03:12.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3ac54a000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4154130 con 0x5629748802f0 2024-09-10T22:03:12.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3ac54a000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a4154780 con 0x562974833e10 2024-09-10T22:03:12.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3a1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297487f9d0 0x5629748f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3a1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297487f9d0 0x5629748f1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43548/0 (socket says 172.21.15.66:43548) 2024-09-10T22:03:12.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3a1400640 1 -- 172.21.15.66:0/4019463834 learned_addr learned my addr 172.21.15.66:0/4019463834 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:12.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3a2800640 1 --2- 172.21.15.66:0/4019463834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629748802f0 0x5629748f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3a1e00640 1 --2- 172.21.15.66:0/4019463834 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562974833e10 0x5629748f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3a1400640 1 -- 172.21.15.66:0/4019463834 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562974833e10 msgr2=0x5629748f13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:12.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3a1400640 1 --2- 172.21.15.66:0/4019463834 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562974833e10 0x5629748f13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3a1400640 1 -- 172.21.15.66:0/4019463834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629748802f0 msgr2=0x5629748f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:12.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3a1400640 1 --2- 172.21.15.66:0/4019463834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629748802f0 0x5629748f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3a1400640 1 -- 172.21.15.66:0/4019463834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629746dcd90 con 0x56297487f9d0 2024-09-10T22:03:12.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3a2800640 1 --2- 172.21.15.66:0/4019463834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629748802f0 0x5629748f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe3a1400640 1 --2- 172.21.15.66:0/4019463834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297487f9d0 0x5629748f1900 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe3900b0040 tx=0x7fe3900be000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ca77f367ebb2ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe38e000640 1 -- 172.21.15.66:0/4019463834 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3900048d0 con 0x56297487f9d0 2024-09-10T22:03:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe38e000640 1 -- 172.21.15.66:0/4019463834 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3900c1040 con 0x56297487f9d0 2024-09-10T22:03:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.033+0000 7fe38e000640 1 -- 172.21.15.66:0/4019463834 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390004cb0 con 0x56297487f9d0 2024-09-10T22:03:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 -- 172.21.15.66:0/4019463834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297487f9d0 msgr2=0x5629748f1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/4019463834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297487f9d0 0x5629748f1900 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe3900b0040 tx=0x7fe3900be000 comp rx=0 tx=0).stop 2024-09-10T22:03:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 -- 172.21.15.66:0/4019463834 shutdown_connections 2024-09-10T22:03:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/4019463834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629748802f0 0x5629748f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/4019463834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297487f9d0 0x5629748f1900 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/4019463834 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562974833e10 0x5629748f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 -- 172.21.15.66:0/4019463834 >> 172.21.15.66:0/4019463834 conn(0x56297471d130 msgr2=0x562974896e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:12.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 -- 172.21.15.66:0/4019463834 shutdown_connections 2024-09-10T22:03:12.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 -- 172.21.15.66:0/4019463834 wait complete. 2024-09-10T22:03:12.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 -- start start 2024-09-10T22:03:12.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 Processor -- start 2024-09-10T22:03:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 0x562974895a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297487f9d0 0x5629748b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629748802f0 0x5629748b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562974684070 con 0x562974833e10 2024-09-10T22:03:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562974689390 con 0x5629748802f0 2024-09-10T22:03:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x5629746908f0 con 0x56297487f9d0 2024-09-10T22:03:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3ac54a000 10 client.0 _ll_register_callbacks cb 0x7fe39400ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3a1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 0x562974895a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3a1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 0x562974895a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43558/0 (socket says 172.21.15.66:43558) 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.037+0000 7fe3a1e00640 1 -- 172.21.15.66:0/110845160 learned_addr learned my addr 172.21.15.66:0/110845160 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a1400640 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297487f9d0 0x5629748b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a2800640 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629748802f0 0x5629748b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a1e00640 1 -- 172.21.15.66:0/110845160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297487f9d0 msgr2=0x5629748b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a1e00640 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297487f9d0 0x5629748b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a1e00640 1 -- 172.21.15.66:0/110845160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629748802f0 msgr2=0x5629748b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a1e00640 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629748802f0 0x5629748b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a1e00640 1 -- 172.21.15.66:0/110845160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562974647a90 con 0x562974833e10 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a1e00640 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 0x562974895a50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe39405c030 tx=0x7fe39405ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a20e593833c17e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe383400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:03:12.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe383400640 1 -- 172.21.15.66:0/110845160 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3940638a0 con 0x562974833e10 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe383400640 1 -- 172.21.15.66:0/110845160 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fe3940940b0 con 0x562974833e10 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 -- 172.21.15.66:0/110845160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3900c1040 con 0x562974833e10 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe383400640 1 -- 172.21.15.66:0/110845160 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3940682c0 con 0x562974833e10 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 10 client.5535 fetch_fsmap learned FSMap version 26 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 -- 172.21.15.66:0/110845160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x562974626050 con 0x562974833e10 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe383400640 1 -- 172.21.15.66:0/110845160 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3900c1040 con 0x562974833e10 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe383400640 1 -- 172.21.15.66:0/110845160 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fe394088030 con 0x562974833e10 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 10 client.5535 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 10 client.5535 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 20 client.5535 populate_metadata read hostname 'smithi066' 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.66:6834/1215985114,v1:172.21.15.66:6835/1215985114] conn(0x5629748b2000 0x5629748f5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 4 client.5535 mds_command: new command op to 5412 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 -- 172.21.15.66:0/110845160 --> [v2:172.21.15.66:6834/1215985114,v1:172.21.15.66:6835/1215985114] -- command(tid 0: {"prefix": "status"}) -- 0x56297468f520 con 0x5629748b2000 2024-09-10T22:03:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a1400640 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.66:6834/1215985114,v1:172.21.15.66:6835/1215985114] conn(0x5629748b2000 0x5629748f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a1400640 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.66:6834/1215985114,v1:172.21.15.66:6835/1215985114] conn(0x5629748b2000 0x5629748f5610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a87da9130200c9d server_cookie=d4188997dd83f7fc in_seq=0 out_seq=0 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe383400640 10 client.5535 ms_handle_connect on v2:172.21.15.66:6834/1215985114 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe383400640 1 -- 172.21.15.66:0/110845160 <== mds.0 v2:172.21.15.66:6834/1215985114 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56297468f520 con 0x5629748b2000 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe383400640 10 client.5535 handle_command_reply: tid=0 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 client.5535 shutdown 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 -- 172.21.15.66:0/110845160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 msgr2=0x562974895a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 0x562974895a50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe39405c030 tx=0x7fe39405ecb0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 -- 172.21.15.66:0/110845160 shutdown_connections 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.66:6834/1215985114,v1:172.21.15.66:6835/1215985114] conn(0x5629748b2000 0x5629748f5610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629748802f0 0x5629748b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297487f9d0 0x5629748b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 --2- 172.21.15.66:0/110845160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562974833e10 0x562974895a50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe383400640 0 client.5535 ms_handle_reset on v2:172.21.15.66:6834/1215985114 2024-09-10T22:03:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 -- 172.21.15.66:0/110845160 >> 172.21.15.66:0/110845160 conn(0x56297471d130 msgr2=0x562974833470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:12.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 -- 172.21.15.66:0/110845160 shutdown_connections 2024-09-10T22:03:12.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 1 -- 172.21.15.66:0/110845160 wait complete. 2024-09-10T22:03:12.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3ac54a000 20 client.5535 trim_cache size 0 max 16384 2024-09-10T22:03:12.048 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:03:12.048 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:03:12.048 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:03:12.048 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5412, 2024-09-10T22:03:12.048 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:03:12.048 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:03:12.048 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:03:12.049 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 6.0194847219999996, 2024-09-10T22:03:12.049 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 26, 2024-09-10T22:03:12.049 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 44, 2024-09-10T22:03:12.049 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 44, 2024-09-10T22:03:12.049 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 10.481069121000001 2024-09-10T22:03:12.049 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:03:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a9200640 1 -- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe384072940 msgr2=0x7fe384074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a9200640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe384072940 0x7fe384074e00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe3900af000 tx=0x7fe39001b060 comp rx=0 tx=0).stop 2024-09-10T22:03:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a9200640 1 -- 172.21.15.66:0/2922623073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a40ae4a0 msgr2=0x7fe3a41550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a9200640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a40ae4a0 0x7fe3a41550c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe39400e7a0 tx=0x7fe39400ec60 comp rx=0 tx=0).stop 2024-09-10T22:03:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a9200640 1 -- 172.21.15.66:0/2922623073 shutdown_connections 2024-09-10T22:03:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a9200640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe384072940 0x7fe384074e00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a9200640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a415def0 0x7fe3a415aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a9200640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a40b0190 0x7fe3a4157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a9200640 1 --2- 172.21.15.66:0/2922623073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3a40ae4a0 0x7fe3a41550c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a9200640 1 -- 172.21.15.66:0/2922623073 >> 172.21.15.66:0/2922623073 conn(0x7fe3a4074240 msgr2=0x7fe3a40b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a9200640 1 -- 172.21.15.66:0/2922623073 shutdown_connections 2024-09-10T22:03:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.041+0000 7fe3a9200640 1 -- 172.21.15.66:0/2922623073 wait complete. 2024-09-10T22:03:12.064 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:03:12.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.209+0000 7f516e800640 1 -- 172.21.15.66:0/3288178007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51680d0d60 msgr2=0x7f51680cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.209+0000 7f516e800640 1 --2- 172.21.15.66:0/3288178007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51680d0d60 0x7f51680cdbb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5168073840 tx=0x7f516001ccf0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.209+0000 7f516e800640 1 -- 172.21.15.66:0/3288178007 shutdown_connections 2024-09-10T22:03:12.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.209+0000 7f516e800640 1 --2- 172.21.15.66:0/3288178007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51680d0d60 0x7f51680cdbb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.209+0000 7f516e800640 1 --2- 172.21.15.66:0/3288178007 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51680d03e0 0x7f51680d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.209+0000 7f516e800640 1 --2- 172.21.15.66:0/3288178007 >> v2:172.21.15.94:3300/0 conn(0x7f516815e010 0x7f51680cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.209+0000 7f516e800640 1 -- 172.21.15.66:0/3288178007 >> 172.21.15.66:0/3288178007 conn(0x7f51680d1c60 msgr2=0x7f51680d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:12.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.209+0000 7f516e800640 1 -- 172.21.15.66:0/3288178007 shutdown_connections 2024-09-10T22:03:12.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.209+0000 7f516e800640 1 -- 172.21.15.66:0/3288178007 wait complete. 2024-09-10T22:03:12.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.209+0000 7f516e800640 1 Processor -- start 2024-09-10T22:03:12.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.209+0000 7f516e800640 1 -- start start 2024-09-10T22:03:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f516e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51680d03e0 0x7f51680c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f516e800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51680d0d60 0x7f51681fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f516e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f516815e010 0x7f51681faa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f516e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f516806e0c0 con 0x7f516815e010 2024-09-10T22:03:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f516e800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f516806df40 con 0x7f51680d03e0 2024-09-10T22:03:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f516e800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f516806e240 con 0x7f51680d0d60 2024-09-10T22:03:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f5167e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f516815e010 0x7f51681faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f5167e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f516815e010 0x7f51681faa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43580/0 (socket says 172.21.15.66:43580) 2024-09-10T22:03:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f5167e00640 1 -- 172.21.15.66:0/219469995 learned_addr learned my addr 172.21.15.66:0/219469995 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f5167400640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51680d03e0 0x7f51680c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f5166a00640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51680d0d60 0x7f51681fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f5167e00640 1 -- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51680d0d60 msgr2=0x7f51681fa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f5167e00640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51680d0d60 0x7f51681fa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f5167e00640 1 -- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51680d03e0 msgr2=0x7f51680c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f5167e00640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51680d03e0 0x7f51680c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f5167e00640 1 -- 172.21.15.66:0/219469995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51680c8e90 con 0x7f516815e010 2024-09-10T22:03:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f5167e00640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f516815e010 0x7f51681faa00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f516001b3a0 tx=0x7f51600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f515f400640 1 -- 172.21.15.66:0/219469995 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51600bb070 con 0x7f516815e010 2024-09-10T22:03:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f515f400640 1 -- 172.21.15.66:0/219469995 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51600044e0 con 0x7f516815e010 2024-09-10T22:03:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f515f400640 1 -- 172.21.15.66:0/219469995 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51600b2040 con 0x7f516815e010 2024-09-10T22:03:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f516e800640 1 -- 172.21.15.66:0/219469995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51680c5ae0 con 0x7f516815e010 2024-09-10T22:03:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f516e800640 1 -- 172.21.15.66:0/219469995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51680c9100 con 0x7f516815e010 2024-09-10T22:03:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.213+0000 7f516e800640 1 -- 172.21.15.66:0/219469995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51680cdbb0 con 0x7f516815e010 2024-09-10T22:03:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.217+0000 7f515f400640 1 -- 172.21.15.66:0/219469995 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f51600b5030 con 0x7f516815e010 2024-09-10T22:03:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.217+0000 7f515f400640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5148072970 0x7f5148074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.217+0000 7f515f400640 1 -- 172.21.15.66:0/219469995 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f516012d510 con 0x7f516815e010 2024-09-10T22:03:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.217+0000 7f5167400640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5148072970 0x7f5148074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:12.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.217+0000 7f5167400640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5148072970 0x7f5148074e30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f51680cb560 tx=0x7f51580091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:12.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.221+0000 7f515f400640 1 -- 172.21.15.66:0/219469995 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5160016970 con 0x7f516815e010 2024-09-10T22:03:12.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.429+0000 7f516e800640 1 -- 172.21.15.66:0/219469995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f51680c8740 con 0x7f516815e010 2024-09-10T22:03:12.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.429+0000 7f515f400640 1 -- 172.21.15.66:0/219469995 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f516012be20 con 0x7f516815e010 2024-09-10T22:03:12.433 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:03:12.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.433+0000 7f516e800640 1 -- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5148072970 msgr2=0x7f5148074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.433+0000 7f516e800640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5148072970 0x7f5148074e30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f51680cb560 tx=0x7f51580091e0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.433+0000 7f516e800640 1 -- 172.21.15.66:0/219469995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f516815e010 msgr2=0x7f51681faa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:12.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.433+0000 7f516e800640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f516815e010 0x7f51681faa00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f516001b3a0 tx=0x7f51600042f0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.433+0000 7f516e800640 1 -- 172.21.15.66:0/219469995 shutdown_connections 2024-09-10T22:03:12.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.433+0000 7f516e800640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5148072970 0x7f5148074e30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.433+0000 7f516e800640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f516815e010 0x7f51681faa00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.433+0000 7f516e800640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51680d0d60 0x7f51681fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.433+0000 7f516e800640 1 --2- 172.21.15.66:0/219469995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51680d03e0 0x7f51680c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:12.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.433+0000 7f516e800640 1 -- 172.21.15.66:0/219469995 >> 172.21.15.66:0/219469995 conn(0x7f51680d1c60 msgr2=0x7f51680b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:12.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.433+0000 7f516e800640 1 -- 172.21.15.66:0/219469995 shutdown_connections 2024-09-10T22:03:12.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12.433+0000 7f516e800640 1 -- 172.21.15.66:0/219469995 wait complete. 2024-09-10T22:03:12.454 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:03:12.454 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:03:12.454 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:03:12.454 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:03:12.454 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:12.454 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:03:12.454 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:03:12.454 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:03:12.455 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:03:12.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:12.455 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:03:12.464 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:03:12.464 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:03:12.464 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:03:12.464 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:03:12.464 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:03:12.464 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:03:12.464 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:03:12.464 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:03:12.464 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:03:12.464 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42282sec preferred_lft 42282sec 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:81:fc:4c:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:03:12.465 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:03:12.466 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:03:12.467 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T22:03:12.467 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:03:12.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:12.468 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:03:12.530 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:03:12.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:12.533 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:03:12.589 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:03:12.589 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:03:12.589 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:03:12.589 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:03:12.589 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:03:12.589 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:03:12.589 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:03:12.589 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:03:12.589 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:03:12.589 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:03:12.589 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:03:12.590 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:03:12.590 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:03:12.590 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:03:12.590 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:03:12.590 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:03:12.590 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:03:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:03:12.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:03:12.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:12.729 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:03:12.738 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:03:12.738 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:03:12.738 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:03:12.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:12.738 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:03:12.801 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T22:03:12.801 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:03:12.801 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:12.853 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:03:12.853 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:03:12.853 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:12.904 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:03:12.964 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:03:13.018 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:03:13.019 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:03:13.019 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:03:13.019 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:03:13.019 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:03:13.019 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:03:13.020 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:03:13.020 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:03:13.020 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:03:13.020 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:03:13.020 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:03:13.020 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:03:13.020 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:03:13.020 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:03:13.021 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:03:13.021 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:03:13.021 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:03:13.021 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:03:13.021 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:03:13.021 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:03:13.021 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:03:13.021 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:03:13.021 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:03:13.021 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:03:13.021 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:03:13.021 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:03:13.022 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:03:13.022 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:03:13.022 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:03:13.022 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:03:13.022 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:03:13.022 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:03:13.022 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:03:13.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:13.023 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:03:13.073 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:03:13.073 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T22:03:13.123 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:03:13.144 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:03:13.194 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:03:13.194 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:03:13.194 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:03:13.194 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:03:13.194 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:03:13.194 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:03:13.194 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:03:13.194 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:03:13.195 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:03:13.196 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:03:13.196 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:03:13.196 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:03:13.196 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:03:13.196 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:03:13.196 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:03:13.196 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:03:13.196 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:03:13.196 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:03:13.196 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:03:13.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:13.197 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:03:13.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-09-10T22:03:13.217+0000 7f3599255580 -1 init, newargv = 0x5620848f3c30 newargc=15 2024-09-10T22:03:13.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[38894]: starting ceph client 2024-09-10T22:03:13.225 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[38894]: starting fuse 2024-09-10T22:03:13.227 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-09-10T22:03:13.227 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T22:03:13.227 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-09-10T22:03:13.228 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-09-10T22:03:13.229 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:03:13.351 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.38894.asok 2024-09-10T22:03:13.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:03:13.362 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38894.asok 2024-09-10T22:03:13.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:13.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38894.asok status 2024-09-10T22:03:13.539 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:03:13.540 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-09-10T22:03:13.540 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:03:13.540 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:03:13.540 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-09-10T22:03:13.540 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-09-10T22:03:13.540 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T22:03:13.540 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "38894", 2024-09-10T22:03:13.540 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-09-10T22:03:13.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:13.540 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-09-10T22:03:13.540 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5547, 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 5547 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1291921178 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:03:13.541 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:03:13.542 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:03:13.542 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1291921178 2024-09-10T22:03:13.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:03:13.542 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.5547 192.168.0.1:0/1291921178", 2024-09-10T22:03:13.542 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/1291921178", 2024-09-10T22:03:13.542 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-09-10T22:03:13.542 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 26, 2024-09-10T22:03:13.542 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 45, 2024-09-10T22:03:13.542 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:03:13.542 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-09-10T22:03:13.542 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-09-10T22:03:13.542 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:03:13.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:13.557 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-09-10T22:03:13.558 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:13.566 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-09-10T22:03:13.567 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:13.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:13.567 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:13.632 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T22:03:13.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.825+0000 7f457be00640 1 -- 172.21.15.66:0/982798684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4560003a90 msgr2=0x7f4560003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:13.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.825+0000 7f457be00640 1 --2- 172.21.15.66:0/982798684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4560003a90 0x7f4560003eb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f456c009b60 tx=0x7f456c01d240 comp rx=0 tx=0).stop 2024-09-10T22:03:13.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.825+0000 7f457b400640 1 -- 172.21.15.66:0/982798684 reap_dead start 2024-09-10T22:03:13.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.825+0000 7f457be00640 1 -- 172.21.15.66:0/982798684 shutdown_connections 2024-09-10T22:03:13.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.825+0000 7f457be00640 1 -- 172.21.15.66:0/982798684 >> 172.21.15.66:0/982798684 conn(0x7f457c09aa50 msgr2=0x7f457c09ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:13.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.825+0000 7f457be00640 1 -- 172.21.15.66:0/982798684 shutdown_connections 2024-09-10T22:03:13.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.825+0000 7f457be00640 1 -- 172.21.15.66:0/982798684 wait complete. 2024-09-10T22:03:13.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457be00640 1 Processor -- start 2024-09-10T22:03:13.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457be00640 1 -- start start 2024-09-10T22:03:13.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457be00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4560003a90 0x7f457c155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:13.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f457c0c7330 0x7f457c1fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:13.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f457c0af980 0x7f457c0afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:13.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f457c169c60 con 0x7f457c0c7330 2024-09-10T22:03:13.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f457c0b2ec0 con 0x7f457c0af980 2024-09-10T22:03:13.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457be00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f457c169de0 con 0x7f4560003a90 2024-09-10T22:03:13.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f4573e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f457c0c7330 0x7f457c1fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:13.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f4573e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f457c0c7330 0x7f457c1fbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59826/0 (socket says 172.21.15.66:59826) 2024-09-10T22:03:13.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f4573e00640 1 -- 172.21.15.66:0/1595672086 learned_addr learned my addr 172.21.15.66:0/1595672086 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:13.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457aa00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4560003a90 0x7f457c155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:13.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457b400640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f457c0af980 0x7f457c0afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:13.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f4573e00640 1 -- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4560003a90 msgr2=0x7f457c155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:13.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f4573e00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4560003a90 0x7f457c155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:13.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f4573e00640 1 -- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f457c0af980 msgr2=0x7f457c0afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:13.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f4573e00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f457c0af980 0x7f457c0afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:13.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f4573e00640 1 -- 172.21.15.66:0/1595672086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c072510 con 0x7f457c0c7330 2024-09-10T22:03:13.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457b400640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f457c0af980 0x7f457c0afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:13.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f4573e00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f457c0c7330 0x7f457c1fbb00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4564003ed0 tx=0x7f4564001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:13.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f4572a00640 1 -- 172.21.15.66:0/1595672086 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564015070 con 0x7f457c0c7330 2024-09-10T22:03:13.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f4572a00640 1 -- 172.21.15.66:0/1595672086 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4564011840 con 0x7f457c0c7330 2024-09-10T22:03:13.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457be00640 1 -- 172.21.15.66:0/1595672086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457c1fc040 con 0x7f457c0c7330 2024-09-10T22:03:13.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f4572a00640 1 -- 172.21.15.66:0/1595672086 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564011c70 con 0x7f457c0c7330 2024-09-10T22:03:13.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.829+0000 7f457be00640 1 -- 172.21.15.66:0/1595672086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457c0c7a90 con 0x7f457c0c7330 2024-09-10T22:03:13.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.833+0000 7f457be00640 1 -- 172.21.15.66:0/1595672086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f457c0d1940 con 0x7f457c0c7330 2024-09-10T22:03:13.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.833+0000 7f4572a00640 1 -- 172.21.15.66:0/1595672086 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f45640103c0 con 0x7f457c0c7330 2024-09-10T22:03:13.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.833+0000 7f4572a00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4558072970 0x7f4558074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:13.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.833+0000 7f4572a00640 1 -- 172.21.15.66:0/1595672086 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f4564094970 con 0x7f457c0c7330 2024-09-10T22:03:13.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.833+0000 7f457aa00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4558072970 0x7f4558074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.833+0000 7f457aa00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4558072970 0x7f4558074e30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f456c0ae000 tx=0x7f456c019fe0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:13.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:13.837+0000 7f4572a00640 1 -- 172.21.15.66:0/1595672086 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f457c1fc040 con 0x7f457c0c7330 2024-09-10T22:03:14.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:14.057+0000 7f457be00640 1 -- 172.21.15.66:0/1595672086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f457c161510 con 0x7f457c0c7330 2024-09-10T22:03:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.053+0000 7f4572a00640 1 -- 172.21.15.66:0/1595672086 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v27) ==== 136+0+0 (secure 0 0 0) 0x7f456405f4e0 con 0x7f457c0c7330 2024-09-10T22:03:15.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.057+0000 7f457be00640 1 -- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4558072970 msgr2=0x7f4558074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.057+0000 7f457be00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4558072970 0x7f4558074e30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f456c0ae000 tx=0x7f456c019fe0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.057+0000 7f457be00640 1 -- 172.21.15.66:0/1595672086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f457c0c7330 msgr2=0x7f457c1fbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.057+0000 7f457be00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f457c0c7330 0x7f457c1fbb00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4564003ed0 tx=0x7f4564001640 comp rx=0 tx=0).stop 2024-09-10T22:03:15.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.057+0000 7f457be00640 1 -- 172.21.15.66:0/1595672086 shutdown_connections 2024-09-10T22:03:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.057+0000 7f457be00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4558072970 0x7f4558074e30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.057+0000 7f457be00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f457c0af980 0x7f457c0afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.057+0000 7f457be00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f457c0c7330 0x7f457c1fbb00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.057+0000 7f457be00640 1 --2- 172.21.15.66:0/1595672086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4560003a90 0x7f457c155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.057+0000 7f457be00640 1 -- 172.21.15.66:0/1595672086 >> 172.21.15.66:0/1595672086 conn(0x7f457c09aa50 msgr2=0x7f457c15cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.061+0000 7f457be00640 1 -- 172.21.15.66:0/1595672086 shutdown_connections 2024-09-10T22:03:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.061+0000 7f457be00640 1 -- 172.21.15.66:0/1595672086 wait complete. 2024-09-10T22:03:15.086 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:03:15.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.233+0000 7fdca6a00640 1 -- 172.21.15.66:0/2395266167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdca00aeec0 msgr2=0x7fdca015df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.233+0000 7fdca6a00640 1 --2- 172.21.15.66:0/2395266167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdca00aeec0 0x7fdca015df60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdca0069da0 tx=0x7fdc9801ccf0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 -- 172.21.15.66:0/2395266167 shutdown_connections 2024-09-10T22:03:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 --2- 172.21.15.66:0/2395266167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdca00aeec0 0x7fdca015df60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 --2- 172.21.15.66:0/2395266167 >> v2:172.21.15.94:3300/0 conn(0x7fdca00d0550 0x7fdca00c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 --2- 172.21.15.66:0/2395266167 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdca015a310 0x7fdca00c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 -- 172.21.15.66:0/2395266167 >> 172.21.15.66:0/2395266167 conn(0x7fdca00cfa30 msgr2=0x7fdca00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 -- 172.21.15.66:0/2395266167 shutdown_connections 2024-09-10T22:03:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 -- 172.21.15.66:0/2395266167 wait complete. 2024-09-10T22:03:15.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 Processor -- start 2024-09-10T22:03:15.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 -- start start 2024-09-10T22:03:15.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdca00aeec0 0x7fdca00b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:15.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdca015a310 0x7fdca014f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:15.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdca01fbe80 0x7fdca00b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:15.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdca015ec40 con 0x7fdca00aeec0 2024-09-10T22:03:15.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdca015eac0 con 0x7fdca015a310 2024-09-10T22:03:15.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca6a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdca015edc0 con 0x7fdca01fbe80 2024-09-10T22:03:15.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdc9fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdca00aeec0 0x7fdca00b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdc9fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdca00aeec0 0x7fdca00b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59846/0 (socket says 172.21.15.66:59846) 2024-09-10T22:03:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdc9fe00640 1 -- 172.21.15.66:0/3418968568 learned_addr learned my addr 172.21.15.66:0/3418968568 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdca4a00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdca01fbe80 0x7fdca00b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdc97e00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdca015a310 0x7fdca014f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdc9fe00640 1 -- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdca01fbe80 msgr2=0x7fdca00b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdc9fe00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdca01fbe80 0x7fdca00b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdc9fe00640 1 -- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdca015a310 msgr2=0x7fdca014f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdc9fe00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdca015a310 0x7fdca014f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdc9fe00640 1 -- 172.21.15.66:0/3418968568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca00b5570 con 0x7fdca00aeec0 2024-09-10T22:03:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdc97e00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdca015a310 0x7fdca014f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.237+0000 7fdc9fe00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdca00aeec0 0x7fdca00b4470 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdc900058d0 tx=0x7fdc9000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.241+0000 7fdc9cc00640 1 -- 172.21.15.66:0/3418968568 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90017070 con 0x7fdca00aeec0 2024-09-10T22:03:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.241+0000 7fdc9cc00640 1 -- 172.21.15.66:0/3418968568 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc90009cc0 con 0x7fdca00aeec0 2024-09-10T22:03:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.241+0000 7fdc9cc00640 1 -- 172.21.15.66:0/3418968568 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc900106c0 con 0x7fdca00aeec0 2024-09-10T22:03:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.241+0000 7fdca6a00640 1 -- 172.21.15.66:0/3418968568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca00b5850 con 0x7fdca00aeec0 2024-09-10T22:03:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.241+0000 7fdca6a00640 1 -- 172.21.15.66:0/3418968568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca00b3c70 con 0x7fdca00aeec0 2024-09-10T22:03:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.241+0000 7fdca6a00640 1 -- 172.21.15.66:0/3418968568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca015df60 con 0x7fdca00aeec0 2024-09-10T22:03:15.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.241+0000 7fdc9cc00640 1 -- 172.21.15.66:0/3418968568 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fdc90010860 con 0x7fdca00aeec0 2024-09-10T22:03:15.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.241+0000 7fdc9cc00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc7c072940 0x7fdc7c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:15.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.241+0000 7fdc9cc00640 1 -- 172.21.15.66:0/3418968568 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fdc90093d90 con 0x7fdca00aeec0 2024-09-10T22:03:15.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.241+0000 7fdc97e00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc7c072940 0x7fdc7c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:15.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.245+0000 7fdc97e00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc7c072940 0x7fdc7c074e00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdc88005fd0 tx=0x7fdc88005f20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:15.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.245+0000 7fdc9cc00640 1 -- 172.21.15.66:0/3418968568 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdc9005e780 con 0x7fdca00aeec0 2024-09-10T22:03:15.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.441+0000 7fdca6a00640 1 -- 172.21.15.66:0/3418968568 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fdca014f6c0 con 0x7fdc7c072940 2024-09-10T22:03:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdc9cc00640 1 -- 172.21.15.66:0/3418968568 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fdca014f6c0 con 0x7fdc7c072940 2024-09-10T22:03:15.446 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:03:15.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:03:15.447 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:03:15.447 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:03:15.447 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:03:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdca6a00640 1 -- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc7c072940 msgr2=0x7fdc7c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdca6a00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc7c072940 0x7fdc7c074e00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdc88005fd0 tx=0x7fdc88005f20 comp rx=0 tx=0).stop 2024-09-10T22:03:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdca6a00640 1 -- 172.21.15.66:0/3418968568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdca00aeec0 msgr2=0x7fdca00b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdca6a00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdca00aeec0 0x7fdca00b4470 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdc900058d0 tx=0x7fdc9000dde0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdca6a00640 1 -- 172.21.15.66:0/3418968568 shutdown_connections 2024-09-10T22:03:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdca6a00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc7c072940 0x7fdc7c074e00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdca6a00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdca01fbe80 0x7fdca00b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdca6a00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdca015a310 0x7fdca014f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdca6a00640 1 --2- 172.21.15.66:0/3418968568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdca00aeec0 0x7fdca00b4470 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdca6a00640 1 -- 172.21.15.66:0/3418968568 >> 172.21.15.66:0/3418968568 conn(0x7fdca00cfa30 msgr2=0x7fdca00ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdca6a00640 1 -- 172.21.15.66:0/3418968568 shutdown_connections 2024-09-10T22:03:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.445+0000 7fdca6a00640 1 -- 172.21.15.66:0/3418968568 wait complete. 2024-09-10T22:03:15.468 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:03:15.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 -- 172.21.15.66:0/284919297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a9406f670 msgr2=0x7f6a9406fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 --2- 172.21.15.66:0/284919297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a9406f670 0x7f6a9406fd30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009840 tx=0x7f6a8401c790 comp rx=0 tx=0).stop 2024-09-10T22:03:15.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 -- 172.21.15.66:0/284919297 shutdown_connections 2024-09-10T22:03:15.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 --2- 172.21.15.66:0/284919297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a9406f670 0x7f6a9406fd30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 -- 172.21.15.66:0/284919297 >> 172.21.15.66:0/284919297 conn(0x7f6a940d0b30 msgr2=0x7f6a940d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:15.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 -- 172.21.15.66:0/284919297 shutdown_connections 2024-09-10T22:03:15.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 -- 172.21.15.66:0/284919297 wait complete. 2024-09-10T22:03:15.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 Processor -- start 2024-09-10T22:03:15.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 -- start start 2024-09-10T22:03:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a9406f670 0x7f6a941f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a940736e0 0x7f6a941518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a9415e000 0x7f6a94150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941696e0 con 0x7f6a940736e0 2024-09-10T22:03:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94169560 con 0x7f6a9415e000 2024-09-10T22:03:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9c800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6a94169860 con 0x7f6a9406f670 2024-09-10T22:03:15.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a99400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a940736e0 0x7f6a941518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:15.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a99400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a940736e0 0x7f6a941518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59856/0 (socket says 172.21.15.66:59856) 2024-09-10T22:03:15.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a99400640 1 -- 172.21.15.66:0/3598399697 learned_addr learned my addr 172.21.15.66:0/3598399697 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:15.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a99e00640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a9406f670 0x7f6a941f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:15.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.616+0000 7f6a9a800640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a9415e000 0x7f6a94150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a99400640 1 -- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a9406f670 msgr2=0x7f6a941f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a99400640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a9406f670 0x7f6a941f8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a99400640 1 -- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a9415e000 msgr2=0x7f6a94150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a99400640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a9415e000 0x7f6a94150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a99400640 1 -- 172.21.15.66:0/3598399697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a94152100 con 0x7f6a940736e0 2024-09-10T22:03:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a99e00640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a9406f670 0x7f6a941f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a9a800640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a9415e000 0x7f6a94150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:15.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a99400640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a940736e0 0x7f6a941518d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009790 tx=0x7f6a84004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:15.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a8e000640 1 -- 172.21.15.66:0/3598399697 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a840bb070 con 0x7f6a940736e0 2024-09-10T22:03:15.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a8e000640 1 -- 172.21.15.66:0/3598399697 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a84005320 con 0x7f6a940736e0 2024-09-10T22:03:15.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a8e000640 1 -- 172.21.15.66:0/3598399697 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a840b2040 con 0x7f6a940736e0 2024-09-10T22:03:15.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a9c800640 1 -- 172.21.15.66:0/3598399697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a941523e0 con 0x7f6a940736e0 2024-09-10T22:03:15.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a9c800640 1 -- 172.21.15.66:0/3598399697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a94151e10 con 0x7f6a940736e0 2024-09-10T22:03:15.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a9c800640 1 -- 172.21.15.66:0/3598399697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a94152900 con 0x7f6a940736e0 2024-09-10T22:03:15.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.620+0000 7f6a8e000640 1 -- 172.21.15.66:0/3598399697 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f6a840b5030 con 0x7f6a940736e0 2024-09-10T22:03:15.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.624+0000 7f6a8e000640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a78072940 0x7f6a78074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:15.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.624+0000 7f6a8e000640 1 -- 172.21.15.66:0/3598399697 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f6a8412d000 con 0x7f6a940736e0 2024-09-10T22:03:15.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.624+0000 7f6a99e00640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a78072940 0x7f6a78074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:15.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.624+0000 7f6a99e00640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a78072940 0x7f6a78074e00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000f70 tx=0x7f6a88009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:15.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.624+0000 7f6a8e000640 1 -- 172.21.15.66:0/3598399697 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6a840f1de0 con 0x7f6a940736e0 2024-09-10T22:03:15.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 -- 172.21.15.66:0/3598399697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6a9406fd30 con 0x7f6a940736e0 2024-09-10T22:03:15.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a8e000640 1 -- 172.21.15.66:0/3598399697 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7f6a84016410 con 0x7f6a940736e0 2024-09-10T22:03:15.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 -- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a78072940 msgr2=0x7f6a78074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a78072940 0x7f6a78074e00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000f70 tx=0x7f6a88009040 comp rx=0 tx=0).stop 2024-09-10T22:03:15.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 -- 172.21.15.66:0/3598399697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a940736e0 msgr2=0x7f6a941518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a940736e0 0x7f6a941518d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009790 tx=0x7f6a84004220 comp rx=0 tx=0).stop 2024-09-10T22:03:15.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 -- 172.21.15.66:0/3598399697 shutdown_connections 2024-09-10T22:03:15.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a78072940 0x7f6a78074e00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a9415e000 0x7f6a94150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a940736e0 0x7f6a941518d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 --2- 172.21.15.66:0/3598399697 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a9406f670 0x7f6a941f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 -- 172.21.15.66:0/3598399697 >> 172.21.15.66:0/3598399697 conn(0x7f6a940d0b30 msgr2=0x7f6a9415bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:15.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 -- 172.21.15.66:0/3598399697 shutdown_connections 2024-09-10T22:03:15.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.800+0000 7f6a9c800640 1 -- 172.21.15.66:0/3598399697 wait complete. 2024-09-10T22:03:15.824 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-10T22:03:15.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 -- 172.21.15.66:0/3029531919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5280cf180 msgr2=0x7fe5280d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 --2- 172.21.15.66:0/3029531919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5280cf180 0x7fe5280d04e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe520009c80 tx=0x7fe52001cd10 comp rx=0 tx=0).stop 2024-09-10T22:03:15.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 -- 172.21.15.66:0/3029531919 shutdown_connections 2024-09-10T22:03:15.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 --2- 172.21.15.66:0/3029531919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5280cf180 0x7fe5280d04e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 --2- 172.21.15.66:0/3029531919 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe52815bd40 0x7fe5280cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 -- 172.21.15.66:0/3029531919 >> 172.21.15.66:0/3029531919 conn(0x7fe5280d1c60 msgr2=0x7fe5280d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:15.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 -- 172.21.15.66:0/3029531919 shutdown_connections 2024-09-10T22:03:15.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 -- 172.21.15.66:0/3029531919 wait complete. 2024-09-10T22:03:15.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 Processor -- start 2024-09-10T22:03:15.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 -- start start 2024-09-10T22:03:15.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5280cf180 0x7fe528144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:15.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5280cf830 0x7fe5280cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:15.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe52815bd40 0x7fe5281f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:15.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe528168ba0 con 0x7fe5280cf830 2024-09-10T22:03:15.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe528168a20 con 0x7fe5280cf180 2024-09-10T22:03:15.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ea00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe528168d20 con 0x7fe52815bd40 2024-09-10T22:03:15.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe51fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5280cf830 0x7fe5280cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:15.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe527e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5280cf180 0x7fe528144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:15.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe51fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5280cf830 0x7fe5280cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59880/0 (socket says 172.21.15.66:59880) 2024-09-10T22:03:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe51fe00640 1 -- 172.21.15.66:0/3101983791 learned_addr learned my addr 172.21.15.66:0/3101983791 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe52ca00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe52815bd40 0x7fe5281f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe51fe00640 1 -- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe52815bd40 msgr2=0x7fe5281f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe51fe00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe52815bd40 0x7fe5281f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe51fe00640 1 -- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5280cf180 msgr2=0x7fe528144340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:15.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe51fe00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5280cf180 0x7fe528144340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:15.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.972+0000 7fe51fe00640 1 -- 172.21.15.66:0/3101983791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5281f8f10 con 0x7fe5280cf830 2024-09-10T22:03:15.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.976+0000 7fe51fe00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5280cf830 0x7fe5280cb670 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c500 tx=0x7fe51000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:15.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.976+0000 7fe524c00640 1 -- 172.21.15.66:0/3101983791 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510017070 con 0x7fe5280cf830 2024-09-10T22:03:15.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.976+0000 7fe524c00640 1 -- 172.21.15.66:0/3101983791 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5100053c0 con 0x7fe5280cf830 2024-09-10T22:03:15.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.976+0000 7fe52ea00640 1 -- 172.21.15.66:0/3101983791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5280cbec0 con 0x7fe5280cf830 2024-09-10T22:03:15.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.976+0000 7fe524c00640 1 -- 172.21.15.66:0/3101983791 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510012890 con 0x7fe5280cf830 2024-09-10T22:03:15.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.976+0000 7fe52ea00640 1 -- 172.21.15.66:0/3101983791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5281f9770 con 0x7fe5280cf830 2024-09-10T22:03:15.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.976+0000 7fe52ea00640 1 -- 172.21.15.66:0/3101983791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5280aec10 con 0x7fe5280cf830 2024-09-10T22:03:15.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.976+0000 7fe524c00640 1 -- 172.21.15.66:0/3101983791 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7fe510003600 con 0x7fe5280cf830 2024-09-10T22:03:15.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.976+0000 7fe524c00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe504072940 0x7fe504074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:15.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.976+0000 7fe524c00640 1 -- 172.21.15.66:0/3101983791 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fe510092e80 con 0x7fe5280cf830 2024-09-10T22:03:15.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.980+0000 7fe527e00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe504072940 0x7fe504074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:15.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.980+0000 7fe527e00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe504072940 0x7fe504074e00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe518009b40 tx=0x7fe5180091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:15.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:15.980+0000 7fe524c00640 1 -- 172.21.15.66:0/3101983791 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe51005d9c0 con 0x7fe5280cf830 2024-09-10T22:03:16.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 -- 172.21.15.66:0/3101983791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fe5280b0df0 con 0x7fe5280cf830 2024-09-10T22:03:16.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe524c00640 1 -- 172.21.15.66:0/3101983791 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v31) ==== 116+0+2 (secure 0 0 0) 0x7fe5100623f0 con 0x7fe5280cf830 2024-09-10T22:03:16.162 INFO:teuthology.orchestra.run.smithi066.stdout:4 2024-09-10T22:03:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 -- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe504072940 msgr2=0x7fe504074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe504072940 0x7fe504074e00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe518009b40 tx=0x7fe5180091e0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 -- 172.21.15.66:0/3101983791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5280cf830 msgr2=0x7fe5280cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5280cf830 0x7fe5280cb670 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c500 tx=0x7fe51000c9c0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 -- 172.21.15.66:0/3101983791 shutdown_connections 2024-09-10T22:03:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe504072940 0x7fe504074e00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe52815bd40 0x7fe5281f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5280cf830 0x7fe5280cb670 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 --2- 172.21.15.66:0/3101983791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5280cf180 0x7fe528144340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 -- 172.21.15.66:0/3101983791 >> 172.21.15.66:0/3101983791 conn(0x7fe5280d1c60 msgr2=0x7fe5280b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:16.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 -- 172.21.15.66:0/3101983791 shutdown_connections 2024-09-10T22:03:16.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.160+0000 7fe52ea00640 1 -- 172.21.15.66:0/3101983791 wait complete. 2024-09-10T22:03:16.183 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.328+0000 7f4b8f200640 1 -- 172.21.15.66:0/904600765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b880c4b10 msgr2=0x7f4b8806e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.328+0000 7f4b8f200640 1 --2- 172.21.15.66:0/904600765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b880c4b10 0x7f4b8806e960 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4b88072560 tx=0x7f4b7801c8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.328+0000 7f4b8f200640 1 -- 172.21.15.66:0/904600765 shutdown_connections 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.328+0000 7f4b8f200640 1 --2- 172.21.15.66:0/904600765 >> v2:172.21.15.94:3300/0 conn(0x7f4b880c4ef0 0x7f4b880c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.328+0000 7f4b8f200640 1 --2- 172.21.15.66:0/904600765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b880c4b10 0x7f4b8806e960 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.328+0000 7f4b8f200640 1 -- 172.21.15.66:0/904600765 >> 172.21.15.66:0/904600765 conn(0x7f4b880cfa30 msgr2=0x7f4b880cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.328+0000 7f4b8f200640 1 -- 172.21.15.66:0/904600765 shutdown_connections 2024-09-10T22:03:16.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.328+0000 7f4b8f200640 1 -- 172.21.15.66:0/904600765 wait complete. 2024-09-10T22:03:16.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b8f200640 1 Processor -- start 2024-09-10T22:03:16.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b8f200640 1 -- start start 2024-09-10T22:03:16.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b8f200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b880ae620 0x7f4b880ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:16.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b8f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b880c4b10 0x7f4b8813cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:16.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b8f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b880c4ef0 0x7f4b8813f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b8f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88167f00 con 0x7f4b880c4b10 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b8f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88167d80 con 0x7f4b880c4ef0 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b8f200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4b88168080 con 0x7f4b880ae620 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b7fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b880c4b10 0x7f4b8813cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b7fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b880c4b10 0x7f4b8813cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59898/0 (socket says 172.21.15.66:59898) 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b7fe00640 1 -- 172.21.15.66:0/4117207247 learned_addr learned my addr 172.21.15.66:0/4117207247 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b8d200640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b880c4ef0 0x7f4b8813f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b87e00640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b880ae620 0x7f4b880ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b7fe00640 1 -- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b880ae620 msgr2=0x7f4b880ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b7fe00640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b880ae620 0x7f4b880ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b7fe00640 1 -- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b880c4ef0 msgr2=0x7f4b8813f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b7fe00640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b880c4ef0 0x7f4b8813f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b7fe00640 1 -- 172.21.15.66:0/4117207247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8813fd60 con 0x7f4b880c4b10 2024-09-10T22:03:16.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b7fe00640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b880c4b10 0x7f4b8813cfd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000c500 tx=0x7f4b7000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:16.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b84c00640 1 -- 172.21.15.66:0/4117207247 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70017070 con 0x7f4b880c4b10 2024-09-10T22:03:16.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b84c00640 1 -- 172.21.15.66:0/4117207247 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b700053c0 con 0x7f4b880c4b10 2024-09-10T22:03:16.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b84c00640 1 -- 172.21.15.66:0/4117207247 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70012890 con 0x7f4b880c4b10 2024-09-10T22:03:16.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b8f200640 1 -- 172.21.15.66:0/4117207247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8813ffe0 con 0x7f4b880c4b10 2024-09-10T22:03:16.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.332+0000 7f4b8f200640 1 -- 172.21.15.66:0/4117207247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881405a0 con 0x7f4b880c4b10 2024-09-10T22:03:16.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.336+0000 7f4b8f200640 1 -- 172.21.15.66:0/4117207247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b8806fae0 con 0x7f4b880c4b10 2024-09-10T22:03:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.336+0000 7f4b84c00640 1 -- 172.21.15.66:0/4117207247 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f4b70003600 con 0x7f4b880c4b10 2024-09-10T22:03:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.336+0000 7f4b84c00640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b64072940 0x7f4b64074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.336+0000 7f4b84c00640 1 -- 172.21.15.66:0/4117207247 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f4b70092e20 con 0x7f4b880c4b10 2024-09-10T22:03:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.336+0000 7f4b87e00640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b64072940 0x7f4b64074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:16.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.340+0000 7f4b87e00640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b64072940 0x7f4b64074e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4b780af000 tx=0x7f4b7801b100 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:16.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.340+0000 7f4b84c00640 1 -- 172.21.15.66:0/4117207247 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4b8813fd60 con 0x7f4b880c4b10 2024-09-10T22:03:16.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.516+0000 7f4b8f200640 1 -- 172.21.15.66:0/4117207247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f4b880cffb0 con 0x7f4b880c4b10 2024-09-10T22:03:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.520+0000 7f4b84c00640 1 -- 172.21.15.66:0/4117207247 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v32)=0 v32) ==== 131+0+0 (secure 0 0 0) 0x7f4b7005d920 con 0x7f4b880c4b10 2024-09-10T22:03:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.524+0000 7f4b8f200640 1 -- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b64072940 msgr2=0x7f4b64074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.524+0000 7f4b8f200640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b64072940 0x7f4b64074e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4b780af000 tx=0x7f4b7801b100 comp rx=0 tx=0).stop 2024-09-10T22:03:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.524+0000 7f4b8f200640 1 -- 172.21.15.66:0/4117207247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b880c4b10 msgr2=0x7f4b8813cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.524+0000 7f4b8f200640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b880c4b10 0x7f4b8813cfd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000c500 tx=0x7f4b7000c9c0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.524+0000 7f4b8f200640 1 -- 172.21.15.66:0/4117207247 shutdown_connections 2024-09-10T22:03:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.524+0000 7f4b8f200640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b64072940 0x7f4b64074e00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.524+0000 7f4b8f200640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b880c4ef0 0x7f4b8813f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.524+0000 7f4b8f200640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b880c4b10 0x7f4b8813cfd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.524+0000 7f4b8f200640 1 --2- 172.21.15.66:0/4117207247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b880ae620 0x7f4b880ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.524+0000 7f4b8f200640 1 -- 172.21.15.66:0/4117207247 >> 172.21.15.66:0/4117207247 conn(0x7f4b880cfa30 msgr2=0x7f4b880af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:16.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.524+0000 7f4b8f200640 1 -- 172.21.15.66:0/4117207247 shutdown_connections 2024-09-10T22:03:16.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.524+0000 7f4b8f200640 1 -- 172.21.15.66:0/4117207247 wait complete. 2024-09-10T22:03:16.541 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:03:16.539+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:03:16.541 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:03:16.539+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:03:16.541 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:03:16.539+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:03:16.541 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:03:16.539+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:03:16.541 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:03:16.539+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:03:16.547 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-10T22:03:16.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.696+0000 7f0cf7000640 1 -- 172.21.15.66:0/2957817474 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cf00ce8d0 msgr2=0x7f0cf00cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.696+0000 7f0cf7000640 1 --2- 172.21.15.66:0/2957817474 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cf00ce8d0 0x7f0cf00cecb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009840 tx=0x7f0ce001c820 comp rx=0 tx=0).stop 2024-09-10T22:03:16.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.696+0000 7f0cf7000640 1 -- 172.21.15.66:0/2957817474 shutdown_connections 2024-09-10T22:03:16.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.696+0000 7f0cf7000640 1 --2- 172.21.15.66:0/2957817474 >> v2:172.21.15.94:3300/0 conn(0x7f0cf00cdf90 0x7f0cf00cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.696+0000 7f0cf7000640 1 --2- 172.21.15.66:0/2957817474 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cf00ce8d0 0x7f0cf00cecb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.696+0000 7f0cf7000640 1 -- 172.21.15.66:0/2957817474 >> 172.21.15.66:0/2957817474 conn(0x7f0cf00d1c60 msgr2=0x7f0cf00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:16.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.696+0000 7f0cf7000640 1 -- 172.21.15.66:0/2957817474 shutdown_connections 2024-09-10T22:03:16.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.696+0000 7f0cf7000640 1 -- 172.21.15.66:0/2957817474 wait complete. 2024-09-10T22:03:16.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.696+0000 7f0cf7000640 1 Processor -- start 2024-09-10T22:03:16.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.696+0000 7f0cf7000640 1 -- start start 2024-09-10T22:03:16.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cf7000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf00cdf90 0x7f0cf00cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:16.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cf7000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf00ce8d0 0x7f0cf014f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:16.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cf7000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cf00cf1f0 0x7f0cf00b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:16.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cf7000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf01691a0 con 0x7f0cf00ce8d0 2024-09-10T22:03:16.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cf7000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0169020 con 0x7f0cf00cdf90 2024-09-10T22:03:16.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cf7000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf0169320 con 0x7f0cf00cf1f0 2024-09-10T22:03:16.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cefe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf00cdf90 0x7f0cf00cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:16.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf00ce8d0 0x7f0cf014f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:16.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cefe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf00cdf90 0x7f0cf00cbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:56900/0 (socket says 172.21.15.66:56900) 2024-09-10T22:03:16.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cefe00640 1 -- 172.21.15.66:0/1142420610 learned_addr learned my addr 172.21.15.66:0/1142420610 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:16.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cf5000640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cf00cf1f0 0x7f0cf00b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cefe00640 1 -- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cf00cf1f0 msgr2=0x7f0cf00b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cefe00640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cf00cf1f0 0x7f0cf00b6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cefe00640 1 -- 172.21.15.66:0/1142420610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf00ce8d0 msgr2=0x7f0cf014f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cefe00640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf00ce8d0 0x7f0cf014f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cefe00640 1 -- 172.21.15.66:0/1142420610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf01f8820 con 0x7f0cf00cdf90 2024-09-10T22:03:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cf5000640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cf00cf1f0 0x7f0cf00b6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0ce7e00640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf00ce8d0 0x7f0cf014f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:03:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cefe00640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf00cdf90 0x7f0cf00cbba0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009790 tx=0x7f0ce00a5b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cecc00640 1 -- 172.21.15.66:0/1142420610 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00b2040 con 0x7f0cf00cdf90 2024-09-10T22:03:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cecc00640 1 -- 172.21.15.66:0/1142420610 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ce0004320 con 0x7f0cf00cdf90 2024-09-10T22:03:16.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cf7000640 1 -- 172.21.15.66:0/1142420610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf014f600 con 0x7f0cf00cdf90 2024-09-10T22:03:16.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cecc00640 1 -- 172.21.15.66:0/1142420610 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00044f0 con 0x7f0cf00cdf90 2024-09-10T22:03:16.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cf7000640 1 -- 172.21.15.66:0/1142420610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf01f7a60 con 0x7f0cf00cdf90 2024-09-10T22:03:16.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.700+0000 7f0cf7000640 1 -- 172.21.15.66:0/1142420610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf00cecb0 con 0x7f0cf00cdf90 2024-09-10T22:03:16.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.704+0000 7f0cecc00640 1 -- 172.21.15.66:0/1142420610 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96700+0+0 (secure 0 0 0) 0x7f0ce00b5030 con 0x7f0cf00cdf90 2024-09-10T22:03:16.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.704+0000 7f0cecc00640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ccc072940 0x7f0ccc074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:16.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.704+0000 7f0cecc00640 1 -- 172.21.15.66:0/1142420610 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f0ce012d100 con 0x7f0cf00cdf90 2024-09-10T22:03:16.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.704+0000 7f0ce7e00640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ccc072940 0x7f0ccc074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:16.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.704+0000 7f0ce7e00640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ccc072940 0x7f0ccc074e00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80096c0 tx=0x7f0cd8009260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:16.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.708+0000 7f0cecc00640 1 -- 172.21.15.66:0/1142420610 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0ce0016410 con 0x7f0cf00cdf90 2024-09-10T22:03:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:16.904+0000 7f0cf7000640 1 -- 172.21.15.66:0/1142420610 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f0cf0070550 con 0x7f0ccc072940 2024-09-10T22:03:18.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.424+0000 7f0cecc00640 1 -- 172.21.15.66:0/1142420610 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 9) ==== 96751+0+0 (secure 0 0 0) 0x7f0ce00f3ec0 con 0x7f0cf00cdf90 2024-09-10T22:03:18.725 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:03:18.723+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:03:18.725 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:03:18.723+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:03:18.725 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:03:18.723+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:03:18.725 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:03:18.723+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:03:18.726 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:03:18.723+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:03:18.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.764+0000 7f0cecc00640 1 -- 172.21.15.66:0/1142420610 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0cf0070550 con 0x7f0ccc072940 2024-09-10T22:03:18.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.768+0000 7f0cf7000640 1 -- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ccc072940 msgr2=0x7f0ccc074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:18.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.768+0000 7f0cf7000640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ccc072940 0x7f0ccc074e00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80096c0 tx=0x7f0cd8009260 comp rx=0 tx=0).stop 2024-09-10T22:03:18.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.768+0000 7f0cf7000640 1 -- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf00cdf90 msgr2=0x7f0cf00cbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:18.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.768+0000 7f0cf7000640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf00cdf90 0x7f0cf00cbba0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009790 tx=0x7f0ce00a5b30 comp rx=0 tx=0).stop 2024-09-10T22:03:18.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.768+0000 7f0cf7000640 1 -- 172.21.15.66:0/1142420610 shutdown_connections 2024-09-10T22:03:18.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.768+0000 7f0cf7000640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ccc072940 0x7f0ccc074e00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.768+0000 7f0cf7000640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0cf00cf1f0 0x7f0cf00b6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.768+0000 7f0cf7000640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf00ce8d0 0x7f0cf014f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.768+0000 7f0cf7000640 1 --2- 172.21.15.66:0/1142420610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf00cdf90 0x7f0cf00cbba0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.768+0000 7f0cf7000640 1 -- 172.21.15.66:0/1142420610 >> 172.21.15.66:0/1142420610 conn(0x7f0cf00d1c60 msgr2=0x7f0cf00af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:18.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.768+0000 7f0cf7000640 1 -- 172.21.15.66:0/1142420610 shutdown_connections 2024-09-10T22:03:18.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.768+0000 7f0cf7000640 1 -- 172.21.15.66:0/1142420610 wait complete. 2024-09-10T22:03:18.795 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:03:18.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.944+0000 7fbbeee00640 1 -- 172.21.15.66:0/3629337846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbcc0052f0 msgr2=0x7fbbcc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:18.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.944+0000 7fbbeee00640 1 --2- 172.21.15.66:0/3629337846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbcc0052f0 0x7fbbcc00b9d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8074240 tx=0x7fbbe001d570 comp rx=0 tx=0).stop 2024-09-10T22:03:18.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.944+0000 7fbbeee00640 1 -- 172.21.15.66:0/3629337846 shutdown_connections 2024-09-10T22:03:18.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.944+0000 7fbbeee00640 1 --2- 172.21.15.66:0/3629337846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbcc0052f0 0x7fbbcc00b9d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.944+0000 7fbbeee00640 1 -- 172.21.15.66:0/3629337846 >> 172.21.15.66:0/3629337846 conn(0x7fbbe80d1c60 msgr2=0x7fbbe80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.944+0000 7fbbeee00640 1 -- 172.21.15.66:0/3629337846 shutdown_connections 2024-09-10T22:03:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbeee00640 1 -- 172.21.15.66:0/3629337846 wait complete. 2024-09-10T22:03:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbeee00640 1 Processor -- start 2024-09-10T22:03:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbeee00640 1 -- start start 2024-09-10T22:03:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbeee00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbbcc0052f0 0x7fbbe81f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbeee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbe80b6b10 0x7fbbe81f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbeee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbe80b6f90 0x7fbbe80b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbeee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe80b32f0 con 0x7fbbe80b6b10 2024-09-10T22:03:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbeee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe80b3170 con 0x7fbbe80b6f90 2024-09-10T22:03:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbeee00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe80b3470 con 0x7fbbcc0052f0 2024-09-10T22:03:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbe80b6b10 0x7fbbe81f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbe80b6b10 0x7fbbe81f5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59930/0 (socket says 172.21.15.66:59930) 2024-09-10T22:03:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbdfe00640 1 -- 172.21.15.66:0/985773592 learned_addr learned my addr 172.21.15.66:0/985773592 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:03:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbe7e00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbbcc0052f0 0x7fbbe81f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbece00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbe80b6f90 0x7fbbe80b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbdfe00640 1 -- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbbcc0052f0 msgr2=0x7fbbe81f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbdfe00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbbcc0052f0 0x7fbbe81f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbdfe00640 1 -- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbe80b6f90 msgr2=0x7fbbe80b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbdfe00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbe80b6f90 0x7fbbe80b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbdfe00640 1 -- 172.21.15.66:0/985773592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe80b7440 con 0x7fbbe80b6b10 2024-09-10T22:03:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbdfe00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbe80b6b10 0x7fbbe81f5350 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0001ec0 tx=0x7fbbd000e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.948+0000 7fbbe4c00640 1 -- 172.21.15.66:0/985773592 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0012850 con 0x7fbbe80b6b10 2024-09-10T22:03:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.952+0000 7fbbe4c00640 1 -- 172.21.15.66:0/985773592 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbd0012ae0 con 0x7fbbe80b6b10 2024-09-10T22:03:18.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.952+0000 7fbbe4c00640 1 -- 172.21.15.66:0/985773592 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd001a400 con 0x7fbbe80b6b10 2024-09-10T22:03:18.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.952+0000 7fbbeee00640 1 -- 172.21.15.66:0/985773592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe80b7720 con 0x7fbbe80b6b10 2024-09-10T22:03:18.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.952+0000 7fbbeee00640 1 -- 172.21.15.66:0/985773592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe806e0b0 con 0x7fbbe80b6b10 2024-09-10T22:03:18.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.952+0000 7fbbeee00640 1 -- 172.21.15.66:0/985773592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe815fce0 con 0x7fbbe80b6b10 2024-09-10T22:03:18.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.952+0000 7fbbe4c00640 1 -- 172.21.15.66:0/985773592 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 9) ==== 96751+0+0 (secure 0 0 0) 0x7fbbd001a5a0 con 0x7fbbe80b6b10 2024-09-10T22:03:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.952+0000 7fbbe4c00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbbc40729d0 0x7fbbc4074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.952+0000 7fbbe4c00640 1 -- 172.21.15.66:0/985773592 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fbbd0096d90 con 0x7fbbe80b6b10 2024-09-10T22:03:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.952+0000 7fbbe7e00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbbc40729d0 0x7fbbc4074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.956+0000 7fbbe7e00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbbc40729d0 0x7fbbc4074e90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8000c10 tx=0x7fbbd8006d20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:18.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:18.956+0000 7fbbe4c00640 1 -- 172.21.15.66:0/985773592 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbbd0016030 con 0x7fbbe80b6b10 2024-09-10T22:03:19.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.156+0000 7fbbeee00640 1 -- 172.21.15.66:0/985773592 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fbbe80d2a40 con 0x7fbbc40729d0 2024-09-10T22:03:19.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbe4c00640 1 -- 172.21.15.66:0/985773592 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fbbe80d2a40 con 0x7fbbc40729d0 2024-09-10T22:03:19.166 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c 2024-09-10T22:03:19.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbeee00640 1 -- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbbc40729d0 msgr2=0x7fbbc4074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:19.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbeee00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbbc40729d0 0x7fbbc4074e90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8000c10 tx=0x7fbbd8006d20 comp rx=0 tx=0).stop 2024-09-10T22:03:19.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbeee00640 1 -- 172.21.15.66:0/985773592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbe80b6b10 msgr2=0x7fbbe81f5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:19.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbeee00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbe80b6b10 0x7fbbe81f5350 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0001ec0 tx=0x7fbbd000e770 comp rx=0 tx=0).stop 2024-09-10T22:03:19.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbeee00640 1 -- 172.21.15.66:0/985773592 shutdown_connections 2024-09-10T22:03:19.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbeee00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbbc40729d0 0x7fbbc4074e90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:19.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbeee00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbe80b6f90 0x7fbbe80b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:19.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbeee00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbe80b6b10 0x7fbbe81f5350 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:19.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbeee00640 1 --2- 172.21.15.66:0/985773592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbbcc0052f0 0x7fbbe81f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:19.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbeee00640 1 -- 172.21.15.66:0/985773592 >> 172.21.15.66:0/985773592 conn(0x7fbbe80d1c60 msgr2=0x7fbbe8071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:19.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbeee00640 1 -- 172.21.15.66:0/985773592 shutdown_connections 2024-09-10T22:03:19.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:03:19.164+0000 7fbbeee00640 1 -- 172.21.15.66:0/985773592 wait complete. 2024-09-10T22:03:19.189 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c 2024-09-10T22:03:19.189 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:03:19.189 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:03:19.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:19.198 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:03:25.372 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:03:25.376 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:03:26.421 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:03:26.421 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:03:26.421 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.15889 s, 150 MB/s 2024-09-10T22:03:26.422 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:03:26.422 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:03:26.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:26.431 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:03:33.371 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:03:33.371 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:03:33.371 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.88194 s, 156 MB/s 2024-09-10T22:03:33.372 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:03:33.372 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:03:33.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:33.381 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:03:40.199 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:03:40.200 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:03:40.200 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.75987 s, 159 MB/s 2024-09-10T22:03:40.201 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:03:40.201 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:03:40.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:40.210 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:03:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:03:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:03:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.84624 s, 157 MB/s 2024-09-10T22:03:47.116 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:03:47.116 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:03:47.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:47.126 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:03:54.092 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:03:54.092 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:03:54.092 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.90994 s, 155 MB/s 2024-09-10T22:03:54.092 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:03:54.093 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:03:54.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:54.102 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:03:55.462 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:03:55.465 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:04:01.061 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:04:01.061 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:04:01.061 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.90396 s, 156 MB/s 2024-09-10T22:04:01.063 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:04:01.063 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:04:01.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:04:01.071 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:04:08.000 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:04:08.000 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:04:08.000 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.87097 s, 156 MB/s 2024-09-10T22:04:08.002 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:04:08.002 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:04:08.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:04:08.011 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:04:14.835 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:04:14.835 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:04:14.836 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.76617 s, 159 MB/s 2024-09-10T22:04:14.837 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:04:14.837 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:04:14.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:04:14.846 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:04:21.773 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:04:21.773 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:04:21.773 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.87193 s, 156 MB/s 2024-09-10T22:04:21.775 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:04:21.775 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:04:21.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:04:21.786 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:04:25.539 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:04:25.542 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:04:28.742 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:04:28.742 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:04:28.742 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.90076 s, 156 MB/s 2024-09-10T22:04:28.744 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-10T22:04:28.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.891+0000 7f18e9800640 1 -- 172.21.15.66:0/3807884805 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18e415a3b0 msgr2=0x7f18e40b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:28.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.891+0000 7f18e9800640 1 --2- 172.21.15.66:0/3807884805 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18e415a3b0 0x7f18e40b2cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009870 tx=0x7f18d401c830 comp rx=0 tx=0).stop 2024-09-10T22:04:28.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.891+0000 7f18e9800640 1 -- 172.21.15.66:0/3807884805 shutdown_connections 2024-09-10T22:04:28.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.891+0000 7f18e9800640 1 --2- 172.21.15.66:0/3807884805 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18e415a3b0 0x7f18e40b2cc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:28.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.891+0000 7f18e9800640 1 -- 172.21.15.66:0/3807884805 >> 172.21.15.66:0/3807884805 conn(0x7f18e40cfaa0 msgr2=0x7f18e40cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:28.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.891+0000 7f18e9800640 1 -- 172.21.15.66:0/3807884805 shutdown_connections 2024-09-10T22:04:28.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.891+0000 7f18e9800640 1 -- 172.21.15.66:0/3807884805 wait complete. 2024-09-10T22:04:28.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e9800640 1 Processor -- start 2024-09-10T22:04:28.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e9800640 1 -- start start 2024-09-10T22:04:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e9800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18e4070730 0x7f18e41460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e9800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18e4072d50 0x7f18e41f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e415a3b0 0x7f18e4154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e9800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4169290 con 0x7f18e415a3b0 2024-09-10T22:04:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e9800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4169110 con 0x7f18e4072d50 2024-09-10T22:04:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e9800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f18e4169410 con 0x7f18e4070730 2024-09-10T22:04:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e415a3b0 0x7f18e4154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e2a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18e4070730 0x7f18e41460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e415a3b0 0x7f18e4154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45746/0 (socket says 172.21.15.66:45746) 2024-09-10T22:04:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e3400640 1 -- 172.21.15.66:0/3643807974 learned_addr learned my addr 172.21.15.66:0/3643807974 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e2000640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18e4072d50 0x7f18e41f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e2a00640 1 -- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18e4072d50 msgr2=0x7f18e41f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e2a00640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18e4072d50 0x7f18e41f6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e2a00640 1 -- 172.21.15.66:0/3643807974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e415a3b0 msgr2=0x7f18e4154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e2a00640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e415a3b0 0x7f18e4154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e2a00640 1 -- 172.21.15.66:0/3643807974 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e4146610 con 0x7f18e4070730 2024-09-10T22:04:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e3400640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e415a3b0 0x7f18e4154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:04:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e2000640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18e4072d50 0x7f18e41f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:04:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e2a00640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18e4070730 0x7f18e41460d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18d40ac000 tx=0x7f18d4005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18daa00640 1 -- 172.21.15.66:0/3643807974 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d40b2040 con 0x7f18e4070730 2024-09-10T22:04:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18daa00640 1 -- 172.21.15.66:0/3643807974 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18d40043f0 con 0x7f18e4070730 2024-09-10T22:04:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18daa00640 1 -- 172.21.15.66:0/3643807974 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4003840 con 0x7f18e4070730 2024-09-10T22:04:28.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e9800640 1 -- 172.21.15.66:0/3643807974 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e41f77c0 con 0x7f18e4070730 2024-09-10T22:04:28.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e9800640 1 -- 172.21.15.66:0/3643807974 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e41f7d80 con 0x7f18e4070730 2024-09-10T22:04:28.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.895+0000 7f18e9800640 1 -- 172.21.15.66:0/3643807974 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e40b2cc0 con 0x7f18e4070730 2024-09-10T22:04:28.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.899+0000 7f18daa00640 1 -- 172.21.15.66:0/3643807974 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 10) ==== 96802+0+0 (secure 0 0 0) 0x7f18d40b5070 con 0x7f18e4070730 2024-09-10T22:04:28.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.899+0000 7f18daa00640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f18c4072a90 0x7f18c4074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:28.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.899+0000 7f18daa00640 1 -- 172.21.15.66:0/3643807974 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f18d412cd20 con 0x7f18e4070730 2024-09-10T22:04:28.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.899+0000 7f18e2000640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f18c4072a90 0x7f18c4074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:28.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.899+0000 7f18e2000640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f18c4072a90 0x7f18c4074f50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f18e40ae9e0 tx=0x7f18d0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:28.903+0000 7f18daa00640 1 -- 172.21.15.66:0/3643807974 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f18d4016450 con 0x7f18e4070730 2024-09-10T22:04:29.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.107+0000 7f18e9800640 1 -- 172.21.15.66:0/3643807974 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f18e40d0700 con 0x7f18c4072a90 2024-09-10T22:04:29.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.123+0000 7f18daa00640 1 -- 172.21.15.66:0/3643807974 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f18e40d0700 con 0x7f18c4072a90 2024-09-10T22:04:29.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.127+0000 7f18e9800640 1 -- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f18c4072a90 msgr2=0x7f18c4074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:29.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.127+0000 7f18e9800640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f18c4072a90 0x7f18c4074f50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f18e40ae9e0 tx=0x7f18d0009040 comp rx=0 tx=0).stop 2024-09-10T22:04:29.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.127+0000 7f18e9800640 1 -- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18e4070730 msgr2=0x7f18e41460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:29.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.127+0000 7f18e9800640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18e4070730 0x7f18e41460d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18d40ac000 tx=0x7f18d4005db0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.127+0000 7f18e9800640 1 -- 172.21.15.66:0/3643807974 shutdown_connections 2024-09-10T22:04:29.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.127+0000 7f18e9800640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f18c4072a90 0x7f18c4074f50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.127+0000 7f18e9800640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e415a3b0 0x7f18e4154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.127+0000 7f18e9800640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18e4072d50 0x7f18e41f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.127+0000 7f18e9800640 1 --2- 172.21.15.66:0/3643807974 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18e4070730 0x7f18e41460d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.127+0000 7f18e9800640 1 -- 172.21.15.66:0/3643807974 >> 172.21.15.66:0/3643807974 conn(0x7f18e40cfaa0 msgr2=0x7f18e406e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:29.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.127+0000 7f18e9800640 1 -- 172.21.15.66:0/3643807974 shutdown_connections 2024-09-10T22:04:29.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.127+0000 7f18e9800640 1 -- 172.21.15.66:0/3643807974 wait complete. 2024-09-10T22:04:29.150 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:04:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.295+0000 7f6663200640 1 -- 172.21.15.66:0/1617841869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c15a310 msgr2=0x7f665c0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.295+0000 7f6663200640 1 --2- 172.21.15.66:0/1617841869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c15a310 0x7f665c0c48b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f664c00a950 tx=0x7f664c01dae0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.295+0000 7f6663200640 1 -- 172.21.15.66:0/1617841869 shutdown_connections 2024-09-10T22:04:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.295+0000 7f6663200640 1 --2- 172.21.15.66:0/1617841869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c15a310 0x7f665c0c48b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.295+0000 7f6663200640 1 -- 172.21.15.66:0/1617841869 >> 172.21.15.66:0/1617841869 conn(0x7f665c0cfa30 msgr2=0x7f665c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.295+0000 7f6663200640 1 -- 172.21.15.66:0/1617841869 shutdown_connections 2024-09-10T22:04:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.295+0000 7f6663200640 1 -- 172.21.15.66:0/1617841869 wait complete. 2024-09-10T22:04:29.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.295+0000 7f6663200640 1 Processor -- start 2024-09-10T22:04:29.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.295+0000 7f6663200640 1 -- start start 2024-09-10T22:04:29.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6663200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f665c0aeec0 0x7f665c13c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6663200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c15a310 0x7f665c0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6663200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f665c148580 0x7f665c0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6663200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f665c168750 con 0x7f665c15a310 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6663200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1685d0 con 0x7f665c0aeec0 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6663200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f665c1688d0 con 0x7f665c148580 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f665b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c15a310 0x7f665c0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f665b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c15a310 0x7f665c0afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45772/0 (socket says 172.21.15.66:45772) 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f665b400640 1 -- 172.21.15.66:0/2379165412 learned_addr learned my addr 172.21.15.66:0/2379165412 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6661200640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f665c148580 0x7f665c0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f665be00640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f665c0aeec0 0x7f665c13c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f665b400640 1 -- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f665c148580 msgr2=0x7f665c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f665b400640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f665c148580 0x7f665c0ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f665b400640 1 -- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f665c0aeec0 msgr2=0x7f665c13c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:29.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f665b400640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f665c0aeec0 0x7f665c13c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f665b400640 1 -- 172.21.15.66:0/2379165412 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c0b0490 con 0x7f665c15a310 2024-09-10T22:04:29.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6661200640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f665c148580 0x7f665c0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:04:29.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f665be00640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f665c0aeec0 0x7f665c13c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:04:29.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f665b400640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c15a310 0x7f665c0afd20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f664800c420 tx=0x7f664800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:29.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6653e00640 1 -- 172.21.15.66:0/2379165412 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648017070 con 0x7f665c15a310 2024-09-10T22:04:29.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6653e00640 1 -- 172.21.15.66:0/2379165412 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6648003600 con 0x7f665c15a310 2024-09-10T22:04:29.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6653e00640 1 -- 172.21.15.66:0/2379165412 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648003a50 con 0x7f665c15a310 2024-09-10T22:04:29.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6663200640 1 -- 172.21.15.66:0/2379165412 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c0b0770 con 0x7f665c15a310 2024-09-10T22:04:29.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6663200640 1 -- 172.21.15.66:0/2379165412 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c0ade00 con 0x7f665c15a310 2024-09-10T22:04:29.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.299+0000 7f6663200640 1 -- 172.21.15.66:0/2379165412 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665c0c48b0 con 0x7f665c15a310 2024-09-10T22:04:29.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.303+0000 7f6653e00640 1 -- 172.21.15.66:0/2379165412 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 10) ==== 96802+0+0 (secure 0 0 0) 0x7f6648003bf0 con 0x7f665c15a310 2024-09-10T22:04:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.303+0000 7f6653e00640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f663c072a60 0x7f663c074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.303+0000 7f6653e00640 1 -- 172.21.15.66:0/2379165412 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f6648093210 con 0x7f665c15a310 2024-09-10T22:04:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.303+0000 7f665be00640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f663c072a60 0x7f663c074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:29.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.303+0000 7f665be00640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f663c072a60 0x7f663c074f20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f664c0b0000 tx=0x7f664c01c320 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.307+0000 7f6653e00640 1 -- 172.21.15.66:0/2379165412 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f664805e0b0 con 0x7f665c15a310 2024-09-10T22:04:29.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.507+0000 7f6663200640 1 -- 172.21.15.66:0/2379165412 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f665c070420 con 0x7f663c072a60 2024-09-10T22:04:29.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6653e00640 1 -- 172.21.15.66:0/2379165412 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f665c070420 con 0x7f663c072a60 2024-09-10T22:04:29.519 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c 2024-09-10T22:04:29.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6663200640 1 -- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f663c072a60 msgr2=0x7f663c074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:29.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6663200640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f663c072a60 0x7f663c074f20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f664c0b0000 tx=0x7f664c01c320 comp rx=0 tx=0).stop 2024-09-10T22:04:29.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6663200640 1 -- 172.21.15.66:0/2379165412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c15a310 msgr2=0x7f665c0afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:29.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6663200640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c15a310 0x7f665c0afd20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f664800c420 tx=0x7f664800c8e0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6663200640 1 -- 172.21.15.66:0/2379165412 shutdown_connections 2024-09-10T22:04:29.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6663200640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f663c072a60 0x7f663c074f20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6663200640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f665c148580 0x7f665c0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6663200640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c15a310 0x7f665c0afd20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6663200640 1 --2- 172.21.15.66:0/2379165412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f665c0aeec0 0x7f665c13c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:29.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6663200640 1 -- 172.21.15.66:0/2379165412 >> 172.21.15.66:0/2379165412 conn(0x7f665c0cfa30 msgr2=0x7f665c073d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:29.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6663200640 1 -- 172.21.15.66:0/2379165412 shutdown_connections 2024-09-10T22:04:29.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:29.515+0000 7f6663200640 1 -- 172.21.15.66:0/2379165412 wait complete. 2024-09-10T22:04:29.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:04:29.541 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c 2024-09-10T22:04:29.541 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:04:29.557 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c 2024-09-10T22:04:29.557 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="10654449664" 2024-09-10T22:04:29.558 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:04:29.558 INFO:tasks.cephfs.test_volumes:r_size = 10654449664 exp_size = 10737418240 2024-09-10T22:04:31.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:04:31.559 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c 2024-09-10T22:04:31.559 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:04:31.574 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/4db39710-6aff-4f48-9e8e-9c1c2a6f7a8c 2024-09-10T22:04:31.574 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="10737418240" 2024-09-10T22:04:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:04:31.575 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-10T22:04:31.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone9020 2024-09-10T22:04:31.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.771+0000 7f7ee5e00640 1 -- 172.21.15.66:0/1378879696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee00cd7a0 msgr2=0x7f7ee00cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:31.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.771+0000 7f7ee5e00640 1 --2- 172.21.15.66:0/1378879696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee00cd7a0 0x7f7ee00cdc30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009c80 tx=0x7f7ed801cda0 comp rx=0 tx=0).stop 2024-09-10T22:04:31.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.771+0000 7f7ee5e00640 1 -- 172.21.15.66:0/1378879696 shutdown_connections 2024-09-10T22:04:31.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.771+0000 7f7ee5e00640 1 --2- 172.21.15.66:0/1378879696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee00cd7a0 0x7f7ee00cdc30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:31.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.771+0000 7f7ee5e00640 1 --2- 172.21.15.66:0/1378879696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ee015ef20 0x7f7ee00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:31.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.771+0000 7f7ee5e00640 1 -- 172.21.15.66:0/1378879696 >> 172.21.15.66:0/1378879696 conn(0x7f7ee00d1b10 msgr2=0x7f7ee00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:31.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.771+0000 7f7ee5e00640 1 -- 172.21.15.66:0/1378879696 shutdown_connections 2024-09-10T22:04:31.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.771+0000 7f7ee5e00640 1 -- 172.21.15.66:0/1378879696 wait complete. 2024-09-10T22:04:31.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.771+0000 7f7ee5e00640 1 Processor -- start 2024-09-10T22:04:31.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.771+0000 7f7ee5e00640 1 -- start start 2024-09-10T22:04:31.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7ee5e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ee00cd7a0 0x7f7ee015b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:31.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7ee5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee015ef20 0x7f7ee006f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:31.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7ee5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ee015f300 0x7f7ee006f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:31.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7ee5e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee0168ee0 con 0x7f7ee015ef20 2024-09-10T22:04:31.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7ee5e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee0168d60 con 0x7f7ee015f300 2024-09-10T22:04:31.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7ee5e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee0169060 con 0x7f7ee00cd7a0 2024-09-10T22:04:31.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee015ef20 0x7f7ee006f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:31.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee015ef20 0x7f7ee006f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45776/0 (socket says 172.21.15.66:45776) 2024-09-10T22:04:31.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7edfe00640 1 -- 172.21.15.66:0/3024121364 learned_addr learned my addr 172.21.15.66:0/3024121364 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:31.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7ee4a00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ee00cd7a0 0x7f7ee015b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:31.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7ee5400640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ee015f300 0x7f7ee006f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7edfe00640 1 -- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ee00cd7a0 msgr2=0x7f7ee015b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7edfe00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ee00cd7a0 0x7f7ee015b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7edfe00640 1 -- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ee015f300 msgr2=0x7f7ee006f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7edfe00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ee015f300 0x7f7ee006f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7edfe00640 1 -- 172.21.15.66:0/3024121364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee00ca0c0 con 0x7f7ee015ef20 2024-09-10T22:04:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7edfe00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee015ef20 0x7f7ee006f320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc001ec0 tx=0x7f7ecc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7edcc00640 1 -- 172.21.15.66:0/3024121364 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc017070 con 0x7f7ee015ef20 2024-09-10T22:04:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7edcc00640 1 -- 172.21.15.66:0/3024121364 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ecc003600 con 0x7f7ee015ef20 2024-09-10T22:04:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7ee5e00640 1 -- 172.21.15.66:0/3024121364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee00ca380 con 0x7f7ee015ef20 2024-09-10T22:04:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7edcc00640 1 -- 172.21.15.66:0/3024121364 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc003a50 con 0x7f7ee015ef20 2024-09-10T22:04:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7ee5e00640 1 -- 172.21.15.66:0/3024121364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee00ca940 con 0x7f7ee015ef20 2024-09-10T22:04:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.775+0000 7f7ee5e00640 1 -- 172.21.15.66:0/3024121364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee00cdc30 con 0x7f7ee015ef20 2024-09-10T22:04:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.779+0000 7f7edcc00640 1 -- 172.21.15.66:0/3024121364 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 10) ==== 96802+0+0 (secure 0 0 0) 0x7f7ecc003bf0 con 0x7f7ee015ef20 2024-09-10T22:04:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.779+0000 7f7edcc00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ec0072a60 0x7f7ec0074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.779+0000 7f7edcc00640 1 -- 172.21.15.66:0/3024121364 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f7ecc093120 con 0x7f7ee015ef20 2024-09-10T22:04:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.779+0000 7f7ee4a00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ec0072a60 0x7f7ec0074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.779+0000 7f7ee4a00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ec0072a60 0x7f7ec0074f20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0075930 tx=0x7f7ed400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:31.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.783+0000 7f7edcc00640 1 -- 172.21.15.66:0/3024121364 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7ecc05d9f0 con 0x7f7ee015ef20 2024-09-10T22:04:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:31.991+0000 7f7ee5e00640 1 -- 172.21.15.66:0/3024121364 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone9020", "target": ["mon-mgr", ""]}) -- 0x7f7ee015c3d0 con 0x7f7ec0072a60 2024-09-10T22:04:32.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7edcc00640 1 -- 172.21.15.66:0/3024121364 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7ee015c3d0 con 0x7f7ec0072a60 2024-09-10T22:04:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7ee5e00640 1 -- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ec0072a60 msgr2=0x7f7ec0074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7ee5e00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ec0072a60 0x7f7ec0074f20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0075930 tx=0x7f7ed400b390 comp rx=0 tx=0).stop 2024-09-10T22:04:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7ee5e00640 1 -- 172.21.15.66:0/3024121364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee015ef20 msgr2=0x7f7ee006f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7ee5e00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee015ef20 0x7f7ee006f320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc001ec0 tx=0x7f7ecc00c890 comp rx=0 tx=0).stop 2024-09-10T22:04:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7ee5e00640 1 -- 172.21.15.66:0/3024121364 shutdown_connections 2024-09-10T22:04:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7ee5e00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ec0072a60 0x7f7ec0074f20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7ee5e00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ee015f300 0x7f7ee006f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7ee5e00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee015ef20 0x7f7ee006f320 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7ee5e00640 1 --2- 172.21.15.66:0/3024121364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ee00cd7a0 0x7f7ee015b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7ee5e00640 1 -- 172.21.15.66:0/3024121364 >> 172.21.15.66:0/3024121364 conn(0x7f7ee00d1b10 msgr2=0x7f7ee00af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7ee5e00640 1 -- 172.21.15.66:0/3024121364 shutdown_connections 2024-09-10T22:04:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.063+0000 7f7ee5e00640 1 -- 172.21.15.66:0/3024121364 wait complete. 2024-09-10T22:04:32.071 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:32.066+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:32.071 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:32.066+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:32.071 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:32.066+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:32.071 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:32.066+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:32.071 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:32.066+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:32.088 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:32.082+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:32.088 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:32.082+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:32.088 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:32.082+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:32.088 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:32.082+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:32.088 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:32.082+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:32.090 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone24791 2024-09-10T22:04:32.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.239+0000 7fc799200640 1 -- 172.21.15.66:0/776942914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc79415f340 msgr2=0x7fc7940cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:32.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.239+0000 7fc799200640 1 --2- 172.21.15.66:0/776942914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc79415f340 0x7fc7940cfb50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc784009930 tx=0x7fc78401c8b0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.239+0000 7fc799200640 1 -- 172.21.15.66:0/776942914 shutdown_connections 2024-09-10T22:04:32.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.239+0000 7fc799200640 1 --2- 172.21.15.66:0/776942914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc79415f340 0x7fc7940cfb50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.239+0000 7fc799200640 1 -- 172.21.15.66:0/776942914 >> 172.21.15.66:0/776942914 conn(0x7fc7940d1c50 msgr2=0x7fc7940d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:32.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.239+0000 7fc799200640 1 -- 172.21.15.66:0/776942914 shutdown_connections 2024-09-10T22:04:32.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.239+0000 7fc799200640 1 -- 172.21.15.66:0/776942914 wait complete. 2024-09-10T22:04:32.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc799200640 1 Processor -- start 2024-09-10T22:04:32.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc799200640 1 -- start start 2024-09-10T22:04:32.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc799200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7940af970 0x7fc79414dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:32.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc799200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc7940b2bf0 0x7fc79415d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:32.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc799200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc79415f340 0x7fc7940cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:32.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc799200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc79416a7a0 con 0x7fc7940af970 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc799200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc79416a620 con 0x7fc79415f340 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc799200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc79416a920 con 0x7fc7940b2bf0 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc791e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7940af970 0x7fc79414dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc791e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7940af970 0x7fc79414dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45800/0 (socket says 172.21.15.66:45800) 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc791e00640 1 -- 172.21.15.66:0/2896756482 learned_addr learned my addr 172.21.15.66:0/2896756482 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc792800640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc79415f340 0x7fc7940cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc78be00640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc7940b2bf0 0x7fc79415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc791e00640 1 -- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc7940b2bf0 msgr2=0x7fc79415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc791e00640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc7940b2bf0 0x7fc79415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc791e00640 1 -- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc79415f340 msgr2=0x7fc7940cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc791e00640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc79415f340 0x7fc7940cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc791e00640 1 -- 172.21.15.66:0/2896756482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7940cf550 con 0x7fc7940af970 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc791e00640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7940af970 0x7fc79414dc30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc7840194c0 tx=0x7fc7840a57e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:32.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc78a000640 1 -- 172.21.15.66:0/2896756482 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7840bd070 con 0x7fc7940af970 2024-09-10T22:04:32.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc78a000640 1 -- 172.21.15.66:0/2896756482 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7840044b0 con 0x7fc7940af970 2024-09-10T22:04:32.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc78a000640 1 -- 172.21.15.66:0/2896756482 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7840b2040 con 0x7fc7940af970 2024-09-10T22:04:32.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc799200640 1 -- 172.21.15.66:0/2896756482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7940ceff0 con 0x7fc7940af970 2024-09-10T22:04:32.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.243+0000 7fc799200640 1 -- 172.21.15.66:0/2896756482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79415cee0 con 0x7fc7940af970 2024-09-10T22:04:32.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.247+0000 7fc799200640 1 -- 172.21.15.66:0/2896756482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7940cfb50 con 0x7fc7940af970 2024-09-10T22:04:32.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.247+0000 7fc78a000640 1 -- 172.21.15.66:0/2896756482 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 10) ==== 96802+0+0 (secure 0 0 0) 0x7fc784003860 con 0x7fc7940af970 2024-09-10T22:04:32.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.247+0000 7fc78a000640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc770072a60 0x7fc770074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:32.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.247+0000 7fc78a000640 1 -- 172.21.15.66:0/2896756482 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fc78412dac0 con 0x7fc7940af970 2024-09-10T22:04:32.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.247+0000 7fc78be00640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc770072a60 0x7fc770074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:32.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.251+0000 7fc78be00640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc770072a60 0x7fc770074f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc794167da0 tx=0x7fc77c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:32.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.251+0000 7fc78a000640 1 -- 172.21.15.66:0/2896756482 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc7940ceff0 con 0x7fc7940af970 2024-09-10T22:04:32.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.455+0000 7fc799200640 1 -- 172.21.15.66:0/2896756482 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone24791", "target": ["mon-mgr", ""]}) -- 0x7fc7940b42c0 con 0x7fc770072a60 2024-09-10T22:04:32.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:32.583+0000 7fc78a000640 1 -- 172.21.15.66:0/2896756482 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 11) ==== 96904+0+0 (secure 0 0 0) 0x7fc7840f44c0 con 0x7fc7940af970 2024-09-10T22:04:33.072 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:33.066+0000 7f5fd5200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:33.072 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:33.066+0000 7f5fd5200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:33.072 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:33.066+0000 7f5fd5200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:33.072 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:33.066+0000 7f5fd5200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:33.072 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:33.066+0000 7f5fd5200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:33.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.195+0000 7fc78a000640 1 -- 172.21.15.66:0/2896756482 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc7940b42c0 con 0x7fc770072a60 2024-09-10T22:04:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.199+0000 7fc799200640 1 -- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc770072a60 msgr2=0x7fc770074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.199+0000 7fc799200640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc770072a60 0x7fc770074f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc794167da0 tx=0x7fc77c009040 comp rx=0 tx=0).stop 2024-09-10T22:04:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.199+0000 7fc799200640 1 -- 172.21.15.66:0/2896756482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7940af970 msgr2=0x7fc79414dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.199+0000 7fc799200640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7940af970 0x7fc79414dc30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc7840194c0 tx=0x7fc7840a57e0 comp rx=0 tx=0).stop 2024-09-10T22:04:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.203+0000 7fc799200640 1 -- 172.21.15.66:0/2896756482 shutdown_connections 2024-09-10T22:04:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.203+0000 7fc799200640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc770072a60 0x7fc770074f20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.203+0000 7fc799200640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc79415f340 0x7fc7940cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.203+0000 7fc799200640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc7940b2bf0 0x7fc79415d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.203+0000 7fc799200640 1 --2- 172.21.15.66:0/2896756482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7940af970 0x7fc79414dc30 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.203+0000 7fc799200640 1 -- 172.21.15.66:0/2896756482 >> 172.21.15.66:0/2896756482 conn(0x7fc7940d1c50 msgr2=0x7fc794072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:33.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.203+0000 7fc799200640 1 -- 172.21.15.66:0/2896756482 shutdown_connections 2024-09-10T22:04:33.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.203+0000 7fc799200640 1 -- 172.21.15.66:0/2896756482 wait complete. 2024-09-10T22:04:33.248 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone21312 2024-09-10T22:04:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.487+0000 7f393ca00640 1 -- 172.21.15.66:0/2470331544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f391c003a60 msgr2=0x7f391c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.487+0000 7f393ca00640 1 --2- 172.21.15.66:0/2470331544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f391c003a60 0x7f391c003e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3928005fd0 tx=0x7f392801cdc0 comp rx=0 tx=0).stop 2024-09-10T22:04:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.487+0000 7f3936800640 1 -- 172.21.15.66:0/2470331544 reap_dead start 2024-09-10T22:04:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.487+0000 7f393ca00640 1 -- 172.21.15.66:0/2470331544 shutdown_connections 2024-09-10T22:04:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.487+0000 7f393ca00640 1 -- 172.21.15.66:0/2470331544 >> 172.21.15.66:0/2470331544 conn(0x7f39380d1c60 msgr2=0x7f39380d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.491+0000 7f393ca00640 1 -- 172.21.15.66:0/2470331544 shutdown_connections 2024-09-10T22:04:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.491+0000 7f393ca00640 1 -- 172.21.15.66:0/2470331544 wait complete. 2024-09-10T22:04:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.495+0000 7f393ca00640 1 Processor -- start 2024-09-10T22:04:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.495+0000 7f393ca00640 1 -- start start 2024-09-10T22:04:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.495+0000 7f393ca00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f391c003a60 0x7f39381675a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.495+0000 7f393ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938161020 0x7f39381f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.495+0000 7f393ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3938070ec0 0x7f3938158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.495+0000 7f393ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3938168e30 con 0x7f3938070ec0 2024-09-10T22:04:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.495+0000 7f393ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3938168cb0 con 0x7f3938161020 2024-09-10T22:04:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.495+0000 7f393ca00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3938168fb0 con 0x7f391c003a60 2024-09-10T22:04:33.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.499+0000 7f3936800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3938070ec0 0x7f3938158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:33.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.499+0000 7f3936800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3938070ec0 0x7f3938158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53038/0 (socket says 172.21.15.66:53038) 2024-09-10T22:04:33.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.499+0000 7f3936800640 1 -- 172.21.15.66:0/4221647779 learned_addr learned my addr 172.21.15.66:0/4221647779 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:33.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.499+0000 7f3935400640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938161020 0x7f39381f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:33.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.499+0000 7f3936800640 1 -- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f391c003a60 msgr2=0x7f39381675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.499+0000 7f3936800640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f391c003a60 0x7f39381675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.499+0000 7f3936800640 1 -- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938161020 msgr2=0x7f39381f8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.499+0000 7f3936800640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938161020 0x7f39381f8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.499+0000 7f3936800640 1 -- 172.21.15.66:0/4221647779 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39381614b0 con 0x7f3938070ec0 2024-09-10T22:04:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.499+0000 7f3936800640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3938070ec0 0x7f3938158880 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3930001340 tx=0x7f393000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:33.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.503+0000 7f392e000640 1 -- 172.21.15.66:0/4221647779 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930015070 con 0x7f3938070ec0 2024-09-10T22:04:33.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.503+0000 7f392e000640 1 -- 172.21.15.66:0/4221647779 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f393000cd10 con 0x7f3938070ec0 2024-09-10T22:04:33.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.503+0000 7f393ca00640 1 -- 172.21.15.66:0/4221647779 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39380c95c0 con 0x7f3938070ec0 2024-09-10T22:04:33.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.503+0000 7f392e000640 1 -- 172.21.15.66:0/4221647779 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39300116c0 con 0x7f3938070ec0 2024-09-10T22:04:33.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.503+0000 7f393ca00640 1 -- 172.21.15.66:0/4221647779 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39380b7830 con 0x7f3938070ec0 2024-09-10T22:04:33.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.503+0000 7f393ca00640 1 -- 172.21.15.66:0/4221647779 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39100052a0 con 0x7f3938070ec0 2024-09-10T22:04:33.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.511+0000 7f392e000640 1 -- 172.21.15.66:0/4221647779 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 11) ==== 96904+0+0 (secure 0 0 0) 0x7f393001a450 con 0x7f3938070ec0 2024-09-10T22:04:33.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.511+0000 7f392e000640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3918072bd0 0x7f3918075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:33.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.511+0000 7f392e000640 1 -- 172.21.15.66:0/4221647779 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f3930094840 con 0x7f3938070ec0 2024-09-10T22:04:33.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.519+0000 7f392e000640 1 -- 172.21.15.66:0/4221647779 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3930001a40 con 0x7f3938070ec0 2024-09-10T22:04:33.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.531+0000 7f3935e00640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3918072bd0 0x7f3918075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:33.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.555+0000 7f3935e00640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3918072bd0 0x7f3918075090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f39280b0000 tx=0x7f391c005840 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:33.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:33.871+0000 7f393ca00640 1 -- 172.21.15.66:0/4221647779 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone21312", "target": ["mon-mgr", ""]}) -- 0x7f3910000c90 con 0x7f3918072bd0 2024-09-10T22:04:34.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.427+0000 7f392e000640 1 -- 172.21.15.66:0/4221647779 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3910000c90 con 0x7f3918072bd0 2024-09-10T22:04:34.435 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:34.430+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:34.435 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:34.430+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:34.435 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:34.430+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:34.436 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:34.430+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:34.436 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:34.430+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.431+0000 7f393ca00640 1 -- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3918072bd0 msgr2=0x7f3918075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.431+0000 7f393ca00640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3918072bd0 0x7f3918075090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f39280b0000 tx=0x7f391c005840 comp rx=0 tx=0).stop 2024-09-10T22:04:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.431+0000 7f393ca00640 1 -- 172.21.15.66:0/4221647779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3938070ec0 msgr2=0x7f3938158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.431+0000 7f393ca00640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3938070ec0 0x7f3938158880 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3930001340 tx=0x7f393000ed10 comp rx=0 tx=0).stop 2024-09-10T22:04:34.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.431+0000 7f393ca00640 1 -- 172.21.15.66:0/4221647779 shutdown_connections 2024-09-10T22:04:34.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.431+0000 7f393ca00640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3918072bd0 0x7f3918075090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:34.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.431+0000 7f393ca00640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3938070ec0 0x7f3938158880 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:34.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.431+0000 7f393ca00640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938161020 0x7f39381f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:34.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.431+0000 7f393ca00640 1 --2- 172.21.15.66:0/4221647779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f391c003a60 0x7f39381675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:34.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.431+0000 7f393ca00640 1 -- 172.21.15.66:0/4221647779 >> 172.21.15.66:0/4221647779 conn(0x7f39380d1c60 msgr2=0x7f3938071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:34.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.431+0000 7f393ca00640 1 -- 172.21.15.66:0/4221647779 shutdown_connections 2024-09-10T22:04:34.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.431+0000 7f393ca00640 1 -- 172.21.15.66:0/4221647779 wait complete. 2024-09-10T22:04:34.477 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone27303 2024-09-10T22:04:34.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.735+0000 7f2bdea00640 1 -- 172.21.15.66:0/3842431015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bd80d06b0 msgr2=0x7f2bd80ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:34.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.735+0000 7f2bdea00640 1 --2- 172.21.15.66:0/3842431015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bd80d06b0 0x7f2bd80ce170 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009840 tx=0x7f2bc401c790 comp rx=0 tx=0).stop 2024-09-10T22:04:34.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.735+0000 7f2bdea00640 1 -- 172.21.15.66:0/3842431015 shutdown_connections 2024-09-10T22:04:34.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.735+0000 7f2bdea00640 1 --2- 172.21.15.66:0/3842431015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bd80d06b0 0x7f2bd80ce170 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:34.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.735+0000 7f2bdea00640 1 --2- 172.21.15.66:0/3842431015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bd80afc20 0x7f2bd80ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:34.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.735+0000 7f2bdea00640 1 -- 172.21.15.66:0/3842431015 >> 172.21.15.66:0/3842431015 conn(0x7f2bd80d1c60 msgr2=0x7f2bd80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:34.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.735+0000 7f2bdea00640 1 -- 172.21.15.66:0/3842431015 shutdown_connections 2024-09-10T22:04:34.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.735+0000 7f2bdea00640 1 -- 172.21.15.66:0/3842431015 wait complete. 2024-09-10T22:04:34.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bdea00640 1 Processor -- start 2024-09-10T22:04:34.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bdea00640 1 -- start start 2024-09-10T22:04:34.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bd80afc20 0x7f2bd806dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:34.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd80d06b0 0x7f2bd81fc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:34.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bd815bd40 0x7f2bd8165c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:34.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bdea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81692d0 con 0x7f2bd80afc20 2024-09-10T22:04:34.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bdea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8169150 con 0x7f2bd80d06b0 2024-09-10T22:04:34.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bdea00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd8169450 con 0x7f2bd815bd40 2024-09-10T22:04:34.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bd80afc20 0x7f2bd806dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:34.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bd80afc20 0x7f2bd806dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53064/0 (socket says 172.21.15.66:53064) 2024-09-10T22:04:34.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bd7e00640 1 -- 172.21.15.66:0/121163123 learned_addr learned my addr 172.21.15.66:0/121163123 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:34.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bdca00640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bd815bd40 0x7f2bd8165c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:34.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bd7400640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd80d06b0 0x7f2bd81fc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:34.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bd7e00640 1 -- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bd815bd40 msgr2=0x7f2bd8165c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:34.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bd7e00640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bd815bd40 0x7f2bd8165c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:34.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bd7e00640 1 -- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd80d06b0 msgr2=0x7f2bd81fc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:34.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bd7e00640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd80d06b0 0x7f2bd81fc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:34.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.739+0000 7f2bd7e00640 1 -- 172.21.15.66:0/121163123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd806e5e0 con 0x7f2bd80afc20 2024-09-10T22:04:34.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.743+0000 7f2bd7e00640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bd80afc20 0x7f2bd806dee0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800c430 tx=0x7f2bc800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:34.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.743+0000 7f2bcfe00640 1 -- 172.21.15.66:0/121163123 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8011790 con 0x7f2bd80afc20 2024-09-10T22:04:34.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.743+0000 7f2bcfe00640 1 -- 172.21.15.66:0/121163123 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bc8017070 con 0x7f2bd80afc20 2024-09-10T22:04:34.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.743+0000 7f2bcfe00640 1 -- 172.21.15.66:0/121163123 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8011cc0 con 0x7f2bd80afc20 2024-09-10T22:04:34.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.743+0000 7f2bdea00640 1 -- 172.21.15.66:0/121163123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd81f8540 con 0x7f2bd80afc20 2024-09-10T22:04:34.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.743+0000 7f2bdea00640 1 -- 172.21.15.66:0/121163123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81f8b00 con 0x7f2bd80afc20 2024-09-10T22:04:34.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.743+0000 7f2bdea00640 1 -- 172.21.15.66:0/121163123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd80cffa0 con 0x7f2bd80afc20 2024-09-10T22:04:34.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.747+0000 7f2bcfe00640 1 -- 172.21.15.66:0/121163123 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 12) ==== 96955+0+0 (secure 0 0 0) 0x7f2bc801c460 con 0x7f2bd80afc20 2024-09-10T22:04:34.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.751+0000 7f2bcfe00640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bb8072c60 0x7f2bb8075120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:34.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.751+0000 7f2bcfe00640 1 -- 172.21.15.66:0/121163123 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f2bc80946c0 con 0x7f2bd80afc20 2024-09-10T22:04:34.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.751+0000 7f2bd7400640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bb8072c60 0x7f2bb8075120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:34.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.751+0000 7f2bcfe00640 1 -- 172.21.15.66:0/121163123 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2bc805f420 con 0x7f2bd80afc20 2024-09-10T22:04:34.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:34.755+0000 7f2bd7400640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bb8072c60 0x7f2bb8075120 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40af000 tx=0x7f2bc401afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:35.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:35.079+0000 7f2bdea00640 1 -- 172.21.15.66:0/121163123 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone27303", "target": ["mon-mgr", ""]}) -- 0x7f2bd80b0df0 con 0x7f2bb8072c60 2024-09-10T22:04:36.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.687+0000 7f2bcfe00640 1 -- 172.21.15.66:0/121163123 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 13) ==== 97006+0+0 (secure 0 0 0) 0x7f2bc805af90 con 0x7f2bd80afc20 2024-09-10T22:04:36.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.707+0000 7f2bcfe00640 1 -- 172.21.15.66:0/121163123 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2bd80b0df0 con 0x7f2bb8072c60 2024-09-10T22:04:36.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.711+0000 7f2bdea00640 1 -- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bb8072c60 msgr2=0x7f2bb8075120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:36.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.711+0000 7f2bdea00640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bb8072c60 0x7f2bb8075120 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40af000 tx=0x7f2bc401afd0 comp rx=0 tx=0).stop 2024-09-10T22:04:36.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.711+0000 7f2bdea00640 1 -- 172.21.15.66:0/121163123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bd80afc20 msgr2=0x7f2bd806dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:36.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.711+0000 7f2bdea00640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bd80afc20 0x7f2bd806dee0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800c430 tx=0x7f2bc800c8f0 comp rx=0 tx=0).stop 2024-09-10T22:04:36.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.711+0000 7f2bdea00640 1 -- 172.21.15.66:0/121163123 shutdown_connections 2024-09-10T22:04:36.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.711+0000 7f2bdea00640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bb8072c60 0x7f2bb8075120 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:36.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.711+0000 7f2bdea00640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bd815bd40 0x7f2bd8165c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:36.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.711+0000 7f2bdea00640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd80d06b0 0x7f2bd81fc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:36.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.711+0000 7f2bdea00640 1 --2- 172.21.15.66:0/121163123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bd80afc20 0x7f2bd806dee0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:36.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.711+0000 7f2bdea00640 1 -- 172.21.15.66:0/121163123 >> 172.21.15.66:0/121163123 conn(0x7f2bd80d1c60 msgr2=0x7f2bd8159500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:36.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.711+0000 7f2bdea00640 1 -- 172.21.15.66:0/121163123 shutdown_connections 2024-09-10T22:04:36.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:36.711+0000 7f2bdea00640 1 -- 172.21.15.66:0/121163123 wait complete. 2024-09-10T22:04:36.753 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:36.750+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:36.754 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:36.750+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:36.754 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:36.750+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:36.754 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:36.750+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:36.754 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:36.750+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:36.761 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:04:37.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.067+0000 7ffb2f000640 1 -- 172.21.15.66:0/265216743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2815dd50 msgr2=0x7ffb280c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:37.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.067+0000 7ffb2f000640 1 --2- 172.21.15.66:0/265216743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2815dd50 0x7ffb280c48b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400afd0 tx=0x7ffb1401dee0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.067+0000 7ffb2f000640 1 -- 172.21.15.66:0/265216743 shutdown_connections 2024-09-10T22:04:37.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.067+0000 7ffb2f000640 1 --2- 172.21.15.66:0/265216743 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb2815e130 0x7ffb280c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.067+0000 7ffb2f000640 1 --2- 172.21.15.66:0/265216743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2815dd50 0x7ffb280c48b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.067+0000 7ffb2f000640 1 -- 172.21.15.66:0/265216743 >> 172.21.15.66:0/265216743 conn(0x7ffb280cfa30 msgr2=0x7ffb280cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:37.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.067+0000 7ffb2f000640 1 -- 172.21.15.66:0/265216743 shutdown_connections 2024-09-10T22:04:37.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.067+0000 7ffb2f000640 1 -- 172.21.15.66:0/265216743 wait complete. 2024-09-10T22:04:37.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.071+0000 7ffb2f000640 1 Processor -- start 2024-09-10T22:04:37.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.071+0000 7ffb2f000640 1 -- start start 2024-09-10T22:04:37.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.071+0000 7ffb2f000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb2806e630 0x7ffb281521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:37.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.071+0000 7ffb2f000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2815dd50 0x7ffb281ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:37.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.071+0000 7ffb2f000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2815e130 0x7ffb2814c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:37.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.071+0000 7ffb2f000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffb28166cf0 con 0x7ffb2815e130 2024-09-10T22:04:37.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.071+0000 7ffb2f000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb28166b70 con 0x7ffb2815dd50 2024-09-10T22:04:37.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.071+0000 7ffb2f000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ffb28166e70 con 0x7ffb2806e630 2024-09-10T22:04:37.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb2d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2815e130 0x7ffb2814c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb2d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2815e130 0x7ffb2814c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53072/0 (socket says 172.21.15.66:53072) 2024-09-10T22:04:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb2d000640 1 -- 172.21.15.66:0/3700464664 learned_addr learned my addr 172.21.15.66:0/3700464664 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb27e00640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb2806e630 0x7ffb281521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb27400640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2815dd50 0x7ffb281ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb2d000640 1 -- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb2806e630 msgr2=0x7ffb281521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb2d000640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb2806e630 0x7ffb281521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb2d000640 1 -- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2815dd50 msgr2=0x7ffb281ffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb2d000640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2815dd50 0x7ffb281ffa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb2d000640 1 -- 172.21.15.66:0/3700464664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb281fff60 con 0x7ffb2815e130 2024-09-10T22:04:37.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb2d000640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2815e130 0x7ffb2814c970 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000cbc0 tx=0x7ffb2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:37.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb1fe00640 1 -- 172.21.15.66:0/3700464664 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20016020 con 0x7ffb2815e130 2024-09-10T22:04:37.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb1fe00640 1 -- 172.21.15.66:0/3700464664 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb2000ab40 con 0x7ffb2815e130 2024-09-10T22:04:37.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb1fe00640 1 -- 172.21.15.66:0/3700464664 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20012420 con 0x7ffb2815e130 2024-09-10T22:04:37.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb2f000640 1 -- 172.21.15.66:0/3700464664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2814d3d0 con 0x7ffb2815e130 2024-09-10T22:04:37.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.075+0000 7ffb2f000640 1 -- 172.21.15.66:0/3700464664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb281516c0 con 0x7ffb2815e130 2024-09-10T22:04:37.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.079+0000 7ffb2f000640 1 -- 172.21.15.66:0/3700464664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb2806ea40 con 0x7ffb2815e130 2024-09-10T22:04:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.079+0000 7ffb1fe00640 1 -- 172.21.15.66:0/3700464664 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 97006+0+0 (secure 0 0 0) 0x7ffb2001b400 con 0x7ffb2815e130 2024-09-10T22:04:37.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.083+0000 7ffb1fe00640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffb08072d70 0x7ffb08075230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:37.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.083+0000 7ffb1fe00640 1 -- 172.21.15.66:0/3700464664 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7ffb200944e0 con 0x7ffb2815e130 2024-09-10T22:04:37.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.099+0000 7ffb1fe00640 1 -- 172.21.15.66:0/3700464664 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffb20020020 con 0x7ffb2815e130 2024-09-10T22:04:37.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.103+0000 7ffb27e00640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffb08072d70 0x7ffb08075230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:37.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.115+0000 7ffb27e00640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffb08072d70 0x7ffb08075230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffb140ae000 tx=0x7ffb1401bcc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:37.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.599+0000 7ffb2f000640 1 -- 172.21.15.66:0/3700464664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffb2814ceb0 con 0x7ffb2815e130 2024-09-10T22:04:37.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.599+0000 7ffb1fe00640 1 -- 172.21.15.66:0/3700464664 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2181 (secure 0 0 0) 0x7ffb281fff60 con 0x7ffb2815e130 2024-09-10T22:04:37.605 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:04:37.605 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:04:37.605 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:04:37.605 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:04:37.605 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:04:37.605 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:04:37.605 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:04:37.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:04:37.606 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:04:37.606 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:04:37.606 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:04:37.606 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:04:37.606 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:04:37.606 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:04:37.606 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:04:37.606 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:04:37.606 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:04:37.606 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:04:37.606 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:04:37.606 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 270, 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 46, 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:04:37.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 2630, 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 10923324840, 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 22106750976, 2024-09-10T22:04:37.608 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 750987362304, 2024-09-10T22:04:37.609 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:04:37.609 INFO:teuthology.orchestra.run.smithi066.stdout: "read_bytes_sec": 14675982, 2024-09-10T22:04:37.609 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 103458462, 2024-09-10T22:04:37.609 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 3, 2024-09-10T22:04:37.609 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 31 2024-09-10T22:04:37.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:04:37.609 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:04:37.609 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 27, 2024-09-10T22:04:37.609 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:03:15:051273+0000", 2024-09-10T22:04:37.609 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 3, 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 3, 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "a", 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 5412 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:04:37.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 8, 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:04:14.360941+0000", 2024-09-10T22:04:37.611 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:04:37.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:04:37.612 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": { 2024-09-10T22:04:37.612 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr-vol-ongoing-clones": { 2024-09-10T22:04:37.612 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "2 ongoing clones - average progress is 8.594% (2s)\n [==..........................] (remaining: 31s)", 2024-09-10T22:04:37.612 INFO:teuthology.orchestra.run.smithi066.stdout: "progress": 0.085940003395080566, 2024-09-10T22:04:37.612 INFO:teuthology.orchestra.run.smithi066.stdout: "add_to_ceph_s": true 2024-09-10T22:04:37.612 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:04:37.612 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:04:37.612 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:04:37.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.603+0000 7ffb2f000640 1 -- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffb08072d70 msgr2=0x7ffb08075230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:37.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.603+0000 7ffb2f000640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffb08072d70 0x7ffb08075230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffb140ae000 tx=0x7ffb1401bcc0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.603+0000 7ffb2f000640 1 -- 172.21.15.66:0/3700464664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2815e130 msgr2=0x7ffb2814c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:37.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.603+0000 7ffb2f000640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2815e130 0x7ffb2814c970 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000cbc0 tx=0x7ffb2000ccb0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.603+0000 7ffb2f000640 1 -- 172.21.15.66:0/3700464664 shutdown_connections 2024-09-10T22:04:37.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.603+0000 7ffb2f000640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffb08072d70 0x7ffb08075230 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.603+0000 7ffb2f000640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2815e130 0x7ffb2814c970 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.603+0000 7ffb2f000640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2815dd50 0x7ffb281ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.603+0000 7ffb2f000640 1 --2- 172.21.15.66:0/3700464664 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb2806e630 0x7ffb281521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.603+0000 7ffb2f000640 1 -- 172.21.15.66:0/3700464664 >> 172.21.15.66:0/3700464664 conn(0x7ffb280cfa30 msgr2=0x7ffb2814d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:37.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.603+0000 7ffb2f000640 1 -- 172.21.15.66:0/3700464664 shutdown_connections 2024-09-10T22:04:37.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.603+0000 7ffb2f000640 1 -- 172.21.15.66:0/3700464664 wait complete. 2024-09-10T22:04:37.639 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone9020 2024-09-10T22:04:37.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.935+0000 7f10aa400640 1 -- 172.21.15.66:0/352249073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1088004810 msgr2=0x7f1088004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:37.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.935+0000 7f10aa400640 1 --2- 172.21.15.66:0/352249073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1088004810 0x7f1088004c90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1094006ba0 tx=0x7f1094023a20 comp rx=0 tx=0).stop 2024-09-10T22:04:37.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.939+0000 7f10aa400640 1 -- 172.21.15.66:0/352249073 shutdown_connections 2024-09-10T22:04:37.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.939+0000 7f10aa400640 1 --2- 172.21.15.66:0/352249073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1088004810 0x7f1088004c90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.939+0000 7f10aa400640 1 -- 172.21.15.66:0/352249073 >> 172.21.15.66:0/352249073 conn(0x7f10a40b26c0 msgr2=0x7f10a40b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:37.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.939+0000 7f10aa400640 1 -- 172.21.15.66:0/352249073 shutdown_connections 2024-09-10T22:04:37.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.939+0000 7f10aa400640 1 -- 172.21.15.66:0/352249073 wait complete. 2024-09-10T22:04:37.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.939+0000 7f10aa400640 1 Processor -- start 2024-09-10T22:04:37.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.943+0000 7f10aa400640 1 -- start start 2024-09-10T22:04:37.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.943+0000 7f10aa400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1088004810 0x7f10a41f9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:37.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.943+0000 7f10aa400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10a414f3a0 0x7f10a4152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:37.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.943+0000 7f10aa400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10a40c5880 0x7f10a4155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:37.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.943+0000 7f10a3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10a40c5880 0x7f10a4155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:37.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.943+0000 7f10a3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10a40c5880 0x7f10a4155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53092/0 (socket says 172.21.15.66:53092) 2024-09-10T22:04:37.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.943+0000 7f10a3e00640 1 -- 172.21.15.66:0/1018331578 learned_addr learned my addr 172.21.15.66:0/1018331578 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:37.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.943+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10a406f650 con 0x7f10a40c5880 2024-09-10T22:04:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.947+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f10a40748d0 con 0x7f1088004810 2024-09-10T22:04:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.947+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f10a406f7d0 con 0x7f10a414f3a0 2024-09-10T22:04:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.947+0000 7f10a3e00640 1 -- 172.21.15.66:0/1018331578 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10a414f3a0 msgr2=0x7f10a4152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.947+0000 7f10a3e00640 1 --2- 172.21.15.66:0/1018331578 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10a414f3a0 0x7f10a4152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.947+0000 7f10a3e00640 1 -- 172.21.15.66:0/1018331578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1088004810 msgr2=0x7f10a41f9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.947+0000 7f10a3e00640 1 --2- 172.21.15.66:0/1018331578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1088004810 0x7f10a41f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.947+0000 7f10a3e00640 1 -- 172.21.15.66:0/1018331578 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a41f9db0 con 0x7f10a40c5880 2024-09-10T22:04:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.947+0000 7f10a3e00640 1 --2- 172.21.15.66:0/1018331578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10a40c5880 0x7f10a4155510 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f109c004270 tx=0x7f109c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.951+0000 7f109b400640 1 -- 172.21.15.66:0/1018331578 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c004b30 con 0x7f10a40c5880 2024-09-10T22:04:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.951+0000 7f109b400640 1 -- 172.21.15.66:0/1018331578 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f109c004cd0 con 0x7f10a40c5880 2024-09-10T22:04:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.951+0000 7f109b400640 1 -- 172.21.15.66:0/1018331578 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c010cc0 con 0x7f10a40c5880 2024-09-10T22:04:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.951+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a41526d0 con 0x7f10a40c5880 2024-09-10T22:04:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.951+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a40c4ac0 con 0x7f10a40c5880 2024-09-10T22:04:37.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.955+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a406cf70 con 0x7f10a40c5880 2024-09-10T22:04:37.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.959+0000 7f109b400640 1 -- 172.21.15.66:0/1018331578 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 97006+0+0 (secure 0 0 0) 0x7f109c00f040 con 0x7f10a40c5880 2024-09-10T22:04:37.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.959+0000 7f109b400640 1 --2- 172.21.15.66:0/1018331578 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1084072d20 0x7f10840751e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:37.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.959+0000 7f109b400640 1 -- 172.21.15.66:0/1018331578 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f109c0944a0 con 0x7f10a40c5880 2024-09-10T22:04:37.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.963+0000 7f10a3400640 1 --2- 172.21.15.66:0/1018331578 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1084072d20 0x7f10840751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:37.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.967+0000 7f10a3400640 1 --2- 172.21.15.66:0/1018331578 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1084072d20 0x7f10840751e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f10a406a000 tx=0x7f1090005af0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:37.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:37.971+0000 7f109b400640 1 -- 172.21.15.66:0/1018331578 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f10a41f9db0 con 0x7f10a40c5880 2024-09-10T22:04:38.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:38.443+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone9020", "target": ["mon-mgr", ""]}) -- 0x7f10a40b2290 con 0x7f1084072d20 2024-09-10T22:04:38.503 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:38.498+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:38.504 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:38.498+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:38.504 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:38.498+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:38.504 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:38.498+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:38.504 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:38.498+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:38.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:38.767+0000 7f109b400640 1 -- 172.21.15.66:0/1018331578 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 14) ==== 97057+0+0 (secure 0 0 0) 0x7f109c05ad50 con 0x7f10a40c5880 2024-09-10T22:04:40.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.171+0000 7f109b400640 1 -- 172.21.15.66:0/1018331578 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f10a40b2290 con 0x7f1084072d20 2024-09-10T22:04:40.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.191+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1084072d20 msgr2=0x7f10840751e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:40.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.191+0000 7f10aa400640 1 --2- 172.21.15.66:0/1018331578 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1084072d20 0x7f10840751e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f10a406a000 tx=0x7f1090005af0 comp rx=0 tx=0).stop 2024-09-10T22:04:40.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.191+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10a40c5880 msgr2=0x7f10a4155510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:40.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.191+0000 7f10aa400640 1 --2- 172.21.15.66:0/1018331578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10a40c5880 0x7f10a4155510 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f109c004270 tx=0x7f109c004730 comp rx=0 tx=0).stop 2024-09-10T22:04:40.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.191+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 shutdown_connections 2024-09-10T22:04:40.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.191+0000 7f10aa400640 1 --2- 172.21.15.66:0/1018331578 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1084072d20 0x7f10840751e0 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f10a406a000 tx=0x7f1090005af0 comp rx=0 tx=0).stop 2024-09-10T22:04:40.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.191+0000 7f10aa400640 1 --2- 172.21.15.66:0/1018331578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10a40c5880 0x7f10a4155510 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:40.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.191+0000 7f10aa400640 1 --2- 172.21.15.66:0/1018331578 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10a414f3a0 0x7f10a4152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:40.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.191+0000 7f10aa400640 1 --2- 172.21.15.66:0/1018331578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1088004810 0x7f10a41f9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:40.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.191+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 >> 172.21.15.66:0/1018331578 conn(0x7f10a40b26c0 msgr2=0x7f10a415e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:40.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.195+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 shutdown_connections 2024-09-10T22:04:40.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.195+0000 7f10aa400640 1 -- 172.21.15.66:0/1018331578 wait complete. 2024-09-10T22:04:40.242 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone24791 2024-09-10T22:04:40.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.539+0000 7ff74b200640 1 -- 172.21.15.66:0/2577620158 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7440d0330 msgr2=0x7ff74415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:40.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.539+0000 7ff74b200640 1 --2- 172.21.15.66:0/2577620158 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7440d0330 0x7ff74415e680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff7340108d0 tx=0x7ff734020610 comp rx=0 tx=0).stop 2024-09-10T22:04:40.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.539+0000 7ff74b200640 1 -- 172.21.15.66:0/2577620158 shutdown_connections 2024-09-10T22:04:40.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.539+0000 7ff74b200640 1 --2- 172.21.15.66:0/2577620158 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7440d0330 0x7ff74415e680 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:40.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.539+0000 7ff74b200640 1 -- 172.21.15.66:0/2577620158 >> 172.21.15.66:0/2577620158 conn(0x7ff7440d0b90 msgr2=0x7ff7440d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:40.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.539+0000 7ff74b200640 1 -- 172.21.15.66:0/2577620158 shutdown_connections 2024-09-10T22:04:40.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.539+0000 7ff74b200640 1 -- 172.21.15.66:0/2577620158 wait complete. 2024-09-10T22:04:40.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.543+0000 7ff74b200640 1 Processor -- start 2024-09-10T22:04:40.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.547+0000 7ff74b200640 1 -- start start 2024-09-10T22:04:40.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.547+0000 7ff74b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440d0330 0x7ff744153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:40.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.547+0000 7ff74b200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7440d0710 0x7ff744157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:40.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.547+0000 7ff74b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74415d370 0x7ff7441584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:40.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.547+0000 7ff74b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff74406d240 con 0x7ff7440d0330 2024-09-10T22:04:40.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.547+0000 7ff74b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff74406d0c0 con 0x7ff74415d370 2024-09-10T22:04:40.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.547+0000 7ff74b200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff74406d3c0 con 0x7ff7440d0710 2024-09-10T22:04:40.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff73b400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7440d0710 0x7ff744157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:40.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff73b400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7440d0710 0x7ff744157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:46304/0 (socket says 172.21.15.66:46304) 2024-09-10T22:04:40.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff73b400640 1 -- 172.21.15.66:0/1011017718 learned_addr learned my addr 172.21.15.66:0/1011017718 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:40.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff743e00640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440d0330 0x7ff744153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:40.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff749200640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74415d370 0x7ff7441584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:40.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff743e00640 1 -- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7440d0710 msgr2=0x7ff744157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:40.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff743e00640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7440d0710 0x7ff744157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:40.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff743e00640 1 -- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74415d370 msgr2=0x7ff7441584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:40.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff743e00640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74415d370 0x7ff7441584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:40.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff743e00640 1 -- 172.21.15.66:0/1011017718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff744153b00 con 0x7ff7440d0330 2024-09-10T22:04:40.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff743e00640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440d0330 0x7ff744153200 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff734004000 tx=0x7ff734010330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:40.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff740c00640 1 -- 172.21.15.66:0/1011017718 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734005230 con 0x7ff7440d0330 2024-09-10T22:04:40.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff740c00640 1 -- 172.21.15.66:0/1011017718 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7340054c0 con 0x7ff7440d0330 2024-09-10T22:04:40.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff740c00640 1 -- 172.21.15.66:0/1011017718 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734004930 con 0x7ff7440d0330 2024-09-10T22:04:40.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff74b200640 1 -- 172.21.15.66:0/1011017718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff744153de0 con 0x7ff7440d0330 2024-09-10T22:04:40.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.551+0000 7ff74b200640 1 -- 172.21.15.66:0/1011017718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7441543a0 con 0x7ff7440d0330 2024-09-10T22:04:40.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.555+0000 7ff74b200640 1 -- 172.21.15.66:0/1011017718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff74406d890 con 0x7ff7440d0330 2024-09-10T22:04:40.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.555+0000 7ff740c00640 1 -- 172.21.15.66:0/1011017718 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 14) ==== 97057+0+0 (secure 0 0 0) 0x7ff734004240 con 0x7ff7440d0330 2024-09-10T22:04:40.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.559+0000 7ff740c00640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff720072d80 0x7ff720075240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:40.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.559+0000 7ff740c00640 1 -- 172.21.15.66:0/1011017718 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7ff73412e9b0 con 0x7ff7440d0330 2024-09-10T22:04:40.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.567+0000 7ff740c00640 1 -- 172.21.15.66:0/1011017718 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff7340b1020 con 0x7ff7440d0330 2024-09-10T22:04:40.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.575+0000 7ff73b400640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff720072d80 0x7ff720075240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:40.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.587+0000 7ff73b400640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff720072d80 0x7ff720075240 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff744155310 tx=0x7ff72c007f60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:40.851+0000 7ff740c00640 1 -- 172.21.15.66:0/1011017718 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 15) ==== 97108+0+0 (secure 0 0 0) 0x7ff7340f5100 con 0x7ff7440d0330 2024-09-10T22:04:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.003+0000 7ff74b200640 1 -- 172.21.15.66:0/1011017718 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone24791", "target": ["mon-mgr", ""]}) -- 0x7ff744076260 con 0x7ff720072d80 2024-09-10T22:04:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.983+0000 7ff740c00640 1 -- 172.21.15.66:0/1011017718 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff72c0015f0 con 0x7ff720072d80 2024-09-10T22:04:41.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.987+0000 7ff74b200640 1 -- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff720072d80 msgr2=0x7ff720075240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:41.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.987+0000 7ff74b200640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff720072d80 0x7ff720075240 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff744155310 tx=0x7ff72c007f60 comp rx=0 tx=0).stop 2024-09-10T22:04:41.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.987+0000 7ff74b200640 1 -- 172.21.15.66:0/1011017718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440d0330 msgr2=0x7ff744153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:41.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.987+0000 7ff74b200640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440d0330 0x7ff744153200 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff734004000 tx=0x7ff734010330 comp rx=0 tx=0).stop 2024-09-10T22:04:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.987+0000 7ff74b200640 1 -- 172.21.15.66:0/1011017718 shutdown_connections 2024-09-10T22:04:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.987+0000 7ff74b200640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff720072d80 0x7ff720075240 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.987+0000 7ff74b200640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74415d370 0x7ff7441584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.987+0000 7ff74b200640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7440d0710 0x7ff744157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.987+0000 7ff74b200640 1 --2- 172.21.15.66:0/1011017718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440d0330 0x7ff744153200 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.987+0000 7ff74b200640 1 -- 172.21.15.66:0/1011017718 >> 172.21.15.66:0/1011017718 conn(0x7ff7440d0b90 msgr2=0x7ff7440b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.987+0000 7ff74b200640 1 -- 172.21.15.66:0/1011017718 shutdown_connections 2024-09-10T22:04:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:41.987+0000 7ff74b200640 1 -- 172.21.15.66:0/1011017718 wait complete. 2024-09-10T22:04:42.039 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone21312 2024-09-10T22:04:42.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.331+0000 7f33ee000640 1 -- 172.21.15.66:0/3804118735 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33e80d49b0 msgr2=0x7f33e8160310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:42.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.331+0000 7f33ee000640 1 --2- 172.21.15.66:0/3804118735 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33e80d49b0 0x7f33e8160310 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009840 tx=0x7f33d401c840 comp rx=0 tx=0).stop 2024-09-10T22:04:42.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.331+0000 7f33ee000640 1 -- 172.21.15.66:0/3804118735 shutdown_connections 2024-09-10T22:04:42.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.331+0000 7f33ee000640 1 --2- 172.21.15.66:0/3804118735 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33e80d49b0 0x7f33e8160310 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:42.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.331+0000 7f33ee000640 1 --2- 172.21.15.66:0/3804118735 >> v2:172.21.15.66:3300/0 conn(0x7f33e81611d0 0x7f33e81615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:42.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.331+0000 7f33ee000640 1 -- 172.21.15.66:0/3804118735 >> 172.21.15.66:0/3804118735 conn(0x7f33e80d3e90 msgr2=0x7f33e80d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:42.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.331+0000 7f33ee000640 1 -- 172.21.15.66:0/3804118735 shutdown_connections 2024-09-10T22:04:42.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.331+0000 7f33ee000640 1 -- 172.21.15.66:0/3804118735 wait complete. 2024-09-10T22:04:42.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.343+0000 7f33ee000640 1 Processor -- start 2024-09-10T22:04:42.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.343+0000 7f33ee000640 1 -- start start 2024-09-10T22:04:42.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33ee000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33e80b2e60 0x7f33e8152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:42.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33ee000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33e80d49b0 0x7f33e81ff640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:42.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33ee000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33e81611d0 0x7f33e81ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:42.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33ee000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33e80700c0 con 0x7f33e81611d0 2024-09-10T22:04:42.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33ee000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33e806ff40 con 0x7f33e80b2e60 2024-09-10T22:04:42.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33ee000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f33e8070240 con 0x7f33e80d49b0 2024-09-10T22:04:42.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33e81611d0 0x7f33e81ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33e6a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33e80d49b0 0x7f33e81ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33e6a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33e80d49b0 0x7f33e81ff640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:46324/0 (socket says 172.21.15.66:46324) 2024-09-10T22:04:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33e7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33e80b2e60 0x7f33e8152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33e7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33e80b2e60 0x7f33e8152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:55458/0 (socket says 172.21.15.66:55458) 2024-09-10T22:04:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33e7400640 1 -- 172.21.15.66:0/2445124822 learned_addr learned my addr 172.21.15.66:0/2445124822 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.347+0000 7f33e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33e81611d0 0x7f33e81ffb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53112/0 (socket says 172.21.15.66:53112) 2024-09-10T22:04:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.351+0000 7f33e7400640 1 -- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33e80d49b0 msgr2=0x7f33e81ff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.351+0000 7f33e7400640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33e80d49b0 0x7f33e81ff640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.351+0000 7f33e7400640 1 -- 172.21.15.66:0/2445124822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33e81611d0 msgr2=0x7f33e81ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.351+0000 7f33e7400640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33e81611d0 0x7f33e81ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:42.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.351+0000 7f33e7400640 1 -- 172.21.15.66:0/2445124822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e8074120 con 0x7f33e80b2e60 2024-09-10T22:04:42.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.351+0000 7f33e6a00640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33e80d49b0 0x7f33e81ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:04:42.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.355+0000 7f33e7400640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33e80b2e60 0x7f33e8152620 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f33d800b410 tx=0x7f33d800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.355+0000 7f33df400640 1 -- 172.21.15.66:0/2445124822 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8015020 con 0x7f33e80b2e60 2024-09-10T22:04:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.355+0000 7f33df400640 1 -- 172.21.15.66:0/2445124822 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33d8011920 con 0x7f33e80b2e60 2024-09-10T22:04:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.355+0000 7f33df400640 1 -- 172.21.15.66:0/2445124822 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8011d70 con 0x7f33e80b2e60 2024-09-10T22:04:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.355+0000 7f33ee000640 1 -- 172.21.15.66:0/2445124822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e80cd880 con 0x7f33e80b2e60 2024-09-10T22:04:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.355+0000 7f33ee000640 1 -- 172.21.15.66:0/2445124822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e80cb100 con 0x7f33e80b2e60 2024-09-10T22:04:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.359+0000 7f33ee000640 1 -- 172.21.15.66:0/2445124822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e80cdda0 con 0x7f33e80b2e60 2024-09-10T22:04:42.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.359+0000 7f33df400640 1 -- 172.21.15.66:0/2445124822 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 15) ==== 97108+0+0 (secure 0 0 0) 0x7f33d801a400 con 0x7f33e80b2e60 2024-09-10T22:04:42.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.359+0000 7f33df400640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f33c8072e10 0x7f33c80752d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:42.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.363+0000 7f33df400640 1 -- 172.21.15.66:0/2445124822 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f33d8094470 con 0x7f33e80b2e60 2024-09-10T22:04:42.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.367+0000 7f33e6a00640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f33c8072e10 0x7f33c80752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:42.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.367+0000 7f33e6a00640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f33c8072e10 0x7f33c80752d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f33d40af000 tx=0x7f33d401b080 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:42.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.375+0000 7f33df400640 1 -- 172.21.15.66:0/2445124822 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f33e80cdda0 con 0x7f33e80b2e60 2024-09-10T22:04:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.727+0000 7f33ee000640 1 -- 172.21.15.66:0/2445124822 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone21312", "target": ["mon-mgr", ""]}) -- 0x7f33e81615b0 con 0x7f33c8072e10 2024-09-10T22:04:42.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:42.915+0000 7f33df400640 1 -- 172.21.15.66:0/2445124822 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 16) ==== 97057+0+0 (secure 0 0 0) 0x7f33d805a630 con 0x7f33e80b2e60 2024-09-10T22:04:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.259+0000 7f33df400640 1 -- 172.21.15.66:0/2445124822 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f33e81615b0 con 0x7f33c8072e10 2024-09-10T22:04:43.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.263+0000 7f33ee000640 1 -- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f33c8072e10 msgr2=0x7f33c80752d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:43.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.263+0000 7f33ee000640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f33c8072e10 0x7f33c80752d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f33d40af000 tx=0x7f33d401b080 comp rx=0 tx=0).stop 2024-09-10T22:04:43.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.263+0000 7f33ee000640 1 -- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33e80b2e60 msgr2=0x7f33e8152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:43.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.263+0000 7f33ee000640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33e80b2e60 0x7f33e8152620 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f33d800b410 tx=0x7f33d800b8d0 comp rx=0 tx=0).stop 2024-09-10T22:04:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.263+0000 7f33ee000640 1 -- 172.21.15.66:0/2445124822 shutdown_connections 2024-09-10T22:04:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.263+0000 7f33ee000640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f33c8072e10 0x7f33c80752d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.263+0000 7f33ee000640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33e81611d0 0x7f33e81ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.263+0000 7f33ee000640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33e80d49b0 0x7f33e81ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.263+0000 7f33ee000640 1 --2- 172.21.15.66:0/2445124822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33e80b2e60 0x7f33e8152620 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.263+0000 7f33ee000640 1 -- 172.21.15.66:0/2445124822 >> 172.21.15.66:0/2445124822 conn(0x7f33e80d3e90 msgr2=0x7f33e80d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.263+0000 7f33ee000640 1 -- 172.21.15.66:0/2445124822 shutdown_connections 2024-09-10T22:04:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.263+0000 7f33ee000640 1 -- 172.21.15.66:0/2445124822 wait complete. 2024-09-10T22:04:43.310 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone27303 2024-09-10T22:04:43.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.551+0000 7f2e14e00640 1 -- 172.21.15.66:0/2290999637 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2df4004810 msgr2=0x7f2df4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:43.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.551+0000 7f2e14e00640 1 --2- 172.21.15.66:0/2290999637 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2df4004810 0x7f2df4004c90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00c710 tx=0x7f2dfc01c690 comp rx=0 tx=0).stop 2024-09-10T22:04:43.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.551+0000 7f2e14e00640 1 -- 172.21.15.66:0/2290999637 shutdown_connections 2024-09-10T22:04:43.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.551+0000 7f2e14e00640 1 --2- 172.21.15.66:0/2290999637 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2df4004810 0x7f2df4004c90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:43.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.551+0000 7f2e14e00640 1 -- 172.21.15.66:0/2290999637 >> 172.21.15.66:0/2290999637 conn(0x7f2e100736d0 msgr2=0x7f2e10073ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:43.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.551+0000 7f2e14e00640 1 -- 172.21.15.66:0/2290999637 shutdown_connections 2024-09-10T22:04:43.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.555+0000 7f2e14e00640 1 -- 172.21.15.66:0/2290999637 wait complete. 2024-09-10T22:04:43.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.555+0000 7f2e14e00640 1 Processor -- start 2024-09-10T22:04:43.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.555+0000 7f2e14e00640 1 -- start start 2024-09-10T22:04:43.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e14e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df4004810 0x7f2e10150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:43.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e14e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10152380 0x7f2e101f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:43.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e14e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e101539f0 0x7f2e10154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:43.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e14e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e10070970 con 0x7f2e10152380 2024-09-10T22:04:43.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e14e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e100707f0 con 0x7f2df4004810 2024-09-10T22:04:43.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e14e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2e100b4e80 con 0x7f2e101539f0 2024-09-10T22:04:43.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e0e800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e101539f0 0x7f2e10154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:43.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e0e800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e101539f0 0x7f2e10154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:60356/0 (socket says 172.21.15.66:60356) 2024-09-10T22:04:43.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e0e800640 1 -- 172.21.15.66:0/2857135162 learned_addr learned my addr 172.21.15.66:0/2857135162 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:43.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e0d400640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10152380 0x7f2e101f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:43.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e0e800640 1 -- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df4004810 msgr2=0x7f2e10150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:43.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e0e800640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df4004810 0x7f2e10150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:43.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e0e800640 1 -- 172.21.15.66:0/2857135162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10152380 msgr2=0x7f2e101f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:43.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e0e800640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10152380 0x7f2e101f8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:43.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.559+0000 7f2e0e800640 1 -- 172.21.15.66:0/2857135162 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e101f8bd0 con 0x7f2e101539f0 2024-09-10T22:04:43.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.563+0000 7f2e0e800640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e101539f0 0x7f2e10154960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2e1015bdb0 tx=0x7f2e0800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:43.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.563+0000 7f2e06000640 1 -- 172.21.15.66:0/2857135162 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0801b070 con 0x7f2e101539f0 2024-09-10T22:04:43.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.563+0000 7f2e06000640 1 -- 172.21.15.66:0/2857135162 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e0800cc80 con 0x7f2e101539f0 2024-09-10T22:04:43.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.563+0000 7f2e06000640 1 -- 172.21.15.66:0/2857135162 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e080146e0 con 0x7f2e101539f0 2024-09-10T22:04:43.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.563+0000 7f2e14e00640 1 -- 172.21.15.66:0/2857135162 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e10152920 con 0x7f2e101539f0 2024-09-10T22:04:43.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.563+0000 7f2e14e00640 1 -- 172.21.15.66:0/2857135162 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e101531f0 con 0x7f2e101539f0 2024-09-10T22:04:43.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.567+0000 7f2e14e00640 1 -- 172.21.15.66:0/2857135162 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e10152d30 con 0x7f2e101539f0 2024-09-10T22:04:43.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.571+0000 7f2e06000640 1 -- 172.21.15.66:0/2857135162 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 16) ==== 97057+0+0 (secure 0 0 0) 0x7f2e08014880 con 0x7f2e101539f0 2024-09-10T22:04:43.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.571+0000 7f2e06000640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2df0072d60 0x7f2df0075220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:43.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.571+0000 7f2e06000640 1 -- 172.21.15.66:0/2857135162 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f2e08094cd0 con 0x7f2e101539f0 2024-09-10T22:04:43.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.575+0000 7f2e0de00640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2df0072d60 0x7f2df0075220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:43.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.579+0000 7f2e0de00640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2df0072d60 0x7f2df0075220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e00001340 tx=0x7f2e00000f60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:43.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.587+0000 7f2e06000640 1 -- 172.21.15.66:0/2857135162 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2e10152d30 con 0x7f2e101539f0 2024-09-10T22:04:43.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:43.967+0000 7f2e14e00640 1 -- 172.21.15.66:0/2857135162 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone27303", "target": ["mon-mgr", ""]}) -- 0x7f2e100b27e0 con 0x7f2df0072d60 2024-09-10T22:04:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.155+0000 7f2e06000640 1 -- 172.21.15.66:0/2857135162 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2e100b27e0 con 0x7f2df0072d60 2024-09-10T22:04:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.155+0000 7f2e14e00640 1 -- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2df0072d60 msgr2=0x7f2df0075220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.155+0000 7f2e14e00640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2df0072d60 0x7f2df0075220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e00001340 tx=0x7f2e00000f60 comp rx=0 tx=0).stop 2024-09-10T22:04:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.155+0000 7f2e14e00640 1 -- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e101539f0 msgr2=0x7f2e10154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.155+0000 7f2e14e00640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e101539f0 0x7f2e10154960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2e1015bdb0 tx=0x7f2e0800ef90 comp rx=0 tx=0).stop 2024-09-10T22:04:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.155+0000 7f2e14e00640 1 -- 172.21.15.66:0/2857135162 shutdown_connections 2024-09-10T22:04:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.155+0000 7f2e14e00640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2df0072d60 0x7f2df0075220 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.155+0000 7f2e14e00640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e101539f0 0x7f2e10154960 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.155+0000 7f2e14e00640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10152380 0x7f2e101f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.155+0000 7f2e14e00640 1 --2- 172.21.15.66:0/2857135162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df4004810 0x7f2e10150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.155+0000 7f2e14e00640 1 -- 172.21.15.66:0/2857135162 >> 172.21.15.66:0/2857135162 conn(0x7f2e100736d0 msgr2=0x7f2e10070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.155+0000 7f2e14e00640 1 -- 172.21.15.66:0/2857135162 shutdown_connections 2024-09-10T22:04:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.159+0000 7f2e14e00640 1 -- 172.21.15.66:0/2857135162 wait complete. 2024-09-10T22:04:44.181 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:04:44.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.323+0000 7fa84f200640 1 -- 172.21.15.66:0/1987031207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85015f300 msgr2=0x7fa8500b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:44.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.323+0000 7fa84f200640 1 --2- 172.21.15.66:0/1987031207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85015f300 0x7fa8500b1090 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa838009840 tx=0x7fa83801c7f0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.323+0000 7fa84f200640 1 -- 172.21.15.66:0/1987031207 shutdown_connections 2024-09-10T22:04:44.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.323+0000 7fa84f200640 1 --2- 172.21.15.66:0/1987031207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85015f300 0x7fa8500b1090 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.323+0000 7fa84f200640 1 --2- 172.21.15.66:0/1987031207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa85015ef20 0x7fa8500cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.323+0000 7fa84f200640 1 -- 172.21.15.66:0/1987031207 >> 172.21.15.66:0/1987031207 conn(0x7fa8500d1b10 msgr2=0x7fa8500d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:44.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84f200640 1 -- 172.21.15.66:0/1987031207 shutdown_connections 2024-09-10T22:04:44.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84f200640 1 -- 172.21.15.66:0/1987031207 wait complete. 2024-09-10T22:04:44.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84f200640 1 Processor -- start 2024-09-10T22:04:44.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84f200640 1 -- start start 2024-09-10T22:04:44.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8500b15d0 0x7fa8501f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:44.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84f200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa85015ef20 0x7fa8501f8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:44.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85015f300 0x7fa850155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:44.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa850168ee0 con 0x7fa8500b15d0 2024-09-10T22:04:44.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa850168d60 con 0x7fa85015f300 2024-09-10T22:04:44.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84f200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa850169060 con 0x7fa85015ef20 2024-09-10T22:04:44.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8500b15d0 0x7fa8501f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:44.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8500b15d0 0x7fa8501f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44400/0 (socket says 172.21.15.66:44400) 2024-09-10T22:04:44.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84de00640 1 -- 172.21.15.66:0/2598905821 learned_addr learned my addr 172.21.15.66:0/2598905821 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:44.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa847e00640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa85015ef20 0x7fa8501f8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:44.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84e800640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85015f300 0x7fa850155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:44.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84de00640 1 -- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa85015ef20 msgr2=0x7fa8501f8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:44.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84de00640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa85015ef20 0x7fa8501f8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84de00640 1 -- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85015f300 msgr2=0x7fa850155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:44.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84de00640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85015f300 0x7fa850155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84de00640 1 -- 172.21.15.66:0/2598905821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa850156190 con 0x7fa8500b15d0 2024-09-10T22:04:44.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa84de00640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8500b15d0 0x7fa8501f56d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa84000ea10 tx=0x7fa84000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:44.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.327+0000 7fa846000640 1 -- 172.21.15.66:0/2598905821 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84000cde0 con 0x7fa8500b15d0 2024-09-10T22:04:44.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.331+0000 7fa846000640 1 -- 172.21.15.66:0/2598905821 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa840004020 con 0x7fa8500b15d0 2024-09-10T22:04:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.331+0000 7fa846000640 1 -- 172.21.15.66:0/2598905821 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840004470 con 0x7fa8500b15d0 2024-09-10T22:04:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.331+0000 7fa84f200640 1 -- 172.21.15.66:0/2598905821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8501f93a0 con 0x7fa8500b15d0 2024-09-10T22:04:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.331+0000 7fa84f200640 1 -- 172.21.15.66:0/2598905821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8501f9960 con 0x7fa8500b15d0 2024-09-10T22:04:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.331+0000 7fa84f200640 1 -- 172.21.15.66:0/2598905821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8500cecc0 con 0x7fa8500b15d0 2024-09-10T22:04:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.331+0000 7fa846000640 1 -- 172.21.15.66:0/2598905821 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 97057+0+0 (secure 0 0 0) 0x7fa840003600 con 0x7fa8500b15d0 2024-09-10T22:04:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.331+0000 7fa846000640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa82c072d30 0x7fa82c0751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.331+0000 7fa846000640 1 -- 172.21.15.66:0/2598905821 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fa8400949a0 con 0x7fa8500b15d0 2024-09-10T22:04:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.331+0000 7fa847e00640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa82c072d30 0x7fa82c0751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.335+0000 7fa847e00640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa82c072d30 0x7fa82c0751f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa8380af000 tx=0x7fa83801b030 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.335+0000 7fa846000640 1 -- 172.21.15.66:0/2598905821 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa84005ee40 con 0x7fa8500b15d0 2024-09-10T22:04:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.531+0000 7fa84f200640 1 -- 172.21.15.66:0/2598905821 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa8500b2f40 con 0x7fa82c072d30 2024-09-10T22:04:44.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa846000640 1 -- 172.21.15.66:0/2598905821 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa8500b2f40 con 0x7fa82c072d30 2024-09-10T22:04:44.538 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:04:44.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:04:44.538 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:04:44.538 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:04:44.538 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:04:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa84f200640 1 -- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa82c072d30 msgr2=0x7fa82c0751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa84f200640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa82c072d30 0x7fa82c0751f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa8380af000 tx=0x7fa83801b030 comp rx=0 tx=0).stop 2024-09-10T22:04:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa84f200640 1 -- 172.21.15.66:0/2598905821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8500b15d0 msgr2=0x7fa8501f56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa84f200640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8500b15d0 0x7fa8501f56d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa84000ea10 tx=0x7fa84000eed0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa84f200640 1 -- 172.21.15.66:0/2598905821 shutdown_connections 2024-09-10T22:04:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa84f200640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa82c072d30 0x7fa82c0751f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa84f200640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85015f300 0x7fa850155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa84f200640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa85015ef20 0x7fa8501f8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa84f200640 1 --2- 172.21.15.66:0/2598905821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8500b15d0 0x7fa8501f56d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa84f200640 1 -- 172.21.15.66:0/2598905821 >> 172.21.15.66:0/2598905821 conn(0x7fa8500d1b10 msgr2=0x7fa8500af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa84f200640 1 -- 172.21.15.66:0/2598905821 shutdown_connections 2024-09-10T22:04:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.535+0000 7fa84f200640 1 -- 172.21.15.66:0/2598905821 wait complete. 2024-09-10T22:04:44.560 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-10T22:04:44.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.703+0000 7fdf27200640 1 -- 172.21.15.66:0/1872111333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf2806cc20 msgr2=0x7fdf280ae9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:44.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.703+0000 7fdf27200640 1 --2- 172.21.15.66:0/1872111333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf2806cc20 0x7fdf280ae9a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdf280d0990 tx=0x7fdf1001c830 comp rx=0 tx=0).stop 2024-09-10T22:04:44.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.703+0000 7fdf27200640 1 -- 172.21.15.66:0/1872111333 shutdown_connections 2024-09-10T22:04:44.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.703+0000 7fdf27200640 1 --2- 172.21.15.66:0/1872111333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf2806cc20 0x7fdf280ae9a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.703+0000 7fdf27200640 1 --2- 172.21.15.66:0/1872111333 >> v2:172.21.15.94:3300/0 conn(0x7fdf28151020 0x7fdf280cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.703+0000 7fdf27200640 1 -- 172.21.15.66:0/1872111333 >> 172.21.15.66:0/1872111333 conn(0x7fdf280cc3c0 msgr2=0x7fdf280cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:44.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.703+0000 7fdf27200640 1 -- 172.21.15.66:0/1872111333 shutdown_connections 2024-09-10T22:04:44.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.703+0000 7fdf27200640 1 -- 172.21.15.66:0/1872111333 wait complete. 2024-09-10T22:04:44.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf27200640 1 Processor -- start 2024-09-10T22:04:44.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf27200640 1 -- start start 2024-09-10T22:04:44.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf27200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2806cc20 0x7fdf2814bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:44.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf27200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdf2806d090 0x7fdf281577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:44.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf27200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf28151020 0x7fdf281f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:44.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf27200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281681e0 con 0x7fdf28151020 2024-09-10T22:04:44.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf27200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28168060 con 0x7fdf2806cc20 2024-09-10T22:04:44.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf27200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdf28168360 con 0x7fdf2806d090 2024-09-10T22:04:44.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf26800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf28151020 0x7fdf281f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:44.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf25e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2806cc20 0x7fdf2814bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:44.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf26800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf28151020 0x7fdf281f6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44414/0 (socket says 172.21.15.66:44414) 2024-09-10T22:04:44.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf26800640 1 -- 172.21.15.66:0/1480952345 learned_addr learned my addr 172.21.15.66:0/1480952345 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf1fe00640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdf2806d090 0x7fdf281577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf26800640 1 -- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdf2806d090 msgr2=0x7fdf281577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf26800640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdf2806d090 0x7fdf281577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf26800640 1 -- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2806cc20 msgr2=0x7fdf2814bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf26800640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2806cc20 0x7fdf2814bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf26800640 1 -- 172.21.15.66:0/1480952345 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf280b3260 con 0x7fdf28151020 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf26800640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf28151020 0x7fdf281f6540 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000c700 tx=0x7fdf2000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf1e000640 1 -- 172.21.15.66:0/1480952345 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20003b80 con 0x7fdf28151020 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf1e000640 1 -- 172.21.15.66:0/1480952345 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf20003d20 con 0x7fdf28151020 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf27200640 1 -- 172.21.15.66:0/1480952345 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2815d530 con 0x7fdf28151020 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf1e000640 1 -- 172.21.15.66:0/1480952345 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2000aab0 con 0x7fdf28151020 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf27200640 1 -- 172.21.15.66:0/1480952345 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf281f5d40 con 0x7fdf28151020 2024-09-10T22:04:44.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.707+0000 7fdf27200640 1 -- 172.21.15.66:0/1480952345 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf280ae9a0 con 0x7fdf28151020 2024-09-10T22:04:44.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.711+0000 7fdf1e000640 1 -- 172.21.15.66:0/1480952345 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 97057+0+0 (secure 0 0 0) 0x7fdf2000ac50 con 0x7fdf28151020 2024-09-10T22:04:44.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.711+0000 7fdf1e000640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdf04072d30 0x7fdf040751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:44.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.711+0000 7fdf1e000640 1 -- 172.21.15.66:0/1480952345 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fdf200941c0 con 0x7fdf28151020 2024-09-10T22:04:44.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.711+0000 7fdf25e00640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdf04072d30 0x7fdf040751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:44.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.711+0000 7fdf25e00640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdf04072d30 0x7fdf040751f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdf18010e60 tx=0x7fdf18012040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:44.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.715+0000 7fdf1e000640 1 -- 172.21.15.66:0/1480952345 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdf2005e290 con 0x7fdf28151020 2024-09-10T22:04:44.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.915+0000 7fdf27200640 1 -- 172.21.15.66:0/1480952345 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf280ceff0 con 0x7fdf04072d30 2024-09-10T22:04:44.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf1e000640 1 -- 172.21.15.66:0/1480952345 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+233 (secure 0 0 0) 0x7fdf280ceff0 con 0x7fdf04072d30 2024-09-10T22:04:44.926 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:04:44.926 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:04:44.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:04:44.926 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sv1" 2024-09-10T22:04:44.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:04:44.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:04:44.926 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone9020" 2024-09-10T22:04:44.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:04:44.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:04:44.927 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone21312" 2024-09-10T22:04:44.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:04:44.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:04:44.927 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone27303" 2024-09-10T22:04:44.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:04:44.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:04:44.927 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone24791" 2024-09-10T22:04:44.927 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:04:44.927 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:04:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf27200640 1 -- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdf04072d30 msgr2=0x7fdf040751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf27200640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdf04072d30 0x7fdf040751f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdf18010e60 tx=0x7fdf18012040 comp rx=0 tx=0).stop 2024-09-10T22:04:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf27200640 1 -- 172.21.15.66:0/1480952345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf28151020 msgr2=0x7fdf281f6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf27200640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf28151020 0x7fdf281f6540 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000c700 tx=0x7fdf2000cbc0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf27200640 1 -- 172.21.15.66:0/1480952345 shutdown_connections 2024-09-10T22:04:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf27200640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdf04072d30 0x7fdf040751f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf27200640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf28151020 0x7fdf281f6540 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf27200640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdf2806d090 0x7fdf281577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf27200640 1 --2- 172.21.15.66:0/1480952345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2806cc20 0x7fdf2814bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf27200640 1 -- 172.21.15.66:0/1480952345 >> 172.21.15.66:0/1480952345 conn(0x7fdf280cc3c0 msgr2=0x7fdf2815a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf27200640 1 -- 172.21.15.66:0/1480952345 shutdown_connections 2024-09-10T22:04:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:44.923+0000 7fdf27200640 1 -- 172.21.15.66:0/1480952345 wait complete. 2024-09-10T22:04:44.948 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-10T22:04:45.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 -- 172.21.15.66:0/936469832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1e415ddd0 msgr2=0x7ff1e415e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 --2- 172.21.15.66:0/936469832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1e415ddd0 0x7ff1e415e1f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009870 tx=0x7ff1cc01f010 comp rx=0 tx=0).stop 2024-09-10T22:04:45.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 -- 172.21.15.66:0/936469832 shutdown_connections 2024-09-10T22:04:45.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 --2- 172.21.15.66:0/936469832 >> v2:172.21.15.94:3300/0 conn(0x7ff1e415b730 0x7ff1e415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 --2- 172.21.15.66:0/936469832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1e415ddd0 0x7ff1e415e1f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 -- 172.21.15.66:0/936469832 >> 172.21.15.66:0/936469832 conn(0x7ff1e40cfaa0 msgr2=0x7ff1e40cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:45.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 -- 172.21.15.66:0/936469832 shutdown_connections 2024-09-10T22:04:45.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 -- 172.21.15.66:0/936469832 wait complete. 2024-09-10T22:04:45.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 Processor -- start 2024-09-10T22:04:45.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 -- start start 2024-09-10T22:04:45.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1e40cee30 0x7ff1e40ab430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:45.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1e415b730 0x7ff1e40acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:45.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1e415ddd0 0x7ff1e40aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:45.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e406e820 con 0x7ff1e415ddd0 2024-09-10T22:04:45.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e406e6a0 con 0x7ff1e415b730 2024-09-10T22:04:45.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.095+0000 7ff1eae00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e406e9a0 con 0x7ff1e40cee30 2024-09-10T22:04:45.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e8e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1e415ddd0 0x7ff1e40aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:45.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e8e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1e415ddd0 0x7ff1e40aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44418/0 (socket says 172.21.15.66:44418) 2024-09-10T22:04:45.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e8e00640 1 -- 172.21.15.66:0/813216241 learned_addr learned my addr 172.21.15.66:0/813216241 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:45.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e3e00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1e40cee30 0x7ff1e40ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:45.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1dbe00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1e415b730 0x7ff1e40acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:45.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e8e00640 1 -- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1e40cee30 msgr2=0x7ff1e40ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e8e00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1e40cee30 0x7ff1e40ab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e8e00640 1 -- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1e415b730 msgr2=0x7ff1e40acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e8e00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1e415b730 0x7ff1e40acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e8e00640 1 -- 172.21.15.66:0/813216241 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e40cd3f0 con 0x7ff1e415ddd0 2024-09-10T22:04:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e8e00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1e415ddd0 0x7ff1e40aed00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00eab0 tx=0x7ff1dc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e0c00640 1 -- 172.21.15.66:0/813216241 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc004b30 con 0x7ff1e415ddd0 2024-09-10T22:04:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e0c00640 1 -- 172.21.15.66:0/813216241 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1dc004cd0 con 0x7ff1e415ddd0 2024-09-10T22:04:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e0c00640 1 -- 172.21.15.66:0/813216241 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc005280 con 0x7ff1e415ddd0 2024-09-10T22:04:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1eae00640 1 -- 172.21.15.66:0/813216241 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e40aafb0 con 0x7ff1e415ddd0 2024-09-10T22:04:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1eae00640 1 -- 172.21.15.66:0/813216241 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e40ccb50 con 0x7ff1e415ddd0 2024-09-10T22:04:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1eae00640 1 -- 172.21.15.66:0/813216241 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e415e1f0 con 0x7ff1e415ddd0 2024-09-10T22:04:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.099+0000 7ff1e0c00640 1 -- 172.21.15.66:0/813216241 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 97057+0+0 (secure 0 0 0) 0x7ff1dc005420 con 0x7ff1e415ddd0 2024-09-10T22:04:45.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.103+0000 7ff1e0c00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff1c0072d30 0x7ff1c00751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:45.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.103+0000 7ff1e0c00640 1 -- 172.21.15.66:0/813216241 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7ff1dc095090 con 0x7ff1e415ddd0 2024-09-10T22:04:45.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.103+0000 7ff1e3e00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff1c0072d30 0x7ff1c00751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:45.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.103+0000 7ff1e3e00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff1c0072d30 0x7ff1c00751f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4001910 tx=0x7ff1d4001420 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:45.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.107+0000 7ff1e0c00640 1 -- 172.21.15.66:0/813216241 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff1dc05f160 con 0x7ff1e415ddd0 2024-09-10T22:04:45.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.311+0000 7ff1eae00640 1 -- 172.21.15.66:0/813216241 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1e41f8230 con 0x7ff1c0072d30 2024-09-10T22:04:45.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.327+0000 7ff1e0c00640 1 -- 172.21.15.66:0/813216241 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff1e41f8230 con 0x7ff1c0072d30 2024-09-10T22:04:45.333 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:04:45.333 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:04:45.333 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:04:45.333 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ss1" 2024-09-10T22:04:45.333 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:04:45.334 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:04:45.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.331+0000 7ff1eae00640 1 -- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff1c0072d30 msgr2=0x7ff1c00751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.331+0000 7ff1eae00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff1c0072d30 0x7ff1c00751f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4001910 tx=0x7ff1d4001420 comp rx=0 tx=0).stop 2024-09-10T22:04:45.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.331+0000 7ff1eae00640 1 -- 172.21.15.66:0/813216241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1e415ddd0 msgr2=0x7ff1e40aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.331+0000 7ff1eae00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1e415ddd0 0x7ff1e40aed00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00eab0 tx=0x7ff1dc00ef70 comp rx=0 tx=0).stop 2024-09-10T22:04:45.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.331+0000 7ff1eae00640 1 -- 172.21.15.66:0/813216241 shutdown_connections 2024-09-10T22:04:45.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.331+0000 7ff1eae00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff1c0072d30 0x7ff1c00751f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.331+0000 7ff1eae00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1e415ddd0 0x7ff1e40aed00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.331+0000 7ff1eae00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1e415b730 0x7ff1e40acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.331+0000 7ff1eae00640 1 --2- 172.21.15.66:0/813216241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1e40cee30 0x7ff1e40ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.331+0000 7ff1eae00640 1 -- 172.21.15.66:0/813216241 >> 172.21.15.66:0/813216241 conn(0x7ff1e40cfaa0 msgr2=0x7ff1e4150190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:45.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.331+0000 7ff1eae00640 1 -- 172.21.15.66:0/813216241 shutdown_connections 2024-09-10T22:04:45.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.331+0000 7ff1eae00640 1 -- 172.21.15.66:0/813216241 wait complete. 2024-09-10T22:04:45.356 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-10T22:04:45.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.503+0000 7f9feac00640 1 -- 172.21.15.66:0/607860260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe40d06b0 msgr2=0x7f9fe40ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.503+0000 7f9feac00640 1 --2- 172.21.15.66:0/607860260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe40d06b0 0x7f9fe40ce170 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009900 tx=0x7f9fcc01c850 comp rx=0 tx=0).stop 2024-09-10T22:04:45.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.503+0000 7f9feac00640 1 -- 172.21.15.66:0/607860260 shutdown_connections 2024-09-10T22:04:45.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.503+0000 7f9feac00640 1 --2- 172.21.15.66:0/607860260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe40d06b0 0x7f9fe40ce170 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.503+0000 7f9feac00640 1 -- 172.21.15.66:0/607860260 >> 172.21.15.66:0/607860260 conn(0x7f9fe40d1c60 msgr2=0x7f9fe40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.503+0000 7f9feac00640 1 -- 172.21.15.66:0/607860260 shutdown_connections 2024-09-10T22:04:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.503+0000 7f9feac00640 1 -- 172.21.15.66:0/607860260 wait complete. 2024-09-10T22:04:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9feac00640 1 Processor -- start 2024-09-10T22:04:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9feac00640 1 -- start start 2024-09-10T22:04:45.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9feac00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9fe40afc20 0x7f9fe41f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:45.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9feac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe40d06b0 0x7f9fe406e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:45.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9feac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fe415bd40 0x7f9fe41fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:45.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9feac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe416a110 con 0x7f9fe40d06b0 2024-09-10T22:04:45.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9feac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe4169f90 con 0x7f9fe415bd40 2024-09-10T22:04:45.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9feac00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe416a290 con 0x7f9fe40afc20 2024-09-10T22:04:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fdbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe40d06b0 0x7f9fe406e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fdbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe40d06b0 0x7f9fe406e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44440/0 (socket says 172.21.15.66:44440) 2024-09-10T22:04:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fdbe00640 1 -- 172.21.15.66:0/1272176398 learned_addr learned my addr 172.21.15.66:0/1272176398 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fe8c00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fe415bd40 0x7f9fe41fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fe3e00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9fe40afc20 0x7f9fe41f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fdbe00640 1 -- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9fe40afc20 msgr2=0x7f9fe41f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fdbe00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9fe40afc20 0x7f9fe41f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fdbe00640 1 -- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fe415bd40 msgr2=0x7f9fe41fbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fdbe00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fe415bd40 0x7f9fe41fbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fdbe00640 1 -- 172.21.15.66:0/1272176398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe406f280 con 0x7f9fe40d06b0 2024-09-10T22:04:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fdbe00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe40d06b0 0x7f9fe406e740 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0b0040 tx=0x7f9fcc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fe0c00640 1 -- 172.21.15.66:0/1272176398 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0b2040 con 0x7f9fe40d06b0 2024-09-10T22:04:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fe0c00640 1 -- 172.21.15.66:0/1272176398 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fcc003600 con 0x7f9fe40d06b0 2024-09-10T22:04:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9feac00640 1 -- 172.21.15.66:0/1272176398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe406f560 con 0x7f9fe40d06b0 2024-09-10T22:04:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9fe0c00640 1 -- 172.21.15.66:0/1272176398 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc003a10 con 0x7f9fe40d06b0 2024-09-10T22:04:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.507+0000 7f9feac00640 1 -- 172.21.15.66:0/1272176398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe41fb660 con 0x7f9fe40d06b0 2024-09-10T22:04:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.511+0000 7f9feac00640 1 -- 172.21.15.66:0/1272176398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe40cffa0 con 0x7f9fe40d06b0 2024-09-10T22:04:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.511+0000 7f9fe0c00640 1 -- 172.21.15.66:0/1272176398 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 97057+0+0 (secure 0 0 0) 0x7f9fcc003bb0 con 0x7f9fe40d06b0 2024-09-10T22:04:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.511+0000 7f9fe0c00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9fc0072d30 0x7f9fc00751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.511+0000 7f9fe0c00640 1 -- 172.21.15.66:0/1272176398 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f9fcc12dee0 con 0x7f9fe40d06b0 2024-09-10T22:04:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.511+0000 7f9fe3e00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9fc0072d30 0x7f9fc00751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.511+0000 7f9fe3e00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9fc0072d30 0x7f9fc00751f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000f70 tx=0x7f9fd4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:45.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.515+0000 7f9fe0c00640 1 -- 172.21.15.66:0/1272176398 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9fcc0164d0 con 0x7f9fe40d06b0 2024-09-10T22:04:45.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.723+0000 7f9feac00640 1 -- 172.21.15.66:0/1272176398 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9fe40b0df0 con 0x7f9fc0072d30 2024-09-10T22:04:45.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.755+0000 7f9fe0c00640 1 -- 172.21.15.66:0/1272176398 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9fe40b0df0 con 0x7f9fc0072d30 2024-09-10T22:04:45.761 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:04:45.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.759+0000 7f9feac00640 1 -- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9fc0072d30 msgr2=0x7f9fc00751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.759+0000 7f9feac00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9fc0072d30 0x7f9fc00751f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000f70 tx=0x7f9fd4009040 comp rx=0 tx=0).stop 2024-09-10T22:04:45.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.759+0000 7f9feac00640 1 -- 172.21.15.66:0/1272176398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe40d06b0 msgr2=0x7f9fe406e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.759+0000 7f9feac00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe40d06b0 0x7f9fe406e740 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0b0040 tx=0x7f9fcc004040 comp rx=0 tx=0).stop 2024-09-10T22:04:45.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.759+0000 7f9feac00640 1 -- 172.21.15.66:0/1272176398 shutdown_connections 2024-09-10T22:04:45.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.759+0000 7f9feac00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9fc0072d30 0x7f9fc00751f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.759+0000 7f9feac00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fe415bd40 0x7f9fe41fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.759+0000 7f9feac00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe40d06b0 0x7f9fe406e740 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.759+0000 7f9feac00640 1 --2- 172.21.15.66:0/1272176398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9fe40afc20 0x7f9fe41f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.759+0000 7f9feac00640 1 -- 172.21.15.66:0/1272176398 >> 172.21.15.66:0/1272176398 conn(0x7f9fe40d1c60 msgr2=0x7f9fe4071ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:45.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.759+0000 7f9feac00640 1 -- 172.21.15.66:0/1272176398 shutdown_connections 2024-09-10T22:04:45.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.759+0000 7f9feac00640 1 -- 172.21.15.66:0/1272176398 wait complete. 2024-09-10T22:04:45.783 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-10T22:04:45.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 -- 172.21.15.66:0/4108901495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdf80d0260 msgr2=0x7fcdf80ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 --2- 172.21.15.66:0/4108901495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdf80d0260 0x7fcdf80ae960 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcdf806d6f0 tx=0x7fcde001c8c0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 -- 172.21.15.66:0/4108901495 shutdown_connections 2024-09-10T22:04:45.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 --2- 172.21.15.66:0/4108901495 >> v2:172.21.15.94:3300/0 conn(0x7fcdf80b2bf0 0x7fcdf80aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 --2- 172.21.15.66:0/4108901495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdf80d0260 0x7fcdf80ae960 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 -- 172.21.15.66:0/4108901495 >> 172.21.15.66:0/4108901495 conn(0x7fcdf80d1c50 msgr2=0x7fcdf80d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:45.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 -- 172.21.15.66:0/4108901495 shutdown_connections 2024-09-10T22:04:45.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 -- 172.21.15.66:0/4108901495 wait complete. 2024-09-10T22:04:45.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 Processor -- start 2024-09-10T22:04:45.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 -- start start 2024-09-10T22:04:45.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcdf80b2bf0 0x7fcdf815c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:45.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdf80d0260 0x7fcdf8155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:45.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdf815f340 0x7fcdf80cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:45.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf81691e0 con 0x7fcdf80d0260 2024-09-10T22:04:45.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf8169060 con 0x7fcdf80b2bf0 2024-09-10T22:04:45.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdfec00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf8169360 con 0x7fcdf815f340 2024-09-10T22:04:45.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdefe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdf80d0260 0x7fcdf8155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:45.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdefe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdf80d0260 0x7fcdf8155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44458/0 (socket says 172.21.15.66:44458) 2024-09-10T22:04:45.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.931+0000 7fcdefe00640 1 -- 172.21.15.66:0/3667668973 learned_addr learned my addr 172.21.15.66:0/3667668973 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:45.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdfcc00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdf815f340 0x7fcdf80cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:45.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdefe00640 1 -- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdf815f340 msgr2=0x7fcdf80cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdf7e00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcdf80b2bf0 0x7fcdf815c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:45.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdefe00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdf815f340 0x7fcdf80cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdefe00640 1 -- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcdf80b2bf0 msgr2=0x7fcdf815c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:45.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdefe00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcdf80b2bf0 0x7fcdf815c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:45.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdefe00640 1 -- 172.21.15.66:0/3667668973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf815c6a0 con 0x7fcdf80d0260 2024-09-10T22:04:45.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdfcc00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdf815f340 0x7fcdf80cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:04:45.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdf7e00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcdf80b2bf0 0x7fcdf815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:04:45.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdefe00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdf80d0260 0x7fcdf8155840 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcde00194d0 tx=0x7fcde0005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:45.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdf4c00640 1 -- 172.21.15.66:0/3667668973 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00bb070 con 0x7fcdf80d0260 2024-09-10T22:04:45.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdf4c00640 1 -- 172.21.15.66:0/3667668973 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcde0005030 con 0x7fcdf80d0260 2024-09-10T22:04:45.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdfec00640 1 -- 172.21.15.66:0/3667668973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf80ce910 con 0x7fcdf80d0260 2024-09-10T22:04:45.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdf4c00640 1 -- 172.21.15.66:0/3667668973 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00b2040 con 0x7fcdf80d0260 2024-09-10T22:04:45.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdfec00640 1 -- 172.21.15.66:0/3667668973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf80ceed0 con 0x7fcdf80d0260 2024-09-10T22:04:45.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdfec00640 1 -- 172.21.15.66:0/3667668973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf80cfb50 con 0x7fcdf80d0260 2024-09-10T22:04:45.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdf4c00640 1 -- 172.21.15.66:0/3667668973 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 97057+0+0 (secure 0 0 0) 0x7fcde0005480 con 0x7fcdf80d0260 2024-09-10T22:04:45.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdf4c00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcdd4072d60 0x7fcdd4075220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:45.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.935+0000 7fcdf4c00640 1 -- 172.21.15.66:0/3667668973 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fcde012dbd0 con 0x7fcdf80d0260 2024-09-10T22:04:45.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.939+0000 7fcdf7e00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcdd4072d60 0x7fcdd4075220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:45.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.939+0000 7fcdf7e00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcdd4072d60 0x7fcdd4075220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcde8001630 tx=0x7fcde8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:45.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:45.939+0000 7fcdf4c00640 1 -- 172.21.15.66:0/3667668973 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcde00164b0 con 0x7fcdf80d0260 2024-09-10T22:04:46.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.139+0000 7fcdfec00640 1 -- 172.21.15.66:0/3667668973 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fcdf80cf520 con 0x7fcdd4072d60 2024-09-10T22:04:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.163+0000 7fcdf4c00640 1 -- 172.21.15.66:0/3667668973 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcdf80cf520 con 0x7fcdd4072d60 2024-09-10T22:04:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.163+0000 7fcdfec00640 1 -- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcdd4072d60 msgr2=0x7fcdd4075220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.163+0000 7fcdfec00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcdd4072d60 0x7fcdd4075220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcde8001630 tx=0x7fcde8009040 comp rx=0 tx=0).stop 2024-09-10T22:04:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.163+0000 7fcdfec00640 1 -- 172.21.15.66:0/3667668973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdf80d0260 msgr2=0x7fcdf8155840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.163+0000 7fcdfec00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdf80d0260 0x7fcdf8155840 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcde00194d0 tx=0x7fcde0005e90 comp rx=0 tx=0).stop 2024-09-10T22:04:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.167+0000 7fcdfec00640 1 -- 172.21.15.66:0/3667668973 shutdown_connections 2024-09-10T22:04:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.167+0000 7fcdfec00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcdd4072d60 0x7fcdd4075220 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.167+0000 7fcdfec00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdf815f340 0x7fcdf80cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.167+0000 7fcdfec00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdf80d0260 0x7fcdf8155840 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.167+0000 7fcdfec00640 1 --2- 172.21.15.66:0/3667668973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcdf80b2bf0 0x7fcdf815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.167+0000 7fcdfec00640 1 -- 172.21.15.66:0/3667668973 >> 172.21.15.66:0/3667668973 conn(0x7fcdf80d1c50 msgr2=0x7fcdf815fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.167+0000 7fcdfec00640 1 -- 172.21.15.66:0/3667668973 shutdown_connections 2024-09-10T22:04:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.167+0000 7fcdfec00640 1 -- 172.21.15.66:0/3667668973 wait complete. 2024-09-10T22:04:46.172 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:46.166+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:46.172 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:46.166+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:46.172 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:46.166+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:46.172 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:46.166+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:46.172 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:46.166+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:46.191 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:46.186+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:46.191 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:46.186+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:46.191 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:46.186+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:46.191 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:46.186+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:46.191 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:04:46.186+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:46.193 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone9020 2024-09-10T22:04:46.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 -- 172.21.15.66:0/1525222609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72d8003a10 msgr2=0x7f72d8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 --2- 172.21.15.66:0/1525222609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72d8003a10 0x7f72d8003e30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f72e400c470 tx=0x7f72e401cfa0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72f8e00640 1 -- 172.21.15.66:0/1525222609 reap_dead start 2024-09-10T22:04:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 -- 172.21.15.66:0/1525222609 shutdown_connections 2024-09-10T22:04:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 -- 172.21.15.66:0/1525222609 >> 172.21.15.66:0/1525222609 conn(0x7f72f40d1c60 msgr2=0x7f72f40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 -- 172.21.15.66:0/1525222609 shutdown_connections 2024-09-10T22:04:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 -- 172.21.15.66:0/1525222609 wait complete. 2024-09-10T22:04:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 Processor -- start 2024-09-10T22:04:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 -- start start 2024-09-10T22:04:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72d8003a10 0x7f72f4156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f72f41627b0 0x7f72f4164410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72f4070c00 0x7f72f41fb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f72f4168dc0 con 0x7f72d8003a10 2024-09-10T22:04:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72f4168c40 con 0x7f72f4070c00 2024-09-10T22:04:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72fae00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f72f4168f40 con 0x7f72f41627b0 2024-09-10T22:04:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72f3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72d8003a10 0x7f72f4156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72f3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72d8003a10 0x7f72f4156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44478/0 (socket says 172.21.15.66:44478) 2024-09-10T22:04:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72f3e00640 1 -- 172.21.15.66:0/722716222 learned_addr learned my addr 172.21.15.66:0/722716222 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72ebe00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f72f41627b0 0x7f72f4164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.347+0000 7f72f8e00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72f4070c00 0x7f72f41fb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72ebe00640 1 -- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72f4070c00 msgr2=0x7f72f41fb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72ebe00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72f4070c00 0x7f72f41fb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72ebe00640 1 -- 172.21.15.66:0/722716222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72d8003a10 msgr2=0x7f72f4156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72ebe00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72d8003a10 0x7f72f4156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72ebe00640 1 -- 172.21.15.66:0/722716222 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f41fc050 con 0x7f72f41627b0 2024-09-10T22:04:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72ebe00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f72f41627b0 0x7f72f4164410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f72dc003ea0 tx=0x7f72dc003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:46.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72f0c00640 1 -- 172.21.15.66:0/722716222 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc005220 con 0x7f72f41627b0 2024-09-10T22:04:46.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72f0c00640 1 -- 172.21.15.66:0/722716222 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72dc0053c0 con 0x7f72f41627b0 2024-09-10T22:04:46.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72f0c00640 1 -- 172.21.15.66:0/722716222 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0047b0 con 0x7f72f41627b0 2024-09-10T22:04:46.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72fae00640 1 -- 172.21.15.66:0/722716222 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f4162cb0 con 0x7f72f41627b0 2024-09-10T22:04:46.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72fae00640 1 -- 172.21.15.66:0/722716222 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f415f5b0 con 0x7f72f41627b0 2024-09-10T22:04:46.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72fae00640 1 -- 172.21.15.66:0/722716222 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72f406dcd0 con 0x7f72f41627b0 2024-09-10T22:04:46.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.351+0000 7f72f0c00640 1 -- 172.21.15.66:0/722716222 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 16) ==== 97057+0+0 (secure 0 0 0) 0x7f72dc03cba0 con 0x7f72f41627b0 2024-09-10T22:04:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.355+0000 7f72f0c00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f72d0072d60 0x7f72d0075220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.355+0000 7f72f0c00640 1 -- 172.21.15.66:0/722716222 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f72dc094b20 con 0x7f72f41627b0 2024-09-10T22:04:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.355+0000 7f72f3e00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f72d0072d60 0x7f72d0075220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.355+0000 7f72f3e00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f72d0072d60 0x7f72d0075220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f72e4000c10 tx=0x7f72e4019de0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:46.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.355+0000 7f72f0c00640 1 -- 172.21.15.66:0/722716222 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f72dc05efb0 con 0x7f72f41627b0 2024-09-10T22:04:46.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.555+0000 7f72fae00640 1 -- 172.21.15.66:0/722716222 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone9020", "target": ["mon-mgr", ""]}) -- 0x7f72f40af8c0 con 0x7f72d0072d60 2024-09-10T22:04:46.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72f0c00640 1 -- 172.21.15.66:0/722716222 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f72f40af8c0 con 0x7f72d0072d60 2024-09-10T22:04:46.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72fae00640 1 -- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f72d0072d60 msgr2=0x7f72d0075220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:46.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72fae00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f72d0072d60 0x7f72d0075220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f72e4000c10 tx=0x7f72e4019de0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72fae00640 1 -- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f72f41627b0 msgr2=0x7f72f4164410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:46.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72fae00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f72f41627b0 0x7f72f4164410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f72dc003ea0 tx=0x7f72dc003f90 comp rx=0 tx=0).stop 2024-09-10T22:04:46.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72fae00640 1 -- 172.21.15.66:0/722716222 shutdown_connections 2024-09-10T22:04:46.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72fae00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f72d0072d60 0x7f72d0075220 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72fae00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72f4070c00 0x7f72f41fb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72fae00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f72f41627b0 0x7f72f4164410 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72fae00640 1 --2- 172.21.15.66:0/722716222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72d8003a10 0x7f72f4156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72fae00640 1 -- 172.21.15.66:0/722716222 >> 172.21.15.66:0/722716222 conn(0x7f72f40d1c60 msgr2=0x7f72f415a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:46.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72fae00640 1 -- 172.21.15.66:0/722716222 shutdown_connections 2024-09-10T22:04:46.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.579+0000 7f72fae00640 1 -- 172.21.15.66:0/722716222 wait complete. 2024-09-10T22:04:46.612 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone21312 2024-09-10T22:04:46.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.763+0000 7fb1cde00640 1 -- 172.21.15.66:0/1752482174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c80cf1f0 msgr2=0x7fb1c80cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:46.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.763+0000 7fb1cde00640 1 --2- 172.21.15.66:0/1752482174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c80cf1f0 0x7fb1c80cff40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009bc0 tx=0x7fb1c001cba0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.763+0000 7fb1cde00640 1 -- 172.21.15.66:0/1752482174 shutdown_connections 2024-09-10T22:04:46.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.763+0000 7fb1cde00640 1 --2- 172.21.15.66:0/1752482174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c80cf1f0 0x7fb1c80cff40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.763+0000 7fb1cde00640 1 --2- 172.21.15.66:0/1752482174 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1c80ce8d0 0x7fb1c80cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.763+0000 7fb1cde00640 1 -- 172.21.15.66:0/1752482174 >> 172.21.15.66:0/1752482174 conn(0x7fb1c80d1c60 msgr2=0x7fb1c80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:46.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.763+0000 7fb1cde00640 1 -- 172.21.15.66:0/1752482174 shutdown_connections 2024-09-10T22:04:46.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.763+0000 7fb1cde00640 1 -- 172.21.15.66:0/1752482174 wait complete. 2024-09-10T22:04:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.763+0000 7fb1cde00640 1 Processor -- start 2024-09-10T22:04:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.763+0000 7fb1cde00640 1 -- start start 2024-09-10T22:04:46.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1cde00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1c80cdf90 0x7fb1c81653a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1cde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c80ce8d0 0x7fb1c8159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1cde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c80cf1f0 0x7fb1c81595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1cde00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81691a0 con 0x7fb1c80cf1f0 2024-09-10T22:04:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1cde00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8169020 con 0x7fb1c80ce8d0 2024-09-10T22:04:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1cde00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c8169320 con 0x7fb1c80cdf90 2024-09-10T22:04:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1c7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c80cf1f0 0x7fb1c81595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1c7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c80cf1f0 0x7fb1c81595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44482/0 (socket says 172.21.15.66:44482) 2024-09-10T22:04:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1c7400640 1 -- 172.21.15.66:0/2464984328 learned_addr learned my addr 172.21.15.66:0/2464984328 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1bfe00640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c80ce8d0 0x7fb1c8159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1c6a00640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1c80cdf90 0x7fb1c81653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1c7400640 1 -- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1c80cdf90 msgr2=0x7fb1c81653a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1c7400640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1c80cdf90 0x7fb1c81653a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1c7400640 1 -- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c80ce8d0 msgr2=0x7fb1c8159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1c7400640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c80ce8d0 0x7fb1c8159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1c7400640 1 -- 172.21.15.66:0/2464984328 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c81fb610 con 0x7fb1c80cf1f0 2024-09-10T22:04:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1c6a00640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1c80cdf90 0x7fb1c81653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:04:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1bfe00640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c80ce8d0 0x7fb1c8159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:04:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1c7400640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c80cf1f0 0x7fb1c81595b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb1c001a7d0 tx=0x7fb1c0003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1bf400640 1 -- 172.21.15.66:0/2464984328 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00bb070 con 0x7fb1c80cf1f0 2024-09-10T22:04:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1bf400640 1 -- 172.21.15.66:0/2464984328 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1c0004e00 con 0x7fb1c80cf1f0 2024-09-10T22:04:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1bf400640 1 -- 172.21.15.66:0/2464984328 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00b2040 con 0x7fb1c80cf1f0 2024-09-10T22:04:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1cde00640 1 -- 172.21.15.66:0/2464984328 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c81fb8f0 con 0x7fb1c80cf1f0 2024-09-10T22:04:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1cde00640 1 -- 172.21.15.66:0/2464984328 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c81fbeb0 con 0x7fb1c80cf1f0 2024-09-10T22:04:46.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.767+0000 7fb1cde00640 1 -- 172.21.15.66:0/2464984328 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c80cecb0 con 0x7fb1c80cf1f0 2024-09-10T22:04:46.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.771+0000 7fb1bf400640 1 -- 172.21.15.66:0/2464984328 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 97057+0+0 (secure 0 0 0) 0x7fb1c00b5030 con 0x7fb1c80cf1f0 2024-09-10T22:04:46.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.771+0000 7fb1bf400640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1a4072d30 0x7fb1a40751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:46.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.771+0000 7fb1bf400640 1 -- 172.21.15.66:0/2464984328 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fb1c012d890 con 0x7fb1c80cf1f0 2024-09-10T22:04:46.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.771+0000 7fb1c6a00640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1a4072d30 0x7fb1a40751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:46.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.771+0000 7fb1c6a00640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1a4072d30 0x7fb1a40751f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8005fd0 tx=0x7fb1b8005e60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:46.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.775+0000 7fb1bf400640 1 -- 172.21.15.66:0/2464984328 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb1c00a5e20 con 0x7fb1c80cf1f0 2024-09-10T22:04:46.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.907+0000 7fb1bf400640 1 -- 172.21.15.66:0/2464984328 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fb1c00f3ea0 con 0x7fb1c80cf1f0 2024-09-10T22:04:46.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:46.987+0000 7fb1cde00640 1 -- 172.21.15.66:0/2464984328 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone21312", "target": ["mon-mgr", ""]}) -- 0x7fb1c80cc100 con 0x7fb1a4072d30 2024-09-10T22:04:47.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1bf400640 1 -- 172.21.15.66:0/2464984328 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb1c80cc100 con 0x7fb1a4072d30 2024-09-10T22:04:47.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1cde00640 1 -- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1a4072d30 msgr2=0x7fb1a40751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:47.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1cde00640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1a4072d30 0x7fb1a40751f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8005fd0 tx=0x7fb1b8005e60 comp rx=0 tx=0).stop 2024-09-10T22:04:47.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1cde00640 1 -- 172.21.15.66:0/2464984328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c80cf1f0 msgr2=0x7fb1c81595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:47.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1cde00640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c80cf1f0 0x7fb1c81595b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb1c001a7d0 tx=0x7fb1c0003ae0 comp rx=0 tx=0).stop 2024-09-10T22:04:47.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1cde00640 1 -- 172.21.15.66:0/2464984328 shutdown_connections 2024-09-10T22:04:47.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1cde00640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1a4072d30 0x7fb1a40751f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:47.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1cde00640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c80cf1f0 0x7fb1c81595b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:47.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1cde00640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c80ce8d0 0x7fb1c8159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:47.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1cde00640 1 --2- 172.21.15.66:0/2464984328 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1c80cdf90 0x7fb1c81653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:47.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1cde00640 1 -- 172.21.15.66:0/2464984328 >> 172.21.15.66:0/2464984328 conn(0x7fb1c80d1c60 msgr2=0x7fb1c815b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:47.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1cde00640 1 -- 172.21.15.66:0/2464984328 shutdown_connections 2024-09-10T22:04:47.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.267+0000 7fb1cde00640 1 -- 172.21.15.66:0/2464984328 wait complete. 2024-09-10T22:04:47.296 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone27303 2024-09-10T22:04:47.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 -- 172.21.15.66:0/2716867857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78540ce8d0 msgr2=0x7f78540cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:47.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 --2- 172.21.15.66:0/2716867857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78540ce8d0 0x7f78540cecb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7844009840 tx=0x7f784401c820 comp rx=0 tx=0).stop 2024-09-10T22:04:47.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 -- 172.21.15.66:0/2716867857 shutdown_connections 2024-09-10T22:04:47.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 --2- 172.21.15.66:0/2716867857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78540cf1f0 0x7f78540cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:47.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 --2- 172.21.15.66:0/2716867857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78540ce8d0 0x7f78540cecb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 -- 172.21.15.66:0/2716867857 >> 172.21.15.66:0/2716867857 conn(0x7f78540d1c60 msgr2=0x7f78540d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 -- 172.21.15.66:0/2716867857 shutdown_connections 2024-09-10T22:04:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 -- 172.21.15.66:0/2716867857 wait complete. 2024-09-10T22:04:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 Processor -- start 2024-09-10T22:04:47.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 -- start start 2024-09-10T22:04:47.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78540cdf90 0x7f7854151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:47.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78540ce8d0 0x7f78541f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:47.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78540cf1f0 0x7f78541f9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:47.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f78541691a0 con 0x7f78540ce8d0 2024-09-10T22:04:47.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7854169020 con 0x7f78540cf1f0 2024-09-10T22:04:47.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f785a600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7854169320 con 0x7f78540cdf90 2024-09-10T22:04:47.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f784be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78540ce8d0 0x7f78541f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:47.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f7853e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78540cf1f0 0x7f78541f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:47.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f784be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78540ce8d0 0x7f78541f6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44498/0 (socket says 172.21.15.66:44498) 2024-09-10T22:04:47.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f7853400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78540cdf90 0x7f7854151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:47.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f784be00640 1 -- 172.21.15.66:0/2015636355 learned_addr learned my addr 172.21.15.66:0/2015636355 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:47.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f784be00640 1 -- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78540cdf90 msgr2=0x7f7854151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:47.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f784be00640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78540cdf90 0x7f7854151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:47.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f784be00640 1 -- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78540cf1f0 msgr2=0x7f78541f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:47.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f784be00640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78540cf1f0 0x7f78541f9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:47.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.447+0000 7f784be00640 1 -- 172.21.15.66:0/2015636355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78541f5d90 con 0x7f78540ce8d0 2024-09-10T22:04:47.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.451+0000 7f7853e00640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78540cf1f0 0x7f78541f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:04:47.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.451+0000 7f7853400640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78540cdf90 0x7f7854151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:04:47.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.451+0000 7f784be00640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78540ce8d0 0x7f78541f6810 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f783c00c430 tx=0x7f783c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:47.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.451+0000 7f784b400640 1 -- 172.21.15.66:0/2015636355 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c011790 con 0x7f78540ce8d0 2024-09-10T22:04:47.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.451+0000 7f784b400640 1 -- 172.21.15.66:0/2015636355 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f783c017070 con 0x7f78540ce8d0 2024-09-10T22:04:47.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.451+0000 7f784b400640 1 -- 172.21.15.66:0/2015636355 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c011d40 con 0x7f78540ce8d0 2024-09-10T22:04:47.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.451+0000 7f785a600640 1 -- 172.21.15.66:0/2015636355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78541f9e20 con 0x7f78540ce8d0 2024-09-10T22:04:47.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.451+0000 7f785a600640 1 -- 172.21.15.66:0/2015636355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78541f6010 con 0x7f78540ce8d0 2024-09-10T22:04:47.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.451+0000 7f785a600640 1 -- 172.21.15.66:0/2015636355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78540cecb0 con 0x7f78540ce8d0 2024-09-10T22:04:47.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.451+0000 7f784b400640 1 -- 172.21.15.66:0/2015636355 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f783c01c460 con 0x7f78540ce8d0 2024-09-10T22:04:47.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.455+0000 7f784b400640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7830072e50 0x7f7830075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:47.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.455+0000 7f784b400640 1 -- 172.21.15.66:0/2015636355 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7f783c091720 con 0x7f78540ce8d0 2024-09-10T22:04:47.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.455+0000 7f7853400640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7830072e50 0x7f7830075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:47.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.455+0000 7f7853400640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7830072e50 0x7f7830075310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f78440af000 tx=0x7f784401b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:47.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.459+0000 7f784b400640 1 -- 172.21.15.66:0/2015636355 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f78541f5d90 con 0x7f78540ce8d0 2024-09-10T22:04:47.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:47.659+0000 7f785a600640 1 -- 172.21.15.66:0/2015636355 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone27303", "target": ["mon-mgr", ""]}) -- 0x7f78541592f0 con 0x7f7830072e50 2024-09-10T22:04:51.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.579+0000 7f784b400640 1 -- 172.21.15.66:0/2015636355 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f78440040c0 con 0x7f7830072e50 2024-09-10T22:04:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.583+0000 7f785a600640 1 -- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7830072e50 msgr2=0x7f7830075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.583+0000 7f785a600640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7830072e50 0x7f7830075310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f78440af000 tx=0x7f784401b060 comp rx=0 tx=0).stop 2024-09-10T22:04:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.583+0000 7f785a600640 1 -- 172.21.15.66:0/2015636355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78540ce8d0 msgr2=0x7f78541f6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.583+0000 7f785a600640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78540ce8d0 0x7f78541f6810 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f783c00c430 tx=0x7f783c00c830 comp rx=0 tx=0).stop 2024-09-10T22:04:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.583+0000 7f785a600640 1 -- 172.21.15.66:0/2015636355 shutdown_connections 2024-09-10T22:04:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.583+0000 7f785a600640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7830072e50 0x7f7830075310 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.583+0000 7f785a600640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78540cf1f0 0x7f78541f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.583+0000 7f785a600640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78540ce8d0 0x7f78541f6810 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.583+0000 7f785a600640 1 --2- 172.21.15.66:0/2015636355 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78540cdf90 0x7f7854151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.583+0000 7f785a600640 1 -- 172.21.15.66:0/2015636355 >> 172.21.15.66:0/2015636355 conn(0x7f78540d1c60 msgr2=0x7f7854159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.583+0000 7f785a600640 1 -- 172.21.15.66:0/2015636355 shutdown_connections 2024-09-10T22:04:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.583+0000 7f785a600640 1 -- 172.21.15.66:0/2015636355 wait complete. 2024-09-10T22:04:51.617 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone24791 2024-09-10T22:04:51.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 -- 172.21.15.66:0/2976520371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b015f470 msgr2=0x7fd8b00cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:51.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 --2- 172.21.15.66:0/2976520371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b015f470 0x7fd8b00cd0a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0073840 tx=0x7fd8a001c890 comp rx=0 tx=0).stop 2024-09-10T22:04:51.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 -- 172.21.15.66:0/2976520371 shutdown_connections 2024-09-10T22:04:51.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 --2- 172.21.15.66:0/2976520371 >> v2:172.21.15.94:3300/0 conn(0x7fd8b00d05c0 0x7fd8b00aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 --2- 172.21.15.66:0/2976520371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b015f470 0x7fd8b00cd0a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 -- 172.21.15.66:0/2976520371 >> 172.21.15.66:0/2976520371 conn(0x7fd8b00d1c60 msgr2=0x7fd8b00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 -- 172.21.15.66:0/2976520371 shutdown_connections 2024-09-10T22:04:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 -- 172.21.15.66:0/2976520371 wait complete. 2024-09-10T22:04:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 Processor -- start 2024-09-10T22:04:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 -- start start 2024-09-10T22:04:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00cfa40 0x7fd8b0157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b00d05c0 0x7fd8b015a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8b015f470 0x7fd8b015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b0168b30 con 0x7fd8b00cfa40 2024-09-10T22:04:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b01689b0 con 0x7fd8b00d05c0 2024-09-10T22:04:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8b5a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b0168cb0 con 0x7fd8b015f470 2024-09-10T22:04:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8aea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00cfa40 0x7fd8b0157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8aea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00cfa40 0x7fd8b0157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44514/0 (socket says 172.21.15.66:44514) 2024-09-10T22:04:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8aea00640 1 -- 172.21.15.66:0/1846513142 learned_addr learned my addr 172.21.15.66:0/1846513142 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8a6000640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b00d05c0 0x7fd8b015a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8af400640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8b015f470 0x7fd8b015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8aea00640 1 -- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8b015f470 msgr2=0x7fd8b015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8aea00640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8b015f470 0x7fd8b015a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8aea00640 1 -- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b00d05c0 msgr2=0x7fd8b015a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8aea00640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b00d05c0 0x7fd8b015a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8aea00640 1 -- 172.21.15.66:0/1846513142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b01f7440 con 0x7fd8b00cfa40 2024-09-10T22:04:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.767+0000 7fd8aea00640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00cfa40 0x7fd8b0157820 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00b0040 tx=0x7fd8a0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.771+0000 7fd8a7400640 1 -- 172.21.15.66:0/1846513142 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00b2040 con 0x7fd8b00cfa40 2024-09-10T22:04:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.771+0000 7fd8a7400640 1 -- 172.21.15.66:0/1846513142 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8a0004430 con 0x7fd8b00cfa40 2024-09-10T22:04:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.771+0000 7fd8a7400640 1 -- 172.21.15.66:0/1846513142 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0005290 con 0x7fd8b00cfa40 2024-09-10T22:04:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.771+0000 7fd8b5a00640 1 -- 172.21.15.66:0/1846513142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b01f7700 con 0x7fd8b00cfa40 2024-09-10T22:04:51.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.771+0000 7fd8b5a00640 1 -- 172.21.15.66:0/1846513142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b01f7cc0 con 0x7fd8b00cfa40 2024-09-10T22:04:51.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.771+0000 7fd8b5a00640 1 -- 172.21.15.66:0/1846513142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b00aebd0 con 0x7fd8b00cfa40 2024-09-10T22:04:51.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.771+0000 7fd8a7400640 1 -- 172.21.15.66:0/1846513142 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fd8a0005430 con 0x7fd8b00cfa40 2024-09-10T22:04:51.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.771+0000 7fd8a7400640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd88c072e50 0x7fd88c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:51.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.771+0000 7fd8a7400640 1 -- 172.21.15.66:0/1846513142 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7fd8a012de80 con 0x7fd8b00cfa40 2024-09-10T22:04:51.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.771+0000 7fd8a6000640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd88c072e50 0x7fd88c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:51.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.775+0000 7fd8a6000640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd88c072e50 0x7fd88c075310 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd8b015ec90 tx=0x7fd898008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:51.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.775+0000 7fd8a7400640 1 -- 172.21.15.66:0/1846513142 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd8a00164b0 con 0x7fd8b00cfa40 2024-09-10T22:04:51.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.979+0000 7fd8b5a00640 1 -- 172.21.15.66:0/1846513142 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone24791", "target": ["mon-mgr", ""]}) -- 0x7fd8b0158580 con 0x7fd88c072e50 2024-09-10T22:04:52.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.995+0000 7fd8a7400640 1 -- 172.21.15.66:0/1846513142 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd8b0158580 con 0x7fd88c072e50 2024-09-10T22:04:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.999+0000 7fd8b5a00640 1 -- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd88c072e50 msgr2=0x7fd88c075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.999+0000 7fd8b5a00640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd88c072e50 0x7fd88c075310 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd8b015ec90 tx=0x7fd898008040 comp rx=0 tx=0).stop 2024-09-10T22:04:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.999+0000 7fd8b5a00640 1 -- 172.21.15.66:0/1846513142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00cfa40 msgr2=0x7fd8b0157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.999+0000 7fd8b5a00640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00cfa40 0x7fd8b0157820 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00b0040 tx=0x7fd8a0004040 comp rx=0 tx=0).stop 2024-09-10T22:04:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.999+0000 7fd8b5a00640 1 -- 172.21.15.66:0/1846513142 shutdown_connections 2024-09-10T22:04:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.999+0000 7fd8b5a00640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd88c072e50 0x7fd88c075310 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.999+0000 7fd8b5a00640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8b015f470 0x7fd8b015a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.999+0000 7fd8b5a00640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b00d05c0 0x7fd8b015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.999+0000 7fd8b5a00640 1 --2- 172.21.15.66:0/1846513142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00cfa40 0x7fd8b0157820 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.999+0000 7fd8b5a00640 1 -- 172.21.15.66:0/1846513142 >> 172.21.15.66:0/1846513142 conn(0x7fd8b00d1c60 msgr2=0x7fd8b006ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.999+0000 7fd8b5a00640 1 -- 172.21.15.66:0/1846513142 shutdown_connections 2024-09-10T22:04:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:51.999+0000 7fd8b5a00640 1 -- 172.21.15.66:0/1846513142 wait complete. 2024-09-10T22:04:52.030 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T22:04:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:04:52.088 INFO:teuthology.orchestra.run.smithi066.stdout:2 2024-09-10T22:04:52.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:04:52.095 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:04:52.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.283+0000 7fd9fbe00640 1 -- 172.21.15.66:0/3383476546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f40706c0 msgr2=0x7fd9f40c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:52.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.283+0000 7fd9fbe00640 1 --2- 172.21.15.66:0/3383476546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f40706c0 0x7fd9f40c48b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800aa10 tx=0x7fd9e801dc00 comp rx=0 tx=0).stop 2024-09-10T22:04:52.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.283+0000 7fd9fbe00640 1 -- 172.21.15.66:0/3383476546 shutdown_connections 2024-09-10T22:04:52.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.283+0000 7fd9fbe00640 1 --2- 172.21.15.66:0/3383476546 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9f40c4df0 0x7fd9f406e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.283+0000 7fd9fbe00640 1 --2- 172.21.15.66:0/3383476546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f40706c0 0x7fd9f40c48b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.283+0000 7fd9fbe00640 1 -- 172.21.15.66:0/3383476546 >> 172.21.15.66:0/3383476546 conn(0x7fd9f40cfa30 msgr2=0x7fd9f40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:52.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.283+0000 7fd9fbe00640 1 -- 172.21.15.66:0/3383476546 shutdown_connections 2024-09-10T22:04:52.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.283+0000 7fd9fbe00640 1 -- 172.21.15.66:0/3383476546 wait complete. 2024-09-10T22:04:52.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9fbe00640 1 Processor -- start 2024-09-10T22:04:52.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9fbe00640 1 -- start start 2024-09-10T22:04:52.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9fbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9f406eb70 0x7fd9f415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:52.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9fbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f40706c0 0x7fd9f413e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:52.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9fbe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9f40c4df0 0x7fd9f40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:52.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9fbe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f415b2a0 con 0x7fd9f40706c0 2024-09-10T22:04:52.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9fbe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f415b120 con 0x7fd9f406eb70 2024-09-10T22:04:52.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9fbe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f4166c50 con 0x7fd9f40c4df0 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f40706c0 0x7fd9f413e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f40706c0 0x7fd9f413e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44538/0 (socket says 172.21.15.66:44538) 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f8a00640 1 -- 172.21.15.66:0/2509243422 learned_addr learned my addr 172.21.15.66:0/2509243422 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f9e00640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9f40c4df0 0x7fd9f40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f9400640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9f406eb70 0x7fd9f415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f8a00640 1 -- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9f40c4df0 msgr2=0x7fd9f40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f8a00640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9f40c4df0 0x7fd9f40aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f8a00640 1 -- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9f406eb70 msgr2=0x7fd9f415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f8a00640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9f406eb70 0x7fd9f415ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f8a00640 1 -- 172.21.15.66:0/2509243422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f415f140 con 0x7fd9f40706c0 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f8a00640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f40706c0 0x7fd9f413e4a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00058d0 tx=0x7fd9e000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f1600640 1 -- 172.21.15.66:0/2509243422 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0017070 con 0x7fd9f40706c0 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f1600640 1 -- 172.21.15.66:0/2509243422 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9e0009d20 con 0x7fd9f40706c0 2024-09-10T22:04:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9f1600640 1 -- 172.21.15.66:0/2509243422 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e00106c0 con 0x7fd9f40706c0 2024-09-10T22:04:52.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9fbe00640 1 -- 172.21.15.66:0/2509243422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f40cf240 con 0x7fd9f40706c0 2024-09-10T22:04:52.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.287+0000 7fd9fbe00640 1 -- 172.21.15.66:0/2509243422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f415fa70 con 0x7fd9f40706c0 2024-09-10T22:04:52.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.291+0000 7fd9fbe00640 1 -- 172.21.15.66:0/2509243422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f40c48b0 con 0x7fd9f40706c0 2024-09-10T22:04:52.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.291+0000 7fd9f1600640 1 -- 172.21.15.66:0/2509243422 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fd9e0010860 con 0x7fd9f40706c0 2024-09-10T22:04:52.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.291+0000 7fd9f1600640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd9d4072e50 0x7fd9d4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:52.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.291+0000 7fd9f1600640 1 -- 172.21.15.66:0/2509243422 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7fd9e0094a00 con 0x7fd9f40706c0 2024-09-10T22:04:52.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.291+0000 7fd9f9400640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd9d4072e50 0x7fd9d4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:52.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.291+0000 7fd9f9400640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd9d4072e50 0x7fd9d4075310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80b0000 tx=0x7fd9e801c440 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:52.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.295+0000 7fd9f1600640 1 -- 172.21.15.66:0/2509243422 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd9e005e9f0 con 0x7fd9f40706c0 2024-09-10T22:04:52.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.467+0000 7fd9fbe00640 1 -- 172.21.15.66:0/2509243422 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fd9f40ce600 con 0x7fd9f40706c0 2024-09-10T22:04:52.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.467+0000 7fd9f1600640 1 -- 172.21.15.66:0/2509243422 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v32)=0 v32) ==== 131+0+0 (secure 0 0 0) 0x7fd9e0063420 con 0x7fd9f40706c0 2024-09-10T22:04:52.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.471+0000 7fd9fbe00640 1 -- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd9d4072e50 msgr2=0x7fd9d4075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:52.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.471+0000 7fd9fbe00640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd9d4072e50 0x7fd9d4075310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80b0000 tx=0x7fd9e801c440 comp rx=0 tx=0).stop 2024-09-10T22:04:52.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.471+0000 7fd9fbe00640 1 -- 172.21.15.66:0/2509243422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f40706c0 msgr2=0x7fd9f413e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:52.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.471+0000 7fd9fbe00640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f40706c0 0x7fd9f413e4a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00058d0 tx=0x7fd9e000dde0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.471+0000 7fd9fbe00640 1 -- 172.21.15.66:0/2509243422 shutdown_connections 2024-09-10T22:04:52.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.471+0000 7fd9fbe00640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd9d4072e50 0x7fd9d4075310 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.471+0000 7fd9fbe00640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9f40c4df0 0x7fd9f40aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.471+0000 7fd9fbe00640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f40706c0 0x7fd9f413e4a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.471+0000 7fd9fbe00640 1 --2- 172.21.15.66:0/2509243422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9f406eb70 0x7fd9f415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.471+0000 7fd9fbe00640 1 -- 172.21.15.66:0/2509243422 >> 172.21.15.66:0/2509243422 conn(0x7fd9f40cfa30 msgr2=0x7fd9f40cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:52.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.471+0000 7fd9fbe00640 1 -- 172.21.15.66:0/2509243422 shutdown_connections 2024-09-10T22:04:52.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.471+0000 7fd9fbe00640 1 -- 172.21.15.66:0/2509243422 wait complete. 2024-09-10T22:04:52.495 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:04:52.495 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:04:52.580 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:04:52.663 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:04:52.663 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:04:52.663 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:04:52.663 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:04:52.672 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:04:52.673 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:04:52.725 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:04:52.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.911+0000 7fb6d2600640 1 -- 172.21.15.66:0/2427597594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6cc15f460 msgr2=0x7fb6cc0cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.911+0000 7fb6d2600640 1 --2- 172.21.15.66:0/2427597594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6cc15f460 0x7fb6cc0cfc00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc069920 tx=0x7fb6bc01c4b0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.911+0000 7fb6d2600640 1 -- 172.21.15.66:0/2427597594 shutdown_connections 2024-09-10T22:04:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.911+0000 7fb6d2600640 1 --2- 172.21.15.66:0/2427597594 >> v2:172.21.15.94:3300/0 conn(0x7fb6cc0b32a0 0x7fb6cc0b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.911+0000 7fb6d2600640 1 --2- 172.21.15.66:0/2427597594 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6cc0b0f60 0x7fb6cc0d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.911+0000 7fb6d2600640 1 --2- 172.21.15.66:0/2427597594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6cc15f460 0x7fb6cc0cfc00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.911+0000 7fb6d2600640 1 -- 172.21.15.66:0/2427597594 >> 172.21.15.66:0/2427597594 conn(0x7fb6cc0d1d00 msgr2=0x7fb6cc0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.911+0000 7fb6d2600640 1 -- 172.21.15.66:0/2427597594 shutdown_connections 2024-09-10T22:04:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.911+0000 7fb6d2600640 1 -- 172.21.15.66:0/2427597594 wait complete. 2024-09-10T22:04:52.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6d2600640 1 Processor -- start 2024-09-10T22:04:52.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6d2600640 1 -- start start 2024-09-10T22:04:52.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6d2600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6cc0b0f60 0x7fb6cc0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:52.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6d2600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6cc0b32a0 0x7fb6cc150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:52.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6d2600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6cc15f460 0x7fb6cc145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:52.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6d2600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc0af720 con 0x7fb6cc0b0f60 2024-09-10T22:04:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6d2600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc0af5a0 con 0x7fb6cc15f460 2024-09-10T22:04:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6d2600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb6cc0af8a0 con 0x7fb6cc0b32a0 2024-09-10T22:04:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6cb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6cc0b0f60 0x7fb6cc0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6cb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6cc0b0f60 0x7fb6cc0b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39316/0 (socket says 172.21.15.66:39316) 2024-09-10T22:04:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6cb400640 1 -- 172.21.15.66:0/3475067152 learned_addr learned my addr 172.21.15.66:0/3475067152 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6cbe00640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6cc15f460 0x7fb6cc145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6caa00640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6cc0b32a0 0x7fb6cc150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6cb400640 1 -- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6cc0b32a0 msgr2=0x7fb6cc150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6cb400640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6cc0b32a0 0x7fb6cc150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6cb400640 1 -- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6cc15f460 msgr2=0x7fb6cc145ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6cb400640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6cc15f460 0x7fb6cc145ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6cb400640 1 -- 172.21.15.66:0/3475067152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6cc146ac0 con 0x7fb6cc0b0f60 2024-09-10T22:04:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6cb400640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6cc0b0f60 0x7fb6cc0b73e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc01c990 tx=0x7fb6bc0a94e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6c3400640 1 -- 172.21.15.66:0/3475067152 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0ad070 con 0x7fb6cc0b0f60 2024-09-10T22:04:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6c3400640 1 -- 172.21.15.66:0/3475067152 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6bc0a9d30 con 0x7fb6cc0b0f60 2024-09-10T22:04:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6c3400640 1 -- 172.21.15.66:0/3475067152 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc005290 con 0x7fb6cc0b0f60 2024-09-10T22:04:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6d2600640 1 -- 172.21.15.66:0/3475067152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6cc0b7920 con 0x7fb6cc0b0f60 2024-09-10T22:04:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6d2600640 1 -- 172.21.15.66:0/3475067152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6cc0b4b30 con 0x7fb6cc0b0f60 2024-09-10T22:04:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.915+0000 7fb6d2600640 1 -- 172.21.15.66:0/3475067152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6cc0b14b0 con 0x7fb6cc0b0f60 2024-09-10T22:04:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.919+0000 7fb6c3400640 1 -- 172.21.15.66:0/3475067152 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fb6bc005430 con 0x7fb6cc0b0f60 2024-09-10T22:04:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.919+0000 7fb6c3400640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6ac072e80 0x7fb6ac075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.919+0000 7fb6c3400640 1 -- 172.21.15.66:0/3475067152 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7fb6bc12c2d0 con 0x7fb6cc0b0f60 2024-09-10T22:04:52.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.919+0000 7fb6caa00640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6ac072e80 0x7fb6ac075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:52.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.919+0000 7fb6caa00640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6ac072e80 0x7fb6ac075340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc14b450 tx=0x7fb6b800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:52.923+0000 7fb6c3400640 1 -- 172.21.15.66:0/3475067152 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb6bc0164b0 con 0x7fb6cc0b0f60 2024-09-10T22:04:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.111+0000 7fb6d2600640 1 -- 172.21.15.66:0/3475067152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb6cc15cb20 con 0x7fb6cc0b0f60 2024-09-10T22:04:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.111+0000 7fb6c3400640 1 -- 172.21.15.66:0/3475067152 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) ==== 80+0+0 (secure 0 0 0) 0x7fb6bc0f8d90 con 0x7fb6cc0b0f60 2024-09-10T22:04:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:04:53.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.111+0000 7fb6d2600640 1 -- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6ac072e80 msgr2=0x7fb6ac075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.111+0000 7fb6d2600640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6ac072e80 0x7fb6ac075340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc14b450 tx=0x7fb6b800b2e0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.111+0000 7fb6d2600640 1 -- 172.21.15.66:0/3475067152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6cc0b0f60 msgr2=0x7fb6cc0b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.111+0000 7fb6d2600640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6cc0b0f60 0x7fb6cc0b73e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc01c990 tx=0x7fb6bc0a94e0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.111+0000 7fb6d2600640 1 -- 172.21.15.66:0/3475067152 shutdown_connections 2024-09-10T22:04:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.111+0000 7fb6d2600640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6ac072e80 0x7fb6ac075340 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.111+0000 7fb6d2600640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6cc15f460 0x7fb6cc145ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.111+0000 7fb6d2600640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6cc0b32a0 0x7fb6cc150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.111+0000 7fb6d2600640 1 --2- 172.21.15.66:0/3475067152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6cc0b0f60 0x7fb6cc0b73e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.111+0000 7fb6d2600640 1 -- 172.21.15.66:0/3475067152 >> 172.21.15.66:0/3475067152 conn(0x7fb6cc0d1d00 msgr2=0x7fb6cc1479c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.115+0000 7fb6d2600640 1 -- 172.21.15.66:0/3475067152 shutdown_connections 2024-09-10T22:04:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.115+0000 7fb6d2600640 1 -- 172.21.15.66:0/3475067152 wait complete. 2024-09-10T22:04:53.137 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-09-10T22:04:53.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:04:53.137 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T22:04:53.158 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:04:53.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:04:53.158 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:04:53.159 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[38894]: fuse finished with error 0 and tester_r 0 2024-09-10T22:04:53.330 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:04:53.331 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:04:53.331 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:04:53.340 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:04:53.340 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:04:53.340 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:04:53.340 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:04:53.381 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:04:53.381 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:04:53.381 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:04:53.382 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:04:53.391 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:04:53.391 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:04:53.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.567+0000 7f9073e00640 1 -- 172.21.15.66:0/851474638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90740d04c0 msgr2=0x7f907415ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:53.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.567+0000 7f9073e00640 1 --2- 172.21.15.66:0/851474638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90740d04c0 0x7f907415ca20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9064009870 tx=0x7f906401c7d0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.567+0000 7f9073e00640 1 -- 172.21.15.66:0/851474638 shutdown_connections 2024-09-10T22:04:53.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.567+0000 7f9073e00640 1 --2- 172.21.15.66:0/851474638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90740d04c0 0x7f907415ca20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.567+0000 7f9073e00640 1 -- 172.21.15.66:0/851474638 >> 172.21.15.66:0/851474638 conn(0x7f90740cd4b0 msgr2=0x7f90740cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:53.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.567+0000 7f9073e00640 1 -- 172.21.15.66:0/851474638 shutdown_connections 2024-09-10T22:04:53.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.567+0000 7f9073e00640 1 -- 172.21.15.66:0/851474638 wait complete. 2024-09-10T22:04:53.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.567+0000 7f9073e00640 1 Processor -- start 2024-09-10T22:04:53.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.567+0000 7f9073e00640 1 -- start start 2024-09-10T22:04:53.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907406cc20 0x7f9074141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:53.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90740ce040 0x7f90740c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740d04c0 0x7f90740ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f907416a2e0 con 0x7f90740d04c0 2024-09-10T22:04:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f907416a160 con 0x7f90740ce040 2024-09-10T22:04:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f907416a460 con 0x7f907406cc20 2024-09-10T22:04:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740d04c0 0x7f90740ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9072a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907406cc20 0x7f9074141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740d04c0 0x7f90740ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39318/0 (socket says 172.21.15.66:39318) 2024-09-10T22:04:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073400640 1 -- 172.21.15.66:0/3550343134 learned_addr learned my addr 172.21.15.66:0/3550343134 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9072000640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90740ce040 0x7f90740c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073400640 1 -- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907406cc20 msgr2=0x7f9074141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073400640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907406cc20 0x7f9074141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073400640 1 -- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90740ce040 msgr2=0x7f90740c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:53.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073400640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90740ce040 0x7f90740c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073400640 1 -- 172.21.15.66:0/3550343134 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9074150bf0 con 0x7f90740d04c0 2024-09-10T22:04:53.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073400640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740d04c0 0x7f90740ca280 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f906c00c790 tx=0x7f906c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:53.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f906aa00640 1 -- 172.21.15.66:0/3550343134 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c003b80 con 0x7f90740d04c0 2024-09-10T22:04:53.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f906aa00640 1 -- 172.21.15.66:0/3550343134 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f906c003d20 con 0x7f90740d04c0 2024-09-10T22:04:53.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073e00640 1 -- 172.21.15.66:0/3550343134 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9074150ed0 con 0x7f90740d04c0 2024-09-10T22:04:53.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f906aa00640 1 -- 172.21.15.66:0/3550343134 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c00aab0 con 0x7f90740d04c0 2024-09-10T22:04:53.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073e00640 1 -- 172.21.15.66:0/3550343134 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9074151490 con 0x7f90740d04c0 2024-09-10T22:04:53.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.571+0000 7f9073e00640 1 -- 172.21.15.66:0/3550343134 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907415ca20 con 0x7f90740d04c0 2024-09-10T22:04:53.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.575+0000 7f906aa00640 1 -- 172.21.15.66:0/3550343134 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f906c00ac50 con 0x7f90740d04c0 2024-09-10T22:04:53.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.575+0000 7f906aa00640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9054072e50 0x7f9054075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.575+0000 7f906aa00640 1 -- 172.21.15.66:0/3550343134 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7f906c0942b0 con 0x7f90740d04c0 2024-09-10T22:04:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.575+0000 7f9072a00640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9054072e50 0x7f9054075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.575+0000 7f9072a00640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9054072e50 0x7f9054075310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f90640af000 tx=0x7f906401b010 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:53.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.579+0000 7f906aa00640 1 -- 172.21.15.66:0/3550343134 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f906c05e330 con 0x7f90740d04c0 2024-09-10T22:04:53.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 -- 172.21.15.66:0/3550343134 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90740cea60 con 0x7f90740d04c0 2024-09-10T22:04:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f906aa00640 1 -- 172.21.15.66:0/3550343134 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5223 (secure 0 0 0) 0x7f906c0670a0 con 0x7f90740d04c0 2024-09-10T22:04:53.791 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:04:53.791 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"btime":"2024-09-10T22:03:15:051273+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5248,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2731842573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2731842573},{"type":"v1","addr":"172.21.15.94:6837","nonce":2731842573}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1324134239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1324134239},{"type":"v1","addr":"172.21.15.66:6833","nonce":1324134239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1694406832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1694406832},{"type":"v1","addr":"172.21.15.94:6836","nonce":1694406832}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:03:05.416734+0000","modified":"2024-09-10T22:03:08.421387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5412":{"gid":5412,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1215985114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1215985114},{"type":"v1","addr":"172.21.15.66:6835","nonce":1215985114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5412,"qdb_cluster":[5412]},"id":3}]} 2024-09-10T22:04:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2024-09-10T22:04:53.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 -- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9054072e50 msgr2=0x7f9054075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9054072e50 0x7f9054075310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f90640af000 tx=0x7f906401b010 comp rx=0 tx=0).stop 2024-09-10T22:04:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 -- 172.21.15.66:0/3550343134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740d04c0 msgr2=0x7f90740ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740d04c0 0x7f90740ca280 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f906c00c790 tx=0x7f906c00cc50 comp rx=0 tx=0).stop 2024-09-10T22:04:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 -- 172.21.15.66:0/3550343134 shutdown_connections 2024-09-10T22:04:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9054072e50 0x7f9054075310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740d04c0 0x7f90740ca280 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90740ce040 0x7f90740c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 --2- 172.21.15.66:0/3550343134 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907406cc20 0x7f9074141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 -- 172.21.15.66:0/3550343134 >> 172.21.15.66:0/3550343134 conn(0x7f90740cd4b0 msgr2=0x7f90740b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 -- 172.21.15.66:0/3550343134 shutdown_connections 2024-09-10T22:04:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.787+0000 7f9073e00640 1 -- 172.21.15.66:0/3550343134 wait complete. 2024-09-10T22:04:53.813 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:04:53.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.955+0000 7f22bb200640 1 -- 172.21.15.66:0/3058545948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22bc15f300 msgr2=0x7f22bc0b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:53.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.955+0000 7f22bb200640 1 --2- 172.21.15.66:0/3058545948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22bc15f300 0x7f22bc0b1090 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009840 tx=0x7f22a401c7e0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.955+0000 7f22bb200640 1 -- 172.21.15.66:0/3058545948 shutdown_connections 2024-09-10T22:04:53.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.955+0000 7f22bb200640 1 --2- 172.21.15.66:0/3058545948 >> v2:172.21.15.94:3300/0 conn(0x7f22bc0b15d0 0x7f22bc0cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.955+0000 7f22bb200640 1 --2- 172.21.15.66:0/3058545948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22bc15f300 0x7f22bc0b1090 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.955+0000 7f22bb200640 1 -- 172.21.15.66:0/3058545948 >> 172.21.15.66:0/3058545948 conn(0x7f22bc0d1b10 msgr2=0x7f22bc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:53.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.955+0000 7f22bb200640 1 -- 172.21.15.66:0/3058545948 shutdown_connections 2024-09-10T22:04:53.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.955+0000 7f22bb200640 1 -- 172.21.15.66:0/3058545948 wait complete. 2024-09-10T22:04:53.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22bb200640 1 Processor -- start 2024-09-10T22:04:53.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22bb200640 1 -- start start 2024-09-10T22:04:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22bb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22bc0b15d0 0x7f22bc15e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22bb200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22bc15ef20 0x7f22bc0cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22bb200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22bc15f300 0x7f22bc0cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22bb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc1692c0 con 0x7f22bc0b15d0 2024-09-10T22:04:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22bb200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc169140 con 0x7f22bc15f300 2024-09-10T22:04:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22bb200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f22bc169440 con 0x7f22bc15ef20 2024-09-10T22:04:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22bc0b15d0 0x7f22bc15e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22bc0b15d0 0x7f22bc15e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39346/0 (socket says 172.21.15.66:39346) 2024-09-10T22:04:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b9e00640 1 -- 172.21.15.66:0/2523711627 learned_addr learned my addr 172.21.15.66:0/2523711627 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b1400640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22bc15ef20 0x7f22bc0cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22ba800640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22bc15f300 0x7f22bc0cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b9e00640 1 -- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22bc15ef20 msgr2=0x7f22bc0cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b9e00640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22bc15ef20 0x7f22bc0cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b9e00640 1 -- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22bc15f300 msgr2=0x7f22bc0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b9e00640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22bc15f300 0x7f22bc0cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b9e00640 1 -- 172.21.15.66:0/2523711627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc0b5d00 con 0x7f22bc0b15d0 2024-09-10T22:04:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b9e00640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22bc0b15d0 0x7f22bc15e140 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c490 tx=0x7f22ac00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b2a00640 1 -- 172.21.15.66:0/2523711627 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac016020 con 0x7f22bc0b15d0 2024-09-10T22:04:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b2a00640 1 -- 172.21.15.66:0/2523711627 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22ac012940 con 0x7f22bc0b15d0 2024-09-10T22:04:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22b2a00640 1 -- 172.21.15.66:0/2523711627 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac012d90 con 0x7f22bc0b15d0 2024-09-10T22:04:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22bb200640 1 -- 172.21.15.66:0/2523711627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22bc0b5fe0 con 0x7f22bc0b15d0 2024-09-10T22:04:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22bb200640 1 -- 172.21.15.66:0/2523711627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22bc0b65a0 con 0x7f22bc0b15d0 2024-09-10T22:04:53.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.959+0000 7f22bb200640 1 -- 172.21.15.66:0/2523711627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22bc0cd7a0 con 0x7f22bc0b15d0 2024-09-10T22:04:53.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.963+0000 7f22b2a00640 1 -- 172.21.15.66:0/2523711627 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f22ac01b400 con 0x7f22bc0b15d0 2024-09-10T22:04:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.963+0000 7f22b2a00640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2298072e50 0x7f2298075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.963+0000 7f22b2a00640 1 -- 172.21.15.66:0/2523711627 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7f22ac0946a0 con 0x7f22bc0b15d0 2024-09-10T22:04:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.963+0000 7f22b1400640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2298072e50 0x7f2298075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.963+0000 7f22b1400640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2298072e50 0x7f2298075310 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f22a40af000 tx=0x7f22a401b020 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:53.967+0000 7f22b2a00640 1 -- 172.21.15.66:0/2523711627 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f22ac020020 con 0x7f22bc0b15d0 2024-09-10T22:04:54.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.223+0000 7f22bb200640 1 -- 172.21.15.66:0/2523711627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22bc072e00 con 0x7f22bc0b15d0 2024-09-10T22:04:54.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.227+0000 7f22b2a00640 1 -- 172.21.15.66:0/2523711627 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5223 (secure 0 0 0) 0x7f22ac05e7e0 con 0x7f22bc0b15d0 2024-09-10T22:04:54.234 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:04:54.234 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"btime":"2024-09-10T22:03:15:051273+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5248,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2731842573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2731842573},{"type":"v1","addr":"172.21.15.94:6837","nonce":2731842573}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1324134239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1324134239},{"type":"v1","addr":"172.21.15.66:6833","nonce":1324134239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1694406832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1694406832},{"type":"v1","addr":"172.21.15.94:6836","nonce":1694406832}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:03:05.416734+0000","modified":"2024-09-10T22:03:08.421387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5412":{"gid":5412,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1215985114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1215985114},{"type":"v1","addr":"172.21.15.66:6835","nonce":1215985114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5412,"qdb_cluster":[5412]},"id":3}]} 2024-09-10T22:04:54.234 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2024-09-10T22:04:54.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.231+0000 7f22bb200640 1 -- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2298072e50 msgr2=0x7f2298075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:54.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.231+0000 7f22bb200640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2298072e50 0x7f2298075310 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f22a40af000 tx=0x7f22a401b020 comp rx=0 tx=0).stop 2024-09-10T22:04:54.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.231+0000 7f22bb200640 1 -- 172.21.15.66:0/2523711627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22bc0b15d0 msgr2=0x7f22bc15e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.231+0000 7f22bb200640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22bc0b15d0 0x7f22bc15e140 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c490 tx=0x7f22ac00c950 comp rx=0 tx=0).stop 2024-09-10T22:04:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.231+0000 7f22bb200640 1 -- 172.21.15.66:0/2523711627 shutdown_connections 2024-09-10T22:04:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.231+0000 7f22bb200640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2298072e50 0x7f2298075310 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.231+0000 7f22bb200640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22bc15f300 0x7f22bc0cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.231+0000 7f22bb200640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22bc15ef20 0x7f22bc0cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.231+0000 7f22bb200640 1 --2- 172.21.15.66:0/2523711627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22bc0b15d0 0x7f22bc15e140 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.231+0000 7f22bb200640 1 -- 172.21.15.66:0/2523711627 >> 172.21.15.66:0/2523711627 conn(0x7f22bc0d1b10 msgr2=0x7f22bc070620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:54.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.231+0000 7f22bb200640 1 -- 172.21.15.66:0/2523711627 shutdown_connections 2024-09-10T22:04:54.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.231+0000 7f22bb200640 1 -- 172.21.15.66:0/2523711627 wait complete. 2024-09-10T22:04:54.272 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:04:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.543+0000 7f0c39e00640 1 -- 172.21.15.66:0/197464169 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c3406ce80 msgr2=0x7f0c340b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:54.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.543+0000 7f0c39e00640 1 --2- 172.21.15.66:0/197464169 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c3406ce80 0x7f0c340b3ac0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009900 tx=0x7f0c2401c8b0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.543+0000 7f0c39e00640 1 -- 172.21.15.66:0/197464169 shutdown_connections 2024-09-10T22:04:54.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.543+0000 7f0c39e00640 1 --2- 172.21.15.66:0/197464169 >> v2:172.21.15.94:3300/0 conn(0x7f0c34071100 0x7f0c34146d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.543+0000 7f0c39e00640 1 --2- 172.21.15.66:0/197464169 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c3406ce80 0x7f0c340b3ac0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.543+0000 7f0c39e00640 1 -- 172.21.15.66:0/197464169 >> 172.21.15.66:0/197464169 conn(0x7f0c3406dd30 msgr2=0x7f0c3406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:54.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.543+0000 7f0c39e00640 1 -- 172.21.15.66:0/197464169 shutdown_connections 2024-09-10T22:04:54.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.543+0000 7f0c39e00640 1 -- 172.21.15.66:0/197464169 wait complete. 2024-09-10T22:04:54.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.547+0000 7f0c39e00640 1 Processor -- start 2024-09-10T22:04:54.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.547+0000 7f0c39e00640 1 -- start start 2024-09-10T22:04:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.547+0000 7f0c39e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c3406ce80 0x7f0c34143350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.547+0000 7f0c39e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c34071100 0x7f0c341fb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.547+0000 7f0c39e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c341fbae0 0x7f0c341413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.547+0000 7f0c39e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c340702e0 con 0x7f0c341fbae0 2024-09-10T22:04:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.547+0000 7f0c39e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34070160 con 0x7f0c34071100 2024-09-10T22:04:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.547+0000 7f0c39e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0c34070460 con 0x7f0c3406ce80 2024-09-10T22:04:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.547+0000 7f0c32a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c3406ce80 0x7f0c34143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.547+0000 7f0c32a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c3406ce80 0x7f0c34143350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:39918/0 (socket says 172.21.15.66:39918) 2024-09-10T22:04:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.547+0000 7f0c32a00640 1 -- 172.21.15.66:0/3233224677 learned_addr learned my addr 172.21.15.66:0/3233224677 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.547+0000 7f0c32000640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c34071100 0x7f0c341fb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.551+0000 7f0c32000640 1 -- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c3406ce80 msgr2=0x7f0c34143350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.551+0000 7f0c32000640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c3406ce80 0x7f0c34143350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.551+0000 7f0c32000640 1 -- 172.21.15.66:0/3233224677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c341fbae0 msgr2=0x7f0c341413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.551+0000 7f0c32000640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c341fbae0 0x7f0c341413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.551+0000 7f0c32000640 1 -- 172.21.15.66:0/3233224677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c34143890 con 0x7f0c34071100 2024-09-10T22:04:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.551+0000 7f0c32000640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c34071100 0x7f0c341fb5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000b3e0 tx=0x7f0c2000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.551+0000 7f0c2b400640 1 -- 172.21.15.66:0/3233224677 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20015020 con 0x7f0c34071100 2024-09-10T22:04:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.551+0000 7f0c2b400640 1 -- 172.21.15.66:0/3233224677 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c20011920 con 0x7f0c34071100 2024-09-10T22:04:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.551+0000 7f0c2b400640 1 -- 172.21.15.66:0/3233224677 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20011d70 con 0x7f0c34071100 2024-09-10T22:04:54.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.551+0000 7f0c39e00640 1 -- 172.21.15.66:0/3233224677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c34142110 con 0x7f0c34071100 2024-09-10T22:04:54.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.551+0000 7f0c39e00640 1 -- 172.21.15.66:0/3233224677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c341426d0 con 0x7f0c34071100 2024-09-10T22:04:54.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.555+0000 7f0c39e00640 1 -- 172.21.15.66:0/3233224677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c340b3ac0 con 0x7f0c34071100 2024-09-10T22:04:54.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.555+0000 7f0c2b400640 1 -- 172.21.15.66:0/3233224677 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f0c20022060 con 0x7f0c34071100 2024-09-10T22:04:54.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.559+0000 7f0c2b400640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0c14072f20 0x7f0c140753e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:54.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.559+0000 7f0c2b400640 1 -- 172.21.15.66:0/3233224677 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7f0c20094170 con 0x7f0c34071100 2024-09-10T22:04:54.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.559+0000 7f0c32a00640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0c14072f20 0x7f0c140753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:54.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.559+0000 7f0c32a00640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0c14072f20 0x7f0c140753e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0c2401c6e0 tx=0x7f0c240a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:54.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.563+0000 7f0c2b400640 1 -- 172.21.15.66:0/3233224677 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0c2005e830 con 0x7f0c34071100 2024-09-10T22:04:54.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.779+0000 7f0c39e00640 1 -- 172.21.15.66:0/3233224677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c340aecd0 con 0x7f0c34071100 2024-09-10T22:04:54.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.779+0000 7f0c2b400640 1 -- 172.21.15.66:0/3233224677 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16861 (secure 0 0 0) 0x7f0c20062c10 con 0x7f0c34071100 2024-09-10T22:04:54.786 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:04:54.786 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":47,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:04:46.895695+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:03:03.140557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T22:03:04.388164+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.37","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:04:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.783+0000 7f0c39e00640 1 -- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0c14072f20 msgr2=0x7f0c140753e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.783+0000 7f0c39e00640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0c14072f20 0x7f0c140753e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0c2401c6e0 tx=0x7f0c240a7040 comp rx=0 tx=0).stop 2024-09-10T22:04:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.783+0000 7f0c39e00640 1 -- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c34071100 msgr2=0x7f0c341fb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.783+0000 7f0c39e00640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c34071100 0x7f0c341fb5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000b3e0 tx=0x7f0c2000b8a0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.783+0000 7f0c39e00640 1 -- 172.21.15.66:0/3233224677 shutdown_connections 2024-09-10T22:04:54.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.783+0000 7f0c39e00640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0c14072f20 0x7f0c140753e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.783+0000 7f0c39e00640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c341fbae0 0x7f0c341413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.783+0000 7f0c39e00640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c34071100 0x7f0c341fb5a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.783+0000 7f0c39e00640 1 --2- 172.21.15.66:0/3233224677 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c3406ce80 0x7f0c34143350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.783+0000 7f0c39e00640 1 -- 172.21.15.66:0/3233224677 >> 172.21.15.66:0/3233224677 conn(0x7f0c3406dd30 msgr2=0x7f0c34160cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:54.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.783+0000 7f0c39e00640 1 -- 172.21.15.66:0/3233224677 shutdown_connections 2024-09-10T22:04:54.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.783+0000 7f0c39e00640 1 -- 172.21.15.66:0/3233224677 wait complete. 2024-09-10T22:04:54.808 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:04:54.808 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:04:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 -- 172.21.15.66:0/1817712599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff31c160280 msgr2=0x7ff31c0b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 --2- 172.21.15.66:0/1817712599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff31c160280 0x7ff31c0b3170 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff304009900 tx=0x7ff30401d280 comp rx=0 tx=0).stop 2024-09-10T22:04:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 -- 172.21.15.66:0/1817712599 shutdown_connections 2024-09-10T22:04:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 --2- 172.21.15.66:0/1817712599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff31c160280 0x7ff31c0b3170 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 -- 172.21.15.66:0/1817712599 >> 172.21.15.66:0/1817712599 conn(0x7ff31c0d1c60 msgr2=0x7ff31c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:54.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 -- 172.21.15.66:0/1817712599 shutdown_connections 2024-09-10T22:04:54.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 -- 172.21.15.66:0/1817712599 wait complete. 2024-09-10T22:04:54.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 Processor -- start 2024-09-10T22:04:54.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 -- start start 2024-09-10T22:04:54.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31c0b1e10 0x7ff31c14aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:54.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff31c15f450 0x7ff31c1656d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:54.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff31c160280 0x7ff31c06f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:54.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c16a040 con 0x7ff31c15f450 2024-09-10T22:04:54.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c169ec0 con 0x7ff31c0b1e10 2024-09-10T22:04:54.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.951+0000 7ff322200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff31c16a1c0 con 0x7ff31c160280 2024-09-10T22:04:54.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff31c15f450 0x7ff31c1656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:54.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff31c15f450 0x7ff31c1656d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39366/0 (socket says 172.21.15.66:39366) 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313e00640 1 -- 172.21.15.66:0/3632009889 learned_addr learned my addr 172.21.15.66:0/3632009889 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff31b400640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31c0b1e10 0x7ff31c14aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff31be00640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff31c160280 0x7ff31c06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313e00640 1 -- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff31c160280 msgr2=0x7ff31c06f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313e00640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff31c160280 0x7ff31c06f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313e00640 1 -- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31c0b1e10 msgr2=0x7ff31c14aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313e00640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31c0b1e10 0x7ff31c14aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313e00640 1 -- 172.21.15.66:0/3632009889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c06df80 con 0x7ff31c15f450 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff31be00640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff31c160280 0x7ff31c06f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313e00640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff31c15f450 0x7ff31c1656d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff304009850 tx=0x7ff304004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313400640 1 -- 172.21.15.66:0/3632009889 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3040b2040 con 0x7ff31c15f450 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313400640 1 -- 172.21.15.66:0/3632009889 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff304003600 con 0x7ff31c15f450 2024-09-10T22:04:54.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313400640 1 -- 172.21.15.66:0/3632009889 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304003a10 con 0x7ff31c15f450 2024-09-10T22:04:54.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff322200640 1 -- 172.21.15.66:0/3632009889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31c06e260 con 0x7ff31c15f450 2024-09-10T22:04:54.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff322200640 1 -- 172.21.15.66:0/3632009889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31c06e820 con 0x7ff31c15f450 2024-09-10T22:04:54.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff322200640 1 -- 172.21.15.66:0/3632009889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31c0b3170 con 0x7ff31c15f450 2024-09-10T22:04:54.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.955+0000 7ff313400640 1 -- 172.21.15.66:0/3632009889 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7ff304003bb0 con 0x7ff31c15f450 2024-09-10T22:04:54.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.959+0000 7ff313400640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff2f8072e50 0x7ff2f8075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:54.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.959+0000 7ff313400640 1 -- 172.21.15.66:0/3632009889 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7ff30412e470 con 0x7ff31c15f450 2024-09-10T22:04:54.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.959+0000 7ff31b400640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff2f8072e50 0x7ff2f8075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:54.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.959+0000 7ff31b400640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff2f8072e50 0x7ff2f8075310 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff31c165660 tx=0x7ff30c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:54.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:54.963+0000 7ff313400640 1 -- 172.21.15.66:0/3632009889 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff3040164d0 con 0x7ff31c15f450 2024-09-10T22:04:55.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.171+0000 7ff322200640 1 -- 172.21.15.66:0/3632009889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff31c0d2a40 con 0x7ff31c15f450 2024-09-10T22:04:55.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.171+0000 7ff313400640 1 -- 172.21.15.66:0/3632009889 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5223 (secure 0 0 0) 0x7ff3040f8ec0 con 0x7ff31c15f450 2024-09-10T22:04:55.176 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:04:55.176 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"btime":"2024-09-10T22:03:15:051273+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5248,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2731842573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2731842573},{"type":"v1","addr":"172.21.15.94:6837","nonce":2731842573}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1324134239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1324134239},{"type":"v1","addr":"172.21.15.66:6833","nonce":1324134239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1694406832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1694406832},{"type":"v1","addr":"172.21.15.94:6836","nonce":1694406832}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:03:05.416734+0000","modified":"2024-09-10T22:03:08.421387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5412":{"gid":5412,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1215985114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1215985114},{"type":"v1","addr":"172.21.15.66:6835","nonce":1215985114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5412,"qdb_cluster":[5412]},"id":3}]} 2024-09-10T22:04:55.176 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2024-09-10T22:04:55.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.175+0000 7ff322200640 1 -- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff2f8072e50 msgr2=0x7ff2f8075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:55.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.175+0000 7ff322200640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff2f8072e50 0x7ff2f8075310 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff31c165660 tx=0x7ff30c009040 comp rx=0 tx=0).stop 2024-09-10T22:04:55.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.175+0000 7ff322200640 1 -- 172.21.15.66:0/3632009889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff31c15f450 msgr2=0x7ff31c1656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:55.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.175+0000 7ff322200640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff31c15f450 0x7ff31c1656d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff304009850 tx=0x7ff304004150 comp rx=0 tx=0).stop 2024-09-10T22:04:55.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.175+0000 7ff322200640 1 -- 172.21.15.66:0/3632009889 shutdown_connections 2024-09-10T22:04:55.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.175+0000 7ff322200640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff2f8072e50 0x7ff2f8075310 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.175+0000 7ff322200640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff31c160280 0x7ff31c06f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.175+0000 7ff322200640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff31c15f450 0x7ff31c1656d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.175+0000 7ff322200640 1 --2- 172.21.15.66:0/3632009889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31c0b1e10 0x7ff31c14aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.175+0000 7ff322200640 1 -- 172.21.15.66:0/3632009889 >> 172.21.15.66:0/3632009889 conn(0x7ff31c0d1c60 msgr2=0x7ff31c071bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:55.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.175+0000 7ff322200640 1 -- 172.21.15.66:0/3632009889 shutdown_connections 2024-09-10T22:04:55.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.175+0000 7ff322200640 1 -- 172.21.15.66:0/3632009889 wait complete. 2024-09-10T22:04:55.198 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:04:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.339+0000 7fbb52200640 1 -- 172.21.15.66:0/4178221146 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb4c15bd40 msgr2=0x7fbb4c0cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.339+0000 7fbb52200640 1 --2- 172.21.15.66:0/4178221146 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb4c15bd40 0x7fbb4c0cffa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009900 tx=0x7fbb3c01c850 comp rx=0 tx=0).stop 2024-09-10T22:04:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.339+0000 7fbb52200640 1 -- 172.21.15.66:0/4178221146 shutdown_connections 2024-09-10T22:04:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.339+0000 7fbb52200640 1 --2- 172.21.15.66:0/4178221146 >> v2:172.21.15.94:3300/0 conn(0x7fbb4c0cf830 0x7fbb4c0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.339+0000 7fbb52200640 1 --2- 172.21.15.66:0/4178221146 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb4c15bd40 0x7fbb4c0cffa0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.339+0000 7fbb52200640 1 -- 172.21.15.66:0/4178221146 >> 172.21.15.66:0/4178221146 conn(0x7fbb4c0d1c60 msgr2=0x7fbb4c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.339+0000 7fbb52200640 1 -- 172.21.15.66:0/4178221146 shutdown_connections 2024-09-10T22:04:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.339+0000 7fbb52200640 1 -- 172.21.15.66:0/4178221146 wait complete. 2024-09-10T22:04:55.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb52200640 1 Processor -- start 2024-09-10T22:04:55.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb52200640 1 -- start start 2024-09-10T22:04:55.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb52200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb4c0aec10 0x7fbb4c151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:55.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb52200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb4c0cf830 0x7fbb4c0b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:55.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb52200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb4c15bd40 0x7fbb4c0cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:55.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb52200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c169070 con 0x7fbb4c15bd40 2024-09-10T22:04:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb52200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c168ef0 con 0x7fbb4c0cf830 2024-09-10T22:04:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb52200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbb4c1691f0 con 0x7fbb4c0aec10 2024-09-10T22:04:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb43e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb4c0cf830 0x7fbb4c0b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb4c15bd40 0x7fbb4c0cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb43e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb4c0cf830 0x7fbb4c0b5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:45616/0 (socket says 172.21.15.66:45616) 2024-09-10T22:04:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb43e00640 1 -- 172.21.15.66:0/2438995558 learned_addr learned my addr 172.21.15.66:0/2438995558 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb4b400640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb4c0aec10 0x7fbb4c151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb43e00640 1 -- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb4c0aec10 msgr2=0x7fbb4c151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb43e00640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb4c0aec10 0x7fbb4c151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb43e00640 1 -- 172.21.15.66:0/2438995558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb4c15bd40 msgr2=0x7fbb4c0cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb43e00640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb4c15bd40 0x7fbb4c0cb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb43e00640 1 -- 172.21.15.66:0/2438995558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4c0b6a20 con 0x7fbb4c0cf830 2024-09-10T22:04:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb43e00640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb4c0cf830 0x7fbb4c0b5fd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400ea20 tx=0x7fbb3400eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb43400640 1 -- 172.21.15.66:0/2438995558 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400cd80 con 0x7fbb4c0cf830 2024-09-10T22:04:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb43400640 1 -- 172.21.15.66:0/2438995558 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb34010430 con 0x7fbb4c0cf830 2024-09-10T22:04:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb43400640 1 -- 172.21.15.66:0/2438995558 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34010860 con 0x7fbb4c0cf830 2024-09-10T22:04:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb52200640 1 -- 172.21.15.66:0/2438995558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4c0b6d00 con 0x7fbb4c0cf830 2024-09-10T22:04:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.343+0000 7fbb52200640 1 -- 172.21.15.66:0/2438995558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb4c0b72c0 con 0x7fbb4c0cf830 2024-09-10T22:04:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.347+0000 7fbb52200640 1 -- 172.21.15.66:0/2438995558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb4c0af040 con 0x7fbb4c0cf830 2024-09-10T22:04:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.347+0000 7fbb43400640 1 -- 172.21.15.66:0/2438995558 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fbb34010a00 con 0x7fbb4c0cf830 2024-09-10T22:04:55.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.347+0000 7fbb43400640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbb28072e50 0x7fbb28075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:55.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.347+0000 7fbb43400640 1 -- 172.21.15.66:0/2438995558 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7fbb340925a0 con 0x7fbb4c0cf830 2024-09-10T22:04:55.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.347+0000 7fbb4b400640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbb28072e50 0x7fbb28075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:55.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.347+0000 7fbb4b400640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbb28072e50 0x7fbb28075310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0af000 tx=0x7fbb3c01b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.351+0000 7fbb43400640 1 -- 172.21.15.66:0/2438995558 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbb3405f0e0 con 0x7fbb4c0cf830 2024-09-10T22:04:55.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.559+0000 7fbb52200640 1 -- 172.21.15.66:0/2438995558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb4c0b0df0 con 0x7fbb4c0cf830 2024-09-10T22:04:55.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.559+0000 7fbb43400640 1 -- 172.21.15.66:0/2438995558 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5223 (secure 0 0 0) 0x7fbb34063b10 con 0x7fbb4c0cf830 2024-09-10T22:04:55.565 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:04:55.565 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"btime":"2024-09-10T22:03:15:051273+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5248,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2731842573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2731842573},{"type":"v1","addr":"172.21.15.94:6837","nonce":2731842573}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1324134239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1324134239},{"type":"v1","addr":"172.21.15.66:6833","nonce":1324134239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1694406832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1694406832},{"type":"v1","addr":"172.21.15.94:6836","nonce":1694406832}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:03:05.416734+0000","modified":"2024-09-10T22:03:08.421387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5412":{"gid":5412,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1215985114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1215985114},{"type":"v1","addr":"172.21.15.66:6835","nonce":1215985114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5412,"qdb_cluster":[5412]},"id":3}]} 2024-09-10T22:04:55.565 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2024-09-10T22:04:55.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.563+0000 7fbb52200640 1 -- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbb28072e50 msgr2=0x7fbb28075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:55.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.563+0000 7fbb52200640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbb28072e50 0x7fbb28075310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0af000 tx=0x7fbb3c01b090 comp rx=0 tx=0).stop 2024-09-10T22:04:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.563+0000 7fbb52200640 1 -- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb4c0cf830 msgr2=0x7fbb4c0b5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.563+0000 7fbb52200640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb4c0cf830 0x7fbb4c0b5fd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400ea20 tx=0x7fbb3400eee0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.563+0000 7fbb52200640 1 -- 172.21.15.66:0/2438995558 shutdown_connections 2024-09-10T22:04:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.563+0000 7fbb52200640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbb28072e50 0x7fbb28075310 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.563+0000 7fbb52200640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb4c15bd40 0x7fbb4c0cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.563+0000 7fbb52200640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb4c0cf830 0x7fbb4c0b5fd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.563+0000 7fbb52200640 1 --2- 172.21.15.66:0/2438995558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb4c0aec10 0x7fbb4c151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.563+0000 7fbb52200640 1 -- 172.21.15.66:0/2438995558 >> 172.21.15.66:0/2438995558 conn(0x7fbb4c0d1c60 msgr2=0x7fbb4c06e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.563+0000 7fbb52200640 1 -- 172.21.15.66:0/2438995558 shutdown_connections 2024-09-10T22:04:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.563+0000 7fbb52200640 1 -- 172.21.15.66:0/2438995558 wait complete. 2024-09-10T22:04:55.588 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:04:55.628 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:04:55.636 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:04:55.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.731+0000 7f40ea400640 1 -- 172.21.15.66:0/2785151256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e415cc80 msgr2=0x7f40e40cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:55.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.731+0000 7f40ea400640 1 --2- 172.21.15.66:0/2785151256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e415cc80 0x7f40e40cd930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f40e4072560 tx=0x7f40dc01ccf0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.731+0000 7f40ea400640 1 -- 172.21.15.66:0/2785151256 shutdown_connections 2024-09-10T22:04:55.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.731+0000 7f40ea400640 1 --2- 172.21.15.66:0/2785151256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e415cc80 0x7f40e40cd930 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.731+0000 7f40ea400640 1 --2- 172.21.15.66:0/2785151256 >> v2:172.21.15.94:3300/0 conn(0x7f40e40aee20 0x7f40e40ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.731+0000 7f40ea400640 1 -- 172.21.15.66:0/2785151256 >> 172.21.15.66:0/2785151256 conn(0x7f40e40cfa30 msgr2=0x7f40e40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:55.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.731+0000 7f40ea400640 1 -- 172.21.15.66:0/2785151256 shutdown_connections 2024-09-10T22:04:55.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.731+0000 7f40ea400640 1 -- 172.21.15.66:0/2785151256 wait complete. 2024-09-10T22:04:55.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.731+0000 7f40ea400640 1 Processor -- start 2024-09-10T22:04:55.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.731+0000 7f40ea400640 1 -- start start 2024-09-10T22:04:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40ea400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e40aee20 0x7f40e415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40ea400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f40e4159900 0x7f40e414f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40ea400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40e415cc80 0x7f40e414fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40ea400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f40e4168f70 con 0x7f40e40aee20 2024-09-10T22:04:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40ea400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40e4168df0 con 0x7f40e415cc80 2024-09-10T22:04:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40ea400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f40e41690f0 con 0x7f40e4159900 2024-09-10T22:04:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40e3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e40aee20 0x7f40e415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40e3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e40aee20 0x7f40e415ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39386/0 (socket says 172.21.15.66:39386) 2024-09-10T22:04:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40e3400640 1 -- 172.21.15.66:0/133350833 learned_addr learned my addr 172.21.15.66:0/133350833 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40e3e00640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40e415cc80 0x7f40e414fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40e2a00640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f40e4159900 0x7f40e414f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40e3400640 1 -- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f40e4159900 msgr2=0x7f40e414f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40e3400640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f40e4159900 0x7f40e414f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40e3400640 1 -- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40e415cc80 msgr2=0x7f40e414fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40e3400640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40e415cc80 0x7f40e414fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40e3400640 1 -- 172.21.15.66:0/133350833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e415dce0 con 0x7f40e40aee20 2024-09-10T22:04:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40e3400640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e40aee20 0x7f40e415ea60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f40d400c480 tx=0x7f40d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:55.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40db400640 1 -- 172.21.15.66:0/133350833 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4012830 con 0x7f40e40aee20 2024-09-10T22:04:55.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40db400640 1 -- 172.21.15.66:0/133350833 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40d4017070 con 0x7f40e40aee20 2024-09-10T22:04:55.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40db400640 1 -- 172.21.15.66:0/133350833 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4012d80 con 0x7f40e40aee20 2024-09-10T22:04:55.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40ea400640 1 -- 172.21.15.66:0/133350833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e415df80 con 0x7f40e40aee20 2024-09-10T22:04:55.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40ea400640 1 -- 172.21.15.66:0/133350833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e415e540 con 0x7f40e40aee20 2024-09-10T22:04:55.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40ea400640 1 -- 172.21.15.66:0/133350833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e40ca080 con 0x7f40e40aee20 2024-09-10T22:04:55.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.735+0000 7f40db400640 1 -- 172.21.15.66:0/133350833 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f40d401c460 con 0x7f40e40aee20 2024-09-10T22:04:55.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.739+0000 7f40db400640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f40c4072e50 0x7f40c4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:55.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.739+0000 7f40db400640 1 -- 172.21.15.66:0/133350833 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7f40d40943c0 con 0x7f40e40aee20 2024-09-10T22:04:55.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.739+0000 7f40e2a00640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f40c4072e50 0x7f40c4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:55.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.739+0000 7f40e2a00640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f40c4072e50 0x7f40c4075310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f40d0005fd0 tx=0x7f40d0005db0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:55.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.743+0000 7f40db400640 1 -- 172.21.15.66:0/133350833 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f40d405ea90 con 0x7f40e40aee20 2024-09-10T22:04:55.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.927+0000 7f40ea400640 1 -- 172.21.15.66:0/133350833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40e4070470 con 0x7f40e40aee20 2024-09-10T22:04:55.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.927+0000 7f40db400640 1 -- 172.21.15.66:0/133350833 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16861 (secure 0 0 0) 0x7f40d405e390 con 0x7f40e40aee20 2024-09-10T22:04:55.934 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:04:55.934 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":47,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:04:46.895695+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:03:03.140557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T22:03:04.388164+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.37","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:04:55.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.931+0000 7f40ea400640 1 -- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f40c4072e50 msgr2=0x7f40c4075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:55.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.931+0000 7f40ea400640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f40c4072e50 0x7f40c4075310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f40d0005fd0 tx=0x7f40d0005db0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.931+0000 7f40ea400640 1 -- 172.21.15.66:0/133350833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e40aee20 msgr2=0x7f40e415ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:55.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.931+0000 7f40ea400640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e40aee20 0x7f40e415ea60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f40d400c480 tx=0x7f40d400c940 comp rx=0 tx=0).stop 2024-09-10T22:04:55.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.931+0000 7f40ea400640 1 -- 172.21.15.66:0/133350833 shutdown_connections 2024-09-10T22:04:55.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.931+0000 7f40ea400640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f40c4072e50 0x7f40c4075310 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.931+0000 7f40ea400640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40e415cc80 0x7f40e414fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.931+0000 7f40ea400640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f40e4159900 0x7f40e414f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.931+0000 7f40ea400640 1 --2- 172.21.15.66:0/133350833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e40aee20 0x7f40e415ea60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:55.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.931+0000 7f40ea400640 1 -- 172.21.15.66:0/133350833 >> 172.21.15.66:0/133350833 conn(0x7f40e40cfa30 msgr2=0x7f40e415f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:55.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.931+0000 7f40ea400640 1 -- 172.21.15.66:0/133350833 shutdown_connections 2024-09-10T22:04:55.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:55.931+0000 7f40ea400640 1 -- 172.21.15.66:0/133350833 wait complete. 2024-09-10T22:04:55.958 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:04:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.103+0000 7fb1c9c00640 1 -- 172.21.15.66:0/2070220012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c415cc80 msgr2=0x7fb1c40cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.103+0000 7fb1c9c00640 1 --2- 172.21.15.66:0/2070220012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c415cc80 0x7fb1c40cd930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4072560 tx=0x7fb1bc01caf0 comp rx=0 tx=0).stop 2024-09-10T22:04:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.103+0000 7fb1c9c00640 1 -- 172.21.15.66:0/2070220012 shutdown_connections 2024-09-10T22:04:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.103+0000 7fb1c9c00640 1 --2- 172.21.15.66:0/2070220012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c415cc80 0x7fb1c40cd930 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.103+0000 7fb1c9c00640 1 --2- 172.21.15.66:0/2070220012 >> v2:172.21.15.94:3300/0 conn(0x7fb1c40aee20 0x7fb1c40ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.103+0000 7fb1c9c00640 1 -- 172.21.15.66:0/2070220012 >> 172.21.15.66:0/2070220012 conn(0x7fb1c40cfa30 msgr2=0x7fb1c40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.103+0000 7fb1c9c00640 1 -- 172.21.15.66:0/2070220012 shutdown_connections 2024-09-10T22:04:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.103+0000 7fb1c9c00640 1 -- 172.21.15.66:0/2070220012 wait complete. 2024-09-10T22:04:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1c9c00640 1 Processor -- start 2024-09-10T22:04:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1c9c00640 1 -- start start 2024-09-10T22:04:56.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1c9c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c40aee20 0x7fb1c415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:56.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1c9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c4159900 0x7fb1c415f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:56.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1c9c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1c415cc80 0x7fb1c415fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:56.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1c9c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c4168f70 con 0x7fb1c4159900 2024-09-10T22:04:56.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1c9c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c4168df0 con 0x7fb1c40aee20 2024-09-10T22:04:56.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1c9c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c41690f0 con 0x7fb1c415cc80 2024-09-10T22:04:56.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c4159900 0x7fb1c415f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:56.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c4159900 0x7fb1c415f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39410/0 (socket says 172.21.15.66:39410) 2024-09-10T22:04:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1bbe00640 1 -- 172.21.15.66:0/3515686832 learned_addr learned my addr 172.21.15.66:0/3515686832 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1c2a00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c40aee20 0x7fb1c415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1c3400640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1c415cc80 0x7fb1c415fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1bbe00640 1 -- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1c415cc80 msgr2=0x7fb1c415fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1bbe00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1c415cc80 0x7fb1c415fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1bbe00640 1 -- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c40aee20 msgr2=0x7fb1c415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1bbe00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c40aee20 0x7fb1c415af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1bbe00640 1 -- 172.21.15.66:0/3515686832 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c40c5c30 con 0x7fb1c4159900 2024-09-10T22:04:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1c2a00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c40aee20 0x7fb1c415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:04:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1bbe00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c4159900 0x7fb1c415f7d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00c9a0 tx=0x7fb1ac00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1baa00640 1 -- 172.21.15.66:0/3515686832 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac007c70 con 0x7fb1c4159900 2024-09-10T22:04:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1baa00640 1 -- 172.21.15.66:0/3515686832 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1ac00f450 con 0x7fb1c4159900 2024-09-10T22:04:56.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.107+0000 7fb1c9c00640 1 -- 172.21.15.66:0/3515686832 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c40c48b0 con 0x7fb1c4159900 2024-09-10T22:04:56.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.111+0000 7fb1baa00640 1 -- 172.21.15.66:0/3515686832 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00f8a0 con 0x7fb1c4159900 2024-09-10T22:04:56.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.111+0000 7fb1c9c00640 1 -- 172.21.15.66:0/3515686832 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c40c4e70 con 0x7fb1c4159900 2024-09-10T22:04:56.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.111+0000 7fb1c9c00640 1 -- 172.21.15.66:0/3515686832 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c40ca080 con 0x7fb1c4159900 2024-09-10T22:04:56.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.111+0000 7fb1baa00640 1 -- 172.21.15.66:0/3515686832 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fb1ac00fa40 con 0x7fb1c4159900 2024-09-10T22:04:56.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.111+0000 7fb1baa00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1a0072e80 0x7fb1a0075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:56.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.111+0000 7fb1baa00640 1 -- 172.21.15.66:0/3515686832 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5823+0+0 (secure 0 0 0) 0x7fb1ac092360 con 0x7fb1c4159900 2024-09-10T22:04:56.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.111+0000 7fb1c2a00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1a0072e80 0x7fb1a0075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.115+0000 7fb1c2a00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1a0072e80 0x7fb1a0075340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb1c415a530 tx=0x7fb1b4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.115+0000 7fb1baa00640 1 -- 172.21.15.66:0/3515686832 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb1ac014030 con 0x7fb1c4159900 2024-09-10T22:04:56.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.327+0000 7fb1c9c00640 1 -- 172.21.15.66:0/3515686832 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb1c4156ad0 con 0x7fb1c4159900 2024-09-10T22:04:56.331 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:04:56.327+0000 7fcc93a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:04:56.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.327+0000 7fb1baa00640 1 -- 172.21.15.66:0/3515686832 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7fb1ac05f470 con 0x7fb1c4159900 2024-09-10T22:04:56.333 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:04:56.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.331+0000 7fb1c9c00640 1 -- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1a0072e80 msgr2=0x7fb1a0075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:56.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.331+0000 7fb1c9c00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1a0072e80 0x7fb1a0075340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb1c415a530 tx=0x7fb1b4009040 comp rx=0 tx=0).stop 2024-09-10T22:04:56.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.331+0000 7fb1c9c00640 1 -- 172.21.15.66:0/3515686832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c4159900 msgr2=0x7fb1c415f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:56.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.331+0000 7fb1c9c00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c4159900 0x7fb1c415f7d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00c9a0 tx=0x7fb1ac00ce60 comp rx=0 tx=0).stop 2024-09-10T22:04:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.331+0000 7fb1c9c00640 1 -- 172.21.15.66:0/3515686832 shutdown_connections 2024-09-10T22:04:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.331+0000 7fb1c9c00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1a0072e80 0x7fb1a0075340 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.331+0000 7fb1c9c00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1c415cc80 0x7fb1c415fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.331+0000 7fb1c9c00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1c4159900 0x7fb1c415f7d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.331+0000 7fb1c9c00640 1 --2- 172.21.15.66:0/3515686832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c40aee20 0x7fb1c415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.331+0000 7fb1c9c00640 1 -- 172.21.15.66:0/3515686832 >> 172.21.15.66:0/3515686832 conn(0x7fb1c40cfa30 msgr2=0x7fb1c406ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.331+0000 7fb1c9c00640 1 -- 172.21.15.66:0/3515686832 shutdown_connections 2024-09-10T22:04:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.331+0000 7fb1c9c00640 1 -- 172.21.15.66:0/3515686832 wait complete. 2024-09-10T22:04:56.360 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:04:56.408 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:04:56.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.511+0000 7f2240800640 1 -- 172.21.15.66:0/3618899084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2238164a70 msgr2=0x7f2238166e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:56.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.511+0000 7f2240800640 1 --2- 172.21.15.66:0/3618899084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2238164a70 0x7f2238166e60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2234009bc0 tx=0x7f223401cb10 comp rx=0 tx=0).stop 2024-09-10T22:04:56.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.511+0000 7f2240800640 1 -- 172.21.15.66:0/3618899084 shutdown_connections 2024-09-10T22:04:56.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.511+0000 7f2240800640 1 --2- 172.21.15.66:0/3618899084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2238164a70 0x7f2238166e60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:56.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.511+0000 7f2240800640 1 -- 172.21.15.66:0/3618899084 >> 172.21.15.66:0/3618899084 conn(0x7f2238074240 msgr2=0x7f2238074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:56.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.511+0000 7f2240800640 1 -- 172.21.15.66:0/3618899084 shutdown_connections 2024-09-10T22:04:56.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.511+0000 7f2240800640 1 -- 172.21.15.66:0/3618899084 wait complete. 2024-09-10T22:04:56.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.511+0000 7f2240800640 1 Processor -- start 2024-09-10T22:04:56.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.511+0000 7f2240800640 1 -- start start 2024-09-10T22:04:56.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2240800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22380b0590 0x7f2238092020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:56.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2240800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f223815bfb0 0x7f22381f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:56.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2240800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2238164a70 0x7f22381f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:56.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2240800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2238169d40 con 0x7f2238164a70 2024-09-10T22:04:56.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2240800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2238169bc0 con 0x7f223815bfb0 2024-09-10T22:04:56.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2240800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2238169ec0 con 0x7f22380b0590 2024-09-10T22:04:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f223e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2238164a70 0x7f22381f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f223e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2238164a70 0x7f22381f95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39448/0 (socket says 172.21.15.66:39448) 2024-09-10T22:04:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f223e800640 1 -- 172.21.15.66:0/2769967753 learned_addr learned my addr 172.21.15.66:0/2769967753 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f223d400640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f223815bfb0 0x7f22381f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f223de00640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22380b0590 0x7f2238092020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f223e800640 1 -- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22380b0590 msgr2=0x7f2238092020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f223e800640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22380b0590 0x7f2238092020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f223e800640 1 -- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f223815bfb0 msgr2=0x7f22381f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f223e800640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f223815bfb0 0x7f22381f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f223e800640 1 -- 172.21.15.66:0/2769967753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22380707f0 con 0x7f2238164a70 2024-09-10T22:04:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f223de00640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22380b0590 0x7f2238092020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:04:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f223e800640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2238164a70 0x7f22381f95a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2234019720 tx=0x7f2234003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2232000640 1 -- 172.21.15.66:0/2769967753 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22340bb070 con 0x7f2238164a70 2024-09-10T22:04:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2232000640 1 -- 172.21.15.66:0/2769967753 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2234004b40 con 0x7f2238164a70 2024-09-10T22:04:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2232000640 1 -- 172.21.15.66:0/2769967753 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22340b2040 con 0x7f2238164a70 2024-09-10T22:04:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2240800640 1 -- 172.21.15.66:0/2769967753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2238070ab0 con 0x7f2238164a70 2024-09-10T22:04:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2240800640 1 -- 172.21.15.66:0/2769967753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223806f5d0 con 0x7f2238164a70 2024-09-10T22:04:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2240800640 1 -- 172.21.15.66:0/2769967753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22380ccd40 con 0x7f2238164a70 2024-09-10T22:04:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.515+0000 7f2232000640 1 -- 172.21.15.66:0/2769967753 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f22340b5030 con 0x7f2238164a70 2024-09-10T22:04:56.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.519+0000 7f2232000640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f221c072e50 0x7f221c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:56.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.519+0000 7f2232000640 1 -- 172.21.15.66:0/2769967753 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5881+0+0 (secure 0 0 0) 0x7f223412da30 con 0x7f2238164a70 2024-09-10T22:04:56.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.519+0000 7f223de00640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f221c072e50 0x7f221c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:56.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.519+0000 7f223de00640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f221c072e50 0x7f221c075310 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f222c0017a0 tx=0x7f222c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:56.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.523+0000 7f2232000640 1 -- 172.21.15.66:0/2769967753 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f22340f1de0 con 0x7f2238164a70 2024-09-10T22:04:56.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:56.731+0000 7f2240800640 1 -- 172.21.15.66:0/2769967753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f22380938f0 con 0x7f2238164a70 2024-09-10T22:04:57.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.331+0000 7f2232000640 1 -- 172.21.15.66:0/2769967753 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f2234016790 con 0x7f2238164a70 2024-09-10T22:04:57.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.331+0000 7f2240800640 1 -- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f221c072e50 msgr2=0x7f221c075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:57.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.331+0000 7f2240800640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f221c072e50 0x7f221c075310 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f222c0017a0 tx=0x7f222c009040 comp rx=0 tx=0).stop 2024-09-10T22:04:57.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.331+0000 7f2240800640 1 -- 172.21.15.66:0/2769967753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2238164a70 msgr2=0x7f22381f95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:57.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.331+0000 7f2240800640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2238164a70 0x7f22381f95a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2234019720 tx=0x7f2234003f60 comp rx=0 tx=0).stop 2024-09-10T22:04:57.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.335+0000 7f2240800640 1 -- 172.21.15.66:0/2769967753 shutdown_connections 2024-09-10T22:04:57.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.335+0000 7f2240800640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f221c072e50 0x7f221c075310 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.335+0000 7f2240800640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2238164a70 0x7f22381f95a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.335+0000 7f2240800640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f223815bfb0 0x7f22381f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.335+0000 7f2240800640 1 --2- 172.21.15.66:0/2769967753 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22380b0590 0x7f2238092020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.335+0000 7f2240800640 1 -- 172.21.15.66:0/2769967753 >> 172.21.15.66:0/2769967753 conn(0x7f2238074240 msgr2=0x7f223815a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.335+0000 7f2240800640 1 -- 172.21.15.66:0/2769967753 shutdown_connections 2024-09-10T22:04:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.335+0000 7f2240800640 1 -- 172.21.15.66:0/2769967753 wait complete. 2024-09-10T22:04:57.361 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:04:57.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.547+0000 7f418be00640 1 -- 172.21.15.66:0/4150100509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c0b1170 msgr2=0x7f418c0ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:57.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.547+0000 7f418be00640 1 --2- 172.21.15.66:0/4150100509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c0b1170 0x7f418c0ce540 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f418c09b620 tx=0x7f418401cac0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.547+0000 7f418be00640 1 -- 172.21.15.66:0/4150100509 shutdown_connections 2024-09-10T22:04:57.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.547+0000 7f418be00640 1 --2- 172.21.15.66:0/4150100509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c0b1170 0x7f418c0ce540 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.547+0000 7f418be00640 1 --2- 172.21.15.66:0/4150100509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f418c15bdc0 0x7f418c0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.547+0000 7f418be00640 1 --2- 172.21.15.66:0/4150100509 >> v2:172.21.15.94:3300/0 conn(0x7f418c15b9e0 0x7f418c0d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.547+0000 7f418be00640 1 -- 172.21.15.66:0/4150100509 >> 172.21.15.66:0/4150100509 conn(0x7f418c09aa50 msgr2=0x7f418c09ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:57.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.547+0000 7f418be00640 1 -- 172.21.15.66:0/4150100509 shutdown_connections 2024-09-10T22:04:57.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.547+0000 7f418be00640 1 -- 172.21.15.66:0/4150100509 wait complete. 2024-09-10T22:04:57.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418be00640 1 Processor -- start 2024-09-10T22:04:57.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418be00640 1 -- start start 2024-09-10T22:04:57.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f418c0b1170 0x7f418c158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:57.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c15b9e0 0x7f418c165ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:57.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418be00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f418c15bdc0 0x7f418c15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:57.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f418c0b2090 con 0x7f418c15b9e0 2024-09-10T22:04:57.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f418c0b1f10 con 0x7f418c0b1170 2024-09-10T22:04:57.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418be00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f418c0b2210 con 0x7f418c15bdc0 2024-09-10T22:04:57.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f4183e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c15b9e0 0x7f418c165ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:57.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f4183e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c15b9e0 0x7f418c165ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39468/0 (socket says 172.21.15.66:39468) 2024-09-10T22:04:57.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f4183e00640 1 -- 172.21.15.66:0/2624858751 learned_addr learned my addr 172.21.15.66:0/2624858751 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418aa00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f418c0b1170 0x7f418c158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418b400640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f418c15bdc0 0x7f418c15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f4183e00640 1 -- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f418c15bdc0 msgr2=0x7f418c15a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f4183e00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f418c15bdc0 0x7f418c15a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f4183e00640 1 -- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f418c0b1170 msgr2=0x7f418c158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f4183e00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f418c0b1170 0x7f418c158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f4183e00640 1 -- 172.21.15.66:0/2624858751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418c14ae10 con 0x7f418c15b9e0 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418b400640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f418c15bdc0 0x7f418c15a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f4183e00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c15b9e0 0x7f418c165ee0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f417400e790 tx=0x7f417400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f4182a00640 1 -- 172.21.15.66:0/2624858751 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417400eea0 con 0x7f418c15b9e0 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f4182a00640 1 -- 172.21.15.66:0/2624858751 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f417400cce0 con 0x7f418c15b9e0 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418be00640 1 -- 172.21.15.66:0/2624858751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418c14b0f0 con 0x7f418c15b9e0 2024-09-10T22:04:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f4182a00640 1 -- 172.21.15.66:0/2624858751 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174004290 con 0x7f418c15b9e0 2024-09-10T22:04:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418be00640 1 -- 172.21.15.66:0/2624858751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f418c166420 con 0x7f418c15b9e0 2024-09-10T22:04:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.551+0000 7f418be00640 1 -- 172.21.15.66:0/2624858751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f418c0ce540 con 0x7f418c15b9e0 2024-09-10T22:04:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.555+0000 7f4182a00640 1 -- 172.21.15.66:0/2624858751 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f4174003600 con 0x7f418c15b9e0 2024-09-10T22:04:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.555+0000 7f4182a00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4168072e50 0x7f4168075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.555+0000 7f4182a00640 1 -- 172.21.15.66:0/2624858751 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5881+0+0 (secure 0 0 0) 0x7f4174094ed0 con 0x7f418c15b9e0 2024-09-10T22:04:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.555+0000 7f418aa00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4168072e50 0x7f4168075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:57.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.555+0000 7f418aa00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4168072e50 0x7f4168075310 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f417c009be0 tx=0x7f417c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.559+0000 7f4182a00640 1 -- 172.21.15.66:0/2624858751 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f417405f4d0 con 0x7f418c15b9e0 2024-09-10T22:04:57.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:57.735+0000 7f418be00640 1 -- 172.21.15.66:0/2624858751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f418c155bd0 con 0x7f418c15b9e0 2024-09-10T22:04:58.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.335+0000 7f4182a00640 1 -- 172.21.15.66:0/2624858751 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) ==== 181+0+0 (secure 0 0 0) 0x7f417405edd0 con 0x7f418c15b9e0 2024-09-10T22:04:58.342 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:04:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.343+0000 7f418be00640 1 -- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4168072e50 msgr2=0x7f4168075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.343+0000 7f418be00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4168072e50 0x7f4168075310 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f417c009be0 tx=0x7f417c0091e0 comp rx=0 tx=0).stop 2024-09-10T22:04:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.343+0000 7f418be00640 1 -- 172.21.15.66:0/2624858751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c15b9e0 msgr2=0x7f418c165ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.343+0000 7f418be00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c15b9e0 0x7f418c165ee0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f417400e790 tx=0x7f417400ec50 comp rx=0 tx=0).stop 2024-09-10T22:04:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.343+0000 7f418be00640 1 -- 172.21.15.66:0/2624858751 shutdown_connections 2024-09-10T22:04:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.343+0000 7f418be00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4168072e50 0x7f4168075310 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.343+0000 7f418be00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f418c15bdc0 0x7f418c15a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.343+0000 7f418be00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c15b9e0 0x7f418c165ee0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.343+0000 7f418be00640 1 --2- 172.21.15.66:0/2624858751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f418c0b1170 0x7f418c158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.343+0000 7f418be00640 1 -- 172.21.15.66:0/2624858751 >> 172.21.15.66:0/2624858751 conn(0x7f418c09aa50 msgr2=0x7f418c1563d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.343+0000 7f418be00640 1 -- 172.21.15.66:0/2624858751 shutdown_connections 2024-09-10T22:04:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.343+0000 7f418be00640 1 -- 172.21.15.66:0/2624858751 wait complete. 2024-09-10T22:04:58.370 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:04:58.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.519+0000 7fc9d7400640 1 -- 172.21.15.66:0/2594130872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9d01605c0 msgr2=0x7fc9d00b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:58.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.519+0000 7fc9d7400640 1 --2- 172.21.15.66:0/2594130872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9d01605c0 0x7fc9d00b1620 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0070630 tx=0x7fc9c401c8c0 comp rx=0 tx=0).stop 2024-09-10T22:04:58.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.519+0000 7fc9d7400640 1 -- 172.21.15.66:0/2594130872 shutdown_connections 2024-09-10T22:04:58.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.519+0000 7fc9d7400640 1 --2- 172.21.15.66:0/2594130872 >> v2:172.21.15.94:3300/0 conn(0x7fc9d0162c80 0x7fc9d0165070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:58.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.519+0000 7fc9d7400640 1 --2- 172.21.15.66:0/2594130872 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9d00b2b10 0x7fc9d00b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:58.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.519+0000 7fc9d7400640 1 --2- 172.21.15.66:0/2594130872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9d01605c0 0x7fc9d00b1620 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:58.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.519+0000 7fc9d7400640 1 -- 172.21.15.66:0/2594130872 >> 172.21.15.66:0/2594130872 conn(0x7fc9d00d1ce0 msgr2=0x7fc9d00d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:58.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.519+0000 7fc9d7400640 1 -- 172.21.15.66:0/2594130872 shutdown_connections 2024-09-10T22:04:58.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.519+0000 7fc9d7400640 1 -- 172.21.15.66:0/2594130872 wait complete. 2024-09-10T22:04:58.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.519+0000 7fc9d7400640 1 Processor -- start 2024-09-10T22:04:58.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.519+0000 7fc9d7400640 1 -- start start 2024-09-10T22:04:58.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00b2b10 0x7fc9d0151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:58.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d7400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9d01605c0 0x7fc9d006f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9d0162c80 0x7fc9d006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d7400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0167dd0 con 0x7fc9d0162c80 2024-09-10T22:04:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d7400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d00b1400 con 0x7fc9d00b2b10 2024-09-10T22:04:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d7400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d0167f50 con 0x7fc9d01605c0 2024-09-10T22:04:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9d0162c80 0x7fc9d006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9d0162c80 0x7fc9d006f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39488/0 (socket says 172.21.15.66:39488) 2024-09-10T22:04:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d5400640 1 -- 172.21.15.66:0/663142796 learned_addr learned my addr 172.21.15.66:0/663142796 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d4a00640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00b2b10 0x7fc9d0151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9cfe00640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9d01605c0 0x7fc9d006f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:58.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d5400640 1 -- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9d01605c0 msgr2=0x7fc9d006f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:58.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d5400640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9d01605c0 0x7fc9d006f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:58.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d5400640 1 -- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00b2b10 msgr2=0x7fc9d0151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:58.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d5400640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00b2b10 0x7fc9d0151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:58.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d5400640 1 -- 172.21.15.66:0/663142796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d0152f00 con 0x7fc9d0162c80 2024-09-10T22:04:58.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d5400640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9d0162c80 0x7fc9d006f740 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800e970 tx=0x7fc9c800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:58.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9ccc00640 1 -- 172.21.15.66:0/663142796 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8004d30 con 0x7fc9d0162c80 2024-09-10T22:04:58.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9ccc00640 1 -- 172.21.15.66:0/663142796 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9c800cb20 con 0x7fc9d0162c80 2024-09-10T22:04:58.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d7400640 1 -- 172.21.15.66:0/663142796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d01531a0 con 0x7fc9d0162c80 2024-09-10T22:04:58.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9ccc00640 1 -- 172.21.15.66:0/663142796 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8005020 con 0x7fc9d0162c80 2024-09-10T22:04:58.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d7400640 1 -- 172.21.15.66:0/663142796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d0153760 con 0x7fc9d0162c80 2024-09-10T22:04:58.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.523+0000 7fc9d7400640 1 -- 172.21.15.66:0/663142796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d00b1620 con 0x7fc9d0162c80 2024-09-10T22:04:58.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.527+0000 7fc9ccc00640 1 -- 172.21.15.66:0/663142796 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fc9c800f040 con 0x7fc9d0162c80 2024-09-10T22:04:58.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.527+0000 7fc9ccc00640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc9b0072e50 0x7fc9b0075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:58.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.527+0000 7fc9ccc00640 1 -- 172.21.15.66:0/663142796 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5415+0+0 (secure 0 0 0) 0x7fc9c8094910 con 0x7fc9d0162c80 2024-09-10T22:04:58.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.527+0000 7fc9d4a00640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc9b0072e50 0x7fc9b0075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:58.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.527+0000 7fc9d4a00640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc9b0072e50 0x7fc9b0075310 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40af000 tx=0x7fc9c401b100 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:58.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.531+0000 7fc9ccc00640 1 -- 172.21.15.66:0/663142796 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc9c8016030 con 0x7fc9d0162c80 2024-09-10T22:04:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:58.703+0000 7fc9d7400640 1 -- 172.21.15.66:0/663142796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc9d00d00a0 con 0x7fc9d0162c80 2024-09-10T22:04:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.343+0000 7fc9ccc00640 1 -- 172.21.15.66:0/663142796 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) ==== 169+0+0 (secure 0 0 0) 0x7fc9c805eb20 con 0x7fc9d0162c80 2024-09-10T22:04:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:04:59.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.347+0000 7fc9d7400640 1 -- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc9b0072e50 msgr2=0x7fc9b0075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:59.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.347+0000 7fc9d7400640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc9b0072e50 0x7fc9b0075310 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40af000 tx=0x7fc9c401b100 comp rx=0 tx=0).stop 2024-09-10T22:04:59.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.347+0000 7fc9d7400640 1 -- 172.21.15.66:0/663142796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9d0162c80 msgr2=0x7fc9d006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.347+0000 7fc9d7400640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9d0162c80 0x7fc9d006f740 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800e970 tx=0x7fc9c800ee30 comp rx=0 tx=0).stop 2024-09-10T22:04:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.347+0000 7fc9d7400640 1 -- 172.21.15.66:0/663142796 shutdown_connections 2024-09-10T22:04:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.347+0000 7fc9d7400640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc9b0072e50 0x7fc9b0075310 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.347+0000 7fc9d7400640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9d0162c80 0x7fc9d006f740 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.347+0000 7fc9d7400640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9d01605c0 0x7fc9d006f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.347+0000 7fc9d7400640 1 --2- 172.21.15.66:0/663142796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00b2b10 0x7fc9d0151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.347+0000 7fc9d7400640 1 -- 172.21.15.66:0/663142796 >> 172.21.15.66:0/663142796 conn(0x7fc9d00d1ce0 msgr2=0x7fc9d01550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.347+0000 7fc9d7400640 1 -- 172.21.15.66:0/663142796 shutdown_connections 2024-09-10T22:04:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.347+0000 7fc9d7400640 1 -- 172.21.15.66:0/663142796 wait complete. 2024-09-10T22:04:59.386 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T22:04:59.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 -- 172.21.15.66:0/3703242361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76b40cfa80 msgr2=0x7f76b40cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:59.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 --2- 172.21.15.66:0/3703242361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76b40cfa80 0x7f76b40cb1f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f76b40ce6c0 tx=0x7f769c01c8c0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 -- 172.21.15.66:0/3703242361 shutdown_connections 2024-09-10T22:04:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 --2- 172.21.15.66:0/3703242361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76b40cb730 0x7f76b40ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 --2- 172.21.15.66:0/3703242361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76b40cfa80 0x7f76b40cb1f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 --2- 172.21.15.66:0/3703242361 >> v2:172.21.15.94:3300/0 conn(0x7f76b415ac70 0x7f76b40d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 -- 172.21.15.66:0/3703242361 >> 172.21.15.66:0/3703242361 conn(0x7f76b40d0ba0 msgr2=0x7f76b40d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 -- 172.21.15.66:0/3703242361 shutdown_connections 2024-09-10T22:04:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 -- 172.21.15.66:0/3703242361 wait complete. 2024-09-10T22:04:59.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 Processor -- start 2024-09-10T22:04:59.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 -- start start 2024-09-10T22:04:59.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76b40cb730 0x7f76b40b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:59.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76b40cfa80 0x7f76b40a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:59.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76b415ac70 0x7f76b40a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:59.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76b406d2f0 con 0x7f76b415ac70 2024-09-10T22:04:59.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76b406d170 con 0x7f76b40cfa80 2024-09-10T22:04:59.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76baa00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f76b406d470 con 0x7f76b40cb730 2024-09-10T22:04:59.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76b8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76b415ac70 0x7f76b40a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:59.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76b8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76b415ac70 0x7f76b40a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39508/0 (socket says 172.21.15.66:39508) 2024-09-10T22:04:59.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76b8a00640 1 -- 172.21.15.66:0/606528163 learned_addr learned my addr 172.21.15.66:0/606528163 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:59.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76b3e00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76b40cb730 0x7f76b40b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:59.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76abe00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76b40cfa80 0x7f76b40a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:59.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76b8a00640 1 -- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76b40cb730 msgr2=0x7f76b40b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:59.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76b8a00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76b40cb730 0x7f76b40b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76b8a00640 1 -- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76b40cfa80 msgr2=0x7f76b40a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:59.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76b8a00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76b40cfa80 0x7f76b40a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76b8a00640 1 -- 172.21.15.66:0/606528163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b4166620 con 0x7f76b415ac70 2024-09-10T22:04:59.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.539+0000 7f76b8a00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76b415ac70 0x7f76b40a0900 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00e990 tx=0x7f76ac00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:59.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.543+0000 7f76b0c00640 1 -- 172.21.15.66:0/606528163 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac004d30 con 0x7f76b415ac70 2024-09-10T22:04:59.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.543+0000 7f76b0c00640 1 -- 172.21.15.66:0/606528163 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76ac00cb00 con 0x7f76b415ac70 2024-09-10T22:04:59.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.543+0000 7f76b0c00640 1 -- 172.21.15.66:0/606528163 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac005020 con 0x7f76b415ac70 2024-09-10T22:04:59.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.543+0000 7f76baa00640 1 -- 172.21.15.66:0/606528163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b4166900 con 0x7f76b415ac70 2024-09-10T22:04:59.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.543+0000 7f76baa00640 1 -- 172.21.15.66:0/606528163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b4166ec0 con 0x7f76b415ac70 2024-09-10T22:04:59.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.543+0000 7f76baa00640 1 -- 172.21.15.66:0/606528163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b406d990 con 0x7f76b415ac70 2024-09-10T22:04:59.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.543+0000 7f76b0c00640 1 -- 172.21.15.66:0/606528163 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f76ac00f040 con 0x7f76b415ac70 2024-09-10T22:04:59.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.543+0000 7f76b0c00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7690072e50 0x7f7690075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:59.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.543+0000 7f76b0c00640 1 -- 172.21.15.66:0/606528163 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f76ac094820 con 0x7f76b415ac70 2024-09-10T22:04:59.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.543+0000 7f76b3e00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7690072e50 0x7f7690075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:59.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.547+0000 7f76b3e00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7690072e50 0x7f7690075310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f76a4005fd0 tx=0x7f76a4005f20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:59.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.547+0000 7f76b0c00640 1 -- 172.21.15.66:0/606528163 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f76ac016030 con 0x7f76b415ac70 2024-09-10T22:04:59.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.723+0000 7f76baa00640 1 -- 172.21.15.66:0/606528163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f76b40b5670 con 0x7f76b415ac70 2024-09-10T22:04:59.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.727+0000 7f76b0c00640 1 -- 172.21.15.66:0/606528163 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f76ac05f260 con 0x7f76b415ac70 2024-09-10T22:04:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.727+0000 7f76baa00640 1 -- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7690072e50 msgr2=0x7f7690075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.727+0000 7f76baa00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7690072e50 0x7f7690075310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f76a4005fd0 tx=0x7f76a4005f20 comp rx=0 tx=0).stop 2024-09-10T22:04:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.727+0000 7f76baa00640 1 -- 172.21.15.66:0/606528163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76b415ac70 msgr2=0x7f76b40a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.727+0000 7f76baa00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76b415ac70 0x7f76b40a0900 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00e990 tx=0x7f76ac00ee50 comp rx=0 tx=0).stop 2024-09-10T22:04:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.731+0000 7f76baa00640 1 -- 172.21.15.66:0/606528163 shutdown_connections 2024-09-10T22:04:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.731+0000 7f76baa00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7690072e50 0x7f7690075310 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.731+0000 7f76baa00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76b415ac70 0x7f76b40a0900 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.731+0000 7f76baa00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76b40cfa80 0x7f76b40a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.731+0000 7f76baa00640 1 --2- 172.21.15.66:0/606528163 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76b40cb730 0x7f76b40b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.731+0000 7f76baa00640 1 -- 172.21.15.66:0/606528163 >> 172.21.15.66:0/606528163 conn(0x7f76b40d0ba0 msgr2=0x7f76b40b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.731+0000 7f76baa00640 1 -- 172.21.15.66:0/606528163 shutdown_connections 2024-09-10T22:04:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.731+0000 7f76baa00640 1 -- 172.21.15.66:0/606528163 wait complete. 2024-09-10T22:04:59.754 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-10T22:04:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.895+0000 7f41f5e00640 1 -- 172.21.15.66:0/2246279772 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f015bd40 msgr2=0x7f41f00cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.895+0000 7f41f5e00640 1 --2- 172.21.15.66:0/2246279772 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f015bd40 0x7f41f00cffa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009900 tx=0x7f41e001c850 comp rx=0 tx=0).stop 2024-09-10T22:04:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.895+0000 7f41f5e00640 1 -- 172.21.15.66:0/2246279772 shutdown_connections 2024-09-10T22:04:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.895+0000 7f41f5e00640 1 --2- 172.21.15.66:0/2246279772 >> v2:172.21.15.94:3300/0 conn(0x7f41f00cf180 0x7f41f00d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.895+0000 7f41f5e00640 1 --2- 172.21.15.66:0/2246279772 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f015bd40 0x7f41f00cffa0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.895+0000 7f41f5e00640 1 -- 172.21.15.66:0/2246279772 >> 172.21.15.66:0/2246279772 conn(0x7f41f00d1c60 msgr2=0x7f41f00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:59.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.895+0000 7f41f5e00640 1 -- 172.21.15.66:0/2246279772 shutdown_connections 2024-09-10T22:04:59.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.895+0000 7f41f5e00640 1 -- 172.21.15.66:0/2246279772 wait complete. 2024-09-10T22:04:59.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41f5e00640 1 Processor -- start 2024-09-10T22:04:59.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41f5e00640 1 -- start start 2024-09-10T22:04:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41f5e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f00cf180 0x7f41f014ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f00cf830 0x7f41f014fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41f5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f015bd40 0x7f41f015add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41f5e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41f0168ba0 con 0x7f41f00cf830 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41f5e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41f0168a20 con 0x7f41f015bd40 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41f5e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f41f0168d20 con 0x7f41f00cf180 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f00cf830 0x7f41f014fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f00cf830 0x7f41f014fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39518/0 (socket says 172.21.15.66:39518) 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41e7e00640 1 -- 172.21.15.66:0/4262702719 learned_addr learned my addr 172.21.15.66:0/4262702719 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41eea00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f00cf180 0x7f41f014ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41ef400640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f015bd40 0x7f41f015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41e7e00640 1 -- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f00cf180 msgr2=0x7f41f014ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41e7e00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f00cf180 0x7f41f014ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41e7e00640 1 -- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f015bd40 msgr2=0x7f41f015add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41e7e00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f015bd40 0x7f41f015add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41e7e00640 1 -- 172.21.15.66:0/4262702719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f0150800 con 0x7f41f00cf830 2024-09-10T22:04:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41eea00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f00cf180 0x7f41f014ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:04:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41e7e00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f00cf830 0x7f41f014fd40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f41d800c500 tx=0x7f41d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41e7400640 1 -- 172.21.15.66:0/4262702719 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8017070 con 0x7f41f00cf830 2024-09-10T22:04:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41e7400640 1 -- 172.21.15.66:0/4262702719 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41d80053c0 con 0x7f41f00cf830 2024-09-10T22:04:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41e7400640 1 -- 172.21.15.66:0/4262702719 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8012890 con 0x7f41f00cf830 2024-09-10T22:04:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41f5e00640 1 -- 172.21.15.66:0/4262702719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f01fbc80 con 0x7f41f00cf830 2024-09-10T22:04:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41f5e00640 1 -- 172.21.15.66:0/4262702719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f01fb480 con 0x7f41f00cf830 2024-09-10T22:04:59.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.899+0000 7f41f5e00640 1 -- 172.21.15.66:0/4262702719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f01fafd0 con 0x7f41f00cf830 2024-09-10T22:04:59.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.903+0000 7f41e7400640 1 -- 172.21.15.66:0/4262702719 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f41d8003600 con 0x7f41f00cf830 2024-09-10T22:04:59.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.903+0000 7f41e7400640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41cc072e50 0x7f41cc075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:59.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.903+0000 7f41e7400640 1 -- 172.21.15.66:0/4262702719 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f41d80938d0 con 0x7f41f00cf830 2024-09-10T22:04:59.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.903+0000 7f41eea00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41cc072e50 0x7f41cc075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:59.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.903+0000 7f41eea00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41cc072e50 0x7f41cc075310 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f41e00af000 tx=0x7f41e001b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:59.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:04:59.907+0000 7f41e7400640 1 -- 172.21.15.66:0/4262702719 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f41d805e290 con 0x7f41f00cf830 2024-09-10T22:05:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.143+0000 7f41f5e00640 1 -- 172.21.15.66:0/4262702719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7f41f00b0df0 con 0x7f41f00cf830 2024-09-10T22:05:00.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.343+0000 7f41e7400640 1 -- 172.21.15.66:0/4262702719 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v210) ==== 166+0+0 (secure 0 0 0) 0x7f41d805db90 con 0x7f41f00cf830 2024-09-10T22:05:00.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.347+0000 7f41f5e00640 1 -- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41cc072e50 msgr2=0x7f41cc075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:00.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.347+0000 7f41f5e00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41cc072e50 0x7f41cc075310 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f41e00af000 tx=0x7f41e001b090 comp rx=0 tx=0).stop 2024-09-10T22:05:00.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.347+0000 7f41f5e00640 1 -- 172.21.15.66:0/4262702719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f00cf830 msgr2=0x7f41f014fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:00.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.347+0000 7f41f5e00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f00cf830 0x7f41f014fd40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f41d800c500 tx=0x7f41d800c9c0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.347+0000 7f41f5e00640 1 -- 172.21.15.66:0/4262702719 shutdown_connections 2024-09-10T22:05:00.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.347+0000 7f41f5e00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41cc072e50 0x7f41cc075310 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.347+0000 7f41f5e00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f015bd40 0x7f41f015add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.347+0000 7f41f5e00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f00cf830 0x7f41f014fd40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.347+0000 7f41f5e00640 1 --2- 172.21.15.66:0/4262702719 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f00cf180 0x7f41f014ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.347+0000 7f41f5e00640 1 -- 172.21.15.66:0/4262702719 >> 172.21.15.66:0/4262702719 conn(0x7f41f00d1c60 msgr2=0x7f41f00b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:00.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.347+0000 7f41f5e00640 1 -- 172.21.15.66:0/4262702719 shutdown_connections 2024-09-10T22:05:00.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.347+0000 7f41f5e00640 1 -- 172.21.15.66:0/4262702719 wait complete. 2024-09-10T22:05:00.373 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-10T22:05:00.373 INFO:tasks.cephfs_test_runner:Starting test: test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-10T22:05:00.374 INFO:tasks.cephfs_test_runner:test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-10T22:05:00.374 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:05:00.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 -- 172.21.15.66:0/2449369626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4ec004810 msgr2=0x7ff4ec004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 --2- 172.21.15.66:0/2449369626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4ec004810 0x7ff4ec004c90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0006f30 tx=0x7ff4f001ce80 comp rx=0 tx=0).stop 2024-09-10T22:05:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff507e00640 1 -- 172.21.15.66:0/2449369626 reap_dead start 2024-09-10T22:05:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 -- 172.21.15.66:0/2449369626 shutdown_connections 2024-09-10T22:05:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 -- 172.21.15.66:0/2449369626 >> 172.21.15.66:0/2449369626 conn(0x7ff5080cfa30 msgr2=0x7ff5080cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 -- 172.21.15.66:0/2449369626 shutdown_connections 2024-09-10T22:05:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 -- 172.21.15.66:0/2449369626 wait complete. 2024-09-10T22:05:00.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 Processor -- start 2024-09-10T22:05:00.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 -- start start 2024-09-10T22:05:00.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4ec004810 0x7ff50814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:00.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff508152920 0x7ff5080ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:00.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5080c6b00 0x7ff5080ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:00.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5080cb8a0 con 0x7ff4ec004810 2024-09-10T22:05:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5080c5d50 con 0x7ff508152920 2024-09-10T22:05:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff50e000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff5080cba20 con 0x7ff5080c6b00 2024-09-10T22:05:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff507400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4ec004810 0x7ff50814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff507400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4ec004810 0x7ff50814d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39534/0 (socket says 172.21.15.66:39534) 2024-09-10T22:05:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff507400640 1 -- 172.21.15.66:0/2350159294 learned_addr learned my addr 172.21.15.66:0/2350159294 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff4ffe00640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff508152920 0x7ff5080ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff507e00640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5080c6b00 0x7ff5080ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff507400640 1 -- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5080c6b00 msgr2=0x7ff5080ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff507400640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5080c6b00 0x7ff5080ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff507400640 1 -- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff508152920 msgr2=0x7ff5080ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff507400640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff508152920 0x7ff5080ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.519+0000 7ff507400640 1 -- 172.21.15.66:0/2350159294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5080c76c0 con 0x7ff4ec004810 2024-09-10T22:05:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.523+0000 7ff507400640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4ec004810 0x7ff50814d490 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8005f80 tx=0x7ff4f8011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.523+0000 7ff4ff400640 1 -- 172.21.15.66:0/2350159294 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8005320 con 0x7ff4ec004810 2024-09-10T22:05:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.523+0000 7ff4ff400640 1 -- 172.21.15.66:0/2350159294 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4f8018020 con 0x7ff4ec004810 2024-09-10T22:05:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.523+0000 7ff50e000640 1 -- 172.21.15.66:0/2350159294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5081f7d50 con 0x7ff4ec004810 2024-09-10T22:05:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.523+0000 7ff4ff400640 1 -- 172.21.15.66:0/2350159294 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8003860 con 0x7ff4ec004810 2024-09-10T22:05:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.523+0000 7ff50e000640 1 -- 172.21.15.66:0/2350159294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5081f8310 con 0x7ff4ec004810 2024-09-10T22:05:00.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.523+0000 7ff50e000640 1 -- 172.21.15.66:0/2350159294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff508070330 con 0x7ff4ec004810 2024-09-10T22:05:00.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.523+0000 7ff4ff400640 1 -- 172.21.15.66:0/2350159294 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7ff4f8010040 con 0x7ff4ec004810 2024-09-10T22:05:00.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.523+0000 7ff4ff400640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff4e4072e50 0x7ff4e4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:00.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.523+0000 7ff4ff400640 1 -- 172.21.15.66:0/2350159294 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7ff4f8097260 con 0x7ff4ec004810 2024-09-10T22:05:00.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.527+0000 7ff4ffe00640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff4e4072e50 0x7ff4e4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:00.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.527+0000 7ff4ffe00640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff4e4072e50 0x7ff4e4075310 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00ac000 tx=0x7ff4f0019cc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:00.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.527+0000 7ff4ff400640 1 -- 172.21.15.66:0/2350159294 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff4f8022020 con 0x7ff4ec004810 2024-09-10T22:05:00.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.703+0000 7ff50e000640 1 -- 172.21.15.66:0/2350159294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff508076650 con 0x7ff4ec004810 2024-09-10T22:05:00.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.703+0000 7ff4ff400640 1 -- 172.21.15.66:0/2350159294 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+12960 (secure 0 0 0) 0x7ff4f8097080 con 0x7ff4ec004810 2024-09-10T22:05:00.709 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:05:00.709 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":51,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:04:59.341254+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/1215985114":"2024-09-11T22:04:56.330594+0000","172.21.15.66:6834/1215985114":"2024-09-11T22:04:56.330594+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:05:00.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.707+0000 7ff50e000640 1 -- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff4e4072e50 msgr2=0x7ff4e4075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:00.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.707+0000 7ff50e000640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff4e4072e50 0x7ff4e4075310 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00ac000 tx=0x7ff4f0019cc0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.707+0000 7ff50e000640 1 -- 172.21.15.66:0/2350159294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4ec004810 msgr2=0x7ff50814d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:00.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.707+0000 7ff50e000640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4ec004810 0x7ff50814d490 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8005f80 tx=0x7ff4f8011800 comp rx=0 tx=0).stop 2024-09-10T22:05:00.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.707+0000 7ff50e000640 1 -- 172.21.15.66:0/2350159294 shutdown_connections 2024-09-10T22:05:00.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.707+0000 7ff50e000640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff4e4072e50 0x7ff4e4075310 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.707+0000 7ff50e000640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5080c6b00 0x7ff5080ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.707+0000 7ff50e000640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff508152920 0x7ff5080ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.707+0000 7ff50e000640 1 --2- 172.21.15.66:0/2350159294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4ec004810 0x7ff50814d490 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.707+0000 7ff50e000640 1 -- 172.21.15.66:0/2350159294 >> 172.21.15.66:0/2350159294 conn(0x7ff5080cfa30 msgr2=0x7ff5080cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:00.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.707+0000 7ff50e000640 1 -- 172.21.15.66:0/2350159294 shutdown_connections 2024-09-10T22:05:00.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.707+0000 7ff50e000640 1 -- 172.21.15.66:0/2350159294 wait complete. 2024-09-10T22:05:00.732 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:00:22.340318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:05:00.733 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:05:00.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.879+0000 7f0011200640 1 -- 172.21.15.66:0/2863051149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f000c0ae620 msgr2=0x7f000c0725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:00.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.879+0000 7f0011200640 1 --2- 172.21.15.66:0/2863051149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f000c0ae620 0x7f000c0725d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efff4009840 tx=0x7efff401c7c0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.879+0000 7f0011200640 1 -- 172.21.15.66:0/2863051149 shutdown_connections 2024-09-10T22:05:00.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.879+0000 7f0011200640 1 --2- 172.21.15.66:0/2863051149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f000c0ae620 0x7f000c0725d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.879+0000 7f0011200640 1 --2- 172.21.15.66:0/2863051149 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f000c0c4b10 0x7f000c06e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.879+0000 7f0011200640 1 -- 172.21.15.66:0/2863051149 >> 172.21.15.66:0/2863051149 conn(0x7f000c0cfa30 msgr2=0x7f000c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:00.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.879+0000 7f0011200640 1 -- 172.21.15.66:0/2863051149 shutdown_connections 2024-09-10T22:05:00.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.879+0000 7f0011200640 1 -- 172.21.15.66:0/2863051149 wait complete. 2024-09-10T22:05:00.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.879+0000 7f0011200640 1 Processor -- start 2024-09-10T22:05:00.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.879+0000 7f0011200640 1 -- start start 2024-09-10T22:05:00.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0011200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f000c0ae620 0x7f000c1fb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:00.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0011200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f000c0c4b10 0x7f000c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:00.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0011200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f000c0c4ef0 0x7f000c153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:00.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0011200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f000c168440 con 0x7f000c0c4ef0 2024-09-10T22:05:00.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0011200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f000c1682c0 con 0x7f000c0ae620 2024-09-10T22:05:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0011200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f000c1685c0 con 0x7f000c0c4b10 2024-09-10T22:05:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f000a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f000c0c4ef0 0x7f000c153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f000a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f000c0c4ef0 0x7f000c153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39550/0 (socket says 172.21.15.66:39550) 2024-09-10T22:05:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f000a800640 1 -- 172.21.15.66:0/337009762 learned_addr learned my addr 172.21.15.66:0/337009762 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0001400640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f000c0c4b10 0x7f000c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0009e00640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f000c0ae620 0x7f000c1fb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f000a800640 1 -- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f000c0c4b10 msgr2=0x7f000c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f000a800640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f000c0c4b10 0x7f000c15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f000a800640 1 -- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f000c0ae620 msgr2=0x7f000c1fb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f000a800640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f000c0ae620 0x7f000c1fb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f000a800640 1 -- 172.21.15.66:0/337009762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c1fbaf0 con 0x7f000c0c4ef0 2024-09-10T22:05:00.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f000a800640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f000c0c4ef0 0x7f000c153450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f000400cb10 tx=0x7f000400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:00.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0002a00640 1 -- 172.21.15.66:0/337009762 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004016020 con 0x7f000c0c4ef0 2024-09-10T22:05:00.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0002a00640 1 -- 172.21.15.66:0/337009762 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f000400ce50 con 0x7f000c0c4ef0 2024-09-10T22:05:00.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0002a00640 1 -- 172.21.15.66:0/337009762 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000400adc0 con 0x7f000c0c4ef0 2024-09-10T22:05:00.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0011200640 1 -- 172.21.15.66:0/337009762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c1fbdd0 con 0x7f000c0c4ef0 2024-09-10T22:05:00.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0011200640 1 -- 172.21.15.66:0/337009762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c153990 con 0x7f000c0c4ef0 2024-09-10T22:05:00.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0011200640 1 -- 172.21.15.66:0/337009762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000c0725d0 con 0x7f000c0c4ef0 2024-09-10T22:05:00.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.883+0000 7f0002a00640 1 -- 172.21.15.66:0/337009762 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f000401a3b0 con 0x7f000c0c4ef0 2024-09-10T22:05:00.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.887+0000 7f0002a00640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7effe8072e50 0x7effe8075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:00.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.887+0000 7f0002a00640 1 -- 172.21.15.66:0/337009762 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f0004094370 con 0x7f000c0c4ef0 2024-09-10T22:05:00.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.887+0000 7f0009e00640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7effe8072e50 0x7effe8075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:00.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.887+0000 7f0009e00640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7effe8072e50 0x7effe8075310 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efffc009be0 tx=0x7efffc0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:00.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:00.891+0000 7f0002a00640 1 -- 172.21.15.66:0/337009762 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0004020030 con 0x7f000c0c4ef0 2024-09-10T22:05:01.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.063+0000 7f0011200640 1 -- 172.21.15.66:0/337009762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f000c0b3a00 con 0x7f000c0c4ef0 2024-09-10T22:05:01.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.063+0000 7f0002a00640 1 -- 172.21.15.66:0/337009762 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7f000407f0a0 con 0x7f000c0c4ef0 2024-09-10T22:05:01.072 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-09-10T22:05:01.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.067+0000 7f0011200640 1 -- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7effe8072e50 msgr2=0x7effe8075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:01.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.067+0000 7f0011200640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7effe8072e50 0x7effe8075310 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efffc009be0 tx=0x7efffc0091e0 comp rx=0 tx=0).stop 2024-09-10T22:05:01.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.067+0000 7f0011200640 1 -- 172.21.15.66:0/337009762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f000c0c4ef0 msgr2=0x7f000c153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:01.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.067+0000 7f0011200640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f000c0c4ef0 0x7f000c153450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f000400cb10 tx=0x7f000400cc00 comp rx=0 tx=0).stop 2024-09-10T22:05:01.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.067+0000 7f0011200640 1 -- 172.21.15.66:0/337009762 shutdown_connections 2024-09-10T22:05:01.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.067+0000 7f0011200640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7effe8072e50 0x7effe8075310 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:01.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.067+0000 7f0011200640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f000c0c4ef0 0x7f000c153450 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:01.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.067+0000 7f0011200640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f000c0c4b10 0x7f000c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:01.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.067+0000 7f0011200640 1 --2- 172.21.15.66:0/337009762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f000c0ae620 0x7f000c1fb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:01.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.067+0000 7f0011200640 1 -- 172.21.15.66:0/337009762 >> 172.21.15.66:0/337009762 conn(0x7f000c0cfa30 msgr2=0x7f000c072e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:01.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.067+0000 7f0011200640 1 -- 172.21.15.66:0/337009762 shutdown_connections 2024-09-10T22:05:01.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.067+0000 7f0011200640 1 -- 172.21.15.66:0/337009762 wait complete. 2024-09-10T22:05:01.090 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-10T22:05:01.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.355+0000 7f7723000640 1 -- 172.21.15.66:0/3040407083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f771c0ae8c0 msgr2=0x7f771c06eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:01.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.355+0000 7f7723000640 1 --2- 172.21.15.66:0/3040407083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f771c0ae8c0 0x7f771c06eba0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7714009bc0 tx=0x7f771401cb70 comp rx=0 tx=0).stop 2024-09-10T22:05:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.355+0000 7f7723000640 1 -- 172.21.15.66:0/3040407083 shutdown_connections 2024-09-10T22:05:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.355+0000 7f7723000640 1 --2- 172.21.15.66:0/3040407083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f771c0ae8c0 0x7f771c06eba0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.355+0000 7f7723000640 1 --2- 172.21.15.66:0/3040407083 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f771c06d330 0x7f771c06e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.355+0000 7f7723000640 1 -- 172.21.15.66:0/3040407083 >> 172.21.15.66:0/3040407083 conn(0x7f771c0cc3c0 msgr2=0x7f771c0cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.355+0000 7f7723000640 1 -- 172.21.15.66:0/3040407083 shutdown_connections 2024-09-10T22:05:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.355+0000 7f7723000640 1 -- 172.21.15.66:0/3040407083 wait complete. 2024-09-10T22:05:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7723000640 1 Processor -- start 2024-09-10T22:05:01.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7723000640 1 -- start start 2024-09-10T22:05:01.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7723000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f771c06d330 0x7f771c16c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:01.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7723000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f771c08bbb0 0x7f771c16cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:01.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7723000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771c0ae8c0 0x7f771c168340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:01.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7723000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f771c178280 con 0x7f771c08bbb0 2024-09-10T22:05:01.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7723000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f771c178100 con 0x7f771c0ae8c0 2024-09-10T22:05:01.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7723000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f771c178400 con 0x7f771c06d330 2024-09-10T22:05:01.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7721200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f771c08bbb0 0x7f771c16cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:01.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7721200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f771c08bbb0 0x7f771c16cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39566/0 (socket says 172.21.15.66:39566) 2024-09-10T22:05:01.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7721200640 1 -- 172.21.15.66:0/2475549181 learned_addr learned my addr 172.21.15.66:0/2475549181 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:01.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7721c00640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f771c06d330 0x7f771c16c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:01.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7722600640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771c0ae8c0 0x7f771c168340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:01.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7721200640 1 -- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f771c06d330 msgr2=0x7f771c16c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:01.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7721200640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f771c06d330 0x7f771c16c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:01.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7721200640 1 -- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771c0ae8c0 msgr2=0x7f771c168340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:01.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7721200640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771c0ae8c0 0x7f771c168340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:01.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7721200640 1 -- 172.21.15.66:0/2475549181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c16b9e0 con 0x7f771c08bbb0 2024-09-10T22:05:01.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7722600640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771c0ae8c0 0x7f771c168340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:05:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7721200640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f771c08bbb0 0x7f771c16cd80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f770800f8d0 tx=0x7f770800fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7719600640 1 -- 172.21.15.66:0/2475549181 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770800dbd0 con 0x7f771c08bbb0 2024-09-10T22:05:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7719600640 1 -- 172.21.15.66:0/2475549181 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7708004020 con 0x7f771c08bbb0 2024-09-10T22:05:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7719600640 1 -- 172.21.15.66:0/2475549181 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770800de60 con 0x7f771c08bbb0 2024-09-10T22:05:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7723000640 1 -- 172.21.15.66:0/2475549181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771c16bcc0 con 0x7f771c08bbb0 2024-09-10T22:05:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7723000640 1 -- 172.21.15.66:0/2475549181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771c168c20 con 0x7f771c08bbb0 2024-09-10T22:05:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.359+0000 7f7723000640 1 -- 172.21.15.66:0/2475549181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f771c06cc20 con 0x7f771c08bbb0 2024-09-10T22:05:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.363+0000 7f7719600640 1 -- 172.21.15.66:0/2475549181 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f7708003600 con 0x7f771c08bbb0 2024-09-10T22:05:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.363+0000 7f7719600640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f76fc072e50 0x7f76fc075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.363+0000 7f7719600640 1 -- 172.21.15.66:0/2475549181 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f7708094390 con 0x7f771c08bbb0 2024-09-10T22:05:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.363+0000 7f7721c00640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f76fc072e50 0x7f76fc075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.363+0000 7f7721c00640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f76fc072e50 0x7f76fc075310 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7710000f70 tx=0x7f7710009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:01.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.367+0000 7f7719600640 1 -- 172.21.15.66:0/2475549181 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f770805f210 con 0x7f771c08bbb0 2024-09-10T22:05:01.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:01.607+0000 7f7723000640 1 -- 172.21.15.66:0/2475549181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7f771c0b54a0 con 0x7f771c08bbb0 2024-09-10T22:05:02.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.351+0000 7f7719600640 1 -- 172.21.15.66:0/2475549181 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v212) ==== 170+0+0 (secure 0 0 0) 0x7f770805eb10 con 0x7f771c08bbb0 2024-09-10T22:05:02.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.355+0000 7f7723000640 1 -- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f76fc072e50 msgr2=0x7f76fc075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:02.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.355+0000 7f7723000640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f76fc072e50 0x7f76fc075310 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7710000f70 tx=0x7f7710009040 comp rx=0 tx=0).stop 2024-09-10T22:05:02.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.355+0000 7f7723000640 1 -- 172.21.15.66:0/2475549181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f771c08bbb0 msgr2=0x7f771c16cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:02.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.355+0000 7f7723000640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f771c08bbb0 0x7f771c16cd80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f770800f8d0 tx=0x7f770800fd90 comp rx=0 tx=0).stop 2024-09-10T22:05:02.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.355+0000 7f7723000640 1 -- 172.21.15.66:0/2475549181 shutdown_connections 2024-09-10T22:05:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.355+0000 7f7723000640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f76fc072e50 0x7f76fc075310 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.355+0000 7f7723000640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771c0ae8c0 0x7f771c168340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.355+0000 7f7723000640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f771c08bbb0 0x7f771c16cd80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.355+0000 7f7723000640 1 --2- 172.21.15.66:0/2475549181 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f771c06d330 0x7f771c16c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.355+0000 7f7723000640 1 -- 172.21.15.66:0/2475549181 >> 172.21.15.66:0/2475549181 conn(0x7f771c0cc3c0 msgr2=0x7f771c06e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.355+0000 7f7723000640 1 -- 172.21.15.66:0/2475549181 shutdown_connections 2024-09-10T22:05:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.355+0000 7f7723000640 1 -- 172.21.15.66:0/2475549181 wait complete. 2024-09-10T22:05:02.384 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T22:05:02.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.531+0000 7f9d02000640 1 -- 172.21.15.66:0/1159412721 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cfc15b880 msgr2=0x7f9cfc0701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:02.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.531+0000 7f9d02000640 1 --2- 172.21.15.66:0/1159412721 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cfc15b880 0x7f9cfc0701f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009840 tx=0x7f9cf001c7f0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.531+0000 7f9d02000640 1 -- 172.21.15.66:0/1159412721 shutdown_connections 2024-09-10T22:05:02.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.531+0000 7f9d02000640 1 --2- 172.21.15.66:0/1159412721 >> v2:172.21.15.94:3300/0 conn(0x7f9cfc0cd240 0x7f9cfc073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.531+0000 7f9d02000640 1 --2- 172.21.15.66:0/1159412721 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cfc15b880 0x7f9cfc0701f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.531+0000 7f9d02000640 1 -- 172.21.15.66:0/1159412721 >> 172.21.15.66:0/1159412721 conn(0x7f9cfc0b1cd0 msgr2=0x7f9cfc0b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:02.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.531+0000 7f9d02000640 1 -- 172.21.15.66:0/1159412721 shutdown_connections 2024-09-10T22:05:02.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.531+0000 7f9d02000640 1 -- 172.21.15.66:0/1159412721 wait complete. 2024-09-10T22:05:02.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d02000640 1 Processor -- start 2024-09-10T22:05:02.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d02000640 1 -- start start 2024-09-10T22:05:02.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d02000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cfc09dc90 0x7f9cfc150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:02.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d02000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cfc0cd240 0x7f9cfc15e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:02.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d02000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cfc15b880 0x7f9cfc0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:02.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d02000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc168690 con 0x7f9cfc09dc90 2024-09-10T22:05:02.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d02000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc074f50 con 0x7f9cfc0cd240 2024-09-10T22:05:02.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d02000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9cfc168810 con 0x7f9cfc15b880 2024-09-10T22:05:02.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d00c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cfc09dc90 0x7f9cfc150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:02.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d00c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cfc09dc90 0x7f9cfc150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35016/0 (socket says 172.21.15.66:35016) 2024-09-10T22:05:02.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d00c00640 1 -- 172.21.15.66:0/1273008941 learned_addr learned my addr 172.21.15.66:0/1273008941 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:02.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d01600640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cfc15b880 0x7f9cfc0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:02.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9cfbe00640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cfc0cd240 0x7f9cfc15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:02.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d00c00640 1 -- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cfc15b880 msgr2=0x7f9cfc0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:02.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d00c00640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cfc15b880 0x7f9cfc0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d00c00640 1 -- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cfc0cd240 msgr2=0x7f9cfc15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:02.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d00c00640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cfc0cd240 0x7f9cfc15e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d00c00640 1 -- 172.21.15.66:0/1273008941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cfc15c160 con 0x7f9cfc09dc90 2024-09-10T22:05:02.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d00c00640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cfc09dc90 0x7f9cfc150ce0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009790 tx=0x7f9cf0005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:02.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9cf8c00640 1 -- 172.21.15.66:0/1273008941 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00b2040 con 0x7f9cfc09dc90 2024-09-10T22:05:02.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9cf8c00640 1 -- 172.21.15.66:0/1273008941 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cf0004330 con 0x7f9cfc09dc90 2024-09-10T22:05:02.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d02000640 1 -- 172.21.15.66:0/1273008941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cfc15ed10 con 0x7f9cfc09dc90 2024-09-10T22:05:02.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9cf8c00640 1 -- 172.21.15.66:0/1273008941 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0003840 con 0x7f9cfc09dc90 2024-09-10T22:05:02.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.535+0000 7f9d02000640 1 -- 172.21.15.66:0/1273008941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cfc15c700 con 0x7f9cfc09dc90 2024-09-10T22:05:02.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.539+0000 7f9d02000640 1 -- 172.21.15.66:0/1273008941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cfc073f10 con 0x7f9cfc09dc90 2024-09-10T22:05:02.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.539+0000 7f9cf8c00640 1 -- 172.21.15.66:0/1273008941 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f9cf00b5070 con 0x7f9cfc09dc90 2024-09-10T22:05:02.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.539+0000 7f9cf8c00640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9cdc072e50 0x7f9cdc075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:02.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.539+0000 7f9cf8c00640 1 -- 172.21.15.66:0/1273008941 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f9cf012b1c0 con 0x7f9cfc09dc90 2024-09-10T22:05:02.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.539+0000 7f9cfbe00640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9cdc072e50 0x7f9cdc075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:02.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.543+0000 7f9cfbe00640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9cdc072e50 0x7f9cdc075310 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc09d530 tx=0x7f9ce8001420 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:02.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.543+0000 7f9cf8c00640 1 -- 172.21.15.66:0/1273008941 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9cf0016410 con 0x7f9cfc09dc90 2024-09-10T22:05:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.719+0000 7f9d02000640 1 -- 172.21.15.66:0/1273008941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f9cfc0b2300 con 0x7f9cfc09dc90 2024-09-10T22:05:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.719+0000 7f9cf8c00640 1 -- 172.21.15.66:0/1273008941 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+1998 (secure 0 0 0) 0x7f9cf00f8350 con 0x7f9cfc09dc90 2024-09-10T22:05:02.724 INFO:teuthology.orchestra.run.smithi066.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T22:05:02.725 INFO:teuthology.orchestra.run.smithi066.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T22:05:02.725 INFO:teuthology.orchestra.run.smithi066.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T22:05:02.725 INFO:teuthology.orchestra.run.smithi066.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T22:05:02.725 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4673.907365 2024-09-10T22:05:02.725 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4537.831230 2024-09-10T22:05:02.725 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 5706.341617 2024-09-10T22:05:02.725 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4484.013907 2024-09-10T22:05:02.725 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3862.285837 2024-09-10T22:05:02.725 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3776.439661 2024-09-10T22:05:02.725 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3820.270368 2024-09-10T22:05:02.725 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3863.602758 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds 20/20 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_ms 2/2 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_frag true 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_scatterstat true 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_verify_scatter true 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced client_mount_timeout 600 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_client 20/20 2024-09-10T22:05:02.726 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_ms 1/1 2024-09-10T22:05:02.727 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T22:05:02.727 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T22:05:02.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.723+0000 7f9d02000640 1 -- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9cdc072e50 msgr2=0x7f9cdc075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.723+0000 7f9d02000640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9cdc072e50 0x7f9cdc075310 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc09d530 tx=0x7f9ce8001420 comp rx=0 tx=0).stop 2024-09-10T22:05:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.723+0000 7f9d02000640 1 -- 172.21.15.66:0/1273008941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cfc09dc90 msgr2=0x7f9cfc150ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.723+0000 7f9d02000640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cfc09dc90 0x7f9cfc150ce0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009790 tx=0x7f9cf0005ca0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.723+0000 7f9d02000640 1 -- 172.21.15.66:0/1273008941 shutdown_connections 2024-09-10T22:05:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.723+0000 7f9d02000640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9cdc072e50 0x7f9cdc075310 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.723+0000 7f9d02000640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cfc15b880 0x7f9cfc0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.723+0000 7f9d02000640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cfc0cd240 0x7f9cfc15e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.723+0000 7f9d02000640 1 --2- 172.21.15.66:0/1273008941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cfc09dc90 0x7f9cfc150ce0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.723+0000 7f9d02000640 1 -- 172.21.15.66:0/1273008941 >> 172.21.15.66:0/1273008941 conn(0x7f9cfc0b1cd0 msgr2=0x7f9cfc0ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.723+0000 7f9d02000640 1 -- 172.21.15.66:0/1273008941 shutdown_connections 2024-09-10T22:05:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.723+0000 7f9d02000640 1 -- 172.21.15.66:0/1273008941 wait complete. 2024-09-10T22:05:02.747 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:05:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.891+0000 7f2966200640 1 -- 172.21.15.66:0/502053334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f296015bd40 msgr2=0x7f29600cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.891+0000 7f2966200640 1 --2- 172.21.15.66:0/502053334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f296015bd40 0x7f29600cffa0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2950009840 tx=0x7f295001c790 comp rx=0 tx=0).stop 2024-09-10T22:05:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.891+0000 7f2966200640 1 -- 172.21.15.66:0/502053334 shutdown_connections 2024-09-10T22:05:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.891+0000 7f2966200640 1 --2- 172.21.15.66:0/502053334 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29600cf180 0x7f29600d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.891+0000 7f2966200640 1 --2- 172.21.15.66:0/502053334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f296015bd40 0x7f29600cffa0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.891+0000 7f2966200640 1 -- 172.21.15.66:0/502053334 >> 172.21.15.66:0/502053334 conn(0x7f29600d1c60 msgr2=0x7f29600d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.891+0000 7f2966200640 1 -- 172.21.15.66:0/502053334 shutdown_connections 2024-09-10T22:05:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.891+0000 7f2966200640 1 -- 172.21.15.66:0/502053334 wait complete. 2024-09-10T22:05:02.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2966200640 1 Processor -- start 2024-09-10T22:05:02.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2966200640 1 -- start start 2024-09-10T22:05:02.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2966200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29600cf180 0x7f2960159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:02.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2966200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29600cf830 0x7f296014ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2966200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f296015bd40 0x7f296014d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2966200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2960168ba0 con 0x7f296015bd40 2024-09-10T22:05:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2966200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2960168a20 con 0x7f29600cf180 2024-09-10T22:05:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2966200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2960168d20 con 0x7f29600cf830 2024-09-10T22:05:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f295fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f296015bd40 0x7f296014d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f295fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f296015bd40 0x7f296014d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35032/0 (socket says 172.21.15.66:35032) 2024-09-10T22:05:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f295fe00640 1 -- 172.21.15.66:0/4095086773 learned_addr learned my addr 172.21.15.66:0/4095086773 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2957e00640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29600cf830 0x7f296014ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f295f400640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29600cf180 0x7f2960159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f295fe00640 1 -- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29600cf830 msgr2=0x7f296014ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f295fe00640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29600cf830 0x7f296014ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f295fe00640 1 -- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29600cf180 msgr2=0x7f2960159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f295fe00640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29600cf180 0x7f2960159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f295fe00640 1 -- 172.21.15.66:0/4095086773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296014a110 con 0x7f296015bd40 2024-09-10T22:05:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f295f400640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29600cf180 0x7f2960159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f295fe00640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f296015bd40 0x7f296014d3f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f295800ea50 tx=0x7f295800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2957400640 1 -- 172.21.15.66:0/4095086773 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295800cdc0 con 0x7f296015bd40 2024-09-10T22:05:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2957400640 1 -- 172.21.15.66:0/4095086773 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2958004c40 con 0x7f296015bd40 2024-09-10T22:05:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2966200640 1 -- 172.21.15.66:0/4095086773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2960149b50 con 0x7f296015bd40 2024-09-10T22:05:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2957400640 1 -- 172.21.15.66:0/4095086773 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958005280 con 0x7f296015bd40 2024-09-10T22:05:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.895+0000 7f2966200640 1 -- 172.21.15.66:0/4095086773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296014c6b0 con 0x7f296015bd40 2024-09-10T22:05:02.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.899+0000 7f2966200640 1 -- 172.21.15.66:0/4095086773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29600aec10 con 0x7f296015bd40 2024-09-10T22:05:02.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.899+0000 7f2957400640 1 -- 172.21.15.66:0/4095086773 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f295800f040 con 0x7f296015bd40 2024-09-10T22:05:02.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.899+0000 7f2957400640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f293c072e50 0x7f293c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:02.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.899+0000 7f2957400640 1 -- 172.21.15.66:0/4095086773 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f2958094860 con 0x7f296015bd40 2024-09-10T22:05:02.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.899+0000 7f295f400640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f293c072e50 0x7f293c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:02.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.899+0000 7f295f400640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f293c072e50 0x7f293c075310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f29500af000 tx=0x7f295001afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:02.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:02.903+0000 7f2957400640 1 -- 172.21.15.66:0/4095086773 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2958016030 con 0x7f296015bd40 2024-09-10T22:05:03.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.079+0000 7f2966200640 1 -- 172.21.15.66:0/4095086773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f29600b0df0 con 0x7f296015bd40 2024-09-10T22:05:03.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.083+0000 7f2957400640 1 -- 172.21.15.66:0/4095086773 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f295805f2a0 con 0x7f296015bd40 2024-09-10T22:05:03.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.087+0000 7f2966200640 1 -- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f293c072e50 msgr2=0x7f293c075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:03.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.087+0000 7f2966200640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f293c072e50 0x7f293c075310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f29500af000 tx=0x7f295001afd0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.087+0000 7f2966200640 1 -- 172.21.15.66:0/4095086773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f296015bd40 msgr2=0x7f296014d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:03.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.087+0000 7f2966200640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f296015bd40 0x7f296014d3f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f295800ea50 tx=0x7f295800ef10 comp rx=0 tx=0).stop 2024-09-10T22:05:03.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.087+0000 7f2966200640 1 -- 172.21.15.66:0/4095086773 shutdown_connections 2024-09-10T22:05:03.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.087+0000 7f2966200640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f293c072e50 0x7f293c075310 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.087+0000 7f2966200640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f296015bd40 0x7f296014d3f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.087+0000 7f2966200640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29600cf830 0x7f296014ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.087+0000 7f2966200640 1 --2- 172.21.15.66:0/4095086773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29600cf180 0x7f2960159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.087+0000 7f2966200640 1 -- 172.21.15.66:0/4095086773 >> 172.21.15.66:0/4095086773 conn(0x7f29600d1c60 msgr2=0x7f29600b6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.087+0000 7f2966200640 1 -- 172.21.15.66:0/4095086773 shutdown_connections 2024-09-10T22:05:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.087+0000 7f2966200640 1 -- 172.21.15.66:0/4095086773 wait complete. 2024-09-10T22:05:03.111 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:05:03.112 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:05:03.204 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:05:03.311 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:05:03.311 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:05:03.311 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:05:03.320 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:05:03.320 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:05:03.321 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:05:03.330 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:05:03.330 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:05:03.330 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:05:03.369 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:05:03.369 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:05:03.370 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:05:03.380 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:05:03.381 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T22:05:03.381 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:05:03.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.555+0000 7efcfae00640 1 -- 172.21.15.66:0/1501387080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf415f100 msgr2=0x7efcf40d1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:03.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.555+0000 7efcfae00640 1 --2- 172.21.15.66:0/1501387080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf415f100 0x7efcf40d1960 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efce8009840 tx=0x7efce801c820 comp rx=0 tx=0).stop 2024-09-10T22:05:03.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.555+0000 7efcfae00640 1 -- 172.21.15.66:0/1501387080 shutdown_connections 2024-09-10T22:05:03.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.555+0000 7efcfae00640 1 --2- 172.21.15.66:0/1501387080 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efcf40b3110 0x7efcf40d1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.555+0000 7efcfae00640 1 --2- 172.21.15.66:0/1501387080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf415f100 0x7efcf40d1960 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.555+0000 7efcfae00640 1 -- 172.21.15.66:0/1501387080 >> 172.21.15.66:0/1501387080 conn(0x7efcf409aa50 msgr2=0x7efcf409ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:03.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.555+0000 7efcfae00640 1 -- 172.21.15.66:0/1501387080 shutdown_connections 2024-09-10T22:05:03.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.555+0000 7efcfae00640 1 -- 172.21.15.66:0/1501387080 wait complete. 2024-09-10T22:05:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfae00640 1 Processor -- start 2024-09-10T22:05:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfae00640 1 -- start start 2024-09-10T22:05:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfae00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efcf40b3110 0x7efcf40c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf40d0110 0x7efcf4157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf415f100 0x7efcf4159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4169290 con 0x7efcf40d0110 2024-09-10T22:05:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4169110 con 0x7efcf415f100 2024-09-10T22:05:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfae00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7efcf4169410 con 0x7efcf40b3110 2024-09-10T22:05:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf40d0110 0x7efcf4157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf40d0110 0x7efcf4157d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35050/0 (socket says 172.21.15.66:35050) 2024-09-10T22:05:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf9000640 1 -- 172.21.15.66:0/2123389527 learned_addr learned my addr 172.21.15.66:0/2123389527 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:03.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf9a00640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efcf40b3110 0x7efcf40c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:03.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfa400640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf415f100 0x7efcf4159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:03.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf9000640 1 -- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efcf40b3110 msgr2=0x7efcf40c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:03.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf9000640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efcf40b3110 0x7efcf40c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf9000640 1 -- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf415f100 msgr2=0x7efcf4159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:03.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf9000640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf415f100 0x7efcf4159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf9000640 1 -- 172.21.15.66:0/2123389527 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf40c7650 con 0x7efcf40d0110 2024-09-10T22:05:03.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfa400640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf415f100 0x7efcf4159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:03.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf9000640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf40d0110 0x7efcf4157d20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efce000c490 tx=0x7efce000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:03.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf1600640 1 -- 172.21.15.66:0/2123389527 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0016020 con 0x7efcf40d0110 2024-09-10T22:05:03.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf1600640 1 -- 172.21.15.66:0/2123389527 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efce0012940 con 0x7efcf40d0110 2024-09-10T22:05:03.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfae00640 1 -- 172.21.15.66:0/2123389527 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf40c9550 con 0x7efcf40d0110 2024-09-10T22:05:03.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcf1600640 1 -- 172.21.15.66:0/2123389527 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0012d90 con 0x7efcf40d0110 2024-09-10T22:05:03.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfae00640 1 -- 172.21.15.66:0/2123389527 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf40c9b10 con 0x7efcf40d0110 2024-09-10T22:05:03.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.559+0000 7efcfae00640 1 -- 172.21.15.66:0/2123389527 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf415f5b0 con 0x7efcf40d0110 2024-09-10T22:05:03.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.563+0000 7efcf1600640 1 -- 172.21.15.66:0/2123389527 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7efce001b400 con 0x7efcf40d0110 2024-09-10T22:05:03.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.563+0000 7efcf1600640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efcd4072e50 0x7efcd4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:03.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.563+0000 7efcf1600640 1 -- 172.21.15.66:0/2123389527 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7efce0094290 con 0x7efcf40d0110 2024-09-10T22:05:03.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.563+0000 7efcf9a00640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efcd4072e50 0x7efcd4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:03.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.563+0000 7efcf9a00640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efcd4072e50 0x7efcd4075310 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efce80af000 tx=0x7efce801b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:03.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.567+0000 7efcf1600640 1 -- 172.21.15.66:0/2123389527 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efce0020030 con 0x7efcf40d0110 2024-09-10T22:05:03.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.775+0000 7efcfae00640 1 -- 172.21.15.66:0/2123389527 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcf40d2970 con 0x7efcf40d0110 2024-09-10T22:05:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.775+0000 7efcf1600640 1 -- 172.21.15.66:0/2123389527 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3908 (secure 0 0 0) 0x7efce005ed10 con 0x7efcf40d0110 2024-09-10T22:05:03.781 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:05:03.781 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":29,"btime":"2024-09-10T22:04:57:332987+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5248,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2731842573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2731842573},{"type":"v1","addr":"172.21.15.94:6837","nonce":2731842573}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1324134239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1324134239},{"type":"v1","addr":"172.21.15.66:6833","nonce":1324134239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1694406832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1694406832},{"type":"v1","addr":"172.21.15.94:6836","nonce":1694406832}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5832,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/960511378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":960511378},{"type":"v1","addr":"172.21.15.66:6835","nonce":960511378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-09-10T22:05:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 29 2024-09-10T22:05:03.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.779+0000 7efcfae00640 1 -- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efcd4072e50 msgr2=0x7efcd4075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:03.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.779+0000 7efcfae00640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efcd4072e50 0x7efcd4075310 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efce80af000 tx=0x7efce801b060 comp rx=0 tx=0).stop 2024-09-10T22:05:03.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.779+0000 7efcfae00640 1 -- 172.21.15.66:0/2123389527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf40d0110 msgr2=0x7efcf4157d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.779+0000 7efcfae00640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf40d0110 0x7efcf4157d20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efce000c490 tx=0x7efce000c950 comp rx=0 tx=0).stop 2024-09-10T22:05:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.779+0000 7efcfae00640 1 -- 172.21.15.66:0/2123389527 shutdown_connections 2024-09-10T22:05:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.779+0000 7efcfae00640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efcd4072e50 0x7efcd4075310 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.779+0000 7efcfae00640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf415f100 0x7efcf4159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.779+0000 7efcfae00640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcf40d0110 0x7efcf4157d20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.779+0000 7efcfae00640 1 --2- 172.21.15.66:0/2123389527 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efcf40b3110 0x7efcf40c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.779+0000 7efcfae00640 1 -- 172.21.15.66:0/2123389527 >> 172.21.15.66:0/2123389527 conn(0x7efcf409aa50 msgr2=0x7efcf415ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.779+0000 7efcfae00640 1 -- 172.21.15.66:0/2123389527 shutdown_connections 2024-09-10T22:05:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:03.779+0000 7efcfae00640 1 -- 172.21.15.66:0/2123389527 wait complete. 2024-09-10T22:05:03.804 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:05:03.804 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:05:03.804 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:05:03.804 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:05:03.804 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:05:03.804 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:05:03.804 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:05:03.805 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:05:03.805 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:05:03.805 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:05:03.805 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:05:03.805 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:05:03.805 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:05:03.805 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:05:03.806 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:05:03.806 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:05:03.819 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:05:03.820 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:05:03.820 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:05:03.820 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:05:03.821 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:05:03.822 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:05:03.823 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:05:03.823 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:05:03.864 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:05:03.864 DEBUG:teuthology.parallel:result is None 2024-09-10T22:05:03.865 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:05:03.866 DEBUG:teuthology.parallel:result is None 2024-09-10T22:05:03.867 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:05:03.867 DEBUG:teuthology.parallel:result is None 2024-09-10T22:05:03.869 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:05:03.869 DEBUG:teuthology.parallel:result is None 2024-09-10T22:05:03.869 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T22:05:03.950 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:05:03.951 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:05:03.958 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:05:03.958 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:05:04.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 -- 172.21.15.66:0/286326581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f80d49b0 msgr2=0x7f00f8160310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 --2- 172.21.15.66:0/286326581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f80d49b0 0x7f00f8160310 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f00f80d02f0 tx=0x7f00e401c8c0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 -- 172.21.15.66:0/286326581 shutdown_connections 2024-09-10T22:05:04.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 --2- 172.21.15.66:0/286326581 >> v2:172.21.15.94:3300/0 conn(0x7f00f80b2e60 0x7f00f8160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 --2- 172.21.15.66:0/286326581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f80d49b0 0x7f00f8160310 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 --2- 172.21.15.66:0/286326581 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00f81611d0 0x7f00f81615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 -- 172.21.15.66:0/286326581 >> 172.21.15.66:0/286326581 conn(0x7f00f80d3e90 msgr2=0x7f00f80d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:04.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 -- 172.21.15.66:0/286326581 shutdown_connections 2024-09-10T22:05:04.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 -- 172.21.15.66:0/286326581 wait complete. 2024-09-10T22:05:04.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 Processor -- start 2024-09-10T22:05:04.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 -- start start 2024-09-10T22:05:04.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f80b2e60 0x7f00f81562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f80d49b0 0x7f00f8156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00f81611d0 0x7f00f8157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f00f80d2bd0 con 0x7f00f80b2e60 2024-09-10T22:05:04.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00f80d2a50 con 0x7f00f80d49b0 2024-09-10T22:05:04.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00ff400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f00f80ceae0 con 0x7f00f81611d0 2024-09-10T22:05:04.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00fca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f80b2e60 0x7f00f81562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00fca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f80b2e60 0x7f00f81562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35094/0 (socket says 172.21.15.66:35094) 2024-09-10T22:05:04.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.019+0000 7f00fca00640 1 -- 172.21.15.66:0/2100876028 learned_addr learned my addr 172.21.15.66:0/2100876028 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:04.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00fca00640 1 -- 172.21.15.66:0/2100876028 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00f81611d0 msgr2=0x7f00f8157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00fca00640 1 --2- 172.21.15.66:0/2100876028 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00f81611d0 0x7f00f8157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00fca00640 1 -- 172.21.15.66:0/2100876028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f80d49b0 msgr2=0x7f00f8156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00fca00640 1 --2- 172.21.15.66:0/2100876028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f80d49b0 0x7f00f8156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00fca00640 1 -- 172.21.15.66:0/2100876028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f81573e0 con 0x7f00f80b2e60 2024-09-10T22:05:04.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00fca00640 1 --2- 172.21.15.66:0/2100876028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f80b2e60 0x7f00f81562d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00e7c0 tx=0x7f00ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:04.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00f4c00640 1 -- 172.21.15.66:0/2100876028 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec00ccc0 con 0x7f00f80b2e60 2024-09-10T22:05:04.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00f4c00640 1 -- 172.21.15.66:0/2100876028 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00ec00ce80 con 0x7f00f80b2e60 2024-09-10T22:05:04.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00f4c00640 1 -- 172.21.15.66:0/2100876028 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec005280 con 0x7f00f80b2e60 2024-09-10T22:05:04.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00ff400640 1 -- 172.21.15.66:0/2100876028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f81fae10 con 0x7f00f80b2e60 2024-09-10T22:05:04.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00ff400640 1 -- 172.21.15.66:0/2100876028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f81fb3d0 con 0x7f00f80b2e60 2024-09-10T22:05:04.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00ff400640 1 -- 172.21.15.66:0/2100876028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f81615b0 con 0x7f00f80b2e60 2024-09-10T22:05:04.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00f4c00640 1 -- 172.21.15.66:0/2100876028 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f00ec005420 con 0x7f00f80b2e60 2024-09-10T22:05:04.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.023+0000 7f00f4c00640 1 --2- 172.21.15.66:0/2100876028 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f00d8072e50 0x7f00d8075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.027+0000 7f00f4c00640 1 -- 172.21.15.66:0/2100876028 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f00ec092420 con 0x7f00f80b2e60 2024-09-10T22:05:04.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.027+0000 7f00f7e00640 1 --2- 172.21.15.66:0/2100876028 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f00d8072e50 0x7f00d8075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.027+0000 7f00f7e00640 1 --2- 172.21.15.66:0/2100876028 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f00d8072e50 0x7f00d8075310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f00e40af000 tx=0x7f00e4019660 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:04.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.031+0000 7f00f4c00640 1 -- 172.21.15.66:0/2100876028 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f00ec05f5b0 con 0x7f00f80b2e60 2024-09-10T22:05:04.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.215+0000 7f00ff400640 1 -- 172.21.15.66:0/2100876028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f00f8158e10 con 0x7f00f80b2e60 2024-09-10T22:05:04.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00f4c00640 1 -- 172.21.15.66:0/2100876028 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v53) ==== 97+0+0 (secure 0 0 0) 0x7f00ec05eeb0 con 0x7f00f80b2e60 2024-09-10T22:05:04.374 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-09-10T22:05:04.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00ff400640 1 -- 172.21.15.66:0/2100876028 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f00d8072e50 msgr2=0x7f00d8075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00ff400640 1 --2- 172.21.15.66:0/2100876028 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f00d8072e50 0x7f00d8075310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f00e40af000 tx=0x7f00e4019660 comp rx=0 tx=0).stop 2024-09-10T22:05:04.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00ff400640 1 -- 172.21.15.66:0/2100876028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f80b2e60 msgr2=0x7f00f81562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00ff400640 1 --2- 172.21.15.66:0/2100876028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f80b2e60 0x7f00f81562d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00e7c0 tx=0x7f00ec00ec80 comp rx=0 tx=0).stop 2024-09-10T22:05:04.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00ff400640 1 -- 172.21.15.66:0/2100876028 shutdown_connections 2024-09-10T22:05:04.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00ff400640 1 --2- 172.21.15.66:0/2100876028 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f00d8072e50 0x7f00d8075310 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00ff400640 1 --2- 172.21.15.66:0/2100876028 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00f81611d0 0x7f00f8157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00ff400640 1 --2- 172.21.15.66:0/2100876028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f80d49b0 0x7f00f8156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00ff400640 1 --2- 172.21.15.66:0/2100876028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f80b2e60 0x7f00f81562d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00ff400640 1 -- 172.21.15.66:0/2100876028 >> 172.21.15.66:0/2100876028 conn(0x7f00f80d3e90 msgr2=0x7f00f80b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:04.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00ff400640 1 -- 172.21.15.66:0/2100876028 shutdown_connections 2024-09-10T22:05:04.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.371+0000 7f00ff400640 1 -- 172.21.15.66:0/2100876028 wait complete. 2024-09-10T22:05:04.398 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T22:05:04.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.539+0000 7fa0e4000640 1 -- 172.21.15.66:0/3458649249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc0afc20 msgr2=0x7fa0dc0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.539+0000 7fa0e4000640 1 --2- 172.21.15.66:0/3458649249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc0afc20 0x7fa0dc0ce6b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009c80 tx=0x7fa0d401cd10 comp rx=0 tx=0).stop 2024-09-10T22:05:04.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.539+0000 7fa0e4000640 1 -- 172.21.15.66:0/3458649249 shutdown_connections 2024-09-10T22:05:04.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.539+0000 7fa0e4000640 1 --2- 172.21.15.66:0/3458649249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc0afc20 0x7fa0dc0ce6b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.539+0000 7fa0e4000640 1 --2- 172.21.15.66:0/3458649249 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0dc0d06b0 0x7fa0dc0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.539+0000 7fa0e4000640 1 -- 172.21.15.66:0/3458649249 >> 172.21.15.66:0/3458649249 conn(0x7fa0dc0d1c60 msgr2=0x7fa0dc0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:04.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.539+0000 7fa0e4000640 1 -- 172.21.15.66:0/3458649249 shutdown_connections 2024-09-10T22:05:04.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.539+0000 7fa0e4000640 1 -- 172.21.15.66:0/3458649249 wait complete. 2024-09-10T22:05:04.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.539+0000 7fa0e4000640 1 Processor -- start 2024-09-10T22:05:04.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e4000640 1 -- start start 2024-09-10T22:05:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e4000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc0afc20 0x7fa0dc14ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e4000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0dc0d06b0 0x7fa0dc1fbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e4000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0dc15bd40 0x7fa0dc15ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e4000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc169250 con 0x7fa0dc0afc20 2024-09-10T22:05:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e4000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1690d0 con 0x7fa0dc0d06b0 2024-09-10T22:05:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e4000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa0dc1693d0 con 0x7fa0dc15bd40 2024-09-10T22:05:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc0afc20 0x7fa0dc14ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc0afc20 0x7fa0dc14ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35104/0 (socket says 172.21.15.66:35104) 2024-09-10T22:05:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e1600640 1 -- 172.21.15.66:0/3519820315 learned_addr learned my addr 172.21.15.66:0/3519820315 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e2000640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0dc15bd40 0x7fa0dc15ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e0c00640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0dc0d06b0 0x7fa0dc1fbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e1600640 1 -- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0dc15bd40 msgr2=0x7fa0dc15ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e1600640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0dc15bd40 0x7fa0dc15ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e1600640 1 -- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0dc0d06b0 msgr2=0x7fa0dc1fbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e1600640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0dc0d06b0 0x7fa0dc1fbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e1600640 1 -- 172.21.15.66:0/3519820315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0dc14f640 con 0x7fa0dc0afc20 2024-09-10T22:05:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e1600640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc0afc20 0x7fa0dc14ec70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000c3c0 tx=0x7fa0d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0d9600640 1 -- 172.21.15.66:0/3519820315 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0017070 con 0x7fa0dc0afc20 2024-09-10T22:05:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0d9600640 1 -- 172.21.15.66:0/3519820315 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0d00053c0 con 0x7fa0dc0afc20 2024-09-10T22:05:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0d9600640 1 -- 172.21.15.66:0/3519820315 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0003860 con 0x7fa0dc0afc20 2024-09-10T22:05:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e4000640 1 -- 172.21.15.66:0/3519820315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0dc1fc360 con 0x7fa0dc0afc20 2024-09-10T22:05:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e4000640 1 -- 172.21.15.66:0/3519820315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0dc1f8270 con 0x7fa0dc0afc20 2024-09-10T22:05:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.543+0000 7fa0e4000640 1 -- 172.21.15.66:0/3519820315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0dc0cffa0 con 0x7fa0dc0afc20 2024-09-10T22:05:04.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.547+0000 7fa0d9600640 1 -- 172.21.15.66:0/3519820315 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fa0d0003a00 con 0x7fa0dc0afc20 2024-09-10T22:05:04.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.547+0000 7fa0d9600640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0bc072e80 0x7fa0bc075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.547+0000 7fa0d9600640 1 -- 172.21.15.66:0/3519820315 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7fa0d00937e0 con 0x7fa0dc0afc20 2024-09-10T22:05:04.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.547+0000 7fa0e0c00640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0bc072e80 0x7fa0bc075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.547+0000 7fa0e0c00640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0bc072e80 0x7fa0bc075340 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc155390 tx=0x7fa0c800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:04.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.551+0000 7fa0d9600640 1 -- 172.21.15.66:0/3519820315 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa0dc14f640 con 0x7fa0dc0afc20 2024-09-10T22:05:04.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.783+0000 7fa0e4000640 1 -- 172.21.15.66:0/3519820315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa0dc1594c0 con 0x7fa0dc0afc20 2024-09-10T22:05:04.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.783+0000 7fa0d9600640 1 -- 172.21.15.66:0/3519820315 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) ==== 80+0+5304 (secure 0 0 0) 0x7fa0d005db90 con 0x7fa0dc0afc20 2024-09-10T22:05:04.789 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:05:04.789 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:05:04.789 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-09-10T22:05:04.789 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.789 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-09-10T22:05:04.789 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmxQD7CBAAttZP0KtdjTI7QWbD7CVinQ==", 2024-09-10T22:05:04.789 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.789 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:05:04.789 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmI7ufFRAANelhvqylKyfGPVfBuY40kg==", 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.790 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQnVoEBAAvsZ/TSOOP/JgJBr/aHwI/g==", 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-09-10T22:05:04.791 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmRQkTHRAAHPIdwqtkkyh31D7IO2vxPA==", 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBCweBmDJkxJBAAVSQNiMAQl5bR3RUwVOuGYw==", 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:05:04.792 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBFweBmwcdsOxAAPPQyzktcfiF8pJNP8uAjwg==", 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-09-10T22:05:04.793 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBJweBmXQhZExAAnEf8mjGX6nnYwWygKjo/2A==", 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBNweBmlauHAxAA/k9xRqbHZsqJq9OYbJKfNQ==", 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:05:04.794 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBRweBmHnFDJRAAgF1ZAPvmoNzeUftZQyL0eg==", 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-09-10T22:05:04.795 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBUweBm/hXpORAA6jpOI/I+QyB2mZsEoAWF0w==", 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBYweBmYu2MExAAPD9WcxrBLzwwJQm19z7YcQ==", 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:05:04.796 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbweBmXfJ9KRAA/AbWlli+Qfg7gy0SblJ7fg==", 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.797 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw==", 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmbS6IKhAAaEmK9irxMipALgkAct/+OQ==", 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:05:04.798 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQ47KJhAAI50ZaSQIhH6FxRM48JZICA==", 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.799 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmUcLCMBAAdf4tfcpVaZoaKjS1UzrxhA==", 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA+weBmPIAzKBAAkyVV875PNhBmEoKsrAkg8w==", 2024-09-10T22:05:04.800 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmLiy1BBAADnraML8IUet7Sw0j83TUVQ==", 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.801 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.802 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:04.802 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:04.802 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-09-10T22:05:04.802 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmItWpABAAlJ+6dFjvtBV3g+jYmUh0tQ==", 2024-09-10T22:05:04.802 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:05:04.802 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:05:04.802 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:05:04.802 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:05:04.802 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.802 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:04.802 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T22:05:04.802 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:05:04.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.787+0000 7fa0e4000640 1 -- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0bc072e80 msgr2=0x7fa0bc075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.787+0000 7fa0e4000640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0bc072e80 0x7fa0bc075340 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc155390 tx=0x7fa0c800b390 comp rx=0 tx=0).stop 2024-09-10T22:05:04.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.787+0000 7fa0e4000640 1 -- 172.21.15.66:0/3519820315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc0afc20 msgr2=0x7fa0dc14ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.787+0000 7fa0e4000640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc0afc20 0x7fa0dc14ec70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000c3c0 tx=0x7fa0d000c880 comp rx=0 tx=0).stop 2024-09-10T22:05:04.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.787+0000 7fa0e4000640 1 -- 172.21.15.66:0/3519820315 shutdown_connections 2024-09-10T22:05:04.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.787+0000 7fa0e4000640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0bc072e80 0x7fa0bc075340 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.787+0000 7fa0e4000640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0dc15bd40 0x7fa0dc15ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.787+0000 7fa0e4000640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0dc0d06b0 0x7fa0dc1fbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.787+0000 7fa0e4000640 1 --2- 172.21.15.66:0/3519820315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc0afc20 0x7fa0dc14ec70 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.787+0000 7fa0e4000640 1 -- 172.21.15.66:0/3519820315 >> 172.21.15.66:0/3519820315 conn(0x7fa0dc0d1c60 msgr2=0x7fa0dc155980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:04.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.787+0000 7fa0e4000640 1 -- 172.21.15.66:0/3519820315 shutdown_connections 2024-09-10T22:05:04.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.787+0000 7fa0e4000640 1 -- 172.21.15.66:0/3519820315 wait complete. 2024-09-10T22:05:04.811 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:05:04.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 -- 172.21.15.66:0/2152701982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1540b15d0 msgr2=0x7fb1540cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 --2- 172.21.15.66:0/2152701982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1540b15d0 0x7fb1540cd7a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009c80 tx=0x7fb14c01cd60 comp rx=0 tx=0).stop 2024-09-10T22:05:04.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 -- 172.21.15.66:0/2152701982 shutdown_connections 2024-09-10T22:05:04.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 --2- 172.21.15.66:0/2152701982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1540b15d0 0x7fb1540cd7a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 --2- 172.21.15.66:0/2152701982 >> v2:172.21.15.94:3300/0 conn(0x7fb15415f300 0x7fb1540b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 -- 172.21.15.66:0/2152701982 >> 172.21.15.66:0/2152701982 conn(0x7fb1540d1b10 msgr2=0x7fb1540d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:04.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 -- 172.21.15.66:0/2152701982 shutdown_connections 2024-09-10T22:05:04.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 -- 172.21.15.66:0/2152701982 wait complete. 2024-09-10T22:05:04.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 Processor -- start 2024-09-10T22:05:04.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 -- start start 2024-09-10T22:05:04.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1540b15d0 0x7fb1541fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb15415ef20 0x7fb15414dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb15415f300 0x7fb1540afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb154169310 con 0x7fb15415f300 2024-09-10T22:05:04.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb154169190 con 0x7fb15415ef20 2024-09-10T22:05:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb153200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb154169490 con 0x7fb1540b15d0 2024-09-10T22:05:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb152800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb15415f300 0x7fb1540afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb151e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1540b15d0 0x7fb1541fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb149400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb15415ef20 0x7fb15414dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb151e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1540b15d0 0x7fb1541fbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:38690/0 (socket says 172.21.15.66:38690) 2024-09-10T22:05:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb151e00640 1 -- 172.21.15.66:0/2979285839 learned_addr learned my addr 172.21.15.66:0/2979285839 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:04.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.955+0000 7fb152800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb15415f300 0x7fb1540afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35136/0 (socket says 172.21.15.66:35136) 2024-09-10T22:05:04.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb152800640 1 -- 172.21.15.66:0/2979285839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1540b15d0 msgr2=0x7fb1541fbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb152800640 1 --2- 172.21.15.66:0/2979285839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1540b15d0 0x7fb1541fbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb152800640 1 -- 172.21.15.66:0/2979285839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb15415ef20 msgr2=0x7fb15414dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb152800640 1 --2- 172.21.15.66:0/2979285839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb15415ef20 0x7fb15414dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb152800640 1 -- 172.21.15.66:0/2979285839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1540b0b10 con 0x7fb15415f300 2024-09-10T22:05:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb152800640 1 --2- 172.21.15.66:0/2979285839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb15415f300 0x7fb1540afd70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0ac000 tx=0x7fb14c0a5a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb14aa00640 1 -- 172.21.15.66:0/2979285839 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0bb070 con 0x7fb15415f300 2024-09-10T22:05:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb14aa00640 1 -- 172.21.15.66:0/2979285839 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb14c004e00 con 0x7fb15415f300 2024-09-10T22:05:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb14aa00640 1 -- 172.21.15.66:0/2979285839 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0b2040 con 0x7fb15415f300 2024-09-10T22:05:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb153200640 1 -- 172.21.15.66:0/2979285839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1540b0830 con 0x7fb15415f300 2024-09-10T22:05:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb153200640 1 -- 172.21.15.66:0/2979285839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1541fb660 con 0x7fb15415f300 2024-09-10T22:05:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb153200640 1 -- 172.21.15.66:0/2979285839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1540cd7a0 con 0x7fb15415f300 2024-09-10T22:05:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb14aa00640 1 -- 172.21.15.66:0/2979285839 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fb14c0b5030 con 0x7fb15415f300 2024-09-10T22:05:04.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb14aa00640 1 --2- 172.21.15.66:0/2979285839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb130072e50 0x7fb130075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.959+0000 7fb14aa00640 1 -- 172.21.15.66:0/2979285839 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7fb14c12d9f0 con 0x7fb15415f300 2024-09-10T22:05:04.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.963+0000 7fb151e00640 1 --2- 172.21.15.66:0/2979285839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb130072e50 0x7fb130075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.963+0000 7fb151e00640 1 --2- 172.21.15.66:0/2979285839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb130072e50 0x7fb130075310 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb1440050b0 tx=0x7fb144003700 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:04.963+0000 7fb14aa00640 1 -- 172.21.15.66:0/2979285839 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb14c016950 con 0x7fb15415f300 2024-09-10T22:05:05.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.139+0000 7fb153200640 1 -- 172.21.15.66:0/2979285839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb1540717d0 con 0x7fb15415f300 2024-09-10T22:05:05.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.139+0000 7fb14aa00640 1 -- 172.21.15.66:0/2979285839 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) ==== 84+0+65 (secure 0 0 0) 0x7fb14c0f81b0 con 0x7fb15415f300 2024-09-10T22:05:05.145 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:05:05.145 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:05:05.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:05.146 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:05:05.146 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:05:05.146 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:05.146 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:05:05.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.143+0000 7fb153200640 1 -- 172.21.15.66:0/2979285839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb130072e50 msgr2=0x7fb130075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.143+0000 7fb153200640 1 --2- 172.21.15.66:0/2979285839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb130072e50 0x7fb130075310 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb1440050b0 tx=0x7fb144003700 comp rx=0 tx=0).stop 2024-09-10T22:05:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.143+0000 7fb153200640 1 -- 172.21.15.66:0/2979285839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb15415f300 msgr2=0x7fb1540afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.143+0000 7fb153200640 1 --2- 172.21.15.66:0/2979285839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb15415f300 0x7fb1540afd70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0ac000 tx=0x7fb14c0a5a80 comp rx=0 tx=0).stop 2024-09-10T22:05:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.143+0000 7fb153200640 1 -- 172.21.15.66:0/2979285839 shutdown_connections 2024-09-10T22:05:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.143+0000 7fb153200640 1 --2- 172.21.15.66:0/2979285839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb130072e50 0x7fb130075310 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.143+0000 7fb153200640 1 --2- 172.21.15.66:0/2979285839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb15415f300 0x7fb1540afd70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.143+0000 7fb153200640 1 --2- 172.21.15.66:0/2979285839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb15415ef20 0x7fb15414dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.143+0000 7fb153200640 1 --2- 172.21.15.66:0/2979285839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1540b15d0 0x7fb1541fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.143+0000 7fb153200640 1 -- 172.21.15.66:0/2979285839 >> 172.21.15.66:0/2979285839 conn(0x7fb1540d1b10 msgr2=0x7fb154071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.143+0000 7fb153200640 1 -- 172.21.15.66:0/2979285839 shutdown_connections 2024-09-10T22:05:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.143+0000 7fb153200640 1 -- 172.21.15.66:0/2979285839 wait complete. 2024-09-10T22:05:05.169 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:05:05.169 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:05:05.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 -- 172.21.15.66:0/1914063934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1d4004d50 msgr2=0x7fa1d40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:05.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 --2- 172.21.15.66:0/1914063934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1d4004d50 0x7fa1d40051d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009ee0 tx=0x7fa1e801d5f0 comp rx=0 tx=0).stop 2024-09-10T22:05:05.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f4a00640 1 -- 172.21.15.66:0/1914063934 reap_dead start 2024-09-10T22:05:05.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 -- 172.21.15.66:0/1914063934 shutdown_connections 2024-09-10T22:05:05.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 -- 172.21.15.66:0/1914063934 >> 172.21.15.66:0/1914063934 conn(0x7fa1f00d1c60 msgr2=0x7fa1f00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:05.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 -- 172.21.15.66:0/1914063934 shutdown_connections 2024-09-10T22:05:05.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 -- 172.21.15.66:0/1914063934 wait complete. 2024-09-10T22:05:05.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 Processor -- start 2024-09-10T22:05:05.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 -- start start 2024-09-10T22:05:05.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1d4004d50 0x7fa1f01f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:05.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1f015a230 0x7fa1f01f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:05.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1f015cf50 0x7fa1f015eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:05.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00d0300 con 0x7fa1d4004d50 2024-09-10T22:05:05.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00d0180 con 0x7fa1f015cf50 2024-09-10T22:05:05.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f6a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f00d0480 con 0x7fa1f015a230 2024-09-10T22:05:05.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1efe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1d4004d50 0x7fa1f01f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:05.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1efe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1d4004d50 0x7fa1f01f85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35158/0 (socket says 172.21.15.66:35158) 2024-09-10T22:05:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1efe00640 1 -- 172.21.15.66:0/2911859062 learned_addr learned my addr 172.21.15.66:0/2911859062 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1e7e00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1f015a230 0x7fa1f01f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1f4a00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1f015cf50 0x7fa1f015eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1efe00640 1 -- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1f015a230 msgr2=0x7fa1f01f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1efe00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1f015a230 0x7fa1f01f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1efe00640 1 -- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1f015cf50 msgr2=0x7fa1f015eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1efe00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1f015cf50 0x7fa1f015eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.319+0000 7fa1efe00640 1 -- 172.21.15.66:0/2911859062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00c83d0 con 0x7fa1d4004d50 2024-09-10T22:05:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.323+0000 7fa1efe00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1d4004d50 0x7fa1f01f85b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000fa00 tx=0x7fa1e000fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.323+0000 7fa1ecc00640 1 -- 172.21.15.66:0/2911859062 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000bcb0 con 0x7fa1d4004d50 2024-09-10T22:05:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.323+0000 7fa1ecc00640 1 -- 172.21.15.66:0/2911859062 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1e0011440 con 0x7fa1d4004d50 2024-09-10T22:05:05.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.323+0000 7fa1f6a00640 1 -- 172.21.15.66:0/2911859062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f01f4750 con 0x7fa1d4004d50 2024-09-10T22:05:05.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.323+0000 7fa1ecc00640 1 -- 172.21.15.66:0/2911859062 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0011810 con 0x7fa1d4004d50 2024-09-10T22:05:05.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.323+0000 7fa1f6a00640 1 -- 172.21.15.66:0/2911859062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f015a6b0 con 0x7fa1d4004d50 2024-09-10T22:05:05.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.323+0000 7fa1f6a00640 1 -- 172.21.15.66:0/2911859062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f00cecb0 con 0x7fa1d4004d50 2024-09-10T22:05:05.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.323+0000 7fa1ecc00640 1 -- 172.21.15.66:0/2911859062 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fa1e0023030 con 0x7fa1d4004d50 2024-09-10T22:05:05.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.323+0000 7fa1ecc00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa1cc072e50 0x7fa1cc075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:05.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.323+0000 7fa1ecc00640 1 -- 172.21.15.66:0/2911859062 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7fa1e0094c60 con 0x7fa1d4004d50 2024-09-10T22:05:05.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.323+0000 7fa1e7e00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa1cc072e50 0x7fa1cc075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:05.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.327+0000 7fa1e7e00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa1cc072e50 0x7fa1cc075310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8001230 tx=0x7fa1d8006730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:05.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.327+0000 7fa1ecc00640 1 -- 172.21.15.66:0/2911859062 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa1e005f010 con 0x7fa1d4004d50 2024-09-10T22:05:05.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:05.507+0000 7fa1f6a00640 1 -- 172.21.15.66:0/2911859062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa1f0070550 con 0x7fa1d4004d50 2024-09-10T22:05:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.383+0000 7fa1ecc00640 1 -- 172.21.15.66:0/2911859062 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) ==== 138+0+0 (secure 0 0 0) 0x7fa1e0063a40 con 0x7fa1d4004d50 2024-09-10T22:05:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:05:06.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.387+0000 7fa1f6a00640 1 -- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa1cc072e50 msgr2=0x7fa1cc075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:06.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.387+0000 7fa1f6a00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa1cc072e50 0x7fa1cc075310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8001230 tx=0x7fa1d8006730 comp rx=0 tx=0).stop 2024-09-10T22:05:06.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.387+0000 7fa1f6a00640 1 -- 172.21.15.66:0/2911859062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1d4004d50 msgr2=0x7fa1f01f85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:06.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.387+0000 7fa1f6a00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1d4004d50 0x7fa1f01f85b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000fa00 tx=0x7fa1e000fec0 comp rx=0 tx=0).stop 2024-09-10T22:05:06.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.387+0000 7fa1f6a00640 1 -- 172.21.15.66:0/2911859062 shutdown_connections 2024-09-10T22:05:06.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.387+0000 7fa1f6a00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa1cc072e50 0x7fa1cc075310 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.387+0000 7fa1f6a00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1f015cf50 0x7fa1f015eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.387+0000 7fa1f6a00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1f015a230 0x7fa1f01f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.387+0000 7fa1f6a00640 1 --2- 172.21.15.66:0/2911859062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1d4004d50 0x7fa1f01f85b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.387+0000 7fa1f6a00640 1 -- 172.21.15.66:0/2911859062 >> 172.21.15.66:0/2911859062 conn(0x7fa1f00d1c60 msgr2=0x7fa1f00aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.387+0000 7fa1f6a00640 1 -- 172.21.15.66:0/2911859062 shutdown_connections 2024-09-10T22:05:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.387+0000 7fa1f6a00640 1 -- 172.21.15.66:0/2911859062 wait complete. 2024-09-10T22:05:06.421 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:05:06.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 -- 172.21.15.66:0/899511906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c070b00 msgr2=0x7fe29c0ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:06.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 --2- 172.21.15.66:0/899511906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c070b00 0x7fe29c0ca4d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe284009900 tx=0x7fe28401c820 comp rx=0 tx=0).stop 2024-09-10T22:05:06.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 -- 172.21.15.66:0/899511906 shutdown_connections 2024-09-10T22:05:06.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 --2- 172.21.15.66:0/899511906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c070b00 0x7fe29c0ca4d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:06.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 -- 172.21.15.66:0/899511906 >> 172.21.15.66:0/899511906 conn(0x7fe29c0cc900 msgr2=0x7fe29c0ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:06.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 -- 172.21.15.66:0/899511906 shutdown_connections 2024-09-10T22:05:06.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 -- 172.21.15.66:0/899511906 wait complete. 2024-09-10T22:05:06.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 Processor -- start 2024-09-10T22:05:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 -- start start 2024-09-10T22:05:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c070b00 0x7fe29c148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c0738d0 0x7fe29c09b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe29c0caa10 0x7fe29c09b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c169070 con 0x7fe29c070b00 2024-09-10T22:05:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c168ef0 con 0x7fe29c0caa10 2024-09-10T22:05:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe2a2800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe29c1691f0 con 0x7fe29c0738d0 2024-09-10T22:05:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe29b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c070b00 0x7fe29c148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe29b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c070b00 0x7fe29c148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35176/0 (socket says 172.21.15.66:35176) 2024-09-10T22:05:06.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe29b400640 1 -- 172.21.15.66:0/2185636940 learned_addr learned my addr 172.21.15.66:0/2185636940 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe293e00640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c0738d0 0x7fe29c09b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe29b400640 1 -- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c0738d0 msgr2=0x7fe29c09b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe29b400640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c0738d0 0x7fe29c09b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe29b400640 1 -- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe29c0caa10 msgr2=0x7fe29c09b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe29b400640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe29c0caa10 0x7fe29c09b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe29b400640 1 -- 172.21.15.66:0/2185636940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c165e60 con 0x7fe29c070b00 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.575+0000 7fe293e00640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c0738d0 0x7fe29c09b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.579+0000 7fe29b400640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c070b00 0x7fe29c148200 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00c530 tx=0x7fe28c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.579+0000 7fe293400640 1 -- 172.21.15.66:0/2185636940 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c017070 con 0x7fe29c070b00 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.579+0000 7fe293400640 1 -- 172.21.15.66:0/2185636940 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe28c0053c0 con 0x7fe29c070b00 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.579+0000 7fe293400640 1 -- 172.21.15.66:0/2185636940 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c012890 con 0x7fe29c070b00 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.579+0000 7fe2a2800640 1 -- 172.21.15.66:0/2185636940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29c09d140 con 0x7fe29c070b00 2024-09-10T22:05:06.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.579+0000 7fe2a2800640 1 -- 172.21.15.66:0/2185636940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29c09d3c0 con 0x7fe29c070b00 2024-09-10T22:05:06.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.579+0000 7fe2a2800640 1 -- 172.21.15.66:0/2185636940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29c0ca4d0 con 0x7fe29c070b00 2024-09-10T22:05:06.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.579+0000 7fe293400640 1 -- 172.21.15.66:0/2185636940 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fe28c003600 con 0x7fe29c070b00 2024-09-10T22:05:06.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.579+0000 7fe293400640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe278072e50 0x7fe278075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:06.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.579+0000 7fe293400640 1 -- 172.21.15.66:0/2185636940 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5227+0+0 (secure 0 0 0) 0x7fe28c093d10 con 0x7fe29c070b00 2024-09-10T22:05:06.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.583+0000 7fe293e00640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe278072e50 0x7fe278075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:06.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.583+0000 7fe293e00640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe278072e50 0x7fe278075310 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe2840af000 tx=0x7fe28401b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:06.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.583+0000 7fe293400640 1 -- 172.21.15.66:0/2185636940 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe28c021020 con 0x7fe29c070b00 2024-09-10T22:05:06.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:06.759+0000 7fe2a2800640 1 -- 172.21.15.66:0/2185636940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe29c0cc150 con 0x7fe29c070b00 2024-09-10T22:05:07.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.387+0000 7fe293400640 1 -- 172.21.15.66:0/2185636940 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) ==== 170+0+0 (secure 0 0 0) 0x7fe29c09d140 con 0x7fe29c070b00 2024-09-10T22:05:07.392 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:05:07.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.391+0000 7fe2a2800640 1 -- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe278072e50 msgr2=0x7fe278075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:07.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.391+0000 7fe2a2800640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe278072e50 0x7fe278075310 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe2840af000 tx=0x7fe28401b060 comp rx=0 tx=0).stop 2024-09-10T22:05:07.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.391+0000 7fe2a2800640 1 -- 172.21.15.66:0/2185636940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c070b00 msgr2=0x7fe29c148200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:07.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.391+0000 7fe2a2800640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c070b00 0x7fe29c148200 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00c530 tx=0x7fe28c00c9f0 comp rx=0 tx=0).stop 2024-09-10T22:05:07.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.391+0000 7fe2a2800640 1 -- 172.21.15.66:0/2185636940 shutdown_connections 2024-09-10T22:05:07.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.391+0000 7fe2a2800640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe278072e50 0x7fe278075310 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:07.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.391+0000 7fe2a2800640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe29c0caa10 0x7fe29c09b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:07.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.391+0000 7fe2a2800640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c0738d0 0x7fe29c09b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:07.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.391+0000 7fe2a2800640 1 --2- 172.21.15.66:0/2185636940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c070b00 0x7fe29c148200 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:07.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.391+0000 7fe2a2800640 1 -- 172.21.15.66:0/2185636940 >> 172.21.15.66:0/2185636940 conn(0x7fe29c0cc900 msgr2=0x7fe29c0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:07.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.391+0000 7fe2a2800640 1 -- 172.21.15.66:0/2185636940 shutdown_connections 2024-09-10T22:05:07.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.391+0000 7fe2a2800640 1 -- 172.21.15.66:0/2185636940 wait complete. 2024-09-10T22:05:07.434 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:05:07.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.579+0000 7fe5df800640 1 -- 172.21.15.66:0/1006188165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d80afc20 msgr2=0x7fe5d80ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:07.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.579+0000 7fe5df800640 1 --2- 172.21.15.66:0/1006188165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d80afc20 0x7fe5d80ce6b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80747e0 tx=0x7fe5d001ccf0 comp rx=0 tx=0).stop 2024-09-10T22:05:07.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.579+0000 7fe5df800640 1 -- 172.21.15.66:0/1006188165 shutdown_connections 2024-09-10T22:05:07.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.579+0000 7fe5df800640 1 --2- 172.21.15.66:0/1006188165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d80afc20 0x7fe5d80ce6b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:07.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.579+0000 7fe5df800640 1 --2- 172.21.15.66:0/1006188165 >> v2:172.21.15.94:3300/0 conn(0x7fe5d80d06b0 0x7fe5d80ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:07.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.579+0000 7fe5df800640 1 -- 172.21.15.66:0/1006188165 >> 172.21.15.66:0/1006188165 conn(0x7fe5d80d1c60 msgr2=0x7fe5d80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:07.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.579+0000 7fe5df800640 1 -- 172.21.15.66:0/1006188165 shutdown_connections 2024-09-10T22:05:07.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.579+0000 7fe5df800640 1 -- 172.21.15.66:0/1006188165 wait complete. 2024-09-10T22:05:07.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5df800640 1 Processor -- start 2024-09-10T22:05:07.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5df800640 1 -- start start 2024-09-10T22:05:07.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5df800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d80afc20 0x7fe5d8163b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:07.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5df800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d80d06b0 0x7fe5d806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:07.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5df800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5d815bd40 0x7fe5d814f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:07.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5df800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d81692a0 con 0x7fe5d80afc20 2024-09-10T22:05:07.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5df800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d8169120 con 0x7fe5d80d06b0 2024-09-10T22:05:07.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5df800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe5d8169420 con 0x7fe5d815bd40 2024-09-10T22:05:07.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5dce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d80afc20 0x7fe5d8163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:07.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5dce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d80afc20 0x7fe5d8163b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35200/0 (socket says 172.21.15.66:35200) 2024-09-10T22:05:07.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5dce00640 1 -- 172.21.15.66:0/2201556750 learned_addr learned my addr 172.21.15.66:0/2201556750 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:07.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5d7e00640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d80d06b0 0x7fe5d806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:07.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5dd800640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5d815bd40 0x7fe5d814f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:07.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5dce00640 1 -- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5d815bd40 msgr2=0x7fe5d814f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:07.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5dce00640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5d815bd40 0x7fe5d814f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:07.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5dce00640 1 -- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d80d06b0 msgr2=0x7fe5d806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:07.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5dce00640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d80d06b0 0x7fe5d806dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:07.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5dce00640 1 -- 172.21.15.66:0/2201556750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d81fba20 con 0x7fe5d80afc20 2024-09-10T22:05:07.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5dd800640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5d815bd40 0x7fe5d814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:05:07.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5dce00640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d80afc20 0x7fe5d8163b60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c3f0 tx=0x7fe5cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:07.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5d4c00640 1 -- 172.21.15.66:0/2201556750 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc017070 con 0x7fe5d80afc20 2024-09-10T22:05:07.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5d4c00640 1 -- 172.21.15.66:0/2201556750 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5cc0053c0 con 0x7fe5d80afc20 2024-09-10T22:05:07.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5df800640 1 -- 172.21.15.66:0/2201556750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d81fbd00 con 0x7fe5d80afc20 2024-09-10T22:05:07.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5d4c00640 1 -- 172.21.15.66:0/2201556750 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc003860 con 0x7fe5d80afc20 2024-09-10T22:05:07.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.583+0000 7fe5df800640 1 -- 172.21.15.66:0/2201556750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d81fc2c0 con 0x7fe5d80afc20 2024-09-10T22:05:07.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.587+0000 7fe5df800640 1 -- 172.21.15.66:0/2201556750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d815d3b0 con 0x7fe5d80afc20 2024-09-10T22:05:07.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.587+0000 7fe5d4c00640 1 -- 172.21.15.66:0/2201556750 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fe5cc003a00 con 0x7fe5d80afc20 2024-09-10T22:05:07.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.587+0000 7fe5d4c00640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe5b8072e50 0x7fe5b8075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:07.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.587+0000 7fe5d4c00640 1 -- 172.21.15.66:0/2201556750 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5609+0+0 (secure 0 0 0) 0x7fe5cc090fe0 con 0x7fe5d80afc20 2024-09-10T22:05:07.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.587+0000 7fe5d7e00640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe5b8072e50 0x7fe5b8075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:07.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.587+0000 7fe5d7e00640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe5b8072e50 0x7fe5b8075310 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe5d81fb5a0 tx=0x7fe5c40091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:07.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.591+0000 7fe5d4c00640 1 -- 172.21.15.66:0/2201556750 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe5cc05db20 con 0x7fe5d80afc20 2024-09-10T22:05:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:07.799+0000 7fe5df800640 1 -- 172.21.15.66:0/2201556750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe5d80cffa0 con 0x7fe5d80afc20 2024-09-10T22:05:08.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.407+0000 7fe5d4c00640 1 -- 172.21.15.66:0/2201556750 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v32) ==== 172+0+0 (secure 0 0 0) 0x7fe5cc062550 con 0x7fe5d80afc20 2024-09-10T22:05:08.425 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 8 and data pool 9 2024-09-10T22:05:08.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.435+0000 7fe5df800640 1 -- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe5b8072e50 msgr2=0x7fe5b8075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:08.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.435+0000 7fe5df800640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe5b8072e50 0x7fe5b8075310 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe5d81fb5a0 tx=0x7fe5c40091e0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.435+0000 7fe5df800640 1 -- 172.21.15.66:0/2201556750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d80afc20 msgr2=0x7fe5d8163b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:08.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.435+0000 7fe5df800640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d80afc20 0x7fe5d8163b60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c3f0 tx=0x7fe5cc00c8b0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.435+0000 7fe5df800640 1 -- 172.21.15.66:0/2201556750 shutdown_connections 2024-09-10T22:05:08.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.435+0000 7fe5df800640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe5b8072e50 0x7fe5b8075310 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe5d81fb5a0 tx=0x7fe5c40091e0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.435+0000 7fe5df800640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5d815bd40 0x7fe5d814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.435+0000 7fe5df800640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d80d06b0 0x7fe5d806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.435+0000 7fe5df800640 1 --2- 172.21.15.66:0/2201556750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d80afc20 0x7fe5d8163b60 secure :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c3f0 tx=0x7fe5cc00c8b0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.435+0000 7fe5df800640 1 -- 172.21.15.66:0/2201556750 >> 172.21.15.66:0/2201556750 conn(0x7fe5d80d1c60 msgr2=0x7fe5d8155cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.439+0000 7fe5df800640 1 -- 172.21.15.66:0/2201556750 shutdown_connections 2024-09-10T22:05:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.439+0000 7fe5df800640 1 -- 172.21.15.66:0/2201556750 wait complete. 2024-09-10T22:05:08.467 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:05:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 -- 172.21.15.66:0/3510964239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce80b4bd0 msgr2=0x7f7ce80b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 --2- 172.21.15.66:0/3510964239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce80b4bd0 0x7f7ce80b3de0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009bc0 tx=0x7f7ce001cac0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 -- 172.21.15.66:0/3510964239 shutdown_connections 2024-09-10T22:05:08.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 --2- 172.21.15.66:0/3510964239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce80b4bd0 0x7f7ce80b3de0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 -- 172.21.15.66:0/3510964239 >> 172.21.15.66:0/3510964239 conn(0x7f7ce80d3da0 msgr2=0x7f7ce80d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:08.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 -- 172.21.15.66:0/3510964239 shutdown_connections 2024-09-10T22:05:08.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 -- 172.21.15.66:0/3510964239 wait complete. 2024-09-10T22:05:08.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 Processor -- start 2024-09-10T22:05:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 -- start start 2024-09-10T22:05:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce80b4bd0 0x7f7ce815efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce8162630 0x7f7ce80ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce80cd0b0 0x7f7ce80a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce816c340 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce816c1c0 con 0x7f7ce80b4bd0 2024-09-10T22:05:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ced400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce816c4c0 con 0x7f7ce8162630 2024-09-10T22:05:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ceca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce80cd0b0 0x7f7ce80a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ceca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce80cd0b0 0x7f7ce80a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35216/0 (socket says 172.21.15.66:35216) 2024-09-10T22:05:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ceca00640 1 -- 172.21.15.66:0/1102999154 learned_addr learned my addr 172.21.15.66:0/1102999154 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ce7e00640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce80b4bd0 0x7f7ce815efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7cdfe00640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce8162630 0x7f7ce80ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ceca00640 1 -- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce8162630 msgr2=0x7f7ce80ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ceca00640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce8162630 0x7f7ce80ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ceca00640 1 -- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce80b4bd0 msgr2=0x7f7ce815efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ceca00640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce80b4bd0 0x7f7ce815efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7ceca00640 1 -- 172.21.15.66:0/1102999154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce80cd550 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.611+0000 7f7cdfe00640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce8162630 0x7f7ce80ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:05:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.615+0000 7f7ceca00640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce80cd0b0 0x7f7ce80a1080 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00196d0 tx=0x7f7ce0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:08.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.615+0000 7f7ce4c00640 1 -- 172.21.15.66:0/1102999154 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00bb070 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.615+0000 7f7ce4c00640 1 -- 172.21.15.66:0/1102999154 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ce0004e00 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.615+0000 7f7ce4c00640 1 -- 172.21.15.66:0/1102999154 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00b2040 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.615+0000 7f7ced400640 1 -- 172.21.15.66:0/1102999154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce80cdb10 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.615+0000 7f7ced400640 1 -- 172.21.15.66:0/1102999154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce80ce0d0 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.615+0000 7f7ced400640 1 -- 172.21.15.66:0/1102999154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce80b9c00 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.615+0000 7f7ce4c00640 1 -- 172.21.15.66:0/1102999154 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f7ce00b5070 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.615+0000 7f7ce4c00640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7cc4072e50 0x7f7cc4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.615+0000 7f7ce4c00640 1 -- 172.21.15.66:0/1102999154 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f7ce012d690 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.615+0000 7f7ce7e00640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7cc4072e50 0x7f7cc4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:08.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.619+0000 7f7ce7e00640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7cc4072e50 0x7f7cc4075310 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8000fc0 tx=0x7f7cd8001c60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:08.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.619+0000 7f7ce4c00640 1 -- 172.21.15.66:0/1102999154 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7ce00166b0 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.795+0000 7f7ced400640 1 -- 172.21.15.66:0/1102999154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ce808dd40 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.795+0000 7f7ce4c00640 1 -- 172.21.15.66:0/1102999154 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16666 (secure 0 0 0) 0x7f7ce00f7c00 con 0x7f7ce80cd0b0 2024-09-10T22:05:08.800 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:05:08.800 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":58,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:05:07.804406+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:05:05.519188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T22:05:06.772731+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:05:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.795+0000 7f7ced400640 1 -- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7cc4072e50 msgr2=0x7f7cc4075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.795+0000 7f7ced400640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7cc4072e50 0x7f7cc4075310 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8000fc0 tx=0x7f7cd8001c60 comp rx=0 tx=0).stop 2024-09-10T22:05:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.795+0000 7f7ced400640 1 -- 172.21.15.66:0/1102999154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce80cd0b0 msgr2=0x7f7ce80a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.795+0000 7f7ced400640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce80cd0b0 0x7f7ce80a1080 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00196d0 tx=0x7f7ce0003f60 comp rx=0 tx=0).stop 2024-09-10T22:05:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.795+0000 7f7ced400640 1 -- 172.21.15.66:0/1102999154 shutdown_connections 2024-09-10T22:05:08.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.795+0000 7f7ced400640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7cc4072e50 0x7f7cc4075310 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.795+0000 7f7ced400640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ce80cd0b0 0x7f7ce80a1080 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.795+0000 7f7ced400640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce8162630 0x7f7ce80ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.795+0000 7f7ced400640 1 --2- 172.21.15.66:0/1102999154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce80b4bd0 0x7f7ce815efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.795+0000 7f7ced400640 1 -- 172.21.15.66:0/1102999154 >> 172.21.15.66:0/1102999154 conn(0x7f7ce80d3da0 msgr2=0x7f7ce806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:08.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.799+0000 7f7ced400640 1 -- 172.21.15.66:0/1102999154 shutdown_connections 2024-09-10T22:05:08.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.799+0000 7f7ced400640 1 -- 172.21.15.66:0/1102999154 wait complete. 2024-09-10T22:05:08.821 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:05:08.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.963+0000 7fd8ae000640 1 -- 172.21.15.66:0/1385470300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a80c4b10 msgr2=0x7fd8a806e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:08.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.963+0000 7fd8ae000640 1 --2- 172.21.15.66:0/1385470300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a80c4b10 0x7fd8a806e960 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd898009900 tx=0x7fd89801c8e0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.963+0000 7fd8ae000640 1 -- 172.21.15.66:0/1385470300 shutdown_connections 2024-09-10T22:05:08.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.963+0000 7fd8ae000640 1 --2- 172.21.15.66:0/1385470300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a80c4b10 0x7fd8a806e960 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.963+0000 7fd8ae000640 1 -- 172.21.15.66:0/1385470300 >> 172.21.15.66:0/1385470300 conn(0x7fd8a80cfa30 msgr2=0x7fd8a80cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:08.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.963+0000 7fd8ae000640 1 -- 172.21.15.66:0/1385470300 shutdown_connections 2024-09-10T22:05:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.963+0000 7fd8ae000640 1 -- 172.21.15.66:0/1385470300 wait complete. 2024-09-10T22:05:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.963+0000 7fd8ae000640 1 Processor -- start 2024-09-10T22:05:08.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.963+0000 7fd8ae000640 1 -- start start 2024-09-10T22:05:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8ae000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a80ae620 0x7fd8a8150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8ae000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8a80c4b10 0x7fd8a815a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8ae000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a80c4ef0 0x7fd8a814d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8ae000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8169610 con 0x7fd8a80c4ef0 2024-09-10T22:05:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8ae000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8169490 con 0x7fd8a80ae620 2024-09-10T22:05:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8ae000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a8169790 con 0x7fd8a80c4b10 2024-09-10T22:05:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a80c4ef0 0x7fd8a814d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a80c4ef0 0x7fd8a814d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35242/0 (socket says 172.21.15.66:35242) 2024-09-10T22:05:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a7e00640 1 -- 172.21.15.66:0/2538486396 learned_addr learned my addr 172.21.15.66:0/2538486396 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a7400640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a80ae620 0x7fd8a8150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a6a00640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8a80c4b10 0x7fd8a815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a7e00640 1 -- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8a80c4b10 msgr2=0x7fd8a815a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a7e00640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8a80c4b10 0x7fd8a815a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a7e00640 1 -- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a80ae620 msgr2=0x7fd8a8150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a7e00640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a80ae620 0x7fd8a8150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a7e00640 1 -- 172.21.15.66:0/2538486396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a8073be0 con 0x7fd8a80c4ef0 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a7400640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a80ae620 0x7fd8a8150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a6a00640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8a80c4b10 0x7fd8a815a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8a7e00640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a80c4ef0 0x7fd8a814d850 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c740 tx=0x7fd8a000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd89f400640 1 -- 172.21.15.66:0/2538486396 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0003b80 con 0x7fd8a80c4ef0 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd89f400640 1 -- 172.21.15.66:0/2538486396 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8a000ce50 con 0x7fd8a80c4ef0 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd89f400640 1 -- 172.21.15.66:0/2538486396 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000a850 con 0x7fd8a80c4ef0 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8ae000640 1 -- 172.21.15.66:0/2538486396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a8073ec0 con 0x7fd8a80c4ef0 2024-09-10T22:05:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8ae000640 1 -- 172.21.15.66:0/2538486396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a8158930 con 0x7fd8a80c4ef0 2024-09-10T22:05:08.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.967+0000 7fd8ae000640 1 -- 172.21.15.66:0/2538486396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a806e960 con 0x7fd8a80c4ef0 2024-09-10T22:05:08.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.971+0000 7fd89f400640 1 -- 172.21.15.66:0/2538486396 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fd8a000a9f0 con 0x7fd8a80c4ef0 2024-09-10T22:05:08.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.971+0000 7fd89f400640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd888072e50 0x7fd888075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:08.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.971+0000 7fd89f400640 1 -- 172.21.15.66:0/2538486396 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fd8a0094330 con 0x7fd8a80c4ef0 2024-09-10T22:05:08.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.971+0000 7fd8a7400640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd888072e50 0x7fd888075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:08.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.971+0000 7fd8a7400640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd888072e50 0x7fd888075310 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd8980af000 tx=0x7fd89801b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:08.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:08.975+0000 7fd89f400640 1 -- 172.21.15.66:0/2538486396 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd8a005ea70 con 0x7fd8a80c4ef0 2024-09-10T22:05:09.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.219+0000 7fd8ae000640 1 -- 172.21.15.66:0/2538486396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8a80cbc60 con 0x7fd8a80c4ef0 2024-09-10T22:05:09.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd89f400640 1 -- 172.21.15.66:0/2538486396 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16666 (secure 0 0 0) 0x7fd8a0067190 con 0x7fd8a80c4ef0 2024-09-10T22:05:09.226 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:05:09.226 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":58,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:05:07.804406+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:05:05.519188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T22:05:06.772731+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:05:09.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd8ae000640 1 -- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd888072e50 msgr2=0x7fd888075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:09.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd8ae000640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd888072e50 0x7fd888075310 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd8980af000 tx=0x7fd89801b120 comp rx=0 tx=0).stop 2024-09-10T22:05:09.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd8ae000640 1 -- 172.21.15.66:0/2538486396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a80c4ef0 msgr2=0x7fd8a814d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:09.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd8ae000640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a80c4ef0 0x7fd8a814d850 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c740 tx=0x7fd8a000cc00 comp rx=0 tx=0).stop 2024-09-10T22:05:09.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd8ae000640 1 -- 172.21.15.66:0/2538486396 shutdown_connections 2024-09-10T22:05:09.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd8ae000640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd888072e50 0x7fd888075310 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:09.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd8ae000640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a80c4ef0 0x7fd8a814d850 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:09.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd8ae000640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8a80c4b10 0x7fd8a815a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:09.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd8ae000640 1 --2- 172.21.15.66:0/2538486396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a80ae620 0x7fd8a8150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:09.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd8ae000640 1 -- 172.21.15.66:0/2538486396 >> 172.21.15.66:0/2538486396 conn(0x7fd8a80cfa30 msgr2=0x7fd8a80af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:09.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd8ae000640 1 -- 172.21.15.66:0/2538486396 shutdown_connections 2024-09-10T22:05:09.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.223+0000 7fd8ae000640 1 -- 172.21.15.66:0/2538486396 wait complete. 2024-09-10T22:05:09.248 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:05:09.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.391+0000 7f5092000640 1 -- 172.21.15.66:0/1521157140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508c0d1b80 msgr2=0x7f508c0b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:09.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.391+0000 7f5092000640 1 --2- 172.21.15.66:0/1521157140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508c0d1b80 0x7f508c0b36d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5074009840 tx=0x7f507401c820 comp rx=0 tx=0).stop 2024-09-10T22:05:09.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.391+0000 7f5092000640 1 -- 172.21.15.66:0/1521157140 shutdown_connections 2024-09-10T22:05:09.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.391+0000 7f5092000640 1 --2- 172.21.15.66:0/1521157140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508c0d1b80 0x7f508c0b36d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:09.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.391+0000 7f5092000640 1 -- 172.21.15.66:0/1521157140 >> 172.21.15.66:0/1521157140 conn(0x7f508c0d3da0 msgr2=0x7f508c0d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:09.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.391+0000 7f5092000640 1 -- 172.21.15.66:0/1521157140 shutdown_connections 2024-09-10T22:05:09.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.391+0000 7f5092000640 1 -- 172.21.15.66:0/1521157140 wait complete. 2024-09-10T22:05:09.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5092000640 1 Processor -- start 2024-09-10T22:05:09.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5092000640 1 -- start start 2024-09-10T22:05:09.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5092000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f508c0b4bd0 0x7f508c0cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:09.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5092000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508c162630 0x7f508c153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:09.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5092000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f508c153fb0 0x7f508c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5092000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f508c16c380 con 0x7f508c162630 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5092000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f508c16c200 con 0x7f508c0b4bd0 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5092000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f508c16c500 con 0x7f508c153fb0 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5083e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508c162630 0x7f508c153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5083e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508c162630 0x7f508c153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35266/0 (socket says 172.21.15.66:35266) 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5083e00640 1 -- 172.21.15.66:0/693469818 learned_addr learned my addr 172.21.15.66:0/693469818 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5090c00640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f508c0b4bd0 0x7f508c0cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5091600640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f508c153fb0 0x7f508c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5083e00640 1 -- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f508c153fb0 msgr2=0x7f508c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5083e00640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f508c153fb0 0x7f508c15db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5083e00640 1 -- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f508c0b4bd0 msgr2=0x7f508c0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5083e00640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f508c0b4bd0 0x7f508c0cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:09.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5083e00640 1 -- 172.21.15.66:0/693469818 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c1548f0 con 0x7f508c162630 2024-09-10T22:05:09.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5090c00640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f508c0b4bd0 0x7f508c0cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:05:09.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5083e00640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508c162630 0x7f508c153a70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5074000c10 tx=0x7f50740a6fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:09.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5089600640 1 -- 172.21.15.66:0/693469818 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50740051c0 con 0x7f508c162630 2024-09-10T22:05:09.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5089600640 1 -- 172.21.15.66:0/693469818 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5074005360 con 0x7f508c162630 2024-09-10T22:05:09.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5092000640 1 -- 172.21.15.66:0/693469818 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508c154bd0 con 0x7f508c162630 2024-09-10T22:05:09.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5089600640 1 -- 172.21.15.66:0/693469818 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50740046d0 con 0x7f508c162630 2024-09-10T22:05:09.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.395+0000 7f5092000640 1 -- 172.21.15.66:0/693469818 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508c154440 con 0x7f508c162630 2024-09-10T22:05:09.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.399+0000 7f5092000640 1 -- 172.21.15.66:0/693469818 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508c08dd40 con 0x7f508c162630 2024-09-10T22:05:09.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.399+0000 7f5089600640 1 -- 172.21.15.66:0/693469818 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f50740c3060 con 0x7f508c162630 2024-09-10T22:05:09.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.399+0000 7f5089600640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5068072e80 0x7f5068075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:09.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.399+0000 7f5089600640 1 -- 172.21.15.66:0/693469818 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f507412b7e0 con 0x7f508c162630 2024-09-10T22:05:09.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.399+0000 7f5090c00640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5068072e80 0x7f5068075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:09.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.403+0000 7f5090c00640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5068072e80 0x7f5068075340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f507c0016b0 tx=0x7f507c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.407+0000 7f5089600640 1 -- 172.21.15.66:0/693469818 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f50740ac030 con 0x7f508c162630 2024-09-10T22:05:09.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:09.615+0000 7f5092000640 1 -- 172.21.15.66:0/693469818 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f508c157dc0 con 0x7f508c162630 2024-09-10T22:05:10.385 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-09-10T22:05:10.379+0000 7f7f2a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:05:10.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.419+0000 7f5089600640 1 -- 172.21.15.66:0/693469818 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7f5074016410 con 0x7f508c162630 2024-09-10T22:05:10.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.423+0000 7f5092000640 1 -- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5068072e80 msgr2=0x7f5068075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:10.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.423+0000 7f5092000640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5068072e80 0x7f5068075340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f507c0016b0 tx=0x7f507c009040 comp rx=0 tx=0).stop 2024-09-10T22:05:10.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.423+0000 7f5092000640 1 -- 172.21.15.66:0/693469818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508c162630 msgr2=0x7f508c153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:10.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.423+0000 7f5092000640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508c162630 0x7f508c153a70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5074000c10 tx=0x7f50740a6fe0 comp rx=0 tx=0).stop 2024-09-10T22:05:10.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.423+0000 7f5092000640 1 -- 172.21.15.66:0/693469818 shutdown_connections 2024-09-10T22:05:10.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.423+0000 7f5092000640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5068072e80 0x7f5068075340 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:10.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.423+0000 7f5092000640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f508c153fb0 0x7f508c15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:10.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.423+0000 7f5092000640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508c162630 0x7f508c153a70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:10.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.423+0000 7f5092000640 1 --2- 172.21.15.66:0/693469818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f508c0b4bd0 0x7f508c0cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:10.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.423+0000 7f5092000640 1 -- 172.21.15.66:0/693469818 >> 172.21.15.66:0/693469818 conn(0x7f508c0d3da0 msgr2=0x7f508c06e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:10.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.423+0000 7f5092000640 1 -- 172.21.15.66:0/693469818 shutdown_connections 2024-09-10T22:05:10.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.423+0000 7f5092000640 1 -- 172.21.15.66:0/693469818 wait complete. 2024-09-10T22:05:10.451 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:05:10.451 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:05:10.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.603+0000 7f3fc7600640 1 -- 172.21.15.66:0/3168763413 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3fc00b32a0 msgr2=0x7f3fc00af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:10.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.603+0000 7f3fc7600640 1 --2- 172.21.15.66:0/3168763413 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3fc00b32a0 0x7f3fc00af390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009c80 tx=0x7f3fb801cc50 comp rx=0 tx=0).stop 2024-09-10T22:05:10.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.603+0000 7f3fc7600640 1 -- 172.21.15.66:0/3168763413 shutdown_connections 2024-09-10T22:05:10.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.603+0000 7f3fc7600640 1 --2- 172.21.15.66:0/3168763413 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3fc00b32a0 0x7f3fc00af390 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:10.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.603+0000 7f3fc7600640 1 --2- 172.21.15.66:0/3168763413 >> v2:172.21.15.94:3300/0 conn(0x7f3fc00b0f60 0x7f3fc00d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:10.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.603+0000 7f3fc7600640 1 -- 172.21.15.66:0/3168763413 >> 172.21.15.66:0/3168763413 conn(0x7f3fc00d1d00 msgr2=0x7f3fc00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:10.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.603+0000 7f3fc7600640 1 -- 172.21.15.66:0/3168763413 shutdown_connections 2024-09-10T22:05:10.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.603+0000 7f3fc7600640 1 -- 172.21.15.66:0/3168763413 wait complete. 2024-09-10T22:05:10.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.603+0000 7f3fc7600640 1 Processor -- start 2024-09-10T22:05:10.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.603+0000 7f3fc7600640 1 -- start start 2024-09-10T22:05:10.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc7600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3fc00b0f60 0x7f3fc00ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:10.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc7600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fc00b32a0 0x7f3fc014cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc7600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fc015f460 0x7f3fc0146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc7600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc00b0630 con 0x7f3fc015f460 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc7600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc00b04b0 con 0x7f3fc00b32a0 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc7600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc0168c60 con 0x7f3fc00b0f60 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fc015f460 0x7f3fc0146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fc015f460 0x7f3fc0146a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35276/0 (socket says 172.21.15.66:35276) 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc5600640 1 -- 172.21.15.66:0/3642834530 learned_addr learned my addr 172.21.15.66:0/3642834530 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fbfe00640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fc00b32a0 0x7f3fc014cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc4c00640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3fc00b0f60 0x7f3fc00ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc5600640 1 -- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3fc00b0f60 msgr2=0x7f3fc00ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc5600640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3fc00b0f60 0x7f3fc00ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc5600640 1 -- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fc00b32a0 msgr2=0x7f3fc014cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc5600640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fc00b32a0 0x7f3fc014cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc5600640 1 -- 172.21.15.66:0/3642834530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc00ce930 con 0x7f3fc015f460 2024-09-10T22:05:10.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fbfe00640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fc00b32a0 0x7f3fc014cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:05:10.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc5600640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fc015f460 0x7f3fc0146a20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009bd0 tx=0x7f3fb8003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:10.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fbcc00640 1 -- 172.21.15.66:0/3642834530 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80bb070 con 0x7f3fc015f460 2024-09-10T22:05:10.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fbcc00640 1 -- 172.21.15.66:0/3642834530 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fb8004d40 con 0x7f3fc015f460 2024-09-10T22:05:10.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc7600640 1 -- 172.21.15.66:0/3642834530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc0147f50 con 0x7f3fc015f460 2024-09-10T22:05:10.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fbcc00640 1 -- 172.21.15.66:0/3642834530 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80b2040 con 0x7f3fc015f460 2024-09-10T22:05:10.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc7600640 1 -- 172.21.15.66:0/3642834530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc0148510 con 0x7f3fc015f460 2024-09-10T22:05:10.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.607+0000 7f3fc7600640 1 -- 172.21.15.66:0/3642834530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc00af390 con 0x7f3fc015f460 2024-09-10T22:05:10.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.611+0000 7f3fbcc00640 1 -- 172.21.15.66:0/3642834530 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f3fb80b5030 con 0x7f3fc015f460 2024-09-10T22:05:10.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.611+0000 7f3fbcc00640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3fa0072e50 0x7f3fa0075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:10.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.611+0000 7f3fbcc00640 1 -- 172.21.15.66:0/3642834530 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f3fb812dbc0 con 0x7f3fc015f460 2024-09-10T22:05:10.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.611+0000 7f3fc4c00640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3fa0072e50 0x7f3fa0075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:10.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.611+0000 7f3fc4c00640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3fa0072e50 0x7f3fa0075310 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00d1cd0 tx=0x7f3fb4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:10.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.615+0000 7f3fbcc00640 1 -- 172.21.15.66:0/3642834530 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3fb8016990 con 0x7f3fc015f460 2024-09-10T22:05:10.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:10.823+0000 7f3fc7600640 1 -- 172.21.15.66:0/3642834530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3fc0074aa0 con 0x7f3fc015f460 2024-09-10T22:05:11.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.431+0000 7f3fbcc00640 1 -- 172.21.15.66:0/3642834530 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f3fb80f8690 con 0x7f3fc015f460 2024-09-10T22:05:11.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.435+0000 7f3fc7600640 1 -- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3fa0072e50 msgr2=0x7f3fa0075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:11.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.435+0000 7f3fc7600640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3fa0072e50 0x7f3fa0075310 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00d1cd0 tx=0x7f3fb4009040 comp rx=0 tx=0).stop 2024-09-10T22:05:11.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.435+0000 7f3fc7600640 1 -- 172.21.15.66:0/3642834530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fc015f460 msgr2=0x7f3fc0146a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:11.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.435+0000 7f3fc7600640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fc015f460 0x7f3fc0146a20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009bd0 tx=0x7f3fb8003f40 comp rx=0 tx=0).stop 2024-09-10T22:05:11.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.435+0000 7f3fc7600640 1 -- 172.21.15.66:0/3642834530 shutdown_connections 2024-09-10T22:05:11.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.435+0000 7f3fc7600640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3fa0072e50 0x7f3fa0075310 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:11.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.435+0000 7f3fc7600640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fc015f460 0x7f3fc0146a20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:11.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.435+0000 7f3fc7600640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fc00b32a0 0x7f3fc014cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:11.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.435+0000 7f3fc7600640 1 --2- 172.21.15.66:0/3642834530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3fc00b0f60 0x7f3fc00ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:11.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.435+0000 7f3fc7600640 1 -- 172.21.15.66:0/3642834530 >> 172.21.15.66:0/3642834530 conn(0x7f3fc00d1d00 msgr2=0x7f3fc0071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:11.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.435+0000 7f3fc7600640 1 -- 172.21.15.66:0/3642834530 shutdown_connections 2024-09-10T22:05:11.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.435+0000 7f3fc7600640 1 -- 172.21.15.66:0/3642834530 wait complete. 2024-09-10T22:05:11.462 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:05:11.462 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:05:11.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.607+0000 7fe355200640 1 -- 172.21.15.66:0/1924856337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe334003a10 msgr2=0x7fe334003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:11.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.607+0000 7fe355200640 1 --2- 172.21.15.66:0/1924856337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe334003a10 0x7fe334003e30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe34000c310 tx=0x7fe34001cfc0 comp rx=0 tx=0).stop 2024-09-10T22:05:11.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.607+0000 7fe34e800640 1 -- 172.21.15.66:0/1924856337 reap_dead start 2024-09-10T22:05:11.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.607+0000 7fe355200640 1 -- 172.21.15.66:0/1924856337 shutdown_connections 2024-09-10T22:05:11.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.607+0000 7fe355200640 1 -- 172.21.15.66:0/1924856337 >> 172.21.15.66:0/1924856337 conn(0x7fe3500d1c60 msgr2=0x7fe3500d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:11.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.607+0000 7fe355200640 1 -- 172.21.15.66:0/1924856337 shutdown_connections 2024-09-10T22:05:11.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.607+0000 7fe355200640 1 -- 172.21.15.66:0/1924856337 wait complete. 2024-09-10T22:05:11.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe355200640 1 Processor -- start 2024-09-10T22:05:11.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe355200640 1 -- start start 2024-09-10T22:05:11.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe355200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe334003a10 0x7fe35015edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:11.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe355200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3501fbb10 0x7fe3500cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:11.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe355200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe350163b60 0x7fe350163fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:11.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe355200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe350166a10 con 0x7fe350163b60 2024-09-10T22:05:11.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe355200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3500cf500 con 0x7fe3501fbb10 2024-09-10T22:05:11.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe355200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe350166b90 con 0x7fe334003a10 2024-09-10T22:05:11.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe34e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe350163b60 0x7fe350163fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:11.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe34e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe350163b60 0x7fe350163fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35284/0 (socket says 172.21.15.66:35284) 2024-09-10T22:05:11.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe34e800640 1 -- 172.21.15.66:0/2404240987 learned_addr learned my addr 172.21.15.66:0/2404240987 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:11.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe34de00640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe334003a10 0x7fe35015edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:11.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe347e00640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3501fbb10 0x7fe3500cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:11.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe34e800640 1 -- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe334003a10 msgr2=0x7fe35015edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:11.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe34e800640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe334003a10 0x7fe35015edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:11.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe34e800640 1 -- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3501fbb10 msgr2=0x7fe3500cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:11.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe34e800640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3501fbb10 0x7fe3500cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:11.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe34e800640 1 -- 172.21.15.66:0/2404240987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3501f7c20 con 0x7fe350163b60 2024-09-10T22:05:11.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe34de00640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe334003a10 0x7fe35015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:05:11.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe347e00640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3501fbb10 0x7fe3500cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:05:11.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe34e800640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe350163b60 0x7fe350163fe0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe348011b20 tx=0x7fe34800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:11.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe346000640 1 -- 172.21.15.66:0/2404240987 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348004b30 con 0x7fe350163b60 2024-09-10T22:05:11.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe346000640 1 -- 172.21.15.66:0/2404240987 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe348004cd0 con 0x7fe350163b60 2024-09-10T22:05:11.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe346000640 1 -- 172.21.15.66:0/2404240987 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348003be0 con 0x7fe350163b60 2024-09-10T22:05:11.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe355200640 1 -- 172.21.15.66:0/2404240987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3501f7f00 con 0x7fe350163b60 2024-09-10T22:05:11.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.611+0000 7fe355200640 1 -- 172.21.15.66:0/2404240987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3501f84c0 con 0x7fe350163b60 2024-09-10T22:05:11.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.615+0000 7fe355200640 1 -- 172.21.15.66:0/2404240987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe350164520 con 0x7fe350163b60 2024-09-10T22:05:11.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.615+0000 7fe346000640 1 -- 172.21.15.66:0/2404240987 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fe348021030 con 0x7fe350163b60 2024-09-10T22:05:11.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.615+0000 7fe346000640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe32c072e50 0x7fe32c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:11.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.615+0000 7fe346000640 1 -- 172.21.15.66:0/2404240987 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fe348094e60 con 0x7fe350163b60 2024-09-10T22:05:11.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.615+0000 7fe34de00640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe32c072e50 0x7fe32c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:11.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.615+0000 7fe34de00640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe32c072e50 0x7fe32c075310 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe3400ac000 tx=0x7fe340019e00 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:11.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.619+0000 7fe346000640 1 -- 172.21.15.66:0/2404240987 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe34805f510 con 0x7fe350163b60 2024-09-10T22:05:11.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 -- 172.21.15.66:0/2404240987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3500ce170 con 0x7fe350163b60 2024-09-10T22:05:11.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe346000640 1 -- 172.21.15.66:0/2404240987 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5220 (secure 0 0 0) 0x7fe34805ee10 con 0x7fe350163b60 2024-09-10T22:05:11.831 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:05:11.831 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":36,"btime":"2024-09-10T22:05:11:423882+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5414,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/135515157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":135515157},{"type":"v1","addr":"172.21.15.94:6837","nonce":135515157}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1614138608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1614138608},{"type":"v1","addr":"172.21.15.94:6836","nonce":1614138608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1060280836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1060280836},{"type":"v1","addr":"172.21.15.66:6833","nonce":1060280836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:05:07.804373+0000","modified":"2024-09-10T22:05:10.828111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5925},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5925":{"gid":5925,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2880381289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2880381289},{"type":"v1","addr":"172.21.15.66:6835","nonce":2880381289}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5925,"qdb_cluster":[5925]},"id":4}]} 2024-09-10T22:05:11.832 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 36 2024-09-10T22:05:11.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 -- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe32c072e50 msgr2=0x7fe32c075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:11.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe32c072e50 0x7fe32c075310 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe3400ac000 tx=0x7fe340019e00 comp rx=0 tx=0).stop 2024-09-10T22:05:11.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 -- 172.21.15.66:0/2404240987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe350163b60 msgr2=0x7fe350163fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:11.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe350163b60 0x7fe350163fe0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe348011b20 tx=0x7fe34800f3c0 comp rx=0 tx=0).stop 2024-09-10T22:05:11.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 -- 172.21.15.66:0/2404240987 shutdown_connections 2024-09-10T22:05:11.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe32c072e50 0x7fe32c075310 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:11.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe350163b60 0x7fe350163fe0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:11.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3501fbb10 0x7fe3500cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:11.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 --2- 172.21.15.66:0/2404240987 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe334003a10 0x7fe35015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:11.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 -- 172.21.15.66:0/2404240987 >> 172.21.15.66:0/2404240987 conn(0x7fe3500d1c60 msgr2=0x7fe3500b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:11.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 -- 172.21.15.66:0/2404240987 shutdown_connections 2024-09-10T22:05:11.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.827+0000 7fe355200640 1 -- 172.21.15.66:0/2404240987 wait complete. 2024-09-10T22:05:11.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:05:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.995+0000 7f382bc00640 1 -- 172.21.15.66:0/532580994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f382415bd40 msgr2=0x7f38240cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.995+0000 7f382bc00640 1 --2- 172.21.15.66:0/532580994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f382415bd40 0x7f38240cffa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3814009840 tx=0x7f381401c790 comp rx=0 tx=0).stop 2024-09-10T22:05:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.995+0000 7f382bc00640 1 -- 172.21.15.66:0/532580994 shutdown_connections 2024-09-10T22:05:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.995+0000 7f382bc00640 1 --2- 172.21.15.66:0/532580994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f382415bd40 0x7f38240cffa0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.995+0000 7f382bc00640 1 -- 172.21.15.66:0/532580994 >> 172.21.15.66:0/532580994 conn(0x7f38240d1c60 msgr2=0x7f38240d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.995+0000 7f382bc00640 1 -- 172.21.15.66:0/532580994 shutdown_connections 2024-09-10T22:05:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.995+0000 7f382bc00640 1 -- 172.21.15.66:0/532580994 wait complete. 2024-09-10T22:05:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f382bc00640 1 Processor -- start 2024-09-10T22:05:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f382bc00640 1 -- start start 2024-09-10T22:05:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f382bc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38240cf180 0x7f382406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f382bc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38240cf830 0x7f3824166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f382bc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f382415bd40 0x7f382406f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f382bc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f382416a580 con 0x7f38240cf180 2024-09-10T22:05:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f382bc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f382416a400 con 0x7f382415bd40 2024-09-10T22:05:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f382bc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f382416a700 con 0x7f38240cf830 2024-09-10T22:05:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3829200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38240cf180 0x7f382406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3829200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38240cf180 0x7f382406dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35304/0 (socket says 172.21.15.66:35304) 2024-09-10T22:05:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3829200640 1 -- 172.21.15.66:0/2217683018 learned_addr learned my addr 172.21.15.66:0/2217683018 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f381be00640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38240cf830 0x7f3824166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3829c00640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f382415bd40 0x7f382406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3829200640 1 -- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38240cf830 msgr2=0x7f3824166ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3829200640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38240cf830 0x7f3824166ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3829200640 1 -- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f382415bd40 msgr2=0x7f382406f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3829200640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f382415bd40 0x7f382406f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3829200640 1 -- 172.21.15.66:0/2217683018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38241fbf90 con 0x7f38240cf180 2024-09-10T22:05:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3829c00640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f382415bd40 0x7f382406f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:05:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3829200640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38240cf180 0x7f382406dd30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3814009790 tx=0x7f38140a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3821600640 1 -- 172.21.15.66:0/2217683018 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38140bb070 con 0x7f38240cf180 2024-09-10T22:05:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3821600640 1 -- 172.21.15.66:0/2217683018 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3814004430 con 0x7f38240cf180 2024-09-10T22:05:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f382bc00640 1 -- 172.21.15.66:0/2217683018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38241fc270 con 0x7f38240cf180 2024-09-10T22:05:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f3821600640 1 -- 172.21.15.66:0/2217683018 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38140b2040 con 0x7f38240cf180 2024-09-10T22:05:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f382bc00640 1 -- 172.21.15.66:0/2217683018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38241fc830 con 0x7f38240cf180 2024-09-10T22:05:12.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:11.999+0000 7f382bc00640 1 -- 172.21.15.66:0/2217683018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38240cffa0 con 0x7f38240cf180 2024-09-10T22:05:12.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.003+0000 7f3821600640 1 -- 172.21.15.66:0/2217683018 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f38140b5030 con 0x7f38240cf180 2024-09-10T22:05:12.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.003+0000 7f3821600640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3800072e80 0x7f3800075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.003+0000 7f3821600640 1 -- 172.21.15.66:0/2217683018 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f381412b000 con 0x7f38240cf180 2024-09-10T22:05:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.003+0000 7f381be00640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3800072e80 0x7f3800075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.003+0000 7f381be00640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3800072e80 0x7f3800075340 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f382415bab0 tx=0x7f380c007ea0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.007+0000 7f3821600640 1 -- 172.21.15.66:0/2217683018 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3814016410 con 0x7f38240cf180 2024-09-10T22:05:12.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.183+0000 7f382bc00640 1 -- 172.21.15.66:0/2217683018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38240b0df0 con 0x7f38240cf180 2024-09-10T22:05:12.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.183+0000 7f3821600640 1 -- 172.21.15.66:0/2217683018 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16653 (secure 0 0 0) 0x7f38140f7b40 con 0x7f38240cf180 2024-09-10T22:05:12.190 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:05:12.190 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":59,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:05:09.401172+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:05:05.519188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T22:05:06.772731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:05:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.187+0000 7f382bc00640 1 -- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3800072e80 msgr2=0x7f3800075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.187+0000 7f382bc00640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3800072e80 0x7f3800075340 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f382415bab0 tx=0x7f380c007ea0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.187+0000 7f382bc00640 1 -- 172.21.15.66:0/2217683018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38240cf180 msgr2=0x7f382406dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.187+0000 7f382bc00640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38240cf180 0x7f382406dd30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3814009790 tx=0x7f38140a5670 comp rx=0 tx=0).stop 2024-09-10T22:05:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.187+0000 7f382bc00640 1 -- 172.21.15.66:0/2217683018 shutdown_connections 2024-09-10T22:05:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.187+0000 7f382bc00640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3800072e80 0x7f3800075340 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.187+0000 7f382bc00640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f382415bd40 0x7f382406f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.187+0000 7f382bc00640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38240cf830 0x7f3824166ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.187+0000 7f382bc00640 1 --2- 172.21.15.66:0/2217683018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38240cf180 0x7f382406dd30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.187+0000 7f382bc00640 1 -- 172.21.15.66:0/2217683018 >> 172.21.15.66:0/2217683018 conn(0x7f38240d1c60 msgr2=0x7f38240b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.187+0000 7f382bc00640 1 -- 172.21.15.66:0/2217683018 shutdown_connections 2024-09-10T22:05:12.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.187+0000 7f382bc00640 1 -- 172.21.15.66:0/2217683018 wait complete. 2024-09-10T22:05:12.213 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:05:12.213 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:05:12.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.355+0000 7f5f6be00640 1 -- 172.21.15.66:0/4248488470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f50004810 msgr2=0x7f5f50004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.355+0000 7f5f6be00640 1 --2- 172.21.15.66:0/4248488470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f50004810 0x7f5f50004c90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c08d190 tx=0x7f5f5401c7e0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.355+0000 7f5f6be00640 1 -- 172.21.15.66:0/4248488470 shutdown_connections 2024-09-10T22:05:12.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.355+0000 7f5f6be00640 1 --2- 172.21.15.66:0/4248488470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f50004810 0x7f5f50004c90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.355+0000 7f5f6be00640 1 -- 172.21.15.66:0/4248488470 >> 172.21.15.66:0/4248488470 conn(0x7f5f6c0d3da0 msgr2=0x7f5f6c0d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:12.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.355+0000 7f5f6be00640 1 -- 172.21.15.66:0/4248488470 shutdown_connections 2024-09-10T22:05:12.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.355+0000 7f5f6be00640 1 -- 172.21.15.66:0/4248488470 wait complete. 2024-09-10T22:05:12.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6be00640 1 Processor -- start 2024-09-10T22:05:12.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6be00640 1 -- start start 2024-09-10T22:05:12.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6be00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f50004810 0x7f5f6c0b9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:12.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c1551d0 0x7f5f6c0c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:12.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f6c1565c0 0x7f5f6c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:12.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c0b15b0 con 0x7f5f6c1565c0 2024-09-10T22:05:12.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c0b1430 con 0x7f5f6c1551d0 2024-09-10T22:05:12.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6be00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6c0b36d0 con 0x7f5f50004810 2024-09-10T22:05:12.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f6c1565c0 0x7f5f6c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:12.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f6c1565c0 0x7f5f6c156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58378/0 (socket says 172.21.15.66:58378) 2024-09-10T22:05:12.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6b400640 1 -- 172.21.15.66:0/416209112 learned_addr learned my addr 172.21.15.66:0/416209112 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:12.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f63e00640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c1551d0 0x7f5f6c0c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:12.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6aa00640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f50004810 0x7f5f6c0b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:12.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6b400640 1 -- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f50004810 msgr2=0x7f5f6c0b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6b400640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f50004810 0x7f5f6c0b9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6b400640 1 -- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c1551d0 msgr2=0x7f5f6c0c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6b400640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c1551d0 0x7f5f6c0c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6b400640 1 -- 172.21.15.66:0/416209112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c159490 con 0x7f5f6c1565c0 2024-09-10T22:05:12.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6b400640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f6c1565c0 0x7f5f6c156a40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0d1870 tx=0x7f5f6400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:12.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f63400640 1 -- 172.21.15.66:0/416209112 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64010890 con 0x7f5f6c1565c0 2024-09-10T22:05:12.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f63400640 1 -- 172.21.15.66:0/416209112 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f64017070 con 0x7f5f6c1565c0 2024-09-10T22:05:12.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f63400640 1 -- 172.21.15.66:0/416209112 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64010d60 con 0x7f5f6c1565c0 2024-09-10T22:05:12.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6be00640 1 -- 172.21.15.66:0/416209112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6c159770 con 0x7f5f6c1565c0 2024-09-10T22:05:12.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6be00640 1 -- 172.21.15.66:0/416209112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6c0b7ae0 con 0x7f5f6c1565c0 2024-09-10T22:05:12.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.359+0000 7f5f6be00640 1 -- 172.21.15.66:0/416209112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f6c0b1730 con 0x7f5f6c1565c0 2024-09-10T22:05:12.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.363+0000 7f5f63400640 1 -- 172.21.15.66:0/416209112 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f5f6401c460 con 0x7f5f6c1565c0 2024-09-10T22:05:12.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.363+0000 7f5f63400640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f48072e50 0x7f5f48075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:12.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.363+0000 7f5f63400640 1 -- 172.21.15.66:0/416209112 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5f64094610 con 0x7f5f6c1565c0 2024-09-10T22:05:12.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.363+0000 7f5f6aa00640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f48072e50 0x7f5f48075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:12.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.363+0000 7f5f6aa00640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f48072e50 0x7f5f48075310 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009ce0 tx=0x7f5f5c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:12.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.367+0000 7f5f63400640 1 -- 172.21.15.66:0/416209112 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f6405ed50 con 0x7f5f6c1565c0 2024-09-10T22:05:12.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.539+0000 7f5f6be00640 1 -- 172.21.15.66:0/416209112 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f6c0d2160 con 0x7f5f48072e50 2024-09-10T22:05:12.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.547+0000 7f5f63400640 1 -- 172.21.15.66:0/416209112 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f5f5c05e840 con 0x7f5f48072e50 2024-09-10T22:05:12.555 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:05:12.557 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:05:12.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.551+0000 7f5f6be00640 1 -- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f48072e50 msgr2=0x7f5f48075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.551+0000 7f5f6be00640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f48072e50 0x7f5f48075310 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009ce0 tx=0x7f5f5c0091e0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.551+0000 7f5f6be00640 1 -- 172.21.15.66:0/416209112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f6c1565c0 msgr2=0x7f5f6c156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.551+0000 7f5f6be00640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f6c1565c0 0x7f5f6c156a40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0d1870 tx=0x7f5f6400ea20 comp rx=0 tx=0).stop 2024-09-10T22:05:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.551+0000 7f5f6be00640 1 -- 172.21.15.66:0/416209112 shutdown_connections 2024-09-10T22:05:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.551+0000 7f5f6be00640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f48072e50 0x7f5f48075310 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.551+0000 7f5f6be00640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f6c1565c0 0x7f5f6c156a40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.551+0000 7f5f6be00640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c1551d0 0x7f5f6c0c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.551+0000 7f5f6be00640 1 --2- 172.21.15.66:0/416209112 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f50004810 0x7f5f6c0b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.551+0000 7f5f6be00640 1 -- 172.21.15.66:0/416209112 >> 172.21.15.66:0/416209112 conn(0x7f5f6c0d3da0 msgr2=0x7f5f6c160930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.551+0000 7f5f6be00640 1 -- 172.21.15.66:0/416209112 shutdown_connections 2024-09-10T22:05:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.551+0000 7f5f6be00640 1 -- 172.21.15.66:0/416209112 wait complete. 2024-09-10T22:05:12.574 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-10T22:05:12.379812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":383888,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":377011,"kb_avail":754590832,"statfs":{"total":773094113280,"available":772701011968,"internally_reserved":0,"allocated":6856704,"data_stored":3510268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":386059828},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.988688"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.405427+0000","last_change":"2024-09-10T22:05:08.436919+0000","last_active":"2024-09-10T22:05:09.405427+0000","last_peered":"2024-09-10T22:05:09.405427+0000","last_clean":"2024-09-10T22:05:09.405427+0000","last_became_active":"2024-09-10T22:05:08.436648+0000","last_became_peered":"2024-09-10T22:05:08.436648+0000","last_unstale":"2024-09-10T22:05:09.405427+0000","last_undegraded":"2024-09-10T22:05:09.405427+0000","last_fullsized":"2024-09-10T22:05:09.405427+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:17.767001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970153+0000","last_change":"2024-09-10T22:05:08.441899+0000","last_active":"2024-09-10T22:05:09.970153+0000","last_peered":"2024-09-10T22:05:09.970153+0000","last_clean":"2024-09-10T22:05:09.970153+0000","last_became_active":"2024-09-10T22:05:08.441635+0000","last_became_peered":"2024-09-10T22:05:08.441635+0000","last_unstale":"2024-09-10T22:05:09.970153+0000","last_undegraded":"2024-09-10T22:05:09.970153+0000","last_fullsized":"2024-09-10T22:05:09.970153+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:33.727735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408904+0000","last_change":"2024-09-10T22:05:08.426344+0000","last_active":"2024-09-10T22:05:09.408904+0000","last_peered":"2024-09-10T22:05:09.408904+0000","last_clean":"2024-09-10T22:05:09.408904+0000","last_became_active":"2024-09-10T22:05:08.426087+0000","last_became_peered":"2024-09-10T22:05:08.426087+0000","last_unstale":"2024-09-10T22:05:09.408904+0000","last_undegraded":"2024-09-10T22:05:09.408904+0000","last_fullsized":"2024-09-10T22:05:09.408904+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:26.579313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888786+0000","last_change":"2024-09-10T22:05:08.422143+0000","last_active":"2024-09-10T22:05:10.888786+0000","last_peered":"2024-09-10T22:05:10.888786+0000","last_clean":"2024-09-10T22:05:10.888786+0000","last_became_active":"2024-09-10T22:05:08.421892+0000","last_became_peered":"2024-09-10T22:05:08.421892+0000","last_unstale":"2024-09-10T22:05:10.888786+0000","last_undegraded":"2024-09-10T22:05:10.888786+0000","last_fullsized":"2024-09-10T22:05:10.888786+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:47.507835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.405069+0000","last_change":"2024-09-10T22:05:08.425476+0000","last_active":"2024-09-10T22:05:09.405069+0000","last_peered":"2024-09-10T22:05:09.405069+0000","last_clean":"2024-09-10T22:05:09.405069+0000","last_became_active":"2024-09-10T22:05:08.425070+0000","last_became_peered":"2024-09-10T22:05:08.425070+0000","last_unstale":"2024-09-10T22:05:09.405069+0000","last_undegraded":"2024-09-10T22:05:09.405069+0000","last_fullsized":"2024-09-10T22:05:09.405069+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:00.368677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424757+0000","last_change":"2024-09-10T22:05:08.435501+0000","last_active":"2024-09-10T22:05:09.424757+0000","last_peered":"2024-09-10T22:05:09.424757+0000","last_clean":"2024-09-10T22:05:09.424757+0000","last_became_active":"2024-09-10T22:05:08.435248+0000","last_became_peered":"2024-09-10T22:05:08.435248+0000","last_unstale":"2024-09-10T22:05:09.424757+0000","last_undegraded":"2024-09-10T22:05:09.424757+0000","last_fullsized":"2024-09-10T22:05:09.424757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:50.499810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365357+0000","last_change":"2024-09-10T22:05:08.422950+0000","last_active":"2024-09-10T22:05:10.365357+0000","last_peered":"2024-09-10T22:05:10.365357+0000","last_clean":"2024-09-10T22:05:10.365357+0000","last_became_active":"2024-09-10T22:05:08.422237+0000","last_became_peered":"2024-09-10T22:05:08.422237+0000","last_unstale":"2024-09-10T22:05:10.365357+0000","last_undegraded":"2024-09-10T22:05:10.365357+0000","last_fullsized":"2024-09-10T22:05:10.365357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:10.588988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970016+0000","last_change":"2024-09-10T22:05:08.432249+0000","last_active":"2024-09-10T22:05:09.970016+0000","last_peered":"2024-09-10T22:05:09.970016+0000","last_clean":"2024-09-10T22:05:09.970016+0000","last_became_active":"2024-09-10T22:05:08.431958+0000","last_became_peered":"2024-09-10T22:05:08.431958+0000","last_unstale":"2024-09-10T22:05:09.970016+0000","last_undegraded":"2024-09-10T22:05:09.970016+0000","last_fullsized":"2024-09-10T22:05:09.970016+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:43.926790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365518+0000","last_change":"2024-09-10T22:05:08.419155+0000","last_active":"2024-09-10T22:05:10.365518+0000","last_peered":"2024-09-10T22:05:10.365518+0000","last_clean":"2024-09-10T22:05:10.365518+0000","last_became_active":"2024-09-10T22:05:08.418403+0000","last_became_peered":"2024-09-10T22:05:08.418403+0000","last_unstale":"2024-09-10T22:05:10.365518+0000","last_undegraded":"2024-09-10T22:05:10.365518+0000","last_fullsized":"2024-09-10T22:05:10.365518+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:50.821502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.886291+0000","last_change":"2024-09-10T22:05:08.411950+0000","last_active":"2024-09-10T22:05:10.886291+0000","last_peered":"2024-09-10T22:05:10.886291+0000","last_clean":"2024-09-10T22:05:10.886291+0000","last_became_active":"2024-09-10T22:05:08.411656+0000","last_became_peered":"2024-09-10T22:05:08.411656+0000","last_unstale":"2024-09-10T22:05:10.886291+0000","last_undegraded":"2024-09-10T22:05:10.886291+0000","last_fullsized":"2024-09-10T22:05:10.886291+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:18.751838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.886264+0000","last_change":"2024-09-10T22:05:08.442199+0000","last_active":"2024-09-10T22:05:10.886264+0000","last_peered":"2024-09-10T22:05:10.886264+0000","last_clean":"2024-09-10T22:05:10.886264+0000","last_became_active":"2024-09-10T22:05:08.440601+0000","last_became_peered":"2024-09-10T22:05:08.440601+0000","last_unstale":"2024-09-10T22:05:10.886264+0000","last_undegraded":"2024-09-10T22:05:10.886264+0000","last_fullsized":"2024-09-10T22:05:10.886264+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:33.457724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888874+0000","last_change":"2024-09-10T22:05:08.432039+0000","last_active":"2024-09-10T22:05:10.888874+0000","last_peered":"2024-09-10T22:05:10.888874+0000","last_clean":"2024-09-10T22:05:10.888874+0000","last_became_active":"2024-09-10T22:05:08.431778+0000","last_became_peered":"2024-09-10T22:05:08.431778+0000","last_unstale":"2024-09-10T22:05:10.888874+0000","last_undegraded":"2024-09-10T22:05:10.888874+0000","last_fullsized":"2024-09-10T22:05:10.888874+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:40.012819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409697+0000","last_change":"2024-09-10T22:05:08.420996+0000","last_active":"2024-09-10T22:05:09.409697+0000","last_peered":"2024-09-10T22:05:09.409697+0000","last_clean":"2024-09-10T22:05:09.409697+0000","last_became_active":"2024-09-10T22:05:08.420728+0000","last_became_peered":"2024-09-10T22:05:08.420728+0000","last_unstale":"2024-09-10T22:05:09.409697+0000","last_undegraded":"2024-09-10T22:05:09.409697+0000","last_fullsized":"2024-09-10T22:05:09.409697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:13.055070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365465+0000","last_change":"2024-09-10T22:05:08.437083+0000","last_active":"2024-09-10T22:05:10.365465+0000","last_peered":"2024-09-10T22:05:10.365465+0000","last_clean":"2024-09-10T22:05:10.365465+0000","last_became_active":"2024-09-10T22:05:08.435695+0000","last_became_peered":"2024-09-10T22:05:08.435695+0000","last_unstale":"2024-09-10T22:05:10.365465+0000","last_undegraded":"2024-09-10T22:05:10.365465+0000","last_fullsized":"2024-09-10T22:05:10.365465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:21.905402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365479+0000","last_change":"2024-09-10T22:05:08.418640+0000","last_active":"2024-09-10T22:05:10.365479+0000","last_peered":"2024-09-10T22:05:10.365479+0000","last_clean":"2024-09-10T22:05:10.365479+0000","last_became_active":"2024-09-10T22:05:08.418331+0000","last_became_peered":"2024-09-10T22:05:08.418331+0000","last_unstale":"2024-09-10T22:05:10.365479+0000","last_undegraded":"2024-09-10T22:05:10.365479+0000","last_fullsized":"2024-09-10T22:05:10.365479+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:50.375916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424458+0000","last_change":"2024-09-10T22:05:08.414602+0000","last_active":"2024-09-10T22:05:09.424458+0000","last_peered":"2024-09-10T22:05:09.424458+0000","last_clean":"2024-09-10T22:05:09.424458+0000","last_became_active":"2024-09-10T22:05:08.414267+0000","last_became_peered":"2024-09-10T22:05:08.414267+0000","last_unstale":"2024-09-10T22:05:09.424458+0000","last_undegraded":"2024-09-10T22:05:09.424458+0000","last_fullsized":"2024-09-10T22:05:09.424458+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:47.779046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970177+0000","last_change":"2024-09-10T22:05:08.436896+0000","last_active":"2024-09-10T22:05:09.970177+0000","last_peered":"2024-09-10T22:05:09.970177+0000","last_clean":"2024-09-10T22:05:09.970177+0000","last_became_active":"2024-09-10T22:05:08.433497+0000","last_became_peered":"2024-09-10T22:05:08.433497+0000","last_unstale":"2024-09-10T22:05:09.970177+0000","last_undegraded":"2024-09-10T22:05:09.970177+0000","last_fullsized":"2024-09-10T22:05:09.970177+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:32.070331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408628+0000","last_change":"2024-09-10T22:05:08.426905+0000","last_active":"2024-09-10T22:05:09.408628+0000","last_peered":"2024-09-10T22:05:09.408628+0000","last_clean":"2024-09-10T22:05:09.408628+0000","last_became_active":"2024-09-10T22:05:08.426639+0000","last_became_peered":"2024-09-10T22:05:08.426639+0000","last_unstale":"2024-09-10T22:05:09.408628+0000","last_undegraded":"2024-09-10T22:05:09.408628+0000","last_fullsized":"2024-09-10T22:05:09.408628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:43.595579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.405398+0000","last_change":"2024-09-10T22:05:08.431297+0000","last_active":"2024-09-10T22:05:09.405398+0000","last_peered":"2024-09-10T22:05:09.405398+0000","last_clean":"2024-09-10T22:05:09.405398+0000","last_became_active":"2024-09-10T22:05:08.431009+0000","last_became_peered":"2024-09-10T22:05:08.431009+0000","last_unstale":"2024-09-10T22:05:09.405398+0000","last_undegraded":"2024-09-10T22:05:09.405398+0000","last_fullsized":"2024-09-10T22:05:09.405398+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:18.655804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969937+0000","last_change":"2024-09-10T22:05:08.439973+0000","last_active":"2024-09-10T22:05:09.969937+0000","last_peered":"2024-09-10T22:05:09.969937+0000","last_clean":"2024-09-10T22:05:09.969937+0000","last_became_active":"2024-09-10T22:05:08.439696+0000","last_became_peered":"2024-09-10T22:05:08.439696+0000","last_unstale":"2024-09-10T22:05:09.969937+0000","last_undegraded":"2024-09-10T22:05:09.969937+0000","last_fullsized":"2024-09-10T22:05:09.969937+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:07.676722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408210+0000","last_change":"2024-09-10T22:05:08.427728+0000","last_active":"2024-09-10T22:05:09.408210+0000","last_peered":"2024-09-10T22:05:09.408210+0000","last_clean":"2024-09-10T22:05:09.408210+0000","last_became_active":"2024-09-10T22:05:08.427333+0000","last_became_peered":"2024-09-10T22:05:08.427333+0000","last_unstale":"2024-09-10T22:05:09.408210+0000","last_undegraded":"2024-09-10T22:05:09.408210+0000","last_fullsized":"2024-09-10T22:05:09.408210+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:34.195763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969819+0000","last_change":"2024-09-10T22:05:08.444049+0000","last_active":"2024-09-10T22:05:09.969819+0000","last_peered":"2024-09-10T22:05:09.969819+0000","last_clean":"2024-09-10T22:05:09.969819+0000","last_became_active":"2024-09-10T22:05:08.443758+0000","last_became_peered":"2024-09-10T22:05:08.443758+0000","last_unstale":"2024-09-10T22:05:09.969819+0000","last_undegraded":"2024-09-10T22:05:09.969819+0000","last_fullsized":"2024-09-10T22:05:09.969819+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:50.298849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408785+0000","last_change":"2024-09-10T22:05:08.429645+0000","last_active":"2024-09-10T22:05:09.408785+0000","last_peered":"2024-09-10T22:05:09.408785+0000","last_clean":"2024-09-10T22:05:09.408785+0000","last_became_active":"2024-09-10T22:05:08.429361+0000","last_became_peered":"2024-09-10T22:05:08.429361+0000","last_unstale":"2024-09-10T22:05:09.408785+0000","last_undegraded":"2024-09-10T22:05:09.408785+0000","last_fullsized":"2024-09-10T22:05:09.408785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:52.783597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409473+0000","last_change":"2024-09-10T22:05:08.419915+0000","last_active":"2024-09-10T22:05:09.409473+0000","last_peered":"2024-09-10T22:05:09.409473+0000","last_clean":"2024-09-10T22:05:09.409473+0000","last_became_active":"2024-09-10T22:05:08.419643+0000","last_became_peered":"2024-09-10T22:05:08.419643+0000","last_unstale":"2024-09-10T22:05:09.409473+0000","last_undegraded":"2024-09-10T22:05:09.409473+0000","last_fullsized":"2024-09-10T22:05:09.409473+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:09.972327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.405578+0000","last_change":"2024-09-10T22:05:08.414753+0000","last_active":"2024-09-10T22:05:09.405578+0000","last_peered":"2024-09-10T22:05:09.405578+0000","last_clean":"2024-09-10T22:05:09.405578+0000","last_became_active":"2024-09-10T22:05:08.414447+0000","last_became_peered":"2024-09-10T22:05:08.414447+0000","last_unstale":"2024-09-10T22:05:09.405578+0000","last_undegraded":"2024-09-10T22:05:09.405578+0000","last_fullsized":"2024-09-10T22:05:09.405578+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:25.991515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888892+0000","last_change":"2024-09-10T22:05:08.426140+0000","last_active":"2024-09-10T22:05:10.888892+0000","last_peered":"2024-09-10T22:05:10.888892+0000","last_clean":"2024-09-10T22:05:10.888892+0000","last_became_active":"2024-09-10T22:05:08.425739+0000","last_became_peered":"2024-09-10T22:05:08.425739+0000","last_unstale":"2024-09-10T22:05:10.888892+0000","last_undegraded":"2024-09-10T22:05:10.888892+0000","last_fullsized":"2024-09-10T22:05:10.888892+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:02.025058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425065+0000","last_change":"2024-09-10T22:05:08.438832+0000","last_active":"2024-09-10T22:05:09.425065+0000","last_peered":"2024-09-10T22:05:09.425065+0000","last_clean":"2024-09-10T22:05:09.425065+0000","last_became_active":"2024-09-10T22:05:08.438638+0000","last_became_peered":"2024-09-10T22:05:08.438638+0000","last_unstale":"2024-09-10T22:05:09.425065+0000","last_undegraded":"2024-09-10T22:05:09.425065+0000","last_fullsized":"2024-09-10T22:05:09.425065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:44.863761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.405327+0000","last_change":"2024-09-10T22:05:08.418899+0000","last_active":"2024-09-10T22:05:09.405327+0000","last_peered":"2024-09-10T22:05:09.405327+0000","last_clean":"2024-09-10T22:05:09.405327+0000","last_became_active":"2024-09-10T22:05:08.418569+0000","last_became_peered":"2024-09-10T22:05:08.418569+0000","last_unstale":"2024-09-10T22:05:09.405327+0000","last_undegraded":"2024-09-10T22:05:09.405327+0000","last_fullsized":"2024-09-10T22:05:09.405327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:29.450972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970350+0000","last_change":"2024-09-10T22:05:08.430551+0000","last_active":"2024-09-10T22:05:09.970350+0000","last_peered":"2024-09-10T22:05:09.970350+0000","last_clean":"2024-09-10T22:05:09.970350+0000","last_became_active":"2024-09-10T22:05:08.426896+0000","last_became_peered":"2024-09-10T22:05:08.426896+0000","last_unstale":"2024-09-10T22:05:09.970350+0000","last_undegraded":"2024-09-10T22:05:09.970350+0000","last_fullsized":"2024-09-10T22:05:09.970350+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:04.278385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408197+0000","last_change":"2024-09-10T22:05:08.426923+0000","last_active":"2024-09-10T22:05:09.408197+0000","last_peered":"2024-09-10T22:05:09.408197+0000","last_clean":"2024-09-10T22:05:09.408197+0000","last_became_active":"2024-09-10T22:05:08.426596+0000","last_became_peered":"2024-09-10T22:05:08.426596+0000","last_unstale":"2024-09-10T22:05:09.408197+0000","last_undegraded":"2024-09-10T22:05:09.408197+0000","last_fullsized":"2024-09-10T22:05:09.408197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:28.155257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.405025+0000","last_change":"2024-09-10T22:05:08.411300+0000","last_active":"2024-09-10T22:05:09.405025+0000","last_peered":"2024-09-10T22:05:09.405025+0000","last_clean":"2024-09-10T22:05:09.405025+0000","last_became_active":"2024-09-10T22:05:08.410898+0000","last_became_peered":"2024-09-10T22:05:08.410898+0000","last_unstale":"2024-09-10T22:05:09.405025+0000","last_undegraded":"2024-09-10T22:05:09.405025+0000","last_fullsized":"2024-09-10T22:05:09.405025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:21.640184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424489+0000","last_change":"2024-09-10T22:05:08.430443+0000","last_active":"2024-09-10T22:05:09.424489+0000","last_peered":"2024-09-10T22:05:09.424489+0000","last_clean":"2024-09-10T22:05:09.424489+0000","last_became_active":"2024-09-10T22:05:08.430138+0000","last_became_peered":"2024-09-10T22:05:08.430138+0000","last_unstale":"2024-09-10T22:05:09.424489+0000","last_undegraded":"2024-09-10T22:05:09.424489+0000","last_fullsized":"2024-09-10T22:05:09.424489+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:48.436292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425267+0000","last_change":"2024-09-10T22:05:08.426171+0000","last_active":"2024-09-10T22:05:09.425267+0000","last_peered":"2024-09-10T22:05:09.425267+0000","last_clean":"2024-09-10T22:05:09.425267+0000","last_became_active":"2024-09-10T22:05:08.425870+0000","last_became_peered":"2024-09-10T22:05:08.425870+0000","last_unstale":"2024-09-10T22:05:09.425267+0000","last_undegraded":"2024-09-10T22:05:09.425267+0000","last_fullsized":"2024-09-10T22:05:09.425267+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:35.925364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425253+0000","last_change":"2024-09-10T22:05:07.413946+0000","last_active":"2024-09-10T22:05:09.425253+0000","last_peered":"2024-09-10T22:05:09.425253+0000","last_clean":"2024-09-10T22:05:09.425253+0000","last_became_active":"2024-09-10T22:05:07.413670+0000","last_became_peered":"2024-09-10T22:05:07.413670+0000","last_unstale":"2024-09-10T22:05:09.425253+0000","last_undegraded":"2024-09-10T22:05:09.425253+0000","last_fullsized":"2024-09-10T22:05:09.425253+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:51.779195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.886388+0000","last_change":"2024-09-10T22:05:08.429659+0000","last_active":"2024-09-10T22:05:10.886388+0000","last_peered":"2024-09-10T22:05:10.886388+0000","last_clean":"2024-09-10T22:05:10.886388+0000","last_became_active":"2024-09-10T22:05:08.429509+0000","last_became_peered":"2024-09-10T22:05:08.429509+0000","last_unstale":"2024-09-10T22:05:10.886388+0000","last_undegraded":"2024-09-10T22:05:10.886388+0000","last_fullsized":"2024-09-10T22:05:10.886388+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:27.181185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969952+0000","last_change":"2024-09-10T22:05:07.415031+0000","last_active":"2024-09-10T22:05:09.969952+0000","last_peered":"2024-09-10T22:05:09.969952+0000","last_clean":"2024-09-10T22:05:09.969952+0000","last_became_active":"2024-09-10T22:05:07.414707+0000","last_became_peered":"2024-09-10T22:05:07.414707+0000","last_unstale":"2024-09-10T22:05:09.969952+0000","last_undegraded":"2024-09-10T22:05:09.969952+0000","last_fullsized":"2024-09-10T22:05:09.969952+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:33.928698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424994+0000","last_change":"2024-09-10T22:05:08.424487+0000","last_active":"2024-09-10T22:05:09.424994+0000","last_peered":"2024-09-10T22:05:09.424994+0000","last_clean":"2024-09-10T22:05:09.424994+0000","last_became_active":"2024-09-10T22:05:08.424205+0000","last_became_peered":"2024-09-10T22:05:08.424205+0000","last_unstale":"2024-09-10T22:05:09.424994+0000","last_undegraded":"2024-09-10T22:05:09.424994+0000","last_fullsized":"2024-09-10T22:05:09.424994+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:18.056158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425143+0000","last_change":"2024-09-10T22:05:07.410708+0000","last_active":"2024-09-10T22:05:09.425143+0000","last_peered":"2024-09-10T22:05:09.425143+0000","last_clean":"2024-09-10T22:05:09.425143+0000","last_became_active":"2024-09-10T22:05:07.410249+0000","last_became_peered":"2024-09-10T22:05:07.410249+0000","last_unstale":"2024-09-10T22:05:09.425143+0000","last_undegraded":"2024-09-10T22:05:09.425143+0000","last_fullsized":"2024-09-10T22:05:09.425143+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:45.733933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425058+0000","last_change":"2024-09-10T22:05:08.424434+0000","last_active":"2024-09-10T22:05:09.425058+0000","last_peered":"2024-09-10T22:05:09.425058+0000","last_clean":"2024-09-10T22:05:09.425058+0000","last_became_active":"2024-09-10T22:05:08.423950+0000","last_became_peered":"2024-09-10T22:05:08.423950+0000","last_unstale":"2024-09-10T22:05:09.425058+0000","last_undegraded":"2024-09-10T22:05:09.425058+0000","last_fullsized":"2024-09-10T22:05:09.425058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:31.151424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888835+0000","last_change":"2024-09-10T22:05:07.416356+0000","last_active":"2024-09-10T22:05:10.888835+0000","last_peered":"2024-09-10T22:05:10.888835+0000","last_clean":"2024-09-10T22:05:10.888835+0000","last_became_active":"2024-09-10T22:05:07.415854+0000","last_became_peered":"2024-09-10T22:05:07.415854+0000","last_unstale":"2024-09-10T22:05:10.888835+0000","last_undegraded":"2024-09-10T22:05:10.888835+0000","last_fullsized":"2024-09-10T22:05:10.888835+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:20.901000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969847+0000","last_change":"2024-09-10T22:05:08.443704+0000","last_active":"2024-09-10T22:05:09.969847+0000","last_peered":"2024-09-10T22:05:09.969847+0000","last_clean":"2024-09-10T22:05:09.969847+0000","last_became_active":"2024-09-10T22:05:08.443359+0000","last_became_peered":"2024-09-10T22:05:08.443359+0000","last_unstale":"2024-09-10T22:05:09.969847+0000","last_undegraded":"2024-09-10T22:05:09.969847+0000","last_fullsized":"2024-09-10T22:05:09.969847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:06.230381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365040+0000","last_change":"2024-09-10T22:05:07.419708+0000","last_active":"2024-09-10T22:05:10.365040+0000","last_peered":"2024-09-10T22:05:10.365040+0000","last_clean":"2024-09-10T22:05:10.365040+0000","last_became_active":"2024-09-10T22:05:07.419341+0000","last_became_peered":"2024-09-10T22:05:07.419341+0000","last_unstale":"2024-09-10T22:05:10.365040+0000","last_undegraded":"2024-09-10T22:05:10.365040+0000","last_fullsized":"2024-09-10T22:05:10.365040+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:08.796119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970297+0000","last_change":"2024-09-10T22:05:08.433589+0000","last_active":"2024-09-10T22:05:09.970297+0000","last_peered":"2024-09-10T22:05:09.970297+0000","last_clean":"2024-09-10T22:05:09.970297+0000","last_became_active":"2024-09-10T22:05:08.433325+0000","last_became_peered":"2024-09-10T22:05:08.433325+0000","last_unstale":"2024-09-10T22:05:09.970297+0000","last_undegraded":"2024-09-10T22:05:09.970297+0000","last_fullsized":"2024-09-10T22:05:09.970297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:59:17.109089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424479+0000","last_change":"2024-09-10T22:05:07.403621+0000","last_active":"2024-09-10T22:05:09.424479+0000","last_peered":"2024-09-10T22:05:09.424479+0000","last_clean":"2024-09-10T22:05:09.424479+0000","last_became_active":"2024-09-10T22:05:07.399475+0000","last_became_peered":"2024-09-10T22:05:07.399475+0000","last_unstale":"2024-09-10T22:05:09.424479+0000","last_undegraded":"2024-09-10T22:05:09.424479+0000","last_fullsized":"2024-09-10T22:05:09.424479+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:59.131489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409358+0000","last_change":"2024-09-10T22:05:08.426518+0000","last_active":"2024-09-10T22:05:09.409358+0000","last_peered":"2024-09-10T22:05:09.409358+0000","last_clean":"2024-09-10T22:05:09.409358+0000","last_became_active":"2024-09-10T22:05:08.424601+0000","last_became_peered":"2024-09-10T22:05:08.424601+0000","last_unstale":"2024-09-10T22:05:09.409358+0000","last_undegraded":"2024-09-10T22:05:09.409358+0000","last_fullsized":"2024-09-10T22:05:09.409358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:27.181185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888418+0000","last_change":"2024-09-10T22:05:07.414884+0000","last_active":"2024-09-10T22:05:10.888418+0000","last_peered":"2024-09-10T22:05:10.888418+0000","last_clean":"2024-09-10T22:05:10.888418+0000","last_became_active":"2024-09-10T22:05:07.414552+0000","last_became_peered":"2024-09-10T22:05:07.414552+0000","last_unstale":"2024-09-10T22:05:10.888418+0000","last_undegraded":"2024-09-10T22:05:10.888418+0000","last_fullsized":"2024-09-10T22:05:10.888418+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:09.584585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969883+0000","last_change":"2024-09-10T22:05:08.438977+0000","last_active":"2024-09-10T22:05:09.969883+0000","last_peered":"2024-09-10T22:05:09.969883+0000","last_clean":"2024-09-10T22:05:09.969883+0000","last_became_active":"2024-09-10T22:05:08.437107+0000","last_became_peered":"2024-09-10T22:05:08.437107+0000","last_unstale":"2024-09-10T22:05:09.969883+0000","last_undegraded":"2024-09-10T22:05:09.969883+0000","last_fullsized":"2024-09-10T22:05:09.969883+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:47.278493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.364967+0000","last_change":"2024-09-10T22:05:07.419662+0000","last_active":"2024-09-10T22:05:10.364967+0000","last_peered":"2024-09-10T22:05:10.364967+0000","last_clean":"2024-09-10T22:05:10.364967+0000","last_became_active":"2024-09-10T22:05:07.419312+0000","last_became_peered":"2024-09-10T22:05:07.419312+0000","last_unstale":"2024-09-10T22:05:10.364967+0000","last_undegraded":"2024-09-10T22:05:10.364967+0000","last_fullsized":"2024-09-10T22:05:10.364967+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:13.991036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365674+0000","last_change":"2024-09-10T22:05:08.418649+0000","last_active":"2024-09-10T22:05:10.365674+0000","last_peered":"2024-09-10T22:05:10.365674+0000","last_clean":"2024-09-10T22:05:10.365674+0000","last_became_active":"2024-09-10T22:05:08.418343+0000","last_became_peered":"2024-09-10T22:05:08.418343+0000","last_unstale":"2024-09-10T22:05:10.365674+0000","last_undegraded":"2024-09-10T22:05:10.365674+0000","last_fullsized":"2024-09-10T22:05:10.365674+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:04.397416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409023+0000","last_change":"2024-09-10T22:05:07.418011+0000","last_active":"2024-09-10T22:05:09.409023+0000","last_peered":"2024-09-10T22:05:09.409023+0000","last_clean":"2024-09-10T22:05:09.409023+0000","last_became_active":"2024-09-10T22:05:07.417671+0000","last_became_peered":"2024-09-10T22:05:07.417671+0000","last_unstale":"2024-09-10T22:05:09.409023+0000","last_undegraded":"2024-09-10T22:05:09.409023+0000","last_fullsized":"2024-09-10T22:05:09.409023+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:07.953831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409450+0000","last_change":"2024-09-10T22:05:08.438782+0000","last_active":"2024-09-10T22:05:09.409450+0000","last_peered":"2024-09-10T22:05:09.409450+0000","last_clean":"2024-09-10T22:05:09.409450+0000","last_became_active":"2024-09-10T22:05:08.438485+0000","last_became_peered":"2024-09-10T22:05:08.438485+0000","last_unstale":"2024-09-10T22:05:09.409450+0000","last_undegraded":"2024-09-10T22:05:09.409450+0000","last_fullsized":"2024-09-10T22:05:09.409450+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:48.436292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970091+0000","last_change":"2024-09-10T22:05:07.417628+0000","last_active":"2024-09-10T22:05:09.970091+0000","last_peered":"2024-09-10T22:05:09.970091+0000","last_clean":"2024-09-10T22:05:09.970091+0000","last_became_active":"2024-09-10T22:05:07.417352+0000","last_became_peered":"2024-09-10T22:05:07.417352+0000","last_unstale":"2024-09-10T22:05:09.970091+0000","last_undegraded":"2024-09-10T22:05:09.970091+0000","last_fullsized":"2024-09-10T22:05:09.970091+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:45.328003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.886023+0000","last_change":"2024-09-10T22:05:08.409963+0000","last_active":"2024-09-10T22:05:10.886023+0000","last_peered":"2024-09-10T22:05:10.886023+0000","last_clean":"2024-09-10T22:05:10.886023+0000","last_became_active":"2024-09-10T22:05:08.409680+0000","last_became_peered":"2024-09-10T22:05:08.409680+0000","last_unstale":"2024-09-10T22:05:10.886023+0000","last_undegraded":"2024-09-10T22:05:10.886023+0000","last_fullsized":"2024-09-10T22:05:10.886023+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:28.279917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409153+0000","last_change":"2024-09-10T22:05:07.418919+0000","last_active":"2024-09-10T22:05:09.409153+0000","last_peered":"2024-09-10T22:05:09.409153+0000","last_clean":"2024-09-10T22:05:09.409153+0000","last_became_active":"2024-09-10T22:05:07.418461+0000","last_became_peered":"2024-09-10T22:05:07.418461+0000","last_unstale":"2024-09-10T22:05:09.409153+0000","last_undegraded":"2024-09-10T22:05:09.409153+0000","last_fullsized":"2024-09-10T22:05:09.409153+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:30.637816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409089+0000","last_change":"2024-09-10T22:05:08.433901+0000","last_active":"2024-09-10T22:05:09.409089+0000","last_peered":"2024-09-10T22:05:09.409089+0000","last_clean":"2024-09-10T22:05:09.409089+0000","last_became_active":"2024-09-10T22:05:08.433623+0000","last_became_peered":"2024-09-10T22:05:08.433623+0000","last_unstale":"2024-09-10T22:05:09.409089+0000","last_undegraded":"2024-09-10T22:05:09.409089+0000","last_fullsized":"2024-09-10T22:05:09.409089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:02.025058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":21,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365352+0000","last_change":"2024-09-10T22:05:07.416575+0000","last_active":"2024-09-10T22:05:10.365352+0000","last_peered":"2024-09-10T22:05:10.365352+0000","last_clean":"2024-09-10T22:05:10.365352+0000","last_became_active":"2024-09-10T22:05:07.416319+0000","last_became_peered":"2024-09-10T22:05:07.416319+0000","last_unstale":"2024-09-10T22:05:10.365352+0000","last_undegraded":"2024-09-10T22:05:10.365352+0000","last_fullsized":"2024-09-10T22:05:10.365352+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:39.277370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888524+0000","last_change":"2024-09-10T22:05:08.427480+0000","last_active":"2024-09-10T22:05:10.888524+0000","last_peered":"2024-09-10T22:05:10.888524+0000","last_clean":"2024-09-10T22:05:10.888524+0000","last_became_active":"2024-09-10T22:05:08.427229+0000","last_became_peered":"2024-09-10T22:05:08.427229+0000","last_unstale":"2024-09-10T22:05:10.888524+0000","last_undegraded":"2024-09-10T22:05:10.888524+0000","last_fullsized":"2024-09-10T22:05:10.888524+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:56.408442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969707+0000","last_change":"2024-09-10T22:05:07.419646+0000","last_active":"2024-09-10T22:05:09.969707+0000","last_peered":"2024-09-10T22:05:09.969707+0000","last_clean":"2024-09-10T22:05:09.969707+0000","last_became_active":"2024-09-10T22:05:07.419363+0000","last_became_peered":"2024-09-10T22:05:07.419363+0000","last_unstale":"2024-09-10T22:05:09.969707+0000","last_undegraded":"2024-09-10T22:05:09.969707+0000","last_fullsized":"2024-09-10T22:05:09.969707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:55.404040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408308+0000","last_change":"2024-09-10T22:05:08.432440+0000","last_active":"2024-09-10T22:05:09.408308+0000","last_peered":"2024-09-10T22:05:09.408308+0000","last_clean":"2024-09-10T22:05:09.408308+0000","last_became_active":"2024-09-10T22:05:08.432165+0000","last_became_peered":"2024-09-10T22:05:08.432165+0000","last_unstale":"2024-09-10T22:05:09.408308+0000","last_undegraded":"2024-09-10T22:05:09.408308+0000","last_fullsized":"2024-09-10T22:05:09.408308+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:29.443096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.407958+0000","last_change":"2024-09-10T22:05:07.408274+0000","last_active":"2024-09-10T22:05:09.407958+0000","last_peered":"2024-09-10T22:05:09.407958+0000","last_clean":"2024-09-10T22:05:09.407958+0000","last_became_active":"2024-09-10T22:05:07.407846+0000","last_became_peered":"2024-09-10T22:05:07.407846+0000","last_unstale":"2024-09-10T22:05:09.407958+0000","last_undegraded":"2024-09-10T22:05:09.407958+0000","last_fullsized":"2024-09-10T22:05:09.407958+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:54.365569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365432+0000","last_change":"2024-09-10T22:05:08.423464+0000","last_active":"2024-09-10T22:05:10.365432+0000","last_peered":"2024-09-10T22:05:10.365432+0000","last_clean":"2024-09-10T22:05:10.365432+0000","last_became_active":"2024-09-10T22:05:08.423195+0000","last_became_peered":"2024-09-10T22:05:08.423195+0000","last_unstale":"2024-09-10T22:05:10.365432+0000","last_undegraded":"2024-09-10T22:05:10.365432+0000","last_fullsized":"2024-09-10T22:05:10.365432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:04.561865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888416+0000","last_change":"2024-09-10T22:05:07.415756+0000","last_active":"2024-09-10T22:05:10.888416+0000","last_peered":"2024-09-10T22:05:10.888416+0000","last_clean":"2024-09-10T22:05:10.888416+0000","last_became_active":"2024-09-10T22:05:07.415453+0000","last_became_peered":"2024-09-10T22:05:07.415453+0000","last_unstale":"2024-09-10T22:05:10.888416+0000","last_undegraded":"2024-09-10T22:05:10.888416+0000","last_fullsized":"2024-09-10T22:05:10.888416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:03.557463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408838+0000","last_change":"2024-09-10T22:05:08.431025+0000","last_active":"2024-09-10T22:05:09.408838+0000","last_peered":"2024-09-10T22:05:09.408838+0000","last_clean":"2024-09-10T22:05:09.408838+0000","last_became_active":"2024-09-10T22:05:08.430727+0000","last_became_peered":"2024-09-10T22:05:08.430727+0000","last_unstale":"2024-09-10T22:05:09.408838+0000","last_undegraded":"2024-09-10T22:05:09.408838+0000","last_fullsized":"2024-09-10T22:05:09.408838+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:59.609250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408287+0000","last_change":"2024-09-10T22:05:07.416172+0000","last_active":"2024-09-10T22:05:09.408287+0000","last_peered":"2024-09-10T22:05:09.408287+0000","last_clean":"2024-09-10T22:05:09.408287+0000","last_became_active":"2024-09-10T22:05:07.415901+0000","last_became_peered":"2024-09-10T22:05:07.415901+0000","last_unstale":"2024-09-10T22:05:09.408287+0000","last_undegraded":"2024-09-10T22:05:09.408287+0000","last_fullsized":"2024-09-10T22:05:09.408287+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:55.370220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409277+0000","last_change":"2024-09-10T22:05:08.429545+0000","last_active":"2024-09-10T22:05:09.409277+0000","last_peered":"2024-09-10T22:05:09.409277+0000","last_clean":"2024-09-10T22:05:09.409277+0000","last_became_active":"2024-09-10T22:05:08.429265+0000","last_became_peered":"2024-09-10T22:05:08.429265+0000","last_unstale":"2024-09-10T22:05:09.409277+0000","last_undegraded":"2024-09-10T22:05:09.409277+0000","last_fullsized":"2024-09-10T22:05:09.409277+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:47.779046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970071+0000","last_change":"2024-09-10T22:05:07.416924+0000","last_active":"2024-09-10T22:05:09.970071+0000","last_peered":"2024-09-10T22:05:09.970071+0000","last_clean":"2024-09-10T22:05:09.970071+0000","last_became_active":"2024-09-10T22:05:07.416672+0000","last_became_peered":"2024-09-10T22:05:07.416672+0000","last_unstale":"2024-09-10T22:05:09.970071+0000","last_undegraded":"2024-09-10T22:05:09.970071+0000","last_fullsized":"2024-09-10T22:05:09.970071+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:43.859359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.889205+0000","last_change":"2024-09-10T22:05:08.423758+0000","last_active":"2024-09-10T22:05:10.889205+0000","last_peered":"2024-09-10T22:05:10.889205+0000","last_clean":"2024-09-10T22:05:10.889205+0000","last_became_active":"2024-09-10T22:05:08.423436+0000","last_became_peered":"2024-09-10T22:05:08.423436+0000","last_unstale":"2024-09-10T22:05:10.889205+0000","last_undegraded":"2024-09-10T22:05:10.889205+0000","last_fullsized":"2024-09-10T22:05:10.889205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:48.436292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425095+0000","last_change":"2024-09-10T22:05:07.411669+0000","last_active":"2024-09-10T22:05:09.425095+0000","last_peered":"2024-09-10T22:05:09.425095+0000","last_clean":"2024-09-10T22:05:09.425095+0000","last_became_active":"2024-09-10T22:05:07.411198+0000","last_became_peered":"2024-09-10T22:05:07.411198+0000","last_unstale":"2024-09-10T22:05:09.425095+0000","last_undegraded":"2024-09-10T22:05:09.425095+0000","last_fullsized":"2024-09-10T22:05:09.425095+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:30.408615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970202+0000","last_change":"2024-09-10T22:05:08.431947+0000","last_active":"2024-09-10T22:05:09.970202+0000","last_peered":"2024-09-10T22:05:09.970202+0000","last_clean":"2024-09-10T22:05:09.970202+0000","last_became_active":"2024-09-10T22:05:08.431577+0000","last_became_peered":"2024-09-10T22:05:08.431577+0000","last_unstale":"2024-09-10T22:05:09.970202+0000","last_undegraded":"2024-09-10T22:05:09.970202+0000","last_fullsized":"2024-09-10T22:05:09.970202+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:29.580261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888720+0000","last_change":"2024-09-10T22:05:07.415938+0000","last_active":"2024-09-10T22:05:10.888720+0000","last_peered":"2024-09-10T22:05:10.888720+0000","last_clean":"2024-09-10T22:05:10.888720+0000","last_became_active":"2024-09-10T22:05:07.415488+0000","last_became_peered":"2024-09-10T22:05:07.415488+0000","last_unstale":"2024-09-10T22:05:10.888720+0000","last_undegraded":"2024-09-10T22:05:10.888720+0000","last_fullsized":"2024-09-10T22:05:10.888720+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:03.393013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409156+0000","last_change":"2024-09-10T22:05:08.432494+0000","last_active":"2024-09-10T22:05:09.409156+0000","last_peered":"2024-09-10T22:05:09.409156+0000","last_clean":"2024-09-10T22:05:09.409156+0000","last_became_active":"2024-09-10T22:05:08.432185+0000","last_became_peered":"2024-09-10T22:05:08.432185+0000","last_unstale":"2024-09-10T22:05:09.409156+0000","last_undegraded":"2024-09-10T22:05:09.409156+0000","last_fullsized":"2024-09-10T22:05:09.409156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:23.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424834+0000","last_change":"2024-09-10T22:05:07.413141+0000","last_active":"2024-09-10T22:05:09.424834+0000","last_peered":"2024-09-10T22:05:09.424834+0000","last_clean":"2024-09-10T22:05:09.424834+0000","last_became_active":"2024-09-10T22:05:07.412606+0000","last_became_peered":"2024-09-10T22:05:07.412606+0000","last_unstale":"2024-09-10T22:05:09.424834+0000","last_undegraded":"2024-09-10T22:05:09.424834+0000","last_fullsized":"2024-09-10T22:05:09.424834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:53.246171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888547+0000","last_change":"2024-09-10T22:05:08.426817+0000","last_active":"2024-09-10T22:05:10.888547+0000","last_peered":"2024-09-10T22:05:10.888547+0000","last_clean":"2024-09-10T22:05:10.888547+0000","last_became_active":"2024-09-10T22:05:08.426537+0000","last_became_peered":"2024-09-10T22:05:08.426537+0000","last_unstale":"2024-09-10T22:05:10.888547+0000","last_undegraded":"2024-09-10T22:05:10.888547+0000","last_fullsized":"2024-09-10T22:05:10.888547+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:29.405607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970341+0000","last_change":"2024-09-10T22:05:07.412724+0000","last_active":"2024-09-10T22:05:09.970341+0000","last_peered":"2024-09-10T22:05:09.970341+0000","last_clean":"2024-09-10T22:05:09.970341+0000","last_became_active":"2024-09-10T22:05:07.412235+0000","last_became_peered":"2024-09-10T22:05:07.412235+0000","last_unstale":"2024-09-10T22:05:09.970341+0000","last_undegraded":"2024-09-10T22:05:09.970341+0000","last_fullsized":"2024-09-10T22:05:09.970341+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:46.503432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424752+0000","last_change":"2024-09-10T22:05:08.437326+0000","last_active":"2024-09-10T22:05:09.424752+0000","last_peered":"2024-09-10T22:05:09.424752+0000","last_clean":"2024-09-10T22:05:09.424752+0000","last_became_active":"2024-09-10T22:05:08.436843+0000","last_became_peered":"2024-09-10T22:05:08.436843+0000","last_unstale":"2024-09-10T22:05:09.424752+0000","last_undegraded":"2024-09-10T22:05:09.424752+0000","last_fullsized":"2024-09-10T22:05:09.424752+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:34.933101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.885846+0000","last_change":"2024-09-10T22:05:07.419669+0000","last_active":"2024-09-10T22:05:10.885846+0000","last_peered":"2024-09-10T22:05:10.885846+0000","last_clean":"2024-09-10T22:05:10.885846+0000","last_became_active":"2024-09-10T22:05:07.419399+0000","last_became_peered":"2024-09-10T22:05:07.419399+0000","last_unstale":"2024-09-10T22:05:10.885846+0000","last_undegraded":"2024-09-10T22:05:10.885846+0000","last_fullsized":"2024-09-10T22:05:10.885846+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:35.612885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424517+0000","last_change":"2024-09-10T22:05:08.434182+0000","last_active":"2024-09-10T22:05:09.424517+0000","last_peered":"2024-09-10T22:05:09.424517+0000","last_clean":"2024-09-10T22:05:09.424517+0000","last_became_active":"2024-09-10T22:05:08.425682+0000","last_became_peered":"2024-09-10T22:05:08.425682+0000","last_unstale":"2024-09-10T22:05:09.424517+0000","last_undegraded":"2024-09-10T22:05:09.424517+0000","last_fullsized":"2024-09-10T22:05:09.424517+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:10.908283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888395+0000","last_change":"2024-09-10T22:05:07.421708+0000","last_active":"2024-09-10T22:05:10.888395+0000","last_peered":"2024-09-10T22:05:10.888395+0000","last_clean":"2024-09-10T22:05:10.888395+0000","last_became_active":"2024-09-10T22:05:07.421421+0000","last_became_peered":"2024-09-10T22:05:07.421421+0000","last_unstale":"2024-09-10T22:05:10.888395+0000","last_undegraded":"2024-09-10T22:05:10.888395+0000","last_fullsized":"2024-09-10T22:05:10.888395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:49.817099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.889099+0000","last_change":"2024-09-10T22:05:08.425626+0000","last_active":"2024-09-10T22:05:10.889099+0000","last_peered":"2024-09-10T22:05:10.889099+0000","last_clean":"2024-09-10T22:05:10.889099+0000","last_became_active":"2024-09-10T22:05:08.425333+0000","last_became_peered":"2024-09-10T22:05:08.425333+0000","last_unstale":"2024-09-10T22:05:10.889099+0000","last_undegraded":"2024-09-10T22:05:10.889099+0000","last_fullsized":"2024-09-10T22:05:10.889099+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:51.427056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.886309+0000","last_change":"2024-09-10T22:05:07.415631+0000","last_active":"2024-09-10T22:05:10.886309+0000","last_peered":"2024-09-10T22:05:10.886309+0000","last_clean":"2024-09-10T22:05:10.886309+0000","last_became_active":"2024-09-10T22:05:07.415345+0000","last_became_peered":"2024-09-10T22:05:07.415345+0000","last_unstale":"2024-09-10T22:05:10.886309+0000","last_undegraded":"2024-09-10T22:05:10.886309+0000","last_fullsized":"2024-09-10T22:05:10.886309+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:04.145998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.889209+0000","last_change":"2024-09-10T22:05:08.433734+0000","last_active":"2024-09-10T22:05:10.889209+0000","last_peered":"2024-09-10T22:05:10.889209+0000","last_clean":"2024-09-10T22:05:10.889209+0000","last_became_active":"2024-09-10T22:05:08.433451+0000","last_became_peered":"2024-09-10T22:05:08.433451+0000","last_unstale":"2024-09-10T22:05:10.889209+0000","last_undegraded":"2024-09-10T22:05:10.889209+0000","last_fullsized":"2024-09-10T22:05:10.889209+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:15.748248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425276+0000","last_change":"2024-09-10T22:05:07.400217+0000","last_active":"2024-09-10T22:05:09.425276+0000","last_peered":"2024-09-10T22:05:09.425276+0000","last_clean":"2024-09-10T22:05:09.425276+0000","last_became_active":"2024-09-10T22:05:07.399944+0000","last_became_peered":"2024-09-10T22:05:07.399944+0000","last_unstale":"2024-09-10T22:05:09.425276+0000","last_undegraded":"2024-09-10T22:05:09.425276+0000","last_fullsized":"2024-09-10T22:05:09.425276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:02.216373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.886051+0000","last_change":"2024-09-10T22:05:08.408878+0000","last_active":"2024-09-10T22:05:10.886051+0000","last_peered":"2024-09-10T22:05:10.886051+0000","last_clean":"2024-09-10T22:05:10.886051+0000","last_became_active":"2024-09-10T22:05:08.408599+0000","last_became_peered":"2024-09-10T22:05:08.408599+0000","last_unstale":"2024-09-10T22:05:10.886051+0000","last_undegraded":"2024-09-10T22:05:10.886051+0000","last_fullsized":"2024-09-10T22:05:10.886051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:57.172129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409585+0000","last_change":"2024-09-10T22:05:07.411290+0000","last_active":"2024-09-10T22:05:09.409585+0000","last_peered":"2024-09-10T22:05:09.409585+0000","last_clean":"2024-09-10T22:05:09.409585+0000","last_became_active":"2024-09-10T22:05:07.410412+0000","last_became_peered":"2024-09-10T22:05:07.410412+0000","last_unstale":"2024-09-10T22:05:09.409585+0000","last_undegraded":"2024-09-10T22:05:09.409585+0000","last_fullsized":"2024-09-10T22:05:09.409585+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:27.275515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425412+0000","last_change":"2024-09-10T22:05:08.424120+0000","last_active":"2024-09-10T22:05:09.425412+0000","last_peered":"2024-09-10T22:05:09.425412+0000","last_clean":"2024-09-10T22:05:09.425412+0000","last_became_active":"2024-09-10T22:05:08.423514+0000","last_became_peered":"2024-09-10T22:05:08.423514+0000","last_unstale":"2024-09-10T22:05:09.425412+0000","last_undegraded":"2024-09-10T22:05:09.425412+0000","last_fullsized":"2024-09-10T22:05:09.425412+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:02.039553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888904+0000","last_change":"2024-09-10T22:05:07.419449+0000","last_active":"2024-09-10T22:05:10.888904+0000","last_peered":"2024-09-10T22:05:10.888904+0000","last_clean":"2024-09-10T22:05:10.888904+0000","last_became_active":"2024-09-10T22:05:07.419092+0000","last_became_peered":"2024-09-10T22:05:07.419092+0000","last_unstale":"2024-09-10T22:05:10.888904+0000","last_undegraded":"2024-09-10T22:05:10.888904+0000","last_fullsized":"2024-09-10T22:05:10.888904+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:49.371514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970336+0000","last_change":"2024-09-10T22:05:08.430561+0000","last_active":"2024-09-10T22:05:09.970336+0000","last_peered":"2024-09-10T22:05:09.970336+0000","last_clean":"2024-09-10T22:05:09.970336+0000","last_became_active":"2024-09-10T22:05:08.427068+0000","last_became_peered":"2024-09-10T22:05:08.427068+0000","last_unstale":"2024-09-10T22:05:09.970336+0000","last_undegraded":"2024-09-10T22:05:09.970336+0000","last_fullsized":"2024-09-10T22:05:09.970336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:05.352080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.885769+0000","last_change":"2024-09-10T22:05:07.414751+0000","last_active":"2024-09-10T22:05:10.885769+0000","last_peered":"2024-09-10T22:05:10.885769+0000","last_clean":"2024-09-10T22:05:10.885769+0000","last_became_active":"2024-09-10T22:05:07.414495+0000","last_became_peered":"2024-09-10T22:05:07.414495+0000","last_unstale":"2024-09-10T22:05:10.885769+0000","last_undegraded":"2024-09-10T22:05:10.885769+0000","last_fullsized":"2024-09-10T22:05:10.885769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:13.559625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408462+0000","last_change":"2024-09-10T22:05:08.430133+0000","last_active":"2024-09-10T22:05:09.408462+0000","last_peered":"2024-09-10T22:05:09.408462+0000","last_clean":"2024-09-10T22:05:09.408462+0000","last_became_active":"2024-09-10T22:05:08.429899+0000","last_became_peered":"2024-09-10T22:05:08.429899+0000","last_unstale":"2024-09-10T22:05:09.408462+0000","last_undegraded":"2024-09-10T22:05:09.408462+0000","last_fullsized":"2024-09-10T22:05:09.408462+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:19.277178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365339+0000","last_change":"2024-09-10T22:05:07.408801+0000","last_active":"2024-09-10T22:05:10.365339+0000","last_peered":"2024-09-10T22:05:10.365339+0000","last_clean":"2024-09-10T22:05:10.365339+0000","last_became_active":"2024-09-10T22:05:07.407117+0000","last_became_peered":"2024-09-10T22:05:07.407117+0000","last_unstale":"2024-09-10T22:05:10.365339+0000","last_undegraded":"2024-09-10T22:05:10.365339+0000","last_fullsized":"2024-09-10T22:05:10.365339+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:35.379752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.885813+0000","last_change":"2024-09-10T22:05:08.412067+0000","last_active":"2024-09-10T22:05:10.885813+0000","last_peered":"2024-09-10T22:05:10.885813+0000","last_clean":"2024-09-10T22:05:10.885813+0000","last_became_active":"2024-09-10T22:05:08.411721+0000","last_became_peered":"2024-09-10T22:05:08.411721+0000","last_unstale":"2024-09-10T22:05:10.885813+0000","last_undegraded":"2024-09-10T22:05:10.885813+0000","last_fullsized":"2024-09-10T22:05:10.885813+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:55.369971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.885802+0000","last_change":"2024-09-10T22:05:07.412991+0000","last_active":"2024-09-10T22:05:10.885802+0000","last_peered":"2024-09-10T22:05:10.885802+0000","last_clean":"2024-09-10T22:05:10.885802+0000","last_became_active":"2024-09-10T22:05:07.412733+0000","last_became_peered":"2024-09-10T22:05:07.412733+0000","last_unstale":"2024-09-10T22:05:10.885802+0000","last_undegraded":"2024-09-10T22:05:10.885802+0000","last_fullsized":"2024-09-10T22:05:10.885802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:31.765144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970132+0000","last_change":"2024-09-10T22:05:08.430515+0000","last_active":"2024-09-10T22:05:09.970132+0000","last_peered":"2024-09-10T22:05:09.970132+0000","last_clean":"2024-09-10T22:05:09.970132+0000","last_became_active":"2024-09-10T22:05:08.426811+0000","last_became_peered":"2024-09-10T22:05:08.426811+0000","last_unstale":"2024-09-10T22:05:09.970132+0000","last_undegraded":"2024-09-10T22:05:09.970132+0000","last_fullsized":"2024-09-10T22:05:09.970132+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:19.998815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969934+0000","last_change":"2024-09-10T22:05:07.412756+0000","last_active":"2024-09-10T22:05:09.969934+0000","last_peered":"2024-09-10T22:05:09.969934+0000","last_clean":"2024-09-10T22:05:09.969934+0000","last_became_active":"2024-09-10T22:05:07.412506+0000","last_became_peered":"2024-09-10T22:05:07.412506+0000","last_unstale":"2024-09-10T22:05:09.969934+0000","last_undegraded":"2024-09-10T22:05:09.969934+0000","last_fullsized":"2024-09-10T22:05:09.969934+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:28.401204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969868+0000","last_change":"2024-09-10T22:05:08.432239+0000","last_active":"2024-09-10T22:05:09.969868+0000","last_peered":"2024-09-10T22:05:09.969868+0000","last_clean":"2024-09-10T22:05:09.969868+0000","last_became_active":"2024-09-10T22:05:08.431807+0000","last_became_peered":"2024-09-10T22:05:08.431807+0000","last_unstale":"2024-09-10T22:05:09.969868+0000","last_undegraded":"2024-09-10T22:05:09.969868+0000","last_fullsized":"2024-09-10T22:05:09.969868+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:12.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.407895+0000","last_change":"2024-09-10T22:05:07.412700+0000","last_active":"2024-09-10T22:05:09.407895+0000","last_peered":"2024-09-10T22:05:09.407895+0000","last_clean":"2024-09-10T22:05:09.407895+0000","last_became_active":"2024-09-10T22:05:07.412453+0000","last_became_peered":"2024-09-10T22:05:07.412453+0000","last_unstale":"2024-09-10T22:05:09.407895+0000","last_undegraded":"2024-09-10T22:05:09.407895+0000","last_fullsized":"2024-09-10T22:05:09.407895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:54.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":181,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408400+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:05:09.408400+0000","last_peered":"2024-09-10T22:05:09.408400+0000","last_clean":"2024-09-10T22:05:09.408400+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:05:09.408400+0000","last_undegraded":"2024-09-10T22:05:09.408400+0000","last_fullsized":"2024-09-10T22:05:09.408400+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509501,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44940,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326900,"statfs":{"total":96636764160,"available":96590745600,"internally_reserved":0,"allocated":1302528,"data_stored":881117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45760,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326080,"statfs":{"total":96636764160,"available":96589905920,"internally_reserved":0,"allocated":700416,"data_stored":290589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509501,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40200,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331640,"statfs":{"total":96636764160,"available":96595599360,"internally_reserved":0,"allocated":708608,"data_stored":291241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45776,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326064,"statfs":{"total":96636764160,"available":96589889536,"internally_reserved":0,"allocated":712704,"data_stored":292201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509500,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45768,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326072,"statfs":{"total":96636764160,"available":96589897728,"internally_reserved":0,"allocated":708608,"data_stored":290645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62404,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309436,"statfs":{"total":96636764160,"available":96572862464,"internally_reserved":0,"allocated":704512,"data_stored":290679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509501,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58836,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313004,"statfs":{"total":96636764160,"available":96576516096,"internally_reserved":0,"allocated":1310720,"data_stored":883151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542206,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":708608,"data_stored":290645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:05:12.579 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:05:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 -- 172.21.15.66:0/3078657018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368c15f340 msgr2=0x7f368c0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 --2- 172.21.15.66:0/3078657018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368c15f340 0x7f368c0cfb50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f367c009900 tx=0x7f367c01c550 comp rx=0 tx=0).stop 2024-09-10T22:05:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 -- 172.21.15.66:0/3078657018 shutdown_connections 2024-09-10T22:05:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 --2- 172.21.15.66:0/3078657018 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f368c0d2770 0x7f368c15fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 --2- 172.21.15.66:0/3078657018 >> v2:172.21.15.94:3300/0 conn(0x7f368c0b2bf0 0x7f368c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 --2- 172.21.15.66:0/3078657018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368c15f340 0x7f368c0cfb50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 -- 172.21.15.66:0/3078657018 >> 172.21.15.66:0/3078657018 conn(0x7f368c0d1c50 msgr2=0x7f368c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:12.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 -- 172.21.15.66:0/3078657018 shutdown_connections 2024-09-10T22:05:12.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 -- 172.21.15.66:0/3078657018 wait complete. 2024-09-10T22:05:12.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 Processor -- start 2024-09-10T22:05:12.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 -- start start 2024-09-10T22:05:12.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c0b2bf0 0x7f368c06f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:12.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f368c0d2770 0x7f368c166010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:12.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368c15f340 0x7f368c166550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:12.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f368c0aecf0 con 0x7f368c15f340 2024-09-10T22:05:12.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f368c0aeb70 con 0x7f368c0b2bf0 2024-09-10T22:05:12.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.759+0000 7f3691800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f368c0aee70 con 0x7f368c0d2770 2024-09-10T22:05:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f368b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368c15f340 0x7f368c166550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f368b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368c15f340 0x7f368c166550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58400/0 (socket says 172.21.15.66:58400) 2024-09-10T22:05:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f368b400640 1 -- 172.21.15.66:0/1508031020 learned_addr learned my addr 172.21.15.66:0/1508031020 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f368aa00640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c0b2bf0 0x7f368c06f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f3683e00640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f368c0d2770 0x7f368c166010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f368b400640 1 -- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f368c0d2770 msgr2=0x7f368c166010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f368b400640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f368c0d2770 0x7f368c166010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f368b400640 1 -- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c0b2bf0 msgr2=0x7f368c06f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f368b400640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c0b2bf0 0x7f368c06f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f368b400640 1 -- 172.21.15.66:0/1508031020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c06e740 con 0x7f368c15f340 2024-09-10T22:05:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f368b400640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368c15f340 0x7f368c166550 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f368400cce0 tx=0x7f368400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f3682a00640 1 -- 172.21.15.66:0/1508031020 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368400f450 con 0x7f368c15f340 2024-09-10T22:05:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f3682a00640 1 -- 172.21.15.66:0/1508031020 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f368400f700 con 0x7f368c15f340 2024-09-10T22:05:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f3682a00640 1 -- 172.21.15.66:0/1508031020 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368400fb50 con 0x7f368c15f340 2024-09-10T22:05:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f3691800640 1 -- 172.21.15.66:0/1508031020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368c0717e0 con 0x7f368c15f340 2024-09-10T22:05:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f3691800640 1 -- 172.21.15.66:0/1508031020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368c06ea00 con 0x7f368c15f340 2024-09-10T22:05:12.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f3691800640 1 -- 172.21.15.66:0/1508031020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f368c071d00 con 0x7f368c15f340 2024-09-10T22:05:12.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.763+0000 7f3682a00640 1 -- 172.21.15.66:0/1508031020 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f368400e3b0 con 0x7f368c15f340 2024-09-10T22:05:12.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.767+0000 7f3682a00640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3668072e50 0x7f3668075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:12.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.767+0000 7f3682a00640 1 -- 172.21.15.66:0/1508031020 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f3684094fe0 con 0x7f368c15f340 2024-09-10T22:05:12.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.767+0000 7f368aa00640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3668072e50 0x7f3668075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:12.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.767+0000 7f368aa00640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3668072e50 0x7f3668075310 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f367c0af000 tx=0x7f367c01ad90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:12.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.771+0000 7f3682a00640 1 -- 172.21.15.66:0/1508031020 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3684014030 con 0x7f368c15f340 2024-09-10T22:05:12.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.939+0000 7f3691800640 1 -- 172.21.15.66:0/1508031020 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f368c15fac0 con 0x7f3668072e50 2024-09-10T22:05:12.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3682a00640 1 -- 172.21.15.66:0/1508031020 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f368c15fac0 con 0x7f3668072e50 2024-09-10T22:05:12.955 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:05:12.957 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:05:12.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3691800640 1 -- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3668072e50 msgr2=0x7f3668075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3691800640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3668072e50 0x7f3668075310 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f367c0af000 tx=0x7f367c01ad90 comp rx=0 tx=0).stop 2024-09-10T22:05:12.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3691800640 1 -- 172.21.15.66:0/1508031020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368c15f340 msgr2=0x7f368c166550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:12.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3691800640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368c15f340 0x7f368c166550 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f368400cce0 tx=0x7f368400a620 comp rx=0 tx=0).stop 2024-09-10T22:05:12.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3691800640 1 -- 172.21.15.66:0/1508031020 shutdown_connections 2024-09-10T22:05:12.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3691800640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3668072e50 0x7f3668075310 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3691800640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368c15f340 0x7f368c166550 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3691800640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f368c0d2770 0x7f368c166010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3691800640 1 --2- 172.21.15.66:0/1508031020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c0b2bf0 0x7f368c06f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:12.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3691800640 1 -- 172.21.15.66:0/1508031020 >> 172.21.15.66:0/1508031020 conn(0x7f368c0d1c50 msgr2=0x7f368c15cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:12.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3691800640 1 -- 172.21.15.66:0/1508031020 shutdown_connections 2024-09-10T22:05:12.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:12.947+0000 7f3691800640 1 -- 172.21.15.66:0/1508031020 wait complete. 2024-09-10T22:05:12.973 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-10T22:05:12.379812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":383888,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":377011,"kb_avail":754590832,"statfs":{"total":773094113280,"available":772701011968,"internally_reserved":0,"allocated":6856704,"data_stored":3510268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":386059828},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.988688"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.405427+0000","last_change":"2024-09-10T22:05:08.436919+0000","last_active":"2024-09-10T22:05:09.405427+0000","last_peered":"2024-09-10T22:05:09.405427+0000","last_clean":"2024-09-10T22:05:09.405427+0000","last_became_active":"2024-09-10T22:05:08.436648+0000","last_became_peered":"2024-09-10T22:05:08.436648+0000","last_unstale":"2024-09-10T22:05:09.405427+0000","last_undegraded":"2024-09-10T22:05:09.405427+0000","last_fullsized":"2024-09-10T22:05:09.405427+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:17.767001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970153+0000","last_change":"2024-09-10T22:05:08.441899+0000","last_active":"2024-09-10T22:05:09.970153+0000","last_peered":"2024-09-10T22:05:09.970153+0000","last_clean":"2024-09-10T22:05:09.970153+0000","last_became_active":"2024-09-10T22:05:08.441635+0000","last_became_peered":"2024-09-10T22:05:08.441635+0000","last_unstale":"2024-09-10T22:05:09.970153+0000","last_undegraded":"2024-09-10T22:05:09.970153+0000","last_fullsized":"2024-09-10T22:05:09.970153+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:33.727735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408904+0000","last_change":"2024-09-10T22:05:08.426344+0000","last_active":"2024-09-10T22:05:09.408904+0000","last_peered":"2024-09-10T22:05:09.408904+0000","last_clean":"2024-09-10T22:05:09.408904+0000","last_became_active":"2024-09-10T22:05:08.426087+0000","last_became_peered":"2024-09-10T22:05:08.426087+0000","last_unstale":"2024-09-10T22:05:09.408904+0000","last_undegraded":"2024-09-10T22:05:09.408904+0000","last_fullsized":"2024-09-10T22:05:09.408904+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:26.579313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888786+0000","last_change":"2024-09-10T22:05:08.422143+0000","last_active":"2024-09-10T22:05:10.888786+0000","last_peered":"2024-09-10T22:05:10.888786+0000","last_clean":"2024-09-10T22:05:10.888786+0000","last_became_active":"2024-09-10T22:05:08.421892+0000","last_became_peered":"2024-09-10T22:05:08.421892+0000","last_unstale":"2024-09-10T22:05:10.888786+0000","last_undegraded":"2024-09-10T22:05:10.888786+0000","last_fullsized":"2024-09-10T22:05:10.888786+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:47.507835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.405069+0000","last_change":"2024-09-10T22:05:08.425476+0000","last_active":"2024-09-10T22:05:09.405069+0000","last_peered":"2024-09-10T22:05:09.405069+0000","last_clean":"2024-09-10T22:05:09.405069+0000","last_became_active":"2024-09-10T22:05:08.425070+0000","last_became_peered":"2024-09-10T22:05:08.425070+0000","last_unstale":"2024-09-10T22:05:09.405069+0000","last_undegraded":"2024-09-10T22:05:09.405069+0000","last_fullsized":"2024-09-10T22:05:09.405069+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:00.368677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424757+0000","last_change":"2024-09-10T22:05:08.435501+0000","last_active":"2024-09-10T22:05:09.424757+0000","last_peered":"2024-09-10T22:05:09.424757+0000","last_clean":"2024-09-10T22:05:09.424757+0000","last_became_active":"2024-09-10T22:05:08.435248+0000","last_became_peered":"2024-09-10T22:05:08.435248+0000","last_unstale":"2024-09-10T22:05:09.424757+0000","last_undegraded":"2024-09-10T22:05:09.424757+0000","last_fullsized":"2024-09-10T22:05:09.424757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:50.499810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365357+0000","last_change":"2024-09-10T22:05:08.422950+0000","last_active":"2024-09-10T22:05:10.365357+0000","last_peered":"2024-09-10T22:05:10.365357+0000","last_clean":"2024-09-10T22:05:10.365357+0000","last_became_active":"2024-09-10T22:05:08.422237+0000","last_became_peered":"2024-09-10T22:05:08.422237+0000","last_unstale":"2024-09-10T22:05:10.365357+0000","last_undegraded":"2024-09-10T22:05:10.365357+0000","last_fullsized":"2024-09-10T22:05:10.365357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:10.588988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970016+0000","last_change":"2024-09-10T22:05:08.432249+0000","last_active":"2024-09-10T22:05:09.970016+0000","last_peered":"2024-09-10T22:05:09.970016+0000","last_clean":"2024-09-10T22:05:09.970016+0000","last_became_active":"2024-09-10T22:05:08.431958+0000","last_became_peered":"2024-09-10T22:05:08.431958+0000","last_unstale":"2024-09-10T22:05:09.970016+0000","last_undegraded":"2024-09-10T22:05:09.970016+0000","last_fullsized":"2024-09-10T22:05:09.970016+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:43.926790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365518+0000","last_change":"2024-09-10T22:05:08.419155+0000","last_active":"2024-09-10T22:05:10.365518+0000","last_peered":"2024-09-10T22:05:10.365518+0000","last_clean":"2024-09-10T22:05:10.365518+0000","last_became_active":"2024-09-10T22:05:08.418403+0000","last_became_peered":"2024-09-10T22:05:08.418403+0000","last_unstale":"2024-09-10T22:05:10.365518+0000","last_undegraded":"2024-09-10T22:05:10.365518+0000","last_fullsized":"2024-09-10T22:05:10.365518+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:50.821502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.886291+0000","last_change":"2024-09-10T22:05:08.411950+0000","last_active":"2024-09-10T22:05:10.886291+0000","last_peered":"2024-09-10T22:05:10.886291+0000","last_clean":"2024-09-10T22:05:10.886291+0000","last_became_active":"2024-09-10T22:05:08.411656+0000","last_became_peered":"2024-09-10T22:05:08.411656+0000","last_unstale":"2024-09-10T22:05:10.886291+0000","last_undegraded":"2024-09-10T22:05:10.886291+0000","last_fullsized":"2024-09-10T22:05:10.886291+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:18.751838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.886264+0000","last_change":"2024-09-10T22:05:08.442199+0000","last_active":"2024-09-10T22:05:10.886264+0000","last_peered":"2024-09-10T22:05:10.886264+0000","last_clean":"2024-09-10T22:05:10.886264+0000","last_became_active":"2024-09-10T22:05:08.440601+0000","last_became_peered":"2024-09-10T22:05:08.440601+0000","last_unstale":"2024-09-10T22:05:10.886264+0000","last_undegraded":"2024-09-10T22:05:10.886264+0000","last_fullsized":"2024-09-10T22:05:10.886264+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:33.457724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888874+0000","last_change":"2024-09-10T22:05:08.432039+0000","last_active":"2024-09-10T22:05:10.888874+0000","last_peered":"2024-09-10T22:05:10.888874+0000","last_clean":"2024-09-10T22:05:10.888874+0000","last_became_active":"2024-09-10T22:05:08.431778+0000","last_became_peered":"2024-09-10T22:05:08.431778+0000","last_unstale":"2024-09-10T22:05:10.888874+0000","last_undegraded":"2024-09-10T22:05:10.888874+0000","last_fullsized":"2024-09-10T22:05:10.888874+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:40.012819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409697+0000","last_change":"2024-09-10T22:05:08.420996+0000","last_active":"2024-09-10T22:05:09.409697+0000","last_peered":"2024-09-10T22:05:09.409697+0000","last_clean":"2024-09-10T22:05:09.409697+0000","last_became_active":"2024-09-10T22:05:08.420728+0000","last_became_peered":"2024-09-10T22:05:08.420728+0000","last_unstale":"2024-09-10T22:05:09.409697+0000","last_undegraded":"2024-09-10T22:05:09.409697+0000","last_fullsized":"2024-09-10T22:05:09.409697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:13.055070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365465+0000","last_change":"2024-09-10T22:05:08.437083+0000","last_active":"2024-09-10T22:05:10.365465+0000","last_peered":"2024-09-10T22:05:10.365465+0000","last_clean":"2024-09-10T22:05:10.365465+0000","last_became_active":"2024-09-10T22:05:08.435695+0000","last_became_peered":"2024-09-10T22:05:08.435695+0000","last_unstale":"2024-09-10T22:05:10.365465+0000","last_undegraded":"2024-09-10T22:05:10.365465+0000","last_fullsized":"2024-09-10T22:05:10.365465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:21.905402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365479+0000","last_change":"2024-09-10T22:05:08.418640+0000","last_active":"2024-09-10T22:05:10.365479+0000","last_peered":"2024-09-10T22:05:10.365479+0000","last_clean":"2024-09-10T22:05:10.365479+0000","last_became_active":"2024-09-10T22:05:08.418331+0000","last_became_peered":"2024-09-10T22:05:08.418331+0000","last_unstale":"2024-09-10T22:05:10.365479+0000","last_undegraded":"2024-09-10T22:05:10.365479+0000","last_fullsized":"2024-09-10T22:05:10.365479+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:50.375916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424458+0000","last_change":"2024-09-10T22:05:08.414602+0000","last_active":"2024-09-10T22:05:09.424458+0000","last_peered":"2024-09-10T22:05:09.424458+0000","last_clean":"2024-09-10T22:05:09.424458+0000","last_became_active":"2024-09-10T22:05:08.414267+0000","last_became_peered":"2024-09-10T22:05:08.414267+0000","last_unstale":"2024-09-10T22:05:09.424458+0000","last_undegraded":"2024-09-10T22:05:09.424458+0000","last_fullsized":"2024-09-10T22:05:09.424458+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:47.779046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970177+0000","last_change":"2024-09-10T22:05:08.436896+0000","last_active":"2024-09-10T22:05:09.970177+0000","last_peered":"2024-09-10T22:05:09.970177+0000","last_clean":"2024-09-10T22:05:09.970177+0000","last_became_active":"2024-09-10T22:05:08.433497+0000","last_became_peered":"2024-09-10T22:05:08.433497+0000","last_unstale":"2024-09-10T22:05:09.970177+0000","last_undegraded":"2024-09-10T22:05:09.970177+0000","last_fullsized":"2024-09-10T22:05:09.970177+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:32.070331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408628+0000","last_change":"2024-09-10T22:05:08.426905+0000","last_active":"2024-09-10T22:05:09.408628+0000","last_peered":"2024-09-10T22:05:09.408628+0000","last_clean":"2024-09-10T22:05:09.408628+0000","last_became_active":"2024-09-10T22:05:08.426639+0000","last_became_peered":"2024-09-10T22:05:08.426639+0000","last_unstale":"2024-09-10T22:05:09.408628+0000","last_undegraded":"2024-09-10T22:05:09.408628+0000","last_fullsized":"2024-09-10T22:05:09.408628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:43.595579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.405398+0000","last_change":"2024-09-10T22:05:08.431297+0000","last_active":"2024-09-10T22:05:09.405398+0000","last_peered":"2024-09-10T22:05:09.405398+0000","last_clean":"2024-09-10T22:05:09.405398+0000","last_became_active":"2024-09-10T22:05:08.431009+0000","last_became_peered":"2024-09-10T22:05:08.431009+0000","last_unstale":"2024-09-10T22:05:09.405398+0000","last_undegraded":"2024-09-10T22:05:09.405398+0000","last_fullsized":"2024-09-10T22:05:09.405398+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:18.655804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969937+0000","last_change":"2024-09-10T22:05:08.439973+0000","last_active":"2024-09-10T22:05:09.969937+0000","last_peered":"2024-09-10T22:05:09.969937+0000","last_clean":"2024-09-10T22:05:09.969937+0000","last_became_active":"2024-09-10T22:05:08.439696+0000","last_became_peered":"2024-09-10T22:05:08.439696+0000","last_unstale":"2024-09-10T22:05:09.969937+0000","last_undegraded":"2024-09-10T22:05:09.969937+0000","last_fullsized":"2024-09-10T22:05:09.969937+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:07.676722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408210+0000","last_change":"2024-09-10T22:05:08.427728+0000","last_active":"2024-09-10T22:05:09.408210+0000","last_peered":"2024-09-10T22:05:09.408210+0000","last_clean":"2024-09-10T22:05:09.408210+0000","last_became_active":"2024-09-10T22:05:08.427333+0000","last_became_peered":"2024-09-10T22:05:08.427333+0000","last_unstale":"2024-09-10T22:05:09.408210+0000","last_undegraded":"2024-09-10T22:05:09.408210+0000","last_fullsized":"2024-09-10T22:05:09.408210+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:34.195763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969819+0000","last_change":"2024-09-10T22:05:08.444049+0000","last_active":"2024-09-10T22:05:09.969819+0000","last_peered":"2024-09-10T22:05:09.969819+0000","last_clean":"2024-09-10T22:05:09.969819+0000","last_became_active":"2024-09-10T22:05:08.443758+0000","last_became_peered":"2024-09-10T22:05:08.443758+0000","last_unstale":"2024-09-10T22:05:09.969819+0000","last_undegraded":"2024-09-10T22:05:09.969819+0000","last_fullsized":"2024-09-10T22:05:09.969819+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:50.298849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408785+0000","last_change":"2024-09-10T22:05:08.429645+0000","last_active":"2024-09-10T22:05:09.408785+0000","last_peered":"2024-09-10T22:05:09.408785+0000","last_clean":"2024-09-10T22:05:09.408785+0000","last_became_active":"2024-09-10T22:05:08.429361+0000","last_became_peered":"2024-09-10T22:05:08.429361+0000","last_unstale":"2024-09-10T22:05:09.408785+0000","last_undegraded":"2024-09-10T22:05:09.408785+0000","last_fullsized":"2024-09-10T22:05:09.408785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:52.783597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409473+0000","last_change":"2024-09-10T22:05:08.419915+0000","last_active":"2024-09-10T22:05:09.409473+0000","last_peered":"2024-09-10T22:05:09.409473+0000","last_clean":"2024-09-10T22:05:09.409473+0000","last_became_active":"2024-09-10T22:05:08.419643+0000","last_became_peered":"2024-09-10T22:05:08.419643+0000","last_unstale":"2024-09-10T22:05:09.409473+0000","last_undegraded":"2024-09-10T22:05:09.409473+0000","last_fullsized":"2024-09-10T22:05:09.409473+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:09.972327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.405578+0000","last_change":"2024-09-10T22:05:08.414753+0000","last_active":"2024-09-10T22:05:09.405578+0000","last_peered":"2024-09-10T22:05:09.405578+0000","last_clean":"2024-09-10T22:05:09.405578+0000","last_became_active":"2024-09-10T22:05:08.414447+0000","last_became_peered":"2024-09-10T22:05:08.414447+0000","last_unstale":"2024-09-10T22:05:09.405578+0000","last_undegraded":"2024-09-10T22:05:09.405578+0000","last_fullsized":"2024-09-10T22:05:09.405578+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:25.991515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888892+0000","last_change":"2024-09-10T22:05:08.426140+0000","last_active":"2024-09-10T22:05:10.888892+0000","last_peered":"2024-09-10T22:05:10.888892+0000","last_clean":"2024-09-10T22:05:10.888892+0000","last_became_active":"2024-09-10T22:05:08.425739+0000","last_became_peered":"2024-09-10T22:05:08.425739+0000","last_unstale":"2024-09-10T22:05:10.888892+0000","last_undegraded":"2024-09-10T22:05:10.888892+0000","last_fullsized":"2024-09-10T22:05:10.888892+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:02.025058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425065+0000","last_change":"2024-09-10T22:05:08.438832+0000","last_active":"2024-09-10T22:05:09.425065+0000","last_peered":"2024-09-10T22:05:09.425065+0000","last_clean":"2024-09-10T22:05:09.425065+0000","last_became_active":"2024-09-10T22:05:08.438638+0000","last_became_peered":"2024-09-10T22:05:08.438638+0000","last_unstale":"2024-09-10T22:05:09.425065+0000","last_undegraded":"2024-09-10T22:05:09.425065+0000","last_fullsized":"2024-09-10T22:05:09.425065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:44.863761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.405327+0000","last_change":"2024-09-10T22:05:08.418899+0000","last_active":"2024-09-10T22:05:09.405327+0000","last_peered":"2024-09-10T22:05:09.405327+0000","last_clean":"2024-09-10T22:05:09.405327+0000","last_became_active":"2024-09-10T22:05:08.418569+0000","last_became_peered":"2024-09-10T22:05:08.418569+0000","last_unstale":"2024-09-10T22:05:09.405327+0000","last_undegraded":"2024-09-10T22:05:09.405327+0000","last_fullsized":"2024-09-10T22:05:09.405327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:29.450972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970350+0000","last_change":"2024-09-10T22:05:08.430551+0000","last_active":"2024-09-10T22:05:09.970350+0000","last_peered":"2024-09-10T22:05:09.970350+0000","last_clean":"2024-09-10T22:05:09.970350+0000","last_became_active":"2024-09-10T22:05:08.426896+0000","last_became_peered":"2024-09-10T22:05:08.426896+0000","last_unstale":"2024-09-10T22:05:09.970350+0000","last_undegraded":"2024-09-10T22:05:09.970350+0000","last_fullsized":"2024-09-10T22:05:09.970350+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:04.278385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408197+0000","last_change":"2024-09-10T22:05:08.426923+0000","last_active":"2024-09-10T22:05:09.408197+0000","last_peered":"2024-09-10T22:05:09.408197+0000","last_clean":"2024-09-10T22:05:09.408197+0000","last_became_active":"2024-09-10T22:05:08.426596+0000","last_became_peered":"2024-09-10T22:05:08.426596+0000","last_unstale":"2024-09-10T22:05:09.408197+0000","last_undegraded":"2024-09-10T22:05:09.408197+0000","last_fullsized":"2024-09-10T22:05:09.408197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:28.155257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.405025+0000","last_change":"2024-09-10T22:05:08.411300+0000","last_active":"2024-09-10T22:05:09.405025+0000","last_peered":"2024-09-10T22:05:09.405025+0000","last_clean":"2024-09-10T22:05:09.405025+0000","last_became_active":"2024-09-10T22:05:08.410898+0000","last_became_peered":"2024-09-10T22:05:08.410898+0000","last_unstale":"2024-09-10T22:05:09.405025+0000","last_undegraded":"2024-09-10T22:05:09.405025+0000","last_fullsized":"2024-09-10T22:05:09.405025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:21.640184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424489+0000","last_change":"2024-09-10T22:05:08.430443+0000","last_active":"2024-09-10T22:05:09.424489+0000","last_peered":"2024-09-10T22:05:09.424489+0000","last_clean":"2024-09-10T22:05:09.424489+0000","last_became_active":"2024-09-10T22:05:08.430138+0000","last_became_peered":"2024-09-10T22:05:08.430138+0000","last_unstale":"2024-09-10T22:05:09.424489+0000","last_undegraded":"2024-09-10T22:05:09.424489+0000","last_fullsized":"2024-09-10T22:05:09.424489+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:48.436292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425267+0000","last_change":"2024-09-10T22:05:08.426171+0000","last_active":"2024-09-10T22:05:09.425267+0000","last_peered":"2024-09-10T22:05:09.425267+0000","last_clean":"2024-09-10T22:05:09.425267+0000","last_became_active":"2024-09-10T22:05:08.425870+0000","last_became_peered":"2024-09-10T22:05:08.425870+0000","last_unstale":"2024-09-10T22:05:09.425267+0000","last_undegraded":"2024-09-10T22:05:09.425267+0000","last_fullsized":"2024-09-10T22:05:09.425267+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:35.925364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425253+0000","last_change":"2024-09-10T22:05:07.413946+0000","last_active":"2024-09-10T22:05:09.425253+0000","last_peered":"2024-09-10T22:05:09.425253+0000","last_clean":"2024-09-10T22:05:09.425253+0000","last_became_active":"2024-09-10T22:05:07.413670+0000","last_became_peered":"2024-09-10T22:05:07.413670+0000","last_unstale":"2024-09-10T22:05:09.425253+0000","last_undegraded":"2024-09-10T22:05:09.425253+0000","last_fullsized":"2024-09-10T22:05:09.425253+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:51.779195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.886388+0000","last_change":"2024-09-10T22:05:08.429659+0000","last_active":"2024-09-10T22:05:10.886388+0000","last_peered":"2024-09-10T22:05:10.886388+0000","last_clean":"2024-09-10T22:05:10.886388+0000","last_became_active":"2024-09-10T22:05:08.429509+0000","last_became_peered":"2024-09-10T22:05:08.429509+0000","last_unstale":"2024-09-10T22:05:10.886388+0000","last_undegraded":"2024-09-10T22:05:10.886388+0000","last_fullsized":"2024-09-10T22:05:10.886388+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:27.181185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969952+0000","last_change":"2024-09-10T22:05:07.415031+0000","last_active":"2024-09-10T22:05:09.969952+0000","last_peered":"2024-09-10T22:05:09.969952+0000","last_clean":"2024-09-10T22:05:09.969952+0000","last_became_active":"2024-09-10T22:05:07.414707+0000","last_became_peered":"2024-09-10T22:05:07.414707+0000","last_unstale":"2024-09-10T22:05:09.969952+0000","last_undegraded":"2024-09-10T22:05:09.969952+0000","last_fullsized":"2024-09-10T22:05:09.969952+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:33.928698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424994+0000","last_change":"2024-09-10T22:05:08.424487+0000","last_active":"2024-09-10T22:05:09.424994+0000","last_peered":"2024-09-10T22:05:09.424994+0000","last_clean":"2024-09-10T22:05:09.424994+0000","last_became_active":"2024-09-10T22:05:08.424205+0000","last_became_peered":"2024-09-10T22:05:08.424205+0000","last_unstale":"2024-09-10T22:05:09.424994+0000","last_undegraded":"2024-09-10T22:05:09.424994+0000","last_fullsized":"2024-09-10T22:05:09.424994+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:18.056158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425143+0000","last_change":"2024-09-10T22:05:07.410708+0000","last_active":"2024-09-10T22:05:09.425143+0000","last_peered":"2024-09-10T22:05:09.425143+0000","last_clean":"2024-09-10T22:05:09.425143+0000","last_became_active":"2024-09-10T22:05:07.410249+0000","last_became_peered":"2024-09-10T22:05:07.410249+0000","last_unstale":"2024-09-10T22:05:09.425143+0000","last_undegraded":"2024-09-10T22:05:09.425143+0000","last_fullsized":"2024-09-10T22:05:09.425143+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:45.733933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425058+0000","last_change":"2024-09-10T22:05:08.424434+0000","last_active":"2024-09-10T22:05:09.425058+0000","last_peered":"2024-09-10T22:05:09.425058+0000","last_clean":"2024-09-10T22:05:09.425058+0000","last_became_active":"2024-09-10T22:05:08.423950+0000","last_became_peered":"2024-09-10T22:05:08.423950+0000","last_unstale":"2024-09-10T22:05:09.425058+0000","last_undegraded":"2024-09-10T22:05:09.425058+0000","last_fullsized":"2024-09-10T22:05:09.425058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:31.151424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888835+0000","last_change":"2024-09-10T22:05:07.416356+0000","last_active":"2024-09-10T22:05:10.888835+0000","last_peered":"2024-09-10T22:05:10.888835+0000","last_clean":"2024-09-10T22:05:10.888835+0000","last_became_active":"2024-09-10T22:05:07.415854+0000","last_became_peered":"2024-09-10T22:05:07.415854+0000","last_unstale":"2024-09-10T22:05:10.888835+0000","last_undegraded":"2024-09-10T22:05:10.888835+0000","last_fullsized":"2024-09-10T22:05:10.888835+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:20.901000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969847+0000","last_change":"2024-09-10T22:05:08.443704+0000","last_active":"2024-09-10T22:05:09.969847+0000","last_peered":"2024-09-10T22:05:09.969847+0000","last_clean":"2024-09-10T22:05:09.969847+0000","last_became_active":"2024-09-10T22:05:08.443359+0000","last_became_peered":"2024-09-10T22:05:08.443359+0000","last_unstale":"2024-09-10T22:05:09.969847+0000","last_undegraded":"2024-09-10T22:05:09.969847+0000","last_fullsized":"2024-09-10T22:05:09.969847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:06.230381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365040+0000","last_change":"2024-09-10T22:05:07.419708+0000","last_active":"2024-09-10T22:05:10.365040+0000","last_peered":"2024-09-10T22:05:10.365040+0000","last_clean":"2024-09-10T22:05:10.365040+0000","last_became_active":"2024-09-10T22:05:07.419341+0000","last_became_peered":"2024-09-10T22:05:07.419341+0000","last_unstale":"2024-09-10T22:05:10.365040+0000","last_undegraded":"2024-09-10T22:05:10.365040+0000","last_fullsized":"2024-09-10T22:05:10.365040+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:08.796119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970297+0000","last_change":"2024-09-10T22:05:08.433589+0000","last_active":"2024-09-10T22:05:09.970297+0000","last_peered":"2024-09-10T22:05:09.970297+0000","last_clean":"2024-09-10T22:05:09.970297+0000","last_became_active":"2024-09-10T22:05:08.433325+0000","last_became_peered":"2024-09-10T22:05:08.433325+0000","last_unstale":"2024-09-10T22:05:09.970297+0000","last_undegraded":"2024-09-10T22:05:09.970297+0000","last_fullsized":"2024-09-10T22:05:09.970297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:59:17.109089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424479+0000","last_change":"2024-09-10T22:05:07.403621+0000","last_active":"2024-09-10T22:05:09.424479+0000","last_peered":"2024-09-10T22:05:09.424479+0000","last_clean":"2024-09-10T22:05:09.424479+0000","last_became_active":"2024-09-10T22:05:07.399475+0000","last_became_peered":"2024-09-10T22:05:07.399475+0000","last_unstale":"2024-09-10T22:05:09.424479+0000","last_undegraded":"2024-09-10T22:05:09.424479+0000","last_fullsized":"2024-09-10T22:05:09.424479+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:59.131489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409358+0000","last_change":"2024-09-10T22:05:08.426518+0000","last_active":"2024-09-10T22:05:09.409358+0000","last_peered":"2024-09-10T22:05:09.409358+0000","last_clean":"2024-09-10T22:05:09.409358+0000","last_became_active":"2024-09-10T22:05:08.424601+0000","last_became_peered":"2024-09-10T22:05:08.424601+0000","last_unstale":"2024-09-10T22:05:09.409358+0000","last_undegraded":"2024-09-10T22:05:09.409358+0000","last_fullsized":"2024-09-10T22:05:09.409358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:27.181185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888418+0000","last_change":"2024-09-10T22:05:07.414884+0000","last_active":"2024-09-10T22:05:10.888418+0000","last_peered":"2024-09-10T22:05:10.888418+0000","last_clean":"2024-09-10T22:05:10.888418+0000","last_became_active":"2024-09-10T22:05:07.414552+0000","last_became_peered":"2024-09-10T22:05:07.414552+0000","last_unstale":"2024-09-10T22:05:10.888418+0000","last_undegraded":"2024-09-10T22:05:10.888418+0000","last_fullsized":"2024-09-10T22:05:10.888418+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:09.584585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969883+0000","last_change":"2024-09-10T22:05:08.438977+0000","last_active":"2024-09-10T22:05:09.969883+0000","last_peered":"2024-09-10T22:05:09.969883+0000","last_clean":"2024-09-10T22:05:09.969883+0000","last_became_active":"2024-09-10T22:05:08.437107+0000","last_became_peered":"2024-09-10T22:05:08.437107+0000","last_unstale":"2024-09-10T22:05:09.969883+0000","last_undegraded":"2024-09-10T22:05:09.969883+0000","last_fullsized":"2024-09-10T22:05:09.969883+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:47.278493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.364967+0000","last_change":"2024-09-10T22:05:07.419662+0000","last_active":"2024-09-10T22:05:10.364967+0000","last_peered":"2024-09-10T22:05:10.364967+0000","last_clean":"2024-09-10T22:05:10.364967+0000","last_became_active":"2024-09-10T22:05:07.419312+0000","last_became_peered":"2024-09-10T22:05:07.419312+0000","last_unstale":"2024-09-10T22:05:10.364967+0000","last_undegraded":"2024-09-10T22:05:10.364967+0000","last_fullsized":"2024-09-10T22:05:10.364967+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:13.991036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365674+0000","last_change":"2024-09-10T22:05:08.418649+0000","last_active":"2024-09-10T22:05:10.365674+0000","last_peered":"2024-09-10T22:05:10.365674+0000","last_clean":"2024-09-10T22:05:10.365674+0000","last_became_active":"2024-09-10T22:05:08.418343+0000","last_became_peered":"2024-09-10T22:05:08.418343+0000","last_unstale":"2024-09-10T22:05:10.365674+0000","last_undegraded":"2024-09-10T22:05:10.365674+0000","last_fullsized":"2024-09-10T22:05:10.365674+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:04.397416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409023+0000","last_change":"2024-09-10T22:05:07.418011+0000","last_active":"2024-09-10T22:05:09.409023+0000","last_peered":"2024-09-10T22:05:09.409023+0000","last_clean":"2024-09-10T22:05:09.409023+0000","last_became_active":"2024-09-10T22:05:07.417671+0000","last_became_peered":"2024-09-10T22:05:07.417671+0000","last_unstale":"2024-09-10T22:05:09.409023+0000","last_undegraded":"2024-09-10T22:05:09.409023+0000","last_fullsized":"2024-09-10T22:05:09.409023+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:07.953831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409450+0000","last_change":"2024-09-10T22:05:08.438782+0000","last_active":"2024-09-10T22:05:09.409450+0000","last_peered":"2024-09-10T22:05:09.409450+0000","last_clean":"2024-09-10T22:05:09.409450+0000","last_became_active":"2024-09-10T22:05:08.438485+0000","last_became_peered":"2024-09-10T22:05:08.438485+0000","last_unstale":"2024-09-10T22:05:09.409450+0000","last_undegraded":"2024-09-10T22:05:09.409450+0000","last_fullsized":"2024-09-10T22:05:09.409450+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:48.436292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970091+0000","last_change":"2024-09-10T22:05:07.417628+0000","last_active":"2024-09-10T22:05:09.970091+0000","last_peered":"2024-09-10T22:05:09.970091+0000","last_clean":"2024-09-10T22:05:09.970091+0000","last_became_active":"2024-09-10T22:05:07.417352+0000","last_became_peered":"2024-09-10T22:05:07.417352+0000","last_unstale":"2024-09-10T22:05:09.970091+0000","last_undegraded":"2024-09-10T22:05:09.970091+0000","last_fullsized":"2024-09-10T22:05:09.970091+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:45.328003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.886023+0000","last_change":"2024-09-10T22:05:08.409963+0000","last_active":"2024-09-10T22:05:10.886023+0000","last_peered":"2024-09-10T22:05:10.886023+0000","last_clean":"2024-09-10T22:05:10.886023+0000","last_became_active":"2024-09-10T22:05:08.409680+0000","last_became_peered":"2024-09-10T22:05:08.409680+0000","last_unstale":"2024-09-10T22:05:10.886023+0000","last_undegraded":"2024-09-10T22:05:10.886023+0000","last_fullsized":"2024-09-10T22:05:10.886023+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:28.279917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409153+0000","last_change":"2024-09-10T22:05:07.418919+0000","last_active":"2024-09-10T22:05:09.409153+0000","last_peered":"2024-09-10T22:05:09.409153+0000","last_clean":"2024-09-10T22:05:09.409153+0000","last_became_active":"2024-09-10T22:05:07.418461+0000","last_became_peered":"2024-09-10T22:05:07.418461+0000","last_unstale":"2024-09-10T22:05:09.409153+0000","last_undegraded":"2024-09-10T22:05:09.409153+0000","last_fullsized":"2024-09-10T22:05:09.409153+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:30.637816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409089+0000","last_change":"2024-09-10T22:05:08.433901+0000","last_active":"2024-09-10T22:05:09.409089+0000","last_peered":"2024-09-10T22:05:09.409089+0000","last_clean":"2024-09-10T22:05:09.409089+0000","last_became_active":"2024-09-10T22:05:08.433623+0000","last_became_peered":"2024-09-10T22:05:08.433623+0000","last_unstale":"2024-09-10T22:05:09.409089+0000","last_undegraded":"2024-09-10T22:05:09.409089+0000","last_fullsized":"2024-09-10T22:05:09.409089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:02.025058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":21,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365352+0000","last_change":"2024-09-10T22:05:07.416575+0000","last_active":"2024-09-10T22:05:10.365352+0000","last_peered":"2024-09-10T22:05:10.365352+0000","last_clean":"2024-09-10T22:05:10.365352+0000","last_became_active":"2024-09-10T22:05:07.416319+0000","last_became_peered":"2024-09-10T22:05:07.416319+0000","last_unstale":"2024-09-10T22:05:10.365352+0000","last_undegraded":"2024-09-10T22:05:10.365352+0000","last_fullsized":"2024-09-10T22:05:10.365352+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:39.277370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888524+0000","last_change":"2024-09-10T22:05:08.427480+0000","last_active":"2024-09-10T22:05:10.888524+0000","last_peered":"2024-09-10T22:05:10.888524+0000","last_clean":"2024-09-10T22:05:10.888524+0000","last_became_active":"2024-09-10T22:05:08.427229+0000","last_became_peered":"2024-09-10T22:05:08.427229+0000","last_unstale":"2024-09-10T22:05:10.888524+0000","last_undegraded":"2024-09-10T22:05:10.888524+0000","last_fullsized":"2024-09-10T22:05:10.888524+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:56.408442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969707+0000","last_change":"2024-09-10T22:05:07.419646+0000","last_active":"2024-09-10T22:05:09.969707+0000","last_peered":"2024-09-10T22:05:09.969707+0000","last_clean":"2024-09-10T22:05:09.969707+0000","last_became_active":"2024-09-10T22:05:07.419363+0000","last_became_peered":"2024-09-10T22:05:07.419363+0000","last_unstale":"2024-09-10T22:05:09.969707+0000","last_undegraded":"2024-09-10T22:05:09.969707+0000","last_fullsized":"2024-09-10T22:05:09.969707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:55.404040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408308+0000","last_change":"2024-09-10T22:05:08.432440+0000","last_active":"2024-09-10T22:05:09.408308+0000","last_peered":"2024-09-10T22:05:09.408308+0000","last_clean":"2024-09-10T22:05:09.408308+0000","last_became_active":"2024-09-10T22:05:08.432165+0000","last_became_peered":"2024-09-10T22:05:08.432165+0000","last_unstale":"2024-09-10T22:05:09.408308+0000","last_undegraded":"2024-09-10T22:05:09.408308+0000","last_fullsized":"2024-09-10T22:05:09.408308+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:29.443096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.407958+0000","last_change":"2024-09-10T22:05:07.408274+0000","last_active":"2024-09-10T22:05:09.407958+0000","last_peered":"2024-09-10T22:05:09.407958+0000","last_clean":"2024-09-10T22:05:09.407958+0000","last_became_active":"2024-09-10T22:05:07.407846+0000","last_became_peered":"2024-09-10T22:05:07.407846+0000","last_unstale":"2024-09-10T22:05:09.407958+0000","last_undegraded":"2024-09-10T22:05:09.407958+0000","last_fullsized":"2024-09-10T22:05:09.407958+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:54.365569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365432+0000","last_change":"2024-09-10T22:05:08.423464+0000","last_active":"2024-09-10T22:05:10.365432+0000","last_peered":"2024-09-10T22:05:10.365432+0000","last_clean":"2024-09-10T22:05:10.365432+0000","last_became_active":"2024-09-10T22:05:08.423195+0000","last_became_peered":"2024-09-10T22:05:08.423195+0000","last_unstale":"2024-09-10T22:05:10.365432+0000","last_undegraded":"2024-09-10T22:05:10.365432+0000","last_fullsized":"2024-09-10T22:05:10.365432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:04.561865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888416+0000","last_change":"2024-09-10T22:05:07.415756+0000","last_active":"2024-09-10T22:05:10.888416+0000","last_peered":"2024-09-10T22:05:10.888416+0000","last_clean":"2024-09-10T22:05:10.888416+0000","last_became_active":"2024-09-10T22:05:07.415453+0000","last_became_peered":"2024-09-10T22:05:07.415453+0000","last_unstale":"2024-09-10T22:05:10.888416+0000","last_undegraded":"2024-09-10T22:05:10.888416+0000","last_fullsized":"2024-09-10T22:05:10.888416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:03.557463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408838+0000","last_change":"2024-09-10T22:05:08.431025+0000","last_active":"2024-09-10T22:05:09.408838+0000","last_peered":"2024-09-10T22:05:09.408838+0000","last_clean":"2024-09-10T22:05:09.408838+0000","last_became_active":"2024-09-10T22:05:08.430727+0000","last_became_peered":"2024-09-10T22:05:08.430727+0000","last_unstale":"2024-09-10T22:05:09.408838+0000","last_undegraded":"2024-09-10T22:05:09.408838+0000","last_fullsized":"2024-09-10T22:05:09.408838+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:59.609250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408287+0000","last_change":"2024-09-10T22:05:07.416172+0000","last_active":"2024-09-10T22:05:09.408287+0000","last_peered":"2024-09-10T22:05:09.408287+0000","last_clean":"2024-09-10T22:05:09.408287+0000","last_became_active":"2024-09-10T22:05:07.415901+0000","last_became_peered":"2024-09-10T22:05:07.415901+0000","last_unstale":"2024-09-10T22:05:09.408287+0000","last_undegraded":"2024-09-10T22:05:09.408287+0000","last_fullsized":"2024-09-10T22:05:09.408287+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:55.370220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409277+0000","last_change":"2024-09-10T22:05:08.429545+0000","last_active":"2024-09-10T22:05:09.409277+0000","last_peered":"2024-09-10T22:05:09.409277+0000","last_clean":"2024-09-10T22:05:09.409277+0000","last_became_active":"2024-09-10T22:05:08.429265+0000","last_became_peered":"2024-09-10T22:05:08.429265+0000","last_unstale":"2024-09-10T22:05:09.409277+0000","last_undegraded":"2024-09-10T22:05:09.409277+0000","last_fullsized":"2024-09-10T22:05:09.409277+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:47.779046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970071+0000","last_change":"2024-09-10T22:05:07.416924+0000","last_active":"2024-09-10T22:05:09.970071+0000","last_peered":"2024-09-10T22:05:09.970071+0000","last_clean":"2024-09-10T22:05:09.970071+0000","last_became_active":"2024-09-10T22:05:07.416672+0000","last_became_peered":"2024-09-10T22:05:07.416672+0000","last_unstale":"2024-09-10T22:05:09.970071+0000","last_undegraded":"2024-09-10T22:05:09.970071+0000","last_fullsized":"2024-09-10T22:05:09.970071+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:43.859359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.889205+0000","last_change":"2024-09-10T22:05:08.423758+0000","last_active":"2024-09-10T22:05:10.889205+0000","last_peered":"2024-09-10T22:05:10.889205+0000","last_clean":"2024-09-10T22:05:10.889205+0000","last_became_active":"2024-09-10T22:05:08.423436+0000","last_became_peered":"2024-09-10T22:05:08.423436+0000","last_unstale":"2024-09-10T22:05:10.889205+0000","last_undegraded":"2024-09-10T22:05:10.889205+0000","last_fullsized":"2024-09-10T22:05:10.889205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:48.436292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425095+0000","last_change":"2024-09-10T22:05:07.411669+0000","last_active":"2024-09-10T22:05:09.425095+0000","last_peered":"2024-09-10T22:05:09.425095+0000","last_clean":"2024-09-10T22:05:09.425095+0000","last_became_active":"2024-09-10T22:05:07.411198+0000","last_became_peered":"2024-09-10T22:05:07.411198+0000","last_unstale":"2024-09-10T22:05:09.425095+0000","last_undegraded":"2024-09-10T22:05:09.425095+0000","last_fullsized":"2024-09-10T22:05:09.425095+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:30.408615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970202+0000","last_change":"2024-09-10T22:05:08.431947+0000","last_active":"2024-09-10T22:05:09.970202+0000","last_peered":"2024-09-10T22:05:09.970202+0000","last_clean":"2024-09-10T22:05:09.970202+0000","last_became_active":"2024-09-10T22:05:08.431577+0000","last_became_peered":"2024-09-10T22:05:08.431577+0000","last_unstale":"2024-09-10T22:05:09.970202+0000","last_undegraded":"2024-09-10T22:05:09.970202+0000","last_fullsized":"2024-09-10T22:05:09.970202+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:29.580261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888720+0000","last_change":"2024-09-10T22:05:07.415938+0000","last_active":"2024-09-10T22:05:10.888720+0000","last_peered":"2024-09-10T22:05:10.888720+0000","last_clean":"2024-09-10T22:05:10.888720+0000","last_became_active":"2024-09-10T22:05:07.415488+0000","last_became_peered":"2024-09-10T22:05:07.415488+0000","last_unstale":"2024-09-10T22:05:10.888720+0000","last_undegraded":"2024-09-10T22:05:10.888720+0000","last_fullsized":"2024-09-10T22:05:10.888720+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:03.393013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409156+0000","last_change":"2024-09-10T22:05:08.432494+0000","last_active":"2024-09-10T22:05:09.409156+0000","last_peered":"2024-09-10T22:05:09.409156+0000","last_clean":"2024-09-10T22:05:09.409156+0000","last_became_active":"2024-09-10T22:05:08.432185+0000","last_became_peered":"2024-09-10T22:05:08.432185+0000","last_unstale":"2024-09-10T22:05:09.409156+0000","last_undegraded":"2024-09-10T22:05:09.409156+0000","last_fullsized":"2024-09-10T22:05:09.409156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:23.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424834+0000","last_change":"2024-09-10T22:05:07.413141+0000","last_active":"2024-09-10T22:05:09.424834+0000","last_peered":"2024-09-10T22:05:09.424834+0000","last_clean":"2024-09-10T22:05:09.424834+0000","last_became_active":"2024-09-10T22:05:07.412606+0000","last_became_peered":"2024-09-10T22:05:07.412606+0000","last_unstale":"2024-09-10T22:05:09.424834+0000","last_undegraded":"2024-09-10T22:05:09.424834+0000","last_fullsized":"2024-09-10T22:05:09.424834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:53.246171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888547+0000","last_change":"2024-09-10T22:05:08.426817+0000","last_active":"2024-09-10T22:05:10.888547+0000","last_peered":"2024-09-10T22:05:10.888547+0000","last_clean":"2024-09-10T22:05:10.888547+0000","last_became_active":"2024-09-10T22:05:08.426537+0000","last_became_peered":"2024-09-10T22:05:08.426537+0000","last_unstale":"2024-09-10T22:05:10.888547+0000","last_undegraded":"2024-09-10T22:05:10.888547+0000","last_fullsized":"2024-09-10T22:05:10.888547+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:29.405607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970341+0000","last_change":"2024-09-10T22:05:07.412724+0000","last_active":"2024-09-10T22:05:09.970341+0000","last_peered":"2024-09-10T22:05:09.970341+0000","last_clean":"2024-09-10T22:05:09.970341+0000","last_became_active":"2024-09-10T22:05:07.412235+0000","last_became_peered":"2024-09-10T22:05:07.412235+0000","last_unstale":"2024-09-10T22:05:09.970341+0000","last_undegraded":"2024-09-10T22:05:09.970341+0000","last_fullsized":"2024-09-10T22:05:09.970341+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:46.503432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424752+0000","last_change":"2024-09-10T22:05:08.437326+0000","last_active":"2024-09-10T22:05:09.424752+0000","last_peered":"2024-09-10T22:05:09.424752+0000","last_clean":"2024-09-10T22:05:09.424752+0000","last_became_active":"2024-09-10T22:05:08.436843+0000","last_became_peered":"2024-09-10T22:05:08.436843+0000","last_unstale":"2024-09-10T22:05:09.424752+0000","last_undegraded":"2024-09-10T22:05:09.424752+0000","last_fullsized":"2024-09-10T22:05:09.424752+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:34.933101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.885846+0000","last_change":"2024-09-10T22:05:07.419669+0000","last_active":"2024-09-10T22:05:10.885846+0000","last_peered":"2024-09-10T22:05:10.885846+0000","last_clean":"2024-09-10T22:05:10.885846+0000","last_became_active":"2024-09-10T22:05:07.419399+0000","last_became_peered":"2024-09-10T22:05:07.419399+0000","last_unstale":"2024-09-10T22:05:10.885846+0000","last_undegraded":"2024-09-10T22:05:10.885846+0000","last_fullsized":"2024-09-10T22:05:10.885846+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:35.612885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.424517+0000","last_change":"2024-09-10T22:05:08.434182+0000","last_active":"2024-09-10T22:05:09.424517+0000","last_peered":"2024-09-10T22:05:09.424517+0000","last_clean":"2024-09-10T22:05:09.424517+0000","last_became_active":"2024-09-10T22:05:08.425682+0000","last_became_peered":"2024-09-10T22:05:08.425682+0000","last_unstale":"2024-09-10T22:05:09.424517+0000","last_undegraded":"2024-09-10T22:05:09.424517+0000","last_fullsized":"2024-09-10T22:05:09.424517+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:10.908283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888395+0000","last_change":"2024-09-10T22:05:07.421708+0000","last_active":"2024-09-10T22:05:10.888395+0000","last_peered":"2024-09-10T22:05:10.888395+0000","last_clean":"2024-09-10T22:05:10.888395+0000","last_became_active":"2024-09-10T22:05:07.421421+0000","last_became_peered":"2024-09-10T22:05:07.421421+0000","last_unstale":"2024-09-10T22:05:10.888395+0000","last_undegraded":"2024-09-10T22:05:10.888395+0000","last_fullsized":"2024-09-10T22:05:10.888395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:49.817099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.889099+0000","last_change":"2024-09-10T22:05:08.425626+0000","last_active":"2024-09-10T22:05:10.889099+0000","last_peered":"2024-09-10T22:05:10.889099+0000","last_clean":"2024-09-10T22:05:10.889099+0000","last_became_active":"2024-09-10T22:05:08.425333+0000","last_became_peered":"2024-09-10T22:05:08.425333+0000","last_unstale":"2024-09-10T22:05:10.889099+0000","last_undegraded":"2024-09-10T22:05:10.889099+0000","last_fullsized":"2024-09-10T22:05:10.889099+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:51.427056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.886309+0000","last_change":"2024-09-10T22:05:07.415631+0000","last_active":"2024-09-10T22:05:10.886309+0000","last_peered":"2024-09-10T22:05:10.886309+0000","last_clean":"2024-09-10T22:05:10.886309+0000","last_became_active":"2024-09-10T22:05:07.415345+0000","last_became_peered":"2024-09-10T22:05:07.415345+0000","last_unstale":"2024-09-10T22:05:10.886309+0000","last_undegraded":"2024-09-10T22:05:10.886309+0000","last_fullsized":"2024-09-10T22:05:10.886309+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:04.145998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.889209+0000","last_change":"2024-09-10T22:05:08.433734+0000","last_active":"2024-09-10T22:05:10.889209+0000","last_peered":"2024-09-10T22:05:10.889209+0000","last_clean":"2024-09-10T22:05:10.889209+0000","last_became_active":"2024-09-10T22:05:08.433451+0000","last_became_peered":"2024-09-10T22:05:08.433451+0000","last_unstale":"2024-09-10T22:05:10.889209+0000","last_undegraded":"2024-09-10T22:05:10.889209+0000","last_fullsized":"2024-09-10T22:05:10.889209+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:15.748248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425276+0000","last_change":"2024-09-10T22:05:07.400217+0000","last_active":"2024-09-10T22:05:09.425276+0000","last_peered":"2024-09-10T22:05:09.425276+0000","last_clean":"2024-09-10T22:05:09.425276+0000","last_became_active":"2024-09-10T22:05:07.399944+0000","last_became_peered":"2024-09-10T22:05:07.399944+0000","last_unstale":"2024-09-10T22:05:09.425276+0000","last_undegraded":"2024-09-10T22:05:09.425276+0000","last_fullsized":"2024-09-10T22:05:09.425276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:02.216373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.886051+0000","last_change":"2024-09-10T22:05:08.408878+0000","last_active":"2024-09-10T22:05:10.886051+0000","last_peered":"2024-09-10T22:05:10.886051+0000","last_clean":"2024-09-10T22:05:10.886051+0000","last_became_active":"2024-09-10T22:05:08.408599+0000","last_became_peered":"2024-09-10T22:05:08.408599+0000","last_unstale":"2024-09-10T22:05:10.886051+0000","last_undegraded":"2024-09-10T22:05:10.886051+0000","last_fullsized":"2024-09-10T22:05:10.886051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:57.172129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.409585+0000","last_change":"2024-09-10T22:05:07.411290+0000","last_active":"2024-09-10T22:05:09.409585+0000","last_peered":"2024-09-10T22:05:09.409585+0000","last_clean":"2024-09-10T22:05:09.409585+0000","last_became_active":"2024-09-10T22:05:07.410412+0000","last_became_peered":"2024-09-10T22:05:07.410412+0000","last_unstale":"2024-09-10T22:05:09.409585+0000","last_undegraded":"2024-09-10T22:05:09.409585+0000","last_fullsized":"2024-09-10T22:05:09.409585+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:27.275515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.425412+0000","last_change":"2024-09-10T22:05:08.424120+0000","last_active":"2024-09-10T22:05:09.425412+0000","last_peered":"2024-09-10T22:05:09.425412+0000","last_clean":"2024-09-10T22:05:09.425412+0000","last_became_active":"2024-09-10T22:05:08.423514+0000","last_became_peered":"2024-09-10T22:05:08.423514+0000","last_unstale":"2024-09-10T22:05:09.425412+0000","last_undegraded":"2024-09-10T22:05:09.425412+0000","last_fullsized":"2024-09-10T22:05:09.425412+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:02.039553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.888904+0000","last_change":"2024-09-10T22:05:07.419449+0000","last_active":"2024-09-10T22:05:10.888904+0000","last_peered":"2024-09-10T22:05:10.888904+0000","last_clean":"2024-09-10T22:05:10.888904+0000","last_became_active":"2024-09-10T22:05:07.419092+0000","last_became_peered":"2024-09-10T22:05:07.419092+0000","last_unstale":"2024-09-10T22:05:10.888904+0000","last_undegraded":"2024-09-10T22:05:10.888904+0000","last_fullsized":"2024-09-10T22:05:10.888904+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:49.371514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970336+0000","last_change":"2024-09-10T22:05:08.430561+0000","last_active":"2024-09-10T22:05:09.970336+0000","last_peered":"2024-09-10T22:05:09.970336+0000","last_clean":"2024-09-10T22:05:09.970336+0000","last_became_active":"2024-09-10T22:05:08.427068+0000","last_became_peered":"2024-09-10T22:05:08.427068+0000","last_unstale":"2024-09-10T22:05:09.970336+0000","last_undegraded":"2024-09-10T22:05:09.970336+0000","last_fullsized":"2024-09-10T22:05:09.970336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:05.352080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.885769+0000","last_change":"2024-09-10T22:05:07.414751+0000","last_active":"2024-09-10T22:05:10.885769+0000","last_peered":"2024-09-10T22:05:10.885769+0000","last_clean":"2024-09-10T22:05:10.885769+0000","last_became_active":"2024-09-10T22:05:07.414495+0000","last_became_peered":"2024-09-10T22:05:07.414495+0000","last_unstale":"2024-09-10T22:05:10.885769+0000","last_undegraded":"2024-09-10T22:05:10.885769+0000","last_fullsized":"2024-09-10T22:05:10.885769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:13.559625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408462+0000","last_change":"2024-09-10T22:05:08.430133+0000","last_active":"2024-09-10T22:05:09.408462+0000","last_peered":"2024-09-10T22:05:09.408462+0000","last_clean":"2024-09-10T22:05:09.408462+0000","last_became_active":"2024-09-10T22:05:08.429899+0000","last_became_peered":"2024-09-10T22:05:08.429899+0000","last_unstale":"2024-09-10T22:05:09.408462+0000","last_undegraded":"2024-09-10T22:05:09.408462+0000","last_fullsized":"2024-09-10T22:05:09.408462+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:19.277178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.365339+0000","last_change":"2024-09-10T22:05:07.408801+0000","last_active":"2024-09-10T22:05:10.365339+0000","last_peered":"2024-09-10T22:05:10.365339+0000","last_clean":"2024-09-10T22:05:10.365339+0000","last_became_active":"2024-09-10T22:05:07.407117+0000","last_became_peered":"2024-09-10T22:05:07.407117+0000","last_unstale":"2024-09-10T22:05:10.365339+0000","last_undegraded":"2024-09-10T22:05:10.365339+0000","last_fullsized":"2024-09-10T22:05:10.365339+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:35.379752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.885813+0000","last_change":"2024-09-10T22:05:08.412067+0000","last_active":"2024-09-10T22:05:10.885813+0000","last_peered":"2024-09-10T22:05:10.885813+0000","last_clean":"2024-09-10T22:05:10.885813+0000","last_became_active":"2024-09-10T22:05:08.411721+0000","last_became_peered":"2024-09-10T22:05:08.411721+0000","last_unstale":"2024-09-10T22:05:10.885813+0000","last_undegraded":"2024-09-10T22:05:10.885813+0000","last_fullsized":"2024-09-10T22:05:10.885813+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:55.369971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:10.885802+0000","last_change":"2024-09-10T22:05:07.412991+0000","last_active":"2024-09-10T22:05:10.885802+0000","last_peered":"2024-09-10T22:05:10.885802+0000","last_clean":"2024-09-10T22:05:10.885802+0000","last_became_active":"2024-09-10T22:05:07.412733+0000","last_became_peered":"2024-09-10T22:05:07.412733+0000","last_unstale":"2024-09-10T22:05:10.885802+0000","last_undegraded":"2024-09-10T22:05:10.885802+0000","last_fullsized":"2024-09-10T22:05:10.885802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:31.765144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.970132+0000","last_change":"2024-09-10T22:05:08.430515+0000","last_active":"2024-09-10T22:05:09.970132+0000","last_peered":"2024-09-10T22:05:09.970132+0000","last_clean":"2024-09-10T22:05:09.970132+0000","last_became_active":"2024-09-10T22:05:08.426811+0000","last_became_peered":"2024-09-10T22:05:08.426811+0000","last_unstale":"2024-09-10T22:05:09.970132+0000","last_undegraded":"2024-09-10T22:05:09.970132+0000","last_fullsized":"2024-09-10T22:05:09.970132+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:19.998815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969934+0000","last_change":"2024-09-10T22:05:07.412756+0000","last_active":"2024-09-10T22:05:09.969934+0000","last_peered":"2024-09-10T22:05:09.969934+0000","last_clean":"2024-09-10T22:05:09.969934+0000","last_became_active":"2024-09-10T22:05:07.412506+0000","last_became_peered":"2024-09-10T22:05:07.412506+0000","last_unstale":"2024-09-10T22:05:09.969934+0000","last_undegraded":"2024-09-10T22:05:09.969934+0000","last_fullsized":"2024-09-10T22:05:09.969934+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:28.401204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.969868+0000","last_change":"2024-09-10T22:05:08.432239+0000","last_active":"2024-09-10T22:05:09.969868+0000","last_peered":"2024-09-10T22:05:09.969868+0000","last_clean":"2024-09-10T22:05:09.969868+0000","last_became_active":"2024-09-10T22:05:08.431807+0000","last_became_peered":"2024-09-10T22:05:08.431807+0000","last_unstale":"2024-09-10T22:05:09.969868+0000","last_undegraded":"2024-09-10T22:05:09.969868+0000","last_fullsized":"2024-09-10T22:05:09.969868+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:07.388179+0000","last_clean_scrub_stamp":"2024-09-10T22:05:07.388179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:12.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.407895+0000","last_change":"2024-09-10T22:05:07.412700+0000","last_active":"2024-09-10T22:05:09.407895+0000","last_peered":"2024-09-10T22:05:09.407895+0000","last_clean":"2024-09-10T22:05:09.407895+0000","last_became_active":"2024-09-10T22:05:07.412453+0000","last_became_peered":"2024-09-10T22:05:07.412453+0000","last_unstale":"2024-09-10T22:05:09.407895+0000","last_undegraded":"2024-09-10T22:05:09.407895+0000","last_fullsized":"2024-09-10T22:05:09.407895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:05:06.383777+0000","last_clean_scrub_stamp":"2024-09-10T22:05:06.383777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:54.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":181,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T22:05:09.408400+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:05:09.408400+0000","last_peered":"2024-09-10T22:05:09.408400+0000","last_clean":"2024-09-10T22:05:09.408400+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:05:09.408400+0000","last_undegraded":"2024-09-10T22:05:09.408400+0000","last_fullsized":"2024-09-10T22:05:09.408400+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509501,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44940,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326900,"statfs":{"total":96636764160,"available":96590745600,"internally_reserved":0,"allocated":1302528,"data_stored":881117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45760,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326080,"statfs":{"total":96636764160,"available":96589905920,"internally_reserved":0,"allocated":700416,"data_stored":290589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509501,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40200,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331640,"statfs":{"total":96636764160,"available":96595599360,"internally_reserved":0,"allocated":708608,"data_stored":291241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45776,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326064,"statfs":{"total":96636764160,"available":96589889536,"internally_reserved":0,"allocated":712704,"data_stored":292201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509500,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45768,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326072,"statfs":{"total":96636764160,"available":96589897728,"internally_reserved":0,"allocated":708608,"data_stored":290645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62404,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309436,"statfs":{"total":96636764160,"available":96572862464,"internally_reserved":0,"allocated":704512,"data_stored":290679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509501,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58836,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313004,"statfs":{"total":96636764160,"available":96576516096,"internally_reserved":0,"allocated":1310720,"data_stored":883151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542206,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":708608,"data_stored":290645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:05:12.978 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:05:12.978 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T22:05:13.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.167+0000 7f8743c00640 1 -- 172.21.15.66:0/1307957704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873c15bd40 msgr2=0x7f873c0cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:13.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.167+0000 7f8743c00640 1 --2- 172.21.15.66:0/1307957704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873c15bd40 0x7f873c0cffa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8730009900 tx=0x7f873001c850 comp rx=0 tx=0).stop 2024-09-10T22:05:13.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.167+0000 7f8743c00640 1 -- 172.21.15.66:0/1307957704 shutdown_connections 2024-09-10T22:05:13.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.167+0000 7f8743c00640 1 --2- 172.21.15.66:0/1307957704 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f873c0cf180 0x7f873c0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.167+0000 7f8743c00640 1 --2- 172.21.15.66:0/1307957704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873c15bd40 0x7f873c0cffa0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.167+0000 7f8743c00640 1 -- 172.21.15.66:0/1307957704 >> 172.21.15.66:0/1307957704 conn(0x7f873c0d1c60 msgr2=0x7f873c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:13.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.167+0000 7f8743c00640 1 -- 172.21.15.66:0/1307957704 shutdown_connections 2024-09-10T22:05:13.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.167+0000 7f8743c00640 1 -- 172.21.15.66:0/1307957704 wait complete. 2024-09-10T22:05:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.167+0000 7f8743c00640 1 Processor -- start 2024-09-10T22:05:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.167+0000 7f8743c00640 1 -- start start 2024-09-10T22:05:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8743c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f873c0cf180 0x7f873c166160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8743c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873c0cf830 0x7f873c1f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8743c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f873c15bd40 0x7f873c0b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8743c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f873c168ba0 con 0x7f873c0cf830 2024-09-10T22:05:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8743c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f873c168a20 con 0x7f873c15bd40 2024-09-10T22:05:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8743c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f873c168d20 con 0x7f873c0cf180 2024-09-10T22:05:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f873be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873c0cf830 0x7f873c1f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f873be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873c0cf830 0x7f873c1f7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58418/0 (socket says 172.21.15.66:58418) 2024-09-10T22:05:13.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8741200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f873c0cf180 0x7f873c166160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:13.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f873be00640 1 -- 172.21.15.66:0/3734504741 learned_addr learned my addr 172.21.15.66:0/3734504741 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:13.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8741c00640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f873c15bd40 0x7f873c0b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:13.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f873be00640 1 -- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f873c0cf180 msgr2=0x7f873c166160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:13.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f873be00640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f873c0cf180 0x7f873c166160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f873be00640 1 -- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f873c15bd40 msgr2=0x7f873c0b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:13.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f873be00640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f873c15bd40 0x7f873c0b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f873be00640 1 -- 172.21.15.66:0/3734504741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873c1fb220 con 0x7f873c0cf830 2024-09-10T22:05:13.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f873be00640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873c0cf830 0x7f873c1f7580 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f872800c500 tx=0x7f872800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:13.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8738c00640 1 -- 172.21.15.66:0/3734504741 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728017070 con 0x7f873c0cf830 2024-09-10T22:05:13.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8738c00640 1 -- 172.21.15.66:0/3734504741 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87280053c0 con 0x7f873c0cf830 2024-09-10T22:05:13.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8738c00640 1 -- 172.21.15.66:0/3734504741 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728012890 con 0x7f873c0cf830 2024-09-10T22:05:13.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8743c00640 1 -- 172.21.15.66:0/3734504741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873c1fb4c0 con 0x7f873c0cf830 2024-09-10T22:05:13.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8743c00640 1 -- 172.21.15.66:0/3734504741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f873c1fbfa0 con 0x7f873c0cf830 2024-09-10T22:05:13.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.171+0000 7f8743c00640 1 -- 172.21.15.66:0/3734504741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f873c0aec10 con 0x7f873c0cf830 2024-09-10T22:05:13.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.175+0000 7f8738c00640 1 -- 172.21.15.66:0/3734504741 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f8728003600 con 0x7f873c0cf830 2024-09-10T22:05:13.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.175+0000 7f8738c00640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f871c072e50 0x7f871c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:13.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.175+0000 7f8738c00640 1 -- 172.21.15.66:0/3734504741 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f8728091040 con 0x7f873c0cf830 2024-09-10T22:05:13.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.175+0000 7f8741200640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f871c072e50 0x7f871c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:13.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.175+0000 7f8741200640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f871c072e50 0x7f871c075310 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87300af000 tx=0x7f873001b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:13.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.179+0000 7f8738c00640 1 -- 172.21.15.66:0/3734504741 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f872805db00 con 0x7f873c0cf830 2024-09-10T22:05:13.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.411+0000 7f8743c00640 1 -- 172.21.15.66:0/3734504741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f873c06c7a0 con 0x7f873c0cf830 2024-09-10T22:05:13.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.415+0000 7f8738c00640 1 -- 172.21.15.66:0/3734504741 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v27) ==== 190+0+147 (secure 0 0 0) 0x7f8728062530 con 0x7f873c0cf830 2024-09-10T22:05:13.420 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T22:05:13.421 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw== 2024-09-10T22:05:13.421 INFO:teuthology.orchestra.run.smithi066.stdout: caps mds = "allow" 2024-09-10T22:05:13.421 INFO:teuthology.orchestra.run.smithi066.stdout: caps mon = "allow r" 2024-09-10T22:05:13.421 INFO:teuthology.orchestra.run.smithi066.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T22:05:13.421 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-09-10T22:05:13.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.419+0000 7f8743c00640 1 -- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f871c072e50 msgr2=0x7f871c075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.419+0000 7f8743c00640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f871c072e50 0x7f871c075310 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87300af000 tx=0x7f873001b090 comp rx=0 tx=0).stop 2024-09-10T22:05:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.419+0000 7f8743c00640 1 -- 172.21.15.66:0/3734504741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873c0cf830 msgr2=0x7f873c1f7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.419+0000 7f8743c00640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873c0cf830 0x7f873c1f7580 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f872800c500 tx=0x7f872800c9c0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.419+0000 7f8743c00640 1 -- 172.21.15.66:0/3734504741 shutdown_connections 2024-09-10T22:05:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.419+0000 7f8743c00640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f871c072e50 0x7f871c075310 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.419+0000 7f8743c00640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f873c15bd40 0x7f873c0b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.419+0000 7f8743c00640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873c0cf830 0x7f873c1f7580 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.419+0000 7f8743c00640 1 --2- 172.21.15.66:0/3734504741 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f873c0cf180 0x7f873c166160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.419+0000 7f8743c00640 1 -- 172.21.15.66:0/3734504741 >> 172.21.15.66:0/3734504741 conn(0x7f873c0d1c60 msgr2=0x7f873c0b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.419+0000 7f8743c00640 1 -- 172.21.15.66:0/3734504741 shutdown_connections 2024-09-10T22:05:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.419+0000 7f8743c00640 1 -- 172.21.15.66:0/3734504741 wait complete. 2024-09-10T22:05:13.443 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:05:13.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.583+0000 7f5a2f200640 1 -- 172.21.15.66:0/1235992287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280aee20 msgr2=0x7f5a280ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.583+0000 7f5a2f200640 1 --2- 172.21.15.66:0/1235992287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280aee20 0x7f5a280ca080 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009900 tx=0x7f5a1001c850 comp rx=0 tx=0).stop 2024-09-10T22:05:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.583+0000 7f5a2f200640 1 -- 172.21.15.66:0/1235992287 shutdown_connections 2024-09-10T22:05:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.583+0000 7f5a2f200640 1 --2- 172.21.15.66:0/1235992287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280aee20 0x7f5a280ca080 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.583+0000 7f5a2f200640 1 -- 172.21.15.66:0/1235992287 >> 172.21.15.66:0/1235992287 conn(0x7f5a280cfa30 msgr2=0x7f5a280cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.583+0000 7f5a2f200640 1 -- 172.21.15.66:0/1235992287 shutdown_connections 2024-09-10T22:05:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.583+0000 7f5a2f200640 1 -- 172.21.15.66:0/1235992287 wait complete. 2024-09-10T22:05:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a2f200640 1 Processor -- start 2024-09-10T22:05:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a2f200640 1 -- start start 2024-09-10T22:05:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a2f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280aee20 0x7f5a2814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a2f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a28159900 0x7f5a280af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a2f200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a2815cc80 0x7f5a280afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a2f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2816a400 con 0x7f5a280aee20 2024-09-10T22:05:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a2f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2816a280 con 0x7f5a28159900 2024-09-10T22:05:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a2f200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5a2816a580 con 0x7f5a2815cc80 2024-09-10T22:05:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a27e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280aee20 0x7f5a2814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a27e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280aee20 0x7f5a2814cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58442/0 (socket says 172.21.15.66:58442) 2024-09-10T22:05:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a27e00640 1 -- 172.21.15.66:0/4209011523 learned_addr learned my addr 172.21.15.66:0/4209011523 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a2d200640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a2815cc80 0x7f5a280afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a1fe00640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a28159900 0x7f5a280af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a27e00640 1 -- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a2815cc80 msgr2=0x7f5a280afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a27e00640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a2815cc80 0x7f5a280afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a27e00640 1 -- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a28159900 msgr2=0x7f5a280af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a27e00640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a28159900 0x7f5a280af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a27e00640 1 -- 172.21.15.66:0/4209011523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2815eac0 con 0x7f5a280aee20 2024-09-10T22:05:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a1fe00640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a28159900 0x7f5a280af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:05:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a27e00640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280aee20 0x7f5a2814cea0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800c4b0 tx=0x7f5a1800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a24c00640 1 -- 172.21.15.66:0/4209011523 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a180052d0 con 0x7f5a280aee20 2024-09-10T22:05:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a24c00640 1 -- 172.21.15.66:0/4209011523 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a18005470 con 0x7f5a280aee20 2024-09-10T22:05:13.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a24c00640 1 -- 172.21.15.66:0/4209011523 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18012870 con 0x7f5a280aee20 2024-09-10T22:05:13.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a2f200640 1 -- 172.21.15.66:0/4209011523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a280c5b70 con 0x7f5a280aee20 2024-09-10T22:05:13.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.587+0000 7f5a2f200640 1 -- 172.21.15.66:0/4209011523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a280c6130 con 0x7f5a280aee20 2024-09-10T22:05:13.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.591+0000 7f5a2f200640 1 -- 172.21.15.66:0/4209011523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a280ca080 con 0x7f5a280aee20 2024-09-10T22:05:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.591+0000 7f5a24c00640 1 -- 172.21.15.66:0/4209011523 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f5a18003600 con 0x7f5a280aee20 2024-09-10T22:05:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.591+0000 7f5a24c00640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5a04072e50 0x7f5a04075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.591+0000 7f5a24c00640 1 -- 172.21.15.66:0/4209011523 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5a18094100 con 0x7f5a280aee20 2024-09-10T22:05:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.591+0000 7f5a1fe00640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5a04072e50 0x7f5a04075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.591+0000 7f5a1fe00640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5a04072e50 0x7f5a04075310 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5a100af000 tx=0x7f5a1001b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.595+0000 7f5a24c00640 1 -- 172.21.15.66:0/4209011523 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5a1805e7b0 con 0x7f5a280aee20 2024-09-10T22:05:13.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 -- 172.21.15.66:0/4209011523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a280cd930 con 0x7f5a280aee20 2024-09-10T22:05:13.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a24c00640 1 -- 172.21.15.66:0/4209011523 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5220 (secure 0 0 0) 0x7f5a1805e0b0 con 0x7f5a280aee20 2024-09-10T22:05:13.807 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:05:13.807 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":36,"btime":"2024-09-10T22:05:11:423882+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5414,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/135515157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":135515157},{"type":"v1","addr":"172.21.15.94:6837","nonce":135515157}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1614138608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1614138608},{"type":"v1","addr":"172.21.15.94:6836","nonce":1614138608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1060280836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1060280836},{"type":"v1","addr":"172.21.15.66:6833","nonce":1060280836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:05:07.804373+0000","modified":"2024-09-10T22:05:10.828111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5925},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5925":{"gid":5925,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2880381289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2880381289},{"type":"v1","addr":"172.21.15.66:6835","nonce":2880381289}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5925,"qdb_cluster":[5925]},"id":4}]} 2024-09-10T22:05:13.807 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 36 2024-09-10T22:05:13.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 -- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5a04072e50 msgr2=0x7f5a04075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:13.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5a04072e50 0x7f5a04075310 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5a100af000 tx=0x7f5a1001b090 comp rx=0 tx=0).stop 2024-09-10T22:05:13.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 -- 172.21.15.66:0/4209011523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280aee20 msgr2=0x7f5a2814cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:13.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280aee20 0x7f5a2814cea0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800c4b0 tx=0x7f5a1800c970 comp rx=0 tx=0).stop 2024-09-10T22:05:13.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 -- 172.21.15.66:0/4209011523 shutdown_connections 2024-09-10T22:05:13.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5a04072e50 0x7f5a04075310 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a2815cc80 0x7f5a280afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a28159900 0x7f5a280af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 --2- 172.21.15.66:0/4209011523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280aee20 0x7f5a2814cea0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:13.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 -- 172.21.15.66:0/4209011523 >> 172.21.15.66:0/4209011523 conn(0x7f5a280cfa30 msgr2=0x7f5a2815fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:13.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 -- 172.21.15.66:0/4209011523 shutdown_connections 2024-09-10T22:05:13.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:13.803+0000 7f5a2f200640 1 -- 172.21.15.66:0/4209011523 wait complete. 2024-09-10T22:05:14.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:05:07.804373+0000', 'modified': '2024-09-10T22:05:10.828111+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5925}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5925': {'gid': 5925, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/2880381289', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 2880381289}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 2880381289}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5925, 'qdb_cluster': [5925]} 2024-09-10T22:05:14.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:05:14.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T22:05:14.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 -- 172.21.15.66:0/2055988542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f500015bd40 msgr2=0x7f50000cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 --2- 172.21.15.66:0/2055988542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f500015bd40 0x7f50000cffa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009840 tx=0x7f4ff001c790 comp rx=0 tx=0).stop 2024-09-10T22:05:14.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 -- 172.21.15.66:0/2055988542 shutdown_connections 2024-09-10T22:05:14.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 --2- 172.21.15.66:0/2055988542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f500015bd40 0x7f50000cffa0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 -- 172.21.15.66:0/2055988542 >> 172.21.15.66:0/2055988542 conn(0x7f50000d1c60 msgr2=0x7f50000d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:14.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 -- 172.21.15.66:0/2055988542 shutdown_connections 2024-09-10T22:05:14.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 -- 172.21.15.66:0/2055988542 wait complete. 2024-09-10T22:05:14.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 Processor -- start 2024-09-10T22:05:14.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 -- start start 2024-09-10T22:05:14.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50000cf180 0x7f5000071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50000cf830 0x7f5000159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f500015bd40 0x7f500015a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f500016a580 con 0x7f50000cf830 2024-09-10T22:05:14.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f500016a400 con 0x7f50000cf180 2024-09-10T22:05:14.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f5005e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f500016a700 con 0x7f500015bd40 2024-09-10T22:05:14.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.379+0000 7f4ff7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50000cf830 0x7f5000159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ff7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50000cf830 0x7f5000159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58448/0 (socket says 172.21.15.66:58448) 2024-09-10T22:05:14.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ff7e00640 1 -- 172.21.15.66:0/276450076 learned_addr learned my addr 172.21.15.66:0/276450076 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:14.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ffea00640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50000cf180 0x7f5000071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ff7e00640 1 -- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f500015bd40 msgr2=0x7f500015a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4fff400640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f500015bd40 0x7f500015a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ff7e00640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f500015bd40 0x7f500015a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ff7e00640 1 -- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50000cf180 msgr2=0x7f5000071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ff7e00640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50000cf180 0x7f5000071e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ff7e00640 1 -- 172.21.15.66:0/276450076 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50000b6390 con 0x7f50000cf830 2024-09-10T22:05:14.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4fff400640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f500015bd40 0x7f500015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:14.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ff7e00640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50000cf830 0x7f5000159d70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800c500 tx=0x7f4fe800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:14.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ff7400640 1 -- 172.21.15.66:0/276450076 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8016020 con 0x7f50000cf830 2024-09-10T22:05:14.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ff7400640 1 -- 172.21.15.66:0/276450076 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fe8012920 con 0x7f50000cf830 2024-09-10T22:05:14.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ff7400640 1 -- 172.21.15.66:0/276450076 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8012d50 con 0x7f50000cf830 2024-09-10T22:05:14.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f5005e00640 1 -- 172.21.15.66:0/276450076 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50000b6670 con 0x7f50000cf830 2024-09-10T22:05:14.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f5005e00640 1 -- 172.21.15.66:0/276450076 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50000b6c30 con 0x7f50000cf830 2024-09-10T22:05:14.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.383+0000 7f4ff7400640 1 -- 172.21.15.66:0/276450076 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f4fe801a3b0 con 0x7f50000cf830 2024-09-10T22:05:14.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ff7400640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4fdc072e50 0x7f4fdc075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ff7400640 1 -- 172.21.15.66:0/276450076 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f4fe80945d0 con 0x7f50000cf830 2024-09-10T22:05:14.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 Processor -- start 2024-09-10T22:05:14.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 -- start start 2024-09-10T22:05:14.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ffea00640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4fdc072e50 0x7f4fdc075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ffea00640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4fdc072e50 0x7f4fdc075310 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00af000 tx=0x7f4ff001afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:14.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa39d0 0x55a5b9aa3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa42f0 0x55a5b9a578d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 0x55a5b9a581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a5b99783f0 con 0x55a5b9a57e10 2024-09-10T22:05:14.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a5b98b5b20 con 0x55a5b9aa42f0 2024-09-10T22:05:14.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55a5b98a4f50 con 0x55a5b9aa39d0 2024-09-10T22:05:14.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4fff400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 0x55a5b9a581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ffea00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa39d0 0x55a5b9aa3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ff7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa42f0 0x55a5b9a578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ffea00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa39d0 0x55a5b9aa3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:38342/0 (socket says 172.21.15.66:38342) 2024-09-10T22:05:14.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ffea00640 1 -- 172.21.15.66:0/3725674157 learned_addr learned my addr 172.21.15.66:0/3725674157 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:14.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ff7e00640 1 -- 172.21.15.66:0/3725674157 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa39d0 msgr2=0x55a5b9aa3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:14.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ff7e00640 1 --2- 172.21.15.66:0/3725674157 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa39d0 0x55a5b9aa3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ff7e00640 1 -- 172.21.15.66:0/3725674157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 msgr2=0x55a5b9a581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:14.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ff7e00640 1 --2- 172.21.15.66:0/3725674157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 0x55a5b9a581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ff7e00640 1 -- 172.21.15.66:0/3725674157 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b986c590 con 0x55a5b9aa42f0 2024-09-10T22:05:14.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4fff400640 1 --2- 172.21.15.66:0/3725674157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 0x55a5b9a581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:05:14.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4ff7e00640 1 --2- 172.21.15.66:0/3725674157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa42f0 0x55a5b9a578d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8091af0 tx=0x7f4fe805dfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d33fc296b46ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:14.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4fefe00640 1 -- 172.21.15.66:0/3725674157 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8094310 con 0x55a5b9aa42f0 2024-09-10T22:05:14.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4fefe00640 1 -- 172.21.15.66:0/3725674157 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fe805ec90 con 0x55a5b9aa42f0 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f4fefe00640 1 -- 172.21.15.66:0/3725674157 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe805d9f0 con 0x55a5b9aa42f0 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 -- 172.21.15.66:0/3725674157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa42f0 msgr2=0x55a5b9a578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 --2- 172.21.15.66:0/3725674157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa42f0 0x55a5b9a578d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8091af0 tx=0x7f4fe805dfa0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 -- 172.21.15.66:0/3725674157 shutdown_connections 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 --2- 172.21.15.66:0/3725674157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 0x55a5b9a581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 --2- 172.21.15.66:0/3725674157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa42f0 0x55a5b9a578d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 --2- 172.21.15.66:0/3725674157 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa39d0 0x55a5b9aa3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 -- 172.21.15.66:0/3725674157 >> 172.21.15.66:0/3725674157 conn(0x55a5b9941130 msgr2=0x55a5b9a86e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 -- 172.21.15.66:0/3725674157 shutdown_connections 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.387+0000 7f500927d000 1 -- 172.21.15.66:0/3725674157 wait complete. 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 1 -- start start 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 1 Processor -- start 2024-09-10T22:05:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9a57e10 0x55a5b9a846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa39d0 0x55a5b9a84e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 0x55a5b9a853b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a5b983f5d0 con 0x55a5b9aa42f0 2024-09-10T22:05:14.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a5b98a8d60 con 0x55a5b9a57e10 2024-09-10T22:05:14.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55a5b984b500 con 0x55a5b9aa39d0 2024-09-10T22:05:14.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 10 client.0 _ll_register_callbacks cb 0x55a5b984f810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:05:14.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fff400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 0x55a5b9a853b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fff400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 0x55a5b9a853b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58460/0 (socket says 172.21.15.66:58460) 2024-09-10T22:05:14.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fff400640 1 -- 172.21.15.66:0/2705962273 learned_addr learned my addr 172.21.15.66:0/2705962273 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:14.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4ffea00640 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9a57e10 0x55a5b9a846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4ff7e00640 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa39d0 0x55a5b9a84e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fff400640 1 -- 172.21.15.66:0/2705962273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa39d0 msgr2=0x55a5b9a84e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fff400640 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa39d0 0x55a5b9a84e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fff400640 1 -- 172.21.15.66:0/2705962273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9a57e10 msgr2=0x55a5b9a846a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fff400640 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9a57e10 0x55a5b9a846a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fff400640 1 -- 172.21.15.66:0/2705962273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b9843170 con 0x55a5b9aa42f0 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fff400640 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 0x55a5b9a853b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8010810 tx=0x7f4ff8010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4010944f80b0d5cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fecc00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fecc00640 1 -- 172.21.15.66:0/2705962273 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8003b80 con 0x55a5b9aa42f0 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fecc00640 1 -- 172.21.15.66:0/2705962273 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f4ff8003d20 con 0x55a5b9aa42f0 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 1 -- 172.21.15.66:0/2705962273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a5b983d0f0 con 0x55a5b9aa42f0 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fecc00640 1 -- 172.21.15.66:0/2705962273 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4ff8021050 con 0x55a5b9aa42f0 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 10 client.6057 fetch_fsmap learned FSMap version 36 2024-09-10T22:05:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 1 -- 172.21.15.66:0/2705962273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55a5b9929ef0 con 0x55a5b9aa42f0 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fecc00640 1 -- 172.21.15.66:0/2705962273 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a5b983d0f0 con 0x55a5b9aa42f0 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4fecc00640 1 -- 172.21.15.66:0/2705962273 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f4ff8024020 con 0x55a5b9aa42f0 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 10 client.6057 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 10 client.6057 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 20 client.6057 populate_metadata read hostname 'smithi066' 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.66:6834/2880381289,v1:172.21.15.66:6835/2880381289] conn(0x55a5b9a860f0 0x55a5b9ab5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 4 client.6057 mds_command: new command op to 5925 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f500927d000 1 -- 172.21.15.66:0/2705962273 --> [v2:172.21.15.66:6834/2880381289,v1:172.21.15.66:6835/2880381289] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5b98b7a70 con 0x55a5b9a860f0 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.391+0000 7f4ffea00640 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.66:6834/2880381289,v1:172.21.15.66:6835/2880381289] conn(0x55a5b9a860f0 0x55a5b9ab5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f4fecc00640 10 client.6057 ms_handle_connect on v2:172.21.15.66:6834/2880381289 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f4ffea00640 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.66:6834/2880381289,v1:172.21.15.66:6835/2880381289] conn(0x55a5b9a860f0 0x55a5b9ab5c90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a97563aa0df7afc server_cookie=f25417a4946bcdc1 in_seq=0 out_seq=0 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f4fecc00640 1 -- 172.21.15.66:0/2705962273 <== mds.0 v2:172.21.15.66:6834/2880381289 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5b98b7a70 con 0x55a5b9a860f0 2024-09-10T22:05:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f4fecc00640 10 client.6057 handle_command_reply: tid=0 2024-09-10T22:05:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f500927d000 1 client.6057 shutdown 2024-09-10T22:05:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f500927d000 1 -- 172.21.15.66:0/2705962273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 msgr2=0x55a5b9a853b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f500927d000 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 0x55a5b9a853b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8010810 tx=0x7f4ff8010cd0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f500927d000 1 -- 172.21.15.66:0/2705962273 shutdown_connections 2024-09-10T22:05:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f500927d000 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.66:6834/2880381289,v1:172.21.15.66:6835/2880381289] conn(0x55a5b9a860f0 0x55a5b9ab5c90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f500927d000 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 0x55a5b9a853b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f500927d000 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa39d0 0x55a5b9a84e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f500927d000 1 --2- 172.21.15.66:0/2705962273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9a57e10 0x55a5b9a846a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f4fecc00640 0 client.6057 ms_handle_reset on v2:172.21.15.66:6834/2880381289 2024-09-10T22:05:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f500927d000 1 -- 172.21.15.66:0/2705962273 >> 172.21.15.66:0/2705962273 conn(0x55a5b9941130 msgr2=0x55a5b9a56fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f500927d000 1 -- 172.21.15.66:0/2705962273 shutdown_connections 2024-09-10T22:05:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f500927d000 1 -- 172.21.15.66:0/2705962273 wait complete. 2024-09-10T22:05:14.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.395+0000 7f500927d000 20 client.6057 trim_cache size 0 max 16384 2024-09-10T22:05:14.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 Processor -- start 2024-09-10T22:05:14.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 -- start start 2024-09-10T22:05:14.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 0x55a5b9b15fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa39d0 0x55a5b9b16520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa42f0 0x55a5b9b16a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5000204620 con 0x55a5b9a57e10 2024-09-10T22:05:14.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5000203650 con 0x55a5b9aa39d0 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f50000c9fe0 con 0x55a5b9aa42f0 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4ffea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 0x55a5b9b15fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4ffea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 0x55a5b9b15fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58466/0 (socket says 172.21.15.66:58466) 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4ffea00640 1 -- 172.21.15.66:0/1345524813 learned_addr learned my addr 172.21.15.66:0/1345524813 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4ff7e00640 1 --2- 172.21.15.66:0/1345524813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa39d0 0x55a5b9b16520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4fff400640 1 --2- 172.21.15.66:0/1345524813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa42f0 0x55a5b9b16a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4ffea00640 1 -- 172.21.15.66:0/1345524813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa42f0 msgr2=0x55a5b9b16a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4ffea00640 1 --2- 172.21.15.66:0/1345524813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa42f0 0x55a5b9b16a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4ffea00640 1 -- 172.21.15.66:0/1345524813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa39d0 msgr2=0x55a5b9b16520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4ffea00640 1 --2- 172.21.15.66:0/1345524813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa39d0 0x55a5b9b16520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4ffea00640 1 -- 172.21.15.66:0/1345524813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b9900d90 con 0x55a5b9a57e10 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4ff7e00640 1 --2- 172.21.15.66:0/1345524813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa39d0 0x55a5b9b16520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:14.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4fff400640 1 --2- 172.21.15.66:0/1345524813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa42f0 0x55a5b9b16a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:14.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4ffea00640 1 --2- 172.21.15.66:0/1345524813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 0x55a5b9b15fe0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009600 tx=0x7f4ff00b6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e190e12258c21f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:14.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4fef400640 1 -- 172.21.15.66:0/1345524813 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00b2040 con 0x55a5b9a57e10 2024-09-10T22:05:14.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4fef400640 1 -- 172.21.15.66:0/1345524813 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ff00c0040 con 0x55a5b9a57e10 2024-09-10T22:05:14.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f4fef400640 1 -- 172.21.15.66:0/1345524813 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0004790 con 0x55a5b9a57e10 2024-09-10T22:05:14.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 -- 172.21.15.66:0/1345524813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 msgr2=0x55a5b9b15fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 --2- 172.21.15.66:0/1345524813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 0x55a5b9b15fe0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009600 tx=0x7f4ff00b6000 comp rx=0 tx=0).stop 2024-09-10T22:05:14.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 -- 172.21.15.66:0/1345524813 shutdown_connections 2024-09-10T22:05:14.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 --2- 172.21.15.66:0/1345524813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9aa42f0 0x55a5b9b16a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 --2- 172.21.15.66:0/1345524813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa39d0 0x55a5b9b16520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 --2- 172.21.15.66:0/1345524813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9a57e10 0x55a5b9b15fe0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 -- 172.21.15.66:0/1345524813 >> 172.21.15.66:0/1345524813 conn(0x55a5b9941130 msgr2=0x55a5b9a5fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:14.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 -- 172.21.15.66:0/1345524813 shutdown_connections 2024-09-10T22:05:14.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 -- 172.21.15.66:0/1345524813 wait complete. 2024-09-10T22:05:14.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.415+0000 7f500927d000 1 -- start start 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 1 Processor -- start 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9a57e10 0x55a5b9b15dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa39d0 0x55a5b9b16310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 0x55a5b9b16850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f50000ca630 con 0x55a5b9aa42f0 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a5b9abace0 con 0x55a5b9aa39d0 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55a5b98b48f0 con 0x55a5b9a57e10 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 10 client.0 _ll_register_callbacks cb 0x7f4ff8001c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fff400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 0x55a5b9b16850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4ffea00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9a57e10 0x55a5b9b15dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fff400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 0x55a5b9b16850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58482/0 (socket says 172.21.15.66:58482) 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fff400640 1 -- 172.21.15.66:0/4206423022 learned_addr learned my addr 172.21.15.66:0/4206423022 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4ff7e00640 1 --2- 172.21.15.66:0/4206423022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa39d0 0x55a5b9b16310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fff400640 1 -- 172.21.15.66:0/4206423022 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9a57e10 msgr2=0x55a5b9b15dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fff400640 1 --2- 172.21.15.66:0/4206423022 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9a57e10 0x55a5b9b15dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fff400640 1 -- 172.21.15.66:0/4206423022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa39d0 msgr2=0x55a5b9b16310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fff400640 1 --2- 172.21.15.66:0/4206423022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa39d0 0x55a5b9b16310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fff400640 1 -- 172.21.15.66:0/4206423022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b986ba90 con 0x55a5b9aa42f0 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fff400640 1 --2- 172.21.15.66:0/4206423022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 0x55a5b9b16850 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800ecf0 tx=0x7f4ff800ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3155622ef57c2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fecc00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fecc00640 1 -- 172.21.15.66:0/4206423022 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80054e0 con 0x55a5b9aa42f0 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fecc00640 1 -- 172.21.15.66:0/4206423022 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f4ff8005680 con 0x55a5b9aa42f0 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 1 -- 172.21.15.66:0/4206423022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff00c0040 con 0x55a5b9aa42f0 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fecc00640 1 -- 172.21.15.66:0/4206423022 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4ff8036050 con 0x55a5b9aa42f0 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 10 client.6063 fetch_fsmap learned FSMap version 36 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 1 -- 172.21.15.66:0/4206423022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55a5b984a050 con 0x55a5b9aa42f0 2024-09-10T22:05:14.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fecc00640 1 -- 172.21.15.66:0/4206423022 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ff00c0040 con 0x55a5b9aa42f0 2024-09-10T22:05:14.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fecc00640 1 -- 172.21.15.66:0/4206423022 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f4ff8039020 con 0x55a5b9aa42f0 2024-09-10T22:05:14.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 10 client.6063 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T22:05:14.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 10 client.6063 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:05:14.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 20 client.6063 populate_metadata read hostname 'smithi066' 2024-09-10T22:05:14.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 1 --2- 172.21.15.66:0/4206423022 >> [v2:172.21.15.66:6834/2880381289,v1:172.21.15.66:6835/2880381289] conn(0x55a5b9ab7fd0 0x55a5b9ab83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 4 client.6063 mds_command: new command op to 5925 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:05:14.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f500927d000 1 -- 172.21.15.66:0/4206423022 --> [v2:172.21.15.66:6834/2880381289,v1:172.21.15.66:6835/2880381289] -- command(tid 0: {"prefix": "status"}) -- 0x55a5b98b3520 con 0x55a5b9ab7fd0 2024-09-10T22:05:14.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4ffea00640 1 --2- 172.21.15.66:0/4206423022 >> [v2:172.21.15.66:6834/2880381289,v1:172.21.15.66:6835/2880381289] conn(0x55a5b9ab7fd0 0x55a5b9ab83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4ffea00640 1 --2- 172.21.15.66:0/4206423022 >> [v2:172.21.15.66:6834/2880381289,v1:172.21.15.66:6835/2880381289] conn(0x55a5b9ab7fd0 0x55a5b9ab83b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53b83ce6fa6d2c3f server_cookie=9c43567da0bbe641 in_seq=0 out_seq=0 2024-09-10T22:05:14.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.419+0000 7f4fecc00640 10 client.6063 ms_handle_connect on v2:172.21.15.66:6834/2880381289 2024-09-10T22:05:14.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f4fecc00640 1 -- 172.21.15.66:0/4206423022 <== mds.0 v2:172.21.15.66:6834/2880381289 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55a5b98b3520 con 0x55a5b9ab7fd0 2024-09-10T22:05:14.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f4fecc00640 10 client.6063 handle_command_reply: tid=0 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f500927d000 1 client.6063 shutdown 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f500927d000 1 -- 172.21.15.66:0/4206423022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 msgr2=0x55a5b9b16850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f500927d000 1 --2- 172.21.15.66:0/4206423022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 0x55a5b9b16850 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800ecf0 tx=0x7f4ff800ea30 comp rx=0 tx=0).stop 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f500927d000 1 -- 172.21.15.66:0/4206423022 shutdown_connections 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f500927d000 1 --2- 172.21.15.66:0/4206423022 >> [v2:172.21.15.66:6834/2880381289,v1:172.21.15.66:6835/2880381289] conn(0x55a5b9ab7fd0 0x55a5b9ab83b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f500927d000 1 --2- 172.21.15.66:0/4206423022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a5b9aa42f0 0x55a5b9b16850 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f500927d000 1 --2- 172.21.15.66:0/4206423022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5b9aa39d0 0x55a5b9b16310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f500927d000 1 --2- 172.21.15.66:0/4206423022 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a5b9a57e10 0x55a5b9b15dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f500927d000 1 -- 172.21.15.66:0/4206423022 >> 172.21.15.66:0/4206423022 conn(0x55a5b9941130 msgr2=0x55a5b9aa25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f4fecc00640 0 client.6063 ms_handle_reset on v2:172.21.15.66:6834/2880381289 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f500927d000 1 -- 172.21.15.66:0/4206423022 shutdown_connections 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f500927d000 1 -- 172.21.15.66:0/4206423022 wait complete. 2024-09-10T22:05:14.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f500927d000 20 client.6063 trim_cache size 0 max 16384 2024-09-10T22:05:14.431 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:05:14.431 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:05:14.431 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:05:14.431 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5925, 2024-09-10T22:05:14.431 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:05:14.431 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:05:14.431 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:05:14.431 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 6.0111810800000001, 2024-09-10T22:05:14.431 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 36, 2024-09-10T22:05:14.431 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 59, 2024-09-10T22:05:14.432 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 59, 2024-09-10T22:05:14.432 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 10.469941064 2024-09-10T22:05:14.432 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:05:14.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f5005e00640 1 -- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4fdc072e50 msgr2=0x7f4fdc075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f5005e00640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4fdc072e50 0x7f4fdc075310 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00af000 tx=0x7f4ff001afd0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f5005e00640 1 -- 172.21.15.66:0/276450076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50000cf830 msgr2=0x7f5000159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f5005e00640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50000cf830 0x7f5000159d70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800c500 tx=0x7f4fe800c9c0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f5005e00640 1 -- 172.21.15.66:0/276450076 shutdown_connections 2024-09-10T22:05:14.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f5005e00640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4fdc072e50 0x7f4fdc075310 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f5005e00640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f500015bd40 0x7f500015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f5005e00640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50000cf830 0x7f5000159d70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f5005e00640 1 --2- 172.21.15.66:0/276450076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50000cf180 0x7f5000071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f5005e00640 1 -- 172.21.15.66:0/276450076 >> 172.21.15.66:0/276450076 conn(0x7f50000d1c60 msgr2=0x7f500006e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:14.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f5005e00640 1 -- 172.21.15.66:0/276450076 shutdown_connections 2024-09-10T22:05:14.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.423+0000 7f5005e00640 1 -- 172.21.15.66:0/276450076 wait complete. 2024-09-10T22:05:14.446 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:05:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 -- 172.21.15.66:0/3228387675 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9338162630 msgr2=0x7f93380d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 --2- 172.21.15.66:0/3228387675 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9338162630 0x7f93380d2380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9328009900 tx=0x7f932801c8e0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 -- 172.21.15.66:0/3228387675 shutdown_connections 2024-09-10T22:05:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 --2- 172.21.15.66:0/3228387675 >> v2:172.21.15.94:3300/0 conn(0x7f93380d28c0 0x7f93380b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 --2- 172.21.15.66:0/3228387675 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9338162630 0x7f93380d2380 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 -- 172.21.15.66:0/3228387675 >> 172.21.15.66:0/3228387675 conn(0x7f93380d3da0 msgr2=0x7f93380d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 -- 172.21.15.66:0/3228387675 shutdown_connections 2024-09-10T22:05:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 -- 172.21.15.66:0/3228387675 wait complete. 2024-09-10T22:05:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 Processor -- start 2024-09-10T22:05:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 -- start start 2024-09-10T22:05:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93380d28c0 0x7f933806e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9338162630 0x7f933806ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93381567e0 0x7f93380b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f933816b910 con 0x7f93381567e0 2024-09-10T22:05:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f933816b790 con 0x7f93380d28c0 2024-09-10T22:05:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9337200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f933816ba90 con 0x7f9338162630 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9336800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93381567e0 0x7f93380b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9336800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93381567e0 0x7f93380b7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58490/0 (socket says 172.21.15.66:58490) 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9336800640 1 -- 172.21.15.66:0/535075749 learned_addr learned my addr 172.21.15.66:0/535075749 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9335e00640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93380d28c0 0x7f933806e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f932d400640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9338162630 0x7f933806ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9336800640 1 -- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9338162630 msgr2=0x7f933806ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9336800640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9338162630 0x7f933806ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9336800640 1 -- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93380d28c0 msgr2=0x7f933806e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9336800640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93380d28c0 0x7f933806e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.591+0000 7f9336800640 1 -- 172.21.15.66:0/535075749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93380cac80 con 0x7f93381567e0 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.595+0000 7f9336800640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93381567e0 0x7f93380b7c20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f933000eb20 tx=0x7f933000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.595+0000 7f932ea00640 1 -- 172.21.15.66:0/535075749 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330004b30 con 0x7f93381567e0 2024-09-10T22:05:14.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.595+0000 7f932ea00640 1 -- 172.21.15.66:0/535075749 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9330004cf0 con 0x7f93381567e0 2024-09-10T22:05:14.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.595+0000 7f932ea00640 1 -- 172.21.15.66:0/535075749 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330005280 con 0x7f93381567e0 2024-09-10T22:05:14.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.595+0000 7f9337200640 1 -- 172.21.15.66:0/535075749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9338156cc0 con 0x7f93381567e0 2024-09-10T22:05:14.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.595+0000 7f9337200640 1 -- 172.21.15.66:0/535075749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9338157280 con 0x7f93381567e0 2024-09-10T22:05:14.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.595+0000 7f9337200640 1 -- 172.21.15.66:0/535075749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93380d3920 con 0x7f93381567e0 2024-09-10T22:05:14.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.595+0000 7f932ea00640 1 -- 172.21.15.66:0/535075749 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f9330005420 con 0x7f93381567e0 2024-09-10T22:05:14.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.595+0000 7f932ea00640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9314072e50 0x7f9314075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:14.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.595+0000 7f932ea00640 1 -- 172.21.15.66:0/535075749 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f9330092680 con 0x7f93381567e0 2024-09-10T22:05:14.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.595+0000 7f9335e00640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9314072e50 0x7f9314075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:14.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.599+0000 7f9335e00640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9314072e50 0x7f9314075310 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f93280af000 tx=0x7f932801b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:14.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.599+0000 7f932ea00640 1 -- 172.21.15.66:0/535075749 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f933005f1c0 con 0x7f93381567e0 2024-09-10T22:05:14.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.815+0000 7f9337200640 1 -- 172.21.15.66:0/535075749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f93380d15d0 con 0x7f93381567e0 2024-09-10T22:05:14.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.815+0000 7f932ea00640 1 -- 172.21.15.66:0/535075749 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f9330063bf0 con 0x7f93381567e0 2024-09-10T22:05:14.822 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:05:14.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.819+0000 7f9337200640 1 -- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9314072e50 msgr2=0x7f9314075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.819+0000 7f9337200640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9314072e50 0x7f9314075310 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f93280af000 tx=0x7f932801b120 comp rx=0 tx=0).stop 2024-09-10T22:05:14.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.819+0000 7f9337200640 1 -- 172.21.15.66:0/535075749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93381567e0 msgr2=0x7f93380b7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:14.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.819+0000 7f9337200640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93381567e0 0x7f93380b7c20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f933000eb20 tx=0x7f933000c2d0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.819+0000 7f9337200640 1 -- 172.21.15.66:0/535075749 shutdown_connections 2024-09-10T22:05:14.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.819+0000 7f9337200640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9314072e50 0x7f9314075310 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.819+0000 7f9337200640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93381567e0 0x7f93380b7c20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.819+0000 7f9337200640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9338162630 0x7f933806ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.819+0000 7f9337200640 1 --2- 172.21.15.66:0/535075749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93380d28c0 0x7f933806e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:14.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.819+0000 7f9337200640 1 -- 172.21.15.66:0/535075749 >> 172.21.15.66:0/535075749 conn(0x7f93380d3da0 msgr2=0x7f93380b8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:14.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.819+0000 7f9337200640 1 -- 172.21.15.66:0/535075749 shutdown_connections 2024-09-10T22:05:14.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:14.819+0000 7f9337200640 1 -- 172.21.15.66:0/535075749 wait complete. 2024-09-10T22:05:14.844 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:05:14.845 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:05:14.845 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:05:14.845 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:05:14.845 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:05:14.845 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:05:14.845 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:05:14.845 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:05:14.845 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:05:14.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:14.845 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:05:14.854 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:05:14.855 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:05:14.855 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:05:14.855 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:05:14.855 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:05:14.855 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:05:14.855 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:05:14.855 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:05:14.855 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:05:14.855 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:05:14.855 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:05:14.856 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:05:14.856 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:05:14.856 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:05:14.856 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:05:14.856 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:05:14.856 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:05:14.856 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42160sec preferred_lft 42160sec 2024-09-10T22:05:14.856 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-09-10T22:05:14.856 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:05:14.856 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:05:14.857 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:81:fc:4c:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:05:14.857 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:05:14.857 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:05:14.857 INFO:teuthology.orchestra.run.smithi066.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:05:14.857 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T22:05:14.857 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:05:14.857 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:05:14.857 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T22:05:14.857 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:05:14.857 INFO:teuthology.orchestra.run.smithi066.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:05:14.857 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:05:14.858 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T22:05:14.858 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:05:14.858 INFO:teuthology.orchestra.run.smithi066.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:05:14.858 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:05:14.858 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T22:05:14.858 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:05:14.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:14.859 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:05:14.919 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:05:14.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:14.922 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:05:14.981 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:05:14.981 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:05:14.981 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:05:14.981 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:05:14.981 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:05:14.981 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:05:14.982 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:05:14.982 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:05:14.982 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:05:14.982 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:05:14.982 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:05:14.982 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:05:14.982 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:05:14.982 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:05:14.982 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:05:14.982 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:05:14.983 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:05:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:05:15.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:05:15.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:15.121 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:05:15.130 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:05:15.130 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:05:15.130 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:05:15.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:15.130 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:05:15.197 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T22:05:15.197 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:05:15.197 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:05:15.245 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:05:15.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:05:15.246 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:05:15.297 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:05:15.358 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:05:15.410 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:05:15.411 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:05:15.411 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:05:15.411 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:05:15.411 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:05:15.411 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:05:15.411 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:05:15.411 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:05:15.411 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:05:15.411 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:05:15.411 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:05:15.411 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:05:15.411 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:05:15.412 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:05:15.412 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:05:15.412 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:05:15.412 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:05:15.412 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:05:15.412 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:05:15.412 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:05:15.412 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:05:15.412 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:05:15.412 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:05:15.412 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:05:15.412 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:05:15.413 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:05:15.413 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:05:15.413 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:05:15.413 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:05:15.413 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:05:15.413 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:05:15.413 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:05:15.413 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:05:15.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:15.414 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:05:15.469 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:05:15.470 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T22:05:15.520 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:05:15.541 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:05:15.594 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:05:15.595 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:05:15.595 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:05:15.595 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:05:15.595 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:05:15.595 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:05:15.595 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:05:15.595 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:05:15.595 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:05:15.595 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:05:15.595 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:05:15.595 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:05:15.595 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:05:15.596 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:05:15.596 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:05:15.596 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:05:15.596 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:05:15.596 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:05:15.596 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:05:15.596 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:05:15.596 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:05:15.596 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:05:15.596 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:05:15.596 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:05:15.596 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:05:15.597 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:05:15.597 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:05:15.597 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:05:15.597 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:05:15.597 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:05:15.597 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:05:15.597 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:05:15.597 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:05:15.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:15.598 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:05:15.616 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-09-10T22:05:15.611+0000 7f4c28246580 -1 init, newargv = 0x5619ce03e050 newargc=15 2024-09-10T22:05:15.616 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[40841]: starting ceph client 2024-09-10T22:05:15.623 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[40841]: starting fuse 2024-09-10T22:05:15.625 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-09-10T22:05:15.625 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T22:05:15.628 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:05:15.628 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-09-10T22:05:15.628 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:05:15.629 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-09-10T22:05:15.630 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:05:15.630 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:05:15.630 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:05:15.630 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-09-10T22:05:15.630 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-09-10T22:05:15.630 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-09-10T22:05:15.630 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:05:15.630 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:05:15.630 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-09-10T22:05:15.630 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:05:15.752 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.40841.asok 2024-09-10T22:05:15.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:05:15.763 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40841.asok 2024-09-10T22:05:15.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:15.763 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40841.asok status 2024-09-10T22:05:15.939 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:05:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-09-10T22:05:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:05:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:05:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-09-10T22:05:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-09-10T22:05:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T22:05:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "40841", 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5489, 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 5489 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:05:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1991542724 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1991542724 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.5489 192.168.0.1:0/1991542724", 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/1991542724", 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 36, 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 59, 2024-09-10T22:05:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:05:15.942 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-09-10T22:05:15.942 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-09-10T22:05:15.942 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:05:15.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:15.954 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-09-10T22:05:15.954 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:05:15.963 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-09-10T22:05:15.964 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T22:05:15.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:15.964 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:05:16.029 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T22:05:16.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.219+0000 7fc8f3a00640 1 -- 172.21.15.66:0/3042927691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cf3f0 msgr2=0x7fc8ec0c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:16.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.219+0000 7fc8f3a00640 1 --2- 172.21.15.66:0/3042927691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cf3f0 0x7fc8ec0c79c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009840 tx=0x7fc8d801c7f0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.219+0000 7fc8f3a00640 1 -- 172.21.15.66:0/3042927691 shutdown_connections 2024-09-10T22:05:16.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.219+0000 7fc8f3a00640 1 --2- 172.21.15.66:0/3042927691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cf3f0 0x7fc8ec0c79c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.219+0000 7fc8f3a00640 1 -- 172.21.15.66:0/3042927691 >> 172.21.15.66:0/3042927691 conn(0x7fc8ec0d3e70 msgr2=0x7fc8ec0d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:16.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.219+0000 7fc8f3a00640 1 -- 172.21.15.66:0/3042927691 shutdown_connections 2024-09-10T22:05:16.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.219+0000 7fc8f3a00640 1 -- 172.21.15.66:0/3042927691 wait complete. 2024-09-10T22:05:16.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.219+0000 7fc8f3a00640 1 Processor -- start 2024-09-10T22:05:16.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.219+0000 7fc8f3a00640 1 -- start start 2024-09-10T22:05:16.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f3a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cf3f0 0x7fc8ec15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:16.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f3a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc8ec0d4990 0x7fc8ec0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:16.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f3a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8ec15df50 0x7fc8ec0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:16.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f3a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec0c7f00 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f3a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec0d0ed0 con 0x7fc8ec15df50 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f3a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ec0c8080 con 0x7fc8ec0d4990 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f1000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cf3f0 0x7fc8ec15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f1000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cf3f0 0x7fc8ec15a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58504/0 (socket says 172.21.15.66:58504) 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f1000640 1 -- 172.21.15.66:0/2281883851 learned_addr learned my addr 172.21.15.66:0/2281883851 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f1000640 1 -- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc8ec0d4990 msgr2=0x7fc8ec0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8ebe00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc8ec0d4990 0x7fc8ec0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f1a00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8ec15df50 0x7fc8ec0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f1000640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc8ec0d4990 0x7fc8ec0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f1000640 1 -- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8ec15df50 msgr2=0x7fc8ec0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f1000640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8ec15df50 0x7fc8ec0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f1000640 1 -- 172.21.15.66:0/2281883851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec1614d0 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8ebe00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc8ec0d4990 0x7fc8ec0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:16.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f1a00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8ec15df50 0x7fc8ec0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:16.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f1000640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cf3f0 0x7fc8ec15a3f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000c3c0 tx=0x7fc8e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:16.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8e8c00640 1 -- 172.21.15.66:0/2281883851 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0017070 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8e8c00640 1 -- 172.21.15.66:0/2281883851 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8e0003600 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8e8c00640 1 -- 172.21.15.66:0/2281883851 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0003a50 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f3a00640 1 -- 172.21.15.66:0/2281883851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec0cb620 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f3a00640 1 -- 172.21.15.66:0/2281883851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec0cbbe0 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8f3a00640 1 -- 172.21.15.66:0/2281883851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ec0c79c0 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.223+0000 7fc8e8c00640 1 -- 172.21.15.66:0/2281883851 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fc8e0003bf0 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.227+0000 7fc8e8c00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc8cc072e80 0x7fc8cc075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:16.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.227+0000 7fc8e8c00640 1 -- 172.21.15.66:0/2281883851 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fc8e0093a90 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.227+0000 7fc8ebe00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc8cc072e80 0x7fc8cc075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:16.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.227+0000 7fc8ebe00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc8cc072e80 0x7fc8cc075340 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80af000 tx=0x7fc8d801b030 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:16.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.231+0000 7fc8e8c00640 1 -- 172.21.15.66:0/2281883851 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc8e005e140 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.443+0000 7fc8f3a00640 1 -- 172.21.15.66:0/2281883851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fc8ec15f140 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8e8c00640 1 -- 172.21.15.66:0/2281883851 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) ==== 136+0+0 (secure 0 0 0) 0x7fc8e005da40 con 0x7fc8ec0cf3f0 2024-09-10T22:05:16.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8f3a00640 1 -- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc8cc072e80 msgr2=0x7fc8cc075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:16.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8f3a00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc8cc072e80 0x7fc8cc075340 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80af000 tx=0x7fc8d801b030 comp rx=0 tx=0).stop 2024-09-10T22:05:16.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8f3a00640 1 -- 172.21.15.66:0/2281883851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cf3f0 msgr2=0x7fc8ec15a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:16.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8f3a00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cf3f0 0x7fc8ec15a3f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000c3c0 tx=0x7fc8e000c880 comp rx=0 tx=0).stop 2024-09-10T22:05:16.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8f3a00640 1 -- 172.21.15.66:0/2281883851 shutdown_connections 2024-09-10T22:05:16.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8f3a00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc8cc072e80 0x7fc8cc075340 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8f3a00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8ec15df50 0x7fc8ec0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8f3a00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc8ec0d4990 0x7fc8ec0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8f3a00640 1 --2- 172.21.15.66:0/2281883851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cf3f0 0x7fc8ec15a3f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8f3a00640 1 -- 172.21.15.66:0/2281883851 >> 172.21.15.66:0/2281883851 conn(0x7fc8ec0d3e70 msgr2=0x7fc8ec0c8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:16.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8f3a00640 1 -- 172.21.15.66:0/2281883851 shutdown_connections 2024-09-10T22:05:16.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.499+0000 7fc8f3a00640 1 -- 172.21.15.66:0/2281883851 wait complete. 2024-09-10T22:05:16.525 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:05:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.667+0000 7fce03e00640 1 -- 172.21.15.66:0/2852588057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0415ef20 msgr2=0x7fce040cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.667+0000 7fce03e00640 1 --2- 172.21.15.66:0/2852588057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0415ef20 0x7fce040cecc0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009840 tx=0x7fcdf401c7e0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.667+0000 7fce03e00640 1 -- 172.21.15.66:0/2852588057 shutdown_connections 2024-09-10T22:05:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.667+0000 7fce03e00640 1 --2- 172.21.15.66:0/2852588057 >> v2:172.21.15.94:3300/0 conn(0x7fce040cd7a0 0x7fce040cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.667+0000 7fce03e00640 1 --2- 172.21.15.66:0/2852588057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0415ef20 0x7fce040cecc0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.667+0000 7fce03e00640 1 -- 172.21.15.66:0/2852588057 >> 172.21.15.66:0/2852588057 conn(0x7fce040d1b10 msgr2=0x7fce040d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.667+0000 7fce03e00640 1 -- 172.21.15.66:0/2852588057 shutdown_connections 2024-09-10T22:05:16.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.667+0000 7fce03e00640 1 -- 172.21.15.66:0/2852588057 wait complete. 2024-09-10T22:05:16.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fce03e00640 1 Processor -- start 2024-09-10T22:05:16.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fce03e00640 1 -- start start 2024-09-10T22:05:16.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fce03e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce040cd7a0 0x7fce04148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:16.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fce03e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0415ef20 0x7fce040b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:16.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fce03e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce0415f300 0x7fce040c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:16.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fce03e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce04168ee0 con 0x7fce0415ef20 2024-09-10T22:05:16.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fce03e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce04168d60 con 0x7fce040cd7a0 2024-09-10T22:05:16.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fce03e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fce04169060 con 0x7fce0415f300 2024-09-10T22:05:16.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fcdfbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0415ef20 0x7fce040b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:16.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fcdfbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0415ef20 0x7fce040b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58512/0 (socket says 172.21.15.66:58512) 2024-09-10T22:05:16.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fcdfbe00640 1 -- 172.21.15.66:0/833176769 learned_addr learned my addr 172.21.15.66:0/833176769 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:16.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fce02a00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce040cd7a0 0x7fce04148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:16.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fce03400640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce0415f300 0x7fce040c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:16.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fcdfbe00640 1 -- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce0415f300 msgr2=0x7fce040c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:16.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fcdfbe00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce0415f300 0x7fce040c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fcdfbe00640 1 -- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce040cd7a0 msgr2=0x7fce04148960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:16.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fcdfbe00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce040cd7a0 0x7fce04148960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fcdfbe00640 1 -- 172.21.15.66:0/833176769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0414b950 con 0x7fce0415ef20 2024-09-10T22:05:16.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fcdfbe00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0415ef20 0x7fce040b7180 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00c3a0 tx=0x7fcdec00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:16.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fcdfaa00640 1 -- 172.21.15.66:0/833176769 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec017070 con 0x7fce0415ef20 2024-09-10T22:05:16.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fcdfaa00640 1 -- 172.21.15.66:0/833176769 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdec003600 con 0x7fce0415ef20 2024-09-10T22:05:16.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fcdfaa00640 1 -- 172.21.15.66:0/833176769 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec003a50 con 0x7fce0415ef20 2024-09-10T22:05:16.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fce03e00640 1 -- 172.21.15.66:0/833176769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce040ae930 con 0x7fce0415ef20 2024-09-10T22:05:16.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.671+0000 7fce03e00640 1 -- 172.21.15.66:0/833176769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce040af190 con 0x7fce0415ef20 2024-09-10T22:05:16.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.675+0000 7fce03e00640 1 -- 172.21.15.66:0/833176769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce040cdc10 con 0x7fce0415ef20 2024-09-10T22:05:16.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.675+0000 7fcdfaa00640 1 -- 172.21.15.66:0/833176769 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fcdec003bf0 con 0x7fce0415ef20 2024-09-10T22:05:16.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.675+0000 7fcdfaa00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcde0072e50 0x7fcde0075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:16.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.675+0000 7fcdfaa00640 1 -- 172.21.15.66:0/833176769 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fcdec093a90 con 0x7fce0415ef20 2024-09-10T22:05:16.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.675+0000 7fce02a00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcde0072e50 0x7fcde0075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:16.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.675+0000 7fce02a00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcde0072e50 0x7fcde0075310 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40af000 tx=0x7fcdf401b020 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:16.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.679+0000 7fcdfaa00640 1 -- 172.21.15.66:0/833176769 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcdec05db80 con 0x7fce0415ef20 2024-09-10T22:05:16.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.875+0000 7fce03e00640 1 -- 172.21.15.66:0/833176769 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fce040af360 con 0x7fcde0072e50 2024-09-10T22:05:16.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.875+0000 7fcdfaa00640 1 -- 172.21.15.66:0/833176769 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fce040af360 con 0x7fcde0072e50 2024-09-10T22:05:16.881 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:05:16.881 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:05:16.881 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:05:16.881 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:05:16.881 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:05:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.879+0000 7fce03e00640 1 -- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcde0072e50 msgr2=0x7fcde0075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.879+0000 7fce03e00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcde0072e50 0x7fcde0075310 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40af000 tx=0x7fcdf401b020 comp rx=0 tx=0).stop 2024-09-10T22:05:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.879+0000 7fce03e00640 1 -- 172.21.15.66:0/833176769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0415ef20 msgr2=0x7fce040b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.879+0000 7fce03e00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0415ef20 0x7fce040b7180 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00c3a0 tx=0x7fcdec00c860 comp rx=0 tx=0).stop 2024-09-10T22:05:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.879+0000 7fce03e00640 1 -- 172.21.15.66:0/833176769 shutdown_connections 2024-09-10T22:05:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.879+0000 7fce03e00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcde0072e50 0x7fcde0075310 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.879+0000 7fce03e00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce0415f300 0x7fce040c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.879+0000 7fce03e00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0415ef20 0x7fce040b7180 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.879+0000 7fce03e00640 1 --2- 172.21.15.66:0/833176769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce040cd7a0 0x7fce04148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.879+0000 7fce03e00640 1 -- 172.21.15.66:0/833176769 >> 172.21.15.66:0/833176769 conn(0x7fce040d1b10 msgr2=0x7fce040b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.879+0000 7fce03e00640 1 -- 172.21.15.66:0/833176769 shutdown_connections 2024-09-10T22:05:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:16.879+0000 7fce03e00640 1 -- 172.21.15.66:0/833176769 wait complete. 2024-09-10T22:05:16.905 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:05:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.051+0000 7f1b3b200640 1 -- 172.21.15.66:0/2046016158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b20004d50 msgr2=0x7f1b200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.051+0000 7f1b3b200640 1 --2- 172.21.15.66:0/2046016158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b20004d50 0x7f1b200051d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009f10 tx=0x7f1b3401d5d0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.051+0000 7f1b3a800640 1 -- 172.21.15.66:0/2046016158 reap_dead start 2024-09-10T22:05:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.051+0000 7f1b3b200640 1 -- 172.21.15.66:0/2046016158 shutdown_connections 2024-09-10T22:05:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.051+0000 7f1b3b200640 1 -- 172.21.15.66:0/2046016158 >> 172.21.15.66:0/2046016158 conn(0x7f1b3c0d1b10 msgr2=0x7f1b3c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.051+0000 7f1b3b200640 1 -- 172.21.15.66:0/2046016158 shutdown_connections 2024-09-10T22:05:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.051+0000 7f1b3b200640 1 -- 172.21.15.66:0/2046016158 wait complete. 2024-09-10T22:05:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.051+0000 7f1b3b200640 1 Processor -- start 2024-09-10T22:05:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.051+0000 7f1b3b200640 1 -- start start 2024-09-10T22:05:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b3b200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b20004d50 0x7f1b3c151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b3b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b3c157130 0x7f1b3c157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b3b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b3c1597b0 0x7f1b3c15ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b3b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c0cf200 con 0x7f1b3c157130 2024-09-10T22:05:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b3b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c0cdc30 con 0x7f1b3c1597b0 2024-09-10T22:05:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b3b200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c1515d0 con 0x7f1b20004d50 2024-09-10T22:05:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b33e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b3c157130 0x7f1b3c157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b33e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b3c157130 0x7f1b3c157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58526/0 (socket says 172.21.15.66:58526) 2024-09-10T22:05:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b33e00640 1 -- 172.21.15.66:0/2179734661 learned_addr learned my addr 172.21.15.66:0/2179734661 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b39e00640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b20004d50 0x7f1b3c151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b3a800640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b3c1597b0 0x7f1b3c15ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b33e00640 1 -- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b20004d50 msgr2=0x7f1b3c151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b33e00640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b20004d50 0x7f1b3c151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b33e00640 1 -- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b3c1597b0 msgr2=0x7f1b3c15ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b33e00640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b3c1597b0 0x7f1b3c15ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b33e00640 1 -- 172.21.15.66:0/2179734661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c158710 con 0x7f1b3c157130 2024-09-10T22:05:17.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b33e00640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b3c157130 0x7f1b3c157ee0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1b24006640 tx=0x7f1b240039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:17.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b32000640 1 -- 172.21.15.66:0/2179734661 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24015070 con 0x7f1b3c157130 2024-09-10T22:05:17.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b32000640 1 -- 172.21.15.66:0/2179734661 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b24011840 con 0x7f1b3c157130 2024-09-10T22:05:17.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b32000640 1 -- 172.21.15.66:0/2179734661 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24011c70 con 0x7f1b3c157130 2024-09-10T22:05:17.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b3b200640 1 -- 172.21.15.66:0/2179734661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c1589f0 con 0x7f1b3c157130 2024-09-10T22:05:17.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b3b200640 1 -- 172.21.15.66:0/2179734661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c158fb0 con 0x7f1b3c157130 2024-09-10T22:05:17.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.055+0000 7f1b3b200640 1 -- 172.21.15.66:0/2179734661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b3c0cc660 con 0x7f1b3c157130 2024-09-10T22:05:17.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.059+0000 7f1b32000640 1 -- 172.21.15.66:0/2179734661 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f1b240103c0 con 0x7f1b3c157130 2024-09-10T22:05:17.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.059+0000 7f1b32000640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b18072e80 0x7f1b18075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:17.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.059+0000 7f1b32000640 1 -- 172.21.15.66:0/2179734661 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f1b24092a30 con 0x7f1b3c157130 2024-09-10T22:05:17.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.059+0000 7f1b39e00640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b18072e80 0x7f1b18075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:17.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.059+0000 7f1b39e00640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b18072e80 0x7f1b18075340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009ce0 tx=0x7f1b2c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:17.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.063+0000 7f1b32000640 1 -- 172.21.15.66:0/2179734661 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1b2405f570 con 0x7f1b3c157130 2024-09-10T22:05:17.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.239+0000 7f1b3b200640 1 -- 172.21.15.66:0/2179734661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1b3c072e00 con 0x7f1b3c157130 2024-09-10T22:05:17.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.239+0000 7f1b32000640 1 -- 172.21.15.66:0/2179734661 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f1b24063fa0 con 0x7f1b3c157130 2024-09-10T22:05:17.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.243+0000 7f1b3b200640 1 -- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b18072e80 msgr2=0x7f1b18075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.243+0000 7f1b3b200640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b18072e80 0x7f1b18075340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009ce0 tx=0x7f1b2c0091e0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.243+0000 7f1b3b200640 1 -- 172.21.15.66:0/2179734661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b3c157130 msgr2=0x7f1b3c157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.243+0000 7f1b3b200640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b3c157130 0x7f1b3c157ee0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1b24006640 tx=0x7f1b240039d0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.243+0000 7f1b3b200640 1 -- 172.21.15.66:0/2179734661 shutdown_connections 2024-09-10T22:05:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.243+0000 7f1b3b200640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b18072e80 0x7f1b18075340 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.243+0000 7f1b3b200640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b3c1597b0 0x7f1b3c15ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.243+0000 7f1b3b200640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b3c157130 0x7f1b3c157ee0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.243+0000 7f1b3b200640 1 --2- 172.21.15.66:0/2179734661 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b20004d50 0x7f1b3c151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.243+0000 7f1b3b200640 1 -- 172.21.15.66:0/2179734661 >> 172.21.15.66:0/2179734661 conn(0x7f1b3c0d1b10 msgr2=0x7f1b3c0b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.243+0000 7f1b3b200640 1 -- 172.21.15.66:0/2179734661 shutdown_connections 2024-09-10T22:05:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.243+0000 7f1b3b200640 1 -- 172.21.15.66:0/2179734661 wait complete. 2024-09-10T22:05:17.266 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-10T22:05:17.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.411+0000 7f6f5e400640 1 -- 172.21.15.66:0/1625435219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815f300 msgr2=0x7f6f580b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.411+0000 7f6f5e400640 1 --2- 172.21.15.66:0/1625435219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815f300 0x7f6f580b1090 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009930 tx=0x7f6f4001c8e0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.411+0000 7f6f5e400640 1 -- 172.21.15.66:0/1625435219 shutdown_connections 2024-09-10T22:05:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.411+0000 7f6f5e400640 1 --2- 172.21.15.66:0/1625435219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f580b15d0 0x7f6f580cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.411+0000 7f6f5e400640 1 --2- 172.21.15.66:0/1625435219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815f300 0x7f6f580b1090 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.411+0000 7f6f5e400640 1 -- 172.21.15.66:0/1625435219 >> 172.21.15.66:0/1625435219 conn(0x7f6f580d1b10 msgr2=0x7f6f580d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.411+0000 7f6f5e400640 1 -- 172.21.15.66:0/1625435219 shutdown_connections 2024-09-10T22:05:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.411+0000 7f6f5e400640 1 -- 172.21.15.66:0/1625435219 wait complete. 2024-09-10T22:05:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.411+0000 7f6f5e400640 1 Processor -- start 2024-09-10T22:05:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.411+0000 7f6f5e400640 1 -- start start 2024-09-10T22:05:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.411+0000 7f6f5e400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f580b15d0 0x7f6f581fbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f5e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815ef20 0x7f6f5806e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f5e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5815f300 0x7f6f58156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f5e400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6f58169360 con 0x7f6f5815ef20 2024-09-10T22:05:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f5e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f581691e0 con 0x7f6f5815f300 2024-09-10T22:05:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f5e400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6f581694e0 con 0x7f6f580b15d0 2024-09-10T22:05:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f4fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815ef20 0x7f6f5806e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f4fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815ef20 0x7f6f5806e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58538/0 (socket says 172.21.15.66:58538) 2024-09-10T22:05:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f4fe00640 1 -- 172.21.15.66:0/3825043942 learned_addr learned my addr 172.21.15.66:0/3825043942 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f5da00640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5815f300 0x7f6f58156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f5d000640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f580b15d0 0x7f6f581fbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f4fe00640 1 -- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f580b15d0 msgr2=0x7f6f581fbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f4fe00640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f580b15d0 0x7f6f581fbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f4fe00640 1 -- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5815f300 msgr2=0x7f6f58156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f4fe00640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5815f300 0x7f6f58156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f4fe00640 1 -- 172.21.15.66:0/3825043942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f580b6940 con 0x7f6f5815ef20 2024-09-10T22:05:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f5da00640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5815f300 0x7f6f58156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:05:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f4fe00640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815ef20 0x7f6f5806e9b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6f4001a510 tx=0x7f6f40005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f55600640 1 -- 172.21.15.66:0/3825043942 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f400b2040 con 0x7f6f5815ef20 2024-09-10T22:05:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f55600640 1 -- 172.21.15.66:0/3825043942 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f40004320 con 0x7f6f5815ef20 2024-09-10T22:05:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f5e400640 1 -- 172.21.15.66:0/3825043942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f580b6c20 con 0x7f6f5815ef20 2024-09-10T22:05:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f55600640 1 -- 172.21.15.66:0/3825043942 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f40005280 con 0x7f6f5815ef20 2024-09-10T22:05:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f5e400640 1 -- 172.21.15.66:0/3825043942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f580c8ff0 con 0x7f6f5815ef20 2024-09-10T22:05:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f5e400640 1 -- 172.21.15.66:0/3825043942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f580cd7a0 con 0x7f6f5815ef20 2024-09-10T22:05:17.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.415+0000 7f6f55600640 1 -- 172.21.15.66:0/3825043942 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f6f580d0100 con 0x7f6f5815ef20 2024-09-10T22:05:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.419+0000 7f6f55600640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f34072e50 0x7f6f34075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.419+0000 7f6f55600640 1 -- 172.21.15.66:0/3825043942 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f6f400bc070 con 0x7f6f5815ef20 2024-09-10T22:05:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.419+0000 7f6f5d000640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f34072e50 0x7f6f34075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.419+0000 7f6f5d000640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f34072e50 0x7f6f34075310 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6f48007fd0 tx=0x7f6f48009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:17.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.423+0000 7f6f55600640 1 -- 172.21.15.66:0/3825043942 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6f400164d0 con 0x7f6f5815ef20 2024-09-10T22:05:17.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 -- 172.21.15.66:0/3825043942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f6f58072e00 con 0x7f6f5815ef20 2024-09-10T22:05:17.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f55600640 1 -- 172.21.15.66:0/3825043942 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v34) ==== 116+0+2 (secure 0 0 0) 0x7f6f400f7680 con 0x7f6f5815ef20 2024-09-10T22:05:17.607 INFO:teuthology.orchestra.run.smithi066.stdout:4 2024-09-10T22:05:17.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 -- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f34072e50 msgr2=0x7f6f34075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f34072e50 0x7f6f34075310 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6f48007fd0 tx=0x7f6f48009040 comp rx=0 tx=0).stop 2024-09-10T22:05:17.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 -- 172.21.15.66:0/3825043942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815ef20 msgr2=0x7f6f5806e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815ef20 0x7f6f5806e9b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6f4001a510 tx=0x7f6f40005de0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 -- 172.21.15.66:0/3825043942 shutdown_connections 2024-09-10T22:05:17.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f34072e50 0x7f6f34075310 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5815f300 0x7f6f58156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815ef20 0x7f6f5806e9b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 --2- 172.21.15.66:0/3825043942 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f580b15d0 0x7f6f581fbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 -- 172.21.15.66:0/3825043942 >> 172.21.15.66:0/3825043942 conn(0x7f6f580d1b10 msgr2=0x7f6f581660d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:17.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 -- 172.21.15.66:0/3825043942 shutdown_connections 2024-09-10T22:05:17.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.603+0000 7f6f5e400640 1 -- 172.21.15.66:0/3825043942 wait complete. 2024-09-10T22:05:17.629 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:05:17.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.775+0000 7fe536800640 1 -- 172.21.15.66:0/620396915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe530071230 msgr2=0x7fe53006d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.775+0000 7fe536800640 1 --2- 172.21.15.66:0/620396915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe530071230 0x7fe53006d630 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe518009840 tx=0x7fe53015d120 comp rx=0 tx=0).stop 2024-09-10T22:05:17.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.775+0000 7fe536800640 1 -- 172.21.15.66:0/620396915 shutdown_connections 2024-09-10T22:05:17.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.775+0000 7fe536800640 1 --2- 172.21.15.66:0/620396915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe530071230 0x7fe53006d630 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.775+0000 7fe536800640 1 -- 172.21.15.66:0/620396915 >> 172.21.15.66:0/620396915 conn(0x7fe5300d0b30 msgr2=0x7fe5300d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:17.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.775+0000 7fe536800640 1 -- 172.21.15.66:0/620396915 shutdown_connections 2024-09-10T22:05:17.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.775+0000 7fe536800640 1 -- 172.21.15.66:0/620396915 wait complete. 2024-09-10T22:05:17.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.775+0000 7fe536800640 1 Processor -- start 2024-09-10T22:05:17.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.775+0000 7fe536800640 1 -- start start 2024-09-10T22:05:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe536800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe530071230 0x7fe5300b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe536800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe530156430 0x7fe53014f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe536800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe530160370 0x7fe5300b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe536800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe53016a150 con 0x7fe530156430 2024-09-10T22:05:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe536800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe530169fd0 con 0x7fe530160370 2024-09-10T22:05:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe536800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe53016a2d0 con 0x7fe530071230 2024-09-10T22:05:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe530156430 0x7fe53014f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe530156430 0x7fe53014f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58552/0 (socket says 172.21.15.66:58552) 2024-09-10T22:05:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527e00640 1 -- 172.21.15.66:0/577469387 learned_addr learned my addr 172.21.15.66:0/577469387 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe52fe00640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe530160370 0x7fe5300b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe52f400640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe530071230 0x7fe5300b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527e00640 1 -- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe530071230 msgr2=0x7fe5300b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527e00640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe530071230 0x7fe5300b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527e00640 1 -- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe530160370 msgr2=0x7fe5300b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527e00640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe530160370 0x7fe5300b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527e00640 1 -- 172.21.15.66:0/577469387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5300b40c0 con 0x7fe530156430 2024-09-10T22:05:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe52fe00640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe530160370 0x7fe5300b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe52f400640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe530071230 0x7fe5300b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527e00640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe530156430 0x7fe53014f900 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe518009790 tx=0x7fe5180a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527400640 1 -- 172.21.15.66:0/577469387 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180bb070 con 0x7fe530156430 2024-09-10T22:05:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527400640 1 -- 172.21.15.66:0/577469387 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe518004430 con 0x7fe530156430 2024-09-10T22:05:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe536800640 1 -- 172.21.15.66:0/577469387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5301659e0 con 0x7fe530156430 2024-09-10T22:05:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527400640 1 -- 172.21.15.66:0/577469387 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180b2040 con 0x7fe530156430 2024-09-10T22:05:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe536800640 1 -- 172.21.15.66:0/577469387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe530165fa0 con 0x7fe530156430 2024-09-10T22:05:17.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe536800640 1 -- 172.21.15.66:0/577469387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53006d630 con 0x7fe530156430 2024-09-10T22:05:17.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.779+0000 7fe527400640 1 -- 172.21.15.66:0/577469387 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7fe5180b5070 con 0x7fe530156430 2024-09-10T22:05:17.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.783+0000 7fe527400640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe50c072e50 0x7fe50c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:17.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.783+0000 7fe527400640 1 -- 172.21.15.66:0/577469387 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fe51812db20 con 0x7fe530156430 2024-09-10T22:05:17.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.783+0000 7fe52f400640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe50c072e50 0x7fe50c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:17.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.783+0000 7fe52f400640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe50c072e50 0x7fe50c075310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe520000f70 tx=0x7fe520009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:17.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.787+0000 7fe527400640 1 -- 172.21.15.66:0/577469387 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe518016410 con 0x7fe530156430 2024-09-10T22:05:17.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.959+0000 7fe536800640 1 -- 172.21.15.66:0/577469387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fe53015a2a0 con 0x7fe530156430 2024-09-10T22:05:17.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.963+0000 7fe527400640 1 -- 172.21.15.66:0/577469387 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v35)=0 v35) ==== 131+0+0 (secure 0 0 0) 0x7fe5180f8670 con 0x7fe530156430 2024-09-10T22:05:17.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.963+0000 7fe536800640 1 -- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe50c072e50 msgr2=0x7fe50c075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.963+0000 7fe536800640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe50c072e50 0x7fe50c075310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe520000f70 tx=0x7fe520009040 comp rx=0 tx=0).stop 2024-09-10T22:05:17.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.963+0000 7fe536800640 1 -- 172.21.15.66:0/577469387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe530156430 msgr2=0x7fe53014f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:17.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.963+0000 7fe536800640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe530156430 0x7fe53014f900 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe518009790 tx=0x7fe5180a5730 comp rx=0 tx=0).stop 2024-09-10T22:05:17.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.967+0000 7fe536800640 1 -- 172.21.15.66:0/577469387 shutdown_connections 2024-09-10T22:05:17.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.967+0000 7fe536800640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe50c072e50 0x7fe50c075310 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.967+0000 7fe536800640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe530160370 0x7fe5300b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.967+0000 7fe536800640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe530156430 0x7fe53014f900 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.967+0000 7fe536800640 1 --2- 172.21.15.66:0/577469387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe530071230 0x7fe5300b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:17.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.967+0000 7fe536800640 1 -- 172.21.15.66:0/577469387 >> 172.21.15.66:0/577469387 conn(0x7fe5300d0b30 msgr2=0x7fe53015aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:17.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.967+0000 7fe536800640 1 -- 172.21.15.66:0/577469387 shutdown_connections 2024-09-10T22:05:17.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:17.967+0000 7fe536800640 1 -- 172.21.15.66:0/577469387 wait complete. 2024-09-10T22:05:17.987 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:05:17.981+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:05:17.987 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:05:17.981+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:05:17.987 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:05:17.981+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:05:17.988 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:05:17.981+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:05:17.988 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:05:17.981+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:05:17.990 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-10T22:05:18.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 -- 172.21.15.66:0/1285245102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c900b2b10 msgr2=0x7f2c901673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:18.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 --2- 172.21.15.66:0/1285245102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c900b2b10 0x7f2c901673c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009bf0 tx=0x7f2c8801cbe0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 -- 172.21.15.66:0/1285245102 shutdown_connections 2024-09-10T22:05:18.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 --2- 172.21.15.66:0/1285245102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c900b2b10 0x7f2c901673c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 -- 172.21.15.66:0/1285245102 >> 172.21.15.66:0/1285245102 conn(0x7f2c900d1ce0 msgr2=0x7f2c900d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:18.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 -- 172.21.15.66:0/1285245102 shutdown_connections 2024-09-10T22:05:18.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 -- 172.21.15.66:0/1285245102 wait complete. 2024-09-10T22:05:18.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 Processor -- start 2024-09-10T22:05:18.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 -- start start 2024-09-10T22:05:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c900b1d30 0x7f2c900cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c900b2b10 0x7f2c90152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c901605c0 0x7f2c9006f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9016a6b0 con 0x7f2c900b1d30 2024-09-10T22:05:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9016a530 con 0x7f2c901605c0 2024-09-10T22:05:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c94a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9016a830 con 0x7f2c900b2b10 2024-09-10T22:05:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c8de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c900b1d30 0x7f2c900cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c8de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c900b1d30 0x7f2c900cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58560/0 (socket says 172.21.15.66:58560) 2024-09-10T22:05:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c8de00640 1 -- 172.21.15.66:0/4000045660 learned_addr learned my addr 172.21.15.66:0/4000045660 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c87e00640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c900b2b10 0x7f2c90152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c8e800640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c901605c0 0x7f2c9006f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c8de00640 1 -- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c900b2b10 msgr2=0x7f2c90152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c8de00640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c900b2b10 0x7f2c90152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c8de00640 1 -- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c901605c0 msgr2=0x7f2c9006f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c8de00640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c901605c0 0x7f2c9006f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c8de00640 1 -- 172.21.15.66:0/4000045660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c900ccd00 con 0x7f2c900b1d30 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c8e800640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c901605c0 0x7f2c9006f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.139+0000 7f2c8de00640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c900b1d30 0x7f2c900cc7c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000c410 tx=0x7f2c8000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.143+0000 7f2c86000640 1 -- 172.21.15.66:0/4000045660 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80011790 con 0x7f2c900b1d30 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.143+0000 7f2c86000640 1 -- 172.21.15.66:0/4000045660 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c80017070 con 0x7f2c900b1d30 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.143+0000 7f2c86000640 1 -- 172.21.15.66:0/4000045660 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80011cc0 con 0x7f2c900b1d30 2024-09-10T22:05:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.143+0000 7f2c94a00640 1 -- 172.21.15.66:0/4000045660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c90153520 con 0x7f2c900b1d30 2024-09-10T22:05:18.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.143+0000 7f2c94a00640 1 -- 172.21.15.66:0/4000045660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c901fb900 con 0x7f2c900b1d30 2024-09-10T22:05:18.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.143+0000 7f2c94a00640 1 -- 172.21.15.66:0/4000045660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c900b1620 con 0x7f2c900b1d30 2024-09-10T22:05:18.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.143+0000 7f2c86000640 1 -- 172.21.15.66:0/4000045660 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f2c8001c460 con 0x7f2c900b1d30 2024-09-10T22:05:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.143+0000 7f2c86000640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c6c072e50 0x7f2c6c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.143+0000 7f2c86000640 1 -- 172.21.15.66:0/4000045660 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f2c80094350 con 0x7f2c900b1d30 2024-09-10T22:05:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.147+0000 7f2c87e00640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c6c072e50 0x7f2c6c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.147+0000 7f2c87e00640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c6c072e50 0x7f2c6c075310 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2c78007fd0 tx=0x7f2c78009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:18.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.147+0000 7f2c86000640 1 -- 172.21.15.66:0/4000045660 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2c8005ea90 con 0x7f2c900b1d30 2024-09-10T22:05:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.347+0000 7f2c94a00640 1 -- 172.21.15.66:0/4000045660 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f2c900c5160 con 0x7f2c6c072e50 2024-09-10T22:05:18.362 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:05:18.357+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:05:18.363 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:05:18.357+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:05:18.363 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:05:18.357+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:05:18.363 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:05:18.357+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:05:18.363 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:05:18.357+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:05:18.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.395+0000 7f2c86000640 1 -- 172.21.15.66:0/4000045660 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2c900c5160 con 0x7f2c6c072e50 2024-09-10T22:05:18.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.399+0000 7f2c94a00640 1 -- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c6c072e50 msgr2=0x7f2c6c075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:18.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.399+0000 7f2c94a00640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c6c072e50 0x7f2c6c075310 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2c78007fd0 tx=0x7f2c78009040 comp rx=0 tx=0).stop 2024-09-10T22:05:18.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.399+0000 7f2c94a00640 1 -- 172.21.15.66:0/4000045660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c900b1d30 msgr2=0x7f2c900cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:18.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.399+0000 7f2c94a00640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c900b1d30 0x7f2c900cc7c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000c410 tx=0x7f2c8000c8d0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.399+0000 7f2c94a00640 1 -- 172.21.15.66:0/4000045660 shutdown_connections 2024-09-10T22:05:18.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.399+0000 7f2c94a00640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c6c072e50 0x7f2c6c075310 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.399+0000 7f2c94a00640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c901605c0 0x7f2c9006f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.399+0000 7f2c94a00640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c900b2b10 0x7f2c90152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.399+0000 7f2c94a00640 1 --2- 172.21.15.66:0/4000045660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c900b1d30 0x7f2c900cc7c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.399+0000 7f2c94a00640 1 -- 172.21.15.66:0/4000045660 >> 172.21.15.66:0/4000045660 conn(0x7f2c900d1ce0 msgr2=0x7f2c900afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:18.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.399+0000 7f2c94a00640 1 -- 172.21.15.66:0/4000045660 shutdown_connections 2024-09-10T22:05:18.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.399+0000 7f2c94a00640 1 -- 172.21.15.66:0/4000045660 wait complete. 2024-09-10T22:05:18.423 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:05:18.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.567+0000 7f61b4a00640 1 -- 172.21.15.66:0/3733378406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b015bd40 msgr2=0x7f61b00cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:18.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.567+0000 7f61b4a00640 1 --2- 172.21.15.66:0/3733378406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b015bd40 0x7f61b00cffa0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f61b00747e0 tx=0x7f61a001c830 comp rx=0 tx=0).stop 2024-09-10T22:05:18.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.567+0000 7f61b4a00640 1 -- 172.21.15.66:0/3733378406 shutdown_connections 2024-09-10T22:05:18.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.567+0000 7f61b4a00640 1 --2- 172.21.15.66:0/3733378406 >> v2:172.21.15.94:3300/0 conn(0x7f61b00cf830 0x7f61b00d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.567+0000 7f61b4a00640 1 --2- 172.21.15.66:0/3733378406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b015bd40 0x7f61b00cffa0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.567+0000 7f61b4a00640 1 -- 172.21.15.66:0/3733378406 >> 172.21.15.66:0/3733378406 conn(0x7f61b00d1c60 msgr2=0x7f61b00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:18.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.567+0000 7f61b4a00640 1 -- 172.21.15.66:0/3733378406 shutdown_connections 2024-09-10T22:05:18.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.567+0000 7f61b4a00640 1 -- 172.21.15.66:0/3733378406 wait complete. 2024-09-10T22:05:18.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61b4a00640 1 Processor -- start 2024-09-10T22:05:18.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61b4a00640 1 -- start start 2024-09-10T22:05:18.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61b4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b00aec10 0x7f61b01f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:18.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61b4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b00cf830 0x7f61b006ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:18.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61b4a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61b015bd40 0x7f61b006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:18.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61b4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0168f80 con 0x7f61b00aec10 2024-09-10T22:05:18.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61b4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0168e00 con 0x7f61b00cf830 2024-09-10T22:05:18.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61b4a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f61b0169100 con 0x7f61b015bd40 2024-09-10T22:05:18.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61ade00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b00aec10 0x7f61b01f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61ade00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b00aec10 0x7f61b01f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58564/0 (socket says 172.21.15.66:58564) 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61ade00640 1 -- 172.21.15.66:0/1686969428 learned_addr learned my addr 172.21.15.66:0/1686969428 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61a7e00640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b00cf830 0x7f61b006ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61ae800640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61b015bd40 0x7f61b006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61ade00640 1 -- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61b015bd40 msgr2=0x7f61b006f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61ade00640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61b015bd40 0x7f61b006f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61ade00640 1 -- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b00cf830 msgr2=0x7f61b006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61ade00640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b00cf830 0x7f61b006ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61ade00640 1 -- 172.21.15.66:0/1686969428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b01f8560 con 0x7f61b00aec10 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61ade00640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b00aec10 0x7f61b01f7c60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f61a00b0040 tx=0x7f61a00a5600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61a6000640 1 -- 172.21.15.66:0/1686969428 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a00b2040 con 0x7f61b00aec10 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61a6000640 1 -- 172.21.15.66:0/1686969428 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61a00043b0 con 0x7f61b00aec10 2024-09-10T22:05:18.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61a6000640 1 -- 172.21.15.66:0/1686969428 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0005280 con 0x7f61b00aec10 2024-09-10T22:05:18.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61b4a00640 1 -- 172.21.15.66:0/1686969428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b01f8840 con 0x7f61b00aec10 2024-09-10T22:05:18.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.571+0000 7f61b4a00640 1 -- 172.21.15.66:0/1686969428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b00ca780 con 0x7f61b00aec10 2024-09-10T22:05:18.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.575+0000 7f61b4a00640 1 -- 172.21.15.66:0/1686969428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b00af0a0 con 0x7f61b00aec10 2024-09-10T22:05:18.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.575+0000 7f61a6000640 1 -- 172.21.15.66:0/1686969428 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 97159+0+0 (secure 0 0 0) 0x7f61a0005420 con 0x7f61b00aec10 2024-09-10T22:05:18.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.575+0000 7f61a6000640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f618c072e50 0x7f618c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:18.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.575+0000 7f61a6000640 1 -- 172.21.15.66:0/1686969428 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f61a012d8e0 con 0x7f61b00aec10 2024-09-10T22:05:18.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.575+0000 7f61a7e00640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f618c072e50 0x7f618c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:18.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.579+0000 7f61a7e00640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f618c072e50 0x7f618c075310 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f61b015ab30 tx=0x7f61980091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:18.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.579+0000 7f61a6000640 1 -- 172.21.15.66:0/1686969428 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f61a00164b0 con 0x7f61b00aec10 2024-09-10T22:05:18.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.779+0000 7f61b4a00640 1 -- 172.21.15.66:0/1686969428 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f61b00b0df0 con 0x7f618c072e50 2024-09-10T22:05:18.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61a6000640 1 -- 172.21.15.66:0/1686969428 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f61b00b0df0 con 0x7f618c072e50 2024-09-10T22:05:18.790 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7 2024-09-10T22:05:18.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61b4a00640 1 -- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f618c072e50 msgr2=0x7f618c075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:18.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61b4a00640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f618c072e50 0x7f618c075310 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f61b015ab30 tx=0x7f61980091e0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61b4a00640 1 -- 172.21.15.66:0/1686969428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b00aec10 msgr2=0x7f61b01f7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:18.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61b4a00640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b00aec10 0x7f61b01f7c60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f61a00b0040 tx=0x7f61a00a5600 comp rx=0 tx=0).stop 2024-09-10T22:05:18.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61b4a00640 1 -- 172.21.15.66:0/1686969428 shutdown_connections 2024-09-10T22:05:18.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61b4a00640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f618c072e50 0x7f618c075310 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61b4a00640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61b015bd40 0x7f61b006f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61b4a00640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b00cf830 0x7f61b006ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61b4a00640 1 --2- 172.21.15.66:0/1686969428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b00aec10 0x7f61b01f7c60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:18.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61b4a00640 1 -- 172.21.15.66:0/1686969428 >> 172.21.15.66:0/1686969428 conn(0x7f61b00d1c60 msgr2=0x7f61b00720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:18.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61b4a00640 1 -- 172.21.15.66:0/1686969428 shutdown_connections 2024-09-10T22:05:18.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:05:18.787+0000 7f61b4a00640 1 -- 172.21.15.66:0/1686969428 wait complete. 2024-09-10T22:05:18.811 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7 2024-09-10T22:05:18.811 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:05:18.811 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:05:18.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:18.821 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:05:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:05:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:05:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.17472 s, 150 MB/s 2024-09-10T22:05:26.062 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:05:26.062 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:05:26.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:26.071 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:05:26.379 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:05:26.383 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:05:32.907 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:05:32.907 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:05:32.907 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.78223 s, 158 MB/s 2024-09-10T22:05:32.908 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:05:32.908 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:05:32.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:32.917 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:05:39.801 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:05:39.801 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:05:39.801 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.82377 s, 157 MB/s 2024-09-10T22:05:39.803 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:05:39.803 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:05:39.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:39.811 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:05:46.723 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:05:46.723 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:05:46.723 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.85796 s, 157 MB/s 2024-09-10T22:05:46.724 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:05:46.725 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:05:46.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:46.734 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:05:53.658 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:05:53.658 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:05:53.658 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.86852 s, 156 MB/s 2024-09-10T22:05:53.659 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:05:53.660 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:05:53.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:05:53.668 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:05:56.471 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:05:56.474 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:06:00.634 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:06:00.635 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:06:00.635 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.91302 s, 155 MB/s 2024-09-10T22:06:00.636 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:06:00.636 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:00.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:06:00.645 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:06:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:06:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:06:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.80206 s, 158 MB/s 2024-09-10T22:06:07.505 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:06:07.505 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:07.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:06:07.514 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:06:14.405 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:06:14.405 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:06:14.405 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.83175 s, 157 MB/s 2024-09-10T22:06:14.407 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:06:14.407 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:14.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:06:14.416 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:06:21.322 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:06:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:06:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.85307 s, 157 MB/s 2024-09-10T22:06:21.324 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:06:21.324 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:21.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:06:21.333 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:06:26.522 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:06:26.527 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:06:28.211 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:06:28.211 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:06:28.212 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.82676 s, 157 MB/s 2024-09-10T22:06:28.213 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-10T22:06:28.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.358+0000 7f4cdf400640 1 -- 172.21.15.66:0/2092583310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc0005240 msgr2=0x7f4cc000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:28.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.358+0000 7f4cdf400640 1 --2- 172.21.15.66:0/2092583310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc0005240 0x7f4cc000b9d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000f6c0 tx=0x7f4cd001e610 comp rx=0 tx=0).stop 2024-09-10T22:06:28.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.358+0000 7f4cdf400640 1 -- 172.21.15.66:0/2092583310 shutdown_connections 2024-09-10T22:06:28.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.358+0000 7f4cdf400640 1 --2- 172.21.15.66:0/2092583310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc0005240 0x7f4cc000b9d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.358+0000 7f4cdf400640 1 -- 172.21.15.66:0/2092583310 >> 172.21.15.66:0/2092583310 conn(0x7f4cd80cfaa0 msgr2=0x7f4cd80cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:28.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.358+0000 7f4cdf400640 1 -- 172.21.15.66:0/2092583310 shutdown_connections 2024-09-10T22:06:28.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.358+0000 7f4cdf400640 1 -- 172.21.15.66:0/2092583310 wait complete. 2024-09-10T22:06:28.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdf400640 1 Processor -- start 2024-09-10T22:06:28.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdf400640 1 -- start start 2024-09-10T22:06:28.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdf400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cc0005240 0x7f4cd8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:28.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cd80cdbb0 0x7f4cd80aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:28.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cd815f3a0 0x7f4cd8161170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:28.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdf400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd80cf760 con 0x7f4cd815f3a0 2024-09-10T22:06:28.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdf400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd80cf5e0 con 0x7f4cd80cdbb0 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdf400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd815e220 con 0x7f4cc0005240 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cd815f3a0 0x7f4cd8161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cd815f3a0 0x7f4cd8161170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51596/0 (socket says 172.21.15.66:51596) 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdd400640 1 -- 172.21.15.66:0/4092423364 learned_addr learned my addr 172.21.15.66:0/4092423364 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdca00640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cc0005240 0x7f4cd8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cd7e00640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cd80cdbb0 0x7f4cd80aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdd400640 1 -- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cc0005240 msgr2=0x7f4cd8160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdd400640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cc0005240 0x7f4cd8160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdd400640 1 -- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cd80cdbb0 msgr2=0x7f4cd80aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdd400640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cd80cdbb0 0x7f4cd80aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdd400640 1 -- 172.21.15.66:0/4092423364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd80ce030 con 0x7f4cd815f3a0 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdd400640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cd815f3a0 0x7f4cd8161170 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000f610 tx=0x7f4cd00abc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cd4c00640 1 -- 172.21.15.66:0/4092423364 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd000ad90 con 0x7f4cd815f3a0 2024-09-10T22:06:28.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cd4c00640 1 -- 172.21.15.66:0/4092423364 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cd00ab070 con 0x7f4cd815f3a0 2024-09-10T22:06:28.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cd4c00640 1 -- 172.21.15.66:0/4092423364 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0003be0 con 0x7f4cd815f3a0 2024-09-10T22:06:28.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.362+0000 7f4cdf400640 1 -- 172.21.15.66:0/4092423364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd815f820 con 0x7f4cd815f3a0 2024-09-10T22:06:28.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.366+0000 7f4cdf400640 1 -- 172.21.15.66:0/4092423364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd815fdd0 con 0x7f4cd815f3a0 2024-09-10T22:06:28.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.366+0000 7f4cdf400640 1 -- 172.21.15.66:0/4092423364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd80af440 con 0x7f4cd815f3a0 2024-09-10T22:06:28.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.366+0000 7f4cd4c00640 1 -- 172.21.15.66:0/4092423364 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 97261+0+0 (secure 0 0 0) 0x7f4cd00bd030 con 0x7f4cd815f3a0 2024-09-10T22:06:28.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.366+0000 7f4cd4c00640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4cb8072fa0 0x7f4cb8075460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:28.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.366+0000 7f4cd4c00640 1 -- 172.21.15.66:0/4092423364 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f4cd012bd00 con 0x7f4cd815f3a0 2024-09-10T22:06:28.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.366+0000 7f4cdca00640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4cb8072fa0 0x7f4cb8075460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:28.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.370+0000 7f4cdca00640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4cb8072fa0 0x7f4cb8075460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc001250 tx=0x7f4ccc001120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:28.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.370+0000 7f4cd4c00640 1 -- 172.21.15.66:0/4092423364 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4cd80af440 con 0x7f4cd815f3a0 2024-09-10T22:06:28.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.578+0000 7f4cdf400640 1 -- 172.21.15.66:0/4092423364 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f4cd815e780 con 0x7f4cb8072fa0 2024-09-10T22:06:28.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.590+0000 7f4cd4c00640 1 -- 172.21.15.66:0/4092423364 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4cd815e780 con 0x7f4cb8072fa0 2024-09-10T22:06:28.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.594+0000 7f4cdf400640 1 -- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4cb8072fa0 msgr2=0x7f4cb8075460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:28.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.594+0000 7f4cdf400640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4cb8072fa0 0x7f4cb8075460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc001250 tx=0x7f4ccc001120 comp rx=0 tx=0).stop 2024-09-10T22:06:28.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.594+0000 7f4cdf400640 1 -- 172.21.15.66:0/4092423364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cd815f3a0 msgr2=0x7f4cd8161170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:28.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.594+0000 7f4cdf400640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cd815f3a0 0x7f4cd8161170 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000f610 tx=0x7f4cd00abc40 comp rx=0 tx=0).stop 2024-09-10T22:06:28.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.594+0000 7f4cdf400640 1 -- 172.21.15.66:0/4092423364 shutdown_connections 2024-09-10T22:06:28.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.594+0000 7f4cdf400640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4cb8072fa0 0x7f4cb8075460 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.594+0000 7f4cdf400640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cd815f3a0 0x7f4cd8161170 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.594+0000 7f4cdf400640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cd80cdbb0 0x7f4cd80aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.594+0000 7f4cdf400640 1 --2- 172.21.15.66:0/4092423364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cc0005240 0x7f4cd8160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.594+0000 7f4cdf400640 1 -- 172.21.15.66:0/4092423364 >> 172.21.15.66:0/4092423364 conn(0x7f4cd80cfaa0 msgr2=0x7f4cd80b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:28.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.594+0000 7f4cdf400640 1 -- 172.21.15.66:0/4092423364 shutdown_connections 2024-09-10T22:06:28.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.594+0000 7f4cdf400640 1 -- 172.21.15.66:0/4092423364 wait complete. 2024-09-10T22:06:28.618 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:06:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.762+0000 7f05c8200640 1 -- 172.21.15.66:0/928133680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c015a310 msgr2=0x7f05c00c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.762+0000 7f05c8200640 1 --2- 172.21.15.66:0/928133680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c015a310 0x7f05c00c48b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009840 tx=0x7f05b401c790 comp rx=0 tx=0).stop 2024-09-10T22:06:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.762+0000 7f05c8200640 1 -- 172.21.15.66:0/928133680 shutdown_connections 2024-09-10T22:06:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.762+0000 7f05c8200640 1 --2- 172.21.15.66:0/928133680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c015a310 0x7f05c00c48b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.762+0000 7f05c8200640 1 -- 172.21.15.66:0/928133680 >> 172.21.15.66:0/928133680 conn(0x7f05c00cfa30 msgr2=0x7f05c00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.762+0000 7f05c8200640 1 -- 172.21.15.66:0/928133680 shutdown_connections 2024-09-10T22:06:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.762+0000 7f05c8200640 1 -- 172.21.15.66:0/928133680 wait complete. 2024-09-10T22:06:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c8200640 1 Processor -- start 2024-09-10T22:06:28.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c8200640 1 -- start start 2024-09-10T22:06:28.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c8200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c00aeec0 0x7f05c014af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:28.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c8200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05c015a310 0x7f05c01342c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:28.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c8200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c0149b60 0x7f05c00b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:28.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c8200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0168750 con 0x7f05c0149b60 2024-09-10T22:06:28.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c8200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01685d0 con 0x7f05c00aeec0 2024-09-10T22:06:28.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c8200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f05c01688d0 con 0x7f05c015a310 2024-09-10T22:06:28.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c0149b60 0x7f05c00b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:28.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c5800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c00aeec0 0x7f05c014af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:28.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c5800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c00aeec0 0x7f05c014af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:52702/0 (socket says 172.21.15.66:52702) 2024-09-10T22:06:28.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c0149b60 0x7f05c00b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51600/0 (socket says 172.21.15.66:51600) 2024-09-10T22:06:28.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c5800640 1 -- 172.21.15.66:0/3732299442 learned_addr learned my addr 172.21.15.66:0/3732299442 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:28.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c4e00640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05c015a310 0x7f05c01342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:28.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c6200640 1 -- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05c015a310 msgr2=0x7f05c01342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:28.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c6200640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05c015a310 0x7f05c01342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c6200640 1 -- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c00aeec0 msgr2=0x7f05c014af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c6200640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c00aeec0 0x7f05c014af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c6200640 1 -- 172.21.15.66:0/3732299442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c00c8770 con 0x7f05c0149b60 2024-09-10T22:06:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c5800640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c00aeec0 0x7f05c014af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:06:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c6200640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c0149b60 0x7f05c00b05d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f05b800cb10 tx=0x7f05b800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05bd600640 1 -- 172.21.15.66:0/3732299442 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8016020 con 0x7f05c0149b60 2024-09-10T22:06:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05bd600640 1 -- 172.21.15.66:0/3732299442 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05b800ce50 con 0x7f05c0149b60 2024-09-10T22:06:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c8200640 1 -- 172.21.15.66:0/3732299442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c00c7310 con 0x7f05c0149b60 2024-09-10T22:06:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05bd600640 1 -- 172.21.15.66:0/3732299442 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b800ada0 con 0x7f05c0149b60 2024-09-10T22:06:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c8200640 1 -- 172.21.15.66:0/3732299442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c00c7030 con 0x7f05c0149b60 2024-09-10T22:06:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.766+0000 7f05c8200640 1 -- 172.21.15.66:0/3732299442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c00c48b0 con 0x7f05c0149b60 2024-09-10T22:06:28.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.770+0000 7f05bd600640 1 -- 172.21.15.66:0/3732299442 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 97261+0+0 (secure 0 0 0) 0x7f05b8012420 con 0x7f05c0149b60 2024-09-10T22:06:28.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.770+0000 7f05bd600640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f05a0072f70 0x7f05a0075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:28.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.770+0000 7f05bd600640 1 -- 172.21.15.66:0/3732299442 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f05b8091db0 con 0x7f05c0149b60 2024-09-10T22:06:28.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.770+0000 7f05c5800640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f05a0072f70 0x7f05a0075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.770+0000 7f05c5800640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f05a0072f70 0x7f05a0075430 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f05b401a3c0 tx=0x7f05b40b2040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.774+0000 7f05bd600640 1 -- 172.21.15.66:0/3732299442 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f05b8020020 con 0x7f05c0149b60 2024-09-10T22:06:28.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.974+0000 7f05c8200640 1 -- 172.21.15.66:0/3732299442 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f05c00cae40 con 0x7f05a0072f70 2024-09-10T22:06:28.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05bd600640 1 -- 172.21.15.66:0/3732299442 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f05c00cae40 con 0x7f05a0072f70 2024-09-10T22:06:28.987 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7 2024-09-10T22:06:28.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05c8200640 1 -- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f05a0072f70 msgr2=0x7f05a0075430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05c8200640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f05a0072f70 0x7f05a0075430 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f05b401a3c0 tx=0x7f05b40b2040 comp rx=0 tx=0).stop 2024-09-10T22:06:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05c8200640 1 -- 172.21.15.66:0/3732299442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c0149b60 msgr2=0x7f05c00b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05c8200640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c0149b60 0x7f05c00b05d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f05b800cb10 tx=0x7f05b800cc00 comp rx=0 tx=0).stop 2024-09-10T22:06:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05c8200640 1 -- 172.21.15.66:0/3732299442 shutdown_connections 2024-09-10T22:06:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05c8200640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f05a0072f70 0x7f05a0075430 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05c8200640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c0149b60 0x7f05c00b05d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05c8200640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05c015a310 0x7f05c01342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05c8200640 1 --2- 172.21.15.66:0/3732299442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05c00aeec0 0x7f05c014af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05c8200640 1 -- 172.21.15.66:0/3732299442 >> 172.21.15.66:0/3732299442 conn(0x7f05c00cfa30 msgr2=0x7f05c014b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05c8200640 1 -- 172.21.15.66:0/3732299442 shutdown_connections 2024-09-10T22:06:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:28.982+0000 7f05c8200640 1 -- 172.21.15.66:0/3732299442 wait complete. 2024-09-10T22:06:29.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:06:29.009 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7 2024-09-10T22:06:29.009 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:06:29.022 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/22931ae5-356d-4afd-826f-abb5ad0cd0b7 2024-09-10T22:06:29.022 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="10737418240" 2024-09-10T22:06:29.022 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:29.023 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-10T22:06:29.023 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-10T22:06:29.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.210+0000 7fd1caa00640 1 -- 172.21.15.66:0/4144890001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c415bd40 msgr2=0x7fd1c40cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:29.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.210+0000 7fd1caa00640 1 --2- 172.21.15.66:0/4144890001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c415bd40 0x7fd1c40cffa0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40747e0 tx=0x7fd1b401c770 comp rx=0 tx=0).stop 2024-09-10T22:06:29.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.210+0000 7fd1caa00640 1 -- 172.21.15.66:0/4144890001 shutdown_connections 2024-09-10T22:06:29.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.210+0000 7fd1caa00640 1 --2- 172.21.15.66:0/4144890001 >> v2:172.21.15.94:3300/0 conn(0x7fd1c40cf830 0x7fd1c40d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.210+0000 7fd1caa00640 1 --2- 172.21.15.66:0/4144890001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c415bd40 0x7fd1c40cffa0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.210+0000 7fd1caa00640 1 -- 172.21.15.66:0/4144890001 >> 172.21.15.66:0/4144890001 conn(0x7fd1c40d1c60 msgr2=0x7fd1c40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:29.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.210+0000 7fd1caa00640 1 -- 172.21.15.66:0/4144890001 shutdown_connections 2024-09-10T22:06:29.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.210+0000 7fd1caa00640 1 -- 172.21.15.66:0/4144890001 wait complete. 2024-09-10T22:06:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1caa00640 1 Processor -- start 2024-09-10T22:06:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1caa00640 1 -- start start 2024-09-10T22:06:29.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1caa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1c40aec10 0x7fd1c41580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:29.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1caa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c40cf830 0x7fd1c4159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:29.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1caa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1c415bd40 0x7fd1c41fb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:29.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1caa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4168f80 con 0x7fd1c40cf830 2024-09-10T22:06:29.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1caa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4168e00 con 0x7fd1c415bd40 2024-09-10T22:06:29.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1caa00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c4169100 con 0x7fd1c40aec10 2024-09-10T22:06:29.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c40cf830 0x7fd1c4159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:29.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c40cf830 0x7fd1c4159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51610/0 (socket says 172.21.15.66:51610) 2024-09-10T22:06:29.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1bbe00640 1 -- 172.21.15.66:0/3931645796 learned_addr learned my addr 172.21.15.66:0/3931645796 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:29.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1c8a00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1c415bd40 0x7fd1c41fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:29.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1c3e00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1c40aec10 0x7fd1c41580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:29.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1bbe00640 1 -- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1c40aec10 msgr2=0x7fd1c41580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:29.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1bbe00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1c40aec10 0x7fd1c41580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1bbe00640 1 -- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1c415bd40 msgr2=0x7fd1c41fb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:29.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1bbe00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1c415bd40 0x7fd1c41fb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1bbe00640 1 -- 172.21.15.66:0/3931645796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c406efd0 con 0x7fd1c40cf830 2024-09-10T22:06:29.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1bbe00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c40cf830 0x7fd1c4159b50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00c9a0 tx=0x7fd1ac00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:29.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1c0c00640 1 -- 172.21.15.66:0/3931645796 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac007c70 con 0x7fd1c40cf830 2024-09-10T22:06:29.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1c0c00640 1 -- 172.21.15.66:0/3931645796 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1ac00f450 con 0x7fd1c40cf830 2024-09-10T22:06:29.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1c0c00640 1 -- 172.21.15.66:0/3931645796 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac00f8a0 con 0x7fd1c40cf830 2024-09-10T22:06:29.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.214+0000 7fd1caa00640 1 -- 172.21.15.66:0/3931645796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c414f990 con 0x7fd1c40cf830 2024-09-10T22:06:29.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.218+0000 7fd1caa00640 1 -- 172.21.15.66:0/3931645796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c415a090 con 0x7fd1c40cf830 2024-09-10T22:06:29.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.218+0000 7fd1caa00640 1 -- 172.21.15.66:0/3931645796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c40af0a0 con 0x7fd1c40cf830 2024-09-10T22:06:29.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.218+0000 7fd1c0c00640 1 -- 172.21.15.66:0/3931645796 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 97261+0+0 (secure 0 0 0) 0x7fd1ac00fa40 con 0x7fd1c40cf830 2024-09-10T22:06:29.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.218+0000 7fd1c0c00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd1a0072f70 0x7fd1a0075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:29.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.218+0000 7fd1c0c00640 1 -- 172.21.15.66:0/3931645796 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd1ac0920b0 con 0x7fd1c40cf830 2024-09-10T22:06:29.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.218+0000 7fd1c3e00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd1a0072f70 0x7fd1a0075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:29.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.222+0000 7fd1c3e00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd1a0072f70 0x7fd1a0075430 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40af000 tx=0x7fd1b401afb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:29.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.222+0000 7fd1c0c00640 1 -- 172.21.15.66:0/3931645796 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd1ac014030 con 0x7fd1c40cf830 2024-09-10T22:06:29.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.430+0000 7fd1caa00640 1 -- 172.21.15.66:0/3931645796 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fd1c40b0df0 con 0x7fd1a0072f70 2024-09-10T22:06:29.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.490+0000 7fd1c0c00640 1 -- 172.21.15.66:0/3931645796 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd1c40b0df0 con 0x7fd1a0072f70 2024-09-10T22:06:29.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.494+0000 7fd1caa00640 1 -- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd1a0072f70 msgr2=0x7fd1a0075430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.494+0000 7fd1caa00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd1a0072f70 0x7fd1a0075430 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40af000 tx=0x7fd1b401afb0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.494+0000 7fd1caa00640 1 -- 172.21.15.66:0/3931645796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c40cf830 msgr2=0x7fd1c4159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.494+0000 7fd1caa00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c40cf830 0x7fd1c4159b50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00c9a0 tx=0x7fd1ac00ce60 comp rx=0 tx=0).stop 2024-09-10T22:06:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.494+0000 7fd1caa00640 1 -- 172.21.15.66:0/3931645796 shutdown_connections 2024-09-10T22:06:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.494+0000 7fd1caa00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd1a0072f70 0x7fd1a0075430 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.494+0000 7fd1caa00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1c415bd40 0x7fd1c41fb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.494+0000 7fd1caa00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c40cf830 0x7fd1c4159b50 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.494+0000 7fd1caa00640 1 --2- 172.21.15.66:0/3931645796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1c40aec10 0x7fd1c41580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.494+0000 7fd1caa00640 1 -- 172.21.15.66:0/3931645796 >> 172.21.15.66:0/3931645796 conn(0x7fd1c40d1c60 msgr2=0x7fd1c40b5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.494+0000 7fd1caa00640 1 -- 172.21.15.66:0/3931645796 shutdown_connections 2024-09-10T22:06:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.494+0000 7fd1caa00640 1 -- 172.21.15.66:0/3931645796 wait complete. 2024-09-10T22:06:29.500 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:29.497+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:29.500 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:29.497+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:29.500 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:29.497+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:29.500 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:29.497+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:29.500 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:29.497+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:29.515 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:29.513+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:29.515 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:29.513+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:29.515 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:29.513+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:29.515 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:29.513+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:29.515 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:29.513+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:29.521 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:06:29.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.662+0000 7fc3f2800640 1 -- 172.21.15.66:0/612455343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3ec0d06b0 msgr2=0x7fc3ec0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.662+0000 7fc3f2800640 1 --2- 172.21.15.66:0/612455343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3ec0d06b0 0x7fc3ec0ce170 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0d1940 tx=0x7fc3d401c830 comp rx=0 tx=0).stop 2024-09-10T22:06:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.662+0000 7fc3f2800640 1 -- 172.21.15.66:0/612455343 shutdown_connections 2024-09-10T22:06:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.662+0000 7fc3f2800640 1 --2- 172.21.15.66:0/612455343 >> v2:172.21.15.94:3300/0 conn(0x7fc3ec0afc20 0x7fc3ec0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.662+0000 7fc3f2800640 1 --2- 172.21.15.66:0/612455343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3ec0d06b0 0x7fc3ec0ce170 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.662+0000 7fc3f2800640 1 -- 172.21.15.66:0/612455343 >> 172.21.15.66:0/612455343 conn(0x7fc3ec0d1c60 msgr2=0x7fc3ec0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.662+0000 7fc3f2800640 1 -- 172.21.15.66:0/612455343 shutdown_connections 2024-09-10T22:06:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.662+0000 7fc3f2800640 1 -- 172.21.15.66:0/612455343 wait complete. 2024-09-10T22:06:29.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3f2800640 1 Processor -- start 2024-09-10T22:06:29.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3f2800640 1 -- start start 2024-09-10T22:06:29.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3f2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3ec0afc20 0x7fc3ec1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:29.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3f2800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc3ec0d06b0 0x7fc3ec0c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:29.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3f2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3ec15bd40 0x7fc3ec0b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:29.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3f2800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1692d0 con 0x7fc3ec0afc20 2024-09-10T22:06:29.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3f2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec169150 con 0x7fc3ec15bd40 2024-09-10T22:06:29.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3f2800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ec169450 con 0x7fc3ec0d06b0 2024-09-10T22:06:29.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3eb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3ec0afc20 0x7fc3ec1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:29.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3eb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3ec0afc20 0x7fc3ec1543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51634/0 (socket says 172.21.15.66:51634) 2024-09-10T22:06:29.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3eb400640 1 -- 172.21.15.66:0/2575069297 learned_addr learned my addr 172.21.15.66:0/2575069297 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:29.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3e3e00640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc3ec0d06b0 0x7fc3ec0c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:29.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3ebe00640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3ec15bd40 0x7fc3ec0b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:29.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3eb400640 1 -- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc3ec0d06b0 msgr2=0x7fc3ec0c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:29.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3eb400640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc3ec0d06b0 0x7fc3ec0c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3eb400640 1 -- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3ec15bd40 msgr2=0x7fc3ec0b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:29.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3eb400640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3ec15bd40 0x7fc3ec0b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3eb400640 1 -- 172.21.15.66:0/2575069297 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec0c94a0 con 0x7fc3ec0afc20 2024-09-10T22:06:29.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3eb400640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3ec0afc20 0x7fc3ec1543c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00b3e0 tx=0x7fc3dc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:29.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3e3400640 1 -- 172.21.15.66:0/2575069297 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc015020 con 0x7fc3ec0afc20 2024-09-10T22:06:29.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3e3400640 1 -- 172.21.15.66:0/2575069297 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3dc011940 con 0x7fc3ec0afc20 2024-09-10T22:06:29.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3e3400640 1 -- 172.21.15.66:0/2575069297 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc011d90 con 0x7fc3ec0afc20 2024-09-10T22:06:29.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3f2800640 1 -- 172.21.15.66:0/2575069297 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec0c9780 con 0x7fc3ec0afc20 2024-09-10T22:06:29.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3f2800640 1 -- 172.21.15.66:0/2575069297 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec0c9d40 con 0x7fc3ec0afc20 2024-09-10T22:06:29.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.666+0000 7fc3f2800640 1 -- 172.21.15.66:0/2575069297 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ec0cffa0 con 0x7fc3ec0afc20 2024-09-10T22:06:29.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.670+0000 7fc3e3400640 1 -- 172.21.15.66:0/2575069297 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 97261+0+0 (secure 0 0 0) 0x7fc3dc01a400 con 0x7fc3ec0afc20 2024-09-10T22:06:29.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.670+0000 7fc3e3400640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3c8072f70 0x7fc3c8075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:29.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.670+0000 7fc3e3400640 1 -- 172.21.15.66:0/2575069297 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fc3dc091af0 con 0x7fc3ec0afc20 2024-09-10T22:06:29.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.670+0000 7fc3e3e00640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3c8072f70 0x7fc3c8075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:29.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.670+0000 7fc3e3e00640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3c8072f70 0x7fc3c8075430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40af000 tx=0x7fc3d401b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:29.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.674+0000 7fc3e3400640 1 -- 172.21.15.66:0/2575069297 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc3dc01f020 con 0x7fc3ec0afc20 2024-09-10T22:06:29.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 -- 172.21.15.66:0/2575069297 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc3ec0b0df0 con 0x7fc3ec0afc20 2024-09-10T22:06:29.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3e3400640 1 -- 172.21.15.66:0/2575069297 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1883 (secure 0 0 0) 0x7fc3dc05e780 con 0x7fc3ec0afc20 2024-09-10T22:06:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:29.915 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:06:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:06:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:06:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:06:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:06:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:06:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:06:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:06:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 382, 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:06:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 59, 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 2535, 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 10528838511, 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 21306949632, 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 751787163648, 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 187314532, 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 0, 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 48 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 37, 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:05:16:499594+0000", 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4, 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 4, 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "c", 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 5925 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:06:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:06:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:06:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:06:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:06:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:06:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:06:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:06:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:06:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:06:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 11, 2024-09-10T22:06:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:06:16.394877+0000", 2024-09-10T22:06:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:06:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": {} 2024-09-10T22:06:29.921 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:06:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 -- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3c8072f70 msgr2=0x7fc3c8075430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3c8072f70 0x7fc3c8075430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40af000 tx=0x7fc3d401b070 comp rx=0 tx=0).stop 2024-09-10T22:06:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 -- 172.21.15.66:0/2575069297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3ec0afc20 msgr2=0x7fc3ec1543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3ec0afc20 0x7fc3ec1543c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00b3e0 tx=0x7fc3dc00b8a0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 -- 172.21.15.66:0/2575069297 shutdown_connections 2024-09-10T22:06:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3c8072f70 0x7fc3c8075430 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3ec15bd40 0x7fc3ec0b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc3ec0d06b0 0x7fc3ec0c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 --2- 172.21.15.66:0/2575069297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3ec0afc20 0x7fc3ec1543c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 -- 172.21.15.66:0/2575069297 >> 172.21.15.66:0/2575069297 conn(0x7fc3ec0d1c60 msgr2=0x7fc3ec06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 -- 172.21.15.66:0/2575069297 shutdown_connections 2024-09-10T22:06:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:29.910+0000 7fc3f2800640 1 -- 172.21.15.66:0/2575069297 wait complete. 2024-09-10T22:06:30.504 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:30.505+0000 7f5fc0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:30.504 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:30.505+0000 7f5fc0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:30.505 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:30.505+0000 7f5fc0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:30.505 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:30.505+0000 7f5fc0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:30.505 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:30.505+0000 7f5fc0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:30.937 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:06:31.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.182+0000 7fe996800640 1 -- 172.21.15.66:0/3391813948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe978003ab0 msgr2=0x7fe978003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:31.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.182+0000 7fe996800640 1 --2- 172.21.15.66:0/3391813948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe978003ab0 0x7fe978003ed0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe984001ec0 tx=0x7fe98401cf70 comp rx=0 tx=0).stop 2024-09-10T22:06:31.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.182+0000 7fe996800640 1 -- 172.21.15.66:0/3391813948 shutdown_connections 2024-09-10T22:06:31.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.182+0000 7fe996800640 1 --2- 172.21.15.66:0/3391813948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe978003ab0 0x7fe978003ed0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:31.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.182+0000 7fe996800640 1 -- 172.21.15.66:0/3391813948 >> 172.21.15.66:0/3391813948 conn(0x7fe9900cfcf0 msgr2=0x7fe9900d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:31.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.182+0000 7fe996800640 1 -- 172.21.15.66:0/3391813948 shutdown_connections 2024-09-10T22:06:31.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.186+0000 7fe996800640 1 -- 172.21.15.66:0/3391813948 wait complete. 2024-09-10T22:06:31.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.186+0000 7fe996800640 1 Processor -- start 2024-09-10T22:06:31.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.186+0000 7fe996800640 1 -- start start 2024-09-10T22:06:31.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe996800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe978003ab0 0x7fe990155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:31.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe996800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9901f9fd0 0x7fe9900aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:31.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe996800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9900af2c0 0x7fe99015c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:31.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe996800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe990073750 con 0x7fe9901f9fd0 2024-09-10T22:06:31.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe996800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe9900735d0 con 0x7fe978003ab0 2024-09-10T22:06:31.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe996800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe9900b2ca0 con 0x7fe9900af2c0 2024-09-10T22:06:31.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe994a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9901f9fd0 0x7fe9900aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:31.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe995e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9900af2c0 0x7fe99015c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:31.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe994a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9901f9fd0 0x7fe9900aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51664/0 (socket says 172.21.15.66:51664) 2024-09-10T22:06:31.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe994a00640 1 -- 172.21.15.66:0/2416539070 learned_addr learned my addr 172.21.15.66:0/2416539070 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:31.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe995400640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe978003ab0 0x7fe990155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:31.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe995e00640 1 -- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe978003ab0 msgr2=0x7fe990155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:31.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe995e00640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe978003ab0 0x7fe990155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:31.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe995e00640 1 -- 172.21.15.66:0/2416539070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9901f9fd0 msgr2=0x7fe9900aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:31.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe995e00640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9901f9fd0 0x7fe9900aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:31.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe995e00640 1 -- 172.21.15.66:0/2416539070 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9900af760 con 0x7fe9900af2c0 2024-09-10T22:06:31.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe994a00640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9901f9fd0 0x7fe9900aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:06:31.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe995e00640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9900af2c0 0x7fe99015c890 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe99006e300 tx=0x7fe98800eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:31.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe98d600640 1 -- 172.21.15.66:0/2416539070 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988004b30 con 0x7fe9900af2c0 2024-09-10T22:06:31.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe996800640 1 -- 172.21.15.66:0/2416539070 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9901f94f0 con 0x7fe9900af2c0 2024-09-10T22:06:31.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe98d600640 1 -- 172.21.15.66:0/2416539070 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe988004dc0 con 0x7fe9900af2c0 2024-09-10T22:06:31.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe98d600640 1 -- 172.21.15.66:0/2416539070 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98800ccd0 con 0x7fe9900af2c0 2024-09-10T22:06:31.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.190+0000 7fe996800640 1 -- 172.21.15.66:0/2416539070 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe990156250 con 0x7fe9900af2c0 2024-09-10T22:06:31.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.194+0000 7fe996800640 1 -- 172.21.15.66:0/2416539070 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9900b3100 con 0x7fe9900af2c0 2024-09-10T22:06:31.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.194+0000 7fe98d600640 1 -- 172.21.15.66:0/2416539070 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 19) ==== 97363+0+0 (secure 0 0 0) 0x7fe988025070 con 0x7fe9900af2c0 2024-09-10T22:06:31.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.198+0000 7fe98d600640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe970073090 0x7fe970075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:31.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.198+0000 7fe98d600640 1 -- 172.21.15.66:0/2416539070 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fe988092470 con 0x7fe9900af2c0 2024-09-10T22:06:31.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.198+0000 7fe995400640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe970073090 0x7fe970075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:31.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.206+0000 7fe98d600640 1 -- 172.21.15.66:0/2416539070 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe988022020 con 0x7fe9900af2c0 2024-09-10T22:06:31.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.210+0000 7fe995400640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe970073090 0x7fe970075550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe9780057f0 tx=0x7fe9840a5f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:31.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.574+0000 7fe996800640 1 -- 172.21.15.66:0/2416539070 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe99009dee0 con 0x7fe9900af2c0 2024-09-10T22:06:31.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.574+0000 7fe98d600640 1 -- 172.21.15.66:0/2416539070 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1883 (secure 0 0 0) 0x7fe98805ef50 con 0x7fe9900af2c0 2024-09-10T22:06:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:31.581 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:06:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:06:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:06:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:06:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:06:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 384, 2024-09-10T22:06:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:06:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:06:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:06:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:06:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:06:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 59, 2024-09-10T22:06:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:06:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:06:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:06:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:06:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:06:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:06:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:06:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:06:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:06:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:06:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:06:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:06:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 2535, 2024-09-10T22:06:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 10528838511, 2024-09-10T22:06:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 21306949632, 2024-09-10T22:06:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 751787163648, 2024-09-10T22:06:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:06:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 129300242, 2024-09-10T22:06:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 0, 2024-09-10T22:06:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 32 2024-09-10T22:06:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:06:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 37, 2024-09-10T22:06:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:05:16:499594+0000", 2024-09-10T22:06:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4, 2024-09-10T22:06:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 4, 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "c", 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 5925 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:06:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 11, 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:06:16.394877+0000", 2024-09-10T22:06:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:06:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": {} 2024-09-10T22:06:31.589 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:06:31.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.578+0000 7fe996800640 1 -- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe970073090 msgr2=0x7fe970075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:31.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.578+0000 7fe996800640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe970073090 0x7fe970075550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe9780057f0 tx=0x7fe9840a5f90 comp rx=0 tx=0).stop 2024-09-10T22:06:31.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.578+0000 7fe996800640 1 -- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9900af2c0 msgr2=0x7fe99015c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:31.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.578+0000 7fe996800640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9900af2c0 0x7fe99015c890 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe99006e300 tx=0x7fe98800eec0 comp rx=0 tx=0).stop 2024-09-10T22:06:31.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.582+0000 7fe996800640 1 -- 172.21.15.66:0/2416539070 shutdown_connections 2024-09-10T22:06:31.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.582+0000 7fe996800640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe970073090 0x7fe970075550 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:31.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.582+0000 7fe996800640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9900af2c0 0x7fe99015c890 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:31.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.582+0000 7fe996800640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9901f9fd0 0x7fe9900aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:31.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.582+0000 7fe996800640 1 --2- 172.21.15.66:0/2416539070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe978003ab0 0x7fe990155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:31.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.582+0000 7fe996800640 1 -- 172.21.15.66:0/2416539070 >> 172.21.15.66:0/2416539070 conn(0x7fe9900cfcf0 msgr2=0x7fe9901e8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:31.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.582+0000 7fe996800640 1 -- 172.21.15.66:0/2416539070 shutdown_connections 2024-09-10T22:06:31.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:31.582+0000 7fe996800640 1 -- 172.21.15.66:0/2416539070 wait complete. 2024-09-10T22:06:32.623 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:06:32.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.854+0000 7f1c8f800640 1 -- 172.21.15.66:0/3087467141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c8815ac80 msgr2=0x7f1c8815e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:32.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.854+0000 7f1c8f800640 1 --2- 172.21.15.66:0/3087467141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c8815ac80 0x7f1c8815e5e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1c880cef70 tx=0x7f1c7c01c890 comp rx=0 tx=0).stop 2024-09-10T22:06:32.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.854+0000 7f1c8f800640 1 -- 172.21.15.66:0/3087467141 shutdown_connections 2024-09-10T22:06:32.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.854+0000 7f1c8f800640 1 --2- 172.21.15.66:0/3087467141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c8815ac80 0x7f1c8815e5e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:32.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.854+0000 7f1c8f800640 1 --2- 172.21.15.66:0/3087467141 >> v2:172.21.15.66:3300/0 conn(0x7f1c8815a8a0 0x7f1c880cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:32.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.854+0000 7f1c8f800640 1 -- 172.21.15.66:0/3087467141 >> 172.21.15.66:0/3087467141 conn(0x7f1c880b26c0 msgr2=0x7f1c880b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:32.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.854+0000 7f1c8f800640 1 -- 172.21.15.66:0/3087467141 shutdown_connections 2024-09-10T22:06:32.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.854+0000 7f1c8f800640 1 -- 172.21.15.66:0/3087467141 wait complete. 2024-09-10T22:06:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.858+0000 7f1c8f800640 1 Processor -- start 2024-09-10T22:06:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.858+0000 7f1c8f800640 1 -- start start 2024-09-10T22:06:32.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8f800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c880cb620 0x7f1c880c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:32.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8f800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c8815a8a0 0x7f1c88151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:32.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8f800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8815ac80 0x7f1c88151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:32.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8f800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1c88167770 con 0x7f1c8815ac80 2024-09-10T22:06:32.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8f800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881678f0 con 0x7f1c8815a8a0 2024-09-10T22:06:32.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8f800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1c88167a70 con 0x7f1c880cb620 2024-09-10T22:06:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8815ac80 0x7f1c88151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8815ac80 0x7f1c88151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44948/0 (socket says 172.21.15.66:44948) 2024-09-10T22:06:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8d800640 1 -- 172.21.15.66:0/1138648532 learned_addr learned my addr 172.21.15.66:0/1138648532 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8ce00640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c880cb620 0x7f1c880c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:32.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8d800640 1 -- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c880cb620 msgr2=0x7f1c880c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:32.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8d800640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c880cb620 0x7f1c880c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:32.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8d800640 1 -- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c8815a8a0 msgr2=0x7f1c88151910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:06:32.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8d800640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c8815a8a0 0x7f1c88151910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:32.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8d800640 1 -- 172.21.15.66:0/1138648532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c88148b30 con 0x7f1c8815ac80 2024-09-10T22:06:32.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8d800640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8815ac80 0x7f1c88151e50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000c7c0 tx=0x7f1c8000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:32.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c84c00640 1 -- 172.21.15.66:0/1138648532 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80003b80 con 0x7f1c8815ac80 2024-09-10T22:06:32.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c84c00640 1 -- 172.21.15.66:0/1138648532 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c80003d20 con 0x7f1c8815ac80 2024-09-10T22:06:32.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8f800640 1 -- 172.21.15.66:0/1138648532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c88147db0 con 0x7f1c8815ac80 2024-09-10T22:06:32.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c84c00640 1 -- 172.21.15.66:0/1138648532 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8000aab0 con 0x7f1c8815ac80 2024-09-10T22:06:32.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.862+0000 7f1c8f800640 1 -- 172.21.15.66:0/1138648532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c88148330 con 0x7f1c8815ac80 2024-09-10T22:06:32.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.866+0000 7f1c8f800640 1 -- 172.21.15.66:0/1138648532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c880cea30 con 0x7f1c8815ac80 2024-09-10T22:06:32.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.866+0000 7f1c84c00640 1 -- 172.21.15.66:0/1138648532 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 97414+0+0 (secure 0 0 0) 0x7f1c8000ac50 con 0x7f1c8815ac80 2024-09-10T22:06:32.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.870+0000 7f1c84c00640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1c68073150 0x7f1c68075610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:32.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.870+0000 7f1c84c00640 1 -- 172.21.15.66:0/1138648532 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f1c80091cd0 con 0x7f1c8815ac80 2024-09-10T22:06:32.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.870+0000 7f1c8ce00640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1c68073150 0x7f1c68075610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:32.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.874+0000 7f1c8ce00640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1c68073150 0x7f1c68075610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1c74005d90 tx=0x7f1c7400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:32.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:32.874+0000 7f1c84c00640 1 -- 172.21.15.66:0/1138648532 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1c8005ebb0 con 0x7f1c8815ac80 2024-09-10T22:06:33.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.282+0000 7f1c8f800640 1 -- 172.21.15.66:0/1138648532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1c881572a0 con 0x7f1c8815ac80 2024-09-10T22:06:33.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.282+0000 7f1c84c00640 1 -- 172.21.15.66:0/1138648532 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2183 (secure 0 0 0) 0x7f1c800672d0 con 0x7f1c8815ac80 2024-09-10T22:06:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:33.287 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:06:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:06:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:06:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:06:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:06:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:06:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:06:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 386, 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:06:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 59, 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 2781, 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 11539776857, 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 22917799936, 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 750176313344, 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "read_bytes_sec": 71637419, 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 213529734, 2024-09-10T22:06:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 17, 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 56 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 37, 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:05:16:499594+0000", 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4, 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 4, 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "c", 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:06:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 5925 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:06:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 11, 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:06:16.394877+0000", 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": { 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr-vol-ongoing-clones": { 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "1 ongoing clones - average progress is 4.766% (1s)\n [=...........................] (remaining: 21s)", 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "progress": 0.047660000622272491, 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "add_to_ceph_s": true 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:33.293 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:06:33.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.286+0000 7f1c8f800640 1 -- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1c68073150 msgr2=0x7f1c68075610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:33.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.286+0000 7f1c8f800640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1c68073150 0x7f1c68075610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1c74005d90 tx=0x7f1c7400b390 comp rx=0 tx=0).stop 2024-09-10T22:06:33.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.286+0000 7f1c8f800640 1 -- 172.21.15.66:0/1138648532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8815ac80 msgr2=0x7f1c88151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:33.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.286+0000 7f1c8f800640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8815ac80 0x7f1c88151e50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000c7c0 tx=0x7f1c8000cc80 comp rx=0 tx=0).stop 2024-09-10T22:06:33.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.290+0000 7f1c8f800640 1 -- 172.21.15.66:0/1138648532 shutdown_connections 2024-09-10T22:06:33.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.290+0000 7f1c8f800640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1c68073150 0x7f1c68075610 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:33.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.290+0000 7f1c8f800640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8815ac80 0x7f1c88151e50 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:33.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.290+0000 7f1c8f800640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c8815a8a0 0x7f1c88151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:33.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.290+0000 7f1c8f800640 1 --2- 172.21.15.66:0/1138648532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c880cb620 0x7f1c880c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:33.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.290+0000 7f1c8f800640 1 -- 172.21.15.66:0/1138648532 >> 172.21.15.66:0/1138648532 conn(0x7f1c880b26c0 msgr2=0x7f1c881529d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:33.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.290+0000 7f1c8f800640 1 -- 172.21.15.66:0/1138648532 shutdown_connections 2024-09-10T22:06:33.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.290+0000 7f1c8f800640 1 -- 172.21.15.66:0/1138648532 wait complete. 2024-09-10T22:06:33.329 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-10T22:06:33.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.550+0000 7fdda7800640 1 -- 172.21.15.66:0/1586693349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdda0154860 msgr2=0x7fdda00b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:33.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.550+0000 7fdda7800640 1 --2- 172.21.15.66:0/1586693349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdda0154860 0x7fdda00b33c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009900 tx=0x7fdd8c01c8e0 comp rx=0 tx=0).stop 2024-09-10T22:06:33.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.550+0000 7fdda7800640 1 -- 172.21.15.66:0/1586693349 shutdown_connections 2024-09-10T22:06:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.550+0000 7fdda7800640 1 --2- 172.21.15.66:0/1586693349 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdda0074fe0 0x7fdda00b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.550+0000 7fdda7800640 1 --2- 172.21.15.66:0/1586693349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdda0154860 0x7fdda00b33c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.550+0000 7fdda7800640 1 -- 172.21.15.66:0/1586693349 >> 172.21.15.66:0/1586693349 conn(0x7fdda00b20c0 msgr2=0x7fdda00b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.550+0000 7fdda7800640 1 -- 172.21.15.66:0/1586693349 shutdown_connections 2024-09-10T22:06:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.550+0000 7fdda7800640 1 -- 172.21.15.66:0/1586693349 wait complete. 2024-09-10T22:06:33.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.554+0000 7fdda7800640 1 Processor -- start 2024-09-10T22:06:33.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.554+0000 7fdda7800640 1 -- start start 2024-09-10T22:06:33.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.554+0000 7fdda7800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdda0074fe0 0x7fdda0152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:33.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.554+0000 7fdda7800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda00b53d0 0x7fdda00b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:33.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.554+0000 7fdda7800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdda0154860 0x7fdda00b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:33.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.554+0000 7fdda7800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdda015e970 con 0x7fdda0154860 2024-09-10T22:06:33.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.554+0000 7fdda7800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdda015e7f0 con 0x7fdda00b53d0 2024-09-10T22:06:33.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.554+0000 7fdda7800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdda015eaf0 con 0x7fdda0074fe0 2024-09-10T22:06:33.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.554+0000 7fdda5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdda0154860 0x7fdda00b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:33.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdda5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdda0154860 0x7fdda00b0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44964/0 (socket says 172.21.15.66:44964) 2024-09-10T22:06:33.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdda5800640 1 -- 172.21.15.66:0/2083895484 learned_addr learned my addr 172.21.15.66:0/2083895484 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:33.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdd9fe00640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda00b53d0 0x7fdda00b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:33.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdda4e00640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdda0074fe0 0x7fdda0152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:33.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdda5800640 1 -- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdda0074fe0 msgr2=0x7fdda0152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:33.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdda5800640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdda0074fe0 0x7fdda0152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:33.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdda5800640 1 -- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda00b53d0 msgr2=0x7fdda00b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:33.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdda5800640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda00b53d0 0x7fdda00b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:33.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdda5800640 1 -- 172.21.15.66:0/2083895484 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda01fb2b0 con 0x7fdda0154860 2024-09-10T22:06:33.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdda5800640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdda0154860 0x7fdda00b0c80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c730 tx=0x7fdd9800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdd9cc00640 1 -- 172.21.15.66:0/2083895484 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98003b80 con 0x7fdda0154860 2024-09-10T22:06:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdd9cc00640 1 -- 172.21.15.66:0/2083895484 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd98003d20 con 0x7fdda0154860 2024-09-10T22:06:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdda7800640 1 -- 172.21.15.66:0/2083895484 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda01fb590 con 0x7fdda0154860 2024-09-10T22:06:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdd9cc00640 1 -- 172.21.15.66:0/2083895484 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9800aa50 con 0x7fdda0154860 2024-09-10T22:06:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdda7800640 1 -- 172.21.15.66:0/2083895484 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda01fbb50 con 0x7fdda0154860 2024-09-10T22:06:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.558+0000 7fdda7800640 1 -- 172.21.15.66:0/2083895484 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda015ded0 con 0x7fdda0154860 2024-09-10T22:06:33.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.562+0000 7fdd9cc00640 1 -- 172.21.15.66:0/2083895484 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 97414+0+0 (secure 0 0 0) 0x7fdd9800abf0 con 0x7fdda0154860 2024-09-10T22:06:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.562+0000 7fdd9cc00640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdd80073150 0x7fdd80075610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.562+0000 7fdd9cc00640 1 -- 172.21.15.66:0/2083895484 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fdd98091c00 con 0x7fdda0154860 2024-09-10T22:06:33.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.566+0000 7fdda4e00640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdd80073150 0x7fdd80075610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:33.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.566+0000 7fdda4e00640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdd80073150 0x7fdd80075610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0af000 tx=0x7fdd8c01b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:33.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.570+0000 7fdd9cc00640 1 -- 172.21.15.66:0/2083895484 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdd9805eae0 con 0x7fdda0154860 2024-09-10T22:06:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:33.878+0000 7fdda7800640 1 -- 172.21.15.66:0/2083895484 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fdda015e270 con 0x7fdd80073150 2024-09-10T22:06:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.050+0000 7fdd9cc00640 1 -- 172.21.15.66:0/2083895484 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdda015e270 con 0x7fdd80073150 2024-09-10T22:06:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.050+0000 7fdda7800640 1 -- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdd80073150 msgr2=0x7fdd80075610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.050+0000 7fdda7800640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdd80073150 0x7fdd80075610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0af000 tx=0x7fdd8c01b120 comp rx=0 tx=0).stop 2024-09-10T22:06:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.054+0000 7fdda7800640 1 -- 172.21.15.66:0/2083895484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdda0154860 msgr2=0x7fdda00b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.054+0000 7fdda7800640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdda0154860 0x7fdda00b0c80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c730 tx=0x7fdd9800cbf0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.054+0000 7fdda7800640 1 -- 172.21.15.66:0/2083895484 shutdown_connections 2024-09-10T22:06:34.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.054+0000 7fdda7800640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdd80073150 0x7fdd80075610 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.054+0000 7fdda7800640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdda0154860 0x7fdda00b0c80 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.054+0000 7fdda7800640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda00b53d0 0x7fdda00b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.054+0000 7fdda7800640 1 --2- 172.21.15.66:0/2083895484 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdda0074fe0 0x7fdda0152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.054+0000 7fdda7800640 1 -- 172.21.15.66:0/2083895484 >> 172.21.15.66:0/2083895484 conn(0x7fdda00b20c0 msgr2=0x7fdda0159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:34.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.054+0000 7fdda7800640 1 -- 172.21.15.66:0/2083895484 shutdown_connections 2024-09-10T22:06:34.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.054+0000 7fdda7800640 1 -- 172.21.15.66:0/2083895484 wait complete. 2024-09-10T22:06:34.077 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:06:34.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.218+0000 7f2057000640 1 -- 172.21.15.66:0/730160482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20500b15d0 msgr2=0x7f20500cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.218+0000 7f2057000640 1 --2- 172.21.15.66:0/730160482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20500b15d0 0x7f20500cd7a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2048009bc0 tx=0x7f204801cba0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.218+0000 7f2057000640 1 -- 172.21.15.66:0/730160482 shutdown_connections 2024-09-10T22:06:34.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.218+0000 7f2057000640 1 --2- 172.21.15.66:0/730160482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20500b15d0 0x7f20500cd7a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.218+0000 7f2057000640 1 --2- 172.21.15.66:0/730160482 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f205015f300 0x7f20500b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.218+0000 7f2057000640 1 -- 172.21.15.66:0/730160482 >> 172.21.15.66:0/730160482 conn(0x7f20500d1b10 msgr2=0x7f20500d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:34.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.218+0000 7f2057000640 1 -- 172.21.15.66:0/730160482 shutdown_connections 2024-09-10T22:06:34.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.218+0000 7f2057000640 1 -- 172.21.15.66:0/730160482 wait complete. 2024-09-10T22:06:34.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2057000640 1 Processor -- start 2024-09-10T22:06:34.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2057000640 1 -- start start 2024-09-10T22:06:34.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2057000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20500b15d0 0x7f2050147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:34.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2057000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f205015ef20 0x7f20500c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:34.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2057000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f205015f300 0x7f20500aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:34.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2057000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f20501693b0 con 0x7f20500b15d0 2024-09-10T22:06:34.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2057000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2050169230 con 0x7f205015ef20 2024-09-10T22:06:34.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2057000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2050169530 con 0x7f205015f300 2024-09-10T22:06:34.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2055c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20500b15d0 0x7f2050147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:34.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2055c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20500b15d0 0x7f2050147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44992/0 (socket says 172.21.15.66:44992) 2024-09-10T22:06:34.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2055c00640 1 -- 172.21.15.66:0/2855961437 learned_addr learned my addr 172.21.15.66:0/2855961437 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:34.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2055200640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f205015ef20 0x7f20500c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:34.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2056600640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f205015f300 0x7f20500aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:34.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2055c00640 1 -- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f205015f300 msgr2=0x7f20500aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2055c00640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f205015f300 0x7f20500aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2055c00640 1 -- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f205015ef20 msgr2=0x7f20500c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2055c00640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f205015ef20 0x7f20500c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2055c00640 1 -- 172.21.15.66:0/2855961437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2050157c70 con 0x7f20500b15d0 2024-09-10T22:06:34.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2055c00640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20500b15d0 0x7f2050147d60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f204400c490 tx=0x7f204400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:34.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f204d600640 1 -- 172.21.15.66:0/2855961437 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2044016020 con 0x7f20500b15d0 2024-09-10T22:06:34.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f204d600640 1 -- 172.21.15.66:0/2855961437 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2044012940 con 0x7f20500b15d0 2024-09-10T22:06:34.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f204d600640 1 -- 172.21.15.66:0/2855961437 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2044012d90 con 0x7f20500b15d0 2024-09-10T22:06:34.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2057000640 1 -- 172.21.15.66:0/2855961437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2050157f50 con 0x7f20500b15d0 2024-09-10T22:06:34.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2057000640 1 -- 172.21.15.66:0/2855961437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2050158510 con 0x7f20500b15d0 2024-09-10T22:06:34.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.222+0000 7f2057000640 1 -- 172.21.15.66:0/2855961437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20500cd7a0 con 0x7f20500b15d0 2024-09-10T22:06:34.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.226+0000 7f204d600640 1 -- 172.21.15.66:0/2855961437 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 97414+0+0 (secure 0 0 0) 0x7f204401b400 con 0x7f20500b15d0 2024-09-10T22:06:34.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.226+0000 7f204d600640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2030073120 0x7f20300755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:34.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.226+0000 7f204d600640 1 -- 172.21.15.66:0/2855961437 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f2044091d40 con 0x7f20500b15d0 2024-09-10T22:06:34.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.226+0000 7f2055200640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2030073120 0x7f20300755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:34.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.226+0000 7f2055200640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2030073120 0x7f20300755e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f20500b5e60 tx=0x7f203c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:34.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.230+0000 7f204d600640 1 -- 172.21.15.66:0/2855961437 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2044020020 con 0x7f20500b15d0 2024-09-10T22:06:34.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.430+0000 7f2057000640 1 -- 172.21.15.66:0/2855961437 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f20500cf020 con 0x7f2030073120 2024-09-10T22:06:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.430+0000 7f204d600640 1 -- 172.21.15.66:0/2855961437 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f20500cf020 con 0x7f2030073120 2024-09-10T22:06:34.437 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:06:34.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:34.437 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:06:34.437 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:34.437 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:06:34.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.434+0000 7f2057000640 1 -- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2030073120 msgr2=0x7f20300755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.434+0000 7f2057000640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2030073120 0x7f20300755e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f20500b5e60 tx=0x7f203c0091e0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.434+0000 7f2057000640 1 -- 172.21.15.66:0/2855961437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20500b15d0 msgr2=0x7f2050147d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.434+0000 7f2057000640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20500b15d0 0x7f2050147d60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f204400c490 tx=0x7f204400c950 comp rx=0 tx=0).stop 2024-09-10T22:06:34.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.434+0000 7f2057000640 1 -- 172.21.15.66:0/2855961437 shutdown_connections 2024-09-10T22:06:34.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.434+0000 7f2057000640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2030073120 0x7f20300755e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.434+0000 7f2057000640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f205015f300 0x7f20500aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.434+0000 7f2057000640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f205015ef20 0x7f20500c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.434+0000 7f2057000640 1 --2- 172.21.15.66:0/2855961437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20500b15d0 0x7f2050147d60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.434+0000 7f2057000640 1 -- 172.21.15.66:0/2855961437 >> 172.21.15.66:0/2855961437 conn(0x7f20500d1b10 msgr2=0x7f20500b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:34.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.434+0000 7f2057000640 1 -- 172.21.15.66:0/2855961437 shutdown_connections 2024-09-10T22:06:34.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.434+0000 7f2057000640 1 -- 172.21.15.66:0/2855961437 wait complete. 2024-09-10T22:06:34.458 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-10T22:06:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.602+0000 7fa914e00640 1 -- 172.21.15.66:0/2196067645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa91006d630 msgr2=0x7fa910160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.602+0000 7fa914e00640 1 --2- 172.21.15.66:0/2196067645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa91006d630 0x7fa910160370 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa908009bc0 tx=0x7fa91015d120 comp rx=0 tx=0).stop 2024-09-10T22:06:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.602+0000 7fa914e00640 1 -- 172.21.15.66:0/2196067645 shutdown_connections 2024-09-10T22:06:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.602+0000 7fa914e00640 1 --2- 172.21.15.66:0/2196067645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa91006d630 0x7fa910160370 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.602+0000 7fa914e00640 1 -- 172.21.15.66:0/2196067645 >> 172.21.15.66:0/2196067645 conn(0x7fa9100d0b30 msgr2=0x7fa9100d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.602+0000 7fa914e00640 1 -- 172.21.15.66:0/2196067645 shutdown_connections 2024-09-10T22:06:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.602+0000 7fa914e00640 1 -- 172.21.15.66:0/2196067645 wait complete. 2024-09-10T22:06:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.602+0000 7fa914e00640 1 Processor -- start 2024-09-10T22:06:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa914e00640 1 -- start start 2024-09-10T22:06:34.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa914e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa91006d630 0x7fa9100c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:34.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa914e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa910071230 0x7fa9100c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:34.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa914e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa910156430 0x7fa9100ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa914e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa91016a180 con 0x7fa91006d630 2024-09-10T22:06:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa914e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa91016a000 con 0x7fa910071230 2024-09-10T22:06:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa914e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa91016a300 con 0x7fa910156430 2024-09-10T22:06:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa90de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa91006d630 0x7fa9100c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa90de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa91006d630 0x7fa9100c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45008/0 (socket says 172.21.15.66:45008) 2024-09-10T22:06:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa90de00640 1 -- 172.21.15.66:0/1946177733 learned_addr learned my addr 172.21.15.66:0/1946177733 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa90d400640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa910071230 0x7fa9100c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa90e800640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa910156430 0x7fa9100ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa90de00640 1 -- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa910156430 msgr2=0x7fa9100ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa90de00640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa910156430 0x7fa9100ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa90de00640 1 -- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa910071230 msgr2=0x7fa9100c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa90de00640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa910071230 0x7fa9100c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa90de00640 1 -- 172.21.15.66:0/1946177733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91013eef0 con 0x7fa91006d630 2024-09-10T22:06:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa90de00640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa91006d630 0x7fa9100c6b40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c430 tx=0x7fa90000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa906000640 1 -- 172.21.15.66:0/1946177733 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900011790 con 0x7fa91006d630 2024-09-10T22:06:34.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa906000640 1 -- 172.21.15.66:0/1946177733 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa900017070 con 0x7fa91006d630 2024-09-10T22:06:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa906000640 1 -- 172.21.15.66:0/1946177733 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900011ce0 con 0x7fa91006d630 2024-09-10T22:06:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa914e00640 1 -- 172.21.15.66:0/1946177733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91013f1d0 con 0x7fa91006d630 2024-09-10T22:06:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa914e00640 1 -- 172.21.15.66:0/1946177733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91013f790 con 0x7fa91006d630 2024-09-10T22:06:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.606+0000 7fa914e00640 1 -- 172.21.15.66:0/1946177733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9100c7080 con 0x7fa91006d630 2024-09-10T22:06:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.610+0000 7fa906000640 1 -- 172.21.15.66:0/1946177733 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 97414+0+0 (secure 0 0 0) 0x7fa90001c460 con 0x7fa91006d630 2024-09-10T22:06:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.610+0000 7fa906000640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa8f0073120 0x7fa8f00755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.610+0000 7fa906000640 1 -- 172.21.15.66:0/1946177733 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fa9000919e0 con 0x7fa91006d630 2024-09-10T22:06:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.610+0000 7fa90d400640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa8f0073120 0x7fa8f00755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.610+0000 7fa90d400640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa8f0073120 0x7fa8f00755e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc000c10 tx=0x7fa8fc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.614+0000 7fa906000640 1 -- 172.21.15.66:0/1946177733 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa90005e4c0 con 0x7fa91006d630 2024-09-10T22:06:34.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.814+0000 7fa914e00640 1 -- 172.21.15.66:0/1946177733 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa910076650 con 0x7fa8f0073120 2024-09-10T22:06:34.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa906000640 1 -- 172.21.15.66:0/1946177733 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7fa910076650 con 0x7fa8f0073120 2024-09-10T22:06:34.822 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:34.822 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:06:34.822 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:34.822 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sv1" 2024-09-10T22:06:34.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:34.823 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:34.823 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ss1clone1" 2024-09-10T22:06:34.823 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:34.823 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:06:34.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa914e00640 1 -- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa8f0073120 msgr2=0x7fa8f00755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa914e00640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa8f0073120 0x7fa8f00755e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc000c10 tx=0x7fa8fc009040 comp rx=0 tx=0).stop 2024-09-10T22:06:34.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa914e00640 1 -- 172.21.15.66:0/1946177733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa91006d630 msgr2=0x7fa9100c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa914e00640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa91006d630 0x7fa9100c6b40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c430 tx=0x7fa90000c8f0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa914e00640 1 -- 172.21.15.66:0/1946177733 shutdown_connections 2024-09-10T22:06:34.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa914e00640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa8f0073120 0x7fa8f00755e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa914e00640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa910156430 0x7fa9100ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa914e00640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa910071230 0x7fa9100c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa914e00640 1 --2- 172.21.15.66:0/1946177733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa91006d630 0x7fa9100c6b40 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa914e00640 1 -- 172.21.15.66:0/1946177733 >> 172.21.15.66:0/1946177733 conn(0x7fa9100d0b30 msgr2=0x7fa91015ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:34.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa914e00640 1 -- 172.21.15.66:0/1946177733 shutdown_connections 2024-09-10T22:06:34.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.818+0000 7fa914e00640 1 -- 172.21.15.66:0/1946177733 wait complete. 2024-09-10T22:06:34.843 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-10T22:06:34.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 -- 172.21.15.66:0/23696091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46400d2050 msgr2=0x7f46400b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 --2- 172.21.15.66:0/23696091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46400d2050 0x7f46400b1a60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4628009900 tx=0x7f462801c8e0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 -- 172.21.15.66:0/23696091 shutdown_connections 2024-09-10T22:06:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 --2- 172.21.15.66:0/23696091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f464015bdc0 0x7f46400b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 --2- 172.21.15.66:0/23696091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46400d2050 0x7f46400b1a60 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 -- 172.21.15.66:0/23696091 >> 172.21.15.66:0/23696091 conn(0x7f464009aa50 msgr2=0x7f464009ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 -- 172.21.15.66:0/23696091 shutdown_connections 2024-09-10T22:06:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 -- 172.21.15.66:0/23696091 wait complete. 2024-09-10T22:06:35.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 Processor -- start 2024-09-10T22:06:35.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 -- start start 2024-09-10T22:06:35.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46400d2050 0x7f46400b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:35.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f464015b9e0 0x7f46401fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:35.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f464015bdc0 0x7f46400b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:35.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4640168ea0 con 0x7f46400d2050 2024-09-10T22:06:35.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4640168d20 con 0x7f464015bdc0 2024-09-10T22:06:35.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463fe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4640169020 con 0x7f464015b9e0 2024-09-10T22:06:35.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46400d2050 0x7f46400b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:35.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46400d2050 0x7f46400b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45018/0 (socket says 172.21.15.66:45018) 2024-09-10T22:06:35.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463ea00640 1 -- 172.21.15.66:0/3691980743 learned_addr learned my addr 172.21.15.66:0/3691980743 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:35.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f4637e00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f464015b9e0 0x7f46401fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:35.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463f400640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f464015bdc0 0x7f46400b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:35.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463ea00640 1 -- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f464015b9e0 msgr2=0x7f46401fbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:35.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463ea00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f464015b9e0 0x7f46401fbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463ea00640 1 -- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f464015bdc0 msgr2=0x7f46400b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:35.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463ea00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f464015bdc0 0x7f46400b7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.990+0000 7f463ea00640 1 -- 172.21.15.66:0/3691980743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46400b79d0 con 0x7f46400d2050 2024-09-10T22:06:35.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.994+0000 7f463ea00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46400d2050 0x7f46400b4960 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f463000c530 tx=0x7f463000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:35.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.994+0000 7f4636a00640 1 -- 172.21.15.66:0/3691980743 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630017070 con 0x7f46400d2050 2024-09-10T22:06:35.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.994+0000 7f4636a00640 1 -- 172.21.15.66:0/3691980743 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46300053c0 con 0x7f46400d2050 2024-09-10T22:06:35.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.994+0000 7f4636a00640 1 -- 172.21.15.66:0/3691980743 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630012890 con 0x7f46400d2050 2024-09-10T22:06:35.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.994+0000 7f463fe00640 1 -- 172.21.15.66:0/3691980743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46400b66d0 con 0x7f46400d2050 2024-09-10T22:06:35.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.994+0000 7f463fe00640 1 -- 172.21.15.66:0/3691980743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46400b6c90 con 0x7f46400d2050 2024-09-10T22:06:35.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.994+0000 7f463fe00640 1 -- 172.21.15.66:0/3691980743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46400d1940 con 0x7f46400d2050 2024-09-10T22:06:35.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.994+0000 7f4636a00640 1 -- 172.21.15.66:0/3691980743 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 97414+0+0 (secure 0 0 0) 0x7f4630003600 con 0x7f46400d2050 2024-09-10T22:06:35.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.994+0000 7f4636a00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f461c073120 0x7f461c0755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:35.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.998+0000 7f4636a00640 1 -- 172.21.15.66:0/3691980743 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f4630091370 con 0x7f46400d2050 2024-09-10T22:06:35.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.998+0000 7f4637e00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f461c073120 0x7f461c0755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:35.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:34.998+0000 7f4637e00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f461c073120 0x7f461c0755e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f46280af000 tx=0x7f462801b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:35.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.002+0000 7f4636a00640 1 -- 172.21.15.66:0/3691980743 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f463005dc00 con 0x7f46400d2050 2024-09-10T22:06:35.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.206+0000 7f463fe00640 1 -- 172.21.15.66:0/3691980743 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46400af830 con 0x7f461c073120 2024-09-10T22:06:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f4636a00640 1 -- 172.21.15.66:0/3691980743 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f46400af830 con 0x7f461c073120 2024-09-10T22:06:35.227 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:35.227 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:06:35.227 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:35.227 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ss1" 2024-09-10T22:06:35.227 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:35.227 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:06:35.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f463fe00640 1 -- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f461c073120 msgr2=0x7f461c0755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:35.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f463fe00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f461c073120 0x7f461c0755e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f46280af000 tx=0x7f462801b120 comp rx=0 tx=0).stop 2024-09-10T22:06:35.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f463fe00640 1 -- 172.21.15.66:0/3691980743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46400d2050 msgr2=0x7f46400b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:35.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f463fe00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46400d2050 0x7f46400b4960 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f463000c530 tx=0x7f463000c9f0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f463fe00640 1 -- 172.21.15.66:0/3691980743 shutdown_connections 2024-09-10T22:06:35.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f463fe00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f461c073120 0x7f461c0755e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f463fe00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f464015bdc0 0x7f46400b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f463fe00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f464015b9e0 0x7f46401fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f463fe00640 1 --2- 172.21.15.66:0/3691980743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46400d2050 0x7f46400b4960 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f463fe00640 1 -- 172.21.15.66:0/3691980743 >> 172.21.15.66:0/3691980743 conn(0x7f464009aa50 msgr2=0x7f46400b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:35.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f463fe00640 1 -- 172.21.15.66:0/3691980743 shutdown_connections 2024-09-10T22:06:35.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.222+0000 7f463fe00640 1 -- 172.21.15.66:0/3691980743 wait complete. 2024-09-10T22:06:35.248 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-10T22:06:35.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.394+0000 7fa269e00640 1 -- 172.21.15.66:0/1324787804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa26415e010 msgr2=0x7fa2640cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:35.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.394+0000 7fa269e00640 1 --2- 172.21.15.66:0/1324787804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa26415e010 0x7fa2640cfea0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa2640d1410 tx=0x7fa25401c810 comp rx=0 tx=0).stop 2024-09-10T22:06:35.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.394+0000 7fa269e00640 1 -- 172.21.15.66:0/1324787804 shutdown_connections 2024-09-10T22:06:35.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.394+0000 7fa269e00640 1 --2- 172.21.15.66:0/1324787804 >> v2:172.21.15.94:3300/0 conn(0x7fa2640d03e0 0x7fa2640d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.394+0000 7fa269e00640 1 --2- 172.21.15.66:0/1324787804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa26415e010 0x7fa2640cfea0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.394+0000 7fa269e00640 1 -- 172.21.15.66:0/1324787804 >> 172.21.15.66:0/1324787804 conn(0x7fa2640d1c60 msgr2=0x7fa2640d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:35.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.394+0000 7fa269e00640 1 -- 172.21.15.66:0/1324787804 shutdown_connections 2024-09-10T22:06:35.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.394+0000 7fa269e00640 1 -- 172.21.15.66:0/1324787804 wait complete. 2024-09-10T22:06:35.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa269e00640 1 Processor -- start 2024-09-10T22:06:35.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa269e00640 1 -- start start 2024-09-10T22:06:35.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa269e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2640d03e0 0x7fa2640cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:35.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa269e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2640d0d90 0x7fa264159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:35.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa269e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa26415e010 0x7fa26414f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:35.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa269e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa26416ae40 con 0x7fa2640d0d90 2024-09-10T22:06:35.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa269e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa26416acc0 con 0x7fa26415e010 2024-09-10T22:06:35.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa269e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa26416afc0 con 0x7fa2640d03e0 2024-09-10T22:06:35.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa25a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2640d0d90 0x7fa264159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:35.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa25a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2640d0d90 0x7fa264159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45040/0 (socket says 172.21.15.66:45040) 2024-09-10T22:06:35.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa25a000640 1 -- 172.21.15.66:0/4134802191 learned_addr learned my addr 172.21.15.66:0/4134802191 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:35.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa262a00640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2640d03e0 0x7fa2640cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:35.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa263400640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa26415e010 0x7fa26414f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:35.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa25a000640 1 -- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2640d03e0 msgr2=0x7fa2640cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:35.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa25a000640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2640d03e0 0x7fa2640cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa25a000640 1 -- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa26415e010 msgr2=0x7fa26414f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:35.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa25a000640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa26415e010 0x7fa26414f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa25a000640 1 -- 172.21.15.66:0/4134802191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2641f8660 con 0x7fa2640d0d90 2024-09-10T22:06:35.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa25a000640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2640d0d90 0x7fa264159070 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00d980 tx=0x7fa24c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:35.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa25be00640 1 -- 172.21.15.66:0/4134802191 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c017070 con 0x7fa2640d0d90 2024-09-10T22:06:35.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa25be00640 1 -- 172.21.15.66:0/4134802191 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa24c009d60 con 0x7fa2640d0d90 2024-09-10T22:06:35.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa269e00640 1 -- 172.21.15.66:0/4134802191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2641f78a0 con 0x7fa2640d0d90 2024-09-10T22:06:35.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa25be00640 1 -- 172.21.15.66:0/4134802191 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c005280 con 0x7fa2640d0d90 2024-09-10T22:06:35.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.398+0000 7fa269e00640 1 -- 172.21.15.66:0/4134802191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2641f7e60 con 0x7fa2640d0d90 2024-09-10T22:06:35.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.402+0000 7fa269e00640 1 -- 172.21.15.66:0/4134802191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2640d0850 con 0x7fa2640d0d90 2024-09-10T22:06:35.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.402+0000 7fa25be00640 1 -- 172.21.15.66:0/4134802191 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 97414+0+0 (secure 0 0 0) 0x7fa24c005420 con 0x7fa2640d0d90 2024-09-10T22:06:35.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.402+0000 7fa25be00640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa240073120 0x7fa2400755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:35.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.402+0000 7fa25be00640 1 -- 172.21.15.66:0/4134802191 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fa24c091f70 con 0x7fa2640d0d90 2024-09-10T22:06:35.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.402+0000 7fa262a00640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa240073120 0x7fa2400755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:35.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.402+0000 7fa262a00640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa240073120 0x7fa2400755e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa2540af000 tx=0x7fa2540a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:35.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.406+0000 7fa25be00640 1 -- 172.21.15.66:0/4134802191 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa24c05e800 con 0x7fa2640d0d90 2024-09-10T22:06:35.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.614+0000 7fa269e00640 1 -- 172.21.15.66:0/4134802191 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa264158360 con 0x7fa240073120 2024-09-10T22:06:35.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa25be00640 1 -- 172.21.15.66:0/4134802191 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa264158360 con 0x7fa240073120 2024-09-10T22:06:35.650 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:35.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa269e00640 1 -- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa240073120 msgr2=0x7fa2400755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:35.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa269e00640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa240073120 0x7fa2400755e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa2540af000 tx=0x7fa2540a7040 comp rx=0 tx=0).stop 2024-09-10T22:06:35.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa269e00640 1 -- 172.21.15.66:0/4134802191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2640d0d90 msgr2=0x7fa264159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:35.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa269e00640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2640d0d90 0x7fa264159070 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00d980 tx=0x7fa24c00de40 comp rx=0 tx=0).stop 2024-09-10T22:06:35.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa269e00640 1 -- 172.21.15.66:0/4134802191 shutdown_connections 2024-09-10T22:06:35.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa269e00640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa240073120 0x7fa2400755e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa269e00640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa26415e010 0x7fa26414f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa269e00640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2640d0d90 0x7fa264159070 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa269e00640 1 --2- 172.21.15.66:0/4134802191 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2640d03e0 0x7fa2640cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa269e00640 1 -- 172.21.15.66:0/4134802191 >> 172.21.15.66:0/4134802191 conn(0x7fa2640d1c60 msgr2=0x7fa2641575c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:35.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa269e00640 1 -- 172.21.15.66:0/4134802191 shutdown_connections 2024-09-10T22:06:35.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.646+0000 7fa269e00640 1 -- 172.21.15.66:0/4134802191 wait complete. 2024-09-10T22:06:35.674 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-10T22:06:35.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.818+0000 7f20e5400640 1 -- 172.21.15.66:0/2597523924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20c4004810 msgr2=0x7f20c4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:35.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.818+0000 7f20e5400640 1 --2- 172.21.15.66:0/2597523924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20c4004810 0x7f20c4004c90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009b30 tx=0x7f20c801d230 comp rx=0 tx=0).stop 2024-09-10T22:06:35.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.818+0000 7f20e5400640 1 -- 172.21.15.66:0/2597523924 shutdown_connections 2024-09-10T22:06:35.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.818+0000 7f20e5400640 1 --2- 172.21.15.66:0/2597523924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20c4004810 0x7f20c4004c90 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.818+0000 7f20e5400640 1 -- 172.21.15.66:0/2597523924 >> 172.21.15.66:0/2597523924 conn(0x7f20e00d1b10 msgr2=0x7f20e00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:35.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.818+0000 7f20e5400640 1 -- 172.21.15.66:0/2597523924 shutdown_connections 2024-09-10T22:06:35.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.818+0000 7f20e5400640 1 -- 172.21.15.66:0/2597523924 wait complete. 2024-09-10T22:06:35.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.818+0000 7f20e5400640 1 Processor -- start 2024-09-10T22:06:35.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.818+0000 7f20e5400640 1 -- start start 2024-09-10T22:06:35.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20e5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20c4004810 0x7f20e01598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:35.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20e5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20e01f5f40 0x7f20e014cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:35.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20e5400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20e01555b0 0x7f20e00718c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:35.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20e5400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f20e00cecc0 con 0x7f20e01f5f40 2024-09-10T22:06:35.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20e5400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20e00b1350 con 0x7f20c4004810 2024-09-10T22:06:35.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20e5400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f20e00cee40 con 0x7f20e01555b0 2024-09-10T22:06:35.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20e01f5f40 0x7f20e014cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:35.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20dfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20c4004810 0x7f20e01598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:35.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20e01f5f40 0x7f20e014cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45050/0 (socket says 172.21.15.66:45050) 2024-09-10T22:06:35.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20d7e00640 1 -- 172.21.15.66:0/984587808 learned_addr learned my addr 172.21.15.66:0/984587808 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:35.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20e4a00640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20e01555b0 0x7f20e00718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:35.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20d7e00640 1 -- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20e01555b0 msgr2=0x7f20e00718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:35.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20d7e00640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20e01555b0 0x7f20e00718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20d7e00640 1 -- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20c4004810 msgr2=0x7f20e01598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:35.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20d7e00640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20c4004810 0x7f20e01598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:35.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20d7e00640 1 -- 172.21.15.66:0/984587808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e0159df0 con 0x7f20e01f5f40 2024-09-10T22:06:35.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20dfe00640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20c4004810 0x7f20e01598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:06:35.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20d7e00640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20e01f5f40 0x7f20e014cec0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f20c80af040 tx=0x7f20c80acd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:35.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20dcc00640 1 -- 172.21.15.66:0/984587808 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c80bc030 con 0x7f20e01f5f40 2024-09-10T22:06:35.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20dcc00640 1 -- 172.21.15.66:0/984587808 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20c80ac400 con 0x7f20e01f5f40 2024-09-10T22:06:35.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20dcc00640 1 -- 172.21.15.66:0/984587808 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c80b1040 con 0x7f20e01f5f40 2024-09-10T22:06:35.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20e5400640 1 -- 172.21.15.66:0/984587808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e01f63d0 con 0x7f20e01f5f40 2024-09-10T22:06:35.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20e5400640 1 -- 172.21.15.66:0/984587808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e01f6990 con 0x7f20e01f5f40 2024-09-10T22:06:35.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.822+0000 7f20e5400640 1 -- 172.21.15.66:0/984587808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e00cd7a0 con 0x7f20e01f5f40 2024-09-10T22:06:35.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.826+0000 7f20dcc00640 1 -- 172.21.15.66:0/984587808 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 97414+0+0 (secure 0 0 0) 0x7f20c8005280 con 0x7f20e01f5f40 2024-09-10T22:06:35.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.826+0000 7f20dcc00640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f20bc073150 0x7f20bc075610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:35.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.826+0000 7f20dcc00640 1 -- 172.21.15.66:0/984587808 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f20c812d0c0 con 0x7f20e01f5f40 2024-09-10T22:06:35.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.826+0000 7f20dfe00640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f20bc073150 0x7f20bc075610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:35.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.826+0000 7f20dfe00640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f20bc073150 0x7f20bc075610 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f20d0001000 tx=0x7f20d0006040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:35.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:35.830+0000 7f20dcc00640 1 -- 172.21.15.66:0/984587808 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f20c8016f40 con 0x7f20e01f5f40 2024-09-10T22:06:36.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.026+0000 7f20e5400640 1 -- 172.21.15.66:0/984587808 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f20e00b0110 con 0x7f20bc073150 2024-09-10T22:06:36.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.046+0000 7f20dcc00640 1 -- 172.21.15.66:0/984587808 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f20e00b0110 con 0x7f20bc073150 2024-09-10T22:06:36.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.046+0000 7f20e5400640 1 -- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f20bc073150 msgr2=0x7f20bc075610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:36.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.046+0000 7f20e5400640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f20bc073150 0x7f20bc075610 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f20d0001000 tx=0x7f20d0006040 comp rx=0 tx=0).stop 2024-09-10T22:06:36.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.046+0000 7f20e5400640 1 -- 172.21.15.66:0/984587808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20e01f5f40 msgr2=0x7f20e014cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:36.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.046+0000 7f20e5400640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20e01f5f40 0x7f20e014cec0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f20c80af040 tx=0x7f20c80acd20 comp rx=0 tx=0).stop 2024-09-10T22:06:36.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.046+0000 7f20e5400640 1 -- 172.21.15.66:0/984587808 shutdown_connections 2024-09-10T22:06:36.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.046+0000 7f20e5400640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f20bc073150 0x7f20bc075610 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:36.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.046+0000 7f20e5400640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20e01555b0 0x7f20e00718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:36.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.046+0000 7f20e5400640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20e01f5f40 0x7f20e014cec0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:36.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.046+0000 7f20e5400640 1 --2- 172.21.15.66:0/984587808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20c4004810 0x7f20e01598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:36.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.046+0000 7f20e5400640 1 -- 172.21.15.66:0/984587808 >> 172.21.15.66:0/984587808 conn(0x7f20e00d1b10 msgr2=0x7f20e00c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:36.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.050+0000 7f20e5400640 1 -- 172.21.15.66:0/984587808 shutdown_connections 2024-09-10T22:06:36.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.050+0000 7f20e5400640 1 -- 172.21.15.66:0/984587808 wait complete. 2024-09-10T22:06:36.065 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:36.065+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:36.065 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:36.065+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:36.065 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:36.065+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:36.066 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:36.065+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:36.066 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:06:36.065+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:06:36.075 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-10T22:06:36.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.230+0000 7fde51400640 1 -- 172.21.15.66:0/1014168284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4c073c40 msgr2=0x7fde4c0cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:36.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.230+0000 7fde51400640 1 --2- 172.21.15.66:0/1014168284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4c073c40 0x7fde4c0cf270 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fde38009840 tx=0x7fde3801c7f0 comp rx=0 tx=0).stop 2024-09-10T22:06:36.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.230+0000 7fde51400640 1 -- 172.21.15.66:0/1014168284 shutdown_connections 2024-09-10T22:06:36.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.230+0000 7fde51400640 1 --2- 172.21.15.66:0/1014168284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4c073c40 0x7fde4c0cf270 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:36.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.230+0000 7fde51400640 1 -- 172.21.15.66:0/1014168284 >> 172.21.15.66:0/1014168284 conn(0x7fde4c0b2730 msgr2=0x7fde4c0b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:36.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.230+0000 7fde51400640 1 -- 172.21.15.66:0/1014168284 shutdown_connections 2024-09-10T22:06:36.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.230+0000 7fde51400640 1 -- 172.21.15.66:0/1014168284 wait complete. 2024-09-10T22:06:36.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.230+0000 7fde51400640 1 Processor -- start 2024-09-10T22:06:36.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.230+0000 7fde51400640 1 -- start start 2024-09-10T22:06:36.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde51400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde4c073c40 0x7fde4c156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:36.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde51400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4c15b8f0 0x7fde4c144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:36.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde51400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c15bcd0 0x7fde4c159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:36.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde51400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c16a3d0 con 0x7fde4c15b8f0 2024-09-10T22:06:36.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde51400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c16a250 con 0x7fde4c15bcd0 2024-09-10T22:06:36.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde51400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fde4c16a550 con 0x7fde4c073c40 2024-09-10T22:06:36.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde4b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4c15b8f0 0x7fde4c144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:36.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde4b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4c15b8f0 0x7fde4c144480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45064/0 (socket says 172.21.15.66:45064) 2024-09-10T22:06:36.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde4b400640 1 -- 172.21.15.66:0/410934018 learned_addr learned my addr 172.21.15.66:0/410934018 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:36.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde4be00640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde4c073c40 0x7fde4c156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:36.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde50a00640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c15bcd0 0x7fde4c159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:36.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde4b400640 1 -- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde4c073c40 msgr2=0x7fde4c156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:36.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde4b400640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde4c073c40 0x7fde4c156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:36.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde4b400640 1 -- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c15bcd0 msgr2=0x7fde4c159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:36.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde4b400640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c15bcd0 0x7fde4c159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:36.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde4b400640 1 -- 172.21.15.66:0/410934018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c1571e0 con 0x7fde4c15b8f0 2024-09-10T22:06:36.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde4b400640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4c15b8f0 0x7fde4c144480 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fde38009790 tx=0x7fde380040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:36.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde43e00640 1 -- 172.21.15.66:0/410934018 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde380bb070 con 0x7fde4c15b8f0 2024-09-10T22:06:36.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde43e00640 1 -- 172.21.15.66:0/410934018 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde38005030 con 0x7fde4c15b8f0 2024-09-10T22:06:36.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde43e00640 1 -- 172.21.15.66:0/410934018 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde380b2040 con 0x7fde4c15b8f0 2024-09-10T22:06:36.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde51400640 1 -- 172.21.15.66:0/410934018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4c158f40 con 0x7fde4c15b8f0 2024-09-10T22:06:36.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde51400640 1 -- 172.21.15.66:0/410934018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4c1569d0 con 0x7fde4c15b8f0 2024-09-10T22:06:36.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.234+0000 7fde51400640 1 -- 172.21.15.66:0/410934018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4c073530 con 0x7fde4c15b8f0 2024-09-10T22:06:36.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.238+0000 7fde43e00640 1 -- 172.21.15.66:0/410934018 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 97414+0+0 (secure 0 0 0) 0x7fde380b5030 con 0x7fde4c15b8f0 2024-09-10T22:06:36.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.238+0000 7fde43e00640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fde2c073120 0x7fde2c0755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:36.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.238+0000 7fde43e00640 1 -- 172.21.15.66:0/410934018 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fde3812b620 con 0x7fde4c15b8f0 2024-09-10T22:06:36.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.238+0000 7fde4be00640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fde2c073120 0x7fde2c0755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:36.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.238+0000 7fde4be00640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fde2c073120 0x7fde2c0755e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fde3c007fd0 tx=0x7fde3c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:36.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.242+0000 7fde43e00640 1 -- 172.21.15.66:0/410934018 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fde38016410 con 0x7fde4c15b8f0 2024-09-10T22:06:36.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.450+0000 7fde51400640 1 -- 172.21.15.66:0/410934018 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fde4c1574c0 con 0x7fde2c073120 2024-09-10T22:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:36.590+0000 7fde43e00640 1 -- 172.21.15.66:0/410934018 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fde380f4120 con 0x7fde4c15b8f0 2024-09-10T22:06:37.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.146+0000 7fde43e00640 1 -- 172.21.15.66:0/410934018 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fde4c1574c0 con 0x7fde2c073120 2024-09-10T22:06:37.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.150+0000 7fde51400640 1 -- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fde2c073120 msgr2=0x7fde2c0755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:37.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.150+0000 7fde51400640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fde2c073120 0x7fde2c0755e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fde3c007fd0 tx=0x7fde3c009040 comp rx=0 tx=0).stop 2024-09-10T22:06:37.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.150+0000 7fde51400640 1 -- 172.21.15.66:0/410934018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4c15b8f0 msgr2=0x7fde4c144480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:37.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.150+0000 7fde51400640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4c15b8f0 0x7fde4c144480 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fde38009790 tx=0x7fde380040c0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.150+0000 7fde51400640 1 -- 172.21.15.66:0/410934018 shutdown_connections 2024-09-10T22:06:37.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.150+0000 7fde51400640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fde2c073120 0x7fde2c0755e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.150+0000 7fde51400640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c15bcd0 0x7fde4c159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.150+0000 7fde51400640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4c15b8f0 0x7fde4c144480 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.150+0000 7fde51400640 1 --2- 172.21.15.66:0/410934018 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde4c073c40 0x7fde4c156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.150+0000 7fde51400640 1 -- 172.21.15.66:0/410934018 >> 172.21.15.66:0/410934018 conn(0x7fde4c0b2730 msgr2=0x7fde4c0b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:37.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.150+0000 7fde51400640 1 -- 172.21.15.66:0/410934018 shutdown_connections 2024-09-10T22:06:37.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.150+0000 7fde51400640 1 -- 172.21.15.66:0/410934018 wait complete. 2024-09-10T22:06:37.176 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T22:06:37.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:06:37.235 INFO:teuthology.orchestra.run.smithi066.stdout:2 2024-09-10T22:06:37.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:06:37.242 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:06:37.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.386+0000 7fa947000640 1 -- 172.21.15.66:0/1325337347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa94015bd40 msgr2=0x7fa9400cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:37.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.386+0000 7fa947000640 1 --2- 172.21.15.66:0/1325337347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa94015bd40 0x7fa9400cffa0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa930009840 tx=0x7fa93001c790 comp rx=0 tx=0).stop 2024-09-10T22:06:37.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.386+0000 7fa947000640 1 -- 172.21.15.66:0/1325337347 shutdown_connections 2024-09-10T22:06:37.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.386+0000 7fa947000640 1 --2- 172.21.15.66:0/1325337347 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9400cf830 0x7fa9400d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.386+0000 7fa947000640 1 --2- 172.21.15.66:0/1325337347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa94015bd40 0x7fa9400cffa0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.386+0000 7fa947000640 1 -- 172.21.15.66:0/1325337347 >> 172.21.15.66:0/1325337347 conn(0x7fa9400d1c60 msgr2=0x7fa9400d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:37.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.386+0000 7fa947000640 1 -- 172.21.15.66:0/1325337347 shutdown_connections 2024-09-10T22:06:37.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.386+0000 7fa947000640 1 -- 172.21.15.66:0/1325337347 wait complete. 2024-09-10T22:06:37.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa947000640 1 Processor -- start 2024-09-10T22:06:37.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa947000640 1 -- start start 2024-09-10T22:06:37.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa947000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9400aec10 0x7fa9400b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:37.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa947000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9400cf830 0x7fa9401fb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:37.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa947000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa94015bd40 0x7fa9401f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:37.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa947000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa940169070 con 0x7fa9400cf830 2024-09-10T22:06:37.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa947000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa940168ef0 con 0x7fa94015bd40 2024-09-10T22:06:37.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa947000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa9401691f0 con 0x7fa9400aec10 2024-09-10T22:06:37.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa937e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9400cf830 0x7fa9401fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:37.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa937e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9400cf830 0x7fa9401fb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45088/0 (socket says 172.21.15.66:45088) 2024-09-10T22:06:37.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa937e00640 1 -- 172.21.15.66:0/701903857 learned_addr learned my addr 172.21.15.66:0/701903857 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:37.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa93fe00640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9400aec10 0x7fa9400b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:37.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa945000640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa94015bd40 0x7fa9401f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa937e00640 1 -- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9400aec10 msgr2=0x7fa9400b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa937e00640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9400aec10 0x7fa9400b73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa937e00640 1 -- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa94015bd40 msgr2=0x7fa9401f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa937e00640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa94015bd40 0x7fa9401f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa937e00640 1 -- 172.21.15.66:0/701903857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9401582e0 con 0x7fa9400cf830 2024-09-10T22:06:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa937e00640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9400cf830 0x7fa9401fb610 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa92800d980 tx=0x7fa92800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa93cc00640 1 -- 172.21.15.66:0/701903857 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928017070 con 0x7fa9400cf830 2024-09-10T22:06:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa93cc00640 1 -- 172.21.15.66:0/701903857 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa928009d20 con 0x7fa9400cf830 2024-09-10T22:06:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa93cc00640 1 -- 172.21.15.66:0/701903857 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9280106c0 con 0x7fa9400cf830 2024-09-10T22:06:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa947000640 1 -- 172.21.15.66:0/701903857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9401585c0 con 0x7fa9400cf830 2024-09-10T22:06:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.390+0000 7fa947000640 1 -- 172.21.15.66:0/701903857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa940163e90 con 0x7fa9400cf830 2024-09-10T22:06:37.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.394+0000 7fa947000640 1 -- 172.21.15.66:0/701903857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9400af010 con 0x7fa9400cf830 2024-09-10T22:06:37.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.394+0000 7fa93cc00640 1 -- 172.21.15.66:0/701903857 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fa928010860 con 0x7fa9400cf830 2024-09-10T22:06:37.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.394+0000 7fa93cc00640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa91c0731b0 0x7fa91c075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:37.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.394+0000 7fa93cc00640 1 -- 172.21.15.66:0/701903857 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fa928092200 con 0x7fa9400cf830 2024-09-10T22:06:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.394+0000 7fa93fe00640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa91c0731b0 0x7fa91c075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.398+0000 7fa93fe00640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa91c0731b0 0x7fa91c075670 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa9300af000 tx=0x7fa93001afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.398+0000 7fa93cc00640 1 -- 172.21.15.66:0/701903857 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa92805ea80 con 0x7fa9400cf830 2024-09-10T22:06:37.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.574+0000 7fa947000640 1 -- 172.21.15.66:0/701903857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fa9400b0df0 con 0x7fa9400cf830 2024-09-10T22:06:37.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.574+0000 7fa93cc00640 1 -- 172.21.15.66:0/701903857 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v35)=0 v35) ==== 131+0+0 (secure 0 0 0) 0x7fa9280634b0 con 0x7fa9400cf830 2024-09-10T22:06:37.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.578+0000 7fa947000640 1 -- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa91c0731b0 msgr2=0x7fa91c075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:37.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.578+0000 7fa947000640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa91c0731b0 0x7fa91c075670 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa9300af000 tx=0x7fa93001afd0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.578+0000 7fa947000640 1 -- 172.21.15.66:0/701903857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9400cf830 msgr2=0x7fa9401fb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:37.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.578+0000 7fa947000640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9400cf830 0x7fa9401fb610 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa92800d980 tx=0x7fa92800de40 comp rx=0 tx=0).stop 2024-09-10T22:06:37.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.578+0000 7fa947000640 1 -- 172.21.15.66:0/701903857 shutdown_connections 2024-09-10T22:06:37.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.578+0000 7fa947000640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa91c0731b0 0x7fa91c075670 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.578+0000 7fa947000640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa94015bd40 0x7fa9401f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.578+0000 7fa947000640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9400cf830 0x7fa9401fb610 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.578+0000 7fa947000640 1 --2- 172.21.15.66:0/701903857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9400aec10 0x7fa9400b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:37.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.578+0000 7fa947000640 1 -- 172.21.15.66:0/701903857 >> 172.21.15.66:0/701903857 conn(0x7fa9400d1c60 msgr2=0x7fa940071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:37.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.578+0000 7fa947000640 1 -- 172.21.15.66:0/701903857 shutdown_connections 2024-09-10T22:06:37.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:37.578+0000 7fa947000640 1 -- 172.21.15.66:0/701903857 wait complete. 2024-09-10T22:06:37.604 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:06:37.604 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:06:37.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:06:37.938 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:06:37.938 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:06:37.938 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:06:37.938 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:37.947 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:06:37.947 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:37.998 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:06:38.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 -- 172.21.15.66:0/1138815050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4190068cb0 msgr2=0x7f41900cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:38.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 --2- 172.21.15.66:0/1138815050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4190068cb0 0x7f41900cc2c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f419009e200 tx=0x7f417801f0b0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 -- 172.21.15.66:0/1138815050 shutdown_connections 2024-09-10T22:06:38.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 --2- 172.21.15.66:0/1138815050 >> v2:172.21.15.94:3300/0 conn(0x7f419006f290 0x7f419015f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 --2- 172.21.15.66:0/1138815050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4190068cb0 0x7f41900cc2c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 -- 172.21.15.66:0/1138815050 >> 172.21.15.66:0/1138815050 conn(0x7f41900d0120 msgr2=0x7f41900d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:38.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 -- 172.21.15.66:0/1138815050 shutdown_connections 2024-09-10T22:06:38.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 -- 172.21.15.66:0/1138815050 wait complete. 2024-09-10T22:06:38.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 Processor -- start 2024-09-10T22:06:38.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 -- start start 2024-09-10T22:06:38.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4190068cb0 0x7f41901f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:38.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f419006f290 0x7f419009d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:38.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419015bb20 0x7f41900c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:38.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f419015fba0 con 0x7f4190068cb0 2024-09-10T22:06:38.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f419015fa20 con 0x7f419015bb20 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4194a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f419015fd20 con 0x7f419006f290 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f418f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4190068cb0 0x7f41901f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f418f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4190068cb0 0x7f41901f8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45108/0 (socket says 172.21.15.66:45108) 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f418f400640 1 -- 172.21.15.66:0/948593021 learned_addr learned my addr 172.21.15.66:0/948593021 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f418fe00640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419015bb20 0x7f41900c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f4187e00640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f419006f290 0x7f419009d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f418f400640 1 -- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f419006f290 msgr2=0x7f419009d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f418f400640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f419006f290 0x7f419009d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f418f400640 1 -- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419015bb20 msgr2=0x7f41900c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f418f400640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419015bb20 0x7f41900c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f418f400640 1 -- 172.21.15.66:0/948593021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41901f5160 con 0x7f4190068cb0 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.186+0000 7f418fe00640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419015bb20 0x7f41900c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:06:38.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.190+0000 7f418f400640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4190068cb0 0x7f41901f8a70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f418000c3c0 tx=0x7f418000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:38.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.190+0000 7f4187400640 1 -- 172.21.15.66:0/948593021 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180017070 con 0x7f4190068cb0 2024-09-10T22:06:38.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.190+0000 7f4187400640 1 -- 172.21.15.66:0/948593021 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41800053c0 con 0x7f4190068cb0 2024-09-10T22:06:38.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.190+0000 7f4194a00640 1 -- 172.21.15.66:0/948593021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41901f5440 con 0x7f4190068cb0 2024-09-10T22:06:38.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.190+0000 7f4187400640 1 -- 172.21.15.66:0/948593021 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180003860 con 0x7f4190068cb0 2024-09-10T22:06:38.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.190+0000 7f4194a00640 1 -- 172.21.15.66:0/948593021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41900c8e60 con 0x7f4190068cb0 2024-09-10T22:06:38.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.190+0000 7f4194a00640 1 -- 172.21.15.66:0/948593021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4190160240 con 0x7f4190068cb0 2024-09-10T22:06:38.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.190+0000 7f4187400640 1 -- 172.21.15.66:0/948593021 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f4180003a00 con 0x7f4190068cb0 2024-09-10T22:06:38.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.190+0000 7f4187400640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f416c0731b0 0x7f416c075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:38.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.190+0000 7f4187400640 1 -- 172.21.15.66:0/948593021 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f4180091450 con 0x7f4190068cb0 2024-09-10T22:06:38.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.194+0000 7f4187e00640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f416c0731b0 0x7f416c075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:38.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.194+0000 7f4187e00640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f416c0731b0 0x7f416c075670 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f41780b2000 tx=0x7f417801d8f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:38.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.198+0000 7f4187400640 1 -- 172.21.15.66:0/948593021 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f418005e380 con 0x7f4190068cb0 2024-09-10T22:06:38.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.382+0000 7f4194a00640 1 -- 172.21.15.66:0/948593021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4190068a20 con 0x7f4190068cb0 2024-09-10T22:06:38.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.382+0000 7f4187400640 1 -- 172.21.15.66:0/948593021 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) ==== 80+0+0 (secure 0 0 0) 0x7f418005dc80 con 0x7f4190068cb0 2024-09-10T22:06:38.389 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:06:38.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.386+0000 7f4194a00640 1 -- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f416c0731b0 msgr2=0x7f416c075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:38.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.386+0000 7f4194a00640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f416c0731b0 0x7f416c075670 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f41780b2000 tx=0x7f417801d8f0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.386+0000 7f4194a00640 1 -- 172.21.15.66:0/948593021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4190068cb0 msgr2=0x7f41901f8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:38.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.386+0000 7f4194a00640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4190068cb0 0x7f41901f8a70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f418000c3c0 tx=0x7f418000c880 comp rx=0 tx=0).stop 2024-09-10T22:06:38.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.386+0000 7f4194a00640 1 -- 172.21.15.66:0/948593021 shutdown_connections 2024-09-10T22:06:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.386+0000 7f4194a00640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f416c0731b0 0x7f416c075670 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.386+0000 7f4194a00640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419015bb20 0x7f41900c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.386+0000 7f4194a00640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f419006f290 0x7f419009d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.386+0000 7f4194a00640 1 --2- 172.21.15.66:0/948593021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4190068cb0 0x7f41901f8a70 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.386+0000 7f4194a00640 1 -- 172.21.15.66:0/948593021 >> 172.21.15.66:0/948593021 conn(0x7f41900d0120 msgr2=0x7f41900ca900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.386+0000 7f4194a00640 1 -- 172.21.15.66:0/948593021 shutdown_connections 2024-09-10T22:06:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.386+0000 7f4194a00640 1 -- 172.21.15.66:0/948593021 wait complete. 2024-09-10T22:06:38.411 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-09-10T22:06:38.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:06:38.412 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T22:06:38.430 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[40841]: fuse finished with error 0 and tester_r 0 2024-09-10T22:06:38.432 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:06:38.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:06:38.433 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:06:38.615 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:06:38.615 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:06:38.615 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:38.628 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:06:38.628 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:06:38.628 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:06:38.628 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:38.675 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:06:38.676 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:06:38.676 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:06:38.676 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:38.689 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:06:38.689 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:06:38.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.982+0000 7f4043000640 1 -- 172.21.15.66:0/807327926 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4020003a10 msgr2=0x7f4020003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:38.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.982+0000 7f4043000640 1 --2- 172.21.15.66:0/807327926 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4020003a10 0x7f4020003e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f402c006c80 tx=0x7f402c01f1a0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.982+0000 7f4043000640 1 -- 172.21.15.66:0/807327926 shutdown_connections 2024-09-10T22:06:38.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.982+0000 7f4043000640 1 --2- 172.21.15.66:0/807327926 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4020003a10 0x7f4020003e10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.982+0000 7f4043000640 1 -- 172.21.15.66:0/807327926 >> 172.21.15.66:0/807327926 conn(0x7f403c0b26c0 msgr2=0x7f403c0b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.982+0000 7f4043000640 1 -- 172.21.15.66:0/807327926 shutdown_connections 2024-09-10T22:06:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.982+0000 7f4043000640 1 -- 172.21.15.66:0/807327926 wait complete. 2024-09-10T22:06:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.986+0000 7f4043000640 1 Processor -- start 2024-09-10T22:06:38.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.986+0000 7f4043000640 1 -- start start 2024-09-10T22:06:38.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.986+0000 7f4043000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4020003a10 0x7f403c0b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:38.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.986+0000 7f4043000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403c14a8a0 0x7f403c14a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:38.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.986+0000 7f4043000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f403c153cf0 0x7f403c154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:38.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.986+0000 7f403b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403c14a8a0 0x7f403c14a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:38.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.986+0000 7f403b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403c14a8a0 0x7f403c14a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:37166/0 (socket says 172.21.15.66:37166) 2024-09-10T22:06:38.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.986+0000 7f403be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4020003a10 0x7f403c0b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:38.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.986+0000 7f403be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4020003a10 0x7f403c0b66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45136/0 (socket says 172.21.15.66:45136) 2024-09-10T22:06:38.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4041000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f403c153cf0 0x7f403c154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:38.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4041000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f403c153cf0 0x7f403c154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:53720/0 (socket says 172.21.15.66:53720) 2024-09-10T22:06:38.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f403b400640 1 -- 172.21.15.66:0/281464890 learned_addr learned my addr 172.21.15.66:0/281464890 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:38.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f403c15e970 con 0x7f4020003a10 2024-09-10T22:06:38.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f403c15e7f0 con 0x7f403c14a8a0 2024-09-10T22:06:38.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f403c15eaf0 con 0x7f403c153cf0 2024-09-10T22:06:38.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4041000640 1 -- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403c14a8a0 msgr2=0x7f403c14a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:38.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4041000640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403c14a8a0 0x7f403c14a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4041000640 1 -- 172.21.15.66:0/281464890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4020003a10 msgr2=0x7f403c0b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:38.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4041000640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4020003a10 0x7f403c0b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4041000640 1 -- 172.21.15.66:0/281464890 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c156ea0 con 0x7f403c153cf0 2024-09-10T22:06:38.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f403b400640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403c14a8a0 0x7f403c14a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:06:38.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4041000640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f403c153cf0 0x7f403c154170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f403c0d1c30 tx=0x7f403400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:38.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4033e00640 1 -- 172.21.15.66:0/281464890 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034003d90 con 0x7f403c153cf0 2024-09-10T22:06:38.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4033e00640 1 -- 172.21.15.66:0/281464890 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4034005070 con 0x7f403c153cf0 2024-09-10T22:06:38.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4033e00640 1 -- 172.21.15.66:0/281464890 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40340054e0 con 0x7f403c153cf0 2024-09-10T22:06:38.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c157180 con 0x7f403c153cf0 2024-09-10T22:06:38.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.990+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c157740 con 0x7f403c153cf0 2024-09-10T22:06:38.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.994+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40140052a0 con 0x7f403c153cf0 2024-09-10T22:06:39.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.994+0000 7f4033e00640 1 -- 172.21.15.66:0/281464890 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f4034005680 con 0x7f403c153cf0 2024-09-10T22:06:39.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.998+0000 7f4033e00640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f401c073230 0x7f401c0756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:39.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.998+0000 7f4033e00640 1 -- 172.21.15.66:0/281464890 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f40340912e0 con 0x7f403c153cf0 2024-09-10T22:06:39.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.998+0000 7f403be00640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f401c073230 0x7f401c0756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:39.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:38.998+0000 7f403be00640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f401c073230 0x7f401c0756f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f40200057f0 tx=0x7f402c01bf90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:39.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.002+0000 7f4033e00640 1 -- 172.21.15.66:0/281464890 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4034019060 con 0x7f403c153cf0 2024-09-10T22:06:39.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.446+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4014005590 con 0x7f403c153cf0 2024-09-10T22:06:39.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.450+0000 7f4033e00640 1 -- 172.21.15.66:0/281464890 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5220 (secure 0 0 0) 0x7f403405db60 con 0x7f403c153cf0 2024-09-10T22:06:39.455 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:39.455 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":37,"btime":"2024-09-10T22:05:16:499594+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5414,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/135515157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":135515157},{"type":"v1","addr":"172.21.15.94:6837","nonce":135515157}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1614138608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1614138608},{"type":"v1","addr":"172.21.15.94:6836","nonce":1614138608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1060280836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1060280836},{"type":"v1","addr":"172.21.15.66:6833","nonce":1060280836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:05:07.804373+0000","modified":"2024-09-10T22:05:10.828111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5925},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5925":{"gid":5925,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2880381289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2880381289},{"type":"v1","addr":"172.21.15.66:6835","nonce":2880381289}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5925,"qdb_cluster":[5925]},"id":4}]} 2024-09-10T22:06:39.456 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 37 2024-09-10T22:06:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.450+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f401c073230 msgr2=0x7f401c0756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.450+0000 7f4043000640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f401c073230 0x7f401c0756f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f40200057f0 tx=0x7f402c01bf90 comp rx=0 tx=0).stop 2024-09-10T22:06:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.450+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f403c153cf0 msgr2=0x7f403c154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.450+0000 7f4043000640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f403c153cf0 0x7f403c154170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f403c0d1c30 tx=0x7f403400c960 comp rx=0 tx=0).stop 2024-09-10T22:06:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.454+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 shutdown_connections 2024-09-10T22:06:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.454+0000 7f4043000640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f401c073230 0x7f401c0756f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.454+0000 7f4043000640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f403c153cf0 0x7f403c154170 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.454+0000 7f4043000640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403c14a8a0 0x7f403c14a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.454+0000 7f4043000640 1 --2- 172.21.15.66:0/281464890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4020003a10 0x7f403c0b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.454+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 >> 172.21.15.66:0/281464890 conn(0x7f403c0b26c0 msgr2=0x7f403c06cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:39.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.454+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 shutdown_connections 2024-09-10T22:06:39.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.454+0000 7f4043000640 1 -- 172.21.15.66:0/281464890 wait complete. 2024-09-10T22:06:39.498 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:06:39.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.766+0000 7f4620c00640 1 -- 172.21.15.66:0/1165111825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f461c0cebc0 msgr2=0x7f461c0d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:39.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.766+0000 7f4620c00640 1 --2- 172.21.15.66:0/1165111825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f461c0cebc0 0x7f461c0d05b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4608009900 tx=0x7f460801c870 comp rx=0 tx=0).stop 2024-09-10T22:06:39.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.766+0000 7f4620c00640 1 -- 172.21.15.66:0/1165111825 shutdown_connections 2024-09-10T22:06:39.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.766+0000 7f4620c00640 1 --2- 172.21.15.66:0/1165111825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f461c0cebc0 0x7f461c0d05b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:39.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.766+0000 7f4620c00640 1 --2- 172.21.15.66:0/1165111825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f461c0d0af0 0x7f461c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:39.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.766+0000 7f4620c00640 1 -- 172.21.15.66:0/1165111825 >> 172.21.15.66:0/1165111825 conn(0x7f461c0d1c60 msgr2=0x7f461c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:39.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.766+0000 7f4620c00640 1 -- 172.21.15.66:0/1165111825 shutdown_connections 2024-09-10T22:06:39.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.766+0000 7f4620c00640 1 -- 172.21.15.66:0/1165111825 wait complete. 2024-09-10T22:06:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.766+0000 7f4620c00640 1 Processor -- start 2024-09-10T22:06:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.766+0000 7f4620c00640 1 -- start start 2024-09-10T22:06:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f4620c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f461c0cebc0 0x7f461c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f4620c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f461c0d0af0 0x7f461c14fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f4620c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c15e010 0x7f461c0b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f4620c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f461c16acc0 con 0x7f461c15e010 2024-09-10T22:06:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f4620c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f461c16ae40 con 0x7f461c0cebc0 2024-09-10T22:06:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f4620c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f461c16afc0 con 0x7f461c0d0af0 2024-09-10T22:06:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f461a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c15e010 0x7f461c0b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f461a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c15e010 0x7f461c0b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45144/0 (socket says 172.21.15.66:45144) 2024-09-10T22:06:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f461a800640 1 -- 172.21.15.66:0/3914346079 learned_addr learned my addr 172.21.15.66:0/3914346079 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f461a800640 1 -- 172.21.15.66:0/3914346079 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f461c0d0af0 msgr2=0x7f461c14fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f461a800640 1 --2- 172.21.15.66:0/3914346079 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f461c0d0af0 0x7f461c14fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f461a800640 1 -- 172.21.15.66:0/3914346079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f461c0cebc0 msgr2=0x7f461c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f461a800640 1 --2- 172.21.15.66:0/3914346079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f461c0cebc0 0x7f461c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f461a800640 1 -- 172.21.15.66:0/3914346079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461c1502e0 con 0x7f461c15e010 2024-09-10T22:06:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.770+0000 7f461a800640 1 --2- 172.21.15.66:0/3914346079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c15e010 0x7f461c0b3a00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f461400b9c0 tx=0x7f461400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.774+0000 7f4612000640 1 -- 172.21.15.66:0/3914346079 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614019070 con 0x7f461c15e010 2024-09-10T22:06:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.774+0000 7f4612000640 1 -- 172.21.15.66:0/3914346079 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f461400cc90 con 0x7f461c15e010 2024-09-10T22:06:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.774+0000 7f4612000640 1 -- 172.21.15.66:0/3914346079 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46140146a0 con 0x7f461c15e010 2024-09-10T22:06:39.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.774+0000 7f4620c00640 1 -- 172.21.15.66:0/3914346079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461c0cb290 con 0x7f461c15e010 2024-09-10T22:06:39.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.774+0000 7f4620c00640 1 -- 172.21.15.66:0/3914346079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f461c0cb850 con 0x7f461c15e010 2024-09-10T22:06:39.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.774+0000 7f4620c00640 1 -- 172.21.15.66:0/3914346079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461c06dd30 con 0x7f461c15e010 2024-09-10T22:06:39.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.774+0000 7f4612000640 1 -- 172.21.15.66:0/3914346079 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f461401d3b0 con 0x7f461c15e010 2024-09-10T22:06:39.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.778+0000 7f4612000640 1 --2- 172.21.15.66:0/3914346079 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45fc073280 0x7f45fc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:39.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.778+0000 7f4612000640 1 -- 172.21.15.66:0/3914346079 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5765+0+0 (secure 0 0 0) 0x7f4614091fa0 con 0x7f461c15e010 2024-09-10T22:06:39.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.778+0000 7f4619e00640 1 --2- 172.21.15.66:0/3914346079 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45fc073280 0x7f45fc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:39.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.778+0000 7f4619e00640 1 --2- 172.21.15.66:0/3914346079 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45fc073280 0x7f45fc075740 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f460c009650 tx=0x7f460c006bb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:39.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:39.782+0000 7f4612000640 1 -- 172.21.15.66:0/3914346079 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f461405ee70 con 0x7f461c15e010 2024-09-10T22:06:40.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 -- 172.21.15.66:0/3914346079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f461c0cbef0 con 0x7f461c15e010 2024-09-10T22:06:40.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4612000640 1 -- 172.21.15.66:0/3914346079 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5220 (secure 0 0 0) 0x7f461405e770 con 0x7f461c15e010 2024-09-10T22:06:40.151 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:40.151 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":37,"btime":"2024-09-10T22:05:16:499594+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5414,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/135515157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":135515157},{"type":"v1","addr":"172.21.15.94:6837","nonce":135515157}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1614138608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1614138608},{"type":"v1","addr":"172.21.15.94:6836","nonce":1614138608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1060280836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1060280836},{"type":"v1","addr":"172.21.15.66:6833","nonce":1060280836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:05:07.804373+0000","modified":"2024-09-10T22:05:10.828111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5925},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5925":{"gid":5925,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2880381289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2880381289},{"type":"v1","addr":"172.21.15.66:6835","nonce":2880381289}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5925,"qdb_cluster":[5925]},"id":4}]} 2024-09-10T22:06:40.151 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 37 2024-09-10T22:06:40.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 -- 172.21.15.66:0/3914346079 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45fc073280 msgr2=0x7f45fc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:40.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 --2- 172.21.15.66:0/3914346079 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45fc073280 0x7f45fc075740 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f460c009650 tx=0x7f460c006bb0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 -- 172.21.15.66:0/3914346079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c15e010 msgr2=0x7f461c0b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:40.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 --2- 172.21.15.66:0/3914346079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c15e010 0x7f461c0b3a00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f461400b9c0 tx=0x7f461400be80 comp rx=0 tx=0).stop 2024-09-10T22:06:40.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 -- 172.21.15.66:0/3914346079 shutdown_connections 2024-09-10T22:06:40.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 --2- 172.21.15.66:0/3914346079 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45fc073280 0x7f45fc075740 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 --2- 172.21.15.66:0/3914346079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c15e010 0x7f461c0b3a00 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 --2- 172.21.15.66:0/3914346079 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f461c0d0af0 0x7f461c14fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 --2- 172.21.15.66:0/3914346079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f461c0cebc0 0x7f461c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 -- 172.21.15.66:0/3914346079 >> 172.21.15.66:0/3914346079 conn(0x7f461c0d1c60 msgr2=0x7f461c15b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:40.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 -- 172.21.15.66:0/3914346079 shutdown_connections 2024-09-10T22:06:40.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.146+0000 7f4620c00640 1 -- 172.21.15.66:0/3914346079 wait complete. 2024-09-10T22:06:40.173 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:06:40.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.314+0000 7f86c9c00640 1 -- 172.21.15.66:0/1683138026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86c40ae620 msgr2=0x7f86c406e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:40.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.314+0000 7f86c9c00640 1 --2- 172.21.15.66:0/1683138026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86c40ae620 0x7f86c406e960 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009870 tx=0x7f86ac01c820 comp rx=0 tx=0).stop 2024-09-10T22:06:40.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.314+0000 7f86c9c00640 1 -- 172.21.15.66:0/1683138026 shutdown_connections 2024-09-10T22:06:40.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.314+0000 7f86c9c00640 1 --2- 172.21.15.66:0/1683138026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86c40ae620 0x7f86c406e960 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.314+0000 7f86c9c00640 1 --2- 172.21.15.66:0/1683138026 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86c40c4ef0 0x7f86c40c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.314+0000 7f86c9c00640 1 -- 172.21.15.66:0/1683138026 >> 172.21.15.66:0/1683138026 conn(0x7f86c40cfa30 msgr2=0x7f86c40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:40.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.314+0000 7f86c9c00640 1 -- 172.21.15.66:0/1683138026 shutdown_connections 2024-09-10T22:06:40.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.314+0000 7f86c9c00640 1 -- 172.21.15.66:0/1683138026 wait complete. 2024-09-10T22:06:40.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.314+0000 7f86c9c00640 1 Processor -- start 2024-09-10T22:06:40.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.314+0000 7f86c9c00640 1 -- start start 2024-09-10T22:06:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86c9c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c40ae620 0x7f86c40b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86c9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86c40c4b10 0x7f86c4144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86c9c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86c40c4ef0 0x7f86c4159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86c9c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4167b80 con 0x7f86c40c4b10 2024-09-10T22:06:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86c9c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4167a00 con 0x7f86c40ae620 2024-09-10T22:06:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86c9c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f86c4167d00 con 0x7f86c40c4ef0 2024-09-10T22:06:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86ba000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86c40c4b10 0x7f86c4144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86ba000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86c40c4b10 0x7f86c4144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45158/0 (socket says 172.21.15.66:45158) 2024-09-10T22:06:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86ba000640 1 -- 172.21.15.66:0/359974626 learned_addr learned my addr 172.21.15.66:0/359974626 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86c2a00640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c40ae620 0x7f86c40b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86c3400640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86c40c4ef0 0x7f86c4159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86ba000640 1 -- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86c40c4ef0 msgr2=0x7f86c4159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86ba000640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86c40c4ef0 0x7f86c4159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86ba000640 1 -- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c40ae620 msgr2=0x7f86c40b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86ba000640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c40ae620 0x7f86c40b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86ba000640 1 -- 172.21.15.66:0/359974626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c40c6e50 con 0x7f86c40c4b10 2024-09-10T22:06:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86ba000640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86c40c4b10 0x7f86c4144520 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f86ac01a450 tx=0x7f86ac0a66d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:40.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86bb400640 1 -- 172.21.15.66:0/359974626 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac004180 con 0x7f86c40c4b10 2024-09-10T22:06:40.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86bb400640 1 -- 172.21.15.66:0/359974626 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86ac004410 con 0x7f86c40c4b10 2024-09-10T22:06:40.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86c9c00640 1 -- 172.21.15.66:0/359974626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c40c7130 con 0x7f86c40c4b10 2024-09-10T22:06:40.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86bb400640 1 -- 172.21.15.66:0/359974626 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac005290 con 0x7f86c40c4b10 2024-09-10T22:06:40.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86c9c00640 1 -- 172.21.15.66:0/359974626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c40c76f0 con 0x7f86c40c4b10 2024-09-10T22:06:40.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.318+0000 7f86c9c00640 1 -- 172.21.15.66:0/359974626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c406e960 con 0x7f86c40c4b10 2024-09-10T22:06:40.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.322+0000 7f86bb400640 1 -- 172.21.15.66:0/359974626 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f86ac005430 con 0x7f86c40c4b10 2024-09-10T22:06:40.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.322+0000 7f86bb400640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86a00731b0 0x7f86a0075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.322+0000 7f86bb400640 1 -- 172.21.15.66:0/359974626 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5765+0+0 (secure 0 0 0) 0x7f86ac12c5f0 con 0x7f86c40c4b10 2024-09-10T22:06:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.322+0000 7f86c2a00640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86a00731b0 0x7f86a0075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.322+0000 7f86c2a00640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86a00731b0 0x7f86a0075670 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f86b40016e0 tx=0x7f86b4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:40.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.326+0000 7f86bb400640 1 -- 172.21.15.66:0/359974626 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f86ac0c7020 con 0x7f86c40c4b10 2024-09-10T22:06:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.502+0000 7f86c9c00640 1 -- 172.21.15.66:0/359974626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86c40b0dc0 con 0x7f86c40c4b10 2024-09-10T22:06:40.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.502+0000 7f86bb400640 1 -- 172.21.15.66:0/359974626 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16761 (secure 0 0 0) 0x7f86ac016410 con 0x7f86c40c4b10 2024-09-10T22:06:40.508 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:40.508 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":60,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:06:39.603852+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:05:05.519188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T22:05:06.772731+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:06:40.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.506+0000 7f86c9c00640 1 -- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86a00731b0 msgr2=0x7f86a0075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:40.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.506+0000 7f86c9c00640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86a00731b0 0x7f86a0075670 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f86b40016e0 tx=0x7f86b4009040 comp rx=0 tx=0).stop 2024-09-10T22:06:40.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.506+0000 7f86c9c00640 1 -- 172.21.15.66:0/359974626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86c40c4b10 msgr2=0x7f86c4144520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:40.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.506+0000 7f86c9c00640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86c40c4b10 0x7f86c4144520 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f86ac01a450 tx=0x7f86ac0a66d0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.506+0000 7f86c9c00640 1 -- 172.21.15.66:0/359974626 shutdown_connections 2024-09-10T22:06:40.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.506+0000 7f86c9c00640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86a00731b0 0x7f86a0075670 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.506+0000 7f86c9c00640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86c40c4ef0 0x7f86c4159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.506+0000 7f86c9c00640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86c40c4b10 0x7f86c4144520 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.506+0000 7f86c9c00640 1 --2- 172.21.15.66:0/359974626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c40ae620 0x7f86c40b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.506+0000 7f86c9c00640 1 -- 172.21.15.66:0/359974626 >> 172.21.15.66:0/359974626 conn(0x7f86c40cfa30 msgr2=0x7f86c4073880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:40.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.506+0000 7f86c9c00640 1 -- 172.21.15.66:0/359974626 shutdown_connections 2024-09-10T22:06:40.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.506+0000 7f86c9c00640 1 -- 172.21.15.66:0/359974626 wait complete. 2024-09-10T22:06:40.530 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:06:40.530 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:06:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.670+0000 7fd5fce00640 1 -- 172.21.15.66:0/1162292392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5f8070b20 msgr2=0x7fd5f8070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.670+0000 7fd5fce00640 1 --2- 172.21.15.66:0/1162292392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5f8070b20 0x7fd5f8070f70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8069e40 tx=0x7fd5f001ccf0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.670+0000 7fd5fce00640 1 -- 172.21.15.66:0/1162292392 shutdown_connections 2024-09-10T22:06:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.670+0000 7fd5fce00640 1 --2- 172.21.15.66:0/1162292392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5f8070b20 0x7fd5f8070f70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.670+0000 7fd5fce00640 1 --2- 172.21.15.66:0/1162292392 >> v2:172.21.15.94:3300/0 conn(0x7fd5f8160370 0x7fd5f80cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.670+0000 7fd5fce00640 1 --2- 172.21.15.66:0/1162292392 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5f8156430 0x7fd5f806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.670+0000 7fd5fce00640 1 -- 172.21.15.66:0/1162292392 >> 172.21.15.66:0/1162292392 conn(0x7fd5f80d0b30 msgr2=0x7fd5f80d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.670+0000 7fd5fce00640 1 -- 172.21.15.66:0/1162292392 shutdown_connections 2024-09-10T22:06:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.670+0000 7fd5fce00640 1 -- 172.21.15.66:0/1162292392 wait complete. 2024-09-10T22:06:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5fce00640 1 Processor -- start 2024-09-10T22:06:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5fce00640 1 -- start start 2024-09-10T22:06:40.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5fce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5f8070b20 0x7fd5f8150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:40.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5fce00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5f8156430 0x7fd5f8151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:40.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5fce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f8160370 0x7fd5f80b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:40.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5fce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f80cebb0 con 0x7fd5f8070b20 2024-09-10T22:06:40.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5fce00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f80cea30 con 0x7fd5f8160370 2024-09-10T22:06:40.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5fce00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f80ced30 con 0x7fd5f8156430 2024-09-10T22:06:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5f8070b20 0x7fd5f8150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5f8070b20 0x7fd5f8150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45180/0 (socket says 172.21.15.66:45180) 2024-09-10T22:06:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5f5e00640 1 -- 172.21.15.66:0/4059994892 learned_addr learned my addr 172.21.15.66:0/4059994892 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5efe00640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5f8156430 0x7fd5f8151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5f6800640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f8160370 0x7fd5f80b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5f5e00640 1 -- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5f8156430 msgr2=0x7fd5f8151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5f5e00640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5f8156430 0x7fd5f8151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5f5e00640 1 -- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f8160370 msgr2=0x7fd5f80b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5f5e00640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f8160370 0x7fd5f80b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5f5e00640 1 -- 172.21.15.66:0/4059994892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f80ca280 con 0x7fd5f8070b20 2024-09-10T22:06:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5f5e00640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5f8070b20 0x7fd5f8150de0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800d720 tx=0x7fd5e800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5ee000640 1 -- 172.21.15.66:0/4059994892 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8017070 con 0x7fd5f8070b20 2024-09-10T22:06:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5ee000640 1 -- 172.21.15.66:0/4059994892 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5e8009d20 con 0x7fd5f8070b20 2024-09-10T22:06:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5ee000640 1 -- 172.21.15.66:0/4059994892 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8004280 con 0x7fd5f8070b20 2024-09-10T22:06:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5fce00640 1 -- 172.21.15.66:0/4059994892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f81594d0 con 0x7fd5f8070b20 2024-09-10T22:06:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.674+0000 7fd5fce00640 1 -- 172.21.15.66:0/4059994892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f80ca540 con 0x7fd5f8070b20 2024-09-10T22:06:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.678+0000 7fd5fce00640 1 -- 172.21.15.66:0/4059994892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f806d630 con 0x7fd5f8070b20 2024-09-10T22:06:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.678+0000 7fd5ee000640 1 -- 172.21.15.66:0/4059994892 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fd5e8005020 con 0x7fd5f8070b20 2024-09-10T22:06:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.678+0000 7fd5ee000640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd5d40731b0 0x7fd5d4075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.678+0000 7fd5ee000640 1 -- 172.21.15.66:0/4059994892 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5765+0+0 (secure 0 0 0) 0x7fd5e8092150 con 0x7fd5f8070b20 2024-09-10T22:06:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.678+0000 7fd5efe00640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd5d40731b0 0x7fd5d4075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.678+0000 7fd5efe00640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd5d40731b0 0x7fd5d4075670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd5f815e860 tx=0x7fd5e00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:40.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.682+0000 7fd5ee000640 1 -- 172.21.15.66:0/4059994892 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd5e8020020 con 0x7fd5f8070b20 2024-09-10T22:06:40.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.898+0000 7fd5fce00640 1 -- 172.21.15.66:0/4059994892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5f8159b00 con 0x7fd5f8070b20 2024-09-10T22:06:40.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.898+0000 7fd5ee000640 1 -- 172.21.15.66:0/4059994892 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5220 (secure 0 0 0) 0x7fd5e805e9d0 con 0x7fd5f8070b20 2024-09-10T22:06:40.904 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:40.904 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":37,"btime":"2024-09-10T22:05:16:499594+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5414,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/135515157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":135515157},{"type":"v1","addr":"172.21.15.94:6837","nonce":135515157}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1614138608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1614138608},{"type":"v1","addr":"172.21.15.94:6836","nonce":1614138608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1060280836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1060280836},{"type":"v1","addr":"172.21.15.66:6833","nonce":1060280836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:05:07.804373+0000","modified":"2024-09-10T22:05:10.828111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5925},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5925":{"gid":5925,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2880381289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2880381289},{"type":"v1","addr":"172.21.15.66:6835","nonce":2880381289}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5925,"qdb_cluster":[5925]},"id":4}]} 2024-09-10T22:06:40.904 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 37 2024-09-10T22:06:40.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.898+0000 7fd5fce00640 1 -- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd5d40731b0 msgr2=0x7fd5d4075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:40.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.898+0000 7fd5fce00640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd5d40731b0 0x7fd5d4075670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd5f815e860 tx=0x7fd5e00091e0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.898+0000 7fd5fce00640 1 -- 172.21.15.66:0/4059994892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5f8070b20 msgr2=0x7fd5f8150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:40.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.898+0000 7fd5fce00640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5f8070b20 0x7fd5f8150de0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800d720 tx=0x7fd5e800dbe0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.902+0000 7fd5fce00640 1 -- 172.21.15.66:0/4059994892 shutdown_connections 2024-09-10T22:06:40.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.902+0000 7fd5fce00640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd5d40731b0 0x7fd5d4075670 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.902+0000 7fd5fce00640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f8160370 0x7fd5f80b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.902+0000 7fd5fce00640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5f8156430 0x7fd5f8151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.902+0000 7fd5fce00640 1 --2- 172.21.15.66:0/4059994892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5f8070b20 0x7fd5f8150de0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:40.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.902+0000 7fd5fce00640 1 -- 172.21.15.66:0/4059994892 >> 172.21.15.66:0/4059994892 conn(0x7fd5f80d0b30 msgr2=0x7fd5f8159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:40.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.902+0000 7fd5fce00640 1 -- 172.21.15.66:0/4059994892 shutdown_connections 2024-09-10T22:06:40.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:40.902+0000 7fd5fce00640 1 -- 172.21.15.66:0/4059994892 wait complete. 2024-09-10T22:06:40.926 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:06:41.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.066+0000 7f26fa200640 1 -- 172.21.15.66:0/1476500002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f40afc20 msgr2=0x7f26f40ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.066+0000 7f26fa200640 1 --2- 172.21.15.66:0/1476500002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f40afc20 0x7f26f40ce6b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f26f40747e0 tx=0x7f26ec01caf0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.066+0000 7f26fa200640 1 -- 172.21.15.66:0/1476500002 shutdown_connections 2024-09-10T22:06:41.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.066+0000 7f26fa200640 1 --2- 172.21.15.66:0/1476500002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f40afc20 0x7f26f40ce6b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.066+0000 7f26fa200640 1 --2- 172.21.15.66:0/1476500002 >> v2:172.21.15.94:3300/0 conn(0x7f26f40d06b0 0x7f26f40ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.066+0000 7f26fa200640 1 -- 172.21.15.66:0/1476500002 >> 172.21.15.66:0/1476500002 conn(0x7f26f40d1c60 msgr2=0x7f26f40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:41.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.066+0000 7f26fa200640 1 -- 172.21.15.66:0/1476500002 shutdown_connections 2024-09-10T22:06:41.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.066+0000 7f26fa200640 1 -- 172.21.15.66:0/1476500002 wait complete. 2024-09-10T22:06:41.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26fa200640 1 Processor -- start 2024-09-10T22:06:41.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26fa200640 1 -- start start 2024-09-10T22:06:41.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26fa200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f40afc20 0x7f26f4155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26fa200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26f40d06b0 0x7f26f414f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26fa200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f415bd40 0x7f26f40ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26fa200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4169200 con 0x7f26f415bd40 2024-09-10T22:06:41.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26fa200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4169080 con 0x7f26f40afc20 2024-09-10T22:06:41.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26fa200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f26f4169380 con 0x7f26f40d06b0 2024-09-10T22:06:41.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26f3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f415bd40 0x7f26f40ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26f3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f40afc20 0x7f26f4155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26f3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f415bd40 0x7f26f40ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45198/0 (socket says 172.21.15.66:45198) 2024-09-10T22:06:41.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26f3e00640 1 -- 172.21.15.66:0/999355662 learned_addr learned my addr 172.21.15.66:0/999355662 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:41.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26ebe00640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26f40d06b0 0x7f26f414f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26f3e00640 1 -- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26f40d06b0 msgr2=0x7f26f414f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26f3e00640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26f40d06b0 0x7f26f414f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26f3e00640 1 -- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f40afc20 msgr2=0x7f26f4155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26f3e00640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f40afc20 0x7f26f4155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26f3e00640 1 -- 172.21.15.66:0/999355662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f40ca620 con 0x7f26f415bd40 2024-09-10T22:06:41.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26f3e00640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f415bd40 0x7f26f40ca0e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0985f0 tx=0x7f26ec005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:41.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26eb400640 1 -- 172.21.15.66:0/999355662 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0b2040 con 0x7f26f415bd40 2024-09-10T22:06:41.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26eb400640 1 -- 172.21.15.66:0/999355662 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26ec0bb070 con 0x7f26f415bd40 2024-09-10T22:06:41.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26eb400640 1 -- 172.21.15.66:0/999355662 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec004330 con 0x7f26f415bd40 2024-09-10T22:06:41.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26fa200640 1 -- 172.21.15.66:0/999355662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f40caf60 con 0x7f26f415bd40 2024-09-10T22:06:41.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.070+0000 7f26fa200640 1 -- 172.21.15.66:0/999355662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f40cb520 con 0x7f26f415bd40 2024-09-10T22:06:41.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.074+0000 7f26fa200640 1 -- 172.21.15.66:0/999355662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f40cb720 con 0x7f26f415bd40 2024-09-10T22:06:41.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.074+0000 7f26eb400640 1 -- 172.21.15.66:0/999355662 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f26ec0b5070 con 0x7f26f415bd40 2024-09-10T22:06:41.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.074+0000 7f26eb400640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f26d00731b0 0x7f26d0075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.074+0000 7f26eb400640 1 -- 172.21.15.66:0/999355662 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5765+0+0 (secure 0 0 0) 0x7f26ec12c260 con 0x7f26f415bd40 2024-09-10T22:06:41.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.074+0000 7f26f3400640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f26d00731b0 0x7f26d0075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.074+0000 7f26f3400640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f26d00731b0 0x7f26d0075670 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f26f41f8d50 tx=0x7f26e40013a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:41.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.078+0000 7f26eb400640 1 -- 172.21.15.66:0/999355662 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f26f40cb720 con 0x7f26f415bd40 2024-09-10T22:06:41.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.286+0000 7f26fa200640 1 -- 172.21.15.66:0/999355662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26f40b0df0 con 0x7f26f415bd40 2024-09-10T22:06:41.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.286+0000 7f26eb400640 1 -- 172.21.15.66:0/999355662 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5220 (secure 0 0 0) 0x7f26ec016770 con 0x7f26f415bd40 2024-09-10T22:06:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:41.292 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":37,"btime":"2024-09-10T22:05:16:499594+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5414,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/135515157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":135515157},{"type":"v1","addr":"172.21.15.94:6837","nonce":135515157}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1614138608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1614138608},{"type":"v1","addr":"172.21.15.94:6836","nonce":1614138608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1060280836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1060280836},{"type":"v1","addr":"172.21.15.66:6833","nonce":1060280836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:05:07.804373+0000","modified":"2024-09-10T22:05:10.828111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5925},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5925":{"gid":5925,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2880381289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2880381289},{"type":"v1","addr":"172.21.15.66:6835","nonce":2880381289}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5925,"qdb_cluster":[5925]},"id":4}]} 2024-09-10T22:06:41.292 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 37 2024-09-10T22:06:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.290+0000 7f26fa200640 1 -- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f26d00731b0 msgr2=0x7f26d0075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.290+0000 7f26fa200640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f26d00731b0 0x7f26d0075670 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f26f41f8d50 tx=0x7f26e40013a0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.290+0000 7f26fa200640 1 -- 172.21.15.66:0/999355662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f415bd40 msgr2=0x7f26f40ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.290+0000 7f26fa200640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f415bd40 0x7f26f40ca0e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0985f0 tx=0x7f26ec005a80 comp rx=0 tx=0).stop 2024-09-10T22:06:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.290+0000 7f26fa200640 1 -- 172.21.15.66:0/999355662 shutdown_connections 2024-09-10T22:06:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.290+0000 7f26fa200640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f26d00731b0 0x7f26d0075670 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.290+0000 7f26fa200640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f415bd40 0x7f26f40ca0e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.290+0000 7f26fa200640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26f40d06b0 0x7f26f414f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.290+0000 7f26fa200640 1 --2- 172.21.15.66:0/999355662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26f40afc20 0x7f26f4155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.290+0000 7f26fa200640 1 -- 172.21.15.66:0/999355662 >> 172.21.15.66:0/999355662 conn(0x7f26f40d1c60 msgr2=0x7f26f40b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:41.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.290+0000 7f26fa200640 1 -- 172.21.15.66:0/999355662 shutdown_connections 2024-09-10T22:06:41.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.290+0000 7f26fa200640 1 -- 172.21.15.66:0/999355662 wait complete. 2024-09-10T22:06:41.313 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:06:41.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.454+0000 7f08b1800640 1 -- 172.21.15.66:0/4294336344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08ac0cf830 msgr2=0x7f08ac0d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.454+0000 7f08b1800640 1 --2- 172.21.15.66:0/4294336344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08ac0cf830 0x7f08ac0d04e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0732c0 tx=0x7f089801c830 comp rx=0 tx=0).stop 2024-09-10T22:06:41.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.454+0000 7f08b1800640 1 -- 172.21.15.66:0/4294336344 shutdown_connections 2024-09-10T22:06:41.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.454+0000 7f08b1800640 1 --2- 172.21.15.66:0/4294336344 >> v2:172.21.15.94:3300/0 conn(0x7f08ac0aec10 0x7f08ac0af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.454+0000 7f08b1800640 1 --2- 172.21.15.66:0/4294336344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08ac0cf830 0x7f08ac0d04e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.454+0000 7f08b1800640 1 --2- 172.21.15.66:0/4294336344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08ac15bd40 0x7f08ac0cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.454+0000 7f08b1800640 1 -- 172.21.15.66:0/4294336344 >> 172.21.15.66:0/4294336344 conn(0x7f08ac0d1c60 msgr2=0x7f08ac0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:41.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.454+0000 7f08b1800640 1 -- 172.21.15.66:0/4294336344 shutdown_connections 2024-09-10T22:06:41.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.454+0000 7f08b1800640 1 -- 172.21.15.66:0/4294336344 wait complete. 2024-09-10T22:06:41.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.454+0000 7f08b1800640 1 Processor -- start 2024-09-10T22:06:41.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.454+0000 7f08b1800640 1 -- start start 2024-09-10T22:06:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08b1800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08ac0aec10 0x7f08ac155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08b1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08ac0cf830 0x7f08ac14bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08b1800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08ac15bd40 0x7f08ac152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08b1800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac0ceab0 con 0x7f08ac0cf830 2024-09-10T22:06:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08b1800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac0ce930 con 0x7f08ac0aec10 2024-09-10T22:06:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08b1800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f08ac0cec30 con 0x7f08ac15bd40 2024-09-10T22:06:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08aa000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08ac0cf830 0x7f08ac14bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08aa000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08ac0cf830 0x7f08ac14bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45210/0 (socket says 172.21.15.66:45210) 2024-09-10T22:06:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08aa000640 1 -- 172.21.15.66:0/3195200579 learned_addr learned my addr 172.21.15.66:0/3195200579 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:41.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08aaa00640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08ac0aec10 0x7f08ac155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08ab400640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08ac15bd40 0x7f08ac152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08aa000640 1 -- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08ac15bd40 msgr2=0x7f08ac152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08aa000640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08ac15bd40 0x7f08ac152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08aa000640 1 -- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08ac0aec10 msgr2=0x7f08ac155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08aa000640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08ac0aec10 0x7f08ac155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08aa000640 1 -- 172.21.15.66:0/3195200579 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac0c58e0 con 0x7f08ac0cf830 2024-09-10T22:06:41.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08aa000640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08ac0cf830 0x7f08ac14bf60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f08980b0040 tx=0x7f0898004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:41.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08a2a00640 1 -- 172.21.15.66:0/3195200579 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08980b2040 con 0x7f08ac0cf830 2024-09-10T22:06:41.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08a2a00640 1 -- 172.21.15.66:0/3195200579 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0898005030 con 0x7f08ac0cf830 2024-09-10T22:06:41.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08a2a00640 1 -- 172.21.15.66:0/3195200579 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898005480 con 0x7f08ac0cf830 2024-09-10T22:06:41.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08b1800640 1 -- 172.21.15.66:0/3195200579 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ac0c5bc0 con 0x7f08ac0cf830 2024-09-10T22:06:41.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08b1800640 1 -- 172.21.15.66:0/3195200579 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ac0c6180 con 0x7f08ac0cf830 2024-09-10T22:06:41.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08b1800640 1 -- 172.21.15.66:0/3195200579 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ac0af040 con 0x7f08ac0cf830 2024-09-10T22:06:41.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.458+0000 7f08a2a00640 1 -- 172.21.15.66:0/3195200579 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f08980046d0 con 0x7f08ac0cf830 2024-09-10T22:06:41.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.462+0000 7f08a2a00640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f088c0731e0 0x7f088c0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.462+0000 7f08a2a00640 1 -- 172.21.15.66:0/3195200579 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5765+0+0 (secure 0 0 0) 0x7f089812b840 con 0x7f08ac0cf830 2024-09-10T22:06:41.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.462+0000 7f08aaa00640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f088c0731e0 0x7f088c0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.462+0000 7f08aaa00640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f088c0731e0 0x7f088c0756a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f089c004630 tx=0x7f089c00b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:41.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.466+0000 7f08a2a00640 1 -- 172.21.15.66:0/3195200579 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f08980164b0 con 0x7f08ac0cf830 2024-09-10T22:06:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.642+0000 7f08b1800640 1 -- 172.21.15.66:0/3195200579 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08ac15abe0 con 0x7f08ac0cf830 2024-09-10T22:06:41.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.642+0000 7f08a2a00640 1 -- 172.21.15.66:0/3195200579 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16761 (secure 0 0 0) 0x7f08980f8710 con 0x7f08ac0cf830 2024-09-10T22:06:41.649 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:41.649 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":60,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:06:39.603852+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:05:05.519188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T22:05:06.772731+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:06:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.646+0000 7f08b1800640 1 -- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f088c0731e0 msgr2=0x7f088c0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.646+0000 7f08b1800640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f088c0731e0 0x7f088c0756a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f089c004630 tx=0x7f089c00b2e0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.646+0000 7f08b1800640 1 -- 172.21.15.66:0/3195200579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08ac0cf830 msgr2=0x7f08ac14bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.646+0000 7f08b1800640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08ac0cf830 0x7f08ac14bf60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f08980b0040 tx=0x7f0898004040 comp rx=0 tx=0).stop 2024-09-10T22:06:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.646+0000 7f08b1800640 1 -- 172.21.15.66:0/3195200579 shutdown_connections 2024-09-10T22:06:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.646+0000 7f08b1800640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f088c0731e0 0x7f088c0756a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.646+0000 7f08b1800640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08ac15bd40 0x7f08ac152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.646+0000 7f08b1800640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08ac0cf830 0x7f08ac14bf60 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.646+0000 7f08b1800640 1 --2- 172.21.15.66:0/3195200579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08ac0aec10 0x7f08ac155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.646+0000 7f08b1800640 1 -- 172.21.15.66:0/3195200579 >> 172.21.15.66:0/3195200579 conn(0x7f08ac0d1c60 msgr2=0x7f08ac15b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.646+0000 7f08b1800640 1 -- 172.21.15.66:0/3195200579 shutdown_connections 2024-09-10T22:06:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.646+0000 7f08b1800640 1 -- 172.21.15.66:0/3195200579 wait complete. 2024-09-10T22:06:41.671 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:06:41.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.814+0000 7f1a7c600640 1 -- 172.21.15.66:0/1839000841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a740d0550 msgr2=0x7f1a740ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.814+0000 7f1a7c600640 1 --2- 172.21.15.66:0/1839000841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a740d0550 0x7f1a740ca1a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009900 tx=0x7f1a6001c850 comp rx=0 tx=0).stop 2024-09-10T22:06:41.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.814+0000 7f1a7c600640 1 -- 172.21.15.66:0/1839000841 shutdown_connections 2024-09-10T22:06:41.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.814+0000 7f1a7c600640 1 --2- 172.21.15.66:0/1839000841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1a740c4fc0 0x7f1a740ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.814+0000 7f1a7c600640 1 --2- 172.21.15.66:0/1839000841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a740d0550 0x7f1a740ca1a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.814+0000 7f1a7c600640 1 -- 172.21.15.66:0/1839000841 >> 172.21.15.66:0/1839000841 conn(0x7f1a740cfa30 msgr2=0x7f1a740cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:41.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.814+0000 7f1a7c600640 1 -- 172.21.15.66:0/1839000841 shutdown_connections 2024-09-10T22:06:41.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.814+0000 7f1a7c600640 1 -- 172.21.15.66:0/1839000841 wait complete. 2024-09-10T22:06:41.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.814+0000 7f1a7c600640 1 Processor -- start 2024-09-10T22:06:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.814+0000 7f1a7c600640 1 -- start start 2024-09-10T22:06:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a7c600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1a740c4fc0 0x7f1a7414d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a7c600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7415a310 0x7f1a740c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a7c600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a7414d970 0x7f1a740c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a7c600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1a74167480 con 0x7f1a7415a310 2024-09-10T22:06:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a7c600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a74167300 con 0x7f1a7414d970 2024-09-10T22:06:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a7c600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1a74167600 con 0x7f1a740c4fc0 2024-09-10T22:06:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a79200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7415a310 0x7f1a740c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a79200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7415a310 0x7f1a740c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45220/0 (socket says 172.21.15.66:45220) 2024-09-10T22:06:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a79200640 1 -- 172.21.15.66:0/3155794713 learned_addr learned my addr 172.21.15.66:0/3155794713 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a7a600640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a7414d970 0x7f1a740c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a79c00640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1a740c4fc0 0x7f1a7414d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a79200640 1 -- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1a740c4fc0 msgr2=0x7f1a7414d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a79200640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1a740c4fc0 0x7f1a7414d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a79200640 1 -- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a7414d970 msgr2=0x7f1a740c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a79200640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a7414d970 0x7f1a740c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a79200640 1 -- 172.21.15.66:0/3155794713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a74072de0 con 0x7f1a7415a310 2024-09-10T22:06:41.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a79200640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7415a310 0x7f1a740c75a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009850 tx=0x7f1a60005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:41.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a71600640 1 -- 172.21.15.66:0/3155794713 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a600bb070 con 0x7f1a7415a310 2024-09-10T22:06:41.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a71600640 1 -- 172.21.15.66:0/3155794713 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a600043f0 con 0x7f1a7415a310 2024-09-10T22:06:41.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a71600640 1 -- 172.21.15.66:0/3155794713 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a600b2040 con 0x7f1a7415a310 2024-09-10T22:06:41.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a7c600640 1 -- 172.21.15.66:0/3155794713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a740730c0 con 0x7f1a7415a310 2024-09-10T22:06:41.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a7c600640 1 -- 172.21.15.66:0/3155794713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a740c6b00 con 0x7f1a7415a310 2024-09-10T22:06:41.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a7c600640 1 -- 172.21.15.66:0/3155794713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a74070420 con 0x7f1a7415a310 2024-09-10T22:06:41.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.818+0000 7f1a71600640 1 -- 172.21.15.66:0/3155794713 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f1a600052d0 con 0x7f1a7415a310 2024-09-10T22:06:41.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.822+0000 7f1a71600640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1a540731e0 0x7f1a540756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.822+0000 7f1a71600640 1 -- 172.21.15.66:0/3155794713 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5765+0+0 (secure 0 0 0) 0x7f1a6012b5a0 con 0x7f1a7415a310 2024-09-10T22:06:41.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.822+0000 7f1a79c00640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1a540731e0 0x7f1a540756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.822+0000 7f1a79c00640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1a540731e0 0x7f1a540756a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1a74072ac0 tx=0x7f1a68009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:41.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:41.826+0000 7f1a71600640 1 -- 172.21.15.66:0/3155794713 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1a600164d0 con 0x7f1a7415a310 2024-09-10T22:06:42.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.034+0000 7f1a7c600640 1 -- 172.21.15.66:0/3155794713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1a740ca130 con 0x7f1a7415a310 2024-09-10T22:06:42.039 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:06:42.034+0000 7fcc93a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:06:42.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.034+0000 7f1a71600640 1 -- 172.21.15.66:0/3155794713 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7f1a600f7e80 con 0x7f1a7415a310 2024-09-10T22:06:42.041 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:06:42.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.038+0000 7f1a7c600640 1 -- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1a540731e0 msgr2=0x7f1a540756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:42.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.038+0000 7f1a7c600640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1a540731e0 0x7f1a540756a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1a74072ac0 tx=0x7f1a68009040 comp rx=0 tx=0).stop 2024-09-10T22:06:42.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.038+0000 7f1a7c600640 1 -- 172.21.15.66:0/3155794713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7415a310 msgr2=0x7f1a740c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:42.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.038+0000 7f1a7c600640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7415a310 0x7f1a740c75a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009850 tx=0x7f1a60005e70 comp rx=0 tx=0).stop 2024-09-10T22:06:42.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.038+0000 7f1a7c600640 1 -- 172.21.15.66:0/3155794713 shutdown_connections 2024-09-10T22:06:42.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.038+0000 7f1a7c600640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1a540731e0 0x7f1a540756a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:42.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.038+0000 7f1a7c600640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a7414d970 0x7f1a740c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:42.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.038+0000 7f1a7c600640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7415a310 0x7f1a740c75a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:42.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.038+0000 7f1a7c600640 1 --2- 172.21.15.66:0/3155794713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1a740c4fc0 0x7f1a7414d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:42.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.038+0000 7f1a7c600640 1 -- 172.21.15.66:0/3155794713 >> 172.21.15.66:0/3155794713 conn(0x7f1a740cfa30 msgr2=0x7f1a74150580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:42.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.038+0000 7f1a7c600640 1 -- 172.21.15.66:0/3155794713 shutdown_connections 2024-09-10T22:06:42.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.038+0000 7f1a7c600640 1 -- 172.21.15.66:0/3155794713 wait complete. 2024-09-10T22:06:42.068 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:06:42.112 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:06:42.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.214+0000 7f41ee600640 1 -- 172.21.15.66:0/1617129576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e815bd40 msgr2=0x7f41e80cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:42.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.214+0000 7f41ee600640 1 --2- 172.21.15.66:0/1617129576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e815bd40 0x7f41e80cffa0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f41e80cc2e0 tx=0x7f41d801c770 comp rx=0 tx=0).stop 2024-09-10T22:06:42.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.214+0000 7f41ee600640 1 -- 172.21.15.66:0/1617129576 shutdown_connections 2024-09-10T22:06:42.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.214+0000 7f41ee600640 1 --2- 172.21.15.66:0/1617129576 >> v2:172.21.15.94:3300/0 conn(0x7f41e80cf180 0x7f41e80d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:42.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.214+0000 7f41ee600640 1 --2- 172.21.15.66:0/1617129576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e815bd40 0x7f41e80cffa0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:42.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.214+0000 7f41ee600640 1 -- 172.21.15.66:0/1617129576 >> 172.21.15.66:0/1617129576 conn(0x7f41e80d1c60 msgr2=0x7f41e80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:42.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.214+0000 7f41ee600640 1 -- 172.21.15.66:0/1617129576 shutdown_connections 2024-09-10T22:06:42.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.214+0000 7f41ee600640 1 -- 172.21.15.66:0/1617129576 wait complete. 2024-09-10T22:06:42.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41ee600640 1 Processor -- start 2024-09-10T22:06:42.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41ee600640 1 -- start start 2024-09-10T22:06:42.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41ee600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41e80cf180 0x7f41e80b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:42.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41ee600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e80cf830 0x7f41e80b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:42.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41ee600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41e815bd40 0x7f41e80b6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:42.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41ee600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41e8168ba0 con 0x7f41e80cf830 2024-09-10T22:06:42.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41ee600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41e8168a20 con 0x7f41e815bd40 2024-09-10T22:06:42.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41ee600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f41e8168d20 con 0x7f41e80cf180 2024-09-10T22:06:42.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41dfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e80cf830 0x7f41e80b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41dfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e80cf830 0x7f41e80b1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45256/0 (socket says 172.21.15.66:45256) 2024-09-10T22:06:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41dfe00640 1 -- 172.21.15.66:0/1655635898 learned_addr learned my addr 172.21.15.66:0/1655635898 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41e7400640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41e80cf180 0x7f41e80b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41e7e00640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41e815bd40 0x7f41e80b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41dfe00640 1 -- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41e80cf180 msgr2=0x7f41e80b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41dfe00640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41e80cf180 0x7f41e80b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41dfe00640 1 -- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41e815bd40 msgr2=0x7f41e80b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41dfe00640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41e815bd40 0x7f41e80b6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41dfe00640 1 -- 172.21.15.66:0/1655635898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e80c9eb0 con 0x7f41e80cf830 2024-09-10T22:06:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41dfe00640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e80cf830 0x7f41e80b1c90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f41d000c450 tx=0x7f41d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41df400640 1 -- 172.21.15.66:0/1655635898 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0011790 con 0x7f41e80cf830 2024-09-10T22:06:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41df400640 1 -- 172.21.15.66:0/1655635898 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41d0017070 con 0x7f41e80cf830 2024-09-10T22:06:42.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41ee600640 1 -- 172.21.15.66:0/1655635898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e80ca190 con 0x7f41e80cf830 2024-09-10T22:06:42.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41df400640 1 -- 172.21.15.66:0/1655635898 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0011cc0 con 0x7f41e80cf830 2024-09-10T22:06:42.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41ee600640 1 -- 172.21.15.66:0/1655635898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e80ca750 con 0x7f41e80cf830 2024-09-10T22:06:42.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.218+0000 7f41ee600640 1 -- 172.21.15.66:0/1655635898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e80aec10 con 0x7f41e80cf830 2024-09-10T22:06:42.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.222+0000 7f41df400640 1 -- 172.21.15.66:0/1655635898 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f41d001c460 con 0x7f41e80cf830 2024-09-10T22:06:42.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.222+0000 7f41df400640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41c40731b0 0x7f41c4075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:42.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.222+0000 7f41df400640 1 -- 172.21.15.66:0/1655635898 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5823+0+0 (secure 0 0 0) 0x7f41d0091d00 con 0x7f41e80cf830 2024-09-10T22:06:42.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.222+0000 7f41e7400640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41c40731b0 0x7f41c4075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:42.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.222+0000 7f41e7400640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41c40731b0 0x7f41c4075670 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f41d80af000 tx=0x7f41d801afb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:42.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.226+0000 7f41df400640 1 -- 172.21.15.66:0/1655635898 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f41d005ebd0 con 0x7f41e80cf830 2024-09-10T22:06:42.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:42.434+0000 7f41ee600640 1 -- 172.21.15.66:0/1655635898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f41e80b0df0 con 0x7f41e80cf830 2024-09-10T22:06:43.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.038+0000 7f41df400640 1 -- 172.21.15.66:0/1655635898 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f41d005e4d0 con 0x7f41e80cf830 2024-09-10T22:06:43.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.042+0000 7f41ee600640 1 -- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41c40731b0 msgr2=0x7f41c4075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:43.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.042+0000 7f41ee600640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41c40731b0 0x7f41c4075670 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f41d80af000 tx=0x7f41d801afb0 comp rx=0 tx=0).stop 2024-09-10T22:06:43.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.042+0000 7f41ee600640 1 -- 172.21.15.66:0/1655635898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e80cf830 msgr2=0x7f41e80b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:43.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.042+0000 7f41ee600640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e80cf830 0x7f41e80b1c90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f41d000c450 tx=0x7f41d000c910 comp rx=0 tx=0).stop 2024-09-10T22:06:43.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.042+0000 7f41ee600640 1 -- 172.21.15.66:0/1655635898 shutdown_connections 2024-09-10T22:06:43.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.042+0000 7f41ee600640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41c40731b0 0x7f41c4075670 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:43.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.042+0000 7f41ee600640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41e815bd40 0x7f41e80b6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:43.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.042+0000 7f41ee600640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e80cf830 0x7f41e80b1c90 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:43.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.042+0000 7f41ee600640 1 --2- 172.21.15.66:0/1655635898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41e80cf180 0x7f41e80b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:43.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.042+0000 7f41ee600640 1 -- 172.21.15.66:0/1655635898 >> 172.21.15.66:0/1655635898 conn(0x7f41e80d1c60 msgr2=0x7f41e80b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:43.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.042+0000 7f41ee600640 1 -- 172.21.15.66:0/1655635898 shutdown_connections 2024-09-10T22:06:43.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.042+0000 7f41ee600640 1 -- 172.21.15.66:0/1655635898 wait complete. 2024-09-10T22:06:43.071 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:06:43.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 -- 172.21.15.66:0/3574038019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3bc0cd020 msgr2=0x7fa3bc0ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:43.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 --2- 172.21.15.66:0/3574038019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3bc0cd020 0x7fa3bc0ce9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8009900 tx=0x7fa3a801c8b0 comp rx=0 tx=0).stop 2024-09-10T22:06:43.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 -- 172.21.15.66:0/3574038019 shutdown_connections 2024-09-10T22:06:43.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 --2- 172.21.15.66:0/3574038019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3bc0cd020 0x7fa3bc0ce9c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:43.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 --2- 172.21.15.66:0/3574038019 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3bc15f240 0x7fa3bc0ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:43.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 -- 172.21.15.66:0/3574038019 >> 172.21.15.66:0/3574038019 conn(0x7fa3bc0d1c50 msgr2=0x7fa3bc0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:43.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 -- 172.21.15.66:0/3574038019 shutdown_connections 2024-09-10T22:06:43.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 -- 172.21.15.66:0/3574038019 wait complete. 2024-09-10T22:06:43.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 Processor -- start 2024-09-10T22:06:43.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 -- start start 2024-09-10T22:06:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3bc0cd020 0x7fa3bc1f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3bc0cef00 0x7fa3bc14bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3bc15f240 0x7fa3bc0afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc169070 con 0x7fa3bc0cef00 2024-09-10T22:06:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc168ef0 con 0x7fa3bc15f240 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c3e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa3bc1691f0 con 0x7fa3bc0cd020 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.262+0000 7fa3c0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3bc0cef00 0x7fa3bc14bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3bc0cef00 0x7fa3bc14bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40118/0 (socket says 172.21.15.66:40118) 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c0a00640 1 -- 172.21.15.66:0/1324948716 learned_addr learned my addr 172.21.15.66:0/1324948716 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c1400640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3bc0cd020 0x7fa3bc1f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c1e00640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3bc15f240 0x7fa3bc0afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c0a00640 1 -- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3bc0cd020 msgr2=0x7fa3bc1f9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c0a00640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3bc0cd020 0x7fa3bc1f9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c0a00640 1 -- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3bc15f240 msgr2=0x7fa3bc0afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c0a00640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3bc15f240 0x7fa3bc0afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c0a00640 1 -- 172.21.15.66:0/1324948716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3bc1fa020 con 0x7fa3bc0cef00 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c1e00640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3bc15f240 0x7fa3bc0afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:06:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c0a00640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3bc0cef00 0x7fa3bc14bb80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8009850 tx=0x7fa3a8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3b9600640 1 -- 172.21.15.66:0/1324948716 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80b2040 con 0x7fa3bc0cef00 2024-09-10T22:06:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3b9600640 1 -- 172.21.15.66:0/1324948716 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3a8005070 con 0x7fa3bc0cef00 2024-09-10T22:06:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3b9600640 1 -- 172.21.15.66:0/1324948716 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8005480 con 0x7fa3bc0cef00 2024-09-10T22:06:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1324948716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3bc14d5c0 con 0x7fa3bc0cef00 2024-09-10T22:06:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1324948716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3bc14c520 con 0x7fa3bc0cef00 2024-09-10T22:06:43.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1324948716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3bc0cfa30 con 0x7fa3bc0cef00 2024-09-10T22:06:43.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.266+0000 7fa3b9600640 1 -- 172.21.15.66:0/1324948716 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fa3a80046d0 con 0x7fa3bc0cef00 2024-09-10T22:06:43.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.270+0000 7fa3b9600640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa39c0731b0 0x7fa39c075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:43.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.270+0000 7fa3b9600640 1 -- 172.21.15.66:0/1324948716 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5823+0+0 (secure 0 0 0) 0x7fa3a812ba80 con 0x7fa3bc0cef00 2024-09-10T22:06:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.270+0000 7fa3c1400640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa39c0731b0 0x7fa39c075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.270+0000 7fa3c1400640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa39c0731b0 0x7fa39c075670 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00058d0 tx=0x7fa3b0005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:43.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.274+0000 7fa3b9600640 1 -- 172.21.15.66:0/1324948716 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa3a80164d0 con 0x7fa3bc0cef00 2024-09-10T22:06:43.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:43.450+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1324948716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa3bc158f80 con 0x7fa3bc0cef00 2024-09-10T22:06:44.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.046+0000 7fa3b9600640 1 -- 172.21.15.66:0/1324948716 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v62) ==== 181+0+0 (secure 0 0 0) 0x7fa3a80f8950 con 0x7fa3bc0cef00 2024-09-10T22:06:44.050 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:06:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.046+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa39c0731b0 msgr2=0x7fa39c075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.046+0000 7fa3c3e00640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa39c0731b0 0x7fa39c075670 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00058d0 tx=0x7fa3b0005860 comp rx=0 tx=0).stop 2024-09-10T22:06:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.046+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1324948716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3bc0cef00 msgr2=0x7fa3bc14bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.046+0000 7fa3c3e00640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3bc0cef00 0x7fa3bc14bb80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8009850 tx=0x7fa3a8004040 comp rx=0 tx=0).stop 2024-09-10T22:06:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.046+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1324948716 shutdown_connections 2024-09-10T22:06:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.046+0000 7fa3c3e00640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa39c0731b0 0x7fa39c075670 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.050+0000 7fa3c3e00640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3bc15f240 0x7fa3bc0afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.050+0000 7fa3c3e00640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3bc0cef00 0x7fa3bc14bb80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:44.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.050+0000 7fa3c3e00640 1 --2- 172.21.15.66:0/1324948716 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3bc0cd020 0x7fa3bc1f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:44.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.050+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1324948716 >> 172.21.15.66:0/1324948716 conn(0x7fa3bc0d1c50 msgr2=0x7fa3bc152130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:44.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.050+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1324948716 shutdown_connections 2024-09-10T22:06:44.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.050+0000 7fa3c3e00640 1 -- 172.21.15.66:0/1324948716 wait complete. 2024-09-10T22:06:44.077 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:06:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 -- 172.21.15.66:0/1247805669 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbdc0cf250 msgr2=0x7fdbdc15f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 --2- 172.21.15.66:0/1247805669 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbdc0cf250 0x7fdbdc15f620 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009c80 tx=0x7fdbd401cc80 comp rx=0 tx=0).stop 2024-09-10T22:06:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 -- 172.21.15.66:0/1247805669 shutdown_connections 2024-09-10T22:06:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 --2- 172.21.15.66:0/1247805669 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbdc0cf250 0x7fdbdc15f620 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 --2- 172.21.15.66:0/1247805669 >> v2:172.21.15.94:3300/0 conn(0x7fdbdc144ac0 0x7fdbdc06e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 -- 172.21.15.66:0/1247805669 >> 172.21.15.66:0/1247805669 conn(0x7fdbdc0cfaa0 msgr2=0x7fdbdc0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 -- 172.21.15.66:0/1247805669 shutdown_connections 2024-09-10T22:06:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 -- 172.21.15.66:0/1247805669 wait complete. 2024-09-10T22:06:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 Processor -- start 2024-09-10T22:06:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 -- start start 2024-09-10T22:06:44.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbdc0cf250 0x7fdbdc157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:44.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbdc144ac0 0x7fdbdc15c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbdc145050 0x7fdbdc13b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc168d30 con 0x7fdbdc0cf250 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc168bb0 con 0x7fdbdc144ac0 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbe0c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdbdc168eb0 con 0x7fdbdc145050 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbd9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbdc0cf250 0x7fdbdc157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbd9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbdc0cf250 0x7fdbdc157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40134/0 (socket says 172.21.15.66:40134) 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbd9e00640 1 -- 172.21.15.66:0/2732236431 learned_addr learned my addr 172.21.15.66:0/2732236431 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbda800640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbdc145050 0x7fdbdc13b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbd3e00640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbdc144ac0 0x7fdbdc15c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbd9e00640 1 -- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbdc145050 msgr2=0x7fdbdc13b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbd9e00640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbdc145050 0x7fdbdc13b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbd9e00640 1 -- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbdc144ac0 msgr2=0x7fdbdc15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbd9e00640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbdc144ac0 0x7fdbdc15c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:44.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.230+0000 7fdbd9e00640 1 -- 172.21.15.66:0/2732236431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc0b2d00 con 0x7fdbdc0cf250 2024-09-10T22:06:44.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.234+0000 7fdbda800640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbdc145050 0x7fdbdc13b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:44.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.234+0000 7fdbd9e00640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbdc0cf250 0x7fdbdc157d60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00ea10 tx=0x7fdbcc00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:44.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.234+0000 7fdbd2000640 1 -- 172.21.15.66:0/2732236431 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00cde0 con 0x7fdbdc0cf250 2024-09-10T22:06:44.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.234+0000 7fdbd2000640 1 -- 172.21.15.66:0/2732236431 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbcc004020 con 0x7fdbdc0cf250 2024-09-10T22:06:44.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.234+0000 7fdbd2000640 1 -- 172.21.15.66:0/2732236431 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc004470 con 0x7fdbdc0cf250 2024-09-10T22:06:44.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.234+0000 7fdbe0c00640 1 -- 172.21.15.66:0/2732236431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc0b2fe0 con 0x7fdbdc0cf250 2024-09-10T22:06:44.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.234+0000 7fdbe0c00640 1 -- 172.21.15.66:0/2732236431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc0b35a0 con 0x7fdbdc0cf250 2024-09-10T22:06:44.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.234+0000 7fdbe0c00640 1 -- 172.21.15.66:0/2732236431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbdc15f620 con 0x7fdbdc0cf250 2024-09-10T22:06:44.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.234+0000 7fdbd2000640 1 -- 172.21.15.66:0/2732236431 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fdbcc003600 con 0x7fdbdc0cf250 2024-09-10T22:06:44.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.234+0000 7fdbd2000640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdbb80731b0 0x7fdbb8075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:44.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.234+0000 7fdbd2000640 1 -- 172.21.15.66:0/2732236431 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5385+0+0 (secure 0 0 0) 0x7fdbcc092660 con 0x7fdbdc0cf250 2024-09-10T22:06:44.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.238+0000 7fdbd3e00640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdbb80731b0 0x7fdbb8075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.238+0000 7fdbd3e00640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdbb80731b0 0x7fdbb8075670 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc165220 tx=0x7fdbc4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:44.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.238+0000 7fdbd2000640 1 -- 172.21.15.66:0/2732236431 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdbcc05f790 con 0x7fdbdc0cf250 2024-09-10T22:06:44.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:44.418+0000 7fdbe0c00640 1 -- 172.21.15.66:0/2732236431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdbdc06ed00 con 0x7fdbdc0cf250 2024-09-10T22:06:45.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.046+0000 7fdbd2000640 1 -- 172.21.15.66:0/2732236431 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v63) ==== 169+0+0 (secure 0 0 0) 0x7fdbcc05f090 con 0x7fdbdc0cf250 2024-09-10T22:06:45.053 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:06:45.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.050+0000 7fdbe0c00640 1 -- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdbb80731b0 msgr2=0x7fdbb8075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:45.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.050+0000 7fdbe0c00640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdbb80731b0 0x7fdbb8075670 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc165220 tx=0x7fdbc4009040 comp rx=0 tx=0).stop 2024-09-10T22:06:45.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.050+0000 7fdbe0c00640 1 -- 172.21.15.66:0/2732236431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbdc0cf250 msgr2=0x7fdbdc157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:45.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.050+0000 7fdbe0c00640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbdc0cf250 0x7fdbdc157d60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00ea10 tx=0x7fdbcc00eed0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.050+0000 7fdbe0c00640 1 -- 172.21.15.66:0/2732236431 shutdown_connections 2024-09-10T22:06:45.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.050+0000 7fdbe0c00640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdbb80731b0 0x7fdbb8075670 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.050+0000 7fdbe0c00640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbdc145050 0x7fdbdc13b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.050+0000 7fdbe0c00640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbdc144ac0 0x7fdbdc15c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.050+0000 7fdbe0c00640 1 --2- 172.21.15.66:0/2732236431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbdc0cf250 0x7fdbdc157d60 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.050+0000 7fdbe0c00640 1 -- 172.21.15.66:0/2732236431 >> 172.21.15.66:0/2732236431 conn(0x7fdbdc0cfaa0 msgr2=0x7fdbdc0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:45.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.050+0000 7fdbe0c00640 1 -- 172.21.15.66:0/2732236431 shutdown_connections 2024-09-10T22:06:45.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.050+0000 7fdbe0c00640 1 -- 172.21.15.66:0/2732236431 wait complete. 2024-09-10T22:06:45.086 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T22:06:45.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.246+0000 7fc93aa00640 1 -- 172.21.15.66:0/1563936149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93415f450 msgr2=0x7fc9340b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:45.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.246+0000 7fc93aa00640 1 --2- 172.21.15.66:0/1563936149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93415f450 0x7fc9340b3170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc924009900 tx=0x7fc92401d280 comp rx=0 tx=0).stop 2024-09-10T22:06:45.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.246+0000 7fc93aa00640 1 -- 172.21.15.66:0/1563936149 shutdown_connections 2024-09-10T22:06:45.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.246+0000 7fc93aa00640 1 --2- 172.21.15.66:0/1563936149 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc934160280 0x7fc9340aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.246+0000 7fc93aa00640 1 --2- 172.21.15.66:0/1563936149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93415f450 0x7fc9340b3170 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.246+0000 7fc93aa00640 1 -- 172.21.15.66:0/1563936149 >> 172.21.15.66:0/1563936149 conn(0x7fc9340d1c60 msgr2=0x7fc9340d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:45.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.246+0000 7fc93aa00640 1 -- 172.21.15.66:0/1563936149 shutdown_connections 2024-09-10T22:06:45.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.246+0000 7fc93aa00640 1 -- 172.21.15.66:0/1563936149 wait complete. 2024-09-10T22:06:45.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc93aa00640 1 Processor -- start 2024-09-10T22:06:45.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc93aa00640 1 -- start start 2024-09-10T22:06:45.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc93aa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9340b1e10 0x7fc9341fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:45.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc93aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93415f450 0x7fc9340b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:45.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc93aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc934160280 0x7fc93414aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:45.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc93aa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340afe80 con 0x7fc934160280 2024-09-10T22:06:45.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc93aa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340afd00 con 0x7fc93415f450 2024-09-10T22:06:45.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc93aa00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc934168eb0 con 0x7fc9340b1e10 2024-09-10T22:06:45.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc938a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc934160280 0x7fc93414aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:45.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc933e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9340b1e10 0x7fc9341fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc933e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9340b1e10 0x7fc9341fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:38552/0 (socket says 172.21.15.66:38552) 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc938a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc934160280 0x7fc93414aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40140/0 (socket says 172.21.15.66:40140) 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc933e00640 1 -- 172.21.15.66:0/384678707 learned_addr learned my addr 172.21.15.66:0/384678707 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc92be00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93415f450 0x7fc9340b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc938a00640 1 -- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9340b1e10 msgr2=0x7fc9341fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc938a00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9340b1e10 0x7fc9341fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc938a00640 1 -- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93415f450 msgr2=0x7fc9340b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc938a00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93415f450 0x7fc9340b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc938a00640 1 -- 172.21.15.66:0/384678707 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9341fc220 con 0x7fc934160280 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc933e00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9340b1e10 0x7fc9341fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc938a00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc934160280 0x7fc93414aea0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00e9f0 tx=0x7fc92c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc930c00640 1 -- 172.21.15.66:0/384678707 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00cde0 con 0x7fc934160280 2024-09-10T22:06:45.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc930c00640 1 -- 172.21.15.66:0/384678707 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc92c004c20 con 0x7fc934160280 2024-09-10T22:06:45.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc930c00640 1 -- 172.21.15.66:0/384678707 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c005020 con 0x7fc934160280 2024-09-10T22:06:45.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc93aa00640 1 -- 172.21.15.66:0/384678707 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9340b6260 con 0x7fc934160280 2024-09-10T22:06:45.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc93aa00640 1 -- 172.21.15.66:0/384678707 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9340b6aa0 con 0x7fc934160280 2024-09-10T22:06:45.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.250+0000 7fc93aa00640 1 -- 172.21.15.66:0/384678707 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9340b3170 con 0x7fc934160280 2024-09-10T22:06:45.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.254+0000 7fc930c00640 1 -- 172.21.15.66:0/384678707 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fc92c0051c0 con 0x7fc934160280 2024-09-10T22:06:45.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.254+0000 7fc930c00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc9100731b0 0x7fc910075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:45.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.254+0000 7fc930c00640 1 -- 172.21.15.66:0/384678707 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4971+0+0 (secure 0 0 0) 0x7fc92c0929a0 con 0x7fc934160280 2024-09-10T22:06:45.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.254+0000 7fc933e00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc9100731b0 0x7fc910075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:45.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.254+0000 7fc933e00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc9100731b0 0x7fc910075670 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc92401aed0 tx=0x7fc92401b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:45.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.258+0000 7fc930c00640 1 -- 172.21.15.66:0/384678707 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc9340b6260 con 0x7fc934160280 2024-09-10T22:06:45.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.430+0000 7fc93aa00640 1 -- 172.21.15.66:0/384678707 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fc9340d2a40 con 0x7fc934160280 2024-09-10T22:06:45.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.434+0000 7fc930c00640 1 -- 172.21.15.66:0/384678707 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v36) ==== 71+0+0 (secure 0 0 0) 0x7fc92c05f380 con 0x7fc934160280 2024-09-10T22:06:45.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.438+0000 7fc93aa00640 1 -- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc9100731b0 msgr2=0x7fc910075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:45.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.438+0000 7fc93aa00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc9100731b0 0x7fc910075670 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc92401aed0 tx=0x7fc92401b060 comp rx=0 tx=0).stop 2024-09-10T22:06:45.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.438+0000 7fc93aa00640 1 -- 172.21.15.66:0/384678707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc934160280 msgr2=0x7fc93414aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:45.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.438+0000 7fc93aa00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc934160280 0x7fc93414aea0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00e9f0 tx=0x7fc92c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.438+0000 7fc93aa00640 1 -- 172.21.15.66:0/384678707 shutdown_connections 2024-09-10T22:06:45.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.438+0000 7fc93aa00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc9100731b0 0x7fc910075670 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.438+0000 7fc93aa00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc934160280 0x7fc93414aea0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.438+0000 7fc93aa00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93415f450 0x7fc9340b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.438+0000 7fc93aa00640 1 --2- 172.21.15.66:0/384678707 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9340b1e10 0x7fc9341fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.438+0000 7fc93aa00640 1 -- 172.21.15.66:0/384678707 >> 172.21.15.66:0/384678707 conn(0x7fc9340d1c60 msgr2=0x7fc9340d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:45.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.438+0000 7fc93aa00640 1 -- 172.21.15.66:0/384678707 shutdown_connections 2024-09-10T22:06:45.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.438+0000 7fc93aa00640 1 -- 172.21.15.66:0/384678707 wait complete. 2024-09-10T22:06:45.463 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-10T22:06:45.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 -- 172.21.15.66:0/2508260250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23280c4ef0 msgr2=0x7f23280c5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:45.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 --2- 172.21.15.66:0/2508260250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23280c4ef0 0x7f23280c5380 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2320009c80 tx=0x7f232001ccb0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 -- 172.21.15.66:0/2508260250 shutdown_connections 2024-09-10T22:06:45.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 --2- 172.21.15.66:0/2508260250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23280c4ef0 0x7f23280c5380 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 --2- 172.21.15.66:0/2508260250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23280c4b10 0x7f232806e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 -- 172.21.15.66:0/2508260250 >> 172.21.15.66:0/2508260250 conn(0x7f23280cfa30 msgr2=0x7f23280cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:45.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 -- 172.21.15.66:0/2508260250 shutdown_connections 2024-09-10T22:06:45.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 -- 172.21.15.66:0/2508260250 wait complete. 2024-09-10T22:06:45.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 Processor -- start 2024-09-10T22:06:45.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 -- start start 2024-09-10T22:06:45.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23280ae620 0x7f2328141360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:45.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23280c4b10 0x7f2328153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:45.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23280c4ef0 0x7f23281535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:45.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2328167c90 con 0x7f23280c4b10 2024-09-10T22:06:45.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2328167b10 con 0x7f23280ae620 2024-09-10T22:06:45.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232f400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2328167e10 con 0x7f23280c4ef0 2024-09-10T22:06:45.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f2327e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23280c4b10 0x7f2328153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:45.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23280ae620 0x7f2328141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:45.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f2327e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23280c4b10 0x7f2328153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40154/0 (socket says 172.21.15.66:40154) 2024-09-10T22:06:45.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f2327e00640 1 -- 172.21.15.66:0/559547138 learned_addr learned my addr 172.21.15.66:0/559547138 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:45.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.606+0000 7f232d400640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23280c4ef0 0x7f23281535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:45.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f2327e00640 1 -- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23280c4ef0 msgr2=0x7f23281535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:45.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f2327e00640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23280c4ef0 0x7f23281535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f2327e00640 1 -- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23280ae620 msgr2=0x7f2328141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:45.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f2327e00640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23280ae620 0x7f2328141360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:45.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f2327e00640 1 -- 172.21.15.66:0/559547138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2328150220 con 0x7f23280c4b10 2024-09-10T22:06:45.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f2327e00640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23280c4b10 0x7f2328153060 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f231400c530 tx=0x7f231400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:45.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f2324c00640 1 -- 172.21.15.66:0/559547138 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314017070 con 0x7f23280c4b10 2024-09-10T22:06:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f2324c00640 1 -- 172.21.15.66:0/559547138 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23140053c0 con 0x7f23280c4b10 2024-09-10T22:06:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f2324c00640 1 -- 172.21.15.66:0/559547138 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314012890 con 0x7f23280c4b10 2024-09-10T22:06:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f232f400640 1 -- 172.21.15.66:0/559547138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23281418a0 con 0x7f23280c4b10 2024-09-10T22:06:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f232f400640 1 -- 172.21.15.66:0/559547138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2328154dc0 con 0x7f23280c4b10 2024-09-10T22:06:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f232f400640 1 -- 172.21.15.66:0/559547138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232806e960 con 0x7f23280c4b10 2024-09-10T22:06:45.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f2324c00640 1 -- 172.21.15.66:0/559547138 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f2314003600 con 0x7f23280c4b10 2024-09-10T22:06:45.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f2324c00640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f23080731b0 0x7f2308075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:45.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.610+0000 7f2324c00640 1 -- 172.21.15.66:0/559547138 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4971+0+0 (secure 0 0 0) 0x7f2314091450 con 0x7f23280c4b10 2024-09-10T22:06:45.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.614+0000 7f232ca00640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f23080731b0 0x7f2308075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:45.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.614+0000 7f232ca00640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f23080731b0 0x7f2308075670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f231c004630 tx=0x7f231c00b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:45.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.614+0000 7f2324c00640 1 -- 172.21.15.66:0/559547138 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2314099020 con 0x7f23280c4b10 2024-09-10T22:06:45.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:45.850+0000 7f232f400640 1 -- 172.21.15.66:0/559547138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7f23280704d0 con 0x7f23280c4b10 2024-09-10T22:06:46.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f2324c00640 1 -- 172.21.15.66:0/559547138 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v280) ==== 167+0+0 (secure 0 0 0) 0x7f2328150220 con 0x7f23280c4b10 2024-09-10T22:06:46.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f232f400640 1 -- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f23080731b0 msgr2=0x7f2308075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f232f400640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f23080731b0 0x7f2308075670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f231c004630 tx=0x7f231c00b390 comp rx=0 tx=0).stop 2024-09-10T22:06:46.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f232f400640 1 -- 172.21.15.66:0/559547138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23280c4b10 msgr2=0x7f2328153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f232f400640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23280c4b10 0x7f2328153060 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f231400c530 tx=0x7f231400c9f0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f232f400640 1 -- 172.21.15.66:0/559547138 shutdown_connections 2024-09-10T22:06:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f232f400640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f23080731b0 0x7f2308075670 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f232f400640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23280c4ef0 0x7f23281535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f232f400640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23280c4b10 0x7f2328153060 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f232f400640 1 --2- 172.21.15.66:0/559547138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23280ae620 0x7f2328141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f232f400640 1 -- 172.21.15.66:0/559547138 >> 172.21.15.66:0/559547138 conn(0x7f23280cfa30 msgr2=0x7f23280b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f232f400640 1 -- 172.21.15.66:0/559547138 shutdown_connections 2024-09-10T22:06:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.058+0000 7f232f400640 1 -- 172.21.15.66:0/559547138 wait complete. 2024-09-10T22:06:46.086 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-10T22:06:46.086 INFO:tasks.cephfs_test_runner:Starting test: test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that 2 progress bars are printed in output of "ceph status" 2024-09-10T22:06:46.086 INFO:tasks.cephfs_test_runner:test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-10T22:06:46.086 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:06:46.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.226+0000 7f062d200640 1 -- 172.21.15.66:0/2347398949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06280ce660 msgr2=0x7f06280cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.226+0000 7f062d200640 1 --2- 172.21.15.66:0/2347398949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06280ce660 0x7f06280cea40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0618009870 tx=0x7f061801c790 comp rx=0 tx=0).stop 2024-09-10T22:06:46.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.226+0000 7f062d200640 1 -- 172.21.15.66:0/2347398949 shutdown_connections 2024-09-10T22:06:46.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.226+0000 7f062d200640 1 --2- 172.21.15.66:0/2347398949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06280ce660 0x7f06280cea40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.226+0000 7f062d200640 1 -- 172.21.15.66:0/2347398949 >> 172.21.15.66:0/2347398949 conn(0x7f06280d1c50 msgr2=0x7f06280d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:46.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.226+0000 7f062d200640 1 -- 172.21.15.66:0/2347398949 shutdown_connections 2024-09-10T22:06:46.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.226+0000 7f062d200640 1 -- 172.21.15.66:0/2347398949 wait complete. 2024-09-10T22:06:46.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f062d200640 1 Processor -- start 2024-09-10T22:06:46.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f062d200640 1 -- start start 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f062d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f062806ebc0 0x7f06281f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f062d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06280ce660 0x7f0628151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f062d200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f062815e460 0x7f06281fbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f062d200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f062816ca00 con 0x7f06280ce660 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f062d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f062806f970 con 0x7f062806ebc0 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f062d200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f062816cb80 con 0x7f062815e460 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f061fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06280ce660 0x7f0628151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f061fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06280ce660 0x7f0628151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40162/0 (socket says 172.21.15.66:40162) 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f061fe00640 1 -- 172.21.15.66:0/3582468147 learned_addr learned my addr 172.21.15.66:0/3582468147 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f0625e00640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f062806ebc0 0x7f06281f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f0626800640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f062815e460 0x7f06281fbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f061fe00640 1 -- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f062815e460 msgr2=0x7f06281fbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f061fe00640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f062815e460 0x7f06281fbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f061fe00640 1 -- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f062806ebc0 msgr2=0x7f06281f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f061fe00640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f062806ebc0 0x7f06281f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f061fe00640 1 -- 172.21.15.66:0/3582468147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0628158350 con 0x7f06280ce660 2024-09-10T22:06:46.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f0625e00640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f062806ebc0 0x7f06281f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:46.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f0626800640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f062815e460 0x7f06281fbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:06:46.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f061fe00640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06280ce660 0x7f0628151060 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f061000c450 tx=0x7f061000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:46.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f061e000640 1 -- 172.21.15.66:0/3582468147 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06100052d0 con 0x7f06280ce660 2024-09-10T22:06:46.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f061e000640 1 -- 172.21.15.66:0/3582468147 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0610005470 con 0x7f06280ce660 2024-09-10T22:06:46.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f062d200640 1 -- 172.21.15.66:0/3582468147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0628158630 con 0x7f06280ce660 2024-09-10T22:06:46.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f061e000640 1 -- 172.21.15.66:0/3582468147 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610012870 con 0x7f06280ce660 2024-09-10T22:06:46.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f062d200640 1 -- 172.21.15.66:0/3582468147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06281f8180 con 0x7f06280ce660 2024-09-10T22:06:46.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.230+0000 7f062d200640 1 -- 172.21.15.66:0/3582468147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06280cea40 con 0x7f06280ce660 2024-09-10T22:06:46.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.234+0000 7f061e000640 1 -- 172.21.15.66:0/3582468147 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f0610003600 con 0x7f06280ce660 2024-09-10T22:06:46.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.234+0000 7f061e000640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f06040731b0 0x7f0604075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:46.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.234+0000 7f061e000640 1 -- 172.21.15.66:0/3582468147 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4971+0+0 (secure 0 0 0) 0x7f06100943a0 con 0x7f06280ce660 2024-09-10T22:06:46.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.234+0000 7f0625e00640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f06040731b0 0x7f0604075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:46.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.234+0000 7f0625e00640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f06040731b0 0x7f0604075670 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f06180af000 tx=0x7f061801afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.238+0000 7f061e000640 1 -- 172.21.15.66:0/3582468147 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f061005e820 con 0x7f06280ce660 2024-09-10T22:06:46.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.414+0000 7f062d200640 1 -- 172.21.15.66:0/3582468147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0628154b70 con 0x7f06280ce660 2024-09-10T22:06:46.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.414+0000 7f061e000640 1 -- 172.21.15.66:0/3582468147 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+13003 (secure 0 0 0) 0x7f061005e120 con 0x7f06280ce660 2024-09-10T22:06:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:46.420 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":63,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:06:45.049147+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/2880381289":"2024-09-11T22:06:42.038582+0000","172.21.15.66:6834/2880381289":"2024-09-11T22:06:42.038582+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:06:46.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.418+0000 7f062d200640 1 -- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f06040731b0 msgr2=0x7f0604075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.418+0000 7f062d200640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f06040731b0 0x7f0604075670 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f06180af000 tx=0x7f061801afd0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.418+0000 7f062d200640 1 -- 172.21.15.66:0/3582468147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06280ce660 msgr2=0x7f0628151060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.418+0000 7f062d200640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06280ce660 0x7f0628151060 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f061000c450 tx=0x7f061000c910 comp rx=0 tx=0).stop 2024-09-10T22:06:46.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.418+0000 7f062d200640 1 -- 172.21.15.66:0/3582468147 shutdown_connections 2024-09-10T22:06:46.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.418+0000 7f062d200640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f06040731b0 0x7f0604075670 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.418+0000 7f062d200640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f062815e460 0x7f06281fbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.418+0000 7f062d200640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06280ce660 0x7f0628151060 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.418+0000 7f062d200640 1 --2- 172.21.15.66:0/3582468147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f062806ebc0 0x7f06281f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.418+0000 7f062d200640 1 -- 172.21.15.66:0/3582468147 >> 172.21.15.66:0/3582468147 conn(0x7f06280d1c50 msgr2=0x7f062815f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:46.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.418+0000 7f062d200640 1 -- 172.21.15.66:0/3582468147 shutdown_connections 2024-09-10T22:06:46.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.418+0000 7f062d200640 1 -- 172.21.15.66:0/3582468147 wait complete. 2024-09-10T22:06:46.443 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:00:22.340318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:06:46.443 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:06:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.586+0000 7f9059800640 1 -- 172.21.15.66:0/3324818222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905415a310 msgr2=0x7f90540c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.586+0000 7f9059800640 1 --2- 172.21.15.66:0/3324818222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905415a310 0x7f90540c48b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f90540cf9e0 tx=0x7f904401c770 comp rx=0 tx=0).stop 2024-09-10T22:06:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.586+0000 7f9059800640 1 -- 172.21.15.66:0/3324818222 shutdown_connections 2024-09-10T22:06:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.586+0000 7f9059800640 1 --2- 172.21.15.66:0/3324818222 >> v2:172.21.15.94:3300/0 conn(0x7f90540d0550 0x7f90540c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.586+0000 7f9059800640 1 --2- 172.21.15.66:0/3324818222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905415a310 0x7f90540c48b0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.586+0000 7f9059800640 1 -- 172.21.15.66:0/3324818222 >> 172.21.15.66:0/3324818222 conn(0x7f90540cfa30 msgr2=0x7f90540cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.586+0000 7f9059800640 1 -- 172.21.15.66:0/3324818222 shutdown_connections 2024-09-10T22:06:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.586+0000 7f9059800640 1 -- 172.21.15.66:0/3324818222 wait complete. 2024-09-10T22:06:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9059800640 1 Processor -- start 2024-09-10T22:06:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9059800640 1 -- start start 2024-09-10T22:06:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9059800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90540aeec0 0x7f90541f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9059800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905415a310 0x7f90540b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9059800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90540c7710 0x7f90541faa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9059800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f905415ed10 con 0x7f90540aeec0 2024-09-10T22:06:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9059800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f905415eb90 con 0x7f90540c7710 2024-09-10T22:06:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9059800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9054167300 con 0x7f905415a310 2024-09-10T22:06:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9052a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90540aeec0 0x7f90541f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9052a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90540aeec0 0x7f90541f8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40188/0 (socket says 172.21.15.66:40188) 2024-09-10T22:06:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9052a00640 1 -- 172.21.15.66:0/4083464639 learned_addr learned my addr 172.21.15.66:0/4083464639 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9053400640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90540c7710 0x7f90541faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f904be00640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905415a310 0x7f90540b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9052a00640 1 -- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905415a310 msgr2=0x7f90540b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9052a00640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905415a310 0x7f90540b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9052a00640 1 -- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90540c7710 msgr2=0x7f90541faa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9052a00640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90540c7710 0x7f90541faa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9052a00640 1 -- 172.21.15.66:0/4083464639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90540b29e0 con 0x7f90540aeec0 2024-09-10T22:06:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9052a00640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90540aeec0 0x7f90541f8d80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f904401a3a0 tx=0x7f90440a8530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f904aa00640 1 -- 172.21.15.66:0/4083464639 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90440be070 con 0x7f90540aeec0 2024-09-10T22:06:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f904aa00640 1 -- 172.21.15.66:0/4083464639 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90440a8a60 con 0x7f90540aeec0 2024-09-10T22:06:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f904aa00640 1 -- 172.21.15.66:0/4083464639 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90440a8e50 con 0x7f90540aeec0 2024-09-10T22:06:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9059800640 1 -- 172.21.15.66:0/4083464639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90540c7b80 con 0x7f90540aeec0 2024-09-10T22:06:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9059800640 1 -- 172.21.15.66:0/4083464639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90540b2f30 con 0x7f90540aeec0 2024-09-10T22:06:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.590+0000 7f9059800640 1 -- 172.21.15.66:0/4083464639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905415df60 con 0x7f90540aeec0 2024-09-10T22:06:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.594+0000 7f904aa00640 1 -- 172.21.15.66:0/4083464639 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f9044005040 con 0x7f90540aeec0 2024-09-10T22:06:46.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.594+0000 7f904aa00640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9030073350 0x7f9030075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:46.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.594+0000 7f904aa00640 1 -- 172.21.15.66:0/4083464639 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4971+0+0 (secure 0 0 0) 0x7f904412bf90 con 0x7f90540aeec0 2024-09-10T22:06:46.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.594+0000 7f904be00640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9030073350 0x7f9030075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:46.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.594+0000 7f904be00640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9030073350 0x7f9030075810 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f903c005d90 tx=0x7f903c005c60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:46.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.598+0000 7f904aa00640 1 -- 172.21.15.66:0/4083464639 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f90440ac030 con 0x7f90540aeec0 2024-09-10T22:06:46.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.770+0000 7f9059800640 1 -- 172.21.15.66:0/4083464639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9054070420 con 0x7f90540aeec0 2024-09-10T22:06:46.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.770+0000 7f904aa00640 1 -- 172.21.15.66:0/4083464639 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7f90440163f0 con 0x7f90540aeec0 2024-09-10T22:06:46.777 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-09-10T22:06:46.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.774+0000 7f9059800640 1 -- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9030073350 msgr2=0x7f9030075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.774+0000 7f9059800640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9030073350 0x7f9030075810 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f903c005d90 tx=0x7f903c005c60 comp rx=0 tx=0).stop 2024-09-10T22:06:46.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.774+0000 7f9059800640 1 -- 172.21.15.66:0/4083464639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90540aeec0 msgr2=0x7f90541f8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.774+0000 7f9059800640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90540aeec0 0x7f90541f8d80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f904401a3a0 tx=0x7f90440a8530 comp rx=0 tx=0).stop 2024-09-10T22:06:46.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.774+0000 7f9059800640 1 -- 172.21.15.66:0/4083464639 shutdown_connections 2024-09-10T22:06:46.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.774+0000 7f9059800640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9030073350 0x7f9030075810 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.774+0000 7f9059800640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90540c7710 0x7f90541faa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.774+0000 7f9059800640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905415a310 0x7f90540b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.774+0000 7f9059800640 1 --2- 172.21.15.66:0/4083464639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90540aeec0 0x7f90541f8d80 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.774+0000 7f9059800640 1 -- 172.21.15.66:0/4083464639 >> 172.21.15.66:0/4083464639 conn(0x7f90540cfa30 msgr2=0x7f90540adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:46.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.774+0000 7f9059800640 1 -- 172.21.15.66:0/4083464639 shutdown_connections 2024-09-10T22:06:46.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.774+0000 7f9059800640 1 -- 172.21.15.66:0/4083464639 wait complete. 2024-09-10T22:06:46.798 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-10T22:06:46.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 -- 172.21.15.66:0/3833550249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48740052f0 msgr2=0x7f487400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 --2- 172.21.15.66:0/3833550249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48740052f0 0x7f487400b9d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4888006fd0 tx=0x7f488801caf0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 -- 172.21.15.66:0/3833550249 shutdown_connections 2024-09-10T22:06:46.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 --2- 172.21.15.66:0/3833550249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48740052f0 0x7f487400b9d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 -- 172.21.15.66:0/3833550249 >> 172.21.15.66:0/3833550249 conn(0x7f48900cfa30 msgr2=0x7f48900cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:46.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 -- 172.21.15.66:0/3833550249 shutdown_connections 2024-09-10T22:06:46.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 -- 172.21.15.66:0/3833550249 wait complete. 2024-09-10T22:06:46.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 Processor -- start 2024-09-10T22:06:46.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 -- start start 2024-09-10T22:06:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48740052f0 0x7f48900b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48900c65c0 0x7f48900c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4890153710 0x7f4890073520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f48900ca200 con 0x7f48740052f0 2024-09-10T22:06:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48900ca080 con 0x7f4890153710 2024-09-10T22:06:46.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4896e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f48900ca380 con 0x7f48900c65c0 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f488fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48740052f0 0x7f48900b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f488fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48740052f0 0x7f48900b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40208/0 (socket says 172.21.15.66:40208) 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f488fe00640 1 -- 172.21.15.66:0/2151031900 learned_addr learned my addr 172.21.15.66:0/2151031900 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4894e00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4890153710 0x7f4890073520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f4887e00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48900c65c0 0x7f48900c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f488fe00640 1 -- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48900c65c0 msgr2=0x7f48900c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f488fe00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48900c65c0 0x7f48900c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f488fe00640 1 -- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4890153710 msgr2=0x7f4890073520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f488fe00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4890153710 0x7f4890073520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.942+0000 7f488fe00640 1 -- 172.21.15.66:0/2151031900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48900c5800 con 0x7f48740052f0 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.946+0000 7f488fe00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48740052f0 0x7f48900b4000 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4880000fc0 tx=0x7f488000e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.946+0000 7f488cc00640 1 -- 172.21.15.66:0/2151031900 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880012850 con 0x7f48740052f0 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.946+0000 7f488cc00640 1 -- 172.21.15.66:0/2151031900 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4880012ae0 con 0x7f48740052f0 2024-09-10T22:06:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.946+0000 7f4896e00640 1 -- 172.21.15.66:0/2151031900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489015b4c0 con 0x7f48740052f0 2024-09-10T22:06:46.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.946+0000 7f488cc00640 1 -- 172.21.15.66:0/2151031900 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488001a400 con 0x7f48740052f0 2024-09-10T22:06:46.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.946+0000 7f4896e00640 1 -- 172.21.15.66:0/2151031900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489015ba80 con 0x7f48740052f0 2024-09-10T22:06:46.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.946+0000 7f4896e00640 1 -- 172.21.15.66:0/2151031900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48900ae940 con 0x7f48740052f0 2024-09-10T22:06:46.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.946+0000 7f488cc00640 1 -- 172.21.15.66:0/2151031900 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f488001a5a0 con 0x7f48740052f0 2024-09-10T22:06:46.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.946+0000 7f488cc00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f486c0731e0 0x7f486c0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:46.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.950+0000 7f488cc00640 1 -- 172.21.15.66:0/2151031900 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4971+0+0 (secure 0 0 0) 0x7f4880095390 con 0x7f48740052f0 2024-09-10T22:06:46.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.950+0000 7f4887e00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f486c0731e0 0x7f486c0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:46.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.950+0000 7f4887e00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f486c0731e0 0x7f486c0756a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f489006fa50 tx=0x7f48780091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:46.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:46.954+0000 7f488cc00640 1 -- 172.21.15.66:0/2151031900 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4880016030 con 0x7f48740052f0 2024-09-10T22:06:47.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:47.194+0000 7f4896e00640 1 -- 172.21.15.66:0/2151031900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7f48900cddb0 con 0x7f48740052f0 2024-09-10T22:06:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f488cc00640 1 -- 172.21.15.66:0/2151031900 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v282) ==== 170+0+0 (secure 0 0 0) 0x7f4880062260 con 0x7f48740052f0 2024-09-10T22:06:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f4896e00640 1 -- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f486c0731e0 msgr2=0x7f486c0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f4896e00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f486c0731e0 0x7f486c0756a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f489006fa50 tx=0x7f48780091e0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f4896e00640 1 -- 172.21.15.66:0/2151031900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48740052f0 msgr2=0x7f48900b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f4896e00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48740052f0 0x7f48900b4000 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4880000fc0 tx=0x7f488000e770 comp rx=0 tx=0).stop 2024-09-10T22:06:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f4896e00640 1 -- 172.21.15.66:0/2151031900 shutdown_connections 2024-09-10T22:06:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f4896e00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f486c0731e0 0x7f486c0756a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f4896e00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4890153710 0x7f4890073520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f4896e00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48900c65c0 0x7f48900c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f4896e00640 1 --2- 172.21.15.66:0/2151031900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48740052f0 0x7f48900b4000 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f4896e00640 1 -- 172.21.15.66:0/2151031900 >> 172.21.15.66:0/2151031900 conn(0x7f48900cfa30 msgr2=0x7f48900ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f4896e00640 1 -- 172.21.15.66:0/2151031900 shutdown_connections 2024-09-10T22:06:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.066+0000 7f4896e00640 1 -- 172.21.15.66:0/2151031900 wait complete. 2024-09-10T22:06:48.098 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T22:06:48.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 -- 172.21.15.66:0/4210260349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd4159900 msgr2=0x7f7bd40ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:48.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 --2- 172.21.15.66:0/4210260349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd4159900 0x7f7bd40ca080 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009900 tx=0x7f7bc401c850 comp rx=0 tx=0).stop 2024-09-10T22:06:48.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 -- 172.21.15.66:0/4210260349 shutdown_connections 2024-09-10T22:06:48.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 --2- 172.21.15.66:0/4210260349 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bd40aee20 0x7f7bd40cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 --2- 172.21.15.66:0/4210260349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd4159900 0x7f7bd40ca080 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 -- 172.21.15.66:0/4210260349 >> 172.21.15.66:0/4210260349 conn(0x7f7bd40cfa30 msgr2=0x7f7bd40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:48.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 -- 172.21.15.66:0/4210260349 shutdown_connections 2024-09-10T22:06:48.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 -- 172.21.15.66:0/4210260349 wait complete. 2024-09-10T22:06:48.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 Processor -- start 2024-09-10T22:06:48.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 -- start start 2024-09-10T22:06:48.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd40aee20 0x7f7bd4159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:48.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bd4159900 0x7f7bd4154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:48.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd415cc80 0x7f7bd40c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:48.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd4168f70 con 0x7f7bd40aee20 2024-09-10T22:06:48.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd4168df0 con 0x7f7bd415cc80 2024-09-10T22:06:48.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bdac00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd41690f0 con 0x7f7bd4159900 2024-09-10T22:06:48.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bd3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd40aee20 0x7f7bd4159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:48.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bd3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd40aee20 0x7f7bd4159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40238/0 (socket says 172.21.15.66:40238) 2024-09-10T22:06:48.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bd3e00640 1 -- 172.21.15.66:0/133757811 learned_addr learned my addr 172.21.15.66:0/133757811 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bd8c00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd415cc80 0x7f7bd40c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bcbe00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bd4159900 0x7f7bd4154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bd3e00640 1 -- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bd4159900 msgr2=0x7f7bd4154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bd3e00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bd4159900 0x7f7bd4154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bd3e00640 1 -- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd415cc80 msgr2=0x7f7bd40c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bd3e00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd415cc80 0x7f7bd40c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.246+0000 7f7bd3e00640 1 -- 172.21.15.66:0/133757811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd4151470 con 0x7f7bd40aee20 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.250+0000 7f7bd3e00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd40aee20 0x7f7bd4159e70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009850 tx=0x7f7bc40a65b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.250+0000 7f7bd0c00640 1 -- 172.21.15.66:0/133757811 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40ac070 con 0x7f7bd40aee20 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.250+0000 7f7bd0c00640 1 -- 172.21.15.66:0/133757811 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bc4004320 con 0x7f7bd40aee20 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.250+0000 7f7bd0c00640 1 -- 172.21.15.66:0/133757811 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40044f0 con 0x7f7bd40aee20 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.250+0000 7f7bdac00640 1 -- 172.21.15.66:0/133757811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd4154940 con 0x7f7bd40aee20 2024-09-10T22:06:48.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.250+0000 7f7bdac00640 1 -- 172.21.15.66:0/133757811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd415a850 con 0x7f7bd40aee20 2024-09-10T22:06:48.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.250+0000 7f7bdac00640 1 -- 172.21.15.66:0/133757811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd40cd930 con 0x7f7bd40aee20 2024-09-10T22:06:48.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.250+0000 7f7bd0c00640 1 -- 172.21.15.66:0/133757811 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f7bc40b9070 con 0x7f7bd40aee20 2024-09-10T22:06:48.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.250+0000 7f7bd0c00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7bb00731b0 0x7f7bb0075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:48.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.250+0000 7f7bd0c00640 1 -- 172.21.15.66:0/133757811 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4971+0+0 (secure 0 0 0) 0x7f7bc412c090 con 0x7f7bd40aee20 2024-09-10T22:06:48.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.250+0000 7f7bcbe00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7bb00731b0 0x7f7bb0075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:48.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.254+0000 7f7bcbe00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7bb00731b0 0x7f7bb0075670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4070110 tx=0x7f7bbc008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:48.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.254+0000 7f7bd0c00640 1 -- 172.21.15.66:0/133757811 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7bd4151470 con 0x7f7bd40aee20 2024-09-10T22:06:48.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.430+0000 7f7bdac00640 1 -- 172.21.15.66:0/133757811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f7bd406ea00 con 0x7f7bd40aee20 2024-09-10T22:06:48.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.430+0000 7f7bd0c00640 1 -- 172.21.15.66:0/133757811 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v36) ==== 59+0+1998 (secure 0 0 0) 0x7f7bc40164d0 con 0x7f7bd40aee20 2024-09-10T22:06:48.436 INFO:teuthology.orchestra.run.smithi066.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T22:06:48.436 INFO:teuthology.orchestra.run.smithi066.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T22:06:48.436 INFO:teuthology.orchestra.run.smithi066.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T22:06:48.436 INFO:teuthology.orchestra.run.smithi066.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T22:06:48.436 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4673.907365 2024-09-10T22:06:48.436 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4537.831230 2024-09-10T22:06:48.436 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 5706.341617 2024-09-10T22:06:48.436 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4484.013907 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3862.285837 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3776.439661 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3820.270368 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3863.602758 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds 20/20 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_ms 2/2 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_frag true 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_scatterstat true 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_verify_scatter true 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T22:06:48.437 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T22:06:48.438 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T22:06:48.438 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced client_mount_timeout 600 2024-09-10T22:06:48.438 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_client 20/20 2024-09-10T22:06:48.438 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_ms 1/1 2024-09-10T22:06:48.438 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T22:06:48.438 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T22:06:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.434+0000 7f7bdac00640 1 -- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7bb00731b0 msgr2=0x7f7bb0075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.434+0000 7f7bdac00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7bb00731b0 0x7f7bb0075670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4070110 tx=0x7f7bbc008040 comp rx=0 tx=0).stop 2024-09-10T22:06:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.434+0000 7f7bdac00640 1 -- 172.21.15.66:0/133757811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd40aee20 msgr2=0x7f7bd4159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.434+0000 7f7bdac00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd40aee20 0x7f7bd4159e70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009850 tx=0x7f7bc40a65b0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.434+0000 7f7bdac00640 1 -- 172.21.15.66:0/133757811 shutdown_connections 2024-09-10T22:06:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.434+0000 7f7bdac00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7bb00731b0 0x7f7bb0075670 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.434+0000 7f7bdac00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd415cc80 0x7f7bd40c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.434+0000 7f7bdac00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bd4159900 0x7f7bd4154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.434+0000 7f7bdac00640 1 --2- 172.21.15.66:0/133757811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd40aee20 0x7f7bd4159e70 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.434+0000 7f7bdac00640 1 -- 172.21.15.66:0/133757811 >> 172.21.15.66:0/133757811 conn(0x7f7bd40cfa30 msgr2=0x7f7bd4154f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.434+0000 7f7bdac00640 1 -- 172.21.15.66:0/133757811 shutdown_connections 2024-09-10T22:06:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.434+0000 7f7bdac00640 1 -- 172.21.15.66:0/133757811 wait complete. 2024-09-10T22:06:48.458 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:06:48.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.602+0000 7f2a62600640 1 -- 172.21.15.66:0/3485548331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c06eea0 msgr2=0x7f2a5c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:48.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.602+0000 7f2a62600640 1 --2- 172.21.15.66:0/3485548331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c06eea0 0x7f2a5c06e4a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c06d650 tx=0x7f2a5401f580 comp rx=0 tx=0).stop 2024-09-10T22:06:48.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.602+0000 7f2a62600640 1 -- 172.21.15.66:0/3485548331 shutdown_connections 2024-09-10T22:06:48.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.602+0000 7f2a62600640 1 --2- 172.21.15.66:0/3485548331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c06eea0 0x7f2a5c06e4a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.602+0000 7f2a62600640 1 --2- 172.21.15.66:0/3485548331 >> v2:172.21.15.94:3300/0 conn(0x7f2a5c0713b0 0x7f2a5c0b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.602+0000 7f2a62600640 1 -- 172.21.15.66:0/3485548331 >> 172.21.15.66:0/3485548331 conn(0x7f2a5c0d1d00 msgr2=0x7f2a5c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:48.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.602+0000 7f2a62600640 1 -- 172.21.15.66:0/3485548331 shutdown_connections 2024-09-10T22:06:48.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.602+0000 7f2a62600640 1 -- 172.21.15.66:0/3485548331 wait complete. 2024-09-10T22:06:48.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a62600640 1 Processor -- start 2024-09-10T22:06:48.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a62600640 1 -- start start 2024-09-10T22:06:48.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a62600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a5c06eea0 0x7f2a5c159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:48.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a62600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c070fd0 0x7f2a5c0b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:48.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a62600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a5c0713b0 0x7f2a5c1f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:48.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a62600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c0d03f0 con 0x7f2a5c070fd0 2024-09-10T22:06:48.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a62600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c0d0270 con 0x7f2a5c0713b0 2024-09-10T22:06:48.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a62600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5c0d0570 con 0x7f2a5c06eea0 2024-09-10T22:06:48.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a53e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c070fd0 0x7f2a5c0b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:48.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a53e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c070fd0 0x7f2a5c0b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40256/0 (socket says 172.21.15.66:40256) 2024-09-10T22:06:48.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a53e00640 1 -- 172.21.15.66:0/390753172 learned_addr learned my addr 172.21.15.66:0/390753172 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a5be00640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a5c0713b0 0x7f2a5c1f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a5b400640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a5c06eea0 0x7f2a5c159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a53e00640 1 -- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a5c06eea0 msgr2=0x7f2a5c159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a53e00640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a5c06eea0 0x7f2a5c159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a53e00640 1 -- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a5c0713b0 msgr2=0x7f2a5c1f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a53e00640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a5c0713b0 0x7f2a5c1f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a53e00640 1 -- 172.21.15.66:0/390753172 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c15c0d0 con 0x7f2a5c070fd0 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a5b400640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a5c06eea0 0x7f2a5c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a53e00640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c070fd0 0x7f2a5c0b3a60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400e7c0 tx=0x7f2a4400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a53400640 1 -- 172.21.15.66:0/390753172 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400ccc0 con 0x7f2a5c070fd0 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a53400640 1 -- 172.21.15.66:0/390753172 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a4400ce80 con 0x7f2a5c070fd0 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a53400640 1 -- 172.21.15.66:0/390753172 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44005280 con 0x7f2a5c070fd0 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a62600640 1 -- 172.21.15.66:0/390753172 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c15c3b0 con 0x7f2a5c070fd0 2024-09-10T22:06:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a62600640 1 -- 172.21.15.66:0/390753172 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c0b3540 con 0x7f2a5c070fd0 2024-09-10T22:06:48.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.606+0000 7f2a62600640 1 -- 172.21.15.66:0/390753172 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5c1596f0 con 0x7f2a5c070fd0 2024-09-10T22:06:48.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.610+0000 7f2a53400640 1 -- 172.21.15.66:0/390753172 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f2a44005420 con 0x7f2a5c070fd0 2024-09-10T22:06:48.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.610+0000 7f2a53400640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a380731b0 0x7f2a38075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:48.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.610+0000 7f2a53400640 1 -- 172.21.15.66:0/390753172 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4971+0+0 (secure 0 0 0) 0x7f2a440925b0 con 0x7f2a5c070fd0 2024-09-10T22:06:48.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.610+0000 7f2a5b400640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a380731b0 0x7f2a38075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:48.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.610+0000 7f2a5b400640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a380731b0 0x7f2a38075670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c001750 tx=0x7f2a4c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:48.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.614+0000 7f2a53400640 1 -- 172.21.15.66:0/390753172 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2a4405f6e0 con 0x7f2a5c070fd0 2024-09-10T22:06:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.786+0000 7f2a62600640 1 -- 172.21.15.66:0/390753172 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2a5c0b1180 con 0x7f2a5c070fd0 2024-09-10T22:06:48.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.790+0000 7f2a53400640 1 -- 172.21.15.66:0/390753172 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7f2a4405efe0 con 0x7f2a5c070fd0 2024-09-10T22:06:48.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.794+0000 7f2a62600640 1 -- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a380731b0 msgr2=0x7f2a38075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:48.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.794+0000 7f2a62600640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a380731b0 0x7f2a38075670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c001750 tx=0x7f2a4c009040 comp rx=0 tx=0).stop 2024-09-10T22:06:48.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.794+0000 7f2a62600640 1 -- 172.21.15.66:0/390753172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c070fd0 msgr2=0x7f2a5c0b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:48.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.794+0000 7f2a62600640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c070fd0 0x7f2a5c0b3a60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400e7c0 tx=0x7f2a4400ec80 comp rx=0 tx=0).stop 2024-09-10T22:06:48.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.794+0000 7f2a62600640 1 -- 172.21.15.66:0/390753172 shutdown_connections 2024-09-10T22:06:48.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.794+0000 7f2a62600640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a380731b0 0x7f2a38075670 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.794+0000 7f2a62600640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a5c0713b0 0x7f2a5c1f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.794+0000 7f2a62600640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c070fd0 0x7f2a5c0b3a60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.794+0000 7f2a62600640 1 --2- 172.21.15.66:0/390753172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a5c06eea0 0x7f2a5c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:48.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.794+0000 7f2a62600640 1 -- 172.21.15.66:0/390753172 >> 172.21.15.66:0/390753172 conn(0x7f2a5c0d1d00 msgr2=0x7f2a5c15fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:48.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.794+0000 7f2a62600640 1 -- 172.21.15.66:0/390753172 shutdown_connections 2024-09-10T22:06:48.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:48.794+0000 7f2a62600640 1 -- 172.21.15.66:0/390753172 wait complete. 2024-09-10T22:06:48.819 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:06:48.819 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:06:48.916 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:06:49.030 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:06:49.030 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:06:49.030 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:49.039 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:06:49.039 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:06:49.039 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:49.048 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:06:49.048 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:06:49.049 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:49.093 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:06:49.093 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:06:49.094 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:06:49.104 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:06:49.104 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T22:06:49.104 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:06:49.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.278+0000 7feafa800640 1 -- 172.21.15.66:0/1647679284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf40ce6e0 msgr2=0x7feaf40ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:49.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.278+0000 7feafa800640 1 --2- 172.21.15.66:0/1647679284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf40ce6e0 0x7feaf40ceac0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7feaf40729d0 tx=0x7feae801db60 comp rx=0 tx=0).stop 2024-09-10T22:06:49.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.278+0000 7feafa800640 1 -- 172.21.15.66:0/1647679284 shutdown_connections 2024-09-10T22:06:49.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.278+0000 7feafa800640 1 --2- 172.21.15.66:0/1647679284 >> v2:172.21.15.94:3300/0 conn(0x7feaf406e780 0x7feaf415e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.278+0000 7feafa800640 1 --2- 172.21.15.66:0/1647679284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf40ce6e0 0x7feaf40ceac0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.278+0000 7feafa800640 1 -- 172.21.15.66:0/1647679284 >> 172.21.15.66:0/1647679284 conn(0x7feaf40cfcf0 msgr2=0x7feaf40d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:49.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.278+0000 7feafa800640 1 -- 172.21.15.66:0/1647679284 shutdown_connections 2024-09-10T22:06:49.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.278+0000 7feafa800640 1 -- 172.21.15.66:0/1647679284 wait complete. 2024-09-10T22:06:49.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feafa800640 1 Processor -- start 2024-09-10T22:06:49.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feafa800640 1 -- start start 2024-09-10T22:06:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feafa800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf406e780 0x7feaf41453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feafa800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feaf40ce6e0 0x7feaf41472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feafa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf40d0a20 0x7feaf4147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feafa800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feaf40b43b0 con 0x7feaf40d0a20 2024-09-10T22:06:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feafa800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feaf40b4230 con 0x7feaf406e780 2024-09-10T22:06:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feafa800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7feaf40b4530 con 0x7feaf40ce6e0 2024-09-10T22:06:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf40d0a20 0x7feaf4147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf40d0a20 0x7feaf4147810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40274/0 (socket says 172.21.15.66:40274) 2024-09-10T22:06:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf9e00640 1 -- 172.21.15.66:0/1561799840 learned_addr learned my addr 172.21.15.66:0/1561799840 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf9400640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf406e780 0x7feaf41453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf8a00640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feaf40ce6e0 0x7feaf41472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf9e00640 1 -- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feaf40ce6e0 msgr2=0x7feaf41472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf9e00640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feaf40ce6e0 0x7feaf41472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf9e00640 1 -- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf406e780 msgr2=0x7feaf41453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf9e00640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf406e780 0x7feaf41453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf9e00640 1 -- 172.21.15.66:0/1561799840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf41547c0 con 0x7feaf40d0a20 2024-09-10T22:06:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf9e00640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf40d0a20 0x7feaf4147810 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7feaec00ea20 tx=0x7feaec00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf1600640 1 -- 172.21.15.66:0/1561799840 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec00cde0 con 0x7feaf40d0a20 2024-09-10T22:06:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf1600640 1 -- 172.21.15.66:0/1561799840 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feaec004c40 con 0x7feaf40d0a20 2024-09-10T22:06:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feaf1600640 1 -- 172.21.15.66:0/1561799840 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec005280 con 0x7feaf40d0a20 2024-09-10T22:06:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feafa800640 1 -- 172.21.15.66:0/1561799840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf415b3e0 con 0x7feaf40d0a20 2024-09-10T22:06:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feafa800640 1 -- 172.21.15.66:0/1561799840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf4155040 con 0x7feaf40d0a20 2024-09-10T22:06:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.282+0000 7feafa800640 1 -- 172.21.15.66:0/1561799840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf40ceac0 con 0x7feaf40d0a20 2024-09-10T22:06:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.286+0000 7feaf1600640 1 -- 172.21.15.66:0/1561799840 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7feaec00f040 con 0x7feaf40d0a20 2024-09-10T22:06:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.286+0000 7feaf1600640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fead40731b0 0x7fead4075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.286+0000 7feaf1600640 1 -- 172.21.15.66:0/1561799840 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4971+0+0 (secure 0 0 0) 0x7feaec0924c0 con 0x7feaf40d0a20 2024-09-10T22:06:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.286+0000 7feaf9400640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fead40731b0 0x7fead4075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.286+0000 7feaf9400640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fead40731b0 0x7fead4075670 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7feae80b0000 tx=0x7feae801c3a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.290+0000 7feaf1600640 1 -- 172.21.15.66:0/1561799840 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feaec098030 con 0x7feaf40d0a20 2024-09-10T22:06:49.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.498+0000 7feafa800640 1 -- 172.21.15.66:0/1561799840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feaf4155210 con 0x7feaf40d0a20 2024-09-10T22:06:49.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.498+0000 7feaf1600640 1 -- 172.21.15.66:0/1561799840 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3908 (secure 0 0 0) 0x7feaec016030 con 0x7feaf40d0a20 2024-09-10T22:06:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:49.506 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:06:43:040798+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5414,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/135515157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":135515157},{"type":"v1","addr":"172.21.15.94:6837","nonce":135515157}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1614138608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1614138608},{"type":"v1","addr":"172.21.15.94:6836","nonce":1614138608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/1060280836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1060280836},{"type":"v1","addr":"172.21.15.66:6833","nonce":1060280836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6028,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1710393751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1710393751},{"type":"v1","addr":"172.21.15.66:6835","nonce":1710393751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-09-10T22:06:49.506 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:06:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.502+0000 7feafa800640 1 -- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fead40731b0 msgr2=0x7fead4075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.502+0000 7feafa800640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fead40731b0 0x7fead4075670 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7feae80b0000 tx=0x7feae801c3a0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.502+0000 7feafa800640 1 -- 172.21.15.66:0/1561799840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf40d0a20 msgr2=0x7feaf4147810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.502+0000 7feafa800640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf40d0a20 0x7feaf4147810 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7feaec00ea20 tx=0x7feaec00eee0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.502+0000 7feafa800640 1 -- 172.21.15.66:0/1561799840 shutdown_connections 2024-09-10T22:06:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.502+0000 7feafa800640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fead40731b0 0x7fead4075670 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.502+0000 7feafa800640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf40d0a20 0x7feaf4147810 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.502+0000 7feafa800640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feaf40ce6e0 0x7feaf41472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.502+0000 7feafa800640 1 --2- 172.21.15.66:0/1561799840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf406e780 0x7feaf41453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.502+0000 7feafa800640 1 -- 172.21.15.66:0/1561799840 >> 172.21.15.66:0/1561799840 conn(0x7feaf40cfcf0 msgr2=0x7feaf4070740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.502+0000 7feafa800640 1 -- 172.21.15.66:0/1561799840 shutdown_connections 2024-09-10T22:06:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.502+0000 7feafa800640 1 -- 172.21.15.66:0/1561799840 wait complete. 2024-09-10T22:06:49.529 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:06:49.529 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:06:49.529 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:06:49.529 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:06:49.529 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:06:49.530 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:06:49.530 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:06:49.530 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:06:49.530 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:06:49.530 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:06:49.531 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:06:49.531 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:06:49.531 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:06:49.531 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:06:49.532 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:06:49.532 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:06:49.541 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:06:49.541 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:06:49.542 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:06:49.542 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:06:49.546 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:06:49.546 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:06:49.547 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:06:49.547 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:06:49.588 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:06:49.589 DEBUG:teuthology.parallel:result is None 2024-09-10T22:06:49.590 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:06:49.590 DEBUG:teuthology.parallel:result is None 2024-09-10T22:06:49.594 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:06:49.594 DEBUG:teuthology.parallel:result is None 2024-09-10T22:06:49.596 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:06:49.596 DEBUG:teuthology.parallel:result is None 2024-09-10T22:06:49.596 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T22:06:49.675 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:06:49.675 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:06:49.686 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:06:49.686 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:06:49.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.742+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2973702779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ae00aeec0 msgr2=0x7f3ae015a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:49.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.742+0000 7f3ae4a00640 1 --2- 172.21.15.66:0/2973702779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ae00aeec0 0x7f3ae015a310 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009870 tx=0x7f3ac801c7d0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.742+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2973702779 shutdown_connections 2024-09-10T22:06:49.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.742+0000 7f3ae4a00640 1 --2- 172.21.15.66:0/2973702779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ae00aeec0 0x7f3ae015a310 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.742+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2973702779 >> 172.21.15.66:0/2973702779 conn(0x7f3ae00cfa30 msgr2=0x7f3ae00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:49.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.742+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2973702779 shutdown_connections 2024-09-10T22:06:49.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.742+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2973702779 wait complete. 2024-09-10T22:06:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ae4a00640 1 Processor -- start 2024-09-10T22:06:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ae4a00640 1 -- start start 2024-09-10T22:06:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ae4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae00ae8e0 0x7f3ae014f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ae4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ae00aeec0 0x7f3ae01f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ae4a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ae0153cc0 0x7f3ae014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ae4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0168d30 con 0x7f3ae00ae8e0 2024-09-10T22:06:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ae4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae006ed40 con 0x7f3ae00aeec0 2024-09-10T22:06:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ae4a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae0168eb0 con 0x7f3ae0153cc0 2024-09-10T22:06:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3adde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae00ae8e0 0x7f3ae014f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3adde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae00ae8e0 0x7f3ae014f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40328/0 (socket says 172.21.15.66:40328) 2024-09-10T22:06:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3adde00640 1 -- 172.21.15.66:0/2371854576 learned_addr learned my addr 172.21.15.66:0/2371854576 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ad7e00640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ae00aeec0 0x7f3ae01f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ade800640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ae0153cc0 0x7f3ae014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3adde00640 1 -- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ae0153cc0 msgr2=0x7f3ae014ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3adde00640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ae0153cc0 0x7f3ae014ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3adde00640 1 -- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ae00aeec0 msgr2=0x7f3ae01f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3adde00640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ae00aeec0 0x7f3ae01f8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3adde00640 1 -- 172.21.15.66:0/2371854576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae00c6c10 con 0x7f3ae00ae8e0 2024-09-10T22:06:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ade800640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ae0153cc0 0x7f3ae014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3adde00640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae00ae8e0 0x7f3ae014f1e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000c430 tx=0x7f3ad000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:49.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ad6000640 1 -- 172.21.15.66:0/2371854576 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0011790 con 0x7f3ae00ae8e0 2024-09-10T22:06:49.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ad6000640 1 -- 172.21.15.66:0/2371854576 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ad0017070 con 0x7f3ae00ae8e0 2024-09-10T22:06:49.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ad6000640 1 -- 172.21.15.66:0/2371854576 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0011cc0 con 0x7f3ae00ae8e0 2024-09-10T22:06:49.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2371854576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae00c6ef0 con 0x7f3ae00ae8e0 2024-09-10T22:06:49.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2371854576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae00c8bb0 con 0x7f3ae00ae8e0 2024-09-10T22:06:49.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.746+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2371854576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae00c7130 con 0x7f3ae00ae8e0 2024-09-10T22:06:49.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.750+0000 7f3ad6000640 1 -- 172.21.15.66:0/2371854576 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f3ad001c460 con 0x7f3ae00ae8e0 2024-09-10T22:06:49.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.750+0000 7f3ad6000640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3abc0731e0 0x7f3abc0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:49.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.750+0000 7f3ad6000640 1 -- 172.21.15.66:0/2371854576 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4971+0+0 (secure 0 0 0) 0x7f3ad0091c20 con 0x7f3ae00ae8e0 2024-09-10T22:06:49.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.750+0000 7f3ad7e00640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3abc0731e0 0x7f3abc0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:49.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.750+0000 7f3ad7e00640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3abc0731e0 0x7f3abc0756a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80af000 tx=0x7f3ac80b2040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:49.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.754+0000 7f3ad6000640 1 -- 172.21.15.66:0/2371854576 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3ad005ea70 con 0x7f3ae00ae8e0 2024-09-10T22:06:49.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:49.942+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2371854576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3ae0154180 con 0x7f3ae00ae8e0 2024-09-10T22:06:50.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.082+0000 7f3ad6000640 1 -- 172.21.15.66:0/2371854576 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v65) ==== 97+0+0 (secure 0 0 0) 0x7f3ad005e370 con 0x7f3ae00ae8e0 2024-09-10T22:06:50.088 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-09-10T22:06:50.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.086+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3abc0731e0 msgr2=0x7f3abc0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:50.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.086+0000 7f3ae4a00640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3abc0731e0 0x7f3abc0756a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80af000 tx=0x7f3ac80b2040 comp rx=0 tx=0).stop 2024-09-10T22:06:50.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.086+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2371854576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae00ae8e0 msgr2=0x7f3ae014f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:50.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.086+0000 7f3ae4a00640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae00ae8e0 0x7f3ae014f1e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000c430 tx=0x7f3ad000c8f0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.086+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2371854576 shutdown_connections 2024-09-10T22:06:50.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.086+0000 7f3ae4a00640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3abc0731e0 0x7f3abc0756a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.086+0000 7f3ae4a00640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ae0153cc0 0x7f3ae014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.086+0000 7f3ae4a00640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ae00aeec0 0x7f3ae01f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.086+0000 7f3ae4a00640 1 --2- 172.21.15.66:0/2371854576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae00ae8e0 0x7f3ae014f1e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.086+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2371854576 >> 172.21.15.66:0/2371854576 conn(0x7f3ae00cfa30 msgr2=0x7f3ae00affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:50.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.086+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2371854576 shutdown_connections 2024-09-10T22:06:50.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.086+0000 7f3ae4a00640 1 -- 172.21.15.66:0/2371854576 wait complete. 2024-09-10T22:06:50.115 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T22:06:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.258+0000 7fa844e00640 1 -- 172.21.15.66:0/1441840100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa824003c40 msgr2=0x7fa824004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.258+0000 7fa844e00640 1 --2- 172.21.15.66:0/1441840100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa824003c40 0x7fa824004040 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa830001860 tx=0x7fa83001c8c0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.258+0000 7fa844e00640 1 -- 172.21.15.66:0/1441840100 shutdown_connections 2024-09-10T22:06:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.258+0000 7fa844e00640 1 --2- 172.21.15.66:0/1441840100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa824003c40 0x7fa824004040 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.258+0000 7fa844e00640 1 -- 172.21.15.66:0/1441840100 >> 172.21.15.66:0/1441840100 conn(0x7fa8400d1c60 msgr2=0x7fa8400d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.258+0000 7fa844e00640 1 -- 172.21.15.66:0/1441840100 shutdown_connections 2024-09-10T22:06:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.258+0000 7fa844e00640 1 -- 172.21.15.66:0/1441840100 wait complete. 2024-09-10T22:06:50.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa844e00640 1 Processor -- start 2024-09-10T22:06:50.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa844e00640 1 -- start start 2024-09-10T22:06:50.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa844e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa824003c40 0x7fa8400cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:50.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa844e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa840155190 0x7fa8400c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:50.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa844e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa840157b70 0x7fa84015b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:50.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa844e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa8400cdbb0 con 0x7fa840157b70 2024-09-10T22:06:50.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa844e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8400d0930 con 0x7fa824003c40 2024-09-10T22:06:50.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa844e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa8400cdd30 con 0x7fa840155190 2024-09-10T22:06:50.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa83e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa840157b70 0x7fa84015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa83e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa840157b70 0x7fa84015b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40344/0 (socket says 172.21.15.66:40344) 2024-09-10T22:06:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa83e800640 1 -- 172.21.15.66:0/876913120 learned_addr learned my addr 172.21.15.66:0/876913120 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa837e00640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa840155190 0x7fa8400c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa83de00640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa824003c40 0x7fa8400cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa83e800640 1 -- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa840155190 msgr2=0x7fa8400c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa83e800640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa840155190 0x7fa8400c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa83e800640 1 -- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa824003c40 msgr2=0x7fa8400cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa83e800640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa824003c40 0x7fa8400cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa83e800640 1 -- 172.21.15.66:0/876913120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8400c8f80 con 0x7fa840157b70 2024-09-10T22:06:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa83e800640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa840157b70 0x7fa84015b600 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa8400cf030 tx=0x7fa838006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa836000640 1 -- 172.21.15.66:0/876913120 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838015070 con 0x7fa840157b70 2024-09-10T22:06:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa836000640 1 -- 172.21.15.66:0/876913120 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa838011940 con 0x7fa840157b70 2024-09-10T22:06:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa836000640 1 -- 172.21.15.66:0/876913120 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838011d90 con 0x7fa840157b70 2024-09-10T22:06:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa844e00640 1 -- 172.21.15.66:0/876913120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8400c9260 con 0x7fa840157b70 2024-09-10T22:06:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa844e00640 1 -- 172.21.15.66:0/876913120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8400c6b60 con 0x7fa840157b70 2024-09-10T22:06:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.262+0000 7fa844e00640 1 -- 172.21.15.66:0/876913120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8400ce290 con 0x7fa840157b70 2024-09-10T22:06:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.266+0000 7fa836000640 1 -- 172.21.15.66:0/876913120 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fa838010420 con 0x7fa840157b70 2024-09-10T22:06:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.266+0000 7fa836000640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa81c0731b0 0x7fa81c075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.266+0000 7fa836000640 1 -- 172.21.15.66:0/876913120 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4885+0+0 (secure 0 0 0) 0x7fa838092b10 con 0x7fa840157b70 2024-09-10T22:06:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.266+0000 7fa83de00640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa81c0731b0 0x7fa81c075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:50.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.270+0000 7fa83de00640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa81c0731b0 0x7fa81c075670 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa830001830 tx=0x7fa830019700 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.270+0000 7fa836000640 1 -- 172.21.15.66:0/876913120 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa83805f3d0 con 0x7fa840157b70 2024-09-10T22:06:50.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 -- 172.21.15.66:0/876913120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa8401eda10 con 0x7fa840157b70 2024-09-10T22:06:50.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa836000640 1 -- 172.21.15.66:0/876913120 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) ==== 80+0+5304 (secure 0 0 0) 0x7fa838063e00 con 0x7fa840157b70 2024-09-10T22:06:50.511 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:50.511 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:06:50.511 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-09-10T22:06:50.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.511 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-09-10T22:06:50.511 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmxQD7CBAAttZP0KtdjTI7QWbD7CVinQ==", 2024-09-10T22:06:50.511 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.511 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:06:50.511 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:06:50.511 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:06:50.511 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmI7ufFRAANelhvqylKyfGPVfBuY40kg==", 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQnVoEBAAvsZ/TSOOP/JgJBr/aHwI/g==", 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmRQkTHRAAHPIdwqtkkyh31D7IO2vxPA==", 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.513 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBCweBmDJkxJBAAVSQNiMAQl5bR3RUwVOuGYw==", 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBFweBmwcdsOxAAPPQyzktcfiF8pJNP8uAjwg==", 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBJweBmXQhZExAAnEf8mjGX6nnYwWygKjo/2A==", 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.515 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBNweBmlauHAxAA/k9xRqbHZsqJq9OYbJKfNQ==", 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBRweBmHnFDJRAAgF1ZAPvmoNzeUftZQyL0eg==", 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBUweBm/hXpORAA6jpOI/I+QyB2mZsEoAWF0w==", 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:06:50.517 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBYweBmYu2MExAAPD9WcxrBLzwwJQm19z7YcQ==", 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbweBmXfJ9KRAA/AbWlli+Qfg7gy0SblJ7fg==", 2024-09-10T22:06:50.518 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw==", 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.519 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmbS6IKhAAaEmK9irxMipALgkAct/+OQ==", 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQ47KJhAAI50ZaSQIhH6FxRM48JZICA==", 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:06:50.520 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmUcLCMBAAdf4tfcpVaZoaKjS1UzrxhA==", 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-09-10T22:06:50.521 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA+weBmPIAzKBAAkyVV875PNhBmEoKsrAkg8w==", 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmLiy1BBAADnraML8IUet7Sw0j83TUVQ==", 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.522 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:06:50.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.523 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-09-10T22:06:50.523 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmItWpABAAlJ+6dFjvtBV3g+jYmUh0tQ==", 2024-09-10T22:06:50.523 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:06:50.523 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:06:50.523 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:06:50.523 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:06:50.523 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.523 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.523 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T22:06:50.523 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:06:50.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 -- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa81c0731b0 msgr2=0x7fa81c075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:50.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa81c0731b0 0x7fa81c075670 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa830001830 tx=0x7fa830019700 comp rx=0 tx=0).stop 2024-09-10T22:06:50.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 -- 172.21.15.66:0/876913120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa840157b70 msgr2=0x7fa84015b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:50.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa840157b70 0x7fa84015b600 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa8400cf030 tx=0x7fa838006430 comp rx=0 tx=0).stop 2024-09-10T22:06:50.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 -- 172.21.15.66:0/876913120 shutdown_connections 2024-09-10T22:06:50.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa81c0731b0 0x7fa81c075670 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa840157b70 0x7fa84015b600 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa840155190 0x7fa8400c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 --2- 172.21.15.66:0/876913120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa824003c40 0x7fa8400cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 -- 172.21.15.66:0/876913120 >> 172.21.15.66:0/876913120 conn(0x7fa8400d1c60 msgr2=0x7fa8400b4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:50.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 -- 172.21.15.66:0/876913120 shutdown_connections 2024-09-10T22:06:50.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.506+0000 7fa844e00640 1 -- 172.21.15.66:0/876913120 wait complete. 2024-09-10T22:06:50.533 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:06:50.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.674+0000 7f5e80a00640 1 -- 172.21.15.66:0/3999541387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c0d0550 msgr2=0x7f5e7c0ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:50.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.674+0000 7f5e80a00640 1 --2- 172.21.15.66:0/3999541387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c0d0550 0x7f5e7c0ca1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009900 tx=0x7f5e6401c850 comp rx=0 tx=0).stop 2024-09-10T22:06:50.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.674+0000 7f5e80a00640 1 -- 172.21.15.66:0/3999541387 shutdown_connections 2024-09-10T22:06:50.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.674+0000 7f5e80a00640 1 --2- 172.21.15.66:0/3999541387 >> v2:172.21.15.94:3300/0 conn(0x7f5e7c0c4fc0 0x7f5e7c0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.674+0000 7f5e80a00640 1 --2- 172.21.15.66:0/3999541387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c0d0550 0x7f5e7c0ca1a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.674+0000 7f5e80a00640 1 -- 172.21.15.66:0/3999541387 >> 172.21.15.66:0/3999541387 conn(0x7f5e7c0cfa30 msgr2=0x7f5e7c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:50.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.674+0000 7f5e80a00640 1 -- 172.21.15.66:0/3999541387 shutdown_connections 2024-09-10T22:06:50.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.674+0000 7f5e80a00640 1 -- 172.21.15.66:0/3999541387 wait complete. 2024-09-10T22:06:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.674+0000 7f5e80a00640 1 Processor -- start 2024-09-10T22:06:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.674+0000 7f5e80a00640 1 -- start start 2024-09-10T22:06:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e80a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c0c4fc0 0x7f5e7c151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e80a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c15a310 0x7f5e7c14f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e80a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c0c7680 0x7f5e7c157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e80a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c167480 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e80a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c167300 con 0x7f5e7c15a310 2024-09-10T22:06:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e80a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7c167600 con 0x7f5e7c0c7680 2024-09-10T22:06:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e79e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c0c4fc0 0x7f5e7c151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e79e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c0c4fc0 0x7f5e7c151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40350/0 (socket says 172.21.15.66:40350) 2024-09-10T22:06:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e79e00640 1 -- 172.21.15.66:0/2195641500 learned_addr learned my addr 172.21.15.66:0/2195641500 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e7a800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c0c7680 0x7f5e7c157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e73e00640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c15a310 0x7f5e7c14f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:50.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e79e00640 1 -- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c0c7680 msgr2=0x7f5e7c157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:50.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e79e00640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c0c7680 0x7f5e7c157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e79e00640 1 -- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c15a310 msgr2=0x7f5e7c14f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:50.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e79e00640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c15a310 0x7f5e7c14f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e79e00640 1 -- 172.21.15.66:0/2195641500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c0c7b30 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e79e00640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c0c4fc0 0x7f5e7c151820 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00c430 tx=0x7f5e6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:50.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e72000640 1 -- 172.21.15.66:0/2195641500 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c017070 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e72000640 1 -- 172.21.15.66:0/2195641500 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e6c0053e0 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e72000640 1 -- 172.21.15.66:0/2195641500 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c003860 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e80a00640 1 -- 172.21.15.66:0/2195641500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7c1576d0 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e80a00640 1 -- 172.21.15.66:0/2195641500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7c155580 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e80a00640 1 -- 172.21.15.66:0/2195641500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7c0c48b0 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.678+0000 7f5e72000640 1 -- 172.21.15.66:0/2195641500 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f5e6c003a00 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.682+0000 7f5e72000640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5e580731b0 0x7f5e58075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:50.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.682+0000 7f5e72000640 1 -- 172.21.15.66:0/2195641500 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4885+0+0 (secure 0 0 0) 0x7f5e6c0914a0 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.682+0000 7f5e73e00640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5e580731b0 0x7f5e58075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:50.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.682+0000 7f5e73e00640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5e580731b0 0x7f5e58075670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5e640af000 tx=0x7f5e6401b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:50.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.686+0000 7f5e72000640 1 -- 172.21.15.66:0/2195641500 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5e6c05de40 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.858+0000 7f5e80a00640 1 -- 172.21.15.66:0/2195641500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5e7c0ca900 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e72000640 1 -- 172.21.15.66:0/2195641500 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) ==== 84+0+65 (secure 0 0 0) 0x7f5e6c062870 con 0x7f5e7c0c4fc0 2024-09-10T22:06:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:50.866 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:06:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:06:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:06:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:06:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:06:50.866 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:06:50.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e80a00640 1 -- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5e580731b0 msgr2=0x7f5e58075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:50.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e80a00640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5e580731b0 0x7f5e58075670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5e640af000 tx=0x7f5e6401b090 comp rx=0 tx=0).stop 2024-09-10T22:06:50.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e80a00640 1 -- 172.21.15.66:0/2195641500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c0c4fc0 msgr2=0x7f5e7c151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:50.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e80a00640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c0c4fc0 0x7f5e7c151820 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00c430 tx=0x7f5e6c00c8f0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e80a00640 1 -- 172.21.15.66:0/2195641500 shutdown_connections 2024-09-10T22:06:50.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e80a00640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5e580731b0 0x7f5e58075670 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e80a00640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c0c7680 0x7f5e7c157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e80a00640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c15a310 0x7f5e7c14f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e80a00640 1 --2- 172.21.15.66:0/2195641500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c0c4fc0 0x7f5e7c151820 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e80a00640 1 -- 172.21.15.66:0/2195641500 >> 172.21.15.66:0/2195641500 conn(0x7f5e7c0cfa30 msgr2=0x7f5e7c0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e80a00640 1 -- 172.21.15.66:0/2195641500 shutdown_connections 2024-09-10T22:06:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:50.862+0000 7f5e80a00640 1 -- 172.21.15.66:0/2195641500 wait complete. 2024-09-10T22:06:50.887 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:06:50.887 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:06:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 -- 172.21.15.66:0/2956007681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff07c003fe0 msgr2=0x7ff07c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 --2- 172.21.15.66:0/2956007681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff07c003fe0 0x7ff07c004810 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff08000c310 tx=0x7ff08001d050 comp rx=0 tx=0).stop 2024-09-10T22:06:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09cc00640 1 -- 172.21.15.66:0/2956007681 reap_dead start 2024-09-10T22:06:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 -- 172.21.15.66:0/2956007681 shutdown_connections 2024-09-10T22:06:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 -- 172.21.15.66:0/2956007681 >> 172.21.15.66:0/2956007681 conn(0x7ff0980cfaa0 msgr2=0x7ff0980cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 -- 172.21.15.66:0/2956007681 shutdown_connections 2024-09-10T22:06:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 -- 172.21.15.66:0/2956007681 wait complete. 2024-09-10T22:06:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 Processor -- start 2024-09-10T22:06:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 -- start start 2024-09-10T22:06:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff07c003fe0 0x7ff0981f8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0980c4d80 0x7ff098145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff09815cd80 0x7ff09815d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff09806e820 con 0x7ff07c003fe0 2024-09-10T22:06:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff09806e6a0 con 0x7ff0980c4d80 2024-09-10T22:06:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff09ec00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff09806e9a0 con 0x7ff09815cd80 2024-09-10T22:06:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff097e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff07c003fe0 0x7ff0981f8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff097e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff07c003fe0 0x7ff0981f8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40362/0 (socket says 172.21.15.66:40362) 2024-09-10T22:06:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.034+0000 7ff097e00640 1 -- 172.21.15.66:0/810076086 learned_addr learned my addr 172.21.15.66:0/810076086 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff097e00640 1 -- 172.21.15.66:0/810076086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff09815cd80 msgr2=0x7ff09815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff097e00640 1 --2- 172.21.15.66:0/810076086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff09815cd80 0x7ff09815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:51.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff097e00640 1 -- 172.21.15.66:0/810076086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0980c4d80 msgr2=0x7ff098145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:51.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff097e00640 1 --2- 172.21.15.66:0/810076086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0980c4d80 0x7ff098145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:51.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff097e00640 1 -- 172.21.15.66:0/810076086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0981f9080 con 0x7ff07c003fe0 2024-09-10T22:06:51.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff097e00640 1 --2- 172.21.15.66:0/810076086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff07c003fe0 0x7ff0981f8b40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff08800e9e0 tx=0x7ff08800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:51.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff094c00640 1 -- 172.21.15.66:0/810076086 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08801c070 con 0x7ff07c003fe0 2024-09-10T22:06:51.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff094c00640 1 -- 172.21.15.66:0/810076086 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff08800cbd0 con 0x7ff07c003fe0 2024-09-10T22:06:51.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff094c00640 1 -- 172.21.15.66:0/810076086 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088017410 con 0x7ff07c003fe0 2024-09-10T22:06:51.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff09ec00640 1 -- 172.21.15.66:0/810076086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0980c96f0 con 0x7ff07c003fe0 2024-09-10T22:06:51.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff09ec00640 1 -- 172.21.15.66:0/810076086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0980c5600 con 0x7ff07c003fe0 2024-09-10T22:06:51.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff09ec00640 1 -- 172.21.15.66:0/810076086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09815e220 con 0x7ff07c003fe0 2024-09-10T22:06:51.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff094c00640 1 -- 172.21.15.66:0/810076086 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7ff08800b040 con 0x7ff07c003fe0 2024-09-10T22:06:51.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff094c00640 1 --2- 172.21.15.66:0/810076086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff0740731b0 0x7ff074075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:51.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.038+0000 7ff094c00640 1 -- 172.21.15.66:0/810076086 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4885+0+0 (secure 0 0 0) 0x7ff088091d10 con 0x7ff07c003fe0 2024-09-10T22:06:51.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.042+0000 7ff08fe00640 1 --2- 172.21.15.66:0/810076086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff0740731b0 0x7ff074075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:51.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.042+0000 7ff08fe00640 1 --2- 172.21.15.66:0/810076086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff0740731b0 0x7ff074075670 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff0800ac000 tx=0x7ff07c005840 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.042+0000 7ff094c00640 1 -- 172.21.15.66:0/810076086 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff0981f9080 con 0x7ff07c003fe0 2024-09-10T22:06:51.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:51.218+0000 7ff09ec00640 1 -- 172.21.15.66:0/810076086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff098159ff0 con 0x7ff07c003fe0 2024-09-10T22:06:52.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.098+0000 7ff094c00640 1 -- 172.21.15.66:0/810076086 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v68) ==== 138+0+0 (secure 0 0 0) 0x7ff08805e7f0 con 0x7ff07c003fe0 2024-09-10T22:06:52.105 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:06:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.102+0000 7ff09ec00640 1 -- 172.21.15.66:0/810076086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff0740731b0 msgr2=0x7ff074075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.102+0000 7ff09ec00640 1 --2- 172.21.15.66:0/810076086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff0740731b0 0x7ff074075670 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff0800ac000 tx=0x7ff07c005840 comp rx=0 tx=0).stop 2024-09-10T22:06:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.102+0000 7ff09ec00640 1 -- 172.21.15.66:0/810076086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff07c003fe0 msgr2=0x7ff0981f8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.102+0000 7ff09ec00640 1 --2- 172.21.15.66:0/810076086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff07c003fe0 0x7ff0981f8b40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff08800e9e0 tx=0x7ff08800eea0 comp rx=0 tx=0).stop 2024-09-10T22:06:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.102+0000 7ff09ec00640 1 -- 172.21.15.66:0/810076086 shutdown_connections 2024-09-10T22:06:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.102+0000 7ff09ec00640 1 --2- 172.21.15.66:0/810076086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff0740731b0 0x7ff074075670 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.102+0000 7ff09ec00640 1 --2- 172.21.15.66:0/810076086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff09815cd80 0x7ff09815d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.102+0000 7ff09ec00640 1 --2- 172.21.15.66:0/810076086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0980c4d80 0x7ff098145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:52.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.102+0000 7ff09ec00640 1 --2- 172.21.15.66:0/810076086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff07c003fe0 0x7ff0981f8b40 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:52.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.102+0000 7ff09ec00640 1 -- 172.21.15.66:0/810076086 >> 172.21.15.66:0/810076086 conn(0x7ff0980cfaa0 msgr2=0x7ff0980aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:52.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.102+0000 7ff09ec00640 1 -- 172.21.15.66:0/810076086 shutdown_connections 2024-09-10T22:06:52.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.102+0000 7ff09ec00640 1 -- 172.21.15.66:0/810076086 wait complete. 2024-09-10T22:06:52.136 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:06:52.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 -- 172.21.15.66:0/2960008053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f756815ef20 msgr2=0x7f75680cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:52.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 --2- 172.21.15.66:0/2960008053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f756815ef20 0x7f75680cecc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7558009840 tx=0x7f755801c7f0 comp rx=0 tx=0).stop 2024-09-10T22:06:52.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 -- 172.21.15.66:0/2960008053 shutdown_connections 2024-09-10T22:06:52.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 --2- 172.21.15.66:0/2960008053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f756815f300 0x7f75680b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:52.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 --2- 172.21.15.66:0/2960008053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f756815ef20 0x7f75680cecc0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 -- 172.21.15.66:0/2960008053 >> 172.21.15.66:0/2960008053 conn(0x7f75680d1b10 msgr2=0x7f75680d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 -- 172.21.15.66:0/2960008053 shutdown_connections 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 -- 172.21.15.66:0/2960008053 wait complete. 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 Processor -- start 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 -- start start 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75680b15d0 0x7f7568157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f756815ef20 0x7f756815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756815f300 0x7f756815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7568168ee0 con 0x7f75680b15d0 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7568168d60 con 0x7f756815f300 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7567200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7568169060 con 0x7f756815ef20 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7565e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75680b15d0 0x7f7568157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7565e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75680b15d0 0x7f7568157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40388/0 (socket says 172.21.15.66:40388) 2024-09-10T22:06:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7565e00640 1 -- 172.21.15.66:0/2693254085 learned_addr learned my addr 172.21.15.66:0/2693254085 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:52.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f755fe00640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f756815ef20 0x7f756815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:52.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7566800640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756815f300 0x7f756815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:52.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7565e00640 1 -- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f756815ef20 msgr2=0x7f756815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:52.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7565e00640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f756815ef20 0x7f756815b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:52.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7565e00640 1 -- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756815f300 msgr2=0x7f756815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:52.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7565e00640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756815f300 0x7f756815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:52.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7565e00640 1 -- 172.21.15.66:0/2693254085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756806e1c0 con 0x7f75680b15d0 2024-09-10T22:06:52.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.290+0000 7f7565e00640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75680b15d0 0x7f7568157c00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7558009790 tx=0x7f7558004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:52.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.294+0000 7f755e000640 1 -- 172.21.15.66:0/2693254085 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580bb070 con 0x7f75680b15d0 2024-09-10T22:06:52.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.294+0000 7f755e000640 1 -- 172.21.15.66:0/2693254085 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7558005320 con 0x7f75680b15d0 2024-09-10T22:06:52.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.294+0000 7f755e000640 1 -- 172.21.15.66:0/2693254085 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580b2040 con 0x7f75680b15d0 2024-09-10T22:06:52.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.294+0000 7f7567200640 1 -- 172.21.15.66:0/2693254085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756806e4a0 con 0x7f75680b15d0 2024-09-10T22:06:52.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.294+0000 7f7567200640 1 -- 172.21.15.66:0/2693254085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756806ea60 con 0x7f75680b15d0 2024-09-10T22:06:52.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.294+0000 7f7567200640 1 -- 172.21.15.66:0/2693254085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75680cecc0 con 0x7f75680b15d0 2024-09-10T22:06:52.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.294+0000 7f755e000640 1 -- 172.21.15.66:0/2693254085 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f75580b5030 con 0x7f75680b15d0 2024-09-10T22:06:52.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.298+0000 7f755e000640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75440731b0 0x7f7544075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:52.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.298+0000 7f755e000640 1 -- 172.21.15.66:0/2693254085 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5255+0+0 (secure 0 0 0) 0x7f755812bae0 con 0x7f75680b15d0 2024-09-10T22:06:52.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.298+0000 7f755fe00640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75440731b0 0x7f7544075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:52.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.298+0000 7f755fe00640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75440731b0 0x7f7544075670 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f756815c940 tx=0x7f755000a2d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:52.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.298+0000 7f755e000640 1 -- 172.21.15.66:0/2693254085 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f75580c4020 con 0x7f75680b15d0 2024-09-10T22:06:52.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:52.474+0000 7f7567200640 1 -- 172.21.15.66:0/2693254085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7568072e00 con 0x7f75680b15d0 2024-09-10T22:06:53.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.106+0000 7f755e000640 1 -- 172.21.15.66:0/2693254085 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v69) ==== 170+0+0 (secure 0 0 0) 0x7f7558016410 con 0x7f75680b15d0 2024-09-10T22:06:53.113 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:06:53.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.110+0000 7f7567200640 1 -- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75440731b0 msgr2=0x7f7544075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:53.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.110+0000 7f7567200640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75440731b0 0x7f7544075670 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f756815c940 tx=0x7f755000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:06:53.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.110+0000 7f7567200640 1 -- 172.21.15.66:0/2693254085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75680b15d0 msgr2=0x7f7568157c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:53.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.110+0000 7f7567200640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75680b15d0 0x7f7568157c00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7558009790 tx=0x7f7558004220 comp rx=0 tx=0).stop 2024-09-10T22:06:53.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.110+0000 7f7567200640 1 -- 172.21.15.66:0/2693254085 shutdown_connections 2024-09-10T22:06:53.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.110+0000 7f7567200640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75440731b0 0x7f7544075670 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:53.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.110+0000 7f7567200640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756815f300 0x7f756815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:53.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.110+0000 7f7567200640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f756815ef20 0x7f756815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:53.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.110+0000 7f7567200640 1 --2- 172.21.15.66:0/2693254085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75680b15d0 0x7f7568157c00 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:53.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.110+0000 7f7567200640 1 -- 172.21.15.66:0/2693254085 >> 172.21.15.66:0/2693254085 conn(0x7f75680d1b10 msgr2=0x7f75680b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:53.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.110+0000 7f7567200640 1 -- 172.21.15.66:0/2693254085 shutdown_connections 2024-09-10T22:06:53.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.110+0000 7f7567200640 1 -- 172.21.15.66:0/2693254085 wait complete. 2024-09-10T22:06:53.152 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:06:53.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.294+0000 7f6afc400640 1 -- 172.21.15.66:0/361588751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af415a310 msgr2=0x7f6af40c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:53.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.294+0000 7f6afc400640 1 --2- 172.21.15.66:0/361588751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af415a310 0x7f6af40c48b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009900 tx=0x7f6ae801c850 comp rx=0 tx=0).stop 2024-09-10T22:06:53.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 -- 172.21.15.66:0/361588751 shutdown_connections 2024-09-10T22:06:53.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 --2- 172.21.15.66:0/361588751 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6af40d0550 0x7f6af40c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:53.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 --2- 172.21.15.66:0/361588751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af415a310 0x7f6af40c48b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:53.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 -- 172.21.15.66:0/361588751 >> 172.21.15.66:0/361588751 conn(0x7f6af40cfa30 msgr2=0x7f6af40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:53.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 -- 172.21.15.66:0/361588751 shutdown_connections 2024-09-10T22:06:53.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 -- 172.21.15.66:0/361588751 wait complete. 2024-09-10T22:06:53.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 Processor -- start 2024-09-10T22:06:53.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 -- start start 2024-09-10T22:06:53.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af40aeec0 0x7f6af41f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6af415a310 0x7f6af4150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af40c6b00 0x7f6af40ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6af415edb0 con 0x7f6af40c6b00 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6af415ec30 con 0x7f6af40aeec0 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afc400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6af4167300 con 0x7f6af415a310 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afa400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af40c6b00 0x7f6af40ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afa400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af40c6b00 0x7f6af40ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42602/0 (socket says 172.21.15.66:42602) 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afa400640 1 -- 172.21.15.66:0/2031195964 learned_addr learned my addr 172.21.15.66:0/2031195964 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6af9a00640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af40aeec0 0x7f6af41f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6af9000640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6af415a310 0x7f6af4150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afa400640 1 -- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6af415a310 msgr2=0x7f6af4150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afa400640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6af415a310 0x7f6af4150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afa400640 1 -- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af40aeec0 msgr2=0x7f6af41f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afa400640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af40aeec0 0x7f6af41f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:53.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afa400640 1 -- 172.21.15.66:0/2031195964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af41f9100 con 0x7f6af40c6b00 2024-09-10T22:06:53.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6afa400640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af40c6b00 0x7f6af40ad8c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00c780 tx=0x7f6aec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:53.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6af1600640 1 -- 172.21.15.66:0/2031195964 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec003b80 con 0x7f6af40c6b00 2024-09-10T22:06:53.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6af1600640 1 -- 172.21.15.66:0/2031195964 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6aec003d20 con 0x7f6af40c6b00 2024-09-10T22:06:53.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.298+0000 7f6af1600640 1 -- 172.21.15.66:0/2031195964 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00aab0 con 0x7f6af40c6b00 2024-09-10T22:06:53.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.302+0000 7f6afc400640 1 -- 172.21.15.66:0/2031195964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af40b0de0 con 0x7f6af40c6b00 2024-09-10T22:06:53.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.302+0000 7f6afc400640 1 -- 172.21.15.66:0/2031195964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af40af4b0 con 0x7f6af40c6b00 2024-09-10T22:06:53.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.302+0000 7f6afc400640 1 -- 172.21.15.66:0/2031195964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af40b1300 con 0x7f6af40c6b00 2024-09-10T22:06:53.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.302+0000 7f6af1600640 1 -- 172.21.15.66:0/2031195964 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f6aec00ac50 con 0x7f6af40c6b00 2024-09-10T22:06:53.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.302+0000 7f6af1600640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6ad40731b0 0x7f6ad4075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:53.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.302+0000 7f6af1600640 1 -- 172.21.15.66:0/2031195964 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5637+0+0 (secure 0 0 0) 0x7f6aec0926c0 con 0x7f6af40c6b00 2024-09-10T22:06:53.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.302+0000 7f6af9a00640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6ad40731b0 0x7f6ad4075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:53.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.306+0000 7f6af9a00640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6ad40731b0 0x7f6ad4075670 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80ae000 tx=0x7f6ae80b3040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:53.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.306+0000 7f6af1600640 1 -- 172.21.15.66:0/2031195964 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6aec05ef40 con 0x7f6af40c6b00 2024-09-10T22:06:53.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:53.514+0000 7f6afc400640 1 -- 172.21.15.66:0/2031195964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6af40c9ef0 con 0x7f6af40c6b00 2024-09-10T22:06:54.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.110+0000 7f6af1600640 1 -- 172.21.15.66:0/2031195964 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) ==== 174+0+0 (secure 0 0 0) 0x7f6aec063970 con 0x7f6af40c6b00 2024-09-10T22:06:54.116 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 10 and data pool 11 2024-09-10T22:06:54.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.118+0000 7f6afc400640 1 -- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6ad40731b0 msgr2=0x7f6ad4075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:54.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.118+0000 7f6afc400640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6ad40731b0 0x7f6ad4075670 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80ae000 tx=0x7f6ae80b3040 comp rx=0 tx=0).stop 2024-09-10T22:06:54.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.118+0000 7f6afc400640 1 -- 172.21.15.66:0/2031195964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af40c6b00 msgr2=0x7f6af40ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:54.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.118+0000 7f6afc400640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af40c6b00 0x7f6af40ad8c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00c780 tx=0x7f6aec00cc40 comp rx=0 tx=0).stop 2024-09-10T22:06:54.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.118+0000 7f6afc400640 1 -- 172.21.15.66:0/2031195964 shutdown_connections 2024-09-10T22:06:54.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.118+0000 7f6afc400640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6ad40731b0 0x7f6ad4075670 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.118+0000 7f6afc400640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af40c6b00 0x7f6af40ad8c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.118+0000 7f6afc400640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6af415a310 0x7f6af4150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.118+0000 7f6afc400640 1 --2- 172.21.15.66:0/2031195964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af40aeec0 0x7f6af41f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.118+0000 7f6afc400640 1 -- 172.21.15.66:0/2031195964 >> 172.21.15.66:0/2031195964 conn(0x7f6af40cfa30 msgr2=0x7f6af4073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:54.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.118+0000 7f6afc400640 1 -- 172.21.15.66:0/2031195964 shutdown_connections 2024-09-10T22:06:54.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.118+0000 7f6afc400640 1 -- 172.21.15.66:0/2031195964 wait complete. 2024-09-10T22:06:54.165 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:06:54.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 -- 172.21.15.66:0/171382335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f719c15f300 msgr2=0x7f719c0b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:54.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 --2- 172.21.15.66:0/171382335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f719c15f300 0x7f719c0b1090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7188009870 tx=0x7f718801c820 comp rx=0 tx=0).stop 2024-09-10T22:06:54.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 -- 172.21.15.66:0/171382335 shutdown_connections 2024-09-10T22:06:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 --2- 172.21.15.66:0/171382335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f719c15f300 0x7f719c0b1090 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 -- 172.21.15.66:0/171382335 >> 172.21.15.66:0/171382335 conn(0x7f719c0d1b10 msgr2=0x7f719c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 -- 172.21.15.66:0/171382335 shutdown_connections 2024-09-10T22:06:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 -- 172.21.15.66:0/171382335 wait complete. 2024-09-10T22:06:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 Processor -- start 2024-09-10T22:06:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 -- start start 2024-09-10T22:06:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f719c0b15d0 0x7f719c0c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f719c15ef20 0x7f719c0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f719c15f300 0x7f719c0af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f719c16a030 con 0x7f719c15ef20 2024-09-10T22:06:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f719c169eb0 con 0x7f719c0b15d0 2024-09-10T22:06:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.310+0000 7f71a2c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f719c16a1b0 con 0x7f719c15f300 2024-09-10T22:06:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f719c15ef20 0x7f719c0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f719c15ef20 0x7f719c0cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42604/0 (socket says 172.21.15.66:42604) 2024-09-10T22:06:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a0e00640 1 -- 172.21.15.66:0/2348622219 learned_addr learned my addr 172.21.15.66:0/2348622219 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a1800640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f719c0b15d0 0x7f719c0c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a2200640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f719c15f300 0x7f719c0af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a0e00640 1 -- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f719c15f300 msgr2=0x7f719c0af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a0e00640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f719c15f300 0x7f719c0af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a0e00640 1 -- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f719c0b15d0 msgr2=0x7f719c0c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a0e00640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f719c0b15d0 0x7f719c0c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a0e00640 1 -- 172.21.15.66:0/2348622219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c1f8900 con 0x7f719c15ef20 2024-09-10T22:06:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a0e00640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f719c15ef20 0x7f719c0cb9e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f718801cd00 tx=0x7f7188005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f7199600640 1 -- 172.21.15.66:0/2348622219 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71880bb070 con 0x7f719c15ef20 2024-09-10T22:06:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f7199600640 1 -- 172.21.15.66:0/2348622219 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7188004330 con 0x7f719c15ef20 2024-09-10T22:06:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f7199600640 1 -- 172.21.15.66:0/2348622219 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71880b2040 con 0x7f719c15ef20 2024-09-10T22:06:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a2c00640 1 -- 172.21.15.66:0/2348622219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719c1f7b40 con 0x7f719c15ef20 2024-09-10T22:06:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a2c00640 1 -- 172.21.15.66:0/2348622219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719c1f8100 con 0x7f719c15ef20 2024-09-10T22:06:54.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f71a2c00640 1 -- 172.21.15.66:0/2348622219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719c0b1090 con 0x7f719c15ef20 2024-09-10T22:06:54.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.314+0000 7f7199600640 1 -- 172.21.15.66:0/2348622219 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f71880b5070 con 0x7f719c15ef20 2024-09-10T22:06:54.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.318+0000 7f7199600640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f717c0731b0 0x7f717c075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:54.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.318+0000 7f7199600640 1 -- 172.21.15.66:0/2348622219 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5737+0+0 (secure 0 0 0) 0x7f718812b8a0 con 0x7f719c15ef20 2024-09-10T22:06:54.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.318+0000 7f71a1800640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f717c0731b0 0x7f717c075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:54.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.318+0000 7f71a1800640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f717c0731b0 0x7f717c075670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f71900046c0 tx=0x7f7190009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:54.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.318+0000 7f7199600640 1 -- 172.21.15.66:0/2348622219 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7188016410 con 0x7f719c15ef20 2024-09-10T22:06:54.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.494+0000 7f71a2c00640 1 -- 172.21.15.66:0/2348622219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f719c072e00 con 0x7f719c15ef20 2024-09-10T22:06:54.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f7199600640 1 -- 172.21.15.66:0/2348622219 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16712 (secure 0 0 0) 0x7f71880f87b0 con 0x7f719c15ef20 2024-09-10T22:06:54.503 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:54.503 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":70,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:06:53.520622+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:06:51.234296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T22:06:52.488635+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:06:54.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f71a2c00640 1 -- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f717c0731b0 msgr2=0x7f717c075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:54.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f71a2c00640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f717c0731b0 0x7f717c075670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f71900046c0 tx=0x7f7190009040 comp rx=0 tx=0).stop 2024-09-10T22:06:54.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f71a2c00640 1 -- 172.21.15.66:0/2348622219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f719c15ef20 msgr2=0x7f719c0cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:54.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f71a2c00640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f719c15ef20 0x7f719c0cb9e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f718801cd00 tx=0x7f7188005d30 comp rx=0 tx=0).stop 2024-09-10T22:06:54.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f71a2c00640 1 -- 172.21.15.66:0/2348622219 shutdown_connections 2024-09-10T22:06:54.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f71a2c00640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f717c0731b0 0x7f717c075670 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f71a2c00640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f719c15f300 0x7f719c0af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f71a2c00640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f719c15ef20 0x7f719c0cb9e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f71a2c00640 1 --2- 172.21.15.66:0/2348622219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f719c0b15d0 0x7f719c0c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f71a2c00640 1 -- 172.21.15.66:0/2348622219 >> 172.21.15.66:0/2348622219 conn(0x7f719c0d1b10 msgr2=0x7f719c0cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:54.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f71a2c00640 1 -- 172.21.15.66:0/2348622219 shutdown_connections 2024-09-10T22:06:54.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.498+0000 7f71a2c00640 1 -- 172.21.15.66:0/2348622219 wait complete. 2024-09-10T22:06:54.524 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:06:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 -- 172.21.15.66:0/3104752643 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f534015a310 msgr2=0x7f53400c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 --2- 172.21.15.66:0/3104752643 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f534015a310 0x7f53400c48b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f533000aa10 tx=0x7f533001dba0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 -- 172.21.15.66:0/3104752643 shutdown_connections 2024-09-10T22:06:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 --2- 172.21.15.66:0/3104752643 >> v2:172.21.15.94:3300/0 conn(0x7f53400d0550 0x7f53400c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 --2- 172.21.15.66:0/3104752643 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f534015a310 0x7f53400c48b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 -- 172.21.15.66:0/3104752643 >> 172.21.15.66:0/3104752643 conn(0x7f53400cfa30 msgr2=0x7f53400cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 -- 172.21.15.66:0/3104752643 shutdown_connections 2024-09-10T22:06:54.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 -- 172.21.15.66:0/3104752643 wait complete. 2024-09-10T22:06:54.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 Processor -- start 2024-09-10T22:06:54.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 -- start start 2024-09-10T22:06:54.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53400aeec0 0x7f534014fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:54.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f534015a310 0x7f534015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:54.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5340150210 0x7f534015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:54.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5340167300 con 0x7f534015a310 2024-09-10T22:06:54.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f534015ec90 con 0x7f53400aeec0 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.666+0000 7f5347800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5340167480 con 0x7f5340150210 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5337e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f534015a310 0x7f534015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5337e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f534015a310 0x7f534015ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42614/0 (socket says 172.21.15.66:42614) 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5337e00640 1 -- 172.21.15.66:0/2276785410 learned_addr learned my addr 172.21.15.66:0/2276785410 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5344e00640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53400aeec0 0x7f534014fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5345800640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5340150210 0x7f534015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5337e00640 1 -- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5340150210 msgr2=0x7f534015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5337e00640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5340150210 0x7f534015cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5337e00640 1 -- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53400aeec0 msgr2=0x7f534014fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5337e00640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53400aeec0 0x7f534014fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5337e00640 1 -- 172.21.15.66:0/2276785410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53400b5540 con 0x7f534015a310 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5345800640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5340150210 0x7f534015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5337e00640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f534015a310 0x7f534015ca90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f532800ea40 tx=0x7f532800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:54.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f533d600640 1 -- 172.21.15.66:0/2276785410 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532800ce40 con 0x7f534015a310 2024-09-10T22:06:54.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5347800640 1 -- 172.21.15.66:0/2276785410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534015b960 con 0x7f534015a310 2024-09-10T22:06:54.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f533d600640 1 -- 172.21.15.66:0/2276785410 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5328004020 con 0x7f534015a310 2024-09-10T22:06:54.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f533d600640 1 -- 172.21.15.66:0/2276785410 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328004470 con 0x7f534015a310 2024-09-10T22:06:54.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5347800640 1 -- 172.21.15.66:0/2276785410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53400b5820 con 0x7f534015a310 2024-09-10T22:06:54.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f5347800640 1 -- 172.21.15.66:0/2276785410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f534015df60 con 0x7f534015a310 2024-09-10T22:06:54.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.670+0000 7f533d600640 1 -- 172.21.15.66:0/2276785410 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f5328003600 con 0x7f534015a310 2024-09-10T22:06:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.674+0000 7f533d600640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f531c0731b0 0x7f531c075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.674+0000 7f533d600640 1 -- 172.21.15.66:0/2276785410 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5737+0+0 (secure 0 0 0) 0x7f5328092660 con 0x7f534015a310 2024-09-10T22:06:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.674+0000 7f5344e00640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f531c0731b0 0x7f531c075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:54.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.674+0000 7f5344e00640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f531c0731b0 0x7f531c075670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f53300b0000 tx=0x7f533001c3e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:54.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.678+0000 7f533d600640 1 -- 172.21.15.66:0/2276785410 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f532805f710 con 0x7f534015a310 2024-09-10T22:06:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.850+0000 7f5347800640 1 -- 172.21.15.66:0/2276785410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5340070420 con 0x7f534015a310 2024-09-10T22:06:54.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.850+0000 7f533d600640 1 -- 172.21.15.66:0/2276785410 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16712 (secure 0 0 0) 0x7f532805f010 con 0x7f534015a310 2024-09-10T22:06:54.857 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:54.857 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":70,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:06:53.520622+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:06:51.234296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T22:06:52.488635+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:06:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.854+0000 7f5347800640 1 -- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f531c0731b0 msgr2=0x7f531c075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.854+0000 7f5347800640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f531c0731b0 0x7f531c075670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f53300b0000 tx=0x7f533001c3e0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.854+0000 7f5347800640 1 -- 172.21.15.66:0/2276785410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f534015a310 msgr2=0x7f534015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.854+0000 7f5347800640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f534015a310 0x7f534015ca90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f532800ea40 tx=0x7f532800ef00 comp rx=0 tx=0).stop 2024-09-10T22:06:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.854+0000 7f5347800640 1 -- 172.21.15.66:0/2276785410 shutdown_connections 2024-09-10T22:06:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.854+0000 7f5347800640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f531c0731b0 0x7f531c075670 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.854+0000 7f5347800640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5340150210 0x7f534015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.854+0000 7f5347800640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f534015a310 0x7f534015ca90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.854+0000 7f5347800640 1 --2- 172.21.15.66:0/2276785410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53400aeec0 0x7f534014fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.854+0000 7f5347800640 1 -- 172.21.15.66:0/2276785410 >> 172.21.15.66:0/2276785410 conn(0x7f53400cfa30 msgr2=0x7f53400ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:54.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.854+0000 7f5347800640 1 -- 172.21.15.66:0/2276785410 shutdown_connections 2024-09-10T22:06:54.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:54.854+0000 7f5347800640 1 -- 172.21.15.66:0/2276785410 wait complete. 2024-09-10T22:06:54.880 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:06:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.022+0000 7f532b600640 1 -- 172.21.15.66:0/943507800 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53240afc20 msgr2=0x7f53240ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.022+0000 7f532b600640 1 --2- 172.21.15.66:0/943507800 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53240afc20 0x7f53240ce880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f531c009bf0 tx=0x7f531c01cb50 comp rx=0 tx=0).stop 2024-09-10T22:06:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.022+0000 7f532b600640 1 -- 172.21.15.66:0/943507800 shutdown_connections 2024-09-10T22:06:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.022+0000 7f532b600640 1 --2- 172.21.15.66:0/943507800 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53240afc20 0x7f53240ce880 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.022+0000 7f532b600640 1 -- 172.21.15.66:0/943507800 >> 172.21.15.66:0/943507800 conn(0x7f53240d1c60 msgr2=0x7f53240d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.022+0000 7f532b600640 1 -- 172.21.15.66:0/943507800 shutdown_connections 2024-09-10T22:06:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.022+0000 7f532b600640 1 -- 172.21.15.66:0/943507800 wait complete. 2024-09-10T22:06:55.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f532b600640 1 Processor -- start 2024-09-10T22:06:55.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f532b600640 1 -- start start 2024-09-10T22:06:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f532b600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53240afc20 0x7f5324167240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f532b600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53240d06b0 0x7f53241f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f532b600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532415bd40 0x7f5324157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f532b600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f532416a0d0 con 0x7f532415bd40 2024-09-10T22:06:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f532b600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5324169f50 con 0x7f53240afc20 2024-09-10T22:06:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f532b600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f532416a250 con 0x7f53240d06b0 2024-09-10T22:06:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5329600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532415bd40 0x7f5324157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5329600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532415bd40 0x7f5324157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42620/0 (socket says 172.21.15.66:42620) 2024-09-10T22:06:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5329600640 1 -- 172.21.15.66:0/1886450866 learned_addr learned my addr 172.21.15.66:0/1886450866 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5328c00640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53240afc20 0x7f5324167240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5323e00640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53240d06b0 0x7f53241f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5329600640 1 -- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53240d06b0 msgr2=0x7f53241f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5329600640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53240d06b0 0x7f53241f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5329600640 1 -- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53240afc20 msgr2=0x7f5324167240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5329600640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53240afc20 0x7f5324167240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5329600640 1 -- 172.21.15.66:0/1886450866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532415cab0 con 0x7f532415bd40 2024-09-10T22:06:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5328c00640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53240afc20 0x7f5324167240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:06:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5323e00640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53240d06b0 0x7f53241f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:06:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5329600640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532415bd40 0x7f5324157c60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f531c019760 tx=0x7f531c009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5320c00640 1 -- 172.21.15.66:0/1886450866 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c0bb070 con 0x7f532415bd40 2024-09-10T22:06:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5320c00640 1 -- 172.21.15.66:0/1886450866 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f531c004d40 con 0x7f532415bd40 2024-09-10T22:06:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f5320c00640 1 -- 172.21.15.66:0/1886450866 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c0b2040 con 0x7f532415bd40 2024-09-10T22:06:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f532b600640 1 -- 172.21.15.66:0/1886450866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532415cd90 con 0x7f532415bd40 2024-09-10T22:06:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.026+0000 7f532b600640 1 -- 172.21.15.66:0/1886450866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f532415d350 con 0x7f532415bd40 2024-09-10T22:06:55.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.030+0000 7f532b600640 1 -- 172.21.15.66:0/1886450866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53240cffa0 con 0x7f532415bd40 2024-09-10T22:06:55.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.030+0000 7f5320c00640 1 -- 172.21.15.66:0/1886450866 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f531c0b5030 con 0x7f532415bd40 2024-09-10T22:06:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.030+0000 7f5320c00640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53040731b0 0x7f5304075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.030+0000 7f5320c00640 1 -- 172.21.15.66:0/1886450866 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5737+0+0 (secure 0 0 0) 0x7f531c12b8b0 con 0x7f532415bd40 2024-09-10T22:06:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.030+0000 7f5328c00640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53040731b0 0x7f5304075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.034+0000 7f5328c00640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53040731b0 0x7f5304075670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5318001230 tx=0x7f5318009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.034+0000 7f5320c00640 1 -- 172.21.15.66:0/1886450866 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f531c0f1de0 con 0x7f532415bd40 2024-09-10T22:06:55.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:55.258+0000 7f532b600640 1 -- 172.21.15.66:0/1886450866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f53240b0df0 con 0x7f532415bd40 2024-09-10T22:06:56.091 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-09-10T22:06:56.086+0000 7ff5b3e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:06:56.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.138+0000 7f5320c00640 1 -- 172.21.15.66:0/1886450866 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7f531c0167d0 con 0x7f532415bd40 2024-09-10T22:06:56.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.142+0000 7f532b600640 1 -- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53040731b0 msgr2=0x7f5304075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:56.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.142+0000 7f532b600640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53040731b0 0x7f5304075670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5318001230 tx=0x7f5318009040 comp rx=0 tx=0).stop 2024-09-10T22:06:56.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.142+0000 7f532b600640 1 -- 172.21.15.66:0/1886450866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532415bd40 msgr2=0x7f5324157c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:56.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.142+0000 7f532b600640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532415bd40 0x7f5324157c60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f531c019760 tx=0x7f531c009200 comp rx=0 tx=0).stop 2024-09-10T22:06:56.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.142+0000 7f532b600640 1 -- 172.21.15.66:0/1886450866 shutdown_connections 2024-09-10T22:06:56.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.142+0000 7f532b600640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53040731b0 0x7f5304075670 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:56.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.142+0000 7f532b600640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f532415bd40 0x7f5324157c60 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:56.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.142+0000 7f532b600640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53240d06b0 0x7f53241f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:56.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.142+0000 7f532b600640 1 --2- 172.21.15.66:0/1886450866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53240afc20 0x7f5324167240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:56.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.142+0000 7f532b600640 1 -- 172.21.15.66:0/1886450866 >> 172.21.15.66:0/1886450866 conn(0x7f53240d1c60 msgr2=0x7f53241545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:56.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.142+0000 7f532b600640 1 -- 172.21.15.66:0/1886450866 shutdown_connections 2024-09-10T22:06:56.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.142+0000 7f532b600640 1 -- 172.21.15.66:0/1886450866 wait complete. 2024-09-10T22:06:56.172 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:06:56.172 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:06:56.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.322+0000 7fd418a00640 1 -- 172.21.15.66:0/1961142758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4140d0000 msgr2=0x7fd4140af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.322+0000 7fd418a00640 1 --2- 172.21.15.66:0/1961142758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4140d0000 0x7fd4140af740 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009bc0 tx=0x7fd40c01ca50 comp rx=0 tx=0).stop 2024-09-10T22:06:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.322+0000 7fd418a00640 1 -- 172.21.15.66:0/1961142758 shutdown_connections 2024-09-10T22:06:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.322+0000 7fd418a00640 1 --2- 172.21.15.66:0/1961142758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4140d0000 0x7fd4140af740 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.322+0000 7fd418a00640 1 -- 172.21.15.66:0/1961142758 >> 172.21.15.66:0/1961142758 conn(0x7fd4140d1a80 msgr2=0x7fd4140d1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.322+0000 7fd418a00640 1 -- 172.21.15.66:0/1961142758 shutdown_connections 2024-09-10T22:06:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.322+0000 7fd418a00640 1 -- 172.21.15.66:0/1961142758 wait complete. 2024-09-10T22:06:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.322+0000 7fd418a00640 1 Processor -- start 2024-09-10T22:06:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.322+0000 7fd418a00640 1 -- start start 2024-09-10T22:06:56.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.322+0000 7fd418a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4140d0000 0x7fd414151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:56.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.322+0000 7fd418a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd4140d25a0 0x7fd4141561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:56.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.322+0000 7fd418a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd41415f170 0x7fd41415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:56.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd418a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd414169a60 con 0x7fd41415f170 2024-09-10T22:06:56.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd418a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4141698e0 con 0x7fd4140d0000 2024-09-10T22:06:56.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd418a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd414169be0 con 0x7fd4140d25a0 2024-09-10T22:06:56.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd413200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd41415f170 0x7fd41415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:56.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd413200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd41415f170 0x7fd41415b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42642/0 (socket says 172.21.15.66:42642) 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd413200640 1 -- 172.21.15.66:0/4025325094 learned_addr learned my addr 172.21.15.66:0/4025325094 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd412800640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4140d0000 0x7fd414151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd40be00640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd4140d25a0 0x7fd4141561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd413200640 1 -- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd4140d25a0 msgr2=0x7fd4141561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd413200640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd4140d25a0 0x7fd4141561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd413200640 1 -- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4140d0000 msgr2=0x7fd414151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd413200640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4140d0000 0x7fd414151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd413200640 1 -- 172.21.15.66:0/4025325094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4141fadf0 con 0x7fd41415f170 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd413200640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd41415f170 0x7fd41415b9d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0b0040 tx=0x7fd40c005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd40aa00640 1 -- 172.21.15.66:0/4025325094 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0b2040 con 0x7fd41415f170 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd40aa00640 1 -- 172.21.15.66:0/4025325094 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd40c004040 con 0x7fd41415f170 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd40aa00640 1 -- 172.21.15.66:0/4025325094 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c004450 con 0x7fd41415f170 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd418a00640 1 -- 172.21.15.66:0/4025325094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4141fb0d0 con 0x7fd41415f170 2024-09-10T22:06:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd418a00640 1 -- 172.21.15.66:0/4025325094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4141fb690 con 0x7fd41415f170 2024-09-10T22:06:56.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd418a00640 1 -- 172.21.15.66:0/4025325094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4140cf8f0 con 0x7fd41415f170 2024-09-10T22:06:56.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.326+0000 7fd40aa00640 1 -- 172.21.15.66:0/4025325094 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fd40c0b5070 con 0x7fd41415f170 2024-09-10T22:06:56.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.330+0000 7fd40aa00640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3f00731b0 0x7fd3f0075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:56.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.330+0000 7fd40aa00640 1 -- 172.21.15.66:0/4025325094 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7fd40c12bbd0 con 0x7fd41415f170 2024-09-10T22:06:56.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.330+0000 7fd412800640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3f00731b0 0x7fd3f0075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:56.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.330+0000 7fd412800640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3f00731b0 0x7fd3f0075670 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd404007fd0 tx=0x7fd404009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.334+0000 7fd40aa00640 1 -- 172.21.15.66:0/4025325094 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd40c0c4060 con 0x7fd41415f170 2024-09-10T22:06:56.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:56.542+0000 7fd418a00640 1 -- 172.21.15.66:0/4025325094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd41415d460 con 0x7fd41415f170 2024-09-10T22:06:56.585 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:06:56.588 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:06:57.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.146+0000 7fd40aa00640 1 -- 172.21.15.66:0/4025325094 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7fd40c016790 con 0x7fd41415f170 2024-09-10T22:06:57.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.150+0000 7fd418a00640 1 -- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3f00731b0 msgr2=0x7fd3f0075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:57.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.150+0000 7fd418a00640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3f00731b0 0x7fd3f0075670 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd404007fd0 tx=0x7fd404009040 comp rx=0 tx=0).stop 2024-09-10T22:06:57.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.150+0000 7fd418a00640 1 -- 172.21.15.66:0/4025325094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd41415f170 msgr2=0x7fd41415b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:57.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.150+0000 7fd418a00640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd41415f170 0x7fd41415b9d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0b0040 tx=0x7fd40c005a60 comp rx=0 tx=0).stop 2024-09-10T22:06:57.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.150+0000 7fd418a00640 1 -- 172.21.15.66:0/4025325094 shutdown_connections 2024-09-10T22:06:57.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.150+0000 7fd418a00640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3f00731b0 0x7fd3f0075670 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.150+0000 7fd418a00640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd41415f170 0x7fd41415b9d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.150+0000 7fd418a00640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd4140d25a0 0x7fd4141561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.150+0000 7fd418a00640 1 --2- 172.21.15.66:0/4025325094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4140d0000 0x7fd414151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.150+0000 7fd418a00640 1 -- 172.21.15.66:0/4025325094 >> 172.21.15.66:0/4025325094 conn(0x7fd4140d1a80 msgr2=0x7fd414070e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:57.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.150+0000 7fd418a00640 1 -- 172.21.15.66:0/4025325094 shutdown_connections 2024-09-10T22:06:57.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.150+0000 7fd418a00640 1 -- 172.21.15.66:0/4025325094 wait complete. 2024-09-10T22:06:57.175 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:06:57.176 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:06:57.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.318+0000 7f6594e00640 1 -- 172.21.15.66:0/1766218873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f659015a8a0 msgr2=0x7f65900cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.318+0000 7f6594e00640 1 --2- 172.21.15.66:0/1766218873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f659015a8a0 0x7f65900cea30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6580009900 tx=0x7f658001c8b0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.318+0000 7f6594e00640 1 -- 172.21.15.66:0/1766218873 shutdown_connections 2024-09-10T22:06:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.318+0000 7f6594e00640 1 --2- 172.21.15.66:0/1766218873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f659015a8a0 0x7f65900cea30 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.318+0000 7f6594e00640 1 -- 172.21.15.66:0/1766218873 >> 172.21.15.66:0/1766218873 conn(0x7f65900b26c0 msgr2=0x7f65900b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.318+0000 7f6594e00640 1 -- 172.21.15.66:0/1766218873 shutdown_connections 2024-09-10T22:06:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.318+0000 7f6594e00640 1 -- 172.21.15.66:0/1766218873 wait complete. 2024-09-10T22:06:57.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.318+0000 7f6594e00640 1 Processor -- start 2024-09-10T22:06:57.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.318+0000 7f6594e00640 1 -- start start 2024-09-10T22:06:57.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6594e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65900cb620 0x7f659014fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:57.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6594e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f659015a8a0 0x7f6590151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:57.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6594e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f659015ac80 0x7f6590151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:57.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6594e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6590168b40 con 0x7f65900cb620 2024-09-10T22:06:57.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6594e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65901689c0 con 0x7f659015ac80 2024-09-10T22:06:57.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6594e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6590168cc0 con 0x7f659015a8a0 2024-09-10T22:06:57.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f658de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65900cb620 0x7f659014fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:57.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f658de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65900cb620 0x7f659014fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42654/0 (socket says 172.21.15.66:42654) 2024-09-10T22:06:57.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f658de00640 1 -- 172.21.15.66:0/142768025 learned_addr learned my addr 172.21.15.66:0/142768025 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:57.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6587e00640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f659015a8a0 0x7f6590151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:57.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f658e800640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f659015ac80 0x7f6590151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:57.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f658de00640 1 -- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f659015a8a0 msgr2=0x7f6590151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:57.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f658de00640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f659015a8a0 0x7f6590151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f658de00640 1 -- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f659015ac80 msgr2=0x7f6590151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:57.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f658de00640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f659015ac80 0x7f6590151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f658de00640 1 -- 172.21.15.66:0/142768025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6590151fc0 con 0x7f65900cb620 2024-09-10T22:06:57.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f658de00640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65900cb620 0x7f659014fa20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6580009850 tx=0x7f65800a58a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:57.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6586000640 1 -- 172.21.15.66:0/142768025 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65800b2040 con 0x7f65900cb620 2024-09-10T22:06:57.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6586000640 1 -- 172.21.15.66:0/142768025 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6580005320 con 0x7f65900cb620 2024-09-10T22:06:57.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6586000640 1 -- 172.21.15.66:0/142768025 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580003700 con 0x7f65900cb620 2024-09-10T22:06:57.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6594e00640 1 -- 172.21.15.66:0/142768025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65900ca940 con 0x7f65900cb620 2024-09-10T22:06:57.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6594e00640 1 -- 172.21.15.66:0/142768025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659014f220 con 0x7f65900cb620 2024-09-10T22:06:57.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6594e00640 1 -- 172.21.15.66:0/142768025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659014f010 con 0x7f65900cb620 2024-09-10T22:06:57.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.322+0000 7f6586000640 1 -- 172.21.15.66:0/142768025 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f65800038c0 con 0x7f65900cb620 2024-09-10T22:06:57.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.326+0000 7f6586000640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f656c0731e0 0x7f656c0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:57.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.326+0000 7f6586000640 1 -- 172.21.15.66:0/142768025 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f658012c180 con 0x7f65900cb620 2024-09-10T22:06:57.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.326+0000 7f6587e00640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f656c0731e0 0x7f656c0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:57.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.326+0000 7f6587e00640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f656c0731e0 0x7f656c0756a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f65900cd5e0 tx=0x7f6578009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:57.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.330+0000 7f6586000640 1 -- 172.21.15.66:0/142768025 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f65800164d0 con 0x7f65900cb620 2024-09-10T22:06:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.538+0000 7f6594e00640 1 -- 172.21.15.66:0/142768025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6590073660 con 0x7f65900cb620 2024-09-10T22:06:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.538+0000 7f6586000640 1 -- 172.21.15.66:0/142768025 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5225 (secure 0 0 0) 0x7f65800f90b0 con 0x7f65900cb620 2024-09-10T22:06:57.544 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:57.545 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":46,"btime":"2024-09-10T22:06:57:144562+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6112,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3780556367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3780556367},{"type":"v1","addr":"172.21.15.94:6836","nonce":3780556367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1892098851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1892098851},{"type":"v1","addr":"172.21.15.94:6837","nonce":1892098851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1548997357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1548997357},{"type":"v1","addr":"172.21.15.66:6834","nonce":1548997357}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:06:53.520589+0000","modified":"2024-09-10T22:06:56.547020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6390":{"gid":6390,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2415586891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2415586891},{"type":"v1","addr":"172.21.15.66:6835","nonce":2415586891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390]},"id":5}]} 2024-09-10T22:06:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 46 2024-09-10T22:06:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.542+0000 7f6594e00640 1 -- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f656c0731e0 msgr2=0x7f656c0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.542+0000 7f6594e00640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f656c0731e0 0x7f656c0756a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f65900cd5e0 tx=0x7f6578009040 comp rx=0 tx=0).stop 2024-09-10T22:06:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.542+0000 7f6594e00640 1 -- 172.21.15.66:0/142768025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65900cb620 msgr2=0x7f659014fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.542+0000 7f6594e00640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65900cb620 0x7f659014fa20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6580009850 tx=0x7f65800a58a0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.542+0000 7f6594e00640 1 -- 172.21.15.66:0/142768025 shutdown_connections 2024-09-10T22:06:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.542+0000 7f6594e00640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f656c0731e0 0x7f656c0756a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.542+0000 7f6594e00640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f659015ac80 0x7f6590151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.542+0000 7f6594e00640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f659015a8a0 0x7f6590151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.542+0000 7f6594e00640 1 --2- 172.21.15.66:0/142768025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65900cb620 0x7f659014fa20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.542+0000 7f6594e00640 1 -- 172.21.15.66:0/142768025 >> 172.21.15.66:0/142768025 conn(0x7f65900b26c0 msgr2=0x7f65900c6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.542+0000 7f6594e00640 1 -- 172.21.15.66:0/142768025 shutdown_connections 2024-09-10T22:06:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.542+0000 7f6594e00640 1 -- 172.21.15.66:0/142768025 wait complete. 2024-09-10T22:06:57.566 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:06:57.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 -- 172.21.15.66:0/3603193159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21c00cd7a0 msgr2=0x7f21c00cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:57.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 --2- 172.21.15.66:0/3603193159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21c00cd7a0 0x7f21c00cdc30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009bc0 tx=0x7f21b801cba0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 -- 172.21.15.66:0/3603193159 shutdown_connections 2024-09-10T22:06:57.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 --2- 172.21.15.66:0/3603193159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21c00cd7a0 0x7f21c00cdc30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 --2- 172.21.15.66:0/3603193159 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21c015ef20 0x7f21c00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 -- 172.21.15.66:0/3603193159 >> 172.21.15.66:0/3603193159 conn(0x7f21c00d1b10 msgr2=0x7f21c00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:57.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 -- 172.21.15.66:0/3603193159 shutdown_connections 2024-09-10T22:06:57.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 -- 172.21.15.66:0/3603193159 wait complete. 2024-09-10T22:06:57.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 Processor -- start 2024-09-10T22:06:57.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 -- start start 2024-09-10T22:06:57.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21c00cd7a0 0x7f21c01f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:57.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21c015ef20 0x7f21c01660f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:57.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21c015f300 0x7f21c0155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:57.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0168ee0 con 0x7f21c00cd7a0 2024-09-10T22:06:57.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0168d60 con 0x7f21c015f300 2024-09-10T22:06:57.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c6a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f21c0169060 con 0x7f21c015ef20 2024-09-10T22:06:57.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21c00cd7a0 0x7f21c01f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:57.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21c00cd7a0 0x7f21c01f8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42658/0 (socket says 172.21.15.66:42658) 2024-09-10T22:06:57.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c5600640 1 -- 172.21.15.66:0/3372233568 learned_addr learned my addr 172.21.15.66:0/3372233568 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:57.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.710+0000 7f21c4c00640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21c015ef20 0x7f21c01660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:57.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21c6000640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21c015f300 0x7f21c0155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21c5600640 1 -- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21c015ef20 msgr2=0x7f21c01660f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21c5600640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21c015ef20 0x7f21c01660f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21c5600640 1 -- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21c015f300 msgr2=0x7f21c0155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21c5600640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21c015f300 0x7f21c0155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21c5600640 1 -- 172.21.15.66:0/3372233568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c0166630 con 0x7f21c00cd7a0 2024-09-10T22:06:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21c5600640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21c00cd7a0 0x7f21c01f8940 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f21b400ea20 tx=0x7f21b400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21bd600640 1 -- 172.21.15.66:0/3372233568 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b400cd80 con 0x7f21c00cd7a0 2024-09-10T22:06:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21bd600640 1 -- 172.21.15.66:0/3372233568 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21b4010430 con 0x7f21c00cd7a0 2024-09-10T22:06:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21c6a00640 1 -- 172.21.15.66:0/3372233568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c006f910 con 0x7f21c00cd7a0 2024-09-10T22:06:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21bd600640 1 -- 172.21.15.66:0/3372233568 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4010860 con 0x7f21c00cd7a0 2024-09-10T22:06:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21c6a00640 1 -- 172.21.15.66:0/3372233568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c01f8140 con 0x7f21c00cd7a0 2024-09-10T22:06:57.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21c6a00640 1 -- 172.21.15.66:0/3372233568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c00cdc30 con 0x7f21c00cd7a0 2024-09-10T22:06:57.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21bd600640 1 -- 172.21.15.66:0/3372233568 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f21b4010a00 con 0x7f21c00cd7a0 2024-09-10T22:06:57.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.714+0000 7f21bd600640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21a00731b0 0x7f21a0075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:57.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.718+0000 7f21bd600640 1 -- 172.21.15.66:0/3372233568 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f21b40927f0 con 0x7f21c00cd7a0 2024-09-10T22:06:57.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.718+0000 7f21c4c00640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21a00731b0 0x7f21a0075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:57.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.718+0000 7f21c4c00640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21a00731b0 0x7f21a0075670 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f21ac007fd0 tx=0x7f21ac009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:57.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.718+0000 7f21bd600640 1 -- 172.21.15.66:0/3372233568 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f21c0166630 con 0x7f21c00cd7a0 2024-09-10T22:06:57.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.894+0000 7f21c6a00640 1 -- 172.21.15.66:0/3372233568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21c0072e00 con 0x7f21c00cd7a0 2024-09-10T22:06:57.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21bd600640 1 -- 172.21.15.66:0/3372233568 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16699 (secure 0 0 0) 0x7f21b405f290 con 0x7f21c00cd7a0 2024-09-10T22:06:57.902 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:57.902 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":71,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:06:55.115079+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:06:51.234296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T22:06:52.488635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:06:57.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21c6a00640 1 -- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21a00731b0 msgr2=0x7f21a0075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:57.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21c6a00640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21a00731b0 0x7f21a0075670 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f21ac007fd0 tx=0x7f21ac009040 comp rx=0 tx=0).stop 2024-09-10T22:06:57.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21c6a00640 1 -- 172.21.15.66:0/3372233568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21c00cd7a0 msgr2=0x7f21c01f8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21c6a00640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21c00cd7a0 0x7f21c01f8940 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f21b400ea20 tx=0x7f21b400eee0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21c6a00640 1 -- 172.21.15.66:0/3372233568 shutdown_connections 2024-09-10T22:06:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21c6a00640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21a00731b0 0x7f21a0075670 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21c6a00640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21c015f300 0x7f21c0155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21c6a00640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21c015ef20 0x7f21c01660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21c6a00640 1 --2- 172.21.15.66:0/3372233568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21c00cd7a0 0x7f21c01f8940 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21c6a00640 1 -- 172.21.15.66:0/3372233568 >> 172.21.15.66:0/3372233568 conn(0x7f21c00d1b10 msgr2=0x7f21c00aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21c6a00640 1 -- 172.21.15.66:0/3372233568 shutdown_connections 2024-09-10T22:06:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:57.898+0000 7f21c6a00640 1 -- 172.21.15.66:0/3372233568 wait complete. 2024-09-10T22:06:57.925 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:06:57.925 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:06:58.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.066+0000 7f67b5800640 1 -- 172.21.15.66:0/3895550752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67b00aee20 msgr2=0x7f67b00cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.066+0000 7f67b5800640 1 --2- 172.21.15.66:0/3895550752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67b00aee20 0x7f67b00cd930 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009bc0 tx=0x7f67a801cb10 comp rx=0 tx=0).stop 2024-09-10T22:06:58.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.066+0000 7f67b5800640 1 -- 172.21.15.66:0/3895550752 shutdown_connections 2024-09-10T22:06:58.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.066+0000 7f67b5800640 1 --2- 172.21.15.66:0/3895550752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67b00aee20 0x7f67b00cd930 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.066+0000 7f67b5800640 1 --2- 172.21.15.66:0/3895550752 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67b0159900 0x7f67b00ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.066+0000 7f67b5800640 1 -- 172.21.15.66:0/3895550752 >> 172.21.15.66:0/3895550752 conn(0x7f67b00cfa30 msgr2=0x7f67b00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:58.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.066+0000 7f67b5800640 1 -- 172.21.15.66:0/3895550752 shutdown_connections 2024-09-10T22:06:58.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.066+0000 7f67b5800640 1 -- 172.21.15.66:0/3895550752 wait complete. 2024-09-10T22:06:58.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.066+0000 7f67b5800640 1 Processor -- start 2024-09-10T22:06:58.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.066+0000 7f67b5800640 1 -- start start 2024-09-10T22:06:58.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67b5800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67b00aee20 0x7f67b015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:58.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67b5800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b0159900 0x7f67b015d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:58.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67b5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67b015cc80 0x7f67b0153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:58.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67b5800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67b0168f70 con 0x7f67b015cc80 2024-09-10T22:06:58.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67b5800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f67b0168df0 con 0x7f67b0159900 2024-09-10T22:06:58.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67b5800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f67b01690f0 con 0x7f67b00aee20 2024-09-10T22:06:58.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67af400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67b015cc80 0x7f67b0153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67af400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67b015cc80 0x7f67b0153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42668/0 (socket says 172.21.15.66:42668) 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67af400640 1 -- 172.21.15.66:0/2459588010 learned_addr learned my addr 172.21.15.66:0/2459588010 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67aea00640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67b00aee20 0x7f67b015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67ae000640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b0159900 0x7f67b015d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67af400640 1 -- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67b00aee20 msgr2=0x7f67b015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67af400640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67b00aee20 0x7f67b015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67af400640 1 -- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b0159900 msgr2=0x7f67b015d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67af400640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b0159900 0x7f67b015d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67af400640 1 -- 172.21.15.66:0/2459588010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b0153180 con 0x7f67b015cc80 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67aea00640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67b00aee20 0x7f67b015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67ae000640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b0159900 0x7f67b015d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67af400640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67b015cc80 0x7f67b0153f00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f67a801a740 tx=0x7f67a80a5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:58.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67a6a00640 1 -- 172.21.15.66:0/2459588010 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a80ba020 con 0x7f67b015cc80 2024-09-10T22:06:58.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67a6a00640 1 -- 172.21.15.66:0/2459588010 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67a8004d40 con 0x7f67b015cc80 2024-09-10T22:06:58.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67a6a00640 1 -- 172.21.15.66:0/2459588010 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a80b2040 con 0x7f67b015cc80 2024-09-10T22:06:58.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67b5800640 1 -- 172.21.15.66:0/2459588010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b015f300 con 0x7f67b015cc80 2024-09-10T22:06:58.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67b5800640 1 -- 172.21.15.66:0/2459588010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b0153700 con 0x7f67b015cc80 2024-09-10T22:06:58.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67b5800640 1 -- 172.21.15.66:0/2459588010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b00cd930 con 0x7f67b015cc80 2024-09-10T22:06:58.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.070+0000 7f67a6a00640 1 -- 172.21.15.66:0/2459588010 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f67a80b5070 con 0x7f67b015cc80 2024-09-10T22:06:58.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.074+0000 7f67a6a00640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f67900731b0 0x7f6790075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:58.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.074+0000 7f67a6a00640 1 -- 172.21.15.66:0/2459588010 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f67a812cca0 con 0x7f67b015cc80 2024-09-10T22:06:58.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.074+0000 7f67aea00640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f67900731b0 0x7f6790075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:58.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.074+0000 7f67aea00640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f67900731b0 0x7f6790075670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f67a00058d0 tx=0x7f67a0005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:58.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.074+0000 7f67a6a00640 1 -- 172.21.15.66:0/2459588010 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f67a80c4020 con 0x7f67b015cc80 2024-09-10T22:06:58.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.246+0000 7f67b5800640 1 -- 172.21.15.66:0/2459588010 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67b0156810 con 0x7f67900731b0 2024-09-10T22:06:58.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.254+0000 7f67a6a00640 1 -- 172.21.15.66:0/2459588010 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239866 (secure 0 0 0) 0x7f67b0156810 con 0x7f67900731b0 2024-09-10T22:06:58.260 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:58.263 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:06:58.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.258+0000 7f67b5800640 1 -- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f67900731b0 msgr2=0x7f6790075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.258+0000 7f67b5800640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f67900731b0 0x7f6790075670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f67a00058d0 tx=0x7f67a0005860 comp rx=0 tx=0).stop 2024-09-10T22:06:58.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.258+0000 7f67b5800640 1 -- 172.21.15.66:0/2459588010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67b015cc80 msgr2=0x7f67b0153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.258+0000 7f67b5800640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67b015cc80 0x7f67b0153f00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f67a801a740 tx=0x7f67a80a5800 comp rx=0 tx=0).stop 2024-09-10T22:06:58.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.258+0000 7f67b5800640 1 -- 172.21.15.66:0/2459588010 shutdown_connections 2024-09-10T22:06:58.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.258+0000 7f67b5800640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f67900731b0 0x7f6790075670 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.258+0000 7f67b5800640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67b015cc80 0x7f67b0153f00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.258+0000 7f67b5800640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b0159900 0x7f67b015d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.258+0000 7f67b5800640 1 --2- 172.21.15.66:0/2459588010 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67b00aee20 0x7f67b015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.258+0000 7f67b5800640 1 -- 172.21.15.66:0/2459588010 >> 172.21.15.66:0/2459588010 conn(0x7f67b00cfa30 msgr2=0x7f67b0154ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:58.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.258+0000 7f67b5800640 1 -- 172.21.15.66:0/2459588010 shutdown_connections 2024-09-10T22:06:58.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.258+0000 7f67b5800640 1 -- 172.21.15.66:0/2459588010 wait complete. 2024-09-10T22:06:58.281 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-10T22:06:56.404009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":38,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":405612,"kb_used_data":7524,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":754569108,"statfs":{"total":773094113280,"available":772678766592,"internally_reserved":0,"allocated":7704576,"data_stored":3888760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":407424564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.295783"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699616+0000","last_change":"2024-09-10T22:06:54.150467+0000","last_active":"2024-09-10T22:06:55.699616+0000","last_peered":"2024-09-10T22:06:55.699616+0000","last_clean":"2024-09-10T22:06:55.699616+0000","last_became_active":"2024-09-10T22:06:54.147124+0000","last_became_peered":"2024-09-10T22:06:54.147124+0000","last_unstale":"2024-09-10T22:06:55.699616+0000","last_undegraded":"2024-09-10T22:06:55.699616+0000","last_fullsized":"2024-09-10T22:06:55.699616+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:51.247348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.700108+0000","last_change":"2024-09-10T22:06:54.152239+0000","last_active":"2024-09-10T22:06:55.700108+0000","last_peered":"2024-09-10T22:06:55.700108+0000","last_clean":"2024-09-10T22:06:55.700108+0000","last_became_active":"2024-09-10T22:06:54.151972+0000","last_became_peered":"2024-09-10T22:06:54.151972+0000","last_unstale":"2024-09-10T22:06:55.700108+0000","last_undegraded":"2024-09-10T22:06:55.700108+0000","last_fullsized":"2024-09-10T22:06:55.700108+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:59.780308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699591+0000","last_change":"2024-09-10T22:06:54.123221+0000","last_active":"2024-09-10T22:06:55.699591+0000","last_peered":"2024-09-10T22:06:55.699591+0000","last_clean":"2024-09-10T22:06:55.699591+0000","last_became_active":"2024-09-10T22:06:54.122763+0000","last_became_peered":"2024-09-10T22:06:54.122763+0000","last_unstale":"2024-09-10T22:06:55.699591+0000","last_undegraded":"2024-09-10T22:06:55.699591+0000","last_fullsized":"2024-09-10T22:06:55.699591+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:10.894363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.700156+0000","last_change":"2024-09-10T22:06:54.143476+0000","last_active":"2024-09-10T22:06:55.700156+0000","last_peered":"2024-09-10T22:06:55.700156+0000","last_clean":"2024-09-10T22:06:55.700156+0000","last_became_active":"2024-09-10T22:06:54.142615+0000","last_became_peered":"2024-09-10T22:06:54.142615+0000","last_unstale":"2024-09-10T22:06:55.700156+0000","last_undegraded":"2024-09-10T22:06:55.700156+0000","last_fullsized":"2024-09-10T22:06:55.700156+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:39.536282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.700215+0000","last_change":"2024-09-10T22:06:54.142413+0000","last_active":"2024-09-10T22:06:55.700215+0000","last_peered":"2024-09-10T22:06:55.700215+0000","last_clean":"2024-09-10T22:06:55.700215+0000","last_became_active":"2024-09-10T22:06:54.141939+0000","last_became_peered":"2024-09-10T22:06:54.141939+0000","last_unstale":"2024-09-10T22:06:55.700215+0000","last_undegraded":"2024-09-10T22:06:55.700215+0000","last_fullsized":"2024-09-10T22:06:55.700215+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:06.161183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-09-10T22:06:52.113903+0000","last_change":"2024-09-10T22:06:52.113903+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.113903+0000","last_undegraded":"2024-09-10T22:06:52.113903+0000","last_fullsized":"2024-09-10T22:06:52.113903+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-09-10T22:06:52.112981+0000","last_change":"2024-09-10T22:06:52.112981+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.112981+0000","last_undegraded":"2024-09-10T22:06:52.112981+0000","last_fullsized":"2024-09-10T22:06:52.112981+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699252+0000","last_change":"2024-09-10T22:06:54.129200+0000","last_active":"2024-09-10T22:06:55.699252+0000","last_peered":"2024-09-10T22:06:55.699252+0000","last_clean":"2024-09-10T22:06:55.699252+0000","last_became_active":"2024-09-10T22:06:54.128869+0000","last_became_peered":"2024-09-10T22:06:54.128869+0000","last_unstale":"2024-09-10T22:06:55.699252+0000","last_undegraded":"2024-09-10T22:06:55.699252+0000","last_fullsized":"2024-09-10T22:06:55.699252+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:43.686391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698943+0000","last_change":"2024-09-10T22:06:53.137342+0000","last_active":"2024-09-10T22:06:55.698943+0000","last_peered":"2024-09-10T22:06:55.698943+0000","last_clean":"2024-09-10T22:06:55.698943+0000","last_became_active":"2024-09-10T22:06:53.137027+0000","last_became_peered":"2024-09-10T22:06:53.137027+0000","last_unstale":"2024-09-10T22:06:55.698943+0000","last_undegraded":"2024-09-10T22:06:55.698943+0000","last_fullsized":"2024-09-10T22:06:55.698943+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:37.043464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-09-10T22:06:52.113553+0000","last_change":"2024-09-10T22:06:52.113553+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.113553+0000","last_undegraded":"2024-09-10T22:06:52.113553+0000","last_fullsized":"2024-09-10T22:06:52.113553+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699668+0000","last_change":"2024-09-10T22:06:54.144950+0000","last_active":"2024-09-10T22:06:55.699668+0000","last_peered":"2024-09-10T22:06:55.699668+0000","last_clean":"2024-09-10T22:06:55.699668+0000","last_became_active":"2024-09-10T22:06:54.144684+0000","last_became_peered":"2024-09-10T22:06:54.144684+0000","last_unstale":"2024-09-10T22:06:55.699668+0000","last_undegraded":"2024-09-10T22:06:55.699668+0000","last_fullsized":"2024-09-10T22:06:55.699668+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:10.943305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698601+0000","last_change":"2024-09-10T22:06:54.141131+0000","last_active":"2024-09-10T22:06:55.698601+0000","last_peered":"2024-09-10T22:06:55.698601+0000","last_clean":"2024-09-10T22:06:55.698601+0000","last_became_active":"2024-09-10T22:06:54.140860+0000","last_became_peered":"2024-09-10T22:06:54.140860+0000","last_unstale":"2024-09-10T22:06:55.698601+0000","last_undegraded":"2024-09-10T22:06:55.698601+0000","last_fullsized":"2024-09-10T22:06:55.698601+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:00.457539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699883+0000","last_change":"2024-09-10T22:06:54.127257+0000","last_active":"2024-09-10T22:06:55.699883+0000","last_peered":"2024-09-10T22:06:55.699883+0000","last_clean":"2024-09-10T22:06:55.699883+0000","last_became_active":"2024-09-10T22:06:54.126989+0000","last_became_peered":"2024-09-10T22:06:54.126989+0000","last_unstale":"2024-09-10T22:06:55.699883+0000","last_undegraded":"2024-09-10T22:06:55.699883+0000","last_fullsized":"2024-09-10T22:06:55.699883+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:18.819733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-09-10T22:06:52.111762+0000","last_change":"2024-09-10T22:06:52.111762+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.111762+0000","last_undegraded":"2024-09-10T22:06:52.111762+0000","last_fullsized":"2024-09-10T22:06:52.111762+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699919+0000","last_change":"2024-09-10T22:06:53.131338+0000","last_active":"2024-09-10T22:06:55.699919+0000","last_peered":"2024-09-10T22:06:55.699919+0000","last_clean":"2024-09-10T22:06:55.699919+0000","last_became_active":"2024-09-10T22:06:53.131066+0000","last_became_peered":"2024-09-10T22:06:53.131066+0000","last_unstale":"2024-09-10T22:06:55.699919+0000","last_undegraded":"2024-09-10T22:06:55.699919+0000","last_fullsized":"2024-09-10T22:06:55.699919+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:38.036420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-09-10T22:06:52.112720+0000","last_change":"2024-09-10T22:06:52.112720+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.112720+0000","last_undegraded":"2024-09-10T22:06:52.112720+0000","last_fullsized":"2024-09-10T22:06:52.112720+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699833+0000","last_change":"2024-09-10T22:06:54.140101+0000","last_active":"2024-09-10T22:06:55.699833+0000","last_peered":"2024-09-10T22:06:55.699833+0000","last_clean":"2024-09-10T22:06:55.699833+0000","last_became_active":"2024-09-10T22:06:54.139827+0000","last_became_peered":"2024-09-10T22:06:54.139827+0000","last_unstale":"2024-09-10T22:06:55.699833+0000","last_undegraded":"2024-09-10T22:06:55.699833+0000","last_fullsized":"2024-09-10T22:06:55.699833+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:28.195270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699727+0000","last_change":"2024-09-10T22:06:53.132217+0000","last_active":"2024-09-10T22:06:55.699727+0000","last_peered":"2024-09-10T22:06:55.699727+0000","last_clean":"2024-09-10T22:06:55.699727+0000","last_became_active":"2024-09-10T22:06:53.131959+0000","last_became_peered":"2024-09-10T22:06:53.131959+0000","last_unstale":"2024-09-10T22:06:55.699727+0000","last_undegraded":"2024-09-10T22:06:55.699727+0000","last_fullsized":"2024-09-10T22:06:55.699727+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:00.370703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:53.104612+0000","last_change":"2024-09-10T22:06:53.104612+0000","last_active":"2024-09-10T22:06:53.104612+0000","last_peered":"2024-09-10T22:06:53.104612+0000","last_clean":"2024-09-10T22:06:53.104612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:53.104612+0000","last_undegraded":"2024-09-10T22:06:53.104612+0000","last_fullsized":"2024-09-10T22:06:53.104612+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:06:52.100292+0000","last_change":"2024-09-10T22:06:52.100292+0000","last_active":"2024-09-10T22:06:52.100292+0000","last_peered":"2024-09-10T22:06:52.100292+0000","last_clean":"2024-09-10T22:06:52.100292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:06:52.100292+0000","last_undegraded":"2024-09-10T22:06:52.100292+0000","last_fullsized":"2024-09-10T22:06:52.100292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":196,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T22:06:51.078401+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:06:51.078401+0000","last_peered":"2024-09-10T22:06:51.078401+0000","last_clean":"2024-09-10T22:06:51.078401+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:06:51.078401+0000","last_undegraded":"2024-09-10T22:06:51.078401+0000","last_fullsized":"2024-09-10T22:06:51.078401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509521,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50608,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321232,"statfs":{"total":96636764160,"available":96584941568,"internally_reserved":0,"allocated":1400832,"data_stored":925570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509522,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50036,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321804,"statfs":{"total":96636764160,"available":96585527296,"internally_reserved":0,"allocated":806912,"data_stored":335202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509521,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50028,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321812,"statfs":{"total":96636764160,"available":96585535488,"internally_reserved":0,"allocated":806912,"data_stored":335202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509522,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50032,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321808,"statfs":{"total":96636764160,"available":96585531392,"internally_reserved":0,"allocated":806912,"data_stored":335202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54184,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317656,"statfs":{"total":96636764160,"available":96581279744,"internally_reserved":0,"allocated":806912,"data_stored":335202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509521,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50028,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321812,"statfs":{"total":96636764160,"available":96585535488,"internally_reserved":0,"allocated":806912,"data_stored":335202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509521,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50624,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321216,"statfs":{"total":96636764160,"available":96584925184,"internally_reserved":0,"allocated":1413120,"data_stored":931415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":50395593},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542227,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":856064,"data_stored":355765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:06:58.286 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:06:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.470+0000 7f2f4e200640 1 -- 172.21.15.66:0/919871324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f4815e000 msgr2=0x7f2f4806fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.470+0000 7f2f4e200640 1 --2- 172.21.15.66:0/919871324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f4815e000 0x7f2f4806fd30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2f480b2650 tx=0x7f2f3801c830 comp rx=0 tx=0).stop 2024-09-10T22:06:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.470+0000 7f2f4e200640 1 -- 172.21.15.66:0/919871324 shutdown_connections 2024-09-10T22:06:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.470+0000 7f2f4e200640 1 --2- 172.21.15.66:0/919871324 >> v2:172.21.15.94:3300/0 conn(0x7f2f480736e0 0x7f2f48073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.470+0000 7f2f4e200640 1 --2- 172.21.15.66:0/919871324 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f4806f670 0x7f2f480ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.470+0000 7f2f4e200640 1 --2- 172.21.15.66:0/919871324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f4815e000 0x7f2f4806fd30 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.470+0000 7f2f4e200640 1 -- 172.21.15.66:0/919871324 >> 172.21.15.66:0/919871324 conn(0x7f2f480d0b30 msgr2=0x7f2f480d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.470+0000 7f2f4e200640 1 -- 172.21.15.66:0/919871324 shutdown_connections 2024-09-10T22:06:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.470+0000 7f2f4e200640 1 -- 172.21.15.66:0/919871324 wait complete. 2024-09-10T22:06:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f4e200640 1 Processor -- start 2024-09-10T22:06:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f4e200640 1 -- start start 2024-09-10T22:06:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f4e200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f4806f670 0x7f2f48149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f4e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f480736e0 0x7f2f481fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f4e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f4815e000 0x7f2f481f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f4e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4815e860 con 0x7f2f4815e000 2024-09-10T22:06:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f4e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4815e6e0 con 0x7f2f480736e0 2024-09-10T22:06:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f4e200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2f4815e9e0 con 0x7f2f4806f670 2024-09-10T22:06:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f47e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f4815e000 0x7f2f481f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f47e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f4815e000 0x7f2f481f7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42698/0 (socket says 172.21.15.66:42698) 2024-09-10T22:06:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f47e00640 1 -- 172.21.15.66:0/2895975978 learned_addr learned my addr 172.21.15.66:0/2895975978 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f3ea00640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f480736e0 0x7f2f481fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f47400640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f4806f670 0x7f2f48149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f47e00640 1 -- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f4806f670 msgr2=0x7f2f48149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f47e00640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f4806f670 0x7f2f48149b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f47e00640 1 -- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f480736e0 msgr2=0x7f2f481fbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f47e00640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f480736e0 0x7f2f481fbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f47e00640 1 -- 172.21.15.66:0/2895975978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f481f8a50 con 0x7f2f4815e000 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f47e00640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f4815e000 0x7f2f481f7fb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000ea20 tx=0x7f2f4000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f3fe00640 1 -- 172.21.15.66:0/2895975978 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000cde0 con 0x7f2f4815e000 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f3fe00640 1 -- 172.21.15.66:0/2895975978 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f40004c40 con 0x7f2f4815e000 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f3fe00640 1 -- 172.21.15.66:0/2895975978 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40005280 con 0x7f2f4815e000 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f4e200640 1 -- 172.21.15.66:0/2895975978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f481f8d30 con 0x7f2f4815e000 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f4e200640 1 -- 172.21.15.66:0/2895975978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4814a250 con 0x7f2f4815e000 2024-09-10T22:06:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.474+0000 7f2f4e200640 1 -- 172.21.15.66:0/2895975978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f48073b70 con 0x7f2f4815e000 2024-09-10T22:06:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.478+0000 7f2f3fe00640 1 -- 172.21.15.66:0/2895975978 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f2f4000f040 con 0x7f2f4815e000 2024-09-10T22:06:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.478+0000 7f2f3fe00640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f240731b0 0x7f2f24075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.478+0000 7f2f3fe00640 1 -- 172.21.15.66:0/2895975978 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f2f40092930 con 0x7f2f4815e000 2024-09-10T22:06:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.478+0000 7f2f47400640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f240731b0 0x7f2f24075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.478+0000 7f2f47400640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f240731b0 0x7f2f24075670 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2f380af000 tx=0x7f2f380a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.482+0000 7f2f3fe00640 1 -- 172.21.15.66:0/2895975978 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2f40016030 con 0x7f2f4815e000 2024-09-10T22:06:58.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.650+0000 7f2f4e200640 1 -- 172.21.15.66:0/2895975978 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2f480c4370 con 0x7f2f240731b0 2024-09-10T22:06:58.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.658+0000 7f2f3fe00640 1 -- 172.21.15.66:0/2895975978 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252057 (secure 0 0 0) 0x7f2f480c4370 con 0x7f2f240731b0 2024-09-10T22:06:58.666 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:58.668 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:06:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.662+0000 7f2f4e200640 1 -- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f240731b0 msgr2=0x7f2f24075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.662+0000 7f2f4e200640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f240731b0 0x7f2f24075670 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2f380af000 tx=0x7f2f380a7040 comp rx=0 tx=0).stop 2024-09-10T22:06:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.662+0000 7f2f4e200640 1 -- 172.21.15.66:0/2895975978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f4815e000 msgr2=0x7f2f481f7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.662+0000 7f2f4e200640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f4815e000 0x7f2f481f7fb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000ea20 tx=0x7f2f4000eee0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.662+0000 7f2f4e200640 1 -- 172.21.15.66:0/2895975978 shutdown_connections 2024-09-10T22:06:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.662+0000 7f2f4e200640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2f240731b0 0x7f2f24075670 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.662+0000 7f2f4e200640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f4815e000 0x7f2f481f7fb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.662+0000 7f2f4e200640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f480736e0 0x7f2f481fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.662+0000 7f2f4e200640 1 --2- 172.21.15.66:0/2895975978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2f4806f670 0x7f2f48149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.662+0000 7f2f4e200640 1 -- 172.21.15.66:0/2895975978 >> 172.21.15.66:0/2895975978 conn(0x7f2f480d0b30 msgr2=0x7f2f480b5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:58.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.662+0000 7f2f4e200640 1 -- 172.21.15.66:0/2895975978 shutdown_connections 2024-09-10T22:06:58.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.662+0000 7f2f4e200640 1 -- 172.21.15.66:0/2895975978 wait complete. 2024-09-10T22:06:58.686 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-09-10T22:06:58.404578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410124,"kb_used_data":7900,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754564596,"statfs":{"total":773094113280,"available":772674146304,"internally_reserved":0,"allocated":8089600,"data_stored":4063076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":411684404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001387"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702808+0000","last_change":"2024-09-10T22:06:54.147904+0000","last_active":"2024-09-10T22:06:55.702808+0000","last_peered":"2024-09-10T22:06:55.702808+0000","last_clean":"2024-09-10T22:06:55.702808+0000","last_became_active":"2024-09-10T22:06:54.145585+0000","last_became_peered":"2024-09-10T22:06:54.145585+0000","last_unstale":"2024-09-10T22:06:55.702808+0000","last_undegraded":"2024-09-10T22:06:55.702808+0000","last_fullsized":"2024-09-10T22:06:55.702808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:26.418988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124701+0000","last_change":"2024-09-10T22:06:54.131537+0000","last_active":"2024-09-10T22:06:55.124701+0000","last_peered":"2024-09-10T22:06:55.124701+0000","last_clean":"2024-09-10T22:06:55.124701+0000","last_became_active":"2024-09-10T22:06:54.130824+0000","last_became_peered":"2024-09-10T22:06:54.130824+0000","last_unstale":"2024-09-10T22:06:55.124701+0000","last_undegraded":"2024-09-10T22:06:55.124701+0000","last_fullsized":"2024-09-10T22:06:55.124701+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:09.948047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699045+0000","last_change":"2024-09-10T22:06:54.138464+0000","last_active":"2024-09-10T22:06:55.699045+0000","last_peered":"2024-09-10T22:06:55.699045+0000","last_clean":"2024-09-10T22:06:55.699045+0000","last_became_active":"2024-09-10T22:06:54.138075+0000","last_became_peered":"2024-09-10T22:06:54.138075+0000","last_unstale":"2024-09-10T22:06:55.699045+0000","last_undegraded":"2024-09-10T22:06:55.699045+0000","last_fullsized":"2024-09-10T22:06:55.699045+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:19.194774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699307+0000","last_change":"2024-09-10T22:06:54.146362+0000","last_active":"2024-09-10T22:06:55.699307+0000","last_peered":"2024-09-10T22:06:55.699307+0000","last_clean":"2024-09-10T22:06:55.699307+0000","last_became_active":"2024-09-10T22:06:54.145870+0000","last_became_peered":"2024-09-10T22:06:54.145870+0000","last_unstale":"2024-09-10T22:06:55.699307+0000","last_undegraded":"2024-09-10T22:06:55.699307+0000","last_fullsized":"2024-09-10T22:06:55.699307+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:08.850779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699616+0000","last_change":"2024-09-10T22:06:54.150467+0000","last_active":"2024-09-10T22:06:55.699616+0000","last_peered":"2024-09-10T22:06:55.699616+0000","last_clean":"2024-09-10T22:06:55.699616+0000","last_became_active":"2024-09-10T22:06:54.147124+0000","last_became_peered":"2024-09-10T22:06:54.147124+0000","last_unstale":"2024-09-10T22:06:55.699616+0000","last_undegraded":"2024-09-10T22:06:55.699616+0000","last_fullsized":"2024-09-10T22:06:55.699616+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:51.247348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124478+0000","last_change":"2024-09-10T22:06:54.132888+0000","last_active":"2024-09-10T22:06:55.124478+0000","last_peered":"2024-09-10T22:06:55.124478+0000","last_clean":"2024-09-10T22:06:55.124478+0000","last_became_active":"2024-09-10T22:06:54.132600+0000","last_became_peered":"2024-09-10T22:06:54.132600+0000","last_unstale":"2024-09-10T22:06:55.124478+0000","last_undegraded":"2024-09-10T22:06:55.124478+0000","last_fullsized":"2024-09-10T22:06:55.124478+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:46.323948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.701839+0000","last_change":"2024-09-10T22:06:54.135554+0000","last_active":"2024-09-10T22:06:55.701839+0000","last_peered":"2024-09-10T22:06:55.701839+0000","last_clean":"2024-09-10T22:06:55.701839+0000","last_became_active":"2024-09-10T22:06:54.135299+0000","last_became_peered":"2024-09-10T22:06:54.135299+0000","last_unstale":"2024-09-10T22:06:55.701839+0000","last_undegraded":"2024-09-10T22:06:55.701839+0000","last_fullsized":"2024-09-10T22:06:55.701839+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:27.312849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.123806+0000","last_change":"2024-09-10T22:06:54.148003+0000","last_active":"2024-09-10T22:06:55.123806+0000","last_peered":"2024-09-10T22:06:55.123806+0000","last_clean":"2024-09-10T22:06:55.123806+0000","last_became_active":"2024-09-10T22:06:54.147663+0000","last_became_peered":"2024-09-10T22:06:54.147663+0000","last_unstale":"2024-09-10T22:06:55.123806+0000","last_undegraded":"2024-09-10T22:06:55.123806+0000","last_fullsized":"2024-09-10T22:06:55.123806+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:26.771041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.700108+0000","last_change":"2024-09-10T22:06:54.152239+0000","last_active":"2024-09-10T22:06:55.700108+0000","last_peered":"2024-09-10T22:06:55.700108+0000","last_clean":"2024-09-10T22:06:55.700108+0000","last_became_active":"2024-09-10T22:06:54.151972+0000","last_became_peered":"2024-09-10T22:06:54.151972+0000","last_unstale":"2024-09-10T22:06:55.700108+0000","last_undegraded":"2024-09-10T22:06:55.700108+0000","last_fullsized":"2024-09-10T22:06:55.700108+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:59.780308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124624+0000","last_change":"2024-09-10T22:06:54.136127+0000","last_active":"2024-09-10T22:06:55.124624+0000","last_peered":"2024-09-10T22:06:55.124624+0000","last_clean":"2024-09-10T22:06:55.124624+0000","last_became_active":"2024-09-10T22:06:54.135868+0000","last_became_peered":"2024-09-10T22:06:54.135868+0000","last_unstale":"2024-09-10T22:06:55.124624+0000","last_undegraded":"2024-09-10T22:06:55.124624+0000","last_fullsized":"2024-09-10T22:06:55.124624+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:12.445633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702726+0000","last_change":"2024-09-10T22:06:54.143898+0000","last_active":"2024-09-10T22:06:55.702726+0000","last_peered":"2024-09-10T22:06:55.702726+0000","last_clean":"2024-09-10T22:06:55.702726+0000","last_became_active":"2024-09-10T22:06:54.139201+0000","last_became_peered":"2024-09-10T22:06:54.139201+0000","last_unstale":"2024-09-10T22:06:55.702726+0000","last_undegraded":"2024-09-10T22:06:55.702726+0000","last_fullsized":"2024-09-10T22:06:55.702726+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:24.031060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698888+0000","last_change":"2024-09-10T22:06:54.145925+0000","last_active":"2024-09-10T22:06:55.698888+0000","last_peered":"2024-09-10T22:06:55.698888+0000","last_clean":"2024-09-10T22:06:55.698888+0000","last_became_active":"2024-09-10T22:06:54.144705+0000","last_became_peered":"2024-09-10T22:06:54.144705+0000","last_unstale":"2024-09-10T22:06:55.698888+0000","last_undegraded":"2024-09-10T22:06:55.698888+0000","last_fullsized":"2024-09-10T22:06:55.698888+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:20.649533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.119590+0000","last_change":"2024-09-10T22:06:54.141614+0000","last_active":"2024-09-10T22:06:55.119590+0000","last_peered":"2024-09-10T22:06:55.119590+0000","last_clean":"2024-09-10T22:06:55.119590+0000","last_became_active":"2024-09-10T22:06:54.140555+0000","last_became_peered":"2024-09-10T22:06:54.140555+0000","last_unstale":"2024-09-10T22:06:55.119590+0000","last_undegraded":"2024-09-10T22:06:55.119590+0000","last_fullsized":"2024-09-10T22:06:55.119590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:19.861881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699591+0000","last_change":"2024-09-10T22:06:54.123221+0000","last_active":"2024-09-10T22:06:55.699591+0000","last_peered":"2024-09-10T22:06:55.699591+0000","last_clean":"2024-09-10T22:06:55.699591+0000","last_became_active":"2024-09-10T22:06:54.122763+0000","last_became_peered":"2024-09-10T22:06:54.122763+0000","last_unstale":"2024-09-10T22:06:55.699591+0000","last_undegraded":"2024-09-10T22:06:55.699591+0000","last_fullsized":"2024-09-10T22:06:55.699591+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:10.894363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.119720+0000","last_change":"2024-09-10T22:06:54.143100+0000","last_active":"2024-09-10T22:06:55.119720+0000","last_peered":"2024-09-10T22:06:55.119720+0000","last_clean":"2024-09-10T22:06:55.119720+0000","last_became_active":"2024-09-10T22:06:54.142815+0000","last_became_peered":"2024-09-10T22:06:54.142815+0000","last_unstale":"2024-09-10T22:06:55.119720+0000","last_undegraded":"2024-09-10T22:06:55.119720+0000","last_fullsized":"2024-09-10T22:06:55.119720+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:28.703813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.123382+0000","last_change":"2024-09-10T22:06:54.133243+0000","last_active":"2024-09-10T22:06:55.123382+0000","last_peered":"2024-09-10T22:06:55.123382+0000","last_clean":"2024-09-10T22:06:55.123382+0000","last_became_active":"2024-09-10T22:06:54.132704+0000","last_became_peered":"2024-09-10T22:06:54.132704+0000","last_unstale":"2024-09-10T22:06:55.123382+0000","last_undegraded":"2024-09-10T22:06:55.123382+0000","last_fullsized":"2024-09-10T22:06:55.123382+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:17.830525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.700156+0000","last_change":"2024-09-10T22:06:54.143476+0000","last_active":"2024-09-10T22:06:55.700156+0000","last_peered":"2024-09-10T22:06:55.700156+0000","last_clean":"2024-09-10T22:06:55.700156+0000","last_became_active":"2024-09-10T22:06:54.142615+0000","last_became_peered":"2024-09-10T22:06:54.142615+0000","last_unstale":"2024-09-10T22:06:55.700156+0000","last_undegraded":"2024-09-10T22:06:55.700156+0000","last_fullsized":"2024-09-10T22:06:55.700156+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:39.536282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699161+0000","last_change":"2024-09-10T22:06:54.135911+0000","last_active":"2024-09-10T22:06:55.699161+0000","last_peered":"2024-09-10T22:06:55.699161+0000","last_clean":"2024-09-10T22:06:55.699161+0000","last_became_active":"2024-09-10T22:06:54.135634+0000","last_became_peered":"2024-09-10T22:06:54.135634+0000","last_unstale":"2024-09-10T22:06:55.699161+0000","last_undegraded":"2024-09-10T22:06:55.699161+0000","last_fullsized":"2024-09-10T22:06:55.699161+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:16.867856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124352+0000","last_change":"2024-09-10T22:06:54.148747+0000","last_active":"2024-09-10T22:06:55.124352+0000","last_peered":"2024-09-10T22:06:55.124352+0000","last_clean":"2024-09-10T22:06:55.124352+0000","last_became_active":"2024-09-10T22:06:54.148480+0000","last_became_peered":"2024-09-10T22:06:54.148480+0000","last_unstale":"2024-09-10T22:06:55.124352+0000","last_undegraded":"2024-09-10T22:06:55.124352+0000","last_fullsized":"2024-09-10T22:06:55.124352+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:35.190335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.119651+0000","last_change":"2024-09-10T22:06:54.150383+0000","last_active":"2024-09-10T22:06:55.119651+0000","last_peered":"2024-09-10T22:06:55.119651+0000","last_clean":"2024-09-10T22:06:55.119651+0000","last_became_active":"2024-09-10T22:06:54.148441+0000","last_became_peered":"2024-09-10T22:06:54.148441+0000","last_unstale":"2024-09-10T22:06:55.119651+0000","last_undegraded":"2024-09-10T22:06:55.119651+0000","last_fullsized":"2024-09-10T22:06:55.119651+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:29.217225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.123939+0000","last_change":"2024-09-10T22:06:54.139288+0000","last_active":"2024-09-10T22:06:55.123939+0000","last_peered":"2024-09-10T22:06:55.123939+0000","last_clean":"2024-09-10T22:06:55.123939+0000","last_became_active":"2024-09-10T22:06:54.138337+0000","last_became_peered":"2024-09-10T22:06:54.138337+0000","last_unstale":"2024-09-10T22:06:55.123939+0000","last_undegraded":"2024-09-10T22:06:55.123939+0000","last_fullsized":"2024-09-10T22:06:55.123939+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:09.858571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.123814+0000","last_change":"2024-09-10T22:06:54.134180+0000","last_active":"2024-09-10T22:06:55.123814+0000","last_peered":"2024-09-10T22:06:55.123814+0000","last_clean":"2024-09-10T22:06:55.123814+0000","last_became_active":"2024-09-10T22:06:54.133852+0000","last_became_peered":"2024-09-10T22:06:54.133852+0000","last_unstale":"2024-09-10T22:06:55.123814+0000","last_undegraded":"2024-09-10T22:06:55.123814+0000","last_fullsized":"2024-09-10T22:06:55.123814+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:10.161265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702128+0000","last_change":"2024-09-10T22:06:54.146614+0000","last_active":"2024-09-10T22:06:55.702128+0000","last_peered":"2024-09-10T22:06:55.702128+0000","last_clean":"2024-09-10T22:06:55.702128+0000","last_became_active":"2024-09-10T22:06:54.146342+0000","last_became_peered":"2024-09-10T22:06:54.146342+0000","last_unstale":"2024-09-10T22:06:55.702128+0000","last_undegraded":"2024-09-10T22:06:55.702128+0000","last_fullsized":"2024-09-10T22:06:55.702128+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:14.988581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.701759+0000","last_change":"2024-09-10T22:06:54.138465+0000","last_active":"2024-09-10T22:06:55.701759+0000","last_peered":"2024-09-10T22:06:55.701759+0000","last_clean":"2024-09-10T22:06:55.701759+0000","last_became_active":"2024-09-10T22:06:54.138198+0000","last_became_peered":"2024-09-10T22:06:54.138198+0000","last_unstale":"2024-09-10T22:06:55.701759+0000","last_undegraded":"2024-09-10T22:06:55.701759+0000","last_fullsized":"2024-09-10T22:06:55.701759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:46.935718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124299+0000","last_change":"2024-09-10T22:06:54.146835+0000","last_active":"2024-09-10T22:06:55.124299+0000","last_peered":"2024-09-10T22:06:55.124299+0000","last_clean":"2024-09-10T22:06:55.124299+0000","last_became_active":"2024-09-10T22:06:54.146568+0000","last_became_peered":"2024-09-10T22:06:54.146568+0000","last_unstale":"2024-09-10T22:06:55.124299+0000","last_undegraded":"2024-09-10T22:06:55.124299+0000","last_fullsized":"2024-09-10T22:06:55.124299+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:23.806771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.701777+0000","last_change":"2024-09-10T22:06:54.142293+0000","last_active":"2024-09-10T22:06:55.701777+0000","last_peered":"2024-09-10T22:06:55.701777+0000","last_clean":"2024-09-10T22:06:55.701777+0000","last_became_active":"2024-09-10T22:06:54.141827+0000","last_became_peered":"2024-09-10T22:06:54.141827+0000","last_unstale":"2024-09-10T22:06:55.701777+0000","last_undegraded":"2024-09-10T22:06:55.701777+0000","last_fullsized":"2024-09-10T22:06:55.701777+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:36.454803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699256+0000","last_change":"2024-09-10T22:06:54.144696+0000","last_active":"2024-09-10T22:06:55.699256+0000","last_peered":"2024-09-10T22:06:55.699256+0000","last_clean":"2024-09-10T22:06:55.699256+0000","last_became_active":"2024-09-10T22:06:54.144449+0000","last_became_peered":"2024-09-10T22:06:54.144449+0000","last_unstale":"2024-09-10T22:06:55.699256+0000","last_undegraded":"2024-09-10T22:06:55.699256+0000","last_fullsized":"2024-09-10T22:06:55.699256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:45.156504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124272+0000","last_change":"2024-09-10T22:06:54.145371+0000","last_active":"2024-09-10T22:06:55.124272+0000","last_peered":"2024-09-10T22:06:55.124272+0000","last_clean":"2024-09-10T22:06:55.124272+0000","last_became_active":"2024-09-10T22:06:54.145117+0000","last_became_peered":"2024-09-10T22:06:54.145117+0000","last_unstale":"2024-09-10T22:06:55.124272+0000","last_undegraded":"2024-09-10T22:06:55.124272+0000","last_fullsized":"2024-09-10T22:06:55.124272+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:46.694321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698541+0000","last_change":"2024-09-10T22:06:54.136365+0000","last_active":"2024-09-10T22:06:55.698541+0000","last_peered":"2024-09-10T22:06:55.698541+0000","last_clean":"2024-09-10T22:06:55.698541+0000","last_became_active":"2024-09-10T22:06:54.136102+0000","last_became_peered":"2024-09-10T22:06:54.136102+0000","last_unstale":"2024-09-10T22:06:55.698541+0000","last_undegraded":"2024-09-10T22:06:55.698541+0000","last_fullsized":"2024-09-10T22:06:55.698541+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:31.473421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.701757+0000","last_change":"2024-09-10T22:06:54.142456+0000","last_active":"2024-09-10T22:06:55.701757+0000","last_peered":"2024-09-10T22:06:55.701757+0000","last_clean":"2024-09-10T22:06:55.701757+0000","last_became_active":"2024-09-10T22:06:54.141733+0000","last_became_peered":"2024-09-10T22:06:54.141733+0000","last_unstale":"2024-09-10T22:06:55.701757+0000","last_undegraded":"2024-09-10T22:06:55.701757+0000","last_fullsized":"2024-09-10T22:06:55.701757+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:56.691456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702144+0000","last_change":"2024-09-10T22:06:54.126822+0000","last_active":"2024-09-10T22:06:55.702144+0000","last_peered":"2024-09-10T22:06:55.702144+0000","last_clean":"2024-09-10T22:06:55.702144+0000","last_became_active":"2024-09-10T22:06:54.126418+0000","last_became_peered":"2024-09-10T22:06:54.126418+0000","last_unstale":"2024-09-10T22:06:55.702144+0000","last_undegraded":"2024-09-10T22:06:55.702144+0000","last_fullsized":"2024-09-10T22:06:55.702144+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:14.988581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124048+0000","last_change":"2024-09-10T22:06:54.154091+0000","last_active":"2024-09-10T22:06:55.124048+0000","last_peered":"2024-09-10T22:06:55.124048+0000","last_clean":"2024-09-10T22:06:55.124048+0000","last_became_active":"2024-09-10T22:06:54.152197+0000","last_became_peered":"2024-09-10T22:06:54.152197+0000","last_unstale":"2024-09-10T22:06:55.124048+0000","last_undegraded":"2024-09-10T22:06:55.124048+0000","last_fullsized":"2024-09-10T22:06:55.124048+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:26.035152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702737+0000","last_change":"2024-09-10T22:06:54.146429+0000","last_active":"2024-09-10T22:06:55.702737+0000","last_peered":"2024-09-10T22:06:55.702737+0000","last_clean":"2024-09-10T22:06:55.702737+0000","last_became_active":"2024-09-10T22:06:54.145619+0000","last_became_peered":"2024-09-10T22:06:54.145619+0000","last_unstale":"2024-09-10T22:06:55.702737+0000","last_undegraded":"2024-09-10T22:06:55.702737+0000","last_fullsized":"2024-09-10T22:06:55.702737+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:11.941772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.703022+0000","last_change":"2024-09-10T22:06:53.134679+0000","last_active":"2024-09-10T22:06:55.703022+0000","last_peered":"2024-09-10T22:06:55.703022+0000","last_clean":"2024-09-10T22:06:55.703022+0000","last_became_active":"2024-09-10T22:06:53.133991+0000","last_became_peered":"2024-09-10T22:06:53.133991+0000","last_unstale":"2024-09-10T22:06:55.703022+0000","last_undegraded":"2024-09-10T22:06:55.703022+0000","last_fullsized":"2024-09-10T22:06:55.703022+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:56.973613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.700215+0000","last_change":"2024-09-10T22:06:54.142413+0000","last_active":"2024-09-10T22:06:55.700215+0000","last_peered":"2024-09-10T22:06:55.700215+0000","last_clean":"2024-09-10T22:06:55.700215+0000","last_became_active":"2024-09-10T22:06:54.141939+0000","last_became_peered":"2024-09-10T22:06:54.141939+0000","last_unstale":"2024-09-10T22:06:55.700215+0000","last_undegraded":"2024-09-10T22:06:55.700215+0000","last_fullsized":"2024-09-10T22:06:55.700215+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:06.161183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.119433+0000","last_change":"2024-09-10T22:06:53.130197+0000","last_active":"2024-09-10T22:06:55.119433+0000","last_peered":"2024-09-10T22:06:55.119433+0000","last_clean":"2024-09-10T22:06:55.119433+0000","last_became_active":"2024-09-10T22:06:53.129936+0000","last_became_peered":"2024-09-10T22:06:53.129936+0000","last_unstale":"2024-09-10T22:06:55.119433+0000","last_undegraded":"2024-09-10T22:06:55.119433+0000","last_fullsized":"2024-09-10T22:06:55.119433+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:20.434317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124132+0000","last_change":"2024-09-10T22:06:54.141403+0000","last_active":"2024-09-10T22:06:55.124132+0000","last_peered":"2024-09-10T22:06:55.124132+0000","last_clean":"2024-09-10T22:06:55.124132+0000","last_became_active":"2024-09-10T22:06:54.141032+0000","last_became_peered":"2024-09-10T22:06:54.141032+0000","last_unstale":"2024-09-10T22:06:55.124132+0000","last_undegraded":"2024-09-10T22:06:55.124132+0000","last_fullsized":"2024-09-10T22:06:55.124132+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:40.311201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.123833+0000","last_change":"2024-09-10T22:06:53.133054+0000","last_active":"2024-09-10T22:06:55.123833+0000","last_peered":"2024-09-10T22:06:55.123833+0000","last_clean":"2024-09-10T22:06:55.123833+0000","last_became_active":"2024-09-10T22:06:53.132748+0000","last_became_peered":"2024-09-10T22:06:53.132748+0000","last_unstale":"2024-09-10T22:06:55.123833+0000","last_undegraded":"2024-09-10T22:06:55.123833+0000","last_fullsized":"2024-09-10T22:06:55.123833+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:42.655090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702735+0000","last_change":"2024-09-10T22:06:54.152068+0000","last_active":"2024-09-10T22:06:55.702735+0000","last_peered":"2024-09-10T22:06:55.702735+0000","last_clean":"2024-09-10T22:06:55.702735+0000","last_became_active":"2024-09-10T22:06:54.151777+0000","last_became_peered":"2024-09-10T22:06:54.151777+0000","last_unstale":"2024-09-10T22:06:55.702735+0000","last_undegraded":"2024-09-10T22:06:55.702735+0000","last_fullsized":"2024-09-10T22:06:55.702735+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:29.217225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124560+0000","last_change":"2024-09-10T22:06:53.136914+0000","last_active":"2024-09-10T22:06:55.124560+0000","last_peered":"2024-09-10T22:06:55.124560+0000","last_clean":"2024-09-10T22:06:55.124560+0000","last_became_active":"2024-09-10T22:06:53.136573+0000","last_became_peered":"2024-09-10T22:06:53.136573+0000","last_unstale":"2024-09-10T22:06:55.124560+0000","last_undegraded":"2024-09-10T22:06:55.124560+0000","last_fullsized":"2024-09-10T22:06:55.124560+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:19.645213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.701986+0000","last_change":"2024-09-10T22:06:54.153420+0000","last_active":"2024-09-10T22:06:55.701986+0000","last_peered":"2024-09-10T22:06:55.701986+0000","last_clean":"2024-09-10T22:06:55.701986+0000","last_became_active":"2024-09-10T22:06:54.152654+0000","last_became_peered":"2024-09-10T22:06:54.152654+0000","last_unstale":"2024-09-10T22:06:55.701986+0000","last_undegraded":"2024-09-10T22:06:55.701986+0000","last_fullsized":"2024-09-10T22:06:55.701986+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:46.935718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699014+0000","last_change":"2024-09-10T22:06:53.139794+0000","last_active":"2024-09-10T22:06:55.699014+0000","last_peered":"2024-09-10T22:06:55.699014+0000","last_clean":"2024-09-10T22:06:55.699014+0000","last_became_active":"2024-09-10T22:06:53.139508+0000","last_became_peered":"2024-09-10T22:06:53.139508+0000","last_unstale":"2024-09-10T22:06:55.699014+0000","last_undegraded":"2024-09-10T22:06:55.699014+0000","last_fullsized":"2024-09-10T22:06:55.699014+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:52.197706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.125005+0000","last_change":"2024-09-10T22:06:54.147486+0000","last_active":"2024-09-10T22:06:55.125005+0000","last_peered":"2024-09-10T22:06:55.125005+0000","last_clean":"2024-09-10T22:06:55.125005+0000","last_became_active":"2024-09-10T22:06:54.147230+0000","last_became_peered":"2024-09-10T22:06:54.147230+0000","last_unstale":"2024-09-10T22:06:55.125005+0000","last_undegraded":"2024-09-10T22:06:55.125005+0000","last_fullsized":"2024-09-10T22:06:55.125005+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:03.985628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124663+0000","last_change":"2024-09-10T22:06:53.135472+0000","last_active":"2024-09-10T22:06:55.124663+0000","last_peered":"2024-09-10T22:06:55.124663+0000","last_clean":"2024-09-10T22:06:55.124663+0000","last_became_active":"2024-09-10T22:06:53.134951+0000","last_became_peered":"2024-09-10T22:06:53.134951+0000","last_unstale":"2024-09-10T22:06:55.124663+0000","last_undegraded":"2024-09-10T22:06:55.124663+0000","last_fullsized":"2024-09-10T22:06:55.124663+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:49.057836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699252+0000","last_change":"2024-09-10T22:06:54.129200+0000","last_active":"2024-09-10T22:06:55.699252+0000","last_peered":"2024-09-10T22:06:55.699252+0000","last_clean":"2024-09-10T22:06:55.699252+0000","last_became_active":"2024-09-10T22:06:54.128869+0000","last_became_peered":"2024-09-10T22:06:54.128869+0000","last_unstale":"2024-09-10T22:06:55.699252+0000","last_undegraded":"2024-09-10T22:06:55.699252+0000","last_fullsized":"2024-09-10T22:06:55.699252+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:43.686391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698647+0000","last_change":"2024-09-10T22:06:53.134359+0000","last_active":"2024-09-10T22:06:55.698647+0000","last_peered":"2024-09-10T22:06:55.698647+0000","last_clean":"2024-09-10T22:06:55.698647+0000","last_became_active":"2024-09-10T22:06:53.132116+0000","last_became_peered":"2024-09-10T22:06:53.132116+0000","last_unstale":"2024-09-10T22:06:55.698647+0000","last_undegraded":"2024-09-10T22:06:55.698647+0000","last_fullsized":"2024-09-10T22:06:55.698647+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:28.307692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.119701+0000","last_change":"2024-09-10T22:06:54.139164+0000","last_active":"2024-09-10T22:06:55.119701+0000","last_peered":"2024-09-10T22:06:55.119701+0000","last_clean":"2024-09-10T22:06:55.119701+0000","last_became_active":"2024-09-10T22:06:54.138854+0000","last_became_peered":"2024-09-10T22:06:54.138854+0000","last_unstale":"2024-09-10T22:06:55.119701+0000","last_undegraded":"2024-09-10T22:06:55.119701+0000","last_fullsized":"2024-09-10T22:06:55.119701+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:34.182152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698943+0000","last_change":"2024-09-10T22:06:53.137342+0000","last_active":"2024-09-10T22:06:55.698943+0000","last_peered":"2024-09-10T22:06:55.698943+0000","last_clean":"2024-09-10T22:06:55.698943+0000","last_became_active":"2024-09-10T22:06:53.137027+0000","last_became_peered":"2024-09-10T22:06:53.137027+0000","last_unstale":"2024-09-10T22:06:55.698943+0000","last_undegraded":"2024-09-10T22:06:55.698943+0000","last_fullsized":"2024-09-10T22:06:55.698943+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:37.043464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702005+0000","last_change":"2024-09-10T22:06:54.142547+0000","last_active":"2024-09-10T22:06:55.702005+0000","last_peered":"2024-09-10T22:06:55.702005+0000","last_clean":"2024-09-10T22:06:55.702005+0000","last_became_active":"2024-09-10T22:06:54.142206+0000","last_became_peered":"2024-09-10T22:06:54.142206+0000","last_unstale":"2024-09-10T22:06:55.702005+0000","last_undegraded":"2024-09-10T22:06:55.702005+0000","last_fullsized":"2024-09-10T22:06:55.702005+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:29.338507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124412+0000","last_change":"2024-09-10T22:06:53.137569+0000","last_active":"2024-09-10T22:06:55.124412+0000","last_peered":"2024-09-10T22:06:55.124412+0000","last_clean":"2024-09-10T22:06:55.124412+0000","last_became_active":"2024-09-10T22:06:53.137288+0000","last_became_peered":"2024-09-10T22:06:53.137288+0000","last_unstale":"2024-09-10T22:06:55.124412+0000","last_undegraded":"2024-09-10T22:06:55.124412+0000","last_fullsized":"2024-09-10T22:06:55.124412+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:50.942493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.703006+0000","last_change":"2024-09-10T22:06:54.146141+0000","last_active":"2024-09-10T22:06:55.703006+0000","last_peered":"2024-09-10T22:06:55.703006+0000","last_clean":"2024-09-10T22:06:55.703006+0000","last_became_active":"2024-09-10T22:06:54.145885+0000","last_became_peered":"2024-09-10T22:06:54.145885+0000","last_unstale":"2024-09-10T22:06:55.703006+0000","last_undegraded":"2024-09-10T22:06:55.703006+0000","last_fullsized":"2024-09-10T22:06:55.703006+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:56.710719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124823+0000","last_change":"2024-09-10T22:06:53.137983+0000","last_active":"2024-09-10T22:06:55.124823+0000","last_peered":"2024-09-10T22:06:55.124823+0000","last_clean":"2024-09-10T22:06:55.124823+0000","last_became_active":"2024-09-10T22:06:53.137737+0000","last_became_peered":"2024-09-10T22:06:53.137737+0000","last_unstale":"2024-09-10T22:06:55.124823+0000","last_undegraded":"2024-09-10T22:06:55.124823+0000","last_fullsized":"2024-09-10T22:06:55.124823+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:37.918473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.125014+0000","last_change":"2024-09-10T22:06:54.133205+0000","last_active":"2024-09-10T22:06:55.125014+0000","last_peered":"2024-09-10T22:06:55.125014+0000","last_clean":"2024-09-10T22:06:55.125014+0000","last_became_active":"2024-09-10T22:06:54.132898+0000","last_became_peered":"2024-09-10T22:06:54.132898+0000","last_unstale":"2024-09-10T22:06:55.125014+0000","last_undegraded":"2024-09-10T22:06:55.125014+0000","last_fullsized":"2024-09-10T22:06:55.125014+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:19.480931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699321+0000","last_change":"2024-09-10T22:06:53.137456+0000","last_active":"2024-09-10T22:06:55.699321+0000","last_peered":"2024-09-10T22:06:55.699321+0000","last_clean":"2024-09-10T22:06:55.699321+0000","last_became_active":"2024-09-10T22:06:53.136026+0000","last_became_peered":"2024-09-10T22:06:53.136026+0000","last_unstale":"2024-09-10T22:06:55.699321+0000","last_undegraded":"2024-09-10T22:06:55.699321+0000","last_fullsized":"2024-09-10T22:06:55.699321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:33.695588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699668+0000","last_change":"2024-09-10T22:06:54.144950+0000","last_active":"2024-09-10T22:06:55.699668+0000","last_peered":"2024-09-10T22:06:55.699668+0000","last_clean":"2024-09-10T22:06:55.699668+0000","last_became_active":"2024-09-10T22:06:54.144684+0000","last_became_peered":"2024-09-10T22:06:54.144684+0000","last_unstale":"2024-09-10T22:06:55.699668+0000","last_undegraded":"2024-09-10T22:06:55.699668+0000","last_fullsized":"2024-09-10T22:06:55.699668+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:10.943305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699222+0000","last_change":"2024-09-10T22:06:53.140853+0000","last_active":"2024-09-10T22:06:55.699222+0000","last_peered":"2024-09-10T22:06:55.699222+0000","last_clean":"2024-09-10T22:06:55.699222+0000","last_became_active":"2024-09-10T22:06:53.140422+0000","last_became_peered":"2024-09-10T22:06:53.140422+0000","last_unstale":"2024-09-10T22:06:55.699222+0000","last_undegraded":"2024-09-10T22:06:55.699222+0000","last_fullsized":"2024-09-10T22:06:55.699222+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:46.737170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702014+0000","last_change":"2024-09-10T22:06:54.136367+0000","last_active":"2024-09-10T22:06:55.702014+0000","last_peered":"2024-09-10T22:06:55.702014+0000","last_clean":"2024-09-10T22:06:55.702014+0000","last_became_active":"2024-09-10T22:06:54.136104+0000","last_became_peered":"2024-09-10T22:06:54.136104+0000","last_unstale":"2024-09-10T22:06:55.702014+0000","last_undegraded":"2024-09-10T22:06:55.702014+0000","last_fullsized":"2024-09-10T22:06:55.702014+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:29.338507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699501+0000","last_change":"2024-09-10T22:06:53.130033+0000","last_active":"2024-09-10T22:06:55.699501+0000","last_peered":"2024-09-10T22:06:55.699501+0000","last_clean":"2024-09-10T22:06:55.699501+0000","last_became_active":"2024-09-10T22:06:53.129737+0000","last_became_peered":"2024-09-10T22:06:53.129737+0000","last_unstale":"2024-09-10T22:06:55.699501+0000","last_undegraded":"2024-09-10T22:06:55.699501+0000","last_fullsized":"2024-09-10T22:06:55.699501+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:37.495709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702384+0000","last_change":"2024-09-10T22:06:54.131832+0000","last_active":"2024-09-10T22:06:55.702384+0000","last_peered":"2024-09-10T22:06:55.702384+0000","last_clean":"2024-09-10T22:06:55.702384+0000","last_became_active":"2024-09-10T22:06:54.130966+0000","last_became_peered":"2024-09-10T22:06:54.130966+0000","last_unstale":"2024-09-10T22:06:55.702384+0000","last_undegraded":"2024-09-10T22:06:55.702384+0000","last_fullsized":"2024-09-10T22:06:55.702384+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:35.494156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124612+0000","last_change":"2024-09-10T22:06:53.131945+0000","last_active":"2024-09-10T22:06:55.124612+0000","last_peered":"2024-09-10T22:06:55.124612+0000","last_clean":"2024-09-10T22:06:55.124612+0000","last_became_active":"2024-09-10T22:06:53.131334+0000","last_became_peered":"2024-09-10T22:06:53.131334+0000","last_unstale":"2024-09-10T22:06:55.124612+0000","last_undegraded":"2024-09-10T22:06:55.124612+0000","last_fullsized":"2024-09-10T22:06:55.124612+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:36.139168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.119515+0000","last_change":"2024-09-10T22:06:54.136544+0000","last_active":"2024-09-10T22:06:55.119515+0000","last_peered":"2024-09-10T22:06:55.119515+0000","last_clean":"2024-09-10T22:06:55.119515+0000","last_became_active":"2024-09-10T22:06:54.136258+0000","last_became_peered":"2024-09-10T22:06:54.136258+0000","last_unstale":"2024-09-10T22:06:55.119515+0000","last_undegraded":"2024-09-10T22:06:55.119515+0000","last_fullsized":"2024-09-10T22:06:55.119515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:54.381722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702052+0000","last_change":"2024-09-10T22:06:53.137639+0000","last_active":"2024-09-10T22:06:55.702052+0000","last_peered":"2024-09-10T22:06:55.702052+0000","last_clean":"2024-09-10T22:06:55.702052+0000","last_became_active":"2024-09-10T22:06:53.137121+0000","last_became_peered":"2024-09-10T22:06:53.137121+0000","last_unstale":"2024-09-10T22:06:55.702052+0000","last_undegraded":"2024-09-10T22:06:55.702052+0000","last_fullsized":"2024-09-10T22:06:55.702052+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:11.441313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698601+0000","last_change":"2024-09-10T22:06:54.141131+0000","last_active":"2024-09-10T22:06:55.698601+0000","last_peered":"2024-09-10T22:06:55.698601+0000","last_clean":"2024-09-10T22:06:55.698601+0000","last_became_active":"2024-09-10T22:06:54.140860+0000","last_became_peered":"2024-09-10T22:06:54.140860+0000","last_unstale":"2024-09-10T22:06:55.698601+0000","last_undegraded":"2024-09-10T22:06:55.698601+0000","last_fullsized":"2024-09-10T22:06:55.698601+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:00.457539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.123336+0000","last_change":"2024-09-10T22:06:53.139658+0000","last_active":"2024-09-10T22:06:55.123336+0000","last_peered":"2024-09-10T22:06:55.123336+0000","last_clean":"2024-09-10T22:06:55.123336+0000","last_became_active":"2024-09-10T22:06:53.139351+0000","last_became_peered":"2024-09-10T22:06:53.139351+0000","last_unstale":"2024-09-10T22:06:55.123336+0000","last_undegraded":"2024-09-10T22:06:55.123336+0000","last_fullsized":"2024-09-10T22:06:55.123336+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:33.601432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124880+0000","last_change":"2024-09-10T22:06:54.135046+0000","last_active":"2024-09-10T22:06:55.124880+0000","last_peered":"2024-09-10T22:06:55.124880+0000","last_clean":"2024-09-10T22:06:55.124880+0000","last_became_active":"2024-09-10T22:06:54.134760+0000","last_became_peered":"2024-09-10T22:06:54.134760+0000","last_unstale":"2024-09-10T22:06:55.124880+0000","last_undegraded":"2024-09-10T22:06:55.124880+0000","last_fullsized":"2024-09-10T22:06:55.124880+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:18:00.550616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124786+0000","last_change":"2024-09-10T22:06:53.135713+0000","last_active":"2024-09-10T22:06:55.124786+0000","last_peered":"2024-09-10T22:06:55.124786+0000","last_clean":"2024-09-10T22:06:55.124786+0000","last_became_active":"2024-09-10T22:06:53.135156+0000","last_became_peered":"2024-09-10T22:06:53.135156+0000","last_unstale":"2024-09-10T22:06:55.124786+0000","last_undegraded":"2024-09-10T22:06:55.124786+0000","last_fullsized":"2024-09-10T22:06:55.124786+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:45.509327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702949+0000","last_change":"2024-09-10T22:06:54.151899+0000","last_active":"2024-09-10T22:06:55.702949+0000","last_peered":"2024-09-10T22:06:55.702949+0000","last_clean":"2024-09-10T22:06:55.702949+0000","last_became_active":"2024-09-10T22:06:54.151557+0000","last_became_peered":"2024-09-10T22:06:54.151557+0000","last_unstale":"2024-09-10T22:06:55.702949+0000","last_undegraded":"2024-09-10T22:06:55.702949+0000","last_fullsized":"2024-09-10T22:06:55.702949+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:38.685171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124737+0000","last_change":"2024-09-10T22:06:53.135398+0000","last_active":"2024-09-10T22:06:55.124737+0000","last_peered":"2024-09-10T22:06:55.124737+0000","last_clean":"2024-09-10T22:06:55.124737+0000","last_became_active":"2024-09-10T22:06:53.134942+0000","last_became_peered":"2024-09-10T22:06:53.134942+0000","last_unstale":"2024-09-10T22:06:55.124737+0000","last_undegraded":"2024-09-10T22:06:55.124737+0000","last_fullsized":"2024-09-10T22:06:55.124737+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:57.573857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699135+0000","last_change":"2024-09-10T22:06:54.146201+0000","last_active":"2024-09-10T22:06:55.699135+0000","last_peered":"2024-09-10T22:06:55.699135+0000","last_clean":"2024-09-10T22:06:55.699135+0000","last_became_active":"2024-09-10T22:06:54.144729+0000","last_became_peered":"2024-09-10T22:06:54.144729+0000","last_unstale":"2024-09-10T22:06:55.699135+0000","last_undegraded":"2024-09-10T22:06:55.699135+0000","last_fullsized":"2024-09-10T22:06:55.699135+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:03.772590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124485+0000","last_change":"2024-09-10T22:06:53.133930+0000","last_active":"2024-09-10T22:06:55.124485+0000","last_peered":"2024-09-10T22:06:55.124485+0000","last_clean":"2024-09-10T22:06:55.124485+0000","last_became_active":"2024-09-10T22:06:53.133566+0000","last_became_peered":"2024-09-10T22:06:53.133566+0000","last_unstale":"2024-09-10T22:06:55.124485+0000","last_undegraded":"2024-09-10T22:06:55.124485+0000","last_fullsized":"2024-09-10T22:06:55.124485+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:31.044518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699883+0000","last_change":"2024-09-10T22:06:54.127257+0000","last_active":"2024-09-10T22:06:55.699883+0000","last_peered":"2024-09-10T22:06:55.699883+0000","last_clean":"2024-09-10T22:06:55.699883+0000","last_became_active":"2024-09-10T22:06:54.126989+0000","last_became_peered":"2024-09-10T22:06:54.126989+0000","last_unstale":"2024-09-10T22:06:55.699883+0000","last_undegraded":"2024-09-10T22:06:55.699883+0000","last_fullsized":"2024-09-10T22:06:55.699883+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:18.819733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702419+0000","last_change":"2024-09-10T22:06:53.132643+0000","last_active":"2024-09-10T22:06:55.702419+0000","last_peered":"2024-09-10T22:06:55.702419+0000","last_clean":"2024-09-10T22:06:55.702419+0000","last_became_active":"2024-09-10T22:06:53.131964+0000","last_became_peered":"2024-09-10T22:06:53.131964+0000","last_unstale":"2024-09-10T22:06:55.702419+0000","last_undegraded":"2024-09-10T22:06:55.702419+0000","last_fullsized":"2024-09-10T22:06:55.702419+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:02.981308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698762+0000","last_change":"2024-09-10T22:06:54.149787+0000","last_active":"2024-09-10T22:06:55.698762+0000","last_peered":"2024-09-10T22:06:55.698762+0000","last_clean":"2024-09-10T22:06:55.698762+0000","last_became_active":"2024-09-10T22:06:54.148812+0000","last_became_peered":"2024-09-10T22:06:54.148812+0000","last_unstale":"2024-09-10T22:06:55.698762+0000","last_undegraded":"2024-09-10T22:06:55.698762+0000","last_fullsized":"2024-09-10T22:06:55.698762+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:34.357465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124726+0000","last_change":"2024-09-10T22:06:53.135201+0000","last_active":"2024-09-10T22:06:55.124726+0000","last_peered":"2024-09-10T22:06:55.124726+0000","last_clean":"2024-09-10T22:06:55.124726+0000","last_became_active":"2024-09-10T22:06:53.134595+0000","last_became_peered":"2024-09-10T22:06:53.134595+0000","last_unstale":"2024-09-10T22:06:55.124726+0000","last_undegraded":"2024-09-10T22:06:55.124726+0000","last_fullsized":"2024-09-10T22:06:55.124726+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:00.460361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698478+0000","last_change":"2024-09-10T22:06:54.151288+0000","last_active":"2024-09-10T22:06:55.698478+0000","last_peered":"2024-09-10T22:06:55.698478+0000","last_clean":"2024-09-10T22:06:55.698478+0000","last_became_active":"2024-09-10T22:06:54.143915+0000","last_became_peered":"2024-09-10T22:06:54.143915+0000","last_unstale":"2024-09-10T22:06:55.698478+0000","last_undegraded":"2024-09-10T22:06:55.698478+0000","last_fullsized":"2024-09-10T22:06:55.698478+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:25.771027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702166+0000","last_change":"2024-09-10T22:06:53.137014+0000","last_active":"2024-09-10T22:06:55.702166+0000","last_peered":"2024-09-10T22:06:55.702166+0000","last_clean":"2024-09-10T22:06:55.702166+0000","last_became_active":"2024-09-10T22:06:53.134407+0000","last_became_peered":"2024-09-10T22:06:53.134407+0000","last_unstale":"2024-09-10T22:06:55.702166+0000","last_undegraded":"2024-09-10T22:06:55.702166+0000","last_fullsized":"2024-09-10T22:06:55.702166+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:59.546296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698883+0000","last_change":"2024-09-10T22:06:54.141129+0000","last_active":"2024-09-10T22:06:55.698883+0000","last_peered":"2024-09-10T22:06:55.698883+0000","last_clean":"2024-09-10T22:06:55.698883+0000","last_became_active":"2024-09-10T22:06:54.140872+0000","last_became_peered":"2024-09-10T22:06:54.140872+0000","last_unstale":"2024-09-10T22:06:55.698883+0000","last_undegraded":"2024-09-10T22:06:55.698883+0000","last_fullsized":"2024-09-10T22:06:55.698883+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:10.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.119075+0000","last_change":"2024-09-10T22:06:53.129881+0000","last_active":"2024-09-10T22:06:55.119075+0000","last_peered":"2024-09-10T22:06:55.119075+0000","last_clean":"2024-09-10T22:06:55.119075+0000","last_became_active":"2024-09-10T22:06:53.129611+0000","last_became_peered":"2024-09-10T22:06:53.129611+0000","last_unstale":"2024-09-10T22:06:55.119075+0000","last_undegraded":"2024-09-10T22:06:55.119075+0000","last_fullsized":"2024-09-10T22:06:55.119075+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:02.081670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.119548+0000","last_change":"2024-09-10T22:06:54.139103+0000","last_active":"2024-09-10T22:06:55.119548+0000","last_peered":"2024-09-10T22:06:55.119548+0000","last_clean":"2024-09-10T22:06:55.119548+0000","last_became_active":"2024-09-10T22:06:54.138834+0000","last_became_peered":"2024-09-10T22:06:54.138834+0000","last_unstale":"2024-09-10T22:06:55.119548+0000","last_undegraded":"2024-09-10T22:06:55.119548+0000","last_fullsized":"2024-09-10T22:06:55.119548+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:40.219523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124899+0000","last_change":"2024-09-10T22:06:53.135399+0000","last_active":"2024-09-10T22:06:55.124899+0000","last_peered":"2024-09-10T22:06:55.124899+0000","last_clean":"2024-09-10T22:06:55.124899+0000","last_became_active":"2024-09-10T22:06:53.134792+0000","last_became_peered":"2024-09-10T22:06:53.134792+0000","last_unstale":"2024-09-10T22:06:55.124899+0000","last_undegraded":"2024-09-10T22:06:55.124899+0000","last_fullsized":"2024-09-10T22:06:55.124899+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:29.575874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.701786+0000","last_change":"2024-09-10T22:06:54.134213+0000","last_active":"2024-09-10T22:06:55.701786+0000","last_peered":"2024-09-10T22:06:55.701786+0000","last_clean":"2024-09-10T22:06:55.701786+0000","last_became_active":"2024-09-10T22:06:54.133941+0000","last_became_peered":"2024-09-10T22:06:54.133941+0000","last_unstale":"2024-09-10T22:06:55.701786+0000","last_undegraded":"2024-09-10T22:06:55.701786+0000","last_fullsized":"2024-09-10T22:06:55.701786+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:13.445394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"71'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:56.127203+0000","last_change":"2024-09-10T22:06:53.142687+0000","last_active":"2024-09-10T22:06:56.127203+0000","last_peered":"2024-09-10T22:06:56.127203+0000","last_clean":"2024-09-10T22:06:56.127203+0000","last_became_active":"2024-09-10T22:06:53.142191+0000","last_became_peered":"2024-09-10T22:06:53.142191+0000","last_unstale":"2024-09-10T22:06:56.127203+0000","last_undegraded":"2024-09-10T22:06:56.127203+0000","last_fullsized":"2024-09-10T22:06:56.127203+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:02.981308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702496+0000","last_change":"2024-09-10T22:06:54.146999+0000","last_active":"2024-09-10T22:06:55.702496+0000","last_peered":"2024-09-10T22:06:55.702496+0000","last_clean":"2024-09-10T22:06:55.702496+0000","last_became_active":"2024-09-10T22:06:54.145997+0000","last_became_peered":"2024-09-10T22:06:54.145997+0000","last_unstale":"2024-09-10T22:06:55.702496+0000","last_undegraded":"2024-09-10T22:06:55.702496+0000","last_fullsized":"2024-09-10T22:06:55.702496+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:35.494156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699919+0000","last_change":"2024-09-10T22:06:53.131338+0000","last_active":"2024-09-10T22:06:55.699919+0000","last_peered":"2024-09-10T22:06:55.699919+0000","last_clean":"2024-09-10T22:06:55.699919+0000","last_became_active":"2024-09-10T22:06:53.131066+0000","last_became_peered":"2024-09-10T22:06:53.131066+0000","last_unstale":"2024-09-10T22:06:55.699919+0000","last_undegraded":"2024-09-10T22:06:55.699919+0000","last_fullsized":"2024-09-10T22:06:55.699919+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:38.036420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.702967+0000","last_change":"2024-09-10T22:06:54.147025+0000","last_active":"2024-09-10T22:06:55.702967+0000","last_peered":"2024-09-10T22:06:55.702967+0000","last_clean":"2024-09-10T22:06:55.702967+0000","last_became_active":"2024-09-10T22:06:54.146757+0000","last_became_peered":"2024-09-10T22:06:54.146757+0000","last_unstale":"2024-09-10T22:06:55.702967+0000","last_undegraded":"2024-09-10T22:06:55.702967+0000","last_fullsized":"2024-09-10T22:06:55.702967+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:28.376694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.119245+0000","last_change":"2024-09-10T22:06:53.136102+0000","last_active":"2024-09-10T22:06:55.119245+0000","last_peered":"2024-09-10T22:06:55.119245+0000","last_clean":"2024-09-10T22:06:55.119245+0000","last_became_active":"2024-09-10T22:06:53.135438+0000","last_became_peered":"2024-09-10T22:06:53.135438+0000","last_unstale":"2024-09-10T22:06:55.119245+0000","last_undegraded":"2024-09-10T22:06:55.119245+0000","last_fullsized":"2024-09-10T22:06:55.119245+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:01.232737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699833+0000","last_change":"2024-09-10T22:06:54.140101+0000","last_active":"2024-09-10T22:06:55.699833+0000","last_peered":"2024-09-10T22:06:55.699833+0000","last_clean":"2024-09-10T22:06:55.699833+0000","last_became_active":"2024-09-10T22:06:54.139827+0000","last_became_peered":"2024-09-10T22:06:54.139827+0000","last_unstale":"2024-09-10T22:06:55.699833+0000","last_undegraded":"2024-09-10T22:06:55.699833+0000","last_fullsized":"2024-09-10T22:06:55.699833+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:28.195270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.699727+0000","last_change":"2024-09-10T22:06:53.132217+0000","last_active":"2024-09-10T22:06:55.699727+0000","last_peered":"2024-09-10T22:06:55.699727+0000","last_clean":"2024-09-10T22:06:55.699727+0000","last_became_active":"2024-09-10T22:06:53.131959+0000","last_became_peered":"2024-09-10T22:06:53.131959+0000","last_unstale":"2024-09-10T22:06:55.699727+0000","last_undegraded":"2024-09-10T22:06:55.699727+0000","last_fullsized":"2024-09-10T22:06:55.699727+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:00.370703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124218+0000","last_change":"2024-09-10T22:06:54.151323+0000","last_active":"2024-09-10T22:06:55.124218+0000","last_peered":"2024-09-10T22:06:55.124218+0000","last_clean":"2024-09-10T22:06:55.124218+0000","last_became_active":"2024-09-10T22:06:54.151005+0000","last_became_peered":"2024-09-10T22:06:54.151005+0000","last_unstale":"2024-09-10T22:06:55.124218+0000","last_undegraded":"2024-09-10T22:06:55.124218+0000","last_fullsized":"2024-09-10T22:06:55.124218+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:29.478512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698629+0000","last_change":"2024-09-10T22:06:53.137989+0000","last_active":"2024-09-10T22:06:55.698629+0000","last_peered":"2024-09-10T22:06:55.698629+0000","last_clean":"2024-09-10T22:06:55.698629+0000","last_became_active":"2024-09-10T22:06:53.137169+0000","last_became_peered":"2024-09-10T22:06:53.137169+0000","last_unstale":"2024-09-10T22:06:55.698629+0000","last_undegraded":"2024-09-10T22:06:55.698629+0000","last_fullsized":"2024-09-10T22:06:55.698629+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:44.321363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698885+0000","last_change":"2024-09-10T22:06:54.156739+0000","last_active":"2024-09-10T22:06:55.698885+0000","last_peered":"2024-09-10T22:06:55.698885+0000","last_clean":"2024-09-10T22:06:55.698885+0000","last_became_active":"2024-09-10T22:06:54.156451+0000","last_became_peered":"2024-09-10T22:06:54.156451+0000","last_unstale":"2024-09-10T22:06:55.698885+0000","last_undegraded":"2024-09-10T22:06:55.698885+0000","last_fullsized":"2024-09-10T22:06:55.698885+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:36.216243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.123519+0000","last_change":"2024-09-10T22:06:53.140521+0000","last_active":"2024-09-10T22:06:55.123519+0000","last_peered":"2024-09-10T22:06:55.123519+0000","last_clean":"2024-09-10T22:06:55.123519+0000","last_became_active":"2024-09-10T22:06:53.140252+0000","last_became_peered":"2024-09-10T22:06:53.140252+0000","last_unstale":"2024-09-10T22:06:55.123519+0000","last_undegraded":"2024-09-10T22:06:55.123519+0000","last_fullsized":"2024-09-10T22:06:55.123519+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:35.010962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.123427+0000","last_change":"2024-09-10T22:06:54.139988+0000","last_active":"2024-09-10T22:06:55.123427+0000","last_peered":"2024-09-10T22:06:55.123427+0000","last_clean":"2024-09-10T22:06:55.123427+0000","last_became_active":"2024-09-10T22:06:54.137303+0000","last_became_peered":"2024-09-10T22:06:54.137303+0000","last_unstale":"2024-09-10T22:06:55.123427+0000","last_undegraded":"2024-09-10T22:06:55.123427+0000","last_fullsized":"2024-09-10T22:06:55.123427+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:02.098281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124357+0000","last_change":"2024-09-10T22:06:53.140125+0000","last_active":"2024-09-10T22:06:55.124357+0000","last_peered":"2024-09-10T22:06:55.124357+0000","last_clean":"2024-09-10T22:06:55.124357+0000","last_became_active":"2024-09-10T22:06:53.139595+0000","last_became_peered":"2024-09-10T22:06:53.139595+0000","last_unstale":"2024-09-10T22:06:55.124357+0000","last_undegraded":"2024-09-10T22:06:55.124357+0000","last_fullsized":"2024-09-10T22:06:55.124357+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:19.353553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.698598+0000","last_change":"2024-09-10T22:06:54.146753+0000","last_active":"2024-09-10T22:06:55.698598+0000","last_peered":"2024-09-10T22:06:55.698598+0000","last_clean":"2024-09-10T22:06:55.698598+0000","last_became_active":"2024-09-10T22:06:54.146458+0000","last_became_peered":"2024-09-10T22:06:54.146458+0000","last_unstale":"2024-09-10T22:06:55.698598+0000","last_undegraded":"2024-09-10T22:06:55.698598+0000","last_fullsized":"2024-09-10T22:06:55.698598+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:53.104612+0000","last_clean_scrub_stamp":"2024-09-10T22:06:53.104612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:57.910123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.124336+0000","last_change":"2024-09-10T22:06:53.130636+0000","last_active":"2024-09-10T22:06:55.124336+0000","last_peered":"2024-09-10T22:06:55.124336+0000","last_clean":"2024-09-10T22:06:55.124336+0000","last_became_active":"2024-09-10T22:06:53.130366+0000","last_became_peered":"2024-09-10T22:06:53.130366+0000","last_unstale":"2024-09-10T22:06:55.124336+0000","last_undegraded":"2024-09-10T22:06:55.124336+0000","last_fullsized":"2024-09-10T22:06:55.124336+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:06:52.100292+0000","last_clean_scrub_stamp":"2024-09-10T22:06:52.100292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:12.498540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":204,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T22:06:55.701834+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:06:55.701834+0000","last_peered":"2024-09-10T22:06:55.701834+0000","last_clean":"2024-09-10T22:06:55.701834+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:06:55.701834+0000","last_undegraded":"2024-09-10T22:06:55.701834+0000","last_fullsized":"2024-09-10T22:06:55.701834+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509522,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50664,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321176,"statfs":{"total":96636764160,"available":96584884224,"internally_reserved":0,"allocated":1458176,"data_stored":951465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509523,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50080,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321760,"statfs":{"total":96636764160,"available":96585482240,"internally_reserved":0,"allocated":864256,"data_stored":360573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509522,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50076,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321764,"statfs":{"total":96636764160,"available":96585486336,"internally_reserved":0,"allocated":860160,"data_stored":360571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509523,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54236,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317604,"statfs":{"total":96636764160,"available":96581226496,"internally_reserved":0,"allocated":860160,"data_stored":360551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54244,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317596,"statfs":{"total":96636764160,"available":96581218304,"internally_reserved":0,"allocated":864256,"data_stored":360593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509522,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50092,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":876544,"data_stored":362709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509522,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50660,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321180,"statfs":{"total":96636764160,"available":96584888320,"internally_reserved":0,"allocated":1449984,"data_stored":950849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":50395593},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542227,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":856064,"data_stored":355765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:06:58.691 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:06:58.691 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T22:06:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 -- 172.21.15.66:0/57403599 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8500d0550 msgr2=0x7fb8500ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 --2- 172.21.15.66:0/57403599 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8500d0550 0x7fb8500ca1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00a980 tx=0x7fb83c01db00 comp rx=0 tx=0).stop 2024-09-10T22:06:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 -- 172.21.15.66:0/57403599 shutdown_connections 2024-09-10T22:06:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 --2- 172.21.15.66:0/57403599 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8500d0550 0x7fb8500ca1a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 -- 172.21.15.66:0/57403599 >> 172.21.15.66:0/57403599 conn(0x7fb8500cfa30 msgr2=0x7fb8500cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:58.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 -- 172.21.15.66:0/57403599 shutdown_connections 2024-09-10T22:06:58.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 -- 172.21.15.66:0/57403599 wait complete. 2024-09-10T22:06:58.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 Processor -- start 2024-09-10T22:06:58.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 -- start start 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8500c4fc0 0x7fb85014e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb85015a310 0x7fb8500ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8500ade00 0x7fb8501fb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb8501686a0 con 0x7fb8500c4fc0 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb850168520 con 0x7fb8500ade00 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb857600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb850168820 con 0x7fb85015a310 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.882+0000 7fb854c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8500c4fc0 0x7fb85014e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb854c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8500c4fc0 0x7fb85014e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42700/0 (socket says 172.21.15.66:42700) 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb854c00640 1 -- 172.21.15.66:0/4146301032 learned_addr learned my addr 172.21.15.66:0/4146301032 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb84fe00640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb85015a310 0x7fb8500ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb855600640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8500ade00 0x7fb8501fb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb854c00640 1 -- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb85015a310 msgr2=0x7fb8500ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb854c00640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb85015a310 0x7fb8500ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb854c00640 1 -- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8500ade00 msgr2=0x7fb8501fb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:58.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb854c00640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8500ade00 0x7fb8501fb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:58.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb854c00640 1 -- 172.21.15.66:0/4146301032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8501fc330 con 0x7fb8500c4fc0 2024-09-10T22:06:58.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb855600640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8500ade00 0x7fb8501fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:58.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb854c00640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8500c4fc0 0x7fb85014e170 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c430 tx=0x7fb84400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:58.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb84cc00640 1 -- 172.21.15.66:0/4146301032 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844017070 con 0x7fb8500c4fc0 2024-09-10T22:06:58.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb84cc00640 1 -- 172.21.15.66:0/4146301032 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8440053c0 con 0x7fb8500c4fc0 2024-09-10T22:06:58.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb857600640 1 -- 172.21.15.66:0/4146301032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb850157910 con 0x7fb8500c4fc0 2024-09-10T22:06:58.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb84cc00640 1 -- 172.21.15.66:0/4146301032 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844003860 con 0x7fb8500c4fc0 2024-09-10T22:06:58.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb857600640 1 -- 172.21.15.66:0/4146301032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb850157ed0 con 0x7fb8500c4fc0 2024-09-10T22:06:58.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb857600640 1 -- 172.21.15.66:0/4146301032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8500c48b0 con 0x7fb8500c4fc0 2024-09-10T22:06:58.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.886+0000 7fb84cc00640 1 -- 172.21.15.66:0/4146301032 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fb844003a00 con 0x7fb8500c4fc0 2024-09-10T22:06:58.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.890+0000 7fb84cc00640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8300731b0 0x7fb830075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:58.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.890+0000 7fb84cc00640 1 -- 172.21.15.66:0/4146301032 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7fb844091490 con 0x7fb8500c4fc0 2024-09-10T22:06:58.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.890+0000 7fb84fe00640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8300731b0 0x7fb830075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:58.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.890+0000 7fb84fe00640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8300731b0 0x7fb830075670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0b0000 tx=0x7fb83c01c340 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:58.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:58.894+0000 7fb84cc00640 1 -- 172.21.15.66:0/4146301032 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb84405dd10 con 0x7fb8500c4fc0 2024-09-10T22:06:59.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.130+0000 7fb857600640 1 -- 172.21.15.66:0/4146301032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb8500cf7a0 con 0x7fb8500c4fc0 2024-09-10T22:06:59.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.134+0000 7fb84cc00640 1 -- 172.21.15.66:0/4146301032 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v28) ==== 190+0+147 (secure 0 0 0) 0x7fb844062740 con 0x7fb8500c4fc0 2024-09-10T22:06:59.140 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T22:06:59.140 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw== 2024-09-10T22:06:59.140 INFO:teuthology.orchestra.run.smithi066.stdout: caps mds = "allow" 2024-09-10T22:06:59.140 INFO:teuthology.orchestra.run.smithi066.stdout: caps mon = "allow r" 2024-09-10T22:06:59.140 INFO:teuthology.orchestra.run.smithi066.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T22:06:59.140 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-09-10T22:06:59.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.138+0000 7fb857600640 1 -- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8300731b0 msgr2=0x7fb830075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:59.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.138+0000 7fb857600640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8300731b0 0x7fb830075670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0b0000 tx=0x7fb83c01c340 comp rx=0 tx=0).stop 2024-09-10T22:06:59.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.138+0000 7fb857600640 1 -- 172.21.15.66:0/4146301032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8500c4fc0 msgr2=0x7fb85014e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:59.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.138+0000 7fb857600640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8500c4fc0 0x7fb85014e170 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c430 tx=0x7fb84400c8f0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.138+0000 7fb857600640 1 -- 172.21.15.66:0/4146301032 shutdown_connections 2024-09-10T22:06:59.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.138+0000 7fb857600640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8300731b0 0x7fb830075670 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.138+0000 7fb857600640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8500ade00 0x7fb8501fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.138+0000 7fb857600640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb85015a310 0x7fb8500ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.138+0000 7fb857600640 1 --2- 172.21.15.66:0/4146301032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8500c4fc0 0x7fb85014e170 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.138+0000 7fb857600640 1 -- 172.21.15.66:0/4146301032 >> 172.21.15.66:0/4146301032 conn(0x7fb8500cfa30 msgr2=0x7fb85014f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.138+0000 7fb857600640 1 -- 172.21.15.66:0/4146301032 shutdown_connections 2024-09-10T22:06:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.138+0000 7fb857600640 1 -- 172.21.15.66:0/4146301032 wait complete. 2024-09-10T22:06:59.164 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:06:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 -- 172.21.15.66:0/637152720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7c15a310 msgr2=0x7f9c7c0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 --2- 172.21.15.66:0/637152720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7c15a310 0x7f9c7c0c48b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00a950 tx=0x7f9c6c01dae0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 -- 172.21.15.66:0/637152720 shutdown_connections 2024-09-10T22:06:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 --2- 172.21.15.66:0/637152720 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c7c0aeec0 0x7f9c7c0c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 --2- 172.21.15.66:0/637152720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7c15a310 0x7f9c7c0c48b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 -- 172.21.15.66:0/637152720 >> 172.21.15.66:0/637152720 conn(0x7f9c7c0cfa30 msgr2=0x7f9c7c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 -- 172.21.15.66:0/637152720 shutdown_connections 2024-09-10T22:06:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 -- 172.21.15.66:0/637152720 wait complete. 2024-09-10T22:06:59.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 Processor -- start 2024-09-10T22:06:59.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 -- start start 2024-09-10T22:06:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c7c0aeec0 0x7f9c7c0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7c15a310 0x7f9c7c0c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c7c0c7230 0x7f9c7c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c167300 con 0x7f9c7c15a310 2024-09-10T22:06:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c15ec90 con 0x7f9c7c0c7230 2024-09-10T22:06:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c83800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7c167480 con 0x7f9c7c0aeec0 2024-09-10T22:06:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.306+0000 7f9c73e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7c15a310 0x7f9c7c0c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c73e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7c15a310 0x7f9c7c0c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42728/0 (socket says 172.21.15.66:42728) 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c73e00640 1 -- 172.21.15.66:0/2182328342 learned_addr learned my addr 172.21.15.66:0/2182328342 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c81800640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c7c0c7230 0x7f9c7c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c80e00640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c7c0aeec0 0x7f9c7c0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c73e00640 1 -- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c7c0aeec0 msgr2=0x7f9c7c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c73e00640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c7c0aeec0 0x7f9c7c0adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c73e00640 1 -- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c7c0c7230 msgr2=0x7f9c7c0b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c73e00640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c7c0c7230 0x7f9c7c0b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c73e00640 1 -- 172.21.15.66:0/2182328342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c074250 con 0x7f9c7c15a310 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c73e00640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7c15a310 0x7f9c7c0c6cf0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c450 tx=0x7f9c6400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c79600640 1 -- 172.21.15.66:0/2182328342 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64016020 con 0x7f9c7c15a310 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c79600640 1 -- 172.21.15.66:0/2182328342 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c64012920 con 0x7f9c7c15a310 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c79600640 1 -- 172.21.15.66:0/2182328342 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64012d70 con 0x7f9c7c15a310 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c83800640 1 -- 172.21.15.66:0/2182328342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7c074530 con 0x7f9c7c15a310 2024-09-10T22:06:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c83800640 1 -- 172.21.15.66:0/2182328342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7c0b47d0 con 0x7f9c7c15a310 2024-09-10T22:06:59.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c83800640 1 -- 172.21.15.66:0/2182328342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c7c15df60 con 0x7f9c7c15a310 2024-09-10T22:06:59.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.310+0000 7f9c79600640 1 -- 172.21.15.66:0/2182328342 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f9c6401b400 con 0x7f9c7c15a310 2024-09-10T22:06:59.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.314+0000 7f9c79600640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9c580731e0 0x7f9c580756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:59.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.314+0000 7f9c79600640 1 -- 172.21.15.66:0/2182328342 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f9c64092110 con 0x7f9c7c15a310 2024-09-10T22:06:59.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.314+0000 7f9c80e00640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9c580731e0 0x7f9c580756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.314+0000 7f9c80e00640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9c580731e0 0x7f9c580756a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c0b0000 tx=0x7f9c6c01c320 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:59.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.318+0000 7f9c79600640 1 -- 172.21.15.66:0/2182328342 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9c64020020 con 0x7f9c7c15a310 2024-09-10T22:06:59.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.526+0000 7f9c83800640 1 -- 172.21.15.66:0/2182328342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c7c0c5110 con 0x7f9c7c15a310 2024-09-10T22:06:59.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.526+0000 7f9c79600640 1 -- 172.21.15.66:0/2182328342 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5225 (secure 0 0 0) 0x7f9c6405efe0 con 0x7f9c7c15a310 2024-09-10T22:06:59.532 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:06:59.532 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":46,"btime":"2024-09-10T22:06:57:144562+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6112,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3780556367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3780556367},{"type":"v1","addr":"172.21.15.94:6836","nonce":3780556367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1892098851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1892098851},{"type":"v1","addr":"172.21.15.94:6837","nonce":1892098851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1548997357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1548997357},{"type":"v1","addr":"172.21.15.66:6834","nonce":1548997357}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:06:53.520589+0000","modified":"2024-09-10T22:06:56.547020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6390":{"gid":6390,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2415586891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2415586891},{"type":"v1","addr":"172.21.15.66:6835","nonce":2415586891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390]},"id":5}]} 2024-09-10T22:06:59.532 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 46 2024-09-10T22:06:59.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.526+0000 7f9c83800640 1 -- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9c580731e0 msgr2=0x7f9c580756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:59.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.526+0000 7f9c83800640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9c580731e0 0x7f9c580756a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c0b0000 tx=0x7f9c6c01c320 comp rx=0 tx=0).stop 2024-09-10T22:06:59.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.530+0000 7f9c83800640 1 -- 172.21.15.66:0/2182328342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7c15a310 msgr2=0x7f9c7c0c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:59.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.530+0000 7f9c83800640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7c15a310 0x7f9c7c0c6cf0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c450 tx=0x7f9c6400c910 comp rx=0 tx=0).stop 2024-09-10T22:06:59.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.530+0000 7f9c83800640 1 -- 172.21.15.66:0/2182328342 shutdown_connections 2024-09-10T22:06:59.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.530+0000 7f9c83800640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9c580731e0 0x7f9c580756a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.530+0000 7f9c83800640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c7c0c7230 0x7f9c7c0b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.530+0000 7f9c83800640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7c15a310 0x7f9c7c0c6cf0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.530+0000 7f9c83800640 1 --2- 172.21.15.66:0/2182328342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c7c0aeec0 0x7f9c7c0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:59.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.530+0000 7f9c83800640 1 -- 172.21.15.66:0/2182328342 >> 172.21.15.66:0/2182328342 conn(0x7f9c7c0cfa30 msgr2=0x7f9c7c0c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:59.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.530+0000 7f9c83800640 1 -- 172.21.15.66:0/2182328342 shutdown_connections 2024-09-10T22:06:59.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:06:59.530+0000 7f9c83800640 1 -- 172.21.15.66:0/2182328342 wait complete. 2024-09-10T22:06:59.958 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:06:53.520589+0000', 'modified': '2024-09-10T22:06:56.547020+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6390}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6390': {'gid': 6390, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/2415586891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6833', 'nonce': 2415586891}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 2415586891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6390, 'qdb_cluster': [6390]} 2024-09-10T22:06:59.958 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:06:59.958 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T22:07:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.102+0000 7feb47a00640 1 -- 172.21.15.66:0/3951539599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb400c4fc0 msgr2=0x7feb400ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.102+0000 7feb47a00640 1 --2- 172.21.15.66:0/3951539599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb400c4fc0 0x7feb400ca8b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feb3800acd0 tx=0x7feb3801dd80 comp rx=0 tx=0).stop 2024-09-10T22:07:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.102+0000 7feb47a00640 1 -- 172.21.15.66:0/3951539599 shutdown_connections 2024-09-10T22:07:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.102+0000 7feb47a00640 1 --2- 172.21.15.66:0/3951539599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb400c4fc0 0x7feb400ca8b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.102+0000 7feb47a00640 1 -- 172.21.15.66:0/3951539599 >> 172.21.15.66:0/3951539599 conn(0x7feb400cfa30 msgr2=0x7feb400cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.102+0000 7feb47a00640 1 -- 172.21.15.66:0/3951539599 shutdown_connections 2024-09-10T22:07:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.102+0000 7feb47a00640 1 -- 172.21.15.66:0/3951539599 wait complete. 2024-09-10T22:07:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.102+0000 7feb47a00640 1 Processor -- start 2024-09-10T22:07:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.102+0000 7feb47a00640 1 -- start start 2024-09-10T22:07:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb47a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb400c4fc0 0x7feb400b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb47a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb4015a310 0x7feb401500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb47a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb40154390 0x7feb40150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb47a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feb401686a0 con 0x7feb4015a310 2024-09-10T22:07:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb47a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb40168520 con 0x7feb400c4fc0 2024-09-10T22:07:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb47a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7feb40168820 con 0x7feb40154390 2024-09-10T22:07:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb4015a310 0x7feb401500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb4015a310 0x7feb401500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42734/0 (socket says 172.21.15.66:42734) 2024-09-10T22:07:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb3fe00640 1 -- 172.21.15.66:0/1817643289 learned_addr learned my addr 172.21.15.66:0/1817643289 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb45a00640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb40154390 0x7feb40150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb45000640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb400c4fc0 0x7feb400b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb3fe00640 1 -- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb40154390 msgr2=0x7feb40150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb3fe00640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb40154390 0x7feb40150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb3fe00640 1 -- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb400c4fc0 msgr2=0x7feb400b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb3fe00640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb400c4fc0 0x7feb400b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb3fe00640 1 -- 172.21.15.66:0/1817643289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb40156900 con 0x7feb4015a310 2024-09-10T22:07:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb3fe00640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb4015a310 0x7feb401500e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00c4c0 tx=0x7feb2c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb3cc00640 1 -- 172.21.15.66:0/1817643289 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0052d0 con 0x7feb4015a310 2024-09-10T22:07:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb3cc00640 1 -- 172.21.15.66:0/1817643289 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb2c005490 con 0x7feb4015a310 2024-09-10T22:07:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb3cc00640 1 -- 172.21.15.66:0/1817643289 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c012870 con 0x7feb4015a310 2024-09-10T22:07:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb47a00640 1 -- 172.21.15.66:0/1817643289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb401fc190 con 0x7feb4015a310 2024-09-10T22:07:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.106+0000 7feb47a00640 1 -- 172.21.15.66:0/1817643289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb40146780 con 0x7feb4015a310 2024-09-10T22:07:00.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb3cc00640 1 -- 172.21.15.66:0/1817643289 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7feb2c012a10 con 0x7feb4015a310 2024-09-10T22:07:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb3cc00640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feb200731b0 0x7feb20075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb3cc00640 1 -- 172.21.15.66:0/1817643289 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7feb2c091900 con 0x7feb4015a310 2024-09-10T22:07:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb4aeb2000 1 Processor -- start 2024-09-10T22:07:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb4aeb2000 1 -- start start 2024-09-10T22:07:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb45000640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feb200731b0 0x7feb20075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb45000640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feb200731b0 0x7feb20075670 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feb34000c10 tx=0x7feb34001730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:00.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb4aeb2000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85c9d0 0x55ef6e85cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb4aeb2000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85d2f0 0x55ef6e8108d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb4aeb2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e8111f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb4aeb2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ef6e7313f0 con 0x55ef6e810e10 2024-09-10T22:07:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb4aeb2000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ef6e66eb20 con 0x55ef6e85c9d0 2024-09-10T22:07:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb4aeb2000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55ef6e65df50 con 0x55ef6e85d2f0 2024-09-10T22:07:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb45a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e8111f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb45a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e8111f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42748/0 (socket says 172.21.15.66:42748) 2024-09-10T22:07:00.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb45000640 1 --2- 172.21.15.66:0/244732886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85c9d0 0x55ef6e85cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb45a00640 1 -- 172.21.15.66:0/244732886 learned_addr learned my addr 172.21.15.66:0/244732886 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb3fe00640 1 --2- 172.21.15.66:0/244732886 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85d2f0 0x55ef6e8108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb45a00640 1 -- 172.21.15.66:0/244732886 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85d2f0 msgr2=0x55ef6e8108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb45a00640 1 --2- 172.21.15.66:0/244732886 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85d2f0 0x55ef6e8108d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb45a00640 1 -- 172.21.15.66:0/244732886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85c9d0 msgr2=0x55ef6e85cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb45a00640 1 --2- 172.21.15.66:0/244732886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85c9d0 0x55ef6e85cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb45a00640 1 -- 172.21.15.66:0/244732886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6e625590 con 0x55ef6e810e10 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.110+0000 7feb45a00640 1 --2- 172.21.15.66:0/244732886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e8111f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7feb380b1040 tx=0x7feb3801ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bebedf5ad14b1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb31600640 1 -- 172.21.15.66:0/244732886 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb380bc020 con 0x55ef6e810e10 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb31600640 1 -- 172.21.15.66:0/244732886 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb38004d40 con 0x55ef6e810e10 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb31600640 1 -- 172.21.15.66:0/244732886 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb380b3040 con 0x55ef6e810e10 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 -- 172.21.15.66:0/244732886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 msgr2=0x55ef6e8111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/244732886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e8111f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7feb380b1040 tx=0x7feb3801ab20 comp rx=0 tx=0).stop 2024-09-10T22:07:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 -- 172.21.15.66:0/244732886 shutdown_connections 2024-09-10T22:07:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/244732886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e8111f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/244732886 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85d2f0 0x55ef6e8108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/244732886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85c9d0 0x55ef6e85cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 -- 172.21.15.66:0/244732886 >> 172.21.15.66:0/244732886 conn(0x55ef6e6fa130 msgr2=0x55ef6e83fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 -- 172.21.15.66:0/244732886 shutdown_connections 2024-09-10T22:07:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 -- 172.21.15.66:0/244732886 wait complete. 2024-09-10T22:07:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 -- start start 2024-09-10T22:07:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 Processor -- start 2024-09-10T22:07:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e83d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85c9d0 0x55ef6e83de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85d2f0 0x55ef6e83e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ef6e5f85d0 con 0x55ef6e810e10 2024-09-10T22:07:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ef6e661d60 con 0x55ef6e85d2f0 2024-09-10T22:07:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55ef6e604500 con 0x55ef6e85c9d0 2024-09-10T22:07:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 10 client.0 _ll_register_callbacks cb 0x55ef6e608810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:07:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb45000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e83d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb45000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e83d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42756/0 (socket says 172.21.15.66:42756) 2024-09-10T22:07:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb45000640 1 -- 172.21.15.66:0/1811768620 learned_addr learned my addr 172.21.15.66:0/1811768620 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb3fe00640 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85c9d0 0x55ef6e83de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb45a00640 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85d2f0 0x55ef6e83e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb3fe00640 1 -- 172.21.15.66:0/1811768620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85d2f0 msgr2=0x55ef6e83e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb3fe00640 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85d2f0 0x55ef6e83e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb3fe00640 1 -- 172.21.15.66:0/1811768620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 msgr2=0x55ef6e83d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb3fe00640 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e83d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb3fe00640 1 -- 172.21.15.66:0/1811768620 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6e5fc170 con 0x55ef6e85c9d0 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb3fe00640 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85c9d0 0x55ef6e83de70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0917d0 tx=0x7feb2c05dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193a25860c02c643 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb26000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3301/0 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb26000640 1 -- 172.21.15.66:0/1811768620 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c083070 con 0x55ef6e85c9d0 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1811768620 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ef6e5f60f0 con 0x55ef6e85c9d0 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.114+0000 7feb26000640 1 -- 172.21.15.66:0/1811768620 <== mon.2 v2:172.21.15.94:3301/0 2 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7feb2c067300 con 0x55ef6e85c9d0 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb26000640 1 -- 172.21.15.66:0/1811768620 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7feb2c091050 con 0x55ef6e85c9d0 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 10 client.5774 fetch_fsmap learned FSMap version 46 2024-09-10T22:07:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1811768620 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55ef6e6e2ef0 con 0x55ef6e85c9d0 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb26000640 1 -- 172.21.15.66:0/1811768620 <== mon.2 v2:172.21.15.94:3301/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ef6e5f60f0 con 0x55ef6e85c9d0 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb26000640 1 -- 172.21.15.66:0/1811768620 <== mon.2 v2:172.21.15.94:3301/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7feb2c073b80 con 0x55ef6e85c9d0 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 10 client.5774 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 10 client.5774 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 20 client.5774 populate_metadata read hostname 'smithi066' 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.66:6833/2415586891,v1:172.21.15.66:6835/2415586891] conn(0x55ef6e83f0f0 0x55ef6e86ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 4 client.5774 mds_command: new command op to 6390 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1811768620 --> [v2:172.21.15.66:6833/2415586891,v1:172.21.15.66:6835/2415586891] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef6e670a70 con 0x55ef6e83f0f0 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb45000640 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.66:6833/2415586891,v1:172.21.15.66:6835/2415586891] conn(0x55ef6e83f0f0 0x55ef6e86ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb45000640 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.66:6833/2415586891,v1:172.21.15.66:6835/2415586891] conn(0x55ef6e83f0f0 0x55ef6e86ec90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d23f794941898ae server_cookie=6c375a2c1b264bf7 in_seq=0 out_seq=0 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb26000640 10 client.5774 ms_handle_connect on v2:172.21.15.66:6833/2415586891 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb26000640 1 -- 172.21.15.66:0/1811768620 <== mds.0 v2:172.21.15.66:6833/2415586891 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ef6e670a70 con 0x55ef6e83f0f0 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb26000640 10 client.5774 handle_command_reply: tid=0 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 client.5774 shutdown 2024-09-10T22:07:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1811768620 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85c9d0 msgr2=0x55ef6e83de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85c9d0 0x55ef6e83de70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0917d0 tx=0x7feb2c05dd10 comp rx=0 tx=0).stop 2024-09-10T22:07:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1811768620 shutdown_connections 2024-09-10T22:07:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.66:6833/2415586891,v1:172.21.15.66:6835/2415586891] conn(0x55ef6e83f0f0 0x55ef6e86ec90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85d2f0 0x55ef6e83e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85c9d0 0x55ef6e83de70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/1811768620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e83d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb26000640 0 client.5774 ms_handle_reset on v2:172.21.15.66:6833/2415586891 2024-09-10T22:07:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1811768620 >> 172.21.15.66:0/1811768620 conn(0x55ef6e6fa130 msgr2=0x55ef6e80ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1811768620 shutdown_connections 2024-09-10T22:07:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1811768620 wait complete. 2024-09-10T22:07:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.118+0000 7feb4aeb2000 20 client.5774 trim_cache size 0 max 16384 2024-09-10T22:07:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb4aeb2000 1 Processor -- start 2024-09-10T22:07:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb4aeb2000 1 -- start start 2024-09-10T22:07:00.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb4aeb2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e85bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb4aeb2000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85c9d0 0x55ef6e85c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb4aeb2000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85d2f0 0x55ef6e80f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb4aeb2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feb40204480 con 0x55ef6e810e10 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb4aeb2000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb401f8c80 con 0x55ef6e85c9d0 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb4aeb2000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7feb401f7fe0 con 0x55ef6e85d2f0 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb45000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e85bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb45000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e85bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42758/0 (socket says 172.21.15.66:42758) 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb45000640 1 -- 172.21.15.66:0/648319955 learned_addr learned my addr 172.21.15.66:0/648319955 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb3fe00640 1 --2- 172.21.15.66:0/648319955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85c9d0 0x55ef6e85c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb45a00640 1 --2- 172.21.15.66:0/648319955 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85d2f0 0x55ef6e80f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb45000640 1 -- 172.21.15.66:0/648319955 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85d2f0 msgr2=0x55ef6e80f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb45000640 1 --2- 172.21.15.66:0/648319955 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85d2f0 0x55ef6e80f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb45000640 1 -- 172.21.15.66:0/648319955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85c9d0 msgr2=0x55ef6e85c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb45000640 1 --2- 172.21.15.66:0/648319955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85c9d0 0x55ef6e85c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb45000640 1 -- 172.21.15.66:0/648319955 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6e6b9d90 con 0x55ef6e810e10 2024-09-10T22:07:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb45a00640 1 --2- 172.21.15.66:0/648319955 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85d2f0 0x55ef6e80f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:07:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb3fe00640 1 --2- 172.21.15.66:0/648319955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85c9d0 0x55ef6e85c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:07:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb45000640 1 --2- 172.21.15.66:0/648319955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e85bcc0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55ef6e714c20 tx=0x7feb34015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dd92422be7c6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb30c00640 1 -- 172.21.15.66:0/648319955 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3401b040 con 0x55ef6e810e10 2024-09-10T22:07:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb30c00640 1 -- 172.21.15.66:0/648319955 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb3401c040 con 0x55ef6e810e10 2024-09-10T22:07:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.138+0000 7feb30c00640 1 -- 172.21.15.66:0/648319955 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb340236a0 con 0x55ef6e810e10 2024-09-10T22:07:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 -- 172.21.15.66:0/648319955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 msgr2=0x55ef6e85bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/648319955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e85bcc0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55ef6e714c20 tx=0x7feb34015000 comp rx=0 tx=0).stop 2024-09-10T22:07:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 -- 172.21.15.66:0/648319955 shutdown_connections 2024-09-10T22:07:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/648319955 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e85d2f0 0x55ef6e80f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/648319955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85c9d0 0x55ef6e85c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/648319955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e810e10 0x55ef6e85bcc0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 -- 172.21.15.66:0/648319955 >> 172.21.15.66:0/648319955 conn(0x55ef6e6fa130 msgr2=0x55ef6e85b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 -- 172.21.15.66:0/648319955 shutdown_connections 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 -- 172.21.15.66:0/648319955 wait complete. 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 -- start start 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 Processor -- start 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e810e10 0x55ef6e8d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e85c9d0 0x55ef6e8d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85d2f0 0x55ef6e8d3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ef6e666390 con 0x55ef6e85c9d0 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ef6e6e2a60 con 0x55ef6e85d2f0 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55ef6e71d730 con 0x55ef6e810e10 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 10 client.0 _ll_register_callbacks cb 0x7feb2c00c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e85c9d0 0x55ef6e8d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e85c9d0 0x55ef6e8d2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42760/0 (socket says 172.21.15.66:42760) 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb3fe00640 1 -- 172.21.15.66:0/1418145067 learned_addr learned my addr 172.21.15.66:0/1418145067 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb45000640 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e810e10 0x55ef6e8d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb45a00640 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85d2f0 0x55ef6e8d3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb3fe00640 1 -- 172.21.15.66:0/1418145067 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e810e10 msgr2=0x55ef6e8d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb3fe00640 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e810e10 0x55ef6e8d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb3fe00640 1 -- 172.21.15.66:0/1418145067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85d2f0 msgr2=0x55ef6e8d3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb3fe00640 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85d2f0 0x55ef6e8d3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb3fe00640 1 -- 172.21.15.66:0/1418145067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6e624a90 con 0x55ef6e85c9d0 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb3fe00640 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e85c9d0 0x55ef6e8d2f30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7feb2c05de40 tx=0x7feb2c091400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8289da46c68d7d52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb26000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb26000640 1 -- 172.21.15.66:0/1418145067 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c083040 con 0x55ef6e85c9d0 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb26000640 1 -- 172.21.15.66:0/1418145067 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7feb2c067100 con 0x55ef6e85c9d0 2024-09-10T22:07:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1418145067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb3401c040 con 0x55ef6e85c9d0 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb26000640 1 -- 172.21.15.66:0/1418145067 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7feb2c065e60 con 0x55ef6e85c9d0 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 10 client.6525 fetch_fsmap learned FSMap version 46 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1418145067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55ef6e603050 con 0x55ef6e85c9d0 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb26000640 1 -- 172.21.15.66:0/1418145067 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb3401c040 con 0x55ef6e85c9d0 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb26000640 1 -- 172.21.15.66:0/1418145067 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7feb2c05d3a0 con 0x55ef6e85c9d0 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 10 client.6525 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 10 client.6525 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 20 client.6525 populate_metadata read hostname 'smithi066' 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.66:6833/2415586891,v1:172.21.15.66:6835/2415586891] conn(0x55ef6e872100 0x55ef6e8711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 4 client.6525 mds_command: new command op to 6390 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.142+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1418145067 --> [v2:172.21.15.66:6833/2415586891,v1:172.21.15.66:6835/2415586891] -- command(tid 0: {"prefix": "status"}) -- 0x55ef6e66c520 con 0x55ef6e872100 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb45000640 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.66:6833/2415586891,v1:172.21.15.66:6835/2415586891] conn(0x55ef6e872100 0x55ef6e8711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb45000640 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.66:6833/2415586891,v1:172.21.15.66:6835/2415586891] conn(0x55ef6e872100 0x55ef6e8711c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ccbb4c4706a09a server_cookie=2484fc519b8f4cd1 in_seq=0 out_seq=0 2024-09-10T22:07:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb26000640 10 client.6525 ms_handle_connect on v2:172.21.15.66:6833/2415586891 2024-09-10T22:07:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb26000640 1 -- 172.21.15.66:0/1418145067 <== mds.0 v2:172.21.15.66:6833/2415586891 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55ef6e66c520 con 0x55ef6e872100 2024-09-10T22:07:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb26000640 10 client.6525 handle_command_reply: tid=0 2024-09-10T22:07:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb4aeb2000 1 client.6525 shutdown 2024-09-10T22:07:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1418145067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e85c9d0 msgr2=0x55ef6e8d2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e85c9d0 0x55ef6e8d2f30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7feb2c05de40 tx=0x7feb2c091400 comp rx=0 tx=0).stop 2024-09-10T22:07:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1418145067 shutdown_connections 2024-09-10T22:07:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.66:6833/2415586891,v1:172.21.15.66:6835/2415586891] conn(0x55ef6e872100 0x55ef6e8711c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef6e85d2f0 0x55ef6e8d3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef6e85c9d0 0x55ef6e8d2f30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb26000640 0 client.6525 ms_handle_reset on v2:172.21.15.66:6833/2415586891 2024-09-10T22:07:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb4aeb2000 1 --2- 172.21.15.66:0/1418145067 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ef6e810e10 0x55ef6e8d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1418145067 >> 172.21.15.66:0/1418145067 conn(0x55ef6e6fa130 msgr2=0x55ef6e80f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1418145067 shutdown_connections 2024-09-10T22:07:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb4aeb2000 1 -- 172.21.15.66:0/1418145067 wait complete. 2024-09-10T22:07:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb4aeb2000 20 client.6525 trim_cache size 0 max 16384 2024-09-10T22:07:00.156 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:07:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:07:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:07:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 6390, 2024-09-10T22:07:00.156 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:07:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:07:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:07:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 6.0340211909999999, 2024-09-10T22:07:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 46, 2024-09-10T22:07:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 71, 2024-09-10T22:07:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 71, 2024-09-10T22:07:00.157 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 10.471807681 2024-09-10T22:07:00.157 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:07:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb47a00640 1 -- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feb200731b0 msgr2=0x7feb20075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb47a00640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feb200731b0 0x7feb20075670 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feb34000c10 tx=0x7feb34001730 comp rx=0 tx=0).stop 2024-09-10T22:07:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb47a00640 1 -- 172.21.15.66:0/1817643289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb4015a310 msgr2=0x7feb401500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb47a00640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb4015a310 0x7feb401500e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00c4c0 tx=0x7feb2c00c980 comp rx=0 tx=0).stop 2024-09-10T22:07:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb47a00640 1 -- 172.21.15.66:0/1817643289 shutdown_connections 2024-09-10T22:07:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb47a00640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feb200731b0 0x7feb20075670 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb47a00640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb40154390 0x7feb40150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb47a00640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb4015a310 0x7feb401500e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb47a00640 1 --2- 172.21.15.66:0/1817643289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb400c4fc0 0x7feb400b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb47a00640 1 -- 172.21.15.66:0/1817643289 >> 172.21.15.66:0/1817643289 conn(0x7feb400cfa30 msgr2=0x7feb400725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:00.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb47a00640 1 -- 172.21.15.66:0/1817643289 shutdown_connections 2024-09-10T22:07:00.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.146+0000 7feb47a00640 1 -- 172.21.15.66:0/1817643289 wait complete. 2024-09-10T22:07:00.171 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:07:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.314+0000 7fec30c00640 1 -- 172.21.15.66:0/741632010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec2c06d630 msgr2=0x7fec2c160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.314+0000 7fec30c00640 1 --2- 172.21.15.66:0/741632010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec2c06d630 0x7fec2c160370 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fec24009c80 tx=0x7fec2c15d120 comp rx=0 tx=0).stop 2024-09-10T22:07:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.314+0000 7fec30c00640 1 -- 172.21.15.66:0/741632010 shutdown_connections 2024-09-10T22:07:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.314+0000 7fec30c00640 1 --2- 172.21.15.66:0/741632010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec2c06d630 0x7fec2c160370 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.314+0000 7fec30c00640 1 -- 172.21.15.66:0/741632010 >> 172.21.15.66:0/741632010 conn(0x7fec2c0d0b30 msgr2=0x7fec2c0d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.314+0000 7fec30c00640 1 -- 172.21.15.66:0/741632010 shutdown_connections 2024-09-10T22:07:00.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.314+0000 7fec30c00640 1 -- 172.21.15.66:0/741632010 wait complete. 2024-09-10T22:07:00.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec30c00640 1 Processor -- start 2024-09-10T22:07:00.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec30c00640 1 -- start start 2024-09-10T22:07:00.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec30c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec2c06d630 0x7fec2c1f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec30c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec2c071230 0x7fec2c152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec30c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec2c156430 0x7fec2c15acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec30c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c16a240 con 0x7fec2c156430 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec30c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c16a0c0 con 0x7fec2c071230 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec30c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fec2c16a3c0 con 0x7fec2c06d630 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec2a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec2c156430 0x7fec2c15acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec2a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec2c156430 0x7fec2c15acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42774/0 (socket says 172.21.15.66:42774) 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec2a800640 1 -- 172.21.15.66:0/2813519016 learned_addr learned my addr 172.21.15.66:0/2813519016 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec29e00640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec2c06d630 0x7fec2c1f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec23e00640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec2c071230 0x7fec2c152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec2a800640 1 -- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec2c06d630 msgr2=0x7fec2c1f8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec2a800640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec2c06d630 0x7fec2c1f8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec2a800640 1 -- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec2c071230 msgr2=0x7fec2c152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec2a800640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec2c071230 0x7fec2c152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec2a800640 1 -- 172.21.15.66:0/2813519016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c0cd150 con 0x7fec2c156430 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec23e00640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec2c071230 0x7fec2c152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:07:00.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec2a800640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec2c156430 0x7fec2c15acd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fec24009bd0 tx=0x7fec240a5ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:00.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec22000640 1 -- 172.21.15.66:0/2813519016 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec240b2040 con 0x7fec2c156430 2024-09-10T22:07:00.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec22000640 1 -- 172.21.15.66:0/2813519016 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec24004d40 con 0x7fec2c156430 2024-09-10T22:07:00.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec22000640 1 -- 172.21.15.66:0/2813519016 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24004320 con 0x7fec2c156430 2024-09-10T22:07:00.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec30c00640 1 -- 172.21.15.66:0/2813519016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2c0cb220 con 0x7fec2c156430 2024-09-10T22:07:00.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.318+0000 7fec30c00640 1 -- 172.21.15.66:0/2813519016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2c0cb7e0 con 0x7fec2c156430 2024-09-10T22:07:00.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.322+0000 7fec30c00640 1 -- 172.21.15.66:0/2813519016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2c160370 con 0x7fec2c156430 2024-09-10T22:07:00.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.322+0000 7fec22000640 1 -- 172.21.15.66:0/2813519016 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fec240b5070 con 0x7fec2c156430 2024-09-10T22:07:00.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.322+0000 7fec22000640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fec080731e0 0x7fec080756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:00.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.322+0000 7fec22000640 1 -- 172.21.15.66:0/2813519016 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7fec2412bbf0 con 0x7fec2c156430 2024-09-10T22:07:00.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.322+0000 7fec29e00640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fec080731e0 0x7fec080756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:00.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.322+0000 7fec29e00640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fec080731e0 0x7fec080756a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec1c007fd0 tx=0x7fec1c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:00.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.326+0000 7fec22000640 1 -- 172.21.15.66:0/2813519016 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fec24016990 con 0x7fec2c156430 2024-09-10T22:07:00.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.538+0000 7fec30c00640 1 -- 172.21.15.66:0/2813519016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fec2c0742e0 con 0x7fec2c156430 2024-09-10T22:07:00.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.538+0000 7fec22000640 1 -- 172.21.15.66:0/2813519016 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7fec240f8ac0 con 0x7fec2c156430 2024-09-10T22:07:00.545 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:07:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.542+0000 7fec30c00640 1 -- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fec080731e0 msgr2=0x7fec080756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.542+0000 7fec30c00640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fec080731e0 0x7fec080756a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec1c007fd0 tx=0x7fec1c009040 comp rx=0 tx=0).stop 2024-09-10T22:07:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.542+0000 7fec30c00640 1 -- 172.21.15.66:0/2813519016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec2c156430 msgr2=0x7fec2c15acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.542+0000 7fec30c00640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec2c156430 0x7fec2c15acd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fec24009bd0 tx=0x7fec240a5ac0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.542+0000 7fec30c00640 1 -- 172.21.15.66:0/2813519016 shutdown_connections 2024-09-10T22:07:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.542+0000 7fec30c00640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fec080731e0 0x7fec080756a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.542+0000 7fec30c00640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec2c156430 0x7fec2c15acd0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.542+0000 7fec30c00640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec2c071230 0x7fec2c152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.542+0000 7fec30c00640 1 --2- 172.21.15.66:0/2813519016 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec2c06d630 0x7fec2c1f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:00.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.542+0000 7fec30c00640 1 -- 172.21.15.66:0/2813519016 >> 172.21.15.66:0/2813519016 conn(0x7fec2c0d0b30 msgr2=0x7fec2c0cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:00.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.542+0000 7fec30c00640 1 -- 172.21.15.66:0/2813519016 shutdown_connections 2024-09-10T22:07:00.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00.542+0000 7fec30c00640 1 -- 172.21.15.66:0/2813519016 wait complete. 2024-09-10T22:07:00.567 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:07:00.567 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:07:00.567 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:07:00.567 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:07:00.567 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:07:00.567 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:07:00.567 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:07:00.567 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:07:00.567 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:07:00.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:00.568 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:07:00.577 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:07:00.577 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:07:00.577 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:07:00.577 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:07:00.577 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:07:00.577 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:07:00.577 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:07:00.577 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:07:00.577 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:07:00.577 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:07:00.577 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:07:00.577 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42054sec preferred_lft 42054sec 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:81:fc:4c:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:07:00.578 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T22:07:00.579 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:07:00.579 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:07:00.579 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T22:07:00.579 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:07:00.579 INFO:teuthology.orchestra.run.smithi066.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:07:00.579 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:07:00.579 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T22:07:00.579 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:07:00.579 INFO:teuthology.orchestra.run.smithi066.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:07:00.579 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:07:00.579 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T22:07:00.579 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:07:00.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:00.581 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:07:00.639 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:07:00.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:00.641 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:07:00.697 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:07:00.698 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:07:00.698 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:07:00.698 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:07:00.698 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:07:00.698 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:07:00.698 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:07:00.698 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:07:00.698 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:07:00.698 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:07:00.698 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:07:00.698 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:07:00.699 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:07:00.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:07:00.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:07:00.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:07:00.699 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:07:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:07:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:07:00.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:00.838 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:07:00.848 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:07:00.848 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:07:00.848 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:07:00.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:00.848 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:07:00.913 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T22:07:00.913 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:07:00.913 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:07:00.965 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:07:00.965 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:07:00.966 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:07:01.016 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:07:01.081 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:07:01.135 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:07:01.136 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:07:01.136 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:07:01.136 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:07:01.136 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:07:01.136 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:07:01.136 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:07:01.136 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:07:01.136 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:07:01.136 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:07:01.136 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:07:01.136 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:07:01.136 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:07:01.137 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:07:01.138 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:07:01.138 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:07:01.138 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:07:01.138 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:07:01.138 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:07:01.138 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:07:01.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:01.139 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:07:01.189 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:07:01.190 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T22:07:01.236 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:07:01.256 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:07:01.307 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:07:01.307 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:07:01.307 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:07:01.307 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:07:01.307 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:07:01.307 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:07:01.307 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:07:01.307 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:07:01.307 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:07:01.307 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:07:01.307 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:07:01.307 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:07:01.308 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:07:01.309 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:07:01.309 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:07:01.309 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:07:01.309 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:07:01.309 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:07:01.309 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:07:01.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:01.310 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:07:01.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-09-10T22:07:01.326+0000 7fa080879580 -1 init, newargv = 0x556a76876c90 newargc=15 2024-09-10T22:07:01.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[42601]: starting ceph client 2024-09-10T22:07:01.338 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[42601]: starting fuse 2024-09-10T22:07:01.339 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-09-10T22:07:01.339 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T22:07:01.339 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-09-10T22:07:01.340 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-09-10T22:07:01.341 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:07:01.341 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-09-10T22:07:01.341 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:07:01.341 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:07:01.341 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:07:01.341 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-09-10T22:07:01.341 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-09-10T22:07:01.341 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-09-10T22:07:01.341 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:07:01.341 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:07:01.341 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-09-10T22:07:01.341 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:07:01.464 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.42601.asok 2024-09-10T22:07:01.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:07:01.475 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42601.asok 2024-09-10T22:07:01.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:01.476 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42601.asok status 2024-09-10T22:07:01.651 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:07:01.651 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-09-10T22:07:01.652 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:07:01.652 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:07:01.652 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-09-10T22:07:01.652 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-09-10T22:07:01.652 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T22:07:01.652 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "42601", 2024-09-10T22:07:01.652 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-09-10T22:07:01.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:01.652 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-09-10T22:07:01.652 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-09-10T22:07:01.652 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 6543, 2024-09-10T22:07:01.652 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 6543 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1267188556 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:07:01.653 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:07:01.654 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1267188556 2024-09-10T22:07:01.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:01.654 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.6543 192.168.0.1:0/1267188556", 2024-09-10T22:07:01.654 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/1267188556", 2024-09-10T22:07:01.654 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-09-10T22:07:01.654 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 46, 2024-09-10T22:07:01.654 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 71, 2024-09-10T22:07:01.654 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:07:01.654 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-09-10T22:07:01.654 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-09-10T22:07:01.654 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:07:01.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:01.667 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-09-10T22:07:01.667 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:07:01.676 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-09-10T22:07:01.676 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T22:07:01.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:01.677 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:07:01.737 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T22:07:01.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.930+0000 7f4571000640 1 -- 172.21.15.66:0/942906275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4550004060 msgr2=0x7f4550004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.930+0000 7f4571000640 1 --2- 172.21.15.66:0/942906275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4550004060 0x7f4550004810 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4558009b60 tx=0x7f455801d280 comp rx=0 tx=0).stop 2024-09-10T22:07:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.930+0000 7f456a800640 1 -- 172.21.15.66:0/942906275 reap_dead start 2024-09-10T22:07:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.930+0000 7f4571000640 1 -- 172.21.15.66:0/942906275 shutdown_connections 2024-09-10T22:07:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.930+0000 7f4571000640 1 -- 172.21.15.66:0/942906275 >> 172.21.15.66:0/942906275 conn(0x7f456c0cfaa0 msgr2=0x7f456c0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.930+0000 7f4571000640 1 -- 172.21.15.66:0/942906275 shutdown_connections 2024-09-10T22:07:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.930+0000 7f4571000640 1 -- 172.21.15.66:0/942906275 wait complete. 2024-09-10T22:07:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4571000640 1 Processor -- start 2024-09-10T22:07:01.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4571000640 1 -- start start 2024-09-10T22:07:01.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4571000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4550004060 0x7f456c148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:01.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4571000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f456c156fd0 0x7f456c0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:01.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4571000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f456c073810 0x7f456c158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:01.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4571000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f456c165a30 con 0x7f456c073810 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4571000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f456c0b46f0 con 0x7f456c156fd0 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4571000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f456c165bb0 con 0x7f4550004060 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f456a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f456c073810 0x7f456c158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f456a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f456c073810 0x7f456c158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42786/0 (socket says 172.21.15.66:42786) 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f456a800640 1 -- 172.21.15.66:0/4076405898 learned_addr learned my addr 172.21.15.66:0/4076405898 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4569400640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f456c156fd0 0x7f456c0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4569e00640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4550004060 0x7f456c148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f456a800640 1 -- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4550004060 msgr2=0x7f456c148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f456a800640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4550004060 0x7f456c148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f456a800640 1 -- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f456c156fd0 msgr2=0x7f456c0732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f456a800640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f456c156fd0 0x7f456c0732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f456a800640 1 -- 172.21.15.66:0/4076405898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c1f7bc0 con 0x7f456c073810 2024-09-10T22:07:01.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4569e00640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4550004060 0x7f456c148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:07:01.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f456a800640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f456c073810 0x7f456c158780 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f456c0cd2e0 tx=0x7f4564011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:01.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4562000640 1 -- 172.21.15.66:0/4076405898 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456400fc60 con 0x7f456c073810 2024-09-10T22:07:01.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4562000640 1 -- 172.21.15.66:0/4076405898 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f456400fe00 con 0x7f456c073810 2024-09-10T22:07:01.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4562000640 1 -- 172.21.15.66:0/4076405898 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564003be0 con 0x7f456c073810 2024-09-10T22:07:01.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4571000640 1 -- 172.21.15.66:0/4076405898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c1f7e40 con 0x7f456c073810 2024-09-10T22:07:01.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4571000640 1 -- 172.21.15.66:0/4076405898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c1f8400 con 0x7f456c073810 2024-09-10T22:07:01.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.934+0000 7f4571000640 1 -- 172.21.15.66:0/4076405898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456c0d07c0 con 0x7f456c073810 2024-09-10T22:07:01.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.938+0000 7f4562000640 1 -- 172.21.15.66:0/4076405898 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f4564019410 con 0x7f456c073810 2024-09-10T22:07:01.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.938+0000 7f4562000640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f454c0731e0 0x7f454c0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:01.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.938+0000 7f4562000640 1 -- 172.21.15.66:0/4076405898 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f4564092600 con 0x7f456c073810 2024-09-10T22:07:01.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.938+0000 7f4569e00640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f454c0731e0 0x7f454c0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:01.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.938+0000 7f4569e00640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f454c0731e0 0x7f454c0756a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f455c001870 tx=0x7f455c001050 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:01.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:01.942+0000 7f4562000640 1 -- 172.21.15.66:0/4076405898 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f456c1f7bc0 con 0x7f456c073810 2024-09-10T22:07:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.154+0000 7f4571000640 1 -- 172.21.15.66:0/4076405898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f456c1629d0 con 0x7f456c073810 2024-09-10T22:07:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.166+0000 7f4562000640 1 -- 172.21.15.66:0/4076405898 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) ==== 136+0+0 (secure 0 0 0) 0x7f456405ee80 con 0x7f456c073810 2024-09-10T22:07:02.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.170+0000 7f4571000640 1 -- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f454c0731e0 msgr2=0x7f454c0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.170+0000 7f4571000640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f454c0731e0 0x7f454c0756a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f455c001870 tx=0x7f455c001050 comp rx=0 tx=0).stop 2024-09-10T22:07:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.170+0000 7f4571000640 1 -- 172.21.15.66:0/4076405898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f456c073810 msgr2=0x7f456c158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.170+0000 7f4571000640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f456c073810 0x7f456c158780 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f456c0cd2e0 tx=0x7f4564011e50 comp rx=0 tx=0).stop 2024-09-10T22:07:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.170+0000 7f4571000640 1 -- 172.21.15.66:0/4076405898 shutdown_connections 2024-09-10T22:07:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.170+0000 7f4571000640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f454c0731e0 0x7f454c0756a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.170+0000 7f4571000640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f456c073810 0x7f456c158780 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.170+0000 7f4571000640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f456c156fd0 0x7f456c0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.170+0000 7f4571000640 1 --2- 172.21.15.66:0/4076405898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4550004060 0x7f456c148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.170+0000 7f4571000640 1 -- 172.21.15.66:0/4076405898 >> 172.21.15.66:0/4076405898 conn(0x7f456c0cfaa0 msgr2=0x7f456c0ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.170+0000 7f4571000640 1 -- 172.21.15.66:0/4076405898 shutdown_connections 2024-09-10T22:07:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.170+0000 7f4571000640 1 -- 172.21.15.66:0/4076405898 wait complete. 2024-09-10T22:07:02.195 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:07:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.338+0000 7fb959000640 1 -- 172.21.15.66:0/2220805725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9380050f0 msgr2=0x7fb938005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.338+0000 7fb959000640 1 --2- 172.21.15.66:0/2220805725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9380050f0 0x7fb938005920 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb940009b60 tx=0x7fb94001d2b0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.338+0000 7fb952800640 1 -- 172.21.15.66:0/2220805725 reap_dead start 2024-09-10T22:07:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.338+0000 7fb959000640 1 -- 172.21.15.66:0/2220805725 shutdown_connections 2024-09-10T22:07:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.338+0000 7fb959000640 1 -- 172.21.15.66:0/2220805725 >> 172.21.15.66:0/2220805725 conn(0x7fb9540cfa30 msgr2=0x7fb9540cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.338+0000 7fb959000640 1 -- 172.21.15.66:0/2220805725 shutdown_connections 2024-09-10T22:07:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.338+0000 7fb959000640 1 -- 172.21.15.66:0/2220805725 wait complete. 2024-09-10T22:07:02.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb959000640 1 Processor -- start 2024-09-10T22:07:02.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb959000640 1 -- start start 2024-09-10T22:07:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb959000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9380050f0 0x7fb954153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb959000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9540c6b00 0x7fb9540c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb959000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9540c8810 0x7fb9540c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb959000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb9540caf70 con 0x7fb9380050f0 2024-09-10T22:07:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb959000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9540cadf0 con 0x7fb9540c6b00 2024-09-10T22:07:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb959000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb9540cb0f0 con 0x7fb9540c8810 2024-09-10T22:07:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb951e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9380050f0 0x7fb954153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb951e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9380050f0 0x7fb954153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42802/0 (socket says 172.21.15.66:42802) 2024-09-10T22:07:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb951e00640 1 -- 172.21.15.66:0/127862933 learned_addr learned my addr 172.21.15.66:0/127862933 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb952800640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9540c8810 0x7fb9540c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb951400640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9540c6b00 0x7fb9540c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb951e00640 1 -- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9540c8810 msgr2=0x7fb9540c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb951e00640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9540c8810 0x7fb9540c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb951e00640 1 -- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9540c6b00 msgr2=0x7fb9540c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb951e00640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9540c6b00 0x7fb9540c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb951e00640 1 -- 172.21.15.66:0/127862933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9540c7580 con 0x7fb9380050f0 2024-09-10T22:07:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb951e00640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9380050f0 0x7fb954153610 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb94400d920 tx=0x7fb94400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb94a000640 1 -- 172.21.15.66:0/127862933 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94400b830 con 0x7fb9380050f0 2024-09-10T22:07:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb94a000640 1 -- 172.21.15.66:0/127862933 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb94400bac0 con 0x7fb9380050f0 2024-09-10T22:07:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb94a000640 1 -- 172.21.15.66:0/127862933 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944003600 con 0x7fb9380050f0 2024-09-10T22:07:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb959000640 1 -- 172.21.15.66:0/127862933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9540c7860 con 0x7fb9380050f0 2024-09-10T22:07:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb959000640 1 -- 172.21.15.66:0/127862933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9540c8310 con 0x7fb9380050f0 2024-09-10T22:07:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.342+0000 7fb959000640 1 -- 172.21.15.66:0/127862933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9540ca1a0 con 0x7fb9380050f0 2024-09-10T22:07:02.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.346+0000 7fb94a000640 1 -- 172.21.15.66:0/127862933 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fb9440037a0 con 0x7fb9380050f0 2024-09-10T22:07:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.346+0000 7fb94a000640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9340731e0 0x7fb9340756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.346+0000 7fb94a000640 1 -- 172.21.15.66:0/127862933 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7fb944092170 con 0x7fb9380050f0 2024-09-10T22:07:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.346+0000 7fb951400640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9340731e0 0x7fb9340756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:02.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.346+0000 7fb951400640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9340731e0 0x7fb9340756a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb9400ae000 tx=0x7fb94001a050 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:02.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.350+0000 7fb94a000640 1 -- 172.21.15.66:0/127862933 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb94405ed10 con 0x7fb9380050f0 2024-09-10T22:07:02.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.554+0000 7fb959000640 1 -- 172.21.15.66:0/127862933 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fb9540724d0 con 0x7fb9340731e0 2024-09-10T22:07:02.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb94a000640 1 -- 172.21.15.66:0/127862933 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fb9540724d0 con 0x7fb9340731e0 2024-09-10T22:07:02.562 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:07:02.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:07:02.562 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:07:02.562 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:07:02.562 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:07:02.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb959000640 1 -- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9340731e0 msgr2=0x7fb9340756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:02.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb959000640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9340731e0 0x7fb9340756a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb9400ae000 tx=0x7fb94001a050 comp rx=0 tx=0).stop 2024-09-10T22:07:02.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb959000640 1 -- 172.21.15.66:0/127862933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9380050f0 msgr2=0x7fb954153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:02.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb959000640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9380050f0 0x7fb954153610 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb94400d920 tx=0x7fb94400dd20 comp rx=0 tx=0).stop 2024-09-10T22:07:02.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb959000640 1 -- 172.21.15.66:0/127862933 shutdown_connections 2024-09-10T22:07:02.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb959000640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9340731e0 0x7fb9340756a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb959000640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9540c8810 0x7fb9540c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb959000640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9540c6b00 0x7fb9540c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb959000640 1 --2- 172.21.15.66:0/127862933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9380050f0 0x7fb954153610 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb959000640 1 -- 172.21.15.66:0/127862933 >> 172.21.15.66:0/127862933 conn(0x7fb9540cfa30 msgr2=0x7fb9540ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:02.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb959000640 1 -- 172.21.15.66:0/127862933 shutdown_connections 2024-09-10T22:07:02.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.558+0000 7fb959000640 1 -- 172.21.15.66:0/127862933 wait complete. 2024-09-10T22:07:02.584 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:07:02.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 -- 172.21.15.66:0/370309514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2c0c4fc0 msgr2=0x7f2a2c0ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:02.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 --2- 172.21.15.66:0/370309514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2c0c4fc0 0x7f2a2c0ca8b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009bf0 tx=0x7f2a2401ca70 comp rx=0 tx=0).stop 2024-09-10T22:07:02.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 -- 172.21.15.66:0/370309514 shutdown_connections 2024-09-10T22:07:02.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 --2- 172.21.15.66:0/370309514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2c0c4fc0 0x7f2a2c0ca8b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 -- 172.21.15.66:0/370309514 >> 172.21.15.66:0/370309514 conn(0x7f2a2c0cfa30 msgr2=0x7f2a2c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:02.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 -- 172.21.15.66:0/370309514 shutdown_connections 2024-09-10T22:07:02.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 -- 172.21.15.66:0/370309514 wait complete. 2024-09-10T22:07:02.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 Processor -- start 2024-09-10T22:07:02.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 -- start start 2024-09-10T22:07:02.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2c0c4fc0 0x7f2a2c150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:02.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a2c15a310 0x7f2a2c153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:02.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a2c14a7c0 0x7f2a2c0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:02.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c1686a0 con 0x7f2a2c15a310 2024-09-10T22:07:02.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c168520 con 0x7f2a2c14a7c0 2024-09-10T22:07:02.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a33a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2c168820 con 0x7f2a2c0c4fc0 2024-09-10T22:07:02.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a2be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a2c15a310 0x7f2a2c153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:02.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a2be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a2c15a310 0x7f2a2c153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46362/0 (socket says 172.21.15.66:46362) 2024-09-10T22:07:02.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a31000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2c0c4fc0 0x7f2a2c150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:02.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a2be00640 1 -- 172.21.15.66:0/2419452839 learned_addr learned my addr 172.21.15.66:0/2419452839 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:02.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.730+0000 7f2a31a00640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a2c14a7c0 0x7f2a2c0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:02.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a2be00640 1 -- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2c0c4fc0 msgr2=0x7f2a2c150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:02.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a2be00640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2c0c4fc0 0x7f2a2c150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a2be00640 1 -- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a2c14a7c0 msgr2=0x7f2a2c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:02.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a2be00640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a2c14a7c0 0x7f2a2c0ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a2be00640 1 -- 172.21.15.66:0/2419452839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2c072750 con 0x7f2a2c15a310 2024-09-10T22:07:02.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a2be00640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a2c15a310 0x7f2a2c153690 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800c730 tx=0x7f2a1800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:02.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a28c00640 1 -- 172.21.15.66:0/2419452839 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18016020 con 0x7f2a2c15a310 2024-09-10T22:07:02.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a28c00640 1 -- 172.21.15.66:0/2419452839 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a1800a8d0 con 0x7f2a2c15a310 2024-09-10T22:07:02.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a28c00640 1 -- 172.21.15.66:0/2419452839 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1800ad20 con 0x7f2a2c15a310 2024-09-10T22:07:02.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a33a00640 1 -- 172.21.15.66:0/2419452839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2c151b40 con 0x7f2a2c15a310 2024-09-10T22:07:02.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a33a00640 1 -- 172.21.15.66:0/2419452839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2c151860 con 0x7f2a2c15a310 2024-09-10T22:07:02.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a33a00640 1 -- 172.21.15.66:0/2419452839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a2c0c48b0 con 0x7f2a2c15a310 2024-09-10T22:07:02.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.734+0000 7f2a28c00640 1 -- 172.21.15.66:0/2419452839 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f2a18012630 con 0x7f2a2c15a310 2024-09-10T22:07:02.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.738+0000 7f2a28c00640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a0c0731e0 0x7f2a0c0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:02.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.738+0000 7f2a28c00640 1 -- 172.21.15.66:0/2419452839 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f2a18092430 con 0x7f2a2c15a310 2024-09-10T22:07:02.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.738+0000 7f2a31000640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a0c0731e0 0x7f2a0c0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:02.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.738+0000 7f2a31000640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a0c0731e0 0x7f2a0c0756a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2a20007fd0 tx=0x7f2a20009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:02.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.742+0000 7f2a28c00640 1 -- 172.21.15.66:0/2419452839 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2a18020020 con 0x7f2a2c15a310 2024-09-10T22:07:02.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.914+0000 7f2a33a00640 1 -- 172.21.15.66:0/2419452839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2a2c0ce750 con 0x7f2a2c15a310 2024-09-10T22:07:02.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a28c00640 1 -- 172.21.15.66:0/2419452839 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7f2a1805ecb0 con 0x7f2a2c15a310 2024-09-10T22:07:02.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a33a00640 1 -- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a0c0731e0 msgr2=0x7f2a0c0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:02.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a33a00640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a0c0731e0 0x7f2a0c0756a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2a20007fd0 tx=0x7f2a20009040 comp rx=0 tx=0).stop 2024-09-10T22:07:02.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a33a00640 1 -- 172.21.15.66:0/2419452839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a2c15a310 msgr2=0x7f2a2c153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:02.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a33a00640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a2c15a310 0x7f2a2c153690 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800c730 tx=0x7f2a1800cbf0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a33a00640 1 -- 172.21.15.66:0/2419452839 shutdown_connections 2024-09-10T22:07:02.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a33a00640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a0c0731e0 0x7f2a0c0756a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a33a00640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a2c14a7c0 0x7f2a2c0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a33a00640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a2c15a310 0x7f2a2c153690 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a33a00640 1 --2- 172.21.15.66:0/2419452839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2c0c4fc0 0x7f2a2c150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:02.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a33a00640 1 -- 172.21.15.66:0/2419452839 >> 172.21.15.66:0/2419452839 conn(0x7f2a2c0cfa30 msgr2=0x7f2a2c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:02.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a33a00640 1 -- 172.21.15.66:0/2419452839 shutdown_connections 2024-09-10T22:07:02.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:02.918+0000 7f2a33a00640 1 -- 172.21.15.66:0/2419452839 wait complete. 2024-09-10T22:07:02.944 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-10T22:07:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.086+0000 7f0f26000640 1 -- 172.21.15.66:0/98203635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f200ae8e0 msgr2=0x7f0f200c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.086+0000 7f0f26000640 1 --2- 172.21.15.66:0/98203635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f200ae8e0 0x7f0f200c48b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0f2006c4f0 tx=0x7f0f1801caf0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.086+0000 7f0f26000640 1 -- 172.21.15.66:0/98203635 shutdown_connections 2024-09-10T22:07:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.086+0000 7f0f26000640 1 --2- 172.21.15.66:0/98203635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f200ae8e0 0x7f0f200c48b0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.086+0000 7f0f26000640 1 --2- 172.21.15.66:0/98203635 >> v2:172.21.15.94:3300/0 conn(0x7f0f200d0550 0x7f0f2015a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.086+0000 7f0f26000640 1 -- 172.21.15.66:0/98203635 >> 172.21.15.66:0/98203635 conn(0x7f0f200cfa30 msgr2=0x7f0f200cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.086+0000 7f0f26000640 1 -- 172.21.15.66:0/98203635 shutdown_connections 2024-09-10T22:07:03.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.086+0000 7f0f26000640 1 -- 172.21.15.66:0/98203635 wait complete. 2024-09-10T22:07:03.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f26000640 1 Processor -- start 2024-09-10T22:07:03.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f26000640 1 -- start start 2024-09-10T22:07:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f26000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f200ae8e0 0x7f0f2014e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f26000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f200aeec0 0x7f0f200b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f26000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f200b5490 0x7f0f200b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f26000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20167ef0 con 0x7f0f200b5490 2024-09-10T22:07:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f26000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20167d70 con 0x7f0f200aeec0 2024-09-10T22:07:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f26000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0f20168070 con 0x7f0f200ae8e0 2024-09-10T22:07:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f1fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f200b5490 0x7f0f200b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f1fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f200b5490 0x7f0f200b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46386/0 (socket says 172.21.15.66:46386) 2024-09-10T22:07:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f1fe00640 1 -- 172.21.15.66:0/1618532226 learned_addr learned my addr 172.21.15.66:0/1618532226 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f1f400640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f200ae8e0 0x7f0f2014e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f17e00640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f200aeec0 0x7f0f200b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f1fe00640 1 -- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f200ae8e0 msgr2=0x7f0f2014e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f1fe00640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f200ae8e0 0x7f0f2014e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f1fe00640 1 -- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f200aeec0 msgr2=0x7f0f200b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f1fe00640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f200aeec0 0x7f0f200b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f1fe00640 1 -- 172.21.15.66:0/1618532226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2014eae0 con 0x7f0f200b5490 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f17e00640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f200aeec0 0x7f0f200b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f1fe00640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f200b5490 0x7f0f200b2df0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0f1801cfd0 tx=0x7f0f18005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f17400640 1 -- 172.21.15.66:0/1618532226 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f180b2040 con 0x7f0f200b5490 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f17400640 1 -- 172.21.15.66:0/1618532226 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f180bb070 con 0x7f0f200b5490 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f17400640 1 -- 172.21.15.66:0/1618532226 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18004530 con 0x7f0f200b5490 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f26000640 1 -- 172.21.15.66:0/1618532226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f20151ac0 con 0x7f0f200b5490 2024-09-10T22:07:03.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.090+0000 7f0f26000640 1 -- 172.21.15.66:0/1618532226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f20152080 con 0x7f0f200b5490 2024-09-10T22:07:03.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.094+0000 7f0f26000640 1 -- 172.21.15.66:0/1618532226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f20070420 con 0x7f0f200b5490 2024-09-10T22:07:03.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.094+0000 7f0f17400640 1 -- 172.21.15.66:0/1618532226 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f0f180b5070 con 0x7f0f200b5490 2024-09-10T22:07:03.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.094+0000 7f0f17400640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0efc0731e0 0x7f0efc0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:03.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.094+0000 7f0f17400640 1 -- 172.21.15.66:0/1618532226 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f0f1812c740 con 0x7f0f200b5490 2024-09-10T22:07:03.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.094+0000 7f0f1f400640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0efc0731e0 0x7f0efc0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:03.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.094+0000 7f0f1f400640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0efc0731e0 0x7f0efc0756a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000bb50 tx=0x7f0f1000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:03.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.098+0000 7f0f17400640 1 -- 172.21.15.66:0/1618532226 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0f18016770 con 0x7f0f200b5490 2024-09-10T22:07:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.270+0000 7f0f26000640 1 -- 172.21.15.66:0/1618532226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f0f200b0990 con 0x7f0f200b5490 2024-09-10T22:07:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.270+0000 7f0f17400640 1 -- 172.21.15.66:0/1618532226 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v37) ==== 116+0+2 (secure 0 0 0) 0x7f0f180f95c0 con 0x7f0f200b5490 2024-09-10T22:07:03.277 INFO:teuthology.orchestra.run.smithi066.stdout:4 2024-09-10T22:07:03.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.274+0000 7f0f26000640 1 -- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0efc0731e0 msgr2=0x7f0efc0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.274+0000 7f0f26000640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0efc0731e0 0x7f0efc0756a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000bb50 tx=0x7f0f1000b2e0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.274+0000 7f0f26000640 1 -- 172.21.15.66:0/1618532226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f200b5490 msgr2=0x7f0f200b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.274+0000 7f0f26000640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f200b5490 0x7f0f200b2df0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0f1801cfd0 tx=0x7f0f18005b00 comp rx=0 tx=0).stop 2024-09-10T22:07:03.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.274+0000 7f0f26000640 1 -- 172.21.15.66:0/1618532226 shutdown_connections 2024-09-10T22:07:03.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.274+0000 7f0f26000640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0efc0731e0 0x7f0efc0756a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.274+0000 7f0f26000640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f200b5490 0x7f0f200b2df0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.274+0000 7f0f26000640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f200aeec0 0x7f0f200b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.274+0000 7f0f26000640 1 --2- 172.21.15.66:0/1618532226 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f200ae8e0 0x7f0f2014e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.274+0000 7f0f26000640 1 -- 172.21.15.66:0/1618532226 >> 172.21.15.66:0/1618532226 conn(0x7f0f200cfa30 msgr2=0x7f0f2006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:03.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.274+0000 7f0f26000640 1 -- 172.21.15.66:0/1618532226 shutdown_connections 2024-09-10T22:07:03.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.274+0000 7f0f26000640 1 -- 172.21.15.66:0/1618532226 wait complete. 2024-09-10T22:07:03.300 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:07:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.446+0000 7fdb35c00640 1 -- 172.21.15.66:0/4139328520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb140051d0 msgr2=0x7fdb1400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.446+0000 7fdb35c00640 1 --2- 172.21.15.66:0/4139328520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb140051d0 0x7fdb1400b9d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800c640 tx=0x7fdb2801f9e0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.446+0000 7fdb35c00640 1 -- 172.21.15.66:0/4139328520 shutdown_connections 2024-09-10T22:07:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.446+0000 7fdb35c00640 1 --2- 172.21.15.66:0/4139328520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb140051d0 0x7fdb1400b9d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.446+0000 7fdb35c00640 1 -- 172.21.15.66:0/4139328520 >> 172.21.15.66:0/4139328520 conn(0x7fdb300d1c50 msgr2=0x7fdb300d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.446+0000 7fdb35c00640 1 -- 172.21.15.66:0/4139328520 shutdown_connections 2024-09-10T22:07:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.446+0000 7fdb35c00640 1 -- 172.21.15.66:0/4139328520 wait complete. 2024-09-10T22:07:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.446+0000 7fdb35c00640 1 Processor -- start 2024-09-10T22:07:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.446+0000 7fdb35c00640 1 -- start start 2024-09-10T22:07:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb35c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb140051d0 0x7fdb3015aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb35c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb300b4a80 0x7fdb300cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb35c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb3015c940 0x7fdb3015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb35c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb300b30d0 con 0x7fdb3015c940 2024-09-10T22:07:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb35c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb300d2a90 con 0x7fdb140051d0 2024-09-10T22:07:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb35c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdb300b3250 con 0x7fdb300b4a80 2024-09-10T22:07:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb2f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb3015c940 0x7fdb3015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb2f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb3015c940 0x7fdb3015cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46410/0 (socket says 172.21.15.66:46410) 2024-09-10T22:07:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb2f400640 1 -- 172.21.15.66:0/7019603 learned_addr learned my addr 172.21.15.66:0/7019603 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb2ea00640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb140051d0 0x7fdb3015aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb2e000640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb300b4a80 0x7fdb300cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb2f400640 1 -- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb300b4a80 msgr2=0x7fdb300cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb2f400640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb300b4a80 0x7fdb300cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb2f400640 1 -- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb140051d0 msgr2=0x7fdb3015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb2f400640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb140051d0 0x7fdb3015aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb2f400640 1 -- 172.21.15.66:0/7019603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb300b4fd0 con 0x7fdb3015c940 2024-09-10T22:07:03.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb2ea00640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb140051d0 0x7fdb3015aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:07:03.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb2f400640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb3015c940 0x7fdb3015cdc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdb28020280 tx=0x7fdb280012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:03.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb26a00640 1 -- 172.21.15.66:0/7019603 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28003980 con 0x7fdb3015c940 2024-09-10T22:07:03.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb26a00640 1 -- 172.21.15.66:0/7019603 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb280b6020 con 0x7fdb3015c940 2024-09-10T22:07:03.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb26a00640 1 -- 172.21.15.66:0/7019603 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb280060d0 con 0x7fdb3015c940 2024-09-10T22:07:03.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb35c00640 1 -- 172.21.15.66:0/7019603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb300b52b0 con 0x7fdb3015c940 2024-09-10T22:07:03.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb35c00640 1 -- 172.21.15.66:0/7019603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb300b5590 con 0x7fdb3015c940 2024-09-10T22:07:03.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb35c00640 1 -- 172.21.15.66:0/7019603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb300cfc30 con 0x7fdb3015c940 2024-09-10T22:07:03.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.450+0000 7fdb26a00640 1 -- 172.21.15.66:0/7019603 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fdb28006270 con 0x7fdb3015c940 2024-09-10T22:07:03.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.454+0000 7fdb26a00640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb100731b0 0x7fdb10075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:03.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.454+0000 7fdb26a00640 1 -- 172.21.15.66:0/7019603 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7fdb2812f2a0 con 0x7fdb3015c940 2024-09-10T22:07:03.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.454+0000 7fdb2ea00640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb100731b0 0x7fdb10075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:03.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.454+0000 7fdb2ea00640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb100731b0 0x7fdb10075670 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdb3009fc80 tx=0x7fdb20006b20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:03.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.454+0000 7fdb26a00640 1 -- 172.21.15.66:0/7019603 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdb280c0070 con 0x7fdb3015c940 2024-09-10T22:07:03.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.630+0000 7fdb35c00640 1 -- 172.21.15.66:0/7019603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fdb30070c50 con 0x7fdb3015c940 2024-09-10T22:07:03.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb26a00640 1 -- 172.21.15.66:0/7019603 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v38)=0 v38) ==== 131+0+0 (secure 0 0 0) 0x7fdb28019790 con 0x7fdb3015c940 2024-09-10T22:07:03.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb35c00640 1 -- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb100731b0 msgr2=0x7fdb10075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb35c00640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb100731b0 0x7fdb10075670 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdb3009fc80 tx=0x7fdb20006b20 comp rx=0 tx=0).stop 2024-09-10T22:07:03.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb35c00640 1 -- 172.21.15.66:0/7019603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb3015c940 msgr2=0x7fdb3015cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb35c00640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb3015c940 0x7fdb3015cdc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdb28020280 tx=0x7fdb280012e0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb35c00640 1 -- 172.21.15.66:0/7019603 shutdown_connections 2024-09-10T22:07:03.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb35c00640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb100731b0 0x7fdb10075670 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb35c00640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb3015c940 0x7fdb3015cdc0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb35c00640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb300b4a80 0x7fdb300cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb35c00640 1 --2- 172.21.15.66:0/7019603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb140051d0 0x7fdb3015aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb35c00640 1 -- 172.21.15.66:0/7019603 >> 172.21.15.66:0/7019603 conn(0x7fdb300d1c50 msgr2=0x7fdb30071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:03.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb35c00640 1 -- 172.21.15.66:0/7019603 shutdown_connections 2024-09-10T22:07:03.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.634+0000 7fdb35c00640 1 -- 172.21.15.66:0/7019603 wait complete. 2024-09-10T22:07:03.655 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:03.656+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:03.655 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:03.656+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:03.655 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:03.656+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:03.656 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:03.656+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:03.656 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:03.656+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:03.659 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-10T22:07:03.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.802+0000 7f3337a00640 1 -- 172.21.15.66:0/520522017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33300c4fc0 msgr2=0x7f33300ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.802+0000 7f3337a00640 1 --2- 172.21.15.66:0/520522017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33300c4fc0 0x7f33300ca6e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3328009bc0 tx=0x7f332801cb10 comp rx=0 tx=0).stop 2024-09-10T22:07:03.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.802+0000 7f3337a00640 1 -- 172.21.15.66:0/520522017 shutdown_connections 2024-09-10T22:07:03.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.802+0000 7f3337a00640 1 --2- 172.21.15.66:0/520522017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33300c4fc0 0x7f33300ca6e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.802+0000 7f3337a00640 1 --2- 172.21.15.66:0/520522017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33300d0550 0x7f33300ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.802+0000 7f3337a00640 1 -- 172.21.15.66:0/520522017 >> 172.21.15.66:0/520522017 conn(0x7f33300cfa30 msgr2=0x7f33300cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:03.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.802+0000 7f3337a00640 1 -- 172.21.15.66:0/520522017 shutdown_connections 2024-09-10T22:07:03.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.802+0000 7f3337a00640 1 -- 172.21.15.66:0/520522017 wait complete. 2024-09-10T22:07:03.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3337a00640 1 Processor -- start 2024-09-10T22:07:03.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3337a00640 1 -- start start 2024-09-10T22:07:03.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3337a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33300c4fc0 0x7f33301f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:03.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3337a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333015a310 0x7f333014cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:03.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3337a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33301f87f0 0x7f333014f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:03.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3337a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3330167480 con 0x7f33300c4fc0 2024-09-10T22:07:03.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3337a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3330167300 con 0x7f333015a310 2024-09-10T22:07:03.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3337a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3330167600 con 0x7f33301f87f0 2024-09-10T22:07:03.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3335000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33300c4fc0 0x7f33301f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:03.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3335000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33300c4fc0 0x7f33301f8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46430/0 (socket says 172.21.15.66:46430) 2024-09-10T22:07:03.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3335000640 1 -- 172.21.15.66:0/2261804982 learned_addr learned my addr 172.21.15.66:0/2261804982 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:03.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3335a00640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33301f87f0 0x7f333014f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:03.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3327e00640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333015a310 0x7f333014cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:03.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3335000640 1 -- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33301f87f0 msgr2=0x7f333014f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3335000640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33301f87f0 0x7f333014f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3335000640 1 -- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333015a310 msgr2=0x7f333014cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:03.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3335000640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333015a310 0x7f333014cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:03.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3335000640 1 -- 172.21.15.66:0/2261804982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33300b59c0 con 0x7f33300c4fc0 2024-09-10T22:07:03.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3327e00640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333015a310 0x7f333014cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:07:03.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3335000640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33300c4fc0 0x7f33301f8f30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f332000c430 tx=0x7f332000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:03.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f332d600640 1 -- 172.21.15.66:0/2261804982 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320011790 con 0x7f33300c4fc0 2024-09-10T22:07:03.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f332d600640 1 -- 172.21.15.66:0/2261804982 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3320017070 con 0x7f33300c4fc0 2024-09-10T22:07:03.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f332d600640 1 -- 172.21.15.66:0/2261804982 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320011ce0 con 0x7f33300c4fc0 2024-09-10T22:07:03.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3337a00640 1 -- 172.21.15.66:0/2261804982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33301f8cf0 con 0x7f33300c4fc0 2024-09-10T22:07:03.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.806+0000 7f3337a00640 1 -- 172.21.15.66:0/2261804982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33300b56e0 con 0x7f33300c4fc0 2024-09-10T22:07:03.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.810+0000 7f3337a00640 1 -- 172.21.15.66:0/2261804982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3330070420 con 0x7f33300c4fc0 2024-09-10T22:07:03.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.810+0000 7f332d600640 1 -- 172.21.15.66:0/2261804982 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f332001c460 con 0x7f33300c4fc0 2024-09-10T22:07:03.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.810+0000 7f332d600640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f330c0731b0 0x7f330c075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:03.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.810+0000 7f332d600640 1 -- 172.21.15.66:0/2261804982 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f3320091be0 con 0x7f33300c4fc0 2024-09-10T22:07:03.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.810+0000 7f3327e00640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f330c0731b0 0x7f330c075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:03.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.810+0000 7f3327e00640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f330c0731b0 0x7f330c075670 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3330164d40 tx=0x7f331800a2f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:03.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.814+0000 7f332d600640 1 -- 172.21.15.66:0/2261804982 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f332005eb70 con 0x7f33300c4fc0 2024-09-10T22:07:03.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.990+0000 7f3337a00640 1 -- 172.21.15.66:0/2261804982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f3330073cc0 con 0x7f33300c4fc0 2024-09-10T22:07:03.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.994+0000 7f332d600640 1 -- 172.21.15.66:0/2261804982 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v39)=0 v39) ==== 136+0+0 (secure 0 0 0) 0x7f332005e470 con 0x7f33300c4fc0 2024-09-10T22:07:04.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.994+0000 7f3337a00640 1 -- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f330c0731b0 msgr2=0x7f330c075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:04.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.994+0000 7f3337a00640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f330c0731b0 0x7f330c075670 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3330164d40 tx=0x7f331800a2f0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.994+0000 7f3337a00640 1 -- 172.21.15.66:0/2261804982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33300c4fc0 msgr2=0x7f33301f8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:04.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.994+0000 7f3337a00640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33300c4fc0 0x7f33301f8f30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f332000c430 tx=0x7f332000c8f0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.994+0000 7f3337a00640 1 -- 172.21.15.66:0/2261804982 shutdown_connections 2024-09-10T22:07:04.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.994+0000 7f3337a00640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f330c0731b0 0x7f330c075670 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.994+0000 7f3337a00640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33301f87f0 0x7f333014f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.994+0000 7f3337a00640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333015a310 0x7f333014cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.994+0000 7f3337a00640 1 --2- 172.21.15.66:0/2261804982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33300c4fc0 0x7f33301f8f30 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.994+0000 7f3337a00640 1 -- 172.21.15.66:0/2261804982 >> 172.21.15.66:0/2261804982 conn(0x7f33300cfa30 msgr2=0x7f3330074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:04.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.998+0000 7f3337a00640 1 -- 172.21.15.66:0/2261804982 shutdown_connections 2024-09-10T22:07:04.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:03.998+0000 7f3337a00640 1 -- 172.21.15.66:0/2261804982 wait complete. 2024-09-10T22:07:04.022 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-10T22:07:04.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.166+0000 7f326da00640 1 -- 172.21.15.66:0/3135684742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3268154b60 msgr2=0x7f326815be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:04.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.166+0000 7f326da00640 1 --2- 172.21.15.66:0/3135684742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3268154b60 0x7f326815be30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3258009840 tx=0x7f325801c790 comp rx=0 tx=0).stop 2024-09-10T22:07:04.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.166+0000 7f326da00640 1 -- 172.21.15.66:0/3135684742 shutdown_connections 2024-09-10T22:07:04.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.166+0000 7f326da00640 1 --2- 172.21.15.66:0/3135684742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3268154b60 0x7f326815be30 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.166+0000 7f326da00640 1 -- 172.21.15.66:0/3135684742 >> 172.21.15.66:0/3135684742 conn(0x7f326806cc20 msgr2=0x7f326806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:04.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.166+0000 7f326da00640 1 -- 172.21.15.66:0/3135684742 shutdown_connections 2024-09-10T22:07:04.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.166+0000 7f326da00640 1 -- 172.21.15.66:0/3135684742 wait complete. 2024-09-10T22:07:04.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f326da00640 1 Processor -- start 2024-09-10T22:07:04.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f326da00640 1 -- start start 2024-09-10T22:07:04.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f326da00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3268073cd0 0x7f32681526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:04.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f326da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3268154b60 0x7f32681426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:04.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f326da00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3268154f40 0x7f326814f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:04.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f326da00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f326806fdb0 con 0x7f3268154b60 2024-09-10T22:07:04.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f326da00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f326806fc30 con 0x7f3268154f40 2024-09-10T22:07:04.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f326da00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f326806ff30 con 0x7f3268073cd0 2024-09-10T22:07:04.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f325fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3268154b60 0x7f32681426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:04.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f325fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3268154b60 0x7f32681426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46448/0 (socket says 172.21.15.66:46448) 2024-09-10T22:07:04.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f325fe00640 1 -- 172.21.15.66:0/3005021180 learned_addr learned my addr 172.21.15.66:0/3005021180 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:04.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f3267e00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3268073cd0 0x7f32681526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f326d000640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3268154f40 0x7f326814f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f325fe00640 1 -- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3268073cd0 msgr2=0x7f32681526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f325fe00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3268073cd0 0x7f32681526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f325fe00640 1 -- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3268154f40 msgr2=0x7f326814f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f325fe00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3268154f40 0x7f326814f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f325fe00640 1 -- 172.21.15.66:0/3005021180 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32680b0360 con 0x7f3268154b60 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f325fe00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3268154b60 0x7f32681426a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f325000c400 tx=0x7f325000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f3264c00640 1 -- 172.21.15.66:0/3005021180 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250011790 con 0x7f3268154b60 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f3264c00640 1 -- 172.21.15.66:0/3005021180 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3250017070 con 0x7f3268154b60 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f326da00640 1 -- 172.21.15.66:0/3005021180 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326814faa0 con 0x7f3268154b60 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f3264c00640 1 -- 172.21.15.66:0/3005021180 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250011cc0 con 0x7f3268154b60 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.170+0000 7f326da00640 1 -- 172.21.15.66:0/3005021180 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3268150060 con 0x7f3268154b60 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.174+0000 7f326da00640 1 -- 172.21.15.66:0/3005021180 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3268150260 con 0x7f3268154b60 2024-09-10T22:07:04.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.174+0000 7f3264c00640 1 -- 172.21.15.66:0/3005021180 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7f325001c460 con 0x7f3268154b60 2024-09-10T22:07:04.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.174+0000 7f3264c00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32440731b0 0x7f3244075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:04.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.174+0000 7f3264c00640 1 -- 172.21.15.66:0/3005021180 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7f3250091c20 con 0x7f3268154b60 2024-09-10T22:07:04.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.174+0000 7f3267e00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32440731b0 0x7f3244075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:04.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.174+0000 7f3267e00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32440731b0 0x7f3244075670 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f32580ae000 tx=0x7f325801afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:04.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.182+0000 7f3264c00640 1 -- 172.21.15.66:0/3005021180 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3268150260 con 0x7f3268154b60 2024-09-10T22:07:04.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.378+0000 7f326da00640 1 -- 172.21.15.66:0/3005021180 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f3268151cc0 con 0x7f32440731b0 2024-09-10T22:07:04.396 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:04.396+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:04.396 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:04.396+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:04.396 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:04.396+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:04.396 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:04.396+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:04.396 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:04.396+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:04.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.422+0000 7f3264c00640 1 -- 172.21.15.66:0/3005021180 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3268151cc0 con 0x7f32440731b0 2024-09-10T22:07:04.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.426+0000 7f326da00640 1 -- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32440731b0 msgr2=0x7f3244075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:04.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.426+0000 7f326da00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32440731b0 0x7f3244075670 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f32580ae000 tx=0x7f325801afd0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.426+0000 7f326da00640 1 -- 172.21.15.66:0/3005021180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3268154b60 msgr2=0x7f32681426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:04.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.426+0000 7f326da00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3268154b60 0x7f32681426a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f325000c400 tx=0x7f325000c8c0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.426+0000 7f326da00640 1 -- 172.21.15.66:0/3005021180 shutdown_connections 2024-09-10T22:07:04.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.426+0000 7f326da00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32440731b0 0x7f3244075670 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.426+0000 7f326da00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3268154f40 0x7f326814f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.426+0000 7f326da00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3268154b60 0x7f32681426a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.426+0000 7f326da00640 1 --2- 172.21.15.66:0/3005021180 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3268073cd0 0x7f32681526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.426+0000 7f326da00640 1 -- 172.21.15.66:0/3005021180 >> 172.21.15.66:0/3005021180 conn(0x7f326806cc20 msgr2=0x7f32681574f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:04.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.426+0000 7f326da00640 1 -- 172.21.15.66:0/3005021180 shutdown_connections 2024-09-10T22:07:04.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.426+0000 7f326da00640 1 -- 172.21.15.66:0/3005021180 wait complete. 2024-09-10T22:07:04.451 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:07:04.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.594+0000 7fb50fc00640 1 -- 172.21.15.66:0/1310362147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5080738d0 msgr2=0x7fb508073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.594+0000 7fb50fc00640 1 --2- 172.21.15.66:0/1310362147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5080738d0 0x7fb508073cb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009900 tx=0x7fb4f801c790 comp rx=0 tx=0).stop 2024-09-10T22:07:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.594+0000 7fb50fc00640 1 -- 172.21.15.66:0/1310362147 shutdown_connections 2024-09-10T22:07:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.594+0000 7fb50fc00640 1 --2- 172.21.15.66:0/1310362147 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb508070b00 0x7fb50815f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.594+0000 7fb50fc00640 1 --2- 172.21.15.66:0/1310362147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5080738d0 0x7fb508073cb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.594+0000 7fb50fc00640 1 -- 172.21.15.66:0/1310362147 >> 172.21.15.66:0/1310362147 conn(0x7fb5080cc900 msgr2=0x7fb5080ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.594+0000 7fb50fc00640 1 -- 172.21.15.66:0/1310362147 shutdown_connections 2024-09-10T22:07:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.594+0000 7fb50fc00640 1 -- 172.21.15.66:0/1310362147 wait complete. 2024-09-10T22:07:04.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50fc00640 1 Processor -- start 2024-09-10T22:07:04.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50fc00640 1 -- start start 2024-09-10T22:07:04.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50fc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb508070b00 0x7fb508153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:04.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50fc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5080738d0 0x7fb50809c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50fc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5080741f0 0x7fb50815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50fc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb5080ae8a0 con 0x7fb5080738d0 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50fc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5080ae720 con 0x7fb5080741f0 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50fc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb5080aea20 con 0x7fb508070b00 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb4ffe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5080738d0 0x7fb50809c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb4ffe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5080738d0 0x7fb50809c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46454/0 (socket says 172.21.15.66:46454) 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb4ffe00640 1 -- 172.21.15.66:0/1594460579 learned_addr learned my addr 172.21.15.66:0/1594460579 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50d200640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb508070b00 0x7fb508153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50dc00640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5080741f0 0x7fb50815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb4ffe00640 1 -- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb508070b00 msgr2=0x7fb508153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb4ffe00640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb508070b00 0x7fb508153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb4ffe00640 1 -- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5080741f0 msgr2=0x7fb50815ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb4ffe00640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5080741f0 0x7fb50815ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb4ffe00640 1 -- 172.21.15.66:0/1594460579 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50809c0e0 con 0x7fb5080738d0 2024-09-10T22:07:04.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50dc00640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5080741f0 0x7fb50815ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:07:04.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb4ffe00640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5080738d0 0x7fb50809c6c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000c460 tx=0x7fb4f000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:04.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb505600640 1 -- 172.21.15.66:0/1594460579 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0017070 con 0x7fb5080738d0 2024-09-10T22:07:04.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb505600640 1 -- 172.21.15.66:0/1594460579 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4f00053c0 con 0x7fb5080738d0 2024-09-10T22:07:04.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb505600640 1 -- 172.21.15.66:0/1594460579 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0003860 con 0x7fb5080738d0 2024-09-10T22:07:04.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50fc00640 1 -- 172.21.15.66:0/1594460579 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50815a8c0 con 0x7fb5080738d0 2024-09-10T22:07:04.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.598+0000 7fb50fc00640 1 -- 172.21.15.66:0/1594460579 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50809be00 con 0x7fb5080738d0 2024-09-10T22:07:04.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.602+0000 7fb50fc00640 1 -- 172.21.15.66:0/1594460579 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb508073cb0 con 0x7fb5080738d0 2024-09-10T22:07:04.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.602+0000 7fb505600640 1 -- 172.21.15.66:0/1594460579 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 97465+0+0 (secure 0 0 0) 0x7fb4f0003a00 con 0x7fb5080738d0 2024-09-10T22:07:04.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.602+0000 7fb505600640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb4e40731b0 0x7fb4e4075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:04.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.602+0000 7fb505600640 1 -- 172.21.15.66:0/1594460579 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5737+0+0 (secure 0 0 0) 0x7fb4f0091440 con 0x7fb5080738d0 2024-09-10T22:07:04.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.602+0000 7fb50d200640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb4e40731b0 0x7fb4e4075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:04.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.606+0000 7fb50d200640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb4e40731b0 0x7fb4e4075670 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80af000 tx=0x7fb4f801afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:04.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.606+0000 7fb505600640 1 -- 172.21.15.66:0/1594460579 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb4f005dcc0 con 0x7fb5080738d0 2024-09-10T22:07:04.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.806+0000 7fb50fc00640 1 -- 172.21.15.66:0/1594460579 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fb50815aa90 con 0x7fb4e40731b0 2024-09-10T22:07:04.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb505600640 1 -- 172.21.15.66:0/1594460579 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fb50815aa90 con 0x7fb4e40731b0 2024-09-10T22:07:04.819 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/33d50645-e72d-4ae5-bd42-6aef7ee1e089 2024-09-10T22:07:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb50fc00640 1 -- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb4e40731b0 msgr2=0x7fb4e4075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb50fc00640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb4e40731b0 0x7fb4e4075670 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80af000 tx=0x7fb4f801afd0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb50fc00640 1 -- 172.21.15.66:0/1594460579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5080738d0 msgr2=0x7fb50809c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb50fc00640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5080738d0 0x7fb50809c6c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000c460 tx=0x7fb4f000c920 comp rx=0 tx=0).stop 2024-09-10T22:07:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb50fc00640 1 -- 172.21.15.66:0/1594460579 shutdown_connections 2024-09-10T22:07:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb50fc00640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb4e40731b0 0x7fb4e4075670 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb50fc00640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5080741f0 0x7fb50815ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb50fc00640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5080738d0 0x7fb50809c6c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb50fc00640 1 --2- 172.21.15.66:0/1594460579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb508070b00 0x7fb508153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:04.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb50fc00640 1 -- 172.21.15.66:0/1594460579 >> 172.21.15.66:0/1594460579 conn(0x7fb5080cc900 msgr2=0x7fb50806e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:04.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb50fc00640 1 -- 172.21.15.66:0/1594460579 shutdown_connections 2024-09-10T22:07:04.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:04.814+0000 7fb50fc00640 1 -- 172.21.15.66:0/1594460579 wait complete. 2024-09-10T22:07:04.840 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/33d50645-e72d-4ae5-bd42-6aef7ee1e089 2024-09-10T22:07:04.840 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:07:04.840 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:07:04.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:04.849 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/33d50645-e72d-4ae5-bd42-6aef7ee1e089/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:07:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:07:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:07:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.14688 s, 150 MB/s 2024-09-10T22:07:12.056 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:07:12.056 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:07:12.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:12.065 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/33d50645-e72d-4ae5-bd42-6aef7ee1e089/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:07:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:07:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:07:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.80466 s, 158 MB/s 2024-09-10T22:07:18.927 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:07:18.927 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:07:18.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:18.936 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/33d50645-e72d-4ae5-bd42-6aef7ee1e089/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:07:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:07:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:07:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.84535 s, 157 MB/s 2024-09-10T22:07:25.836 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-10T22:07:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.982+0000 7fd8b3800640 1 -- 172.21.15.66:0/3863826517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac0cf250 msgr2=0x7fd8ac15f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.982+0000 7fd8b3800640 1 --2- 172.21.15.66:0/3863826517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac0cf250 0x7fd8ac15f620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009c80 tx=0x7fd8a401cce0 comp rx=0 tx=0).stop 2024-09-10T22:07:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.982+0000 7fd8b3800640 1 -- 172.21.15.66:0/3863826517 shutdown_connections 2024-09-10T22:07:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.982+0000 7fd8b3800640 1 --2- 172.21.15.66:0/3863826517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac0cf250 0x7fd8ac15f620 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.982+0000 7fd8b3800640 1 --2- 172.21.15.66:0/3863826517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8ac144ac0 0x7fd8ac06e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.982+0000 7fd8b3800640 1 -- 172.21.15.66:0/3863826517 >> 172.21.15.66:0/3863826517 conn(0x7fd8ac0cfaa0 msgr2=0x7fd8ac0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.982+0000 7fd8b3800640 1 -- 172.21.15.66:0/3863826517 shutdown_connections 2024-09-10T22:07:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.982+0000 7fd8b3800640 1 -- 172.21.15.66:0/3863826517 wait complete. 2024-09-10T22:07:25.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b3800640 1 Processor -- start 2024-09-10T22:07:25.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b3800640 1 -- start start 2024-09-10T22:07:25.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b3800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ac0cf250 0x7fd8ac13e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:25.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b3800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8ac144ac0 0x7fd8ac15d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:25.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b3800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac145050 0x7fd8ac0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:25.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b3800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac168de0 con 0x7fd8ac0cf250 2024-09-10T22:07:25.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b3800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac168c60 con 0x7fd8ac145050 2024-09-10T22:07:25.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b3800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ac168f60 con 0x7fd8ac144ac0 2024-09-10T22:07:25.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ac0cf250 0x7fd8ac13e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:25.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ac0cf250 0x7fd8ac13e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41940/0 (socket says 172.21.15.66:41940) 2024-09-10T22:07:25.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b0e00640 1 -- 172.21.15.66:0/4186343709 learned_addr learned my addr 172.21.15.66:0/4186343709 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:25.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8a3e00640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8ac144ac0 0x7fd8ac15d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:25.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b1800640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac145050 0x7fd8ac0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:25.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b0e00640 1 -- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8ac144ac0 msgr2=0x7fd8ac15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:25.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b0e00640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8ac144ac0 0x7fd8ac15d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:25.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b0e00640 1 -- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac145050 msgr2=0x7fd8ac0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:25.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b0e00640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac145050 0x7fd8ac0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:25.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b0e00640 1 -- 172.21.15.66:0/4186343709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac15c0e0 con 0x7fd8ac0cf250 2024-09-10T22:07:25.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b1800640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac145050 0x7fd8ac0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:07:25.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b0e00640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ac0cf250 0x7fd8ac13e980 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00e9f0 tx=0x7fd89c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:25.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8a9600640 1 -- 172.21.15.66:0/4186343709 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c00cd80 con 0x7fd8ac0cf250 2024-09-10T22:07:25.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8a9600640 1 -- 172.21.15.66:0/4186343709 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd89c010430 con 0x7fd8ac0cf250 2024-09-10T22:07:25.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8a9600640 1 -- 172.21.15.66:0/4186343709 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c010860 con 0x7fd8ac0cf250 2024-09-10T22:07:25.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.986+0000 7fd8b3800640 1 -- 172.21.15.66:0/4186343709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ac13e6e0 con 0x7fd8ac0cf250 2024-09-10T22:07:25.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.990+0000 7fd8b3800640 1 -- 172.21.15.66:0/4186343709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ac15c3a0 con 0x7fd8ac0cf250 2024-09-10T22:07:25.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.990+0000 7fd8b3800640 1 -- 172.21.15.66:0/4186343709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ac15f620 con 0x7fd8ac0cf250 2024-09-10T22:07:25.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.990+0000 7fd8a9600640 1 -- 172.21.15.66:0/4186343709 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 22) ==== 97567+0+0 (secure 0 0 0) 0x7fd89c010a00 con 0x7fd8ac0cf250 2024-09-10T22:07:25.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.990+0000 7fd8a9600640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd8880732d0 0x7fd888075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:25.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.990+0000 7fd8a9600640 1 -- 172.21.15.66:0/4186343709 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fd89c0928a0 con 0x7fd8ac0cf250 2024-09-10T22:07:25.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.990+0000 7fd8a3e00640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd8880732d0 0x7fd888075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:25.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.990+0000 7fd8a3e00640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd8880732d0 0x7fd888075790 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac131ad0 tx=0x7fd894009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:26.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:25.994+0000 7fd8a9600640 1 -- 172.21.15.66:0/4186343709 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd89c05fa90 con 0x7fd8ac0cf250 2024-09-10T22:07:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.202+0000 7fd8b3800640 1 -- 172.21.15.66:0/4186343709 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fd8ac1663e0 con 0x7fd8880732d0 2024-09-10T22:07:26.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.214+0000 7fd8a9600640 1 -- 172.21.15.66:0/4186343709 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd8ac1663e0 con 0x7fd8880732d0 2024-09-10T22:07:26.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.218+0000 7fd8b3800640 1 -- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd8880732d0 msgr2=0x7fd888075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:26.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.218+0000 7fd8b3800640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd8880732d0 0x7fd888075790 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac131ad0 tx=0x7fd894009040 comp rx=0 tx=0).stop 2024-09-10T22:07:26.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.218+0000 7fd8b3800640 1 -- 172.21.15.66:0/4186343709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ac0cf250 msgr2=0x7fd8ac13e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:26.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.218+0000 7fd8b3800640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ac0cf250 0x7fd8ac13e980 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00e9f0 tx=0x7fd89c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.218+0000 7fd8b3800640 1 -- 172.21.15.66:0/4186343709 shutdown_connections 2024-09-10T22:07:26.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.218+0000 7fd8b3800640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd8880732d0 0x7fd888075790 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.218+0000 7fd8b3800640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac145050 0x7fd8ac0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.218+0000 7fd8b3800640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8ac144ac0 0x7fd8ac15d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.218+0000 7fd8b3800640 1 --2- 172.21.15.66:0/4186343709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ac0cf250 0x7fd8ac13e980 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.218+0000 7fd8b3800640 1 -- 172.21.15.66:0/4186343709 >> 172.21.15.66:0/4186343709 conn(0x7fd8ac0cfaa0 msgr2=0x7fd8ac0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:26.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.218+0000 7fd8b3800640 1 -- 172.21.15.66:0/4186343709 shutdown_connections 2024-09-10T22:07:26.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.218+0000 7fd8b3800640 1 -- 172.21.15.66:0/4186343709 wait complete. 2024-09-10T22:07:26.242 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:07:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.386+0000 7fb536800640 1 -- 172.21.15.66:0/2323219784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb530071700 msgr2=0x7fb5300b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.386+0000 7fb536800640 1 --2- 172.21.15.66:0/2323219784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb530071700 0x7fb5300b0410 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb530073480 tx=0x7fb51c01c7a0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.386+0000 7fb536800640 1 -- 172.21.15.66:0/2323219784 shutdown_connections 2024-09-10T22:07:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.386+0000 7fb536800640 1 --2- 172.21.15.66:0/2323219784 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5300b0ef0 0x7fb5300d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.386+0000 7fb536800640 1 --2- 172.21.15.66:0/2323219784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb530071700 0x7fb5300b0410 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.386+0000 7fb536800640 1 --2- 172.21.15.66:0/2323219784 >> v2:172.21.15.94:3300/0 conn(0x7fb530070ac0 0x7fb530070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.386+0000 7fb536800640 1 -- 172.21.15.66:0/2323219784 >> 172.21.15.66:0/2323219784 conn(0x7fb5300d1d00 msgr2=0x7fb5300d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.386+0000 7fb536800640 1 -- 172.21.15.66:0/2323219784 shutdown_connections 2024-09-10T22:07:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.386+0000 7fb536800640 1 -- 172.21.15.66:0/2323219784 wait complete. 2024-09-10T22:07:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.386+0000 7fb536800640 1 Processor -- start 2024-09-10T22:07:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.386+0000 7fb536800640 1 -- start start 2024-09-10T22:07:26.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb536800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb530070ac0 0x7fb53006e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:26.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb536800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb530071700 0x7fb5301f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb536800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5300b0ef0 0x7fb5300c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb536800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb5300af590 con 0x7fb5300b0ef0 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb536800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5300af410 con 0x7fb530071700 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb536800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb5300af710 con 0x7fb530070ac0 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb52fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5300b0ef0 0x7fb5300c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb52fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5300b0ef0 0x7fb5300c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41954/0 (socket says 172.21.15.66:41954) 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb52fe00640 1 -- 172.21.15.66:0/2330268161 learned_addr learned my addr 172.21.15.66:0/2330268161 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb52ea00640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb530071700 0x7fb5301f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb52f400640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb530070ac0 0x7fb53006e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb52fe00640 1 -- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb530070ac0 msgr2=0x7fb53006e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb52fe00640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb530070ac0 0x7fb53006e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb52fe00640 1 -- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb530071700 msgr2=0x7fb5301f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb52fe00640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb530071700 0x7fb5301f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb52fe00640 1 -- 172.21.15.66:0/2330268161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5301f8c30 con 0x7fb5300b0ef0 2024-09-10T22:07:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb52fe00640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5300b0ef0 0x7fb5300c80d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb52800e970 tx=0x7fb52800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb527400640 1 -- 172.21.15.66:0/2330268161 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528004d30 con 0x7fb5300b0ef0 2024-09-10T22:07:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb527400640 1 -- 172.21.15.66:0/2330268161 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb52800cb20 con 0x7fb5300b0ef0 2024-09-10T22:07:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb527400640 1 -- 172.21.15.66:0/2330268161 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528005020 con 0x7fb5300b0ef0 2024-09-10T22:07:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb536800640 1 -- 172.21.15.66:0/2330268161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5300b75e0 con 0x7fb5300b0ef0 2024-09-10T22:07:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb536800640 1 -- 172.21.15.66:0/2330268161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb53006dd30 con 0x7fb5300b0ef0 2024-09-10T22:07:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb536800640 1 -- 172.21.15.66:0/2330268161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb530165410 con 0x7fb5300b0ef0 2024-09-10T22:07:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.390+0000 7fb527400640 1 -- 172.21.15.66:0/2330268161 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 22) ==== 97567+0+0 (secure 0 0 0) 0x7fb52800f040 con 0x7fb5300b0ef0 2024-09-10T22:07:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.394+0000 7fb527400640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb5100732d0 0x7fb510075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.394+0000 7fb527400640 1 -- 172.21.15.66:0/2330268161 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fb5280920f0 con 0x7fb5300b0ef0 2024-09-10T22:07:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.394+0000 7fb52f400640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb5100732d0 0x7fb510075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.394+0000 7fb52f400640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb5100732d0 0x7fb510075790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb530051110 tx=0x7fb52000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.398+0000 7fb527400640 1 -- 172.21.15.66:0/2330268161 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb528016030 con 0x7fb5300b0ef0 2024-09-10T22:07:26.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.598+0000 7fb536800640 1 -- 172.21.15.66:0/2330268161 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fb5300ce2c0 con 0x7fb5100732d0 2024-09-10T22:07:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.602+0000 7fb527400640 1 -- 172.21.15.66:0/2330268161 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fb5300ce2c0 con 0x7fb5100732d0 2024-09-10T22:07:26.607 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/33d50645-e72d-4ae5-bd42-6aef7ee1e089 2024-09-10T22:07:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.602+0000 7fb536800640 1 -- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb5100732d0 msgr2=0x7fb510075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.602+0000 7fb536800640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb5100732d0 0x7fb510075790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb530051110 tx=0x7fb52000b2e0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.602+0000 7fb536800640 1 -- 172.21.15.66:0/2330268161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5300b0ef0 msgr2=0x7fb5300c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.602+0000 7fb536800640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5300b0ef0 0x7fb5300c80d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb52800e970 tx=0x7fb52800ee30 comp rx=0 tx=0).stop 2024-09-10T22:07:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.602+0000 7fb536800640 1 -- 172.21.15.66:0/2330268161 shutdown_connections 2024-09-10T22:07:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.602+0000 7fb536800640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb5100732d0 0x7fb510075790 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.602+0000 7fb536800640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5300b0ef0 0x7fb5300c80d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.602+0000 7fb536800640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb530071700 0x7fb5301f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.602+0000 7fb536800640 1 --2- 172.21.15.66:0/2330268161 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb530070ac0 0x7fb53006e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.602+0000 7fb536800640 1 -- 172.21.15.66:0/2330268161 >> 172.21.15.66:0/2330268161 conn(0x7fb5300d1d00 msgr2=0x7fb5300d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.606+0000 7fb536800640 1 -- 172.21.15.66:0/2330268161 shutdown_connections 2024-09-10T22:07:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:26.606+0000 7fb536800640 1 -- 172.21.15.66:0/2330268161 wait complete. 2024-09-10T22:07:26.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:26.628 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/33d50645-e72d-4ae5-bd42-6aef7ee1e089 2024-09-10T22:07:26.628 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:07:26.636 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:07:26.641 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/33d50645-e72d-4ae5-bd42-6aef7ee1e089 2024-09-10T22:07:26.641 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3138256896" 2024-09-10T22:07:26.641 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:07:26.642 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-09-10T22:07:26.644 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:07:28.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:28.643 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/33d50645-e72d-4ae5-bd42-6aef7ee1e089 2024-09-10T22:07:28.643 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:07:28.659 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/33d50645-e72d-4ae5-bd42-6aef7ee1e089 2024-09-10T22:07:28.659 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3138256896" 2024-09-10T22:07:28.659 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:07:28.660 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-09-10T22:07:30.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:07:30.661 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/33d50645-e72d-4ae5-bd42-6aef7ee1e089 2024-09-10T22:07:30.661 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:07:30.675 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/33d50645-e72d-4ae5-bd42-6aef7ee1e089 2024-09-10T22:07:30.675 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3221225472" 2024-09-10T22:07:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:07:30.676 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-10T22:07:30.677 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone70890 2024-09-10T22:07:30.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.878+0000 7f9d88400640 1 -- 172.21.15.66:0/3029295735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800c4b10 msgr2=0x7f9d8006e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:30.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.878+0000 7f9d88400640 1 --2- 172.21.15.66:0/3029295735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800c4b10 0x7f9d8006e960 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9d800ae5b0 tx=0x7f9d7401c830 comp rx=0 tx=0).stop 2024-09-10T22:07:30.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.878+0000 7f9d88400640 1 -- 172.21.15.66:0/3029295735 shutdown_connections 2024-09-10T22:07:30.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.878+0000 7f9d88400640 1 --2- 172.21.15.66:0/3029295735 >> v2:172.21.15.94:3300/0 conn(0x7f9d800c4ef0 0x7f9d800c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:30.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.878+0000 7f9d88400640 1 --2- 172.21.15.66:0/3029295735 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d800ae620 0x7f9d800725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:30.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.878+0000 7f9d88400640 1 --2- 172.21.15.66:0/3029295735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800c4b10 0x7f9d8006e960 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:30.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.878+0000 7f9d88400640 1 -- 172.21.15.66:0/3029295735 >> 172.21.15.66:0/3029295735 conn(0x7f9d800cfa30 msgr2=0x7f9d800cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:30.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.878+0000 7f9d88400640 1 -- 172.21.15.66:0/3029295735 shutdown_connections 2024-09-10T22:07:30.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.878+0000 7f9d88400640 1 -- 172.21.15.66:0/3029295735 wait complete. 2024-09-10T22:07:30.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d88400640 1 Processor -- start 2024-09-10T22:07:30.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d88400640 1 -- start start 2024-09-10T22:07:30.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d88400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d800ae620 0x7f9d8015b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d88400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800c4b10 0x7f9d8014d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d88400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d800c4ef0 0x7f9d801550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d88400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d80166d80 con 0x7f9d800c4b10 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d88400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9d80166c00 con 0x7f9d800c4ef0 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d88400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9d80166f00 con 0x7f9d800ae620 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d85000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800c4b10 0x7f9d8014d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d86400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d800c4ef0 0x7f9d801550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d85a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d800ae620 0x7f9d8015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d85000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800c4b10 0x7f9d8014d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41974/0 (socket says 172.21.15.66:41974) 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d85000640 1 -- 172.21.15.66:0/3096918268 learned_addr learned my addr 172.21.15.66:0/3096918268 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d85000640 1 -- 172.21.15.66:0/3096918268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d800ae620 msgr2=0x7f9d8015b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d85000640 1 --2- 172.21.15.66:0/3096918268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d800ae620 0x7f9d8015b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d85000640 1 -- 172.21.15.66:0/3096918268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d800c4ef0 msgr2=0x7f9d801550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d85000640 1 --2- 172.21.15.66:0/3096918268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d800c4ef0 0x7f9d801550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d85000640 1 -- 172.21.15.66:0/3096918268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8015b760 con 0x7f9d800c4b10 2024-09-10T22:07:30.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d85000640 1 --2- 172.21.15.66:0/3096918268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800c4b10 0x7f9d8014d260 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00c940 tx=0x7f9d6c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:30.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d7d600640 1 -- 172.21.15.66:0/3096918268 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c007bb0 con 0x7f9d800c4b10 2024-09-10T22:07:30.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d7d600640 1 -- 172.21.15.66:0/3096918268 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d6c007e40 con 0x7f9d800c4b10 2024-09-10T22:07:30.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d7d600640 1 -- 172.21.15.66:0/3096918268 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c005240 con 0x7f9d800c4b10 2024-09-10T22:07:30.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d88400640 1 -- 172.21.15.66:0/3096918268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8014d7a0 con 0x7f9d800c4b10 2024-09-10T22:07:30.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d88400640 1 -- 172.21.15.66:0/3096918268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d800afb20 con 0x7f9d800c4b10 2024-09-10T22:07:30.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.882+0000 7f9d88400640 1 -- 172.21.15.66:0/3096918268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8006e960 con 0x7f9d800c4b10 2024-09-10T22:07:30.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.886+0000 7f9d7d600640 1 -- 172.21.15.66:0/3096918268 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 22) ==== 97567+0+0 (secure 0 0 0) 0x7f9d6c0053e0 con 0x7f9d800c4b10 2024-09-10T22:07:30.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.886+0000 7f9d7d600640 1 --2- 172.21.15.66:0/3096918268 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d600732d0 0x7f9d60075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:30.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.886+0000 7f9d7d600640 1 -- 172.21.15.66:0/3096918268 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f9d6c092130 con 0x7f9d800c4b10 2024-09-10T22:07:30.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.886+0000 7f9d85a00640 1 --2- 172.21.15.66:0/3096918268 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d600732d0 0x7f9d60075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:30.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.886+0000 7f9d85a00640 1 --2- 172.21.15.66:0/3096918268 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d600732d0 0x7f9d60075790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9d740af000 tx=0x7f9d7401b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:30.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:30.890+0000 7f9d7d600640 1 -- 172.21.15.66:0/3096918268 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9d6c014030 con 0x7f9d800c4b10 2024-09-10T22:07:31.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.094+0000 7f9d88400640 1 -- 172.21.15.66:0/3096918268 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone70890", "target": ["mon-mgr", ""]}) -- 0x7f9d80073e40 con 0x7f9d600732d0 2024-09-10T22:07:31.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d7d600640 1 -- 172.21.15.66:0/3096918268 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d80073e40 con 0x7f9d600732d0 2024-09-10T22:07:31.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d88400640 1 -- 172.21.15.66:0/3096918268 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d600732d0 msgr2=0x7f9d60075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:31.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d88400640 1 --2- 172.21.15.66:0/3096918268 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d600732d0 0x7f9d60075790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9d740af000 tx=0x7f9d7401b070 comp rx=0 tx=0).stop 2024-09-10T22:07:31.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d88400640 1 -- 172.21.15.66:0/3096918268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800c4b10 msgr2=0x7f9d8014d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:31.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d88400640 1 --2- 172.21.15.66:0/3096918268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800c4b10 0x7f9d8014d260 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00c940 tx=0x7f9d6c00ce00 comp rx=0 tx=0).stop 2024-09-10T22:07:31.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d88400640 1 -- 172.21.15.66:0/3096918268 shutdown_connections 2024-09-10T22:07:31.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d88400640 1 --2- 172.21.15.66:0/3096918268 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d600732d0 0x7f9d60075790 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:31.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d88400640 1 --2- 172.21.15.66:0/3096918268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d800c4ef0 0x7f9d801550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:31.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d88400640 1 --2- 172.21.15.66:0/3096918268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800c4b10 0x7f9d8014d260 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:31.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d88400640 1 --2- 172.21.15.66:0/3096918268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d800ae620 0x7f9d8015b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:31.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d88400640 1 -- 172.21.15.66:0/3096918268 >> 172.21.15.66:0/3096918268 conn(0x7f9d800cfa30 msgr2=0x7f9d800cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:31.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d88400640 1 -- 172.21.15.66:0/3096918268 shutdown_connections 2024-09-10T22:07:31.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.154+0000 7f9d88400640 1 -- 172.21.15.66:0/3096918268 wait complete. 2024-09-10T22:07:31.164 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:31.164+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:31.165 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:31.164+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:31.165 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:31.164+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:31.165 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:31.164+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:31.165 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:31.164+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:31.181 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone93841 2024-09-10T22:07:31.182 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:31.180+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:31.182 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:31.180+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:31.182 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:31.180+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:31.182 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:31.180+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:31.182 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:31.180+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:31.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.334+0000 7f78cce00640 1 -- 172.21.15.66:0/1247506420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78c815ef20 msgr2=0x7f78c80cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:31.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.334+0000 7f78cce00640 1 --2- 172.21.15.66:0/1247506420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78c815ef20 0x7f78c80cecc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009870 tx=0x7f78b801c820 comp rx=0 tx=0).stop 2024-09-10T22:07:31.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.334+0000 7f78cce00640 1 -- 172.21.15.66:0/1247506420 shutdown_connections 2024-09-10T22:07:31.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.334+0000 7f78cce00640 1 --2- 172.21.15.66:0/1247506420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78c80cd7a0 0x7f78c80cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:31.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.334+0000 7f78cce00640 1 --2- 172.21.15.66:0/1247506420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78c815ef20 0x7f78c80cecc0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:31.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.334+0000 7f78cce00640 1 -- 172.21.15.66:0/1247506420 >> 172.21.15.66:0/1247506420 conn(0x7f78c80d1b10 msgr2=0x7f78c80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:31.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.334+0000 7f78cce00640 1 -- 172.21.15.66:0/1247506420 shutdown_connections 2024-09-10T22:07:31.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.334+0000 7f78cce00640 1 -- 172.21.15.66:0/1247506420 wait complete. 2024-09-10T22:07:31.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78cce00640 1 Processor -- start 2024-09-10T22:07:31.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78cce00640 1 -- start start 2024-09-10T22:07:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78cce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78c80cd7a0 0x7f78c81647c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78cce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78c815ef20 0x7f78c8164f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78cce00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78c815f300 0x7f78c81522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78cce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8168ee0 con 0x7f78c80cd7a0 2024-09-10T22:07:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78cce00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8168d60 con 0x7f78c815ef20 2024-09-10T22:07:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78cce00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f78c8169060 con 0x7f78c815f300 2024-09-10T22:07:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78c7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78c80cd7a0 0x7f78c81647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78c7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78c80cd7a0 0x7f78c81647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41990/0 (socket says 172.21.15.66:41990) 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78c7400640 1 -- 172.21.15.66:0/4012399738 learned_addr learned my addr 172.21.15.66:0/4012399738 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78c7e00640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78c815f300 0x7f78c81522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78bfe00640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78c815ef20 0x7f78c8164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78c7400640 1 -- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78c815f300 msgr2=0x7f78c81522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78c7400640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78c815f300 0x7f78c81522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78c7400640 1 -- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78c815ef20 msgr2=0x7f78c8164f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78c7400640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78c815ef20 0x7f78c8164f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78c7400640 1 -- 172.21.15.66:0/4012399738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c8165480 con 0x7f78c80cd7a0 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78bfe00640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78c815ef20 0x7f78c8164f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78c7400640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78c80cd7a0 0x7f78c81647c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009790 tx=0x7f78b80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78bf400640 1 -- 172.21.15.66:0/4012399738 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b80b2040 con 0x7f78c80cd7a0 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78bf400640 1 -- 172.21.15.66:0/4012399738 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78b8005020 con 0x7f78c80cd7a0 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78bf400640 1 -- 172.21.15.66:0/4012399738 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8005450 con 0x7f78c80cd7a0 2024-09-10T22:07:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78cce00640 1 -- 172.21.15.66:0/4012399738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c8165750 con 0x7f78c80cd7a0 2024-09-10T22:07:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.338+0000 7f78cce00640 1 -- 172.21.15.66:0/4012399738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c8163fc0 con 0x7f78c80cd7a0 2024-09-10T22:07:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.342+0000 7f78cce00640 1 -- 172.21.15.66:0/4012399738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c80cdc30 con 0x7f78c80cd7a0 2024-09-10T22:07:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.342+0000 7f78bf400640 1 -- 172.21.15.66:0/4012399738 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 22) ==== 97567+0+0 (secure 0 0 0) 0x7f78b80b5070 con 0x7f78c80cd7a0 2024-09-10T22:07:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.342+0000 7f78bf400640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f78a40732d0 0x7f78a4075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.342+0000 7f78bf400640 1 -- 172.21.15.66:0/4012399738 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f78b812b4e0 con 0x7f78c80cd7a0 2024-09-10T22:07:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.342+0000 7f78bfe00640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f78a40732d0 0x7f78a4075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:31.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.342+0000 7f78bfe00640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f78a40732d0 0x7f78a4075790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f78c8159260 tx=0x7f78b0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:31.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.346+0000 7f78bf400640 1 -- 172.21.15.66:0/4012399738 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f78b8016410 con 0x7f78c80cd7a0 2024-09-10T22:07:31.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:31.550+0000 7f78cce00640 1 -- 172.21.15.66:0/4012399738 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone93841", "target": ["mon-mgr", ""]}) -- 0x7f78c8072e00 con 0x7f78a40732d0 2024-09-10T22:07:32.164 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:32.164+0000 7f5fc2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:32.164 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:32.164+0000 7f5fc2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:32.164 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:32.164+0000 7f5fc2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:32.165 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:32.164+0000 7f5fc2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:32.165 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:32.164+0000 7f5fc2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:32.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.250+0000 7f78bf400640 1 -- 172.21.15.66:0/4012399738 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f78c8072e00 con 0x7f78a40732d0 2024-09-10T22:07:32.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.254+0000 7f78cce00640 1 -- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f78a40732d0 msgr2=0x7f78a4075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:32.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.254+0000 7f78cce00640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f78a40732d0 0x7f78a4075790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f78c8159260 tx=0x7f78b0009040 comp rx=0 tx=0).stop 2024-09-10T22:07:32.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.254+0000 7f78cce00640 1 -- 172.21.15.66:0/4012399738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78c80cd7a0 msgr2=0x7f78c81647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:32.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.254+0000 7f78cce00640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78c80cd7a0 0x7f78c81647c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009790 tx=0x7f78b80040c0 comp rx=0 tx=0).stop 2024-09-10T22:07:32.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.254+0000 7f78cce00640 1 -- 172.21.15.66:0/4012399738 shutdown_connections 2024-09-10T22:07:32.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.254+0000 7f78cce00640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f78a40732d0 0x7f78a4075790 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f78c8159260 tx=0x7f78b0009040 comp rx=0 tx=0).stop 2024-09-10T22:07:32.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.254+0000 7f78cce00640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78c815f300 0x7f78c81522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:32.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.254+0000 7f78cce00640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78c815ef20 0x7f78c8164f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:32.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.254+0000 7f78cce00640 1 --2- 172.21.15.66:0/4012399738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78c80cd7a0 0x7f78c81647c0 secure :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009790 tx=0x7f78b80040c0 comp rx=0 tx=0).stop 2024-09-10T22:07:32.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.254+0000 7f78cce00640 1 -- 172.21.15.66:0/4012399738 >> 172.21.15.66:0/4012399738 conn(0x7f78c80d1b10 msgr2=0x7f78c806f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:32.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.254+0000 7f78cce00640 1 -- 172.21.15.66:0/4012399738 shutdown_connections 2024-09-10T22:07:32.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.258+0000 7f78cce00640 1 -- 172.21.15.66:0/4012399738 wait complete. 2024-09-10T22:07:32.307 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone33762 2024-09-10T22:07:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 -- 172.21.15.66:0/757790141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847415dd50 msgr2=0x7f84740c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:32.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 --2- 172.21.15.66:0/757790141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847415dd50 0x7f84740c48b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f846800afd0 tx=0x7f846801dee0 comp rx=0 tx=0).stop 2024-09-10T22:07:32.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 -- 172.21.15.66:0/757790141 shutdown_connections 2024-09-10T22:07:32.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 --2- 172.21.15.66:0/757790141 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f847415e130 0x7f84740c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:32.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 --2- 172.21.15.66:0/757790141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847415dd50 0x7f84740c48b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:32.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 -- 172.21.15.66:0/757790141 >> 172.21.15.66:0/757790141 conn(0x7f84740cfa30 msgr2=0x7f84740cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:32.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 -- 172.21.15.66:0/757790141 shutdown_connections 2024-09-10T22:07:32.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 -- 172.21.15.66:0/757790141 wait complete. 2024-09-10T22:07:32.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 Processor -- start 2024-09-10T22:07:32.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 -- start start 2024-09-10T22:07:32.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f847406e630 0x7f847414e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:32.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f847415dd50 0x7f84741500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:32.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847415e130 0x7f8474150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:32.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8474166cf0 con 0x7f847415e130 2024-09-10T22:07:32.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8474166b70 con 0x7f847415dd50 2024-09-10T22:07:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f847b600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8474166e70 con 0x7f847406e630 2024-09-10T22:07:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.538+0000 7f8479600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847415e130 0x7f8474150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8479600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847415e130 0x7f8474150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53074/0 (socket says 172.21.15.66:53074) 2024-09-10T22:07:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8479600640 1 -- 172.21.15.66:0/2914752518 learned_addr learned my addr 172.21.15.66:0/2914752518 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8473e00640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f847415dd50 0x7f84741500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8479600640 1 -- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f847406e630 msgr2=0x7f847414e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8479600640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f847406e630 0x7f847414e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8479600640 1 -- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f847415dd50 msgr2=0x7f84741500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8479600640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f847415dd50 0x7f84741500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8479600640 1 -- 172.21.15.66:0/2914752518 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84740b2dd0 con 0x7f847415e130 2024-09-10T22:07:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8479600640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847415e130 0x7f8474150600 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f846c00cbc0 tx=0x7f846c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8470c00640 1 -- 172.21.15.66:0/2914752518 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c016020 con 0x7f847415e130 2024-09-10T22:07:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8470c00640 1 -- 172.21.15.66:0/2914752518 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f846c00ab40 con 0x7f847415e130 2024-09-10T22:07:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f847b600640 1 -- 172.21.15.66:0/2914752518 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84740b30b0 con 0x7f847415e130 2024-09-10T22:07:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8470c00640 1 -- 172.21.15.66:0/2914752518 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c012420 con 0x7f847415e130 2024-09-10T22:07:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f847b600640 1 -- 172.21.15.66:0/2914752518 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84740b3670 con 0x7f847415e130 2024-09-10T22:07:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f847b600640 1 -- 172.21.15.66:0/2914752518 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84740d0150 con 0x7f847415e130 2024-09-10T22:07:32.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.542+0000 7f8470c00640 1 -- 172.21.15.66:0/2914752518 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 22) ==== 97567+0+0 (secure 0 0 0) 0x7f846c0125c0 con 0x7f847415e130 2024-09-10T22:07:32.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.546+0000 7f8470c00640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84540732d0 0x7f8454075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:32.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.546+0000 7f8470c00640 1 -- 172.21.15.66:0/2914752518 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f846c091e20 con 0x7f847415e130 2024-09-10T22:07:32.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.554+0000 7f8470c00640 1 -- 172.21.15.66:0/2914752518 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f846c020030 con 0x7f847415e130 2024-09-10T22:07:32.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.558+0000 7f8478c00640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84540732d0 0x7f8454075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:32.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.562+0000 7f8478c00640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84540732d0 0x7f8454075790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f84680ae000 tx=0x7f846801a220 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:32.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.578+0000 7f8470c00640 1 -- 172.21.15.66:0/2914752518 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 23) ==== 97720+0+0 (secure 0 0 0) 0x7f846c05ab20 con 0x7f847415e130 2024-09-10T22:07:32.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:32.902+0000 7f847b600640 1 -- 172.21.15.66:0/2914752518 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone33762", "target": ["mon-mgr", ""]}) -- 0x7f8474069850 con 0x7f84540732d0 2024-09-10T22:07:33.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.446+0000 7f8470c00640 1 -- 172.21.15.66:0/2914752518 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8474069850 con 0x7f84540732d0 2024-09-10T22:07:33.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.450+0000 7f847b600640 1 -- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84540732d0 msgr2=0x7f8454075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:33.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.450+0000 7f847b600640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84540732d0 0x7f8454075790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f84680ae000 tx=0x7f846801a220 comp rx=0 tx=0).stop 2024-09-10T22:07:33.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.450+0000 7f847b600640 1 -- 172.21.15.66:0/2914752518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847415e130 msgr2=0x7f8474150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:33.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.450+0000 7f847b600640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847415e130 0x7f8474150600 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f846c00cbc0 tx=0x7f846c00ccb0 comp rx=0 tx=0).stop 2024-09-10T22:07:33.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.450+0000 7f847b600640 1 -- 172.21.15.66:0/2914752518 shutdown_connections 2024-09-10T22:07:33.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.450+0000 7f847b600640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84540732d0 0x7f8454075790 secure :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f84680ae000 tx=0x7f846801a220 comp rx=0 tx=0).stop 2024-09-10T22:07:33.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.450+0000 7f847b600640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847415e130 0x7f8474150600 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:33.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.450+0000 7f847b600640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f847415dd50 0x7f84741500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:33.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.450+0000 7f847b600640 1 --2- 172.21.15.66:0/2914752518 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f847406e630 0x7f847414e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:33.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.450+0000 7f847b600640 1 -- 172.21.15.66:0/2914752518 >> 172.21.15.66:0/2914752518 conn(0x7f84740cfa30 msgr2=0x7f84740738e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:33.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.450+0000 7f847b600640 1 -- 172.21.15.66:0/2914752518 shutdown_connections 2024-09-10T22:07:33.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.450+0000 7f847b600640 1 -- 172.21.15.66:0/2914752518 wait complete. 2024-09-10T22:07:33.486 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:33.484+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:33.486 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:33.484+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:33.486 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:33.484+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:33.487 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:33.484+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:33.487 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:33.484+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:33.500 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone16323 2024-09-10T22:07:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.726+0000 7f3482000640 1 -- 172.21.15.66:0/499429462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3460004810 msgr2=0x7f3460004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.726+0000 7f3482000640 1 --2- 172.21.15.66:0/499429462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3460004810 0x7f3460004c90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3468006f90 tx=0x7f346801cf20 comp rx=0 tx=0).stop 2024-09-10T22:07:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.730+0000 7f3482000640 1 -- 172.21.15.66:0/499429462 shutdown_connections 2024-09-10T22:07:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.730+0000 7f3482000640 1 --2- 172.21.15.66:0/499429462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3460004810 0x7f3460004c90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.730+0000 7f3482000640 1 -- 172.21.15.66:0/499429462 >> 172.21.15.66:0/499429462 conn(0x7f347c0cfa30 msgr2=0x7f347c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.730+0000 7f3482000640 1 -- 172.21.15.66:0/499429462 shutdown_connections 2024-09-10T22:07:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.730+0000 7f3482000640 1 -- 172.21.15.66:0/499429462 wait complete. 2024-09-10T22:07:33.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.730+0000 7f3482000640 1 Processor -- start 2024-09-10T22:07:33.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.730+0000 7f3482000640 1 -- start start 2024-09-10T22:07:33.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f3482000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3460004810 0x7f347c148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:33.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f3482000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f347c0c5940 0x7f347c0c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:33.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f3482000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f347c0c7a50 0x7f347c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:33.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f3482000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f347c1603e0 con 0x7f347c0c5940 2024-09-10T22:07:33.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f3482000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f347c160260 con 0x7f347c0c7a50 2024-09-10T22:07:33.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f3482000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f347c160560 con 0x7f3460004810 2024-09-10T22:07:33.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f347aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f347c0c5940 0x7f347c0c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:33.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f347aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f347c0c5940 0x7f347c0c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53092/0 (socket says 172.21.15.66:53092) 2024-09-10T22:07:33.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f347aa00640 1 -- 172.21.15.66:0/3753273583 learned_addr learned my addr 172.21.15.66:0/3753273583 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:33.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f347aa00640 1 -- 172.21.15.66:0/3753273583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3460004810 msgr2=0x7f347c148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:33.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f347aa00640 1 --2- 172.21.15.66:0/3753273583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3460004810 0x7f347c148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:33.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f347aa00640 1 -- 172.21.15.66:0/3753273583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f347c0c7a50 msgr2=0x7f347c0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:33.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f347aa00640 1 --2- 172.21.15.66:0/3753273583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f347c0c7a50 0x7f347c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:33.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f347aa00640 1 -- 172.21.15.66:0/3753273583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c0c7f60 con 0x7f347c0c5940 2024-09-10T22:07:33.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f347aa00640 1 --2- 172.21.15.66:0/3753273583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f347c0c5940 0x7f347c0c7510 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f346801b620 tx=0x7f3468006d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:33.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.734+0000 7f3473400640 1 -- 172.21.15.66:0/3753273583 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468001500 con 0x7f347c0c5940 2024-09-10T22:07:33.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.738+0000 7f3473400640 1 -- 172.21.15.66:0/3753273583 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34680016a0 con 0x7f347c0c5940 2024-09-10T22:07:33.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.738+0000 7f3482000640 1 -- 172.21.15.66:0/3753273583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347c0c4b80 con 0x7f347c0c5940 2024-09-10T22:07:33.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.738+0000 7f3473400640 1 -- 172.21.15.66:0/3753273583 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468006330 con 0x7f347c0c5940 2024-09-10T22:07:33.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.738+0000 7f3482000640 1 -- 172.21.15.66:0/3753273583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347c0c5140 con 0x7f347c0c5940 2024-09-10T22:07:33.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.738+0000 7f3482000640 1 -- 172.21.15.66:0/3753273583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347c0cdb10 con 0x7f347c0c5940 2024-09-10T22:07:33.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.742+0000 7f3473400640 1 -- 172.21.15.66:0/3753273583 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 97720+0+0 (secure 0 0 0) 0x7f3468003600 con 0x7f347c0c5940 2024-09-10T22:07:33.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.742+0000 7f3473400640 1 --2- 172.21.15.66:0/3753273583 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f345c0734b0 0x7f345c075970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:33.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.742+0000 7f3473400640 1 -- 172.21.15.66:0/3753273583 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f346812ac50 con 0x7f347c0c5940 2024-09-10T22:07:33.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.746+0000 7f3473400640 1 -- 172.21.15.66:0/3753273583 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3468016cd0 con 0x7f347c0c5940 2024-09-10T22:07:33.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.750+0000 7f347b400640 1 --2- 172.21.15.66:0/3753273583 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f345c0734b0 0x7f345c075970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:33.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:33.754+0000 7f347b400640 1 --2- 172.21.15.66:0/3753273583 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f345c0734b0 0x7f345c075970 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f346c005fd0 tx=0x7f346c005c40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:34.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.106+0000 7f3482000640 1 -- 172.21.15.66:0/3753273583 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone16323", "target": ["mon-mgr", ""]}) -- 0x7f347c06f780 con 0x7f345c0734b0 2024-09-10T22:07:34.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.650+0000 7f3473400640 1 -- 172.21.15.66:0/3753273583 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 24) ==== 97771+0+0 (secure 0 0 0) 0x7f34680f3410 con 0x7f347c0c5940 2024-09-10T22:07:34.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.666+0000 7f3473400640 1 -- 172.21.15.66:0/3753273583 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f347c06f780 con 0x7f345c0734b0 2024-09-10T22:07:34.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.670+0000 7f3482000640 1 -- 172.21.15.66:0/3753273583 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f345c0734b0 msgr2=0x7f345c075970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:34.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.670+0000 7f3482000640 1 --2- 172.21.15.66:0/3753273583 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f345c0734b0 0x7f345c075970 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f346c005fd0 tx=0x7f346c005c40 comp rx=0 tx=0).stop 2024-09-10T22:07:34.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.670+0000 7f3482000640 1 -- 172.21.15.66:0/3753273583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f347c0c5940 msgr2=0x7f347c0c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:34.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.670+0000 7f3482000640 1 --2- 172.21.15.66:0/3753273583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f347c0c5940 0x7f347c0c7510 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f346801b620 tx=0x7f3468006d00 comp rx=0 tx=0).stop 2024-09-10T22:07:34.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.670+0000 7f3482000640 1 -- 172.21.15.66:0/3753273583 shutdown_connections 2024-09-10T22:07:34.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.670+0000 7f3482000640 1 --2- 172.21.15.66:0/3753273583 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f345c0734b0 0x7f345c075970 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:34.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.670+0000 7f3482000640 1 --2- 172.21.15.66:0/3753273583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f347c0c7a50 0x7f347c0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:34.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.670+0000 7f3482000640 1 --2- 172.21.15.66:0/3753273583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f347c0c5940 0x7f347c0c7510 secure :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f346801b620 tx=0x7f3468006d00 comp rx=0 tx=0).stop 2024-09-10T22:07:34.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.670+0000 7f3482000640 1 --2- 172.21.15.66:0/3753273583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3460004810 0x7f347c148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:34.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.670+0000 7f3482000640 1 -- 172.21.15.66:0/3753273583 >> 172.21.15.66:0/3753273583 conn(0x7f347c0cfa30 msgr2=0x7f347c0cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:34.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.670+0000 7f3482000640 1 -- 172.21.15.66:0/3753273583 shutdown_connections 2024-09-10T22:07:34.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.670+0000 7f3482000640 1 -- 172.21.15.66:0/3753273583 wait complete. 2024-09-10T22:07:34.701 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone87134 2024-09-10T22:07:34.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.954+0000 7f446bc00640 1 -- 172.21.15.66:0/3037477259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446415dd50 msgr2=0x7f44640c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:34.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.954+0000 7f446bc00640 1 --2- 172.21.15.66:0/3037477259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446415dd50 0x7f44640c48b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f446415b510 tx=0x7f445c01e2e0 comp rx=0 tx=0).stop 2024-09-10T22:07:34.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.954+0000 7f446bc00640 1 -- 172.21.15.66:0/3037477259 shutdown_connections 2024-09-10T22:07:34.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.954+0000 7f446bc00640 1 --2- 172.21.15.66:0/3037477259 >> v2:172.21.15.94:3300/0 conn(0x7f446406ed40 0x7f44640c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:34.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.954+0000 7f446bc00640 1 --2- 172.21.15.66:0/3037477259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446415dd50 0x7f44640c48b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:34.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.954+0000 7f446bc00640 1 -- 172.21.15.66:0/3037477259 >> 172.21.15.66:0/3037477259 conn(0x7f44640cfa30 msgr2=0x7f44640cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:34.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.954+0000 7f446bc00640 1 -- 172.21.15.66:0/3037477259 shutdown_connections 2024-09-10T22:07:34.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.954+0000 7f446bc00640 1 -- 172.21.15.66:0/3037477259 wait complete. 2024-09-10T22:07:34.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.954+0000 7f446bc00640 1 Processor -- start 2024-09-10T22:07:34.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f446bc00640 1 -- start start 2024-09-10T22:07:34.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f446bc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f446406ed40 0x7f44640b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:34.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f446bc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446415dd50 0x7f4464073700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:34.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f446bc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f446415e130 0x7f4464073c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:34.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f446bc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4464166d40 con 0x7f446415dd50 2024-09-10T22:07:34.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f446bc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4464166bc0 con 0x7f446406ed40 2024-09-10T22:07:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f446bc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4464166ec0 con 0x7f446415e130 2024-09-10T22:07:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4463e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446415dd50 0x7f4464073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4463e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446415dd50 0x7f4464073700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53114/0 (socket says 172.21.15.66:53114) 2024-09-10T22:07:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4463e00640 1 -- 172.21.15.66:0/1363915937 learned_addr learned my addr 172.21.15.66:0/1363915937 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4469200640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f446406ed40 0x7f44640b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4469c00640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f446415e130 0x7f4464073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4463e00640 1 -- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f446415e130 msgr2=0x7f4464073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4463e00640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f446415e130 0x7f4464073c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4463e00640 1 -- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f446406ed40 msgr2=0x7f44640b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4463e00640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f446406ed40 0x7f44640b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4463e00640 1 -- 172.21.15.66:0/1363915937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44640b46f0 con 0x7f446415dd50 2024-09-10T22:07:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4463e00640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446415dd50 0x7f4464073700 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f445400b440 tx=0x7f445400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:34.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4460c00640 1 -- 172.21.15.66:0/1363915937 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454015020 con 0x7f446415dd50 2024-09-10T22:07:34.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4460c00640 1 -- 172.21.15.66:0/1363915937 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4454011920 con 0x7f446415dd50 2024-09-10T22:07:34.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f4460c00640 1 -- 172.21.15.66:0/1363915937 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454011d70 con 0x7f446415dd50 2024-09-10T22:07:34.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f446bc00640 1 -- 172.21.15.66:0/1363915937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44641f8c30 con 0x7f446415dd50 2024-09-10T22:07:34.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f446bc00640 1 -- 172.21.15.66:0/1363915937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44641f91f0 con 0x7f446415dd50 2024-09-10T22:07:34.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.958+0000 7f446bc00640 1 -- 172.21.15.66:0/1363915937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44641f93f0 con 0x7f446415dd50 2024-09-10T22:07:34.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.962+0000 7f4460c00640 1 -- 172.21.15.66:0/1363915937 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 24) ==== 97771+0+0 (secure 0 0 0) 0x7f445401a400 con 0x7f446415dd50 2024-09-10T22:07:34.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.962+0000 7f4460c00640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4444073540 0x7f4444075a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:34.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.962+0000 7f4460c00640 1 -- 172.21.15.66:0/1363915937 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f4454091b70 con 0x7f446415dd50 2024-09-10T22:07:34.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.970+0000 7f4460c00640 1 -- 172.21.15.66:0/1363915937 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f44641f93f0 con 0x7f446415dd50 2024-09-10T22:07:34.984 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:34.984+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:34.984 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:34.984+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:34.984 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:34.984+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:34.984 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:34.984+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:34.984 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:34.984+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:34.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.982+0000 7f4469200640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4444073540 0x7f4444075a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:34.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:34.990+0000 7f4469200640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4444073540 0x7f4444075a00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f445c0ae000 tx=0x7f445c01c0c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:35.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:35.314+0000 7f446bc00640 1 -- 172.21.15.66:0/1363915937 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone87134", "target": ["mon-mgr", ""]}) -- 0x7f446406e630 con 0x7f4444073540 2024-09-10T22:07:36.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.174+0000 7f4460c00640 1 -- 172.21.15.66:0/1363915937 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f446406e630 con 0x7f4444073540 2024-09-10T22:07:36.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.178+0000 7f446bc00640 1 -- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4444073540 msgr2=0x7f4444075a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:36.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.178+0000 7f446bc00640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4444073540 0x7f4444075a00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f445c0ae000 tx=0x7f445c01c0c0 comp rx=0 tx=0).stop 2024-09-10T22:07:36.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.178+0000 7f446bc00640 1 -- 172.21.15.66:0/1363915937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446415dd50 msgr2=0x7f4464073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:36.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.178+0000 7f446bc00640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446415dd50 0x7f4464073700 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f445400b440 tx=0x7f445400b840 comp rx=0 tx=0).stop 2024-09-10T22:07:36.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.178+0000 7f446bc00640 1 -- 172.21.15.66:0/1363915937 shutdown_connections 2024-09-10T22:07:36.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.178+0000 7f446bc00640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4444073540 0x7f4444075a00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:36.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.178+0000 7f446bc00640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f446415e130 0x7f4464073c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:36.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.178+0000 7f446bc00640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446415dd50 0x7f4464073700 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:36.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.178+0000 7f446bc00640 1 --2- 172.21.15.66:0/1363915937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f446406ed40 0x7f44640b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:36.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.178+0000 7f446bc00640 1 -- 172.21.15.66:0/1363915937 >> 172.21.15.66:0/1363915937 conn(0x7f44640cfa30 msgr2=0x7f446415be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:36.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.178+0000 7f446bc00640 1 -- 172.21.15.66:0/1363915937 shutdown_connections 2024-09-10T22:07:36.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.182+0000 7f446bc00640 1 -- 172.21.15.66:0/1363915937 wait complete. 2024-09-10T22:07:36.218 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone62795 2024-09-10T22:07:36.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.438+0000 7f6101400640 1 -- 172.21.15.66:0/1109945221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e00052f0 msgr2=0x7f60e000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:36.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.438+0000 7f6101400640 1 --2- 172.21.15.66:0/1109945221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e00052f0 0x7f60e000b9d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f60f4006fd0 tx=0x7f60f401ca00 comp rx=0 tx=0).stop 2024-09-10T22:07:36.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.438+0000 7f6101400640 1 -- 172.21.15.66:0/1109945221 shutdown_connections 2024-09-10T22:07:36.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.438+0000 7f6101400640 1 --2- 172.21.15.66:0/1109945221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e00052f0 0x7f60e000b9d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:36.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.438+0000 7f6101400640 1 -- 172.21.15.66:0/1109945221 >> 172.21.15.66:0/1109945221 conn(0x7f60fc0b26c0 msgr2=0x7f60fc0b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:36.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.438+0000 7f6101400640 1 -- 172.21.15.66:0/1109945221 shutdown_connections 2024-09-10T22:07:36.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.438+0000 7f6101400640 1 -- 172.21.15.66:0/1109945221 wait complete. 2024-09-10T22:07:36.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f6101400640 1 Processor -- start 2024-09-10T22:07:36.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f6101400640 1 -- start start 2024-09-10T22:07:36.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f6101400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e00052f0 0x7f60fc1e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:36.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f6101400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc1e5480 0x7f60fc142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:36.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f6101400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60fc140a10 0x7f60fc140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f6101400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc06f650 con 0x7f60e00052f0 2024-09-10T22:07:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f6101400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc06d2e0 con 0x7f60fc1e5480 2024-09-10T22:07:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f6101400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc06f7d0 con 0x7f60fc140a10 2024-09-10T22:07:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60fa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e00052f0 0x7f60fc1e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60fa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e00052f0 0x7f60fc1e69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53120/0 (socket says 172.21.15.66:53120) 2024-09-10T22:07:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60fa800640 1 -- 172.21.15.66:0/2288630017 learned_addr learned my addr 172.21.15.66:0/2288630017 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60fb200640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60fc140a10 0x7f60fc140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60fa800640 1 -- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60fc140a10 msgr2=0x7f60fc140ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60fa800640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60fc140a10 0x7f60fc140ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60fa800640 1 -- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc1e5480 msgr2=0x7f60fc142d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60fa800640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc1e5480 0x7f60fc142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60fa800640 1 -- 172.21.15.66:0/2288630017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc15d630 con 0x7f60e00052f0 2024-09-10T22:07:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60fa800640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e00052f0 0x7f60fc1e69e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f60ec000f70 tx=0x7f60ec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60f2a00640 1 -- 172.21.15.66:0/2288630017 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec017070 con 0x7f60e00052f0 2024-09-10T22:07:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60f2a00640 1 -- 172.21.15.66:0/2288630017 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60ec00a640 con 0x7f60e00052f0 2024-09-10T22:07:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f6101400640 1 -- 172.21.15.66:0/2288630017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc1e5940 con 0x7f60e00052f0 2024-09-10T22:07:36.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f60f2a00640 1 -- 172.21.15.66:0/2288630017 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00aa90 con 0x7f60e00052f0 2024-09-10T22:07:36.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.442+0000 7f6101400640 1 -- 172.21.15.66:0/2288630017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc1e5f00 con 0x7f60e00052f0 2024-09-10T22:07:36.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.446+0000 7f6101400640 1 -- 172.21.15.66:0/2288630017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc073f40 con 0x7f60e00052f0 2024-09-10T22:07:36.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.446+0000 7f60f2a00640 1 -- 172.21.15.66:0/2288630017 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 24) ==== 97771+0+0 (secure 0 0 0) 0x7f60ec00ac30 con 0x7f60e00052f0 2024-09-10T22:07:36.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.450+0000 7f60f2a00640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f60d8073510 0x7f60d80759d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:36.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.450+0000 7f60f2a00640 1 -- 172.21.15.66:0/2288630017 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f60ec091520 con 0x7f60e00052f0 2024-09-10T22:07:36.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.450+0000 7f60f3e00640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f60d8073510 0x7f60d80759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:36.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.454+0000 7f60f2a00640 1 -- 172.21.15.66:0/2288630017 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f60ec05e060 con 0x7f60e00052f0 2024-09-10T22:07:36.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.454+0000 7f60f3e00640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f60d8073510 0x7f60d80759d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f60fc1e7a30 tx=0x7f60e4006ca0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.750+0000 7f6101400640 1 -- 172.21.15.66:0/2288630017 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone62795", "target": ["mon-mgr", ""]}) -- 0x7f60fc1e6680 con 0x7f60d8073510 2024-09-10T22:07:36.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:36.914+0000 7f60f2a00640 1 -- 172.21.15.66:0/2288630017 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 25) ==== 97822+0+0 (secure 0 0 0) 0x7f60ec05a2d0 con 0x7f60e00052f0 2024-09-10T22:07:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.598+0000 7f60f2a00640 1 -- 172.21.15.66:0/2288630017 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f60fc1e6680 con 0x7f60d8073510 2024-09-10T22:07:37.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.602+0000 7f6101400640 1 -- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f60d8073510 msgr2=0x7f60d80759d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:37.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.602+0000 7f6101400640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f60d8073510 0x7f60d80759d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f60fc1e7a30 tx=0x7f60e4006ca0 comp rx=0 tx=0).stop 2024-09-10T22:07:37.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.602+0000 7f6101400640 1 -- 172.21.15.66:0/2288630017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e00052f0 msgr2=0x7f60fc1e69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:37.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.602+0000 7f6101400640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e00052f0 0x7f60fc1e69e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f60ec000f70 tx=0x7f60ec00cc20 comp rx=0 tx=0).stop 2024-09-10T22:07:37.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.602+0000 7f6101400640 1 -- 172.21.15.66:0/2288630017 shutdown_connections 2024-09-10T22:07:37.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.602+0000 7f6101400640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f60d8073510 0x7f60d80759d0 secure :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f60fc1e7a30 tx=0x7f60e4006ca0 comp rx=0 tx=0).stop 2024-09-10T22:07:37.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.602+0000 7f6101400640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60fc140a10 0x7f60fc140ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:37.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.602+0000 7f6101400640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60fc1e5480 0x7f60fc142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:37.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.602+0000 7f6101400640 1 --2- 172.21.15.66:0/2288630017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e00052f0 0x7f60fc1e69e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:37.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.602+0000 7f6101400640 1 -- 172.21.15.66:0/2288630017 >> 172.21.15.66:0/2288630017 conn(0x7f60fc0b26c0 msgr2=0x7f60fc15d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:37.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.602+0000 7f6101400640 1 -- 172.21.15.66:0/2288630017 shutdown_connections 2024-09-10T22:07:37.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.602+0000 7f6101400640 1 -- 172.21.15.66:0/2288630017 wait complete. 2024-09-10T22:07:37.640 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone17856 2024-09-10T22:07:37.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.934+0000 7f36dac00640 1 -- 172.21.15.66:0/2332031982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36d40ae620 msgr2=0x7f36d40725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:37.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.934+0000 7f36dac00640 1 --2- 172.21.15.66:0/2332031982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36d40ae620 0x7f36d40725d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f36d40cd550 tx=0x7f36bc01c8c0 comp rx=0 tx=0).stop 2024-09-10T22:07:37.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.934+0000 7f36dac00640 1 -- 172.21.15.66:0/2332031982 shutdown_connections 2024-09-10T22:07:37.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.934+0000 7f36dac00640 1 --2- 172.21.15.66:0/2332031982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36d40ae620 0x7f36d40725d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:37.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.934+0000 7f36dac00640 1 --2- 172.21.15.66:0/2332031982 >> v2:172.21.15.94:3300/0 conn(0x7f36d40c4b10 0x7f36d406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:37.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.934+0000 7f36dac00640 1 -- 172.21.15.66:0/2332031982 >> 172.21.15.66:0/2332031982 conn(0x7f36d40cfa30 msgr2=0x7f36d40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:37.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.934+0000 7f36dac00640 1 -- 172.21.15.66:0/2332031982 shutdown_connections 2024-09-10T22:07:37.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.934+0000 7f36dac00640 1 -- 172.21.15.66:0/2332031982 wait complete. 2024-09-10T22:07:37.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36dac00640 1 Processor -- start 2024-09-10T22:07:37.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36dac00640 1 -- start start 2024-09-10T22:07:37.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36dac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36d40ae620 0x7f36d40b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:37.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36dac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d40c4b10 0x7f36d4146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:37.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36dac00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36d40c4ef0 0x7f36d40c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:37.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36dac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4168390 con 0x7f36d40ae620 2024-09-10T22:07:37.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36dac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4168210 con 0x7f36d40c4b10 2024-09-10T22:07:37.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36dac00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f36d4168510 con 0x7f36d40c4ef0 2024-09-10T22:07:37.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36d40ae620 0x7f36d40b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:37.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36d40ae620 0x7f36d40b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53140/0 (socket says 172.21.15.66:53140) 2024-09-10T22:07:37.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36d3e00640 1 -- 172.21.15.66:0/3852912298 learned_addr learned my addr 172.21.15.66:0/3852912298 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:37.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36d3e00640 1 -- 172.21.15.66:0/3852912298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36d40c4ef0 msgr2=0x7f36d40c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:37.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36d3e00640 1 --2- 172.21.15.66:0/3852912298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36d40c4ef0 0x7f36d40c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:37.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36d3e00640 1 -- 172.21.15.66:0/3852912298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d40c4b10 msgr2=0x7f36d4146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:37.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36d3e00640 1 --2- 172.21.15.66:0/3852912298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d40c4b10 0x7f36d4146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:37.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36d3e00640 1 -- 172.21.15.66:0/3852912298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d4074000 con 0x7f36d40ae620 2024-09-10T22:07:37.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36d3e00640 1 --2- 172.21.15.66:0/3852912298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36d40ae620 0x7f36d40b2940 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f36c400d980 tx=0x7f36c400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:37.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36d0c00640 1 -- 172.21.15.66:0/3852912298 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4017070 con 0x7f36d40ae620 2024-09-10T22:07:37.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36d0c00640 1 -- 172.21.15.66:0/3852912298 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36c4009d60 con 0x7f36d40ae620 2024-09-10T22:07:37.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36d0c00640 1 -- 172.21.15.66:0/3852912298 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4005280 con 0x7f36d40ae620 2024-09-10T22:07:37.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36dac00640 1 -- 172.21.15.66:0/3852912298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d40742e0 con 0x7f36d40ae620 2024-09-10T22:07:37.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.938+0000 7f36dac00640 1 -- 172.21.15.66:0/3852912298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d40c7040 con 0x7f36d40ae620 2024-09-10T22:07:37.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.942+0000 7f36dac00640 1 -- 172.21.15.66:0/3852912298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d40725d0 con 0x7f36d40ae620 2024-09-10T22:07:37.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.942+0000 7f36d0c00640 1 -- 172.21.15.66:0/3852912298 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 97822+0+0 (secure 0 0 0) 0x7f36c4005420 con 0x7f36d40ae620 2024-09-10T22:07:37.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.942+0000 7f36d0c00640 1 --2- 172.21.15.66:0/3852912298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f36b00735a0 0x7f36b0075a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:37.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.946+0000 7f36d0c00640 1 -- 172.21.15.66:0/3852912298 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f36c4091f60 con 0x7f36d40ae620 2024-09-10T22:07:37.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.950+0000 7f36cbe00640 1 --2- 172.21.15.66:0/3852912298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f36b00735a0 0x7f36b0075a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:37.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.950+0000 7f36cbe00640 1 --2- 172.21.15.66:0/3852912298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f36b00735a0 0x7f36b0075a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0af000 tx=0x7f36bc019660 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:37.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:37.950+0000 7f36d0c00640 1 -- 172.21.15.66:0/3852912298 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f36c405f090 con 0x7f36d40ae620 2024-09-10T22:07:38.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:38.294+0000 7f36dac00640 1 -- 172.21.15.66:0/3852912298 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone17856", "target": ["mon-mgr", ""]}) -- 0x7f36d40b0260 con 0x7f36b00735a0 2024-09-10T22:07:40.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36d0c00640 1 -- 172.21.15.66:0/3852912298 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f36d40b0260 con 0x7f36b00735a0 2024-09-10T22:07:40.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36dac00640 1 -- 172.21.15.66:0/3852912298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f36b00735a0 msgr2=0x7f36b0075a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:40.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36dac00640 1 --2- 172.21.15.66:0/3852912298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f36b00735a0 0x7f36b0075a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0af000 tx=0x7f36bc019660 comp rx=0 tx=0).stop 2024-09-10T22:07:40.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36dac00640 1 -- 172.21.15.66:0/3852912298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36d40ae620 msgr2=0x7f36d40b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:40.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36dac00640 1 --2- 172.21.15.66:0/3852912298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36d40ae620 0x7f36d40b2940 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f36c400d980 tx=0x7f36c400de40 comp rx=0 tx=0).stop 2024-09-10T22:07:40.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36dac00640 1 -- 172.21.15.66:0/3852912298 shutdown_connections 2024-09-10T22:07:40.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36dac00640 1 --2- 172.21.15.66:0/3852912298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f36b00735a0 0x7f36b0075a60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36dac00640 1 --2- 172.21.15.66:0/3852912298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36d40c4ef0 0x7f36d40c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36dac00640 1 --2- 172.21.15.66:0/3852912298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d40c4b10 0x7f36d4146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36dac00640 1 --2- 172.21.15.66:0/3852912298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36d40ae620 0x7f36d40b2940 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36dac00640 1 -- 172.21.15.66:0/3852912298 >> 172.21.15.66:0/3852912298 conn(0x7f36d40cfa30 msgr2=0x7f36d415d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:40.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36dac00640 1 -- 172.21.15.66:0/3852912298 shutdown_connections 2024-09-10T22:07:40.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.206+0000 7f36dac00640 1 -- 172.21.15.66:0/3852912298 wait complete. 2024-09-10T22:07:40.247 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:07:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.510+0000 7ff504400640 1 -- 172.21.15.66:0/926485602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4fc0d03e0 msgr2=0x7ff4fc0d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.510+0000 7ff504400640 1 --2- 172.21.15.66:0/926485602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4fc0d03e0 0x7ff4fc0d0820 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009900 tx=0x7ff4f001c850 comp rx=0 tx=0).stop 2024-09-10T22:07:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.510+0000 7ff504400640 1 -- 172.21.15.66:0/926485602 shutdown_connections 2024-09-10T22:07:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.510+0000 7ff504400640 1 --2- 172.21.15.66:0/926485602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4fc0d03e0 0x7ff4fc0d0820 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.510+0000 7ff504400640 1 --2- 172.21.15.66:0/926485602 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4fc15e010 0x7ff4fc0cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.510+0000 7ff504400640 1 -- 172.21.15.66:0/926485602 >> 172.21.15.66:0/926485602 conn(0x7ff4fc0d1c60 msgr2=0x7ff4fc0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.510+0000 7ff504400640 1 -- 172.21.15.66:0/926485602 shutdown_connections 2024-09-10T22:07:40.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.510+0000 7ff504400640 1 -- 172.21.15.66:0/926485602 wait complete. 2024-09-10T22:07:40.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.514+0000 7ff504400640 1 Processor -- start 2024-09-10T22:07:40.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.518+0000 7ff504400640 1 -- start start 2024-09-10T22:07:40.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.518+0000 7ff504400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4fc0d03e0 0x7ff4fc09bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:40.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff504400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4fc0d0d60 0x7ff4fc1f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:40.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff504400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4fc15e010 0x7ff4fc09ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:40.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff501a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4fc0d03e0 0x7ff4fc09bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:40.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff501a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4fc0d03e0 0x7ff4fc09bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53160/0 (socket says 172.21.15.66:53160) 2024-09-10T22:07:40.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff504400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc16aea0 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff504400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc16ad20 con 0x7ff4fc0d0d60 2024-09-10T22:07:40.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff504400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff4fc16b020 con 0x7ff4fc15e010 2024-09-10T22:07:40.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff501a00640 1 -- 172.21.15.66:0/16796379 learned_addr learned my addr 172.21.15.66:0/16796379 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff501a00640 1 -- 172.21.15.66:0/16796379 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4fc15e010 msgr2=0x7ff4fc09ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff501a00640 1 --2- 172.21.15.66:0/16796379 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4fc15e010 0x7ff4fc09ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff501a00640 1 -- 172.21.15.66:0/16796379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4fc0d0d60 msgr2=0x7ff4fc1f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:40.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff501a00640 1 --2- 172.21.15.66:0/16796379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4fc0d0d60 0x7ff4fc1f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.522+0000 7ff501a00640 1 -- 172.21.15.66:0/16796379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc1f80a0 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.526+0000 7ff501a00640 1 --2- 172.21.15.66:0/16796379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4fc0d03e0 0x7ff4fc09bdc0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800b410 tx=0x7ff4e800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:40.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.526+0000 7ff4f9600640 1 -- 172.21.15.66:0/16796379 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8015020 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.526+0000 7ff4f9600640 1 -- 172.21.15.66:0/16796379 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4e8011920 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.526+0000 7ff4f9600640 1 -- 172.21.15.66:0/16796379 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8011d70 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.526+0000 7ff504400640 1 -- 172.21.15.66:0/16796379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4fc1f8380 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.526+0000 7ff504400640 1 -- 172.21.15.66:0/16796379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4fc1609a0 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.530+0000 7ff504400640 1 -- 172.21.15.66:0/16796379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4fc0d0820 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.530+0000 7ff4f9600640 1 -- 172.21.15.66:0/16796379 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 97822+0+0 (secure 0 0 0) 0x7ff4e801a400 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.534+0000 7ff4f9600640 1 --2- 172.21.15.66:0/16796379 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff4dc073670 0x7ff4dc075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:40.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.534+0000 7ff4f9600640 1 -- 172.21.15.66:0/16796379 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7ff4e8092010 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.542+0000 7ff501000640 1 --2- 172.21.15.66:0/16796379 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff4dc073670 0x7ff4dc075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:40.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.542+0000 7ff501000640 1 --2- 172.21.15.66:0/16796379 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff4dc073670 0x7ff4dc075b30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00af000 tx=0x7ff4f00195f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:40.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.546+0000 7ff4f9600640 1 -- 172.21.15.66:0/16796379 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff4e801f020 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.938+0000 7ff504400640 1 -- 172.21.15.66:0/16796379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff4fc0cbef0 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.938+0000 7ff4f9600640 1 -- 172.21.15.66:0/16796379 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2422 (secure 0 0 0) 0x7ff4e805f140 con 0x7ff4fc0d03e0 2024-09-10T22:07:40.943 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:07:40.943 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:07:40.943 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:07:40.943 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:07:40.943 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:07:40.943 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:07:40.943 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:07:40.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:40.943 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:07:40.943 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:07:40.943 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:07:40.944 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:07:40.944 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:07:40.944 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:07:40.944 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:07:40.944 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:07:40.944 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:07:40.944 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:07:40.944 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:07:40.944 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 454, 2024-09-10T22:07:40.944 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:07:40.944 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:07:40.944 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 73, 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:07:40.945 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:07:40.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:07:40.946 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:07:40.946 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:07:40.946 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:07:40.946 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:07:40.946 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:07:40.946 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:07:40.946 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 1590, 2024-09-10T22:07:40.946 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 6519159416, 2024-09-10T22:07:40.946 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 13092556800, 2024-09-10T22:07:40.946 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 760001556480, 2024-09-10T22:07:40.946 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:07:40.947 INFO:teuthology.orchestra.run.smithi066.stdout: "read_bytes_sec": 293821258, 2024-09-10T22:07:40.947 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 274646271, 2024-09-10T22:07:40.947 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 71, 2024-09-10T22:07:40.947 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 77 2024-09-10T22:07:40.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:40.947 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:07:40.947 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 47, 2024-09-10T22:07:40.947 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:07:02:166078+0000", 2024-09-10T22:07:40.947 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5, 2024-09-10T22:07:40.947 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:07:40.947 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 5, 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "c", 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 6390 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:40.948 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:07:40.949 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:07:40.949 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:07:40.949 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:07:40.949 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:07:40.949 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:07:40.949 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:07:40.949 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:07:40.949 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:07:40.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:40.949 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:07:40.950 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 11, 2024-09-10T22:07:40.950 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:06:16.394877+0000", 2024-09-10T22:07:40.950 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:07:40.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:40.950 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": { 2024-09-10T22:07:40.950 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr-vol-ongoing-clones": { 2024-09-10T22:07:40.950 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "4 ongoing clones - average progress is 30.599% (6s)\n [========....................] (remaining: 14s)", 2024-09-10T22:07:40.950 INFO:teuthology.orchestra.run.smithi066.stdout: "progress": 0.30599001049995422, 2024-09-10T22:07:40.950 INFO:teuthology.orchestra.run.smithi066.stdout: "add_to_ceph_s": true 2024-09-10T22:07:40.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:40.950 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr-vol-total-clones": { 2024-09-10T22:07:40.951 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "Total 5 clones - average progress is 27.656% (0s)\n [=======.....................] ", 2024-09-10T22:07:40.951 INFO:teuthology.orchestra.run.smithi066.stdout: "progress": 0.27656000852584839, 2024-09-10T22:07:40.951 INFO:teuthology.orchestra.run.smithi066.stdout: "add_to_ceph_s": true 2024-09-10T22:07:40.951 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:07:40.951 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:07:40.951 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:07:40.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.942+0000 7ff504400640 1 -- 172.21.15.66:0/16796379 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff4dc073670 msgr2=0x7ff4dc075b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:40.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.942+0000 7ff504400640 1 --2- 172.21.15.66:0/16796379 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff4dc073670 0x7ff4dc075b30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00af000 tx=0x7ff4f00195f0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.942+0000 7ff504400640 1 -- 172.21.15.66:0/16796379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4fc0d03e0 msgr2=0x7ff4fc09bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:40.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.942+0000 7ff504400640 1 --2- 172.21.15.66:0/16796379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4fc0d03e0 0x7ff4fc09bdc0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800b410 tx=0x7ff4e800b8d0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.942+0000 7ff504400640 1 -- 172.21.15.66:0/16796379 shutdown_connections 2024-09-10T22:07:40.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.942+0000 7ff504400640 1 --2- 172.21.15.66:0/16796379 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff4dc073670 0x7ff4dc075b30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.942+0000 7ff504400640 1 --2- 172.21.15.66:0/16796379 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4fc15e010 0x7ff4fc09ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.942+0000 7ff504400640 1 --2- 172.21.15.66:0/16796379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4fc0d0d60 0x7ff4fc1f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.942+0000 7ff504400640 1 --2- 172.21.15.66:0/16796379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4fc0d03e0 0x7ff4fc09bdc0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:40.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.942+0000 7ff504400640 1 -- 172.21.15.66:0/16796379 >> 172.21.15.66:0/16796379 conn(0x7ff4fc0d1c60 msgr2=0x7ff4fc0b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:40.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.942+0000 7ff504400640 1 -- 172.21.15.66:0/16796379 shutdown_connections 2024-09-10T22:07:40.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:40.942+0000 7ff504400640 1 -- 172.21.15.66:0/16796379 wait complete. 2024-09-10T22:07:40.975 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone70890 2024-09-10T22:07:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.230+0000 7fa069000640 1 -- 172.21.15.66:0/2529688014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0640cf500 msgr2=0x7fa06415acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.230+0000 7fa069000640 1 --2- 172.21.15.66:0/2529688014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0640cf500 0x7fa06415acd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009c80 tx=0x7fa05c01cdc0 comp rx=0 tx=0).stop 2024-09-10T22:07:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.234+0000 7fa069000640 1 -- 172.21.15.66:0/2529688014 shutdown_connections 2024-09-10T22:07:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.234+0000 7fa069000640 1 --2- 172.21.15.66:0/2529688014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0640cf500 0x7fa06415acd0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.234+0000 7fa069000640 1 --2- 172.21.15.66:0/2529688014 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0640c5a80 0x7fa0640d0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.234+0000 7fa069000640 1 -- 172.21.15.66:0/2529688014 >> 172.21.15.66:0/2529688014 conn(0x7fa0640cfa30 msgr2=0x7fa0640cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.234+0000 7fa069000640 1 -- 172.21.15.66:0/2529688014 shutdown_connections 2024-09-10T22:07:41.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.238+0000 7fa069000640 1 -- 172.21.15.66:0/2529688014 wait complete. 2024-09-10T22:07:41.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.242+0000 7fa069000640 1 Processor -- start 2024-09-10T22:07:41.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.242+0000 7fa069000640 1 -- start start 2024-09-10T22:07:41.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa069000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0640c5a80 0x7fa06414c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:41.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa069000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0640cf500 0x7fa064151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:41.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa069000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0641503e0 0x7fa064155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:41.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa069000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa06406f0a0 con 0x7fa0640c5a80 2024-09-10T22:07:41.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa069000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa06406ef20 con 0x7fa0641503e0 2024-09-10T22:07:41.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa069000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa06406f220 con 0x7fa0640cf500 2024-09-10T22:07:41.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa061e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0640c5a80 0x7fa06414c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:41.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa061e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0640c5a80 0x7fa06414c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53174/0 (socket says 172.21.15.66:53174) 2024-09-10T22:07:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa061e00640 1 -- 172.21.15.66:0/3349484326 learned_addr learned my addr 172.21.15.66:0/3349484326 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa061400640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0640cf500 0x7fa064151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa061e00640 1 -- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0640cf500 msgr2=0x7fa064151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa061e00640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0640cf500 0x7fa064151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa061e00640 1 -- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0641503e0 msgr2=0x7fa064155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa061e00640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0641503e0 0x7fa064155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa061e00640 1 -- 172.21.15.66:0/3349484326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0641f75f0 con 0x7fa0640c5a80 2024-09-10T22:07:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.246+0000 7fa061400640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0640cf500 0x7fa064151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:07:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.250+0000 7fa061e00640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0640c5a80 0x7fa06414c370 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c450 tx=0x7fa05400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.250+0000 7fa05a000640 1 -- 172.21.15.66:0/3349484326 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054017070 con 0x7fa0640c5a80 2024-09-10T22:07:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.250+0000 7fa05a000640 1 -- 172.21.15.66:0/3349484326 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa054003600 con 0x7fa0640c5a80 2024-09-10T22:07:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.250+0000 7fa05a000640 1 -- 172.21.15.66:0/3349484326 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054003a50 con 0x7fa0640c5a80 2024-09-10T22:07:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.250+0000 7fa069000640 1 -- 172.21.15.66:0/3349484326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0641f78d0 con 0x7fa0640c5a80 2024-09-10T22:07:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.250+0000 7fa069000640 1 -- 172.21.15.66:0/3349484326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0641f7e90 con 0x7fa0640c5a80 2024-09-10T22:07:41.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.254+0000 7fa069000640 1 -- 172.21.15.66:0/3349484326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0640d0550 con 0x7fa0640c5a80 2024-09-10T22:07:41.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.254+0000 7fa05a000640 1 -- 172.21.15.66:0/3349484326 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 97822+0+0 (secure 0 0 0) 0x7fa054003bf0 con 0x7fa0640c5a80 2024-09-10T22:07:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.262+0000 7fa05a000640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0440735d0 0x7fa044075a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.262+0000 7fa05a000640 1 -- 172.21.15.66:0/3349484326 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fa0540915d0 con 0x7fa0640c5a80 2024-09-10T22:07:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.270+0000 7fa05a000640 1 -- 172.21.15.66:0/3349484326 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa05405e450 con 0x7fa0640c5a80 2024-09-10T22:07:41.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.318+0000 7fa061400640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0440735d0 0x7fa044075a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:41.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.322+0000 7fa061400640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0440735d0 0x7fa044075a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa064073840 tx=0x7fa050008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:41.642+0000 7fa069000640 1 -- 172.21.15.66:0/3349484326 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone70890", "target": ["mon-mgr", ""]}) -- 0x7fa064075130 con 0x7fa0440735d0 2024-09-10T22:07:42.176 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:42.176+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:42.176 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:42.176+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:42.176 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:42.176+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:42.176 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:42.176+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:42.176 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:42.176+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:42.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:42.858+0000 7fa05a000640 1 -- 172.21.15.66:0/3349484326 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7fa054059fc0 con 0x7fa0640c5a80 2024-09-10T22:07:44.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.210+0000 7fa05a000640 1 -- 172.21.15.66:0/3349484326 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa064075130 con 0x7fa0440735d0 2024-09-10T22:07:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.218+0000 7fa069000640 1 -- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0440735d0 msgr2=0x7fa044075a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.218+0000 7fa069000640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0440735d0 0x7fa044075a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa064073840 tx=0x7fa050008040 comp rx=0 tx=0).stop 2024-09-10T22:07:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.218+0000 7fa069000640 1 -- 172.21.15.66:0/3349484326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0640c5a80 msgr2=0x7fa06414c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.218+0000 7fa069000640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0640c5a80 0x7fa06414c370 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c450 tx=0x7fa05400c910 comp rx=0 tx=0).stop 2024-09-10T22:07:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.218+0000 7fa069000640 1 -- 172.21.15.66:0/3349484326 shutdown_connections 2024-09-10T22:07:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.218+0000 7fa069000640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa0440735d0 0x7fa044075a90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.218+0000 7fa069000640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0641503e0 0x7fa064155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.218+0000 7fa069000640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0640cf500 0x7fa064151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.218+0000 7fa069000640 1 --2- 172.21.15.66:0/3349484326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0640c5a80 0x7fa06414c370 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.218+0000 7fa069000640 1 -- 172.21.15.66:0/3349484326 >> 172.21.15.66:0/3349484326 conn(0x7fa0640cfa30 msgr2=0x7fa0641571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:44.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.222+0000 7fa069000640 1 -- 172.21.15.66:0/3349484326 shutdown_connections 2024-09-10T22:07:44.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.222+0000 7fa069000640 1 -- 172.21.15.66:0/3349484326 wait complete. 2024-09-10T22:07:44.271 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone93841 2024-09-10T22:07:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.526+0000 7f65fea00640 1 -- 172.21.15.66:0/2177380463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65dc004810 msgr2=0x7f65dc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.526+0000 7f65fea00640 1 --2- 172.21.15.66:0/2177380463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65dc004810 0x7f65dc004c90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f65e400c730 tx=0x7f65e401c6c0 comp rx=0 tx=0).stop 2024-09-10T22:07:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.526+0000 7f65fca00640 1 -- 172.21.15.66:0/2177380463 reap_dead start 2024-09-10T22:07:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.530+0000 7f65fea00640 1 -- 172.21.15.66:0/2177380463 shutdown_connections 2024-09-10T22:07:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.530+0000 7f65fea00640 1 -- 172.21.15.66:0/2177380463 >> 172.21.15.66:0/2177380463 conn(0x7f65f80d0310 msgr2=0x7f65f80d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.530+0000 7f65fea00640 1 -- 172.21.15.66:0/2177380463 shutdown_connections 2024-09-10T22:07:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.530+0000 7f65fea00640 1 -- 172.21.15.66:0/2177380463 wait complete. 2024-09-10T22:07:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.530+0000 7f65fea00640 1 Processor -- start 2024-09-10T22:07:44.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.530+0000 7f65fea00640 1 -- start start 2024-09-10T22:07:44.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65fea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65dc004810 0x7f65f809c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:44.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65fea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65f815a5a0 0x7f65f814d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:44.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65fea00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65f806f6d0 0x7f65f806fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:44.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65fea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f65f80d0fb0 con 0x7f65f815a5a0 2024-09-10T22:07:44.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65fea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65f80d0e30 con 0x7f65dc004810 2024-09-10T22:07:44.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65fea00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f65f8166ec0 con 0x7f65f806f6d0 2024-09-10T22:07:44.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65f7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65f815a5a0 0x7f65f814d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65f7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65f815a5a0 0x7f65f814d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34864/0 (socket says 172.21.15.66:34864) 2024-09-10T22:07:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65f7400640 1 -- 172.21.15.66:0/4288628718 learned_addr learned my addr 172.21.15.66:0/4288628718 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65f7400640 1 -- 172.21.15.66:0/4288628718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65f806f6d0 msgr2=0x7f65f806fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65f7400640 1 --2- 172.21.15.66:0/4288628718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65f806f6d0 0x7f65f806fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65f7400640 1 -- 172.21.15.66:0/4288628718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65dc004810 msgr2=0x7f65f809c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65f7400640 1 --2- 172.21.15.66:0/4288628718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65dc004810 0x7f65f809c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65f7400640 1 -- 172.21.15.66:0/4288628718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f809c8b0 con 0x7f65f815a5a0 2024-09-10T22:07:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65f7400640 1 --2- 172.21.15.66:0/4288628718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65f815a5a0 0x7f65f814d260 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f65e400c650 tx=0x7f65e4004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65efe00640 1 -- 172.21.15.66:0/4288628718 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4006930 con 0x7f65f815a5a0 2024-09-10T22:07:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65efe00640 1 -- 172.21.15.66:0/4288628718 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65e40bb070 con 0x7f65f815a5a0 2024-09-10T22:07:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65efe00640 1 -- 172.21.15.66:0/4288628718 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e400c300 con 0x7f65f815a5a0 2024-09-10T22:07:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65fea00640 1 -- 172.21.15.66:0/4288628718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f809cb70 con 0x7f65f815a5a0 2024-09-10T22:07:44.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.534+0000 7f65fea00640 1 -- 172.21.15.66:0/4288628718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f809d130 con 0x7f65f815a5a0 2024-09-10T22:07:44.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.538+0000 7f65fea00640 1 -- 172.21.15.66:0/4288628718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f80cb030 con 0x7f65f815a5a0 2024-09-10T22:07:44.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.550+0000 7f65efe00640 1 -- 172.21.15.66:0/4288628718 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7f65e40b5470 con 0x7f65f815a5a0 2024-09-10T22:07:44.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.554+0000 7f65efe00640 1 --2- 172.21.15.66:0/4288628718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65d8073660 0x7f65d8075b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.554+0000 7f65efe00640 1 -- 172.21.15.66:0/4288628718 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f65e412d700 con 0x7f65f815a5a0 2024-09-10T22:07:44.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.558+0000 7f65efe00640 1 -- 172.21.15.66:0/4288628718 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f65e40163d0 con 0x7f65f815a5a0 2024-09-10T22:07:44.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.566+0000 7f65f7e00640 1 --2- 172.21.15.66:0/4288628718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65d8073660 0x7f65d8075b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:44.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.574+0000 7f65f7e00640 1 --2- 172.21.15.66:0/4288628718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65d8073660 0x7f65d8075b20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009730 tx=0x7f65e8006d20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:44.930+0000 7f65fea00640 1 -- 172.21.15.66:0/4288628718 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone93841", "target": ["mon-mgr", ""]}) -- 0x7f65f8070090 con 0x7f65d8073660 2024-09-10T22:07:46.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.770+0000 7f65efe00640 1 -- 172.21.15.66:0/4288628718 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f65f8070090 con 0x7f65d8073660 2024-09-10T22:07:46.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.774+0000 7f65fea00640 1 -- 172.21.15.66:0/4288628718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65d8073660 msgr2=0x7f65d8075b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:46.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.774+0000 7f65fea00640 1 --2- 172.21.15.66:0/4288628718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65d8073660 0x7f65d8075b20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009730 tx=0x7f65e8006d20 comp rx=0 tx=0).stop 2024-09-10T22:07:46.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.774+0000 7f65fea00640 1 -- 172.21.15.66:0/4288628718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65f815a5a0 msgr2=0x7f65f814d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:46.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.774+0000 7f65fea00640 1 --2- 172.21.15.66:0/4288628718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65f815a5a0 0x7f65f814d260 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f65e400c650 tx=0x7f65e4004cb0 comp rx=0 tx=0).stop 2024-09-10T22:07:46.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.774+0000 7f65fea00640 1 -- 172.21.15.66:0/4288628718 shutdown_connections 2024-09-10T22:07:46.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.774+0000 7f65fea00640 1 --2- 172.21.15.66:0/4288628718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65d8073660 0x7f65d8075b20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:46.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.774+0000 7f65fea00640 1 --2- 172.21.15.66:0/4288628718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65f806f6d0 0x7f65f806fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:46.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.774+0000 7f65fea00640 1 --2- 172.21.15.66:0/4288628718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65f815a5a0 0x7f65f814d260 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:46.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.774+0000 7f65fea00640 1 --2- 172.21.15.66:0/4288628718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65dc004810 0x7f65f809c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:46.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.774+0000 7f65fea00640 1 -- 172.21.15.66:0/4288628718 >> 172.21.15.66:0/4288628718 conn(0x7f65f80d0310 msgr2=0x7f65f8070710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:46.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.774+0000 7f65fea00640 1 -- 172.21.15.66:0/4288628718 shutdown_connections 2024-09-10T22:07:46.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:46.774+0000 7f65fea00640 1 -- 172.21.15.66:0/4288628718 wait complete. 2024-09-10T22:07:46.805 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone33762 2024-09-10T22:07:47.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.038+0000 7f8d35c00640 1 -- 172.21.15.66:0/890218878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d140052f0 msgr2=0x7f8d1400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:47.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.038+0000 7f8d35c00640 1 --2- 172.21.15.66:0/890218878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d140052f0 0x7f8d1400b9d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8d28001c80 tx=0x7f8d2801ca20 comp rx=0 tx=0).stop 2024-09-10T22:07:47.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.038+0000 7f8d35c00640 1 -- 172.21.15.66:0/890218878 shutdown_connections 2024-09-10T22:07:47.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.038+0000 7f8d35c00640 1 --2- 172.21.15.66:0/890218878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d140052f0 0x7f8d1400b9d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:47.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.038+0000 7f8d35c00640 1 -- 172.21.15.66:0/890218878 >> 172.21.15.66:0/890218878 conn(0x7f8d300cfcf0 msgr2=0x7f8d300d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:47.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.038+0000 7f8d35c00640 1 -- 172.21.15.66:0/890218878 shutdown_connections 2024-09-10T22:07:47.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.038+0000 7f8d35c00640 1 -- 172.21.15.66:0/890218878 wait complete. 2024-09-10T22:07:47.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.042+0000 7f8d35c00640 1 Processor -- start 2024-09-10T22:07:47.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.042+0000 7f8d35c00640 1 -- start start 2024-09-10T22:07:47.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d35c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d140052f0 0x7f8d3009b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:47.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d35c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d300c50b0 0x7f8d3015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:47.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d35c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d301f8780 0x7f8d300b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:47.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d35c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3015ef30 con 0x7f8d300c50b0 2024-09-10T22:07:47.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d35c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3015edb0 con 0x7f8d301f8780 2024-09-10T22:07:47.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d35c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8d3015f0b0 con 0x7f8d140052f0 2024-09-10T22:07:47.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d2f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d300c50b0 0x7f8d3015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:47.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d2f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d300c50b0 0x7f8d3015ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34884/0 (socket says 172.21.15.66:34884) 2024-09-10T22:07:47.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d2f400640 1 -- 172.21.15.66:0/2962852388 learned_addr learned my addr 172.21.15.66:0/2962852388 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:47.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d2fe00640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d140052f0 0x7f8d3009b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:47.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d35200640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d301f8780 0x7f8d300b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:47.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d2f400640 1 -- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d140052f0 msgr2=0x7f8d3009b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:47.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d2f400640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d140052f0 0x7f8d3009b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:47.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d2f400640 1 -- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d301f8780 msgr2=0x7f8d300b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:47.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d2f400640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d301f8780 0x7f8d300b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:47.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d2f400640 1 -- 172.21.15.66:0/2962852388 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d301f90d0 con 0x7f8d300c50b0 2024-09-10T22:07:47.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d2f400640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d300c50b0 0x7f8d3015ad90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c730 tx=0x7f8d1c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:47.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d27e00640 1 -- 172.21.15.66:0/2962852388 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c017070 con 0x7f8d300c50b0 2024-09-10T22:07:47.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d27e00640 1 -- 172.21.15.66:0/2962852388 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d1c005380 con 0x7f8d300c50b0 2024-09-10T22:07:47.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d35c00640 1 -- 172.21.15.66:0/2962852388 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d301f93b0 con 0x7f8d300c50b0 2024-09-10T22:07:47.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d27e00640 1 -- 172.21.15.66:0/2962852388 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00a8a0 con 0x7f8d300c50b0 2024-09-10T22:07:47.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d35c00640 1 -- 172.21.15.66:0/2962852388 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d3009ae60 con 0x7f8d300c50b0 2024-09-10T22:07:47.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.046+0000 7f8d35c00640 1 -- 172.21.15.66:0/2962852388 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d300b4940 con 0x7f8d300c50b0 2024-09-10T22:07:47.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.050+0000 7f8d27e00640 1 -- 172.21.15.66:0/2962852388 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7f8d1c00aa40 con 0x7f8d300c50b0 2024-09-10T22:07:47.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.050+0000 7f8d27e00640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d10073630 0x7f8d10075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:47.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.050+0000 7f8d27e00640 1 -- 172.21.15.66:0/2962852388 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f8d1c0917a0 con 0x7f8d300c50b0 2024-09-10T22:07:47.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.054+0000 7f8d27e00640 1 -- 172.21.15.66:0/2962852388 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8d1c05e5b0 con 0x7f8d300c50b0 2024-09-10T22:07:47.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.062+0000 7f8d2fe00640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d10073630 0x7f8d10075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:47.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.066+0000 7f8d2fe00640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d10073630 0x7f8d10075af0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000b830 tx=0x7f8d20008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:47.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:47.390+0000 7f8d35c00640 1 -- 172.21.15.66:0/2962852388 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone33762", "target": ["mon-mgr", ""]}) -- 0x7f8d3009b030 con 0x7f8d10073630 2024-09-10T22:07:48.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d27e00640 1 -- 172.21.15.66:0/2962852388 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8d3009b030 con 0x7f8d10073630 2024-09-10T22:07:48.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d35c00640 1 -- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d10073630 msgr2=0x7f8d10075af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:48.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d35c00640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d10073630 0x7f8d10075af0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000b830 tx=0x7f8d20008040 comp rx=0 tx=0).stop 2024-09-10T22:07:48.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d35c00640 1 -- 172.21.15.66:0/2962852388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d300c50b0 msgr2=0x7f8d3015ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:48.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d35c00640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d300c50b0 0x7f8d3015ad90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c730 tx=0x7f8d1c00cbf0 comp rx=0 tx=0).stop 2024-09-10T22:07:48.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d35c00640 1 -- 172.21.15.66:0/2962852388 shutdown_connections 2024-09-10T22:07:48.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d35c00640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d10073630 0x7f8d10075af0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:48.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d35c00640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d301f8780 0x7f8d300b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:48.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d35c00640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d300c50b0 0x7f8d3015ad90 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:48.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d35c00640 1 --2- 172.21.15.66:0/2962852388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d140052f0 0x7f8d3009b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:48.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d35c00640 1 -- 172.21.15.66:0/2962852388 >> 172.21.15.66:0/2962852388 conn(0x7f8d300cfcf0 msgr2=0x7f8d3006ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:48.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d35c00640 1 -- 172.21.15.66:0/2962852388 shutdown_connections 2024-09-10T22:07:48.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.574+0000 7f8d35c00640 1 -- 172.21.15.66:0/2962852388 wait complete. 2024-09-10T22:07:48.618 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone16323 2024-09-10T22:07:48.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.862+0000 7f9794e00640 1 -- 172.21.15.66:0/1066882209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9790073de0 msgr2=0x7f97900cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:48.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.862+0000 7f9794e00640 1 --2- 172.21.15.66:0/1066882209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9790073de0 0x7f97900cc910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9780009840 tx=0x7f978001c7f0 comp rx=0 tx=0).stop 2024-09-10T22:07:48.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.862+0000 7f9794e00640 1 -- 172.21.15.66:0/1066882209 shutdown_connections 2024-09-10T22:07:48.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.862+0000 7f9794e00640 1 --2- 172.21.15.66:0/1066882209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9790073de0 0x7f97900cc910 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:48.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.862+0000 7f9794e00640 1 -- 172.21.15.66:0/1066882209 >> 172.21.15.66:0/1066882209 conn(0x7f97900b1e20 msgr2=0x7f97900b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:48.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.862+0000 7f9794e00640 1 -- 172.21.15.66:0/1066882209 shutdown_connections 2024-09-10T22:07:48.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.862+0000 7f9794e00640 1 -- 172.21.15.66:0/1066882209 wait complete. 2024-09-10T22:07:48.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9794e00640 1 Processor -- start 2024-09-10T22:07:48.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9794e00640 1 -- start start 2024-09-10T22:07:48.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9794e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9790073de0 0x7f97901fca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:48.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9794e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97900741c0 0x7f97901fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:48.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9794e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97900b38e0 0x7f9790072a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:48.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9794e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9790151810 con 0x7f97900741c0 2024-09-10T22:07:48.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9794e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9790151690 con 0x7f97900b38e0 2024-09-10T22:07:48.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9794e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9790151990 con 0x7f9790073de0 2024-09-10T22:07:48.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f978d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97900741c0 0x7f97901fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:48.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f978d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97900741c0 0x7f97901fed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34886/0 (socket says 172.21.15.66:34886) 2024-09-10T22:07:48.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f978d400640 1 -- 172.21.15.66:0/1020368015 learned_addr learned my addr 172.21.15.66:0/1020368015 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:48.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f978d400640 1 -- 172.21.15.66:0/1020368015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9790073de0 msgr2=0x7f97901fca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:48.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f978d400640 1 --2- 172.21.15.66:0/1020368015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9790073de0 0x7f97901fca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:48.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f978d400640 1 -- 172.21.15.66:0/1020368015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97900b38e0 msgr2=0x7f9790072a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:48.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f978d400640 1 --2- 172.21.15.66:0/1020368015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97900b38e0 0x7f9790072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:48.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f978d400640 1 -- 172.21.15.66:0/1020368015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97901fcf70 con 0x7f97900741c0 2024-09-10T22:07:48.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f978d400640 1 --2- 172.21.15.66:0/1020368015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97900741c0 0x7f97901fed30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c490 tx=0x7f977c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:48.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9786000640 1 -- 172.21.15.66:0/1020368015 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c016020 con 0x7f97900741c0 2024-09-10T22:07:48.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9786000640 1 -- 172.21.15.66:0/1020368015 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f977c012940 con 0x7f97900741c0 2024-09-10T22:07:48.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9786000640 1 -- 172.21.15.66:0/1020368015 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c012d90 con 0x7f97900741c0 2024-09-10T22:07:48.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9794e00640 1 -- 172.21.15.66:0/1020368015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979015c550 con 0x7f97900741c0 2024-09-10T22:07:48.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.866+0000 7f9794e00640 1 -- 172.21.15.66:0/1020368015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f979015c810 con 0x7f97900741c0 2024-09-10T22:07:48.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.870+0000 7f9794e00640 1 -- 172.21.15.66:0/1020368015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97900760a0 con 0x7f97900741c0 2024-09-10T22:07:48.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.870+0000 7f9786000640 1 -- 172.21.15.66:0/1020368015 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7f977c01b400 con 0x7f97900741c0 2024-09-10T22:07:48.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.874+0000 7f9786000640 1 --2- 172.21.15.66:0/1020368015 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9770073630 0x7f9770075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:48.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.874+0000 7f9786000640 1 -- 172.21.15.66:0/1020368015 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f977c092110 con 0x7f97900741c0 2024-09-10T22:07:48.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.874+0000 7f9786000640 1 -- 172.21.15.66:0/1020368015 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f977c020030 con 0x7f97900741c0 2024-09-10T22:07:48.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.878+0000 7f978de00640 1 --2- 172.21.15.66:0/1020368015 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9770073630 0x7f9770075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:48.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:48.882+0000 7f978de00640 1 --2- 172.21.15.66:0/1020368015 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9770073630 0x7f9770075af0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f97800af000 tx=0x7f9780019590 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:49.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.258+0000 7f9794e00640 1 -- 172.21.15.66:0/1020368015 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone16323", "target": ["mon-mgr", ""]}) -- 0x7f979015a750 con 0x7f9770073630 2024-09-10T22:07:49.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.910+0000 7f9786000640 1 -- 172.21.15.66:0/1020368015 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f979015a750 con 0x7f9770073630 2024-09-10T22:07:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.914+0000 7f9794e00640 1 -- 172.21.15.66:0/1020368015 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9770073630 msgr2=0x7f9770075af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.914+0000 7f9794e00640 1 --2- 172.21.15.66:0/1020368015 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9770073630 0x7f9770075af0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f97800af000 tx=0x7f9780019590 comp rx=0 tx=0).stop 2024-09-10T22:07:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.914+0000 7f9794e00640 1 -- 172.21.15.66:0/1020368015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97900741c0 msgr2=0x7f97901fed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.914+0000 7f9794e00640 1 --2- 172.21.15.66:0/1020368015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97900741c0 0x7f97901fed30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c490 tx=0x7f977c00c950 comp rx=0 tx=0).stop 2024-09-10T22:07:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.914+0000 7f9794e00640 1 -- 172.21.15.66:0/1020368015 shutdown_connections 2024-09-10T22:07:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.914+0000 7f9794e00640 1 --2- 172.21.15.66:0/1020368015 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9770073630 0x7f9770075af0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.914+0000 7f9794e00640 1 --2- 172.21.15.66:0/1020368015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97900b38e0 0x7f9790072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.914+0000 7f9794e00640 1 --2- 172.21.15.66:0/1020368015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97900741c0 0x7f97901fed30 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:49.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.914+0000 7f9794e00640 1 --2- 172.21.15.66:0/1020368015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9790073de0 0x7f97901fca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:49.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.914+0000 7f9794e00640 1 -- 172.21.15.66:0/1020368015 >> 172.21.15.66:0/1020368015 conn(0x7f97900b1e20 msgr2=0x7f97900cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:49.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.914+0000 7f9794e00640 1 -- 172.21.15.66:0/1020368015 shutdown_connections 2024-09-10T22:07:49.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:49.914+0000 7f9794e00640 1 -- 172.21.15.66:0/1020368015 wait complete. 2024-09-10T22:07:49.964 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone87134 2024-09-10T22:07:50.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.222+0000 7fc4c6200640 1 -- 172.21.15.66:0/943106302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a4004810 msgr2=0x7fc4a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:50.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.222+0000 7fc4c6200640 1 --2- 172.21.15.66:0/943106302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a4004810 0x7fc4a4004c90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009910 tx=0x7fc4b001c740 comp rx=0 tx=0).stop 2024-09-10T22:07:50.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.226+0000 7fc4c6200640 1 -- 172.21.15.66:0/943106302 shutdown_connections 2024-09-10T22:07:50.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.226+0000 7fc4c6200640 1 --2- 172.21.15.66:0/943106302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a4004810 0x7fc4a4004c90 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:50.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.226+0000 7fc4c6200640 1 -- 172.21.15.66:0/943106302 >> 172.21.15.66:0/943106302 conn(0x7fc4c00b26c0 msgr2=0x7fc4c00b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:50.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.226+0000 7fc4c6200640 1 -- 172.21.15.66:0/943106302 shutdown_connections 2024-09-10T22:07:50.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.226+0000 7fc4c6200640 1 -- 172.21.15.66:0/943106302 wait complete. 2024-09-10T22:07:50.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.226+0000 7fc4c6200640 1 Processor -- start 2024-09-10T22:07:50.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.226+0000 7fc4c6200640 1 -- start start 2024-09-10T22:07:50.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4c6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a4004810 0x7fc4c01fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:50.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4c6200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4c00b4070 0x7fc4c00b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:50.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4c6200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4c00b6160 0x7fc4c0158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:50.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4c6200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c015e970 con 0x7fc4a4004810 2024-09-10T22:07:50.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4c6200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c015e7f0 con 0x7fc4c00b6160 2024-09-10T22:07:50.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4c6200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c015eaf0 con 0x7fc4c00b4070 2024-09-10T22:07:50.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4bf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a4004810 0x7fc4c01fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:50.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4bf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a4004810 0x7fc4c01fcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34908/0 (socket says 172.21.15.66:34908) 2024-09-10T22:07:50.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4bf400640 1 -- 172.21.15.66:0/2929183002 learned_addr learned my addr 172.21.15.66:0/2929183002 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:50.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4bf400640 1 -- 172.21.15.66:0/2929183002 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4c00b4070 msgr2=0x7fc4c00b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:50.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4bf400640 1 --2- 172.21.15.66:0/2929183002 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4c00b4070 0x7fc4c00b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:50.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4bf400640 1 -- 172.21.15.66:0/2929183002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4c00b6160 msgr2=0x7fc4c0158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:50.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4bf400640 1 --2- 172.21.15.66:0/2929183002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4c00b6160 0x7fc4c0158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:50.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4bf400640 1 -- 172.21.15.66:0/2929183002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c01fc040 con 0x7fc4a4004810 2024-09-10T22:07:50.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4bf400640 1 --2- 172.21.15.66:0/2929183002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a4004810 0x7fc4c01fcae0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00f430 tx=0x7fc4ac00f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:50.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4b7400640 1 -- 172.21.15.66:0/2929183002 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac01a070 con 0x7fc4a4004810 2024-09-10T22:07:50.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4b7400640 1 -- 172.21.15.66:0/2929183002 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4ac0084e0 con 0x7fc4a4004810 2024-09-10T22:07:50.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.230+0000 7fc4b7400640 1 -- 172.21.15.66:0/2929183002 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac006980 con 0x7fc4a4004810 2024-09-10T22:07:50.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.234+0000 7fc4c6200640 1 -- 172.21.15.66:0/2929183002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c00b6a80 con 0x7fc4a4004810 2024-09-10T22:07:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.234+0000 7fc4c6200640 1 -- 172.21.15.66:0/2929183002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c01fc2e0 con 0x7fc4a4004810 2024-09-10T22:07:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.234+0000 7fc4c6200640 1 -- 172.21.15.66:0/2929183002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c00cf270 con 0x7fc4a4004810 2024-09-10T22:07:50.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.234+0000 7fc4b7400640 1 -- 172.21.15.66:0/2929183002 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7fc4ac006b20 con 0x7fc4a4004810 2024-09-10T22:07:50.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.238+0000 7fc4b7400640 1 --2- 172.21.15.66:0/2929183002 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc4a0073680 0x7fc4a0075b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:50.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.238+0000 7fc4b7400640 1 -- 172.21.15.66:0/2929183002 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fc4ac0946c0 con 0x7fc4a4004810 2024-09-10T22:07:50.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.242+0000 7fc4b7400640 1 -- 172.21.15.66:0/2929183002 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc4ac0614d0 con 0x7fc4a4004810 2024-09-10T22:07:50.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.254+0000 7fc4bea00640 1 --2- 172.21.15.66:0/2929183002 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc4a0073680 0x7fc4a0075b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:50.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.258+0000 7fc4bea00640 1 --2- 172.21.15.66:0/2929183002 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc4a0073680 0x7fc4a0075b40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00af000 tx=0x7fc4a4005840 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:50.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:50.658+0000 7fc4c6200640 1 -- 172.21.15.66:0/2929183002 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone87134", "target": ["mon-mgr", ""]}) -- 0x7fc4c0155bd0 con 0x7fc4a0073680 2024-09-10T22:07:51.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.686+0000 7fc4b7400640 1 -- 172.21.15.66:0/2929183002 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc4c0155bd0 con 0x7fc4a0073680 2024-09-10T22:07:51.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.690+0000 7fc4c6200640 1 -- 172.21.15.66:0/2929183002 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc4a0073680 msgr2=0x7fc4a0075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:51.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.690+0000 7fc4c6200640 1 --2- 172.21.15.66:0/2929183002 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc4a0073680 0x7fc4a0075b40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00af000 tx=0x7fc4a4005840 comp rx=0 tx=0).stop 2024-09-10T22:07:51.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.690+0000 7fc4c6200640 1 -- 172.21.15.66:0/2929183002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a4004810 msgr2=0x7fc4c01fcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:51.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.690+0000 7fc4c6200640 1 --2- 172.21.15.66:0/2929183002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a4004810 0x7fc4c01fcae0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00f430 tx=0x7fc4ac00f8f0 comp rx=0 tx=0).stop 2024-09-10T22:07:51.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.690+0000 7fc4c6200640 1 -- 172.21.15.66:0/2929183002 shutdown_connections 2024-09-10T22:07:51.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.690+0000 7fc4c6200640 1 --2- 172.21.15.66:0/2929183002 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc4a0073680 0x7fc4a0075b40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:51.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.690+0000 7fc4c6200640 1 --2- 172.21.15.66:0/2929183002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4c00b6160 0x7fc4c0158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:51.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.690+0000 7fc4c6200640 1 --2- 172.21.15.66:0/2929183002 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4c00b4070 0x7fc4c00b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:51.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.690+0000 7fc4c6200640 1 --2- 172.21.15.66:0/2929183002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a4004810 0x7fc4c01fcae0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:51.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.690+0000 7fc4c6200640 1 -- 172.21.15.66:0/2929183002 >> 172.21.15.66:0/2929183002 conn(0x7fc4c00b26c0 msgr2=0x7fc4c00d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:51.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.690+0000 7fc4c6200640 1 -- 172.21.15.66:0/2929183002 shutdown_connections 2024-09-10T22:07:51.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.690+0000 7fc4c6200640 1 -- 172.21.15.66:0/2929183002 wait complete. 2024-09-10T22:07:51.728 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone62795 2024-09-10T22:07:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.974+0000 7f3028e00640 1 -- 172.21.15.66:0/2503507131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f302415ac80 msgr2=0x7f30240cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.974+0000 7f3028e00640 1 --2- 172.21.15.66:0/2503507131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f302415ac80 0x7f30240cea30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f300c009840 tx=0x7f300c01c7f0 comp rx=0 tx=0).stop 2024-09-10T22:07:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.974+0000 7f3028e00640 1 -- 172.21.15.66:0/2503507131 shutdown_connections 2024-09-10T22:07:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.974+0000 7f3028e00640 1 --2- 172.21.15.66:0/2503507131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f302415ac80 0x7f30240cea30 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.974+0000 7f3028e00640 1 -- 172.21.15.66:0/2503507131 >> 172.21.15.66:0/2503507131 conn(0x7f30240b26c0 msgr2=0x7f30240b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.974+0000 7f3028e00640 1 -- 172.21.15.66:0/2503507131 shutdown_connections 2024-09-10T22:07:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.974+0000 7f3028e00640 1 -- 172.21.15.66:0/2503507131 wait complete. 2024-09-10T22:07:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.982+0000 7f3028e00640 1 Processor -- start 2024-09-10T22:07:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.982+0000 7f3028e00640 1 -- start start 2024-09-10T22:07:51.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.982+0000 7f3028e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30240cb620 0x7f3024149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:51.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.982+0000 7f3028e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415a8a0 0x7f302415dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:51.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.982+0000 7f3028e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f302415ac80 0x7f302415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:51.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.982+0000 7f3028e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f302406cfb0 con 0x7f30240cb620 2024-09-10T22:07:51.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.982+0000 7f3028e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f302406ce30 con 0x7f302415a8a0 2024-09-10T22:07:51.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.982+0000 7f3028e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f302406d130 con 0x7f302415ac80 2024-09-10T22:07:51.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f3021e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30240cb620 0x7f3024149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:51.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f3021e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30240cb620 0x7f3024149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34926/0 (socket says 172.21.15.66:34926) 2024-09-10T22:07:51.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f3021e00640 1 -- 172.21.15.66:0/1772978854 learned_addr learned my addr 172.21.15.66:0/1772978854 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:51.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f3021e00640 1 -- 172.21.15.66:0/1772978854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f302415ac80 msgr2=0x7f302415cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:51.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f3021e00640 1 --2- 172.21.15.66:0/1772978854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f302415ac80 0x7f302415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:51.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f3021e00640 1 -- 172.21.15.66:0/1772978854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415a8a0 msgr2=0x7f302415dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:51.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f3021e00640 1 --2- 172.21.15.66:0/1772978854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415a8a0 0x7f302415dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:51.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f3021e00640 1 -- 172.21.15.66:0/1772978854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302415e150 con 0x7f30240cb620 2024-09-10T22:07:51.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f3021e00640 1 --2- 172.21.15.66:0/1772978854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30240cb620 0x7f3024149240 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f301400c4e0 tx=0x7f301400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f301a000640 1 -- 172.21.15.66:0/1772978854 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30140052d0 con 0x7f30240cb620 2024-09-10T22:07:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f301a000640 1 -- 172.21.15.66:0/1772978854 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3014005470 con 0x7f30240cb620 2024-09-10T22:07:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f301a000640 1 -- 172.21.15.66:0/1772978854 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014012890 con 0x7f30240cb620 2024-09-10T22:07:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f3028e00640 1 -- 172.21.15.66:0/1772978854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30240b4020 con 0x7f30240cb620 2024-09-10T22:07:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.986+0000 7f3028e00640 1 -- 172.21.15.66:0/1772978854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302415b4e0 con 0x7f30240cb620 2024-09-10T22:07:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.990+0000 7f3028e00640 1 -- 172.21.15.66:0/1772978854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30240cea30 con 0x7f30240cb620 2024-09-10T22:07:51.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.990+0000 7f301a000640 1 -- 172.21.15.66:0/1772978854 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7f3014003600 con 0x7f30240cb620 2024-09-10T22:07:51.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.994+0000 7f301a000640 1 --2- 172.21.15.66:0/1772978854 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3000073680 0x7f3000075b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:51.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.994+0000 7f301a000640 1 -- 172.21.15.66:0/1772978854 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f3014091bc0 con 0x7f30240cb620 2024-09-10T22:07:52.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:51.998+0000 7f301a000640 1 -- 172.21.15.66:0/1772978854 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f302415e150 con 0x7f30240cb620 2024-09-10T22:07:52.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.006+0000 7f301be00640 1 --2- 172.21.15.66:0/1772978854 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3000073680 0x7f3000075b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.006+0000 7f301be00640 1 --2- 172.21.15.66:0/1772978854 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3000073680 0x7f3000075b40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f300c0af000 tx=0x7f300c019590 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:52.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.358+0000 7f3028e00640 1 -- 172.21.15.66:0/1772978854 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone62795", "target": ["mon-mgr", ""]}) -- 0x7f30240cd590 con 0x7f3000073680 2024-09-10T22:07:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.938+0000 7f301a000640 1 -- 172.21.15.66:0/1772978854 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f30240cd590 con 0x7f3000073680 2024-09-10T22:07:52.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.942+0000 7f3028e00640 1 -- 172.21.15.66:0/1772978854 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3000073680 msgr2=0x7f3000075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:52.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.942+0000 7f3028e00640 1 --2- 172.21.15.66:0/1772978854 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3000073680 0x7f3000075b40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f300c0af000 tx=0x7f300c019590 comp rx=0 tx=0).stop 2024-09-10T22:07:52.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.942+0000 7f3028e00640 1 -- 172.21.15.66:0/1772978854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30240cb620 msgr2=0x7f3024149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:52.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.942+0000 7f3028e00640 1 --2- 172.21.15.66:0/1772978854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30240cb620 0x7f3024149240 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f301400c4e0 tx=0x7f301400c9a0 comp rx=0 tx=0).stop 2024-09-10T22:07:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.942+0000 7f3028e00640 1 -- 172.21.15.66:0/1772978854 shutdown_connections 2024-09-10T22:07:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.942+0000 7f3028e00640 1 --2- 172.21.15.66:0/1772978854 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3000073680 0x7f3000075b40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.942+0000 7f3028e00640 1 --2- 172.21.15.66:0/1772978854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f302415ac80 0x7f302415cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.942+0000 7f3028e00640 1 --2- 172.21.15.66:0/1772978854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415a8a0 0x7f302415dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.942+0000 7f3028e00640 1 --2- 172.21.15.66:0/1772978854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30240cb620 0x7f3024149240 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.942+0000 7f3028e00640 1 -- 172.21.15.66:0/1772978854 >> 172.21.15.66:0/1772978854 conn(0x7f30240b26c0 msgr2=0x7f30241541f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:52.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.942+0000 7f3028e00640 1 -- 172.21.15.66:0/1772978854 shutdown_connections 2024-09-10T22:07:52.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:52.942+0000 7f3028e00640 1 -- 172.21.15.66:0/1772978854 wait complete. 2024-09-10T22:07:52.974 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone17856 2024-09-10T22:07:53.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.166+0000 7f8d27c00640 1 -- 172.21.15.66:0/3522602284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d200cf560 msgr2=0x7f8d20073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:53.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.166+0000 7f8d27c00640 1 --2- 172.21.15.66:0/3522602284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d200cf560 0x7f8d20073750 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009840 tx=0x7f8d0c01c820 comp rx=0 tx=0).stop 2024-09-10T22:07:53.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.166+0000 7f8d27c00640 1 -- 172.21.15.66:0/3522602284 shutdown_connections 2024-09-10T22:07:53.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.166+0000 7f8d27c00640 1 --2- 172.21.15.66:0/3522602284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d200cf560 0x7f8d20073750 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.166+0000 7f8d27c00640 1 -- 172.21.15.66:0/3522602284 >> 172.21.15.66:0/3522602284 conn(0x7f8d200d0b50 msgr2=0x7f8d200d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:53.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.166+0000 7f8d27c00640 1 -- 172.21.15.66:0/3522602284 shutdown_connections 2024-09-10T22:07:53.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.166+0000 7f8d27c00640 1 -- 172.21.15.66:0/3522602284 wait complete. 2024-09-10T22:07:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.170+0000 7f8d27c00640 1 Processor -- start 2024-09-10T22:07:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.170+0000 7f8d27c00640 1 -- start start 2024-09-10T22:07:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d27c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d20073c90 0x7f8d2015bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d27c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d2015ab20 0x7f8d201399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d27c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d2014a1f0 0x7f8d20153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d27c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20168c40 con 0x7f8d20073c90 2024-09-10T22:07:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d27c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20168ac0 con 0x7f8d2015ab20 2024-09-10T22:07:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d27c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8d20168dc0 con 0x7f8d2014a1f0 2024-09-10T22:07:53.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d25200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d20073c90 0x7f8d2015bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:53.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d25200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d20073c90 0x7f8d2015bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37842/0 (socket says 172.21.15.66:37842) 2024-09-10T22:07:53.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d25200640 1 -- 172.21.15.66:0/1550348685 learned_addr learned my addr 172.21.15.66:0/1550348685 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:53.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d25200640 1 -- 172.21.15.66:0/1550348685 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d2014a1f0 msgr2=0x7f8d20153fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:07:53.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d25200640 1 --2- 172.21.15.66:0/1550348685 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d2014a1f0 0x7f8d20153fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d25200640 1 -- 172.21.15.66:0/1550348685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d2015ab20 msgr2=0x7f8d201399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:53.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d25200640 1 --2- 172.21.15.66:0/1550348685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d2015ab20 0x7f8d201399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d25200640 1 -- 172.21.15.66:0/1550348685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2013a330 con 0x7f8d20073c90 2024-09-10T22:07:53.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.174+0000 7f8d25200640 1 --2- 172.21.15.66:0/1550348685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d20073c90 0x7f8d2015bc30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800c740 tx=0x7f8d1800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.178+0000 7f8d1cc00640 1 -- 172.21.15.66:0/1550348685 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18003db0 con 0x7f8d20073c90 2024-09-10T22:07:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.178+0000 7f8d1cc00640 1 -- 172.21.15.66:0/1550348685 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d1800ce50 con 0x7f8d20073c90 2024-09-10T22:07:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.178+0000 7f8d1cc00640 1 -- 172.21.15.66:0/1550348685 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d180052d0 con 0x7f8d20073c90 2024-09-10T22:07:53.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.178+0000 7f8d27c00640 1 -- 172.21.15.66:0/1550348685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2013a610 con 0x7f8d20073c90 2024-09-10T22:07:53.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.182+0000 7f8d27c00640 1 -- 172.21.15.66:0/1550348685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2015c170 con 0x7f8d20073c90 2024-09-10T22:07:53.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.182+0000 7f8d27c00640 1 -- 172.21.15.66:0/1550348685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d20073750 con 0x7f8d20073c90 2024-09-10T22:07:53.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.182+0000 7f8d1cc00640 1 -- 172.21.15.66:0/1550348685 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7f8d18005470 con 0x7f8d20073c90 2024-09-10T22:07:53.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.186+0000 7f8d1cc00640 1 --2- 172.21.15.66:0/1550348685 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d00073630 0x7f8d00075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:53.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.186+0000 7f8d1cc00640 1 -- 172.21.15.66:0/1550348685 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f8d18091c70 con 0x7f8d20073c90 2024-09-10T22:07:53.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.186+0000 7f8d1fe00640 1 --2- 172.21.15.66:0/1550348685 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d00073630 0x7f8d00075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:53.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.186+0000 7f8d1fe00640 1 --2- 172.21.15.66:0/1550348685 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d00073630 0x7f8d00075af0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0af000 tx=0x7f8d0c0195c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:53.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.190+0000 7f8d1cc00640 1 -- 172.21.15.66:0/1550348685 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8d18020060 con 0x7f8d20073c90 2024-09-10T22:07:53.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.466+0000 7f8d27c00640 1 -- 172.21.15.66:0/1550348685 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone17856", "target": ["mon-mgr", ""]}) -- 0x7f8d200c5c10 con 0x7f8d00073630 2024-09-10T22:07:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d1cc00640 1 -- 172.21.15.66:0/1550348685 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8d200c5c10 con 0x7f8d00073630 2024-09-10T22:07:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d27c00640 1 -- 172.21.15.66:0/1550348685 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d00073630 msgr2=0x7f8d00075af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d27c00640 1 --2- 172.21.15.66:0/1550348685 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d00073630 0x7f8d00075af0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0af000 tx=0x7f8d0c0195c0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d27c00640 1 -- 172.21.15.66:0/1550348685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d20073c90 msgr2=0x7f8d2015bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d27c00640 1 --2- 172.21.15.66:0/1550348685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d20073c90 0x7f8d2015bc30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800c740 tx=0x7f8d1800cc00 comp rx=0 tx=0).stop 2024-09-10T22:07:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d27c00640 1 -- 172.21.15.66:0/1550348685 shutdown_connections 2024-09-10T22:07:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d27c00640 1 --2- 172.21.15.66:0/1550348685 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8d00073630 0x7f8d00075af0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d27c00640 1 --2- 172.21.15.66:0/1550348685 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d2014a1f0 0x7f8d20153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d27c00640 1 --2- 172.21.15.66:0/1550348685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d2015ab20 0x7f8d201399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d27c00640 1 --2- 172.21.15.66:0/1550348685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d20073c90 0x7f8d2015bc30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d27c00640 1 -- 172.21.15.66:0/1550348685 >> 172.21.15.66:0/1550348685 conn(0x7f8d200d0b50 msgr2=0x7f8d200d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d27c00640 1 -- 172.21.15.66:0/1550348685 shutdown_connections 2024-09-10T22:07:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.598+0000 7f8d27c00640 1 -- 172.21.15.66:0/1550348685 wait complete. 2024-09-10T22:07:53.626 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:07:53.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 -- 172.21.15.66:0/1857269897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40ae620 msgr2=0x7fbad40725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:53.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 --2- 172.21.15.66:0/1857269897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40ae620 0x7fbad40725d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009840 tx=0x7fbabc01c7c0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 -- 172.21.15.66:0/1857269897 shutdown_connections 2024-09-10T22:07:53.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 --2- 172.21.15.66:0/1857269897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40ae620 0x7fbad40725d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 --2- 172.21.15.66:0/1857269897 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbad40c4b10 0x7fbad406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 -- 172.21.15.66:0/1857269897 >> 172.21.15.66:0/1857269897 conn(0x7fbad40cfa30 msgr2=0x7fbad40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:53.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 -- 172.21.15.66:0/1857269897 shutdown_connections 2024-09-10T22:07:53.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 -- 172.21.15.66:0/1857269897 wait complete. 2024-09-10T22:07:53.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 Processor -- start 2024-09-10T22:07:53.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 -- start start 2024-09-10T22:07:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbad40ae620 0x7fbad41fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40c4b10 0x7fbad415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbad40c4ef0 0x7fbad4155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4168440 con 0x7fbad40c4b10 2024-09-10T22:07:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41682c0 con 0x7fbad40ae620 2024-09-10T22:07:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbada200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbad41685c0 con 0x7fbad40c4ef0 2024-09-10T22:07:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbacbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40c4b10 0x7fbad415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbacbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40c4b10 0x7fbad415b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37862/0 (socket says 172.21.15.66:37862) 2024-09-10T22:07:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbacbe00640 1 -- 172.21.15.66:0/347818307 learned_addr learned my addr 172.21.15.66:0/347818307 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:53.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbad3400640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbad40ae620 0x7fbad41fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:53.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.770+0000 7fbad3e00640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbad40c4ef0 0x7fbad4155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:53.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbacbe00640 1 -- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbad40c4ef0 msgr2=0x7fbad4155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:53.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbacbe00640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbad40c4ef0 0x7fbad4155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbacbe00640 1 -- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbad40ae620 msgr2=0x7fbad41fbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:53.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbacbe00640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbad40ae620 0x7fbad41fbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbacbe00640 1 -- 172.21.15.66:0/347818307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad40b2ca0 con 0x7fbad40c4b10 2024-09-10T22:07:53.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbacbe00640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40c4b10 0x7fbad415b260 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbabc000c10 tx=0x7fbabc0a6680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:53.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbacb400640 1 -- 172.21.15.66:0/347818307 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc0050c0 con 0x7fbad40c4b10 2024-09-10T22:07:53.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbacb400640 1 -- 172.21.15.66:0/347818307 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbabc005260 con 0x7fbad40c4b10 2024-09-10T22:07:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbacb400640 1 -- 172.21.15.66:0/347818307 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc005650 con 0x7fbad40c4b10 2024-09-10T22:07:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbada200640 1 -- 172.21.15.66:0/347818307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad40b3260 con 0x7fbad40c4b10 2024-09-10T22:07:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbada200640 1 -- 172.21.15.66:0/347818307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad40b3820 con 0x7fbad40c4b10 2024-09-10T22:07:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbada200640 1 -- 172.21.15.66:0/347818307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad40725d0 con 0x7fbad40c4b10 2024-09-10T22:07:53.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbacb400640 1 -- 172.21.15.66:0/347818307 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7fbabc0046d0 con 0x7fbad40c4b10 2024-09-10T22:07:53.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbacb400640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbab00737d0 0x7fbab0075c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:53.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.774+0000 7fbacb400640 1 -- 172.21.15.66:0/347818307 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fbabc12bbd0 con 0x7fbad40c4b10 2024-09-10T22:07:53.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.778+0000 7fbad3400640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbab00737d0 0x7fbab0075c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:53.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.778+0000 7fbad3400640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbab00737d0 0x7fbab0075c90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009be0 tx=0x7fbac40091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.778+0000 7fbacb400640 1 -- 172.21.15.66:0/347818307 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbabc0ac030 con 0x7fbad40c4b10 2024-09-10T22:07:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.978+0000 7fbada200640 1 -- 172.21.15.66:0/347818307 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fbad40b0990 con 0x7fbab00737d0 2024-09-10T22:07:53.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.978+0000 7fbacb400640 1 -- 172.21.15.66:0/347818307 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fbad40b0990 con 0x7fbab00737d0 2024-09-10T22:07:53.985 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:07:53.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:07:53.985 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:07:53.985 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:07:53.985 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:07:53.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.982+0000 7fbada200640 1 -- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbab00737d0 msgr2=0x7fbab0075c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:53.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.982+0000 7fbada200640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbab00737d0 0x7fbab0075c90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009be0 tx=0x7fbac40091e0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.982+0000 7fbada200640 1 -- 172.21.15.66:0/347818307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40c4b10 msgr2=0x7fbad415b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:53.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.982+0000 7fbada200640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40c4b10 0x7fbad415b260 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbabc000c10 tx=0x7fbabc0a6680 comp rx=0 tx=0).stop 2024-09-10T22:07:53.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.982+0000 7fbada200640 1 -- 172.21.15.66:0/347818307 shutdown_connections 2024-09-10T22:07:53.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.982+0000 7fbada200640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbab00737d0 0x7fbab0075c90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.982+0000 7fbada200640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbad40c4ef0 0x7fbad4155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.982+0000 7fbada200640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40c4b10 0x7fbad415b260 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.982+0000 7fbada200640 1 --2- 172.21.15.66:0/347818307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbad40ae620 0x7fbad41fbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:53.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.982+0000 7fbada200640 1 -- 172.21.15.66:0/347818307 >> 172.21.15.66:0/347818307 conn(0x7fbad40cfa30 msgr2=0x7fbad40af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:53.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.982+0000 7fbada200640 1 -- 172.21.15.66:0/347818307 shutdown_connections 2024-09-10T22:07:53.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:53.982+0000 7fbada200640 1 -- 172.21.15.66:0/347818307 wait complete. 2024-09-10T22:07:54.006 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-10T22:07:54.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.150+0000 7f9818000640 1 -- 172.21.15.66:0/3084399696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98100b2bf0 msgr2=0x7f98100d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:54.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.150+0000 7f9818000640 1 --2- 172.21.15.66:0/3084399696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98100b2bf0 0x7f98100d0090 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009840 tx=0x7f97fc01c820 comp rx=0 tx=0).stop 2024-09-10T22:07:54.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.150+0000 7f9818000640 1 -- 172.21.15.66:0/3084399696 shutdown_connections 2024-09-10T22:07:54.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.150+0000 7f9818000640 1 --2- 172.21.15.66:0/3084399696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98100b2bf0 0x7f98100d0090 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.150+0000 7f9818000640 1 --2- 172.21.15.66:0/3084399696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f981015f340 0x7f98100cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.150+0000 7f9818000640 1 -- 172.21.15.66:0/3084399696 >> 172.21.15.66:0/3084399696 conn(0x7f98100d1c50 msgr2=0x7f98100d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:54.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.150+0000 7f9818000640 1 -- 172.21.15.66:0/3084399696 shutdown_connections 2024-09-10T22:07:54.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.150+0000 7f9818000640 1 -- 172.21.15.66:0/3084399696 wait complete. 2024-09-10T22:07:54.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9818000640 1 Processor -- start 2024-09-10T22:07:54.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9818000640 1 -- start start 2024-09-10T22:07:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9818000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98100b2bf0 0x7f9810159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9818000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98100d2770 0x7f98101f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9818000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981015f340 0x7f98101fb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9818000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9810169190 con 0x7f98100b2bf0 2024-09-10T22:07:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9818000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9810169010 con 0x7f981015f340 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9818000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9810169310 con 0x7f98100d2770 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9815600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98100b2bf0 0x7f9810159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9815600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98100b2bf0 0x7f9810159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37894/0 (socket says 172.21.15.66:37894) 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9815600640 1 -- 172.21.15.66:0/3356800197 learned_addr learned my addr 172.21.15.66:0/3356800197 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9814c00640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98100d2770 0x7f98101f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9816000640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981015f340 0x7f98101fb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9815600640 1 -- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98100d2770 msgr2=0x7f98101f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9815600640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98100d2770 0x7f98101f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9815600640 1 -- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981015f340 msgr2=0x7f98101fb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9815600640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981015f340 0x7f98101fb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9815600640 1 -- 172.21.15.66:0/3356800197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98101f7d50 con 0x7f98100b2bf0 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9815600640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98100b2bf0 0x7f9810159ea0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f980400e7a0 tx=0x7f980400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f980d600640 1 -- 172.21.15.66:0/3356800197 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980400cc60 con 0x7f98100b2bf0 2024-09-10T22:07:54.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f980d600640 1 -- 172.21.15.66:0/3356800197 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f980400ee50 con 0x7f98100b2bf0 2024-09-10T22:07:54.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9818000640 1 -- 172.21.15.66:0/3356800197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98101fbd90 con 0x7f98100b2bf0 2024-09-10T22:07:54.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f980d600640 1 -- 172.21.15.66:0/3356800197 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804010670 con 0x7f98100b2bf0 2024-09-10T22:07:54.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9818000640 1 -- 172.21.15.66:0/3356800197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98101f8010 con 0x7f98100b2bf0 2024-09-10T22:07:54.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.154+0000 7f9818000640 1 -- 172.21.15.66:0/3356800197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f981015fac0 con 0x7f98100b2bf0 2024-09-10T22:07:54.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.158+0000 7f980d600640 1 -- 172.21.15.66:0/3356800197 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7f9804010810 con 0x7f98100b2bf0 2024-09-10T22:07:54.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.158+0000 7f980d600640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f97f0073660 0x7f97f0075b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:54.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.158+0000 7f980d600640 1 -- 172.21.15.66:0/3356800197 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f9804092970 con 0x7f98100b2bf0 2024-09-10T22:07:54.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.158+0000 7f9814c00640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f97f0073660 0x7f97f0075b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:54.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.158+0000 7f9814c00640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f97f0073660 0x7f97f0075b20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0af000 tx=0x7f97fc01b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:54.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.162+0000 7f980d600640 1 -- 172.21.15.66:0/3356800197 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f980405f130 con 0x7f98100b2bf0 2024-09-10T22:07:54.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.362+0000 7f9818000640 1 -- 172.21.15.66:0/3356800197 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9810072180 con 0x7f97f0073660 2024-09-10T22:07:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.370+0000 7f980d600640 1 -- 172.21.15.66:0/3356800197 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7f9810072180 con 0x7f97f0073660 2024-09-10T22:07:54.377 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:07:54.377 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:07:54.377 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:07:54.377 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone16323" 2024-09-10T22:07:54.377 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:54.377 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sv1" 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone70890" 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone17856" 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone33762" 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:07:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone93841" 2024-09-10T22:07:54.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:54.379 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:07:54.379 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone62795" 2024-09-10T22:07:54.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:07:54.379 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:07:54.379 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone87134" 2024-09-10T22:07:54.379 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:07:54.379 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:07:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.374+0000 7f9818000640 1 -- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f97f0073660 msgr2=0x7f97f0075b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.374+0000 7f9818000640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f97f0073660 0x7f97f0075b20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0af000 tx=0x7f97fc01b060 comp rx=0 tx=0).stop 2024-09-10T22:07:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.374+0000 7f9818000640 1 -- 172.21.15.66:0/3356800197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98100b2bf0 msgr2=0x7f9810159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.374+0000 7f9818000640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98100b2bf0 0x7f9810159ea0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f980400e7a0 tx=0x7f980400ec60 comp rx=0 tx=0).stop 2024-09-10T22:07:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.374+0000 7f9818000640 1 -- 172.21.15.66:0/3356800197 shutdown_connections 2024-09-10T22:07:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.374+0000 7f9818000640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f97f0073660 0x7f97f0075b20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.374+0000 7f9818000640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981015f340 0x7f98101fb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.374+0000 7f9818000640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98100d2770 0x7f98101f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.374+0000 7f9818000640 1 --2- 172.21.15.66:0/3356800197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98100b2bf0 0x7f9810159ea0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.374+0000 7f9818000640 1 -- 172.21.15.66:0/3356800197 >> 172.21.15.66:0/3356800197 conn(0x7f98100d1c50 msgr2=0x7f981015d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:54.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.374+0000 7f9818000640 1 -- 172.21.15.66:0/3356800197 shutdown_connections 2024-09-10T22:07:54.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.374+0000 7f9818000640 1 -- 172.21.15.66:0/3356800197 wait complete. 2024-09-10T22:07:54.399 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone16323 2024-09-10T22:07:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.546+0000 7fd3e6400640 1 -- 172.21.15.66:0/4279923105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3e00cd020 msgr2=0x7fd3e00ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.546+0000 7fd3e6400640 1 --2- 172.21.15.66:0/4279923105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3e00cd020 0x7fd3e00ce9c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009840 tx=0x7fd3c801c7f0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.546+0000 7fd3e6400640 1 -- 172.21.15.66:0/4279923105 shutdown_connections 2024-09-10T22:07:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.546+0000 7fd3e6400640 1 --2- 172.21.15.66:0/4279923105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3e00cd020 0x7fd3e00ce9c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.546+0000 7fd3e6400640 1 --2- 172.21.15.66:0/4279923105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3e015f240 0x7fd3e00ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.546+0000 7fd3e6400640 1 -- 172.21.15.66:0/4279923105 >> 172.21.15.66:0/4279923105 conn(0x7fd3e00d1c50 msgr2=0x7fd3e00d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.546+0000 7fd3e6400640 1 -- 172.21.15.66:0/4279923105 shutdown_connections 2024-09-10T22:07:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.546+0000 7fd3e6400640 1 -- 172.21.15.66:0/4279923105 wait complete. 2024-09-10T22:07:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.546+0000 7fd3e6400640 1 Processor -- start 2024-09-10T22:07:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.546+0000 7fd3e6400640 1 -- start start 2024-09-10T22:07:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3e6400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3e00cd020 0x7fd3e015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3e6400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3e00cef00 0x7fd3e01581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3e6400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3e015f240 0x7fd3e0071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3e6400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e0169070 con 0x7fd3e00cd020 2024-09-10T22:07:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3e6400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e0168ef0 con 0x7fd3e00cef00 2024-09-10T22:07:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3e6400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e01691f0 con 0x7fd3e015f240 2024-09-10T22:07:54.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3df400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3e00cd020 0x7fd3e015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3df400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3e00cd020 0x7fd3e015a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37898/0 (socket says 172.21.15.66:37898) 2024-09-10T22:07:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3df400640 1 -- 172.21.15.66:0/3556722902 learned_addr learned my addr 172.21.15.66:0/3556722902 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3d7e00640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3e00cef00 0x7fd3e01581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3dfe00640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3e015f240 0x7fd3e0071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3df400640 1 -- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3e015f240 msgr2=0x7fd3e0071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3df400640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3e015f240 0x7fd3e0071af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3df400640 1 -- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3e00cef00 msgr2=0x7fd3e01581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3df400640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3e00cef00 0x7fd3e01581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3df400640 1 -- 172.21.15.66:0/3556722902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e00c7630 con 0x7fd3e00cd020 2024-09-10T22:07:54.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3df400640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3e00cd020 0x7fd3e015a1e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000e7a0 tx=0x7fd3d000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:54.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3d7400640 1 -- 172.21.15.66:0/3556722902 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d000ccc0 con 0x7fd3e00cd020 2024-09-10T22:07:54.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3d7400640 1 -- 172.21.15.66:0/3556722902 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3d0010430 con 0x7fd3e00cd020 2024-09-10T22:07:54.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3e6400640 1 -- 172.21.15.66:0/3556722902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e00c7910 con 0x7fd3e00cd020 2024-09-10T22:07:54.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3d7400640 1 -- 172.21.15.66:0/3556722902 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0010860 con 0x7fd3e00cd020 2024-09-10T22:07:54.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3e6400640 1 -- 172.21.15.66:0/3556722902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e00ae960 con 0x7fd3e00cd020 2024-09-10T22:07:54.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.550+0000 7fd3e6400640 1 -- 172.21.15.66:0/3556722902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e00cfa30 con 0x7fd3e00cd020 2024-09-10T22:07:54.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.554+0000 7fd3d7400640 1 -- 172.21.15.66:0/3556722902 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7fd3d0010a00 con 0x7fd3e00cd020 2024-09-10T22:07:54.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.554+0000 7fd3d7400640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3bc073630 0x7fd3bc075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:54.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.554+0000 7fd3d7400640 1 -- 172.21.15.66:0/3556722902 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fd3d0092af0 con 0x7fd3e00cd020 2024-09-10T22:07:54.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.554+0000 7fd3d7e00640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3bc073630 0x7fd3bc075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:54.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.554+0000 7fd3d7e00640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3bc073630 0x7fd3bc075af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80af000 tx=0x7fd3c801b030 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:54.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.558+0000 7fd3d7400640 1 -- 172.21.15.66:0/3556722902 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd3d005f2b0 con 0x7fd3e00cd020 2024-09-10T22:07:54.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:54.762+0000 7fd3e6400640 1 -- 172.21.15.66:0/3556722902 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone16323", "target": ["mon-mgr", ""]}) -- 0x7fd3e00cc110 con 0x7fd3bc073630 2024-09-10T22:07:56.692 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:07:56.696 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:07:59.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3d7400640 1 -- 172.21.15.66:0/3556722902 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd3e00cc110 con 0x7fd3bc073630 2024-09-10T22:07:59.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3e6400640 1 -- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3bc073630 msgr2=0x7fd3bc075af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:59.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3e6400640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3bc073630 0x7fd3bc075af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80af000 tx=0x7fd3c801b030 comp rx=0 tx=0).stop 2024-09-10T22:07:59.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3e6400640 1 -- 172.21.15.66:0/3556722902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3e00cd020 msgr2=0x7fd3e015a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:59.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3e6400640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3e00cd020 0x7fd3e015a1e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000e7a0 tx=0x7fd3d000ec60 comp rx=0 tx=0).stop 2024-09-10T22:07:59.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3e6400640 1 -- 172.21.15.66:0/3556722902 shutdown_connections 2024-09-10T22:07:59.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3e6400640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3bc073630 0x7fd3bc075af0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3e6400640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3e015f240 0x7fd3e0071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3e6400640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3e00cef00 0x7fd3e01581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3e6400640 1 --2- 172.21.15.66:0/3556722902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3e00cd020 0x7fd3e015a1e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3e6400640 1 -- 172.21.15.66:0/3556722902 >> 172.21.15.66:0/3556722902 conn(0x7fd3e00d1c50 msgr2=0x7fd3e015d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:59.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3e6400640 1 -- 172.21.15.66:0/3556722902 shutdown_connections 2024-09-10T22:07:59.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.702+0000 7fd3e6400640 1 -- 172.21.15.66:0/3556722902 wait complete. 2024-09-10T22:07:59.720 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:59.720+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:59.720 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:59.720+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:59.720 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:59.720+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:59.720 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:59.720+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:59.720 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:07:59.720+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:07:59.736 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-10T22:07:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.886+0000 7f00b9000640 1 -- 172.21.15.66:0/2960609129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00b40cf1f0 msgr2=0x7f00b40cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.886+0000 7f00b9000640 1 --2- 172.21.15.66:0/2960609129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00b40cf1f0 0x7f00b40cfd70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f00b4073ae0 tx=0x7f00ac01c8c0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.886+0000 7f00b9000640 1 -- 172.21.15.66:0/2960609129 shutdown_connections 2024-09-10T22:07:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.886+0000 7f00b9000640 1 --2- 172.21.15.66:0/2960609129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00b40cf1f0 0x7f00b40cfd70 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.886+0000 7f00b9000640 1 --2- 172.21.15.66:0/2960609129 >> v2:172.21.15.94:3300/0 conn(0x7f00b40cdf90 0x7f00b40cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.886+0000 7f00b9000640 1 --2- 172.21.15.66:0/2960609129 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00b40ce8d0 0x7f00b40cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.886+0000 7f00b9000640 1 -- 172.21.15.66:0/2960609129 >> 172.21.15.66:0/2960609129 conn(0x7f00b40d1c60 msgr2=0x7f00b40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.886+0000 7f00b9000640 1 -- 172.21.15.66:0/2960609129 shutdown_connections 2024-09-10T22:07:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.886+0000 7f00b9000640 1 -- 172.21.15.66:0/2960609129 wait complete. 2024-09-10T22:07:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b9000640 1 Processor -- start 2024-09-10T22:07:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b9000640 1 -- start start 2024-09-10T22:07:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b9000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00b40cdf90 0x7f00b40c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00b40ce8d0 0x7f00b40b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b40cf1f0 0x7f00b4158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:59.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b9000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f00b40b1850 con 0x7f00b40ce8d0 2024-09-10T22:07:59.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b9000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00b40b16d0 con 0x7f00b40cf1f0 2024-09-10T22:07:59.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b9000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f00b40b19d0 con 0x7f00b40cdf90 2024-09-10T22:07:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00a9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00b40ce8d0 0x7f00b40b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00a9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00b40ce8d0 0x7f00b40b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37924/0 (socket says 172.21.15.66:37924) 2024-09-10T22:07:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00a9400640 1 -- 172.21.15.66:0/3172954039 learned_addr learned my addr 172.21.15.66:0/3172954039 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:07:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b1e00640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00b40cdf90 0x7f00b40c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b2800640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b40cf1f0 0x7f00b4158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00a9400640 1 -- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00b40cdf90 msgr2=0x7f00b40c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00a9400640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00b40cdf90 0x7f00b40c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00a9400640 1 -- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b40cf1f0 msgr2=0x7f00b4158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00a9400640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b40cf1f0 0x7f00b4158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00a9400640 1 -- 172.21.15.66:0/3172954039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b4158fc0 con 0x7f00b40ce8d0 2024-09-10T22:07:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00a9400640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00b40ce8d0 0x7f00b40b76c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f009c00d750 tx=0x7f009c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00aaa00640 1 -- 172.21.15.66:0/3172954039 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c017070 con 0x7f00b40ce8d0 2024-09-10T22:07:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00aaa00640 1 -- 172.21.15.66:0/3172954039 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f009c009d20 con 0x7f00b40ce8d0 2024-09-10T22:07:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00aaa00640 1 -- 172.21.15.66:0/3172954039 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c00de60 con 0x7f00b40ce8d0 2024-09-10T22:07:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b9000640 1 -- 172.21.15.66:0/3172954039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b415d9a0 con 0x7f00b40ce8d0 2024-09-10T22:07:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b9000640 1 -- 172.21.15.66:0/3172954039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b41592a0 con 0x7f00b40ce8d0 2024-09-10T22:07:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.890+0000 7f00b9000640 1 -- 172.21.15.66:0/3172954039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b415d790 con 0x7f00b40ce8d0 2024-09-10T22:07:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.894+0000 7f00aaa00640 1 -- 172.21.15.66:0/3172954039 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7f009c0106c0 con 0x7f00b40ce8d0 2024-09-10T22:07:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.894+0000 7f00aaa00640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0090073630 0x7f0090075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.894+0000 7f00aaa00640 1 -- 172.21.15.66:0/3172954039 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f009c0924b0 con 0x7f00b40ce8d0 2024-09-10T22:07:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.894+0000 7f00b1e00640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0090073630 0x7f0090075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.894+0000 7f00b1e00640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0090073630 0x7f0090075af0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f00b40c8ad0 tx=0x7f00a4009650 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:07:59.898+0000 7f00aaa00640 1 -- 172.21.15.66:0/3172954039 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f009c05ec70 con 0x7f00b40ce8d0 2024-09-10T22:08:00.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.102+0000 7f00b9000640 1 -- 172.21.15.66:0/3172954039 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00b40cecb0 con 0x7f0090073630 2024-09-10T22:08:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.114+0000 7f00aaa00640 1 -- 172.21.15.66:0/3172954039 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f00b40cecb0 con 0x7f0090073630 2024-09-10T22:08:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:00.122 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:08:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ss1" 2024-09-10T22:08:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:00.122 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:08:00.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.118+0000 7f00b9000640 1 -- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0090073630 msgr2=0x7f0090075af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:00.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.118+0000 7f00b9000640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0090073630 0x7f0090075af0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f00b40c8ad0 tx=0x7f00a4009650 comp rx=0 tx=0).stop 2024-09-10T22:08:00.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.118+0000 7f00b9000640 1 -- 172.21.15.66:0/3172954039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00b40ce8d0 msgr2=0x7f00b40b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:00.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.118+0000 7f00b9000640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00b40ce8d0 0x7f00b40b76c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f009c00d750 tx=0x7f009c00dc10 comp rx=0 tx=0).stop 2024-09-10T22:08:00.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.118+0000 7f00b9000640 1 -- 172.21.15.66:0/3172954039 shutdown_connections 2024-09-10T22:08:00.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.118+0000 7f00b9000640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0090073630 0x7f0090075af0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.118+0000 7f00b9000640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b40cf1f0 0x7f00b4158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.118+0000 7f00b9000640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00b40ce8d0 0x7f00b40b76c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.118+0000 7f00b9000640 1 --2- 172.21.15.66:0/3172954039 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00b40cdf90 0x7f00b40c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.118+0000 7f00b9000640 1 -- 172.21.15.66:0/3172954039 >> 172.21.15.66:0/3172954039 conn(0x7f00b40d1c60 msgr2=0x7f00b40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.118+0000 7f00b9000640 1 -- 172.21.15.66:0/3172954039 shutdown_connections 2024-09-10T22:08:00.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.118+0000 7f00b9000640 1 -- 172.21.15.66:0/3172954039 wait complete. 2024-09-10T22:08:00.143 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-10T22:08:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.290+0000 7fafc2400640 1 -- 172.21.15.66:0/2902482322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafbc15f300 msgr2=0x7fafbc0b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.290+0000 7fafc2400640 1 --2- 172.21.15.66:0/2902482322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafbc15f300 0x7fafbc0b1090 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009900 tx=0x7fafa801c870 comp rx=0 tx=0).stop 2024-09-10T22:08:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.290+0000 7fafc2400640 1 -- 172.21.15.66:0/2902482322 shutdown_connections 2024-09-10T22:08:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.290+0000 7fafc2400640 1 --2- 172.21.15.66:0/2902482322 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafbc0b15d0 0x7fafbc0cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.290+0000 7fafc2400640 1 --2- 172.21.15.66:0/2902482322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafbc15f300 0x7fafbc0b1090 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.290+0000 7fafc2400640 1 --2- 172.21.15.66:0/2902482322 >> v2:172.21.15.94:3300/0 conn(0x7fafbc15ef20 0x7fafbc0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.290+0000 7fafc2400640 1 -- 172.21.15.66:0/2902482322 >> 172.21.15.66:0/2902482322 conn(0x7fafbc0d1b10 msgr2=0x7fafbc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.290+0000 7fafc2400640 1 -- 172.21.15.66:0/2902482322 shutdown_connections 2024-09-10T22:08:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.290+0000 7fafc2400640 1 -- 172.21.15.66:0/2902482322 wait complete. 2024-09-10T22:08:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafc2400640 1 Processor -- start 2024-09-10T22:08:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafc2400640 1 -- start start 2024-09-10T22:08:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafc2400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc0b15d0 0x7fafbc151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafc2400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafbc15ef20 0x7fafbc1518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafc2400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafbc15f300 0x7fafbc0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafc2400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc15fd90 con 0x7fafbc15ef20 2024-09-10T22:08:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafc2400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc15fc10 con 0x7fafbc0b15d0 2024-09-10T22:08:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafc2400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc15ff10 con 0x7fafbc15f300 2024-09-10T22:08:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafbbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafbc15ef20 0x7fafbc1518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafbbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafbc15ef20 0x7fafbc1518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37950/0 (socket says 172.21.15.66:37950) 2024-09-10T22:08:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafc1000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc0b15d0 0x7fafbc151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafbbe00640 1 -- 172.21.15.66:0/3797909951 learned_addr learned my addr 172.21.15.66:0/3797909951 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafc1a00640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafbc15f300 0x7fafbc0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafbbe00640 1 -- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafbc15f300 msgr2=0x7fafbc0aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafbbe00640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafbc15f300 0x7fafbc0aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafbbe00640 1 -- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc0b15d0 msgr2=0x7fafbc151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafbbe00640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc0b15d0 0x7fafbc151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafbbe00640 1 -- 172.21.15.66:0/3797909951 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc0cb2d0 con 0x7fafbc15ef20 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafc1a00640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafbc15f300 0x7fafbc0aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafbbe00640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafbc15ef20 0x7fafbc1518f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009850 tx=0x7fafa8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafb8c00640 1 -- 172.21.15.66:0/3797909951 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa80b2040 con 0x7fafbc15ef20 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafb8c00640 1 -- 172.21.15.66:0/3797909951 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafa8004330 con 0x7fafbc15ef20 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafb8c00640 1 -- 172.21.15.66:0/3797909951 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8005290 con 0x7fafbc15ef20 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.294+0000 7fafc2400640 1 -- 172.21.15.66:0/3797909951 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc0afdf0 con 0x7fafbc15ef20 2024-09-10T22:08:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.298+0000 7fafc2400640 1 -- 172.21.15.66:0/3797909951 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc0cb570 con 0x7fafbc15ef20 2024-09-10T22:08:00.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.298+0000 7fafc2400640 1 -- 172.21.15.66:0/3797909951 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafbc0b0310 con 0x7fafbc15ef20 2024-09-10T22:08:00.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.298+0000 7fafb8c00640 1 -- 172.21.15.66:0/3797909951 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7fafbc0d0100 con 0x7fafbc15ef20 2024-09-10T22:08:00.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.298+0000 7fafb8c00640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faf9c073630 0x7faf9c075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:00.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.298+0000 7fafb8c00640 1 -- 172.21.15.66:0/3797909951 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fafa812b260 con 0x7fafbc15ef20 2024-09-10T22:08:00.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.298+0000 7fafc1000640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faf9c073630 0x7faf9c075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:00.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.302+0000 7fafc1000640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faf9c073630 0x7faf9c075af0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009be0 tx=0x7fafb00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:00.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.302+0000 7fafb8c00640 1 -- 172.21.15.66:0/3797909951 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fafbc0b0310 con 0x7fafbc15ef20 2024-09-10T22:08:00.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.506+0000 7fafc2400640 1 -- 172.21.15.66:0/3797909951 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fafbc160430 con 0x7faf9c073630 2024-09-10T22:08:00.546 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:00.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafb8c00640 1 -- 172.21.15.66:0/3797909951 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fafbc160430 con 0x7faf9c073630 2024-09-10T22:08:00.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafc2400640 1 -- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faf9c073630 msgr2=0x7faf9c075af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:00.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafc2400640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faf9c073630 0x7faf9c075af0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009be0 tx=0x7fafb00091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafc2400640 1 -- 172.21.15.66:0/3797909951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafbc15ef20 msgr2=0x7fafbc1518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:00.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafc2400640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafbc15ef20 0x7fafbc1518f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009850 tx=0x7fafa8005de0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafc2400640 1 -- 172.21.15.66:0/3797909951 shutdown_connections 2024-09-10T22:08:00.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafc2400640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faf9c073630 0x7faf9c075af0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafc2400640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafbc15f300 0x7fafbc0aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafc2400640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafbc15ef20 0x7fafbc1518f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafc2400640 1 --2- 172.21.15.66:0/3797909951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc0b15d0 0x7fafbc151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafc2400640 1 -- 172.21.15.66:0/3797909951 >> 172.21.15.66:0/3797909951 conn(0x7fafbc0d1b10 msgr2=0x7fafbc165680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:00.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafc2400640 1 -- 172.21.15.66:0/3797909951 shutdown_connections 2024-09-10T22:08:00.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.542+0000 7fafc2400640 1 -- 172.21.15.66:0/3797909951 wait complete. 2024-09-10T22:08:00.569 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-10T22:08:00.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.714+0000 7f1752c00640 1 -- 172.21.15.66:0/1178867755 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f174c0ce660 msgr2=0x7f174c0cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:00.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.714+0000 7f1752c00640 1 --2- 172.21.15.66:0/1178867755 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f174c0ce660 0x7f174c0cea40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f173c009840 tx=0x7f173c01c790 comp rx=0 tx=0).stop 2024-09-10T22:08:00.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.714+0000 7f1752c00640 1 -- 172.21.15.66:0/1178867755 shutdown_connections 2024-09-10T22:08:00.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.714+0000 7f1752c00640 1 --2- 172.21.15.66:0/1178867755 >> v2:172.21.15.94:3300/0 conn(0x7f174c15e460 0x7f174c0ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.714+0000 7f1752c00640 1 --2- 172.21.15.66:0/1178867755 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f174c0ce660 0x7f174c0cea40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.714+0000 7f1752c00640 1 -- 172.21.15.66:0/1178867755 >> 172.21.15.66:0/1178867755 conn(0x7f174c0d1c50 msgr2=0x7f174c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:00.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.714+0000 7f1752c00640 1 -- 172.21.15.66:0/1178867755 shutdown_connections 2024-09-10T22:08:00.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.714+0000 7f1752c00640 1 -- 172.21.15.66:0/1178867755 wait complete. 2024-09-10T22:08:00.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1752c00640 1 Processor -- start 2024-09-10T22:08:00.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1752c00640 1 -- start start 2024-09-10T22:08:00.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1752c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f174c06ebc0 0x7f174c0c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:00.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1752c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f174c0ce660 0x7f174c0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:00.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1752c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f174c15e460 0x7f174c0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:00.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1752c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f174c0d0520 con 0x7f174c15e460 2024-09-10T22:08:00.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1752c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f174c0d03a0 con 0x7f174c0ce660 2024-09-10T22:08:00.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1752c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f174c0d06a0 con 0x7f174c06ebc0 2024-09-10T22:08:00.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f174be00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f174c06ebc0 0x7f174c0c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:00.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1750c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f174c15e460 0x7f174c0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:00.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1743e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f174c0ce660 0x7f174c0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:00.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f174be00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f174c06ebc0 0x7f174c0c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:51872/0 (socket says 172.21.15.66:51872) 2024-09-10T22:08:00.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f174be00640 1 -- 172.21.15.66:0/2809936319 learned_addr learned my addr 172.21.15.66:0/2809936319 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:00.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f174be00640 1 -- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f174c0ce660 msgr2=0x7f174c0cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:00.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f174be00640 1 --2- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f174c0ce660 0x7f174c0cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f174be00640 1 -- 172.21.15.66:0/2809936319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f174c15e460 msgr2=0x7f174c0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:00.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f174be00640 1 --2- 172.21.15.66:0/2809936319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f174c15e460 0x7f174c0af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f174be00640 1 -- 172.21.15.66:0/2809936319 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c0c9c70 con 0x7f174c06ebc0 2024-09-10T22:08:00.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f174be00640 1 --2- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f174c06ebc0 0x7f174c0c9730 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f173c01d030 tx=0x7f173c0a6f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:00.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1748c00640 1 -- 172.21.15.66:0/2809936319 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c0ac070 con 0x7f174c06ebc0 2024-09-10T22:08:00.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1748c00640 1 -- 172.21.15.66:0/2809936319 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f173c005130 con 0x7f174c06ebc0 2024-09-10T22:08:00.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1748c00640 1 -- 172.21.15.66:0/2809936319 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c005560 con 0x7f174c06ebc0 2024-09-10T22:08:00.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1752c00640 1 -- 172.21.15.66:0/2809936319 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c0ca8d0 con 0x7f174c06ebc0 2024-09-10T22:08:00.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1752c00640 1 -- 172.21.15.66:0/2809936319 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c0cae90 con 0x7f174c06ebc0 2024-09-10T22:08:00.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.718+0000 7f1752c00640 1 -- 172.21.15.66:0/2809936319 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174c0d0b90 con 0x7f174c06ebc0 2024-09-10T22:08:00.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.722+0000 7f1748c00640 1 -- 172.21.15.66:0/2809936319 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 26) ==== 97873+0+0 (secure 0 0 0) 0x7f173c0046d0 con 0x7f174c06ebc0 2024-09-10T22:08:00.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.722+0000 7f1748c00640 1 --2- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1728073630 0x7f1728075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:00.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.722+0000 7f1748c00640 1 -- 172.21.15.66:0/2809936319 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f173c12c120 con 0x7f174c06ebc0 2024-09-10T22:08:00.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.722+0000 7f1743e00640 1 --2- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1728073630 0x7f1728075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:00.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.722+0000 7f1743e00640 1 --2- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1728073630 0x7f1728075af0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f174c15bcc0 tx=0x7f17340091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:00.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.726+0000 7f1748c00640 1 -- 172.21.15.66:0/2809936319 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f173c0c0070 con 0x7f174c06ebc0 2024-09-10T22:08:00.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.930+0000 7f1752c00640 1 -- 172.21.15.66:0/2809936319 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f174c06fd20 con 0x7f1728073630 2024-09-10T22:08:00.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1748c00640 1 -- 172.21.15.66:0/2809936319 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f174c06fd20 con 0x7f1728073630 2024-09-10T22:08:00.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1752c00640 1 -- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1728073630 msgr2=0x7f1728075af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:00.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1752c00640 1 --2- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1728073630 0x7f1728075af0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f174c15bcc0 tx=0x7f17340091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1752c00640 1 -- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f174c06ebc0 msgr2=0x7f174c0c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:00.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1752c00640 1 --2- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f174c06ebc0 0x7f174c0c9730 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f173c01d030 tx=0x7f173c0a6f90 comp rx=0 tx=0).stop 2024-09-10T22:08:00.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1752c00640 1 -- 172.21.15.66:0/2809936319 shutdown_connections 2024-09-10T22:08:00.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1752c00640 1 --2- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1728073630 0x7f1728075af0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1752c00640 1 --2- 172.21.15.66:0/2809936319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f174c15e460 0x7f174c0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1752c00640 1 --2- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f174c0ce660 0x7f174c0cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1752c00640 1 --2- 172.21.15.66:0/2809936319 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f174c06ebc0 0x7f174c0c9730 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:00.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1752c00640 1 -- 172.21.15.66:0/2809936319 >> 172.21.15.66:0/2809936319 conn(0x7f174c0d1c50 msgr2=0x7f174c072060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:00.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1752c00640 1 -- 172.21.15.66:0/2809936319 shutdown_connections 2024-09-10T22:08:00.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:00.946+0000 7f1752c00640 1 -- 172.21.15.66:0/2809936319 wait complete. 2024-09-10T22:08:00.975 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone70890 2024-09-10T22:08:01.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 -- 172.21.15.66:0/1567758559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4594004d50 msgr2=0x7f45940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 --2- 172.21.15.66:0/1567758559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4594004d50 0x7f45940051d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009eb0 tx=0x7f45a801d680 comp rx=0 tx=0).stop 2024-09-10T22:08:01.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45ae800640 1 -- 172.21.15.66:0/1567758559 reap_dead start 2024-09-10T22:08:01.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 -- 172.21.15.66:0/1567758559 shutdown_connections 2024-09-10T22:08:01.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 -- 172.21.15.66:0/1567758559 >> 172.21.15.66:0/1567758559 conn(0x7f45b009aa50 msgr2=0x7f45b009ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:01.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 -- 172.21.15.66:0/1567758559 shutdown_connections 2024-09-10T22:08:01.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 -- 172.21.15.66:0/1567758559 wait complete. 2024-09-10T22:08:01.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 Processor -- start 2024-09-10T22:08:01.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 -- start start 2024-09-10T22:08:01.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4594004d50 0x7f45b0155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:01.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b015ad40 0x7f45b015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:01.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45b00b7050 0x7f45b00b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45b00b3010 con 0x7f45b015ad40 2024-09-10T22:08:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f45b00b2e90 con 0x7f4594004d50 2024-09-10T22:08:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45af200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f45b00cc8e0 con 0x7f45b00b7050 2024-09-10T22:08:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.130+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b015ad40 0x7f45b015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45ae800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45b00b7050 0x7f45b00b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b015ad40 0x7f45b015b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37964/0 (socket says 172.21.15.66:37964) 2024-09-10T22:08:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45a7e00640 1 -- 172.21.15.66:0/1034527110 learned_addr learned my addr 172.21.15.66:0/1034527110 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45ade00640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4594004d50 0x7f45b0155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45a7e00640 1 -- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45b00b7050 msgr2=0x7f45b00b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45a7e00640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45b00b7050 0x7f45b00b74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45a7e00640 1 -- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4594004d50 msgr2=0x7f45b0155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45a7e00640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4594004d50 0x7f45b0155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45a7e00640 1 -- 172.21.15.66:0/1034527110 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b01596a0 con 0x7f45b015ad40 2024-09-10T22:08:01.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45a7e00640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b015ad40 0x7f45b015b1c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4598006660 tx=0x7f45980039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:01.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45a6000640 1 -- 172.21.15.66:0/1034527110 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598015070 con 0x7f45b015ad40 2024-09-10T22:08:01.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45a6000640 1 -- 172.21.15.66:0/1034527110 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4598011920 con 0x7f45b015ad40 2024-09-10T22:08:01.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45a6000640 1 -- 172.21.15.66:0/1034527110 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598011d70 con 0x7f45b015ad40 2024-09-10T22:08:01.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45af200640 1 -- 172.21.15.66:0/1034527110 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b0159960 con 0x7f45b015ad40 2024-09-10T22:08:01.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45af200640 1 -- 172.21.15.66:0/1034527110 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b0159f20 con 0x7f45b015ad40 2024-09-10T22:08:01.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.134+0000 7f45af200640 1 -- 172.21.15.66:0/1034527110 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b015b700 con 0x7f45b015ad40 2024-09-10T22:08:01.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.138+0000 7f45a6000640 1 -- 172.21.15.66:0/1034527110 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f45980103c0 con 0x7f45b015ad40 2024-09-10T22:08:01.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.138+0000 7f45a6000640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f458c0736f0 0x7f458c075bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:01.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.138+0000 7f45a6000640 1 -- 172.21.15.66:0/1034527110 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f4598092cd0 con 0x7f45b015ad40 2024-09-10T22:08:01.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.138+0000 7f45ade00640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f458c0736f0 0x7f458c075bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:01.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.138+0000 7f45ade00640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f458c0736f0 0x7f458c075bb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009c60 tx=0x7f45a00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:01.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.142+0000 7f45a6000640 1 -- 172.21.15.66:0/1034527110 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f459805f530 con 0x7f45b015ad40 2024-09-10T22:08:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.342+0000 7f45af200640 1 -- 172.21.15.66:0/1034527110 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone70890", "target": ["mon-mgr", ""]}) -- 0x7f45b009f430 con 0x7f458c0736f0 2024-09-10T22:08:01.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.358+0000 7f45a6000640 1 -- 172.21.15.66:0/1034527110 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f45b009f430 con 0x7f458c0736f0 2024-09-10T22:08:01.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.362+0000 7f45af200640 1 -- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f458c0736f0 msgr2=0x7f458c075bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.362+0000 7f45af200640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f458c0736f0 0x7f458c075bb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009c60 tx=0x7f45a00091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.362+0000 7f45af200640 1 -- 172.21.15.66:0/1034527110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b015ad40 msgr2=0x7f45b015b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.362+0000 7f45af200640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b015ad40 0x7f45b015b1c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4598006660 tx=0x7f45980039d0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.362+0000 7f45af200640 1 -- 172.21.15.66:0/1034527110 shutdown_connections 2024-09-10T22:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.362+0000 7f45af200640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f458c0736f0 0x7f458c075bb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.362+0000 7f45af200640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45b00b7050 0x7f45b00b74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.362+0000 7f45af200640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b015ad40 0x7f45b015b1c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.362+0000 7f45af200640 1 --2- 172.21.15.66:0/1034527110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4594004d50 0x7f45b0155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.362+0000 7f45af200640 1 -- 172.21.15.66:0/1034527110 >> 172.21.15.66:0/1034527110 conn(0x7f45b009aa50 msgr2=0x7f45b00ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.362+0000 7f45af200640 1 -- 172.21.15.66:0/1034527110 shutdown_connections 2024-09-10T22:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.362+0000 7f45af200640 1 -- 172.21.15.66:0/1034527110 wait complete. 2024-09-10T22:08:01.388 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone17856 2024-09-10T22:08:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 -- 172.21.15.66:0/3899537225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9100ce8d0 msgr2=0x7ff9100cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 --2- 172.21.15.66:0/3899537225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9100ce8d0 0x7ff9100cecb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff900009900 tx=0x7ff90001c8e0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 -- 172.21.15.66:0/3899537225 shutdown_connections 2024-09-10T22:08:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 --2- 172.21.15.66:0/3899537225 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9100cf1f0 0x7ff9100cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 --2- 172.21.15.66:0/3899537225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9100ce8d0 0x7ff9100cecb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 -- 172.21.15.66:0/3899537225 >> 172.21.15.66:0/3899537225 conn(0x7ff9100d1c60 msgr2=0x7ff9100d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 -- 172.21.15.66:0/3899537225 shutdown_connections 2024-09-10T22:08:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 -- 172.21.15.66:0/3899537225 wait complete. 2024-09-10T22:08:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 Processor -- start 2024-09-10T22:08:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 -- start start 2024-09-10T22:08:01.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9100cdf90 0x7ff9101fbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:01.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9100ce8d0 0x7ff910159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:01.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9100cf1f0 0x7ff9100cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:01.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101691a0 con 0x7ff9100cf1f0 2024-09-10T22:08:01.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff910169020 con 0x7ff9100cdf90 2024-09-10T22:08:01.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.538+0000 7ff916800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff910169320 con 0x7ff9100ce8d0 2024-09-10T22:08:01.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff90fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9100cf1f0 0x7ff9100cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:01.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff90fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9100cf1f0 0x7ff9100cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37968/0 (socket says 172.21.15.66:37968) 2024-09-10T22:08:01.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff90fe00640 1 -- 172.21.15.66:0/87692094 learned_addr learned my addr 172.21.15.66:0/87692094 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:01.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff907e00640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9100ce8d0 0x7ff910159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:01.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff90f400640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9100cdf90 0x7ff9101fbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:01.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff90fe00640 1 -- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9100ce8d0 msgr2=0x7ff910159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff90fe00640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9100ce8d0 0x7ff910159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff90fe00640 1 -- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9100cdf90 msgr2=0x7ff9101fbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff90fe00640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9100cdf90 0x7ff9101fbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff90fe00640 1 -- 172.21.15.66:0/87692094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9100c8a20 con 0x7ff9100cf1f0 2024-09-10T22:08:01.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff90fe00640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9100cf1f0 0x7ff9100cb900 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff90800ea80 tx=0x7ff90800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff907400640 1 -- 172.21.15.66:0/87692094 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908004b30 con 0x7ff9100cf1f0 2024-09-10T22:08:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff907400640 1 -- 172.21.15.66:0/87692094 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff908004cf0 con 0x7ff9100cf1f0 2024-09-10T22:08:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff916800640 1 -- 172.21.15.66:0/87692094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9100c6480 con 0x7ff9100cf1f0 2024-09-10T22:08:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff907400640 1 -- 172.21.15.66:0/87692094 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908005280 con 0x7ff9100cf1f0 2024-09-10T22:08:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff916800640 1 -- 172.21.15.66:0/87692094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9101fb780 con 0x7ff9100cf1f0 2024-09-10T22:08:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff916800640 1 -- 172.21.15.66:0/87692094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9100cecb0 con 0x7ff9100cf1f0 2024-09-10T22:08:01.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.542+0000 7ff907400640 1 -- 172.21.15.66:0/87692094 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7ff908005420 con 0x7ff9100cf1f0 2024-09-10T22:08:01.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.546+0000 7ff907400640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8ec0736f0 0x7ff8ec075bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:01.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.546+0000 7ff907400640 1 -- 172.21.15.66:0/87692094 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7ff908092c40 con 0x7ff9100cf1f0 2024-09-10T22:08:01.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.546+0000 7ff90f400640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8ec0736f0 0x7ff8ec075bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:01.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.546+0000 7ff90f400640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8ec0736f0 0x7ff8ec075bb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff9000af000 tx=0x7ff90001b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:01.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.554+0000 7ff907400640 1 -- 172.21.15.66:0/87692094 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff90805f3e0 con 0x7ff9100cf1f0 2024-09-10T22:08:01.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.754+0000 7ff916800640 1 -- 172.21.15.66:0/87692094 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone17856", "target": ["mon-mgr", ""]}) -- 0x7ff9101fc4a0 con 0x7ff8ec0736f0 2024-09-10T22:08:01.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.774+0000 7ff907400640 1 -- 172.21.15.66:0/87692094 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff900004140 con 0x7ff8ec0736f0 2024-09-10T22:08:01.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.778+0000 7ff916800640 1 -- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8ec0736f0 msgr2=0x7ff8ec075bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.778+0000 7ff916800640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8ec0736f0 0x7ff8ec075bb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff9000af000 tx=0x7ff90001b120 comp rx=0 tx=0).stop 2024-09-10T22:08:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.778+0000 7ff916800640 1 -- 172.21.15.66:0/87692094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9100cf1f0 msgr2=0x7ff9100cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.778+0000 7ff916800640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9100cf1f0 0x7ff9100cb900 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff90800ea80 tx=0x7ff90800ef40 comp rx=0 tx=0).stop 2024-09-10T22:08:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.778+0000 7ff916800640 1 -- 172.21.15.66:0/87692094 shutdown_connections 2024-09-10T22:08:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.778+0000 7ff916800640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8ec0736f0 0x7ff8ec075bb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.778+0000 7ff916800640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9100cf1f0 0x7ff9100cb900 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.778+0000 7ff916800640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9100ce8d0 0x7ff910159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.778+0000 7ff916800640 1 --2- 172.21.15.66:0/87692094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9100cdf90 0x7ff9101fbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.778+0000 7ff916800640 1 -- 172.21.15.66:0/87692094 >> 172.21.15.66:0/87692094 conn(0x7ff9100d1c60 msgr2=0x7ff91015f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.778+0000 7ff916800640 1 -- 172.21.15.66:0/87692094 shutdown_connections 2024-09-10T22:08:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.778+0000 7ff916800640 1 -- 172.21.15.66:0/87692094 wait complete. 2024-09-10T22:08:01.810 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone33762 2024-09-10T22:08:01.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 -- 172.21.15.66:0/245996143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa924072640 msgr2=0x7fa924072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 --2- 172.21.15.66:0/245996143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa924072640 0x7fa924072ad0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa924074ea0 tx=0x7fa91c01cc60 comp rx=0 tx=0).stop 2024-09-10T22:08:01.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 -- 172.21.15.66:0/245996143 shutdown_connections 2024-09-10T22:08:01.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 --2- 172.21.15.66:0/245996143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa924072640 0x7fa924072ad0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 --2- 172.21.15.66:0/245996143 >> v2:172.21.15.94:3300/0 conn(0x7fa9240cce70 0x7fa9240cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 --2- 172.21.15.66:0/245996143 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa92415ab00 0x7fa92406e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 -- 172.21.15.66:0/245996143 >> 172.21.15.66:0/245996143 conn(0x7fa9240cfaa0 msgr2=0x7fa9240cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:01.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 -- 172.21.15.66:0/245996143 shutdown_connections 2024-09-10T22:08:01.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 -- 172.21.15.66:0/245996143 wait complete. 2024-09-10T22:08:01.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 Processor -- start 2024-09-10T22:08:01.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 -- start start 2024-09-10T22:08:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa924072640 0x7fa9241faff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.958+0000 7fa929400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9240cce70 0x7fa92414cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa929400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92415ab00 0x7fa92414d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa929400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa9240b3c30 con 0x7fa92415ab00 2024-09-10T22:08:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa929400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9240b3ab0 con 0x7fa9240cce70 2024-09-10T22:08:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa929400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa9240b3db0 con 0x7fa924072640 2024-09-10T22:08:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa923200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92415ab00 0x7fa92414d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa923200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92415ab00 0x7fa92414d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37996/0 (socket says 172.21.15.66:37996) 2024-09-10T22:08:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa923200640 1 -- 172.21.15.66:0/2388482556 learned_addr learned my addr 172.21.15.66:0/2388482556 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa922800640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa924072640 0x7fa9241faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa919e00640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9240cce70 0x7fa92414cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa923200640 1 -- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa924072640 msgr2=0x7fa9241faff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa923200640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa924072640 0x7fa9241faff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa923200640 1 -- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9240cce70 msgr2=0x7fa92414cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa923200640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9240cce70 0x7fa92414cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa923200640 1 -- 172.21.15.66:0/2388482556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9241fb6d0 con 0x7fa92415ab00 2024-09-10T22:08:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa919e00640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9240cce70 0x7fa92414cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa922800640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa924072640 0x7fa9241faff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa923200640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92415ab00 0x7fa92414d290 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009bd0 tx=0x7fa91c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa91b400640 1 -- 172.21.15.66:0/2388482556 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0bb070 con 0x7fa92415ab00 2024-09-10T22:08:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa91b400640 1 -- 172.21.15.66:0/2388482556 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa91c004d40 con 0x7fa92415ab00 2024-09-10T22:08:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa91b400640 1 -- 172.21.15.66:0/2388482556 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0b2040 con 0x7fa92415ab00 2024-09-10T22:08:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa929400640 1 -- 172.21.15.66:0/2388482556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9241fb9b0 con 0x7fa92415ab00 2024-09-10T22:08:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa929400640 1 -- 172.21.15.66:0/2388482556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92414c830 con 0x7fa92415ab00 2024-09-10T22:08:01.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa929400640 1 -- 172.21.15.66:0/2388482556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9240cbfa0 con 0x7fa92415ab00 2024-09-10T22:08:01.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.962+0000 7fa91b400640 1 -- 172.21.15.66:0/2388482556 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fa91c0b5030 con 0x7fa92415ab00 2024-09-10T22:08:01.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.966+0000 7fa91b400640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa9000736c0 0x7fa900075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:01.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.966+0000 7fa91b400640 1 -- 172.21.15.66:0/2388482556 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fa91c12c080 con 0x7fa92415ab00 2024-09-10T22:08:01.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.966+0000 7fa922800640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa9000736c0 0x7fa900075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.966+0000 7fa922800640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa9000736c0 0x7fa900075b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa91400bbd0 tx=0x7fa91400b5b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:01.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:01.966+0000 7fa91b400640 1 -- 172.21.15.66:0/2388482556 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa91c016970 con 0x7fa92415ab00 2024-09-10T22:08:02.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.170+0000 7fa929400640 1 -- 172.21.15.66:0/2388482556 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone33762", "target": ["mon-mgr", ""]}) -- 0x7fa9240d07c0 con 0x7fa9000736c0 2024-09-10T22:08:02.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.190+0000 7fa91b400640 1 -- 172.21.15.66:0/2388482556 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa9240d07c0 con 0x7fa9000736c0 2024-09-10T22:08:02.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.190+0000 7fa929400640 1 -- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa9000736c0 msgr2=0x7fa900075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.190+0000 7fa929400640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa9000736c0 0x7fa900075b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa91400bbd0 tx=0x7fa91400b5b0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.190+0000 7fa929400640 1 -- 172.21.15.66:0/2388482556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92415ab00 msgr2=0x7fa92414d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.190+0000 7fa929400640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92415ab00 0x7fa92414d290 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009bd0 tx=0x7fa91c009300 comp rx=0 tx=0).stop 2024-09-10T22:08:02.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.194+0000 7fa929400640 1 -- 172.21.15.66:0/2388482556 shutdown_connections 2024-09-10T22:08:02.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.194+0000 7fa929400640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa9000736c0 0x7fa900075b80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.194+0000 7fa929400640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92415ab00 0x7fa92414d290 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.194+0000 7fa929400640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9240cce70 0x7fa92414cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.194+0000 7fa929400640 1 --2- 172.21.15.66:0/2388482556 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa924072640 0x7fa9241faff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.194+0000 7fa929400640 1 -- 172.21.15.66:0/2388482556 >> 172.21.15.66:0/2388482556 conn(0x7fa9240cfaa0 msgr2=0x7fa92414f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:02.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.194+0000 7fa929400640 1 -- 172.21.15.66:0/2388482556 shutdown_connections 2024-09-10T22:08:02.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.194+0000 7fa929400640 1 -- 172.21.15.66:0/2388482556 wait complete. 2024-09-10T22:08:02.236 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone93841 2024-09-10T22:08:02.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.466+0000 7f3c72400640 1 -- 172.21.15.66:0/324407778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c6c0a1c50 msgr2=0x7f3c6c06f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.466+0000 7f3c72400640 1 --2- 172.21.15.66:0/324407778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c6c0a1c50 0x7f3c6c06f870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009bc0 tx=0x7f3c6401cb90 comp rx=0 tx=0).stop 2024-09-10T22:08:02.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.466+0000 7f3c72400640 1 -- 172.21.15.66:0/324407778 shutdown_connections 2024-09-10T22:08:02.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.466+0000 7f3c72400640 1 --2- 172.21.15.66:0/324407778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c6c0a1c50 0x7f3c6c06f870 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.466+0000 7f3c72400640 1 --2- 172.21.15.66:0/324407778 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c6c0cb640 0x7f3c6c0cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.466+0000 7f3c72400640 1 -- 172.21.15.66:0/324407778 >> 172.21.15.66:0/324407778 conn(0x7f3c6c0d0b10 msgr2=0x7f3c6c0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:02.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.466+0000 7f3c72400640 1 -- 172.21.15.66:0/324407778 shutdown_connections 2024-09-10T22:08:02.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.466+0000 7f3c72400640 1 -- 172.21.15.66:0/324407778 wait complete. 2024-09-10T22:08:02.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.470+0000 7f3c72400640 1 Processor -- start 2024-09-10T22:08:02.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.470+0000 7f3c72400640 1 -- start start 2024-09-10T22:08:02.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c72400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c0a1c50 0x7f3c6c16b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c72400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c6c0cb160 0x7f3c6c0ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c72400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c6c0cb640 0x7f3c6c204680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c72400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c06cfb0 con 0x7f3c6c0a1c50 2024-09-10T22:08:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c72400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c06ce30 con 0x7f3c6c0cb160 2024-09-10T22:08:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c72400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6c06d130 con 0x7f3c6c0cb640 2024-09-10T22:08:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c6aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c6c0cb160 0x7f3c6c0ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c6b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c0a1c50 0x7f3c6c16b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c6b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c0a1c50 0x7f3c6c16b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56538/0 (socket says 172.21.15.66:56538) 2024-09-10T22:08:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c6b400640 1 -- 172.21.15.66:0/85449027 learned_addr learned my addr 172.21.15.66:0/85449027 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c6be00640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c6c0cb640 0x7f3c6c204680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c6aa00640 1 -- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c6c0cb640 msgr2=0x7f3c6c204680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c6aa00640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c6c0cb640 0x7f3c6c204680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c6aa00640 1 -- 172.21.15.66:0/85449027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c0a1c50 msgr2=0x7f3c6c16b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c6aa00640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c0a1c50 0x7f3c6c16b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c6aa00640 1 -- 172.21.15.66:0/85449027 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6c205130 con 0x7f3c6c0cb160 2024-09-10T22:08:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c6b400640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c0a1c50 0x7f3c6c16b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c6aa00640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c6c0cb160 0x7f3c6c0ca340 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c3e0 tx=0x7f3c5c00c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c63400640 1 -- 172.21.15.66:0/85449027 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c017070 con 0x7f3c6c0cb160 2024-09-10T22:08:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c63400640 1 -- 172.21.15.66:0/85449027 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c5c003600 con 0x7f3c6c0cb160 2024-09-10T22:08:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c63400640 1 -- 172.21.15.66:0/85449027 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c003a90 con 0x7f3c6c0cb160 2024-09-10T22:08:02.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.474+0000 7f3c72400640 1 -- 172.21.15.66:0/85449027 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6c205410 con 0x7f3c6c0cb160 2024-09-10T22:08:02.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.478+0000 7f3c72400640 1 -- 172.21.15.66:0/85449027 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c6c0c9e20 con 0x7f3c6c0cb160 2024-09-10T22:08:02.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.478+0000 7f3c72400640 1 -- 172.21.15.66:0/85449027 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c6c06f870 con 0x7f3c6c0cb160 2024-09-10T22:08:02.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.478+0000 7f3c63400640 1 -- 172.21.15.66:0/85449027 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f3c5c003c30 con 0x7f3c6c0cb160 2024-09-10T22:08:02.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.482+0000 7f3c63400640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c4c073710 0x7f3c4c075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.482+0000 7f3c63400640 1 -- 172.21.15.66:0/85449027 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f3c5c0916d0 con 0x7f3c6c0cb160 2024-09-10T22:08:02.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.482+0000 7f3c6b400640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c4c073710 0x7f3c4c075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.482+0000 7f3c6b400640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c4c073710 0x7f3c4c075bd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3c58011440 tx=0x7f3c580091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:02.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.494+0000 7f3c63400640 1 -- 172.21.15.66:0/85449027 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3c5c007e20 con 0x7f3c6c0cb160 2024-09-10T22:08:02.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.734+0000 7f3c72400640 1 -- 172.21.15.66:0/85449027 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone93841", "target": ["mon-mgr", ""]}) -- 0x7f3c6c16ba20 con 0x7f3c4c073710 2024-09-10T22:08:02.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c63400640 1 -- 172.21.15.66:0/85449027 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3c58005020 con 0x7f3c4c073710 2024-09-10T22:08:02.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c72400640 1 -- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c4c073710 msgr2=0x7f3c4c075bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c72400640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c4c073710 0x7f3c4c075bd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3c58011440 tx=0x7f3c580091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c72400640 1 -- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c6c0cb160 msgr2=0x7f3c6c0ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c72400640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c6c0cb160 0x7f3c6c0ca340 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c3e0 tx=0x7f3c5c00c8a0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c72400640 1 -- 172.21.15.66:0/85449027 shutdown_connections 2024-09-10T22:08:02.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c72400640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c4c073710 0x7f3c4c075bd0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c72400640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c6c0cb640 0x7f3c6c204680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c72400640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c6c0cb160 0x7f3c6c0ca340 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c72400640 1 --2- 172.21.15.66:0/85449027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c0a1c50 0x7f3c6c16b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c72400640 1 -- 172.21.15.66:0/85449027 >> 172.21.15.66:0/85449027 conn(0x7f3c6c0d0b10 msgr2=0x7f3c6c159500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:02.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c72400640 1 -- 172.21.15.66:0/85449027 shutdown_connections 2024-09-10T22:08:02.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.754+0000 7f3c72400640 1 -- 172.21.15.66:0/85449027 wait complete. 2024-09-10T22:08:02.786 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone62795 2024-09-10T22:08:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.934+0000 7fb261a00640 1 -- 172.21.15.66:0/1292505247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb25c15b9e0 msgr2=0x7fb25c0d1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.934+0000 7fb261a00640 1 --2- 172.21.15.66:0/1292505247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb25c15b9e0 0x7fb25c0d1940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009870 tx=0x7fb24c01c860 comp rx=0 tx=0).stop 2024-09-10T22:08:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.934+0000 7fb261a00640 1 -- 172.21.15.66:0/1292505247 shutdown_connections 2024-09-10T22:08:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.934+0000 7fb261a00640 1 --2- 172.21.15.66:0/1292505247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb25c15b9e0 0x7fb25c0d1940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.934+0000 7fb261a00640 1 -- 172.21.15.66:0/1292505247 >> 172.21.15.66:0/1292505247 conn(0x7fb25c09aa50 msgr2=0x7fb25c09ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:02.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.934+0000 7fb261a00640 1 -- 172.21.15.66:0/1292505247 shutdown_connections 2024-09-10T22:08:02.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.934+0000 7fb261a00640 1 -- 172.21.15.66:0/1292505247 wait complete. 2024-09-10T22:08:02.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.934+0000 7fb261a00640 1 Processor -- start 2024-09-10T22:08:02.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.934+0000 7fb261a00640 1 -- start start 2024-09-10T22:08:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb261a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c0b2a70 0x7fb25c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb261a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb25c15b9e0 0x7fb25c1fc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb261a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb25c15bdc0 0x7fb25c06f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb261a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c16a390 con 0x7fb25c0b2a70 2024-09-10T22:08:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb261a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c16a210 con 0x7fb25c15bdc0 2024-09-10T22:08:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb261a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb25c16a510 con 0x7fb25c15b9e0 2024-09-10T22:08:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb25be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c0b2a70 0x7fb25c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb25be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c0b2a70 0x7fb25c06dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56550/0 (socket says 172.21.15.66:56550) 2024-09-10T22:08:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb25be00640 1 -- 172.21.15.66:0/2407516501 learned_addr learned my addr 172.21.15.66:0/2407516501 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb25b400640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb25c15b9e0 0x7fb25c1fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb261000640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb25c15bdc0 0x7fb25c06f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb25be00640 1 -- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb25c15b9e0 msgr2=0x7fb25c1fc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb25be00640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb25c15b9e0 0x7fb25c1fc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb25be00640 1 -- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb25c15bdc0 msgr2=0x7fb25c06f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb25be00640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb25c15bdc0 0x7fb25c06f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb25be00640 1 -- 172.21.15.66:0/2407516501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25c0c8e50 con 0x7fb25c0b2a70 2024-09-10T22:08:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb25be00640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c0b2a70 0x7fb25c06dd30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb24c019470 tx=0x7fb24c0a5720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb253e00640 1 -- 172.21.15.66:0/2407516501 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c0b2040 con 0x7fb25c0b2a70 2024-09-10T22:08:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb253e00640 1 -- 172.21.15.66:0/2407516501 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb24c004380 con 0x7fb25c0b2a70 2024-09-10T22:08:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb261a00640 1 -- 172.21.15.66:0/2407516501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25c0c9130 con 0x7fb25c0b2a70 2024-09-10T22:08:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb253e00640 1 -- 172.21.15.66:0/2407516501 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c005260 con 0x7fb25c0b2a70 2024-09-10T22:08:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb261a00640 1 -- 172.21.15.66:0/2407516501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb25c0c96f0 con 0x7fb25c0b2a70 2024-09-10T22:08:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.938+0000 7fb261a00640 1 -- 172.21.15.66:0/2407516501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb25c0d1940 con 0x7fb25c0b2a70 2024-09-10T22:08:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.942+0000 7fb253e00640 1 -- 172.21.15.66:0/2407516501 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fb24c0b5070 con 0x7fb25c0b2a70 2024-09-10T22:08:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.942+0000 7fb253e00640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb23c0736c0 0x7fb23c075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.942+0000 7fb253e00640 1 -- 172.21.15.66:0/2407516501 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fb24c12b900 con 0x7fb25c0b2a70 2024-09-10T22:08:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.942+0000 7fb25b400640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb23c0736c0 0x7fb23c075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.942+0000 7fb25b400640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb23c0736c0 0x7fb23c075b80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb25c15b5b0 tx=0x7fb248009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:02.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:02.946+0000 7fb253e00640 1 -- 172.21.15.66:0/2407516501 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb24c016450 con 0x7fb25c0b2a70 2024-09-10T22:08:03.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.150+0000 7fb261a00640 1 -- 172.21.15.66:0/2407516501 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone62795", "target": ["mon-mgr", ""]}) -- 0x7fb25c1fc560 con 0x7fb23c0736c0 2024-09-10T22:08:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.170+0000 7fb253e00640 1 -- 172.21.15.66:0/2407516501 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb248001c70 con 0x7fb23c0736c0 2024-09-10T22:08:03.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.174+0000 7fb261a00640 1 -- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb23c0736c0 msgr2=0x7fb23c075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:03.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.174+0000 7fb261a00640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb23c0736c0 0x7fb23c075b80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb25c15b5b0 tx=0x7fb248009040 comp rx=0 tx=0).stop 2024-09-10T22:08:03.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.174+0000 7fb261a00640 1 -- 172.21.15.66:0/2407516501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c0b2a70 msgr2=0x7fb25c06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:03.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.174+0000 7fb261a00640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c0b2a70 0x7fb25c06dd30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb24c019470 tx=0x7fb24c0a5720 comp rx=0 tx=0).stop 2024-09-10T22:08:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.174+0000 7fb261a00640 1 -- 172.21.15.66:0/2407516501 shutdown_connections 2024-09-10T22:08:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.174+0000 7fb261a00640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb23c0736c0 0x7fb23c075b80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.174+0000 7fb261a00640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb25c15bdc0 0x7fb25c06f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.174+0000 7fb261a00640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb25c15b9e0 0x7fb25c1fc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.174+0000 7fb261a00640 1 --2- 172.21.15.66:0/2407516501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c0b2a70 0x7fb25c06dd30 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.174+0000 7fb261a00640 1 -- 172.21.15.66:0/2407516501 >> 172.21.15.66:0/2407516501 conn(0x7fb25c09aa50 msgr2=0x7fb25c0b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.174+0000 7fb261a00640 1 -- 172.21.15.66:0/2407516501 shutdown_connections 2024-09-10T22:08:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.174+0000 7fb261a00640 1 -- 172.21.15.66:0/2407516501 wait complete. 2024-09-10T22:08:03.203 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone87134 2024-09-10T22:08:03.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.394+0000 7fa825600640 1 -- 172.21.15.66:0/566401431 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8200cd0a0 msgr2=0x7fa8200d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:03.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.394+0000 7fa825600640 1 --2- 172.21.15.66:0/566401431 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8200cd0a0 0x7fa8200d0150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa808009840 tx=0x7fa80801c7f0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.394+0000 7fa825600640 1 -- 172.21.15.66:0/566401431 shutdown_connections 2024-09-10T22:08:03.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.394+0000 7fa825600640 1 --2- 172.21.15.66:0/566401431 >> v2:172.21.15.94:3300/0 conn(0x7fa8200aebd0 0x7fa8200af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.394+0000 7fa825600640 1 --2- 172.21.15.66:0/566401431 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8200cd0a0 0x7fa8200d0150 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.394+0000 7fa825600640 1 -- 172.21.15.66:0/566401431 >> 172.21.15.66:0/566401431 conn(0x7fa8200d1c60 msgr2=0x7fa8200d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:03.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.394+0000 7fa825600640 1 -- 172.21.15.66:0/566401431 shutdown_connections 2024-09-10T22:08:03.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.394+0000 7fa825600640 1 -- 172.21.15.66:0/566401431 wait complete. 2024-09-10T22:08:03.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa825600640 1 Processor -- start 2024-09-10T22:08:03.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa825600640 1 -- start start 2024-09-10T22:08:03.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa825600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8200aebd0 0x7fa820156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa825600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8200cd0a0 0x7fa8200c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa825600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82015f470 0x7fa82006f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa825600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa820168d40 con 0x7fa82015f470 2024-09-10T22:08:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa825600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa820168bc0 con 0x7fa8200aebd0 2024-09-10T22:08:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa825600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa820168ec0 con 0x7fa8200cd0a0 2024-09-10T22:08:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa81f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82015f470 0x7fa82006f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa81f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82015f470 0x7fa82006f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56558/0 (socket says 172.21.15.66:56558) 2024-09-10T22:08:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa81f400640 1 -- 172.21.15.66:0/3211920247 learned_addr learned my addr 172.21.15.66:0/3211920247 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa817e00640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8200cd0a0 0x7fa8200c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa81ea00640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8200aebd0 0x7fa820156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa81f400640 1 -- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8200cd0a0 msgr2=0x7fa8200c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa81f400640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8200cd0a0 0x7fa8200c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa81f400640 1 -- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8200aebd0 msgr2=0x7fa820156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:03.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa81f400640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8200aebd0 0x7fa820156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa81f400640 1 -- 172.21.15.66:0/3211920247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82006e120 con 0x7fa82015f470 2024-09-10T22:08:03.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa81f400640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82015f470 0x7fa82006f180 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa81800efd0 tx=0x7fa81800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:03.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa816a00640 1 -- 172.21.15.66:0/3211920247 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818003b80 con 0x7fa82015f470 2024-09-10T22:08:03.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa816a00640 1 -- 172.21.15.66:0/3211920247 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa818003d20 con 0x7fa82015f470 2024-09-10T22:08:03.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa825600640 1 -- 172.21.15.66:0/3211920247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82006e6a0 con 0x7fa82015f470 2024-09-10T22:08:03.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa816a00640 1 -- 172.21.15.66:0/3211920247 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818004280 con 0x7fa82015f470 2024-09-10T22:08:03.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.398+0000 7fa825600640 1 -- 172.21.15.66:0/3211920247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82006ec60 con 0x7fa82015f470 2024-09-10T22:08:03.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.402+0000 7fa825600640 1 -- 172.21.15.66:0/3211920247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8200af060 con 0x7fa82015f470 2024-09-10T22:08:03.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.402+0000 7fa816a00640 1 -- 172.21.15.66:0/3211920247 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fa818004420 con 0x7fa82015f470 2024-09-10T22:08:03.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.402+0000 7fa816a00640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa7fc0736f0 0x7fa7fc075bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:03.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.402+0000 7fa816a00640 1 -- 172.21.15.66:0/3211920247 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7fa818092d00 con 0x7fa82015f470 2024-09-10T22:08:03.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.402+0000 7fa81ea00640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa7fc0736f0 0x7fa7fc075bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:03.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.402+0000 7fa81ea00640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa7fc0736f0 0x7fa7fc075bb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa810001870 tx=0x7fa810001380 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:03.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.406+0000 7fa816a00640 1 -- 172.21.15.66:0/3211920247 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa81805f4a0 con 0x7fa82015f470 2024-09-10T22:08:03.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.606+0000 7fa825600640 1 -- 172.21.15.66:0/3211920247 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone87134", "target": ["mon-mgr", ""]}) -- 0x7fa82006eea0 con 0x7fa7fc0736f0 2024-09-10T22:08:03.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.626+0000 7fa816a00640 1 -- 172.21.15.66:0/3211920247 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa81000f6b0 con 0x7fa7fc0736f0 2024-09-10T22:08:03.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.630+0000 7fa825600640 1 -- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa7fc0736f0 msgr2=0x7fa7fc075bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:03.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.630+0000 7fa825600640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa7fc0736f0 0x7fa7fc075bb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa810001870 tx=0x7fa810001380 comp rx=0 tx=0).stop 2024-09-10T22:08:03.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.630+0000 7fa825600640 1 -- 172.21.15.66:0/3211920247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82015f470 msgr2=0x7fa82006f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:03.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.630+0000 7fa825600640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82015f470 0x7fa82006f180 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa81800efd0 tx=0x7fa81800c6a0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.630+0000 7fa825600640 1 -- 172.21.15.66:0/3211920247 shutdown_connections 2024-09-10T22:08:03.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.630+0000 7fa825600640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa7fc0736f0 0x7fa7fc075bb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.630+0000 7fa825600640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82015f470 0x7fa82006f180 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.630+0000 7fa825600640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8200cd0a0 0x7fa8200c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.630+0000 7fa825600640 1 --2- 172.21.15.66:0/3211920247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8200aebd0 0x7fa820156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.630+0000 7fa825600640 1 -- 172.21.15.66:0/3211920247 >> 172.21.15.66:0/3211920247 conn(0x7fa8200d1c60 msgr2=0x7fa82015bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:03.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.630+0000 7fa825600640 1 -- 172.21.15.66:0/3211920247 shutdown_connections 2024-09-10T22:08:03.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.630+0000 7fa825600640 1 -- 172.21.15.66:0/3211920247 wait complete. 2024-09-10T22:08:03.656 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T22:08:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:08:03.714 INFO:teuthology.orchestra.run.smithi066.stdout:2 2024-09-10T22:08:03.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:08:03.721 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:08:03.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.866+0000 7f835be00640 1 -- 172.21.15.66:0/4039633525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354160280 msgr2=0x7f83540aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:03.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.866+0000 7f835be00640 1 --2- 172.21.15.66:0/4039633525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354160280 0x7f83540aec10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f834c009bc0 tx=0x7f834c01d540 comp rx=0 tx=0).stop 2024-09-10T22:08:03.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.866+0000 7f835be00640 1 -- 172.21.15.66:0/4039633525 shutdown_connections 2024-09-10T22:08:03.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.866+0000 7f835be00640 1 --2- 172.21.15.66:0/4039633525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354160280 0x7f83540aec10 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.866+0000 7f835be00640 1 --2- 172.21.15.66:0/4039633525 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f835415f450 0x7f83540b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.866+0000 7f835be00640 1 -- 172.21.15.66:0/4039633525 >> 172.21.15.66:0/4039633525 conn(0x7f83540d1c60 msgr2=0x7f83540d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:03.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.866+0000 7f835be00640 1 -- 172.21.15.66:0/4039633525 shutdown_connections 2024-09-10T22:08:03.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.866+0000 7f835be00640 1 -- 172.21.15.66:0/4039633525 wait complete. 2024-09-10T22:08:03.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.866+0000 7f835be00640 1 Processor -- start 2024-09-10T22:08:03.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.866+0000 7f835be00640 1 -- start start 2024-09-10T22:08:03.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f835be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540b1e10 0x7f83541f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:03.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f835be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f835415f450 0x7f835406f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:03.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f835be00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8354160280 0x7f83540ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:03.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f835be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8354168eb0 con 0x7f83540b1e10 2024-09-10T22:08:03.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f835be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83540afe40 con 0x7f835415f450 2024-09-10T22:08:03.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f835be00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8354169030 con 0x7f8354160280 2024-09-10T22:08:03.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8359400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540b1e10 0x7f83541f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:03.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8359400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540b1e10 0x7f83541f8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56566/0 (socket says 172.21.15.66:56566) 2024-09-10T22:08:03.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8359400640 1 -- 172.21.15.66:0/3498268705 learned_addr learned my addr 172.21.15.66:0/3498268705 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8359e00640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8354160280 0x7f83540ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8358a00640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f835415f450 0x7f835406f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8359400640 1 -- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8354160280 msgr2=0x7f83540ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8359400640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8354160280 0x7f83540ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8359400640 1 -- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f835415f450 msgr2=0x7f835406f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8359400640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f835415f450 0x7f835406f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8359400640 1 -- 172.21.15.66:0/3498268705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83541f8960 con 0x7f83540b1e10 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8359e00640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8354160280 0x7f83540ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8359400640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540b1e10 0x7f83541f8420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f834800e9f0 tx=0x7f834800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8351600640 1 -- 172.21.15.66:0/3498268705 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834800cd80 con 0x7f83540b1e10 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8351600640 1 -- 172.21.15.66:0/3498268705 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8348010430 con 0x7f83540b1e10 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f835be00640 1 -- 172.21.15.66:0/3498268705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835406fa20 con 0x7f83540b1e10 2024-09-10T22:08:03.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8351600640 1 -- 172.21.15.66:0/3498268705 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348010860 con 0x7f83540b1e10 2024-09-10T22:08:03.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f835be00640 1 -- 172.21.15.66:0/3498268705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83541f7940 con 0x7f83540b1e10 2024-09-10T22:08:03.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f835be00640 1 -- 172.21.15.66:0/3498268705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83540b3170 con 0x7f83540b1e10 2024-09-10T22:08:03.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.870+0000 7f8351600640 1 -- 172.21.15.66:0/3498268705 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f8348010a00 con 0x7f83540b1e10 2024-09-10T22:08:03.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.874+0000 7f8351600640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f83340736c0 0x7f8334075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:03.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.874+0000 7f8351600640 1 -- 172.21.15.66:0/3498268705 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f83480929f0 con 0x7f83540b1e10 2024-09-10T22:08:03.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.874+0000 7f8358a00640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f83340736c0 0x7f8334075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:03.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.874+0000 7f8358a00640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f83340736c0 0x7f8334075b80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f83540cbf30 tx=0x7f8340009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:03.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:03.878+0000 7f8351600640 1 -- 172.21.15.66:0/3498268705 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f834805fbe0 con 0x7f83540b1e10 2024-09-10T22:08:04.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.050+0000 7f835be00640 1 -- 172.21.15.66:0/3498268705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f83540d2a40 con 0x7f83540b1e10 2024-09-10T22:08:04.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.050+0000 7f8351600640 1 -- 172.21.15.66:0/3498268705 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v39)=0 v39) ==== 131+0+0 (secure 0 0 0) 0x7f834805f4e0 con 0x7f83540b1e10 2024-09-10T22:08:04.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.054+0000 7f835be00640 1 -- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f83340736c0 msgr2=0x7f8334075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:04.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.054+0000 7f835be00640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f83340736c0 0x7f8334075b80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f83540cbf30 tx=0x7f8340009040 comp rx=0 tx=0).stop 2024-09-10T22:08:04.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.054+0000 7f835be00640 1 -- 172.21.15.66:0/3498268705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540b1e10 msgr2=0x7f83541f8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:04.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.054+0000 7f835be00640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540b1e10 0x7f83541f8420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f834800e9f0 tx=0x7f834800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:08:04.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.054+0000 7f835be00640 1 -- 172.21.15.66:0/3498268705 shutdown_connections 2024-09-10T22:08:04.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.054+0000 7f835be00640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f83340736c0 0x7f8334075b80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:04.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.054+0000 7f835be00640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8354160280 0x7f83540ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:04.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.054+0000 7f835be00640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f835415f450 0x7f835406f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:04.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.054+0000 7f835be00640 1 --2- 172.21.15.66:0/3498268705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540b1e10 0x7f83541f8420 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:04.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.054+0000 7f835be00640 1 -- 172.21.15.66:0/3498268705 >> 172.21.15.66:0/3498268705 conn(0x7f83540d1c60 msgr2=0x7f83541553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:04.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.054+0000 7f835be00640 1 -- 172.21.15.66:0/3498268705 shutdown_connections 2024-09-10T22:08:04.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.054+0000 7f835be00640 1 -- 172.21.15.66:0/3498268705 wait complete. 2024-09-10T22:08:04.081 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:08:04.081 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:08:04.192 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:08:04.307 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:08:04.307 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:08:04.307 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:08:04.307 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:08:04.320 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:08:04.320 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:08:04.374 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:08:04.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.658+0000 7f239a400640 1 -- 172.21.15.66:0/3311630300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f238c009a30 msgr2=0x7f238c009ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:04.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.658+0000 7f239a400640 1 --2- 172.21.15.66:0/3311630300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f238c009a30 0x7f238c009ea0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2380009900 tx=0x7f238001c8a0 comp rx=0 tx=0).stop 2024-09-10T22:08:04.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.658+0000 7f239a400640 1 -- 172.21.15.66:0/3311630300 shutdown_connections 2024-09-10T22:08:04.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.658+0000 7f239a400640 1 --2- 172.21.15.66:0/3311630300 >> v2:172.21.15.94:3300/0 conn(0x7f238c040650 0x7f238c040a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:04.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.658+0000 7f239a400640 1 --2- 172.21.15.66:0/3311630300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f238c009a30 0x7f238c009ea0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:04.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.658+0000 7f239a400640 1 -- 172.21.15.66:0/3311630300 >> 172.21.15.66:0/3311630300 conn(0x7f238c05cd10 msgr2=0x7f238c05d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:04.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.658+0000 7f239a400640 1 -- 172.21.15.66:0/3311630300 shutdown_connections 2024-09-10T22:08:04.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.658+0000 7f239a400640 1 -- 172.21.15.66:0/3311630300 wait complete. 2024-09-10T22:08:04.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.662+0000 7f239a400640 1 Processor -- start 2024-09-10T22:08:04.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.662+0000 7f239a400640 1 -- start start 2024-09-10T22:08:04.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.662+0000 7f239a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f238c009a30 0x7f238c183ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:04.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.662+0000 7f239a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f238c040650 0x7f238c03eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:04.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.662+0000 7f239a400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f238c0e6e80 0x7f238c180fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:04.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.662+0000 7f239a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f238c0f3e60 con 0x7f238c040650 2024-09-10T22:08:04.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.662+0000 7f239a400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f238c0f3ce0 con 0x7f238c009a30 2024-09-10T22:08:04.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.662+0000 7f239a400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f238c0f3fe0 con 0x7f238c0e6e80 2024-09-10T22:08:04.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2399000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f238c009a30 0x7f238c183ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:04.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2393e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f238c040650 0x7f238c03eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:04.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2399000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f238c009a30 0x7f238c183ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:51034/0 (socket says 172.21.15.66:51034) 2024-09-10T22:08:04.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2399000640 1 -- 172.21.15.66:0/1096668362 learned_addr learned my addr 172.21.15.66:0/1096668362 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:04.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2399a00640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f238c0e6e80 0x7f238c180fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:04.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2393e00640 1 -- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f238c0e6e80 msgr2=0x7f238c180fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:04.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2393e00640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f238c0e6e80 0x7f238c180fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:04.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2393e00640 1 -- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f238c009a30 msgr2=0x7f238c183ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:04.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2393e00640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f238c009a30 0x7f238c183ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:04.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2393e00640 1 -- 172.21.15.66:0/1096668362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238c1845f0 con 0x7f238c040650 2024-09-10T22:08:04.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2399000640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f238c009a30 0x7f238c183ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:04.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2393e00640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f238c040650 0x7f238c03eec0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f23800ac000 tx=0x7f2380005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:04.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2390c00640 1 -- 172.21.15.66:0/1096668362 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23800b2040 con 0x7f238c040650 2024-09-10T22:08:04.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2390c00640 1 -- 172.21.15.66:0/1096668362 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23800044b0 con 0x7f238c040650 2024-09-10T22:08:04.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f2390c00640 1 -- 172.21.15.66:0/1096668362 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380003860 con 0x7f238c040650 2024-09-10T22:08:04.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f239a400640 1 -- 172.21.15.66:0/1096668362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238c1848d0 con 0x7f238c040650 2024-09-10T22:08:04.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.666+0000 7f239a400640 1 -- 172.21.15.66:0/1096668362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238c180ac0 con 0x7f238c040650 2024-09-10T22:08:04.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.670+0000 7f239a400640 1 -- 172.21.15.66:0/1096668362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f238c184df0 con 0x7f238c040650 2024-09-10T22:08:04.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.670+0000 7f2390c00640 1 -- 172.21.15.66:0/1096668362 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f23800b5070 con 0x7f238c040650 2024-09-10T22:08:04.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.670+0000 7f2390c00640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f23740736c0 0x7f2374075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:04.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.674+0000 7f2399000640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f23740736c0 0x7f2374075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:04.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.674+0000 7f2399000640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f23740736c0 0x7f2374075b80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f238c03f7e0 tx=0x7f2388009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:04.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.674+0000 7f2390c00640 1 -- 172.21.15.66:0/1096668362 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5795+0+0 (secure 0 0 0) 0x7f23800a5720 con 0x7f238c040650 2024-09-10T22:08:04.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:04.686+0000 7f2390c00640 1 -- 172.21.15.66:0/1096668362 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f238c184df0 con 0x7f238c040650 2024-09-10T22:08:05.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.070+0000 7f239a400640 1 -- 172.21.15.66:0/1096668362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f238c1808b0 con 0x7f238c040650 2024-09-10T22:08:05.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.070+0000 7f2390c00640 1 -- 172.21.15.66:0/1096668362 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v73) ==== 80+0+0 (secure 0 0 0) 0x7f2380016490 con 0x7f238c040650 2024-09-10T22:08:05.077 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:08:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.074+0000 7f239a400640 1 -- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f23740736c0 msgr2=0x7f2374075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.074+0000 7f239a400640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f23740736c0 0x7f2374075b80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f238c03f7e0 tx=0x7f2388009040 comp rx=0 tx=0).stop 2024-09-10T22:08:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.074+0000 7f239a400640 1 -- 172.21.15.66:0/1096668362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f238c040650 msgr2=0x7f238c03eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.074+0000 7f239a400640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f238c040650 0x7f238c03eec0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f23800ac000 tx=0x7f2380005770 comp rx=0 tx=0).stop 2024-09-10T22:08:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.074+0000 7f239a400640 1 -- 172.21.15.66:0/1096668362 shutdown_connections 2024-09-10T22:08:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.074+0000 7f239a400640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f23740736c0 0x7f2374075b80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.074+0000 7f239a400640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f238c0e6e80 0x7f238c180fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.074+0000 7f239a400640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f238c040650 0x7f238c03eec0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.074+0000 7f239a400640 1 --2- 172.21.15.66:0/1096668362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f238c009a30 0x7f238c183ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.074+0000 7f239a400640 1 -- 172.21.15.66:0/1096668362 >> 172.21.15.66:0/1096668362 conn(0x7f238c05cd10 msgr2=0x7f238c0e3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:05.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.074+0000 7f239a400640 1 -- 172.21.15.66:0/1096668362 shutdown_connections 2024-09-10T22:08:05.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.074+0000 7f239a400640 1 -- 172.21.15.66:0/1096668362 wait complete. 2024-09-10T22:08:05.114 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-09-10T22:08:05.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:05.114 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T22:08:05.164 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:08:05.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:05.165 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:08:05.175 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[42601]: fuse finished with error 0 and tester_r 0 2024-09-10T22:08:05.330 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:08:05.330 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:08:05.330 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:08:05.345 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:08:05.345 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:08:05.345 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:08:05.346 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:08:05.389 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:08:05.389 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:08:05.389 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:08:05.389 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:08:05.401 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:08:05.401 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:08:05.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.646+0000 7f5057600640 1 -- 172.21.15.66:0/2643368477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50500b0ef0 msgr2=0x7f50500d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:05.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.646+0000 7f5057600640 1 --2- 172.21.15.66:0/2643368477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50500b0ef0 0x7f50500d02c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5048009bc0 tx=0x7f504801ca80 comp rx=0 tx=0).stop 2024-09-10T22:08:05.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.646+0000 7f5057600640 1 -- 172.21.15.66:0/2643368477 shutdown_connections 2024-09-10T22:08:05.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.646+0000 7f5057600640 1 --2- 172.21.15.66:0/2643368477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50500b0ef0 0x7f50500d02c0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.646+0000 7f5057600640 1 --2- 172.21.15.66:0/2643368477 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5050070ac0 0x7f5050070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.646+0000 7f5057600640 1 -- 172.21.15.66:0/2643368477 >> 172.21.15.66:0/2643368477 conn(0x7f50500d1d00 msgr2=0x7f50500d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:05.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.646+0000 7f5057600640 1 -- 172.21.15.66:0/2643368477 shutdown_connections 2024-09-10T22:08:05.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.646+0000 7f5057600640 1 -- 172.21.15.66:0/2643368477 wait complete. 2024-09-10T22:08:05.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.646+0000 7f5057600640 1 Processor -- start 2024-09-10T22:08:05.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.646+0000 7f5057600640 1 -- start start 2024-09-10T22:08:05.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.646+0000 7f5057600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5050070ac0 0x7f50500b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.646+0000 7f5057600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050071700 0x7f50501f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5057600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50500b0ef0 0x7f5050159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5057600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5050160b50 con 0x7f5050071700 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5057600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50501609d0 con 0x7f5050070ac0 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5057600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5050160cd0 con 0x7f50500b0ef0 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5047e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050071700 0x7f50501f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5054c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5050070ac0 0x7f50500b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5055600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50500b0ef0 0x7f5050159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5054c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5050070ac0 0x7f50500b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:51040/0 (socket says 172.21.15.66:51040) 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5054c00640 1 -- 172.21.15.66:0/499437353 learned_addr learned my addr 172.21.15.66:0/499437353 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5055600640 1 -- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5050070ac0 msgr2=0x7f50500b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5055600640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5050070ac0 0x7f50500b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5055600640 1 -- 172.21.15.66:0/499437353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050071700 msgr2=0x7f50501f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:05.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5055600640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050071700 0x7f50501f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5055600640 1 -- 172.21.15.66:0/499437353 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50500c8fe0 con 0x7f50500b0ef0 2024-09-10T22:08:05.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5047e00640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050071700 0x7f50501f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:05.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5054c00640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5050070ac0 0x7f50500b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:05.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5055600640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50500b0ef0 0x7f5050159bd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f50480ac000 tx=0x7f5048003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:05.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f504d600640 1 -- 172.21.15.66:0/499437353 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50480b2040 con 0x7f50500b0ef0 2024-09-10T22:08:05.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f504d600640 1 -- 172.21.15.66:0/499437353 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5048004040 con 0x7f50500b0ef0 2024-09-10T22:08:05.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f504d600640 1 -- 172.21.15.66:0/499437353 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048004490 con 0x7f50500b0ef0 2024-09-10T22:08:05.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5057600640 1 -- 172.21.15.66:0/499437353 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50500c92c0 con 0x7f50500b0ef0 2024-09-10T22:08:05.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5057600640 1 -- 172.21.15.66:0/499437353 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f50500c5e60 con 0x7f50500b0ef0 2024-09-10T22:08:05.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f5057600640 1 -- 172.21.15.66:0/499437353 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5050070ea0 con 0x7f50500b0ef0 2024-09-10T22:08:05.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.650+0000 7f504d600640 1 -- 172.21.15.66:0/499437353 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f50480b5070 con 0x7f50500b0ef0 2024-09-10T22:08:05.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.654+0000 7f504d600640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f502c077ad0 0x7f502c079f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:05.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.654+0000 7f504d600640 1 -- 172.21.15.66:0/499437353 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7f504812b620 con 0x7f50500b0ef0 2024-09-10T22:08:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.654+0000 7f5054c00640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f502c077ad0 0x7f502c079f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.654+0000 7f5054c00640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f502c077ad0 0x7f502c079f90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f50500b25a0 tx=0x7f50400091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:05.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.658+0000 7f504d600640 1 -- 172.21.15.66:0/499437353 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5048016790 con 0x7f50500b0ef0 2024-09-10T22:08:05.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.866+0000 7f5057600640 1 -- 172.21.15.66:0/499437353 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50500b4700 con 0x7f50500b0ef0 2024-09-10T22:08:05.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.866+0000 7f504d600640 1 -- 172.21.15.66:0/499437353 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5225 (secure 0 0 0) 0x7f50480f7dc0 con 0x7f50500b0ef0 2024-09-10T22:08:05.873 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:05.873 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":47,"btime":"2024-09-10T22:07:02:166078+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6112,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3780556367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3780556367},{"type":"v1","addr":"172.21.15.94:6836","nonce":3780556367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1892098851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1892098851},{"type":"v1","addr":"172.21.15.94:6837","nonce":1892098851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1548997357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1548997357},{"type":"v1","addr":"172.21.15.66:6834","nonce":1548997357}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:06:53.520589+0000","modified":"2024-09-10T22:06:56.547020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6390":{"gid":6390,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2415586891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2415586891},{"type":"v1","addr":"172.21.15.66:6835","nonce":2415586891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390]},"id":5}]} 2024-09-10T22:08:05.873 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 47 2024-09-10T22:08:05.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.870+0000 7f5057600640 1 -- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f502c077ad0 msgr2=0x7f502c079f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:05.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.870+0000 7f5057600640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f502c077ad0 0x7f502c079f90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f50500b25a0 tx=0x7f50400091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.870+0000 7f5057600640 1 -- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50500b0ef0 msgr2=0x7f5050159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:05.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.870+0000 7f5057600640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50500b0ef0 0x7f5050159bd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f50480ac000 tx=0x7f5048003c30 comp rx=0 tx=0).stop 2024-09-10T22:08:05.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.870+0000 7f5057600640 1 -- 172.21.15.66:0/499437353 shutdown_connections 2024-09-10T22:08:05.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.870+0000 7f5057600640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f502c077ad0 0x7f502c079f90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.870+0000 7f5057600640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50500b0ef0 0x7f5050159bd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.870+0000 7f5057600640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050071700 0x7f50501f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.870+0000 7f5057600640 1 --2- 172.21.15.66:0/499437353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5050070ac0 0x7f50500b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:05.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.870+0000 7f5057600640 1 -- 172.21.15.66:0/499437353 >> 172.21.15.66:0/499437353 conn(0x7f50500d1d00 msgr2=0x7f50500d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:05.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.870+0000 7f5057600640 1 -- 172.21.15.66:0/499437353 shutdown_connections 2024-09-10T22:08:05.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:05.870+0000 7f5057600640 1 -- 172.21.15.66:0/499437353 wait complete. 2024-09-10T22:08:05.894 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:08:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.034+0000 7f80f2800640 1 -- 172.21.15.66:0/3146902249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80ec0d28c0 msgr2=0x7f80ec0b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.034+0000 7f80f2800640 1 --2- 172.21.15.66:0/3146902249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80ec0d28c0 0x7f80ec0b1120 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f80ec0b6970 tx=0x7f80e401c8c0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.034+0000 7f80f2800640 1 -- 172.21.15.66:0/3146902249 shutdown_connections 2024-09-10T22:08:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.034+0000 7f80f2800640 1 --2- 172.21.15.66:0/3146902249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80ec0d28c0 0x7f80ec0b1120 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.034+0000 7f80f2800640 1 --2- 172.21.15.66:0/3146902249 >> v2:172.21.15.94:3300/0 conn(0x7f80ec162630 0x7f80ec0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.034+0000 7f80f2800640 1 -- 172.21.15.66:0/3146902249 >> 172.21.15.66:0/3146902249 conn(0x7f80ec0d3da0 msgr2=0x7f80ec0d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.034+0000 7f80f2800640 1 -- 172.21.15.66:0/3146902249 shutdown_connections 2024-09-10T22:08:06.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.034+0000 7f80f2800640 1 -- 172.21.15.66:0/3146902249 wait complete. 2024-09-10T22:08:06.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f2800640 1 Processor -- start 2024-09-10T22:08:06.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f2800640 1 -- start start 2024-09-10T22:08:06.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80ec0d28c0 0x7f80ec161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f2800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f80ec162630 0x7f80ec154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80ec153bb0 0x7f80ec1604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f2800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec16b9c0 con 0x7f80ec0d28c0 2024-09-10T22:08:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec16b840 con 0x7f80ec153bb0 2024-09-10T22:08:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f2800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f80ec16bb40 con 0x7f80ec162630 2024-09-10T22:08:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80ec0d28c0 0x7f80ec161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80ec0d28c0 0x7f80ec161ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56624/0 (socket says 172.21.15.66:56624) 2024-09-10T22:08:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f1400640 1 -- 172.21.15.66:0/109988710 learned_addr learned my addr 172.21.15.66:0/109988710 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f0a00640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f80ec162630 0x7f80ec154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f1e00640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80ec153bb0 0x7f80ec1604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f1400640 1 -- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f80ec162630 msgr2=0x7f80ec154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f1400640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f80ec162630 0x7f80ec154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f1400640 1 -- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80ec153bb0 msgr2=0x7f80ec1604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f1400640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80ec153bb0 0x7f80ec1604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f1400640 1 -- 172.21.15.66:0/109988710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec15fc30 con 0x7f80ec0d28c0 2024-09-10T22:08:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f1400640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80ec0d28c0 0x7f80ec161ab0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f80e000ebc0 tx=0x7f80e000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80e9600640 1 -- 172.21.15.66:0/109988710 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0010630 con 0x7f80ec0d28c0 2024-09-10T22:08:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80e9600640 1 -- 172.21.15.66:0/109988710 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80e00108c0 con 0x7f80ec0d28c0 2024-09-10T22:08:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80e9600640 1 -- 172.21.15.66:0/109988710 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0010cf0 con 0x7f80ec0d28c0 2024-09-10T22:08:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f2800640 1 -- 172.21.15.66:0/109988710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ec15ff10 con 0x7f80ec0d28c0 2024-09-10T22:08:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.038+0000 7f80f2800640 1 -- 172.21.15.66:0/109988710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ec15a280 con 0x7f80ec0d28c0 2024-09-10T22:08:06.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.042+0000 7f80f2800640 1 -- 172.21.15.66:0/109988710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80ec160280 con 0x7f80ec0d28c0 2024-09-10T22:08:06.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.042+0000 7f80e9600640 1 -- 172.21.15.66:0/109988710 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f80e0018420 con 0x7f80ec0d28c0 2024-09-10T22:08:06.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.042+0000 7f80e9600640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f80cc0736c0 0x7f80cc075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:06.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.042+0000 7f80e9600640 1 -- 172.21.15.66:0/109988710 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7f80e0092a90 con 0x7f80ec0d28c0 2024-09-10T22:08:06.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.042+0000 7f80f0a00640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f80cc0736c0 0x7f80cc075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:06.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.042+0000 7f80f0a00640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f80cc0736c0 0x7f80cc075b80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f80ec161ff0 tx=0x7f80d8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:06.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.046+0000 7f80e9600640 1 -- 172.21.15.66:0/109988710 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f80e005f570 con 0x7f80ec0d28c0 2024-09-10T22:08:06.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.326+0000 7f80f2800640 1 -- 172.21.15.66:0/109988710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80ec08dd40 con 0x7f80ec0d28c0 2024-09-10T22:08:06.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.326+0000 7f80e9600640 1 -- 172.21.15.66:0/109988710 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5225 (secure 0 0 0) 0x7f80e0063fa0 con 0x7f80ec0d28c0 2024-09-10T22:08:06.333 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:06.333 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":47,"btime":"2024-09-10T22:07:02:166078+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6112,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3780556367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3780556367},{"type":"v1","addr":"172.21.15.94:6836","nonce":3780556367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1892098851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1892098851},{"type":"v1","addr":"172.21.15.94:6837","nonce":1892098851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1548997357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1548997357},{"type":"v1","addr":"172.21.15.66:6834","nonce":1548997357}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:06:53.520589+0000","modified":"2024-09-10T22:06:56.547020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6390":{"gid":6390,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2415586891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2415586891},{"type":"v1","addr":"172.21.15.66:6835","nonce":2415586891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390]},"id":5}]} 2024-09-10T22:08:06.333 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 47 2024-09-10T22:08:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.330+0000 7f80f2800640 1 -- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f80cc0736c0 msgr2=0x7f80cc075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.330+0000 7f80f2800640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f80cc0736c0 0x7f80cc075b80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f80ec161ff0 tx=0x7f80d8009040 comp rx=0 tx=0).stop 2024-09-10T22:08:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.330+0000 7f80f2800640 1 -- 172.21.15.66:0/109988710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80ec0d28c0 msgr2=0x7f80ec161ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.330+0000 7f80f2800640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80ec0d28c0 0x7f80ec161ab0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f80e000ebc0 tx=0x7f80e000c6a0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.330+0000 7f80f2800640 1 -- 172.21.15.66:0/109988710 shutdown_connections 2024-09-10T22:08:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.330+0000 7f80f2800640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f80cc0736c0 0x7f80cc075b80 secure :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f80ec161ff0 tx=0x7f80d8009040 comp rx=0 tx=0).stop 2024-09-10T22:08:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.330+0000 7f80f2800640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80ec153bb0 0x7f80ec1604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.330+0000 7f80f2800640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f80ec162630 0x7f80ec154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.330+0000 7f80f2800640 1 --2- 172.21.15.66:0/109988710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80ec0d28c0 0x7f80ec161ab0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.330+0000 7f80f2800640 1 -- 172.21.15.66:0/109988710 >> 172.21.15.66:0/109988710 conn(0x7f80ec0d3da0 msgr2=0x7f80ec0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.330+0000 7f80f2800640 1 -- 172.21.15.66:0/109988710 shutdown_connections 2024-09-10T22:08:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.330+0000 7f80f2800640 1 -- 172.21.15.66:0/109988710 wait complete. 2024-09-10T22:08:06.372 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:08:06.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 -- 172.21.15.66:0/4287612628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b4815f300 msgr2=0x7f0b480b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 --2- 172.21.15.66:0/4287612628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b4815f300 0x7f0b480b1090 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009930 tx=0x7f0b3801c8e0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 -- 172.21.15.66:0/4287612628 shutdown_connections 2024-09-10T22:08:06.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 --2- 172.21.15.66:0/4287612628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b4815f300 0x7f0b480b1090 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 -- 172.21.15.66:0/4287612628 >> 172.21.15.66:0/4287612628 conn(0x7f0b480d1b10 msgr2=0x7f0b480d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:06.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 -- 172.21.15.66:0/4287612628 shutdown_connections 2024-09-10T22:08:06.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 -- 172.21.15.66:0/4287612628 wait complete. 2024-09-10T22:08:06.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 Processor -- start 2024-09-10T22:08:06.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 -- start start 2024-09-10T22:08:06.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b480b15d0 0x7f0b4814b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:06.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b4815ef20 0x7f0b481f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:06.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b4815f300 0x7f0b480ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:06.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4816a080 con 0x7f0b4815f300 2024-09-10T22:08:06.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b48169f00 con 0x7f0b480b15d0 2024-09-10T22:08:06.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.562+0000 7f0b4f200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4816a200 con 0x7f0b4815ef20 2024-09-10T22:08:06.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b4815f300 0x7f0b480ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:06.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b4815f300 0x7f0b480ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56644/0 (socket says 172.21.15.66:56644) 2024-09-10T22:08:06.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4e800640 1 -- 172.21.15.66:0/4210393334 learned_addr learned my addr 172.21.15.66:0/4210393334 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:06.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4d400640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b4815ef20 0x7f0b481f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:06.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4de00640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b480b15d0 0x7f0b4814b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:06.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4e800640 1 -- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b4815ef20 msgr2=0x7f0b481f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4e800640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b4815ef20 0x7f0b481f8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4e800640 1 -- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b480b15d0 msgr2=0x7f0b4814b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4e800640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b480b15d0 0x7f0b4814b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4e800640 1 -- 172.21.15.66:0/4210393334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b481f9350 con 0x7f0b4815f300 2024-09-10T22:08:06.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4e800640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b4815f300 0x7f0b480ca740 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400c780 tx=0x7f0b4400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:06.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b42000640 1 -- 172.21.15.66:0/4210393334 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44003b80 con 0x7f0b4815f300 2024-09-10T22:08:06.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b42000640 1 -- 172.21.15.66:0/4210393334 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b44003d20 con 0x7f0b4815f300 2024-09-10T22:08:06.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b42000640 1 -- 172.21.15.66:0/4210393334 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400aab0 con 0x7f0b4815f300 2024-09-10T22:08:06.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4f200640 1 -- 172.21.15.66:0/4210393334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b481f95d0 con 0x7f0b4815f300 2024-09-10T22:08:06.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4f200640 1 -- 172.21.15.66:0/4210393334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b481f9b90 con 0x7f0b4815f300 2024-09-10T22:08:06.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b4f200640 1 -- 172.21.15.66:0/4210393334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b480b1090 con 0x7f0b4815f300 2024-09-10T22:08:06.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.566+0000 7f0b42000640 1 -- 172.21.15.66:0/4210393334 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f0b4400ac50 con 0x7f0b4815f300 2024-09-10T22:08:06.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.570+0000 7f0b42000640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b2c0736c0 0x7f0b2c075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:06.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.570+0000 7f0b42000640 1 -- 172.21.15.66:0/4210393334 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7f0b44091e10 con 0x7f0b4815f300 2024-09-10T22:08:06.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.570+0000 7f0b4de00640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b2c0736c0 0x7f0b2c075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:06.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.570+0000 7f0b4de00640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b2c0736c0 0x7f0b2c075b80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c007fd0 tx=0x7f0b3c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:06.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.574+0000 7f0b42000640 1 -- 172.21.15.66:0/4210393334 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0b481f9350 con 0x7f0b4815f300 2024-09-10T22:08:06.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.746+0000 7f0b4f200640 1 -- 172.21.15.66:0/4210393334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b48072e00 con 0x7f0b4815f300 2024-09-10T22:08:06.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b42000640 1 -- 172.21.15.66:0/4210393334 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16908 (secure 0 0 0) 0x7f0b44067320 con 0x7f0b4815f300 2024-09-10T22:08:06.754 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:06.755 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":74,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:08:05.076199+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:06:51.234296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T22:06:52.488635+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":74,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:08:06.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b4f200640 1 -- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b2c0736c0 msgr2=0x7f0b2c075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b4f200640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b2c0736c0 0x7f0b2c075b80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c007fd0 tx=0x7f0b3c009040 comp rx=0 tx=0).stop 2024-09-10T22:08:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b4f200640 1 -- 172.21.15.66:0/4210393334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b4815f300 msgr2=0x7f0b480ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b4f200640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b4815f300 0x7f0b480ca740 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400c780 tx=0x7f0b4400cc40 comp rx=0 tx=0).stop 2024-09-10T22:08:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b4f200640 1 -- 172.21.15.66:0/4210393334 shutdown_connections 2024-09-10T22:08:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b4f200640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b2c0736c0 0x7f0b2c075b80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b4f200640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b4815f300 0x7f0b480ca740 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b4f200640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b4815ef20 0x7f0b481f8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b4f200640 1 --2- 172.21.15.66:0/4210393334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b480b15d0 0x7f0b4814b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b4f200640 1 -- 172.21.15.66:0/4210393334 >> 172.21.15.66:0/4210393334 conn(0x7f0b480d1b10 msgr2=0x7f0b4815df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b4f200640 1 -- 172.21.15.66:0/4210393334 shutdown_connections 2024-09-10T22:08:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.750+0000 7f0b4f200640 1 -- 172.21.15.66:0/4210393334 wait complete. 2024-09-10T22:08:06.778 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:08:06.778 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:08:06.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.918+0000 7fa8e1200640 1 -- 172.21.15.66:0/1476377910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8dc0b1e10 msgr2=0x7fa8dc15fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.918+0000 7fa8e1200640 1 --2- 172.21.15.66:0/1476377910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8dc0b1e10 0x7fa8dc15fb70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009900 tx=0x7fa8c401d2b0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.918+0000 7fa8e1200640 1 -- 172.21.15.66:0/1476377910 shutdown_connections 2024-09-10T22:08:06.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.918+0000 7fa8e1200640 1 --2- 172.21.15.66:0/1476377910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8dc0b1e10 0x7fa8dc15fb70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.918+0000 7fa8e1200640 1 --2- 172.21.15.66:0/1476377910 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8dc15f450 0x7fa8dc0b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.918+0000 7fa8e1200640 1 -- 172.21.15.66:0/1476377910 >> 172.21.15.66:0/1476377910 conn(0x7fa8dc0d1c60 msgr2=0x7fa8dc0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:06.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.918+0000 7fa8e1200640 1 -- 172.21.15.66:0/1476377910 shutdown_connections 2024-09-10T22:08:06.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.918+0000 7fa8e1200640 1 -- 172.21.15.66:0/1476377910 wait complete. 2024-09-10T22:08:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8e1200640 1 Processor -- start 2024-09-10T22:08:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8e1200640 1 -- start start 2024-09-10T22:08:06.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8e1200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8dc0b1e10 0x7fa8dc06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8e1200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8dc15f450 0x7fa8dc1fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8e1200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8dc1600b0 0x7fa8dc1f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8e1200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc1692f0 con 0x7fa8dc15f450 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8e1200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc169170 con 0x7fa8dc0b1e10 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8e1200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa8dc169470 con 0x7fa8dc1600b0 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8dc15f450 0x7fa8dc1fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8dc15f450 0x7fa8dc1fc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56658/0 (socket says 172.21.15.66:56658) 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8dc0b1e10 0x7fa8dc06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d1400640 1 -- 172.21.15.66:0/3841215297 learned_addr learned my addr 172.21.15.66:0/3841215297 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8da800640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8dc1600b0 0x7fa8dc1f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d9e00640 1 -- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8dc1600b0 msgr2=0x7fa8dc1f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d9e00640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8dc1600b0 0x7fa8dc1f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d9e00640 1 -- 172.21.15.66:0/3841215297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8dc15f450 msgr2=0x7fa8dc1fc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d9e00640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8dc15f450 0x7fa8dc1fc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d9e00640 1 -- 172.21.15.66:0/3841215297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc1f8b70 con 0x7fa8dc0b1e10 2024-09-10T22:08:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d1400640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8dc15f450 0x7fa8dc1fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d9e00640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8dc0b1e10 0x7fa8dc06e5e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00e9f0 tx=0x7fa8cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d2a00640 1 -- 172.21.15.66:0/3841215297 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc00cd80 con 0x7fa8dc0b1e10 2024-09-10T22:08:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d2a00640 1 -- 172.21.15.66:0/3841215297 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8cc004020 con 0x7fa8dc0b1e10 2024-09-10T22:08:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8e1200640 1 -- 172.21.15.66:0/3841215297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8dc1f8e50 con 0x7fa8dc0b1e10 2024-09-10T22:08:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8d2a00640 1 -- 172.21.15.66:0/3841215297 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc004450 con 0x7fa8dc0b1e10 2024-09-10T22:08:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.922+0000 7fa8e1200640 1 -- 172.21.15.66:0/3841215297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8dc1575d0 con 0x7fa8dc0b1e10 2024-09-10T22:08:06.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.926+0000 7fa8e1200640 1 -- 172.21.15.66:0/3841215297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8dc1f9070 con 0x7fa8dc0b1e10 2024-09-10T22:08:06.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.926+0000 7fa8d2a00640 1 -- 172.21.15.66:0/3841215297 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fa8cc003600 con 0x7fa8dc0b1e10 2024-09-10T22:08:06.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.926+0000 7fa8d2a00640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa8b80736c0 0x7fa8b8075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:06.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.926+0000 7fa8d2a00640 1 -- 172.21.15.66:0/3841215297 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7fa8cc092960 con 0x7fa8dc0b1e10 2024-09-10T22:08:06.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.926+0000 7fa8d1400640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa8b80736c0 0x7fa8b8075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:06.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.926+0000 7fa8d1400640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa8b80736c0 0x7fa8b8075b80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009230 tx=0x7fa8c401b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:06.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:06.930+0000 7fa8d2a00640 1 -- 172.21.15.66:0/3841215297 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa8cc05f100 con 0x7fa8dc0b1e10 2024-09-10T22:08:07.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.142+0000 7fa8e1200640 1 -- 172.21.15.66:0/3841215297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8dc15fb70 con 0x7fa8dc0b1e10 2024-09-10T22:08:07.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.142+0000 7fa8d2a00640 1 -- 172.21.15.66:0/3841215297 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5225 (secure 0 0 0) 0x7fa8cc066e20 con 0x7fa8dc0b1e10 2024-09-10T22:08:07.150 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:07.150 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":47,"btime":"2024-09-10T22:07:02:166078+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6112,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3780556367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3780556367},{"type":"v1","addr":"172.21.15.94:6836","nonce":3780556367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1892098851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1892098851},{"type":"v1","addr":"172.21.15.94:6837","nonce":1892098851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1548997357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1548997357},{"type":"v1","addr":"172.21.15.66:6834","nonce":1548997357}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:06:53.520589+0000","modified":"2024-09-10T22:06:56.547020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6390":{"gid":6390,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2415586891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2415586891},{"type":"v1","addr":"172.21.15.66:6835","nonce":2415586891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390]},"id":5}]} 2024-09-10T22:08:07.150 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 47 2024-09-10T22:08:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.146+0000 7fa8e1200640 1 -- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa8b80736c0 msgr2=0x7fa8b8075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.146+0000 7fa8e1200640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa8b80736c0 0x7fa8b8075b80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009230 tx=0x7fa8c401b090 comp rx=0 tx=0).stop 2024-09-10T22:08:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.146+0000 7fa8e1200640 1 -- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8dc0b1e10 msgr2=0x7fa8dc06e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.146+0000 7fa8e1200640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8dc0b1e10 0x7fa8dc06e5e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00e9f0 tx=0x7fa8cc00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.146+0000 7fa8e1200640 1 -- 172.21.15.66:0/3841215297 shutdown_connections 2024-09-10T22:08:07.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.146+0000 7fa8e1200640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa8b80736c0 0x7fa8b8075b80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.146+0000 7fa8e1200640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8dc1600b0 0x7fa8dc1f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.146+0000 7fa8e1200640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8dc15f450 0x7fa8dc1fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.146+0000 7fa8e1200640 1 --2- 172.21.15.66:0/3841215297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8dc0b1e10 0x7fa8dc06e5e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.146+0000 7fa8e1200640 1 -- 172.21.15.66:0/3841215297 >> 172.21.15.66:0/3841215297 conn(0x7fa8dc0d1c60 msgr2=0x7fa8dc0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:07.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.146+0000 7fa8e1200640 1 -- 172.21.15.66:0/3841215297 shutdown_connections 2024-09-10T22:08:07.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.146+0000 7fa8e1200640 1 -- 172.21.15.66:0/3841215297 wait complete. 2024-09-10T22:08:07.171 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:08:07.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 -- 172.21.15.66:0/2210047551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21940b0ef0 msgr2=0x7f21940d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 --2- 172.21.15.66:0/2210047551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21940b0ef0 0x7f21940d02c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f218c009c80 tx=0x7f218c01cc80 comp rx=0 tx=0).stop 2024-09-10T22:08:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 -- 172.21.15.66:0/2210047551 shutdown_connections 2024-09-10T22:08:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 --2- 172.21.15.66:0/2210047551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21940b0ef0 0x7f21940d02c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 --2- 172.21.15.66:0/2210047551 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2194071700 0x7f2194070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 -- 172.21.15.66:0/2210047551 >> 172.21.15.66:0/2210047551 conn(0x7f21940d1d00 msgr2=0x7f21940d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 -- 172.21.15.66:0/2210047551 shutdown_connections 2024-09-10T22:08:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 -- 172.21.15.66:0/2210047551 wait complete. 2024-09-10T22:08:07.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 Processor -- start 2024-09-10T22:08:07.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 -- start start 2024-09-10T22:08:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2194070ac0 0x7f21940cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2194071700 0x7f219415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21940b0ef0 0x7f21940b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2194160b50 con 0x7f2194071700 2024-09-10T22:08:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f21941609d0 con 0x7f2194070ac0 2024-09-10T22:08:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219c600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2194160cd0 con 0x7f21940b0ef0 2024-09-10T22:08:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f2199200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2194071700 0x7f219415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f2199c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2194070ac0 0x7f21940cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f2199200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2194071700 0x7f219415ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56660/0 (socket says 172.21.15.66:56660) 2024-09-10T22:08:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f2199c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2194070ac0 0x7f21940cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:51116/0 (socket says 172.21.15.66:51116) 2024-09-10T22:08:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f2199200640 1 -- 172.21.15.66:0/2804757950 learned_addr learned my addr 172.21.15.66:0/2804757950 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.314+0000 7f219a600640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21940b0ef0 0x7f21940b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f2199c00640 1 -- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21940b0ef0 msgr2=0x7f21940b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f2199c00640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21940b0ef0 0x7f21940b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f2199c00640 1 -- 172.21.15.66:0/2804757950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2194071700 msgr2=0x7f219415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f2199c00640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2194071700 0x7f219415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f2199c00640 1 -- 172.21.15.66:0/2804757950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21940cbd00 con 0x7f2194070ac0 2024-09-10T22:08:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f2199c00640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2194070ac0 0x7f21940cb790 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f218800c390 tx=0x7f218800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f2191600640 1 -- 172.21.15.66:0/2804757950 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188017070 con 0x7f2194070ac0 2024-09-10T22:08:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f2191600640 1 -- 172.21.15.66:0/2804757950 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21880053c0 con 0x7f2194070ac0 2024-09-10T22:08:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f2191600640 1 -- 172.21.15.66:0/2804757950 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188003840 con 0x7f2194070ac0 2024-09-10T22:08:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f219c600640 1 -- 172.21.15.66:0/2804757950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21940b2d10 con 0x7f2194070ac0 2024-09-10T22:08:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f219c600640 1 -- 172.21.15.66:0/2804757950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21940b32d0 con 0x7f2194070ac0 2024-09-10T22:08:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f219c600640 1 -- 172.21.15.66:0/2804757950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2194070ea0 con 0x7f2194070ac0 2024-09-10T22:08:07.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f2191600640 1 -- 172.21.15.66:0/2804757950 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f21880039e0 con 0x7f2194070ac0 2024-09-10T22:08:07.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.318+0000 7f2191600640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21740736c0 0x7f2174075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:07.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.322+0000 7f2191600640 1 -- 172.21.15.66:0/2804757950 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7f2188091740 con 0x7f2194070ac0 2024-09-10T22:08:07.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.322+0000 7f2199200640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21740736c0 0x7f2174075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:07.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.322+0000 7f2199200640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21740736c0 0x7f2174075b80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f21940c8fd0 tx=0x7f21800091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:07.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.326+0000 7f2191600640 1 -- 172.21.15.66:0/2804757950 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f218805deb0 con 0x7f2194070ac0 2024-09-10T22:08:07.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.534+0000 7f219c600640 1 -- 172.21.15.66:0/2804757950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21940ce280 con 0x7f2194070ac0 2024-09-10T22:08:07.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f2191600640 1 -- 172.21.15.66:0/2804757950 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5225 (secure 0 0 0) 0x7f21880628e0 con 0x7f2194070ac0 2024-09-10T22:08:07.542 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:07.542 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":47,"btime":"2024-09-10T22:07:02:166078+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6112,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3780556367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3780556367},{"type":"v1","addr":"172.21.15.94:6836","nonce":3780556367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1892098851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1892098851},{"type":"v1","addr":"172.21.15.94:6837","nonce":1892098851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1548997357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1548997357},{"type":"v1","addr":"172.21.15.66:6834","nonce":1548997357}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:06:53.520589+0000","modified":"2024-09-10T22:06:56.547020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6390":{"gid":6390,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2415586891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2415586891},{"type":"v1","addr":"172.21.15.66:6835","nonce":2415586891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390]},"id":5}]} 2024-09-10T22:08:07.543 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 47 2024-09-10T22:08:07.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f219c600640 1 -- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21740736c0 msgr2=0x7f2174075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:07.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f219c600640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21740736c0 0x7f2174075b80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f21940c8fd0 tx=0x7f21800091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f219c600640 1 -- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2194070ac0 msgr2=0x7f21940cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:07.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f219c600640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2194070ac0 0x7f21940cb790 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f218800c390 tx=0x7f218800c850 comp rx=0 tx=0).stop 2024-09-10T22:08:07.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f219c600640 1 -- 172.21.15.66:0/2804757950 shutdown_connections 2024-09-10T22:08:07.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f219c600640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21740736c0 0x7f2174075b80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f219c600640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21940b0ef0 0x7f21940b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f219c600640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2194071700 0x7f219415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f219c600640 1 --2- 172.21.15.66:0/2804757950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2194070ac0 0x7f21940cb790 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f219c600640 1 -- 172.21.15.66:0/2804757950 >> 172.21.15.66:0/2804757950 conn(0x7f21940d1d00 msgr2=0x7f21940afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:07.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f219c600640 1 -- 172.21.15.66:0/2804757950 shutdown_connections 2024-09-10T22:08:07.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.538+0000 7f219c600640 1 -- 172.21.15.66:0/2804757950 wait complete. 2024-09-10T22:08:07.565 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:08:07.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.706+0000 7f0d44a00640 1 -- 172.21.15.66:0/596761623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d400741f0 msgr2=0x7f0d4015ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:07.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.706+0000 7f0d44a00640 1 --2- 172.21.15.66:0/596761623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d400741f0 0x7f0d4015ed70 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0d40072650 tx=0x7f0d2801c770 comp rx=0 tx=0).stop 2024-09-10T22:08:07.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.706+0000 7f0d44a00640 1 -- 172.21.15.66:0/596761623 shutdown_connections 2024-09-10T22:08:07.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.706+0000 7f0d44a00640 1 --2- 172.21.15.66:0/596761623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d400741f0 0x7f0d4015ed70 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.706+0000 7f0d44a00640 1 --2- 172.21.15.66:0/596761623 >> v2:172.21.15.94:3300/0 conn(0x7f0d400738d0 0x7f0d40073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.706+0000 7f0d44a00640 1 -- 172.21.15.66:0/596761623 >> 172.21.15.66:0/596761623 conn(0x7f0d400cc900 msgr2=0x7f0d400ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:07.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.706+0000 7f0d44a00640 1 -- 172.21.15.66:0/596761623 shutdown_connections 2024-09-10T22:08:07.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.706+0000 7f0d44a00640 1 -- 172.21.15.66:0/596761623 wait complete. 2024-09-10T22:08:07.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.706+0000 7f0d44a00640 1 Processor -- start 2024-09-10T22:08:07.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.706+0000 7f0d44a00640 1 -- start start 2024-09-10T22:08:07.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.706+0000 7f0d44a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d40070b00 0x7f0d4009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:07.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.706+0000 7f0d44a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d400738d0 0x7f0d4015e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:07.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d44a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d400741f0 0x7f0d40150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:07.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d44a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d400ae8a0 con 0x7f0d400738d0 2024-09-10T22:08:07.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d44a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d400ae720 con 0x7f0d400741f0 2024-09-10T22:08:07.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d44a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0d400aea20 con 0x7f0d40070b00 2024-09-10T22:08:07.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d37e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d400738d0 0x7f0d4015e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:07.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d37e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d400738d0 0x7f0d4015e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56688/0 (socket says 172.21.15.66:56688) 2024-09-10T22:08:07.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d37e00640 1 -- 172.21.15.66:0/3955818298 learned_addr learned my addr 172.21.15.66:0/3955818298 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:07.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d3de00640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d40070b00 0x7f0d4009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:07.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d3e800640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d400741f0 0x7f0d40150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:07.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d37e00640 1 -- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d40070b00 msgr2=0x7f0d4009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:07.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d37e00640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d40070b00 0x7f0d4009aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d37e00640 1 -- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d400741f0 msgr2=0x7f0d40150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:07.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d37e00640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d400741f0 0x7f0d40150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d37e00640 1 -- 172.21.15.66:0/3955818298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4015edc0 con 0x7f0d400738d0 2024-09-10T22:08:07.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d37e00640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d400738d0 0x7f0d4015e880 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0d280b0040 tx=0x7f0d280a5540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:07.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d36000640 1 -- 172.21.15.66:0/3955818298 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d280b2040 con 0x7f0d400738d0 2024-09-10T22:08:07.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d36000640 1 -- 172.21.15.66:0/3955818298 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d280042c0 con 0x7f0d400738d0 2024-09-10T22:08:07.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d36000640 1 -- 172.21.15.66:0/3955818298 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28005290 con 0x7f0d400738d0 2024-09-10T22:08:07.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d44a00640 1 -- 172.21.15.66:0/3955818298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4009d230 con 0x7f0d400738d0 2024-09-10T22:08:07.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d44a00640 1 -- 172.21.15.66:0/3955818298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4009d7f0 con 0x7f0d400738d0 2024-09-10T22:08:07.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d44a00640 1 -- 172.21.15.66:0/3955818298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d400cc150 con 0x7f0d400738d0 2024-09-10T22:08:07.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.710+0000 7f0d36000640 1 -- 172.21.15.66:0/3955818298 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f0d28005430 con 0x7f0d400738d0 2024-09-10T22:08:07.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.714+0000 7f0d36000640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d1c0736c0 0x7f0d1c075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:07.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.714+0000 7f0d36000640 1 -- 172.21.15.66:0/3955818298 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7f0d2812bad0 con 0x7f0d400738d0 2024-09-10T22:08:07.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.714+0000 7f0d3de00640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d1c0736c0 0x7f0d1c075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:07.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.714+0000 7f0d3de00640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d1c0736c0 0x7f0d1c075b80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000bc00 tx=0x7f0d3000b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:07.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.718+0000 7f0d36000640 1 -- 172.21.15.66:0/3955818298 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0d4015edc0 con 0x7f0d400738d0 2024-09-10T22:08:07.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.890+0000 7f0d44a00640 1 -- 172.21.15.66:0/3955818298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d4009dd40 con 0x7f0d400738d0 2024-09-10T22:08:07.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d36000640 1 -- 172.21.15.66:0/3955818298 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16908 (secure 0 0 0) 0x7f0d280164b0 con 0x7f0d400738d0 2024-09-10T22:08:07.899 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:07.899 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":74,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:08:05.076199+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:06:51.234296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T22:06:52.488635+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":74,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:08:07.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d44a00640 1 -- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d1c0736c0 msgr2=0x7f0d1c075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:07.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d44a00640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d1c0736c0 0x7f0d1c075b80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000bc00 tx=0x7f0d3000b390 comp rx=0 tx=0).stop 2024-09-10T22:08:07.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d44a00640 1 -- 172.21.15.66:0/3955818298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d400738d0 msgr2=0x7f0d4015e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:07.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d44a00640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d400738d0 0x7f0d4015e880 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0d280b0040 tx=0x7f0d280a5540 comp rx=0 tx=0).stop 2024-09-10T22:08:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d44a00640 1 -- 172.21.15.66:0/3955818298 shutdown_connections 2024-09-10T22:08:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d44a00640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d1c0736c0 0x7f0d1c075b80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d44a00640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d400741f0 0x7f0d40150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d44a00640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d400738d0 0x7f0d4015e880 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d44a00640 1 --2- 172.21.15.66:0/3955818298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d40070b00 0x7f0d4009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d44a00640 1 -- 172.21.15.66:0/3955818298 >> 172.21.15.66:0/3955818298 conn(0x7f0d400cc900 msgr2=0x7f0d400affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d44a00640 1 -- 172.21.15.66:0/3955818298 shutdown_connections 2024-09-10T22:08:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:07.894+0000 7f0d44a00640 1 -- 172.21.15.66:0/3955818298 wait complete. 2024-09-10T22:08:07.921 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:08:08.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.062+0000 7f87be600640 1 -- 172.21.15.66:0/1620830003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b80b2bf0 msgr2=0x7f87b80aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:08.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.062+0000 7f87be600640 1 --2- 172.21.15.66:0/1620830003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b80b2bf0 0x7f87b80aeea0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f87b8073790 tx=0x7f87b001cd80 comp rx=0 tx=0).stop 2024-09-10T22:08:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.062+0000 7f87be600640 1 -- 172.21.15.66:0/1620830003 shutdown_connections 2024-09-10T22:08:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.062+0000 7f87be600640 1 --2- 172.21.15.66:0/1620830003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b80b2bf0 0x7f87b80aeea0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.062+0000 7f87be600640 1 --2- 172.21.15.66:0/1620830003 >> v2:172.21.15.94:3300/0 conn(0x7f87b80d0260 0x7f87b80ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.062+0000 7f87be600640 1 -- 172.21.15.66:0/1620830003 >> 172.21.15.66:0/1620830003 conn(0x7f87b80d1c50 msgr2=0x7f87b80d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.062+0000 7f87be600640 1 -- 172.21.15.66:0/1620830003 shutdown_connections 2024-09-10T22:08:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.062+0000 7f87be600640 1 -- 172.21.15.66:0/1620830003 wait complete. 2024-09-10T22:08:08.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87be600640 1 Processor -- start 2024-09-10T22:08:08.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87be600640 1 -- start start 2024-09-10T22:08:08.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87be600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87b80b2bf0 0x7f87b80cf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:08.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87be600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87b80d0260 0x7f87b815e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:08.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87be600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b815f340 0x7f87b815e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:08.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87be600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81692a0 con 0x7f87b815f340 2024-09-10T22:08:08.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87be600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8169120 con 0x7f87b80b2bf0 2024-09-10T22:08:08.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87be600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f87b8169420 con 0x7f87b80d0260 2024-09-10T22:08:08.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b815f340 0x7f87b815e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:08.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b815f340 0x7f87b815e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56710/0 (socket says 172.21.15.66:56710) 2024-09-10T22:08:08.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87b7e00640 1 -- 172.21.15.66:0/180685960 learned_addr learned my addr 172.21.15.66:0/180685960 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:08.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87afe00640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87b80d0260 0x7f87b815e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:08.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87b7400640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87b80b2bf0 0x7f87b80cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:08.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87b7e00640 1 -- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87b80d0260 msgr2=0x7f87b815e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:08.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87b7e00640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87b80d0260 0x7f87b815e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:08.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87b7e00640 1 -- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87b80b2bf0 msgr2=0x7f87b80cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:08.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87b7e00640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87b80b2bf0 0x7f87b80cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:08.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87b7e00640 1 -- 172.21.15.66:0/180685960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b8069a70 con 0x7f87b815f340 2024-09-10T22:08:08.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87b7400640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87b80b2bf0 0x7f87b80cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:08.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87b7e00640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b815f340 0x7f87b815e5a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f87b0019990 tx=0x7f87b0009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87af400640 1 -- 172.21.15.66:0/180685960 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b00bb070 con 0x7f87b815f340 2024-09-10T22:08:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87af400640 1 -- 172.21.15.66:0/180685960 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87b0004c00 con 0x7f87b815f340 2024-09-10T22:08:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87af400640 1 -- 172.21.15.66:0/180685960 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b00b2040 con 0x7f87b815f340 2024-09-10T22:08:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87be600640 1 -- 172.21.15.66:0/180685960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b80b61f0 con 0x7f87b815f340 2024-09-10T22:08:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87be600640 1 -- 172.21.15.66:0/180685960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b80b67b0 con 0x7f87b815f340 2024-09-10T22:08:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.066+0000 7f87be600640 1 -- 172.21.15.66:0/180685960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b80cfb50 con 0x7f87b815f340 2024-09-10T22:08:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.070+0000 7f87af400640 1 -- 172.21.15.66:0/180685960 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f87b00b5030 con 0x7f87b815f340 2024-09-10T22:08:08.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.070+0000 7f87af400640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f87940736b0 0x7f8794075b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:08.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.070+0000 7f87af400640 1 -- 172.21.15.66:0/180685960 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5851+0+0 (secure 0 0 0) 0x7f87b012b840 con 0x7f87b815f340 2024-09-10T22:08:08.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.070+0000 7f87b7400640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f87940736b0 0x7f8794075b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:08.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.070+0000 7f87b7400640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f87940736b0 0x7f8794075b70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f87a80016b0 tx=0x7f87a8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:08.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.074+0000 7f87af400640 1 -- 172.21.15.66:0/180685960 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f87b0016970 con 0x7f87b815f340 2024-09-10T22:08:08.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.282+0000 7f87be600640 1 -- 172.21.15.66:0/180685960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f87b80c50d0 con 0x7f87b815f340 2024-09-10T22:08:08.288 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:08:08.282+0000 7fcc93a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:08:08.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87af400640 1 -- 172.21.15.66:0/180685960 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7f87b0101090 con 0x7f87b815f340 2024-09-10T22:08:08.290 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:08:08.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87be600640 1 -- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f87940736b0 msgr2=0x7f8794075b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:08.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87be600640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f87940736b0 0x7f8794075b70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f87a80016b0 tx=0x7f87a8009040 comp rx=0 tx=0).stop 2024-09-10T22:08:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87be600640 1 -- 172.21.15.66:0/180685960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b815f340 msgr2=0x7f87b815e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87be600640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b815f340 0x7f87b815e5a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f87b0019990 tx=0x7f87b0009300 comp rx=0 tx=0).stop 2024-09-10T22:08:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87be600640 1 -- 172.21.15.66:0/180685960 shutdown_connections 2024-09-10T22:08:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87be600640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f87940736b0 0x7f8794075b70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87be600640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b815f340 0x7f87b815e5a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87be600640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87b80d0260 0x7f87b815e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87be600640 1 --2- 172.21.15.66:0/180685960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87b80b2bf0 0x7f87b80cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87be600640 1 -- 172.21.15.66:0/180685960 >> 172.21.15.66:0/180685960 conn(0x7f87b80d1c50 msgr2=0x7f87b8071550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87be600640 1 -- 172.21.15.66:0/180685960 shutdown_connections 2024-09-10T22:08:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.286+0000 7f87be600640 1 -- 172.21.15.66:0/180685960 wait complete. 2024-09-10T22:08:08.316 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:08:08.361 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:08:08.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.462+0000 7f81db200640 1 -- 172.21.15.66:0/1856069262 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81dc0cd7a0 msgr2=0x7f81dc0cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:08.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.462+0000 7f81db200640 1 --2- 172.21.15.66:0/1856069262 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81dc0cd7a0 0x7f81dc0cdc30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009c20 tx=0x7f81d401cbc0 comp rx=0 tx=0).stop 2024-09-10T22:08:08.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.462+0000 7f81db200640 1 -- 172.21.15.66:0/1856069262 shutdown_connections 2024-09-10T22:08:08.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.462+0000 7f81db200640 1 --2- 172.21.15.66:0/1856069262 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81dc0cd7a0 0x7f81dc0cdc30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:08.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.462+0000 7f81db200640 1 -- 172.21.15.66:0/1856069262 >> 172.21.15.66:0/1856069262 conn(0x7f81dc0d1b10 msgr2=0x7f81dc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:08.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.462+0000 7f81db200640 1 -- 172.21.15.66:0/1856069262 shutdown_connections 2024-09-10T22:08:08.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.462+0000 7f81db200640 1 -- 172.21.15.66:0/1856069262 wait complete. 2024-09-10T22:08:08.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.462+0000 7f81db200640 1 Processor -- start 2024-09-10T22:08:08.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.462+0000 7f81db200640 1 -- start start 2024-09-10T22:08:08.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81db200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81dc0cd7a0 0x7f81dc1f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81db200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81dc15ef20 0x7f81dc167520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81db200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81dc15f300 0x7f81dc0b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81db200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc16a1c0 con 0x7f81dc15f300 2024-09-10T22:08:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81db200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc16a040 con 0x7f81dc0cd7a0 2024-09-10T22:08:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81db200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f81dc16a340 con 0x7f81dc15ef20 2024-09-10T22:08:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81da800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81dc15f300 0x7f81dc0b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81da800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81dc15f300 0x7f81dc0b6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56730/0 (socket says 172.21.15.66:56730) 2024-09-10T22:08:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81da800640 1 -- 172.21.15.66:0/2616916924 learned_addr learned my addr 172.21.15.66:0/2616916924 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81d3e00640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81dc15ef20 0x7f81dc167520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81d9e00640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81dc0cd7a0 0x7f81dc1f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81da800640 1 -- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81dc15ef20 msgr2=0x7f81dc167520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81da800640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81dc15ef20 0x7f81dc167520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:08.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81da800640 1 -- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81dc0cd7a0 msgr2=0x7f81dc1f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:08.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81da800640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81dc0cd7a0 0x7f81dc1f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:08.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81da800640 1 -- 172.21.15.66:0/2616916924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc0cb700 con 0x7f81dc15f300 2024-09-10T22:08:08.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81da800640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81dc15f300 0x7f81dc0b6b10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f81d401c630 tx=0x7f81d4003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:08.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81d2000640 1 -- 172.21.15.66:0/2616916924 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40bb070 con 0x7f81dc15f300 2024-09-10T22:08:08.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81d2000640 1 -- 172.21.15.66:0/2616916924 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81d4004d60 con 0x7f81dc15f300 2024-09-10T22:08:08.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81d2000640 1 -- 172.21.15.66:0/2616916924 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40b2040 con 0x7f81dc15f300 2024-09-10T22:08:08.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81db200640 1 -- 172.21.15.66:0/2616916924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc0c9150 con 0x7f81dc15f300 2024-09-10T22:08:08.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81db200640 1 -- 172.21.15.66:0/2616916924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc0cbc60 con 0x7f81dc15f300 2024-09-10T22:08:08.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.466+0000 7f81db200640 1 -- 172.21.15.66:0/2616916924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81dc0cdc30 con 0x7f81dc15f300 2024-09-10T22:08:08.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.470+0000 7f81d2000640 1 -- 172.21.15.66:0/2616916924 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f81d40b5070 con 0x7f81dc15f300 2024-09-10T22:08:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.470+0000 7f81d2000640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f81b80736c0 0x7f81b8075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.470+0000 7f81d2000640 1 -- 172.21.15.66:0/2616916924 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5909+0+0 (secure 0 0 0) 0x7f81d412bba0 con 0x7f81dc15f300 2024-09-10T22:08:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.470+0000 7f81d9e00640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f81b80736c0 0x7f81b8075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.470+0000 7f81d9e00640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f81b80736c0 0x7f81b8075b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f81cc007a80 tx=0x7f81cc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:08.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.474+0000 7f81d2000640 1 -- 172.21.15.66:0/2616916924 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f81dc0cb700 con 0x7f81dc15f300 2024-09-10T22:08:08.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:08.682+0000 7f81db200640 1 -- 172.21.15.66:0/2616916924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f81dc15bfe0 con 0x7f81dc15f300 2024-09-10T22:08:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.290+0000 7f81d2000640 1 -- 172.21.15.66:0/2616916924 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7f81d40167b0 con 0x7f81dc15f300 2024-09-10T22:08:09.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.294+0000 7f81db200640 1 -- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f81b80736c0 msgr2=0x7f81b8075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:09.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.294+0000 7f81db200640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f81b80736c0 0x7f81b8075b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f81cc007a80 tx=0x7f81cc009040 comp rx=0 tx=0).stop 2024-09-10T22:08:09.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.294+0000 7f81db200640 1 -- 172.21.15.66:0/2616916924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81dc15f300 msgr2=0x7f81dc0b6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:09.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.294+0000 7f81db200640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81dc15f300 0x7f81dc0b6b10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f81d401c630 tx=0x7f81d4003f40 comp rx=0 tx=0).stop 2024-09-10T22:08:09.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.294+0000 7f81db200640 1 -- 172.21.15.66:0/2616916924 shutdown_connections 2024-09-10T22:08:09.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.294+0000 7f81db200640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f81b80736c0 0x7f81b8075b80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:09.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.294+0000 7f81db200640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81dc15f300 0x7f81dc0b6b10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:09.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.294+0000 7f81db200640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81dc15ef20 0x7f81dc167520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:09.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.294+0000 7f81db200640 1 --2- 172.21.15.66:0/2616916924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81dc0cd7a0 0x7f81dc1f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:09.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.294+0000 7f81db200640 1 -- 172.21.15.66:0/2616916924 >> 172.21.15.66:0/2616916924 conn(0x7f81dc0d1b10 msgr2=0x7f81dc06e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:09.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.294+0000 7f81db200640 1 -- 172.21.15.66:0/2616916924 shutdown_connections 2024-09-10T22:08:09.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.294+0000 7f81db200640 1 -- 172.21.15.66:0/2616916924 wait complete. 2024-09-10T22:08:09.371 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:08:09.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.562+0000 7f458c000640 1 -- 172.21.15.66:0/3746622727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4584145050 msgr2=0x7f45840ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:09.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.562+0000 7f458c000640 1 --2- 172.21.15.66:0/3746622727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4584145050 0x7f45840ceb40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4584076880 tx=0x7f457001c830 comp rx=0 tx=0).stop 2024-09-10T22:08:09.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.562+0000 7f458c000640 1 -- 172.21.15.66:0/3746622727 shutdown_connections 2024-09-10T22:08:09.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.562+0000 7f458c000640 1 --2- 172.21.15.66:0/3746622727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4584145050 0x7f45840ceb40 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:09.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.562+0000 7f458c000640 1 --2- 172.21.15.66:0/3746622727 >> v2:172.21.15.94:3300/0 conn(0x7f4584144ac0 0x7f458406e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:09.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.562+0000 7f458c000640 1 -- 172.21.15.66:0/3746622727 >> 172.21.15.66:0/3746622727 conn(0x7f45840cfaa0 msgr2=0x7f45840cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:09.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.562+0000 7f458c000640 1 -- 172.21.15.66:0/3746622727 shutdown_connections 2024-09-10T22:08:09.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.562+0000 7f458c000640 1 -- 172.21.15.66:0/3746622727 wait complete. 2024-09-10T22:08:09.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.562+0000 7f458c000640 1 Processor -- start 2024-09-10T22:08:09.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.562+0000 7f458c000640 1 -- start start 2024-09-10T22:08:09.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f458c000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45840cf080 0x7f458413c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:09.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f458c000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4584144ac0 0x7f458406cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:09.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f458c000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4584145050 0x7f458406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:09.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f458c000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4584168fa0 con 0x7f4584144ac0 2024-09-10T22:08:09.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f458c000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4584168e20 con 0x7f45840cf080 2024-09-10T22:08:09.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f458c000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4584169120 con 0x7f4584145050 2024-09-10T22:08:09.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4588c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4584144ac0 0x7f458406cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:09.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4588c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4584144ac0 0x7f458406cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56740/0 (socket says 172.21.15.66:56740) 2024-09-10T22:08:09.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4588c00640 1 -- 172.21.15.66:0/1628584853 learned_addr learned my addr 172.21.15.66:0/1628584853 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:09.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4589600640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45840cf080 0x7f458413c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:09.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f458a000640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4584145050 0x7f458406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:09.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4588c00640 1 -- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4584145050 msgr2=0x7f458406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:09.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4588c00640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4584145050 0x7f458406d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:09.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4588c00640 1 -- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45840cf080 msgr2=0x7f458413c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:09.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4588c00640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45840cf080 0x7f458413c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:09.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4588c00640 1 -- 172.21.15.66:0/1628584853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45841609a0 con 0x7f4584144ac0 2024-09-10T22:08:09.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4588c00640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4584144ac0 0x7f458406cc20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4570019440 tx=0x7f45700a5600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:09.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4581600640 1 -- 172.21.15.66:0/1628584853 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45700bb070 con 0x7f4584144ac0 2024-09-10T22:08:09.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4581600640 1 -- 172.21.15.66:0/1628584853 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4570004320 con 0x7f4584144ac0 2024-09-10T22:08:09.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f4581600640 1 -- 172.21.15.66:0/1628584853 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45700b2040 con 0x7f4584144ac0 2024-09-10T22:08:09.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f458c000640 1 -- 172.21.15.66:0/1628584853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4584160c80 con 0x7f4584144ac0 2024-09-10T22:08:09.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f458c000640 1 -- 172.21.15.66:0/1628584853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4584161240 con 0x7f4584144ac0 2024-09-10T22:08:09.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.566+0000 7f458c000640 1 -- 172.21.15.66:0/1628584853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45840ce1c0 con 0x7f4584144ac0 2024-09-10T22:08:09.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.570+0000 7f4581600640 1 -- 172.21.15.66:0/1628584853 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f45700052d0 con 0x7f4584144ac0 2024-09-10T22:08:09.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.570+0000 7f4581600640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45640736c0 0x7f4564075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:09.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.570+0000 7f4581600640 1 -- 172.21.15.66:0/1628584853 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5909+0+0 (secure 0 0 0) 0x7f457012b5f0 con 0x7f4584144ac0 2024-09-10T22:08:09.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.570+0000 7f4589600640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45640736c0 0x7f4564075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:09.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.570+0000 7f4589600640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45640736c0 0x7f4564075b80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4578005d90 tx=0x7f457800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:09.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.574+0000 7f4581600640 1 -- 172.21.15.66:0/1628584853 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f45700164b0 con 0x7f4584144ac0 2024-09-10T22:08:09.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:09.750+0000 7f458c000640 1 -- 172.21.15.66:0/1628584853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f45840d0290 con 0x7f4584144ac0 2024-09-10T22:08:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.294+0000 7f4581600640 1 -- 172.21.15.66:0/1628584853 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) ==== 181+0+0 (secure 0 0 0) 0x7f45700f7d90 con 0x7f4584144ac0 2024-09-10T22:08:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:08:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.298+0000 7f458c000640 1 -- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45640736c0 msgr2=0x7f4564075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.298+0000 7f458c000640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45640736c0 0x7f4564075b80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4578005d90 tx=0x7f457800b390 comp rx=0 tx=0).stop 2024-09-10T22:08:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.298+0000 7f458c000640 1 -- 172.21.15.66:0/1628584853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4584144ac0 msgr2=0x7f458406cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.298+0000 7f458c000640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4584144ac0 0x7f458406cc20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4570019440 tx=0x7f45700a5600 comp rx=0 tx=0).stop 2024-09-10T22:08:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.298+0000 7f458c000640 1 -- 172.21.15.66:0/1628584853 shutdown_connections 2024-09-10T22:08:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.298+0000 7f458c000640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45640736c0 0x7f4564075b80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.298+0000 7f458c000640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4584145050 0x7f458406d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.298+0000 7f458c000640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4584144ac0 0x7f458406cc20 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.298+0000 7f458c000640 1 --2- 172.21.15.66:0/1628584853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45840cf080 0x7f458413c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.298+0000 7f458c000640 1 -- 172.21.15.66:0/1628584853 >> 172.21.15.66:0/1628584853 conn(0x7f45840cfaa0 msgr2=0x7f45840b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.298+0000 7f458c000640 1 -- 172.21.15.66:0/1628584853 shutdown_connections 2024-09-10T22:08:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.298+0000 7f458c000640 1 -- 172.21.15.66:0/1628584853 wait complete. 2024-09-10T22:08:10.325 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:08:10.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 -- 172.21.15.66:0/2585665605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce98004030 msgr2=0x7fce98004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:10.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 --2- 172.21.15.66:0/2585665605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce98004030 0x7fce98004810 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fceb0073e20 tx=0x7fce9c01f610 comp rx=0 tx=0).stop 2024-09-10T22:08:10.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb6400640 1 -- 172.21.15.66:0/2585665605 reap_dead start 2024-09-10T22:08:10.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 -- 172.21.15.66:0/2585665605 shutdown_connections 2024-09-10T22:08:10.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 -- 172.21.15.66:0/2585665605 >> 172.21.15.66:0/2585665605 conn(0x7fceb00d1c60 msgr2=0x7fceb00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:10.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 -- 172.21.15.66:0/2585665605 shutdown_connections 2024-09-10T22:08:10.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 -- 172.21.15.66:0/2585665605 wait complete. 2024-09-10T22:08:10.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 Processor -- start 2024-09-10T22:08:10.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 -- start start 2024-09-10T22:08:10.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce98004030 0x7fceb01590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:10.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fceb014f130 0x7fceb014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:10.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fceb014bd60 0x7fceb014c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:10.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fceb00d0350 con 0x7fceb014f130 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fceb00d01d0 con 0x7fce98004030 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb8400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fceb00d04d0 con 0x7fceb014bd60 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fceb014f130 0x7fceb014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fceb014f130 0x7fceb014b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56754/0 (socket says 172.21.15.66:56754) 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb5000640 1 -- 172.21.15.66:0/618098525 learned_addr learned my addr 172.21.15.66:0/618098525 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb5a00640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce98004030 0x7fceb01590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb6400640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fceb014bd60 0x7fceb014c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb5000640 1 -- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fceb014bd60 msgr2=0x7fceb014c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb5000640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fceb014bd60 0x7fceb014c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb5000640 1 -- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce98004030 msgr2=0x7fceb01590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb5000640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce98004030 0x7fceb01590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb5000640 1 -- 172.21.15.66:0/618098525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb015c2e0 con 0x7fceb014f130 2024-09-10T22:08:10.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.474+0000 7fceb5000640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fceb014f130 0x7fceb014b820 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fce9c026320 tx=0x7fce9c001cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:10.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.478+0000 7fcead600640 1 -- 172.21.15.66:0/618098525 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0b5040 con 0x7fceb014f130 2024-09-10T22:08:10.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.478+0000 7fcead600640 1 -- 172.21.15.66:0/618098525 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce9c0bf070 con 0x7fceb014f130 2024-09-10T22:08:10.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.478+0000 7fcead600640 1 -- 172.21.15.66:0/618098525 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c001620 con 0x7fceb014f130 2024-09-10T22:08:10.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.478+0000 7fceb8400640 1 -- 172.21.15.66:0/618098525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb015b6d0 con 0x7fceb014f130 2024-09-10T22:08:10.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.478+0000 7fceb8400640 1 -- 172.21.15.66:0/618098525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb015bc90 con 0x7fceb014f130 2024-09-10T22:08:10.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.478+0000 7fceb8400640 1 -- 172.21.15.66:0/618098525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb00cecb0 con 0x7fceb014f130 2024-09-10T22:08:10.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.478+0000 7fcead600640 1 -- 172.21.15.66:0/618098525 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fce9c003600 con 0x7fceb014f130 2024-09-10T22:08:10.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.478+0000 7fcead600640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce900736c0 0x7fce90075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:10.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.478+0000 7fcead600640 1 -- 172.21.15.66:0/618098525 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5471+0+0 (secure 0 0 0) 0x7fce9c12f8d0 con 0x7fceb014f130 2024-09-10T22:08:10.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.478+0000 7fceb5a00640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce900736c0 0x7fce90075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:10.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.482+0000 7fceb5a00640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce900736c0 0x7fce90075b80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009d80 tx=0x7fcea40091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:10.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.482+0000 7fcead600640 1 -- 172.21.15.66:0/618098525 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fce9c0193c0 con 0x7fceb014f130 2024-09-10T22:08:10.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:10.662+0000 7fceb8400640 1 -- 172.21.15.66:0/618098525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fceb0070550 con 0x7fceb014f130 2024-09-10T22:08:11.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.302+0000 7fcead600640 1 -- 172.21.15.66:0/618098525 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) ==== 169+0+0 (secure 0 0 0) 0x7fce9c0fca40 con 0x7fceb014f130 2024-09-10T22:08:11.309 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:08:11.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.306+0000 7fceb8400640 1 -- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce900736c0 msgr2=0x7fce90075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:11.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.306+0000 7fceb8400640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce900736c0 0x7fce90075b80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009d80 tx=0x7fcea40091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.306+0000 7fceb8400640 1 -- 172.21.15.66:0/618098525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fceb014f130 msgr2=0x7fceb014b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:11.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.306+0000 7fceb8400640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fceb014f130 0x7fceb014b820 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fce9c026320 tx=0x7fce9c001cd0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.306+0000 7fceb8400640 1 -- 172.21.15.66:0/618098525 shutdown_connections 2024-09-10T22:08:11.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.306+0000 7fceb8400640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce900736c0 0x7fce90075b80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.306+0000 7fceb8400640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fceb014bd60 0x7fceb014c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.306+0000 7fceb8400640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fceb014f130 0x7fceb014b820 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.306+0000 7fceb8400640 1 --2- 172.21.15.66:0/618098525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce98004030 0x7fceb01590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.306+0000 7fceb8400640 1 -- 172.21.15.66:0/618098525 >> 172.21.15.66:0/618098525 conn(0x7fceb00d1c60 msgr2=0x7fceb006f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:11.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.310+0000 7fceb8400640 1 -- 172.21.15.66:0/618098525 shutdown_connections 2024-09-10T22:08:11.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.310+0000 7fceb8400640 1 -- 172.21.15.66:0/618098525 wait complete. 2024-09-10T22:08:11.346 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T22:08:11.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.490+0000 7f442f200640 1 -- 172.21.15.66:0/3023538268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44280b15d0 msgr2=0x7f44280cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:11.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.490+0000 7f442f200640 1 --2- 172.21.15.66:0/3023538268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44280b15d0 0x7f44280cd7a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4428073890 tx=0x7f442401cd50 comp rx=0 tx=0).stop 2024-09-10T22:08:11.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.490+0000 7f442f200640 1 -- 172.21.15.66:0/3023538268 shutdown_connections 2024-09-10T22:08:11.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.490+0000 7f442f200640 1 --2- 172.21.15.66:0/3023538268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44280b15d0 0x7f44280cd7a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.490+0000 7f442f200640 1 --2- 172.21.15.66:0/3023538268 >> v2:172.21.15.94:3300/0 conn(0x7f442815f300 0x7f44280b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.490+0000 7f442f200640 1 --2- 172.21.15.66:0/3023538268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f442815ef20 0x7f44280cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.490+0000 7f442f200640 1 -- 172.21.15.66:0/3023538268 >> 172.21.15.66:0/3023538268 conn(0x7f44280d1b10 msgr2=0x7f44280d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:11.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442f200640 1 -- 172.21.15.66:0/3023538268 shutdown_connections 2024-09-10T22:08:11.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442f200640 1 -- 172.21.15.66:0/3023538268 wait complete. 2024-09-10T22:08:11.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442f200640 1 Processor -- start 2024-09-10T22:08:11.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442f200640 1 -- start start 2024-09-10T22:08:11.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442f200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44280b15d0 0x7f44280c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:11.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f442815ef20 0x7f4428153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:11.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f442815f300 0x7f4428165640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:11.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f442815fd90 con 0x7f442815ef20 2024-09-10T22:08:11.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f442815fc10 con 0x7f442815f300 2024-09-10T22:08:11.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442f200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f442815ff10 con 0x7f44280b15d0 2024-09-10T22:08:11.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f442815ef20 0x7f4428153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:11.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f442815ef20 0x7f4428153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56772/0 (socket says 172.21.15.66:56772) 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442d400640 1 -- 172.21.15.66:0/3543223632 learned_addr learned my addr 172.21.15.66:0/3543223632 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442de00640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44280b15d0 0x7f44280c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442e800640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f442815f300 0x7f4428165640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442d400640 1 -- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44280b15d0 msgr2=0x7f44280c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442d400640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44280b15d0 0x7f44280c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442d400640 1 -- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f442815f300 msgr2=0x7f4428165640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442d400640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f442815f300 0x7f4428165640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442d400640 1 -- 172.21.15.66:0/3543223632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44280ca2b0 con 0x7f442815ef20 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.494+0000 7f442d400640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f442815ef20 0x7f4428153c90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f441800d950 tx=0x7f441800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.498+0000 7f4422000640 1 -- 172.21.15.66:0/3543223632 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418017070 con 0x7f442815ef20 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.498+0000 7f4422000640 1 -- 172.21.15.66:0/3543223632 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4418009d20 con 0x7f442815ef20 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.498+0000 7f4422000640 1 -- 172.21.15.66:0/3543223632 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44180106c0 con 0x7f442815ef20 2024-09-10T22:08:11.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.498+0000 7f442f200640 1 -- 172.21.15.66:0/3543223632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44280ca590 con 0x7f442815ef20 2024-09-10T22:08:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.498+0000 7f442f200640 1 -- 172.21.15.66:0/3543223632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44280cab50 con 0x7f442815ef20 2024-09-10T22:08:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.498+0000 7f442f200640 1 -- 172.21.15.66:0/3543223632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4428160430 con 0x7f442815ef20 2024-09-10T22:08:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.498+0000 7f4422000640 1 -- 172.21.15.66:0/3543223632 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f4418010860 con 0x7f442815ef20 2024-09-10T22:08:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.498+0000 7f4422000640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f440c0736f0 0x7f440c075bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.498+0000 7f4422000640 1 -- 172.21.15.66:0/3543223632 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7f4418092480 con 0x7f442815ef20 2024-09-10T22:08:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.498+0000 7f442de00640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f440c0736f0 0x7f440c075bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.502+0000 7f442de00640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f440c0736f0 0x7f440c075bb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f441c004630 tx=0x7f441c00b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:11.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.502+0000 7f4422000640 1 -- 172.21.15.66:0/3543223632 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f441805ec20 con 0x7f442815ef20 2024-09-10T22:08:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.678+0000 7f442f200640 1 -- 172.21.15.66:0/3543223632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f4428072e00 con 0x7f442815ef20 2024-09-10T22:08:11.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.682+0000 7f4422000640 1 -- 172.21.15.66:0/3543223632 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v40) ==== 71+0+0 (secure 0 0 0) 0x7f4418063650 con 0x7f442815ef20 2024-09-10T22:08:11.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.686+0000 7f442f200640 1 -- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f440c0736f0 msgr2=0x7f440c075bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:11.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.686+0000 7f442f200640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f440c0736f0 0x7f440c075bb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f441c004630 tx=0x7f441c00b2e0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.686+0000 7f442f200640 1 -- 172.21.15.66:0/3543223632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f442815ef20 msgr2=0x7f4428153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:11.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.686+0000 7f442f200640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f442815ef20 0x7f4428153c90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f441800d950 tx=0x7f441800de10 comp rx=0 tx=0).stop 2024-09-10T22:08:11.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.686+0000 7f442f200640 1 -- 172.21.15.66:0/3543223632 shutdown_connections 2024-09-10T22:08:11.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.686+0000 7f442f200640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f440c0736f0 0x7f440c075bb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.686+0000 7f442f200640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f442815f300 0x7f4428165640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.686+0000 7f442f200640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f442815ef20 0x7f4428153c90 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.686+0000 7f442f200640 1 --2- 172.21.15.66:0/3543223632 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44280b15d0 0x7f44280c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.686+0000 7f442f200640 1 -- 172.21.15.66:0/3543223632 >> 172.21.15.66:0/3543223632 conn(0x7f44280d1b10 msgr2=0x7f44280b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:11.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.686+0000 7f442f200640 1 -- 172.21.15.66:0/3543223632 shutdown_connections 2024-09-10T22:08:11.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.686+0000 7f442f200640 1 -- 172.21.15.66:0/3543223632 wait complete. 2024-09-10T22:08:11.712 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-10T22:08:11.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 -- 172.21.15.66:0/3814754561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdacc156430 msgr2=0x7fdacc06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:11.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 --2- 172.21.15.66:0/3814754561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdacc156430 0x7fdacc06d630 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fdacc06db70 tx=0x7fdabc01c830 comp rx=0 tx=0).stop 2024-09-10T22:08:11.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 -- 172.21.15.66:0/3814754561 shutdown_connections 2024-09-10T22:08:11.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 --2- 172.21.15.66:0/3814754561 >> v2:172.21.15.94:3300/0 conn(0x7fdacc071230 0x7fdacc0cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 --2- 172.21.15.66:0/3814754561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdacc156430 0x7fdacc06d630 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 -- 172.21.15.66:0/3814754561 >> 172.21.15.66:0/3814754561 conn(0x7fdacc0d0b30 msgr2=0x7fdacc0d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:11.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 -- 172.21.15.66:0/3814754561 shutdown_connections 2024-09-10T22:08:11.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 -- 172.21.15.66:0/3814754561 wait complete. 2024-09-10T22:08:11.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 Processor -- start 2024-09-10T22:08:11.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 -- start start 2024-09-10T22:08:11.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdacc071230 0x7fdacc150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:11.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdacc156430 0x7fdacc0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:11.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdacc160370 0x7fdacc0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:11.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc168d10 con 0x7fdacc156430 2024-09-10T22:08:11.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc168b90 con 0x7fdacc071230 2024-09-10T22:08:11.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad3200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdacc168e90 con 0x7fdacc160370 2024-09-10T22:08:11.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdacc156430 0x7fdacc0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:11.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdacc156430 0x7fdacc0ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56792/0 (socket says 172.21.15.66:56792) 2024-09-10T22:08:11.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdac3e00640 1 -- 172.21.15.66:0/4064245029 learned_addr learned my addr 172.21.15.66:0/4064245029 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:11.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdacbe00640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdacc071230 0x7fdacc150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:11.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad1200640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdacc160370 0x7fdacc0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:11.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdac3e00640 1 -- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdacc160370 msgr2=0x7fdacc0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:11.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdac3e00640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdacc160370 0x7fdacc0cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdac3e00640 1 -- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdacc071230 msgr2=0x7fdacc150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:11.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdac3e00640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdacc071230 0x7fdacc150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:11.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdac3e00640 1 -- 172.21.15.66:0/4064245029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdacc0b51c0 con 0x7fdacc156430 2024-09-10T22:08:11.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.854+0000 7fdad1200640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdacc160370 0x7fdacc0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:08:11.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.858+0000 7fdac3e00640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdacc156430 0x7fdacc0ccbf0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fdab400c400 tx=0x7fdab400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:11.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.858+0000 7fdac8c00640 1 -- 172.21.15.66:0/4064245029 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4017070 con 0x7fdacc156430 2024-09-10T22:08:11.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.858+0000 7fdac8c00640 1 -- 172.21.15.66:0/4064245029 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdab4003600 con 0x7fdacc156430 2024-09-10T22:08:11.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.858+0000 7fdad3200640 1 -- 172.21.15.66:0/4064245029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdacc0b54a0 con 0x7fdacc156430 2024-09-10T22:08:11.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.858+0000 7fdac8c00640 1 -- 172.21.15.66:0/4064245029 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4003a50 con 0x7fdacc156430 2024-09-10T22:08:11.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.858+0000 7fdad3200640 1 -- 172.21.15.66:0/4064245029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdacc0b5a60 con 0x7fdacc156430 2024-09-10T22:08:11.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.858+0000 7fdad3200640 1 -- 172.21.15.66:0/4064245029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdacc0cfe80 con 0x7fdacc156430 2024-09-10T22:08:11.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.858+0000 7fdac8c00640 1 -- 172.21.15.66:0/4064245029 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fdab4003bf0 con 0x7fdacc156430 2024-09-10T22:08:11.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.858+0000 7fdac8c00640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdaa80736c0 0x7fdaa8075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:11.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.858+0000 7fdac8c00640 1 -- 172.21.15.66:0/4064245029 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7fdab4091710 con 0x7fdacc156430 2024-09-10T22:08:11.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.858+0000 7fdacbe00640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdaa80736c0 0x7fdaa8075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:11.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.862+0000 7fdacbe00640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdaa80736c0 0x7fdaa8075b80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0af000 tx=0x7fdabc01b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:11.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:11.862+0000 7fdac8c00640 1 -- 172.21.15.66:0/4064245029 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdab405e500 con 0x7fdacc156430 2024-09-10T22:08:12.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.106+0000 7fdad3200640 1 -- 172.21.15.66:0/4064245029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7fdacc050ef0 con 0x7fdacc156430 2024-09-10T22:08:12.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdac8c00640 1 -- 172.21.15.66:0/4064245029 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v345) ==== 167+0+0 (secure 0 0 0) 0x7fdab405de00 con 0x7fdacc156430 2024-09-10T22:08:12.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdad3200640 1 -- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdaa80736c0 msgr2=0x7fdaa8075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:12.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdad3200640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdaa80736c0 0x7fdaa8075b80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0af000 tx=0x7fdabc01b070 comp rx=0 tx=0).stop 2024-09-10T22:08:12.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdad3200640 1 -- 172.21.15.66:0/4064245029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdacc156430 msgr2=0x7fdacc0ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:12.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdad3200640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdacc156430 0x7fdacc0ccbf0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fdab400c400 tx=0x7fdab400c8c0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdad3200640 1 -- 172.21.15.66:0/4064245029 shutdown_connections 2024-09-10T22:08:12.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdad3200640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdaa80736c0 0x7fdaa8075b80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdad3200640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdacc160370 0x7fdacc0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdad3200640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdacc156430 0x7fdacc0ccbf0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdad3200640 1 --2- 172.21.15.66:0/4064245029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdacc071230 0x7fdacc150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdad3200640 1 -- 172.21.15.66:0/4064245029 >> 172.21.15.66:0/4064245029 conn(0x7fdacc0d0b30 msgr2=0x7fdacc15b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:12.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdad3200640 1 -- 172.21.15.66:0/4064245029 shutdown_connections 2024-09-10T22:08:12.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.306+0000 7fdad3200640 1 -- 172.21.15.66:0/4064245029 wait complete. 2024-09-10T22:08:12.333 INFO:tasks.cephfs_test_runner:Test that 2 progress bars are printed in output of "ceph status" ... ok 2024-09-10T22:08:12.333 INFO:tasks.cephfs_test_runner:Starting test: test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress indicated by progress bar for ongoing+pending clones 2024-09-10T22:08:12.333 INFO:tasks.cephfs_test_runner:test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-10T22:08:12.334 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:08:12.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.490+0000 7ff83ca00640 1 -- 172.21.15.66:0/4285564109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8380aee20 msgr2=0x7ff8380cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:12.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.490+0000 7ff83ca00640 1 --2- 172.21.15.66:0/4285564109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8380aee20 0x7ff8380cd930 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff838071740 tx=0x7ff83001ccf0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.490+0000 7ff83ca00640 1 -- 172.21.15.66:0/4285564109 shutdown_connections 2024-09-10T22:08:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.490+0000 7ff83ca00640 1 --2- 172.21.15.66:0/4285564109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8380aee20 0x7ff8380cd930 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.490+0000 7ff83ca00640 1 --2- 172.21.15.66:0/4285564109 >> v2:172.21.15.94:3300/0 conn(0x7ff838159900 0x7ff8380ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.490+0000 7ff83ca00640 1 -- 172.21.15.66:0/4285564109 >> 172.21.15.66:0/4285564109 conn(0x7ff8380cfa30 msgr2=0x7ff8380cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.490+0000 7ff83ca00640 1 -- 172.21.15.66:0/4285564109 shutdown_connections 2024-09-10T22:08:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.490+0000 7ff83ca00640 1 -- 172.21.15.66:0/4285564109 wait complete. 2024-09-10T22:08:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff83ca00640 1 Processor -- start 2024-09-10T22:08:12.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff83ca00640 1 -- start start 2024-09-10T22:08:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff83ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8380aee20 0x7ff838160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff83ca00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff838159900 0x7ff83814ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff83ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff83815cc80 0x7ff8381f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff83ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff838168f70 con 0x7ff8380aee20 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff83ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff838168df0 con 0x7ff83815cc80 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff83ca00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff8381690f0 con 0x7ff838159900 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff835e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8380aee20 0x7ff838160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff835e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8380aee20 0x7ff838160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47252/0 (socket says 172.21.15.66:47252) 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff835e00640 1 -- 172.21.15.66:0/2746052508 learned_addr learned my addr 172.21.15.66:0/2746052508 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff836800640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff83815cc80 0x7ff8381f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff82fe00640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff838159900 0x7ff83814ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff835e00640 1 -- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff838159900 msgr2=0x7ff83814ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff835e00640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff838159900 0x7ff83814ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff835e00640 1 -- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff83815cc80 msgr2=0x7ff8381f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff835e00640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff83815cc80 0x7ff8381f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff835e00640 1 -- 172.21.15.66:0/2746052508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff838159170 con 0x7ff8380aee20 2024-09-10T22:08:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff835e00640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8380aee20 0x7ff838160120 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff82800e940 tx=0x7ff82800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff82e000640 1 -- 172.21.15.66:0/2746052508 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82800cd80 con 0x7ff8380aee20 2024-09-10T22:08:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff82e000640 1 -- 172.21.15.66:0/2746052508 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff828005020 con 0x7ff8380aee20 2024-09-10T22:08:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff83ca00640 1 -- 172.21.15.66:0/2746052508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83815f900 con 0x7ff8380aee20 2024-09-10T22:08:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff82e000640 1 -- 172.21.15.66:0/2746052508 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828005450 con 0x7ff8380aee20 2024-09-10T22:08:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.494+0000 7ff83ca00640 1 -- 172.21.15.66:0/2746052508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83815fec0 con 0x7ff8380aee20 2024-09-10T22:08:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.498+0000 7ff83ca00640 1 -- 172.21.15.66:0/2746052508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff838070470 con 0x7ff8380aee20 2024-09-10T22:08:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.498+0000 7ff82e000640 1 -- 172.21.15.66:0/2746052508 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7ff8280047b0 con 0x7ff8380aee20 2024-09-10T22:08:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.498+0000 7ff82e000640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8140736c0 0x7ff814075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.498+0000 7ff82e000640 1 -- 172.21.15.66:0/2746052508 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7ff828092f50 con 0x7ff8380aee20 2024-09-10T22:08:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.498+0000 7ff82fe00640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8140736c0 0x7ff814075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.502+0000 7ff82fe00640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8140736c0 0x7ff814075b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff8380c9ec0 tx=0x7ff820009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.502+0000 7ff82e000640 1 -- 172.21.15.66:0/2746052508 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff838159170 con 0x7ff8380aee20 2024-09-10T22:08:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.678+0000 7ff83ca00640 1 -- 172.21.15.66:0/2746052508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff838076650 con 0x7ff8380aee20 2024-09-10T22:08:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff82e000640 1 -- 172.21.15.66:0/2746052508 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+13049 (secure 0 0 0) 0x7ff82805f6f0 con 0x7ff8380aee20 2024-09-10T22:08:12.687 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:12.687 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":77,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:08:11.300970+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/2415586891":"2024-09-11T22:08:08.287780+0000","172.21.15.66:6833/2415586891":"2024-09-11T22:08:08.287780+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:08:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff83ca00640 1 -- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8140736c0 msgr2=0x7ff814075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff83ca00640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8140736c0 0x7ff814075b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff8380c9ec0 tx=0x7ff820009060 comp rx=0 tx=0).stop 2024-09-10T22:08:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff83ca00640 1 -- 172.21.15.66:0/2746052508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8380aee20 msgr2=0x7ff838160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff83ca00640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8380aee20 0x7ff838160120 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff82800e940 tx=0x7ff82800ee00 comp rx=0 tx=0).stop 2024-09-10T22:08:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff83ca00640 1 -- 172.21.15.66:0/2746052508 shutdown_connections 2024-09-10T22:08:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff83ca00640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8140736c0 0x7ff814075b80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff83ca00640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff83815cc80 0x7ff8381f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff83ca00640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff838159900 0x7ff83814ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff83ca00640 1 --2- 172.21.15.66:0/2746052508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8380aee20 0x7ff838160120 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff83ca00640 1 -- 172.21.15.66:0/2746052508 >> 172.21.15.66:0/2746052508 conn(0x7ff8380cfa30 msgr2=0x7ff83814f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff83ca00640 1 -- 172.21.15.66:0/2746052508 shutdown_connections 2024-09-10T22:08:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.682+0000 7ff83ca00640 1 -- 172.21.15.66:0/2746052508 wait complete. 2024-09-10T22:08:12.709 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:00:22.340318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:08:12.710 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:08:12.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.854+0000 7f4229e00640 1 -- 172.21.15.66:0/415198902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4208004810 msgr2=0x7f4208004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:12.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.854+0000 7f4229e00640 1 --2- 172.21.15.66:0/415198902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4208004810 0x7f4208004c90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f42240af2e0 tx=0x7f421001cf20 comp rx=0 tx=0).stop 2024-09-10T22:08:12.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.854+0000 7f4223400640 1 -- 172.21.15.66:0/415198902 reap_dead start 2024-09-10T22:08:12.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.854+0000 7f4229e00640 1 -- 172.21.15.66:0/415198902 shutdown_connections 2024-09-10T22:08:12.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.854+0000 7f4229e00640 1 -- 172.21.15.66:0/415198902 >> 172.21.15.66:0/415198902 conn(0x7f42240cfa30 msgr2=0x7f42240cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:12.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.854+0000 7f4229e00640 1 -- 172.21.15.66:0/415198902 shutdown_connections 2024-09-10T22:08:12.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.854+0000 7f4229e00640 1 -- 172.21.15.66:0/415198902 wait complete. 2024-09-10T22:08:12.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4229e00640 1 Processor -- start 2024-09-10T22:08:12.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4229e00640 1 -- start start 2024-09-10T22:08:12.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4229e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4208004810 0x7f422414e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:12.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4229e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f42240a9ec0 0x7f42240aa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:12.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4229e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42240725f0 0x7f4224072a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:12.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4229e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f422415a310 con 0x7f4208004810 2024-09-10T22:08:12.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4229e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f422406ed40 con 0x7f42240725f0 2024-09-10T22:08:12.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4229e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f422415a490 con 0x7f42240a9ec0 2024-09-10T22:08:12.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4222a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4208004810 0x7f422414e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:12.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4222a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4208004810 0x7f422414e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47272/0 (socket says 172.21.15.66:47272) 2024-09-10T22:08:12.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4222a00640 1 -- 172.21.15.66:0/1689327030 learned_addr learned my addr 172.21.15.66:0/1689327030 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:12.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4222000640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f42240a9ec0 0x7f42240aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:12.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4223400640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42240725f0 0x7f4224072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:12.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4222a00640 1 -- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f42240a9ec0 msgr2=0x7f42240aa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:12.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4222a00640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f42240a9ec0 0x7f42240aa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4222a00640 1 -- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42240725f0 msgr2=0x7f4224072a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:12.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4222a00640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42240725f0 0x7f4224072a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:12.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4222a00640 1 -- 172.21.15.66:0/1689327030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42240517d0 con 0x7f4208004810 2024-09-10T22:08:12.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4222a00640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4208004810 0x7f422414e4a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4214011b10 tx=0x7f4214009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:12.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f421b400640 1 -- 172.21.15.66:0/1689327030 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421401e070 con 0x7f4208004810 2024-09-10T22:08:12.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f421b400640 1 -- 172.21.15.66:0/1689327030 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f421400fe40 con 0x7f4208004810 2024-09-10T22:08:12.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4229e00640 1 -- 172.21.15.66:0/1689327030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4224073390 con 0x7f4208004810 2024-09-10T22:08:12.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f421b400640 1 -- 172.21.15.66:0/1689327030 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42140042b0 con 0x7f4208004810 2024-09-10T22:08:12.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.858+0000 7f4229e00640 1 -- 172.21.15.66:0/1689327030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4224073950 con 0x7f4208004810 2024-09-10T22:08:12.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.862+0000 7f4229e00640 1 -- 172.21.15.66:0/1689327030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42240d0550 con 0x7f4208004810 2024-09-10T22:08:12.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.862+0000 7f421b400640 1 -- 172.21.15.66:0/1689327030 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f421402b080 con 0x7f4208004810 2024-09-10T22:08:12.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.862+0000 7f421b400640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f42040736c0 0x7f4204075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:12.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.862+0000 7f421b400640 1 -- 172.21.15.66:0/1689327030 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7f4214093390 con 0x7f4208004810 2024-09-10T22:08:12.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.862+0000 7f4222000640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f42040736c0 0x7f4204075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:12.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.866+0000 7f4222000640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f42040736c0 0x7f4204075b80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f42100ac000 tx=0x7f4210019d60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:12.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:12.866+0000 7f421b400640 1 -- 172.21.15.66:0/1689327030 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f421405fbf0 con 0x7f4208004810 2024-09-10T22:08:13.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.038+0000 7f4229e00640 1 -- 172.21.15.66:0/1689327030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4224070420 con 0x7f4208004810 2024-09-10T22:08:13.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.038+0000 7f421b400640 1 -- 172.21.15.66:0/1689327030 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f4214064620 con 0x7f4208004810 2024-09-10T22:08:13.045 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-09-10T22:08:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.042+0000 7f4229e00640 1 -- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f42040736c0 msgr2=0x7f4204075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.042+0000 7f4229e00640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f42040736c0 0x7f4204075b80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f42100ac000 tx=0x7f4210019d60 comp rx=0 tx=0).stop 2024-09-10T22:08:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.042+0000 7f4229e00640 1 -- 172.21.15.66:0/1689327030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4208004810 msgr2=0x7f422414e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.042+0000 7f4229e00640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4208004810 0x7f422414e4a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4214011b10 tx=0x7f4214009ea0 comp rx=0 tx=0).stop 2024-09-10T22:08:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.042+0000 7f4229e00640 1 -- 172.21.15.66:0/1689327030 shutdown_connections 2024-09-10T22:08:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.042+0000 7f4229e00640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f42040736c0 0x7f4204075b80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.042+0000 7f4229e00640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42240725f0 0x7f4224072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:13.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.042+0000 7f4229e00640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f42240a9ec0 0x7f42240aa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:13.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.042+0000 7f4229e00640 1 --2- 172.21.15.66:0/1689327030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4208004810 0x7f422414e4a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:13.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.042+0000 7f4229e00640 1 -- 172.21.15.66:0/1689327030 >> 172.21.15.66:0/1689327030 conn(0x7f42240cfa30 msgr2=0x7f42240cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:13.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.042+0000 7f4229e00640 1 -- 172.21.15.66:0/1689327030 shutdown_connections 2024-09-10T22:08:13.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.042+0000 7f4229e00640 1 -- 172.21.15.66:0/1689327030 wait complete. 2024-09-10T22:08:13.066 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-10T22:08:13.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.206+0000 7f2bf2e00640 1 -- 172.21.15.66:0/812337535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bec159900 msgr2=0x7f2bec0ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:13.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.206+0000 7f2bf2e00640 1 --2- 172.21.15.66:0/812337535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bec159900 0x7f2bec0ca080 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009840 tx=0x7f2bdc01c790 comp rx=0 tx=0).stop 2024-09-10T22:08:13.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.206+0000 7f2bf2e00640 1 -- 172.21.15.66:0/812337535 shutdown_connections 2024-09-10T22:08:13.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.206+0000 7f2bf2e00640 1 --2- 172.21.15.66:0/812337535 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bec15cc80 0x7f2bec0ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:13.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.206+0000 7f2bf2e00640 1 --2- 172.21.15.66:0/812337535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bec159900 0x7f2bec0ca080 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:13.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.206+0000 7f2bf2e00640 1 -- 172.21.15.66:0/812337535 >> 172.21.15.66:0/812337535 conn(0x7f2bec0cfa30 msgr2=0x7f2bec0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:13.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bf2e00640 1 -- 172.21.15.66:0/812337535 shutdown_connections 2024-09-10T22:08:13.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bf2e00640 1 -- 172.21.15.66:0/812337535 wait complete. 2024-09-10T22:08:13.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bf2e00640 1 Processor -- start 2024-09-10T22:08:13.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bf2e00640 1 -- start start 2024-09-10T22:08:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bf2e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bec0aec50 0x7f2bec15b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bf2e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bec159900 0x7f2bec1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bf2e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bec15cc80 0x7f2bec073840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bf2e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec168f70 con 0x7f2bec0aec50 2024-09-10T22:08:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bf2e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec168df0 con 0x7f2bec15cc80 2024-09-10T22:08:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bf2e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2bec1690f0 con 0x7f2bec159900 2024-09-10T22:08:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bec0aec50 0x7f2bec15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bec0aec50 0x7f2bec15b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47290/0 (socket says 172.21.15.66:47290) 2024-09-10T22:08:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bebe00640 1 -- 172.21.15.66:0/3335333183 learned_addr learned my addr 172.21.15.66:0/3335333183 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2be3e00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bec159900 0x7f2bec1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bf0e00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bec15cc80 0x7f2bec073840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bebe00640 1 -- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bec159900 msgr2=0x7f2bec1517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bebe00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bec159900 0x7f2bec1517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bebe00640 1 -- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bec15cc80 msgr2=0x7f2bec073840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bebe00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bec15cc80 0x7f2bec073840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bebe00640 1 -- 172.21.15.66:0/3335333183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec15b5c0 con 0x7f2bec0aec50 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2be3e00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bec159900 0x7f2bec1517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.210+0000 7f2bebe00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bec0aec50 0x7f2bec15b080 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc01cc70 tx=0x7f2bdc005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.214+0000 7f2be8c00640 1 -- 172.21.15.66:0/3335333183 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0c1070 con 0x7f2bec0aec50 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.214+0000 7f2be8c00640 1 -- 172.21.15.66:0/3335333183 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bdc005040 con 0x7f2bec0aec50 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.214+0000 7f2be8c00640 1 -- 172.21.15.66:0/3335333183 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0ad050 con 0x7f2bec0aec50 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.214+0000 7f2bf2e00640 1 -- 172.21.15.66:0/3335333183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bec151d30 con 0x7f2bec0aec50 2024-09-10T22:08:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.214+0000 7f2bf2e00640 1 -- 172.21.15.66:0/3335333183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bec1528a0 con 0x7f2bec0aec50 2024-09-10T22:08:13.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.214+0000 7f2bf2e00640 1 -- 172.21.15.66:0/3335333183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bec070470 con 0x7f2bec0aec50 2024-09-10T22:08:13.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.214+0000 7f2be8c00640 1 -- 172.21.15.66:0/3335333183 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f2bdc005450 con 0x7f2bec0aec50 2024-09-10T22:08:13.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.214+0000 7f2be8c00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bc80736c0 0x7f2bc8075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:13.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.214+0000 7f2be8c00640 1 -- 172.21.15.66:0/3335333183 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7f2bdc12c7d0 con 0x7f2bec0aec50 2024-09-10T22:08:13.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.214+0000 7f2be3e00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bc80736c0 0x7f2bc8075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:13.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.218+0000 7f2be3e00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bc80736c0 0x7f2bc8075b80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0ca540 tx=0x7f2bd400b5b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:13.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.218+0000 7f2be8c00640 1 -- 172.21.15.66:0/3335333183 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2bdc016410 con 0x7f2bec0aec50 2024-09-10T22:08:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:13.462+0000 7f2bf2e00640 1 -- 172.21.15.66:0/3335333183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f2bec0ccc20 con 0x7f2bec0aec50 2024-09-10T22:08:14.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2be8c00640 1 -- 172.21.15.66:0/3335333183 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v347) ==== 177+0+0 (secure 0 0 0) 0x7f2bdc0f9900 con 0x7f2bec0aec50 2024-09-10T22:08:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2bf2e00640 1 -- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bc80736c0 msgr2=0x7f2bc8075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2bf2e00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bc80736c0 0x7f2bc8075b80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0ca540 tx=0x7f2bd400b5b0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2bf2e00640 1 -- 172.21.15.66:0/3335333183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bec0aec50 msgr2=0x7f2bec15b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2bf2e00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bec0aec50 0x7f2bec15b080 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc01cc70 tx=0x7f2bdc005d70 comp rx=0 tx=0).stop 2024-09-10T22:08:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2bf2e00640 1 -- 172.21.15.66:0/3335333183 shutdown_connections 2024-09-10T22:08:14.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2bf2e00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bc80736c0 0x7f2bc8075b80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2bf2e00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bec15cc80 0x7f2bec073840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2bf2e00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bec159900 0x7f2bec1517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2bf2e00640 1 --2- 172.21.15.66:0/3335333183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bec0aec50 0x7f2bec15b080 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2bf2e00640 1 -- 172.21.15.66:0/3335333183 >> 172.21.15.66:0/3335333183 conn(0x7f2bec0cfa30 msgr2=0x7f2bec0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:14.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2bf2e00640 1 -- 172.21.15.66:0/3335333183 shutdown_connections 2024-09-10T22:08:14.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.314+0000 7f2bf2e00640 1 -- 172.21.15.66:0/3335333183 wait complete. 2024-09-10T22:08:14.344 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T22:08:14.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.490+0000 7f358ea00640 1 -- 172.21.15.66:0/1361811149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358815f190 msgr2=0x7f35880b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:14.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.490+0000 7f358ea00640 1 --2- 172.21.15.66:0/1361811149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358815f190 0x7f35880b2f30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3588074c20 tx=0x7f357c01c740 comp rx=0 tx=0).stop 2024-09-10T22:08:14.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.490+0000 7f358ea00640 1 -- 172.21.15.66:0/1361811149 shutdown_connections 2024-09-10T22:08:14.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.490+0000 7f358ea00640 1 --2- 172.21.15.66:0/1361811149 >> v2:172.21.15.94:3300/0 conn(0x7f358815f570 0x7f35880cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.490+0000 7f358ea00640 1 --2- 172.21.15.66:0/1361811149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358815f190 0x7f35880b2f30 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.490+0000 7f358ea00640 1 -- 172.21.15.66:0/1361811149 >> 172.21.15.66:0/1361811149 conn(0x7f35880d1c50 msgr2=0x7f35880d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:14.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.490+0000 7f358ea00640 1 -- 172.21.15.66:0/1361811149 shutdown_connections 2024-09-10T22:08:14.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.490+0000 7f358ea00640 1 -- 172.21.15.66:0/1361811149 wait complete. 2024-09-10T22:08:14.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.490+0000 7f358ea00640 1 Processor -- start 2024-09-10T22:08:14.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358ea00640 1 -- start start 2024-09-10T22:08:14.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358ea00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35880cff70 0x7f3588148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:14.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358815f190 0x7f35881fb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:14.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358815f570 0x7f35881f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:14.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3588168da0 con 0x7f358815f570 2024-09-10T22:08:14.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358ea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3588168c20 con 0x7f358815f190 2024-09-10T22:08:14.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358ea00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3588168f20 con 0x7f35880cff70 2024-09-10T22:08:14.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358815f570 0x7f35881f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:14.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358815f570 0x7f35881f7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47310/0 (socket says 172.21.15.66:47310) 2024-09-10T22:08:14.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358e000640 1 -- 172.21.15.66:0/3693019367 learned_addr learned my addr 172.21.15.66:0/3693019367 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:14.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358cc00640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358815f190 0x7f35881fb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:14.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358d600640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35880cff70 0x7f3588148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:14.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358e000640 1 -- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35880cff70 msgr2=0x7f3588148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:14.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358e000640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35880cff70 0x7f3588148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358e000640 1 -- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358815f190 msgr2=0x7f35881fb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:14.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358e000640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358815f190 0x7f35881fb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358e000640 1 -- 172.21.15.66:0/3693019367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35881f8190 con 0x7f358815f570 2024-09-10T22:08:14.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358cc00640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358815f190 0x7f35881fb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:14.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358e000640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358815f570 0x7f35881f7850 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f358000c7a0 tx=0x7f358000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:14.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f3585600640 1 -- 172.21.15.66:0/3693019367 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580017070 con 0x7f358815f570 2024-09-10T22:08:14.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f3585600640 1 -- 172.21.15.66:0/3693019367 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3580005070 con 0x7f358815f570 2024-09-10T22:08:14.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f3585600640 1 -- 172.21.15.66:0/3693019367 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800054c0 con 0x7f358815f570 2024-09-10T22:08:14.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358ea00640 1 -- 172.21.15.66:0/3693019367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35881f8470 con 0x7f358815f570 2024-09-10T22:08:14.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358ea00640 1 -- 172.21.15.66:0/3693019367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35881faf40 con 0x7f358815f570 2024-09-10T22:08:14.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.494+0000 7f358ea00640 1 -- 172.21.15.66:0/3693019367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35880b2f30 con 0x7f358815f570 2024-09-10T22:08:14.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.498+0000 7f3585600640 1 -- 172.21.15.66:0/3693019367 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f3580005660 con 0x7f358815f570 2024-09-10T22:08:14.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.498+0000 7f3585600640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35680736c0 0x7f3568075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:14.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.498+0000 7f3585600640 1 -- 172.21.15.66:0/3693019367 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7f3580091a10 con 0x7f358815f570 2024-09-10T22:08:14.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.498+0000 7f358d600640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35680736c0 0x7f3568075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:14.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.498+0000 7f358d600640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35680736c0 0x7f3568075b80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f357c0af000 tx=0x7f357c01af80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:14.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.502+0000 7f3585600640 1 -- 172.21.15.66:0/3693019367 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f358005e1b0 con 0x7f358815f570 2024-09-10T22:08:14.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.674+0000 7f358ea00640 1 -- 172.21.15.66:0/3693019367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f35880cc210 con 0x7f358815f570 2024-09-10T22:08:14.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.674+0000 7f3585600640 1 -- 172.21.15.66:0/3693019367 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v40) ==== 59+0+1998 (secure 0 0 0) 0x7f3580062be0 con 0x7f358815f570 2024-09-10T22:08:14.681 INFO:teuthology.orchestra.run.smithi066.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T22:08:14.681 INFO:teuthology.orchestra.run.smithi066.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T22:08:14.681 INFO:teuthology.orchestra.run.smithi066.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T22:08:14.681 INFO:teuthology.orchestra.run.smithi066.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4673.907365 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4537.831230 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 5706.341617 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4484.013907 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3862.285837 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3776.439661 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3820.270368 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3863.602758 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds 20/20 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_ms 2/2 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_frag true 2024-09-10T22:08:14.682 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_scatterstat true 2024-09-10T22:08:14.683 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T22:08:14.683 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_verify_scatter true 2024-09-10T22:08:14.683 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T22:08:14.683 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T22:08:14.683 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T22:08:14.683 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced client_mount_timeout 600 2024-09-10T22:08:14.683 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_client 20/20 2024-09-10T22:08:14.683 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_ms 1/1 2024-09-10T22:08:14.683 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T22:08:14.683 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T22:08:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.678+0000 7f358ea00640 1 -- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35680736c0 msgr2=0x7f3568075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.678+0000 7f358ea00640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35680736c0 0x7f3568075b80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f357c0af000 tx=0x7f357c01af80 comp rx=0 tx=0).stop 2024-09-10T22:08:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.678+0000 7f358ea00640 1 -- 172.21.15.66:0/3693019367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358815f570 msgr2=0x7f35881f7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.678+0000 7f358ea00640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358815f570 0x7f35881f7850 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f358000c7a0 tx=0x7f358000cc60 comp rx=0 tx=0).stop 2024-09-10T22:08:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.678+0000 7f358ea00640 1 -- 172.21.15.66:0/3693019367 shutdown_connections 2024-09-10T22:08:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.678+0000 7f358ea00640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35680736c0 0x7f3568075b80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.678+0000 7f358ea00640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358815f570 0x7f35881f7850 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.678+0000 7f358ea00640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358815f190 0x7f35881fb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.678+0000 7f358ea00640 1 --2- 172.21.15.66:0/3693019367 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35880cff70 0x7f3588148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.678+0000 7f358ea00640 1 -- 172.21.15.66:0/3693019367 >> 172.21.15.66:0/3693019367 conn(0x7f35880d1c50 msgr2=0x7f35880d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.678+0000 7f358ea00640 1 -- 172.21.15.66:0/3693019367 shutdown_connections 2024-09-10T22:08:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.678+0000 7f358ea00640 1 -- 172.21.15.66:0/3693019367 wait complete. 2024-09-10T22:08:14.703 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:08:14.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.846+0000 7fb6d9000640 1 -- 172.21.15.66:0/1088772886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d40afc20 msgr2=0x7fb6d40ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:14.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.846+0000 7fb6d9000640 1 --2- 172.21.15.66:0/1088772886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d40afc20 0x7fb6d40ce880 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009bc0 tx=0x7fb6cc01cb10 comp rx=0 tx=0).stop 2024-09-10T22:08:14.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.846+0000 7fb6d9000640 1 -- 172.21.15.66:0/1088772886 shutdown_connections 2024-09-10T22:08:14.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.846+0000 7fb6d9000640 1 --2- 172.21.15.66:0/1088772886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d40afc20 0x7fb6d40ce880 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.846+0000 7fb6d9000640 1 -- 172.21.15.66:0/1088772886 >> 172.21.15.66:0/1088772886 conn(0x7fb6d40d1c60 msgr2=0x7fb6d40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:14.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.846+0000 7fb6d9000640 1 -- 172.21.15.66:0/1088772886 shutdown_connections 2024-09-10T22:08:14.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.846+0000 7fb6d9000640 1 -- 172.21.15.66:0/1088772886 wait complete. 2024-09-10T22:08:14.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.846+0000 7fb6d9000640 1 Processor -- start 2024-09-10T22:08:14.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d9000640 1 -- start start 2024-09-10T22:08:14.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d9000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6d40afc20 0x7fb6d415f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:14.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d40d06b0 0x7fb6d415a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:14.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d415bd40 0x7fb6d415d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:14.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d9000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d416a1b0 con 0x7fb6d415bd40 2024-09-10T22:08:14.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d9000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d416a030 con 0x7fb6d40d06b0 2024-09-10T22:08:14.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d9000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d416a330 con 0x7fb6d40afc20 2024-09-10T22:08:14.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d415bd40 0x7fb6d415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:14.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d415bd40 0x7fb6d415d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47318/0 (socket says 172.21.15.66:47318) 2024-09-10T22:08:14.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d2800640 1 -- 172.21.15.66:0/3318333775 learned_addr learned my addr 172.21.15.66:0/3318333775 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:14.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6cbe00640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d40d06b0 0x7fb6d415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:14.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d1e00640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6d40afc20 0x7fb6d415f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:14.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d2800640 1 -- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6d40afc20 msgr2=0x7fb6d415f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:14.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d2800640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6d40afc20 0x7fb6d415f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d2800640 1 -- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d40d06b0 msgr2=0x7fb6d415a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:14.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d2800640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d40d06b0 0x7fb6d415a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:14.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d2800640 1 -- 172.21.15.66:0/3318333775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d40caea0 con 0x7fb6d415bd40 2024-09-10T22:08:14.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d2800640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d415bd40 0x7fb6d415d050 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc019720 tx=0x7fb6cc0a5a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:14.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6ca000640 1 -- 172.21.15.66:0/3318333775 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0bb070 con 0x7fb6d415bd40 2024-09-10T22:08:14.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6ca000640 1 -- 172.21.15.66:0/3318333775 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6cc004e00 con 0x7fb6d415bd40 2024-09-10T22:08:14.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d9000640 1 -- 172.21.15.66:0/3318333775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d40cb180 con 0x7fb6d415bd40 2024-09-10T22:08:14.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6ca000640 1 -- 172.21.15.66:0/3318333775 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0b2040 con 0x7fb6d415bd40 2024-09-10T22:08:14.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d9000640 1 -- 172.21.15.66:0/3318333775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d40cb740 con 0x7fb6d415bd40 2024-09-10T22:08:14.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.850+0000 7fb6d9000640 1 -- 172.21.15.66:0/3318333775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d40cffa0 con 0x7fb6d415bd40 2024-09-10T22:08:14.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.854+0000 7fb6ca000640 1 -- 172.21.15.66:0/3318333775 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fb6cc0b5030 con 0x7fb6d415bd40 2024-09-10T22:08:14.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.854+0000 7fb6ca000640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6b00736c0 0x7fb6b0075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:14.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.854+0000 7fb6ca000640 1 -- 172.21.15.66:0/3318333775 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7fb6cc12b990 con 0x7fb6d415bd40 2024-09-10T22:08:14.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.854+0000 7fb6d1e00640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6b00736c0 0x7fb6b0075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:14.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.854+0000 7fb6d1e00640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6b00736c0 0x7fb6b0075b80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40b2320 tx=0x7fb6c4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:14.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:14.858+0000 7fb6ca000640 1 -- 172.21.15.66:0/3318333775 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb6cc0f1de0 con 0x7fb6d415bd40 2024-09-10T22:08:15.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.030+0000 7fb6d9000640 1 -- 172.21.15.66:0/3318333775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb6d40b0df0 con 0x7fb6d415bd40 2024-09-10T22:08:15.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.034+0000 7fb6ca000640 1 -- 172.21.15.66:0/3318333775 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7fb6cc016790 con 0x7fb6d415bd40 2024-09-10T22:08:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.038+0000 7fb6d9000640 1 -- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6b00736c0 msgr2=0x7fb6b0075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.038+0000 7fb6d9000640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6b00736c0 0x7fb6b0075b80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40b2320 tx=0x7fb6c4009040 comp rx=0 tx=0).stop 2024-09-10T22:08:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.038+0000 7fb6d9000640 1 -- 172.21.15.66:0/3318333775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d415bd40 msgr2=0x7fb6d415d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.038+0000 7fb6d9000640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d415bd40 0x7fb6d415d050 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc019720 tx=0x7fb6cc0a5a70 comp rx=0 tx=0).stop 2024-09-10T22:08:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.038+0000 7fb6d9000640 1 -- 172.21.15.66:0/3318333775 shutdown_connections 2024-09-10T22:08:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.038+0000 7fb6d9000640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6b00736c0 0x7fb6b0075b80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.038+0000 7fb6d9000640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d415bd40 0x7fb6d415d050 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.038+0000 7fb6d9000640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d40d06b0 0x7fb6d415a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.038+0000 7fb6d9000640 1 --2- 172.21.15.66:0/3318333775 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6d40afc20 0x7fb6d415f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.038+0000 7fb6d9000640 1 -- 172.21.15.66:0/3318333775 >> 172.21.15.66:0/3318333775 conn(0x7fb6d40d1c60 msgr2=0x7fb6d406f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.038+0000 7fb6d9000640 1 -- 172.21.15.66:0/3318333775 shutdown_connections 2024-09-10T22:08:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.038+0000 7fb6d9000640 1 -- 172.21.15.66:0/3318333775 wait complete. 2024-09-10T22:08:15.062 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:08:15.062 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:08:15.144 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:08:15.245 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:08:15.246 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:08:15.246 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:08:15.255 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:08:15.255 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:08:15.255 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:08:15.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:08:15.265 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:08:15.265 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:08:15.309 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:08:15.309 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:08:15.309 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:08:15.319 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:08:15.319 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T22:08:15.319 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:08:15.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.494+0000 7f9f67a00640 1 -- 172.21.15.66:0/2344568979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f600d0550 msgr2=0x7f9f600ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:15.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.494+0000 7f9f67a00640 1 --2- 172.21.15.66:0/2344568979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f600d0550 0x7f9f600ca1a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9f600af550 tx=0x7f9f4c01c830 comp rx=0 tx=0).stop 2024-09-10T22:08:15.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.494+0000 7f9f67a00640 1 -- 172.21.15.66:0/2344568979 shutdown_connections 2024-09-10T22:08:15.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.494+0000 7f9f67a00640 1 --2- 172.21.15.66:0/2344568979 >> v2:172.21.15.94:3300/0 conn(0x7f9f600c4fc0 0x7f9f600ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.494+0000 7f9f67a00640 1 --2- 172.21.15.66:0/2344568979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f600d0550 0x7f9f600ca1a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.494+0000 7f9f67a00640 1 -- 172.21.15.66:0/2344568979 >> 172.21.15.66:0/2344568979 conn(0x7f9f600cfa30 msgr2=0x7f9f600cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:15.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.494+0000 7f9f67a00640 1 -- 172.21.15.66:0/2344568979 shutdown_connections 2024-09-10T22:08:15.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.494+0000 7f9f67a00640 1 -- 172.21.15.66:0/2344568979 wait complete. 2024-09-10T22:08:15.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.494+0000 7f9f67a00640 1 Processor -- start 2024-09-10T22:08:15.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.494+0000 7f9f67a00640 1 -- start start 2024-09-10T22:08:15.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f67a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f600c4fc0 0x7f9f60140550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:15.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f67a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f6015a310 0x7f9f60156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:15.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f67a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f60150620 0x7f9f600ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:15.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f67a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9f60167480 con 0x7f9f60150620 2024-09-10T22:08:15.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f67a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f60167300 con 0x7f9f600c4fc0 2024-09-10T22:08:15.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f67a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9f60167600 con 0x7f9f6015a310 2024-09-10T22:08:15.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f65a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f60150620 0x7f9f600ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:15.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f65a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f60150620 0x7f9f600ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47334/0 (socket says 172.21.15.66:47334) 2024-09-10T22:08:15.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f65a00640 1 -- 172.21.15.66:0/3642325415 learned_addr learned my addr 172.21.15.66:0/3642325415 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:15.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f65000640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f600c4fc0 0x7f9f60140550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:15.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f5fe00640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f6015a310 0x7f9f60156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:15.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f65a00640 1 -- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f6015a310 msgr2=0x7f9f60156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:15.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f65a00640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f6015a310 0x7f9f60156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f65a00640 1 -- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f600c4fc0 msgr2=0x7f9f60140550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:15.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f65a00640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f600c4fc0 0x7f9f60140550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f65a00640 1 -- 172.21.15.66:0/3642325415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f601557b0 con 0x7f9f60150620 2024-09-10T22:08:15.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f65a00640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f60150620 0x7f9f600ad8c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800dd80 tx=0x7f9f5800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:15.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f5cc00640 1 -- 172.21.15.66:0/3642325415 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58004d30 con 0x7f9f60150620 2024-09-10T22:08:15.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f5cc00640 1 -- 172.21.15.66:0/3642325415 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f5800be40 con 0x7f9f60150620 2024-09-10T22:08:15.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f67a00640 1 -- 172.21.15.66:0/3642325415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f60155a90 con 0x7f9f60150620 2024-09-10T22:08:15.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f5cc00640 1 -- 172.21.15.66:0/3642325415 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800e8c0 con 0x7f9f60150620 2024-09-10T22:08:15.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f67a00640 1 -- 172.21.15.66:0/3642325415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f60156050 con 0x7f9f60150620 2024-09-10T22:08:15.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.498+0000 7f9f67a00640 1 -- 172.21.15.66:0/3642325415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f60070420 con 0x7f9f60150620 2024-09-10T22:08:15.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.502+0000 7f9f5cc00640 1 -- 172.21.15.66:0/3642325415 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f9f5800b040 con 0x7f9f60150620 2024-09-10T22:08:15.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.502+0000 7f9f5cc00640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9f400736f0 0x7f9f40075bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:15.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.502+0000 7f9f5cc00640 1 -- 172.21.15.66:0/3642325415 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7f9f580928c0 con 0x7f9f60150620 2024-09-10T22:08:15.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.502+0000 7f9f65000640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9f400736f0 0x7f9f40075bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:15.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.502+0000 7f9f65000640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9f400736f0 0x7f9f40075bb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9f54007fd0 tx=0x7f9f54009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:15.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.506+0000 7f9f5cc00640 1 -- 172.21.15.66:0/3642325415 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9f5805f060 con 0x7f9f60150620 2024-09-10T22:08:15.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.714+0000 7f9f67a00640 1 -- 172.21.15.66:0/3642325415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f600c4c40 con 0x7f9f60150620 2024-09-10T22:08:15.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.714+0000 7f9f5cc00640 1 -- 172.21.15.66:0/3642325415 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+3908 (secure 0 0 0) 0x7f9f58063a90 con 0x7f9f60150620 2024-09-10T22:08:15.721 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:15.721 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":49,"btime":"2024-09-10T22:08:09:290253+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6112,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3780556367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3780556367},{"type":"v1","addr":"172.21.15.94:6836","nonce":3780556367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1892098851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1892098851},{"type":"v1","addr":"172.21.15.94:6837","nonce":1892098851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1548997357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1548997357},{"type":"v1","addr":"172.21.15.66:6834","nonce":1548997357}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6888,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/571940318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":571940318},{"type":"v1","addr":"172.21.15.66:6835","nonce":571940318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[]} 2024-09-10T22:08:15.722 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 49 2024-09-10T22:08:15.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.718+0000 7f9f67a00640 1 -- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9f400736f0 msgr2=0x7f9f40075bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:15.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.718+0000 7f9f67a00640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9f400736f0 0x7f9f40075bb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9f54007fd0 tx=0x7f9f54009040 comp rx=0 tx=0).stop 2024-09-10T22:08:15.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.718+0000 7f9f67a00640 1 -- 172.21.15.66:0/3642325415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f60150620 msgr2=0x7f9f600ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:15.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.718+0000 7f9f67a00640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f60150620 0x7f9f600ad8c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800dd80 tx=0x7f9f5800b730 comp rx=0 tx=0).stop 2024-09-10T22:08:15.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.718+0000 7f9f67a00640 1 -- 172.21.15.66:0/3642325415 shutdown_connections 2024-09-10T22:08:15.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.718+0000 7f9f67a00640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9f400736f0 0x7f9f40075bb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.718+0000 7f9f67a00640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f60150620 0x7f9f600ad8c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.718+0000 7f9f67a00640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f6015a310 0x7f9f60156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.718+0000 7f9f67a00640 1 --2- 172.21.15.66:0/3642325415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f600c4fc0 0x7f9f60140550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.718+0000 7f9f67a00640 1 -- 172.21.15.66:0/3642325415 >> 172.21.15.66:0/3642325415 conn(0x7f9f600cfa30 msgr2=0x7f9f60073120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:15.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.718+0000 7f9f67a00640 1 -- 172.21.15.66:0/3642325415 shutdown_connections 2024-09-10T22:08:15.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.718+0000 7f9f67a00640 1 -- 172.21.15.66:0/3642325415 wait complete. 2024-09-10T22:08:15.744 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:08:15.744 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:08:15.744 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:08:15.744 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:08:15.744 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:08:15.745 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:08:15.745 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:08:15.745 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:08:15.745 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:08:15.745 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:08:15.745 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:08:15.745 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:08:15.746 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:08:15.746 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:08:15.746 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:08:15.746 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:08:15.759 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:08:15.759 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:08:15.760 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:08:15.760 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:08:15.761 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:08:15.761 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:08:15.762 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:08:15.762 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:08:15.804 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:08:15.804 DEBUG:teuthology.parallel:result is None 2024-09-10T22:08:15.805 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:08:15.805 DEBUG:teuthology.parallel:result is None 2024-09-10T22:08:15.806 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:08:15.806 DEBUG:teuthology.parallel:result is None 2024-09-10T22:08:15.807 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:08:15.807 DEBUG:teuthology.parallel:result is None 2024-09-10T22:08:15.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T22:08:15.892 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:08:15.892 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:08:15.895 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:08:15.895 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:08:15.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.954+0000 7fe192800640 1 -- 172.21.15.66:0/34257536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c15a310 msgr2=0x7fe18c0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:15.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.954+0000 7fe192800640 1 --2- 172.21.15.66:0/34257536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c15a310 0x7fe18c0c48b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0af550 tx=0x7fe17c01c830 comp rx=0 tx=0).stop 2024-09-10T22:08:15.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.954+0000 7fe192800640 1 -- 172.21.15.66:0/34257536 shutdown_connections 2024-09-10T22:08:15.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.954+0000 7fe192800640 1 --2- 172.21.15.66:0/34257536 >> v2:172.21.15.94:3300/0 conn(0x7fe18c0aeec0 0x7fe18c0c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.954+0000 7fe192800640 1 --2- 172.21.15.66:0/34257536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c15a310 0x7fe18c0c48b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.954+0000 7fe192800640 1 -- 172.21.15.66:0/34257536 >> 172.21.15.66:0/34257536 conn(0x7fe18c0cfa30 msgr2=0x7fe18c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:15.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.954+0000 7fe192800640 1 -- 172.21.15.66:0/34257536 shutdown_connections 2024-09-10T22:08:15.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.954+0000 7fe192800640 1 -- 172.21.15.66:0/34257536 wait complete. 2024-09-10T22:08:15.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.954+0000 7fe192800640 1 Processor -- start 2024-09-10T22:08:15.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.954+0000 7fe192800640 1 -- start start 2024-09-10T22:08:15.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe192800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe18c0aeec0 0x7fe18c0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:15.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe192800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe18c15a310 0x7fe18c144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:15.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe192800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c0afdb0 0x7fe18c144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe192800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1679c0 con 0x7fe18c0afdb0 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe192800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c167840 con 0x7fe18c0aeec0 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe192800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe18c167b40 con 0x7fe18c15a310 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe18be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c0afdb0 0x7fe18c144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe18b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe18c0aeec0 0x7fe18c0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe18be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c0afdb0 0x7fe18c144640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47386/0 (socket says 172.21.15.66:47386) 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe18be00640 1 -- 172.21.15.66:0/3707501786 learned_addr learned my addr 172.21.15.66:0/3707501786 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe182a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe18c15a310 0x7fe18c144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe18be00640 1 -- 172.21.15.66:0/3707501786 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe18c15a310 msgr2=0x7fe18c144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe18be00640 1 --2- 172.21.15.66:0/3707501786 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe18c15a310 0x7fe18c144100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe18be00640 1 -- 172.21.15.66:0/3707501786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe18c0aeec0 msgr2=0x7fe18c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe18be00640 1 --2- 172.21.15.66:0/3707501786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe18c0aeec0 0x7fe18c0adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe18be00640 1 -- 172.21.15.66:0/3707501786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c1496e0 con 0x7fe18c0afdb0 2024-09-10T22:08:15.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe18be00640 1 --2- 172.21.15.66:0/3707501786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c0afdb0 0x7fe18c144640 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe18400b970 tx=0x7fe18400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:15.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe183e00640 1 -- 172.21.15.66:0/3707501786 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184019070 con 0x7fe18c0afdb0 2024-09-10T22:08:15.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe183e00640 1 -- 172.21.15.66:0/3707501786 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe18400cc50 con 0x7fe18c0afdb0 2024-09-10T22:08:15.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe183e00640 1 -- 172.21.15.66:0/3707501786 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840146a0 con 0x7fe18c0afdb0 2024-09-10T22:08:15.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe192800640 1 -- 172.21.15.66:0/3707501786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c1499c0 con 0x7fe18c0afdb0 2024-09-10T22:08:15.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe192800640 1 -- 172.21.15.66:0/3707501786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18c149f80 con 0x7fe18c0afdb0 2024-09-10T22:08:15.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.958+0000 7fe192800640 1 -- 172.21.15.66:0/3707501786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18c15df60 con 0x7fe18c0afdb0 2024-09-10T22:08:15.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.962+0000 7fe183e00640 1 -- 172.21.15.66:0/3707501786 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fe18401d3b0 con 0x7fe18c0afdb0 2024-09-10T22:08:15.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.962+0000 7fe183e00640 1 --2- 172.21.15.66:0/3707501786 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe1680736c0 0x7fe168075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:15.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.962+0000 7fe183e00640 1 -- 172.21.15.66:0/3707501786 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4999+0+0 (secure 0 0 0) 0x7fe1840920c0 con 0x7fe18c0afdb0 2024-09-10T22:08:15.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.962+0000 7fe18b400640 1 --2- 172.21.15.66:0/3707501786 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe1680736c0 0x7fe168075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:15.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.962+0000 7fe18b400640 1 --2- 172.21.15.66:0/3707501786 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe1680736c0 0x7fe168075b80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0af000 tx=0x7fe17c01b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:15.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:15.966+0000 7fe183e00640 1 -- 172.21.15.66:0/3707501786 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe18405e860 con 0x7fe18c0afdb0 2024-09-10T22:08:16.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.154+0000 7fe192800640 1 -- 172.21.15.66:0/3707501786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe18c14f9e0 con 0x7fe18c0afdb0 2024-09-10T22:08:16.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.338+0000 7fe183e00640 1 -- 172.21.15.66:0/3707501786 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) ==== 97+0+0 (secure 0 0 0) 0x7fe184063290 con 0x7fe18c0afdb0 2024-09-10T22:08:16.344 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-09-10T22:08:16.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.342+0000 7fe192800640 1 -- 172.21.15.66:0/3707501786 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe1680736c0 msgr2=0x7fe168075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:16.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.342+0000 7fe192800640 1 --2- 172.21.15.66:0/3707501786 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe1680736c0 0x7fe168075b80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0af000 tx=0x7fe17c01b070 comp rx=0 tx=0).stop 2024-09-10T22:08:16.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.342+0000 7fe192800640 1 -- 172.21.15.66:0/3707501786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c0afdb0 msgr2=0x7fe18c144640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:16.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.342+0000 7fe192800640 1 --2- 172.21.15.66:0/3707501786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c0afdb0 0x7fe18c144640 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe18400b970 tx=0x7fe18400be30 comp rx=0 tx=0).stop 2024-09-10T22:08:16.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.342+0000 7fe192800640 1 -- 172.21.15.66:0/3707501786 shutdown_connections 2024-09-10T22:08:16.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.342+0000 7fe192800640 1 --2- 172.21.15.66:0/3707501786 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe1680736c0 0x7fe168075b80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.342+0000 7fe192800640 1 --2- 172.21.15.66:0/3707501786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c0afdb0 0x7fe18c144640 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.342+0000 7fe192800640 1 --2- 172.21.15.66:0/3707501786 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe18c15a310 0x7fe18c144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.342+0000 7fe192800640 1 --2- 172.21.15.66:0/3707501786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe18c0aeec0 0x7fe18c0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.342+0000 7fe192800640 1 -- 172.21.15.66:0/3707501786 >> 172.21.15.66:0/3707501786 conn(0x7fe18c0cfa30 msgr2=0x7fe18c0c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:16.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.342+0000 7fe192800640 1 -- 172.21.15.66:0/3707501786 shutdown_connections 2024-09-10T22:08:16.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.342+0000 7fe192800640 1 -- 172.21.15.66:0/3707501786 wait complete. 2024-09-10T22:08:16.370 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T22:08:16.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 -- 172.21.15.66:0/2648866678 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19f015f340 msgr2=0x7f19f00cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:16.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 --2- 172.21.15.66:0/2648866678 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19f015f340 0x7f19f00cfb50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009840 tx=0x7f19e001c820 comp rx=0 tx=0).stop 2024-09-10T22:08:16.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 -- 172.21.15.66:0/2648866678 shutdown_connections 2024-09-10T22:08:16.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 --2- 172.21.15.66:0/2648866678 >> v2:172.21.15.94:3300/0 conn(0x7f19f00b2bf0 0x7f19f00d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 --2- 172.21.15.66:0/2648866678 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19f015f340 0x7f19f00cfb50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 -- 172.21.15.66:0/2648866678 >> 172.21.15.66:0/2648866678 conn(0x7f19f00d1c50 msgr2=0x7f19f00d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:16.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 -- 172.21.15.66:0/2648866678 shutdown_connections 2024-09-10T22:08:16.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 -- 172.21.15.66:0/2648866678 wait complete. 2024-09-10T22:08:16.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 Processor -- start 2024-09-10T22:08:16.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 -- start start 2024-09-10T22:08:16.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f00b2bf0 0x7f19f0165c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f00d2770 0x7f19f01f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19f015f340 0x7f19f006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0169190 con 0x7f19f00d2770 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0169010 con 0x7f19f00b2bf0 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f6e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f19f0169310 con 0x7f19f015f340 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f00d2770 0x7f19f01f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f00d2770 0x7f19f01f8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47392/0 (socket says 172.21.15.66:47392) 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19e7e00640 1 -- 172.21.15.66:0/3115505696 learned_addr learned my addr 172.21.15.66:0/3115505696 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19efe00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f00b2bf0 0x7f19f0165c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f4e00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19f015f340 0x7f19f006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19e7e00640 1 -- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19f015f340 msgr2=0x7f19f006f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19e7e00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19f015f340 0x7f19f006f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19e7e00640 1 -- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f00b2bf0 msgr2=0x7f19f0165c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19e7e00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f00b2bf0 0x7f19f0165c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19e7e00640 1 -- 172.21.15.66:0/3115505696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f0166250 con 0x7f19f00d2770 2024-09-10T22:08:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.514+0000 7f19f4e00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19f015f340 0x7f19f006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.518+0000 7f19e7e00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f00d2770 0x7f19f01f8b50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f19d800e9c0 tx=0x7f19d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.518+0000 7f19ecc00640 1 -- 172.21.15.66:0/3115505696 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d800cde0 con 0x7f19f00d2770 2024-09-10T22:08:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.518+0000 7f19ecc00640 1 -- 172.21.15.66:0/3115505696 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d8010520 con 0x7f19f00d2770 2024-09-10T22:08:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.518+0000 7f19f6e00640 1 -- 172.21.15.66:0/3115505696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f0166530 con 0x7f19f00d2770 2024-09-10T22:08:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.518+0000 7f19ecc00640 1 -- 172.21.15.66:0/3115505696 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8010970 con 0x7f19f00d2770 2024-09-10T22:08:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.518+0000 7f19f6e00640 1 -- 172.21.15.66:0/3115505696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f006fcf0 con 0x7f19f00d2770 2024-09-10T22:08:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.518+0000 7f19f6e00640 1 -- 172.21.15.66:0/3115505696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f015fac0 con 0x7f19f00d2770 2024-09-10T22:08:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.518+0000 7f19ecc00640 1 -- 172.21.15.66:0/3115505696 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f19d8010b10 con 0x7f19f00d2770 2024-09-10T22:08:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.518+0000 7f19ecc00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19cc0736c0 0x7f19cc075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.522+0000 7f19ecc00640 1 -- 172.21.15.66:0/3115505696 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4913+0+0 (secure 0 0 0) 0x7f19d80929d0 con 0x7f19f00d2770 2024-09-10T22:08:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.522+0000 7f19efe00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19cc0736c0 0x7f19cc075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.522+0000 7f19efe00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19cc0736c0 0x7f19cc075b80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f19e00af000 tx=0x7f19e001b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:16.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.526+0000 7f19ecc00640 1 -- 172.21.15.66:0/3115505696 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f19d805fa80 con 0x7f19f00d2770 2024-09-10T22:08:16.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.762+0000 7f19f6e00640 1 -- 172.21.15.66:0/3115505696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f19f015c3e0 con 0x7f19f00d2770 2024-09-10T22:08:16.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.762+0000 7f19ecc00640 1 -- 172.21.15.66:0/3115505696 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v29) ==== 80+0+5304 (secure 0 0 0) 0x7f19d805f380 con 0x7f19f00d2770 2024-09-10T22:08:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:16.769 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:08:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-09-10T22:08:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-09-10T22:08:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmxQD7CBAAttZP0KtdjTI7QWbD7CVinQ==", 2024-09-10T22:08:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:08:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:08:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:08:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-09-10T22:08:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmI7ufFRAANelhvqylKyfGPVfBuY40kg==", 2024-09-10T22:08:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:08:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:08:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:08:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-09-10T22:08:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQnVoEBAAvsZ/TSOOP/JgJBr/aHwI/g==", 2024-09-10T22:08:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:08:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:08:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:08:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-09-10T22:08:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmRQkTHRAAHPIdwqtkkyh31D7IO2vxPA==", 2024-09-10T22:08:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:08:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:08:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBCweBmDJkxJBAAVSQNiMAQl5bR3RUwVOuGYw==", 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBFweBmwcdsOxAAPPQyzktcfiF8pJNP8uAjwg==", 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBJweBmXQhZExAAnEf8mjGX6nnYwWygKjo/2A==", 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:08:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBNweBmlauHAxAA/k9xRqbHZsqJq9OYbJKfNQ==", 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-09-10T22:08:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBRweBmHnFDJRAAgF1ZAPvmoNzeUftZQyL0eg==", 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBUweBm/hXpORAA6jpOI/I+QyB2mZsEoAWF0w==", 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBYweBmYu2MExAAPD9WcxrBLzwwJQm19z7YcQ==", 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbweBmXfJ9KRAA/AbWlli+Qfg7gy0SblJ7fg==", 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw==", 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmbS6IKhAAaEmK9irxMipALgkAct/+OQ==", 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQ47KJhAAI50ZaSQIhH6FxRM48JZICA==", 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:08:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmUcLCMBAAdf4tfcpVaZoaKjS1UzrxhA==", 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA+weBmPIAzKBAAkyVV875PNhBmEoKsrAkg8w==", 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmLiy1BBAADnraML8IUet7Sw0j83TUVQ==", 2024-09-10T22:08:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmItWpABAAlJ+6dFjvtBV3g+jYmUh0tQ==", 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:08:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.783 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:16.783 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T22:08:16.783 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:08:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.766+0000 7f19f6e00640 1 -- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19cc0736c0 msgr2=0x7f19cc075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.766+0000 7f19f6e00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19cc0736c0 0x7f19cc075b80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f19e00af000 tx=0x7f19e001b060 comp rx=0 tx=0).stop 2024-09-10T22:08:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.766+0000 7f19f6e00640 1 -- 172.21.15.66:0/3115505696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f00d2770 msgr2=0x7f19f01f8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.766+0000 7f19f6e00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f00d2770 0x7f19f01f8b50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f19d800e9c0 tx=0x7f19d800ee80 comp rx=0 tx=0).stop 2024-09-10T22:08:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.766+0000 7f19f6e00640 1 -- 172.21.15.66:0/3115505696 shutdown_connections 2024-09-10T22:08:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.766+0000 7f19f6e00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19cc0736c0 0x7f19cc075b80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.766+0000 7f19f6e00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19f015f340 0x7f19f006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.766+0000 7f19f6e00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f00d2770 0x7f19f01f8b50 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.766+0000 7f19f6e00640 1 --2- 172.21.15.66:0/3115505696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f00b2bf0 0x7f19f0165c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.766+0000 7f19f6e00640 1 -- 172.21.15.66:0/3115505696 >> 172.21.15.66:0/3115505696 conn(0x7f19f00d1c50 msgr2=0x7f19f0072120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.766+0000 7f19f6e00640 1 -- 172.21.15.66:0/3115505696 shutdown_connections 2024-09-10T22:08:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.766+0000 7f19f6e00640 1 -- 172.21.15.66:0/3115505696 wait complete. 2024-09-10T22:08:16.792 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:08:16.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 -- 172.21.15.66:0/1851395626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2480c4fc0 msgr2=0x7fc2480ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:16.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 --2- 172.21.15.66:0/1851395626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2480c4fc0 0x7fc2480ca6e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc24000ad90 tx=0x7fc24001dc20 comp rx=0 tx=0).stop 2024-09-10T22:08:16.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 -- 172.21.15.66:0/1851395626 shutdown_connections 2024-09-10T22:08:16.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 --2- 172.21.15.66:0/1851395626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2480c4fc0 0x7fc2480ca6e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 --2- 172.21.15.66:0/1851395626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2480d0550 0x7fc2480ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 -- 172.21.15.66:0/1851395626 >> 172.21.15.66:0/1851395626 conn(0x7fc2480cfa30 msgr2=0x7fc2480cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:16.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 -- 172.21.15.66:0/1851395626 shutdown_connections 2024-09-10T22:08:16.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 -- 172.21.15.66:0/1851395626 wait complete. 2024-09-10T22:08:16.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 Processor -- start 2024-09-10T22:08:16.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 -- start start 2024-09-10T22:08:16.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2480c4fc0 0x7fc248147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:16.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc24815a310 0x7fc248152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:16.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2481f6710 0x7fc24814ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:16.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc248167480 con 0x7fc24815a310 2024-09-10T22:08:16.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc248167300 con 0x7fc2480c4fc0 2024-09-10T22:08:16.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24f000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc248167600 con 0x7fc2481f6710 2024-09-10T22:08:16.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc23fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc24815a310 0x7fc248152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:16.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc23fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc24815a310 0x7fc248152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47422/0 (socket says 172.21.15.66:47422) 2024-09-10T22:08:16.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc23fe00640 1 -- 172.21.15.66:0/440165399 learned_addr learned my addr 172.21.15.66:0/440165399 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:16.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc247e00640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2480c4fc0 0x7fc248147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:16.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc24d000640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2481f6710 0x7fc24814ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:16.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc23fe00640 1 -- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2481f6710 msgr2=0x7fc24814ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:16.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc23fe00640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2481f6710 0x7fc24814ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc23fe00640 1 -- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2480c4fc0 msgr2=0x7fc248147a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:16.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc23fe00640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2480c4fc0 0x7fc248147a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:16.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc23fe00640 1 -- 172.21.15.66:0/440165399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc248150e10 con 0x7fc24815a310 2024-09-10T22:08:16.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.934+0000 7fc23fe00640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc24815a310 0x7fc248152890 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc23000e9f0 tx=0x7fc23000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:16.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.938+0000 7fc244c00640 1 -- 172.21.15.66:0/440165399 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23000cd20 con 0x7fc24815a310 2024-09-10T22:08:16.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.938+0000 7fc244c00640 1 -- 172.21.15.66:0/440165399 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc230010430 con 0x7fc24815a310 2024-09-10T22:08:16.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.938+0000 7fc244c00640 1 -- 172.21.15.66:0/440165399 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230010860 con 0x7fc24815a310 2024-09-10T22:08:16.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.938+0000 7fc24f000640 1 -- 172.21.15.66:0/440165399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2480b2ac0 con 0x7fc24815a310 2024-09-10T22:08:16.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.938+0000 7fc24f000640 1 -- 172.21.15.66:0/440165399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2480b3080 con 0x7fc24815a310 2024-09-10T22:08:16.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.938+0000 7fc24f000640 1 -- 172.21.15.66:0/440165399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2480c48b0 con 0x7fc24815a310 2024-09-10T22:08:16.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.938+0000 7fc244c00640 1 -- 172.21.15.66:0/440165399 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fc230010a00 con 0x7fc24815a310 2024-09-10T22:08:16.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.938+0000 7fc244c00640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc224077ad0 0x7fc224079f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:16.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.942+0000 7fc244c00640 1 -- 172.21.15.66:0/440165399 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4913+0+0 (secure 0 0 0) 0x7fc2300928c0 con 0x7fc24815a310 2024-09-10T22:08:16.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.942+0000 7fc247e00640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc224077ad0 0x7fc224079f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:16.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.942+0000 7fc247e00640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc224077ad0 0x7fc224079f90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc238007fd0 tx=0x7fc238009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:16.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:16.946+0000 7fc244c00640 1 -- 172.21.15.66:0/440165399 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc23005f460 con 0x7fc24815a310 2024-09-10T22:08:17.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 -- 172.21.15.66:0/440165399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc248070420 con 0x7fc24815a310 2024-09-10T22:08:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc244c00640 1 -- 172.21.15.66:0/440165399 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fc230063e90 con 0x7fc24815a310 2024-09-10T22:08:17.127 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:17.127 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:08:17.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:17.127 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:08:17.127 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:08:17.127 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:17.127 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:08:17.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 -- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc224077ad0 msgr2=0x7fc224079f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:17.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc224077ad0 0x7fc224079f90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc238007fd0 tx=0x7fc238009040 comp rx=0 tx=0).stop 2024-09-10T22:08:17.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 -- 172.21.15.66:0/440165399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc24815a310 msgr2=0x7fc248152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:17.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc24815a310 0x7fc248152890 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc23000e9f0 tx=0x7fc23000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:08:17.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 -- 172.21.15.66:0/440165399 shutdown_connections 2024-09-10T22:08:17.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc224077ad0 0x7fc224079f90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:17.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2481f6710 0x7fc24814ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:17.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc24815a310 0x7fc248152890 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:17.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 --2- 172.21.15.66:0/440165399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2480c4fc0 0x7fc248147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:17.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 -- 172.21.15.66:0/440165399 >> 172.21.15.66:0/440165399 conn(0x7fc2480cfa30 msgr2=0x7fc2480724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:17.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 -- 172.21.15.66:0/440165399 shutdown_connections 2024-09-10T22:08:17.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.122+0000 7fc24f000640 1 -- 172.21.15.66:0/440165399 wait complete. 2024-09-10T22:08:17.148 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:08:17.149 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:08:17.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 -- 172.21.15.66:0/3644692329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e015ddd0 msgr2=0x7f09e015e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:17.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 --2- 172.21.15.66:0/3644692329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e015ddd0 0x7f09e015e220 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f09c800aa40 tx=0x7f09c8020400 comp rx=0 tx=0).stop 2024-09-10T22:08:17.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 -- 172.21.15.66:0/3644692329 shutdown_connections 2024-09-10T22:08:17.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 --2- 172.21.15.66:0/3644692329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e015ddd0 0x7f09e015e220 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:17.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 -- 172.21.15.66:0/3644692329 >> 172.21.15.66:0/3644692329 conn(0x7f09e00cfaa0 msgr2=0x7f09e00cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:17.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 -- 172.21.15.66:0/3644692329 shutdown_connections 2024-09-10T22:08:17.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 -- 172.21.15.66:0/3644692329 wait complete. 2024-09-10T22:08:17.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 Processor -- start 2024-09-10T22:08:17.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 -- start start 2024-09-10T22:08:17.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09e00cee30 0x7f09e00c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:17.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e015b730 0x7f09e00c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:17.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09e015ddd0 0x7f09e00c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:17.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09e015c620 con 0x7f09e015b730 2024-09-10T22:08:17.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f09e015c4a0 con 0x7f09e00cee30 2024-09-10T22:08:17.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.294+0000 7f09e4a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f09e015c7a0 con 0x7f09e015ddd0 2024-09-10T22:08:17.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e015b730 0x7f09e00c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:17.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e015b730 0x7f09e00c9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47442/0 (socket says 172.21.15.66:47442) 2024-09-10T22:08:17.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d7e00640 1 -- 172.21.15.66:0/2719946516 learned_addr learned my addr 172.21.15.66:0/2719946516 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:17.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09de800640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09e015ddd0 0x7f09e00c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:17.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09dde00640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09e00cee30 0x7f09e00c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:17.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d7e00640 1 -- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09e015ddd0 msgr2=0x7f09e00c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:17.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d7e00640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09e015ddd0 0x7f09e00c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:17.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d7e00640 1 -- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09e00cee30 msgr2=0x7f09e00c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:17.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d7e00640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09e00cee30 0x7f09e00c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:17.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d7e00640 1 -- 172.21.15.66:0/2719946516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e00c55a0 con 0x7f09e015b730 2024-09-10T22:08:17.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d7e00640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e015b730 0x7f09e00c9160 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f09c801d010 tx=0x7f09c80b0400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:17.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d6000640 1 -- 172.21.15.66:0/2719946516 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c80b5040 con 0x7f09e015b730 2024-09-10T22:08:17.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d6000640 1 -- 172.21.15.66:0/2719946516 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09c80b0c10 con 0x7f09e015b730 2024-09-10T22:08:17.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09e4a00640 1 -- 172.21.15.66:0/2719946516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e00c5880 con 0x7f09e015b730 2024-09-10T22:08:17.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d6000640 1 -- 172.21.15.66:0/2719946516 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8003600 con 0x7f09e015b730 2024-09-10T22:08:17.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09e4a00640 1 -- 172.21.15.66:0/2719946516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e00c5e40 con 0x7f09e015b730 2024-09-10T22:08:17.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09e4a00640 1 -- 172.21.15.66:0/2719946516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e015e220 con 0x7f09e015b730 2024-09-10T22:08:17.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.298+0000 7f09d6000640 1 -- 172.21.15.66:0/2719946516 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f09c80b8070 con 0x7f09e015b730 2024-09-10T22:08:17.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.302+0000 7f09d6000640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09bc0736b0 0x7f09bc075b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:17.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.302+0000 7f09d6000640 1 -- 172.21.15.66:0/2719946516 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4913+0+0 (secure 0 0 0) 0x7f09c812f940 con 0x7f09e015b730 2024-09-10T22:08:17.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.302+0000 7f09dde00640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09bc0736b0 0x7f09bc075b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:17.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.302+0000 7f09dde00640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09bc0736b0 0x7f09bc075b70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f09d00016f0 tx=0x7f09d0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:17.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.302+0000 7f09d6000640 1 -- 172.21.15.66:0/2719946516 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f09c8019f60 con 0x7f09e015b730 2024-09-10T22:08:17.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:17.478+0000 7f09e4a00640 1 -- 172.21.15.66:0/2719946516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f09e0150b40 con 0x7f09e015b730 2024-09-10T22:08:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.350+0000 7f09d6000640 1 -- 172.21.15.66:0/2719946516 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f09c80fc770 con 0x7f09e015b730 2024-09-10T22:08:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:08:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.354+0000 7f09e4a00640 1 -- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09bc0736b0 msgr2=0x7f09bc075b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.354+0000 7f09e4a00640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09bc0736b0 0x7f09bc075b70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f09d00016f0 tx=0x7f09d0009040 comp rx=0 tx=0).stop 2024-09-10T22:08:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.354+0000 7f09e4a00640 1 -- 172.21.15.66:0/2719946516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e015b730 msgr2=0x7f09e00c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.354+0000 7f09e4a00640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e015b730 0x7f09e00c9160 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f09c801d010 tx=0x7f09c80b0400 comp rx=0 tx=0).stop 2024-09-10T22:08:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.354+0000 7f09e4a00640 1 -- 172.21.15.66:0/2719946516 shutdown_connections 2024-09-10T22:08:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.354+0000 7f09e4a00640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09bc0736b0 0x7f09bc075b70 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.354+0000 7f09e4a00640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09e015ddd0 0x7f09e00c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:18.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.354+0000 7f09e4a00640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e015b730 0x7f09e00c9160 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:18.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.354+0000 7f09e4a00640 1 --2- 172.21.15.66:0/2719946516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09e00cee30 0x7f09e00c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:18.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.354+0000 7f09e4a00640 1 -- 172.21.15.66:0/2719946516 >> 172.21.15.66:0/2719946516 conn(0x7f09e00cfaa0 msgr2=0x7f09e00ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:18.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.354+0000 7f09e4a00640 1 -- 172.21.15.66:0/2719946516 shutdown_connections 2024-09-10T22:08:18.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.358+0000 7f09e4a00640 1 -- 172.21.15.66:0/2719946516 wait complete. 2024-09-10T22:08:18.389 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:08:18.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.542+0000 7f0961e00640 1 -- 172.21.15.66:0/214271504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0940004810 msgr2=0x7f0940004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:18.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.542+0000 7f0961e00640 1 --2- 172.21.15.66:0/214271504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0940004810 0x7f0940004c90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f094400c310 tx=0x7f094401cfc0 comp rx=0 tx=0).stop 2024-09-10T22:08:18.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.542+0000 7f095b400640 1 -- 172.21.15.66:0/214271504 reap_dead start 2024-09-10T22:08:18.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0961e00640 1 -- 172.21.15.66:0/214271504 shutdown_connections 2024-09-10T22:08:18.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0961e00640 1 -- 172.21.15.66:0/214271504 >> 172.21.15.66:0/214271504 conn(0x7f095c0cfa30 msgr2=0x7f095c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:18.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0961e00640 1 -- 172.21.15.66:0/214271504 shutdown_connections 2024-09-10T22:08:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0961e00640 1 -- 172.21.15.66:0/214271504 wait complete. 2024-09-10T22:08:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0961e00640 1 Processor -- start 2024-09-10T22:08:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0961e00640 1 -- start start 2024-09-10T22:08:18.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0961e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0940004810 0x7f095c14c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:18.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0961e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c0cdf40 0x7f095c0ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:18.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0961e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f095c15d130 0x7f095c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:18.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0961e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f095c15ebf0 con 0x7f095c15d130 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0961e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f095c15ea70 con 0x7f0940004810 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0961e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f095c15ed70 con 0x7f095c0cdf40 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f095b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f095c15d130 0x7f095c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f095b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f095c15d130 0x7f095c15d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47474/0 (socket says 172.21.15.66:47474) 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f095b400640 1 -- 172.21.15.66:0/2279616244 learned_addr learned my addr 172.21.15.66:0/2279616244 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0952000640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c0cdf40 0x7f095c0ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f095aa00640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0940004810 0x7f095c14c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f095b400640 1 -- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c0cdf40 msgr2=0x7f095c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f095b400640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c0cdf40 0x7f095c0ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f095b400640 1 -- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0940004810 msgr2=0x7f095c14c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f095b400640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0940004810 0x7f095c14c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f095b400640 1 -- 172.21.15.66:0/2279616244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c1f8160 con 0x7f095c15d130 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f0952000640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c0cdf40 0x7f095c0ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.546+0000 7f095b400640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f095c15d130 0x7f095c15d5e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f095c06ded0 tx=0x7f0954004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.550+0000 7f0953e00640 1 -- 172.21.15.66:0/2279616244 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954004b30 con 0x7f095c15d130 2024-09-10T22:08:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.550+0000 7f0953e00640 1 -- 172.21.15.66:0/2279616244 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0954004cd0 con 0x7f095c15d130 2024-09-10T22:08:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.550+0000 7f0953e00640 1 -- 172.21.15.66:0/2279616244 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954010cc0 con 0x7f095c15d130 2024-09-10T22:08:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.550+0000 7f0961e00640 1 -- 172.21.15.66:0/2279616244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c1f8400 con 0x7f095c15d130 2024-09-10T22:08:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.550+0000 7f0961e00640 1 -- 172.21.15.66:0/2279616244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c1f89c0 con 0x7f095c15d130 2024-09-10T22:08:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.550+0000 7f0961e00640 1 -- 172.21.15.66:0/2279616244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f095c0c4df0 con 0x7f095c15d130 2024-09-10T22:08:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.550+0000 7f0953e00640 1 -- 172.21.15.66:0/2279616244 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f095400f040 con 0x7f095c15d130 2024-09-10T22:08:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.550+0000 7f0953e00640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09380736f0 0x7f0938075bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.550+0000 7f0953e00640 1 -- 172.21.15.66:0/2279616244 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5283+0+0 (secure 0 0 0) 0x7f09540923e0 con 0x7f095c15d130 2024-09-10T22:08:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.550+0000 7f095aa00640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09380736f0 0x7f0938075bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:18.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.554+0000 7f095aa00640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09380736f0 0x7f0938075bb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f095c158f90 tx=0x7f094c00b360 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:18.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.554+0000 7f0953e00640 1 -- 172.21.15.66:0/2279616244 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0954010e60 con 0x7f095c15d130 2024-09-10T22:08:18.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:18.730+0000 7f0961e00640 1 -- 172.21.15.66:0/2279616244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f095c070420 con 0x7f095c15d130 2024-09-10T22:08:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.358+0000 7f0953e00640 1 -- 172.21.15.66:0/2279616244 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f095405f510 con 0x7f095c15d130 2024-09-10T22:08:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.362+0000 7f0961e00640 1 -- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09380736f0 msgr2=0x7f0938075bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.362+0000 7f0961e00640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09380736f0 0x7f0938075bb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f095c158f90 tx=0x7f094c00b360 comp rx=0 tx=0).stop 2024-09-10T22:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.362+0000 7f0961e00640 1 -- 172.21.15.66:0/2279616244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f095c15d130 msgr2=0x7f095c15d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.362+0000 7f0961e00640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f095c15d130 0x7f095c15d5e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f095c06ded0 tx=0x7f0954004640 comp rx=0 tx=0).stop 2024-09-10T22:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.362+0000 7f0961e00640 1 -- 172.21.15.66:0/2279616244 shutdown_connections 2024-09-10T22:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.362+0000 7f0961e00640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09380736f0 0x7f0938075bb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.362+0000 7f0961e00640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f095c15d130 0x7f095c15d5e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.362+0000 7f0961e00640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c0cdf40 0x7f095c0ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.362+0000 7f0961e00640 1 --2- 172.21.15.66:0/2279616244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0940004810 0x7f095c14c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.362+0000 7f0961e00640 1 -- 172.21.15.66:0/2279616244 >> 172.21.15.66:0/2279616244 conn(0x7f095c0cfa30 msgr2=0x7f095c0cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.362+0000 7f0961e00640 1 -- 172.21.15.66:0/2279616244 shutdown_connections 2024-09-10T22:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.362+0000 7f0961e00640 1 -- 172.21.15.66:0/2279616244 wait complete. 2024-09-10T22:08:19.403 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:08:19.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.546+0000 7f5feae00640 1 -- 172.21.15.66:0/1933682878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe40aee20 msgr2=0x7f5fe40cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:19.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.546+0000 7f5feae00640 1 --2- 172.21.15.66:0/1933682878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe40aee20 0x7f5fe40cd930 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009c80 tx=0x7f5fdc01cd10 comp rx=0 tx=0).stop 2024-09-10T22:08:19.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.546+0000 7f5feae00640 1 -- 172.21.15.66:0/1933682878 shutdown_connections 2024-09-10T22:08:19.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.546+0000 7f5feae00640 1 --2- 172.21.15.66:0/1933682878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe40aee20 0x7f5fe40cd930 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:19.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.546+0000 7f5feae00640 1 --2- 172.21.15.66:0/1933682878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fe4159900 0x7f5fe40ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:19.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.546+0000 7f5feae00640 1 -- 172.21.15.66:0/1933682878 >> 172.21.15.66:0/1933682878 conn(0x7f5fe40cfa30 msgr2=0x7f5fe40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:19.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.546+0000 7f5feae00640 1 -- 172.21.15.66:0/1933682878 shutdown_connections 2024-09-10T22:08:19.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.546+0000 7f5feae00640 1 -- 172.21.15.66:0/1933682878 wait complete. 2024-09-10T22:08:19.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5feae00640 1 Processor -- start 2024-09-10T22:08:19.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5feae00640 1 -- start start 2024-09-10T22:08:19.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5feae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fe40aee20 0x7f5fe41464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:19.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5feae00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fe4159900 0x7f5fe40c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:19.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5feae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe415cc80 0x7f5fe415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:19.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5feae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4168f70 con 0x7f5fe415cc80 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5feae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4168df0 con 0x7f5fe40aee20 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5feae00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe41690f0 con 0x7f5fe4159900 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe8e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe415cc80 0x7f5fe415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe8e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe415cc80 0x7f5fe415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47498/0 (socket says 172.21.15.66:47498) 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe8e00640 1 -- 172.21.15.66:0/1690913948 learned_addr learned my addr 172.21.15.66:0/1690913948 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe3e00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fe40aee20 0x7f5fe41464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fdb400640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fe4159900 0x7f5fe40c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe8e00640 1 -- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fe4159900 msgr2=0x7f5fe40c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe8e00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fe4159900 0x7f5fe40c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe8e00640 1 -- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fe40aee20 msgr2=0x7f5fe41464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe8e00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fe40aee20 0x7f5fe41464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe8e00640 1 -- 172.21.15.66:0/1690913948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe41469f0 con 0x7f5fe415cc80 2024-09-10T22:08:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe8e00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe415cc80 0x7f5fe415aa80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc01b3c0 tx=0x7f5fdc009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:19.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe0c00640 1 -- 172.21.15.66:0/1690913948 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0b2040 con 0x7f5fe415cc80 2024-09-10T22:08:19.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe0c00640 1 -- 172.21.15.66:0/1690913948 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fdc004200 con 0x7f5fe415cc80 2024-09-10T22:08:19.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5fe0c00640 1 -- 172.21.15.66:0/1690913948 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc004610 con 0x7f5fe415cc80 2024-09-10T22:08:19.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5feae00640 1 -- 172.21.15.66:0/1690913948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe40c6380 con 0x7f5fe415cc80 2024-09-10T22:08:19.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.550+0000 7f5feae00640 1 -- 172.21.15.66:0/1690913948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe4159fa0 con 0x7f5fe415cc80 2024-09-10T22:08:19.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.554+0000 7f5feae00640 1 -- 172.21.15.66:0/1690913948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe40cd930 con 0x7f5fe415cc80 2024-09-10T22:08:19.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.554+0000 7f5fe0c00640 1 -- 172.21.15.66:0/1690913948 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f5fdc0b5070 con 0x7f5fe415cc80 2024-09-10T22:08:19.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.554+0000 7f5fe0c00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5fc00736c0 0x7f5fc0075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:19.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.554+0000 7f5fe0c00640 1 -- 172.21.15.66:0/1690913948 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5665+0+0 (secure 0 0 0) 0x7f5fdc12cf10 con 0x7f5fe415cc80 2024-09-10T22:08:19.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.558+0000 7f5fe3e00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5fc00736c0 0x7f5fc0075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:19.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.558+0000 7f5fe3e00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5fc00736c0 0x7f5fc0075b80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009be0 tx=0x7f5fd40091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:19.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.558+0000 7f5fe0c00640 1 -- 172.21.15.66:0/1690913948 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5fdc016990 con 0x7f5fe415cc80 2024-09-10T22:08:19.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:19.770+0000 7f5feae00640 1 -- 172.21.15.66:0/1690913948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5fe406fc70 con 0x7f5fe415cc80 2024-09-10T22:08:20.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.362+0000 7f5fe0c00640 1 -- 172.21.15.66:0/1690913948 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v52) ==== 174+0+0 (secure 0 0 0) 0x7f5fdc0f96f0 con 0x7f5fe415cc80 2024-09-10T22:08:20.367 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 12 and data pool 13 2024-09-10T22:08:20.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.366+0000 7f5feae00640 1 -- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5fc00736c0 msgr2=0x7f5fc0075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:20.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.366+0000 7f5feae00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5fc00736c0 0x7f5fc0075b80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009be0 tx=0x7f5fd40091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.366+0000 7f5feae00640 1 -- 172.21.15.66:0/1690913948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe415cc80 msgr2=0x7f5fe415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:20.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.366+0000 7f5feae00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe415cc80 0x7f5fe415aa80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc01b3c0 tx=0x7f5fdc009320 comp rx=0 tx=0).stop 2024-09-10T22:08:20.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.366+0000 7f5feae00640 1 -- 172.21.15.66:0/1690913948 shutdown_connections 2024-09-10T22:08:20.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.366+0000 7f5feae00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5fc00736c0 0x7f5fc0075b80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.366+0000 7f5feae00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe415cc80 0x7f5fe415aa80 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.366+0000 7f5feae00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fe4159900 0x7f5fe40c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.366+0000 7f5feae00640 1 --2- 172.21.15.66:0/1690913948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fe40aee20 0x7f5fe41464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.366+0000 7f5feae00640 1 -- 172.21.15.66:0/1690913948 >> 172.21.15.66:0/1690913948 conn(0x7f5fe40cfa30 msgr2=0x7f5fe415fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.366+0000 7f5feae00640 1 -- 172.21.15.66:0/1690913948 shutdown_connections 2024-09-10T22:08:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.366+0000 7f5feae00640 1 -- 172.21.15.66:0/1690913948 wait complete. 2024-09-10T22:08:20.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:08:20.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.562+0000 7fde23600640 1 -- 172.21.15.66:0/2639633522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde04004d50 msgr2=0x7fde040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:20.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.562+0000 7fde23600640 1 --2- 172.21.15.66:0/2639633522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde04004d50 0x7fde040051d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fde14009ee0 tx=0x7fde1401d630 comp rx=0 tx=0).stop 2024-09-10T22:08:20.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.562+0000 7fde21600640 1 -- 172.21.15.66:0/2639633522 reap_dead start 2024-09-10T22:08:20.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.562+0000 7fde23600640 1 -- 172.21.15.66:0/2639633522 shutdown_connections 2024-09-10T22:08:20.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.562+0000 7fde23600640 1 -- 172.21.15.66:0/2639633522 >> 172.21.15.66:0/2639633522 conn(0x7fde1c0d1c60 msgr2=0x7fde1c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:20.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.562+0000 7fde23600640 1 -- 172.21.15.66:0/2639633522 shutdown_connections 2024-09-10T22:08:20.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.562+0000 7fde23600640 1 -- 172.21.15.66:0/2639633522 wait complete. 2024-09-10T22:08:20.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde23600640 1 Processor -- start 2024-09-10T22:08:20.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde23600640 1 -- start start 2024-09-10T22:08:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde23600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde04004d50 0x7fde1c09f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde23600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde1c15c460 0x7fde1c0cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde23600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde1c15c8e0 0x7fde1c0cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde23600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c06e270 con 0x7fde1c15c460 2024-09-10T22:08:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde23600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c0d0850 con 0x7fde04004d50 2024-09-10T22:08:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde23600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fde1c06e3f0 con 0x7fde1c15c8e0 2024-09-10T22:08:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde1c15c460 0x7fde1c0cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde20c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde04004d50 0x7fde1c09f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde1c15c460 0x7fde1c0cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47518/0 (socket says 172.21.15.66:47518) 2024-09-10T22:08:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde1be00640 1 -- 172.21.15.66:0/1658326833 learned_addr learned my addr 172.21.15.66:0/1658326833 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:20.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde21600640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde1c15c8e0 0x7fde1c0cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:20.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde1be00640 1 -- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde1c15c8e0 msgr2=0x7fde1c0cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:20.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde1be00640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde1c15c8e0 0x7fde1c0cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde1be00640 1 -- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde04004d50 msgr2=0x7fde1c09f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:20.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde1be00640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde04004d50 0x7fde1c09f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde1be00640 1 -- 172.21.15.66:0/1658326833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1c09e630 con 0x7fde1c15c460 2024-09-10T22:08:20.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde1be00640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde1c15c460 0x7fde1c0cb340 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fde08011a50 tx=0x7fde08011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:20.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde18c00640 1 -- 172.21.15.66:0/1658326833 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0801f070 con 0x7fde1c15c460 2024-09-10T22:08:20.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde18c00640 1 -- 172.21.15.66:0/1658326833 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde0800fe80 con 0x7fde1c15c460 2024-09-10T22:08:20.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde18c00640 1 -- 172.21.15.66:0/1658326833 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde08012690 con 0x7fde1c15c460 2024-09-10T22:08:20.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde23600640 1 -- 172.21.15.66:0/1658326833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1c09f8d0 con 0x7fde1c15c460 2024-09-10T22:08:20.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde23600640 1 -- 172.21.15.66:0/1658326833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1c09e8b0 con 0x7fde1c15c460 2024-09-10T22:08:20.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.566+0000 7fde23600640 1 -- 172.21.15.66:0/1658326833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde1c06dd30 con 0x7fde1c15c460 2024-09-10T22:08:20.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.570+0000 7fde18c00640 1 -- 172.21.15.66:0/1658326833 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fde08012830 con 0x7fde1c15c460 2024-09-10T22:08:20.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.570+0000 7fde18c00640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fddfc0736c0 0x7fddfc075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:20.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.570+0000 7fde18c00640 1 -- 172.21.15.66:0/1658326833 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5765+0+0 (secure 0 0 0) 0x7fde08092fe0 con 0x7fde1c15c460 2024-09-10T22:08:20.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.570+0000 7fde20c00640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fddfc0736c0 0x7fddfc075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:20.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.570+0000 7fde20c00640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fddfc0736c0 0x7fddfc075b80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fde10001ec0 tx=0x7fde10006730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:20.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.574+0000 7fde18c00640 1 -- 172.21.15.66:0/1658326833 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fde1c09e630 con 0x7fde1c15c460 2024-09-10T22:08:20.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 -- 172.21.15.66:0/1658326833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde1c0cbef0 con 0x7fde1c15c460 2024-09-10T22:08:20.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde18c00640 1 -- 172.21.15.66:0/1658326833 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16749 (secure 0 0 0) 0x7fde0805f700 con 0x7fde1c15c460 2024-09-10T22:08:20.756 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:20.756 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":84,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:08:19.775692+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:08:17.494722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T22:08:18.745603+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:08:20.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 -- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fddfc0736c0 msgr2=0x7fddfc075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:20.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fddfc0736c0 0x7fddfc075b80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fde10001ec0 tx=0x7fde10006730 comp rx=0 tx=0).stop 2024-09-10T22:08:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 -- 172.21.15.66:0/1658326833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde1c15c460 msgr2=0x7fde1c0cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde1c15c460 0x7fde1c0cb340 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fde08011a50 tx=0x7fde08011f10 comp rx=0 tx=0).stop 2024-09-10T22:08:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 -- 172.21.15.66:0/1658326833 shutdown_connections 2024-09-10T22:08:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fddfc0736c0 0x7fddfc075b80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde1c15c8e0 0x7fde1c0cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde1c15c460 0x7fde1c0cb340 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 --2- 172.21.15.66:0/1658326833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde04004d50 0x7fde1c09f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 -- 172.21.15.66:0/1658326833 >> 172.21.15.66:0/1658326833 conn(0x7fde1c0d1c60 msgr2=0x7fde1c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 -- 172.21.15.66:0/1658326833 shutdown_connections 2024-09-10T22:08:20.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.750+0000 7fde23600640 1 -- 172.21.15.66:0/1658326833 wait complete. 2024-09-10T22:08:20.777 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:08:20.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.918+0000 7f2800e00640 1 -- 172.21.15.66:0/1383301302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27fc15f100 msgr2=0x7f27fc0d1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:20.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.918+0000 7f2800e00640 1 --2- 172.21.15.66:0/1383301302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27fc15f100 0x7f27fc0d1960 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009840 tx=0x7f27ec01c820 comp rx=0 tx=0).stop 2024-09-10T22:08:20.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.918+0000 7f2800e00640 1 -- 172.21.15.66:0/1383301302 shutdown_connections 2024-09-10T22:08:20.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.918+0000 7f2800e00640 1 --2- 172.21.15.66:0/1383301302 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f27fc0b3110 0x7f27fc0d1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.918+0000 7f2800e00640 1 --2- 172.21.15.66:0/1383301302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27fc15f100 0x7f27fc0d1960 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.918+0000 7f2800e00640 1 -- 172.21.15.66:0/1383301302 >> 172.21.15.66:0/1383301302 conn(0x7f27fc09aa50 msgr2=0x7f27fc09ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:20.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.918+0000 7f2800e00640 1 -- 172.21.15.66:0/1383301302 shutdown_connections 2024-09-10T22:08:20.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.918+0000 7f2800e00640 1 -- 172.21.15.66:0/1383301302 wait complete. 2024-09-10T22:08:20.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f2800e00640 1 Processor -- start 2024-09-10T22:08:20.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f2800e00640 1 -- start start 2024-09-10T22:08:20.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f2800e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27fc0b3110 0x7f27fc0b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:20.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f2800e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27fc0d0110 0x7f27fc1fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:20.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f2800e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f27fc15f100 0x7f27fc165f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:20.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f2800e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc1692e0 con 0x7f27fc0b3110 2024-09-10T22:08:20.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f2800e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc169160 con 0x7f27fc0d0110 2024-09-10T22:08:20.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f2800e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f27fc169460 con 0x7f27fc15f100 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27fc0b3110 0x7f27fc0b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27fc0b3110 0x7f27fc0b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47526/0 (socket says 172.21.15.66:47526) 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27fb400640 1 -- 172.21.15.66:0/981157435 learned_addr learned my addr 172.21.15.66:0/981157435 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27faa00640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27fc0d0110 0x7f27fc1fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27fbe00640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f27fc15f100 0x7f27fc165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27fb400640 1 -- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f27fc15f100 msgr2=0x7f27fc165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27fb400640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f27fc15f100 0x7f27fc165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27fb400640 1 -- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27fc0d0110 msgr2=0x7f27fc1fbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27fb400640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27fc0d0110 0x7f27fc1fbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27fb400640 1 -- 172.21.15.66:0/981157435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc0b4b70 con 0x7f27fc0b3110 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27faa00640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27fc0d0110 0x7f27fc1fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27fb400640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27fc0b3110 0x7f27fc0b5930 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009790 tx=0x7f27ec005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:20.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27f3400640 1 -- 172.21.15.66:0/981157435 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0bb070 con 0x7f27fc0b3110 2024-09-10T22:08:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27f3400640 1 -- 172.21.15.66:0/981157435 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27ec0a5d50 con 0x7f27fc0b3110 2024-09-10T22:08:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f27f3400640 1 -- 172.21.15.66:0/981157435 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0b2040 con 0x7f27fc0b3110 2024-09-10T22:08:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f2800e00640 1 -- 172.21.15.66:0/981157435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc0b4e50 con 0x7f27fc0b3110 2024-09-10T22:08:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f2800e00640 1 -- 172.21.15.66:0/981157435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc154ba0 con 0x7f27fc0b3110 2024-09-10T22:08:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.922+0000 7f2800e00640 1 -- 172.21.15.66:0/981157435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27fc15f5b0 con 0x7f27fc0b3110 2024-09-10T22:08:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.926+0000 7f27f3400640 1 -- 172.21.15.66:0/981157435 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f27ec0b5030 con 0x7f27fc0b3110 2024-09-10T22:08:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.926+0000 7f27f3400640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f27dc0736f0 0x7f27dc075bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.926+0000 7f27f3400640 1 -- 172.21.15.66:0/981157435 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5765+0+0 (secure 0 0 0) 0x7f27ec12b220 con 0x7f27fc0b3110 2024-09-10T22:08:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.926+0000 7f27faa00640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f27dc0736f0 0x7f27dc075bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:20.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.926+0000 7f27faa00640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f27dc0736f0 0x7f27dc075bb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f27e8007fd0 tx=0x7f27e8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:20.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:20.930+0000 7f27f3400640 1 -- 172.21.15.66:0/981157435 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f27ec0f1de0 con 0x7f27fc0b3110 2024-09-10T22:08:21.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 -- 172.21.15.66:0/981157435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27fc0d2970 con 0x7f27fc0b3110 2024-09-10T22:08:21.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f27f3400640 1 -- 172.21.15.66:0/981157435 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16749 (secure 0 0 0) 0x7f27ec016410 con 0x7f27fc0b3110 2024-09-10T22:08:21.111 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:21.111 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":84,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:08:19.775692+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:08:17.494722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T22:08:18.745603+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:08:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 -- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f27dc0736f0 msgr2=0x7f27dc075bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f27dc0736f0 0x7f27dc075bb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f27e8007fd0 tx=0x7f27e8009040 comp rx=0 tx=0).stop 2024-09-10T22:08:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 -- 172.21.15.66:0/981157435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27fc0b3110 msgr2=0x7f27fc0b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27fc0b3110 0x7f27fc0b5930 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009790 tx=0x7f27ec005db0 comp rx=0 tx=0).stop 2024-09-10T22:08:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 -- 172.21.15.66:0/981157435 shutdown_connections 2024-09-10T22:08:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f27dc0736f0 0x7f27dc075bb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f27fc15f100 0x7f27fc165f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27fc0d0110 0x7f27fc1fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 --2- 172.21.15.66:0/981157435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27fc0b3110 0x7f27fc0b5930 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 -- 172.21.15.66:0/981157435 >> 172.21.15.66:0/981157435 conn(0x7f27fc09aa50 msgr2=0x7f27fc0d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 -- 172.21.15.66:0/981157435 shutdown_connections 2024-09-10T22:08:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.106+0000 7f2800e00640 1 -- 172.21.15.66:0/981157435 wait complete. 2024-09-10T22:08:21.133 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:08:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 -- 172.21.15.66:0/2589214365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc015ada0 msgr2=0x7f4cc00caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 --2- 172.21.15.66:0/2589214365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc015ada0 0x7f4cc00caa90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4cc006eb10 tx=0x7f4cb401c8c0 comp rx=0 tx=0).stop 2024-09-10T22:08:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 -- 172.21.15.66:0/2589214365 shutdown_connections 2024-09-10T22:08:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 --2- 172.21.15.66:0/2589214365 >> v2:172.21.15.94:3300/0 conn(0x7f4cc00cbb40 0x7f4cc00ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 --2- 172.21.15.66:0/2589214365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc015ada0 0x7f4cc00caa90 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 -- 172.21.15.66:0/2589214365 >> 172.21.15.66:0/2589214365 conn(0x7f4cc0074240 msgr2=0x7f4cc0074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 -- 172.21.15.66:0/2589214365 shutdown_connections 2024-09-10T22:08:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 -- 172.21.15.66:0/2589214365 wait complete. 2024-09-10T22:08:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 Processor -- start 2024-09-10T22:08:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 -- start start 2024-09-10T22:08:21.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cc00cbb40 0x7f4cc00c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:21.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc00ce770 0x7f4cc0162b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:21.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc015ada0 0x7f4cc0163090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc01680b0 con 0x7f4cc00ce770 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0167f30 con 0x7f4cc015ada0 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc8200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc0168230 con 0x7f4cc00cbb40 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc00ce770 0x7f4cc0162b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc00ce770 0x7f4cc0162b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47540/0 (socket says 172.21.15.66:47540) 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc4e00640 1 -- 172.21.15.66:0/876491613 learned_addr learned my addr 172.21.15.66:0/876491613 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc5800640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cc00cbb40 0x7f4cc00c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc6200640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc015ada0 0x7f4cc0163090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.278+0000 7f4cc4e00640 1 -- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cc00cbb40 msgr2=0x7f4cc00c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cc4e00640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cc00cbb40 0x7f4cc00c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cc4e00640 1 -- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc015ada0 msgr2=0x7f4cc0163090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cc4e00640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc015ada0 0x7f4cc0163090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cc4e00640 1 -- 172.21.15.66:0/876491613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc00c6a80 con 0x7f4cc00ce770 2024-09-10T22:08:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cc5800640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cc00cbb40 0x7f4cc00c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cc6200640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc015ada0 0x7f4cc0163090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cc4e00640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc00ce770 0x7f4cc0162b50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00d9b0 tx=0x7f4cac00de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cbd600640 1 -- 172.21.15.66:0/876491613 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac017070 con 0x7f4cc00ce770 2024-09-10T22:08:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cbd600640 1 -- 172.21.15.66:0/876491613 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cac009cc0 con 0x7f4cc00ce770 2024-09-10T22:08:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cbd600640 1 -- 172.21.15.66:0/876491613 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac005280 con 0x7f4cc00ce770 2024-09-10T22:08:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cc8200640 1 -- 172.21.15.66:0/876491613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc00ad040 con 0x7f4cc00ce770 2024-09-10T22:08:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cc8200640 1 -- 172.21.15.66:0/876491613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc00ad600 con 0x7f4cc00ce770 2024-09-10T22:08:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cc8200640 1 -- 172.21.15.66:0/876491613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc00ce230 con 0x7f4cc00ce770 2024-09-10T22:08:21.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.282+0000 7f4cbd600640 1 -- 172.21.15.66:0/876491613 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f4cac005420 con 0x7f4cc00ce770 2024-09-10T22:08:21.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.286+0000 7f4cbd600640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4ca00736c0 0x7f4ca0075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:21.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.286+0000 7f4cbd600640 1 -- 172.21.15.66:0/876491613 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5765+0+0 (secure 0 0 0) 0x7f4cac092080 con 0x7f4cc00ce770 2024-09-10T22:08:21.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.286+0000 7f4cc5800640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4ca00736c0 0x7f4ca0075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:21.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.286+0000 7f4cc5800640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4ca00736c0 0x7f4ca0075b80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40af000 tx=0x7f4cb401b100 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:21.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.290+0000 7f4cbd600640 1 -- 172.21.15.66:0/876491613 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4cac05f1b0 con 0x7f4cc00ce770 2024-09-10T22:08:21.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:21.502+0000 7f4cc8200640 1 -- 172.21.15.66:0/876491613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4cc00c6df0 con 0x7f4cc00ce770 2024-09-10T22:08:22.348 INFO:tasks.ceph.mds.d.smithi094.stderr:2024-09-10T22:08:22.348+0000 7ffbb0a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:08:22.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.385+0000 7f4cbd600640 1 -- 172.21.15.66:0/876491613 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) ==== 118+0+0 (secure 0 0 0) 0x7f4cac05eab0 con 0x7f4cc00ce770 2024-09-10T22:08:22.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.389+0000 7f4cc8200640 1 -- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4ca00736c0 msgr2=0x7f4ca0075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.389+0000 7f4cc8200640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4ca00736c0 0x7f4ca0075b80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40af000 tx=0x7f4cb401b100 comp rx=0 tx=0).stop 2024-09-10T22:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.389+0000 7f4cc8200640 1 -- 172.21.15.66:0/876491613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc00ce770 msgr2=0x7f4cc0162b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.389+0000 7f4cc8200640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc00ce770 0x7f4cc0162b50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00d9b0 tx=0x7f4cac00de70 comp rx=0 tx=0).stop 2024-09-10T22:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.389+0000 7f4cc8200640 1 -- 172.21.15.66:0/876491613 shutdown_connections 2024-09-10T22:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.389+0000 7f4cc8200640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4ca00736c0 0x7f4ca0075b80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.389+0000 7f4cc8200640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc015ada0 0x7f4cc0163090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.389+0000 7f4cc8200640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cc00ce770 0x7f4cc0162b50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.389+0000 7f4cc8200640 1 --2- 172.21.15.66:0/876491613 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cc00cbb40 0x7f4cc00c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.389+0000 7f4cc8200640 1 -- 172.21.15.66:0/876491613 >> 172.21.15.66:0/876491613 conn(0x7f4cc0074240 msgr2=0x7f4cc0159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.389+0000 7f4cc8200640 1 -- 172.21.15.66:0/876491613 shutdown_connections 2024-09-10T22:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.389+0000 7f4cc8200640 1 -- 172.21.15.66:0/876491613 wait complete. 2024-09-10T22:08:22.419 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:08:22.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:08:22.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 -- 172.21.15.66:0/2784333392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa4156430 msgr2=0x7effa406d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:22.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 --2- 172.21.15.66:0/2784333392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa4156430 0x7effa406d630 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7eff98009840 tx=0x7effa415d120 comp rx=0 tx=0).stop 2024-09-10T22:08:22.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 -- 172.21.15.66:0/2784333392 shutdown_connections 2024-09-10T22:08:22.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 --2- 172.21.15.66:0/2784333392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa4156430 0x7effa406d630 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:22.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 -- 172.21.15.66:0/2784333392 >> 172.21.15.66:0/2784333392 conn(0x7effa40d0b30 msgr2=0x7effa40d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:22.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 -- 172.21.15.66:0/2784333392 shutdown_connections 2024-09-10T22:08:22.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 -- 172.21.15.66:0/2784333392 wait complete. 2024-09-10T22:08:22.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 Processor -- start 2024-09-10T22:08:22.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 -- start start 2024-09-10T22:08:22.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa4071230 0x7effa4154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:22.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa4156430 0x7effa41532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:22.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7effa4160370 0x7effa4151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:22.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7effa416a380 con 0x7effa4156430 2024-09-10T22:08:22.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7effa416a200 con 0x7effa4071230 2024-09-10T22:08:22.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effac600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7effa416a500 con 0x7effa4160370 2024-09-10T22:08:22.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effa9200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa4156430 0x7effa41532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:22.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effa9200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa4156430 0x7effa41532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42128/0 (socket says 172.21.15.66:42128) 2024-09-10T22:08:22.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effa9200640 1 -- 172.21.15.66:0/487302317 learned_addr learned my addr 172.21.15.66:0/487302317 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:22.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.569+0000 7effa9c00640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa4071230 0x7effa4154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:22.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effaa600640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7effa4160370 0x7effa4151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:22.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effa9200640 1 -- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7effa4160370 msgr2=0x7effa4151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:22.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effa9200640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7effa4160370 0x7effa4151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:22.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effa9200640 1 -- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa4071230 msgr2=0x7effa4154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:22.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effa9200640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa4071230 0x7effa4154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:22.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effa9200640 1 -- 172.21.15.66:0/487302317 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa4154630 con 0x7effa4156430 2024-09-10T22:08:22.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effa9200640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa4156430 0x7effa41532a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7eff9000c400 tx=0x7eff9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:22.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effa1600640 1 -- 172.21.15.66:0/487302317 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90017070 con 0x7effa4156430 2024-09-10T22:08:22.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effa1600640 1 -- 172.21.15.66:0/487302317 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff90005400 con 0x7effa4156430 2024-09-10T22:08:22.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effa1600640 1 -- 172.21.15.66:0/487302317 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90003860 con 0x7effa4156430 2024-09-10T22:08:22.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effac600640 1 -- 172.21.15.66:0/487302317 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa4154910 con 0x7effa4156430 2024-09-10T22:08:22.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effac600640 1 -- 172.21.15.66:0/487302317 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa4152d80 con 0x7effa4156430 2024-09-10T22:08:22.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effac600640 1 -- 172.21.15.66:0/487302317 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa406d630 con 0x7effa4156430 2024-09-10T22:08:22.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effa1600640 1 -- 172.21.15.66:0/487302317 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7eff90003a00 con 0x7effa4156430 2024-09-10T22:08:22.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.573+0000 7effa1600640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7eff840736c0 0x7eff84075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:22.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.577+0000 7effa1600640 1 -- 172.21.15.66:0/487302317 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7eff90091750 con 0x7effa4156430 2024-09-10T22:08:22.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.577+0000 7effa9c00640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7eff840736c0 0x7eff84075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:22.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.577+0000 7effa9c00640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7eff840736c0 0x7eff84075b80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7eff980af000 tx=0x7eff980b2040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:22.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.581+0000 7effa1600640 1 -- 172.21.15.66:0/487302317 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7eff9005def0 con 0x7effa4156430 2024-09-10T22:08:22.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:22.789+0000 7effac600640 1 -- 172.21.15.66:0/487302317 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7effa4154ae0 con 0x7effa4156430 2024-09-10T22:08:23.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effa1600640 1 -- 172.21.15.66:0/487302317 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) ==== 122+0+0 (secure 0 0 0) 0x7eff90062920 con 0x7effa4156430 2024-09-10T22:08:23.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effac600640 1 -- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7eff840736c0 msgr2=0x7eff84075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:23.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effac600640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7eff840736c0 0x7eff84075b80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7eff980af000 tx=0x7eff980b2040 comp rx=0 tx=0).stop 2024-09-10T22:08:23.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effac600640 1 -- 172.21.15.66:0/487302317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa4156430 msgr2=0x7effa41532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:23.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effac600640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa4156430 0x7effa41532a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7eff9000c400 tx=0x7eff9000c8c0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effac600640 1 -- 172.21.15.66:0/487302317 shutdown_connections 2024-09-10T22:08:23.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effac600640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7eff840736c0 0x7eff84075b80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effac600640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7effa4160370 0x7effa4151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effac600640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa4156430 0x7effa41532a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effac600640 1 --2- 172.21.15.66:0/487302317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa4071230 0x7effa4154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effac600640 1 -- 172.21.15.66:0/487302317 >> 172.21.15.66:0/487302317 conn(0x7effa40d0b30 msgr2=0x7effa414c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:23.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effac600640 1 -- 172.21.15.66:0/487302317 shutdown_connections 2024-09-10T22:08:23.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.393+0000 7effac600640 1 -- 172.21.15.66:0/487302317 wait complete. 2024-09-10T22:08:23.426 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:08:23.426 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:08:23.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.569+0000 7f028ae00640 1 -- 172.21.15.66:0/3353478586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f028415ef20 msgr2=0x7f02840cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:23.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.569+0000 7f028ae00640 1 --2- 172.21.15.66:0/3353478586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f028415ef20 0x7f02840cecc0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0278009840 tx=0x7f027801c7b0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.569+0000 7f028ae00640 1 -- 172.21.15.66:0/3353478586 shutdown_connections 2024-09-10T22:08:23.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.569+0000 7f028ae00640 1 --2- 172.21.15.66:0/3353478586 >> v2:172.21.15.94:3300/0 conn(0x7f028415f300 0x7f02840b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.569+0000 7f028ae00640 1 --2- 172.21.15.66:0/3353478586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f028415ef20 0x7f02840cecc0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.569+0000 7f028ae00640 1 -- 172.21.15.66:0/3353478586 >> 172.21.15.66:0/3353478586 conn(0x7f02840d1b10 msgr2=0x7f02840d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:23.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.569+0000 7f028ae00640 1 -- 172.21.15.66:0/3353478586 shutdown_connections 2024-09-10T22:08:23.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.569+0000 7f028ae00640 1 -- 172.21.15.66:0/3353478586 wait complete. 2024-09-10T22:08:23.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028ae00640 1 Processor -- start 2024-09-10T22:08:23.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028ae00640 1 -- start start 2024-09-10T22:08:23.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028ae00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02840b15d0 0x7f02840b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:23.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f028415ef20 0x7f028415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:23.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028ae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f028415f300 0x7f02840cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:23.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0284168d60 con 0x7f028415ef20 2024-09-10T22:08:23.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028ae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0284168ee0 con 0x7f028415f300 2024-09-10T22:08:23.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028ae00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0284169060 con 0x7f02840b15d0 2024-09-10T22:08:23.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0289000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f028415ef20 0x7f028415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:23.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0289000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f028415ef20 0x7f028415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42142/0 (socket says 172.21.15.66:42142) 2024-09-10T22:08:23.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0289000640 1 -- 172.21.15.66:0/2489872402 learned_addr learned my addr 172.21.15.66:0/2489872402 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:23.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0289a00640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02840b15d0 0x7f02840b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:23.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028a400640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f028415f300 0x7f02840cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:23.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0289000640 1 -- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02840b15d0 msgr2=0x7f02840b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:23.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0289000640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02840b15d0 0x7f02840b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0289000640 1 -- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f028415f300 msgr2=0x7f02840cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:23.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0289000640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f028415f300 0x7f02840cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0289000640 1 -- 172.21.15.66:0/2489872402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02840b7900 con 0x7f028415ef20 2024-09-10T22:08:23.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028a400640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f028415f300 0x7f02840cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:08:23.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0289000640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f028415ef20 0x7f028415b1c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f027000e7c0 tx=0x7f027000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:23.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0281600640 1 -- 172.21.15.66:0/2489872402 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027000cc60 con 0x7f028415ef20 2024-09-10T22:08:23.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0281600640 1 -- 172.21.15.66:0/2489872402 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f027000ce00 con 0x7f028415ef20 2024-09-10T22:08:23.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f0281600640 1 -- 172.21.15.66:0/2489872402 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270005280 con 0x7f028415ef20 2024-09-10T22:08:23.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028ae00640 1 -- 172.21.15.66:0/2489872402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028415b700 con 0x7f028415ef20 2024-09-10T22:08:23.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028ae00640 1 -- 172.21.15.66:0/2489872402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02840c98b0 con 0x7f028415ef20 2024-09-10T22:08:23.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.573+0000 7f028ae00640 1 -- 172.21.15.66:0/2489872402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02840cecc0 con 0x7f028415ef20 2024-09-10T22:08:23.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.577+0000 7f0281600640 1 -- 172.21.15.66:0/2489872402 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f0270005420 con 0x7f028415ef20 2024-09-10T22:08:23.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.577+0000 7f0281600640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f02640736c0 0x7f0264075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:23.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.577+0000 7f0281600640 1 -- 172.21.15.66:0/2489872402 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f0270092b20 con 0x7f028415ef20 2024-09-10T22:08:23.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.577+0000 7f0289a00640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f02640736c0 0x7f0264075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:23.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.577+0000 7f0289a00640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f02640736c0 0x7f0264075b80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f02780af000 tx=0x7f027801aff0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:23.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.581+0000 7f0281600640 1 -- 172.21.15.66:0/2489872402 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f02840b7900 con 0x7f028415ef20 2024-09-10T22:08:23.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 -- 172.21.15.66:0/2489872402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0284072e00 con 0x7f028415ef20 2024-09-10T22:08:23.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f0281600640 1 -- 172.21.15.66:0/2489872402 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5222 (secure 0 0 0) 0x7f027005f910 con 0x7f028415ef20 2024-09-10T22:08:23.795 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:23.795 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":56,"btime":"2024-09-10T22:08:23:391913+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3360242011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3360242011},{"type":"v1","addr":"172.21.15.94:6837","nonce":3360242011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3211574496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3211574496},{"type":"v1","addr":"172.21.15.66:6835","nonce":3211574496}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/553876835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":553876835},{"type":"v1","addr":"172.21.15.66:6834","nonce":553876835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:08:19.775657+0000","modified":"2024-09-10T22:08:22.793838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6836/3187502512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3187502512},{"type":"v1","addr":"172.21.15.94:6836","nonce":3187502512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":6}]} 2024-09-10T22:08:23.796 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 56 2024-09-10T22:08:23.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 -- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f02640736c0 msgr2=0x7f0264075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:23.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f02640736c0 0x7f0264075b80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f02780af000 tx=0x7f027801aff0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 -- 172.21.15.66:0/2489872402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f028415ef20 msgr2=0x7f028415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:23.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f028415ef20 0x7f028415b1c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f027000e7c0 tx=0x7f027000ec80 comp rx=0 tx=0).stop 2024-09-10T22:08:23.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 -- 172.21.15.66:0/2489872402 shutdown_connections 2024-09-10T22:08:23.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f02640736c0 0x7f0264075b80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f028415f300 0x7f02840cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f028415ef20 0x7f028415b1c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 --2- 172.21.15.66:0/2489872402 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02840b15d0 0x7f02840b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 -- 172.21.15.66:0/2489872402 >> 172.21.15.66:0/2489872402 conn(0x7f02840d1b10 msgr2=0x7f0284166120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:23.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 -- 172.21.15.66:0/2489872402 shutdown_connections 2024-09-10T22:08:23.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.789+0000 7f028ae00640 1 -- 172.21.15.66:0/2489872402 wait complete. 2024-09-10T22:08:23.817 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:08:23.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.957+0000 7fc359c00640 1 -- 172.21.15.66:0/3877882649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc354159900 msgr2=0x7fc3540ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:23.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.957+0000 7fc359c00640 1 --2- 172.21.15.66:0/3877882649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc354159900 0x7fc3540ca080 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc344009900 tx=0x7fc34401c850 comp rx=0 tx=0).stop 2024-09-10T22:08:23.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.957+0000 7fc359c00640 1 -- 172.21.15.66:0/3877882649 shutdown_connections 2024-09-10T22:08:23.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.957+0000 7fc359c00640 1 --2- 172.21.15.66:0/3877882649 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc35415cc80 0x7fc3540ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.957+0000 7fc359c00640 1 --2- 172.21.15.66:0/3877882649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc354159900 0x7fc3540ca080 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.957+0000 7fc359c00640 1 -- 172.21.15.66:0/3877882649 >> 172.21.15.66:0/3877882649 conn(0x7fc3540cfa30 msgr2=0x7fc3540cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:23.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.957+0000 7fc359c00640 1 -- 172.21.15.66:0/3877882649 shutdown_connections 2024-09-10T22:08:23.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.957+0000 7fc359c00640 1 -- 172.21.15.66:0/3877882649 wait complete. 2024-09-10T22:08:23.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.957+0000 7fc359c00640 1 Processor -- start 2024-09-10T22:08:23.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.957+0000 7fc359c00640 1 -- start start 2024-09-10T22:08:23.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc359c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3540aec50 0x7fc3540b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:23.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc359c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc354159900 0x7fc35415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:23.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc359c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc35415cc80 0x7fc3541f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:23.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc359c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc354168f70 con 0x7fc3540aec50 2024-09-10T22:08:23.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc359c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc354168df0 con 0x7fc354159900 2024-09-10T22:08:23.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc359c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc3541690f0 con 0x7fc35415cc80 2024-09-10T22:08:23.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc352a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3540aec50 0x7fc3540b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:23.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc352a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3540aec50 0x7fc3540b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42162/0 (socket says 172.21.15.66:42162) 2024-09-10T22:08:23.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc352a00640 1 -- 172.21.15.66:0/4028938589 learned_addr learned my addr 172.21.15.66:0/4028938589 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:23.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc34be00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc354159900 0x7fc35415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:23.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc353400640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc35415cc80 0x7fc3541f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:23.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc352a00640 1 -- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc35415cc80 msgr2=0x7fc3541f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:23.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc352a00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc35415cc80 0x7fc3541f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc352a00640 1 -- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc354159900 msgr2=0x7fc35415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:23.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc352a00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc354159900 0x7fc35415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:23.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc352a00640 1 -- 172.21.15.66:0/4028938589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3540b4d90 con 0x7fc3540aec50 2024-09-10T22:08:23.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc352a00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3540aec50 0x7fc3540b2db0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc34401a480 tx=0x7fc3440a6ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:23.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc34aa00640 1 -- 172.21.15.66:0/4028938589 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3440ac070 con 0x7fc3540aec50 2024-09-10T22:08:23.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc34aa00640 1 -- 172.21.15.66:0/4028938589 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc344004030 con 0x7fc3540aec50 2024-09-10T22:08:23.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc34aa00640 1 -- 172.21.15.66:0/4028938589 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344004480 con 0x7fc3540aec50 2024-09-10T22:08:23.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc359c00640 1 -- 172.21.15.66:0/4028938589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3540b5070 con 0x7fc3540aec50 2024-09-10T22:08:23.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc359c00640 1 -- 172.21.15.66:0/4028938589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3540b5630 con 0x7fc3540aec50 2024-09-10T22:08:23.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.961+0000 7fc359c00640 1 -- 172.21.15.66:0/4028938589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3541fb0a0 con 0x7fc3540aec50 2024-09-10T22:08:23.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.965+0000 7fc34aa00640 1 -- 172.21.15.66:0/4028938589 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fc3440b8020 con 0x7fc3540aec50 2024-09-10T22:08:23.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.965+0000 7fc34aa00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3300736c0 0x7fc330075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:23.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.965+0000 7fc34aa00640 1 -- 172.21.15.66:0/4028938589 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fc34412b910 con 0x7fc3540aec50 2024-09-10T22:08:23.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.965+0000 7fc34be00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3300736c0 0x7fc330075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:23.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.965+0000 7fc34be00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3300736c0 0x7fc330075b80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc3540adcf0 tx=0x7fc33c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:23.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:23.969+0000 7fc34aa00640 1 -- 172.21.15.66:0/4028938589 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc3541fb0a0 con 0x7fc3540aec50 2024-09-10T22:08:24.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.141+0000 7fc359c00640 1 -- 172.21.15.66:0/4028938589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3540ca080 con 0x7fc3540aec50 2024-09-10T22:08:24.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc34aa00640 1 -- 172.21.15.66:0/4028938589 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16736 (secure 0 0 0) 0x7fc3440bf070 con 0x7fc3540aec50 2024-09-10T22:08:24.151 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:24.151 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":85,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:08:21.375302+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:08:17.494722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T22:08:18.745603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:08:24.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc359c00640 1 -- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3300736c0 msgr2=0x7fc330075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:24.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc359c00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3300736c0 0x7fc330075b80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc3540adcf0 tx=0x7fc33c0091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc359c00640 1 -- 172.21.15.66:0/4028938589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3540aec50 msgr2=0x7fc3540b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:24.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc359c00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3540aec50 0x7fc3540b2db0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc34401a480 tx=0x7fc3440a6ac0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc359c00640 1 -- 172.21.15.66:0/4028938589 shutdown_connections 2024-09-10T22:08:24.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc359c00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc3300736c0 0x7fc330075b80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc359c00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc35415cc80 0x7fc3541f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc359c00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc354159900 0x7fc35415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc359c00640 1 --2- 172.21.15.66:0/4028938589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3540aec50 0x7fc3540b2db0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc359c00640 1 -- 172.21.15.66:0/4028938589 >> 172.21.15.66:0/4028938589 conn(0x7fc3540cfa30 msgr2=0x7fc354072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:24.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc359c00640 1 -- 172.21.15.66:0/4028938589 shutdown_connections 2024-09-10T22:08:24.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.145+0000 7fc359c00640 1 -- 172.21.15.66:0/4028938589 wait complete. 2024-09-10T22:08:24.173 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:08:24.173 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:08:24.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.313+0000 7f41ea200640 1 -- 172.21.15.66:0/3757034769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e415f460 msgr2=0x7f41e40cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:24.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.313+0000 7f41ea200640 1 --2- 172.21.15.66:0/3757034769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e415f460 0x7f41e40cfc00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009900 tx=0x7f41d401c730 comp rx=0 tx=0).stop 2024-09-10T22:08:24.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.313+0000 7f41ea200640 1 -- 172.21.15.66:0/3757034769 shutdown_connections 2024-09-10T22:08:24.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.313+0000 7f41ea200640 1 --2- 172.21.15.66:0/3757034769 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41e40b32a0 0x7f41e40af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.313+0000 7f41ea200640 1 --2- 172.21.15.66:0/3757034769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e415f460 0x7f41e40cfc00 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.313+0000 7f41ea200640 1 -- 172.21.15.66:0/3757034769 >> 172.21.15.66:0/3757034769 conn(0x7f41e40d1d00 msgr2=0x7f41e40d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:24.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.313+0000 7f41ea200640 1 -- 172.21.15.66:0/3757034769 shutdown_connections 2024-09-10T22:08:24.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.313+0000 7f41ea200640 1 -- 172.21.15.66:0/3757034769 wait complete. 2024-09-10T22:08:24.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.313+0000 7f41ea200640 1 Processor -- start 2024-09-10T22:08:24.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41ea200640 1 -- start start 2024-09-10T22:08:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41ea200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41e40b0f60 0x7f41e415c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41ea200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41e40b32a0 0x7f41e40c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41ea200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e415f460 0x7f41e4070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41ea200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41e40b0600 con 0x7f41e415f460 2024-09-10T22:08:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41ea200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41e40b0480 con 0x7f41e40b0f60 2024-09-10T22:08:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41ea200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f41e4168c60 con 0x7f41e40b32a0 2024-09-10T22:08:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41e3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e415f460 0x7f41e4070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41e3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e415f460 0x7f41e4070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42186/0 (socket says 172.21.15.66:42186) 2024-09-10T22:08:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41e3e00640 1 -- 172.21.15.66:0/1228648448 learned_addr learned my addr 172.21.15.66:0/1228648448 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41e3400640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41e40b0f60 0x7f41e415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41dbe00640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41e40b32a0 0x7f41e40c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41e3e00640 1 -- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41e40b32a0 msgr2=0x7f41e40c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41e3e00640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41e40b32a0 0x7f41e40c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41e3e00640 1 -- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41e40b0f60 msgr2=0x7f41e415c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:24.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41e3e00640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41e40b0f60 0x7f41e415c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41e3e00640 1 -- 172.21.15.66:0/1228648448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e406e7a0 con 0x7f41e415f460 2024-09-10T22:08:24.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41e3e00640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e415f460 0x7f41e4070ac0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00ea20 tx=0x7f41dc00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:24.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41db400640 1 -- 172.21.15.66:0/1228648448 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00cde0 con 0x7f41e415f460 2024-09-10T22:08:24.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41db400640 1 -- 172.21.15.66:0/1228648448 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41dc004c40 con 0x7f41e415f460 2024-09-10T22:08:24.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41ea200640 1 -- 172.21.15.66:0/1228648448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e41fb4b0 con 0x7f41e415f460 2024-09-10T22:08:24.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41db400640 1 -- 172.21.15.66:0/1228648448 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc005280 con 0x7f41e415f460 2024-09-10T22:08:24.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41ea200640 1 -- 172.21.15.66:0/1228648448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e41fbd50 con 0x7f41e415f460 2024-09-10T22:08:24.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.317+0000 7f41ea200640 1 -- 172.21.15.66:0/1228648448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e40af390 con 0x7f41e415f460 2024-09-10T22:08:24.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.321+0000 7f41db400640 1 -- 172.21.15.66:0/1228648448 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f41dc00f040 con 0x7f41e415f460 2024-09-10T22:08:24.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.321+0000 7f41db400640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41c00736f0 0x7f41c0075bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:24.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.321+0000 7f41db400640 1 -- 172.21.15.66:0/1228648448 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f41dc092310 con 0x7f41e415f460 2024-09-10T22:08:24.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.321+0000 7f41e3400640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41c00736f0 0x7f41c0075bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:24.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.321+0000 7f41e3400640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41c00736f0 0x7f41c0075bb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f41d40af000 tx=0x7f41d401afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:24.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.325+0000 7f41db400640 1 -- 172.21.15.66:0/1228648448 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f41dc09c080 con 0x7f41e415f460 2024-09-10T22:08:24.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.497+0000 7f41ea200640 1 -- 172.21.15.66:0/1228648448 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41e4074aa0 con 0x7f41c00736f0 2024-09-10T22:08:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.501+0000 7f41db400640 1 -- 172.21.15.66:0/1228648448 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252081 (secure 0 0 0) 0x7f41e4074aa0 con 0x7f41c00736f0 2024-09-10T22:08:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:24.514 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:08:24.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.505+0000 7f41ea200640 1 -- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41c00736f0 msgr2=0x7f41c0075bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:24.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.505+0000 7f41ea200640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41c00736f0 0x7f41c0075bb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f41d40af000 tx=0x7f41d401afd0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.505+0000 7f41ea200640 1 -- 172.21.15.66:0/1228648448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e415f460 msgr2=0x7f41e4070ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:24.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.505+0000 7f41ea200640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e415f460 0x7f41e4070ac0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00ea20 tx=0x7f41dc00eee0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.505+0000 7f41ea200640 1 -- 172.21.15.66:0/1228648448 shutdown_connections 2024-09-10T22:08:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.505+0000 7f41ea200640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41c00736f0 0x7f41c0075bb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.505+0000 7f41ea200640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e415f460 0x7f41e4070ac0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.505+0000 7f41ea200640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41e40b32a0 0x7f41e40c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.505+0000 7f41ea200640 1 --2- 172.21.15.66:0/1228648448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41e40b0f60 0x7f41e415c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.505+0000 7f41ea200640 1 -- 172.21.15.66:0/1228648448 >> 172.21.15.66:0/1228648448 conn(0x7f41e40d1d00 msgr2=0x7f41e406df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.505+0000 7f41ea200640 1 -- 172.21.15.66:0/1228648448 shutdown_connections 2024-09-10T22:08:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.505+0000 7f41ea200640 1 -- 172.21.15.66:0/1228648448 wait complete. 2024-09-10T22:08:24.533 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-10T22:08:24.437654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":472488,"kb_used_data":9308,"kb_used_omap":12,"kb_used_meta":462963,"kb_avail":754502232,"statfs":{"total":773094113280,"available":772610285568,"internally_reserved":0,"allocated":9531392,"data_stored":4723048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":474074676},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001442"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421860+0000","last_change":"2024-09-10T22:08:20.404492+0000","last_active":"2024-09-10T22:08:21.421860+0000","last_peered":"2024-09-10T22:08:21.421860+0000","last_clean":"2024-09-10T22:08:21.421860+0000","last_became_active":"2024-09-10T22:08:20.404111+0000","last_became_peered":"2024-09-10T22:08:20.404111+0000","last_unstale":"2024-09-10T22:08:21.421860+0000","last_undegraded":"2024-09-10T22:08:21.421860+0000","last_fullsized":"2024-09-10T22:08:21.421860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:36.965088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418122+0000","last_change":"2024-09-10T22:08:20.404842+0000","last_active":"2024-09-10T22:08:21.418122+0000","last_peered":"2024-09-10T22:08:21.418122+0000","last_clean":"2024-09-10T22:08:21.418122+0000","last_became_active":"2024-09-10T22:08:20.404635+0000","last_became_peered":"2024-09-10T22:08:20.404635+0000","last_unstale":"2024-09-10T22:08:21.418122+0000","last_undegraded":"2024-09-10T22:08:21.418122+0000","last_fullsized":"2024-09-10T22:08:21.418122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:50.283801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388789+0000","last_change":"2024-09-10T22:08:20.397882+0000","last_active":"2024-09-10T22:08:21.388789+0000","last_peered":"2024-09-10T22:08:21.388789+0000","last_clean":"2024-09-10T22:08:21.388789+0000","last_became_active":"2024-09-10T22:08:20.397590+0000","last_became_peered":"2024-09-10T22:08:20.397590+0000","last_unstale":"2024-09-10T22:08:21.388789+0000","last_undegraded":"2024-09-10T22:08:21.388789+0000","last_fullsized":"2024-09-10T22:08:21.388789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:15.201174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.392068+0000","last_change":"2024-09-10T22:08:20.385143+0000","last_active":"2024-09-10T22:08:20.392068+0000","last_peered":"2024-09-10T22:08:20.392068+0000","last_clean":"2024-09-10T22:08:20.392068+0000","last_became_active":"2024-09-10T22:08:20.384842+0000","last_became_peered":"2024-09-10T22:08:20.384842+0000","last_unstale":"2024-09-10T22:08:20.392068+0000","last_undegraded":"2024-09-10T22:08:20.392068+0000","last_fullsized":"2024-09-10T22:08:20.392068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:01.935933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420827+0000","last_change":"2024-09-10T22:08:20.397875+0000","last_active":"2024-09-10T22:08:21.420827+0000","last_peered":"2024-09-10T22:08:21.420827+0000","last_clean":"2024-09-10T22:08:21.420827+0000","last_became_active":"2024-09-10T22:08:20.397513+0000","last_became_peered":"2024-09-10T22:08:20.397513+0000","last_unstale":"2024-09-10T22:08:21.420827+0000","last_undegraded":"2024-09-10T22:08:21.420827+0000","last_fullsized":"2024-09-10T22:08:21.420827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:44.962036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388116+0000","last_change":"2024-09-10T22:08:20.397096+0000","last_active":"2024-09-10T22:08:21.388116+0000","last_peered":"2024-09-10T22:08:21.388116+0000","last_clean":"2024-09-10T22:08:21.388116+0000","last_became_active":"2024-09-10T22:08:20.396527+0000","last_became_peered":"2024-09-10T22:08:20.396527+0000","last_unstale":"2024-09-10T22:08:21.388116+0000","last_undegraded":"2024-09-10T22:08:21.388116+0000","last_fullsized":"2024-09-10T22:08:21.388116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:59.646735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.394206+0000","last_change":"2024-09-10T22:08:20.387260+0000","last_active":"2024-09-10T22:08:20.394206+0000","last_peered":"2024-09-10T22:08:20.394206+0000","last_clean":"2024-09-10T22:08:20.394206+0000","last_became_active":"2024-09-10T22:08:20.386961+0000","last_became_peered":"2024-09-10T22:08:20.386961+0000","last_unstale":"2024-09-10T22:08:20.394206+0000","last_undegraded":"2024-09-10T22:08:20.394206+0000","last_fullsized":"2024-09-10T22:08:20.394206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:32.021077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.387924+0000","last_change":"2024-09-10T22:08:20.390994+0000","last_active":"2024-09-10T22:08:21.387924+0000","last_peered":"2024-09-10T22:08:21.387924+0000","last_clean":"2024-09-10T22:08:21.387924+0000","last_became_active":"2024-09-10T22:08:20.390722+0000","last_became_peered":"2024-09-10T22:08:20.390722+0000","last_unstale":"2024-09-10T22:08:21.387924+0000","last_undegraded":"2024-09-10T22:08:21.387924+0000","last_fullsized":"2024-09-10T22:08:21.387924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:56.171917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389040+0000","last_change":"2024-09-10T22:08:20.388810+0000","last_active":"2024-09-10T22:08:21.389040+0000","last_peered":"2024-09-10T22:08:21.389040+0000","last_clean":"2024-09-10T22:08:21.389040+0000","last_became_active":"2024-09-10T22:08:20.388544+0000","last_became_peered":"2024-09-10T22:08:20.388544+0000","last_unstale":"2024-09-10T22:08:21.389040+0000","last_undegraded":"2024-09-10T22:08:21.389040+0000","last_fullsized":"2024-09-10T22:08:21.389040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:36.538229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421769+0000","last_change":"2024-09-10T22:08:20.401994+0000","last_active":"2024-09-10T22:08:21.421769+0000","last_peered":"2024-09-10T22:08:21.421769+0000","last_clean":"2024-09-10T22:08:21.421769+0000","last_became_active":"2024-09-10T22:08:20.399202+0000","last_became_peered":"2024-09-10T22:08:20.399202+0000","last_unstale":"2024-09-10T22:08:21.421769+0000","last_undegraded":"2024-09-10T22:08:21.421769+0000","last_fullsized":"2024-09-10T22:08:21.421769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:45.376214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418787+0000","last_change":"2024-09-10T22:08:20.386662+0000","last_active":"2024-09-10T22:08:21.418787+0000","last_peered":"2024-09-10T22:08:21.418787+0000","last_clean":"2024-09-10T22:08:21.418787+0000","last_became_active":"2024-09-10T22:08:20.386345+0000","last_became_peered":"2024-09-10T22:08:20.386345+0000","last_unstale":"2024-09-10T22:08:21.418787+0000","last_undegraded":"2024-09-10T22:08:21.418787+0000","last_fullsized":"2024-09-10T22:08:21.418787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:54.956555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421853+0000","last_change":"2024-09-10T22:08:20.400052+0000","last_active":"2024-09-10T22:08:21.421853+0000","last_peered":"2024-09-10T22:08:21.421853+0000","last_clean":"2024-09-10T22:08:21.421853+0000","last_became_active":"2024-09-10T22:08:20.399281+0000","last_became_peered":"2024-09-10T22:08:20.399281+0000","last_unstale":"2024-09-10T22:08:21.421853+0000","last_undegraded":"2024-09-10T22:08:21.421853+0000","last_fullsized":"2024-09-10T22:08:21.421853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:37.083764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389156+0000","last_change":"2024-09-10T22:08:20.387024+0000","last_active":"2024-09-10T22:08:21.389156+0000","last_peered":"2024-09-10T22:08:21.389156+0000","last_clean":"2024-09-10T22:08:21.389156+0000","last_became_active":"2024-09-10T22:08:20.386713+0000","last_became_peered":"2024-09-10T22:08:20.386713+0000","last_unstale":"2024-09-10T22:08:21.389156+0000","last_undegraded":"2024-09-10T22:08:21.389156+0000","last_fullsized":"2024-09-10T22:08:21.389156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:23.083038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.395135+0000","last_change":"2024-09-10T22:08:20.388339+0000","last_active":"2024-09-10T22:08:20.395135+0000","last_peered":"2024-09-10T22:08:20.395135+0000","last_clean":"2024-09-10T22:08:20.395135+0000","last_became_active":"2024-09-10T22:08:20.387900+0000","last_became_peered":"2024-09-10T22:08:20.387900+0000","last_unstale":"2024-09-10T22:08:20.395135+0000","last_undegraded":"2024-09-10T22:08:20.395135+0000","last_fullsized":"2024-09-10T22:08:20.395135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:09:03.735880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418866+0000","last_change":"2024-09-10T22:08:20.390422+0000","last_active":"2024-09-10T22:08:21.418866+0000","last_peered":"2024-09-10T22:08:21.418866+0000","last_clean":"2024-09-10T22:08:21.418866+0000","last_became_active":"2024-09-10T22:08:20.389809+0000","last_became_peered":"2024-09-10T22:08:20.389809+0000","last_unstale":"2024-09-10T22:08:21.418866+0000","last_undegraded":"2024-09-10T22:08:21.418866+0000","last_fullsized":"2024-09-10T22:08:21.418866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:45.292576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420568+0000","last_change":"2024-09-10T22:08:20.398103+0000","last_active":"2024-09-10T22:08:21.420568+0000","last_peered":"2024-09-10T22:08:21.420568+0000","last_clean":"2024-09-10T22:08:21.420568+0000","last_became_active":"2024-09-10T22:08:20.397670+0000","last_became_peered":"2024-09-10T22:08:20.397670+0000","last_unstale":"2024-09-10T22:08:21.420568+0000","last_undegraded":"2024-09-10T22:08:21.420568+0000","last_fullsized":"2024-09-10T22:08:21.420568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:24.713807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421297+0000","last_change":"2024-09-10T22:08:20.392928+0000","last_active":"2024-09-10T22:08:21.421297+0000","last_peered":"2024-09-10T22:08:21.421297+0000","last_clean":"2024-09-10T22:08:21.421297+0000","last_became_active":"2024-09-10T22:08:20.392250+0000","last_became_peered":"2024-09-10T22:08:20.392250+0000","last_unstale":"2024-09-10T22:08:21.421297+0000","last_undegraded":"2024-09-10T22:08:21.421297+0000","last_fullsized":"2024-09-10T22:08:21.421297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:41.724028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.400133+0000","last_change":"2024-09-10T22:08:20.393671+0000","last_active":"2024-09-10T22:08:20.400133+0000","last_peered":"2024-09-10T22:08:20.400133+0000","last_clean":"2024-09-10T22:08:20.400133+0000","last_became_active":"2024-09-10T22:08:20.393403+0000","last_became_peered":"2024-09-10T22:08:20.393403+0000","last_unstale":"2024-09-10T22:08:20.400133+0000","last_undegraded":"2024-09-10T22:08:20.400133+0000","last_fullsized":"2024-09-10T22:08:20.400133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:12.408747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.419376+0000","last_change":"2024-09-10T22:08:20.384344+0000","last_active":"2024-09-10T22:08:21.419376+0000","last_peered":"2024-09-10T22:08:21.419376+0000","last_clean":"2024-09-10T22:08:21.419376+0000","last_became_active":"2024-09-10T22:08:20.384097+0000","last_became_peered":"2024-09-10T22:08:20.384097+0000","last_unstale":"2024-09-10T22:08:21.419376+0000","last_undegraded":"2024-09-10T22:08:21.419376+0000","last_fullsized":"2024-09-10T22:08:21.419376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:15.201174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.395930+0000","last_change":"2024-09-10T22:08:20.389239+0000","last_active":"2024-09-10T22:08:20.395930+0000","last_peered":"2024-09-10T22:08:20.395930+0000","last_clean":"2024-09-10T22:08:20.395930+0000","last_became_active":"2024-09-10T22:08:20.388915+0000","last_became_peered":"2024-09-10T22:08:20.388915+0000","last_unstale":"2024-09-10T22:08:20.395930+0000","last_undegraded":"2024-09-10T22:08:20.395930+0000","last_fullsized":"2024-09-10T22:08:20.395930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:28.209965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.399248+0000","last_change":"2024-09-10T22:08:20.391351+0000","last_active":"2024-09-10T22:08:20.399248+0000","last_peered":"2024-09-10T22:08:20.399248+0000","last_clean":"2024-09-10T22:08:20.399248+0000","last_became_active":"2024-09-10T22:08:20.391077+0000","last_became_peered":"2024-09-10T22:08:20.391077+0000","last_unstale":"2024-09-10T22:08:20.399248+0000","last_undegraded":"2024-09-10T22:08:20.399248+0000","last_fullsized":"2024-09-10T22:08:20.399248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:24.427232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417482+0000","last_change":"2024-09-10T22:08:20.389584+0000","last_active":"2024-09-10T22:08:21.417482+0000","last_peered":"2024-09-10T22:08:21.417482+0000","last_clean":"2024-09-10T22:08:21.417482+0000","last_became_active":"2024-09-10T22:08:20.389241+0000","last_became_peered":"2024-09-10T22:08:20.389241+0000","last_unstale":"2024-09-10T22:08:21.417482+0000","last_undegraded":"2024-09-10T22:08:21.417482+0000","last_fullsized":"2024-09-10T22:08:21.417482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:44.998796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421474+0000","last_change":"2024-09-10T22:08:20.398880+0000","last_active":"2024-09-10T22:08:21.421474+0000","last_peered":"2024-09-10T22:08:21.421474+0000","last_clean":"2024-09-10T22:08:21.421474+0000","last_became_active":"2024-09-10T22:08:20.396815+0000","last_became_peered":"2024-09-10T22:08:20.396815+0000","last_unstale":"2024-09-10T22:08:21.421474+0000","last_undegraded":"2024-09-10T22:08:21.421474+0000","last_fullsized":"2024-09-10T22:08:21.421474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:39.381939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.393067+0000","last_change":"2024-09-10T22:08:20.386430+0000","last_active":"2024-09-10T22:08:20.393067+0000","last_peered":"2024-09-10T22:08:20.393067+0000","last_clean":"2024-09-10T22:08:20.393067+0000","last_became_active":"2024-09-10T22:08:20.386181+0000","last_became_peered":"2024-09-10T22:08:20.386181+0000","last_unstale":"2024-09-10T22:08:20.393067+0000","last_undegraded":"2024-09-10T22:08:20.393067+0000","last_fullsized":"2024-09-10T22:08:20.393067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:54.442771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418131+0000","last_change":"2024-09-10T22:08:20.405318+0000","last_active":"2024-09-10T22:08:21.418131+0000","last_peered":"2024-09-10T22:08:21.418131+0000","last_clean":"2024-09-10T22:08:21.418131+0000","last_became_active":"2024-09-10T22:08:20.404288+0000","last_became_peered":"2024-09-10T22:08:20.404288+0000","last_unstale":"2024-09-10T22:08:21.418131+0000","last_undegraded":"2024-09-10T22:08:21.418131+0000","last_fullsized":"2024-09-10T22:08:21.418131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:46.114623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389420+0000","last_change":"2024-09-10T22:08:20.395279+0000","last_active":"2024-09-10T22:08:21.389420+0000","last_peered":"2024-09-10T22:08:21.389420+0000","last_clean":"2024-09-10T22:08:21.389420+0000","last_became_active":"2024-09-10T22:08:20.394954+0000","last_became_peered":"2024-09-10T22:08:20.394954+0000","last_unstale":"2024-09-10T22:08:21.389420+0000","last_undegraded":"2024-09-10T22:08:21.389420+0000","last_fullsized":"2024-09-10T22:08:21.389420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:56.845631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420915+0000","last_change":"2024-09-10T22:08:20.396652+0000","last_active":"2024-09-10T22:08:21.420915+0000","last_peered":"2024-09-10T22:08:21.420915+0000","last_clean":"2024-09-10T22:08:21.420915+0000","last_became_active":"2024-09-10T22:08:20.393903+0000","last_became_peered":"2024-09-10T22:08:20.393903+0000","last_unstale":"2024-09-10T22:08:21.420915+0000","last_undegraded":"2024-09-10T22:08:21.420915+0000","last_fullsized":"2024-09-10T22:08:21.420915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:39.853371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421595+0000","last_change":"2024-09-10T22:08:20.403589+0000","last_active":"2024-09-10T22:08:21.421595+0000","last_peered":"2024-09-10T22:08:21.421595+0000","last_clean":"2024-09-10T22:08:21.421595+0000","last_became_active":"2024-09-10T22:08:20.398045+0000","last_became_peered":"2024-09-10T22:08:20.398045+0000","last_unstale":"2024-09-10T22:08:21.421595+0000","last_undegraded":"2024-09-10T22:08:21.421595+0000","last_fullsized":"2024-09-10T22:08:21.421595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:20.977215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417719+0000","last_change":"2024-09-10T22:08:20.387470+0000","last_active":"2024-09-10T22:08:21.417719+0000","last_peered":"2024-09-10T22:08:21.417719+0000","last_clean":"2024-09-10T22:08:21.417719+0000","last_became_active":"2024-09-10T22:08:20.387160+0000","last_became_peered":"2024-09-10T22:08:20.387160+0000","last_unstale":"2024-09-10T22:08:21.417719+0000","last_undegraded":"2024-09-10T22:08:21.417719+0000","last_fullsized":"2024-09-10T22:08:21.417719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:06.472265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417597+0000","last_change":"2024-09-10T22:08:20.395935+0000","last_active":"2024-09-10T22:08:21.417597+0000","last_peered":"2024-09-10T22:08:21.417597+0000","last_clean":"2024-09-10T22:08:21.417597+0000","last_became_active":"2024-09-10T22:08:20.395483+0000","last_became_peered":"2024-09-10T22:08:20.395483+0000","last_unstale":"2024-09-10T22:08:21.417597+0000","last_undegraded":"2024-09-10T22:08:21.417597+0000","last_fullsized":"2024-09-10T22:08:21.417597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:12.906612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421067+0000","last_change":"2024-09-10T22:08:20.393478+0000","last_active":"2024-09-10T22:08:21.421067+0000","last_peered":"2024-09-10T22:08:21.421067+0000","last_clean":"2024-09-10T22:08:21.421067+0000","last_became_active":"2024-09-10T22:08:20.393193+0000","last_became_peered":"2024-09-10T22:08:20.393193+0000","last_unstale":"2024-09-10T22:08:21.421067+0000","last_undegraded":"2024-09-10T22:08:21.421067+0000","last_fullsized":"2024-09-10T22:08:21.421067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:53.151316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420962+0000","last_change":"2024-09-10T22:08:20.399665+0000","last_active":"2024-09-10T22:08:21.420962+0000","last_peered":"2024-09-10T22:08:21.420962+0000","last_clean":"2024-09-10T22:08:21.420962+0000","last_became_active":"2024-09-10T22:08:20.399327+0000","last_became_peered":"2024-09-10T22:08:20.399327+0000","last_unstale":"2024-09-10T22:08:21.420962+0000","last_undegraded":"2024-09-10T22:08:21.420962+0000","last_fullsized":"2024-09-10T22:08:21.420962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:59.063752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420869+0000","last_change":"2024-09-10T22:08:20.397480+0000","last_active":"2024-09-10T22:08:21.420869+0000","last_peered":"2024-09-10T22:08:21.420869+0000","last_clean":"2024-09-10T22:08:21.420869+0000","last_became_active":"2024-09-10T22:08:20.397225+0000","last_became_peered":"2024-09-10T22:08:20.397225+0000","last_unstale":"2024-09-10T22:08:21.420869+0000","last_undegraded":"2024-09-10T22:08:21.420869+0000","last_fullsized":"2024-09-10T22:08:21.420869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:36.965088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422002+0000","last_change":"2024-09-10T22:08:19.396127+0000","last_active":"2024-09-10T22:08:21.422002+0000","last_peered":"2024-09-10T22:08:21.422002+0000","last_clean":"2024-09-10T22:08:21.422002+0000","last_became_active":"2024-09-10T22:08:19.395865+0000","last_became_peered":"2024-09-10T22:08:19.395865+0000","last_unstale":"2024-09-10T22:08:21.422002+0000","last_undegraded":"2024-09-10T22:08:21.422002+0000","last_fullsized":"2024-09-10T22:08:21.422002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:15.667695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421848+0000","last_change":"2024-09-10T22:08:20.404141+0000","last_active":"2024-09-10T22:08:21.421848+0000","last_peered":"2024-09-10T22:08:21.421848+0000","last_clean":"2024-09-10T22:08:21.421848+0000","last_became_active":"2024-09-10T22:08:20.402992+0000","last_became_peered":"2024-09-10T22:08:20.402992+0000","last_unstale":"2024-09-10T22:08:21.421848+0000","last_undegraded":"2024-09-10T22:08:21.421848+0000","last_fullsized":"2024-09-10T22:08:21.421848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:45.059450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421090+0000","last_change":"2024-09-10T22:08:19.377851+0000","last_active":"2024-09-10T22:08:21.421090+0000","last_peered":"2024-09-10T22:08:21.421090+0000","last_clean":"2024-09-10T22:08:21.421090+0000","last_became_active":"2024-09-10T22:08:19.377586+0000","last_became_peered":"2024-09-10T22:08:19.377586+0000","last_unstale":"2024-09-10T22:08:21.421090+0000","last_undegraded":"2024-09-10T22:08:21.421090+0000","last_fullsized":"2024-09-10T22:08:21.421090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:27:42.338875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422063+0000","last_change":"2024-09-10T22:08:20.404336+0000","last_active":"2024-09-10T22:08:21.422063+0000","last_peered":"2024-09-10T22:08:21.422063+0000","last_clean":"2024-09-10T22:08:21.422063+0000","last_became_active":"2024-09-10T22:08:20.402184+0000","last_became_peered":"2024-09-10T22:08:20.402184+0000","last_unstale":"2024-09-10T22:08:21.422063+0000","last_undegraded":"2024-09-10T22:08:21.422063+0000","last_fullsized":"2024-09-10T22:08:21.422063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:56.865067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.419316+0000","last_change":"2024-09-10T22:08:19.385068+0000","last_active":"2024-09-10T22:08:21.419316+0000","last_peered":"2024-09-10T22:08:21.419316+0000","last_clean":"2024-09-10T22:08:21.419316+0000","last_became_active":"2024-09-10T22:08:19.384808+0000","last_became_peered":"2024-09-10T22:08:19.384808+0000","last_unstale":"2024-09-10T22:08:21.419316+0000","last_undegraded":"2024-09-10T22:08:21.419316+0000","last_fullsized":"2024-09-10T22:08:21.419316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:07.605108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388887+0000","last_change":"2024-09-10T22:08:20.393174+0000","last_active":"2024-09-10T22:08:21.388887+0000","last_peered":"2024-09-10T22:08:21.388887+0000","last_clean":"2024-09-10T22:08:21.388887+0000","last_became_active":"2024-09-10T22:08:20.392018+0000","last_became_peered":"2024-09-10T22:08:20.392018+0000","last_unstale":"2024-09-10T22:08:21.388887+0000","last_undegraded":"2024-09-10T22:08:21.388887+0000","last_fullsized":"2024-09-10T22:08:21.388887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:22.828177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418517+0000","last_change":"2024-09-10T22:08:19.382251+0000","last_active":"2024-09-10T22:08:21.418517+0000","last_peered":"2024-09-10T22:08:21.418517+0000","last_clean":"2024-09-10T22:08:21.418517+0000","last_became_active":"2024-09-10T22:08:19.381959+0000","last_became_peered":"2024-09-10T22:08:19.381959+0000","last_unstale":"2024-09-10T22:08:21.418517+0000","last_undegraded":"2024-09-10T22:08:21.418517+0000","last_fullsized":"2024-09-10T22:08:21.418517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:14.608934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417659+0000","last_change":"2024-09-10T22:08:20.396441+0000","last_active":"2024-09-10T22:08:21.417659+0000","last_peered":"2024-09-10T22:08:21.417659+0000","last_clean":"2024-09-10T22:08:21.417659+0000","last_became_active":"2024-09-10T22:08:20.396164+0000","last_became_peered":"2024-09-10T22:08:20.396164+0000","last_unstale":"2024-09-10T22:08:21.417659+0000","last_undegraded":"2024-09-10T22:08:21.417659+0000","last_fullsized":"2024-09-10T22:08:21.417659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:20.634464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421024+0000","last_change":"2024-09-10T22:08:19.383171+0000","last_active":"2024-09-10T22:08:21.421024+0000","last_peered":"2024-09-10T22:08:21.421024+0000","last_clean":"2024-09-10T22:08:21.421024+0000","last_became_active":"2024-09-10T22:08:19.382913+0000","last_became_peered":"2024-09-10T22:08:19.382913+0000","last_unstale":"2024-09-10T22:08:21.421024+0000","last_undegraded":"2024-09-10T22:08:21.421024+0000","last_fullsized":"2024-09-10T22:08:21.421024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:09.942929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420926+0000","last_change":"2024-09-10T22:08:20.400350+0000","last_active":"2024-09-10T22:08:21.420926+0000","last_peered":"2024-09-10T22:08:21.420926+0000","last_clean":"2024-09-10T22:08:21.420926+0000","last_became_active":"2024-09-10T22:08:20.398557+0000","last_became_peered":"2024-09-10T22:08:20.398557+0000","last_unstale":"2024-09-10T22:08:21.420926+0000","last_undegraded":"2024-09-10T22:08:21.420926+0000","last_fullsized":"2024-09-10T22:08:21.420926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:51.366407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420548+0000","last_change":"2024-09-10T22:08:19.379617+0000","last_active":"2024-09-10T22:08:21.420548+0000","last_peered":"2024-09-10T22:08:21.420548+0000","last_clean":"2024-09-10T22:08:21.420548+0000","last_became_active":"2024-09-10T22:08:19.379304+0000","last_became_peered":"2024-09-10T22:08:19.379304+0000","last_unstale":"2024-09-10T22:08:21.420548+0000","last_undegraded":"2024-09-10T22:08:21.420548+0000","last_fullsized":"2024-09-10T22:08:21.420548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:51.901590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388115+0000","last_change":"2024-09-10T22:08:20.387939+0000","last_active":"2024-09-10T22:08:21.388115+0000","last_peered":"2024-09-10T22:08:21.388115+0000","last_clean":"2024-09-10T22:08:21.388115+0000","last_became_active":"2024-09-10T22:08:20.387683+0000","last_became_peered":"2024-09-10T22:08:20.387683+0000","last_unstale":"2024-09-10T22:08:21.388115+0000","last_undegraded":"2024-09-10T22:08:21.388115+0000","last_fullsized":"2024-09-10T22:08:21.388115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:31.307804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417831+0000","last_change":"2024-09-10T22:08:19.373523+0000","last_active":"2024-09-10T22:08:21.417831+0000","last_peered":"2024-09-10T22:08:21.417831+0000","last_clean":"2024-09-10T22:08:21.417831+0000","last_became_active":"2024-09-10T22:08:19.373226+0000","last_became_peered":"2024-09-10T22:08:19.373226+0000","last_unstale":"2024-09-10T22:08:21.417831+0000","last_undegraded":"2024-09-10T22:08:21.417831+0000","last_fullsized":"2024-09-10T22:08:21.417831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:38.850072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418893+0000","last_change":"2024-09-10T22:08:20.390669+0000","last_active":"2024-09-10T22:08:21.418893+0000","last_peered":"2024-09-10T22:08:21.418893+0000","last_clean":"2024-09-10T22:08:21.418893+0000","last_became_active":"2024-09-10T22:08:20.390173+0000","last_became_peered":"2024-09-10T22:08:20.390173+0000","last_unstale":"2024-09-10T22:08:21.418893+0000","last_undegraded":"2024-09-10T22:08:21.418893+0000","last_fullsized":"2024-09-10T22:08:21.418893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:42.611617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417351+0000","last_change":"2024-09-10T22:08:19.380471+0000","last_active":"2024-09-10T22:08:21.417351+0000","last_peered":"2024-09-10T22:08:21.417351+0000","last_clean":"2024-09-10T22:08:21.417351+0000","last_became_active":"2024-09-10T22:08:19.379244+0000","last_became_peered":"2024-09-10T22:08:19.379244+0000","last_unstale":"2024-09-10T22:08:21.417351+0000","last_undegraded":"2024-09-10T22:08:21.417351+0000","last_fullsized":"2024-09-10T22:08:21.417351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:32.221509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.419276+0000","last_change":"2024-09-10T22:08:20.383095+0000","last_active":"2024-09-10T22:08:21.419276+0000","last_peered":"2024-09-10T22:08:21.419276+0000","last_clean":"2024-09-10T22:08:21.419276+0000","last_became_active":"2024-09-10T22:08:20.382811+0000","last_became_peered":"2024-09-10T22:08:20.382811+0000","last_unstale":"2024-09-10T22:08:21.419276+0000","last_undegraded":"2024-09-10T22:08:21.419276+0000","last_fullsized":"2024-09-10T22:08:21.419276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:33.630831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422111+0000","last_change":"2024-09-10T22:08:19.388513+0000","last_active":"2024-09-10T22:08:21.422111+0000","last_peered":"2024-09-10T22:08:21.422111+0000","last_clean":"2024-09-10T22:08:21.422111+0000","last_became_active":"2024-09-10T22:08:19.388253+0000","last_became_peered":"2024-09-10T22:08:19.388253+0000","last_unstale":"2024-09-10T22:08:21.422111+0000","last_undegraded":"2024-09-10T22:08:21.422111+0000","last_fullsized":"2024-09-10T22:08:21.422111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:58.672220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389639+0000","last_change":"2024-09-10T22:08:20.393304+0000","last_active":"2024-09-10T22:08:21.389639+0000","last_peered":"2024-09-10T22:08:21.389639+0000","last_clean":"2024-09-10T22:08:21.389639+0000","last_became_active":"2024-09-10T22:08:20.393016+0000","last_became_peered":"2024-09-10T22:08:20.393016+0000","last_unstale":"2024-09-10T22:08:21.389639+0000","last_undegraded":"2024-09-10T22:08:21.389639+0000","last_fullsized":"2024-09-10T22:08:21.389639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:12.016944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421714+0000","last_change":"2024-09-10T22:08:19.390129+0000","last_active":"2024-09-10T22:08:21.421714+0000","last_peered":"2024-09-10T22:08:21.421714+0000","last_clean":"2024-09-10T22:08:21.421714+0000","last_became_active":"2024-09-10T22:08:19.389393+0000","last_became_peered":"2024-09-10T22:08:19.389393+0000","last_unstale":"2024-09-10T22:08:21.421714+0000","last_undegraded":"2024-09-10T22:08:21.421714+0000","last_fullsized":"2024-09-10T22:08:21.421714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:45.887767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418455+0000","last_change":"2024-09-10T22:08:20.401232+0000","last_active":"2024-09-10T22:08:21.418455+0000","last_peered":"2024-09-10T22:08:21.418455+0000","last_clean":"2024-09-10T22:08:21.418455+0000","last_became_active":"2024-09-10T22:08:20.400916+0000","last_became_peered":"2024-09-10T22:08:20.400916+0000","last_unstale":"2024-09-10T22:08:21.418455+0000","last_undegraded":"2024-09-10T22:08:21.418455+0000","last_fullsized":"2024-09-10T22:08:21.418455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:37.889053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421850+0000","last_change":"2024-09-10T22:08:19.384198+0000","last_active":"2024-09-10T22:08:21.421850+0000","last_peered":"2024-09-10T22:08:21.421850+0000","last_clean":"2024-09-10T22:08:21.421850+0000","last_became_active":"2024-09-10T22:08:19.383891+0000","last_became_peered":"2024-09-10T22:08:19.383891+0000","last_unstale":"2024-09-10T22:08:21.421850+0000","last_undegraded":"2024-09-10T22:08:21.421850+0000","last_fullsized":"2024-09-10T22:08:21.421850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:55.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421122+0000","last_change":"2024-09-10T22:08:20.392699+0000","last_active":"2024-09-10T22:08:21.421122+0000","last_peered":"2024-09-10T22:08:21.421122+0000","last_clean":"2024-09-10T22:08:21.421122+0000","last_became_active":"2024-09-10T22:08:20.392423+0000","last_became_peered":"2024-09-10T22:08:20.392423+0000","last_unstale":"2024-09-10T22:08:21.421122+0000","last_undegraded":"2024-09-10T22:08:21.421122+0000","last_fullsized":"2024-09-10T22:08:21.421122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:42.171548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388852+0000","last_change":"2024-09-10T22:08:19.378809+0000","last_active":"2024-09-10T22:08:21.388852+0000","last_peered":"2024-09-10T22:08:21.388852+0000","last_clean":"2024-09-10T22:08:21.388852+0000","last_became_active":"2024-09-10T22:08:19.378103+0000","last_became_peered":"2024-09-10T22:08:19.378103+0000","last_unstale":"2024-09-10T22:08:21.388852+0000","last_undegraded":"2024-09-10T22:08:21.388852+0000","last_fullsized":"2024-09-10T22:08:21.388852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.162991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418107+0000","last_change":"2024-09-10T22:08:20.384026+0000","last_active":"2024-09-10T22:08:21.418107+0000","last_peered":"2024-09-10T22:08:21.418107+0000","last_clean":"2024-09-10T22:08:21.418107+0000","last_became_active":"2024-09-10T22:08:20.383640+0000","last_became_peered":"2024-09-10T22:08:20.383640+0000","last_unstale":"2024-09-10T22:08:21.418107+0000","last_undegraded":"2024-09-10T22:08:21.418107+0000","last_fullsized":"2024-09-10T22:08:21.418107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:43.825862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417655+0000","last_change":"2024-09-10T22:08:19.392580+0000","last_active":"2024-09-10T22:08:21.417655+0000","last_peered":"2024-09-10T22:08:21.417655+0000","last_clean":"2024-09-10T22:08:21.417655+0000","last_became_active":"2024-09-10T22:08:19.392294+0000","last_became_peered":"2024-09-10T22:08:19.392294+0000","last_unstale":"2024-09-10T22:08:21.417655+0000","last_undegraded":"2024-09-10T22:08:21.417655+0000","last_fullsized":"2024-09-10T22:08:21.417655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:35.245280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389379+0000","last_change":"2024-09-10T22:08:20.395871+0000","last_active":"2024-09-10T22:08:21.389379+0000","last_peered":"2024-09-10T22:08:21.389379+0000","last_clean":"2024-09-10T22:08:21.389379+0000","last_became_active":"2024-09-10T22:08:20.395620+0000","last_became_peered":"2024-09-10T22:08:20.395620+0000","last_unstale":"2024-09-10T22:08:21.389379+0000","last_undegraded":"2024-09-10T22:08:21.389379+0000","last_fullsized":"2024-09-10T22:08:21.389379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:38.019367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418904+0000","last_change":"2024-09-10T22:08:19.391439+0000","last_active":"2024-09-10T22:08:21.418904+0000","last_peered":"2024-09-10T22:08:21.418904+0000","last_clean":"2024-09-10T22:08:21.418904+0000","last_became_active":"2024-09-10T22:08:19.390589+0000","last_became_peered":"2024-09-10T22:08:19.390589+0000","last_unstale":"2024-09-10T22:08:21.418904+0000","last_undegraded":"2024-09-10T22:08:21.418904+0000","last_fullsized":"2024-09-10T22:08:21.418904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:28.192491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417403+0000","last_change":"2024-09-10T22:08:20.392365+0000","last_active":"2024-09-10T22:08:21.417403+0000","last_peered":"2024-09-10T22:08:21.417403+0000","last_clean":"2024-09-10T22:08:21.417403+0000","last_became_active":"2024-09-10T22:08:20.391859+0000","last_became_peered":"2024-09-10T22:08:20.391859+0000","last_unstale":"2024-09-10T22:08:21.417403+0000","last_undegraded":"2024-09-10T22:08:21.417403+0000","last_fullsized":"2024-09-10T22:08:21.417403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:14.296616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388026+0000","last_change":"2024-09-10T22:08:19.380051+0000","last_active":"2024-09-10T22:08:21.388026+0000","last_peered":"2024-09-10T22:08:21.388026+0000","last_clean":"2024-09-10T22:08:21.388026+0000","last_became_active":"2024-09-10T22:08:19.379794+0000","last_became_peered":"2024-09-10T22:08:19.379794+0000","last_unstale":"2024-09-10T22:08:21.388026+0000","last_undegraded":"2024-09-10T22:08:21.388026+0000","last_fullsized":"2024-09-10T22:08:21.388026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:58.060452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417406+0000","last_change":"2024-09-10T22:08:20.405143+0000","last_active":"2024-09-10T22:08:21.417406+0000","last_peered":"2024-09-10T22:08:21.417406+0000","last_clean":"2024-09-10T22:08:21.417406+0000","last_became_active":"2024-09-10T22:08:20.404264+0000","last_became_peered":"2024-09-10T22:08:20.404264+0000","last_unstale":"2024-09-10T22:08:21.417406+0000","last_undegraded":"2024-09-10T22:08:21.417406+0000","last_fullsized":"2024-09-10T22:08:21.417406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:39.698136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417041+0000","last_change":"2024-09-10T22:08:19.386983+0000","last_active":"2024-09-10T22:08:21.417041+0000","last_peered":"2024-09-10T22:08:21.417041+0000","last_clean":"2024-09-10T22:08:21.417041+0000","last_became_active":"2024-09-10T22:08:19.386678+0000","last_became_peered":"2024-09-10T22:08:19.386678+0000","last_unstale":"2024-09-10T22:08:21.417041+0000","last_undegraded":"2024-09-10T22:08:21.417041+0000","last_fullsized":"2024-09-10T22:08:21.417041+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:35.961788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389015+0000","last_change":"2024-09-10T22:08:20.390517+0000","last_active":"2024-09-10T22:08:21.389015+0000","last_peered":"2024-09-10T22:08:21.389015+0000","last_clean":"2024-09-10T22:08:21.389015+0000","last_became_active":"2024-09-10T22:08:20.389571+0000","last_became_peered":"2024-09-10T22:08:20.389571+0000","last_unstale":"2024-09-10T22:08:21.389015+0000","last_undegraded":"2024-09-10T22:08:21.389015+0000","last_fullsized":"2024-09-10T22:08:21.389015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:29.195791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422154+0000","last_change":"2024-09-10T22:08:19.387428+0000","last_active":"2024-09-10T22:08:21.422154+0000","last_peered":"2024-09-10T22:08:21.422154+0000","last_clean":"2024-09-10T22:08:21.422154+0000","last_became_active":"2024-09-10T22:08:19.387142+0000","last_became_peered":"2024-09-10T22:08:19.387142+0000","last_unstale":"2024-09-10T22:08:21.422154+0000","last_undegraded":"2024-09-10T22:08:21.422154+0000","last_fullsized":"2024-09-10T22:08:21.422154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:18.451468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422016+0000","last_change":"2024-09-10T22:08:20.400798+0000","last_active":"2024-09-10T22:08:21.422016+0000","last_peered":"2024-09-10T22:08:21.422016+0000","last_clean":"2024-09-10T22:08:21.422016+0000","last_became_active":"2024-09-10T22:08:20.400312+0000","last_became_peered":"2024-09-10T22:08:20.400312+0000","last_unstale":"2024-09-10T22:08:21.422016+0000","last_undegraded":"2024-09-10T22:08:21.422016+0000","last_fullsized":"2024-09-10T22:08:21.422016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:18.222180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.419395+0000","last_change":"2024-09-10T22:08:19.374585+0000","last_active":"2024-09-10T22:08:21.419395+0000","last_peered":"2024-09-10T22:08:21.419395+0000","last_clean":"2024-09-10T22:08:21.419395+0000","last_became_active":"2024-09-10T22:08:19.374289+0000","last_became_peered":"2024-09-10T22:08:19.374289+0000","last_unstale":"2024-09-10T22:08:21.419395+0000","last_undegraded":"2024-09-10T22:08:21.419395+0000","last_fullsized":"2024-09-10T22:08:21.419395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:36.518328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421248+0000","last_change":"2024-09-10T22:08:20.393428+0000","last_active":"2024-09-10T22:08:21.421248+0000","last_peered":"2024-09-10T22:08:21.421248+0000","last_clean":"2024-09-10T22:08:21.421248+0000","last_became_active":"2024-09-10T22:08:20.393097+0000","last_became_peered":"2024-09-10T22:08:20.393097+0000","last_unstale":"2024-09-10T22:08:21.421248+0000","last_undegraded":"2024-09-10T22:08:21.421248+0000","last_fullsized":"2024-09-10T22:08:21.421248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:08.608408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418562+0000","last_change":"2024-09-10T22:08:19.376319+0000","last_active":"2024-09-10T22:08:21.418562+0000","last_peered":"2024-09-10T22:08:21.418562+0000","last_clean":"2024-09-10T22:08:21.418562+0000","last_became_active":"2024-09-10T22:08:19.375527+0000","last_became_peered":"2024-09-10T22:08:19.375527+0000","last_unstale":"2024-09-10T22:08:21.418562+0000","last_undegraded":"2024-09-10T22:08:21.418562+0000","last_fullsized":"2024-09-10T22:08:21.418562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:41.168249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.402592+0000","last_change":"2024-09-10T22:08:20.395005+0000","last_active":"2024-09-10T22:08:20.402592+0000","last_peered":"2024-09-10T22:08:20.402592+0000","last_clean":"2024-09-10T22:08:20.402592+0000","last_became_active":"2024-09-10T22:08:20.387603+0000","last_became_peered":"2024-09-10T22:08:20.387603+0000","last_unstale":"2024-09-10T22:08:20.402592+0000","last_undegraded":"2024-09-10T22:08:20.402592+0000","last_fullsized":"2024-09-10T22:08:20.402592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:41.264580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.419150+0000","last_change":"2024-09-10T22:08:19.390007+0000","last_active":"2024-09-10T22:08:21.419150+0000","last_peered":"2024-09-10T22:08:21.419150+0000","last_clean":"2024-09-10T22:08:21.419150+0000","last_became_active":"2024-09-10T22:08:19.389763+0000","last_became_peered":"2024-09-10T22:08:19.389763+0000","last_unstale":"2024-09-10T22:08:21.419150+0000","last_undegraded":"2024-09-10T22:08:21.419150+0000","last_fullsized":"2024-09-10T22:08:21.419150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:14.197875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388371+0000","last_change":"2024-09-10T22:08:20.388425+0000","last_active":"2024-09-10T22:08:21.388371+0000","last_peered":"2024-09-10T22:08:21.388371+0000","last_clean":"2024-09-10T22:08:21.388371+0000","last_became_active":"2024-09-10T22:08:20.386962+0000","last_became_peered":"2024-09-10T22:08:20.386962+0000","last_unstale":"2024-09-10T22:08:21.388371+0000","last_undegraded":"2024-09-10T22:08:21.388371+0000","last_fullsized":"2024-09-10T22:08:21.388371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:33.775899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.376025+0000","last_change":"2024-09-10T22:08:19.392746+0000","last_active":"2024-09-10T22:08:20.376025+0000","last_peered":"2024-09-10T22:08:20.376025+0000","last_clean":"2024-09-10T22:08:20.376025+0000","last_became_active":"2024-09-10T22:08:19.392360+0000","last_became_peered":"2024-09-10T22:08:19.392360+0000","last_unstale":"2024-09-10T22:08:20.376025+0000","last_undegraded":"2024-09-10T22:08:20.376025+0000","last_fullsized":"2024-09-10T22:08:20.376025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:07.828645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.393022+0000","last_change":"2024-09-10T22:08:20.386333+0000","last_active":"2024-09-10T22:08:20.393022+0000","last_peered":"2024-09-10T22:08:20.393022+0000","last_clean":"2024-09-10T22:08:20.393022+0000","last_became_active":"2024-09-10T22:08:20.385854+0000","last_became_peered":"2024-09-10T22:08:20.385854+0000","last_unstale":"2024-09-10T22:08:20.393022+0000","last_undegraded":"2024-09-10T22:08:20.393022+0000","last_fullsized":"2024-09-10T22:08:20.393022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:42.971872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.375796+0000","last_change":"2024-09-10T22:08:19.387806+0000","last_active":"2024-09-10T22:08:20.375796+0000","last_peered":"2024-09-10T22:08:20.375796+0000","last_clean":"2024-09-10T22:08:20.375796+0000","last_became_active":"2024-09-10T22:08:19.386727+0000","last_became_peered":"2024-09-10T22:08:19.386727+0000","last_unstale":"2024-09-10T22:08:20.375796+0000","last_undegraded":"2024-09-10T22:08:20.375796+0000","last_fullsized":"2024-09-10T22:08:20.375796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:06.050175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388109+0000","last_change":"2024-09-10T22:08:20.387821+0000","last_active":"2024-09-10T22:08:21.388109+0000","last_peered":"2024-09-10T22:08:21.388109+0000","last_clean":"2024-09-10T22:08:21.388109+0000","last_became_active":"2024-09-10T22:08:20.387524+0000","last_became_peered":"2024-09-10T22:08:20.387524+0000","last_unstale":"2024-09-10T22:08:21.388109+0000","last_undegraded":"2024-09-10T22:08:21.388109+0000","last_fullsized":"2024-09-10T22:08:21.388109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:56.865067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417971+0000","last_change":"2024-09-10T22:08:19.385790+0000","last_active":"2024-09-10T22:08:21.417971+0000","last_peered":"2024-09-10T22:08:21.417971+0000","last_clean":"2024-09-10T22:08:21.417971+0000","last_became_active":"2024-09-10T22:08:19.385464+0000","last_became_peered":"2024-09-10T22:08:19.385464+0000","last_unstale":"2024-09-10T22:08:21.417971+0000","last_undegraded":"2024-09-10T22:08:21.417971+0000","last_fullsized":"2024-09-10T22:08:21.417971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:21.824878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421020+0000","last_change":"2024-09-10T22:08:20.390271+0000","last_active":"2024-09-10T22:08:21.421020+0000","last_peered":"2024-09-10T22:08:21.421020+0000","last_clean":"2024-09-10T22:08:21.421020+0000","last_became_active":"2024-09-10T22:08:20.389966+0000","last_became_peered":"2024-09-10T22:08:20.389966+0000","last_unstale":"2024-09-10T22:08:21.421020+0000","last_undegraded":"2024-09-10T22:08:21.421020+0000","last_fullsized":"2024-09-10T22:08:21.421020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:15.612234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418244+0000","last_change":"2024-09-10T22:08:19.389431+0000","last_active":"2024-09-10T22:08:21.418244+0000","last_peered":"2024-09-10T22:08:21.418244+0000","last_clean":"2024-09-10T22:08:21.418244+0000","last_became_active":"2024-09-10T22:08:19.389099+0000","last_became_peered":"2024-09-10T22:08:19.389099+0000","last_unstale":"2024-09-10T22:08:21.418244+0000","last_undegraded":"2024-09-10T22:08:21.418244+0000","last_fullsized":"2024-09-10T22:08:21.418244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:28.946507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389172+0000","last_change":"2024-09-10T22:08:20.399532+0000","last_active":"2024-09-10T22:08:21.389172+0000","last_peered":"2024-09-10T22:08:21.389172+0000","last_clean":"2024-09-10T22:08:21.389172+0000","last_became_active":"2024-09-10T22:08:20.399228+0000","last_became_peered":"2024-09-10T22:08:20.399228+0000","last_unstale":"2024-09-10T22:08:21.389172+0000","last_undegraded":"2024-09-10T22:08:21.389172+0000","last_fullsized":"2024-09-10T22:08:21.389172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:42.189021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418056+0000","last_change":"2024-09-10T22:08:19.382102+0000","last_active":"2024-09-10T22:08:21.418056+0000","last_peered":"2024-09-10T22:08:21.418056+0000","last_clean":"2024-09-10T22:08:21.418056+0000","last_became_active":"2024-09-10T22:08:19.380748+0000","last_became_peered":"2024-09-10T22:08:19.380748+0000","last_unstale":"2024-09-10T22:08:21.418056+0000","last_undegraded":"2024-09-10T22:08:21.418056+0000","last_fullsized":"2024-09-10T22:08:21.418056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:46.688764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422021+0000","last_change":"2024-09-10T22:08:20.402492+0000","last_active":"2024-09-10T22:08:21.422021+0000","last_peered":"2024-09-10T22:08:21.422021+0000","last_clean":"2024-09-10T22:08:21.422021+0000","last_became_active":"2024-09-10T22:08:20.399700+0000","last_became_peered":"2024-09-10T22:08:20.399700+0000","last_unstale":"2024-09-10T22:08:21.422021+0000","last_undegraded":"2024-09-10T22:08:21.422021+0000","last_fullsized":"2024-09-10T22:08:21.422021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:15.612234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421624+0000","last_change":"2024-09-10T22:08:19.396211+0000","last_active":"2024-09-10T22:08:21.421624+0000","last_peered":"2024-09-10T22:08:21.421624+0000","last_clean":"2024-09-10T22:08:21.421624+0000","last_became_active":"2024-09-10T22:08:19.395898+0000","last_became_peered":"2024-09-10T22:08:19.395898+0000","last_unstale":"2024-09-10T22:08:21.421624+0000","last_undegraded":"2024-09-10T22:08:21.421624+0000","last_fullsized":"2024-09-10T22:08:21.421624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:04.841115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421318+0000","last_change":"2024-09-10T22:08:20.391315+0000","last_active":"2024-09-10T22:08:21.421318+0000","last_peered":"2024-09-10T22:08:21.421318+0000","last_clean":"2024-09-10T22:08:21.421318+0000","last_became_active":"2024-09-10T22:08:20.391054+0000","last_became_peered":"2024-09-10T22:08:20.391054+0000","last_unstale":"2024-09-10T22:08:21.421318+0000","last_undegraded":"2024-09-10T22:08:21.421318+0000","last_fullsized":"2024-09-10T22:08:21.421318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:43.318891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422086+0000","last_change":"2024-09-10T22:08:19.389121+0000","last_active":"2024-09-10T22:08:21.422086+0000","last_peered":"2024-09-10T22:08:21.422086+0000","last_clean":"2024-09-10T22:08:21.422086+0000","last_became_active":"2024-09-10T22:08:19.388493+0000","last_became_peered":"2024-09-10T22:08:19.388493+0000","last_unstale":"2024-09-10T22:08:21.422086+0000","last_undegraded":"2024-09-10T22:08:21.422086+0000","last_fullsized":"2024-09-10T22:08:21.422086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:44.551044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388341+0000","last_change":"2024-09-10T22:08:20.397358+0000","last_active":"2024-09-10T22:08:21.388341+0000","last_peered":"2024-09-10T22:08:21.388341+0000","last_clean":"2024-09-10T22:08:21.388341+0000","last_became_active":"2024-09-10T22:08:20.396856+0000","last_became_peered":"2024-09-10T22:08:20.396856+0000","last_unstale":"2024-09-10T22:08:21.388341+0000","last_undegraded":"2024-09-10T22:08:21.388341+0000","last_fullsized":"2024-09-10T22:08:21.388341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:36:34.698550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418505+0000","last_change":"2024-09-10T22:08:19.387812+0000","last_active":"2024-09-10T22:08:21.418505+0000","last_peered":"2024-09-10T22:08:21.418505+0000","last_clean":"2024-09-10T22:08:21.418505+0000","last_became_active":"2024-09-10T22:08:19.387485+0000","last_became_peered":"2024-09-10T22:08:19.387485+0000","last_unstale":"2024-09-10T22:08:21.418505+0000","last_undegraded":"2024-09-10T22:08:21.418505+0000","last_fullsized":"2024-09-10T22:08:21.418505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:50.363107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418376+0000","last_change":"2024-09-10T22:08:20.385282+0000","last_active":"2024-09-10T22:08:21.418376+0000","last_peered":"2024-09-10T22:08:21.418376+0000","last_clean":"2024-09-10T22:08:21.418376+0000","last_became_active":"2024-09-10T22:08:20.384999+0000","last_became_peered":"2024-09-10T22:08:20.384999+0000","last_unstale":"2024-09-10T22:08:21.418376+0000","last_undegraded":"2024-09-10T22:08:21.418376+0000","last_fullsized":"2024-09-10T22:08:21.418376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:31.307804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420662+0000","last_change":"2024-09-10T22:08:19.379104+0000","last_active":"2024-09-10T22:08:21.420662+0000","last_peered":"2024-09-10T22:08:21.420662+0000","last_clean":"2024-09-10T22:08:21.420662+0000","last_became_active":"2024-09-10T22:08:19.378785+0000","last_became_peered":"2024-09-10T22:08:19.378785+0000","last_unstale":"2024-09-10T22:08:21.420662+0000","last_undegraded":"2024-09-10T22:08:21.420662+0000","last_fullsized":"2024-09-10T22:08:21.420662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:35.529232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388301+0000","last_change":"2024-09-10T22:08:20.391674+0000","last_active":"2024-09-10T22:08:21.388301+0000","last_peered":"2024-09-10T22:08:21.388301+0000","last_clean":"2024-09-10T22:08:21.388301+0000","last_became_active":"2024-09-10T22:08:20.391352+0000","last_became_peered":"2024-09-10T22:08:20.391352+0000","last_unstale":"2024-09-10T22:08:21.388301+0000","last_undegraded":"2024-09-10T22:08:21.388301+0000","last_fullsized":"2024-09-10T22:08:21.388301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:37.889053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421618+0000","last_change":"2024-09-10T22:08:19.389989+0000","last_active":"2024-09-10T22:08:21.421618+0000","last_peered":"2024-09-10T22:08:21.421618+0000","last_clean":"2024-09-10T22:08:21.421618+0000","last_became_active":"2024-09-10T22:08:19.388786+0000","last_became_peered":"2024-09-10T22:08:19.388786+0000","last_unstale":"2024-09-10T22:08:21.421618+0000","last_undegraded":"2024-09-10T22:08:21.421618+0000","last_fullsized":"2024-09-10T22:08:21.421618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:21.960161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421473+0000","last_change":"2024-09-10T22:08:20.399594+0000","last_active":"2024-09-10T22:08:21.421473+0000","last_peered":"2024-09-10T22:08:21.421473+0000","last_clean":"2024-09-10T22:08:21.421473+0000","last_became_active":"2024-09-10T22:08:20.396259+0000","last_became_peered":"2024-09-10T22:08:20.396259+0000","last_unstale":"2024-09-10T22:08:21.421473+0000","last_undegraded":"2024-09-10T22:08:21.421473+0000","last_fullsized":"2024-09-10T22:08:21.421473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:51.166291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420481+0000","last_change":"2024-09-10T22:08:19.388762+0000","last_active":"2024-09-10T22:08:21.420481+0000","last_peered":"2024-09-10T22:08:21.420481+0000","last_clean":"2024-09-10T22:08:21.420481+0000","last_became_active":"2024-09-10T22:08:19.388528+0000","last_became_peered":"2024-09-10T22:08:19.388528+0000","last_unstale":"2024-09-10T22:08:21.420481+0000","last_undegraded":"2024-09-10T22:08:21.420481+0000","last_fullsized":"2024-09-10T22:08:21.420481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:58.672220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418427+0000","last_change":"2024-09-10T22:08:20.404988+0000","last_active":"2024-09-10T22:08:21.418427+0000","last_peered":"2024-09-10T22:08:21.418427+0000","last_clean":"2024-09-10T22:08:21.418427+0000","last_became_active":"2024-09-10T22:08:20.398466+0000","last_became_peered":"2024-09-10T22:08:20.398466+0000","last_unstale":"2024-09-10T22:08:21.418427+0000","last_undegraded":"2024-09-10T22:08:21.418427+0000","last_fullsized":"2024-09-10T22:08:21.418427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:00.434894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388935+0000","last_change":"2024-09-10T22:08:19.394006+0000","last_active":"2024-09-10T22:08:21.388935+0000","last_peered":"2024-09-10T22:08:21.388935+0000","last_clean":"2024-09-10T22:08:21.388935+0000","last_became_active":"2024-09-10T22:08:19.393741+0000","last_became_peered":"2024-09-10T22:08:19.393741+0000","last_unstale":"2024-09-10T22:08:21.388935+0000","last_undegraded":"2024-09-10T22:08:21.388935+0000","last_fullsized":"2024-09-10T22:08:21.388935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:50.363107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":230,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388249+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:08:21.388249+0000","last_peered":"2024-09-10T22:08:21.388249+0000","last_clean":"2024-09-10T22:08:21.388249+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:08:21.388249+0000","last_undegraded":"2024-09-10T22:08:21.388249+0000","last_fullsized":"2024-09-10T22:08:21.388249+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509539,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46680,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325160,"statfs":{"total":96636764160,"available":96588963840,"internally_reserved":0,"allocated":1634304,"data_stored":1033419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46104,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325736,"statfs":{"total":96636764160,"available":96589553664,"internally_reserved":0,"allocated":1048576,"data_stored":443675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509539,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58588,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313252,"statfs":{"total":96636764160,"available":96576770048,"internally_reserved":0,"allocated":1052672,"data_stored":444615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509540,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58572,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313268,"statfs":{"total":96636764160,"available":96576786432,"internally_reserved":0,"allocated":1036288,"data_stored":442981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509538,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64092,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307748,"statfs":{"total":96636764160,"available":96571133952,"internally_reserved":0,"allocated":1048576,"data_stored":444593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64551367},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80732,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94291108,"statfs":{"total":96636764160,"available":96554094592,"internally_reserved":0,"allocated":1048576,"data_stored":443175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81590725},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509539,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59156,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312684,"statfs":{"total":96636764160,"available":96576188416,"internally_reserved":0,"allocated":1634304,"data_stored":1033383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542244,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58564,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313276,"statfs":{"total":96636764160,"available":96576794624,"internally_reserved":0,"allocated":1028096,"data_stored":437207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:08:24.539 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:08:24.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.717+0000 7f6561400640 1 -- 172.21.15.66:0/3200470129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f655c06fcb0 msgr2=0x7f655c15c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:24.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.717+0000 7f6561400640 1 --2- 172.21.15.66:0/3200470129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f655c06fcb0 0x7f655c15c320 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6554009c80 tx=0x7f655401cd10 comp rx=0 tx=0).stop 2024-09-10T22:08:24.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.717+0000 7f6561400640 1 -- 172.21.15.66:0/3200470129 shutdown_connections 2024-09-10T22:08:24.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.717+0000 7f6561400640 1 --2- 172.21.15.66:0/3200470129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f655c06fcb0 0x7f655c15c320 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.717+0000 7f6561400640 1 --2- 172.21.15.66:0/3200470129 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f655c06cc20 0x7f655c06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.717+0000 7f6561400640 1 -- 172.21.15.66:0/3200470129 >> 172.21.15.66:0/3200470129 conn(0x7f655c0cc3c0 msgr2=0x7f655c0cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:24.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.717+0000 7f6561400640 1 -- 172.21.15.66:0/3200470129 shutdown_connections 2024-09-10T22:08:24.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.717+0000 7f6561400640 1 -- 172.21.15.66:0/3200470129 wait complete. 2024-09-10T22:08:24.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.717+0000 7f6561400640 1 Processor -- start 2024-09-10T22:08:24.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.717+0000 7f6561400640 1 -- start start 2024-09-10T22:08:24.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6561400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c06cc20 0x7f655c165150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:24.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6561400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f655c06fcb0 0x7f655c1554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:24.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6561400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f655c0ae990 0x7f655c0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:24.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6561400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f655c168a50 con 0x7f655c06fcb0 2024-09-10T22:08:24.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6561400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f655c1688d0 con 0x7f655c06cc20 2024-09-10T22:08:24.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6561400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f655c168bd0 con 0x7f655c0ae990 2024-09-10T22:08:24.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6553e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f655c06fcb0 0x7f655c1554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:24.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6553e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f655c06fcb0 0x7f655c1554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42202/0 (socket says 172.21.15.66:42202) 2024-09-10T22:08:24.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6553e00640 1 -- 172.21.15.66:0/3097898352 learned_addr learned my addr 172.21.15.66:0/3097898352 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:24.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f655be00640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c06cc20 0x7f655c165150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:24.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6560a00640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f655c0ae990 0x7f655c0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:24.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6553e00640 1 -- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f655c0ae990 msgr2=0x7f655c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:24.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6553e00640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f655c0ae990 0x7f655c0c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6553e00640 1 -- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c06cc20 msgr2=0x7f655c165150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:24.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6553e00640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c06cc20 0x7f655c165150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6553e00640 1 -- 172.21.15.66:0/3097898352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c157450 con 0x7f655c06fcb0 2024-09-10T22:08:24.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6553e00640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f655c06fcb0 0x7f655c1554b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f654400c3a0 tx=0x7f654400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:24.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6558c00640 1 -- 172.21.15.66:0/3097898352 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544017070 con 0x7f655c06fcb0 2024-09-10T22:08:24.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6558c00640 1 -- 172.21.15.66:0/3097898352 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6544003600 con 0x7f655c06fcb0 2024-09-10T22:08:24.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6558c00640 1 -- 172.21.15.66:0/3097898352 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544003a50 con 0x7f655c06fcb0 2024-09-10T22:08:24.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6561400640 1 -- 172.21.15.66:0/3097898352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655c0b4200 con 0x7f655c06fcb0 2024-09-10T22:08:24.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6561400640 1 -- 172.21.15.66:0/3097898352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655c0b47c0 con 0x7f655c06fcb0 2024-09-10T22:08:24.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6561400640 1 -- 172.21.15.66:0/3097898352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f655c15c320 con 0x7f655c06fcb0 2024-09-10T22:08:24.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.721+0000 7f6558c00640 1 -- 172.21.15.66:0/3097898352 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f6544003bf0 con 0x7f655c06fcb0 2024-09-10T22:08:24.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.725+0000 7f6558c00640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65380736c0 0x7f6538075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:24.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.725+0000 7f6558c00640 1 -- 172.21.15.66:0/3097898352 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f65440916c0 con 0x7f655c06fcb0 2024-09-10T22:08:24.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.725+0000 7f655be00640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65380736c0 0x7f6538075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:24.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.725+0000 7f655be00640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65380736c0 0x7f6538075b80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f654c00bc00 tx=0x7f654c00b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:24.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.729+0000 7f6558c00640 1 -- 172.21.15.66:0/3097898352 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f655c157450 con 0x7f655c06fcb0 2024-09-10T22:08:24.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.901+0000 7f6561400640 1 -- 172.21.15.66:0/3097898352 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f655c1559f0 con 0x7f65380736c0 2024-09-10T22:08:24.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.905+0000 7f6558c00640 1 -- 172.21.15.66:0/3097898352 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252081 (secure 0 0 0) 0x7f655c1559f0 con 0x7f65380736c0 2024-09-10T22:08:24.915 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:24.916 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:08:24.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.909+0000 7f6561400640 1 -- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65380736c0 msgr2=0x7f6538075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:24.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.909+0000 7f6561400640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65380736c0 0x7f6538075b80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f654c00bc00 tx=0x7f654c00b390 comp rx=0 tx=0).stop 2024-09-10T22:08:24.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.909+0000 7f6561400640 1 -- 172.21.15.66:0/3097898352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f655c06fcb0 msgr2=0x7f655c1554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:24.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.909+0000 7f6561400640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f655c06fcb0 0x7f655c1554b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f654400c3a0 tx=0x7f654400c860 comp rx=0 tx=0).stop 2024-09-10T22:08:24.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.909+0000 7f6561400640 1 -- 172.21.15.66:0/3097898352 shutdown_connections 2024-09-10T22:08:24.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.909+0000 7f6561400640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65380736c0 0x7f6538075b80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.909+0000 7f6561400640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f655c0ae990 0x7f655c0c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.909+0000 7f6561400640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f655c06fcb0 0x7f655c1554b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.909+0000 7f6561400640 1 --2- 172.21.15.66:0/3097898352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655c06cc20 0x7f655c165150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:24.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.909+0000 7f6561400640 1 -- 172.21.15.66:0/3097898352 >> 172.21.15.66:0/3097898352 conn(0x7f655c0cc3c0 msgr2=0x7f655c0cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:24.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.909+0000 7f6561400640 1 -- 172.21.15.66:0/3097898352 shutdown_connections 2024-09-10T22:08:24.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:24.909+0000 7f6561400640 1 -- 172.21.15.66:0/3097898352 wait complete. 2024-09-10T22:08:24.936 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-10T22:08:24.437654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":472488,"kb_used_data":9308,"kb_used_omap":12,"kb_used_meta":462963,"kb_avail":754502232,"statfs":{"total":773094113280,"available":772610285568,"internally_reserved":0,"allocated":9531392,"data_stored":4723048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":474074676},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001442"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421860+0000","last_change":"2024-09-10T22:08:20.404492+0000","last_active":"2024-09-10T22:08:21.421860+0000","last_peered":"2024-09-10T22:08:21.421860+0000","last_clean":"2024-09-10T22:08:21.421860+0000","last_became_active":"2024-09-10T22:08:20.404111+0000","last_became_peered":"2024-09-10T22:08:20.404111+0000","last_unstale":"2024-09-10T22:08:21.421860+0000","last_undegraded":"2024-09-10T22:08:21.421860+0000","last_fullsized":"2024-09-10T22:08:21.421860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:36.965088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418122+0000","last_change":"2024-09-10T22:08:20.404842+0000","last_active":"2024-09-10T22:08:21.418122+0000","last_peered":"2024-09-10T22:08:21.418122+0000","last_clean":"2024-09-10T22:08:21.418122+0000","last_became_active":"2024-09-10T22:08:20.404635+0000","last_became_peered":"2024-09-10T22:08:20.404635+0000","last_unstale":"2024-09-10T22:08:21.418122+0000","last_undegraded":"2024-09-10T22:08:21.418122+0000","last_fullsized":"2024-09-10T22:08:21.418122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:50.283801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388789+0000","last_change":"2024-09-10T22:08:20.397882+0000","last_active":"2024-09-10T22:08:21.388789+0000","last_peered":"2024-09-10T22:08:21.388789+0000","last_clean":"2024-09-10T22:08:21.388789+0000","last_became_active":"2024-09-10T22:08:20.397590+0000","last_became_peered":"2024-09-10T22:08:20.397590+0000","last_unstale":"2024-09-10T22:08:21.388789+0000","last_undegraded":"2024-09-10T22:08:21.388789+0000","last_fullsized":"2024-09-10T22:08:21.388789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:15.201174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.392068+0000","last_change":"2024-09-10T22:08:20.385143+0000","last_active":"2024-09-10T22:08:20.392068+0000","last_peered":"2024-09-10T22:08:20.392068+0000","last_clean":"2024-09-10T22:08:20.392068+0000","last_became_active":"2024-09-10T22:08:20.384842+0000","last_became_peered":"2024-09-10T22:08:20.384842+0000","last_unstale":"2024-09-10T22:08:20.392068+0000","last_undegraded":"2024-09-10T22:08:20.392068+0000","last_fullsized":"2024-09-10T22:08:20.392068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:01.935933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420827+0000","last_change":"2024-09-10T22:08:20.397875+0000","last_active":"2024-09-10T22:08:21.420827+0000","last_peered":"2024-09-10T22:08:21.420827+0000","last_clean":"2024-09-10T22:08:21.420827+0000","last_became_active":"2024-09-10T22:08:20.397513+0000","last_became_peered":"2024-09-10T22:08:20.397513+0000","last_unstale":"2024-09-10T22:08:21.420827+0000","last_undegraded":"2024-09-10T22:08:21.420827+0000","last_fullsized":"2024-09-10T22:08:21.420827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:44.962036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388116+0000","last_change":"2024-09-10T22:08:20.397096+0000","last_active":"2024-09-10T22:08:21.388116+0000","last_peered":"2024-09-10T22:08:21.388116+0000","last_clean":"2024-09-10T22:08:21.388116+0000","last_became_active":"2024-09-10T22:08:20.396527+0000","last_became_peered":"2024-09-10T22:08:20.396527+0000","last_unstale":"2024-09-10T22:08:21.388116+0000","last_undegraded":"2024-09-10T22:08:21.388116+0000","last_fullsized":"2024-09-10T22:08:21.388116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:59.646735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.394206+0000","last_change":"2024-09-10T22:08:20.387260+0000","last_active":"2024-09-10T22:08:20.394206+0000","last_peered":"2024-09-10T22:08:20.394206+0000","last_clean":"2024-09-10T22:08:20.394206+0000","last_became_active":"2024-09-10T22:08:20.386961+0000","last_became_peered":"2024-09-10T22:08:20.386961+0000","last_unstale":"2024-09-10T22:08:20.394206+0000","last_undegraded":"2024-09-10T22:08:20.394206+0000","last_fullsized":"2024-09-10T22:08:20.394206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:32.021077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.387924+0000","last_change":"2024-09-10T22:08:20.390994+0000","last_active":"2024-09-10T22:08:21.387924+0000","last_peered":"2024-09-10T22:08:21.387924+0000","last_clean":"2024-09-10T22:08:21.387924+0000","last_became_active":"2024-09-10T22:08:20.390722+0000","last_became_peered":"2024-09-10T22:08:20.390722+0000","last_unstale":"2024-09-10T22:08:21.387924+0000","last_undegraded":"2024-09-10T22:08:21.387924+0000","last_fullsized":"2024-09-10T22:08:21.387924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:56.171917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389040+0000","last_change":"2024-09-10T22:08:20.388810+0000","last_active":"2024-09-10T22:08:21.389040+0000","last_peered":"2024-09-10T22:08:21.389040+0000","last_clean":"2024-09-10T22:08:21.389040+0000","last_became_active":"2024-09-10T22:08:20.388544+0000","last_became_peered":"2024-09-10T22:08:20.388544+0000","last_unstale":"2024-09-10T22:08:21.389040+0000","last_undegraded":"2024-09-10T22:08:21.389040+0000","last_fullsized":"2024-09-10T22:08:21.389040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:36.538229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421769+0000","last_change":"2024-09-10T22:08:20.401994+0000","last_active":"2024-09-10T22:08:21.421769+0000","last_peered":"2024-09-10T22:08:21.421769+0000","last_clean":"2024-09-10T22:08:21.421769+0000","last_became_active":"2024-09-10T22:08:20.399202+0000","last_became_peered":"2024-09-10T22:08:20.399202+0000","last_unstale":"2024-09-10T22:08:21.421769+0000","last_undegraded":"2024-09-10T22:08:21.421769+0000","last_fullsized":"2024-09-10T22:08:21.421769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:45.376214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418787+0000","last_change":"2024-09-10T22:08:20.386662+0000","last_active":"2024-09-10T22:08:21.418787+0000","last_peered":"2024-09-10T22:08:21.418787+0000","last_clean":"2024-09-10T22:08:21.418787+0000","last_became_active":"2024-09-10T22:08:20.386345+0000","last_became_peered":"2024-09-10T22:08:20.386345+0000","last_unstale":"2024-09-10T22:08:21.418787+0000","last_undegraded":"2024-09-10T22:08:21.418787+0000","last_fullsized":"2024-09-10T22:08:21.418787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:54.956555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421853+0000","last_change":"2024-09-10T22:08:20.400052+0000","last_active":"2024-09-10T22:08:21.421853+0000","last_peered":"2024-09-10T22:08:21.421853+0000","last_clean":"2024-09-10T22:08:21.421853+0000","last_became_active":"2024-09-10T22:08:20.399281+0000","last_became_peered":"2024-09-10T22:08:20.399281+0000","last_unstale":"2024-09-10T22:08:21.421853+0000","last_undegraded":"2024-09-10T22:08:21.421853+0000","last_fullsized":"2024-09-10T22:08:21.421853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:37.083764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389156+0000","last_change":"2024-09-10T22:08:20.387024+0000","last_active":"2024-09-10T22:08:21.389156+0000","last_peered":"2024-09-10T22:08:21.389156+0000","last_clean":"2024-09-10T22:08:21.389156+0000","last_became_active":"2024-09-10T22:08:20.386713+0000","last_became_peered":"2024-09-10T22:08:20.386713+0000","last_unstale":"2024-09-10T22:08:21.389156+0000","last_undegraded":"2024-09-10T22:08:21.389156+0000","last_fullsized":"2024-09-10T22:08:21.389156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:23.083038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.395135+0000","last_change":"2024-09-10T22:08:20.388339+0000","last_active":"2024-09-10T22:08:20.395135+0000","last_peered":"2024-09-10T22:08:20.395135+0000","last_clean":"2024-09-10T22:08:20.395135+0000","last_became_active":"2024-09-10T22:08:20.387900+0000","last_became_peered":"2024-09-10T22:08:20.387900+0000","last_unstale":"2024-09-10T22:08:20.395135+0000","last_undegraded":"2024-09-10T22:08:20.395135+0000","last_fullsized":"2024-09-10T22:08:20.395135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:09:03.735880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418866+0000","last_change":"2024-09-10T22:08:20.390422+0000","last_active":"2024-09-10T22:08:21.418866+0000","last_peered":"2024-09-10T22:08:21.418866+0000","last_clean":"2024-09-10T22:08:21.418866+0000","last_became_active":"2024-09-10T22:08:20.389809+0000","last_became_peered":"2024-09-10T22:08:20.389809+0000","last_unstale":"2024-09-10T22:08:21.418866+0000","last_undegraded":"2024-09-10T22:08:21.418866+0000","last_fullsized":"2024-09-10T22:08:21.418866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:45.292576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420568+0000","last_change":"2024-09-10T22:08:20.398103+0000","last_active":"2024-09-10T22:08:21.420568+0000","last_peered":"2024-09-10T22:08:21.420568+0000","last_clean":"2024-09-10T22:08:21.420568+0000","last_became_active":"2024-09-10T22:08:20.397670+0000","last_became_peered":"2024-09-10T22:08:20.397670+0000","last_unstale":"2024-09-10T22:08:21.420568+0000","last_undegraded":"2024-09-10T22:08:21.420568+0000","last_fullsized":"2024-09-10T22:08:21.420568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:24.713807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421297+0000","last_change":"2024-09-10T22:08:20.392928+0000","last_active":"2024-09-10T22:08:21.421297+0000","last_peered":"2024-09-10T22:08:21.421297+0000","last_clean":"2024-09-10T22:08:21.421297+0000","last_became_active":"2024-09-10T22:08:20.392250+0000","last_became_peered":"2024-09-10T22:08:20.392250+0000","last_unstale":"2024-09-10T22:08:21.421297+0000","last_undegraded":"2024-09-10T22:08:21.421297+0000","last_fullsized":"2024-09-10T22:08:21.421297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:41.724028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.400133+0000","last_change":"2024-09-10T22:08:20.393671+0000","last_active":"2024-09-10T22:08:20.400133+0000","last_peered":"2024-09-10T22:08:20.400133+0000","last_clean":"2024-09-10T22:08:20.400133+0000","last_became_active":"2024-09-10T22:08:20.393403+0000","last_became_peered":"2024-09-10T22:08:20.393403+0000","last_unstale":"2024-09-10T22:08:20.400133+0000","last_undegraded":"2024-09-10T22:08:20.400133+0000","last_fullsized":"2024-09-10T22:08:20.400133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:12.408747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.419376+0000","last_change":"2024-09-10T22:08:20.384344+0000","last_active":"2024-09-10T22:08:21.419376+0000","last_peered":"2024-09-10T22:08:21.419376+0000","last_clean":"2024-09-10T22:08:21.419376+0000","last_became_active":"2024-09-10T22:08:20.384097+0000","last_became_peered":"2024-09-10T22:08:20.384097+0000","last_unstale":"2024-09-10T22:08:21.419376+0000","last_undegraded":"2024-09-10T22:08:21.419376+0000","last_fullsized":"2024-09-10T22:08:21.419376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:15.201174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.395930+0000","last_change":"2024-09-10T22:08:20.389239+0000","last_active":"2024-09-10T22:08:20.395930+0000","last_peered":"2024-09-10T22:08:20.395930+0000","last_clean":"2024-09-10T22:08:20.395930+0000","last_became_active":"2024-09-10T22:08:20.388915+0000","last_became_peered":"2024-09-10T22:08:20.388915+0000","last_unstale":"2024-09-10T22:08:20.395930+0000","last_undegraded":"2024-09-10T22:08:20.395930+0000","last_fullsized":"2024-09-10T22:08:20.395930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:28.209965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.399248+0000","last_change":"2024-09-10T22:08:20.391351+0000","last_active":"2024-09-10T22:08:20.399248+0000","last_peered":"2024-09-10T22:08:20.399248+0000","last_clean":"2024-09-10T22:08:20.399248+0000","last_became_active":"2024-09-10T22:08:20.391077+0000","last_became_peered":"2024-09-10T22:08:20.391077+0000","last_unstale":"2024-09-10T22:08:20.399248+0000","last_undegraded":"2024-09-10T22:08:20.399248+0000","last_fullsized":"2024-09-10T22:08:20.399248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:24.427232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417482+0000","last_change":"2024-09-10T22:08:20.389584+0000","last_active":"2024-09-10T22:08:21.417482+0000","last_peered":"2024-09-10T22:08:21.417482+0000","last_clean":"2024-09-10T22:08:21.417482+0000","last_became_active":"2024-09-10T22:08:20.389241+0000","last_became_peered":"2024-09-10T22:08:20.389241+0000","last_unstale":"2024-09-10T22:08:21.417482+0000","last_undegraded":"2024-09-10T22:08:21.417482+0000","last_fullsized":"2024-09-10T22:08:21.417482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:44.998796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421474+0000","last_change":"2024-09-10T22:08:20.398880+0000","last_active":"2024-09-10T22:08:21.421474+0000","last_peered":"2024-09-10T22:08:21.421474+0000","last_clean":"2024-09-10T22:08:21.421474+0000","last_became_active":"2024-09-10T22:08:20.396815+0000","last_became_peered":"2024-09-10T22:08:20.396815+0000","last_unstale":"2024-09-10T22:08:21.421474+0000","last_undegraded":"2024-09-10T22:08:21.421474+0000","last_fullsized":"2024-09-10T22:08:21.421474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:39.381939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.393067+0000","last_change":"2024-09-10T22:08:20.386430+0000","last_active":"2024-09-10T22:08:20.393067+0000","last_peered":"2024-09-10T22:08:20.393067+0000","last_clean":"2024-09-10T22:08:20.393067+0000","last_became_active":"2024-09-10T22:08:20.386181+0000","last_became_peered":"2024-09-10T22:08:20.386181+0000","last_unstale":"2024-09-10T22:08:20.393067+0000","last_undegraded":"2024-09-10T22:08:20.393067+0000","last_fullsized":"2024-09-10T22:08:20.393067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:54.442771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418131+0000","last_change":"2024-09-10T22:08:20.405318+0000","last_active":"2024-09-10T22:08:21.418131+0000","last_peered":"2024-09-10T22:08:21.418131+0000","last_clean":"2024-09-10T22:08:21.418131+0000","last_became_active":"2024-09-10T22:08:20.404288+0000","last_became_peered":"2024-09-10T22:08:20.404288+0000","last_unstale":"2024-09-10T22:08:21.418131+0000","last_undegraded":"2024-09-10T22:08:21.418131+0000","last_fullsized":"2024-09-10T22:08:21.418131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:46.114623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389420+0000","last_change":"2024-09-10T22:08:20.395279+0000","last_active":"2024-09-10T22:08:21.389420+0000","last_peered":"2024-09-10T22:08:21.389420+0000","last_clean":"2024-09-10T22:08:21.389420+0000","last_became_active":"2024-09-10T22:08:20.394954+0000","last_became_peered":"2024-09-10T22:08:20.394954+0000","last_unstale":"2024-09-10T22:08:21.389420+0000","last_undegraded":"2024-09-10T22:08:21.389420+0000","last_fullsized":"2024-09-10T22:08:21.389420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:56.845631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420915+0000","last_change":"2024-09-10T22:08:20.396652+0000","last_active":"2024-09-10T22:08:21.420915+0000","last_peered":"2024-09-10T22:08:21.420915+0000","last_clean":"2024-09-10T22:08:21.420915+0000","last_became_active":"2024-09-10T22:08:20.393903+0000","last_became_peered":"2024-09-10T22:08:20.393903+0000","last_unstale":"2024-09-10T22:08:21.420915+0000","last_undegraded":"2024-09-10T22:08:21.420915+0000","last_fullsized":"2024-09-10T22:08:21.420915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:39.853371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421595+0000","last_change":"2024-09-10T22:08:20.403589+0000","last_active":"2024-09-10T22:08:21.421595+0000","last_peered":"2024-09-10T22:08:21.421595+0000","last_clean":"2024-09-10T22:08:21.421595+0000","last_became_active":"2024-09-10T22:08:20.398045+0000","last_became_peered":"2024-09-10T22:08:20.398045+0000","last_unstale":"2024-09-10T22:08:21.421595+0000","last_undegraded":"2024-09-10T22:08:21.421595+0000","last_fullsized":"2024-09-10T22:08:21.421595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:20.977215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417719+0000","last_change":"2024-09-10T22:08:20.387470+0000","last_active":"2024-09-10T22:08:21.417719+0000","last_peered":"2024-09-10T22:08:21.417719+0000","last_clean":"2024-09-10T22:08:21.417719+0000","last_became_active":"2024-09-10T22:08:20.387160+0000","last_became_peered":"2024-09-10T22:08:20.387160+0000","last_unstale":"2024-09-10T22:08:21.417719+0000","last_undegraded":"2024-09-10T22:08:21.417719+0000","last_fullsized":"2024-09-10T22:08:21.417719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:06.472265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417597+0000","last_change":"2024-09-10T22:08:20.395935+0000","last_active":"2024-09-10T22:08:21.417597+0000","last_peered":"2024-09-10T22:08:21.417597+0000","last_clean":"2024-09-10T22:08:21.417597+0000","last_became_active":"2024-09-10T22:08:20.395483+0000","last_became_peered":"2024-09-10T22:08:20.395483+0000","last_unstale":"2024-09-10T22:08:21.417597+0000","last_undegraded":"2024-09-10T22:08:21.417597+0000","last_fullsized":"2024-09-10T22:08:21.417597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:12.906612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421067+0000","last_change":"2024-09-10T22:08:20.393478+0000","last_active":"2024-09-10T22:08:21.421067+0000","last_peered":"2024-09-10T22:08:21.421067+0000","last_clean":"2024-09-10T22:08:21.421067+0000","last_became_active":"2024-09-10T22:08:20.393193+0000","last_became_peered":"2024-09-10T22:08:20.393193+0000","last_unstale":"2024-09-10T22:08:21.421067+0000","last_undegraded":"2024-09-10T22:08:21.421067+0000","last_fullsized":"2024-09-10T22:08:21.421067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:53.151316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420962+0000","last_change":"2024-09-10T22:08:20.399665+0000","last_active":"2024-09-10T22:08:21.420962+0000","last_peered":"2024-09-10T22:08:21.420962+0000","last_clean":"2024-09-10T22:08:21.420962+0000","last_became_active":"2024-09-10T22:08:20.399327+0000","last_became_peered":"2024-09-10T22:08:20.399327+0000","last_unstale":"2024-09-10T22:08:21.420962+0000","last_undegraded":"2024-09-10T22:08:21.420962+0000","last_fullsized":"2024-09-10T22:08:21.420962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:59.063752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420869+0000","last_change":"2024-09-10T22:08:20.397480+0000","last_active":"2024-09-10T22:08:21.420869+0000","last_peered":"2024-09-10T22:08:21.420869+0000","last_clean":"2024-09-10T22:08:21.420869+0000","last_became_active":"2024-09-10T22:08:20.397225+0000","last_became_peered":"2024-09-10T22:08:20.397225+0000","last_unstale":"2024-09-10T22:08:21.420869+0000","last_undegraded":"2024-09-10T22:08:21.420869+0000","last_fullsized":"2024-09-10T22:08:21.420869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:36.965088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422002+0000","last_change":"2024-09-10T22:08:19.396127+0000","last_active":"2024-09-10T22:08:21.422002+0000","last_peered":"2024-09-10T22:08:21.422002+0000","last_clean":"2024-09-10T22:08:21.422002+0000","last_became_active":"2024-09-10T22:08:19.395865+0000","last_became_peered":"2024-09-10T22:08:19.395865+0000","last_unstale":"2024-09-10T22:08:21.422002+0000","last_undegraded":"2024-09-10T22:08:21.422002+0000","last_fullsized":"2024-09-10T22:08:21.422002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:15.667695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421848+0000","last_change":"2024-09-10T22:08:20.404141+0000","last_active":"2024-09-10T22:08:21.421848+0000","last_peered":"2024-09-10T22:08:21.421848+0000","last_clean":"2024-09-10T22:08:21.421848+0000","last_became_active":"2024-09-10T22:08:20.402992+0000","last_became_peered":"2024-09-10T22:08:20.402992+0000","last_unstale":"2024-09-10T22:08:21.421848+0000","last_undegraded":"2024-09-10T22:08:21.421848+0000","last_fullsized":"2024-09-10T22:08:21.421848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:45.059450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421090+0000","last_change":"2024-09-10T22:08:19.377851+0000","last_active":"2024-09-10T22:08:21.421090+0000","last_peered":"2024-09-10T22:08:21.421090+0000","last_clean":"2024-09-10T22:08:21.421090+0000","last_became_active":"2024-09-10T22:08:19.377586+0000","last_became_peered":"2024-09-10T22:08:19.377586+0000","last_unstale":"2024-09-10T22:08:21.421090+0000","last_undegraded":"2024-09-10T22:08:21.421090+0000","last_fullsized":"2024-09-10T22:08:21.421090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:27:42.338875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422063+0000","last_change":"2024-09-10T22:08:20.404336+0000","last_active":"2024-09-10T22:08:21.422063+0000","last_peered":"2024-09-10T22:08:21.422063+0000","last_clean":"2024-09-10T22:08:21.422063+0000","last_became_active":"2024-09-10T22:08:20.402184+0000","last_became_peered":"2024-09-10T22:08:20.402184+0000","last_unstale":"2024-09-10T22:08:21.422063+0000","last_undegraded":"2024-09-10T22:08:21.422063+0000","last_fullsized":"2024-09-10T22:08:21.422063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:56.865067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.419316+0000","last_change":"2024-09-10T22:08:19.385068+0000","last_active":"2024-09-10T22:08:21.419316+0000","last_peered":"2024-09-10T22:08:21.419316+0000","last_clean":"2024-09-10T22:08:21.419316+0000","last_became_active":"2024-09-10T22:08:19.384808+0000","last_became_peered":"2024-09-10T22:08:19.384808+0000","last_unstale":"2024-09-10T22:08:21.419316+0000","last_undegraded":"2024-09-10T22:08:21.419316+0000","last_fullsized":"2024-09-10T22:08:21.419316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:07.605108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388887+0000","last_change":"2024-09-10T22:08:20.393174+0000","last_active":"2024-09-10T22:08:21.388887+0000","last_peered":"2024-09-10T22:08:21.388887+0000","last_clean":"2024-09-10T22:08:21.388887+0000","last_became_active":"2024-09-10T22:08:20.392018+0000","last_became_peered":"2024-09-10T22:08:20.392018+0000","last_unstale":"2024-09-10T22:08:21.388887+0000","last_undegraded":"2024-09-10T22:08:21.388887+0000","last_fullsized":"2024-09-10T22:08:21.388887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:22.828177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418517+0000","last_change":"2024-09-10T22:08:19.382251+0000","last_active":"2024-09-10T22:08:21.418517+0000","last_peered":"2024-09-10T22:08:21.418517+0000","last_clean":"2024-09-10T22:08:21.418517+0000","last_became_active":"2024-09-10T22:08:19.381959+0000","last_became_peered":"2024-09-10T22:08:19.381959+0000","last_unstale":"2024-09-10T22:08:21.418517+0000","last_undegraded":"2024-09-10T22:08:21.418517+0000","last_fullsized":"2024-09-10T22:08:21.418517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:14.608934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417659+0000","last_change":"2024-09-10T22:08:20.396441+0000","last_active":"2024-09-10T22:08:21.417659+0000","last_peered":"2024-09-10T22:08:21.417659+0000","last_clean":"2024-09-10T22:08:21.417659+0000","last_became_active":"2024-09-10T22:08:20.396164+0000","last_became_peered":"2024-09-10T22:08:20.396164+0000","last_unstale":"2024-09-10T22:08:21.417659+0000","last_undegraded":"2024-09-10T22:08:21.417659+0000","last_fullsized":"2024-09-10T22:08:21.417659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:20.634464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421024+0000","last_change":"2024-09-10T22:08:19.383171+0000","last_active":"2024-09-10T22:08:21.421024+0000","last_peered":"2024-09-10T22:08:21.421024+0000","last_clean":"2024-09-10T22:08:21.421024+0000","last_became_active":"2024-09-10T22:08:19.382913+0000","last_became_peered":"2024-09-10T22:08:19.382913+0000","last_unstale":"2024-09-10T22:08:21.421024+0000","last_undegraded":"2024-09-10T22:08:21.421024+0000","last_fullsized":"2024-09-10T22:08:21.421024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:09.942929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420926+0000","last_change":"2024-09-10T22:08:20.400350+0000","last_active":"2024-09-10T22:08:21.420926+0000","last_peered":"2024-09-10T22:08:21.420926+0000","last_clean":"2024-09-10T22:08:21.420926+0000","last_became_active":"2024-09-10T22:08:20.398557+0000","last_became_peered":"2024-09-10T22:08:20.398557+0000","last_unstale":"2024-09-10T22:08:21.420926+0000","last_undegraded":"2024-09-10T22:08:21.420926+0000","last_fullsized":"2024-09-10T22:08:21.420926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:51.366407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420548+0000","last_change":"2024-09-10T22:08:19.379617+0000","last_active":"2024-09-10T22:08:21.420548+0000","last_peered":"2024-09-10T22:08:21.420548+0000","last_clean":"2024-09-10T22:08:21.420548+0000","last_became_active":"2024-09-10T22:08:19.379304+0000","last_became_peered":"2024-09-10T22:08:19.379304+0000","last_unstale":"2024-09-10T22:08:21.420548+0000","last_undegraded":"2024-09-10T22:08:21.420548+0000","last_fullsized":"2024-09-10T22:08:21.420548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:51.901590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388115+0000","last_change":"2024-09-10T22:08:20.387939+0000","last_active":"2024-09-10T22:08:21.388115+0000","last_peered":"2024-09-10T22:08:21.388115+0000","last_clean":"2024-09-10T22:08:21.388115+0000","last_became_active":"2024-09-10T22:08:20.387683+0000","last_became_peered":"2024-09-10T22:08:20.387683+0000","last_unstale":"2024-09-10T22:08:21.388115+0000","last_undegraded":"2024-09-10T22:08:21.388115+0000","last_fullsized":"2024-09-10T22:08:21.388115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:31.307804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417831+0000","last_change":"2024-09-10T22:08:19.373523+0000","last_active":"2024-09-10T22:08:21.417831+0000","last_peered":"2024-09-10T22:08:21.417831+0000","last_clean":"2024-09-10T22:08:21.417831+0000","last_became_active":"2024-09-10T22:08:19.373226+0000","last_became_peered":"2024-09-10T22:08:19.373226+0000","last_unstale":"2024-09-10T22:08:21.417831+0000","last_undegraded":"2024-09-10T22:08:21.417831+0000","last_fullsized":"2024-09-10T22:08:21.417831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:38.850072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418893+0000","last_change":"2024-09-10T22:08:20.390669+0000","last_active":"2024-09-10T22:08:21.418893+0000","last_peered":"2024-09-10T22:08:21.418893+0000","last_clean":"2024-09-10T22:08:21.418893+0000","last_became_active":"2024-09-10T22:08:20.390173+0000","last_became_peered":"2024-09-10T22:08:20.390173+0000","last_unstale":"2024-09-10T22:08:21.418893+0000","last_undegraded":"2024-09-10T22:08:21.418893+0000","last_fullsized":"2024-09-10T22:08:21.418893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:42.611617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417351+0000","last_change":"2024-09-10T22:08:19.380471+0000","last_active":"2024-09-10T22:08:21.417351+0000","last_peered":"2024-09-10T22:08:21.417351+0000","last_clean":"2024-09-10T22:08:21.417351+0000","last_became_active":"2024-09-10T22:08:19.379244+0000","last_became_peered":"2024-09-10T22:08:19.379244+0000","last_unstale":"2024-09-10T22:08:21.417351+0000","last_undegraded":"2024-09-10T22:08:21.417351+0000","last_fullsized":"2024-09-10T22:08:21.417351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:32.221509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.419276+0000","last_change":"2024-09-10T22:08:20.383095+0000","last_active":"2024-09-10T22:08:21.419276+0000","last_peered":"2024-09-10T22:08:21.419276+0000","last_clean":"2024-09-10T22:08:21.419276+0000","last_became_active":"2024-09-10T22:08:20.382811+0000","last_became_peered":"2024-09-10T22:08:20.382811+0000","last_unstale":"2024-09-10T22:08:21.419276+0000","last_undegraded":"2024-09-10T22:08:21.419276+0000","last_fullsized":"2024-09-10T22:08:21.419276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:33.630831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422111+0000","last_change":"2024-09-10T22:08:19.388513+0000","last_active":"2024-09-10T22:08:21.422111+0000","last_peered":"2024-09-10T22:08:21.422111+0000","last_clean":"2024-09-10T22:08:21.422111+0000","last_became_active":"2024-09-10T22:08:19.388253+0000","last_became_peered":"2024-09-10T22:08:19.388253+0000","last_unstale":"2024-09-10T22:08:21.422111+0000","last_undegraded":"2024-09-10T22:08:21.422111+0000","last_fullsized":"2024-09-10T22:08:21.422111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:58.672220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389639+0000","last_change":"2024-09-10T22:08:20.393304+0000","last_active":"2024-09-10T22:08:21.389639+0000","last_peered":"2024-09-10T22:08:21.389639+0000","last_clean":"2024-09-10T22:08:21.389639+0000","last_became_active":"2024-09-10T22:08:20.393016+0000","last_became_peered":"2024-09-10T22:08:20.393016+0000","last_unstale":"2024-09-10T22:08:21.389639+0000","last_undegraded":"2024-09-10T22:08:21.389639+0000","last_fullsized":"2024-09-10T22:08:21.389639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:12.016944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421714+0000","last_change":"2024-09-10T22:08:19.390129+0000","last_active":"2024-09-10T22:08:21.421714+0000","last_peered":"2024-09-10T22:08:21.421714+0000","last_clean":"2024-09-10T22:08:21.421714+0000","last_became_active":"2024-09-10T22:08:19.389393+0000","last_became_peered":"2024-09-10T22:08:19.389393+0000","last_unstale":"2024-09-10T22:08:21.421714+0000","last_undegraded":"2024-09-10T22:08:21.421714+0000","last_fullsized":"2024-09-10T22:08:21.421714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:45.887767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418455+0000","last_change":"2024-09-10T22:08:20.401232+0000","last_active":"2024-09-10T22:08:21.418455+0000","last_peered":"2024-09-10T22:08:21.418455+0000","last_clean":"2024-09-10T22:08:21.418455+0000","last_became_active":"2024-09-10T22:08:20.400916+0000","last_became_peered":"2024-09-10T22:08:20.400916+0000","last_unstale":"2024-09-10T22:08:21.418455+0000","last_undegraded":"2024-09-10T22:08:21.418455+0000","last_fullsized":"2024-09-10T22:08:21.418455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:37.889053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421850+0000","last_change":"2024-09-10T22:08:19.384198+0000","last_active":"2024-09-10T22:08:21.421850+0000","last_peered":"2024-09-10T22:08:21.421850+0000","last_clean":"2024-09-10T22:08:21.421850+0000","last_became_active":"2024-09-10T22:08:19.383891+0000","last_became_peered":"2024-09-10T22:08:19.383891+0000","last_unstale":"2024-09-10T22:08:21.421850+0000","last_undegraded":"2024-09-10T22:08:21.421850+0000","last_fullsized":"2024-09-10T22:08:21.421850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:55.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421122+0000","last_change":"2024-09-10T22:08:20.392699+0000","last_active":"2024-09-10T22:08:21.421122+0000","last_peered":"2024-09-10T22:08:21.421122+0000","last_clean":"2024-09-10T22:08:21.421122+0000","last_became_active":"2024-09-10T22:08:20.392423+0000","last_became_peered":"2024-09-10T22:08:20.392423+0000","last_unstale":"2024-09-10T22:08:21.421122+0000","last_undegraded":"2024-09-10T22:08:21.421122+0000","last_fullsized":"2024-09-10T22:08:21.421122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:42.171548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388852+0000","last_change":"2024-09-10T22:08:19.378809+0000","last_active":"2024-09-10T22:08:21.388852+0000","last_peered":"2024-09-10T22:08:21.388852+0000","last_clean":"2024-09-10T22:08:21.388852+0000","last_became_active":"2024-09-10T22:08:19.378103+0000","last_became_peered":"2024-09-10T22:08:19.378103+0000","last_unstale":"2024-09-10T22:08:21.388852+0000","last_undegraded":"2024-09-10T22:08:21.388852+0000","last_fullsized":"2024-09-10T22:08:21.388852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.162991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418107+0000","last_change":"2024-09-10T22:08:20.384026+0000","last_active":"2024-09-10T22:08:21.418107+0000","last_peered":"2024-09-10T22:08:21.418107+0000","last_clean":"2024-09-10T22:08:21.418107+0000","last_became_active":"2024-09-10T22:08:20.383640+0000","last_became_peered":"2024-09-10T22:08:20.383640+0000","last_unstale":"2024-09-10T22:08:21.418107+0000","last_undegraded":"2024-09-10T22:08:21.418107+0000","last_fullsized":"2024-09-10T22:08:21.418107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:43.825862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417655+0000","last_change":"2024-09-10T22:08:19.392580+0000","last_active":"2024-09-10T22:08:21.417655+0000","last_peered":"2024-09-10T22:08:21.417655+0000","last_clean":"2024-09-10T22:08:21.417655+0000","last_became_active":"2024-09-10T22:08:19.392294+0000","last_became_peered":"2024-09-10T22:08:19.392294+0000","last_unstale":"2024-09-10T22:08:21.417655+0000","last_undegraded":"2024-09-10T22:08:21.417655+0000","last_fullsized":"2024-09-10T22:08:21.417655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:35.245280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389379+0000","last_change":"2024-09-10T22:08:20.395871+0000","last_active":"2024-09-10T22:08:21.389379+0000","last_peered":"2024-09-10T22:08:21.389379+0000","last_clean":"2024-09-10T22:08:21.389379+0000","last_became_active":"2024-09-10T22:08:20.395620+0000","last_became_peered":"2024-09-10T22:08:20.395620+0000","last_unstale":"2024-09-10T22:08:21.389379+0000","last_undegraded":"2024-09-10T22:08:21.389379+0000","last_fullsized":"2024-09-10T22:08:21.389379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:38.019367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418904+0000","last_change":"2024-09-10T22:08:19.391439+0000","last_active":"2024-09-10T22:08:21.418904+0000","last_peered":"2024-09-10T22:08:21.418904+0000","last_clean":"2024-09-10T22:08:21.418904+0000","last_became_active":"2024-09-10T22:08:19.390589+0000","last_became_peered":"2024-09-10T22:08:19.390589+0000","last_unstale":"2024-09-10T22:08:21.418904+0000","last_undegraded":"2024-09-10T22:08:21.418904+0000","last_fullsized":"2024-09-10T22:08:21.418904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:28.192491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417403+0000","last_change":"2024-09-10T22:08:20.392365+0000","last_active":"2024-09-10T22:08:21.417403+0000","last_peered":"2024-09-10T22:08:21.417403+0000","last_clean":"2024-09-10T22:08:21.417403+0000","last_became_active":"2024-09-10T22:08:20.391859+0000","last_became_peered":"2024-09-10T22:08:20.391859+0000","last_unstale":"2024-09-10T22:08:21.417403+0000","last_undegraded":"2024-09-10T22:08:21.417403+0000","last_fullsized":"2024-09-10T22:08:21.417403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:14.296616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388026+0000","last_change":"2024-09-10T22:08:19.380051+0000","last_active":"2024-09-10T22:08:21.388026+0000","last_peered":"2024-09-10T22:08:21.388026+0000","last_clean":"2024-09-10T22:08:21.388026+0000","last_became_active":"2024-09-10T22:08:19.379794+0000","last_became_peered":"2024-09-10T22:08:19.379794+0000","last_unstale":"2024-09-10T22:08:21.388026+0000","last_undegraded":"2024-09-10T22:08:21.388026+0000","last_fullsized":"2024-09-10T22:08:21.388026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:58.060452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417406+0000","last_change":"2024-09-10T22:08:20.405143+0000","last_active":"2024-09-10T22:08:21.417406+0000","last_peered":"2024-09-10T22:08:21.417406+0000","last_clean":"2024-09-10T22:08:21.417406+0000","last_became_active":"2024-09-10T22:08:20.404264+0000","last_became_peered":"2024-09-10T22:08:20.404264+0000","last_unstale":"2024-09-10T22:08:21.417406+0000","last_undegraded":"2024-09-10T22:08:21.417406+0000","last_fullsized":"2024-09-10T22:08:21.417406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:39.698136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417041+0000","last_change":"2024-09-10T22:08:19.386983+0000","last_active":"2024-09-10T22:08:21.417041+0000","last_peered":"2024-09-10T22:08:21.417041+0000","last_clean":"2024-09-10T22:08:21.417041+0000","last_became_active":"2024-09-10T22:08:19.386678+0000","last_became_peered":"2024-09-10T22:08:19.386678+0000","last_unstale":"2024-09-10T22:08:21.417041+0000","last_undegraded":"2024-09-10T22:08:21.417041+0000","last_fullsized":"2024-09-10T22:08:21.417041+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:35.961788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389015+0000","last_change":"2024-09-10T22:08:20.390517+0000","last_active":"2024-09-10T22:08:21.389015+0000","last_peered":"2024-09-10T22:08:21.389015+0000","last_clean":"2024-09-10T22:08:21.389015+0000","last_became_active":"2024-09-10T22:08:20.389571+0000","last_became_peered":"2024-09-10T22:08:20.389571+0000","last_unstale":"2024-09-10T22:08:21.389015+0000","last_undegraded":"2024-09-10T22:08:21.389015+0000","last_fullsized":"2024-09-10T22:08:21.389015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:29.195791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422154+0000","last_change":"2024-09-10T22:08:19.387428+0000","last_active":"2024-09-10T22:08:21.422154+0000","last_peered":"2024-09-10T22:08:21.422154+0000","last_clean":"2024-09-10T22:08:21.422154+0000","last_became_active":"2024-09-10T22:08:19.387142+0000","last_became_peered":"2024-09-10T22:08:19.387142+0000","last_unstale":"2024-09-10T22:08:21.422154+0000","last_undegraded":"2024-09-10T22:08:21.422154+0000","last_fullsized":"2024-09-10T22:08:21.422154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:18.451468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422016+0000","last_change":"2024-09-10T22:08:20.400798+0000","last_active":"2024-09-10T22:08:21.422016+0000","last_peered":"2024-09-10T22:08:21.422016+0000","last_clean":"2024-09-10T22:08:21.422016+0000","last_became_active":"2024-09-10T22:08:20.400312+0000","last_became_peered":"2024-09-10T22:08:20.400312+0000","last_unstale":"2024-09-10T22:08:21.422016+0000","last_undegraded":"2024-09-10T22:08:21.422016+0000","last_fullsized":"2024-09-10T22:08:21.422016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:18.222180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.419395+0000","last_change":"2024-09-10T22:08:19.374585+0000","last_active":"2024-09-10T22:08:21.419395+0000","last_peered":"2024-09-10T22:08:21.419395+0000","last_clean":"2024-09-10T22:08:21.419395+0000","last_became_active":"2024-09-10T22:08:19.374289+0000","last_became_peered":"2024-09-10T22:08:19.374289+0000","last_unstale":"2024-09-10T22:08:21.419395+0000","last_undegraded":"2024-09-10T22:08:21.419395+0000","last_fullsized":"2024-09-10T22:08:21.419395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:36.518328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421248+0000","last_change":"2024-09-10T22:08:20.393428+0000","last_active":"2024-09-10T22:08:21.421248+0000","last_peered":"2024-09-10T22:08:21.421248+0000","last_clean":"2024-09-10T22:08:21.421248+0000","last_became_active":"2024-09-10T22:08:20.393097+0000","last_became_peered":"2024-09-10T22:08:20.393097+0000","last_unstale":"2024-09-10T22:08:21.421248+0000","last_undegraded":"2024-09-10T22:08:21.421248+0000","last_fullsized":"2024-09-10T22:08:21.421248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:08.608408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418562+0000","last_change":"2024-09-10T22:08:19.376319+0000","last_active":"2024-09-10T22:08:21.418562+0000","last_peered":"2024-09-10T22:08:21.418562+0000","last_clean":"2024-09-10T22:08:21.418562+0000","last_became_active":"2024-09-10T22:08:19.375527+0000","last_became_peered":"2024-09-10T22:08:19.375527+0000","last_unstale":"2024-09-10T22:08:21.418562+0000","last_undegraded":"2024-09-10T22:08:21.418562+0000","last_fullsized":"2024-09-10T22:08:21.418562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:41.168249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.402592+0000","last_change":"2024-09-10T22:08:20.395005+0000","last_active":"2024-09-10T22:08:20.402592+0000","last_peered":"2024-09-10T22:08:20.402592+0000","last_clean":"2024-09-10T22:08:20.402592+0000","last_became_active":"2024-09-10T22:08:20.387603+0000","last_became_peered":"2024-09-10T22:08:20.387603+0000","last_unstale":"2024-09-10T22:08:20.402592+0000","last_undegraded":"2024-09-10T22:08:20.402592+0000","last_fullsized":"2024-09-10T22:08:20.402592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:41.264580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.419150+0000","last_change":"2024-09-10T22:08:19.390007+0000","last_active":"2024-09-10T22:08:21.419150+0000","last_peered":"2024-09-10T22:08:21.419150+0000","last_clean":"2024-09-10T22:08:21.419150+0000","last_became_active":"2024-09-10T22:08:19.389763+0000","last_became_peered":"2024-09-10T22:08:19.389763+0000","last_unstale":"2024-09-10T22:08:21.419150+0000","last_undegraded":"2024-09-10T22:08:21.419150+0000","last_fullsized":"2024-09-10T22:08:21.419150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:14.197875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388371+0000","last_change":"2024-09-10T22:08:20.388425+0000","last_active":"2024-09-10T22:08:21.388371+0000","last_peered":"2024-09-10T22:08:21.388371+0000","last_clean":"2024-09-10T22:08:21.388371+0000","last_became_active":"2024-09-10T22:08:20.386962+0000","last_became_peered":"2024-09-10T22:08:20.386962+0000","last_unstale":"2024-09-10T22:08:21.388371+0000","last_undegraded":"2024-09-10T22:08:21.388371+0000","last_fullsized":"2024-09-10T22:08:21.388371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:33.775899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.376025+0000","last_change":"2024-09-10T22:08:19.392746+0000","last_active":"2024-09-10T22:08:20.376025+0000","last_peered":"2024-09-10T22:08:20.376025+0000","last_clean":"2024-09-10T22:08:20.376025+0000","last_became_active":"2024-09-10T22:08:19.392360+0000","last_became_peered":"2024-09-10T22:08:19.392360+0000","last_unstale":"2024-09-10T22:08:20.376025+0000","last_undegraded":"2024-09-10T22:08:20.376025+0000","last_fullsized":"2024-09-10T22:08:20.376025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:07.828645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.393022+0000","last_change":"2024-09-10T22:08:20.386333+0000","last_active":"2024-09-10T22:08:20.393022+0000","last_peered":"2024-09-10T22:08:20.393022+0000","last_clean":"2024-09-10T22:08:20.393022+0000","last_became_active":"2024-09-10T22:08:20.385854+0000","last_became_peered":"2024-09-10T22:08:20.385854+0000","last_unstale":"2024-09-10T22:08:20.393022+0000","last_undegraded":"2024-09-10T22:08:20.393022+0000","last_fullsized":"2024-09-10T22:08:20.393022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:42.971872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T22:08:20.375796+0000","last_change":"2024-09-10T22:08:19.387806+0000","last_active":"2024-09-10T22:08:20.375796+0000","last_peered":"2024-09-10T22:08:20.375796+0000","last_clean":"2024-09-10T22:08:20.375796+0000","last_became_active":"2024-09-10T22:08:19.386727+0000","last_became_peered":"2024-09-10T22:08:19.386727+0000","last_unstale":"2024-09-10T22:08:20.375796+0000","last_undegraded":"2024-09-10T22:08:20.375796+0000","last_fullsized":"2024-09-10T22:08:20.375796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:06.050175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388109+0000","last_change":"2024-09-10T22:08:20.387821+0000","last_active":"2024-09-10T22:08:21.388109+0000","last_peered":"2024-09-10T22:08:21.388109+0000","last_clean":"2024-09-10T22:08:21.388109+0000","last_became_active":"2024-09-10T22:08:20.387524+0000","last_became_peered":"2024-09-10T22:08:20.387524+0000","last_unstale":"2024-09-10T22:08:21.388109+0000","last_undegraded":"2024-09-10T22:08:21.388109+0000","last_fullsized":"2024-09-10T22:08:21.388109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:56.865067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.417971+0000","last_change":"2024-09-10T22:08:19.385790+0000","last_active":"2024-09-10T22:08:21.417971+0000","last_peered":"2024-09-10T22:08:21.417971+0000","last_clean":"2024-09-10T22:08:21.417971+0000","last_became_active":"2024-09-10T22:08:19.385464+0000","last_became_peered":"2024-09-10T22:08:19.385464+0000","last_unstale":"2024-09-10T22:08:21.417971+0000","last_undegraded":"2024-09-10T22:08:21.417971+0000","last_fullsized":"2024-09-10T22:08:21.417971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:21.824878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421020+0000","last_change":"2024-09-10T22:08:20.390271+0000","last_active":"2024-09-10T22:08:21.421020+0000","last_peered":"2024-09-10T22:08:21.421020+0000","last_clean":"2024-09-10T22:08:21.421020+0000","last_became_active":"2024-09-10T22:08:20.389966+0000","last_became_peered":"2024-09-10T22:08:20.389966+0000","last_unstale":"2024-09-10T22:08:21.421020+0000","last_undegraded":"2024-09-10T22:08:21.421020+0000","last_fullsized":"2024-09-10T22:08:21.421020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:15.612234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418244+0000","last_change":"2024-09-10T22:08:19.389431+0000","last_active":"2024-09-10T22:08:21.418244+0000","last_peered":"2024-09-10T22:08:21.418244+0000","last_clean":"2024-09-10T22:08:21.418244+0000","last_became_active":"2024-09-10T22:08:19.389099+0000","last_became_peered":"2024-09-10T22:08:19.389099+0000","last_unstale":"2024-09-10T22:08:21.418244+0000","last_undegraded":"2024-09-10T22:08:21.418244+0000","last_fullsized":"2024-09-10T22:08:21.418244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:28.946507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.389172+0000","last_change":"2024-09-10T22:08:20.399532+0000","last_active":"2024-09-10T22:08:21.389172+0000","last_peered":"2024-09-10T22:08:21.389172+0000","last_clean":"2024-09-10T22:08:21.389172+0000","last_became_active":"2024-09-10T22:08:20.399228+0000","last_became_peered":"2024-09-10T22:08:20.399228+0000","last_unstale":"2024-09-10T22:08:21.389172+0000","last_undegraded":"2024-09-10T22:08:21.389172+0000","last_fullsized":"2024-09-10T22:08:21.389172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:42.189021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418056+0000","last_change":"2024-09-10T22:08:19.382102+0000","last_active":"2024-09-10T22:08:21.418056+0000","last_peered":"2024-09-10T22:08:21.418056+0000","last_clean":"2024-09-10T22:08:21.418056+0000","last_became_active":"2024-09-10T22:08:19.380748+0000","last_became_peered":"2024-09-10T22:08:19.380748+0000","last_unstale":"2024-09-10T22:08:21.418056+0000","last_undegraded":"2024-09-10T22:08:21.418056+0000","last_fullsized":"2024-09-10T22:08:21.418056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:46.688764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422021+0000","last_change":"2024-09-10T22:08:20.402492+0000","last_active":"2024-09-10T22:08:21.422021+0000","last_peered":"2024-09-10T22:08:21.422021+0000","last_clean":"2024-09-10T22:08:21.422021+0000","last_became_active":"2024-09-10T22:08:20.399700+0000","last_became_peered":"2024-09-10T22:08:20.399700+0000","last_unstale":"2024-09-10T22:08:21.422021+0000","last_undegraded":"2024-09-10T22:08:21.422021+0000","last_fullsized":"2024-09-10T22:08:21.422021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:15.612234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421624+0000","last_change":"2024-09-10T22:08:19.396211+0000","last_active":"2024-09-10T22:08:21.421624+0000","last_peered":"2024-09-10T22:08:21.421624+0000","last_clean":"2024-09-10T22:08:21.421624+0000","last_became_active":"2024-09-10T22:08:19.395898+0000","last_became_peered":"2024-09-10T22:08:19.395898+0000","last_unstale":"2024-09-10T22:08:21.421624+0000","last_undegraded":"2024-09-10T22:08:21.421624+0000","last_fullsized":"2024-09-10T22:08:21.421624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:04.841115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421318+0000","last_change":"2024-09-10T22:08:20.391315+0000","last_active":"2024-09-10T22:08:21.421318+0000","last_peered":"2024-09-10T22:08:21.421318+0000","last_clean":"2024-09-10T22:08:21.421318+0000","last_became_active":"2024-09-10T22:08:20.391054+0000","last_became_peered":"2024-09-10T22:08:20.391054+0000","last_unstale":"2024-09-10T22:08:21.421318+0000","last_undegraded":"2024-09-10T22:08:21.421318+0000","last_fullsized":"2024-09-10T22:08:21.421318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:43.318891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.422086+0000","last_change":"2024-09-10T22:08:19.389121+0000","last_active":"2024-09-10T22:08:21.422086+0000","last_peered":"2024-09-10T22:08:21.422086+0000","last_clean":"2024-09-10T22:08:21.422086+0000","last_became_active":"2024-09-10T22:08:19.388493+0000","last_became_peered":"2024-09-10T22:08:19.388493+0000","last_unstale":"2024-09-10T22:08:21.422086+0000","last_undegraded":"2024-09-10T22:08:21.422086+0000","last_fullsized":"2024-09-10T22:08:21.422086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:44.551044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388341+0000","last_change":"2024-09-10T22:08:20.397358+0000","last_active":"2024-09-10T22:08:21.388341+0000","last_peered":"2024-09-10T22:08:21.388341+0000","last_clean":"2024-09-10T22:08:21.388341+0000","last_became_active":"2024-09-10T22:08:20.396856+0000","last_became_peered":"2024-09-10T22:08:20.396856+0000","last_unstale":"2024-09-10T22:08:21.388341+0000","last_undegraded":"2024-09-10T22:08:21.388341+0000","last_fullsized":"2024-09-10T22:08:21.388341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:36:34.698550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418505+0000","last_change":"2024-09-10T22:08:19.387812+0000","last_active":"2024-09-10T22:08:21.418505+0000","last_peered":"2024-09-10T22:08:21.418505+0000","last_clean":"2024-09-10T22:08:21.418505+0000","last_became_active":"2024-09-10T22:08:19.387485+0000","last_became_peered":"2024-09-10T22:08:19.387485+0000","last_unstale":"2024-09-10T22:08:21.418505+0000","last_undegraded":"2024-09-10T22:08:21.418505+0000","last_fullsized":"2024-09-10T22:08:21.418505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:50.363107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418376+0000","last_change":"2024-09-10T22:08:20.385282+0000","last_active":"2024-09-10T22:08:21.418376+0000","last_peered":"2024-09-10T22:08:21.418376+0000","last_clean":"2024-09-10T22:08:21.418376+0000","last_became_active":"2024-09-10T22:08:20.384999+0000","last_became_peered":"2024-09-10T22:08:20.384999+0000","last_unstale":"2024-09-10T22:08:21.418376+0000","last_undegraded":"2024-09-10T22:08:21.418376+0000","last_fullsized":"2024-09-10T22:08:21.418376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:31.307804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420662+0000","last_change":"2024-09-10T22:08:19.379104+0000","last_active":"2024-09-10T22:08:21.420662+0000","last_peered":"2024-09-10T22:08:21.420662+0000","last_clean":"2024-09-10T22:08:21.420662+0000","last_became_active":"2024-09-10T22:08:19.378785+0000","last_became_peered":"2024-09-10T22:08:19.378785+0000","last_unstale":"2024-09-10T22:08:21.420662+0000","last_undegraded":"2024-09-10T22:08:21.420662+0000","last_fullsized":"2024-09-10T22:08:21.420662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:35.529232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388301+0000","last_change":"2024-09-10T22:08:20.391674+0000","last_active":"2024-09-10T22:08:21.388301+0000","last_peered":"2024-09-10T22:08:21.388301+0000","last_clean":"2024-09-10T22:08:21.388301+0000","last_became_active":"2024-09-10T22:08:20.391352+0000","last_became_peered":"2024-09-10T22:08:20.391352+0000","last_unstale":"2024-09-10T22:08:21.388301+0000","last_undegraded":"2024-09-10T22:08:21.388301+0000","last_fullsized":"2024-09-10T22:08:21.388301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:37.889053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421618+0000","last_change":"2024-09-10T22:08:19.389989+0000","last_active":"2024-09-10T22:08:21.421618+0000","last_peered":"2024-09-10T22:08:21.421618+0000","last_clean":"2024-09-10T22:08:21.421618+0000","last_became_active":"2024-09-10T22:08:19.388786+0000","last_became_peered":"2024-09-10T22:08:19.388786+0000","last_unstale":"2024-09-10T22:08:21.421618+0000","last_undegraded":"2024-09-10T22:08:21.421618+0000","last_fullsized":"2024-09-10T22:08:21.421618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:21.960161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.421473+0000","last_change":"2024-09-10T22:08:20.399594+0000","last_active":"2024-09-10T22:08:21.421473+0000","last_peered":"2024-09-10T22:08:21.421473+0000","last_clean":"2024-09-10T22:08:21.421473+0000","last_became_active":"2024-09-10T22:08:20.396259+0000","last_became_peered":"2024-09-10T22:08:20.396259+0000","last_unstale":"2024-09-10T22:08:21.421473+0000","last_undegraded":"2024-09-10T22:08:21.421473+0000","last_fullsized":"2024-09-10T22:08:21.421473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:51.166291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.420481+0000","last_change":"2024-09-10T22:08:19.388762+0000","last_active":"2024-09-10T22:08:21.420481+0000","last_peered":"2024-09-10T22:08:21.420481+0000","last_clean":"2024-09-10T22:08:21.420481+0000","last_became_active":"2024-09-10T22:08:19.388528+0000","last_became_peered":"2024-09-10T22:08:19.388528+0000","last_unstale":"2024-09-10T22:08:21.420481+0000","last_undegraded":"2024-09-10T22:08:21.420481+0000","last_fullsized":"2024-09-10T22:08:21.420481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:58.672220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.418427+0000","last_change":"2024-09-10T22:08:20.404988+0000","last_active":"2024-09-10T22:08:21.418427+0000","last_peered":"2024-09-10T22:08:21.418427+0000","last_clean":"2024-09-10T22:08:21.418427+0000","last_became_active":"2024-09-10T22:08:20.398466+0000","last_became_peered":"2024-09-10T22:08:20.398466+0000","last_unstale":"2024-09-10T22:08:21.418427+0000","last_undegraded":"2024-09-10T22:08:21.418427+0000","last_fullsized":"2024-09-10T22:08:21.418427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:19.357354+0000","last_clean_scrub_stamp":"2024-09-10T22:08:19.357354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:00.434894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388935+0000","last_change":"2024-09-10T22:08:19.394006+0000","last_active":"2024-09-10T22:08:21.388935+0000","last_peered":"2024-09-10T22:08:21.388935+0000","last_clean":"2024-09-10T22:08:21.388935+0000","last_became_active":"2024-09-10T22:08:19.393741+0000","last_became_peered":"2024-09-10T22:08:19.393741+0000","last_unstale":"2024-09-10T22:08:21.388935+0000","last_undegraded":"2024-09-10T22:08:21.388935+0000","last_fullsized":"2024-09-10T22:08:21.388935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:08:18.354054+0000","last_clean_scrub_stamp":"2024-09-10T22:08:18.354054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:50.363107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":230,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:08:21.388249+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:08:21.388249+0000","last_peered":"2024-09-10T22:08:21.388249+0000","last_clean":"2024-09-10T22:08:21.388249+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:08:21.388249+0000","last_undegraded":"2024-09-10T22:08:21.388249+0000","last_fullsized":"2024-09-10T22:08:21.388249+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509539,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46680,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325160,"statfs":{"total":96636764160,"available":96588963840,"internally_reserved":0,"allocated":1634304,"data_stored":1033419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46104,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325736,"statfs":{"total":96636764160,"available":96589553664,"internally_reserved":0,"allocated":1048576,"data_stored":443675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509539,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58588,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313252,"statfs":{"total":96636764160,"available":96576770048,"internally_reserved":0,"allocated":1052672,"data_stored":444615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509540,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58572,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313268,"statfs":{"total":96636764160,"available":96576786432,"internally_reserved":0,"allocated":1036288,"data_stored":442981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509538,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64092,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307748,"statfs":{"total":96636764160,"available":96571133952,"internally_reserved":0,"allocated":1048576,"data_stored":444593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64551367},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80732,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94291108,"statfs":{"total":96636764160,"available":96554094592,"internally_reserved":0,"allocated":1048576,"data_stored":443175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81590725},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509539,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59156,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312684,"statfs":{"total":96636764160,"available":96576188416,"internally_reserved":0,"allocated":1634304,"data_stored":1033383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542244,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58564,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313276,"statfs":{"total":96636764160,"available":96576794624,"internally_reserved":0,"allocated":1028096,"data_stored":437207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:08:24.940 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:08:24.940 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T22:08:25.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.129+0000 7fbb41c00640 1 -- 172.21.15.66:0/1869882545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3c0d06b0 msgr2=0x7fbb3c0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:25.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.129+0000 7fbb41c00640 1 --2- 172.21.15.66:0/1869882545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3c0d06b0 0x7fbb3c0ce170 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009900 tx=0x7fbb2401c850 comp rx=0 tx=0).stop 2024-09-10T22:08:25.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.129+0000 7fbb41c00640 1 -- 172.21.15.66:0/1869882545 shutdown_connections 2024-09-10T22:08:25.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.129+0000 7fbb41c00640 1 --2- 172.21.15.66:0/1869882545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3c0d06b0 0x7fbb3c0ce170 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.129+0000 7fbb41c00640 1 -- 172.21.15.66:0/1869882545 >> 172.21.15.66:0/1869882545 conn(0x7fbb3c0d1c60 msgr2=0x7fbb3c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:25.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.129+0000 7fbb41c00640 1 -- 172.21.15.66:0/1869882545 shutdown_connections 2024-09-10T22:08:25.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.129+0000 7fbb41c00640 1 -- 172.21.15.66:0/1869882545 wait complete. 2024-09-10T22:08:25.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb41c00640 1 Processor -- start 2024-09-10T22:08:25.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb41c00640 1 -- start start 2024-09-10T22:08:25.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb41c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb3c0afc20 0x7fbb3c1f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:25.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb41c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3c0d06b0 0x7fbb3c06e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:25.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb41c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb3c15bd40 0x7fbb3c1fbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:25.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb41c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c16a1b0 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb41c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c16a030 con 0x7fbb3c15bd40 2024-09-10T22:08:25.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb41c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3c16a330 con 0x7fbb3c0afc20 2024-09-10T22:08:25.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb32000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3c0d06b0 0x7fbb3c06e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:25.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb32000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3c0d06b0 0x7fbb3c06e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42230/0 (socket says 172.21.15.66:42230) 2024-09-10T22:08:25.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb32000640 1 -- 172.21.15.66:0/1272075826 learned_addr learned my addr 172.21.15.66:0/1272075826 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:25.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb3b400640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb3c15bd40 0x7fbb3c1fbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:25.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb3aa00640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb3c0afc20 0x7fbb3c1f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:25.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb32000640 1 -- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb3c0afc20 msgr2=0x7fbb3c1f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:25.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb32000640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb3c0afc20 0x7fbb3c1f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb32000640 1 -- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb3c15bd40 msgr2=0x7fbb3c1fbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:25.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb32000640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb3c15bd40 0x7fbb3c1fbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb32000640 1 -- 172.21.15.66:0/1272075826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c06f280 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb32000640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3c0d06b0 0x7fbb3c06e740 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fbb240b0040 tx=0x7fbb24004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:25.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb33400640 1 -- 172.21.15.66:0/1272075826 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb240b2040 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb33400640 1 -- 172.21.15.66:0/1272075826 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb24004430 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb33400640 1 -- 172.21.15.66:0/1272075826 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24005290 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb41c00640 1 -- 172.21.15.66:0/1272075826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3c06f560 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb41c00640 1 -- 172.21.15.66:0/1272075826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3c1fb700 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.133+0000 7fbb41c00640 1 -- 172.21.15.66:0/1272075826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3c0cffa0 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.137+0000 7fbb33400640 1 -- 172.21.15.66:0/1272075826 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fbb240046d0 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.137+0000 7fbb33400640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbb180736c0 0x7fbb18075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:25.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.137+0000 7fbb33400640 1 -- 172.21.15.66:0/1272075826 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fbb2412c2c0 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.137+0000 7fbb3aa00640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbb180736c0 0x7fbb18075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:25.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.137+0000 7fbb3aa00640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbb180736c0 0x7fbb18075b80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c001650 tx=0x7fbb2c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:25.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.141+0000 7fbb33400640 1 -- 172.21.15.66:0/1272075826 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbb240164d0 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.393+0000 7fbb41c00640 1 -- 172.21.15.66:0/1272075826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbb3c0b0df0 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.401+0000 7fbb33400640 1 -- 172.21.15.66:0/1272075826 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v30) ==== 190+0+147 (secure 0 0 0) 0x7fbb240f90b0 con 0x7fbb3c0d06b0 2024-09-10T22:08:25.410 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T22:08:25.410 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw== 2024-09-10T22:08:25.410 INFO:teuthology.orchestra.run.smithi066.stdout: caps mds = "allow" 2024-09-10T22:08:25.410 INFO:teuthology.orchestra.run.smithi066.stdout: caps mon = "allow r" 2024-09-10T22:08:25.410 INFO:teuthology.orchestra.run.smithi066.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T22:08:25.410 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-09-10T22:08:25.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.405+0000 7fbb41c00640 1 -- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbb180736c0 msgr2=0x7fbb18075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:25.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.405+0000 7fbb41c00640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbb180736c0 0x7fbb18075b80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c001650 tx=0x7fbb2c009040 comp rx=0 tx=0).stop 2024-09-10T22:08:25.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.405+0000 7fbb41c00640 1 -- 172.21.15.66:0/1272075826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3c0d06b0 msgr2=0x7fbb3c06e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:25.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.405+0000 7fbb41c00640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3c0d06b0 0x7fbb3c06e740 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fbb240b0040 tx=0x7fbb24004040 comp rx=0 tx=0).stop 2024-09-10T22:08:25.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.405+0000 7fbb41c00640 1 -- 172.21.15.66:0/1272075826 shutdown_connections 2024-09-10T22:08:25.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.405+0000 7fbb41c00640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbb180736c0 0x7fbb18075b80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.405+0000 7fbb41c00640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb3c15bd40 0x7fbb3c1fbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.405+0000 7fbb41c00640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3c0d06b0 0x7fbb3c06e740 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.405+0000 7fbb41c00640 1 --2- 172.21.15.66:0/1272075826 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb3c0afc20 0x7fbb3c1f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.405+0000 7fbb41c00640 1 -- 172.21.15.66:0/1272075826 >> 172.21.15.66:0/1272075826 conn(0x7fbb3c0d1c60 msgr2=0x7fbb3c071ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:25.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.405+0000 7fbb41c00640 1 -- 172.21.15.66:0/1272075826 shutdown_connections 2024-09-10T22:08:25.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.405+0000 7fbb41c00640 1 -- 172.21.15.66:0/1272075826 wait complete. 2024-09-10T22:08:25.433 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:08:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.573+0000 7f5bb7200640 1 -- 172.21.15.66:0/3191200207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb815f300 msgr2=0x7f5bb80b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.573+0000 7f5bb7200640 1 --2- 172.21.15.66:0/3191200207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb815f300 0x7f5bb80b1090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009840 tx=0x7f5ba401c7f0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.573+0000 7f5bb7200640 1 -- 172.21.15.66:0/3191200207 shutdown_connections 2024-09-10T22:08:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.573+0000 7f5bb7200640 1 --2- 172.21.15.66:0/3191200207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb815f300 0x7f5bb80b1090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.573+0000 7f5bb7200640 1 --2- 172.21.15.66:0/3191200207 >> v2:172.21.15.94:3300/0 conn(0x7f5bb815ef20 0x7f5bb80cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.573+0000 7f5bb7200640 1 -- 172.21.15.66:0/3191200207 >> 172.21.15.66:0/3191200207 conn(0x7f5bb80d1b10 msgr2=0x7f5bb80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.573+0000 7f5bb7200640 1 -- 172.21.15.66:0/3191200207 shutdown_connections 2024-09-10T22:08:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.573+0000 7f5bb7200640 1 -- 172.21.15.66:0/3191200207 wait complete. 2024-09-10T22:08:25.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb7200640 1 Processor -- start 2024-09-10T22:08:25.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb7200640 1 -- start start 2024-09-10T22:08:25.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb7200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb80b15d0 0x7f5bb80c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:25.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb7200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bb815ef20 0x7f5bb81f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:25.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bb815f300 0x7f5bb815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:25.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8168ee0 con 0x7f5bb815f300 2024-09-10T22:08:25.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb7200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8168d60 con 0x7f5bb815ef20 2024-09-10T22:08:25.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb7200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb8169060 con 0x7f5bb80b15d0 2024-09-10T22:08:25.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bb815f300 0x7f5bb815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:25.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb5e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb80b15d0 0x7f5bb80c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:25.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bb815f300 0x7f5bb815ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42234/0 (socket says 172.21.15.66:42234) 2024-09-10T22:08:25.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb6800640 1 -- 172.21.15.66:0/2537173787 learned_addr learned my addr 172.21.15.66:0/2537173787 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:25.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb5400640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bb815ef20 0x7f5bb81f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:25.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb6800640 1 -- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb80b15d0 msgr2=0x7f5bb80c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:25.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb6800640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb80b15d0 0x7f5bb80c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb6800640 1 -- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bb815ef20 msgr2=0x7f5bb81f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:25.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb6800640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bb815ef20 0x7f5bb81f5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb6800640 1 -- 172.21.15.66:0/2537173787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb80b5ec0 con 0x7f5bb815f300 2024-09-10T22:08:25.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb6800640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bb815f300 0x7f5bb815ae60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000cb30 tx=0x7f5bb000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:25.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bae000640 1 -- 172.21.15.66:0/2537173787 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0016020 con 0x7f5bb815f300 2024-09-10T22:08:25.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bae000640 1 -- 172.21.15.66:0/2537173787 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bb000ab40 con 0x7f5bb815f300 2024-09-10T22:08:25.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb7200640 1 -- 172.21.15.66:0/2537173787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb80caaf0 con 0x7f5bb815f300 2024-09-10T22:08:25.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bae000640 1 -- 172.21.15.66:0/2537173787 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0012420 con 0x7f5bb815f300 2024-09-10T22:08:25.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb7200640 1 -- 172.21.15.66:0/2537173787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb80cb0b0 con 0x7f5bb815f300 2024-09-10T22:08:25.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.577+0000 7f5bb7200640 1 -- 172.21.15.66:0/2537173787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb80cecc0 con 0x7f5bb815f300 2024-09-10T22:08:25.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.581+0000 7f5bae000640 1 -- 172.21.15.66:0/2537173787 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f5bb00125c0 con 0x7f5bb815f300 2024-09-10T22:08:25.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.581+0000 7f5bae000640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b980736c0 0x7f5b98075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:25.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.581+0000 7f5bae000640 1 -- 172.21.15.66:0/2537173787 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f5bb0092180 con 0x7f5bb815f300 2024-09-10T22:08:25.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.581+0000 7f5bb5e00640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b980736c0 0x7f5b98075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:25.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.581+0000 7f5bb5e00640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b980736c0 0x7f5b98075b80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009be0 tx=0x7f5ba80091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.585+0000 7f5bae000640 1 -- 172.21.15.66:0/2537173787 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5bb0020020 con 0x7f5bb815f300 2024-09-10T22:08:25.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.793+0000 7f5bb7200640 1 -- 172.21.15.66:0/2537173787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bb8072e00 con 0x7f5bb815f300 2024-09-10T22:08:25.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bae000640 1 -- 172.21.15.66:0/2537173787 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5222 (secure 0 0 0) 0x7f5bb005ecc0 con 0x7f5bb815f300 2024-09-10T22:08:25.802 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:25.802 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":56,"btime":"2024-09-10T22:08:23:391913+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3360242011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3360242011},{"type":"v1","addr":"172.21.15.94:6837","nonce":3360242011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3211574496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3211574496},{"type":"v1","addr":"172.21.15.66:6835","nonce":3211574496}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/553876835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":553876835},{"type":"v1","addr":"172.21.15.66:6834","nonce":553876835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:08:19.775657+0000","modified":"2024-09-10T22:08:22.793838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6836/3187502512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3187502512},{"type":"v1","addr":"172.21.15.94:6836","nonce":3187502512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":6}]} 2024-09-10T22:08:25.803 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 56 2024-09-10T22:08:25.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bb7200640 1 -- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b980736c0 msgr2=0x7f5b98075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:25.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bb7200640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b980736c0 0x7f5b98075b80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009be0 tx=0x7f5ba80091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bb7200640 1 -- 172.21.15.66:0/2537173787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bb815f300 msgr2=0x7f5bb815ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:25.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bb7200640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bb815f300 0x7f5bb815ae60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000cb30 tx=0x7f5bb000cc20 comp rx=0 tx=0).stop 2024-09-10T22:08:25.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bb7200640 1 -- 172.21.15.66:0/2537173787 shutdown_connections 2024-09-10T22:08:25.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bb7200640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b980736c0 0x7f5b98075b80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bb7200640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bb815f300 0x7f5bb815ae60 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bb7200640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bb815ef20 0x7f5bb81f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bb7200640 1 --2- 172.21.15.66:0/2537173787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb80b15d0 0x7f5bb80c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:25.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bb7200640 1 -- 172.21.15.66:0/2537173787 >> 172.21.15.66:0/2537173787 conn(0x7f5bb80d1b10 msgr2=0x7f5bb80b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:25.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bb7200640 1 -- 172.21.15.66:0/2537173787 shutdown_connections 2024-09-10T22:08:25.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:25.797+0000 7f5bb7200640 1 -- 172.21.15.66:0/2537173787 wait complete. 2024-09-10T22:08:26.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:08:19.775657+0000', 'modified': '2024-09-10T22:08:22.793838+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6990': {'gid': 6990, 'name': 'd', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6836/3187502512', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6834', 'nonce': 3187502512}, {'type': 'v1', 'addr': '172.21.15.94:6836', 'nonce': 3187502512}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6990, 'qdb_cluster': [6990]} 2024-09-10T22:08:26.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:08:26.227 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T22:08:26.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.373+0000 7f17b9200640 1 -- 172.21.15.66:0/3864074233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17b4159900 msgr2=0x7f17b40ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.373+0000 7f17b9200640 1 --2- 172.21.15.66:0/3864074233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17b4159900 0x7f17b40ca080 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f17b41608e0 tx=0x7f17a401c830 comp rx=0 tx=0).stop 2024-09-10T22:08:26.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.373+0000 7f17b9200640 1 -- 172.21.15.66:0/3864074233 shutdown_connections 2024-09-10T22:08:26.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.373+0000 7f17b9200640 1 --2- 172.21.15.66:0/3864074233 >> v2:172.21.15.94:3300/0 conn(0x7f17b415cc80 0x7f17b40ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.373+0000 7f17b9200640 1 --2- 172.21.15.66:0/3864074233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17b4159900 0x7f17b40ca080 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.373+0000 7f17b9200640 1 -- 172.21.15.66:0/3864074233 >> 172.21.15.66:0/3864074233 conn(0x7f17b40cfa30 msgr2=0x7f17b40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:26.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.373+0000 7f17b9200640 1 -- 172.21.15.66:0/3864074233 shutdown_connections 2024-09-10T22:08:26.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.373+0000 7f17b9200640 1 -- 172.21.15.66:0/3864074233 wait complete. 2024-09-10T22:08:26.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.373+0000 7f17b9200640 1 Processor -- start 2024-09-10T22:08:26.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.373+0000 7f17b9200640 1 -- start start 2024-09-10T22:08:26.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17b9200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b40aec50 0x7f17b40c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17b9200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17b4159900 0x7f17b41fb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17b9200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17b415cc80 0x7f17b415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17b9200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f17b4168f70 con 0x7f17b4159900 2024-09-10T22:08:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17b9200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17b4168df0 con 0x7f17b40aec50 2024-09-10T22:08:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17b9200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f17b41690f0 con 0x7f17b415cc80 2024-09-10T22:08:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17abe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17b4159900 0x7f17b41fb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17abe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17b4159900 0x7f17b41fb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42254/0 (socket says 172.21.15.66:42254) 2024-09-10T22:08:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17abe00640 1 -- 172.21.15.66:0/3745699021 learned_addr learned my addr 172.21.15.66:0/3745699021 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17b1e00640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b40aec50 0x7f17b40c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17b2800640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17b415cc80 0x7f17b415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17abe00640 1 -- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17b415cc80 msgr2=0x7f17b415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17abe00640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17b415cc80 0x7f17b415afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17abe00640 1 -- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b40aec50 msgr2=0x7f17b40c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17abe00640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b40aec50 0x7f17b40c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17abe00640 1 -- 172.21.15.66:0/3745699021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b40c5390 con 0x7f17b4159900 2024-09-10T22:08:26.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17abe00640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17b4159900 0x7f17b41fb4c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f179c00c940 tx=0x7f179c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:26.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17aa000640 1 -- 172.21.15.66:0/3745699021 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c004220 con 0x7f17b4159900 2024-09-10T22:08:26.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17aa000640 1 -- 172.21.15.66:0/3745699021 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f179c0079b0 con 0x7f17b4159900 2024-09-10T22:08:26.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17aa000640 1 -- 172.21.15.66:0/3745699021 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c0044b0 con 0x7f17b4159900 2024-09-10T22:08:26.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17b9200640 1 -- 172.21.15.66:0/3745699021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b40c5670 con 0x7f17b4159900 2024-09-10T22:08:26.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.377+0000 7f17b9200640 1 -- 172.21.15.66:0/3745699021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b415aad0 con 0x7f17b4159900 2024-09-10T22:08:26.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17aa000640 1 -- 172.21.15.66:0/3745699021 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f179c020060 con 0x7f17b4159900 2024-09-10T22:08:26.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17bc6c8000 1 Processor -- start 2024-09-10T22:08:26.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17aa000640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f17900736c0 0x7f1790075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17bc6c8000 1 -- start start 2024-09-10T22:08:26.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17aa000640 1 -- 172.21.15.66:0/3745699021 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f179c091b50 con 0x7f17b4159900 2024-09-10T22:08:26.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17b2800640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f17900736c0 0x7f1790075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17bc6c8000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940a9d0 0x55ca0940adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17bc6c8000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca093be8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17bc6c8000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca093bee10 0x55ca093bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17bc6c8000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ca092df3f0 con 0x55ca0940b2f0 2024-09-10T22:08:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17bc6c8000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ca0921cb20 con 0x55ca093bee10 2024-09-10T22:08:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17bc6c8000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55ca0920bf50 con 0x55ca0940a9d0 2024-09-10T22:08:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17abe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca093be8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17abe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca093be8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42256/0 (socket says 172.21.15.66:42256) 2024-09-10T22:08:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17abe00640 1 -- 172.21.15.66:0/3263004608 learned_addr learned my addr 172.21.15.66:0/3263004608 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17b1e00640 1 --2- 172.21.15.66:0/3263004608 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940a9d0 0x55ca0940adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17b2800640 1 --2- 172.21.15.66:0/3263004608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca093bee10 0x55ca093bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17b2800640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f17900736c0 0x7f1790075b80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005650 tx=0x7f17ac00f5a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17abe00640 1 -- 172.21.15.66:0/3263004608 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940a9d0 msgr2=0x55ca0940adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17abe00640 1 --2- 172.21.15.66:0/3263004608 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940a9d0 0x55ca0940adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17abe00640 1 -- 172.21.15.66:0/3263004608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca093bee10 msgr2=0x55ca093bf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17abe00640 1 --2- 172.21.15.66:0/3263004608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca093bee10 0x55ca093bf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17abe00640 1 -- 172.21.15.66:0/3263004608 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca091d3590 con 0x55ca0940b2f0 2024-09-10T22:08:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.381+0000 7f17abe00640 1 --2- 172.21.15.66:0/3263004608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca093be8d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f179c005fd0 tx=0x7f179c05de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e05c97fa3cb2ca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17a2a00640 1 -- 172.21.15.66:0/3263004608 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c083070 con 0x55ca0940b2f0 2024-09-10T22:08:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17a2a00640 1 -- 172.21.15.66:0/3263004608 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f179c05e630 con 0x55ca0940b2f0 2024-09-10T22:08:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17a2a00640 1 -- 172.21.15.66:0/3263004608 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c01d070 con 0x55ca0940b2f0 2024-09-10T22:08:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3263004608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 msgr2=0x55ca093be8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/3263004608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca093be8d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f179c005fd0 tx=0x7f179c05de60 comp rx=0 tx=0).stop 2024-09-10T22:08:26.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3263004608 shutdown_connections 2024-09-10T22:08:26.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/3263004608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca093bee10 0x55ca093bf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/3263004608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca093be8d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/3263004608 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940a9d0 0x55ca0940adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3263004608 >> 172.21.15.66:0/3263004608 conn(0x55ca092a8130 msgr2=0x55ca093ede80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:26.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3263004608 shutdown_connections 2024-09-10T22:08:26.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3263004608 wait complete. 2024-09-10T22:08:26.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 -- start start 2024-09-10T22:08:26.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 Processor -- start 2024-09-10T22:08:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca093bee10 0x55ca093eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0940a9d0 0x55ca093ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940b2f0 0x55ca093ec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ca091a65d0 con 0x55ca093bee10 2024-09-10T22:08:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ca0920fd60 con 0x55ca0940a9d0 2024-09-10T22:08:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55ca091b2500 con 0x55ca0940b2f0 2024-09-10T22:08:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17bc6c8000 10 client.0 _ll_register_callbacks cb 0x55ca091b6810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:08:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17b1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca093bee10 0x55ca093eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17b1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca093bee10 0x55ca093eb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42266/0 (socket says 172.21.15.66:42266) 2024-09-10T22:08:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17b1e00640 1 -- 172.21.15.66:0/2425146439 learned_addr learned my addr 172.21.15.66:0/2425146439 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17b1e00640 1 -- 172.21.15.66:0/2425146439 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940b2f0 msgr2=0x55ca093ec3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17b1e00640 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940b2f0 0x55ca093ec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17b1e00640 1 -- 172.21.15.66:0/2425146439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0940a9d0 msgr2=0x55ca093ebe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17abe00640 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0940a9d0 0x55ca093ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17b1e00640 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0940a9d0 0x55ca093ebe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17b1e00640 1 -- 172.21.15.66:0/2425146439 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca091aa170 con 0x55ca093bee10 2024-09-10T22:08:26.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17abe00640 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0940a9d0 0x55ca093ebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:08:26.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f17b1e00640 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca093bee10 0x55ca093eb6a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f17a4000c10 tx=0x7f17a40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37447f360712c27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:26.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.385+0000 7f1797400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:08:26.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f1797400640 1 -- 172.21.15.66:0/2425146439 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4003900 con 0x55ca093bee10 2024-09-10T22:08:26.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f1797400640 1 -- 172.21.15.66:0/2425146439 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f17a4003aa0 con 0x55ca093bee10 2024-09-10T22:08:26.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 -- 172.21.15.66:0/2425146439 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ca091a40f0 con 0x55ca093bee10 2024-09-10T22:08:26.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f1797400640 1 -- 172.21.15.66:0/2425146439 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f17a40c6050 con 0x55ca093bee10 2024-09-10T22:08:26.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 10 client.7110 fetch_fsmap learned FSMap version 56 2024-09-10T22:08:26.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 -- 172.21.15.66:0/2425146439 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55ca09290ef0 con 0x55ca093bee10 2024-09-10T22:08:26.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f1797400640 1 -- 172.21.15.66:0/2425146439 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ca091a40f0 con 0x55ca093bee10 2024-09-10T22:08:26.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f1797400640 1 -- 172.21.15.66:0/2425146439 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7f17a40c9020 con 0x55ca093bee10 2024-09-10T22:08:26.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 10 client.7110 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T22:08:26.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 10 client.7110 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T22:08:26.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 20 client.7110 populate_metadata read hostname 'smithi066' 2024-09-10T22:08:26.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.94:6834/3187502512,v1:172.21.15.94:6836/3187502512] conn(0x55ca093ed0f0 0x55ca0941cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 4 client.7110 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:08:26.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 -- 172.21.15.66:0/2425146439 --> [v2:172.21.15.94:6834/3187502512,v1:172.21.15.94:6836/3187502512] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca0921ea70 con 0x55ca093ed0f0 2024-09-10T22:08:26.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17abe00640 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.94:6834/3187502512,v1:172.21.15.94:6836/3187502512] conn(0x55ca093ed0f0 0x55ca0941cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17abe00640 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.94:6834/3187502512,v1:172.21.15.94:6836/3187502512] conn(0x55ca093ed0f0 0x55ca0941cc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab3c341d4e2ab20 server_cookie=c7de437c02cb5359 in_seq=0 out_seq=0 2024-09-10T22:08:26.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f1797400640 10 client.7110 ms_handle_connect on v2:172.21.15.94:6834/3187502512 2024-09-10T22:08:26.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f1797400640 1 -- 172.21.15.66:0/2425146439 <== mds.0 v2:172.21.15.94:6834/3187502512 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ca0921ea70 con 0x55ca093ed0f0 2024-09-10T22:08:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f1797400640 10 client.7110 handle_command_reply: tid=0 2024-09-10T22:08:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 client.7110 shutdown 2024-09-10T22:08:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 -- 172.21.15.66:0/2425146439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca093bee10 msgr2=0x55ca093eb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca093bee10 0x55ca093eb6a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f17a4000c10 tx=0x7f17a40045e0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 -- 172.21.15.66:0/2425146439 shutdown_connections 2024-09-10T22:08:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.94:6834/3187502512,v1:172.21.15.94:6836/3187502512] conn(0x55ca093ed0f0 0x55ca0941cc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940b2f0 0x55ca093ec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0940a9d0 0x55ca093ebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f1797400640 0 client.7110 ms_handle_reset on v2:172.21.15.94:6834/3187502512 2024-09-10T22:08:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/2425146439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca093bee10 0x55ca093eb6a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 -- 172.21.15.66:0/2425146439 >> 172.21.15.66:0/2425146439 conn(0x55ca092a8130 msgr2=0x55ca093bdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:26.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 -- 172.21.15.66:0/2425146439 shutdown_connections 2024-09-10T22:08:26.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 1 -- 172.21.15.66:0/2425146439 wait complete. 2024-09-10T22:08:26.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.389+0000 7f17bc6c8000 20 client.7110 trim_cache size 0 max 16384 2024-09-10T22:08:26.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17bc6c8000 1 Processor -- start 2024-09-10T22:08:26.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17bc6c8000 1 -- start start 2024-09-10T22:08:26.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17bc6c8000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca093bee10 0x55ca0947cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17bc6c8000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0940a9d0 0x55ca0947d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17bc6c8000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca0947d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17bc6c8000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f17b4204050 con 0x55ca0940b2f0 2024-09-10T22:08:26.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17bc6c8000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41f88a0 con 0x55ca0940a9d0 2024-09-10T22:08:26.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17bc6c8000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f17b41f7c00 con 0x55ca093bee10 2024-09-10T22:08:26.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17b2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca0947d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17b2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca0947d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42282/0 (socket says 172.21.15.66:42282) 2024-09-10T22:08:26.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17b2800640 1 -- 172.21.15.66:0/164610339 learned_addr learned my addr 172.21.15.66:0/164610339 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:26.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17b1e00640 1 --2- 172.21.15.66:0/164610339 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca093bee10 0x55ca0947cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17abe00640 1 --2- 172.21.15.66:0/164610339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0940a9d0 0x55ca0947d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17b2800640 1 -- 172.21.15.66:0/164610339 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca093bee10 msgr2=0x55ca0947cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:26.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17b2800640 1 --2- 172.21.15.66:0/164610339 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca093bee10 0x55ca0947cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17b2800640 1 -- 172.21.15.66:0/164610339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0940a9d0 msgr2=0x55ca0947d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:26.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17b2800640 1 --2- 172.21.15.66:0/164610339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0940a9d0 0x55ca0947d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.409+0000 7f17b2800640 1 -- 172.21.15.66:0/164610339 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca09267d90 con 0x55ca0940b2f0 2024-09-10T22:08:26.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17b2800640 1 --2- 172.21.15.66:0/164610339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca0947d850 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005950 tx=0x7f17ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a55732bd6d6752 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:26.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17a2000640 1 -- 172.21.15.66:0/164610339 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac01b040 con 0x55ca0940b2f0 2024-09-10T22:08:26.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17a2000640 1 -- 172.21.15.66:0/164610339 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17ac025070 con 0x55ca0940b2f0 2024-09-10T22:08:26.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17a2000640 1 -- 172.21.15.66:0/164610339 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac020650 con 0x55ca0940b2f0 2024-09-10T22:08:26.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 -- 172.21.15.66:0/164610339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 msgr2=0x55ca0947d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/164610339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca0947d850 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005950 tx=0x7f17ac014000 comp rx=0 tx=0).stop 2024-09-10T22:08:26.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 -- 172.21.15.66:0/164610339 shutdown_connections 2024-09-10T22:08:26.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/164610339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca0947d850 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/164610339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0940a9d0 0x55ca0947d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/164610339 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca093bee10 0x55ca0947cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 -- 172.21.15.66:0/164610339 >> 172.21.15.66:0/164610339 conn(0x55ca092a8130 msgr2=0x55ca093c6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:26.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 -- 172.21.15.66:0/164610339 shutdown_connections 2024-09-10T22:08:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 -- 172.21.15.66:0/164610339 wait complete. 2024-09-10T22:08:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 -- start start 2024-09-10T22:08:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 Processor -- start 2024-09-10T22:08:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca093bee10 0x55ca094809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940a9d0 0x55ca09480f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca09481470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ca092cb730 con 0x55ca0940b2f0 2024-09-10T22:08:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ca09214390 con 0x55ca093bee10 2024-09-10T22:08:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55ca0921b8f0 con 0x55ca0940a9d0 2024-09-10T22:08:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17bc6c8000 10 client.0 _ll_register_callbacks cb 0x7f17a4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:08:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17b2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca09481470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17b2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca09481470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42298/0 (socket says 172.21.15.66:42298) 2024-09-10T22:08:26.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17b2800640 1 -- 172.21.15.66:0/3705521690 learned_addr learned my addr 172.21.15.66:0/3705521690 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:26.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17b1e00640 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca093bee10 0x55ca094809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17abe00640 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940a9d0 0x55ca09480f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17b2800640 1 -- 172.21.15.66:0/3705521690 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940a9d0 msgr2=0x55ca09480f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:26.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17b2800640 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940a9d0 0x55ca09480f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17b2800640 1 -- 172.21.15.66:0/3705521690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca093bee10 msgr2=0x55ca094809f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:26.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17b2800640 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca093bee10 0x55ca094809f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.413+0000 7f17b2800640 1 -- 172.21.15.66:0/3705521690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca091d2a90 con 0x55ca0940b2f0 2024-09-10T22:08:26.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b2800640 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca09481470 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0050c0 tx=0x7f17ac020a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0526fb1073cde6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:26.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f1797400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:08:26.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f1797400640 1 -- 172.21.15.66:0/3705521690 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac01b050 con 0x55ca0940b2f0 2024-09-10T22:08:26.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f1797400640 1 -- 172.21.15.66:0/3705521690 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f17ac0315f0 con 0x55ca0940b2f0 2024-09-10T22:08:26.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3705521690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ac025070 con 0x55ca0940b2f0 2024-09-10T22:08:26.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f1797400640 1 -- 172.21.15.66:0/3705521690 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f17ac038050 con 0x55ca0940b2f0 2024-09-10T22:08:26.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 10 client.7116 fetch_fsmap learned FSMap version 56 2024-09-10T22:08:26.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3705521690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55ca091b1050 con 0x55ca0940b2f0 2024-09-10T22:08:26.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f1797400640 1 -- 172.21.15.66:0/3705521690 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17ac025070 con 0x55ca0940b2f0 2024-09-10T22:08:26.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f1797400640 1 -- 172.21.15.66:0/3705521690 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7f17ac03b020 con 0x55ca0940b2f0 2024-09-10T22:08:26.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 10 client.7116 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T22:08:26.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 10 client.7116 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T22:08:26.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 20 client.7116 populate_metadata read hostname 'smithi066' 2024-09-10T22:08:26.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.94:6834/3187502512,v1:172.21.15.94:6836/3187502512] conn(0x55ca09420230 0x55ca09489c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 4 client.7116 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:08:26.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3705521690 --> [v2:172.21.15.94:6834/3187502512,v1:172.21.15.94:6836/3187502512] -- command(tid 0: {"prefix": "status"}) -- 0x55ca0921a520 con 0x55ca09420230 2024-09-10T22:08:26.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b1e00640 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.94:6834/3187502512,v1:172.21.15.94:6836/3187502512] conn(0x55ca09420230 0x55ca09489c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b1e00640 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.94:6834/3187502512,v1:172.21.15.94:6836/3187502512] conn(0x55ca09420230 0x55ca09489c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf7ad79f35dbc435 server_cookie=b8fe649f27097f8e in_seq=0 out_seq=0 2024-09-10T22:08:26.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f1797400640 10 client.7116 ms_handle_connect on v2:172.21.15.94:6834/3187502512 2024-09-10T22:08:26.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f1797400640 1 -- 172.21.15.66:0/3705521690 <== mds.0 v2:172.21.15.94:6834/3187502512 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ca0921a520 con 0x55ca09420230 2024-09-10T22:08:26.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f1797400640 10 client.7116 handle_command_reply: tid=0 2024-09-10T22:08:26.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 client.7116 shutdown 2024-09-10T22:08:26.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3705521690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 msgr2=0x55ca09481470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca09481470 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0050c0 tx=0x7f17ac020a20 comp rx=0 tx=0).stop 2024-09-10T22:08:26.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3705521690 shutdown_connections 2024-09-10T22:08:26.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.94:6834/3187502512,v1:172.21.15.94:6836/3187502512] conn(0x55ca09420230 0x55ca09489c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca0940b2f0 0x55ca09481470 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ca0940a9d0 0x55ca09480f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 --2- 172.21.15.66:0/3705521690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca093bee10 0x55ca094809f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3705521690 >> 172.21.15.66:0/3705521690 conn(0x55ca092a8130 msgr2=0x55ca094095b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:26.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f1797400640 0 client.7116 ms_handle_reset on v2:172.21.15.94:6834/3187502512 2024-09-10T22:08:26.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3705521690 shutdown_connections 2024-09-10T22:08:26.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 1 -- 172.21.15.66:0/3705521690 wait complete. 2024-09-10T22:08:26.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17bc6c8000 20 client.7116 trim_cache size 0 max 16384 2024-09-10T22:08:26.430 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:08:26.430 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:08:26.430 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:08:26.430 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 6990, 2024-09-10T22:08:26.430 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:08:26.430 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:08:26.430 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:08:26.430 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 6.0563735579999998, 2024-09-10T22:08:26.430 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 56, 2024-09-10T22:08:26.431 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 85, 2024-09-10T22:08:26.431 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 85, 2024-09-10T22:08:26.431 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 10.524908422999999 2024-09-10T22:08:26.431 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:08:26.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b9200640 1 -- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f17900736c0 msgr2=0x7f1790075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b9200640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f17900736c0 0x7f1790075b80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005650 tx=0x7f17ac00f5a0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b9200640 1 -- 172.21.15.66:0/3745699021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17b4159900 msgr2=0x7f17b41fb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b9200640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17b4159900 0x7f17b41fb4c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f179c00c940 tx=0x7f179c00ce00 comp rx=0 tx=0).stop 2024-09-10T22:08:26.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b9200640 1 -- 172.21.15.66:0/3745699021 shutdown_connections 2024-09-10T22:08:26.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b9200640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f17900736c0 0x7f1790075b80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b9200640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17b415cc80 0x7f17b415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b9200640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17b4159900 0x7f17b41fb4c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b9200640 1 --2- 172.21.15.66:0/3745699021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b40aec50 0x7f17b40c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b9200640 1 -- 172.21.15.66:0/3745699021 >> 172.21.15.66:0/3745699021 conn(0x7f17b40cfa30 msgr2=0x7f17b41554b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:26.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b9200640 1 -- 172.21.15.66:0/3745699021 shutdown_connections 2024-09-10T22:08:26.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.417+0000 7f17b9200640 1 -- 172.21.15.66:0/3745699021 wait complete. 2024-09-10T22:08:26.445 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:08:26.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.585+0000 7fd575400640 1 -- 172.21.15.66:0/3505009433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5700cf180 msgr2=0x7fd5700d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.585+0000 7fd575400640 1 --2- 172.21.15.66:0/3505009433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5700cf180 0x7fd5700d04e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd568009bc0 tx=0x7fd56801cb10 comp rx=0 tx=0).stop 2024-09-10T22:08:26.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.585+0000 7fd575400640 1 -- 172.21.15.66:0/3505009433 shutdown_connections 2024-09-10T22:08:26.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.585+0000 7fd575400640 1 --2- 172.21.15.66:0/3505009433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5700cf180 0x7fd5700d04e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.585+0000 7fd575400640 1 --2- 172.21.15.66:0/3505009433 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd57015bd40 0x7fd5700cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.585+0000 7fd575400640 1 -- 172.21.15.66:0/3505009433 >> 172.21.15.66:0/3505009433 conn(0x7fd5700d1c60 msgr2=0x7fd5700d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.585+0000 7fd575400640 1 -- 172.21.15.66:0/3505009433 shutdown_connections 2024-09-10T22:08:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.585+0000 7fd575400640 1 -- 172.21.15.66:0/3505009433 wait complete. 2024-09-10T22:08:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.585+0000 7fd575400640 1 Processor -- start 2024-09-10T22:08:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.585+0000 7fd575400640 1 -- start start 2024-09-10T22:08:26.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd575400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5700cf180 0x7fd5701550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd575400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5700cf830 0x7fd5700b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd575400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd57015bd40 0x7fd570156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd575400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd570168ba0 con 0x7fd5700cf180 2024-09-10T22:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd575400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd570168a20 con 0x7fd5700cf830 2024-09-10T22:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd575400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd570168d20 con 0x7fd57015bd40 2024-09-10T22:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd56e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5700cf180 0x7fd5701550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd56e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5700cf180 0x7fd5701550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42314/0 (socket says 172.21.15.66:42314) 2024-09-10T22:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd56e800640 1 -- 172.21.15.66:0/1876370609 learned_addr learned my addr 172.21.15.66:0/1876370609 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd56f200640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd57015bd40 0x7fd570156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd56de00640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5700cf830 0x7fd5700b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd56e800640 1 -- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd57015bd40 msgr2=0x7fd570156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd56e800640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd57015bd40 0x7fd570156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd56e800640 1 -- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5700cf830 msgr2=0x7fd5700b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd56e800640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5700cf830 0x7fd5700b6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd56e800640 1 -- 172.21.15.66:0/1876370609 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5701f7fc0 con 0x7fd5700cf180 2024-09-10T22:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd56e800640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5700cf180 0x7fd5701550b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd56000e7a0 tx=0x7fd56000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd566a00640 1 -- 172.21.15.66:0/1876370609 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56000ccc0 con 0x7fd5700cf180 2024-09-10T22:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd566a00640 1 -- 172.21.15.66:0/1876370609 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd560010430 con 0x7fd5700cf180 2024-09-10T22:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd566a00640 1 -- 172.21.15.66:0/1876370609 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560010860 con 0x7fd5700cf180 2024-09-10T22:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd575400640 1 -- 172.21.15.66:0/1876370609 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5701f82a0 con 0x7fd5700cf180 2024-09-10T22:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd575400640 1 -- 172.21.15.66:0/1876370609 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5701f8860 con 0x7fd5700cf180 2024-09-10T22:08:26.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.589+0000 7fd575400640 1 -- 172.21.15.66:0/1876370609 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5700aec10 con 0x7fd5700cf180 2024-09-10T22:08:26.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.593+0000 7fd566a00640 1 -- 172.21.15.66:0/1876370609 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fd560010a00 con 0x7fd5700cf180 2024-09-10T22:08:26.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.593+0000 7fd566a00640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd5500736c0 0x7fd550075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:26.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.593+0000 7fd566a00640 1 -- 172.21.15.66:0/1876370609 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fd560092760 con 0x7fd5700cf180 2024-09-10T22:08:26.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.593+0000 7fd56de00640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd5500736c0 0x7fd550075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:26.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.593+0000 7fd56de00640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd5500736c0 0x7fd550075b80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd5701518a0 tx=0x7fd55c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:26.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.597+0000 7fd566a00640 1 -- 172.21.15.66:0/1876370609 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd56005f240 con 0x7fd5700cf180 2024-09-10T22:08:26.775 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:08:26.779 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:08:26.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 -- 172.21.15.66:0/1876370609 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd5700b3390 con 0x7fd5700cf180 2024-09-10T22:08:26.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd566a00640 1 -- 172.21.15.66:0/1876370609 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7fd560063c70 con 0x7fd5700cf180 2024-09-10T22:08:26.814 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:08:26.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 -- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd5500736c0 msgr2=0x7fd550075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd5500736c0 0x7fd550075b80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd5701518a0 tx=0x7fd55c009040 comp rx=0 tx=0).stop 2024-09-10T22:08:26.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 -- 172.21.15.66:0/1876370609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5700cf180 msgr2=0x7fd5701550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:26.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5700cf180 0x7fd5701550b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd56000e7a0 tx=0x7fd56000ec60 comp rx=0 tx=0).stop 2024-09-10T22:08:26.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 -- 172.21.15.66:0/1876370609 shutdown_connections 2024-09-10T22:08:26.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd5500736c0 0x7fd550075b80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd57015bd40 0x7fd570156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5700cf830 0x7fd5700b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 --2- 172.21.15.66:0/1876370609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5700cf180 0x7fd5701550b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:26.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 -- 172.21.15.66:0/1876370609 >> 172.21.15.66:0/1876370609 conn(0x7fd5700d1c60 msgr2=0x7fd57006f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:26.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 -- 172.21.15.66:0/1876370609 shutdown_connections 2024-09-10T22:08:26.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:26.809+0000 7fd575400640 1 -- 172.21.15.66:0/1876370609 wait complete. 2024-09-10T22:08:26.836 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:08:26.836 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:08:26.836 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:08:26.836 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:08:26.836 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:08:26.836 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:08:26.836 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:08:26.837 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:08:26.837 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:08:26.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:26.837 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:08:26.890 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:08:26.891 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:08:26.891 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:08:26.891 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:08:26.891 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:08:26.891 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:08:26.891 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:08:26.891 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:08:26.891 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:08:26.891 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:08:26.891 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:08:26.891 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:08:26.891 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41968sec preferred_lft 41968sec 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:81:fc:4c:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T22:08:26.892 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:08:26.893 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:08:26.893 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T22:08:26.893 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:08:26.893 INFO:teuthology.orchestra.run.smithi066.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:08:26.893 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:08:26.893 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T22:08:26.893 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:08:26.893 INFO:teuthology.orchestra.run.smithi066.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:08:26.893 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:08:26.893 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T22:08:26.893 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:08:26.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:26.894 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:08:26.951 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:08:26.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:26.953 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:08:27.013 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:08:27.013 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:08:27.013 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:08:27.013 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:08:27.013 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:08:27.013 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:08:27.013 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:08:27.013 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:08:27.013 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:08:27.013 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:08:27.013 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:08:27.014 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:08:27.014 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:08:27.014 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:08:27.014 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:08:27.014 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:08:27.014 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:08:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:08:27.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:08:27.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:27.150 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:08:27.159 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:08:27.159 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:08:27.159 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:08:27.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:27.160 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:08:27.225 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T22:08:27.226 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:08:27.226 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:08:27.277 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:08:27.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:08:27.278 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:08:27.329 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:08:27.395 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:08:27.447 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:08:27.447 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:08:27.448 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:08:27.448 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:08:27.448 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:08:27.448 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:08:27.448 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:08:27.448 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:08:27.448 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:08:27.449 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:08:27.450 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:08:27.450 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:08:27.450 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:08:27.450 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:08:27.450 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:08:27.450 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:08:27.450 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:08:27.450 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:08:27.450 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:08:27.450 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:08:27.450 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:08:27.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:27.451 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:08:27.501 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:08:27.502 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T22:08:27.552 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:08:27.573 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:08:27.623 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:08:27.623 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:08:27.623 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:08:27.623 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:08:27.623 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:08:27.623 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:08:27.624 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:08:27.625 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:08:27.625 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:08:27.625 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:08:27.625 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:08:27.625 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:08:27.625 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:08:27.625 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:08:27.625 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:08:27.625 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:08:27.625 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:08:27.625 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:08:27.626 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:08:27.626 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:08:27.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:27.626 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:08:27.649 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-09-10T22:08:27.641+0000 7ffa73759580 -1 init, newargv = 0x55dc7c0c97e0 newargc=15 2024-09-10T22:08:27.649 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[44770]: starting ceph client 2024-09-10T22:08:27.656 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[44770]: starting fuse 2024-09-10T22:08:27.657 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-09-10T22:08:27.657 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T22:08:27.657 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-09-10T22:08:27.658 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:08:27.659 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-09-10T22:08:27.659 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:08:27.659 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:08:27.659 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:08:27.659 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-09-10T22:08:27.659 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-09-10T22:08:27.659 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-09-10T22:08:27.659 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:08:27.659 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:08:27.659 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-09-10T22:08:27.659 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:08:27.784 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.44770.asok 2024-09-10T22:08:27.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:08:27.796 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44770.asok 2024-09-10T22:08:27.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:27.796 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44770.asok status 2024-09-10T22:08:27.974 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:08:27.974 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-09-10T22:08:27.974 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "44770", 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 7131, 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-09-10T22:08:27.975 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 7131 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 219126170 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 219126170 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.7131 192.168.0.1:0/219126170", 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/219126170", 2024-09-10T22:08:27.976 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-09-10T22:08:27.977 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 56, 2024-09-10T22:08:27.977 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 85, 2024-09-10T22:08:27.977 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:08:27.977 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-09-10T22:08:27.977 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-09-10T22:08:27.977 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:08:27.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:27.990 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-09-10T22:08:27.990 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:08:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-09-10T22:08:27.999 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T22:08:27.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:27.999 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:08:28.062 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T22:08:28.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.253+0000 7f32cdc00640 1 -- 172.21.15.66:0/699519297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32c8162f90 msgr2=0x7f32c8165380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.253+0000 7f32cdc00640 1 --2- 172.21.15.66:0/699519297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32c8162f90 0x7f32c8165380 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f32c000ad00 tx=0x7f32c00206b0 comp rx=0 tx=0).stop 2024-09-10T22:08:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.253+0000 7f32cdc00640 1 -- 172.21.15.66:0/699519297 shutdown_connections 2024-09-10T22:08:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.253+0000 7f32cdc00640 1 --2- 172.21.15.66:0/699519297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32c8162f90 0x7f32c8165380 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.253+0000 7f32cdc00640 1 -- 172.21.15.66:0/699519297 >> 172.21.15.66:0/699519297 conn(0x7f32c80cfaa0 msgr2=0x7f32c80cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.253+0000 7f32cdc00640 1 -- 172.21.15.66:0/699519297 shutdown_connections 2024-09-10T22:08:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.253+0000 7f32cdc00640 1 -- 172.21.15.66:0/699519297 wait complete. 2024-09-10T22:08:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.253+0000 7f32cdc00640 1 Processor -- start 2024-09-10T22:08:28.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.253+0000 7f32cdc00640 1 -- start start 2024-09-10T22:08:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32cdc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32c80cee30 0x7f32c81f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32cdc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32c815ddd0 0x7f32c8148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32cdc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c8162f90 0x7f32c815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32cdc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f32c806e6a0 con 0x7f32c815ddd0 2024-09-10T22:08:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32cdc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32c815c950 con 0x7f32c8162f90 2024-09-10T22:08:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32cdc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f32c806e820 con 0x7f32c80cee30 2024-09-10T22:08:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32be000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32c815ddd0 0x7f32c8148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32be000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32c815ddd0 0x7f32c8148960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42336/0 (socket says 172.21.15.66:42336) 2024-09-10T22:08:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32be000640 1 -- 172.21.15.66:0/28564825 learned_addr learned my addr 172.21.15.66:0/28564825 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32c6a00640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32c80cee30 0x7f32c81f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32c7400640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c8162f90 0x7f32c815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32be000640 1 -- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32c80cee30 msgr2=0x7f32c81f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32be000640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32c80cee30 0x7f32c81f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32be000640 1 -- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c8162f90 msgr2=0x7f32c815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32be000640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c8162f90 0x7f32c815b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32be000640 1 -- 172.21.15.66:0/28564825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c814f540 con 0x7f32c815ddd0 2024-09-10T22:08:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32be000640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32c815ddd0 0x7f32c8148960 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f32b000c490 tx=0x7f32b000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32bf400640 1 -- 172.21.15.66:0/28564825 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b00052d0 con 0x7f32c815ddd0 2024-09-10T22:08:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32bf400640 1 -- 172.21.15.66:0/28564825 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32b0005490 con 0x7f32c815ddd0 2024-09-10T22:08:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32bf400640 1 -- 172.21.15.66:0/28564825 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0012870 con 0x7f32c815ddd0 2024-09-10T22:08:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32cdc00640 1 -- 172.21.15.66:0/28564825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c8148ea0 con 0x7f32c815ddd0 2024-09-10T22:08:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32cdc00640 1 -- 172.21.15.66:0/28564825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c80b3610 con 0x7f32c815ddd0 2024-09-10T22:08:28.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.257+0000 7f32cdc00640 1 -- 172.21.15.66:0/28564825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c80cf210 con 0x7f32c815ddd0 2024-09-10T22:08:28.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.261+0000 7f32bf400640 1 -- 172.21.15.66:0/28564825 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f32b0012a10 con 0x7f32c815ddd0 2024-09-10T22:08:28.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.261+0000 7f32bf400640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32a40736c0 0x7f32a4075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:28.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.261+0000 7f32bf400640 1 -- 172.21.15.66:0/28564825 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f32b0091bf0 con 0x7f32c815ddd0 2024-09-10T22:08:28.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.261+0000 7f32c6a00640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32a40736c0 0x7f32a4075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:28.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.261+0000 7f32c6a00640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32a40736c0 0x7f32a4075b80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f32b8007fd0 tx=0x7f32b8007ed0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:28.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.265+0000 7f32bf400640 1 -- 172.21.15.66:0/28564825 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f32b005e390 con 0x7f32c815ddd0 2024-09-10T22:08:28.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:28.489+0000 7f32cdc00640 1 -- 172.21.15.66:0/28564825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f32c806f1e0 con 0x7f32c815ddd0 2024-09-10T22:08:29.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.489+0000 7f32bf400640 1 -- 172.21.15.66:0/28564825 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v57) ==== 136+0+0 (secure 0 0 0) 0x7f32b0062dc0 con 0x7f32c815ddd0 2024-09-10T22:08:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.493+0000 7f32cdc00640 1 -- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32a40736c0 msgr2=0x7f32a4075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:29.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.493+0000 7f32cdc00640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32a40736c0 0x7f32a4075b80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f32b8007fd0 tx=0x7f32b8007ed0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.493+0000 7f32cdc00640 1 -- 172.21.15.66:0/28564825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32c815ddd0 msgr2=0x7f32c8148960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:29.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.493+0000 7f32cdc00640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32c815ddd0 0x7f32c8148960 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f32b000c490 tx=0x7f32b000c950 comp rx=0 tx=0).stop 2024-09-10T22:08:29.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.493+0000 7f32cdc00640 1 -- 172.21.15.66:0/28564825 shutdown_connections 2024-09-10T22:08:29.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.493+0000 7f32cdc00640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f32a40736c0 0x7f32a4075b80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.493+0000 7f32cdc00640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c8162f90 0x7f32c815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.493+0000 7f32cdc00640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32c815ddd0 0x7f32c8148960 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.493+0000 7f32cdc00640 1 --2- 172.21.15.66:0/28564825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32c80cee30 0x7f32c81f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.493+0000 7f32cdc00640 1 -- 172.21.15.66:0/28564825 >> 172.21.15.66:0/28564825 conn(0x7f32c80cfaa0 msgr2=0x7f32c80728e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:29.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.493+0000 7f32cdc00640 1 -- 172.21.15.66:0/28564825 shutdown_connections 2024-09-10T22:08:29.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.493+0000 7f32cdc00640 1 -- 172.21.15.66:0/28564825 wait complete. 2024-09-10T22:08:29.525 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:08:29.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.673+0000 7f3214c00640 1 -- 172.21.15.66:0/1401147856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f321015f450 msgr2=0x7f32100b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:29.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.673+0000 7f3214c00640 1 --2- 172.21.15.66:0/1401147856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f321015f450 0x7f32100b3170 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f32100d13f0 tx=0x7f320001cfd0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.673+0000 7f3214c00640 1 -- 172.21.15.66:0/1401147856 shutdown_connections 2024-09-10T22:08:29.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.673+0000 7f3214c00640 1 --2- 172.21.15.66:0/1401147856 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32101600b0 0x7f32100cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.673+0000 7f3214c00640 1 --2- 172.21.15.66:0/1401147856 >> v2:172.21.15.94:3300/0 conn(0x7f32100b1e10 0x7f321015fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.673+0000 7f3214c00640 1 --2- 172.21.15.66:0/1401147856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f321015f450 0x7f32100b3170 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.673+0000 7f3214c00640 1 -- 172.21.15.66:0/1401147856 >> 172.21.15.66:0/1401147856 conn(0x7f32100d1c60 msgr2=0x7f32100d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:29.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.673+0000 7f3214c00640 1 -- 172.21.15.66:0/1401147856 shutdown_connections 2024-09-10T22:08:29.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.673+0000 7f3214c00640 1 -- 172.21.15.66:0/1401147856 wait complete. 2024-09-10T22:08:29.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.673+0000 7f3214c00640 1 Processor -- start 2024-09-10T22:08:29.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.673+0000 7f3214c00640 1 -- start start 2024-09-10T22:08:29.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3214c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32100b1e10 0x7f3210166750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:29.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3214c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f321015f450 0x7f321015ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:29.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3214c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32101600b0 0x7f321006f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:29.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3214c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f32100aefa0 con 0x7f32101600b0 2024-09-10T22:08:29.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3214c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32100aee20 con 0x7f321015f450 2024-09-10T22:08:29.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3214c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f32100af120 con 0x7f32100b1e10 2024-09-10T22:08:29.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f320e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32101600b0 0x7f321006f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:29.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f320e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32101600b0 0x7f321006f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42350/0 (socket says 172.21.15.66:42350) 2024-09-10T22:08:29.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f320e800640 1 -- 172.21.15.66:0/1807494449 learned_addr learned my addr 172.21.15.66:0/1807494449 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:29.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f320de00640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32100b1e10 0x7f3210166750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3205400640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f321015f450 0x7f321015ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f320e800640 1 -- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32100b1e10 msgr2=0x7f3210166750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f320e800640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32100b1e10 0x7f3210166750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f320e800640 1 -- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f321015f450 msgr2=0x7f321015ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f320e800640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f321015f450 0x7f321015ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f320e800640 1 -- 172.21.15.66:0/1807494449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321014c510 con 0x7f32101600b0 2024-09-10T22:08:29.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f320e800640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32101600b0 0x7f321006f170 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f320800cce0 tx=0x7f320800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:29.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3206a00640 1 -- 172.21.15.66:0/1807494449 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320800f450 con 0x7f32101600b0 2024-09-10T22:08:29.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3206a00640 1 -- 172.21.15.66:0/1807494449 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f320800f6e0 con 0x7f32101600b0 2024-09-10T22:08:29.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3206a00640 1 -- 172.21.15.66:0/1807494449 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320800fb30 con 0x7f32101600b0 2024-09-10T22:08:29.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3214c00640 1 -- 172.21.15.66:0/1807494449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321015e410 con 0x7f32101600b0 2024-09-10T22:08:29.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3214c00640 1 -- 172.21.15.66:0/1807494449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32101fb760 con 0x7f32101600b0 2024-09-10T22:08:29.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.677+0000 7f3214c00640 1 -- 172.21.15.66:0/1807494449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32100b3170 con 0x7f32101600b0 2024-09-10T22:08:29.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.681+0000 7f3206a00640 1 -- 172.21.15.66:0/1807494449 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f320800fcd0 con 0x7f32101600b0 2024-09-10T22:08:29.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.681+0000 7f3206a00640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f31ec0736c0 0x7f31ec075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:29.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.681+0000 7f3206a00640 1 -- 172.21.15.66:0/1807494449 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f3208092b90 con 0x7f32101600b0 2024-09-10T22:08:29.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.681+0000 7f320de00640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f31ec0736c0 0x7f31ec075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:29.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.681+0000 7f320de00640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f31ec0736c0 0x7f31ec075b80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f32000ae000 tx=0x7f320001adb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:29.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.685+0000 7f3206a00640 1 -- 172.21.15.66:0/1807494449 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3208014030 con 0x7f32101600b0 2024-09-10T22:08:29.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.881+0000 7f3214c00640 1 -- 172.21.15.66:0/1807494449 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f321014c810 con 0x7f31ec0736c0 2024-09-10T22:08:29.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.881+0000 7f3206a00640 1 -- 172.21.15.66:0/1807494449 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f321014c810 con 0x7f31ec0736c0 2024-09-10T22:08:29.889 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:08:29.890 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:08:29.890 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:08:29.890 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:08:29.890 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:08:29.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.885+0000 7f3214c00640 1 -- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f31ec0736c0 msgr2=0x7f31ec075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:29.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.885+0000 7f3214c00640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f31ec0736c0 0x7f31ec075b80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f32000ae000 tx=0x7f320001adb0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.885+0000 7f3214c00640 1 -- 172.21.15.66:0/1807494449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32101600b0 msgr2=0x7f321006f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:29.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.885+0000 7f3214c00640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32101600b0 0x7f321006f170 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f320800cce0 tx=0x7f320800a620 comp rx=0 tx=0).stop 2024-09-10T22:08:29.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.885+0000 7f3214c00640 1 -- 172.21.15.66:0/1807494449 shutdown_connections 2024-09-10T22:08:29.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.885+0000 7f3214c00640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f31ec0736c0 0x7f31ec075b80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.885+0000 7f3214c00640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32101600b0 0x7f321006f170 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.885+0000 7f3214c00640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f321015f450 0x7f321015ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.885+0000 7f3214c00640 1 --2- 172.21.15.66:0/1807494449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32100b1e10 0x7f3210166750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.885+0000 7f3214c00640 1 -- 172.21.15.66:0/1807494449 >> 172.21.15.66:0/1807494449 conn(0x7f32100d1c60 msgr2=0x7f32100b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:29.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.885+0000 7f3214c00640 1 -- 172.21.15.66:0/1807494449 shutdown_connections 2024-09-10T22:08:29.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:29.885+0000 7f3214c00640 1 -- 172.21.15.66:0/1807494449 wait complete. 2024-09-10T22:08:29.913 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:08:30.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.057+0000 7f0f16000640 1 -- 172.21.15.66:0/1013807576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f100b32a0 msgr2=0x7f0f100b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.057+0000 7f0f16000640 1 --2- 172.21.15.66:0/1013807576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f100b32a0 0x7f0f100b14b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0f10069970 tx=0x7f0f0801c970 comp rx=0 tx=0).stop 2024-09-10T22:08:30.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.057+0000 7f0f16000640 1 -- 172.21.15.66:0/1013807576 shutdown_connections 2024-09-10T22:08:30.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.057+0000 7f0f16000640 1 --2- 172.21.15.66:0/1013807576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f100b32a0 0x7f0f100b14b0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.057+0000 7f0f16000640 1 --2- 172.21.15.66:0/1013807576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f100b0f60 0x7f0f100d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.057+0000 7f0f16000640 1 --2- 172.21.15.66:0/1013807576 >> v2:172.21.15.94:3300/0 conn(0x7f0f1015f460 0x7f0f100cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.057+0000 7f0f16000640 1 -- 172.21.15.66:0/1013807576 >> 172.21.15.66:0/1013807576 conn(0x7f0f100d1d00 msgr2=0x7f0f100d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:30.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.057+0000 7f0f16000640 1 -- 172.21.15.66:0/1013807576 shutdown_connections 2024-09-10T22:08:30.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.057+0000 7f0f16000640 1 -- 172.21.15.66:0/1013807576 wait complete. 2024-09-10T22:08:30.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f16000640 1 Processor -- start 2024-09-10T22:08:30.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f16000640 1 -- start start 2024-09-10T22:08:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f16000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f100b0f60 0x7f0f10155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f16000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f100b32a0 0x7f0f1014ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f16000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f1015f460 0x7f0f1015d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f16000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0f100af750 con 0x7f0f100b0f60 2024-09-10T22:08:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f16000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f100af5d0 con 0x7f0f100b32a0 2024-09-10T22:08:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f16000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0f100af8d0 con 0x7f0f1015f460 2024-09-10T22:08:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f0f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f100b0f60 0x7f0f10155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f0f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f100b0f60 0x7f0f10155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42372/0 (socket says 172.21.15.66:42372) 2024-09-10T22:08:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f0f400640 1 -- 172.21.15.66:0/1279975402 learned_addr learned my addr 172.21.15.66:0/1279975402 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f0ea00640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f100b32a0 0x7f0f1014ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f0fe00640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f1015f460 0x7f0f1015d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f0f400640 1 -- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f1015f460 msgr2=0x7f0f1015d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f0f400640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f1015f460 0x7f0f1015d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f0f400640 1 -- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f100b32a0 msgr2=0x7f0f1014ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f0f400640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f100b32a0 0x7f0f1014ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f0f400640 1 -- 172.21.15.66:0/1279975402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f100cf620 con 0x7f0f100b0f60 2024-09-10T22:08:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f0f400640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f100b0f60 0x7f0f10155b40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000e7a0 tx=0x7f0f0000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f07400640 1 -- 172.21.15.66:0/1279975402 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000ccc0 con 0x7f0f100b0f60 2024-09-10T22:08:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f07400640 1 -- 172.21.15.66:0/1279975402 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f00010430 con 0x7f0f100b0f60 2024-09-10T22:08:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f07400640 1 -- 172.21.15.66:0/1279975402 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00010860 con 0x7f0f100b0f60 2024-09-10T22:08:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f16000640 1 -- 172.21.15.66:0/1279975402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f101f6130 con 0x7f0f100b0f60 2024-09-10T22:08:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f16000640 1 -- 172.21.15.66:0/1279975402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f101f66f0 con 0x7f0f100b0f60 2024-09-10T22:08:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.061+0000 7f0f16000640 1 -- 172.21.15.66:0/1279975402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f100b14b0 con 0x7f0f100b0f60 2024-09-10T22:08:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.065+0000 7f0f07400640 1 -- 172.21.15.66:0/1279975402 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f0f00010a00 con 0x7f0f100b0f60 2024-09-10T22:08:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.065+0000 7f0f07400640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ef00736c0 0x7f0ef0075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.065+0000 7f0f07400640 1 -- 172.21.15.66:0/1279975402 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f0f00092810 con 0x7f0f100b0f60 2024-09-10T22:08:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.065+0000 7f0f0ea00640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ef00736c0 0x7f0ef0075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.065+0000 7f0f0ea00640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ef00736c0 0x7f0ef0075b80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0f100cc1c0 tx=0x7f0efc0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:30.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.069+0000 7f0f07400640 1 -- 172.21.15.66:0/1279975402 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0f0005f940 con 0x7f0f100b0f60 2024-09-10T22:08:30.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.241+0000 7f0f16000640 1 -- 172.21.15.66:0/1279975402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0f10074aa0 con 0x7f0f100b0f60 2024-09-10T22:08:30.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.241+0000 7f0f07400640 1 -- 172.21.15.66:0/1279975402 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7f0f0005f240 con 0x7f0f100b0f60 2024-09-10T22:08:30.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.245+0000 7f0f16000640 1 -- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ef00736c0 msgr2=0x7f0ef0075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.245+0000 7f0f16000640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ef00736c0 0x7f0ef0075b80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0f100cc1c0 tx=0x7f0efc0091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.245+0000 7f0f16000640 1 -- 172.21.15.66:0/1279975402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f100b0f60 msgr2=0x7f0f10155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.245+0000 7f0f16000640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f100b0f60 0x7f0f10155b40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000e7a0 tx=0x7f0f0000ec60 comp rx=0 tx=0).stop 2024-09-10T22:08:30.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.245+0000 7f0f16000640 1 -- 172.21.15.66:0/1279975402 shutdown_connections 2024-09-10T22:08:30.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.245+0000 7f0f16000640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ef00736c0 0x7f0ef0075b80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.245+0000 7f0f16000640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f1015f460 0x7f0f1015d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.245+0000 7f0f16000640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f100b32a0 0x7f0f1014ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.245+0000 7f0f16000640 1 --2- 172.21.15.66:0/1279975402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f100b0f60 0x7f0f10155b40 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.245+0000 7f0f16000640 1 -- 172.21.15.66:0/1279975402 >> 172.21.15.66:0/1279975402 conn(0x7f0f100d1d00 msgr2=0x7f0f10070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:30.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.245+0000 7f0f16000640 1 -- 172.21.15.66:0/1279975402 shutdown_connections 2024-09-10T22:08:30.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.245+0000 7f0f16000640 1 -- 172.21.15.66:0/1279975402 wait complete. 2024-09-10T22:08:30.271 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-10T22:08:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.413+0000 7fa4e1c00640 1 -- 172.21.15.66:0/1369384694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4dc0c4b10 msgr2=0x7fa4dc06e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.413+0000 7fa4e1c00640 1 --2- 172.21.15.66:0/1369384694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4dc0c4b10 0x7fa4dc06e960 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0c6a10 tx=0x7fa4cc01c8c0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.413+0000 7fa4e1c00640 1 -- 172.21.15.66:0/1369384694 shutdown_connections 2024-09-10T22:08:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.413+0000 7fa4e1c00640 1 --2- 172.21.15.66:0/1369384694 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4dc0c4ef0 0x7fa4dc0c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.413+0000 7fa4e1c00640 1 --2- 172.21.15.66:0/1369384694 >> v2:172.21.15.94:3300/0 conn(0x7fa4dc0ae620 0x7fa4dc0725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.413+0000 7fa4e1c00640 1 --2- 172.21.15.66:0/1369384694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4dc0c4b10 0x7fa4dc06e960 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.413+0000 7fa4e1c00640 1 -- 172.21.15.66:0/1369384694 >> 172.21.15.66:0/1369384694 conn(0x7fa4dc0cfa30 msgr2=0x7fa4dc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.413+0000 7fa4e1c00640 1 -- 172.21.15.66:0/1369384694 shutdown_connections 2024-09-10T22:08:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.413+0000 7fa4e1c00640 1 -- 172.21.15.66:0/1369384694 wait complete. 2024-09-10T22:08:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.413+0000 7fa4e1c00640 1 Processor -- start 2024-09-10T22:08:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.413+0000 7fa4e1c00640 1 -- start start 2024-09-10T22:08:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4e1c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4dc0ae620 0x7fa4dc0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4e1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4dc0c4b10 0x7fa4dc0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4e1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4dc0c4ef0 0x7fa4dc0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4e1c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc166cd0 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4e1c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc166b50 con 0x7fa4dc0c4b10 2024-09-10T22:08:30.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4e1c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa4dc166e50 con 0x7fa4dc0ae620 2024-09-10T22:08:30.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4db400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4dc0c4ef0 0x7fa4dc0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:30.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4db400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4dc0c4ef0 0x7fa4dc0cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42392/0 (socket says 172.21.15.66:42392) 2024-09-10T22:08:30.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4db400640 1 -- 172.21.15.66:0/2974662255 learned_addr learned my addr 172.21.15.66:0/2974662255 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:30.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4d3e00640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4dc0c4b10 0x7fa4dc0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:30.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4daa00640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4dc0ae620 0x7fa4dc0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:30.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4db400640 1 -- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4dc0ae620 msgr2=0x7fa4dc0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4db400640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4dc0ae620 0x7fa4dc0adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4db400640 1 -- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4dc0c4b10 msgr2=0x7fa4dc0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4db400640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4dc0c4b10 0x7fa4dc0c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4db400640 1 -- 172.21.15.66:0/2974662255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc0c7e50 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4d3e00640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4dc0c4b10 0x7fa4dc0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:08:30.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4db400640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4dc0c4ef0 0x7fa4dc0cdac0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400ea80 tx=0x7fa4d400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:30.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4d2a00640 1 -- 172.21.15.66:0/2974662255 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4004b30 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4d2a00640 1 -- 172.21.15.66:0/2974662255 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4d4004cf0 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4d2a00640 1 -- 172.21.15.66:0/2974662255 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4005280 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4e1c00640 1 -- 172.21.15.66:0/2974662255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4dc0c8130 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4e1c00640 1 -- 172.21.15.66:0/2974662255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4dc0ae060 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4e1c00640 1 -- 172.21.15.66:0/2974662255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4dc06e960 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.417+0000 7fa4d2a00640 1 -- 172.21.15.66:0/2974662255 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7fa4d4005420 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.421+0000 7fa4d2a00640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa4b80736c0 0x7fa4b8075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.421+0000 7fa4d2a00640 1 -- 172.21.15.66:0/2974662255 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fa4d4092e10 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.421+0000 7fa4daa00640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa4b80736c0 0x7fa4b8075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.421+0000 7fa4daa00640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa4b80736c0 0x7fa4b8075b80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0af000 tx=0x7fa4cc01b100 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:30.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.425+0000 7fa4d2a00640 1 -- 172.21.15.66:0/2974662255 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa4d405fc00 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.601+0000 7fa4e1c00640 1 -- 172.21.15.66:0/2974662255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fa4dc072750 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.601+0000 7fa4d2a00640 1 -- 172.21.15.66:0/2974662255 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v41) ==== 116+0+2 (secure 0 0 0) 0x7fa4d4068320 con 0x7fa4dc0c4ef0 2024-09-10T22:08:30.609 INFO:teuthology.orchestra.run.smithi066.stdout:4 2024-09-10T22:08:30.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.605+0000 7fa4e1c00640 1 -- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa4b80736c0 msgr2=0x7fa4b8075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.605+0000 7fa4e1c00640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa4b80736c0 0x7fa4b8075b80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0af000 tx=0x7fa4cc01b100 comp rx=0 tx=0).stop 2024-09-10T22:08:30.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.605+0000 7fa4e1c00640 1 -- 172.21.15.66:0/2974662255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4dc0c4ef0 msgr2=0x7fa4dc0cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.605+0000 7fa4e1c00640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4dc0c4ef0 0x7fa4dc0cdac0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400ea80 tx=0x7fa4d400ef40 comp rx=0 tx=0).stop 2024-09-10T22:08:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.605+0000 7fa4e1c00640 1 -- 172.21.15.66:0/2974662255 shutdown_connections 2024-09-10T22:08:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.605+0000 7fa4e1c00640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa4b80736c0 0x7fa4b8075b80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.605+0000 7fa4e1c00640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4dc0c4ef0 0x7fa4dc0cdac0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.605+0000 7fa4e1c00640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4dc0c4b10 0x7fa4dc0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.605+0000 7fa4e1c00640 1 --2- 172.21.15.66:0/2974662255 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4dc0ae620 0x7fa4dc0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.605+0000 7fa4e1c00640 1 -- 172.21.15.66:0/2974662255 >> 172.21.15.66:0/2974662255 conn(0x7fa4dc0cfa30 msgr2=0x7fa4dc15d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.605+0000 7fa4e1c00640 1 -- 172.21.15.66:0/2974662255 shutdown_connections 2024-09-10T22:08:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.605+0000 7fa4e1c00640 1 -- 172.21.15.66:0/2974662255 wait complete. 2024-09-10T22:08:30.630 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:08:30.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.773+0000 7f267a200640 1 -- 172.21.15.66:0/209464440 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26740aee20 msgr2=0x7f26740ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.773+0000 7f267a200640 1 --2- 172.21.15.66:0/209464440 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26740aee20 0x7f26740ca080 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f265c009870 tx=0x7f265c01c7d0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.773+0000 7f267a200640 1 -- 172.21.15.66:0/209464440 shutdown_connections 2024-09-10T22:08:30.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.773+0000 7f267a200640 1 --2- 172.21.15.66:0/209464440 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26740aee20 0x7f26740ca080 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.773+0000 7f267a200640 1 -- 172.21.15.66:0/209464440 >> 172.21.15.66:0/209464440 conn(0x7f26740cfa30 msgr2=0x7f26740cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:30.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.773+0000 7f267a200640 1 -- 172.21.15.66:0/209464440 shutdown_connections 2024-09-10T22:08:30.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.773+0000 7f267a200640 1 -- 172.21.15.66:0/209464440 wait complete. 2024-09-10T22:08:30.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.773+0000 7f267a200640 1 Processor -- start 2024-09-10T22:08:30.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.773+0000 7f267a200640 1 -- start start 2024-09-10T22:08:30.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f267a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26740aee20 0x7f267415e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:30.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f267a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2674159900 0x7f267415e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:30.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f267a200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f267415cc80 0x7f2674158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:30.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f267a200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f267416a300 con 0x7f2674159900 2024-09-10T22:08:30.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f267a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f267416a180 con 0x7f26740aee20 2024-09-10T22:08:30.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f267a200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f267416a480 con 0x7f267415cc80 2024-09-10T22:08:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f266be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2674159900 0x7f267415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f266be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2674159900 0x7f267415e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42400/0 (socket says 172.21.15.66:42400) 2024-09-10T22:08:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f266be00640 1 -- 172.21.15.66:0/2944309365 learned_addr learned my addr 172.21.15.66:0/2944309365 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f2673e00640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f267415cc80 0x7f2674158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f2673400640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26740aee20 0x7f267415e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f266be00640 1 -- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f267415cc80 msgr2=0x7f2674158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f266be00640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f267415cc80 0x7f2674158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f266be00640 1 -- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26740aee20 msgr2=0x7f267415e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f266be00640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26740aee20 0x7f267415e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f266be00640 1 -- 172.21.15.66:0/2944309365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26741f8500 con 0x7f2674159900 2024-09-10T22:08:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f2673e00640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f267415cc80 0x7f2674158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f266be00640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2674159900 0x7f267415e050 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f265c0193e0 tx=0x7f265c0a6530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f266b400640 1 -- 172.21.15.66:0/2944309365 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c005040 con 0x7f2674159900 2024-09-10T22:08:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f266b400640 1 -- 172.21.15.66:0/2944309365 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f265c0051e0 con 0x7f2674159900 2024-09-10T22:08:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f267a200640 1 -- 172.21.15.66:0/2944309365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26741f87e0 con 0x7f2674159900 2024-09-10T22:08:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f266b400640 1 -- 172.21.15.66:0/2944309365 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c005630 con 0x7f2674159900 2024-09-10T22:08:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f267a200640 1 -- 172.21.15.66:0/2944309365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267415d850 con 0x7f2674159900 2024-09-10T22:08:30.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.777+0000 7f267a200640 1 -- 172.21.15.66:0/2944309365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26740ca080 con 0x7f2674159900 2024-09-10T22:08:30.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.781+0000 7f266b400640 1 -- 172.21.15.66:0/2944309365 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f26740ce850 con 0x7f2674159900 2024-09-10T22:08:30.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.781+0000 7f266b400640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f26500736c0 0x7f2650075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:30.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.781+0000 7f266b400640 1 -- 172.21.15.66:0/2944309365 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f265c12c3d0 con 0x7f2674159900 2024-09-10T22:08:30.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.781+0000 7f2673400640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f26500736c0 0x7f2650075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:30.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.781+0000 7f2673400640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f26500736c0 0x7f2650075b80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2674154f40 tx=0x7f2664009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:30.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.785+0000 7f266b400640 1 -- 172.21.15.66:0/2944309365 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f265c0ac030 con 0x7f2674159900 2024-09-10T22:08:30.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.961+0000 7f267a200640 1 -- 172.21.15.66:0/2944309365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f2674070470 con 0x7f2674159900 2024-09-10T22:08:30.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f266b400640 1 -- 172.21.15.66:0/2944309365 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v42)=0 v42) ==== 131+0+0 (secure 0 0 0) 0x7f265c016450 con 0x7f2674159900 2024-09-10T22:08:30.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f267a200640 1 -- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f26500736c0 msgr2=0x7f2650075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f267a200640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f26500736c0 0x7f2650075b80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2674154f40 tx=0x7f2664009040 comp rx=0 tx=0).stop 2024-09-10T22:08:30.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f267a200640 1 -- 172.21.15.66:0/2944309365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2674159900 msgr2=0x7f267415e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:30.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f267a200640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2674159900 0x7f267415e050 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f265c0193e0 tx=0x7f265c0a6530 comp rx=0 tx=0).stop 2024-09-10T22:08:30.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f267a200640 1 -- 172.21.15.66:0/2944309365 shutdown_connections 2024-09-10T22:08:30.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f267a200640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f26500736c0 0x7f2650075b80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f267a200640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f267415cc80 0x7f2674158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f267a200640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2674159900 0x7f267415e050 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f267a200640 1 --2- 172.21.15.66:0/2944309365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26740aee20 0x7f267415e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:30.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f267a200640 1 -- 172.21.15.66:0/2944309365 >> 172.21.15.66:0/2944309365 conn(0x7f26740cfa30 msgr2=0x7f26740740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:30.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f267a200640 1 -- 172.21.15.66:0/2944309365 shutdown_connections 2024-09-10T22:08:30.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:30.965+0000 7f267a200640 1 -- 172.21.15.66:0/2944309365 wait complete. 2024-09-10T22:08:30.992 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:30.992+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:30.992 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:30.992+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:30.992 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:30.992+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:30.992 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:30.992+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:30.993 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:30.992+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:30.993 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-10T22:08:31.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.137+0000 7f1986a00640 1 -- 172.21.15.66:0/2486206673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1980164a70 msgr2=0x7f1980166e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.137+0000 7f1986a00640 1 --2- 172.21.15.66:0/2486206673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1980164a70 0x7f1980166e60 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1978009bc0 tx=0x7f197801cb10 comp rx=0 tx=0).stop 2024-09-10T22:08:31.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.137+0000 7f1986a00640 1 -- 172.21.15.66:0/2486206673 shutdown_connections 2024-09-10T22:08:31.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.137+0000 7f1986a00640 1 --2- 172.21.15.66:0/2486206673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1980164a70 0x7f1980166e60 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.137+0000 7f1986a00640 1 -- 172.21.15.66:0/2486206673 >> 172.21.15.66:0/2486206673 conn(0x7f1980074240 msgr2=0x7f1980074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:31.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.137+0000 7f1986a00640 1 -- 172.21.15.66:0/2486206673 shutdown_connections 2024-09-10T22:08:31.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.137+0000 7f1986a00640 1 -- 172.21.15.66:0/2486206673 wait complete. 2024-09-10T22:08:31.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.137+0000 7f1986a00640 1 Processor -- start 2024-09-10T22:08:31.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1986a00640 1 -- start start 2024-09-10T22:08:31.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1986a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19800b0590 0x7f19800c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1986a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f198015bfb0 0x7f19800aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1986a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1980164a70 0x7f1980094670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1986a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1980169d40 con 0x7f19800b0590 2024-09-10T22:08:31.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1986a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1980169bc0 con 0x7f198015bfb0 2024-09-10T22:08:31.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1986a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1980169ec0 con 0x7f1980164a70 2024-09-10T22:08:31.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f197fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19800b0590 0x7f19800c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f197fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19800b0590 0x7f19800c30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42416/0 (socket says 172.21.15.66:42416) 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f197fe00640 1 -- 172.21.15.66:0/1469729169 learned_addr learned my addr 172.21.15.66:0/1469729169 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1984a00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1980164a70 0x7f1980094670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1977e00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f198015bfb0 0x7f19800aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f197fe00640 1 -- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1980164a70 msgr2=0x7f1980094670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f197fe00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1980164a70 0x7f1980094670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f197fe00640 1 -- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f198015bfb0 msgr2=0x7f19800aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f197fe00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f198015bfb0 0x7f19800aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f197fe00640 1 -- 172.21.15.66:0/1469729169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19800cc780 con 0x7f19800b0590 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1977e00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f198015bfb0 0x7f19800aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f197fe00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19800b0590 0x7f19800c30f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f197000c500 tx=0x7f197000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f197cc00640 1 -- 172.21.15.66:0/1469729169 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970017070 con 0x7f19800b0590 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f197cc00640 1 -- 172.21.15.66:0/1469729169 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19700053c0 con 0x7f19800b0590 2024-09-10T22:08:31.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f197cc00640 1 -- 172.21.15.66:0/1469729169 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970012890 con 0x7f19800b0590 2024-09-10T22:08:31.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1986a00640 1 -- 172.21.15.66:0/1469729169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19800ae250 con 0x7f19800b0590 2024-09-10T22:08:31.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1986a00640 1 -- 172.21.15.66:0/1469729169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19800c3630 con 0x7f19800b0590 2024-09-10T22:08:31.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.141+0000 7f1986a00640 1 -- 172.21.15.66:0/1469729169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19800ccd40 con 0x7f19800b0590 2024-09-10T22:08:31.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.145+0000 7f197cc00640 1 -- 172.21.15.66:0/1469729169 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f1970003600 con 0x7f19800b0590 2024-09-10T22:08:31.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.145+0000 7f197cc00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f195c0736c0 0x7f195c075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.145+0000 7f197cc00640 1 -- 172.21.15.66:0/1469729169 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f1970091770 con 0x7f19800b0590 2024-09-10T22:08:31.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.145+0000 7f1977e00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f195c0736c0 0x7f195c075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.145+0000 7f1977e00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f195c0736c0 0x7f195c075b80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1968007a60 tx=0x7f1968009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:31.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.149+0000 7f197cc00640 1 -- 172.21.15.66:0/1469729169 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f197005de90 con 0x7f19800b0590 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.325+0000 7f1986a00640 1 -- 172.21.15.66:0/1469729169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f19800cc950 con 0x7f19800b0590 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f197cc00640 1 -- 172.21.15.66:0/1469729169 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v43)=0 v43) ==== 136+0+0 (secure 0 0 0) 0x7f19700628c0 con 0x7f19800b0590 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f1986a00640 1 -- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f195c0736c0 msgr2=0x7f195c075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f1986a00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f195c0736c0 0x7f195c075b80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1968007a60 tx=0x7f1968009040 comp rx=0 tx=0).stop 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f1986a00640 1 -- 172.21.15.66:0/1469729169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19800b0590 msgr2=0x7f19800c30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f1986a00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19800b0590 0x7f19800c30f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f197000c500 tx=0x7f197000c9c0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f1986a00640 1 -- 172.21.15.66:0/1469729169 shutdown_connections 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f1986a00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f195c0736c0 0x7f195c075b80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f1986a00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1980164a70 0x7f1980094670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f1986a00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f198015bfb0 0x7f19800aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f1986a00640 1 --2- 172.21.15.66:0/1469729169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19800b0590 0x7f19800c30f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f1986a00640 1 -- 172.21.15.66:0/1469729169 >> 172.21.15.66:0/1469729169 conn(0x7f1980074240 msgr2=0x7f19800d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f1986a00640 1 -- 172.21.15.66:0/1469729169 shutdown_connections 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.329+0000 7f1986a00640 1 -- 172.21.15.66:0/1469729169 wait complete. 2024-09-10T22:08:31.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-10T22:08:31.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.505+0000 7f3114c00640 1 -- 172.21.15.66:0/934010217 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31100d22c0 msgr2=0x7f31101668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.505+0000 7f3114c00640 1 --2- 172.21.15.66:0/934010217 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31100d22c0 0x7f31101668d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f31100cdc80 tx=0x7f310801cb30 comp rx=0 tx=0).stop 2024-09-10T22:08:31.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.505+0000 7f3114c00640 1 -- 172.21.15.66:0/934010217 shutdown_connections 2024-09-10T22:08:31.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.505+0000 7f3114c00640 1 --2- 172.21.15.66:0/934010217 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31100d22c0 0x7f31101668d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.505+0000 7f3114c00640 1 -- 172.21.15.66:0/934010217 >> 172.21.15.66:0/934010217 conn(0x7f311009aa50 msgr2=0x7f311009ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:31.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.505+0000 7f3114c00640 1 -- 172.21.15.66:0/934010217 shutdown_connections 2024-09-10T22:08:31.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.505+0000 7f3114c00640 1 -- 172.21.15.66:0/934010217 wait complete. 2024-09-10T22:08:31.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.505+0000 7f3114c00640 1 Processor -- start 2024-09-10T22:08:31.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.505+0000 7f3114c00640 1 -- start start 2024-09-10T22:08:31.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f3114c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31100d22c0 0x7f3110147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f3114c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f311015bb40 0x7f31100c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f3114c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f311015bf20 0x7f31100c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f3114c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f311016a110 con 0x7f311015bb40 2024-09-10T22:08:31.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f3114c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3110169f90 con 0x7f31100d22c0 2024-09-10T22:08:31.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f3114c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f311016a290 con 0x7f311015bf20 2024-09-10T22:08:31.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f310d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f311015bb40 0x7f31100c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f310d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f311015bb40 0x7f31100c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42424/0 (socket says 172.21.15.66:42424) 2024-09-10T22:08:31.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f310d400640 1 -- 172.21.15.66:0/1322115635 learned_addr learned my addr 172.21.15.66:0/1322115635 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:31.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f310e800640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f311015bf20 0x7f31100c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f310de00640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31100d22c0 0x7f3110147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f310d400640 1 -- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f311015bf20 msgr2=0x7f31100c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f310d400640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f311015bf20 0x7f31100c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f310d400640 1 -- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31100d22c0 msgr2=0x7f3110147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f310d400640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31100d22c0 0x7f3110147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f310d400640 1 -- 172.21.15.66:0/1322115635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311009e7d0 con 0x7f311015bb40 2024-09-10T22:08:31.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f310de00640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31100d22c0 0x7f3110147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:31.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f310d400640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f311015bb40 0x7f31100c77c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00c500 tx=0x7f30fc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:31.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f3106000640 1 -- 172.21.15.66:0/1322115635 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc017070 con 0x7f311015bb40 2024-09-10T22:08:31.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f3106000640 1 -- 172.21.15.66:0/1322115635 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30fc0053c0 con 0x7f311015bb40 2024-09-10T22:08:31.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f3106000640 1 -- 172.21.15.66:0/1322115635 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc012890 con 0x7f311015bb40 2024-09-10T22:08:31.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f3114c00640 1 -- 172.21.15.66:0/1322115635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311009eab0 con 0x7f311015bb40 2024-09-10T22:08:31.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f3114c00640 1 -- 172.21.15.66:0/1322115635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311009f070 con 0x7f311015bb40 2024-09-10T22:08:31.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.509+0000 7f3114c00640 1 -- 172.21.15.66:0/1322115635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31100cc730 con 0x7f311015bb40 2024-09-10T22:08:31.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.513+0000 7f3106000640 1 -- 172.21.15.66:0/1322115635 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f30fc003600 con 0x7f311015bb40 2024-09-10T22:08:31.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.513+0000 7f3106000640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f30f00736c0 0x7f30f0075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.513+0000 7f3106000640 1 -- 172.21.15.66:0/1322115635 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f30fc091700 con 0x7f311015bb40 2024-09-10T22:08:31.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.513+0000 7f310de00640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f30f00736c0 0x7f30f0075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.513+0000 7f310de00640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f30f00736c0 0x7f30f0075b80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f31000046c0 tx=0x7f3100009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:31.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.517+0000 7f3106000640 1 -- 172.21.15.66:0/1322115635 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f30fc05de20 con 0x7f311015bb40 2024-09-10T22:08:31.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.713+0000 7f3114c00640 1 -- 172.21.15.66:0/1322115635 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f31100b63f0 con 0x7f30f00736c0 2024-09-10T22:08:31.732 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:31.732+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:31.732 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:31.732+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:31.732 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:31.732+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:31.732 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:31.732+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:31.732 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:31.732+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:31.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3106000640 1 -- 172.21.15.66:0/1322115635 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f31100b63f0 con 0x7f30f00736c0 2024-09-10T22:08:31.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3114c00640 1 -- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f30f00736c0 msgr2=0x7f30f0075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3114c00640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f30f00736c0 0x7f30f0075b80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f31000046c0 tx=0x7f3100009040 comp rx=0 tx=0).stop 2024-09-10T22:08:31.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3114c00640 1 -- 172.21.15.66:0/1322115635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f311015bb40 msgr2=0x7f31100c77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3114c00640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f311015bb40 0x7f31100c77c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00c500 tx=0x7f30fc00c9c0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3114c00640 1 -- 172.21.15.66:0/1322115635 shutdown_connections 2024-09-10T22:08:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3114c00640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f30f00736c0 0x7f30f0075b80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3114c00640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f311015bf20 0x7f31100c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3114c00640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f311015bb40 0x7f31100c77c0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3114c00640 1 --2- 172.21.15.66:0/1322115635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31100d22c0 0x7f3110147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3114c00640 1 -- 172.21.15.66:0/1322115635 >> 172.21.15.66:0/1322115635 conn(0x7f311009aa50 msgr2=0x7f31100d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3114c00640 1 -- 172.21.15.66:0/1322115635 shutdown_connections 2024-09-10T22:08:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.761+0000 7f3114c00640 1 -- 172.21.15.66:0/1322115635 wait complete. 2024-09-10T22:08:31.788 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:08:31.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 -- 172.21.15.66:0/3522150757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f689c003a60 msgr2=0x7f689c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 --2- 172.21.15.66:0/3522150757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f689c003a60 0x7f689c003e80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009b30 tx=0x7f68a801d1e0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 -- 172.21.15.66:0/3522150757 shutdown_connections 2024-09-10T22:08:31.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 --2- 172.21.15.66:0/3522150757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f689c003a60 0x7f689c003e80 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 -- 172.21.15.66:0/3522150757 >> 172.21.15.66:0/3522150757 conn(0x7f68b40d1c60 msgr2=0x7f68b40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:31.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 -- 172.21.15.66:0/3522150757 shutdown_connections 2024-09-10T22:08:31.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 -- 172.21.15.66:0/3522150757 wait complete. 2024-09-10T22:08:31.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 Processor -- start 2024-09-10T22:08:31.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 -- start start 2024-09-10T22:08:31.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f689c003a60 0x7f68b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68b41fb270 0x7f68b41ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b41fb6d0 0x7f68b41ec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f68b40d0670 con 0x7f689c003a60 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68b40d04f0 con 0x7f68b41fb6d0 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68bb600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f68b40d07f0 con 0x7f68b41fb270 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68b8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f689c003a60 0x7f68b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68b8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f689c003a60 0x7f68b406f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42442/0 (socket says 172.21.15.66:42442) 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68b8c00640 1 -- 172.21.15.66:0/3307219082 learned_addr learned my addr 172.21.15.66:0/3307219082 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68b3e00640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68b41fb270 0x7f68b41ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68b9600640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b41fb6d0 0x7f68b41ec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68b8c00640 1 -- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68b41fb270 msgr2=0x7f68b41ebfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68b8c00640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68b41fb270 0x7f68b41ebfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68b8c00640 1 -- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b41fb6d0 msgr2=0x7f68b41ec4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68b8c00640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b41fb6d0 0x7f68b41ec4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.933+0000 7f68b8c00640 1 -- 172.21.15.66:0/3307219082 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b406f6e0 con 0x7f689c003a60 2024-09-10T22:08:31.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.937+0000 7f68b8c00640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f689c003a60 0x7f68b406f1a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f68a80af040 tx=0x7f68a8005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:31.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.937+0000 7f68b0c00640 1 -- 172.21.15.66:0/3307219082 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a80b1040 con 0x7f689c003a60 2024-09-10T22:08:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.937+0000 7f68b0c00640 1 -- 172.21.15.66:0/3307219082 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68a8004120 con 0x7f689c003a60 2024-09-10T22:08:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.937+0000 7f68b0c00640 1 -- 172.21.15.66:0/3307219082 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a80ac7b0 con 0x7f689c003a60 2024-09-10T22:08:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.937+0000 7f68bb600640 1 -- 172.21.15.66:0/3307219082 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b406dd30 con 0x7f689c003a60 2024-09-10T22:08:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.937+0000 7f68bb600640 1 -- 172.21.15.66:0/3307219082 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b406e2f0 con 0x7f689c003a60 2024-09-10T22:08:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.937+0000 7f68bb600640 1 -- 172.21.15.66:0/3307219082 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b40aedd0 con 0x7f689c003a60 2024-09-10T22:08:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.937+0000 7f68b0c00640 1 -- 172.21.15.66:0/3307219082 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 27) ==== 97924+0+0 (secure 0 0 0) 0x7f68a80ac950 con 0x7f689c003a60 2024-09-10T22:08:31.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.937+0000 7f68b0c00640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f68940736c0 0x7f6894075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.937+0000 7f68b0c00640 1 -- 172.21.15.66:0/3307219082 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f68a812cf90 con 0x7f689c003a60 2024-09-10T22:08:31.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.937+0000 7f68b3e00640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f68940736c0 0x7f6894075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.941+0000 7f68b3e00640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f68940736c0 0x7f6894075b80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f68b406f130 tx=0x7f68a00067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:31.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:31.941+0000 7f68b0c00640 1 -- 172.21.15.66:0/3307219082 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f68a8016f20 con 0x7f689c003a60 2024-09-10T22:08:32.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.141+0000 7f68bb600640 1 -- 172.21.15.66:0/3307219082 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f68b40d2a40 con 0x7f68940736c0 2024-09-10T22:08:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68b0c00640 1 -- 172.21.15.66:0/3307219082 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f68b40d2a40 con 0x7f68940736c0 2024-09-10T22:08:32.155 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/bd8f889e-ea1f-4e34-9b9e-798a339d5687 2024-09-10T22:08:32.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68bb600640 1 -- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f68940736c0 msgr2=0x7f6894075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68bb600640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f68940736c0 0x7f6894075b80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f68b406f130 tx=0x7f68a00067b0 comp rx=0 tx=0).stop 2024-09-10T22:08:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68bb600640 1 -- 172.21.15.66:0/3307219082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f689c003a60 msgr2=0x7f68b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68bb600640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f689c003a60 0x7f68b406f1a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f68a80af040 tx=0x7f68a8005520 comp rx=0 tx=0).stop 2024-09-10T22:08:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68bb600640 1 -- 172.21.15.66:0/3307219082 shutdown_connections 2024-09-10T22:08:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68bb600640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f68940736c0 0x7f6894075b80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68bb600640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b41fb6d0 0x7f68b41ec4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68bb600640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68b41fb270 0x7f68b41ebfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68bb600640 1 --2- 172.21.15.66:0/3307219082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f689c003a60 0x7f68b406f1a0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68bb600640 1 -- 172.21.15.66:0/3307219082 >> 172.21.15.66:0/3307219082 conn(0x7f68b40d1c60 msgr2=0x7f68b40b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68bb600640 1 -- 172.21.15.66:0/3307219082 shutdown_connections 2024-09-10T22:08:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:32.149+0000 7f68bb600640 1 -- 172.21.15.66:0/3307219082 wait complete. 2024-09-10T22:08:32.177 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/bd8f889e-ea1f-4e34-9b9e-798a339d5687 2024-09-10T22:08:32.177 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:08:32.177 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:08:32.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:32.186 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bd8f889e-ea1f-4e34-9b9e-798a339d5687/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:08:39.383 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:08:39.383 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:08:39.383 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.12142 s, 151 MB/s 2024-09-10T22:08:39.384 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:08:39.384 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:08:39.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:39.393 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bd8f889e-ea1f-4e34-9b9e-798a339d5687/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:08:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:08:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:08:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.86995 s, 156 MB/s 2024-09-10T22:08:46.317 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:08:46.318 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:08:46.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:46.326 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bd8f889e-ea1f-4e34-9b9e-798a339d5687/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:08:53.289 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:08:53.289 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:08:53.290 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.907 s, 155 MB/s 2024-09-10T22:08:53.291 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-10T22:08:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 -- 172.21.15.66:0/1523906846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905c0cce70 msgr2=0x7f905c0cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 --2- 172.21.15.66:0/1523906846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905c0cce70 0x7f905c0cbfa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9054009c80 tx=0x7f905401cc50 comp rx=0 tx=0).stop 2024-09-10T22:08:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 -- 172.21.15.66:0/1523906846 shutdown_connections 2024-09-10T22:08:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 --2- 172.21.15.66:0/1523906846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905c0cce70 0x7f905c0cbfa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 --2- 172.21.15.66:0/1523906846 >> v2:172.21.15.94:3300/0 conn(0x7f905c072d50 0x7f905c06e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 -- 172.21.15.66:0/1523906846 >> 172.21.15.66:0/1523906846 conn(0x7f905c0cfaa0 msgr2=0x7f905c0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 -- 172.21.15.66:0/1523906846 shutdown_connections 2024-09-10T22:08:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 -- 172.21.15.66:0/1523906846 wait complete. 2024-09-10T22:08:53.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 Processor -- start 2024-09-10T22:08:53.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 -- start start 2024-09-10T22:08:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905c072d50 0x7f905c146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c0cce70 0x7f905c14b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c15ab00 0x7f905c1633b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f905c167bc0 con 0x7f905c15ab00 2024-09-10T22:08:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f905c167a40 con 0x7f905c0cce70 2024-09-10T22:08:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.437+0000 7f9061400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f905c167d40 con 0x7f905c072d50 2024-09-10T22:08:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f9053e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c0cce70 0x7f905c14b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f905a800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905c072d50 0x7f905c146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f905b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c15ab00 0x7f905c1633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f9053e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c0cce70 0x7f905c14b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:39278/0 (socket says 172.21.15.66:39278) 2024-09-10T22:08:53.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f9053e00640 1 -- 172.21.15.66:0/1693446278 learned_addr learned my addr 172.21.15.66:0/1693446278 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:53.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f905a800640 1 -- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c0cce70 msgr2=0x7f905c14b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:53.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f905a800640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c0cce70 0x7f905c14b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:53.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f905a800640 1 -- 172.21.15.66:0/1693446278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c15ab00 msgr2=0x7f905c1633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:53.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f905a800640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c15ab00 0x7f905c1633b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:53.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f905a800640 1 -- 172.21.15.66:0/1693446278 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905c1473f0 con 0x7f905c072d50 2024-09-10T22:08:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f905b200640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c15ab00 0x7f905c1633b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f905a800640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905c072d50 0x7f905c146eb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c390 tx=0x7f904c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f9052a00640 1 -- 172.21.15.66:0/1693446278 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c017070 con 0x7f905c072d50 2024-09-10T22:08:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f9052a00640 1 -- 172.21.15.66:0/1693446278 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f904c0053c0 con 0x7f905c072d50 2024-09-10T22:08:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f9052a00640 1 -- 172.21.15.66:0/1693446278 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c003840 con 0x7f905c072d50 2024-09-10T22:08:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f9061400640 1 -- 172.21.15.66:0/1693446278 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905c159260 con 0x7f905c072d50 2024-09-10T22:08:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f9061400640 1 -- 172.21.15.66:0/1693446278 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f905c1628d0 con 0x7f905c072d50 2024-09-10T22:08:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f9061400640 1 -- 172.21.15.66:0/1693446278 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905c0d07c0 con 0x7f905c072d50 2024-09-10T22:08:53.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.441+0000 7f9052a00640 1 -- 172.21.15.66:0/1693446278 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 28) ==== 98026+0+0 (secure 0 0 0) 0x7f904c0039e0 con 0x7f905c072d50 2024-09-10T22:08:53.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.445+0000 7f9052a00640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f90380737e0 0x7f9038075ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:53.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.445+0000 7f9052a00640 1 -- 172.21.15.66:0/1693446278 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f904c0916d0 con 0x7f905c072d50 2024-09-10T22:08:53.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.445+0000 7f9053e00640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f90380737e0 0x7f9038075ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:53.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.445+0000 7f9053e00640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f90380737e0 0x7f9038075ca0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f905c14cef0 tx=0x7f90440091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.449+0000 7f9052a00640 1 -- 172.21.15.66:0/1693446278 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f905c1473f0 con 0x7f905c072d50 2024-09-10T22:08:53.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.653+0000 7f9061400640 1 -- 172.21.15.66:0/1693446278 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f905c158f40 con 0x7f90380737e0 2024-09-10T22:08:53.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.665+0000 7f9052a00640 1 -- 172.21.15.66:0/1693446278 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f905c158f40 con 0x7f90380737e0 2024-09-10T22:08:53.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.665+0000 7f9061400640 1 -- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f90380737e0 msgr2=0x7f9038075ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:53.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.665+0000 7f9061400640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f90380737e0 0x7f9038075ca0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f905c14cef0 tx=0x7f90440091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:53.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.665+0000 7f9061400640 1 -- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905c072d50 msgr2=0x7f905c146eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:53.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.665+0000 7f9061400640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905c072d50 0x7f905c146eb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c390 tx=0x7f904c00c850 comp rx=0 tx=0).stop 2024-09-10T22:08:53.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.665+0000 7f9061400640 1 -- 172.21.15.66:0/1693446278 shutdown_connections 2024-09-10T22:08:53.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.665+0000 7f9061400640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f90380737e0 0x7f9038075ca0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:53.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.665+0000 7f9061400640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c15ab00 0x7f905c1633b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:53.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.665+0000 7f9061400640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c0cce70 0x7f905c14b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:53.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.665+0000 7f9061400640 1 --2- 172.21.15.66:0/1693446278 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f905c072d50 0x7f905c146eb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:53.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.665+0000 7f9061400640 1 -- 172.21.15.66:0/1693446278 >> 172.21.15.66:0/1693446278 conn(0x7f905c0cfaa0 msgr2=0x7f905c0cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:53.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.669+0000 7f9061400640 1 -- 172.21.15.66:0/1693446278 shutdown_connections 2024-09-10T22:08:53.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.669+0000 7f9061400640 1 -- 172.21.15.66:0/1693446278 wait complete. 2024-09-10T22:08:53.693 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:08:53.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.837+0000 7fd5bb800640 1 -- 172.21.15.66:0/3371200383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd59c004d50 msgr2=0x7fd59c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:53.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.837+0000 7fd5bb800640 1 --2- 172.21.15.66:0/3371200383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd59c004d50 0x7fd59c0051d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009eb0 tx=0x7fd5ac01d5b0 comp rx=0 tx=0).stop 2024-09-10T22:08:53.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.837+0000 7fd5b9800640 1 -- 172.21.15.66:0/3371200383 reap_dead start 2024-09-10T22:08:53.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.837+0000 7fd5bb800640 1 -- 172.21.15.66:0/3371200383 shutdown_connections 2024-09-10T22:08:53.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.837+0000 7fd5bb800640 1 -- 172.21.15.66:0/3371200383 >> 172.21.15.66:0/3371200383 conn(0x7fd5b40cfa30 msgr2=0x7fd5b40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:53.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.837+0000 7fd5bb800640 1 -- 172.21.15.66:0/3371200383 shutdown_connections 2024-09-10T22:08:53.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.837+0000 7fd5bb800640 1 -- 172.21.15.66:0/3371200383 wait complete. 2024-09-10T22:08:53.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5bb800640 1 Processor -- start 2024-09-10T22:08:53.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5bb800640 1 -- start start 2024-09-10T22:08:53.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5bb800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd59c004d50 0x7fd5b415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:53.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5bb800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b40b2e30 0x7fd5b4154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:53.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5bb800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5b41f2e80 0x7fd5b40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:53.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5bb800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b40cdab0 con 0x7fd5b41f2e80 2024-09-10T22:08:53.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5bb800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b40cd930 con 0x7fd5b40b2e30 2024-09-10T22:08:53.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5bb800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b40cdc30 con 0x7fd59c004d50 2024-09-10T22:08:53.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5b41f2e80 0x7fd5b40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:53.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5b41f2e80 0x7fd5b40736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59610/0 (socket says 172.21.15.66:59610) 2024-09-10T22:08:53.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b8e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd59c004d50 0x7fd5b415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:53.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b9800640 1 -- 172.21.15.66:0/2677170337 learned_addr learned my addr 172.21.15.66:0/2677170337 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:53.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b3e00640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b40b2e30 0x7fd5b4154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:53.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b9800640 1 -- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd59c004d50 msgr2=0x7fd5b415b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:53.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b9800640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd59c004d50 0x7fd5b415b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:53.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b9800640 1 -- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b40b2e30 msgr2=0x7fd5b4154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:53.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b9800640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b40b2e30 0x7fd5b4154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:53.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b9800640 1 -- 172.21.15.66:0/2677170337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b415b0b0 con 0x7fd5b41f2e80 2024-09-10T22:08:53.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b8e00640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd59c004d50 0x7fd5b415b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:53.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b9800640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5b41f2e80 0x7fd5b40736e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac01d020 tx=0x7fd5ac009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:53.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b0c00640 1 -- 172.21.15.66:0/2677170337 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac009ab0 con 0x7fd5b41f2e80 2024-09-10T22:08:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b0c00640 1 -- 172.21.15.66:0/2677170337 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5ac0a94f0 con 0x7fd5b41f2e80 2024-09-10T22:08:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5bb800640 1 -- 172.21.15.66:0/2677170337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b415b390 con 0x7fd5b41f2e80 2024-09-10T22:08:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5b0c00640 1 -- 172.21.15.66:0/2677170337 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac009200 con 0x7fd5b41f2e80 2024-09-10T22:08:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.841+0000 7fd5bb800640 1 -- 172.21.15.66:0/2677170337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b41f2680 con 0x7fd5b41f2e80 2024-09-10T22:08:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.845+0000 7fd5bb800640 1 -- 172.21.15.66:0/2677170337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b40b3210 con 0x7fd5b41f2e80 2024-09-10T22:08:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.845+0000 7fd5b0c00640 1 -- 172.21.15.66:0/2677170337 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 28) ==== 98026+0+0 (secure 0 0 0) 0x7fd5ac003b80 con 0x7fd5b41f2e80 2024-09-10T22:08:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.845+0000 7fd5b0c00640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd594073810 0x7fd594075cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.845+0000 7fd5b0c00640 1 -- 172.21.15.66:0/2677170337 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fd5ac12ccd0 con 0x7fd5b41f2e80 2024-09-10T22:08:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.849+0000 7fd5b8e00640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd594073810 0x7fd594075cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:53.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.849+0000 7fd5b8e00640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd594073810 0x7fd594075cd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8012560 tx=0x7fd5a80091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:53.849+0000 7fd5b0c00640 1 -- 172.21.15.66:0/2677170337 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd5ac0ad030 con 0x7fd5b41f2e80 2024-09-10T22:08:54.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.053+0000 7fd5bb800640 1 -- 172.21.15.66:0/2677170337 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fd5b40ca080 con 0x7fd594073810 2024-09-10T22:08:54.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.057+0000 7fd5b0c00640 1 -- 172.21.15.66:0/2677170337 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fd5b40ca080 con 0x7fd594073810 2024-09-10T22:08:54.064 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/bd8f889e-ea1f-4e34-9b9e-798a339d5687 2024-09-10T22:08:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.061+0000 7fd5bb800640 1 -- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd594073810 msgr2=0x7fd594075cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.061+0000 7fd5bb800640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd594073810 0x7fd594075cd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8012560 tx=0x7fd5a80091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.061+0000 7fd5bb800640 1 -- 172.21.15.66:0/2677170337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5b41f2e80 msgr2=0x7fd5b40736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.061+0000 7fd5bb800640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5b41f2e80 0x7fd5b40736e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac01d020 tx=0x7fd5ac009480 comp rx=0 tx=0).stop 2024-09-10T22:08:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.061+0000 7fd5bb800640 1 -- 172.21.15.66:0/2677170337 shutdown_connections 2024-09-10T22:08:54.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.061+0000 7fd5bb800640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd594073810 0x7fd594075cd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:54.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.061+0000 7fd5bb800640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5b41f2e80 0x7fd5b40736e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:54.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.061+0000 7fd5bb800640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b40b2e30 0x7fd5b4154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:54.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.061+0000 7fd5bb800640 1 --2- 172.21.15.66:0/2677170337 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd59c004d50 0x7fd5b415b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:54.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.061+0000 7fd5bb800640 1 -- 172.21.15.66:0/2677170337 >> 172.21.15.66:0/2677170337 conn(0x7fd5b40cfa30 msgr2=0x7fd5b40725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:54.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.061+0000 7fd5bb800640 1 -- 172.21.15.66:0/2677170337 shutdown_connections 2024-09-10T22:08:54.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:54.061+0000 7fd5bb800640 1 -- 172.21.15.66:0/2677170337 wait complete. 2024-09-10T22:08:54.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:54.086 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/bd8f889e-ea1f-4e34-9b9e-798a339d5687 2024-09-10T22:08:54.086 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:08:54.099 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/bd8f889e-ea1f-4e34-9b9e-798a339d5687 2024-09-10T22:08:54.099 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3138256896" 2024-09-10T22:08:54.100 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:54.100 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-09-10T22:08:56.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:08:56.101 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/bd8f889e-ea1f-4e34-9b9e-798a339d5687 2024-09-10T22:08:56.101 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:08:56.116 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/bd8f889e-ea1f-4e34-9b9e-798a339d5687 2024-09-10T22:08:56.116 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3221225472" 2024-09-10T22:08:56.116 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:08:56.117 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-10T22:08:56.117 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone33350 2024-09-10T22:08:56.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.317+0000 7fb74de00640 1 -- 172.21.15.66:0/1145319592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb748156430 msgr2=0x7fb74806d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:56.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.317+0000 7fb74de00640 1 --2- 172.21.15.66:0/1145319592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb748156430 0x7fb74806d630 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb738009840 tx=0x7fb74815d120 comp rx=0 tx=0).stop 2024-09-10T22:08:56.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.317+0000 7fb74de00640 1 -- 172.21.15.66:0/1145319592 shutdown_connections 2024-09-10T22:08:56.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.317+0000 7fb74de00640 1 --2- 172.21.15.66:0/1145319592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb748156430 0x7fb74806d630 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:56.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.317+0000 7fb74de00640 1 -- 172.21.15.66:0/1145319592 >> 172.21.15.66:0/1145319592 conn(0x7fb7480d0b30 msgr2=0x7fb7480d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:56.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.317+0000 7fb74de00640 1 -- 172.21.15.66:0/1145319592 shutdown_connections 2024-09-10T22:08:56.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.317+0000 7fb74de00640 1 -- 172.21.15.66:0/1145319592 wait complete. 2024-09-10T22:08:56.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb74de00640 1 Processor -- start 2024-09-10T22:08:56.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb74de00640 1 -- start start 2024-09-10T22:08:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb74de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb748071230 0x7fb7480cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb74de00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb748156430 0x7fb74815b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb74de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb748160370 0x7fb74815b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb74de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb74816a330 con 0x7fb748160370 2024-09-10T22:08:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb74de00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb74816a1b0 con 0x7fb748071230 2024-09-10T22:08:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb74de00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb74816a4b0 con 0x7fb748156430 2024-09-10T22:08:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb747400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb748160370 0x7fb74815b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb747400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb748160370 0x7fb74815b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59634/0 (socket says 172.21.15.66:59634) 2024-09-10T22:08:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb747400640 1 -- 172.21.15.66:0/4285011922 learned_addr learned my addr 172.21.15.66:0/4285011922 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb746a00640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb748071230 0x7fb7480cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb73fe00640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb748156430 0x7fb74815b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb747400640 1 -- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb748156430 msgr2=0x7fb74815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:56.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb747400640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb748156430 0x7fb74815b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:56.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb747400640 1 -- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb748071230 msgr2=0x7fb7480cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:56.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb747400640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb748071230 0x7fb7480cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:56.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb747400640 1 -- 172.21.15.66:0/4285011922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7480ce1d0 con 0x7fb748160370 2024-09-10T22:08:56.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb746a00640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb748071230 0x7fb7480cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:56.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb747400640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb748160370 0x7fb74815b6f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb74000c780 tx=0x7fb74000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:56.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb73f400640 1 -- 172.21.15.66:0/4285011922 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740004b70 con 0x7fb748160370 2024-09-10T22:08:56.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb73f400640 1 -- 172.21.15.66:0/4285011922 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb740004e00 con 0x7fb748160370 2024-09-10T22:08:56.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb73f400640 1 -- 172.21.15.66:0/4285011922 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74000aab0 con 0x7fb748160370 2024-09-10T22:08:56.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb74de00640 1 -- 172.21.15.66:0/4285011922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb748151790 con 0x7fb748160370 2024-09-10T22:08:56.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb74de00640 1 -- 172.21.15.66:0/4285011922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7480cc700 con 0x7fb748160370 2024-09-10T22:08:56.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.321+0000 7fb74de00640 1 -- 172.21.15.66:0/4285011922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb74806d630 con 0x7fb748160370 2024-09-10T22:08:56.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.325+0000 7fb73f400640 1 -- 172.21.15.66:0/4285011922 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 28) ==== 98026+0+0 (secure 0 0 0) 0x7fb74000ac50 con 0x7fb748160370 2024-09-10T22:08:56.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.325+0000 7fb73f400640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb7240737e0 0x7fb724075ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:56.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.325+0000 7fb73f400640 1 -- 172.21.15.66:0/4285011922 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fb740092490 con 0x7fb748160370 2024-09-10T22:08:56.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.325+0000 7fb746a00640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb7240737e0 0x7fb724075ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:56.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.325+0000 7fb746a00640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb7240737e0 0x7fb724075ca0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb7380ae000 tx=0x7fb73801afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:56.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.329+0000 7fb73f400640 1 -- 172.21.15.66:0/4285011922 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb740020070 con 0x7fb748160370 2024-09-10T22:08:56.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.533+0000 7fb74de00640 1 -- 172.21.15.66:0/4285011922 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone33350", "target": ["mon-mgr", ""]}) -- 0x7fb7480745f0 con 0x7fb7240737e0 2024-09-10T22:08:56.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.597+0000 7fb73f400640 1 -- 172.21.15.66:0/4285011922 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb7480745f0 con 0x7fb7240737e0 2024-09-10T22:08:56.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.601+0000 7fb74de00640 1 -- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb7240737e0 msgr2=0x7fb724075ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:56.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.601+0000 7fb74de00640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb7240737e0 0x7fb724075ca0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb7380ae000 tx=0x7fb73801afd0 comp rx=0 tx=0).stop 2024-09-10T22:08:56.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.601+0000 7fb74de00640 1 -- 172.21.15.66:0/4285011922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb748160370 msgr2=0x7fb74815b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:56.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.601+0000 7fb74de00640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb748160370 0x7fb74815b6f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb74000c780 tx=0x7fb74000cc40 comp rx=0 tx=0).stop 2024-09-10T22:08:56.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.601+0000 7fb74de00640 1 -- 172.21.15.66:0/4285011922 shutdown_connections 2024-09-10T22:08:56.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.601+0000 7fb74de00640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb7240737e0 0x7fb724075ca0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:56.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.601+0000 7fb74de00640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb748160370 0x7fb74815b6f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:56.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.601+0000 7fb74de00640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb748156430 0x7fb74815b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:56.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.601+0000 7fb74de00640 1 --2- 172.21.15.66:0/4285011922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb748071230 0x7fb7480cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.601+0000 7fb74de00640 1 -- 172.21.15.66:0/4285011922 >> 172.21.15.66:0/4285011922 conn(0x7fb7480d0b30 msgr2=0x7fb748159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.601+0000 7fb74de00640 1 -- 172.21.15.66:0/4285011922 shutdown_connections 2024-09-10T22:08:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.601+0000 7fb74de00640 1 -- 172.21.15.66:0/4285011922 wait complete. 2024-09-10T22:08:56.613 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:56.611+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:56.614 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:56.611+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:56.614 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:56.611+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:56.614 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:56.611+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:56.614 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:56.611+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:56.627 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone75881 2024-09-10T22:08:56.630 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:56.627+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:56.630 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:56.627+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:56.630 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:56.627+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:56.630 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:56.627+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:56.630 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:56.627+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 -- 172.21.15.66:0/331355450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa668004810 msgr2=0x7fa668004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 --2- 172.21.15.66:0/331355450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa668004810 0x7fa668004c90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa6800b1360 tx=0x7fa66c01d1c0 comp rx=0 tx=0).stop 2024-09-10T22:08:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa686000640 1 -- 172.21.15.66:0/331355450 reap_dead start 2024-09-10T22:08:56.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 -- 172.21.15.66:0/331355450 shutdown_connections 2024-09-10T22:08:56.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 -- 172.21.15.66:0/331355450 >> 172.21.15.66:0/331355450 conn(0x7fa6800d1d00 msgr2=0x7fa6800d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:56.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 -- 172.21.15.66:0/331355450 shutdown_connections 2024-09-10T22:08:56.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 -- 172.21.15.66:0/331355450 wait complete. 2024-09-10T22:08:56.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 Processor -- start 2024-09-10T22:08:56.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 -- start start 2024-09-10T22:08:56.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa668004810 0x7fa6801fa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:56.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6800c8150 0x7fa6801fb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:56.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800b3e70 0x7fa6801fb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:56.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa6800b2200 con 0x7fa6800b3e70 2024-09-10T22:08:56.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6800b2080 con 0x7fa6800c8150 2024-09-10T22:08:56.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa688000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa680166c50 con 0x7fa668004810 2024-09-10T22:08:56.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa686000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800b3e70 0x7fa6801fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa686000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800b3e70 0x7fa6801fb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59646/0 (socket says 172.21.15.66:59646) 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa686000640 1 -- 172.21.15.66:0/2152373475 learned_addr learned my addr 172.21.15.66:0/2152373475 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa685600640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa668004810 0x7fa6801fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa684c00640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6800c8150 0x7fa6801fb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa686000640 1 -- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa668004810 msgr2=0x7fa6801fa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa686000640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa668004810 0x7fa6801fa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa686000640 1 -- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6800c8150 msgr2=0x7fa6801fb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa686000640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6800c8150 0x7fa6801fb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa686000640 1 -- 172.21.15.66:0/2152373475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa680071af0 con 0x7fa6800b3e70 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.777+0000 7fa686000640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800b3e70 0x7fa6801fb680 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa6780014f0 tx=0x7fa67800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.781+0000 7fa67d600640 1 -- 172.21.15.66:0/2152373475 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678015020 con 0x7fa6800b3e70 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.781+0000 7fa67d600640 1 -- 172.21.15.66:0/2152373475 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa67800ca40 con 0x7fa6800b3e70 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.781+0000 7fa67d600640 1 -- 172.21.15.66:0/2152373475 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67800ce90 con 0x7fa6800b3e70 2024-09-10T22:08:56.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.781+0000 7fa688000640 1 -- 172.21.15.66:0/2152373475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6801fbdf0 con 0x7fa6800b3e70 2024-09-10T22:08:56.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.781+0000 7fa688000640 1 -- 172.21.15.66:0/2152373475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6801fae80 con 0x7fa6800b3e70 2024-09-10T22:08:56.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.781+0000 7fa688000640 1 -- 172.21.15.66:0/2152373475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6800cfc00 con 0x7fa6800b3e70 2024-09-10T22:08:56.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.781+0000 7fa67d600640 1 -- 172.21.15.66:0/2152373475 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 28) ==== 98026+0+0 (secure 0 0 0) 0x7fa678011460 con 0x7fa6800b3e70 2024-09-10T22:08:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.781+0000 7fa67d600640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa660073980 0x7fa660075e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.781+0000 7fa67d600640 1 -- 172.21.15.66:0/2152373475 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fa6780926c0 con 0x7fa6800b3e70 2024-09-10T22:08:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.781+0000 7fa685600640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa660073980 0x7fa660075e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:56.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.785+0000 7fa685600640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa660073980 0x7fa660075e40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa674009ce0 tx=0x7fa6740091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:56.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.785+0000 7fa67d600640 1 -- 172.21.15.66:0/2152373475 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa67801f030 con 0x7fa6800b3e70 2024-09-10T22:08:56.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:08:56.852 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:08:57.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:56.993+0000 7fa688000640 1 -- 172.21.15.66:0/2152373475 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone75881", "target": ["mon-mgr", ""]}) -- 0x7fa680074aa0 con 0x7fa660073980 2024-09-10T22:08:57.610 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:57.607+0000 7f5faa000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:57.610 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:57.607+0000 7f5faa000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:57.610 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:57.607+0000 7f5faa000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:57.610 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:57.607+0000 7f5faa000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:57.610 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:57.607+0000 7f5faa000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:57.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.689+0000 7fa67d600640 1 -- 172.21.15.66:0/2152373475 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa680074aa0 con 0x7fa660073980 2024-09-10T22:08:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.689+0000 7fa688000640 1 -- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa660073980 msgr2=0x7fa660075e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.689+0000 7fa688000640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa660073980 0x7fa660075e40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa674009ce0 tx=0x7fa6740091e0 comp rx=0 tx=0).stop 2024-09-10T22:08:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.689+0000 7fa688000640 1 -- 172.21.15.66:0/2152373475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800b3e70 msgr2=0x7fa6801fb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.689+0000 7fa688000640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800b3e70 0x7fa6801fb680 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa6780014f0 tx=0x7fa67800ec50 comp rx=0 tx=0).stop 2024-09-10T22:08:57.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.693+0000 7fa688000640 1 -- 172.21.15.66:0/2152373475 shutdown_connections 2024-09-10T22:08:57.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.693+0000 7fa688000640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa660073980 0x7fa660075e40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:57.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.693+0000 7fa688000640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6800b3e70 0x7fa6801fb680 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:57.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.693+0000 7fa688000640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6800c8150 0x7fa6801fb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:57.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.693+0000 7fa688000640 1 --2- 172.21.15.66:0/2152373475 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa668004810 0x7fa6801fa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:57.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.693+0000 7fa688000640 1 -- 172.21.15.66:0/2152373475 >> 172.21.15.66:0/2152373475 conn(0x7fa6800d1d00 msgr2=0x7fa6800b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:57.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.693+0000 7fa688000640 1 -- 172.21.15.66:0/2152373475 shutdown_connections 2024-09-10T22:08:57.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.693+0000 7fa688000640 1 -- 172.21.15.66:0/2152373475 wait complete. 2024-09-10T22:08:57.730 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone94742 2024-09-10T22:08:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 -- 172.21.15.66:0/59435422 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e0c15bd40 msgr2=0x7f1e0c0cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 --2- 172.21.15.66:0/59435422 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e0c15bd40 0x7f1e0c0cffa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009840 tx=0x7f1e0001c790 comp rx=0 tx=0).stop 2024-09-10T22:08:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 -- 172.21.15.66:0/59435422 shutdown_connections 2024-09-10T22:08:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 --2- 172.21.15.66:0/59435422 >> v2:172.21.15.94:3300/0 conn(0x7f1e0c0cf180 0x7f1e0c0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 --2- 172.21.15.66:0/59435422 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e0c15bd40 0x7f1e0c0cffa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 -- 172.21.15.66:0/59435422 >> 172.21.15.66:0/59435422 conn(0x7f1e0c0d1c60 msgr2=0x7f1e0c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 -- 172.21.15.66:0/59435422 shutdown_connections 2024-09-10T22:08:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 -- 172.21.15.66:0/59435422 wait complete. 2024-09-10T22:08:57.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 Processor -- start 2024-09-10T22:08:57.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 -- start start 2024-09-10T22:08:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c0cf180 0x7f1e0c14ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e0c0cf830 0x7f1e0c14fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e0c15bd40 0x7f1e0c15add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c168ba0 con 0x7f1e0c0cf180 2024-09-10T22:08:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c168a20 con 0x7f1e0c15bd40 2024-09-10T22:08:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.965+0000 7f1e14200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0c168d20 con 0x7f1e0c0cf830 2024-09-10T22:08:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e11800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c0cf180 0x7f1e0c14ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e11800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c0cf180 0x7f1e0c14ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59662/0 (socket says 172.21.15.66:59662) 2024-09-10T22:08:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e11800640 1 -- 172.21.15.66:0/3372604622 learned_addr learned my addr 172.21.15.66:0/3372604622 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e10e00640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e0c0cf830 0x7f1e0c14fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e12200640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e0c15bd40 0x7f1e0c15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e11800640 1 -- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e0c0cf830 msgr2=0x7f1e0c14fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e11800640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e0c0cf830 0x7f1e0c14fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e11800640 1 -- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e0c15bd40 msgr2=0x7f1e0c15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e11800640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e0c15bd40 0x7f1e0c15add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e11800640 1 -- 172.21.15.66:0/3372604622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c150800 con 0x7f1e0c0cf180 2024-09-10T22:08:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e11800640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c0cf180 0x7f1e0c14ebe0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009790 tx=0x7f1e000a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e09600640 1 -- 172.21.15.66:0/3372604622 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e000bb070 con 0x7f1e0c0cf180 2024-09-10T22:08:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e09600640 1 -- 172.21.15.66:0/3372604622 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e00005030 con 0x7f1e0c0cf180 2024-09-10T22:08:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e14200640 1 -- 172.21.15.66:0/3372604622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0c1fbc80 con 0x7f1e0c0cf180 2024-09-10T22:08:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e14200640 1 -- 172.21.15.66:0/3372604622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e0c1fb1a0 con 0x7f1e0c0cf180 2024-09-10T22:08:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e09600640 1 -- 172.21.15.66:0/3372604622 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e000b2040 con 0x7f1e0c0cf180 2024-09-10T22:08:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.969+0000 7f1e14200640 1 -- 172.21.15.66:0/3372604622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de80052a0 con 0x7f1e0c0cf180 2024-09-10T22:08:57.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.977+0000 7f1e09600640 1 -- 172.21.15.66:0/3372604622 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 28) ==== 98026+0+0 (secure 0 0 0) 0x7f1e000b5030 con 0x7f1e0c0cf180 2024-09-10T22:08:57.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.977+0000 7f1e09600640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1dec0737e0 0x7f1dec075ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:57.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.977+0000 7f1e09600640 1 -- 172.21.15.66:0/3372604622 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f1e0012b6b0 con 0x7f1e0c0cf180 2024-09-10T22:08:57.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.977+0000 7f1e09600640 1 -- 172.21.15.66:0/3372604622 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1e000c4020 con 0x7f1e0c0cf180 2024-09-10T22:08:57.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.989+0000 7f1e10e00640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1dec0737e0 0x7f1dec075ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:57.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:57.989+0000 7f1e10e00640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1dec0737e0 0x7f1dec075ca0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c1fad80 tx=0x7f1df8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:58.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.261+0000 7f1e14200640 1 -- 172.21.15.66:0/3372604622 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone94742", "target": ["mon-mgr", ""]}) -- 0x7f1de8000c90 con 0x7f1dec0737e0 2024-09-10T22:08:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.897+0000 7f1e09600640 1 -- 172.21.15.66:0/3372604622 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1de8000c90 con 0x7f1dec0737e0 2024-09-10T22:08:58.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.901+0000 7f1e14200640 1 -- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1dec0737e0 msgr2=0x7f1dec075ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:58.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.901+0000 7f1e14200640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1dec0737e0 0x7f1dec075ca0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c1fad80 tx=0x7f1df8009040 comp rx=0 tx=0).stop 2024-09-10T22:08:58.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.901+0000 7f1e14200640 1 -- 172.21.15.66:0/3372604622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c0cf180 msgr2=0x7f1e0c14ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:58.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.901+0000 7f1e14200640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c0cf180 0x7f1e0c14ebe0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009790 tx=0x7f1e000a57a0 comp rx=0 tx=0).stop 2024-09-10T22:08:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.901+0000 7f1e14200640 1 -- 172.21.15.66:0/3372604622 shutdown_connections 2024-09-10T22:08:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.901+0000 7f1e14200640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1dec0737e0 0x7f1dec075ca0 secure :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c1fad80 tx=0x7f1df8009040 comp rx=0 tx=0).stop 2024-09-10T22:08:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.901+0000 7f1e14200640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e0c15bd40 0x7f1e0c15add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.901+0000 7f1e14200640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e0c0cf830 0x7f1e0c14fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.901+0000 7f1e14200640 1 --2- 172.21.15.66:0/3372604622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c0cf180 0x7f1e0c14ebe0 secure :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009790 tx=0x7f1e000a57a0 comp rx=0 tx=0).stop 2024-09-10T22:08:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.901+0000 7f1e14200640 1 -- 172.21.15.66:0/3372604622 >> 172.21.15.66:0/3372604622 conn(0x7f1e0c0d1c60 msgr2=0x7f1e0c0b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.905+0000 7f1e14200640 1 -- 172.21.15.66:0/3372604622 shutdown_connections 2024-09-10T22:08:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:58.905+0000 7f1e14200640 1 -- 172.21.15.66:0/3372604622 wait complete. 2024-09-10T22:08:58.943 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone723 2024-09-10T22:08:58.945 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:58.943+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:58.945 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:58.943+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:58.945 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:58.943+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:58.945 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:58.943+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:58.945 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:08:58.943+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:08:59.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.153+0000 7f3035200640 1 -- 172.21.15.66:0/4269991109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30300cf830 msgr2=0x7f30300d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:59.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.153+0000 7f3035200640 1 --2- 172.21.15.66:0/4269991109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30300cf830 0x7f30300d04e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3030073840 tx=0x7f301801c830 comp rx=0 tx=0).stop 2024-09-10T22:08:59.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.153+0000 7f3035200640 1 -- 172.21.15.66:0/4269991109 shutdown_connections 2024-09-10T22:08:59.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.153+0000 7f3035200640 1 --2- 172.21.15.66:0/4269991109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30300cf830 0x7f30300d04e0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:59.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.153+0000 7f3035200640 1 --2- 172.21.15.66:0/4269991109 >> v2:172.21.15.94:3300/0 conn(0x7f303015bd40 0x7f30300cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:59.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.153+0000 7f3035200640 1 -- 172.21.15.66:0/4269991109 >> 172.21.15.66:0/4269991109 conn(0x7f30300d1c60 msgr2=0x7f30300d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:59.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.153+0000 7f3035200640 1 -- 172.21.15.66:0/4269991109 shutdown_connections 2024-09-10T22:08:59.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.153+0000 7f3035200640 1 -- 172.21.15.66:0/4269991109 wait complete. 2024-09-10T22:08:59.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3035200640 1 Processor -- start 2024-09-10T22:08:59.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3035200640 1 -- start start 2024-09-10T22:08:59.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3035200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f30300aec10 0x7f30300c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:59.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3035200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30300cf830 0x7f30301f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:59.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3035200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f303015bd40 0x7f30301f9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:59.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3035200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3030169020 con 0x7f30300cf830 2024-09-10T22:08:59.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3035200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3030168ea0 con 0x7f303015bd40 2024-09-10T22:08:59.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3035200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f30301691a0 con 0x7f30300aec10 2024-09-10T22:08:59.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3027e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30300cf830 0x7f30301f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:59.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3027e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30300cf830 0x7f30301f90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59678/0 (socket says 172.21.15.66:59678) 2024-09-10T22:08:59.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3027e00640 1 -- 172.21.15.66:0/3524084375 learned_addr learned my addr 172.21.15.66:0/3524084375 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:08:59.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3027e00640 1 -- 172.21.15.66:0/3524084375 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f30300aec10 msgr2=0x7f30300c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:59.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3027e00640 1 --2- 172.21.15.66:0/3524084375 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f30300aec10 0x7f30300c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:59.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3027e00640 1 -- 172.21.15.66:0/3524084375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f303015bd40 msgr2=0x7f30301f9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:59.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3027e00640 1 --2- 172.21.15.66:0/3524084375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f303015bd40 0x7f30301f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:59.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.157+0000 7f3027e00640 1 -- 172.21.15.66:0/3524084375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30300c8170 con 0x7f30300cf830 2024-09-10T22:08:59.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.161+0000 7f3027e00640 1 --2- 172.21.15.66:0/3524084375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30300cf830 0x7f30301f90f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f30180b0040 tx=0x7f3018004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:59.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.161+0000 7f3026000640 1 -- 172.21.15.66:0/3524084375 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30180b2040 con 0x7f30300cf830 2024-09-10T22:08:59.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.161+0000 7f3026000640 1 -- 172.21.15.66:0/3524084375 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3018005030 con 0x7f30300cf830 2024-09-10T22:08:59.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.161+0000 7f3026000640 1 -- 172.21.15.66:0/3524084375 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018005480 con 0x7f30300cf830 2024-09-10T22:08:59.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.161+0000 7f3035200640 1 -- 172.21.15.66:0/3524084375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303014edd0 con 0x7f30300cf830 2024-09-10T22:08:59.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.161+0000 7f3035200640 1 -- 172.21.15.66:0/3524084375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303014f390 con 0x7f30300cf830 2024-09-10T22:08:59.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.161+0000 7f3035200640 1 -- 172.21.15.66:0/3524084375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30300af0a0 con 0x7f30300cf830 2024-09-10T22:08:59.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.165+0000 7f3026000640 1 -- 172.21.15.66:0/3524084375 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 29) ==== 98179+0+0 (secure 0 0 0) 0x7f30180046d0 con 0x7f30300cf830 2024-09-10T22:08:59.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.165+0000 7f3026000640 1 --2- 172.21.15.66:0/3524084375 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f300c073990 0x7f300c075e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:59.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.165+0000 7f3026000640 1 -- 172.21.15.66:0/3524084375 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f301812bc80 con 0x7f30300cf830 2024-09-10T22:08:59.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.169+0000 7f3026000640 1 -- 172.21.15.66:0/3524084375 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f30180164b0 con 0x7f30300cf830 2024-09-10T22:08:59.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.177+0000 7f302de00640 1 --2- 172.21.15.66:0/3524084375 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f300c073990 0x7f300c075e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:59.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.197+0000 7f302de00640 1 --2- 172.21.15.66:0/3524084375 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f300c073990 0x7f300c075e50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3020009690 tx=0x7f3020006c00 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:59.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:08:59.545+0000 7f3035200640 1 -- 172.21.15.66:0/3524084375 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone723", "target": ["mon-mgr", ""]}) -- 0x7f30300b0df0 con 0x7f300c073990 2024-09-10T22:09:00.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.405+0000 7f3026000640 1 -- 172.21.15.66:0/3524084375 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f30300b0df0 con 0x7f300c073990 2024-09-10T22:09:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.409+0000 7f3035200640 1 -- 172.21.15.66:0/3524084375 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f300c073990 msgr2=0x7f300c075e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.409+0000 7f3035200640 1 --2- 172.21.15.66:0/3524084375 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f300c073990 0x7f300c075e50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3020009690 tx=0x7f3020006c00 comp rx=0 tx=0).stop 2024-09-10T22:09:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.409+0000 7f3035200640 1 -- 172.21.15.66:0/3524084375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30300cf830 msgr2=0x7f30301f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.409+0000 7f3035200640 1 --2- 172.21.15.66:0/3524084375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30300cf830 0x7f30301f90f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f30180b0040 tx=0x7f3018004040 comp rx=0 tx=0).stop 2024-09-10T22:09:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.409+0000 7f3035200640 1 -- 172.21.15.66:0/3524084375 shutdown_connections 2024-09-10T22:09:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.409+0000 7f3035200640 1 --2- 172.21.15.66:0/3524084375 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f300c073990 0x7f300c075e50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.409+0000 7f3035200640 1 --2- 172.21.15.66:0/3524084375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f303015bd40 0x7f30301f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.409+0000 7f3035200640 1 --2- 172.21.15.66:0/3524084375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30300cf830 0x7f30301f90f0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.409+0000 7f3035200640 1 --2- 172.21.15.66:0/3524084375 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f30300aec10 0x7f30300c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.409+0000 7f3035200640 1 -- 172.21.15.66:0/3524084375 >> 172.21.15.66:0/3524084375 conn(0x7f30300d1c60 msgr2=0x7f30300c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.409+0000 7f3035200640 1 -- 172.21.15.66:0/3524084375 shutdown_connections 2024-09-10T22:09:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.409+0000 7f3035200640 1 -- 172.21.15.66:0/3524084375 wait complete. 2024-09-10T22:09:00.423 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:00.423+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:09:00.423 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:00.423+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:09:00.423 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:00.423+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:09:00.423 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:00.423+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:09:00.424 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:00.423+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:09:00.454 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone37214 2024-09-10T22:09:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.693+0000 7ffa1b400640 1 -- 172.21.15.66:0/4053426584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa140c4ef0 msgr2=0x7ffa140c5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.693+0000 7ffa1b400640 1 --2- 172.21.15.66:0/4053426584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa140c4ef0 0x7ffa140c5320 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ffa140cd550 tx=0x7ffa0c01cd80 comp rx=0 tx=0).stop 2024-09-10T22:09:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.693+0000 7ffa1b400640 1 -- 172.21.15.66:0/4053426584 shutdown_connections 2024-09-10T22:09:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.693+0000 7ffa1b400640 1 --2- 172.21.15.66:0/4053426584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa140c4ef0 0x7ffa140c5320 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.693+0000 7ffa1b400640 1 --2- 172.21.15.66:0/4053426584 >> v2:172.21.15.94:3300/0 conn(0x7ffa140c4b10 0x7ffa1406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.693+0000 7ffa1b400640 1 -- 172.21.15.66:0/4053426584 >> 172.21.15.66:0/4053426584 conn(0x7ffa140cfa30 msgr2=0x7ffa140cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.693+0000 7ffa1b400640 1 -- 172.21.15.66:0/4053426584 shutdown_connections 2024-09-10T22:09:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.693+0000 7ffa1b400640 1 -- 172.21.15.66:0/4053426584 wait complete. 2024-09-10T22:09:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa1b400640 1 Processor -- start 2024-09-10T22:09:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa1b400640 1 -- start start 2024-09-10T22:09:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa1b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa140ae620 0x7ffa141fac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa1b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa140c4b10 0x7ffa14151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa1b400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa140c4ef0 0x7ffa140b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa1b400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14167fb0 con 0x7ffa140c4b10 2024-09-10T22:09:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa1b400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14167e30 con 0x7ffa140ae620 2024-09-10T22:09:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa1b400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ffa14168130 con 0x7ffa140c4ef0 2024-09-10T22:09:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa13e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa140c4b10 0x7ffa14151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa13e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa140c4b10 0x7ffa14151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59692/0 (socket says 172.21.15.66:59692) 2024-09-10T22:09:00.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa13e00640 1 -- 172.21.15.66:0/1520409113 learned_addr learned my addr 172.21.15.66:0/1520409113 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:00.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa13e00640 1 -- 172.21.15.66:0/1520409113 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa140c4ef0 msgr2=0x7ffa140b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:00.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.697+0000 7ffa13e00640 1 --2- 172.21.15.66:0/1520409113 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa140c4ef0 0x7ffa140b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:00.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.701+0000 7ffa13e00640 1 -- 172.21.15.66:0/1520409113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa140ae620 msgr2=0x7ffa141fac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:00.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.701+0000 7ffa13e00640 1 --2- 172.21.15.66:0/1520409113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa140ae620 0x7ffa141fac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:00.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.701+0000 7ffa13e00640 1 -- 172.21.15.66:0/1520409113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa140b2d10 con 0x7ffa140c4b10 2024-09-10T22:09:00.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.701+0000 7ffa13e00640 1 --2- 172.21.15.66:0/1520409113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa140c4b10 0x7ffa14151330 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400c490 tx=0x7ffa04001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:00.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.701+0000 7ffa10c00640 1 -- 172.21.15.66:0/1520409113 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04017070 con 0x7ffa140c4b10 2024-09-10T22:09:00.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.701+0000 7ffa10c00640 1 -- 172.21.15.66:0/1520409113 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa04003600 con 0x7ffa140c4b10 2024-09-10T22:09:00.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.701+0000 7ffa10c00640 1 -- 172.21.15.66:0/1520409113 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04003a90 con 0x7ffa140c4b10 2024-09-10T22:09:00.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.701+0000 7ffa1b400640 1 -- 172.21.15.66:0/1520409113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa140b2ff0 con 0x7ffa140c4b10 2024-09-10T22:09:00.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.701+0000 7ffa1b400640 1 -- 172.21.15.66:0/1520409113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa140b35b0 con 0x7ffa140c4b10 2024-09-10T22:09:00.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.701+0000 7ffa1b400640 1 -- 172.21.15.66:0/1520409113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa140b39e0 con 0x7ffa140c4b10 2024-09-10T22:09:00.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.701+0000 7ffa10c00640 1 -- 172.21.15.66:0/1520409113 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 29) ==== 98179+0+0 (secure 0 0 0) 0x7ffa04003c30 con 0x7ffa140c4b10 2024-09-10T22:09:00.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.705+0000 7ffa10c00640 1 --2- 172.21.15.66:0/1520409113 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff9f40739c0 0x7ff9f4075e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:00.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.705+0000 7ffa10c00640 1 -- 172.21.15.66:0/1520409113 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7ffa040914c0 con 0x7ffa140c4b10 2024-09-10T22:09:00.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.705+0000 7ffa18a00640 1 --2- 172.21.15.66:0/1520409113 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff9f40739c0 0x7ff9f4075e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:00.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.709+0000 7ffa18a00640 1 --2- 172.21.15.66:0/1520409113 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff9f40739c0 0x7ff9f4075e80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009730 tx=0x7ffa00006ca0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:00.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:00.709+0000 7ffa10c00640 1 -- 172.21.15.66:0/1520409113 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffa0405e240 con 0x7ffa140c4b10 2024-09-10T22:09:01.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:01.093+0000 7ffa1b400640 1 -- 172.21.15.66:0/1520409113 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone37214", "target": ["mon-mgr", ""]}) -- 0x7ffa14159030 con 0x7ff9f40739c0 2024-09-10T22:09:01.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:01.129+0000 7ffa10c00640 1 -- 172.21.15.66:0/1520409113 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7ffa04059db0 con 0x7ffa140c4b10 2024-09-10T22:09:02.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.301+0000 7ffa10c00640 1 -- 172.21.15.66:0/1520409113 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffa14159030 con 0x7ff9f40739c0 2024-09-10T22:09:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.305+0000 7ffa1b400640 1 -- 172.21.15.66:0/1520409113 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff9f40739c0 msgr2=0x7ff9f4075e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.305+0000 7ffa1b400640 1 --2- 172.21.15.66:0/1520409113 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff9f40739c0 0x7ff9f4075e80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009730 tx=0x7ffa00006ca0 comp rx=0 tx=0).stop 2024-09-10T22:09:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.305+0000 7ffa1b400640 1 -- 172.21.15.66:0/1520409113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa140c4b10 msgr2=0x7ffa14151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.305+0000 7ffa1b400640 1 --2- 172.21.15.66:0/1520409113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa140c4b10 0x7ffa14151330 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400c490 tx=0x7ffa04001e80 comp rx=0 tx=0).stop 2024-09-10T22:09:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.305+0000 7ffa1b400640 1 -- 172.21.15.66:0/1520409113 shutdown_connections 2024-09-10T22:09:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.305+0000 7ffa1b400640 1 --2- 172.21.15.66:0/1520409113 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff9f40739c0 0x7ff9f4075e80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.305+0000 7ffa1b400640 1 --2- 172.21.15.66:0/1520409113 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa140c4ef0 0x7ffa140b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.305+0000 7ffa1b400640 1 --2- 172.21.15.66:0/1520409113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa140c4b10 0x7ffa14151330 secure :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400c490 tx=0x7ffa04001e80 comp rx=0 tx=0).stop 2024-09-10T22:09:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.305+0000 7ffa1b400640 1 --2- 172.21.15.66:0/1520409113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa140ae620 0x7ffa141fac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.305+0000 7ffa1b400640 1 -- 172.21.15.66:0/1520409113 >> 172.21.15.66:0/1520409113 conn(0x7ffa140cfa30 msgr2=0x7ffa140cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.305+0000 7ffa1b400640 1 -- 172.21.15.66:0/1520409113 shutdown_connections 2024-09-10T22:09:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.305+0000 7ffa1b400640 1 -- 172.21.15.66:0/1520409113 wait complete. 2024-09-10T22:09:02.370 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:09:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.605+0000 7ff26d000640 1 -- 172.21.15.66:0/1168929834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2680afc20 msgr2=0x7ff2680ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.605+0000 7ff26d000640 1 --2- 172.21.15.66:0/1168929834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2680afc20 0x7ff2680ce880 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ff260009bc0 tx=0x7ff26001cb10 comp rx=0 tx=0).stop 2024-09-10T22:09:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.605+0000 7ff26d000640 1 -- 172.21.15.66:0/1168929834 shutdown_connections 2024-09-10T22:09:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.605+0000 7ff26d000640 1 --2- 172.21.15.66:0/1168929834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2680afc20 0x7ff2680ce880 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.605+0000 7ff26d000640 1 -- 172.21.15.66:0/1168929834 >> 172.21.15.66:0/1168929834 conn(0x7ff2680d1c60 msgr2=0x7ff2680d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.605+0000 7ff26d000640 1 -- 172.21.15.66:0/1168929834 shutdown_connections 2024-09-10T22:09:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.605+0000 7ff26d000640 1 -- 172.21.15.66:0/1168929834 wait complete. 2024-09-10T22:09:02.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff26d000640 1 Processor -- start 2024-09-10T22:09:02.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff26d000640 1 -- start start 2024-09-10T22:09:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff26d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2680afc20 0x7ff26806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff26d000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2680d06b0 0x7ff2680b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff26d000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff26815bd40 0x7ff2680b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff26d000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff26816a1b0 con 0x7ff2680afc20 2024-09-10T22:09:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff26d000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff26816a030 con 0x7ff2680d06b0 2024-09-10T22:09:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff26d000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff26816a330 con 0x7ff26815bd40 2024-09-10T22:09:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff265e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2680afc20 0x7ff26806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff265e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2680afc20 0x7ff26806f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40480/0 (socket says 172.21.15.66:40480) 2024-09-10T22:09:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff265e00640 1 -- 172.21.15.66:0/616120543 learned_addr learned my addr 172.21.15.66:0/616120543 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff25fe00640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2680d06b0 0x7ff2680b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff265e00640 1 -- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff26815bd40 msgr2=0x7ff2680b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff265e00640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff26815bd40 0x7ff2680b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff265e00640 1 -- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2680d06b0 msgr2=0x7ff2680b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff265e00640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2680d06b0 0x7ff2680b70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff265e00640 1 -- 172.21.15.66:0/616120543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26815b910 con 0x7ff2680afc20 2024-09-10T22:09:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff265e00640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2680afc20 0x7ff26806f6b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff25800b410 tx=0x7ff25800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.609+0000 7ff25e000640 1 -- 172.21.15.66:0/616120543 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258015030 con 0x7ff2680afc20 2024-09-10T22:09:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.613+0000 7ff25e000640 1 -- 172.21.15.66:0/616120543 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff258004020 con 0x7ff2680afc20 2024-09-10T22:09:02.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.613+0000 7ff26d000640 1 -- 172.21.15.66:0/616120543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2681fb540 con 0x7ff2680afc20 2024-09-10T22:09:02.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.613+0000 7ff25e000640 1 -- 172.21.15.66:0/616120543 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2580044b0 con 0x7ff2680afc20 2024-09-10T22:09:02.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.613+0000 7ff26d000640 1 -- 172.21.15.66:0/616120543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2681fbac0 con 0x7ff2680afc20 2024-09-10T22:09:02.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.613+0000 7ff26d000640 1 -- 172.21.15.66:0/616120543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2680cffa0 con 0x7ff2680afc20 2024-09-10T22:09:02.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.621+0000 7ff25e000640 1 -- 172.21.15.66:0/616120543 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7ff258022060 con 0x7ff2680afc20 2024-09-10T22:09:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.621+0000 7ff25e000640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff244073ab0 0x7ff244075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.621+0000 7ff25e000640 1 -- 172.21.15.66:0/616120543 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7ff258091ae0 con 0x7ff2680afc20 2024-09-10T22:09:02.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.625+0000 7ff25e000640 1 -- 172.21.15.66:0/616120543 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff25805ec10 con 0x7ff2680afc20 2024-09-10T22:09:02.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.629+0000 7ff25fe00640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff244073ab0 0x7ff244075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:02.653+0000 7ff25fe00640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff244073ab0 0x7ff244075f70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff268167370 tx=0x7ff250009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.005+0000 7ff26d000640 1 -- 172.21.15.66:0/616120543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff26806fbf0 con 0x7ff2680afc20 2024-09-10T22:09:03.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.005+0000 7ff25e000640 1 -- 172.21.15.66:0/616120543 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2172 (secure 0 0 0) 0x7ff25805e510 con 0x7ff2680afc20 2024-09-10T22:09:03.015 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:09:03.015 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:09:03.016 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:09:03.016 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:09:03.016 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:09:03.016 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:09:03.016 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:09:03.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:03.017 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:09:03.017 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:09:03.017 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:09:03.017 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:09:03.017 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:09:03.018 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:03.018 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:09:03.018 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:09:03.018 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:09:03.018 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:09:03.018 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:03.019 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 536, 2024-09-10T22:09:03.019 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:09:03.019 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:09:03.019 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:09:03.019 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:09:03.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:03.020 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:09:03.020 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 85, 2024-09-10T22:09:03.020 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:09:03.020 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:09:03.020 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:09:03.020 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:09:03.021 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:09:03.021 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:09:03.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:03.021 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:09:03.021 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:09:03.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:09:03.022 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:09:03.022 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:09:03.022 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:03.022 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:03.022 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:09:03.022 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:09:03.023 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 796, 2024-09-10T22:09:03.023 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 3222062215, 2024-09-10T22:09:03.023 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 6954971136, 2024-09-10T22:09:03.023 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 766139142144, 2024-09-10T22:09:03.023 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:09:03.023 INFO:teuthology.orchestra.run.smithi066.stdout: "read_bytes_sec": 85, 2024-09-10T22:09:03.023 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 81364647, 2024-09-10T22:09:03.024 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 0, 2024-09-10T22:09:03.024 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 22 2024-09-10T22:09:03.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:03.024 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:09:03.024 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 57, 2024-09-10T22:09:03.025 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:08:29:488488+0000", 2024-09-10T22:09:03.025 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 6, 2024-09-10T22:09:03.025 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:09:03.025 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:09:03.025 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:09:03.026 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:09:03.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:09:03.026 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 6, 2024-09-10T22:09:03.026 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:09:03.026 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "d", 2024-09-10T22:09:03.026 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:09:03.027 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 6990 2024-09-10T22:09:03.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:03.027 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:03.027 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:09:03.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:03.027 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:09:03.027 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:09:03.027 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:09:03.028 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:09:03.028 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:09:03.028 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:09:03.028 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:09:03.028 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:03.028 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:09:03.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:03.029 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:09:03.029 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 14, 2024-09-10T22:09:03.029 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:08:02.433118+0000", 2024-09-10T22:09:03.029 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:09:03.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:03.029 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": { 2024-09-10T22:09:03.029 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr-vol-ongoing-clones": { 2024-09-10T22:09:03.029 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "3 ongoing clones - average progress is 11.372% (1s)\n [===.........................] (remaining: 14s)", 2024-09-10T22:09:03.029 INFO:teuthology.orchestra.run.smithi066.stdout: "progress": 0.11371999979019165, 2024-09-10T22:09:03.029 INFO:teuthology.orchestra.run.smithi066.stdout: "add_to_ceph_s": true 2024-09-10T22:09:03.030 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:03.030 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:03.030 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:09:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.013+0000 7ff26d000640 1 -- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff244073ab0 msgr2=0x7ff244075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.013+0000 7ff26d000640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff244073ab0 0x7ff244075f70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff268167370 tx=0x7ff250009040 comp rx=0 tx=0).stop 2024-09-10T22:09:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.013+0000 7ff26d000640 1 -- 172.21.15.66:0/616120543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2680afc20 msgr2=0x7ff26806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.013+0000 7ff26d000640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2680afc20 0x7ff26806f6b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff25800b410 tx=0x7ff25800b8d0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.013+0000 7ff26d000640 1 -- 172.21.15.66:0/616120543 shutdown_connections 2024-09-10T22:09:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.013+0000 7ff26d000640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff244073ab0 0x7ff244075f70 secure :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff268167370 tx=0x7ff250009040 comp rx=0 tx=0).stop 2024-09-10T22:09:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.013+0000 7ff26d000640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff26815bd40 0x7ff2680b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.013+0000 7ff26d000640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2680d06b0 0x7ff2680b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.013+0000 7ff26d000640 1 --2- 172.21.15.66:0/616120543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2680afc20 0x7ff26806f6b0 secure :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff25800b410 tx=0x7ff25800b8d0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.013+0000 7ff26d000640 1 -- 172.21.15.66:0/616120543 >> 172.21.15.66:0/616120543 conn(0x7ff2680d1c60 msgr2=0x7ff26806e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.013+0000 7ff26d000640 1 -- 172.21.15.66:0/616120543 shutdown_connections 2024-09-10T22:09:03.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:03.013+0000 7ff26d000640 1 -- 172.21.15.66:0/616120543 wait complete. 2024-09-10T22:09:04.052 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:09:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.273+0000 7fa748400640 1 -- 172.21.15.66:0/3193464980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728004810 msgr2=0x7fa728004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.273+0000 7fa748400640 1 --2- 172.21.15.66:0/3193464980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728004810 0x7fa728004c90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00c310 tx=0x7fa72c01f710 comp rx=0 tx=0).stop 2024-09-10T22:09:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.273+0000 7fa748400640 1 -- 172.21.15.66:0/3193464980 shutdown_connections 2024-09-10T22:09:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.273+0000 7fa748400640 1 --2- 172.21.15.66:0/3193464980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728004810 0x7fa728004c90 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.273+0000 7fa748400640 1 -- 172.21.15.66:0/3193464980 >> 172.21.15.66:0/3193464980 conn(0x7fa7400d1c50 msgr2=0x7fa7400d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.273+0000 7fa748400640 1 -- 172.21.15.66:0/3193464980 shutdown_connections 2024-09-10T22:09:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.273+0000 7fa748400640 1 -- 172.21.15.66:0/3193464980 wait complete. 2024-09-10T22:09:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa748400640 1 Processor -- start 2024-09-10T22:09:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa748400640 1 -- start start 2024-09-10T22:09:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa748400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa728004810 0x7fa7401f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa748400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7401f3ed0 0x7fa7400b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa748400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7401f4350 0x7fa740158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa748400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7400d28f0 con 0x7fa7401f4350 2024-09-10T22:09:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa748400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa7400d2770 con 0x7fa728004810 2024-09-10T22:09:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa748400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa7400d2a70 con 0x7fa7401f3ed0 2024-09-10T22:09:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa746400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7401f4350 0x7fa740158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa746400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7401f4350 0x7fa740158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40496/0 (socket says 172.21.15.66:40496) 2024-09-10T22:09:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa746400640 1 -- 172.21.15.66:0/3956133225 learned_addr learned my addr 172.21.15.66:0/3956133225 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa746400640 1 -- 172.21.15.66:0/3956133225 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7401f3ed0 msgr2=0x7fa7400b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa746400640 1 --2- 172.21.15.66:0/3956133225 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7401f3ed0 0x7fa7400b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa746400640 1 -- 172.21.15.66:0/3956133225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa728004810 msgr2=0x7fa7401f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa746400640 1 --2- 172.21.15.66:0/3956133225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa728004810 0x7fa7401f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa746400640 1 -- 172.21.15.66:0/3956133225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa740157980 con 0x7fa7401f4350 2024-09-10T22:09:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa746400640 1 --2- 172.21.15.66:0/3956133225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7401f4350 0x7fa740158a20 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa73800d900 tx=0x7fa73800ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa73d600640 1 -- 172.21.15.66:0/3956133225 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738004b30 con 0x7fa7401f4350 2024-09-10T22:09:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa73d600640 1 -- 172.21.15.66:0/3956133225 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa738004de0 con 0x7fa7401f4350 2024-09-10T22:09:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa73d600640 1 -- 172.21.15.66:0/3956133225 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73800bcb0 con 0x7fa7401f4350 2024-09-10T22:09:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.277+0000 7fa748400640 1 -- 172.21.15.66:0/3956133225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa740157c60 con 0x7fa7401f4350 2024-09-10T22:09:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.281+0000 7fa748400640 1 -- 172.21.15.66:0/3956133225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa740158220 con 0x7fa7401f4350 2024-09-10T22:09:04.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.281+0000 7fa748400640 1 -- 172.21.15.66:0/3956133225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7400cfc70 con 0x7fa7401f4350 2024-09-10T22:09:04.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.281+0000 7fa73d600640 1 -- 172.21.15.66:0/3956133225 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7fa738010460 con 0x7fa7401f4350 2024-09-10T22:09:04.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.285+0000 7fa73d600640 1 --2- 172.21.15.66:0/3956133225 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa720073ab0 0x7fa720075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:04.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.285+0000 7fa73d600640 1 -- 172.21.15.66:0/3956133225 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fa7380929b0 con 0x7fa7401f4350 2024-09-10T22:09:04.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.285+0000 7fa73d600640 1 -- 172.21.15.66:0/3956133225 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa73805fae0 con 0x7fa7401f4350 2024-09-10T22:09:04.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.309+0000 7fa745a00640 1 --2- 172.21.15.66:0/3956133225 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa720073ab0 0x7fa720075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:04.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.317+0000 7fa745a00640 1 --2- 172.21.15.66:0/3956133225 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa720073ab0 0x7fa720075f70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa734007fd0 tx=0x7fa7340079c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:04.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.701+0000 7fa748400640 1 -- 172.21.15.66:0/3956133225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa74006e330 con 0x7fa7401f4350 2024-09-10T22:09:04.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.701+0000 7fa73d600640 1 -- 172.21.15.66:0/3956133225 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2183 (secure 0 0 0) 0x7fa738068030 con 0x7fa7401f4350 2024-09-10T22:09:04.710 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:09:04.710 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:09:04.710 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:09:04.710 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:09:04.710 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:09:04.710 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:09:04.710 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:09:04.710 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 537, 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:09:04.711 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 85, 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:09:04.712 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 1501, 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 6150015829, 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 12405559296, 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 760688553984, 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: "read_bytes_sec": 262892415, 2024-09-10T22:09:04.713 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 261882631, 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 64, 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 74 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 57, 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:08:29:488488+0000", 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 6, 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 6, 2024-09-10T22:09:04.714 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "d", 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 6990 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:09:04.715 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 14, 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:08:02.433118+0000", 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": { 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr-vol-ongoing-clones": { 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "4 ongoing clones - average progress is 25.977% (5s)\n [=======.....................] (remaining: 14s)", 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: "progress": 0.25977000594139099, 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: "add_to_ceph_s": true 2024-09-10T22:09:04.716 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:04.717 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:04.717 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:09:04.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.709+0000 7fa748400640 1 -- 172.21.15.66:0/3956133225 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa720073ab0 msgr2=0x7fa720075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:04.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.709+0000 7fa748400640 1 --2- 172.21.15.66:0/3956133225 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa720073ab0 0x7fa720075f70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa734007fd0 tx=0x7fa7340079c0 comp rx=0 tx=0).stop 2024-09-10T22:09:04.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.709+0000 7fa748400640 1 -- 172.21.15.66:0/3956133225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7401f4350 msgr2=0x7fa740158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:04.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.709+0000 7fa748400640 1 --2- 172.21.15.66:0/3956133225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7401f4350 0x7fa740158a20 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa73800d900 tx=0x7fa73800ddc0 comp rx=0 tx=0).stop 2024-09-10T22:09:04.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.709+0000 7fa748400640 1 -- 172.21.15.66:0/3956133225 shutdown_connections 2024-09-10T22:09:04.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.709+0000 7fa748400640 1 --2- 172.21.15.66:0/3956133225 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa720073ab0 0x7fa720075f70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:04.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.709+0000 7fa748400640 1 --2- 172.21.15.66:0/3956133225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7401f4350 0x7fa740158a20 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:04.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.709+0000 7fa748400640 1 --2- 172.21.15.66:0/3956133225 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7401f3ed0 0x7fa7400b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:04.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.709+0000 7fa748400640 1 --2- 172.21.15.66:0/3956133225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa728004810 0x7fa7401f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:04.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.709+0000 7fa748400640 1 -- 172.21.15.66:0/3956133225 >> 172.21.15.66:0/3956133225 conn(0x7fa7400d1c50 msgr2=0x7fa74015b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:04.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.709+0000 7fa748400640 1 -- 172.21.15.66:0/3956133225 shutdown_connections 2024-09-10T22:09:04.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:04.709+0000 7fa748400640 1 -- 172.21.15.66:0/3956133225 wait complete. 2024-09-10T22:09:05.748 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:09:05.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.965+0000 7f63d3200640 1 -- 172.21.15.66:0/1158191779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63cc036700 msgr2=0x7f63cc006110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:05.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.965+0000 7f63d3200640 1 --2- 172.21.15.66:0/1158191779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63cc036700 0x7f63cc006110 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009900 tx=0x7f63c001c640 comp rx=0 tx=0).stop 2024-09-10T22:09:05.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.965+0000 7f63d3200640 1 -- 172.21.15.66:0/1158191779 shutdown_connections 2024-09-10T22:09:05.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.965+0000 7f63d3200640 1 --2- 172.21.15.66:0/1158191779 >> v2:172.21.15.94:3300/0 conn(0x7f63cc01c370 0x7f63cc006650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:05.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.965+0000 7f63d3200640 1 --2- 172.21.15.66:0/1158191779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63cc036700 0x7f63cc006110 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:05.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.965+0000 7f63d3200640 1 -- 172.21.15.66:0/1158191779 >> 172.21.15.66:0/1158191779 conn(0x7f63cc054570 msgr2=0x7f63cc054980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:05.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.965+0000 7f63d3200640 1 -- 172.21.15.66:0/1158191779 shutdown_connections 2024-09-10T22:09:05.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.965+0000 7f63d3200640 1 -- 172.21.15.66:0/1158191779 wait complete. 2024-09-10T22:09:05.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.969+0000 7f63d3200640 1 Processor -- start 2024-09-10T22:09:05.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.969+0000 7f63d3200640 1 -- start start 2024-09-10T22:09:05.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.973+0000 7f63d3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63cc00ba10 0x7f63cc0e0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:05.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.973+0000 7f63d3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63cc01c370 0x7f63cc0db4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:05.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.973+0000 7f63d3200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63cc036700 0x7f63cc050a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:05.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.973+0000 7f63d3200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc0eb400 con 0x7f63cc00ba10 2024-09-10T22:09:05.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.973+0000 7f63d3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc0eb280 con 0x7f63cc01c370 2024-09-10T22:09:05.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.973+0000 7f63d3200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f63cc0eb580 con 0x7f63cc036700 2024-09-10T22:09:05.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.973+0000 7f63d1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63cc00ba10 0x7f63cc0e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:05.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.973+0000 7f63d1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63cc00ba10 0x7f63cc0e0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40504/0 (socket says 172.21.15.66:40504) 2024-09-10T22:09:05.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.973+0000 7f63d1e00640 1 -- 172.21.15.66:0/3582933465 learned_addr learned my addr 172.21.15.66:0/3582933465 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:05.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.973+0000 7f63d2800640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63cc036700 0x7f63cc050a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:05.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.973+0000 7f63d1400640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63cc01c370 0x7f63cc0db4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:05.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.977+0000 7f63d1e00640 1 -- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63cc036700 msgr2=0x7f63cc050a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:05.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.977+0000 7f63d1e00640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63cc036700 0x7f63cc050a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:05.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.977+0000 7f63d1e00640 1 -- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63cc01c370 msgr2=0x7f63cc0db4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:05.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.977+0000 7f63d1e00640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63cc01c370 0x7f63cc0db4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:05.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.977+0000 7f63d1e00640 1 -- 172.21.15.66:0/3582933465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63cc0e0a00 con 0x7f63cc00ba10 2024-09-10T22:09:05.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.977+0000 7f63d1e00640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63cc00ba10 0x7f63cc0e0390 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f63c400c430 tx=0x7f63c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.977+0000 7f63ca000640 1 -- 172.21.15.66:0/3582933465 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4011790 con 0x7f63cc00ba10 2024-09-10T22:09:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.977+0000 7f63d3200640 1 -- 172.21.15.66:0/3582933465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63cc0e0ce0 con 0x7f63cc00ba10 2024-09-10T22:09:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.977+0000 7f63d3200640 1 -- 172.21.15.66:0/3582933465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63cc0e12a0 con 0x7f63cc00ba10 2024-09-10T22:09:05.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.977+0000 7f63ca000640 1 -- 172.21.15.66:0/3582933465 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63c4017070 con 0x7f63cc00ba10 2024-09-10T22:09:05.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.977+0000 7f63ca000640 1 -- 172.21.15.66:0/3582933465 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4011b70 con 0x7f63cc00ba10 2024-09-10T22:09:05.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.981+0000 7f63d3200640 1 -- 172.21.15.66:0/3582933465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ac0052a0 con 0x7f63cc00ba10 2024-09-10T22:09:05.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.985+0000 7f63ca000640 1 -- 172.21.15.66:0/3582933465 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7f63c4006830 con 0x7f63cc00ba10 2024-09-10T22:09:05.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.985+0000 7f63ca000640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f63b4073ae0 0x7f63b4075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.985+0000 7f63ca000640 1 -- 172.21.15.66:0/3582933465 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f63c4091ce0 con 0x7f63cc00ba10 2024-09-10T22:09:05.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.993+0000 7f63d1400640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f63b4073ae0 0x7f63b4075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:05.993+0000 7f63ca000640 1 -- 172.21.15.66:0/3582933465 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f63c4021020 con 0x7f63cc00ba10 2024-09-10T22:09:06.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.017+0000 7f63d1400640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f63b4073ae0 0x7f63b4075fa0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f63c00af000 tx=0x7f63c001ae80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:06.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.373+0000 7f63d3200640 1 -- 172.21.15.66:0/3582933465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f63ac005860 con 0x7f63cc00ba10 2024-09-10T22:09:06.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.373+0000 7f63ca000640 1 -- 172.21.15.66:0/3582933465 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2183 (secure 0 0 0) 0x7f63c40670d0 con 0x7f63cc00ba10 2024-09-10T22:09:06.380 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:09:06.380 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:09:06.380 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:09:06.380 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:09:06.380 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:09:06.380 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:09:06.380 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:06.381 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 539, 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 85, 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:09:06.382 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 1501, 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 6150015829, 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 12405559296, 2024-09-10T22:09:06.383 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 760688553984, 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "read_bytes_sec": 262892415, 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 261882631, 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 64, 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 74 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 57, 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:08:29:488488+0000", 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 6, 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:09:06.384 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 6, 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "d", 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 6990 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:09:06.385 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 14, 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:08:02.433118+0000", 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": { 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr-vol-ongoing-clones": { 2024-09-10T22:09:06.386 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "4 ongoing clones - average progress is 25.977% (5s)\n [=======.....................] (remaining: 14s)", 2024-09-10T22:09:06.387 INFO:teuthology.orchestra.run.smithi066.stdout: "progress": 0.25977000594139099, 2024-09-10T22:09:06.387 INFO:teuthology.orchestra.run.smithi066.stdout: "add_to_ceph_s": true 2024-09-10T22:09:06.387 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:06.387 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:06.387 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:09:06.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.377+0000 7f63d3200640 1 -- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f63b4073ae0 msgr2=0x7f63b4075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:06.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.377+0000 7f63d3200640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f63b4073ae0 0x7f63b4075fa0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f63c00af000 tx=0x7f63c001ae80 comp rx=0 tx=0).stop 2024-09-10T22:09:06.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.377+0000 7f63d3200640 1 -- 172.21.15.66:0/3582933465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63cc00ba10 msgr2=0x7f63cc0e0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:06.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.377+0000 7f63d3200640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63cc00ba10 0x7f63cc0e0390 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f63c400c430 tx=0x7f63c400c8f0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.377+0000 7f63d3200640 1 -- 172.21.15.66:0/3582933465 shutdown_connections 2024-09-10T22:09:06.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.377+0000 7f63d3200640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f63b4073ae0 0x7f63b4075fa0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.377+0000 7f63d3200640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63cc036700 0x7f63cc050a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.377+0000 7f63d3200640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63cc01c370 0x7f63cc0db4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.377+0000 7f63d3200640 1 --2- 172.21.15.66:0/3582933465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63cc00ba10 0x7f63cc0e0390 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.377+0000 7f63d3200640 1 -- 172.21.15.66:0/3582933465 >> 172.21.15.66:0/3582933465 conn(0x7f63cc054570 msgr2=0x7f63cc0de120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.377+0000 7f63d3200640 1 -- 172.21.15.66:0/3582933465 shutdown_connections 2024-09-10T22:09:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:06.377+0000 7f63d3200640 1 -- 172.21.15.66:0/3582933465 wait complete. 2024-09-10T22:09:07.408 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:09:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.661+0000 7f5528a00640 1 -- 172.21.15.66:0/3828002816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552415f570 msgr2=0x7f55240cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.661+0000 7f5528a00640 1 --2- 172.21.15.66:0/3828002816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552415f570 0x7f55240cfa30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f550c009900 tx=0x7f550c01c850 comp rx=0 tx=0).stop 2024-09-10T22:09:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.661+0000 7f5528a00640 1 -- 172.21.15.66:0/3828002816 shutdown_connections 2024-09-10T22:09:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.661+0000 7f5528a00640 1 --2- 172.21.15.66:0/3828002816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552415f570 0x7f55240cfa30 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.661+0000 7f5528a00640 1 --2- 172.21.15.66:0/3828002816 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f552415f190 0x7f55240b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:07.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.661+0000 7f5528a00640 1 -- 172.21.15.66:0/3828002816 >> 172.21.15.66:0/3828002816 conn(0x7f55240d1c50 msgr2=0x7f55240d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:07.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.661+0000 7f5528a00640 1 -- 172.21.15.66:0/3828002816 shutdown_connections 2024-09-10T22:09:07.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.661+0000 7f5528a00640 1 -- 172.21.15.66:0/3828002816 wait complete. 2024-09-10T22:09:07.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.665+0000 7f5528a00640 1 Processor -- start 2024-09-10T22:09:07.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.665+0000 7f5528a00640 1 -- start start 2024-09-10T22:09:07.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.665+0000 7f5528a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55240cff70 0x7f5524159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:07.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.665+0000 7f5528a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552415f190 0x7f55241fb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:07.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.665+0000 7f5528a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f552415f570 0x7f5524154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:07.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.665+0000 7f5528a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5524168f60 con 0x7f552415f190 2024-09-10T22:09:07.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.665+0000 7f5528a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5524168de0 con 0x7f55240cff70 2024-09-10T22:09:07.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.665+0000 7f5528a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f55241690e0 con 0x7f552415f570 2024-09-10T22:09:07.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.669+0000 7f551be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552415f190 0x7f55241fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:07.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.669+0000 7f551be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552415f190 0x7f55241fb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40510/0 (socket says 172.21.15.66:40510) 2024-09-10T22:09:07.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.669+0000 7f551be00640 1 -- 172.21.15.66:0/4038039205 learned_addr learned my addr 172.21.15.66:0/4038039205 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:07.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.669+0000 7f5523400640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55240cff70 0x7f5524159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:07.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.669+0000 7f551be00640 1 -- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f552415f570 msgr2=0x7f5524154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:07.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.669+0000 7f551be00640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f552415f570 0x7f5524154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:07.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.669+0000 7f551be00640 1 -- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55240cff70 msgr2=0x7f5524159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:07.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.669+0000 7f551be00640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55240cff70 0x7f5524159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:07.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.669+0000 7f551be00640 1 -- 172.21.15.66:0/4038039205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552415a030 con 0x7f552415f190 2024-09-10T22:09:07.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.673+0000 7f551be00640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552415f190 0x7f55241fb890 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f550c01a480 tx=0x7f550c0a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:07.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.673+0000 7f551b400640 1 -- 172.21.15.66:0/4038039205 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c0ba020 con 0x7f552415f190 2024-09-10T22:09:07.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.673+0000 7f551b400640 1 -- 172.21.15.66:0/4038039205 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f550c004430 con 0x7f552415f190 2024-09-10T22:09:07.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.673+0000 7f551b400640 1 -- 172.21.15.66:0/4038039205 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c0b2040 con 0x7f552415f190 2024-09-10T22:09:07.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.673+0000 7f5528a00640 1 -- 172.21.15.66:0/4038039205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552415a310 con 0x7f552415f190 2024-09-10T22:09:07.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.673+0000 7f5528a00640 1 -- 172.21.15.66:0/4038039205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552415a8d0 con 0x7f552415f190 2024-09-10T22:09:07.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.677+0000 7f5528a00640 1 -- 172.21.15.66:0/4038039205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55240b2f30 con 0x7f552415f190 2024-09-10T22:09:07.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.677+0000 7f551b400640 1 -- 172.21.15.66:0/4038039205 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7f550c003860 con 0x7f552415f190 2024-09-10T22:09:07.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.681+0000 7f551b400640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5500073b00 0x7f5500075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:07.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.681+0000 7f551b400640 1 -- 172.21.15.66:0/4038039205 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f550c12baa0 con 0x7f552415f190 2024-09-10T22:09:07.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.685+0000 7f5523400640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5500073b00 0x7f5500075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:07.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.685+0000 7f551b400640 1 -- 172.21.15.66:0/4038039205 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f550c0164d0 con 0x7f552415f190 2024-09-10T22:09:07.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:07.713+0000 7f5523400640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5500073b00 0x7f5500075fc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5514006fd0 tx=0x7f5514008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:08.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.105+0000 7f5528a00640 1 -- 172.21.15.66:0/4038039205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f552415e9c0 con 0x7f552415f190 2024-09-10T22:09:08.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.113+0000 7f551b400640 1 -- 172.21.15.66:0/4038039205 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2421 (secure 0 0 0) 0x7f550c0f8cd0 con 0x7f552415f190 2024-09-10T22:09:08.120 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:09:08.121 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:09:08.121 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:09:08.121 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:09:08.121 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:09:08.121 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:09:08.121 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:09:08.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:08.121 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:09:08.121 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:09:08.121 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:09:08.121 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:09:08.121 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 541, 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:08.122 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 85, 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:09:08.123 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:09:08.124 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:08.124 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:08.124 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:09:08.124 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:09:08.124 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 1615, 2024-09-10T22:09:08.124 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 6628166485, 2024-09-10T22:09:08.124 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 13403541504, 2024-09-10T22:09:08.124 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 759690571776, 2024-09-10T22:09:08.124 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:09:08.124 INFO:teuthology.orchestra.run.smithi066.stdout: "read_bytes_sec": 303192988, 2024-09-10T22:09:08.124 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 301486068, 2024-09-10T22:09:08.125 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 73, 2024-09-10T22:09:08.125 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 83 2024-09-10T22:09:08.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:08.126 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:09:08.126 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 57, 2024-09-10T22:09:08.126 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:08:29:488488+0000", 2024-09-10T22:09:08.126 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 6, 2024-09-10T22:09:08.126 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:09:08.126 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:09:08.126 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:09:08.126 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:09:08.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:09:08.126 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 6, 2024-09-10T22:09:08.126 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "d", 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 6990 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:09:08.127 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 14, 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:08:02.433118+0000", 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": { 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr-vol-ongoing-clones": { 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "4 ongoing clones - average progress is 30.989% (7s)\n [========....................] (remaining: 15s)", 2024-09-10T22:09:08.128 INFO:teuthology.orchestra.run.smithi066.stdout: "progress": 0.30989000201225281, 2024-09-10T22:09:08.129 INFO:teuthology.orchestra.run.smithi066.stdout: "add_to_ceph_s": true 2024-09-10T22:09:08.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:08.129 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr-vol-total-clones": { 2024-09-10T22:09:08.129 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "Total 5 clones - average progress is 27.24% (0s)\n [=======.....................] ", 2024-09-10T22:09:08.129 INFO:teuthology.orchestra.run.smithi066.stdout: "progress": 0.27239999175071716, 2024-09-10T22:09:08.129 INFO:teuthology.orchestra.run.smithi066.stdout: "add_to_ceph_s": true 2024-09-10T22:09:08.129 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:08.129 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:08.129 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:09:08.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.117+0000 7f5528a00640 1 -- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5500073b00 msgr2=0x7f5500075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:08.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.117+0000 7f5528a00640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5500073b00 0x7f5500075fc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5514006fd0 tx=0x7f5514008040 comp rx=0 tx=0).stop 2024-09-10T22:09:08.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.117+0000 7f5528a00640 1 -- 172.21.15.66:0/4038039205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552415f190 msgr2=0x7f55241fb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:08.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.117+0000 7f5528a00640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552415f190 0x7f55241fb890 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f550c01a480 tx=0x7f550c0a5710 comp rx=0 tx=0).stop 2024-09-10T22:09:08.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.121+0000 7f5528a00640 1 -- 172.21.15.66:0/4038039205 shutdown_connections 2024-09-10T22:09:08.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.121+0000 7f5528a00640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5500073b00 0x7f5500075fc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:08.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.121+0000 7f5528a00640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f552415f570 0x7f5524154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:08.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.121+0000 7f5528a00640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552415f190 0x7f55241fb890 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:08.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.121+0000 7f5528a00640 1 --2- 172.21.15.66:0/4038039205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55240cff70 0x7f5524159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:08.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.121+0000 7f5528a00640 1 -- 172.21.15.66:0/4038039205 >> 172.21.15.66:0/4038039205 conn(0x7f55240d1c50 msgr2=0x7f552406e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:08.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.121+0000 7f5528a00640 1 -- 172.21.15.66:0/4038039205 shutdown_connections 2024-09-10T22:09:08.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.121+0000 7f5528a00640 1 -- 172.21.15.66:0/4038039205 wait complete. 2024-09-10T22:09:08.183 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 30.989% (7s)\n [========....................] (remaining: 15s)', 'progress': 0.3098900020122528, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 5 clones - average progress is 27.24% (0s)\n [=======.....................] ', 'progress': 0.27239999175071716, 'add_to_ceph_s': True}} 2024-09-10T22:09:08.184 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone51775 2024-09-10T22:09:08.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.473+0000 7fcd27000640 1 -- 172.21.15.66:0/1282086390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20073610 msgr2=0x7fcd20073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:08.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.473+0000 7fcd27000640 1 --2- 172.21.15.66:0/1282086390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20073610 0x7fcd20073a40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009c80 tx=0x7fcd1801cda0 comp rx=0 tx=0).stop 2024-09-10T22:09:08.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.473+0000 7fcd27000640 1 -- 172.21.15.66:0/1282086390 shutdown_connections 2024-09-10T22:09:08.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.473+0000 7fcd27000640 1 --2- 172.21.15.66:0/1282086390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20073610 0x7fcd20073a40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:08.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.473+0000 7fcd27000640 1 --2- 172.21.15.66:0/1282086390 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd20071120 0x7fcd200705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:08.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.473+0000 7fcd27000640 1 -- 172.21.15.66:0/1282086390 >> 172.21.15.66:0/1282086390 conn(0x7fcd200cd5d0 msgr2=0x7fcd200cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:08.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.473+0000 7fcd27000640 1 -- 172.21.15.66:0/1282086390 shutdown_connections 2024-09-10T22:09:08.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.477+0000 7fcd27000640 1 -- 172.21.15.66:0/1282086390 wait complete. 2024-09-10T22:09:08.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.477+0000 7fcd27000640 1 Processor -- start 2024-09-10T22:09:08.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.477+0000 7fcd27000640 1 -- start start 2024-09-10T22:09:08.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd27000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20071120 0x7fcd20149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:08.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd27000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd20073610 0x7fcd201511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:08.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd27000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd2015ac10 0x7fcd20151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:08.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd27000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20165c90 con 0x7fcd20073610 2024-09-10T22:09:08.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd27000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20165b10 con 0x7fcd20071120 2024-09-10T22:09:08.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd27000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcd20165e10 con 0x7fcd2015ac10 2024-09-10T22:09:08.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd1f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd20073610 0x7fcd201511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:08.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd1f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd20073610 0x7fcd201511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40518/0 (socket says 172.21.15.66:40518) 2024-09-10T22:09:08.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd1f400640 1 -- 172.21.15.66:0/4289317551 learned_addr learned my addr 172.21.15.66:0/4289317551 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:08.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd1fe00640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20071120 0x7fcd20149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:08.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd25000640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd2015ac10 0x7fcd20151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:08.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd1f400640 1 -- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd2015ac10 msgr2=0x7fcd20151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:08.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd1f400640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd2015ac10 0x7fcd20151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:08.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd1f400640 1 -- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20071120 msgr2=0x7fcd20149060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:08.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd1f400640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20071120 0x7fcd20149060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:08.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd1f400640 1 -- 172.21.15.66:0/4289317551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd200ca650 con 0x7fcd20073610 2024-09-10T22:09:08.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd1fe00640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20071120 0x7fcd20149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:09:08.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd25000640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd2015ac10 0x7fcd20151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:09:08.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd1f400640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd20073610 0x7fcd201511e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00e940 tx=0x7fcd0c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:08.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd17e00640 1 -- 172.21.15.66:0/4289317551 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c00cd80 con 0x7fcd20073610 2024-09-10T22:09:08.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd17e00640 1 -- 172.21.15.66:0/4289317551 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd0c005020 con 0x7fcd20073610 2024-09-10T22:09:08.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd27000640 1 -- 172.21.15.66:0/4289317551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd200ca930 con 0x7fcd20073610 2024-09-10T22:09:08.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd17e00640 1 -- 172.21.15.66:0/4289317551 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c005450 con 0x7fcd20073610 2024-09-10T22:09:08.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.481+0000 7fcd27000640 1 -- 172.21.15.66:0/4289317551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2014c340 con 0x7fcd20073610 2024-09-10T22:09:08.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.485+0000 7fcd27000640 1 -- 172.21.15.66:0/4289317551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd200705a0 con 0x7fcd20073610 2024-09-10T22:09:08.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.485+0000 7fcd17e00640 1 -- 172.21.15.66:0/4289317551 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7fcd0c0047b0 con 0x7fcd20073610 2024-09-10T22:09:08.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.489+0000 7fcd17e00640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd00073ab0 0x7fcd00075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:08.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.489+0000 7fcd17e00640 1 -- 172.21.15.66:0/4289317551 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7fcd0c092ea0 con 0x7fcd20073610 2024-09-10T22:09:08.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.497+0000 7fcd17e00640 1 -- 172.21.15.66:0/4289317551 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcd0c05fbf0 con 0x7fcd20073610 2024-09-10T22:09:08.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.497+0000 7fcd1fe00640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd00073ab0 0x7fcd00075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:08.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.537+0000 7fcd1fe00640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd00073ab0 0x7fcd00075f70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcd200ced50 tx=0x7fcd10009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:08.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:08.897+0000 7fcd27000640 1 -- 172.21.15.66:0/4289317551 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone51775", "target": ["mon-mgr", ""]}) -- 0x7fcd2015e520 con 0x7fcd00073ab0 2024-09-10T22:09:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.917+0000 7fcd17e00640 1 -- 172.21.15.66:0/4289317551 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcd2015e520 con 0x7fcd00073ab0 2024-09-10T22:09:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.917+0000 7fcd27000640 1 -- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd00073ab0 msgr2=0x7fcd00075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.917+0000 7fcd27000640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd00073ab0 0x7fcd00075f70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcd200ced50 tx=0x7fcd10009040 comp rx=0 tx=0).stop 2024-09-10T22:09:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.917+0000 7fcd27000640 1 -- 172.21.15.66:0/4289317551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd20073610 msgr2=0x7fcd201511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.917+0000 7fcd27000640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd20073610 0x7fcd201511e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00e940 tx=0x7fcd0c00ee00 comp rx=0 tx=0).stop 2024-09-10T22:09:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.921+0000 7fcd27000640 1 -- 172.21.15.66:0/4289317551 shutdown_connections 2024-09-10T22:09:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.921+0000 7fcd27000640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd00073ab0 0x7fcd00075f70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.921+0000 7fcd27000640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd2015ac10 0x7fcd20151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.921+0000 7fcd27000640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd20073610 0x7fcd201511e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.921+0000 7fcd27000640 1 --2- 172.21.15.66:0/4289317551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20071120 0x7fcd20149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.921+0000 7fcd27000640 1 -- 172.21.15.66:0/4289317551 >> 172.21.15.66:0/4289317551 conn(0x7fcd200cd5d0 msgr2=0x7fcd200b5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.921+0000 7fcd27000640 1 -- 172.21.15.66:0/4289317551 shutdown_connections 2024-09-10T22:09:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:09.921+0000 7fcd27000640 1 -- 172.21.15.66:0/4289317551 wait complete. 2024-09-10T22:09:09.974 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone23716 2024-09-10T22:09:10.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.209+0000 7f2bf3e00640 1 -- 172.21.15.66:0/3522737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bf415ab30 msgr2=0x7f2bf40cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:10.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.209+0000 7f2bf3e00640 1 --2- 172.21.15.66:0/3522737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bf415ab30 0x7f2bf40cc620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009930 tx=0x7f2be401d340 comp rx=0 tx=0).stop 2024-09-10T22:09:10.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.209+0000 7f2bf3e00640 1 -- 172.21.15.66:0/3522737 shutdown_connections 2024-09-10T22:09:10.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.209+0000 7f2bf3e00640 1 --2- 172.21.15.66:0/3522737 >> v2:172.21.15.94:3300/0 conn(0x7f2bf40cc0d0 0x7f2bf40b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:10.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.209+0000 7f2bf3e00640 1 --2- 172.21.15.66:0/3522737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bf415ab30 0x7f2bf40cc620 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:10.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.209+0000 7f2bf3e00640 1 -- 172.21.15.66:0/3522737 >> 172.21.15.66:0/3522737 conn(0x7f2bf4074060 msgr2=0x7f2bf4074470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:10.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.209+0000 7f2bf3e00640 1 -- 172.21.15.66:0/3522737 shutdown_connections 2024-09-10T22:09:10.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.209+0000 7f2bf3e00640 1 -- 172.21.15.66:0/3522737 wait complete. 2024-09-10T22:09:10.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf3e00640 1 Processor -- start 2024-09-10T22:09:10.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf3e00640 1 -- start start 2024-09-10T22:09:10.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bf415ab30 0x7f2bf409db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:10.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bf414ebd0 0x7f2bf4153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:10.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bf41540e0 0x7f2bf406fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:10.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4168280 con 0x7f2bf414ebd0 2024-09-10T22:09:10.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4168100 con 0x7f2bf41540e0 2024-09-10T22:09:10.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf3e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf4168400 con 0x7f2bf415ab30 2024-09-10T22:09:10.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bf414ebd0 0x7f2bf4153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:10.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bf414ebd0 0x7f2bf4153ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40534/0 (socket says 172.21.15.66:40534) 2024-09-10T22:09:10.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf2000640 1 -- 172.21.15.66:0/4152472202 learned_addr learned my addr 172.21.15.66:0/4152472202 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:10.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf3400640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bf41540e0 0x7f2bf406fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:10.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf2a00640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bf415ab30 0x7f2bf409db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:10.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf2000640 1 -- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bf415ab30 msgr2=0x7f2bf409db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:10.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf2000640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bf415ab30 0x7f2bf409db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:10.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf2000640 1 -- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bf41540e0 msgr2=0x7f2bf406fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:10.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf2000640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bf41540e0 0x7f2bf406fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:10.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf2000640 1 -- 172.21.15.66:0/4152472202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf414cb00 con 0x7f2bf414ebd0 2024-09-10T22:09:10.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf2a00640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bf415ab30 0x7f2bf409db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:09:10.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf2000640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bf414ebd0 0x7f2bf4153ba0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2be000c970 tx=0x7f2be000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:10.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2beaa00640 1 -- 172.21.15.66:0/4152472202 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0007bb0 con 0x7f2bf414ebd0 2024-09-10T22:09:10.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2beaa00640 1 -- 172.21.15.66:0/4152472202 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2be0005020 con 0x7f2bf414ebd0 2024-09-10T22:09:10.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf3e00640 1 -- 172.21.15.66:0/4152472202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf41f5fc0 con 0x7f2bf414ebd0 2024-09-10T22:09:10.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2beaa00640 1 -- 172.21.15.66:0/4152472202 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0007e60 con 0x7f2bf414ebd0 2024-09-10T22:09:10.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf3e00640 1 -- 172.21.15.66:0/4152472202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf41f6580 con 0x7f2bf414ebd0 2024-09-10T22:09:10.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.213+0000 7f2bf3e00640 1 -- 172.21.15.66:0/4152472202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf40b39e0 con 0x7f2bf414ebd0 2024-09-10T22:09:10.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.217+0000 7f2beaa00640 1 -- 172.21.15.66:0/4152472202 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7f2be0005470 con 0x7f2bf414ebd0 2024-09-10T22:09:10.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.217+0000 7f2beaa00640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bd4073ae0 0x7f2bd4075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:10.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.217+0000 7f2beaa00640 1 -- 172.21.15.66:0/4152472202 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f2be0092240 con 0x7f2bf414ebd0 2024-09-10T22:09:10.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.221+0000 7f2bf2a00640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bd4073ae0 0x7f2bd4075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:10.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.221+0000 7f2bf2a00640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bd4073ae0 0x7f2bd4075fa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2be40af000 tx=0x7f2be401b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:10.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.225+0000 7f2beaa00640 1 -- 172.21.15.66:0/4152472202 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2be0014030 con 0x7f2bf414ebd0 2024-09-10T22:09:10.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:10.665+0000 7f2bf3e00640 1 -- 172.21.15.66:0/4152472202 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone23716", "target": ["mon-mgr", ""]}) -- 0x7f2bf40cc890 con 0x7f2bd4073ae0 2024-09-10T22:09:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.457+0000 7f2beaa00640 1 -- 172.21.15.66:0/4152472202 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2bf40cc890 con 0x7f2bd4073ae0 2024-09-10T22:09:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.461+0000 7f2bf3e00640 1 -- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bd4073ae0 msgr2=0x7f2bd4075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.461+0000 7f2bf3e00640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bd4073ae0 0x7f2bd4075fa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2be40af000 tx=0x7f2be401b120 comp rx=0 tx=0).stop 2024-09-10T22:09:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.461+0000 7f2bf3e00640 1 -- 172.21.15.66:0/4152472202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bf414ebd0 msgr2=0x7f2bf4153ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.461+0000 7f2bf3e00640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bf414ebd0 0x7f2bf4153ba0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2be000c970 tx=0x7f2be000ce30 comp rx=0 tx=0).stop 2024-09-10T22:09:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.461+0000 7f2bf3e00640 1 -- 172.21.15.66:0/4152472202 shutdown_connections 2024-09-10T22:09:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.461+0000 7f2bf3e00640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2bd4073ae0 0x7f2bd4075fa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.461+0000 7f2bf3e00640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bf41540e0 0x7f2bf406fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.461+0000 7f2bf3e00640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bf414ebd0 0x7f2bf4153ba0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.461+0000 7f2bf3e00640 1 --2- 172.21.15.66:0/4152472202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bf415ab30 0x7f2bf409db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.461+0000 7f2bf3e00640 1 -- 172.21.15.66:0/4152472202 >> 172.21.15.66:0/4152472202 conn(0x7f2bf4074060 msgr2=0x7f2bf415da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.465+0000 7f2bf3e00640 1 -- 172.21.15.66:0/4152472202 shutdown_connections 2024-09-10T22:09:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.465+0000 7f2bf3e00640 1 -- 172.21.15.66:0/4152472202 wait complete. 2024-09-10T22:09:11.517 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone5127 2024-09-10T22:09:11.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.813+0000 7f3d43200640 1 -- 172.21.15.66:0/3190559925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d44161240 msgr2=0x7f3d440b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:11.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.813+0000 7f3d43200640 1 --2- 172.21.15.66:0/3190559925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d44161240 0x7f3d440b2f70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009900 tx=0x7f3d3401c880 comp rx=0 tx=0).stop 2024-09-10T22:09:11.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.813+0000 7f3d43200640 1 -- 172.21.15.66:0/3190559925 shutdown_connections 2024-09-10T22:09:11.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.813+0000 7f3d43200640 1 --2- 172.21.15.66:0/3190559925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d440b21d0 0x7f3d440b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:11.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.813+0000 7f3d43200640 1 --2- 172.21.15.66:0/3190559925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d44161240 0x7f3d440b2f70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:11.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.813+0000 7f3d43200640 1 -- 172.21.15.66:0/3190559925 >> 172.21.15.66:0/3190559925 conn(0x7f3d4409aa50 msgr2=0x7f3d4409ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:11.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.813+0000 7f3d43200640 1 -- 172.21.15.66:0/3190559925 shutdown_connections 2024-09-10T22:09:11.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.813+0000 7f3d43200640 1 -- 172.21.15.66:0/3190559925 wait complete. 2024-09-10T22:09:11.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.813+0000 7f3d43200640 1 Processor -- start 2024-09-10T22:09:11.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d43200640 1 -- start start 2024-09-10T22:09:11.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d43200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d440b21d0 0x7f3d44155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:11.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d43200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d440cedc0 0x7f3d4414a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:11.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d43200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d44161240 0x7f3d440c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:11.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d43200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4416b090 con 0x7f3d440b21d0 2024-09-10T22:09:11.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d43200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4416af10 con 0x7f3d440cedc0 2024-09-10T22:09:11.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d43200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3d4416b210 con 0x7f3d44161240 2024-09-10T22:09:11.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d41e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d440b21d0 0x7f3d44155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:11.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d41e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d440b21d0 0x7f3d44155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40538/0 (socket says 172.21.15.66:40538) 2024-09-10T22:09:11.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d41e00640 1 -- 172.21.15.66:0/3818454037 learned_addr learned my addr 172.21.15.66:0/3818454037 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:11.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d41400640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d440cedc0 0x7f3d4414a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:11.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d42800640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d44161240 0x7f3d440c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:11.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d41e00640 1 -- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d44161240 msgr2=0x7f3d440c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:11.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d41e00640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d44161240 0x7f3d440c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:11.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d41e00640 1 -- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d440cedc0 msgr2=0x7f3d4414a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:11.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d41e00640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d440cedc0 0x7f3d4414a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:11.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d41e00640 1 -- 172.21.15.66:0/3818454037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d44158d70 con 0x7f3d440b21d0 2024-09-10T22:09:11.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d41e00640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d440b21d0 0x7f3d44155930 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3d340194f0 tx=0x7f3d34004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:11.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d3a000640 1 -- 172.21.15.66:0/3818454037 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d340b2040 con 0x7f3d440b21d0 2024-09-10T22:09:11.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d3a000640 1 -- 172.21.15.66:0/3818454037 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d34003600 con 0x7f3d440b21d0 2024-09-10T22:09:11.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d43200640 1 -- 172.21.15.66:0/3818454037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d44154e50 con 0x7f3d440b21d0 2024-09-10T22:09:11.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d3a000640 1 -- 172.21.15.66:0/3818454037 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d34003a10 con 0x7f3d440b21d0 2024-09-10T22:09:11.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.817+0000 7f3d43200640 1 -- 172.21.15.66:0/3818454037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d44155410 con 0x7f3d440b21d0 2024-09-10T22:09:11.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.821+0000 7f3d43200640 1 -- 172.21.15.66:0/3818454037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d440b2660 con 0x7f3d440b21d0 2024-09-10T22:09:11.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.821+0000 7f3d3a000640 1 -- 172.21.15.66:0/3818454037 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7f3d440d1eb0 con 0x7f3d440b21d0 2024-09-10T22:09:11.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.825+0000 7f3d3a000640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3d24073ab0 0x7f3d24075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:11.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.825+0000 7f3d3a000640 1 -- 172.21.15.66:0/3818454037 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f3d3412c120 con 0x7f3d440b21d0 2024-09-10T22:09:11.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.829+0000 7f3d3a000640 1 -- 172.21.15.66:0/3818454037 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3d340164d0 con 0x7f3d440b21d0 2024-09-10T22:09:11.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.837+0000 7f3d41400640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3d24073ab0 0x7f3d24075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:11.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:11.841+0000 7f3d41400640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3d24073ab0 0x7f3d24075f70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3d441503f0 tx=0x7f3d30009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:12.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.217+0000 7f3d43200640 1 -- 172.21.15.66:0/3818454037 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone5127", "target": ["mon-mgr", ""]}) -- 0x7f3d440d2970 con 0x7f3d24073ab0 2024-09-10T22:09:12.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.593+0000 7f3d3a000640 1 -- 172.21.15.66:0/3818454037 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3d440d2970 con 0x7f3d24073ab0 2024-09-10T22:09:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.597+0000 7f3d43200640 1 -- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3d24073ab0 msgr2=0x7f3d24075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.597+0000 7f3d43200640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3d24073ab0 0x7f3d24075f70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3d441503f0 tx=0x7f3d30009040 comp rx=0 tx=0).stop 2024-09-10T22:09:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.597+0000 7f3d43200640 1 -- 172.21.15.66:0/3818454037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d440b21d0 msgr2=0x7f3d44155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.597+0000 7f3d43200640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d440b21d0 0x7f3d44155930 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3d340194f0 tx=0x7f3d34004040 comp rx=0 tx=0).stop 2024-09-10T22:09:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.597+0000 7f3d43200640 1 -- 172.21.15.66:0/3818454037 shutdown_connections 2024-09-10T22:09:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.597+0000 7f3d43200640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3d24073ab0 0x7f3d24075f70 secure :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3d441503f0 tx=0x7f3d30009040 comp rx=0 tx=0).stop 2024-09-10T22:09:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.597+0000 7f3d43200640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3d44161240 0x7f3d440c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.597+0000 7f3d43200640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d440cedc0 0x7f3d4414a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.597+0000 7f3d43200640 1 --2- 172.21.15.66:0/3818454037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d440b21d0 0x7f3d44155930 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.597+0000 7f3d43200640 1 -- 172.21.15.66:0/3818454037 >> 172.21.15.66:0/3818454037 conn(0x7f3d4409aa50 msgr2=0x7f3d440b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.597+0000 7f3d43200640 1 -- 172.21.15.66:0/3818454037 shutdown_connections 2024-09-10T22:09:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.597+0000 7f3d43200640 1 -- 172.21.15.66:0/3818454037 wait complete. 2024-09-10T22:09:12.649 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone86448 2024-09-10T22:09:12.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.877+0000 7f71b9800640 1 -- 172.21.15.66:0/2322176676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b40cee70 msgr2=0x7f71b415ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:12.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.877+0000 7f71b9800640 1 --2- 172.21.15.66:0/2322176676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b40cee70 0x7f71b415ae90 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f71b4051480 tx=0x7f71ac01cd80 comp rx=0 tx=0).stop 2024-09-10T22:09:12.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 -- 172.21.15.66:0/2322176676 shutdown_connections 2024-09-10T22:09:12.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 --2- 172.21.15.66:0/2322176676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b40cee70 0x7f71b415ae90 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:12.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 --2- 172.21.15.66:0/2322176676 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f71b40d0450 0x7f71b40ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:12.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 --2- 172.21.15.66:0/2322176676 >> v2:172.21.15.94:3300/0 conn(0x7f71b41585c0 0x7f71b4074320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:12.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 -- 172.21.15.66:0/2322176676 >> 172.21.15.66:0/2322176676 conn(0x7f71b40d0a30 msgr2=0x7f71b40d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:12.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 -- 172.21.15.66:0/2322176676 shutdown_connections 2024-09-10T22:09:12.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 -- 172.21.15.66:0/2322176676 wait complete. 2024-09-10T22:09:12.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 Processor -- start 2024-09-10T22:09:12.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 -- start start 2024-09-10T22:09:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b40cee70 0x7f71b415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b40d0450 0x7f71b415c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f71b41585c0 0x7f71b415cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f71b415ed40 con 0x7f71b40cee70 2024-09-10T22:09:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71b415ebc0 con 0x7f71b40d0450 2024-09-10T22:09:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b9800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f71b415eec0 con 0x7f71b41585c0 2024-09-10T22:09:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b40cee70 0x7f71b415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b40cee70 0x7f71b415a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37254/0 (socket says 172.21.15.66:37254) 2024-09-10T22:09:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.881+0000 7f71b3e00640 1 -- 172.21.15.66:0/4046496517 learned_addr learned my addr 172.21.15.66:0/4046496517 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71b3400640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b40d0450 0x7f71b415c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71b8e00640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f71b41585c0 0x7f71b415cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71b3e00640 1 -- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f71b41585c0 msgr2=0x7f71b415cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71b3e00640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f71b41585c0 0x7f71b415cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71b3e00640 1 -- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b40d0450 msgr2=0x7f71b415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71b3e00640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b40d0450 0x7f71b415c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71b3e00640 1 -- 172.21.15.66:0/4046496517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b41f8730 con 0x7f71b40cee70 2024-09-10T22:09:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71b3e00640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b40cee70 0x7f71b415a680 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f71a40058d0 tx=0x7f71a400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:12.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71abe00640 1 -- 172.21.15.66:0/4046496517 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4017070 con 0x7f71b40cee70 2024-09-10T22:09:12.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71abe00640 1 -- 172.21.15.66:0/4046496517 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a4009d20 con 0x7f71b40cee70 2024-09-10T22:09:12.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71b9800640 1 -- 172.21.15.66:0/4046496517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b41f8a10 con 0x7f71b40cee70 2024-09-10T22:09:12.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71abe00640 1 -- 172.21.15.66:0/4046496517 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a40106c0 con 0x7f71b40cee70 2024-09-10T22:09:12.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71b9800640 1 -- 172.21.15.66:0/4046496517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b41f8fd0 con 0x7f71b40cee70 2024-09-10T22:09:12.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.885+0000 7f71b9800640 1 -- 172.21.15.66:0/4046496517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b40ce930 con 0x7f71b40cee70 2024-09-10T22:09:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.889+0000 7f71abe00640 1 -- 172.21.15.66:0/4046496517 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7f71a4010860 con 0x7f71b40cee70 2024-09-10T22:09:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.889+0000 7f71abe00640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7194073ab0 0x7f7194075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.889+0000 7f71abe00640 1 -- 172.21.15.66:0/4046496517 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5765+0+0 (secure 0 0 0) 0x7f71a4092310 con 0x7f71b40cee70 2024-09-10T22:09:12.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.893+0000 7f71b3400640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7194073ab0 0x7f7194075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.893+0000 7f71b3400640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7194073ab0 0x7f7194075f70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f71b415b1b0 tx=0x7f71a000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:12.897+0000 7f71abe00640 1 -- 172.21.15.66:0/4046496517 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f71a405ee50 con 0x7f71b40cee70 2024-09-10T22:09:13.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.297+0000 7f71b9800640 1 -- 172.21.15.66:0/4046496517 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone86448", "target": ["mon-mgr", ""]}) -- 0x7f71b41fb930 con 0x7f7194073ab0 2024-09-10T22:09:13.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.693+0000 7f71abe00640 1 -- 172.21.15.66:0/4046496517 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f71b41fb930 con 0x7f7194073ab0 2024-09-10T22:09:13.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.709+0000 7f71b9800640 1 -- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7194073ab0 msgr2=0x7f7194075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:13.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.709+0000 7f71b9800640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7194073ab0 0x7f7194075f70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f71b415b1b0 tx=0x7f71a000b2e0 comp rx=0 tx=0).stop 2024-09-10T22:09:13.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.709+0000 7f71b9800640 1 -- 172.21.15.66:0/4046496517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b40cee70 msgr2=0x7f71b415a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:13.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.709+0000 7f71b9800640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b40cee70 0x7f71b415a680 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f71a40058d0 tx=0x7f71a400dde0 comp rx=0 tx=0).stop 2024-09-10T22:09:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.709+0000 7f71b9800640 1 -- 172.21.15.66:0/4046496517 shutdown_connections 2024-09-10T22:09:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.709+0000 7f71b9800640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7194073ab0 0x7f7194075f70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.709+0000 7f71b9800640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f71b41585c0 0x7f71b415cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.709+0000 7f71b9800640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b40d0450 0x7f71b415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.709+0000 7f71b9800640 1 --2- 172.21.15.66:0/4046496517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b40cee70 0x7f71b415a680 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.709+0000 7f71b9800640 1 -- 172.21.15.66:0/4046496517 >> 172.21.15.66:0/4046496517 conn(0x7f71b40d0a30 msgr2=0x7f71b41541b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.709+0000 7f71b9800640 1 -- 172.21.15.66:0/4046496517 shutdown_connections 2024-09-10T22:09:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:13.709+0000 7f71b9800640 1 -- 172.21.15.66:0/4046496517 wait complete. 2024-09-10T22:09:13.760 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone86889 2024-09-10T22:09:14.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.073+0000 7f1da6000640 1 -- 172.21.15.66:0/3197858638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1da015a310 msgr2=0x7f1da00c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:14.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.073+0000 7f1da6000640 1 --2- 172.21.15.66:0/3197858638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1da015a310 0x7f1da00c48b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000a950 tx=0x7f1d9001dae0 comp rx=0 tx=0).stop 2024-09-10T22:09:14.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.073+0000 7f1da6000640 1 -- 172.21.15.66:0/3197858638 shutdown_connections 2024-09-10T22:09:14.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.073+0000 7f1da6000640 1 --2- 172.21.15.66:0/3197858638 >> v2:172.21.15.94:3300/0 conn(0x7f1da00d0550 0x7f1da00c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:14.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.073+0000 7f1da6000640 1 --2- 172.21.15.66:0/3197858638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1da015a310 0x7f1da00c48b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:14.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.073+0000 7f1da6000640 1 -- 172.21.15.66:0/3197858638 >> 172.21.15.66:0/3197858638 conn(0x7f1da00cfa30 msgr2=0x7f1da00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:14.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.073+0000 7f1da6000640 1 -- 172.21.15.66:0/3197858638 shutdown_connections 2024-09-10T22:09:14.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.073+0000 7f1da6000640 1 -- 172.21.15.66:0/3197858638 wait complete. 2024-09-10T22:09:14.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.073+0000 7f1da6000640 1 Processor -- start 2024-09-10T22:09:14.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1da6000640 1 -- start start 2024-09-10T22:09:14.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1da6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da00aeec0 0x7f1da01f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:14.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1da6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1da015a310 0x7f1da00b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:14.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1da6000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1da00c7620 0x7f1da00b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:14.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1da6000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1da015ed60 con 0x7f1da00aeec0 2024-09-10T22:09:14.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1da6000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1da015ebe0 con 0x7f1da015a310 2024-09-10T22:09:14.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1da6000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1da0167300 con 0x7f1da00c7620 2024-09-10T22:09:14.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d9f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da00aeec0 0x7f1da01f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:14.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d9f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da00aeec0 0x7f1da01f6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37260/0 (socket says 172.21.15.66:37260) 2024-09-10T22:09:14.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d9f400640 1 -- 172.21.15.66:0/1025936351 learned_addr learned my addr 172.21.15.66:0/1025936351 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:14.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d9fe00640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1da00c7620 0x7f1da00b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:14.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d9ea00640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1da015a310 0x7f1da00b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:14.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d9f400640 1 -- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1da00c7620 msgr2=0x7f1da00b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:14.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d9f400640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1da00c7620 0x7f1da00b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:14.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d9f400640 1 -- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1da015a310 msgr2=0x7f1da00b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:14.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d9f400640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1da015a310 0x7f1da00b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:14.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d9f400640 1 -- 172.21.15.66:0/1025936351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da00c99e0 con 0x7f1da00aeec0 2024-09-10T22:09:14.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d9fe00640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1da00c7620 0x7f1da00b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:09:14.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d9f400640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da00aeec0 0x7f1da01f6c20 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000a8a0 tx=0x7f1d900a6740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:14.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d97400640 1 -- 172.21.15.66:0/1025936351 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d900bb020 con 0x7f1da00aeec0 2024-09-10T22:09:14.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d97400640 1 -- 172.21.15.66:0/1025936351 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d900053e0 con 0x7f1da00aeec0 2024-09-10T22:09:14.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1d97400640 1 -- 172.21.15.66:0/1025936351 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d900b3040 con 0x7f1da00aeec0 2024-09-10T22:09:14.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1da6000640 1 -- 172.21.15.66:0/1025936351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da00c7b40 con 0x7f1da00aeec0 2024-09-10T22:09:14.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.077+0000 7f1da6000640 1 -- 172.21.15.66:0/1025936351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da00c8100 con 0x7f1da00aeec0 2024-09-10T22:09:14.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.081+0000 7f1da6000640 1 -- 172.21.15.66:0/1025936351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da0070420 con 0x7f1da00aeec0 2024-09-10T22:09:14.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.081+0000 7f1d97400640 1 -- 172.21.15.66:0/1025936351 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7f1d9000b040 con 0x7f1da00aeec0 2024-09-10T22:09:14.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.081+0000 7f1d97400640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1d80073ad0 0x7f1d80075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:14.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.081+0000 7f1d97400640 1 -- 172.21.15.66:0/1025936351 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5823+0+0 (secure 0 0 0) 0x7f1d9012c580 con 0x7f1da00aeec0 2024-09-10T22:09:14.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.085+0000 7f1d9ea00640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1d80073ad0 0x7f1d80075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:14.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.089+0000 7f1d9ea00640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1d80073ad0 0x7f1d80075f90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c0058d0 tx=0x7f1d8c005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:14.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.089+0000 7f1d97400640 1 -- 172.21.15.66:0/1025936351 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1d900c5020 con 0x7f1da00aeec0 2024-09-10T22:09:14.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.409+0000 7f1da6000640 1 -- 172.21.15.66:0/1025936351 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone86889", "target": ["mon-mgr", ""]}) -- 0x7f1da014cfd0 con 0x7f1d80073ad0 2024-09-10T22:09:14.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.897+0000 7f1d97400640 1 -- 172.21.15.66:0/1025936351 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1da014cfd0 con 0x7f1d80073ad0 2024-09-10T22:09:14.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.901+0000 7f1da6000640 1 -- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1d80073ad0 msgr2=0x7f1d80075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:14.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.901+0000 7f1da6000640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1d80073ad0 0x7f1d80075f90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c0058d0 tx=0x7f1d8c005860 comp rx=0 tx=0).stop 2024-09-10T22:09:14.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.901+0000 7f1da6000640 1 -- 172.21.15.66:0/1025936351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da00aeec0 msgr2=0x7f1da01f6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:14.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.901+0000 7f1da6000640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da00aeec0 0x7f1da01f6c20 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000a8a0 tx=0x7f1d900a6740 comp rx=0 tx=0).stop 2024-09-10T22:09:14.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.901+0000 7f1da6000640 1 -- 172.21.15.66:0/1025936351 shutdown_connections 2024-09-10T22:09:14.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.901+0000 7f1da6000640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1d80073ad0 0x7f1d80075f90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:14.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.901+0000 7f1da6000640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1da00c7620 0x7f1da00b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:14.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.901+0000 7f1da6000640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1da015a310 0x7f1da00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:14.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.901+0000 7f1da6000640 1 --2- 172.21.15.66:0/1025936351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da00aeec0 0x7f1da01f6c20 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:14.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.901+0000 7f1da6000640 1 -- 172.21.15.66:0/1025936351 >> 172.21.15.66:0/1025936351 conn(0x7f1da00cfa30 msgr2=0x7f1da00ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:14.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.901+0000 7f1da6000640 1 -- 172.21.15.66:0/1025936351 shutdown_connections 2024-09-10T22:09:14.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:14.901+0000 7f1da6000640 1 -- 172.21.15.66:0/1025936351 wait complete. 2024-09-10T22:09:14.941 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone201510 2024-09-10T22:09:15.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.221+0000 7f12dd600640 1 -- 172.21.15.66:0/337313326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12bc004810 msgr2=0x7f12bc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:15.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.221+0000 7f12dd600640 1 --2- 172.21.15.66:0/337313326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12bc004810 0x7f12bc004c90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009bf0 tx=0x7f12c401d310 comp rx=0 tx=0).stop 2024-09-10T22:09:15.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.221+0000 7f12dd600640 1 -- 172.21.15.66:0/337313326 shutdown_connections 2024-09-10T22:09:15.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.221+0000 7f12dd600640 1 --2- 172.21.15.66:0/337313326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12bc004810 0x7f12bc004c90 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:15.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.221+0000 7f12dd600640 1 -- 172.21.15.66:0/337313326 >> 172.21.15.66:0/337313326 conn(0x7f12d80d1c50 msgr2=0x7f12d80d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:15.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.225+0000 7f12dd600640 1 -- 172.21.15.66:0/337313326 shutdown_connections 2024-09-10T22:09:15.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.229+0000 7f12dd600640 1 -- 172.21.15.66:0/337313326 wait complete. 2024-09-10T22:09:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.233+0000 7f12dd600640 1 Processor -- start 2024-09-10T22:09:15.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.233+0000 7f12dd600640 1 -- start start 2024-09-10T22:09:15.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.233+0000 7f12dd600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12bc004810 0x7f12d80ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:15.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.233+0000 7f12dd600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12d80cf2e0 0x7f12d81f1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:15.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.233+0000 7f12dd600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d80b6880 0x7f12d81f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:15.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.233+0000 7f12dd600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12d815f8a0 con 0x7f12bc004810 2024-09-10T22:09:15.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.233+0000 7f12dd600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f12d815f720 con 0x7f12d80cf2e0 2024-09-10T22:09:15.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.233+0000 7f12dd600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f12d80cfb50 con 0x7f12d80b6880 2024-09-10T22:09:15.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12d6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12bc004810 0x7f12d80ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:15.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12d6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12bc004810 0x7f12d80ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37276/0 (socket says 172.21.15.66:37276) 2024-09-10T22:09:15.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12d6a00640 1 -- 172.21.15.66:0/2515764742 learned_addr learned my addr 172.21.15.66:0/2515764742 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:15.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12d6a00640 1 -- 172.21.15.66:0/2515764742 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d80b6880 msgr2=0x7f12d81f1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12d6a00640 1 --2- 172.21.15.66:0/2515764742 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d80b6880 0x7f12d81f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12d6a00640 1 -- 172.21.15.66:0/2515764742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12d80cf2e0 msgr2=0x7f12d81f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12d6a00640 1 --2- 172.21.15.66:0/2515764742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12d80cf2e0 0x7f12d81f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12d6a00640 1 -- 172.21.15.66:0/2515764742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d80b6d90 con 0x7f12bc004810 2024-09-10T22:09:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12d6a00640 1 --2- 172.21.15.66:0/2515764742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12bc004810 0x7f12d80ceda0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f12c800b400 tx=0x7f12c800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12cea00640 1 -- 172.21.15.66:0/2515764742 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8015020 con 0x7f12bc004810 2024-09-10T22:09:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12cea00640 1 -- 172.21.15.66:0/2515764742 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12c8011920 con 0x7f12bc004810 2024-09-10T22:09:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12cea00640 1 -- 172.21.15.66:0/2515764742 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8011d50 con 0x7f12bc004810 2024-09-10T22:09:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12dd600640 1 -- 172.21.15.66:0/2515764742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d80b7070 con 0x7f12bc004810 2024-09-10T22:09:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12dd600640 1 -- 172.21.15.66:0/2515764742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d80b7630 con 0x7f12bc004810 2024-09-10T22:09:15.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.237+0000 7f12dd600640 1 -- 172.21.15.66:0/2515764742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d80d00b0 con 0x7f12bc004810 2024-09-10T22:09:15.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.241+0000 7f12cea00640 1 -- 172.21.15.66:0/2515764742 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7f12c801a400 con 0x7f12bc004810 2024-09-10T22:09:15.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.241+0000 7f12cea00640 1 --2- 172.21.15.66:0/2515764742 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12b8073ae0 0x7f12b8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:15.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.241+0000 7f12cea00640 1 -- 172.21.15.66:0/2515764742 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f12c80920e0 con 0x7f12bc004810 2024-09-10T22:09:15.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.253+0000 7f12cea00640 1 -- 172.21.15.66:0/2515764742 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f12c801f020 con 0x7f12bc004810 2024-09-10T22:09:15.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.265+0000 7f12d6000640 1 --2- 172.21.15.66:0/2515764742 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12b8073ae0 0x7f12b8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:15.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.281+0000 7f12d6000640 1 --2- 172.21.15.66:0/2515764742 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12b8073ae0 0x7f12b8075fa0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f12c40ad000 tx=0x7f12bc005840 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:15.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:15.597+0000 7f12dd600640 1 -- 172.21.15.66:0/2515764742 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone201510", "target": ["mon-mgr", ""]}) -- 0x7f12d80afb80 con 0x7f12b8073ae0 2024-09-10T22:09:16.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.357+0000 7f12cea00640 1 -- 172.21.15.66:0/2515764742 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f12d80afb80 con 0x7f12b8073ae0 2024-09-10T22:09:16.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.365+0000 7f12dd600640 1 -- 172.21.15.66:0/2515764742 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12b8073ae0 msgr2=0x7f12b8075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:16.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.365+0000 7f12dd600640 1 --2- 172.21.15.66:0/2515764742 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12b8073ae0 0x7f12b8075fa0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f12c40ad000 tx=0x7f12bc005840 comp rx=0 tx=0).stop 2024-09-10T22:09:16.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.365+0000 7f12dd600640 1 -- 172.21.15.66:0/2515764742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12bc004810 msgr2=0x7f12d80ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:16.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.365+0000 7f12dd600640 1 --2- 172.21.15.66:0/2515764742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12bc004810 0x7f12d80ceda0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f12c800b400 tx=0x7f12c800b8c0 comp rx=0 tx=0).stop 2024-09-10T22:09:16.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.369+0000 7f12dd600640 1 -- 172.21.15.66:0/2515764742 shutdown_connections 2024-09-10T22:09:16.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.369+0000 7f12dd600640 1 --2- 172.21.15.66:0/2515764742 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12b8073ae0 0x7f12b8075fa0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:16.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.369+0000 7f12dd600640 1 --2- 172.21.15.66:0/2515764742 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d80b6880 0x7f12d81f1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:16.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.369+0000 7f12dd600640 1 --2- 172.21.15.66:0/2515764742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12d80cf2e0 0x7f12d81f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:16.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.369+0000 7f12dd600640 1 --2- 172.21.15.66:0/2515764742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12bc004810 0x7f12d80ceda0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:16.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.369+0000 7f12dd600640 1 -- 172.21.15.66:0/2515764742 >> 172.21.15.66:0/2515764742 conn(0x7f12d80d1c50 msgr2=0x7f12d8071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:16.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.369+0000 7f12dd600640 1 -- 172.21.15.66:0/2515764742 shutdown_connections 2024-09-10T22:09:16.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.369+0000 7f12dd600640 1 -- 172.21.15.66:0/2515764742 wait complete. 2024-09-10T22:09:16.416 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone403911 2024-09-10T22:09:16.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.705+0000 7fd41de00640 1 -- 172.21.15.66:0/1092076114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd41815e010 msgr2=0x7fd4180cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:16.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.705+0000 7fd41de00640 1 --2- 172.21.15.66:0/1092076114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd41815e010 0x7fd4180cfea0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd408009c80 tx=0x7fd40801cd10 comp rx=0 tx=0).stop 2024-09-10T22:09:16.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.705+0000 7fd41de00640 1 -- 172.21.15.66:0/1092076114 shutdown_connections 2024-09-10T22:09:16.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.705+0000 7fd41de00640 1 --2- 172.21.15.66:0/1092076114 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd4180d03e0 0x7fd4180d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:16.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.705+0000 7fd41de00640 1 --2- 172.21.15.66:0/1092076114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd41815e010 0x7fd4180cfea0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:16.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.705+0000 7fd41de00640 1 -- 172.21.15.66:0/1092076114 >> 172.21.15.66:0/1092076114 conn(0x7fd4180d1c60 msgr2=0x7fd4180d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:16.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.705+0000 7fd41de00640 1 -- 172.21.15.66:0/1092076114 shutdown_connections 2024-09-10T22:09:16.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.705+0000 7fd41de00640 1 -- 172.21.15.66:0/1092076114 wait complete. 2024-09-10T22:09:16.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.709+0000 7fd41de00640 1 Processor -- start 2024-09-10T22:09:16.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.709+0000 7fd41de00640 1 -- start start 2024-09-10T22:09:16.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.709+0000 7fd41de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4180d03e0 0x7fd418154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:16.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.709+0000 7fd41de00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd4180d0d60 0x7fd4180c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:16.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.709+0000 7fd41de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41815e010 0x7fd4180b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:16.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.709+0000 7fd41de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd41816aea0 con 0x7fd4180d03e0 2024-09-10T22:09:16.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.709+0000 7fd41de00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd41816ad20 con 0x7fd41815e010 2024-09-10T22:09:16.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.709+0000 7fd41de00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd41816b020 con 0x7fd4180d0d60 2024-09-10T22:09:16.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.709+0000 7fd416a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4180d03e0 0x7fd418154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:16.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.709+0000 7fd416a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4180d03e0 0x7fd418154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37304/0 (socket says 172.21.15.66:37304) 2024-09-10T22:09:16.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.709+0000 7fd416a00640 1 -- 172.21.15.66:0/1224552135 learned_addr learned my addr 172.21.15.66:0/1224552135 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:16.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.713+0000 7fd416a00640 1 -- 172.21.15.66:0/1224552135 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd4180d0d60 msgr2=0x7fd4180c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:16.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.713+0000 7fd416a00640 1 --2- 172.21.15.66:0/1224552135 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd4180d0d60 0x7fd4180c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:16.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.713+0000 7fd416a00640 1 -- 172.21.15.66:0/1224552135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41815e010 msgr2=0x7fd4180b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:16.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.713+0000 7fd416a00640 1 --2- 172.21.15.66:0/1224552135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41815e010 0x7fd4180b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:16.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.713+0000 7fd416a00640 1 -- 172.21.15.66:0/1224552135 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41815f5b0 con 0x7fd4180d03e0 2024-09-10T22:09:16.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.713+0000 7fd416a00640 1 --2- 172.21.15.66:0/1224552135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4180d03e0 0x7fd418154cd0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd408000f90 tx=0x7fd4080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:16.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.713+0000 7fd40f400640 1 -- 172.21.15.66:0/1224552135 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4080b2040 con 0x7fd4180d03e0 2024-09-10T22:09:16.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.713+0000 7fd40f400640 1 -- 172.21.15.66:0/1224552135 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4080044e0 con 0x7fd4180d03e0 2024-09-10T22:09:16.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.713+0000 7fd41de00640 1 -- 172.21.15.66:0/1224552135 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4180c8780 con 0x7fd4180d03e0 2024-09-10T22:09:16.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.713+0000 7fd40f400640 1 -- 172.21.15.66:0/1224552135 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4080ad440 con 0x7fd4180d03e0 2024-09-10T22:09:16.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.713+0000 7fd41de00640 1 -- 172.21.15.66:0/1224552135 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd418155c80 con 0x7fd4180d03e0 2024-09-10T22:09:16.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.713+0000 7fd41de00640 1 -- 172.21.15.66:0/1224552135 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4180d0820 con 0x7fd4180d03e0 2024-09-10T22:09:16.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.717+0000 7fd40f400640 1 -- 172.21.15.66:0/1224552135 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7fd4080b5070 con 0x7fd4180d03e0 2024-09-10T22:09:16.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.721+0000 7fd40f400640 1 --2- 172.21.15.66:0/1224552135 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3f8073ab0 0x7fd3f8075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:16.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.721+0000 7fd40f400640 1 -- 172.21.15.66:0/1224552135 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fd40812cba0 con 0x7fd4180d03e0 2024-09-10T22:09:16.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.721+0000 7fd416000640 1 --2- 172.21.15.66:0/1224552135 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3f8073ab0 0x7fd3f8075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:16.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.725+0000 7fd416000640 1 --2- 172.21.15.66:0/1224552135 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3f8073ab0 0x7fd3f8075f70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd4180b7c10 tx=0x7fd410006ca0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:16.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:16.729+0000 7fd40f400640 1 -- 172.21.15.66:0/1224552135 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd408016990 con 0x7fd4180d03e0 2024-09-10T22:09:17.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:17.101+0000 7fd41de00640 1 -- 172.21.15.66:0/1224552135 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone403911", "target": ["mon-mgr", ""]}) -- 0x7fd4180cbef0 con 0x7fd3f8073ab0 2024-09-10T22:09:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.469+0000 7fd40f400640 1 -- 172.21.15.66:0/1224552135 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd4180cbef0 con 0x7fd3f8073ab0 2024-09-10T22:09:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.469+0000 7fd41de00640 1 -- 172.21.15.66:0/1224552135 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3f8073ab0 msgr2=0x7fd3f8075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.469+0000 7fd41de00640 1 --2- 172.21.15.66:0/1224552135 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3f8073ab0 0x7fd3f8075f70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd4180b7c10 tx=0x7fd410006ca0 comp rx=0 tx=0).stop 2024-09-10T22:09:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.469+0000 7fd41de00640 1 -- 172.21.15.66:0/1224552135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4180d03e0 msgr2=0x7fd418154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.469+0000 7fd41de00640 1 --2- 172.21.15.66:0/1224552135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4180d03e0 0x7fd418154cd0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd408000f90 tx=0x7fd4080042f0 comp rx=0 tx=0).stop 2024-09-10T22:09:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.469+0000 7fd41de00640 1 -- 172.21.15.66:0/1224552135 shutdown_connections 2024-09-10T22:09:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.469+0000 7fd41de00640 1 --2- 172.21.15.66:0/1224552135 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd3f8073ab0 0x7fd3f8075f70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.469+0000 7fd41de00640 1 --2- 172.21.15.66:0/1224552135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41815e010 0x7fd4180b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.469+0000 7fd41de00640 1 --2- 172.21.15.66:0/1224552135 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd4180d0d60 0x7fd4180c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.469+0000 7fd41de00640 1 --2- 172.21.15.66:0/1224552135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4180d03e0 0x7fd418154cd0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.469+0000 7fd41de00640 1 -- 172.21.15.66:0/1224552135 >> 172.21.15.66:0/1224552135 conn(0x7fd4180d1c60 msgr2=0x7fd4180b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:18.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.473+0000 7fd41de00640 1 -- 172.21.15.66:0/1224552135 shutdown_connections 2024-09-10T22:09:18.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.473+0000 7fd41de00640 1 -- 172.21.15.66:0/1224552135 wait complete. 2024-09-10T22:09:18.511 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone82612 2024-09-10T22:09:18.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.769+0000 7fd00d200640 1 -- 172.21.15.66:0/2122856807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfec005e60 msgr2=0x7fcfec0062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:18.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.769+0000 7fd00d200640 1 --2- 172.21.15.66:0/2122856807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfec005e60 0x7fcfec0062e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009b40 tx=0x7fcff801d290 comp rx=0 tx=0).stop 2024-09-10T22:09:18.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.769+0000 7fd006800640 1 -- 172.21.15.66:0/2122856807 reap_dead start 2024-09-10T22:09:18.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.769+0000 7fd00d200640 1 -- 172.21.15.66:0/2122856807 shutdown_connections 2024-09-10T22:09:18.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.769+0000 7fd00d200640 1 -- 172.21.15.66:0/2122856807 >> 172.21.15.66:0/2122856807 conn(0x7fd0080cfa30 msgr2=0x7fd0080cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:18.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.769+0000 7fd00d200640 1 -- 172.21.15.66:0/2122856807 shutdown_connections 2024-09-10T22:09:18.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.769+0000 7fd00d200640 1 -- 172.21.15.66:0/2122856807 wait complete. 2024-09-10T22:09:18.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.773+0000 7fd00d200640 1 Processor -- start 2024-09-10T22:09:18.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.773+0000 7fd00d200640 1 -- start start 2024-09-10T22:09:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.773+0000 7fd00d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfec005e60 0x7fd00813f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.773+0000 7fd00d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0081f30c0 0x7fd0081586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:18.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.773+0000 7fd00d200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0081f5c60 0x7fd0081f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:18.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.773+0000 7fd00d200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd00806e9c0 con 0x7fcfec005e60 2024-09-10T22:09:18.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.773+0000 7fd00d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd00806e840 con 0x7fd0081f30c0 2024-09-10T22:09:18.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.773+0000 7fd00d200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd00815dd50 con 0x7fd0081f5c60 2024-09-10T22:09:18.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.773+0000 7fd005e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfec005e60 0x7fd00813f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:18.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.773+0000 7fd005e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfec005e60 0x7fd00813f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37326/0 (socket says 172.21.15.66:37326) 2024-09-10T22:09:18.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.773+0000 7fd005e00640 1 -- 172.21.15.66:0/2396471114 learned_addr learned my addr 172.21.15.66:0/2396471114 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:18.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.777+0000 7fd005e00640 1 -- 172.21.15.66:0/2396471114 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0081f5c60 msgr2=0x7fd0081f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:18.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.777+0000 7fd005e00640 1 --2- 172.21.15.66:0/2396471114 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0081f5c60 0x7fd0081f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:18.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.777+0000 7fd005e00640 1 -- 172.21.15.66:0/2396471114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0081f30c0 msgr2=0x7fd0081586b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:09:18.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.777+0000 7fd005e00640 1 --2- 172.21.15.66:0/2396471114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0081f30c0 0x7fd0081586b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:18.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.777+0000 7fd005e00640 1 -- 172.21.15.66:0/2396471114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0081f3540 con 0x7fcfec005e60 2024-09-10T22:09:18.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.777+0000 7fd005e00640 1 --2- 172.21.15.66:0/2396471114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfec005e60 0x7fd00813f2b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd00000caa0 tx=0x7fd00000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:18.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.777+0000 7fcffe000640 1 -- 172.21.15.66:0/2396471114 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000017070 con 0x7fcfec005e60 2024-09-10T22:09:18.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.777+0000 7fd00d200640 1 -- 172.21.15.66:0/2396471114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00813fd80 con 0x7fcfec005e60 2024-09-10T22:09:18.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.777+0000 7fd00d200640 1 -- 172.21.15.66:0/2396471114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0081e7490 con 0x7fcfec005e60 2024-09-10T22:09:18.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.777+0000 7fcffe000640 1 -- 172.21.15.66:0/2396471114 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd00000a730 con 0x7fcfec005e60 2024-09-10T22:09:18.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.777+0000 7fcffe000640 1 -- 172.21.15.66:0/2396471114 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00000ab20 con 0x7fcfec005e60 2024-09-10T22:09:18.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.777+0000 7fd00d200640 1 -- 172.21.15.66:0/2396471114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe00052a0 con 0x7fcfec005e60 2024-09-10T22:09:18.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.781+0000 7fcffe000640 1 -- 172.21.15.66:0/2396471114 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7fd00000acc0 con 0x7fcfec005e60 2024-09-10T22:09:18.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.785+0000 7fcffe000640 1 --2- 172.21.15.66:0/2396471114 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcfe8073ae0 0x7fcfe8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:18.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.785+0000 7fcffe000640 1 -- 172.21.15.66:0/2396471114 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fd000091d80 con 0x7fcfec005e60 2024-09-10T22:09:18.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.789+0000 7fcffe000640 1 -- 172.21.15.66:0/2396471114 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd00005e8c0 con 0x7fcfec005e60 2024-09-10T22:09:18.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.797+0000 7fd005400640 1 --2- 172.21.15.66:0/2396471114 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcfe8073ae0 0x7fcfe8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:18.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:18.801+0000 7fd005400640 1 --2- 172.21.15.66:0/2396471114 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcfe8073ae0 0x7fcfe8075fa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd0080b5a20 tx=0x7fcff40095a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:19.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:19.109+0000 7fd00d200640 1 -- 172.21.15.66:0/2396471114 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone82612", "target": ["mon-mgr", ""]}) -- 0x7fcfe0000c90 con 0x7fcfe8073ae0 2024-09-10T22:09:20.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.389+0000 7fcffe000640 1 -- 172.21.15.66:0/2396471114 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcfe0000c90 con 0x7fcfe8073ae0 2024-09-10T22:09:20.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.397+0000 7fd00d200640 1 -- 172.21.15.66:0/2396471114 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcfe8073ae0 msgr2=0x7fcfe8075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:20.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.397+0000 7fd00d200640 1 --2- 172.21.15.66:0/2396471114 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcfe8073ae0 0x7fcfe8075fa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd0080b5a20 tx=0x7fcff40095a0 comp rx=0 tx=0).stop 2024-09-10T22:09:20.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.397+0000 7fd00d200640 1 -- 172.21.15.66:0/2396471114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfec005e60 msgr2=0x7fd00813f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:20.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.397+0000 7fd00d200640 1 --2- 172.21.15.66:0/2396471114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfec005e60 0x7fd00813f2b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd00000caa0 tx=0x7fd00000cf60 comp rx=0 tx=0).stop 2024-09-10T22:09:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.397+0000 7fd00d200640 1 -- 172.21.15.66:0/2396471114 shutdown_connections 2024-09-10T22:09:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.397+0000 7fd00d200640 1 --2- 172.21.15.66:0/2396471114 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcfe8073ae0 0x7fcfe8075fa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.397+0000 7fd00d200640 1 --2- 172.21.15.66:0/2396471114 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0081f5c60 0x7fd0081f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.397+0000 7fd00d200640 1 --2- 172.21.15.66:0/2396471114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0081f30c0 0x7fd0081586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.397+0000 7fd00d200640 1 --2- 172.21.15.66:0/2396471114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfec005e60 0x7fd00813f2b0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.397+0000 7fd00d200640 1 -- 172.21.15.66:0/2396471114 >> 172.21.15.66:0/2396471114 conn(0x7fd0080cfa30 msgr2=0x7fd008159680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.397+0000 7fd00d200640 1 -- 172.21.15.66:0/2396471114 shutdown_connections 2024-09-10T22:09:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.397+0000 7fd00d200640 1 -- 172.21.15.66:0/2396471114 wait complete. 2024-09-10T22:09:20.455 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone16013 2024-09-10T22:09:20.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.721+0000 7fa5a2a00640 1 -- 172.21.15.66:0/96049062 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa59c0d0af0 msgr2=0x7fa59c06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:20.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.721+0000 7fa5a2a00640 1 --2- 172.21.15.66:0/96049062 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa59c0d0af0 0x7fa59c06dd30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa594009bc0 tx=0x7fa59401cb30 comp rx=0 tx=0).stop 2024-09-10T22:09:20.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.721+0000 7fa5a2a00640 1 -- 172.21.15.66:0/96049062 shutdown_connections 2024-09-10T22:09:20.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.721+0000 7fa5a2a00640 1 --2- 172.21.15.66:0/96049062 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa59c0d0af0 0x7fa59c06dd30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:20.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.721+0000 7fa5a2a00640 1 --2- 172.21.15.66:0/96049062 >> v2:172.21.15.94:3300/0 conn(0x7fa59c0cebc0 0x7fa59c0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:20.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.721+0000 7fa5a2a00640 1 -- 172.21.15.66:0/96049062 >> 172.21.15.66:0/96049062 conn(0x7fa59c0d1c60 msgr2=0x7fa59c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:20.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.721+0000 7fa5a2a00640 1 -- 172.21.15.66:0/96049062 shutdown_connections 2024-09-10T22:09:20.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.721+0000 7fa5a2a00640 1 -- 172.21.15.66:0/96049062 wait complete. 2024-09-10T22:09:20.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.721+0000 7fa5a2a00640 1 Processor -- start 2024-09-10T22:09:20.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.721+0000 7fa5a2a00640 1 -- start start 2024-09-10T22:09:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa5a2a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa59c0cebc0 0x7fa59c1511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa5a2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa59c0d0af0 0x7fa59c155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa5a2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa59c15e010 0x7fa59c156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa5a2a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c16acc0 con 0x7fa59c15e010 2024-09-10T22:09:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa5a2a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c06f040 con 0x7fa59c0d0af0 2024-09-10T22:09:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa5a2a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa59c16ae40 con 0x7fa59c0cebc0 2024-09-10T22:09:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa5a0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa59c15e010 0x7fa59c156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa5a0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa59c15e010 0x7fa59c156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37330/0 (socket says 172.21.15.66:37330) 2024-09-10T22:09:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa5a0a00640 1 -- 172.21.15.66:0/474951959 learned_addr learned my addr 172.21.15.66:0/474951959 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa59b400640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa59c0d0af0 0x7fa59c155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa59be00640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa59c0cebc0 0x7fa59c1511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa59b400640 1 -- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa59c0cebc0 msgr2=0x7fa59c1511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa59b400640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa59c0cebc0 0x7fa59c1511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:20.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa59b400640 1 -- 172.21.15.66:0/474951959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa59c15e010 msgr2=0x7fa59c156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:20.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa59b400640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa59c15e010 0x7fa59c156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:20.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa59b400640 1 -- 172.21.15.66:0/474951959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c1f9b50 con 0x7fa59c0d0af0 2024-09-10T22:09:20.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa5a0a00640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa59c15e010 0x7fa59c156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:09:20.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa59b400640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa59c0d0af0 0x7fa59c155f50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00ea20 tx=0x7fa58c00eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:20.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa593e00640 1 -- 172.21.15.66:0/474951959 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00cd80 con 0x7fa59c0d0af0 2024-09-10T22:09:20.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa593e00640 1 -- 172.21.15.66:0/474951959 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa58c010430 con 0x7fa59c0d0af0 2024-09-10T22:09:20.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa593e00640 1 -- 172.21.15.66:0/474951959 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c010860 con 0x7fa59c0d0af0 2024-09-10T22:09:20.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa5a2a00640 1 -- 172.21.15.66:0/474951959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59c1f9e30 con 0x7fa59c0d0af0 2024-09-10T22:09:20.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.725+0000 7fa5a2a00640 1 -- 172.21.15.66:0/474951959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59c1fa3f0 con 0x7fa59c0d0af0 2024-09-10T22:09:20.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.729+0000 7fa5a2a00640 1 -- 172.21.15.66:0/474951959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59c1fa7d0 con 0x7fa59c0d0af0 2024-09-10T22:09:20.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.729+0000 7fa593e00640 1 -- 172.21.15.66:0/474951959 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7fa58c010a00 con 0x7fa59c0d0af0 2024-09-10T22:09:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.729+0000 7fa593e00640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa57c073ab0 0x7fa57c075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.729+0000 7fa593e00640 1 -- 172.21.15.66:0/474951959 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fa58c092e40 con 0x7fa59c0d0af0 2024-09-10T22:09:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.729+0000 7fa59be00640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa57c073ab0 0x7fa57c075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.729+0000 7fa59be00640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa57c073ab0 0x7fa57c075f70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0b7bf0 tx=0x7fa588009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:20.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:20.737+0000 7fa593e00640 1 -- 172.21.15.66:0/474951959 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa59c1fa7d0 con 0x7fa59c0d0af0 2024-09-10T22:09:21.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:21.093+0000 7fa5a2a00640 1 -- 172.21.15.66:0/474951959 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone16013", "target": ["mon-mgr", ""]}) -- 0x7fa59c06dd30 con 0x7fa57c073ab0 2024-09-10T22:09:22.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.145+0000 7fa593e00640 1 -- 172.21.15.66:0/474951959 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa59c06dd30 con 0x7fa57c073ab0 2024-09-10T22:09:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.149+0000 7fa5a2a00640 1 -- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa57c073ab0 msgr2=0x7fa57c075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.149+0000 7fa5a2a00640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa57c073ab0 0x7fa57c075f70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0b7bf0 tx=0x7fa588009040 comp rx=0 tx=0).stop 2024-09-10T22:09:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.149+0000 7fa5a2a00640 1 -- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa59c0d0af0 msgr2=0x7fa59c155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.149+0000 7fa5a2a00640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa59c0d0af0 0x7fa59c155f50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00ea20 tx=0x7fa58c00eee0 comp rx=0 tx=0).stop 2024-09-10T22:09:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.149+0000 7fa5a2a00640 1 -- 172.21.15.66:0/474951959 shutdown_connections 2024-09-10T22:09:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.149+0000 7fa5a2a00640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa57c073ab0 0x7fa57c075f70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.149+0000 7fa5a2a00640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa59c15e010 0x7fa59c156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.149+0000 7fa5a2a00640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa59c0d0af0 0x7fa59c155f50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.149+0000 7fa5a2a00640 1 --2- 172.21.15.66:0/474951959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa59c0cebc0 0x7fa59c1511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.149+0000 7fa5a2a00640 1 -- 172.21.15.66:0/474951959 >> 172.21.15.66:0/474951959 conn(0x7fa59c0d1c60 msgr2=0x7fa59c071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.149+0000 7fa5a2a00640 1 -- 172.21.15.66:0/474951959 shutdown_connections 2024-09-10T22:09:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.149+0000 7fa5a2a00640 1 -- 172.21.15.66:0/474951959 wait complete. 2024-09-10T22:09:22.197 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone344114 2024-09-10T22:09:22.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.457+0000 7f4e81800640 1 -- 172.21.15.66:0/232877869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60004010 msgr2=0x7f4e60004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:22.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.457+0000 7f4e81800640 1 --2- 172.21.15.66:0/232877869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60004010 0x7f4e60004810 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c2e0 tx=0x7f4e6801f6e0 comp rx=0 tx=0).stop 2024-09-10T22:09:22.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.457+0000 7f4e80e00640 1 -- 172.21.15.66:0/232877869 reap_dead start 2024-09-10T22:09:22.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.457+0000 7f4e81800640 1 -- 172.21.15.66:0/232877869 shutdown_connections 2024-09-10T22:09:22.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.457+0000 7f4e81800640 1 -- 172.21.15.66:0/232877869 >> 172.21.15.66:0/232877869 conn(0x7f4e7c0cc3c0 msgr2=0x7f4e7c0cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:22.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.457+0000 7f4e81800640 1 -- 172.21.15.66:0/232877869 shutdown_connections 2024-09-10T22:09:22.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.457+0000 7f4e81800640 1 -- 172.21.15.66:0/232877869 wait complete. 2024-09-10T22:09:22.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.457+0000 7f4e81800640 1 Processor -- start 2024-09-10T22:09:22.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.457+0000 7f4e81800640 1 -- start start 2024-09-10T22:09:22.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e81800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60004010 0x7f4e7c1405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:22.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e81800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e7c142630 0x7f4e7c1f9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:22.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e81800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e7c1f99d0 0x7f4e7c159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:22.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e81800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c074360 con 0x7f4e60004010 2024-09-10T22:09:22.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e81800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c0741e0 con 0x7f4e7c142630 2024-09-10T22:09:22.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e81800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7c0744e0 con 0x7f4e7c1f99d0 2024-09-10T22:09:22.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e7be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60004010 0x7f4e7c1405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:22.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e7be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60004010 0x7f4e7c1405d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51862/0 (socket says 172.21.15.66:51862) 2024-09-10T22:09:22.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e7be00640 1 -- 172.21.15.66:0/891138181 learned_addr learned my addr 172.21.15.66:0/891138181 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:22.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e7be00640 1 -- 172.21.15.66:0/891138181 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e7c1f99d0 msgr2=0x7f4e7c159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:22.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e7be00640 1 --2- 172.21.15.66:0/891138181 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e7c1f99d0 0x7f4e7c159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:22.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e7be00640 1 -- 172.21.15.66:0/891138181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e7c142630 msgr2=0x7f4e7c1f9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:22.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e7be00640 1 --2- 172.21.15.66:0/891138181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e7c142630 0x7f4e7c1f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:22.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e7be00640 1 -- 172.21.15.66:0/891138181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c140b10 con 0x7f4e60004010 2024-09-10T22:09:22.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e7be00640 1 --2- 172.21.15.66:0/891138181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60004010 0x7f4e7c1405d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00c4d0 tx=0x7f4e6c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:22.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e73e00640 1 -- 172.21.15.66:0/891138181 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0052d0 con 0x7f4e60004010 2024-09-10T22:09:22.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e73e00640 1 -- 172.21.15.66:0/891138181 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e6c005470 con 0x7f4e60004010 2024-09-10T22:09:22.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e81800640 1 -- 172.21.15.66:0/891138181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c142af0 con 0x7f4e60004010 2024-09-10T22:09:22.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e73e00640 1 -- 172.21.15.66:0/891138181 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c012890 con 0x7f4e60004010 2024-09-10T22:09:22.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e81800640 1 -- 172.21.15.66:0/891138181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c1430b0 con 0x7f4e60004010 2024-09-10T22:09:22.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.461+0000 7f4e81800640 1 -- 172.21.15.66:0/891138181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e540052a0 con 0x7f4e60004010 2024-09-10T22:09:22.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.465+0000 7f4e73e00640 1 -- 172.21.15.66:0/891138181 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7f4e6c003600 con 0x7f4e60004010 2024-09-10T22:09:22.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.465+0000 7f4e73e00640 1 --2- 172.21.15.66:0/891138181 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4e5c073ab0 0x7f4e5c075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:22.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.465+0000 7f4e73e00640 1 -- 172.21.15.66:0/891138181 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f4e6c0928f0 con 0x7f4e60004010 2024-09-10T22:09:22.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.473+0000 7f4e73e00640 1 -- 172.21.15.66:0/891138181 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4e6c05f640 con 0x7f4e60004010 2024-09-10T22:09:22.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.473+0000 7f4e7b400640 1 --2- 172.21.15.66:0/891138181 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4e5c073ab0 0x7f4e5c075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:22.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.477+0000 7f4e7b400640 1 --2- 172.21.15.66:0/891138181 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4e5c073ab0 0x7f4e5c075f70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c2e0 tx=0x7f4e60005840 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:22.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:22.793+0000 7f4e81800640 1 -- 172.21.15.66:0/891138181 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone344114", "target": ["mon-mgr", ""]}) -- 0x7f4e54000c90 con 0x7f4e5c073ab0 2024-09-10T22:09:23.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.557+0000 7f4e73e00640 1 -- 172.21.15.66:0/891138181 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e54000c90 con 0x7f4e5c073ab0 2024-09-10T22:09:23.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.561+0000 7f4e81800640 1 -- 172.21.15.66:0/891138181 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4e5c073ab0 msgr2=0x7f4e5c075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:23.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.561+0000 7f4e81800640 1 --2- 172.21.15.66:0/891138181 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4e5c073ab0 0x7f4e5c075f70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c2e0 tx=0x7f4e60005840 comp rx=0 tx=0).stop 2024-09-10T22:09:23.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.561+0000 7f4e81800640 1 -- 172.21.15.66:0/891138181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60004010 msgr2=0x7f4e7c1405d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:23.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.561+0000 7f4e81800640 1 --2- 172.21.15.66:0/891138181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60004010 0x7f4e7c1405d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00c4d0 tx=0x7f4e6c00c990 comp rx=0 tx=0).stop 2024-09-10T22:09:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.565+0000 7f4e81800640 1 -- 172.21.15.66:0/891138181 shutdown_connections 2024-09-10T22:09:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.565+0000 7f4e81800640 1 --2- 172.21.15.66:0/891138181 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4e5c073ab0 0x7f4e5c075f70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.565+0000 7f4e81800640 1 --2- 172.21.15.66:0/891138181 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e7c1f99d0 0x7f4e7c159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.565+0000 7f4e81800640 1 --2- 172.21.15.66:0/891138181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e7c142630 0x7f4e7c1f9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.565+0000 7f4e81800640 1 --2- 172.21.15.66:0/891138181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e60004010 0x7f4e7c1405d0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.565+0000 7f4e81800640 1 -- 172.21.15.66:0/891138181 >> 172.21.15.66:0/891138181 conn(0x7f4e7c0cc3c0 msgr2=0x7f4e7c0ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.565+0000 7f4e81800640 1 -- 172.21.15.66:0/891138181 shutdown_connections 2024-09-10T22:09:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.565+0000 7f4e81800640 1 -- 172.21.15.66:0/891138181 wait complete. 2024-09-10T22:09:23.624 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8415 2024-09-10T22:09:23.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.877+0000 7fba08600640 1 -- 172.21.15.66:0/3736195748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba0006e120 msgr2=0x7fba0006e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:23.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.877+0000 7fba08600640 1 --2- 172.21.15.66:0/3736195748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba0006e120 0x7fba0006e500 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fba0006d9c0 tx=0x7fb9f001c740 comp rx=0 tx=0).stop 2024-09-10T22:09:23.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.881+0000 7fba08600640 1 -- 172.21.15.66:0/3736195748 shutdown_connections 2024-09-10T22:09:23.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.881+0000 7fba08600640 1 --2- 172.21.15.66:0/3736195748 >> v2:172.21.15.94:3300/0 conn(0x7fba00161c40 0x7fba001620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:23.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.881+0000 7fba08600640 1 --2- 172.21.15.66:0/3736195748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba000b2c40 0x7fba0006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:23.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.881+0000 7fba08600640 1 --2- 172.21.15.66:0/3736195748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba0006e120 0x7fba0006e500 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:23.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.881+0000 7fba08600640 1 -- 172.21.15.66:0/3736195748 >> 172.21.15.66:0/3736195748 conn(0x7fba000d1d10 msgr2=0x7fba000d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:23.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.881+0000 7fba08600640 1 -- 172.21.15.66:0/3736195748 shutdown_connections 2024-09-10T22:09:23.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.881+0000 7fba08600640 1 -- 172.21.15.66:0/3736195748 wait complete. 2024-09-10T22:09:23.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.885+0000 7fba08600640 1 Processor -- start 2024-09-10T22:09:23.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.885+0000 7fba08600640 1 -- start start 2024-09-10T22:09:23.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba08600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba0006e120 0x7fba00160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:23.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba08600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba000b2c40 0x7fba00160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:23.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba08600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba00161c40 0x7fba00208140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:23.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba08600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fba0015d840 con 0x7fba000b2c40 2024-09-10T22:09:23.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba08600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba0015d6c0 con 0x7fba00161c40 2024-09-10T22:09:23.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba08600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fba0015d9c0 con 0x7fba0006e120 2024-09-10T22:09:23.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba05200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba000b2c40 0x7fba00160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:23.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba05200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba000b2c40 0x7fba00160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51868/0 (socket says 172.21.15.66:51868) 2024-09-10T22:09:23.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba05200640 1 -- 172.21.15.66:0/3617301352 learned_addr learned my addr 172.21.15.66:0/3617301352 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:23.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba05200640 1 -- 172.21.15.66:0/3617301352 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba0006e120 msgr2=0x7fba00160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:23.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba05200640 1 --2- 172.21.15.66:0/3617301352 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba0006e120 0x7fba00160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:23.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba05200640 1 -- 172.21.15.66:0/3617301352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba00161c40 msgr2=0x7fba00208140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:23.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba05200640 1 --2- 172.21.15.66:0/3617301352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba00161c40 0x7fba00208140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:23.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.889+0000 7fba05200640 1 -- 172.21.15.66:0/3617301352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0015dcd0 con 0x7fba000b2c40 2024-09-10T22:09:23.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.893+0000 7fba05200640 1 --2- 172.21.15.66:0/3617301352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba000b2c40 0x7fba00160790 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00e9f0 tx=0x7fb9ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:23.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.893+0000 7fb9fd600640 1 -- 172.21.15.66:0/3617301352 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec00ccc0 con 0x7fba000b2c40 2024-09-10T22:09:23.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.893+0000 7fb9fd600640 1 -- 172.21.15.66:0/3617301352 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9ec00ce60 con 0x7fba000b2c40 2024-09-10T22:09:23.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.893+0000 7fba08600640 1 -- 172.21.15.66:0/3617301352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba000b5fc0 con 0x7fba000b2c40 2024-09-10T22:09:23.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.893+0000 7fb9fd600640 1 -- 172.21.15.66:0/3617301352 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec005280 con 0x7fba000b2c40 2024-09-10T22:09:23.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.893+0000 7fba08600640 1 -- 172.21.15.66:0/3617301352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba000b6580 con 0x7fba000b2c40 2024-09-10T22:09:23.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.897+0000 7fba08600640 1 -- 172.21.15.66:0/3617301352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba001620d0 con 0x7fba000b2c40 2024-09-10T22:09:23.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.897+0000 7fb9fd600640 1 -- 172.21.15.66:0/3617301352 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7fb9ec005420 con 0x7fba000b2c40 2024-09-10T22:09:23.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.897+0000 7fb9fd600640 1 --2- 172.21.15.66:0/3617301352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9e0073ab0 0x7fb9e0075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:23.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.901+0000 7fb9fd600640 1 -- 172.21.15.66:0/3617301352 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fb9ec092e70 con 0x7fba000b2c40 2024-09-10T22:09:23.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.901+0000 7fba05c00640 1 --2- 172.21.15.66:0/3617301352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9e0073ab0 0x7fb9e0075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:23.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.905+0000 7fb9fd600640 1 -- 172.21.15.66:0/3617301352 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb9ec05fbc0 con 0x7fba000b2c40 2024-09-10T22:09:23.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:23.905+0000 7fba05c00640 1 --2- 172.21.15.66:0/3617301352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9e0073ab0 0x7fb9e0075f70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00af000 tx=0x7fb9f00194e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:24.337+0000 7fba08600640 1 -- 172.21.15.66:0/3617301352 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8415", "target": ["mon-mgr", ""]}) -- 0x7fba000726d0 con 0x7fb9e0073ab0 2024-09-10T22:09:25.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fb9fd600640 1 -- 172.21.15.66:0/3617301352 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fba000726d0 con 0x7fb9e0073ab0 2024-09-10T22:09:25.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fba08600640 1 -- 172.21.15.66:0/3617301352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9e0073ab0 msgr2=0x7fb9e0075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:25.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fba08600640 1 --2- 172.21.15.66:0/3617301352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9e0073ab0 0x7fb9e0075f70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00af000 tx=0x7fb9f00194e0 comp rx=0 tx=0).stop 2024-09-10T22:09:25.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fba08600640 1 -- 172.21.15.66:0/3617301352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba000b2c40 msgr2=0x7fba00160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:25.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fba08600640 1 --2- 172.21.15.66:0/3617301352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba000b2c40 0x7fba00160790 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00e9f0 tx=0x7fb9ec00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:09:25.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fba08600640 1 -- 172.21.15.66:0/3617301352 shutdown_connections 2024-09-10T22:09:25.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fba08600640 1 --2- 172.21.15.66:0/3617301352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9e0073ab0 0x7fb9e0075f70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:25.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fba08600640 1 --2- 172.21.15.66:0/3617301352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba00161c40 0x7fba00208140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:25.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fba08600640 1 --2- 172.21.15.66:0/3617301352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba000b2c40 0x7fba00160790 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:25.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fba08600640 1 --2- 172.21.15.66:0/3617301352 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba0006e120 0x7fba00160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:25.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fba08600640 1 -- 172.21.15.66:0/3617301352 >> 172.21.15.66:0/3617301352 conn(0x7fba000d1d10 msgr2=0x7fba000af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:25.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fba08600640 1 -- 172.21.15.66:0/3617301352 shutdown_connections 2024-09-10T22:09:25.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.145+0000 7fba08600640 1 -- 172.21.15.66:0/3617301352 wait complete. 2024-09-10T22:09:25.199 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone302916 2024-09-10T22:09:25.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.493+0000 7f8431600640 1 -- 172.21.15.66:0/637177913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c15acd0 msgr2=0x7f842c0d0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:25.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.493+0000 7f8431600640 1 --2- 172.21.15.66:0/637177913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c15acd0 0x7f842c0d0550 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f841c009900 tx=0x7f841c01c8e0 comp rx=0 tx=0).stop 2024-09-10T22:09:25.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.493+0000 7f8431600640 1 -- 172.21.15.66:0/637177913 shutdown_connections 2024-09-10T22:09:25.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.493+0000 7f8431600640 1 --2- 172.21.15.66:0/637177913 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f842c0cbcd0 0x7f842c0c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:25.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.493+0000 7f8431600640 1 --2- 172.21.15.66:0/637177913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c15acd0 0x7f842c0d0550 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:25.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.493+0000 7f8431600640 1 -- 172.21.15.66:0/637177913 >> 172.21.15.66:0/637177913 conn(0x7f842c0cfa30 msgr2=0x7f842c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:25.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.493+0000 7f8431600640 1 -- 172.21.15.66:0/637177913 shutdown_connections 2024-09-10T22:09:25.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.493+0000 7f8431600640 1 -- 172.21.15.66:0/637177913 wait complete. 2024-09-10T22:09:25.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f8431600640 1 Processor -- start 2024-09-10T22:09:25.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f8431600640 1 -- start start 2024-09-10T22:09:25.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f8431600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f842c0cf500 0x7f842c156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:25.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f8431600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f842c15acd0 0x7f842c166120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:25.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f8431600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c1fc450 0x7f842c1fb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:25.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f8431600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f842c06f5c0 con 0x7f842c1fc450 2024-09-10T22:09:25.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f8431600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f842c06f440 con 0x7f842c0cf500 2024-09-10T22:09:25.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f8431600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f842c06f740 con 0x7f842c15acd0 2024-09-10T22:09:25.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f842b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c1fc450 0x7f842c1fb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:25.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f842b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c1fc450 0x7f842c1fb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51874/0 (socket says 172.21.15.66:51874) 2024-09-10T22:09:25.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f842b400640 1 -- 172.21.15.66:0/3681413862 learned_addr learned my addr 172.21.15.66:0/3681413862 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:25.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f842b400640 1 -- 172.21.15.66:0/3681413862 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f842c15acd0 msgr2=0x7f842c166120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:25.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f842b400640 1 --2- 172.21.15.66:0/3681413862 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f842c15acd0 0x7f842c166120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:25.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f842b400640 1 -- 172.21.15.66:0/3681413862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f842c0cf500 msgr2=0x7f842c156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:25.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f842b400640 1 --2- 172.21.15.66:0/3681413862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f842c0cf500 0x7f842c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:25.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.497+0000 7f842b400640 1 -- 172.21.15.66:0/3681413862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c164480 con 0x7f842c1fc450 2024-09-10T22:09:25.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.501+0000 7f842b400640 1 --2- 172.21.15.66:0/3681413862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c1fc450 0x7f842c1fb840 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f842400c7b0 tx=0x7f842400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:25.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.501+0000 7f8422a00640 1 -- 172.21.15.66:0/3681413862 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424017070 con 0x7f842c1fc450 2024-09-10T22:09:25.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.501+0000 7f8431600640 1 -- 172.21.15.66:0/3681413862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842c164760 con 0x7f842c1fc450 2024-09-10T22:09:25.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.501+0000 7f8422a00640 1 -- 172.21.15.66:0/3681413862 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8424005070 con 0x7f842c1fc450 2024-09-10T22:09:25.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.501+0000 7f8422a00640 1 -- 172.21.15.66:0/3681413862 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842400ce60 con 0x7f842c1fc450 2024-09-10T22:09:25.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.501+0000 7f8431600640 1 -- 172.21.15.66:0/3681413862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842c0725b0 con 0x7f842c1fc450 2024-09-10T22:09:25.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.501+0000 7f8431600640 1 -- 172.21.15.66:0/3681413862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f842c06fd20 con 0x7f842c1fc450 2024-09-10T22:09:25.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.505+0000 7f8422a00640 1 -- 172.21.15.66:0/3681413862 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7f84240054c0 con 0x7f842c1fc450 2024-09-10T22:09:25.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.505+0000 7f8422a00640 1 --2- 172.21.15.66:0/3681413862 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f840c073ab0 0x7f840c075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:25.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.505+0000 7f8422a00640 1 -- 172.21.15.66:0/3681413862 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f8424091d30 con 0x7f842c1fc450 2024-09-10T22:09:25.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.517+0000 7f8422a00640 1 -- 172.21.15.66:0/3681413862 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8424021020 con 0x7f842c1fc450 2024-09-10T22:09:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.545+0000 7f842aa00640 1 --2- 172.21.15.66:0/3681413862 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f840c073ab0 0x7f840c075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:25.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.553+0000 7f842aa00640 1 --2- 172.21.15.66:0/3681413862 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f840c073ab0 0x7f840c075f70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f841c0af000 tx=0x7f841c019680 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:25.889+0000 7f8431600640 1 -- 172.21.15.66:0/3681413862 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone302916", "target": ["mon-mgr", ""]}) -- 0x7f842c075130 con 0x7f840c073ab0 2024-09-10T22:09:26.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.677+0000 7f8422a00640 1 -- 172.21.15.66:0/3681413862 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f842c075130 con 0x7f840c073ab0 2024-09-10T22:09:26.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.677+0000 7f8431600640 1 -- 172.21.15.66:0/3681413862 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f840c073ab0 msgr2=0x7f840c075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:26.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.677+0000 7f8431600640 1 --2- 172.21.15.66:0/3681413862 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f840c073ab0 0x7f840c075f70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f841c0af000 tx=0x7f841c019680 comp rx=0 tx=0).stop 2024-09-10T22:09:26.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.677+0000 7f8431600640 1 -- 172.21.15.66:0/3681413862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c1fc450 msgr2=0x7f842c1fb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:26.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.677+0000 7f8431600640 1 --2- 172.21.15.66:0/3681413862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c1fc450 0x7f842c1fb840 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f842400c7b0 tx=0x7f842400cc70 comp rx=0 tx=0).stop 2024-09-10T22:09:26.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.681+0000 7f8431600640 1 -- 172.21.15.66:0/3681413862 shutdown_connections 2024-09-10T22:09:26.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.681+0000 7f8431600640 1 --2- 172.21.15.66:0/3681413862 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f840c073ab0 0x7f840c075f70 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:26.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.681+0000 7f8431600640 1 --2- 172.21.15.66:0/3681413862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c1fc450 0x7f842c1fb840 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:26.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.681+0000 7f8431600640 1 --2- 172.21.15.66:0/3681413862 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f842c15acd0 0x7f842c166120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:26.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.681+0000 7f8431600640 1 --2- 172.21.15.66:0/3681413862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f842c0cf500 0x7f842c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:26.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.681+0000 7f8431600640 1 -- 172.21.15.66:0/3681413862 >> 172.21.15.66:0/3681413862 conn(0x7f842c0cfa30 msgr2=0x7f842c0b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:26.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.681+0000 7f8431600640 1 -- 172.21.15.66:0/3681413862 shutdown_connections 2024-09-10T22:09:26.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.681+0000 7f8431600640 1 -- 172.21.15.66:0/3681413862 wait complete. 2024-09-10T22:09:26.729 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone885517 2024-09-10T22:09:26.900 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:09:26.905 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:09:26.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.993+0000 7fef03400640 1 -- 172.21.15.66:0/1418915034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feefc0c4fc0 msgr2=0x7feefc06e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:26.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.993+0000 7fef03400640 1 --2- 172.21.15.66:0/1418915034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feefc0c4fc0 0x7feefc06e630 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7feefc0c5f10 tx=0x7feee801e620 comp rx=0 tx=0).stop 2024-09-10T22:09:26.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.993+0000 7fef03400640 1 -- 172.21.15.66:0/1418915034 shutdown_connections 2024-09-10T22:09:26.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.993+0000 7fef03400640 1 --2- 172.21.15.66:0/1418915034 >> v2:172.21.15.94:3300/0 conn(0x7feefc06eb70 0x7feefc163c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:26.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.993+0000 7fef03400640 1 --2- 172.21.15.66:0/1418915034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feefc0c4fc0 0x7feefc06e630 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:26.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.993+0000 7fef03400640 1 -- 172.21.15.66:0/1418915034 >> 172.21.15.66:0/1418915034 conn(0x7feefc0cfa30 msgr2=0x7feefc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:27.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.993+0000 7fef03400640 1 -- 172.21.15.66:0/1418915034 shutdown_connections 2024-09-10T22:09:27.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.993+0000 7fef03400640 1 -- 172.21.15.66:0/1418915034 wait complete. 2024-09-10T22:09:27.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.993+0000 7fef03400640 1 Processor -- start 2024-09-10T22:09:27.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.993+0000 7fef03400640 1 -- start start 2024-09-10T22:09:27.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7fef03400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feefc06eb70 0x7feefc15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:27.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7fef03400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feefc0706c0 0x7feefc1f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:27.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7fef03400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feefc0c4fc0 0x7feefc137200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:27.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7fef03400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feefc166980 con 0x7feefc0706c0 2024-09-10T22:09:27.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7fef03400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feefc166800 con 0x7feefc0c4fc0 2024-09-10T22:09:27.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7fef03400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7feefc166b00 con 0x7feefc06eb70 2024-09-10T22:09:27.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7feefbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feefc0706c0 0x7feefc1f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:27.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7feefbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feefc0706c0 0x7feefc1f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51898/0 (socket says 172.21.15.66:51898) 2024-09-10T22:09:27.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7feefbe00640 1 -- 172.21.15.66:0/2504246313 learned_addr learned my addr 172.21.15.66:0/2504246313 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:27.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7feefbe00640 1 -- 172.21.15.66:0/2504246313 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feefc06eb70 msgr2=0x7feefc15cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:27.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7feefbe00640 1 --2- 172.21.15.66:0/2504246313 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feefc06eb70 0x7feefc15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:27.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7feefbe00640 1 -- 172.21.15.66:0/2504246313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feefc0c4fc0 msgr2=0x7feefc137200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:09:27.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7feefbe00640 1 --2- 172.21.15.66:0/2504246313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feefc0c4fc0 0x7feefc137200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:27.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:26.997+0000 7feefbe00640 1 -- 172.21.15.66:0/2504246313 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc15d100 con 0x7feefc0706c0 2024-09-10T22:09:27.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.001+0000 7feefbe00640 1 --2- 172.21.15.66:0/2504246313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feefc0706c0 0x7feefc1f72c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7feee80b1040 tx=0x7feee8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:27.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.001+0000 7feef8c00640 1 -- 172.21.15.66:0/2504246313 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee80b3070 con 0x7feefc0706c0 2024-09-10T22:09:27.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.001+0000 7feef8c00640 1 -- 172.21.15.66:0/2504246313 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feee8005220 con 0x7feefc0706c0 2024-09-10T22:09:27.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.001+0000 7feef8c00640 1 -- 172.21.15.66:0/2504246313 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8003810 con 0x7feefc0706c0 2024-09-10T22:09:27.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.001+0000 7fef03400640 1 -- 172.21.15.66:0/2504246313 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc15afd0 con 0x7feefc0706c0 2024-09-10T22:09:27.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.001+0000 7fef03400640 1 -- 172.21.15.66:0/2504246313 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc152910 con 0x7feefc0706c0 2024-09-10T22:09:27.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.005+0000 7feef8c00640 1 -- 172.21.15.66:0/2504246313 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7feee800b040 con 0x7feefc0706c0 2024-09-10T22:09:27.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.005+0000 7fef03400640 1 -- 172.21.15.66:0/2504246313 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feefc06e630 con 0x7feefc0706c0 2024-09-10T22:09:27.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.005+0000 7feef8c00640 1 --2- 172.21.15.66:0/2504246313 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feedc073ab0 0x7feedc075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:27.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.009+0000 7feef8c00640 1 -- 172.21.15.66:0/2504246313 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7feee812c0e0 con 0x7feefc0706c0 2024-09-10T22:09:27.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.017+0000 7fef00a00640 1 --2- 172.21.15.66:0/2504246313 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feedc073ab0 0x7feedc075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:27.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.021+0000 7feef8c00640 1 -- 172.21.15.66:0/2504246313 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feee80177e0 con 0x7feefc0706c0 2024-09-10T22:09:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.021+0000 7fef00a00640 1 --2- 172.21.15.66:0/2504246313 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feedc073ab0 0x7feedc075f70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7feeec007fd0 tx=0x7feeec0079a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:27.445+0000 7fef03400640 1 -- 172.21.15.66:0/2504246313 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone885517", "target": ["mon-mgr", ""]}) -- 0x7feefc15b480 con 0x7feedc073ab0 2024-09-10T22:09:28.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7feef8c00640 1 -- 172.21.15.66:0/2504246313 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feefc15b480 con 0x7feedc073ab0 2024-09-10T22:09:28.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7fef03400640 1 -- 172.21.15.66:0/2504246313 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feedc073ab0 msgr2=0x7feedc075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:28.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7fef03400640 1 --2- 172.21.15.66:0/2504246313 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feedc073ab0 0x7feedc075f70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7feeec007fd0 tx=0x7feeec0079a0 comp rx=0 tx=0).stop 2024-09-10T22:09:28.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7fef03400640 1 -- 172.21.15.66:0/2504246313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feefc0706c0 msgr2=0x7feefc1f72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:28.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7fef03400640 1 --2- 172.21.15.66:0/2504246313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feefc0706c0 0x7feefc1f72c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7feee80b1040 tx=0x7feee8005de0 comp rx=0 tx=0).stop 2024-09-10T22:09:28.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7fef03400640 1 -- 172.21.15.66:0/2504246313 shutdown_connections 2024-09-10T22:09:28.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7fef03400640 1 --2- 172.21.15.66:0/2504246313 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feedc073ab0 0x7feedc075f70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:28.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7fef03400640 1 --2- 172.21.15.66:0/2504246313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feefc0c4fc0 0x7feefc137200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:28.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7fef03400640 1 --2- 172.21.15.66:0/2504246313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feefc0706c0 0x7feefc1f72c0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:28.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7fef03400640 1 --2- 172.21.15.66:0/2504246313 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feefc06eb70 0x7feefc15cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:28.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7fef03400640 1 -- 172.21.15.66:0/2504246313 >> 172.21.15.66:0/2504246313 conn(0x7feefc0cfa30 msgr2=0x7feefc0b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:28.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7fef03400640 1 -- 172.21.15.66:0/2504246313 shutdown_connections 2024-09-10T22:09:28.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.597+0000 7fef03400640 1 -- 172.21.15.66:0/2504246313 wait complete. 2024-09-10T22:09:28.635 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone931018 2024-09-10T22:09:28.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.909+0000 7f17c3400640 1 -- 172.21.15.66:0/2102664752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17bc159900 msgr2=0x7f17bc0ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:28.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.909+0000 7f17c3400640 1 --2- 172.21.15.66:0/2102664752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17bc159900 0x7f17bc0ca080 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009900 tx=0x7f17b001c850 comp rx=0 tx=0).stop 2024-09-10T22:09:28.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.909+0000 7f17c3400640 1 -- 172.21.15.66:0/2102664752 shutdown_connections 2024-09-10T22:09:28.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.909+0000 7f17c3400640 1 --2- 172.21.15.66:0/2102664752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17bc159900 0x7f17bc0ca080 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:28.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.909+0000 7f17c3400640 1 -- 172.21.15.66:0/2102664752 >> 172.21.15.66:0/2102664752 conn(0x7f17bc0cfa30 msgr2=0x7f17bc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:28.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.913+0000 7f17c3400640 1 -- 172.21.15.66:0/2102664752 shutdown_connections 2024-09-10T22:09:28.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.913+0000 7f17c3400640 1 -- 172.21.15.66:0/2102664752 wait complete. 2024-09-10T22:09:28.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c3400640 1 Processor -- start 2024-09-10T22:09:28.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c3400640 1 -- start start 2024-09-10T22:09:28.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17bc0aee20 0x7f17bc149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:28.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17bc159900 0x7f17bc155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:28.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c3400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17bc15cc80 0x7f17bc15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:28.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c3400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc16a360 con 0x7f17bc0aee20 2024-09-10T22:09:28.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c3400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc16a1e0 con 0x7f17bc159900 2024-09-10T22:09:28.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c3400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f17bc16a4e0 con 0x7f17bc15cc80 2024-09-10T22:09:28.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17bc0aee20 0x7f17bc149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:28.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17bc0aee20 0x7f17bc149f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51924/0 (socket says 172.21.15.66:51924) 2024-09-10T22:09:28.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c0a00640 1 -- 172.21.15.66:0/244517606 learned_addr learned my addr 172.21.15.66:0/244517606 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:28.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17bbe00640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17bc159900 0x7f17bc155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:28.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c1400640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17bc15cc80 0x7f17bc15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:28.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c0a00640 1 -- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17bc15cc80 msgr2=0x7f17bc15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:28.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c0a00640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17bc15cc80 0x7f17bc15a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:28.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c0a00640 1 -- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17bc159900 msgr2=0x7f17bc155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:28.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c0a00640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17bc159900 0x7f17bc155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:28.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c0a00640 1 -- 172.21.15.66:0/244517606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17bc153f80 con 0x7f17bc0aee20 2024-09-10T22:09:28.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c0a00640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17bc0aee20 0x7f17bc149f80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009850 tx=0x7f17b00a65b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:28.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17b8c00640 1 -- 172.21.15.66:0/244517606 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b00ac070 con 0x7f17bc0aee20 2024-09-10T22:09:28.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17b8c00640 1 -- 172.21.15.66:0/244517606 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17b0004310 con 0x7f17bc0aee20 2024-09-10T22:09:28.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17b8c00640 1 -- 172.21.15.66:0/244517606 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b00053a0 con 0x7f17bc0aee20 2024-09-10T22:09:28.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c3400640 1 -- 172.21.15.66:0/244517606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17bc154260 con 0x7f17bc0aee20 2024-09-10T22:09:28.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.917+0000 7f17c3400640 1 -- 172.21.15.66:0/244517606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17bc154820 con 0x7f17bc0aee20 2024-09-10T22:09:28.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.921+0000 7f17c3400640 1 -- 172.21.15.66:0/244517606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17bc070470 con 0x7f17bc0aee20 2024-09-10T22:09:28.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.921+0000 7f17b8c00640 1 -- 172.21.15.66:0/244517606 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7f17bc0ce850 con 0x7f17bc0aee20 2024-09-10T22:09:28.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.925+0000 7f17b8c00640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f179c073ae0 0x7f179c075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:28.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.925+0000 7f17b8c00640 1 -- 172.21.15.66:0/244517606 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f17b012b990 con 0x7f17bc0aee20 2024-09-10T22:09:28.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.933+0000 7f17b8c00640 1 -- 172.21.15.66:0/244517606 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f17b00bf060 con 0x7f17bc0aee20 2024-09-10T22:09:28.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.937+0000 7f17bbe00640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f179c073ae0 0x7f179c075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:28.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:28.973+0000 7f17bbe00640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f179c073ae0 0x7f179c075fa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0af600 tx=0x7f17a8008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:29.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:29.269+0000 7f17c3400640 1 -- 172.21.15.66:0/244517606 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone931018", "target": ["mon-mgr", ""]}) -- 0x7f17bc072f80 con 0x7f179c073ae0 2024-09-10T22:09:30.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.277+0000 7f17b8c00640 1 -- 172.21.15.66:0/244517606 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f17bc072f80 con 0x7f179c073ae0 2024-09-10T22:09:30.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.285+0000 7f17c3400640 1 -- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f179c073ae0 msgr2=0x7f179c075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:30.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.285+0000 7f17c3400640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f179c073ae0 0x7f179c075fa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0af600 tx=0x7f17a8008040 comp rx=0 tx=0).stop 2024-09-10T22:09:30.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.285+0000 7f17c3400640 1 -- 172.21.15.66:0/244517606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17bc0aee20 msgr2=0x7f17bc149f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:30.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.285+0000 7f17c3400640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17bc0aee20 0x7f17bc149f80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009850 tx=0x7f17b00a65b0 comp rx=0 tx=0).stop 2024-09-10T22:09:30.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.285+0000 7f17c3400640 1 -- 172.21.15.66:0/244517606 shutdown_connections 2024-09-10T22:09:30.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.285+0000 7f17c3400640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f179c073ae0 0x7f179c075fa0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:30.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.285+0000 7f17c3400640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17bc15cc80 0x7f17bc15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:30.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.285+0000 7f17c3400640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17bc159900 0x7f17bc155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:30.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.285+0000 7f17c3400640 1 --2- 172.21.15.66:0/244517606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17bc0aee20 0x7f17bc149f80 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:30.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.285+0000 7f17c3400640 1 -- 172.21.15.66:0/244517606 >> 172.21.15.66:0/244517606 conn(0x7f17bc0cfa30 msgr2=0x7f17bc073f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:30.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.285+0000 7f17c3400640 1 -- 172.21.15.66:0/244517606 shutdown_connections 2024-09-10T22:09:30.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.285+0000 7f17c3400640 1 -- 172.21.15.66:0/244517606 wait complete. 2024-09-10T22:09:30.343 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone533619 2024-09-10T22:09:30.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.561+0000 7f61d0e00640 1 -- 172.21.15.66:0/2954167332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc0aee20 msgr2=0x7f61cc0ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.561+0000 7f61d0e00640 1 --2- 172.21.15.66:0/2954167332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc0aee20 0x7f61cc0ca080 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009840 tx=0x7f61b801c7d0 comp rx=0 tx=0).stop 2024-09-10T22:09:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.561+0000 7f61d0e00640 1 -- 172.21.15.66:0/2954167332 shutdown_connections 2024-09-10T22:09:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.561+0000 7f61d0e00640 1 --2- 172.21.15.66:0/2954167332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc0aee20 0x7f61cc0ca080 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.561+0000 7f61d0e00640 1 --2- 172.21.15.66:0/2954167332 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61cc15cc80 0x7f61cc0cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.561+0000 7f61d0e00640 1 -- 172.21.15.66:0/2954167332 >> 172.21.15.66:0/2954167332 conn(0x7f61cc0cfa30 msgr2=0x7f61cc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.561+0000 7f61d0e00640 1 -- 172.21.15.66:0/2954167332 shutdown_connections 2024-09-10T22:09:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.561+0000 7f61d0e00640 1 -- 172.21.15.66:0/2954167332 wait complete. 2024-09-10T22:09:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.565+0000 7f61d0e00640 1 Processor -- start 2024-09-10T22:09:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.565+0000 7f61d0e00640 1 -- start start 2024-09-10T22:09:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.565+0000 7f61d0e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61cc159900 0x7f61cc155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.565+0000 7f61d0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc15cc80 0x7f61cc0b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61d0e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61cc15bc30 0x7f61cc156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61c9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc15cc80 0x7f61cc0b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61d0e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc168f70 con 0x7f61cc15cc80 2024-09-10T22:09:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61c9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc15cc80 0x7f61cc0b5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51938/0 (socket says 172.21.15.66:51938) 2024-09-10T22:09:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61c9400640 1 -- 172.21.15.66:0/338977019 learned_addr learned my addr 172.21.15.66:0/338977019 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61d0e00640 1 -- 172.21.15.66:0/338977019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc168df0 con 0x7f61cc15bc30 2024-09-10T22:09:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61d0e00640 1 -- 172.21.15.66:0/338977019 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f61cc1690f0 con 0x7f61cc159900 2024-09-10T22:09:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61c9400640 1 -- 172.21.15.66:0/338977019 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61cc159900 msgr2=0x7f61cc155710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:09:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61c9400640 1 --2- 172.21.15.66:0/338977019 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61cc159900 0x7f61cc155710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:30.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61c9400640 1 -- 172.21.15.66:0/338977019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61cc15bc30 msgr2=0x7f61cc156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:30.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61c9400640 1 --2- 172.21.15.66:0/338977019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61cc15bc30 0x7f61cc156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:30.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61c9400640 1 -- 172.21.15.66:0/338977019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc0b5750 con 0x7f61cc15cc80 2024-09-10T22:09:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61c9400640 1 --2- 172.21.15.66:0/338977019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc15cc80 0x7f61cc0b5210 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f61b801ccb0 tx=0x7f61b80a6500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61c2000640 1 -- 172.21.15.66:0/338977019 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b80ac070 con 0x7f61cc15cc80 2024-09-10T22:09:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61c2000640 1 -- 172.21.15.66:0/338977019 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61b80a6c90 con 0x7f61cc15cc80 2024-09-10T22:09:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61c2000640 1 -- 172.21.15.66:0/338977019 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8003600 con 0x7f61cc15cc80 2024-09-10T22:09:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61d0e00640 1 -- 172.21.15.66:0/338977019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61cc154670 con 0x7f61cc15cc80 2024-09-10T22:09:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61d0e00640 1 -- 172.21.15.66:0/338977019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61cc154c30 con 0x7f61cc15cc80 2024-09-10T22:09:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.569+0000 7f61d0e00640 1 -- 172.21.15.66:0/338977019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61cc070470 con 0x7f61cc15cc80 2024-09-10T22:09:30.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.573+0000 7f61c2000640 1 -- 172.21.15.66:0/338977019 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7f61b80b8070 con 0x7f61cc15cc80 2024-09-10T22:09:30.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.573+0000 7f61c2000640 1 --2- 172.21.15.66:0/338977019 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f61ac073ab0 0x7f61ac075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:30.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.573+0000 7f61c2000640 1 -- 172.21.15.66:0/338977019 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f61b812c640 con 0x7f61cc15cc80 2024-09-10T22:09:30.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.577+0000 7f61c2000640 1 -- 172.21.15.66:0/338977019 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f61b8016450 con 0x7f61cc15cc80 2024-09-10T22:09:30.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.597+0000 7f61c9e00640 1 --2- 172.21.15.66:0/338977019 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f61ac073ab0 0x7f61ac075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:30.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.617+0000 7f61c9e00640 1 --2- 172.21.15.66:0/338977019 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f61ac073ab0 0x7f61ac075f70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f61bc001550 tx=0x7f61bc007a20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:30.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:30.953+0000 7f61d0e00640 1 -- 172.21.15.66:0/338977019 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone533619", "target": ["mon-mgr", ""]}) -- 0x7f61cc153850 con 0x7f61ac073ab0 2024-09-10T22:09:32.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61c2000640 1 -- 172.21.15.66:0/338977019 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f61cc153850 con 0x7f61ac073ab0 2024-09-10T22:09:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61d0e00640 1 -- 172.21.15.66:0/338977019 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f61ac073ab0 msgr2=0x7f61ac075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61d0e00640 1 --2- 172.21.15.66:0/338977019 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f61ac073ab0 0x7f61ac075f70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f61bc001550 tx=0x7f61bc007a20 comp rx=0 tx=0).stop 2024-09-10T22:09:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61d0e00640 1 -- 172.21.15.66:0/338977019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc15cc80 msgr2=0x7f61cc0b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61d0e00640 1 --2- 172.21.15.66:0/338977019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc15cc80 0x7f61cc0b5210 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f61b801ccb0 tx=0x7f61b80a6500 comp rx=0 tx=0).stop 2024-09-10T22:09:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61d0e00640 1 -- 172.21.15.66:0/338977019 shutdown_connections 2024-09-10T22:09:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61d0e00640 1 --2- 172.21.15.66:0/338977019 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f61ac073ab0 0x7f61ac075f70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61d0e00640 1 --2- 172.21.15.66:0/338977019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61cc15bc30 0x7f61cc156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61d0e00640 1 --2- 172.21.15.66:0/338977019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc15cc80 0x7f61cc0b5210 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61d0e00640 1 --2- 172.21.15.66:0/338977019 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61cc159900 0x7f61cc155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61d0e00640 1 -- 172.21.15.66:0/338977019 >> 172.21.15.66:0/338977019 conn(0x7f61cc0cfa30 msgr2=0x7f61cc152e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61d0e00640 1 -- 172.21.15.66:0/338977019 shutdown_connections 2024-09-10T22:09:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:32.325+0000 7f61d0e00640 1 -- 172.21.15.66:0/338977019 wait complete. 2024-09-10T22:09:34.367 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:09:34.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.617+0000 7fd58e200640 1 -- 172.21.15.66:0/3204954103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5700052f0 msgr2=0x7fd57000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:34.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.617+0000 7fd58e200640 1 --2- 172.21.15.66:0/3204954103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5700052f0 0x7fd57000b9d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd58000c590 tx=0x7fd58001d2a0 comp rx=0 tx=0).stop 2024-09-10T22:09:34.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.617+0000 7fd58d800640 1 -- 172.21.15.66:0/3204954103 reap_dead start 2024-09-10T22:09:34.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.617+0000 7fd58e200640 1 -- 172.21.15.66:0/3204954103 shutdown_connections 2024-09-10T22:09:34.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.617+0000 7fd58e200640 1 -- 172.21.15.66:0/3204954103 >> 172.21.15.66:0/3204954103 conn(0x7fd5880b2730 msgr2=0x7fd5880b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:34.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.621+0000 7fd58e200640 1 -- 172.21.15.66:0/3204954103 shutdown_connections 2024-09-10T22:09:34.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.621+0000 7fd58e200640 1 -- 172.21.15.66:0/3204954103 wait complete. 2024-09-10T22:09:34.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.621+0000 7fd58e200640 1 Processor -- start 2024-09-10T22:09:34.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.625+0000 7fd58e200640 1 -- start start 2024-09-10T22:09:34.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.625+0000 7fd58e200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5700052f0 0x7fd5881537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:34.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.625+0000 7fd58e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd588153d10 0x7fd58814dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:34.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.625+0000 7fd58e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1b40 0x7fd5880b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:34.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.625+0000 7fd58e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5880b3e70 con 0x7fd5880b1b40 2024-09-10T22:09:34.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.625+0000 7fd58e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5880b3cf0 con 0x7fd588153d10 2024-09-10T22:09:34.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.625+0000 7fd58e200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd58806f4a0 con 0x7fd5700052f0 2024-09-10T22:09:34.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd58d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1b40 0x7fd5880b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:34.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd58d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1b40 0x7fd5880b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54910/0 (socket says 172.21.15.66:54910) 2024-09-10T22:09:34.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd58d800640 1 -- 172.21.15.66:0/3495388784 learned_addr learned my addr 172.21.15.66:0/3495388784 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:34.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd58d800640 1 -- 172.21.15.66:0/3495388784 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5700052f0 msgr2=0x7fd5881537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:34.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd58d800640 1 --2- 172.21.15.66:0/3495388784 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5700052f0 0x7fd5881537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:34.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd58d800640 1 -- 172.21.15.66:0/3495388784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd588153d10 msgr2=0x7fd58814dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:34.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd58d800640 1 --2- 172.21.15.66:0/3495388784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd588153d10 0x7fd58814dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:34.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd58d800640 1 -- 172.21.15.66:0/3495388784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5881fbeb0 con 0x7fd5880b1b40 2024-09-10T22:09:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd58d800640 1 --2- 172.21.15.66:0/3495388784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1b40 0x7fd5880b1fc0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd580098e70 tx=0x7fd580006610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd584c00640 1 -- 172.21.15.66:0/3495388784 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580001ff0 con 0x7fd5880b1b40 2024-09-10T22:09:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd584c00640 1 -- 172.21.15.66:0/3495388784 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd580004e40 con 0x7fd5880b1b40 2024-09-10T22:09:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd584c00640 1 -- 172.21.15.66:0/3495388784 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580001a80 con 0x7fd5880b1b40 2024-09-10T22:09:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd58e200640 1 -- 172.21.15.66:0/3495388784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5881fc190 con 0x7fd5880b1b40 2024-09-10T22:09:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.629+0000 7fd58e200640 1 -- 172.21.15.66:0/3495388784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5881fc750 con 0x7fd5880b1b40 2024-09-10T22:09:34.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.633+0000 7fd58e200640 1 -- 172.21.15.66:0/3495388784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58806fa30 con 0x7fd5880b1b40 2024-09-10T22:09:34.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.633+0000 7fd584c00640 1 -- 172.21.15.66:0/3495388784 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7fd5800c2070 con 0x7fd5880b1b40 2024-09-10T22:09:34.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.633+0000 7fd584c00640 1 --2- 172.21.15.66:0/3495388784 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd568073ae0 0x7fd568075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:34.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.637+0000 7fd584c00640 1 -- 172.21.15.66:0/3495388784 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fd58012c4b0 con 0x7fd5880b1b40 2024-09-10T22:09:34.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.641+0000 7fd58ce00640 1 --2- 172.21.15.66:0/3495388784 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd568073ae0 0x7fd568075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:34.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.641+0000 7fd58ce00640 1 --2- 172.21.15.66:0/3495388784 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd568073ae0 0x7fd568075fa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd57c006fd0 tx=0x7fd57c006240 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:34.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:34.645+0000 7fd584c00640 1 -- 172.21.15.66:0/3495388784 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd5800bf060 con 0x7fd5880b1b40 2024-09-10T22:09:35.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.057+0000 7fd58e200640 1 -- 172.21.15.66:0/3495388784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd58806d2f0 con 0x7fd5880b1b40 2024-09-10T22:09:35.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.057+0000 7fd584c00640 1 -- 172.21.15.66:0/3495388784 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2677 (secure 0 0 0) 0x7fd580017050 con 0x7fd5880b1b40 2024-09-10T22:09:35.065 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:09:35.065 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:09:35.065 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:09:35.065 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:09:35.065 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:09:35.066 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:09:35.066 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:09:35.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:35.066 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:09:35.066 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:09:35.066 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:09:35.066 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:09:35.066 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:09:35.066 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:35.066 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:09:35.066 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:09:35.067 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:09:35.067 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:09:35.067 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:35.067 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 568, 2024-09-10T22:09:35.067 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:09:35.067 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:09:35.067 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:09:35.067 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:09:35.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:35.067 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:09:35.067 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 87, 2024-09-10T22:09:35.067 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:35.068 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:35.069 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:09:35.069 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:09:35.069 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 6731, 2024-09-10T22:09:35.069 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 28016781810, 2024-09-10T22:09:35.069 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 55775133696, 2024-09-10T22:09:35.069 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 717318979584, 2024-09-10T22:09:35.069 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:09:35.069 INFO:teuthology.orchestra.run.smithi066.stdout: "recovering_objects_per_sec": 1, 2024-09-10T22:09:35.069 INFO:teuthology.orchestra.run.smithi066.stdout: "recovering_bytes_per_sec": 4535040, 2024-09-10T22:09:35.069 INFO:teuthology.orchestra.run.smithi066.stdout: "recovering_keys_per_sec": 0, 2024-09-10T22:09:35.069 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects_recovered": 13, 2024-09-10T22:09:35.069 INFO:teuthology.orchestra.run.smithi066.stdout: "num_bytes_recovered": 54525952, 2024-09-10T22:09:35.070 INFO:teuthology.orchestra.run.smithi066.stdout: "num_keys_recovered": 0, 2024-09-10T22:09:35.070 INFO:teuthology.orchestra.run.smithi066.stdout: "read_bytes_sec": 594092663, 2024-09-10T22:09:35.070 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 601478116, 2024-09-10T22:09:35.070 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 143, 2024-09-10T22:09:35.070 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 162 2024-09-10T22:09:35.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:35.070 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:09:35.070 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 57, 2024-09-10T22:09:35.070 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:08:29:488488+0000", 2024-09-10T22:09:35.070 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 6, 2024-09-10T22:09:35.070 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:09:35.071 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:09:35.071 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:09:35.071 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:09:35.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:09:35.071 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 6, 2024-09-10T22:09:35.071 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:09:35.071 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "d", 2024-09-10T22:09:35.071 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:09:35.071 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 6990 2024-09-10T22:09:35.072 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:35.072 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:35.072 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:09:35.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:35.072 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:09:35.072 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:09:35.072 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:09:35.073 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:09:35.073 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:09:35.073 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:09:35.073 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:09:35.073 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:09:35.073 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:09:35.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:35.074 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:09:35.074 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 17, 2024-09-10T22:09:35.074 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:09:28.497849+0000", 2024-09-10T22:09:35.074 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:09:35.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:35.075 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": { 2024-09-10T22:09:35.075 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr-vol-ongoing-clones": { 2024-09-10T22:09:35.075 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "4 ongoing clones - average progress is 20.052% (35s)\n [=====.......................] (remaining: 2m)", 2024-09-10T22:09:35.075 INFO:teuthology.orchestra.run.smithi066.stdout: "progress": 0.20051999390125275, 2024-09-10T22:09:35.075 INFO:teuthology.orchestra.run.smithi066.stdout: "add_to_ceph_s": true 2024-09-10T22:09:35.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:09:35.075 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr-vol-total-clones": { 2024-09-10T22:09:35.076 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "Total 12 clones - average progress is 15.365% (28s)\n [====........................] (remaining: 2m)", 2024-09-10T22:09:35.076 INFO:teuthology.orchestra.run.smithi066.stdout: "progress": 0.15365000069141388, 2024-09-10T22:09:35.076 INFO:teuthology.orchestra.run.smithi066.stdout: "add_to_ceph_s": true 2024-09-10T22:09:35.076 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:35.076 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:09:35.076 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:09:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.061+0000 7fd58e200640 1 -- 172.21.15.66:0/3495388784 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd568073ae0 msgr2=0x7fd568075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.061+0000 7fd58e200640 1 --2- 172.21.15.66:0/3495388784 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd568073ae0 0x7fd568075fa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd57c006fd0 tx=0x7fd57c006240 comp rx=0 tx=0).stop 2024-09-10T22:09:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.061+0000 7fd58e200640 1 -- 172.21.15.66:0/3495388784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1b40 msgr2=0x7fd5880b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.061+0000 7fd58e200640 1 --2- 172.21.15.66:0/3495388784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1b40 0x7fd5880b1fc0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd580098e70 tx=0x7fd580006610 comp rx=0 tx=0).stop 2024-09-10T22:09:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.061+0000 7fd58e200640 1 -- 172.21.15.66:0/3495388784 shutdown_connections 2024-09-10T22:09:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.061+0000 7fd58e200640 1 --2- 172.21.15.66:0/3495388784 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd568073ae0 0x7fd568075fa0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.061+0000 7fd58e200640 1 --2- 172.21.15.66:0/3495388784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1b40 0x7fd5880b1fc0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.061+0000 7fd58e200640 1 --2- 172.21.15.66:0/3495388784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd588153d10 0x7fd58814dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.061+0000 7fd58e200640 1 --2- 172.21.15.66:0/3495388784 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5700052f0 0x7fd5881537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.061+0000 7fd58e200640 1 -- 172.21.15.66:0/3495388784 >> 172.21.15.66:0/3495388784 conn(0x7fd5880b2730 msgr2=0x7fd5880b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.061+0000 7fd58e200640 1 -- 172.21.15.66:0/3495388784 shutdown_connections 2024-09-10T22:09:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.061+0000 7fd58e200640 1 -- 172.21.15.66:0/3495388784 wait complete. 2024-09-10T22:09:35.110 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 20.052% (35s)\n [=====.......................] (remaining: 2m)', 'progress': 0.20051999390125275, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 12 clones - average progress is 15.365% (28s)\n [====........................] (remaining: 2m)', 'progress': 0.15365000069141388, 'add_to_ceph_s': True}} 2024-09-10T22:09:35.110 INFO:tasks.cephfs.test_volumes:new_onpen_p is less than onpen_p. 2024-09-10T22:09:35.110 INFO:tasks.cephfs.test_volumes:new_onpen_p = 0.15365000069141388; onpen_p = 0.27239999175071716 2024-09-10T22:09:35.111 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone33350 2024-09-10T22:09:35.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.385+0000 7ff8fcc00640 1 -- 172.21.15.66:0/413621905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8f80c6280 msgr2=0x7ff8f80cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:35.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.385+0000 7ff8fcc00640 1 --2- 172.21.15.66:0/413621905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8f80c6280 0x7ff8f80cf2e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000afd0 tx=0x7ff8e001ec10 comp rx=0 tx=0).stop 2024-09-10T22:09:35.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.385+0000 7ff8fcc00640 1 -- 172.21.15.66:0/413621905 shutdown_connections 2024-09-10T22:09:35.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.385+0000 7ff8fcc00640 1 --2- 172.21.15.66:0/413621905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8f80c6280 0x7ff8f80cf2e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:35.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.385+0000 7ff8fcc00640 1 --2- 172.21.15.66:0/413621905 >> v2:172.21.15.94:3300/0 conn(0x7ff8f80cef00 0x7ff8f80d0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:35.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.385+0000 7ff8fcc00640 1 -- 172.21.15.66:0/413621905 >> 172.21.15.66:0/413621905 conn(0x7ff8f80cfa30 msgr2=0x7ff8f80cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:35.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.385+0000 7ff8fcc00640 1 -- 172.21.15.66:0/413621905 shutdown_connections 2024-09-10T22:09:35.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.385+0000 7ff8fcc00640 1 -- 172.21.15.66:0/413621905 wait complete. 2024-09-10T22:09:35.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.389+0000 7ff8fcc00640 1 Processor -- start 2024-09-10T22:09:35.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.389+0000 7ff8fcc00640 1 -- start start 2024-09-10T22:09:35.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.389+0000 7ff8fcc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8f80c6280 0x7ff8f8163020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:35.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.389+0000 7ff8fcc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8f80cd930 0x7ff8f80c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:35.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.389+0000 7ff8fcc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8f80cef00 0x7ff8f80ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:35.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.389+0000 7ff8fcc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f80c4a30 con 0x7ff8f80c6280 2024-09-10T22:09:35.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.389+0000 7ff8fcc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f80c48b0 con 0x7ff8f80cd930 2024-09-10T22:09:35.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.389+0000 7ff8fcc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f80ce4b0 con 0x7ff8f80cef00 2024-09-10T22:09:35.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8f80c6280 0x7ff8f8163020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:35.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8f80c6280 0x7ff8f8163020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54926/0 (socket says 172.21.15.66:54926) 2024-09-10T22:09:35.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8f5e00640 1 -- 172.21.15.66:0/2230898586 learned_addr learned my addr 172.21.15.66:0/2230898586 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:35.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8f6800640 1 --2- 172.21.15.66:0/2230898586 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8f80cef00 0x7ff8f80ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:35.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8efe00640 1 --2- 172.21.15.66:0/2230898586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8f80cd930 0x7ff8f80c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:35.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8f5e00640 1 -- 172.21.15.66:0/2230898586 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8f80cef00 msgr2=0x7ff8f80ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:35.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8f5e00640 1 --2- 172.21.15.66:0/2230898586 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8f80cef00 0x7ff8f80ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:35.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8f5e00640 1 -- 172.21.15.66:0/2230898586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8f80cd930 msgr2=0x7ff8f80c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:35.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8f5e00640 1 --2- 172.21.15.66:0/2230898586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8f80cd930 0x7ff8f80c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:35.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8f5e00640 1 -- 172.21.15.66:0/2230898586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f8163560 con 0x7ff8f80c6280 2024-09-10T22:09:35.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8f5e00640 1 --2- 172.21.15.66:0/2230898586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8f80c6280 0x7ff8f8163020 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80058d0 tx=0x7ff8e800ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:35.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8ee000640 1 -- 172.21.15.66:0/2230898586 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8019070 con 0x7ff8f80c6280 2024-09-10T22:09:35.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8ee000640 1 -- 172.21.15.66:0/2230898586 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8e80079b0 con 0x7ff8f80c6280 2024-09-10T22:09:35.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8ee000640 1 -- 172.21.15.66:0/2230898586 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8007de0 con 0x7ff8f80c6280 2024-09-10T22:09:35.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8fcc00640 1 -- 172.21.15.66:0/2230898586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f8165a80 con 0x7ff8f80c6280 2024-09-10T22:09:35.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.393+0000 7ff8fcc00640 1 -- 172.21.15.66:0/2230898586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f8166040 con 0x7ff8f80c6280 2024-09-10T22:09:35.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.397+0000 7ff8ee000640 1 -- 172.21.15.66:0/2230898586 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7ff8e8014410 con 0x7ff8f80c6280 2024-09-10T22:09:35.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.397+0000 7ff8fcc00640 1 -- 172.21.15.66:0/2230898586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f80d0550 con 0x7ff8f80c6280 2024-09-10T22:09:35.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.401+0000 7ff8ee000640 1 --2- 172.21.15.66:0/2230898586 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8d4073ab0 0x7ff8d4075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:35.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.401+0000 7ff8ee000640 1 -- 172.21.15.66:0/2230898586 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7ff8e80919d0 con 0x7ff8f80c6280 2024-09-10T22:09:35.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.409+0000 7ff8ee000640 1 -- 172.21.15.66:0/2230898586 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff8e805e510 con 0x7ff8f80c6280 2024-09-10T22:09:35.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.417+0000 7ff8efe00640 1 --2- 172.21.15.66:0/2230898586 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8d4073ab0 0x7ff8d4075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:35.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.433+0000 7ff8efe00640 1 --2- 172.21.15.66:0/2230898586 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8d4073ab0 0x7ff8d4075f70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00b1000 tx=0x7ff8e001b9b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:35.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:35.781+0000 7ff8fcc00640 1 -- 172.21.15.66:0/2230898586 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone33350", "target": ["mon-mgr", ""]}) -- 0x7ff8f80c7c40 con 0x7ff8d4073ab0 2024-09-10T22:09:36.043 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:36.039+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:09:36.043 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:36.039+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:09:36.043 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:36.039+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:09:36.043 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:36.039+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:09:36.043 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:36.039+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:09:36.330 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:36.327+0000 7f5fe8800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:36.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.325+0000 7ff8ee000640 1 -- 172.21.15.66:0/2230898586 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7ff8f80c7c40 con 0x7ff8d4073ab0 2024-09-10T22:09:36.331 INFO:teuthology.orchestra.run.smithi066.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:36.349 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T22:09:36.350 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone75881 2024-09-10T22:09:36.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.613+0000 7fb12a200640 1 -- 172.21.15.66:0/3790246828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040051d0 msgr2=0x7fb10400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:36.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.613+0000 7fb12a200640 1 --2- 172.21.15.66:0/3790246828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040051d0 0x7fb10400b9d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009ee0 tx=0x7fb11c01d630 comp rx=0 tx=0).stop 2024-09-10T22:09:36.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.613+0000 7fb12a200640 1 -- 172.21.15.66:0/3790246828 shutdown_connections 2024-09-10T22:09:36.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.613+0000 7fb12a200640 1 --2- 172.21.15.66:0/3790246828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040051d0 0x7fb10400b9d0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:36.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.613+0000 7fb12a200640 1 -- 172.21.15.66:0/3790246828 >> 172.21.15.66:0/3790246828 conn(0x7fb1240d1b10 msgr2=0x7fb1240d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:36.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.613+0000 7fb12a200640 1 -- 172.21.15.66:0/3790246828 shutdown_connections 2024-09-10T22:09:36.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.613+0000 7fb12a200640 1 -- 172.21.15.66:0/3790246828 wait complete. 2024-09-10T22:09:36.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.613+0000 7fb12a200640 1 Processor -- start 2024-09-10T22:09:36.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.613+0000 7fb12a200640 1 -- start start 2024-09-10T22:09:36.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb12a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1040051d0 0x7fb124156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:36.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb12a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1240c9f00 0x7fb1240b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:36.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb12a200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1240ca380 0x7fb1240b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:36.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb12a200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb1240cdfa0 con 0x7fb1240c9f00 2024-09-10T22:09:36.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb12a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1240cde20 con 0x7fb1040051d0 2024-09-10T22:09:36.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb12a200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb1240cecc0 con 0x7fb1240ca380 2024-09-10T22:09:36.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb11be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1240c9f00 0x7fb1240b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:36.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb11be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1240c9f00 0x7fb1240b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54948/0 (socket says 172.21.15.66:54948) 2024-09-10T22:09:36.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb11be00640 1 -- 172.21.15.66:0/3010040693 learned_addr learned my addr 172.21.15.66:0/3010040693 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:36.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb129800640 1 --2- 172.21.15.66:0/3010040693 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1240ca380 0x7fb1240b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:36.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb11be00640 1 -- 172.21.15.66:0/3010040693 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1240ca380 msgr2=0x7fb1240b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:36.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb11be00640 1 --2- 172.21.15.66:0/3010040693 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1240ca380 0x7fb1240b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:36.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb11be00640 1 -- 172.21.15.66:0/3010040693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1040051d0 msgr2=0x7fb124156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:36.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb11be00640 1 --2- 172.21.15.66:0/3010040693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1040051d0 0x7fb124156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:36.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb11be00640 1 -- 172.21.15.66:0/3010040693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1241f7da0 con 0x7fb1240c9f00 2024-09-10T22:09:36.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb129800640 1 --2- 172.21.15.66:0/3010040693 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1240ca380 0x7fb1240b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:09:36.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb11be00640 1 --2- 172.21.15.66:0/3010040693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1240c9f00 0x7fb1240b5460 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c390 tx=0x7fb11400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:36.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb121600640 1 -- 172.21.15.66:0/3010040693 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114011790 con 0x7fb1240c9f00 2024-09-10T22:09:36.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb121600640 1 -- 172.21.15.66:0/3010040693 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb114017070 con 0x7fb1240c9f00 2024-09-10T22:09:36.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb12a200640 1 -- 172.21.15.66:0/3010040693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1241f8080 con 0x7fb1240c9f00 2024-09-10T22:09:36.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb121600640 1 -- 172.21.15.66:0/3010040693 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114011cc0 con 0x7fb1240c9f00 2024-09-10T22:09:36.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb12a200640 1 -- 172.21.15.66:0/3010040693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1241f8640 con 0x7fb1240c9f00 2024-09-10T22:09:36.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.617+0000 7fb12a200640 1 -- 172.21.15.66:0/3010040693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1240cc5c0 con 0x7fb1240c9f00 2024-09-10T22:09:36.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.621+0000 7fb121600640 1 -- 172.21.15.66:0/3010040693 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 30) ==== 98281+0+0 (secure 0 0 0) 0x7fb11401c460 con 0x7fb1240c9f00 2024-09-10T22:09:36.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.621+0000 7fb121600640 1 --2- 172.21.15.66:0/3010040693 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb100073ae0 0x7fb100075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:36.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.621+0000 7fb121600640 1 -- 172.21.15.66:0/3010040693 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fb114091eb0 con 0x7fb1240c9f00 2024-09-10T22:09:36.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.629+0000 7fb121600640 1 -- 172.21.15.66:0/3010040693 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb114021020 con 0x7fb1240c9f00 2024-09-10T22:09:36.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.641+0000 7fb128e00640 1 --2- 172.21.15.66:0/3010040693 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb100073ae0 0x7fb100075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:36.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.653+0000 7fb128e00640 1 --2- 172.21.15.66:0/3010040693 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb100073ae0 0x7fb100075fa0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb1240b10e0 tx=0x7fb10c001b50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:36.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.889+0000 7fb121600640 1 -- 172.21.15.66:0/3010040693 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 31) ==== 98332+0+0 (secure 0 0 0) 0x7fb11405a7d0 con 0x7fb1240c9f00 2024-09-10T22:09:36.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:36.961+0000 7fb12a200640 1 -- 172.21.15.66:0/3010040693 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone75881", "target": ["mon-mgr", ""]}) -- 0x7fb124072240 con 0x7fb100073ae0 2024-09-10T22:09:37.456 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:37.455+0000 7f5fe8800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:37.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.457+0000 7fb121600640 1 -- 172.21.15.66:0/3010040693 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fb124072240 con 0x7fb100073ae0 2024-09-10T22:09:37.463 INFO:teuthology.orchestra.run.smithi066.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:37.487 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T22:09:37.488 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone94742 2024-09-10T22:09:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.701+0000 7fef77c00640 1 -- 172.21.15.66:0/2423982403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef700c4fc0 msgr2=0x7fef700ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.701+0000 7fef77c00640 1 --2- 172.21.15.66:0/2423982403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef700c4fc0 0x7fef700ca6e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fef6800ad90 tx=0x7fef6801dc20 comp rx=0 tx=0).stop 2024-09-10T22:09:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.701+0000 7fef77c00640 1 -- 172.21.15.66:0/2423982403 shutdown_connections 2024-09-10T22:09:37.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.701+0000 7fef77c00640 1 --2- 172.21.15.66:0/2423982403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef700c4fc0 0x7fef700ca6e0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:37.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.701+0000 7fef77c00640 1 --2- 172.21.15.66:0/2423982403 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef700d0550 0x7fef700ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:37.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.701+0000 7fef77c00640 1 -- 172.21.15.66:0/2423982403 >> 172.21.15.66:0/2423982403 conn(0x7fef700cfa30 msgr2=0x7fef700cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:37.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.701+0000 7fef77c00640 1 -- 172.21.15.66:0/2423982403 shutdown_connections 2024-09-10T22:09:37.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.701+0000 7fef77c00640 1 -- 172.21.15.66:0/2423982403 wait complete. 2024-09-10T22:09:37.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef77c00640 1 Processor -- start 2024-09-10T22:09:37.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef77c00640 1 -- start start 2024-09-10T22:09:37.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef77c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef700c4fc0 0x7fef70150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:37.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef77c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef7015a310 0x7fef70144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:37.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef77c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef70150e10 0x7fef701512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:37.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef77c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fef70167480 con 0x7fef7015a310 2024-09-10T22:09:37.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef77c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef70167300 con 0x7fef70150e10 2024-09-10T22:09:37.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef77c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fef70167600 con 0x7fef700c4fc0 2024-09-10T22:09:37.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef6fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef7015a310 0x7fef70144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:37.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef6fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef7015a310 0x7fef70144490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54972/0 (socket says 172.21.15.66:54972) 2024-09-10T22:09:37.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef6fe00640 1 -- 172.21.15.66:0/3549040551 learned_addr learned my addr 172.21.15.66:0/3549040551 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef6fe00640 1 -- 172.21.15.66:0/3549040551 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef700c4fc0 msgr2=0x7fef70150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef6fe00640 1 --2- 172.21.15.66:0/3549040551 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef700c4fc0 0x7fef70150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef6fe00640 1 -- 172.21.15.66:0/3549040551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef70150e10 msgr2=0x7fef701512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef6fe00640 1 --2- 172.21.15.66:0/3549040551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef70150e10 0x7fef701512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.705+0000 7fef6fe00640 1 -- 172.21.15.66:0/3549040551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef701507c0 con 0x7fef7015a310 2024-09-10T22:09:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.709+0000 7fef6fe00640 1 --2- 172.21.15.66:0/3549040551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef7015a310 0x7fef70144490 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00b550 tx=0x7fef5c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.709+0000 7fef6cc00640 1 -- 172.21.15.66:0/3549040551 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c014070 con 0x7fef7015a310 2024-09-10T22:09:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.709+0000 7fef6cc00640 1 -- 172.21.15.66:0/3549040551 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef5c010980 con 0x7fef7015a310 2024-09-10T22:09:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.709+0000 7fef6cc00640 1 -- 172.21.15.66:0/3549040551 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c010dd0 con 0x7fef7015a310 2024-09-10T22:09:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.709+0000 7fef77c00640 1 -- 172.21.15.66:0/3549040551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7014a0d0 con 0x7fef7015a310 2024-09-10T22:09:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.709+0000 7fef77c00640 1 -- 172.21.15.66:0/3549040551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7014fa80 con 0x7fef7015a310 2024-09-10T22:09:37.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.709+0000 7fef77c00640 1 -- 172.21.15.66:0/3549040551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7014f870 con 0x7fef7015a310 2024-09-10T22:09:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.713+0000 7fef6cc00640 1 -- 172.21.15.66:0/3549040551 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 31) ==== 98332+0+0 (secure 0 0 0) 0x7fef5c00f3c0 con 0x7fef7015a310 2024-09-10T22:09:37.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.713+0000 7fef6cc00640 1 --2- 172.21.15.66:0/3549040551 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fef50073c40 0x7fef50076100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:37.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.713+0000 7fef6cc00640 1 -- 172.21.15.66:0/3549040551 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fef5c092440 con 0x7fef7015a310 2024-09-10T22:09:37.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.713+0000 7fef75200640 1 --2- 172.21.15.66:0/3549040551 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fef50073c40 0x7fef50076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:37.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.717+0000 7fef75200640 1 --2- 172.21.15.66:0/3549040551 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fef50073c40 0x7fef50076100 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fef64009650 tx=0x7fef64006bb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:37.721+0000 7fef6cc00640 1 -- 172.21.15.66:0/3549040551 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fef701507c0 con 0x7fef7015a310 2024-09-10T22:09:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.045+0000 7fef77c00640 1 -- 172.21.15.66:0/3549040551 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone94742", "target": ["mon-mgr", ""]}) -- 0x7fef700c48b0 con 0x7fef50073c40 2024-09-10T22:09:38.518 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:38.515+0000 7f5fe8800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:38.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.513+0000 7fef6cc00640 1 -- 172.21.15.66:0/3549040551 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fef700c48b0 con 0x7fef50073c40 2024-09-10T22:09:38.521 INFO:teuthology.orchestra.run.smithi066.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:38.546 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T22:09:38.546 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone723 2024-09-10T22:09:38.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.781+0000 7f064f400640 1 -- 172.21.15.66:0/2600350488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064806e630 msgr2=0x7f064806ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:38.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.781+0000 7f064f400640 1 --2- 172.21.15.66:0/2600350488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064806e630 0x7f064806ea80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f06480cd570 tx=0x7f06400203f0 comp rx=0 tx=0).stop 2024-09-10T22:09:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.781+0000 7f064f400640 1 -- 172.21.15.66:0/2600350488 shutdown_connections 2024-09-10T22:09:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.781+0000 7f064f400640 1 --2- 172.21.15.66:0/2600350488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064806e630 0x7f064806ea80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.781+0000 7f064f400640 1 --2- 172.21.15.66:0/2600350488 >> v2:172.21.15.94:3300/0 conn(0x7f06480cef00 0x7f06480d0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.781+0000 7f064f400640 1 -- 172.21.15.66:0/2600350488 >> 172.21.15.66:0/2600350488 conn(0x7f06480cfa30 msgr2=0x7f06480cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.785+0000 7f064f400640 1 -- 172.21.15.66:0/2600350488 shutdown_connections 2024-09-10T22:09:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.785+0000 7f064f400640 1 -- 172.21.15.66:0/2600350488 wait complete. 2024-09-10T22:09:38.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.785+0000 7f064f400640 1 Processor -- start 2024-09-10T22:09:38.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.785+0000 7f064f400640 1 -- start start 2024-09-10T22:09:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064f400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f064806e630 0x7f0648200f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06480c6280 0x7f06480cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06480cef00 0x7f06480c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f06480c67e0 con 0x7f06480cef00 2024-09-10T22:09:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064f400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06480c4ac0 con 0x7f06480c6280 2024-09-10T22:09:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064f400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f064806efc0 con 0x7f064806e630 2024-09-10T22:09:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06480cef00 0x7f06480c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:38.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06480cef00 0x7f06480c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54986/0 (socket says 172.21.15.66:54986) 2024-09-10T22:09:38.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064d400640 1 -- 172.21.15.66:0/4233059751 learned_addr learned my addr 172.21.15.66:0/4233059751 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:38.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064d400640 1 -- 172.21.15.66:0/4233059751 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f064806e630 msgr2=0x7f0648200f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:38.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064d400640 1 --2- 172.21.15.66:0/4233059751 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f064806e630 0x7f0648200f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:38.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064d400640 1 -- 172.21.15.66:0/4233059751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06480c6280 msgr2=0x7f06480cb810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:09:38.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064d400640 1 --2- 172.21.15.66:0/4233059751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06480c6280 0x7f06480cb810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:38.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064d400640 1 -- 172.21.15.66:0/4233059751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064816cb90 con 0x7f06480cef00 2024-09-10T22:09:38.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064d400640 1 --2- 172.21.15.66:0/4233059751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06480cef00 0x7f06480c9530 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f064001d000 tx=0x7f06400b0560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:38.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f0645600640 1 -- 172.21.15.66:0/4233059751 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400c0070 con 0x7f06480cef00 2024-09-10T22:09:38.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f0645600640 1 -- 172.21.15.66:0/4233059751 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06400b6040 con 0x7f06480cef00 2024-09-10T22:09:38.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f0645600640 1 -- 172.21.15.66:0/4233059751 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400b5070 con 0x7f06480cef00 2024-09-10T22:09:38.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064f400640 1 -- 172.21.15.66:0/4233059751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064816ce70 con 0x7f06480cef00 2024-09-10T22:09:38.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.789+0000 7f064f400640 1 -- 172.21.15.66:0/4233059751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064816d430 con 0x7f06480cef00 2024-09-10T22:09:38.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.793+0000 7f064f400640 1 -- 172.21.15.66:0/4233059751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06480c6c60 con 0x7f06480cef00 2024-09-10T22:09:38.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.793+0000 7f0645600640 1 -- 172.21.15.66:0/4233059751 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 31) ==== 98332+0+0 (secure 0 0 0) 0x7f064000b070 con 0x7f06480cef00 2024-09-10T22:09:38.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.793+0000 7f0645600640 1 --2- 172.21.15.66:0/4233059751 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0624073b40 0x7f0624076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:38.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.793+0000 7f0645600640 1 -- 172.21.15.66:0/4233059751 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f064012fab0 con 0x7f06480cef00 2024-09-10T22:09:38.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.801+0000 7f064ca00640 1 --2- 172.21.15.66:0/4233059751 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0624073b40 0x7f0624076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:38.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.801+0000 7f064ca00640 1 --2- 172.21.15.66:0/4233059751 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0624073b40 0x7f0624076000 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f063800a860 tx=0x7f0638005b60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:38.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:38.809+0000 7f0645600640 1 -- 172.21.15.66:0/4233059751 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0640019f50 con 0x7f06480cef00 2024-09-10T22:09:39.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.289+0000 7f064f400640 1 -- 172.21.15.66:0/4233059751 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone723", "target": ["mon-mgr", ""]}) -- 0x7f0648075130 con 0x7f0624073b40 2024-09-10T22:09:39.561 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:39.559+0000 7f5fe8800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:39.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.557+0000 7f0645600640 1 -- 172.21.15.66:0/4233059751 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f0648075130 con 0x7f0624073b40 2024-09-10T22:09:39.562 INFO:teuthology.orchestra.run.smithi066.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:39.581 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T22:09:39.582 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone37214 2024-09-10T22:09:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.801+0000 7f4d17a00640 1 -- 172.21.15.66:0/26158438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d100aee20 msgr2=0x7f4d100ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.801+0000 7f4d17a00640 1 --2- 172.21.15.66:0/26158438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d100aee20 0x7f4d100ca080 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009900 tx=0x7f4cfc01c850 comp rx=0 tx=0).stop 2024-09-10T22:09:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 -- 172.21.15.66:0/26158438 shutdown_connections 2024-09-10T22:09:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 --2- 172.21.15.66:0/26158438 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d1015cc80 0x7f4d100cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 --2- 172.21.15.66:0/26158438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d100aee20 0x7f4d100ca080 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 -- 172.21.15.66:0/26158438 >> 172.21.15.66:0/26158438 conn(0x7f4d100cfa30 msgr2=0x7f4d100cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 -- 172.21.15.66:0/26158438 shutdown_connections 2024-09-10T22:09:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 -- 172.21.15.66:0/26158438 wait complete. 2024-09-10T22:09:39.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 Processor -- start 2024-09-10T22:09:39.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 -- start start 2024-09-10T22:09:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d100aee20 0x7f4d101554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d10159900 0x7f4d10151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d1015cc80 0x7f4d1015e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4d10168f70 con 0x7f4d10159900 2024-09-10T22:09:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d10168df0 con 0x7f4d100aee20 2024-09-10T22:09:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.805+0000 7f4d17a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4d101690f0 con 0x7f4d1015cc80 2024-09-10T22:09:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d0fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d10159900 0x7f4d10151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d0fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d10159900 0x7f4d10151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55008/0 (socket says 172.21.15.66:55008) 2024-09-10T22:09:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d0fe00640 1 -- 172.21.15.66:0/467085173 learned_addr learned my addr 172.21.15.66:0/467085173 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d15000640 1 --2- 172.21.15.66:0/467085173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d100aee20 0x7f4d101554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d0fe00640 1 -- 172.21.15.66:0/467085173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d1015cc80 msgr2=0x7f4d1015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d15a00640 1 --2- 172.21.15.66:0/467085173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d1015cc80 0x7f4d1015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d0fe00640 1 --2- 172.21.15.66:0/467085173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d1015cc80 0x7f4d1015e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d0fe00640 1 -- 172.21.15.66:0/467085173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d100aee20 msgr2=0x7f4d101554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d0fe00640 1 --2- 172.21.15.66:0/467085173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d100aee20 0x7f4d101554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d0fe00640 1 -- 172.21.15.66:0/467085173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d101506a0 con 0x7f4d10159900 2024-09-10T22:09:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d15000640 1 --2- 172.21.15.66:0/467085173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d100aee20 0x7f4d101554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:09:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d15a00640 1 --2- 172.21.15.66:0/467085173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d1015cc80 0x7f4d1015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:09:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d0fe00640 1 --2- 172.21.15.66:0/467085173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d10159900 0x7f4d10151fc0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009850 tx=0x7f4cfc0a65b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d0cc00640 1 -- 172.21.15.66:0/467085173 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc0ac070 con 0x7f4d10159900 2024-09-10T22:09:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d17a00640 1 -- 172.21.15.66:0/467085173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d101f92e0 con 0x7f4d10159900 2024-09-10T22:09:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d0cc00640 1 -- 172.21.15.66:0/467085173 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cfc004310 con 0x7f4d10159900 2024-09-10T22:09:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d0cc00640 1 -- 172.21.15.66:0/467085173 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc0053a0 con 0x7f4d10159900 2024-09-10T22:09:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.809+0000 7f4d17a00640 1 -- 172.21.15.66:0/467085173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d101f98a0 con 0x7f4d10159900 2024-09-10T22:09:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.813+0000 7f4d17a00640 1 -- 172.21.15.66:0/467085173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d100ca080 con 0x7f4d10159900 2024-09-10T22:09:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.813+0000 7f4d0cc00640 1 -- 172.21.15.66:0/467085173 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 31) ==== 98332+0+0 (secure 0 0 0) 0x7f4cfc0046d0 con 0x7f4d10159900 2024-09-10T22:09:39.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.817+0000 7f4d0cc00640 1 --2- 172.21.15.66:0/467085173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4cf0073b40 0x7f4cf0076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:39.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.817+0000 7f4d0cc00640 1 -- 172.21.15.66:0/467085173 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f4cfc12c930 con 0x7f4d10159900 2024-09-10T22:09:39.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.821+0000 7f4d0cc00640 1 -- 172.21.15.66:0/467085173 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4cfc0bf020 con 0x7f4d10159900 2024-09-10T22:09:39.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.821+0000 7f4d15000640 1 --2- 172.21.15.66:0/467085173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4cf0073b40 0x7f4cf0076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:39.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:39.825+0000 7f4d15000640 1 --2- 172.21.15.66:0/467085173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4cf0073b40 0x7f4cf0076000 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4d04003ec0 tx=0x7f4d04009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:40.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:40.209+0000 7f4d17a00640 1 -- 172.21.15.66:0/467085173 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone37214", "target": ["mon-mgr", ""]}) -- 0x7f4d10070470 con 0x7f4cf0073b40 2024-09-10T22:09:40.690 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:40.687+0000 7f5fe8800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:40.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:40.721+0000 7f4d0cc00640 1 -- 172.21.15.66:0/467085173 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f4d10070470 con 0x7f4cf0073b40 2024-09-10T22:09:40.729 INFO:teuthology.orchestra.run.smithi066.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:40.747 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T22:09:40.748 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone51775 2024-09-10T22:09:41.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.037+0000 7f8cea200640 1 -- 172.21.15.66:0/2123093371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cdc035d10 msgr2=0x7f8cdc039e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:41.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.037+0000 7f8cea200640 1 --2- 172.21.15.66:0/2123093371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cdc035d10 0x7f8cdc039e20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00edb0 tx=0x7f8cd001c8e0 comp rx=0 tx=0).stop 2024-09-10T22:09:41.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.037+0000 7f8cea200640 1 -- 172.21.15.66:0/2123093371 shutdown_connections 2024-09-10T22:09:41.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.037+0000 7f8cea200640 1 --2- 172.21.15.66:0/2123093371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cdc035d10 0x7f8cdc039e20 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:41.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.037+0000 7f8cea200640 1 --2- 172.21.15.66:0/2123093371 >> v2:172.21.15.94:3300/0 conn(0x7f8cdc035930 0x7f8cdc033c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:41.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.037+0000 7f8cea200640 1 -- 172.21.15.66:0/2123093371 >> 172.21.15.66:0/2123093371 conn(0x7f8cdc01dbd0 msgr2=0x7f8cdc0549c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:41.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.037+0000 7f8cea200640 1 -- 172.21.15.66:0/2123093371 shutdown_connections 2024-09-10T22:09:41.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.037+0000 7f8cea200640 1 -- 172.21.15.66:0/2123093371 wait complete. 2024-09-10T22:09:41.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.041+0000 7f8cea200640 1 Processor -- start 2024-09-10T22:09:41.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.041+0000 7f8cea200640 1 -- start start 2024-09-10T22:09:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.045+0000 7f8cea200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc035930 0x7f8cdc049a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.045+0000 7f8cea200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cdc035d10 0x7f8cdc0d30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.045+0000 7f8cea200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8cdc055570 0x7f8cdc0df170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.045+0000 7f8cea200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc0329c0 con 0x7f8cdc035d10 2024-09-10T22:09:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.045+0000 7f8cea200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc032840 con 0x7f8cdc035930 2024-09-10T22:09:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.045+0000 7f8cea200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc0eda90 con 0x7f8cdc055570 2024-09-10T22:09:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.045+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cdc035d10 0x7f8cdc0d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.045+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cdc035d10 0x7f8cdc0d30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55016/0 (socket says 172.21.15.66:55016) 2024-09-10T22:09:41.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.045+0000 7f8ce3e00640 1 -- 172.21.15.66:0/2433922692 learned_addr learned my addr 172.21.15.66:0/2433922692 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:41.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.045+0000 7f8ce8e00640 1 --2- 172.21.15.66:0/2433922692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc035930 0x7f8cdc049a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:41.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.045+0000 7f8ce3e00640 1 -- 172.21.15.66:0/2433922692 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8cdc055570 msgr2=0x7f8cdc0df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:41.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.045+0000 7f8ce3e00640 1 --2- 172.21.15.66:0/2433922692 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8cdc055570 0x7f8cdc0df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:41.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.049+0000 7f8ce3e00640 1 -- 172.21.15.66:0/2433922692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc035930 msgr2=0x7f8cdc049a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:41.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.049+0000 7f8ce3e00640 1 --2- 172.21.15.66:0/2433922692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc035930 0x7f8cdc049a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:41.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.049+0000 7f8ce3e00640 1 -- 172.21.15.66:0/2433922692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc047a40 con 0x7f8cdc035d10 2024-09-10T22:09:41.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.049+0000 7f8ce8e00640 1 --2- 172.21.15.66:0/2433922692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cdc035930 0x7f8cdc049a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:09:41.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.049+0000 7f8ce3e00640 1 --2- 172.21.15.66:0/2433922692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cdc035d10 0x7f8cdc0d30c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4066720 tx=0x7f8ce406ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.049+0000 7f8ce0c00640 1 -- 172.21.15.66:0/2433922692 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce408f070 con 0x7f8cdc035d10 2024-09-10T22:09:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.049+0000 7f8ce0c00640 1 -- 172.21.15.66:0/2433922692 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ce4074700 con 0x7f8cdc035d10 2024-09-10T22:09:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.049+0000 7f8ce0c00640 1 -- 172.21.15.66:0/2433922692 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4074b90 con 0x7f8cdc035d10 2024-09-10T22:09:41.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.049+0000 7f8cea200640 1 -- 172.21.15.66:0/2433922692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cdc047ce0 con 0x7f8cdc035d10 2024-09-10T22:09:41.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.049+0000 7f8cea200640 1 -- 172.21.15.66:0/2433922692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cdc04f400 con 0x7f8cdc035d10 2024-09-10T22:09:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.053+0000 7f8ce0c00640 1 -- 172.21.15.66:0/2433922692 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 31) ==== 98332+0+0 (secure 0 0 0) 0x7f8ce4094450 con 0x7f8cdc035d10 2024-09-10T22:09:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.057+0000 7f8cea200640 1 -- 172.21.15.66:0/2433922692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cdc033c80 con 0x7f8cdc035d10 2024-09-10T22:09:41.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.057+0000 7f8ce0c00640 1 --2- 172.21.15.66:0/2433922692 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8cc4073b40 0x7f8cc4076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:41.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.057+0000 7f8ce0c00640 1 -- 172.21.15.66:0/2433922692 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f8ce410d640 con 0x7f8cdc035d10 2024-09-10T22:09:41.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.057+0000 7f8ce8e00640 1 --2- 172.21.15.66:0/2433922692 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8cc4073b40 0x7f8cc4076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:41.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.061+0000 7f8ce8e00640 1 --2- 172.21.15.66:0/2433922692 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8cc4073b40 0x7f8cc4076000 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8006fd0 tx=0x7f8cd80090e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:41.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.061+0000 7f8ce0c00640 1 -- 172.21.15.66:0/2433922692 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ce410db10 con 0x7f8cdc035d10 2024-09-10T22:09:41.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:41.413+0000 7f8cea200640 1 -- 172.21.15.66:0/2433922692 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone51775", "target": ["mon-mgr", ""]}) -- 0x7f8cdc03a570 con 0x7f8cc4073b40 2024-09-10T22:09:42.071 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:42.071+0000 7f5fe8800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:42.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.073+0000 7f8ce0c00640 1 -- 172.21.15.66:0/2433922692 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f8cdc03a570 con 0x7f8cc4073b40 2024-09-10T22:09:42.078 INFO:teuthology.orchestra.run.smithi066.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:42.093 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T22:09:42.094 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone23716 2024-09-10T22:09:42.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.393+0000 7f99d4000640 1 -- 172.21.15.66:0/2656766450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99cc15bd40 msgr2=0x7f99cc0cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:42.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.393+0000 7f99d4000640 1 --2- 172.21.15.66:0/2656766450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99cc15bd40 0x7f99cc0cffa0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f99cc073290 tx=0x7f99c001c830 comp rx=0 tx=0).stop 2024-09-10T22:09:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.393+0000 7f99d4000640 1 -- 172.21.15.66:0/2656766450 shutdown_connections 2024-09-10T22:09:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.393+0000 7f99d4000640 1 --2- 172.21.15.66:0/2656766450 >> v2:172.21.15.94:3300/0 conn(0x7f99cc0aec10 0x7f99cc0af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.393+0000 7f99d4000640 1 --2- 172.21.15.66:0/2656766450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99cc0cf830 0x7f99cc0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.393+0000 7f99d4000640 1 --2- 172.21.15.66:0/2656766450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99cc15bd40 0x7f99cc0cffa0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.393+0000 7f99d4000640 1 -- 172.21.15.66:0/2656766450 >> 172.21.15.66:0/2656766450 conn(0x7f99cc0d1c60 msgr2=0x7f99cc0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.393+0000 7f99d4000640 1 -- 172.21.15.66:0/2656766450 shutdown_connections 2024-09-10T22:09:42.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.393+0000 7f99d4000640 1 -- 172.21.15.66:0/2656766450 wait complete. 2024-09-10T22:09:42.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.397+0000 7f99d4000640 1 Processor -- start 2024-09-10T22:09:42.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.397+0000 7f99d4000640 1 -- start start 2024-09-10T22:09:42.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.397+0000 7f99d4000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99cc0aec10 0x7f99cc0c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.397+0000 7f99d4000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99cc0cf830 0x7f99cc06ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.397+0000 7f99d4000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99cc15bd40 0x7f99cc06f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.397+0000 7f99d4000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc0cea80 con 0x7f99cc0cf830 2024-09-10T22:09:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.397+0000 7f99d4000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc0ce900 con 0x7f99cc15bd40 2024-09-10T22:09:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.397+0000 7f99d4000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f99cc0cec00 con 0x7f99cc0aec10 2024-09-10T22:09:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99d0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99cc0cf830 0x7f99cc06ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99d0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99cc0cf830 0x7f99cc06ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58588/0 (socket says 172.21.15.66:58588) 2024-09-10T22:09:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99d0c00640 1 -- 172.21.15.66:0/820211555 learned_addr learned my addr 172.21.15.66:0/820211555 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99d1600640 1 --2- 172.21.15.66:0/820211555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99cc0aec10 0x7f99cc0c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:42.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99d0c00640 1 -- 172.21.15.66:0/820211555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99cc0aec10 msgr2=0x7f99cc0c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:42.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99d0c00640 1 --2- 172.21.15.66:0/820211555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99cc0aec10 0x7f99cc0c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:42.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99d0c00640 1 -- 172.21.15.66:0/820211555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99cc15bd40 msgr2=0x7f99cc06f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:42.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99d0c00640 1 --2- 172.21.15.66:0/820211555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99cc15bd40 0x7f99cc06f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:42.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99d0c00640 1 -- 172.21.15.66:0/820211555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc15b0a0 con 0x7f99cc0cf830 2024-09-10T22:09:42.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99d0c00640 1 --2- 172.21.15.66:0/820211555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99cc0cf830 0x7f99cc06ed70 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f99b800d980 tx=0x7f99b800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:42.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99c9600640 1 -- 172.21.15.66:0/820211555 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8017070 con 0x7f99cc0cf830 2024-09-10T22:09:42.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99c9600640 1 -- 172.21.15.66:0/820211555 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99b8009d60 con 0x7f99cc0cf830 2024-09-10T22:09:42.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99c9600640 1 -- 172.21.15.66:0/820211555 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8005280 con 0x7f99cc0cf830 2024-09-10T22:09:42.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99d4000640 1 -- 172.21.15.66:0/820211555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99cc15b380 con 0x7f99cc0cf830 2024-09-10T22:09:42.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.401+0000 7f99d4000640 1 -- 172.21.15.66:0/820211555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99cc0ca830 con 0x7f99cc0cf830 2024-09-10T22:09:42.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.405+0000 7f99d4000640 1 -- 172.21.15.66:0/820211555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99cc0caa30 con 0x7f99cc0cf830 2024-09-10T22:09:42.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.405+0000 7f99c9600640 1 -- 172.21.15.66:0/820211555 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 31) ==== 98332+0+0 (secure 0 0 0) 0x7f99b8005420 con 0x7f99cc0cf830 2024-09-10T22:09:42.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.409+0000 7f99c9600640 1 --2- 172.21.15.66:0/820211555 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f99ac073b90 0x7f99ac076050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:42.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.409+0000 7f99c9600640 1 -- 172.21.15.66:0/820211555 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f99b8092520 con 0x7f99cc0cf830 2024-09-10T22:09:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.409+0000 7f99d1600640 1 --2- 172.21.15.66:0/820211555 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f99ac073b90 0x7f99ac076050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.413+0000 7f99d1600640 1 --2- 172.21.15.66:0/820211555 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f99ac073b90 0x7f99ac076050 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f99c00af000 tx=0x7f99c001b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.413+0000 7f99c9600640 1 -- 172.21.15.66:0/820211555 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f99cc0caa30 con 0x7f99cc0cf830 2024-09-10T22:09:42.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:42.821+0000 7f99d4000640 1 -- 172.21.15.66:0/820211555 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone23716", "target": ["mon-mgr", ""]}) -- 0x7f99cc0af010 con 0x7f99ac073b90 2024-09-10T22:09:43.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.065+0000 7f99c9600640 1 -- 172.21.15.66:0/820211555 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f99b805ae00 con 0x7f99cc0cf830 2024-09-10T22:09:43.553 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:43.547+0000 7f5fe8800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:43.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.561+0000 7f99c9600640 1 -- 172.21.15.66:0/820211555 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f99cc0af010 con 0x7f99ac073b90 2024-09-10T22:09:43.567 INFO:teuthology.orchestra.run.smithi066.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:43.589 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T22:09:43.589 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone5127 2024-09-10T22:09:43.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.889+0000 7f1b59e00640 1 -- 172.21.15.66:0/3873068499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b54158300 msgr2=0x7f1b5406fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:43.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.889+0000 7f1b59e00640 1 --2- 172.21.15.66:0/3873068499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b54158300 0x7f1b5406fd30 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009900 tx=0x7f1b4401c7f0 comp rx=0 tx=0).stop 2024-09-10T22:09:43.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.889+0000 7f1b59e00640 1 -- 172.21.15.66:0/3873068499 shutdown_connections 2024-09-10T22:09:43.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.889+0000 7f1b59e00640 1 --2- 172.21.15.66:0/3873068499 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b540ca450 0x7f1b54074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:43.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.889+0000 7f1b59e00640 1 --2- 172.21.15.66:0/3873068499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b54158300 0x7f1b5406fd30 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:43.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.889+0000 7f1b59e00640 1 -- 172.21.15.66:0/3873068499 >> 172.21.15.66:0/3873068499 conn(0x7f1b540cd4b0 msgr2=0x7f1b540cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:43.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.889+0000 7f1b59e00640 1 -- 172.21.15.66:0/3873068499 shutdown_connections 2024-09-10T22:09:43.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.889+0000 7f1b59e00640 1 -- 172.21.15.66:0/3873068499 wait complete. 2024-09-10T22:09:43.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.893+0000 7f1b59e00640 1 Processor -- start 2024-09-10T22:09:43.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.893+0000 7f1b59e00640 1 -- start start 2024-09-10T22:09:43.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.893+0000 7f1b59e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b54073fe0 0x7f1b54150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:43.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.893+0000 7f1b59e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b54158300 0x7f1b5414c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:43.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.893+0000 7f1b59e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b5414d360 0x7f1b5414d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:43.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.893+0000 7f1b59e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5416adf0 con 0x7f1b54073fe0 2024-09-10T22:09:43.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.893+0000 7f1b59e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5416ac70 con 0x7f1b54158300 2024-09-10T22:09:43.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.893+0000 7f1b59e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5416af70 con 0x7f1b5414d360 2024-09-10T22:09:43.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.893+0000 7f1b58a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b54073fe0 0x7f1b54150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:43.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.893+0000 7f1b58a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b54073fe0 0x7f1b54150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58608/0 (socket says 172.21.15.66:58608) 2024-09-10T22:09:43.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.893+0000 7f1b58a00640 1 -- 172.21.15.66:0/2250364088 learned_addr learned my addr 172.21.15.66:0/2250364088 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:43.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.893+0000 7f1b53e00640 1 --2- 172.21.15.66:0/2250364088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b54158300 0x7f1b5414c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:43.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b59400640 1 --2- 172.21.15.66:0/2250364088 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b5414d360 0x7f1b5414d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:43.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b58a00640 1 -- 172.21.15.66:0/2250364088 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b5414d360 msgr2=0x7f1b5414d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:43.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b58a00640 1 --2- 172.21.15.66:0/2250364088 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b5414d360 0x7f1b5414d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:43.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b58a00640 1 -- 172.21.15.66:0/2250364088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b54158300 msgr2=0x7f1b5414c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:43.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b58a00640 1 --2- 172.21.15.66:0/2250364088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b54158300 0x7f1b5414c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:43.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b58a00640 1 -- 172.21.15.66:0/2250364088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5414ef80 con 0x7f1b54073fe0 2024-09-10T22:09:43.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b53e00640 1 --2- 172.21.15.66:0/2250364088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b54158300 0x7f1b5414c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:09:43.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b58a00640 1 --2- 172.21.15.66:0/2250364088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b54073fe0 0x7f1b54150020 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1b4401af00 tx=0x7f1b44005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:43.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b50c00640 1 -- 172.21.15.66:0/2250364088 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b440b2040 con 0x7f1b54073fe0 2024-09-10T22:09:43.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b50c00640 1 -- 172.21.15.66:0/2250364088 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b44003600 con 0x7f1b54073fe0 2024-09-10T22:09:43.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b50c00640 1 -- 172.21.15.66:0/2250364088 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b440039f0 con 0x7f1b54073fe0 2024-09-10T22:09:43.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b59e00640 1 -- 172.21.15.66:0/2250364088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5414f260 con 0x7f1b54073fe0 2024-09-10T22:09:43.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.897+0000 7f1b59e00640 1 -- 172.21.15.66:0/2250364088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5414f820 con 0x7f1b54073fe0 2024-09-10T22:09:43.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.901+0000 7f1b59e00640 1 -- 172.21.15.66:0/2250364088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b300052a0 con 0x7f1b54073fe0 2024-09-10T22:09:43.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.905+0000 7f1b50c00640 1 -- 172.21.15.66:0/2250364088 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f1b440b5070 con 0x7f1b54073fe0 2024-09-10T22:09:43.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.909+0000 7f1b50c00640 1 --2- 172.21.15.66:0/2250364088 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b38073ab0 0x7f1b38075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:43.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.909+0000 7f1b50c00640 1 -- 172.21.15.66:0/2250364088 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f1b4412bb90 con 0x7f1b54073fe0 2024-09-10T22:09:43.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.913+0000 7f1b50c00640 1 -- 172.21.15.66:0/2250364088 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1b440164d0 con 0x7f1b54073fe0 2024-09-10T22:09:43.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.917+0000 7f1b53e00640 1 --2- 172.21.15.66:0/2250364088 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b38073ab0 0x7f1b38075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:43.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:43.917+0000 7f1b53e00640 1 --2- 172.21.15.66:0/2250364088 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b38073ab0 0x7f1b38075f70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1b541fb040 tx=0x7f1b48009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:44.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:44.365+0000 7f1b59e00640 1 -- 172.21.15.66:0/2250364088 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone5127", "target": ["mon-mgr", ""]}) -- 0x7f1b30000bd0 con 0x7f1b38073ab0 2024-09-10T22:09:44.690 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:44.687+0000 7f5fe8800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:44.685+0000 7f1b50c00640 1 -- 172.21.15.66:0/2250364088 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f1b30000bd0 con 0x7f1b38073ab0 2024-09-10T22:09:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:44.706 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T22:09:44.706 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone86448 2024-09-10T22:09:45.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.005+0000 7f7ad5e00640 1 -- 172.21.15.66:0/4228970588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ab8003a60 msgr2=0x7f7ab8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:45.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.005+0000 7f7ad5e00640 1 --2- 172.21.15.66:0/4228970588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ab8003a60 0x7f7ab8003e60 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400c260 tx=0x7f7ac401cf40 comp rx=0 tx=0).stop 2024-09-10T22:09:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.005+0000 7f7ad5e00640 1 -- 172.21.15.66:0/4228970588 shutdown_connections 2024-09-10T22:09:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.005+0000 7f7ad5e00640 1 --2- 172.21.15.66:0/4228970588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ab8003a60 0x7f7ab8003e60 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.005+0000 7f7ad5e00640 1 -- 172.21.15.66:0/4228970588 >> 172.21.15.66:0/4228970588 conn(0x7f7ad009aa50 msgr2=0x7f7ad009ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:45.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.009+0000 7f7ad5e00640 1 -- 172.21.15.66:0/4228970588 shutdown_connections 2024-09-10T22:09:45.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.009+0000 7f7ad5e00640 1 -- 172.21.15.66:0/4228970588 wait complete. 2024-09-10T22:09:45.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.009+0000 7f7ad5e00640 1 Processor -- start 2024-09-10T22:09:45.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.009+0000 7f7ad5e00640 1 -- start start 2024-09-10T22:09:45.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.013+0000 7f7ad5e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ab8003a60 0x7f7ad0156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:45.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.013+0000 7f7ad5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad015a620 0x7f7ad015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:45.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.013+0000 7f7ad5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ad00ce760 0x7f7ad00cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:45.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.013+0000 7f7ad5e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad00af190 con 0x7f7ad00ce760 2024-09-10T22:09:45.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.013+0000 7f7ad5e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad00af010 con 0x7f7ad015a620 2024-09-10T22:09:45.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.013+0000 7f7ad5e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad00af310 con 0x7f7ab8003a60 2024-09-10T22:09:45.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.013+0000 7f7ad5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ad00ce760 0x7f7ad00cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:45.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.013+0000 7f7ad5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ad00ce760 0x7f7ad00cebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58628/0 (socket says 172.21.15.66:58628) 2024-09-10T22:09:45.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.013+0000 7f7ad5400640 1 -- 172.21.15.66:0/3456104269 learned_addr learned my addr 172.21.15.66:0/3456104269 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:45.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.013+0000 7f7ad4a00640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ab8003a60 0x7f7ad0156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:45.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.017+0000 7f7ad5400640 1 -- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ab8003a60 msgr2=0x7f7ad0156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:45.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.017+0000 7f7ad5400640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ab8003a60 0x7f7ad0156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:45.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.017+0000 7f7ad5400640 1 -- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad015a620 msgr2=0x7f7ad015aa80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:09:45.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.017+0000 7f7ad5400640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad015a620 0x7f7ad015aa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:45.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.017+0000 7f7ad5400640 1 -- 172.21.15.66:0/3456104269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad006ce80 con 0x7f7ad00ce760 2024-09-10T22:09:45.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.017+0000 7f7ad5400640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ad00ce760 0x7f7ad00cebe0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00d2c80 tx=0x7f7ac800ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:45.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.021+0000 7f7accc00640 1 -- 172.21.15.66:0/3456104269 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8003db0 con 0x7f7ad00ce760 2024-09-10T22:09:45.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.021+0000 7f7accc00640 1 -- 172.21.15.66:0/3456104269 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ac8019070 con 0x7f7ad00ce760 2024-09-10T22:09:45.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.021+0000 7f7ad5e00640 1 -- 172.21.15.66:0/3456104269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad00c6240 con 0x7f7ad00ce760 2024-09-10T22:09:45.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.021+0000 7f7ad5e00640 1 -- 172.21.15.66:0/3456104269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad00c6800 con 0x7f7ad00ce760 2024-09-10T22:09:45.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.021+0000 7f7accc00640 1 -- 172.21.15.66:0/3456104269 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac800caf0 con 0x7f7ad00ce760 2024-09-10T22:09:45.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.021+0000 7f7ad5e00640 1 -- 172.21.15.66:0/3456104269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aac0052a0 con 0x7f7ad00ce760 2024-09-10T22:09:45.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.025+0000 7f7accc00640 1 -- 172.21.15.66:0/3456104269 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f7ac800cc90 con 0x7f7ad00ce760 2024-09-10T22:09:45.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.025+0000 7f7accc00640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ab0073ae0 0x7f7ab0075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:45.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.029+0000 7f7accc00640 1 -- 172.21.15.66:0/3456104269 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f7ac805a640 con 0x7f7ad00ce760 2024-09-10T22:09:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.029+0000 7f7ad4a00640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ab0073ae0 0x7f7ab0075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:45.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.037+0000 7f7ad4a00640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ab0073ae0 0x7f7ab0075fa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40ab000 tx=0x7f7ac40a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:45.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.037+0000 7f7accc00640 1 -- 172.21.15.66:0/3456104269 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7ac805ead0 con 0x7f7ad00ce760 2024-09-10T22:09:45.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:45.421+0000 7f7ad5e00640 1 -- 172.21.15.66:0/3456104269 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone86448", "target": ["mon-mgr", ""]}) -- 0x7f7aac000bd0 con 0x7f7ab0073ae0 2024-09-10T22:09:47.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.001+0000 7f7accc00640 1 -- 172.21.15.66:0/3456104269 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7aac000bd0 con 0x7f7ab0073ae0 2024-09-10T22:09:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.009+0000 7f7ad5e00640 1 -- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ab0073ae0 msgr2=0x7f7ab0075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.009+0000 7f7ad5e00640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ab0073ae0 0x7f7ab0075fa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40ab000 tx=0x7f7ac40a7040 comp rx=0 tx=0).stop 2024-09-10T22:09:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.009+0000 7f7ad5e00640 1 -- 172.21.15.66:0/3456104269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ad00ce760 msgr2=0x7f7ad00cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.009+0000 7f7ad5e00640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ad00ce760 0x7f7ad00cebe0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00d2c80 tx=0x7f7ac800ee10 comp rx=0 tx=0).stop 2024-09-10T22:09:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.009+0000 7f7ad5e00640 1 -- 172.21.15.66:0/3456104269 shutdown_connections 2024-09-10T22:09:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.009+0000 7f7ad5e00640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ab0073ae0 0x7f7ab0075fa0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.009+0000 7f7ad5e00640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ad00ce760 0x7f7ad00cebe0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.009+0000 7f7ad5e00640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad015a620 0x7f7ad015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:47.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.009+0000 7f7ad5e00640 1 --2- 172.21.15.66:0/3456104269 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ab8003a60 0x7f7ad0156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:47.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.009+0000 7f7ad5e00640 1 -- 172.21.15.66:0/3456104269 >> 172.21.15.66:0/3456104269 conn(0x7f7ad009aa50 msgr2=0x7f7ad006df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:47.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.009+0000 7f7ad5e00640 1 -- 172.21.15.66:0/3456104269 shutdown_connections 2024-09-10T22:09:47.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.009+0000 7f7ad5e00640 1 -- 172.21.15.66:0/3456104269 wait complete. 2024-09-10T22:09:47.054 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone86889 2024-09-10T22:09:47.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.209+0000 7fa9e6e00640 1 -- 172.21.15.66:0/112686330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e00afc20 msgr2=0x7fa9e00ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:47.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.209+0000 7fa9e6e00640 1 --2- 172.21.15.66:0/112686330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e00afc20 0x7fa9e00ce880 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009c80 tx=0x7fa9d801cd10 comp rx=0 tx=0).stop 2024-09-10T22:09:47.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.209+0000 7fa9e6e00640 1 -- 172.21.15.66:0/112686330 shutdown_connections 2024-09-10T22:09:47.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.209+0000 7fa9e6e00640 1 --2- 172.21.15.66:0/112686330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e00afc20 0x7fa9e00ce880 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:47.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.209+0000 7fa9e6e00640 1 -- 172.21.15.66:0/112686330 >> 172.21.15.66:0/112686330 conn(0x7fa9e00d1c60 msgr2=0x7fa9e00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:47.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.209+0000 7fa9e6e00640 1 -- 172.21.15.66:0/112686330 shutdown_connections 2024-09-10T22:09:47.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.209+0000 7fa9e6e00640 1 -- 172.21.15.66:0/112686330 wait complete. 2024-09-10T22:09:47.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.213+0000 7fa9e6e00640 1 Processor -- start 2024-09-10T22:09:47.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.213+0000 7fa9e6e00640 1 -- start start 2024-09-10T22:09:47.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e6e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9e00afc20 0x7fa9e0159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:47.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e6e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9e00d06b0 0x7fa9e0167020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:47.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e6e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e015bd40 0x7fa9e015da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:47.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e6e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e016a160 con 0x7fa9e015bd40 2024-09-10T22:09:47.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e6e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0169fe0 con 0x7fa9e00d06b0 2024-09-10T22:09:47.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e6e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e016a2e0 con 0x7fa9e00afc20 2024-09-10T22:09:47.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e015bd40 0x7fa9e015da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:47.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e015bd40 0x7fa9e015da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58648/0 (socket says 172.21.15.66:58648) 2024-09-10T22:09:47.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e4e00640 1 -- 172.21.15.66:0/280402573 learned_addr learned my addr 172.21.15.66:0/280402573 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e4e00640 1 -- 172.21.15.66:0/280402573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9e00afc20 msgr2=0x7fa9e0159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e4e00640 1 --2- 172.21.15.66:0/280402573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9e00afc20 0x7fa9e0159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e4e00640 1 -- 172.21.15.66:0/280402573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9e00d06b0 msgr2=0x7fa9e0167020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e4e00640 1 --2- 172.21.15.66:0/280402573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9e00d06b0 0x7fa9e0167020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e4e00640 1 -- 172.21.15.66:0/280402573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e00b4fc0 con 0x7fa9e015bd40 2024-09-10T22:09:47.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e4e00640 1 --2- 172.21.15.66:0/280402573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e015bd40 0x7fa9e015da30 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009bd0 tx=0x7fa9d80052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:47.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9d7e00640 1 -- 172.21.15.66:0/280402573 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80bc020 con 0x7fa9e015bd40 2024-09-10T22:09:47.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9d7e00640 1 -- 172.21.15.66:0/280402573 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9d8005800 con 0x7fa9e015bd40 2024-09-10T22:09:47.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9d7e00640 1 -- 172.21.15.66:0/280402573 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80b2040 con 0x7fa9e015bd40 2024-09-10T22:09:47.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e6e00640 1 -- 172.21.15.66:0/280402573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e00b52a0 con 0x7fa9e015bd40 2024-09-10T22:09:47.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.217+0000 7fa9e6e00640 1 -- 172.21.15.66:0/280402573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e00b5860 con 0x7fa9e015bd40 2024-09-10T22:09:47.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.221+0000 7fa9d7e00640 1 -- 172.21.15.66:0/280402573 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7fa9d80b5070 con 0x7fa9e015bd40 2024-09-10T22:09:47.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.221+0000 7fa9d7e00640 1 --2- 172.21.15.66:0/280402573 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa9c0073ab0 0x7fa9c0075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:47.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.225+0000 7fa9d7e00640 1 -- 172.21.15.66:0/280402573 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fa9d8002020 con 0x7fa9e015bd40 2024-09-10T22:09:47.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.225+0000 7fa9e6e00640 1 -- 172.21.15.66:0/280402573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b80052a0 con 0x7fa9e015bd40 2024-09-10T22:09:47.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.233+0000 7fa9dfe00640 1 --2- 172.21.15.66:0/280402573 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa9c0073ab0 0x7fa9c0075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:47.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.233+0000 7fa9d7e00640 1 -- 172.21.15.66:0/280402573 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa9d8016990 con 0x7fa9e015bd40 2024-09-10T22:09:47.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.237+0000 7fa9dfe00640 1 --2- 172.21.15.66:0/280402573 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa9c0073ab0 0x7fa9c0075f70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0000f70 tx=0x7fa9d00079c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:47.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.589+0000 7fa9e6e00640 1 -- 172.21.15.66:0/280402573 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone86889", "target": ["mon-mgr", ""]}) -- 0x7fa9b8000bd0 con 0x7fa9c0073ab0 2024-09-10T22:09:47.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:47.721+0000 7fa9d7e00640 1 -- 172.21.15.66:0/280402573 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fa9b8000bd0 con 0x7fa9c0073ab0 2024-09-10T22:09:47.730 INFO:teuthology.orchestra.run.smithi066.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:47.733 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:09:47.727+0000 7f5fe8800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-10T22:09:47.748 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T22:09:47.749 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone201510 2024-09-10T22:09:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.001+0000 7fe053e00640 1 -- 172.21.15.66:0/2780447641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe04c15ac80 msgr2=0x7fe04c15e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.001+0000 7fe053e00640 1 --2- 172.21.15.66:0/2780447641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe04c15ac80 0x7fe04c15e5e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe04c06ec50 tx=0x7fe04001c890 comp rx=0 tx=0).stop 2024-09-10T22:09:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.001+0000 7fe053e00640 1 -- 172.21.15.66:0/2780447641 shutdown_connections 2024-09-10T22:09:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.001+0000 7fe053e00640 1 --2- 172.21.15.66:0/2780447641 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe04c0cb620 0x7fe04c0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.001+0000 7fe053e00640 1 --2- 172.21.15.66:0/2780447641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe04c15ac80 0x7fe04c15e5e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.001+0000 7fe053e00640 1 --2- 172.21.15.66:0/2780447641 >> v2:172.21.15.94:3300/0 conn(0x7fe04c15a8a0 0x7fe04c0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.001+0000 7fe053e00640 1 -- 172.21.15.66:0/2780447641 >> 172.21.15.66:0/2780447641 conn(0x7fe04c0b26c0 msgr2=0x7fe04c0b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.001+0000 7fe053e00640 1 -- 172.21.15.66:0/2780447641 shutdown_connections 2024-09-10T22:09:48.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.001+0000 7fe053e00640 1 -- 172.21.15.66:0/2780447641 wait complete. 2024-09-10T22:09:48.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe053e00640 1 Processor -- start 2024-09-10T22:09:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe053e00640 1 -- start start 2024-09-10T22:09:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe053e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe04c0cb620 0x7fe04c167270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe053e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe04c15a8a0 0x7fe04c1522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe053e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe04c15ac80 0x7fe04c0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe053e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c0739c0 con 0x7fe04c0cb620 2024-09-10T22:09:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe053e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c073840 con 0x7fe04c15ac80 2024-09-10T22:09:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe053e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe04c073b40 con 0x7fe04c15a8a0 2024-09-10T22:09:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe051400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe04c0cb620 0x7fe04c167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe051400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe04c0cb620 0x7fe04c167270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58668/0 (socket says 172.21.15.66:58668) 2024-09-10T22:09:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe051400640 1 -- 172.21.15.66:0/242696663 learned_addr learned my addr 172.21.15.66:0/242696663 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:48.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe051e00640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe04c15ac80 0x7fe04c0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:48.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe051400640 1 -- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe04c15a8a0 msgr2=0x7fe04c1522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:48.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe051400640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe04c15a8a0 0x7fe04c1522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:48.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe051400640 1 -- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe04c15ac80 msgr2=0x7fe04c0ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:48.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe051400640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe04c15ac80 0x7fe04c0ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:48.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.005+0000 7fe051400640 1 -- 172.21.15.66:0/242696663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04c154730 con 0x7fe04c0cb620 2024-09-10T22:09:48.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.009+0000 7fe051400640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe04c0cb620 0x7fe04c167270 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b550 tx=0x7fe03800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:48.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.009+0000 7fe049600640 1 -- 172.21.15.66:0/242696663 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038014070 con 0x7fe04c0cb620 2024-09-10T22:09:48.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.009+0000 7fe049600640 1 -- 172.21.15.66:0/242696663 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe038010980 con 0x7fe04c0cb620 2024-09-10T22:09:48.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.009+0000 7fe049600640 1 -- 172.21.15.66:0/242696663 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038010dd0 con 0x7fe04c0cb620 2024-09-10T22:09:48.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.009+0000 7fe053e00640 1 -- 172.21.15.66:0/242696663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04c154170 con 0x7fe04c0cb620 2024-09-10T22:09:48.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.009+0000 7fe053e00640 1 -- 172.21.15.66:0/242696663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe04c166a90 con 0x7fe04c0cb620 2024-09-10T22:09:48.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.009+0000 7fe053e00640 1 -- 172.21.15.66:0/242696663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0280052a0 con 0x7fe04c0cb620 2024-09-10T22:09:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.013+0000 7fe049600640 1 -- 172.21.15.66:0/242696663 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7fe038023480 con 0x7fe04c0cb620 2024-09-10T22:09:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.013+0000 7fe049600640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe02c073b00 0x7fe02c075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.017+0000 7fe049600640 1 -- 172.21.15.66:0/242696663 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fe0380923d0 con 0x7fe04c0cb620 2024-09-10T22:09:48.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.021+0000 7fe049600640 1 -- 172.21.15.66:0/242696663 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe03805f540 con 0x7fe04c0cb620 2024-09-10T22:09:48.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.025+0000 7fe050a00640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe02c073b00 0x7fe02c075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:48.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.041+0000 7fe050a00640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe02c073b00 0x7fe02c075fc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe0400af000 tx=0x7fe040019630 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:48.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:48.349+0000 7fe053e00640 1 -- 172.21.15.66:0/242696663 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone201510", "target": ["mon-mgr", ""]}) -- 0x7fe028000bd0 con 0x7fe02c073b00 2024-09-10T22:09:50.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.157+0000 7fe049600640 1 -- 172.21.15.66:0/242696663 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe028000bd0 con 0x7fe02c073b00 2024-09-10T22:09:50.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.161+0000 7fe053e00640 1 -- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe02c073b00 msgr2=0x7fe02c075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:50.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.161+0000 7fe053e00640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe02c073b00 0x7fe02c075fc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe0400af000 tx=0x7fe040019630 comp rx=0 tx=0).stop 2024-09-10T22:09:50.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.161+0000 7fe053e00640 1 -- 172.21.15.66:0/242696663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe04c0cb620 msgr2=0x7fe04c167270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:50.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.161+0000 7fe053e00640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe04c0cb620 0x7fe04c167270 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b550 tx=0x7fe03800ba10 comp rx=0 tx=0).stop 2024-09-10T22:09:50.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.161+0000 7fe053e00640 1 -- 172.21.15.66:0/242696663 shutdown_connections 2024-09-10T22:09:50.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.161+0000 7fe053e00640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe02c073b00 0x7fe02c075fc0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:50.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.161+0000 7fe053e00640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe04c15ac80 0x7fe04c0ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:50.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.161+0000 7fe053e00640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe04c15a8a0 0x7fe04c1522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:50.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.161+0000 7fe053e00640 1 --2- 172.21.15.66:0/242696663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe04c0cb620 0x7fe04c167270 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:50.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.161+0000 7fe053e00640 1 -- 172.21.15.66:0/242696663 >> 172.21.15.66:0/242696663 conn(0x7fe04c0b26c0 msgr2=0x7fe04c157fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:50.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.161+0000 7fe053e00640 1 -- 172.21.15.66:0/242696663 shutdown_connections 2024-09-10T22:09:50.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.161+0000 7fe053e00640 1 -- 172.21.15.66:0/242696663 wait complete. 2024-09-10T22:09:50.247 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone403911 2024-09-10T22:09:50.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.577+0000 7fdb60000640 1 -- 172.21.15.66:0/1540778306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb5815bd40 msgr2=0x7fdb580cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:50.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.577+0000 7fdb60000640 1 --2- 172.21.15.66:0/1540778306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb5815bd40 0x7fdb580cffa0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009900 tx=0x7fdb4401c850 comp rx=0 tx=0).stop 2024-09-10T22:09:50.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.577+0000 7fdb60000640 1 -- 172.21.15.66:0/1540778306 shutdown_connections 2024-09-10T22:09:50.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.577+0000 7fdb60000640 1 --2- 172.21.15.66:0/1540778306 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb580cf830 0x7fdb580d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:50.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.577+0000 7fdb60000640 1 --2- 172.21.15.66:0/1540778306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb5815bd40 0x7fdb580cffa0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:50.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.577+0000 7fdb60000640 1 -- 172.21.15.66:0/1540778306 >> 172.21.15.66:0/1540778306 conn(0x7fdb580d1c60 msgr2=0x7fdb580d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:50.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.577+0000 7fdb60000640 1 -- 172.21.15.66:0/1540778306 shutdown_connections 2024-09-10T22:09:50.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.577+0000 7fdb60000640 1 -- 172.21.15.66:0/1540778306 wait complete. 2024-09-10T22:09:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb60000640 1 Processor -- start 2024-09-10T22:09:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb60000640 1 -- start start 2024-09-10T22:09:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb60000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb580aec10 0x7fdb581fb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb60000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb580cf830 0x7fdb580b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb60000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb5815bd40 0x7fdb581f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb60000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb581690c0 con 0x7fdb580cf830 2024-09-10T22:09:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb60000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58168f40 con 0x7fdb5815bd40 2024-09-10T22:09:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb60000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdb58169240 con 0x7fdb580aec10 2024-09-10T22:09:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb5cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb580cf830 0x7fdb580b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb5cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb580cf830 0x7fdb580b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58688/0 (socket says 172.21.15.66:58688) 2024-09-10T22:09:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb5cc00640 1 -- 172.21.15.66:0/1198770226 learned_addr learned my addr 172.21.15.66:0/1198770226 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb5d600640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb580aec10 0x7fdb581fb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.581+0000 7fdb5e000640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb5815bd40 0x7fdb581f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.585+0000 7fdb5cc00640 1 -- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb580aec10 msgr2=0x7fdb581fb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.585+0000 7fdb5cc00640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb580aec10 0x7fdb581fb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.585+0000 7fdb5cc00640 1 -- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb5815bd40 msgr2=0x7fdb581f8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.585+0000 7fdb5cc00640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb5815bd40 0x7fdb581f8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.585+0000 7fdb5cc00640 1 -- 172.21.15.66:0/1198770226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb580b6e10 con 0x7fdb580cf830 2024-09-10T22:09:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.585+0000 7fdb5cc00640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb580cf830 0x7fdb580b68d0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00b580 tx=0x7fdb4c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.585+0000 7fdb55600640 1 -- 172.21.15.66:0/1198770226 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c014070 con 0x7fdb580cf830 2024-09-10T22:09:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.585+0000 7fdb60000640 1 -- 172.21.15.66:0/1198770226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb581fbff0 con 0x7fdb580cf830 2024-09-10T22:09:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.585+0000 7fdb60000640 1 -- 172.21.15.66:0/1198770226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb581fc5b0 con 0x7fdb580cf830 2024-09-10T22:09:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.585+0000 7fdb55600640 1 -- 172.21.15.66:0/1198770226 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb4c010980 con 0x7fdb580cf830 2024-09-10T22:09:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.585+0000 7fdb55600640 1 -- 172.21.15.66:0/1198770226 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c010dd0 con 0x7fdb580cf830 2024-09-10T22:09:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.585+0000 7fdb60000640 1 -- 172.21.15.66:0/1198770226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb340052a0 con 0x7fdb580cf830 2024-09-10T22:09:50.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.589+0000 7fdb55600640 1 -- 172.21.15.66:0/1198770226 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7fdb4c00f3c0 con 0x7fdb580cf830 2024-09-10T22:09:50.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.589+0000 7fdb55600640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb3807c2c0 0x7fdb3807e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:50.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.589+0000 7fdb55600640 1 -- 172.21.15.66:0/1198770226 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fdb4c091eb0 con 0x7fdb580cf830 2024-09-10T22:09:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.593+0000 7fdb55600640 1 -- 172.21.15.66:0/1198770226 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdb4c05f190 con 0x7fdb580cf830 2024-09-10T22:09:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.629+0000 7fdb5d600640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb3807c2c0 0x7fdb3807e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:50.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.645+0000 7fdb5d600640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb3807c2c0 0x7fdb3807e780 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdb440af000 tx=0x7fdb4401b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:50.941+0000 7fdb60000640 1 -- 172.21.15.66:0/1198770226 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone403911", "target": ["mon-mgr", ""]}) -- 0x7fdb34000bd0 con 0x7fdb3807c2c0 2024-09-10T22:09:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.489+0000 7fdb55600640 1 -- 172.21.15.66:0/1198770226 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdb34000bd0 con 0x7fdb3807c2c0 2024-09-10T22:09:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.493+0000 7fdb60000640 1 -- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb3807c2c0 msgr2=0x7fdb3807e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.493+0000 7fdb60000640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb3807c2c0 0x7fdb3807e780 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdb440af000 tx=0x7fdb4401b090 comp rx=0 tx=0).stop 2024-09-10T22:09:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.493+0000 7fdb60000640 1 -- 172.21.15.66:0/1198770226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb580cf830 msgr2=0x7fdb580b68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.493+0000 7fdb60000640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb580cf830 0x7fdb580b68d0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00b580 tx=0x7fdb4c00ba40 comp rx=0 tx=0).stop 2024-09-10T22:09:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.493+0000 7fdb60000640 1 -- 172.21.15.66:0/1198770226 shutdown_connections 2024-09-10T22:09:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.493+0000 7fdb60000640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb3807c2c0 0x7fdb3807e780 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.493+0000 7fdb60000640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb5815bd40 0x7fdb581f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.493+0000 7fdb60000640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb580cf830 0x7fdb580b68d0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.493+0000 7fdb60000640 1 --2- 172.21.15.66:0/1198770226 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb580aec10 0x7fdb581fb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.493+0000 7fdb60000640 1 -- 172.21.15.66:0/1198770226 >> 172.21.15.66:0/1198770226 conn(0x7fdb580d1c60 msgr2=0x7fdb58071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.493+0000 7fdb60000640 1 -- 172.21.15.66:0/1198770226 shutdown_connections 2024-09-10T22:09:52.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.493+0000 7fdb60000640 1 -- 172.21.15.66:0/1198770226 wait complete. 2024-09-10T22:09:52.526 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone82612 2024-09-10T22:09:52.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.785+0000 7f5e3be00640 1 -- 172.21.15.66:0/2009112285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e3c06fde0 msgr2=0x7f5e3c06cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:52.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.785+0000 7f5e3be00640 1 --2- 172.21.15.66:0/2009112285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e3c06fde0 0x7f5e3c06cc20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c154740 tx=0x7f5e2401c830 comp rx=0 tx=0).stop 2024-09-10T22:09:52.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.789+0000 7f5e3be00640 1 -- 172.21.15.66:0/2009112285 shutdown_connections 2024-09-10T22:09:52.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.789+0000 7f5e3be00640 1 --2- 172.21.15.66:0/2009112285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e3c06fde0 0x7f5e3c06cc20 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:52.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.789+0000 7f5e3be00640 1 --2- 172.21.15.66:0/2009112285 >> v2:172.21.15.94:3300/0 conn(0x7f5e3c070580 0x7f5e3c0cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:52.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.789+0000 7f5e3be00640 1 -- 172.21.15.66:0/2009112285 >> 172.21.15.66:0/2009112285 conn(0x7f5e3c0cc3c0 msgr2=0x7f5e3c0cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:52.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.789+0000 7f5e3be00640 1 -- 172.21.15.66:0/2009112285 shutdown_connections 2024-09-10T22:09:52.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.793+0000 7f5e3be00640 1 -- 172.21.15.66:0/2009112285 wait complete. 2024-09-10T22:09:52.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.793+0000 7f5e3be00640 1 Processor -- start 2024-09-10T22:09:52.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.793+0000 7f5e3be00640 1 -- start start 2024-09-10T22:09:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e3c06fde0 0x7f5e3c157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e3c070580 0x7f5e3c142e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c150df0 0x7f5e3c1433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c167d80 con 0x7f5e3c06fde0 2024-09-10T22:09:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c06f4c0 con 0x7f5e3c150df0 2024-09-10T22:09:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3be00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3c167f00 con 0x7f5e3c070580 2024-09-10T22:09:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e3c06fde0 0x7f5e3c157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e3c06fde0 0x7f5e3c157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38572/0 (socket says 172.21.15.66:38572) 2024-09-10T22:09:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3aa00640 1 -- 172.21.15.66:0/4127922662 learned_addr learned my addr 172.21.15.66:0/4127922662 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3aa00640 1 -- 172.21.15.66:0/4127922662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e3c070580 msgr2=0x7f5e3c142e70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:09:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3aa00640 1 --2- 172.21.15.66:0/4127922662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e3c070580 0x7f5e3c142e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3aa00640 1 -- 172.21.15.66:0/4127922662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c150df0 msgr2=0x7f5e3c1433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3aa00640 1 --2- 172.21.15.66:0/4127922662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c150df0 0x7f5e3c1433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.797+0000 7f5e3aa00640 1 -- 172.21.15.66:0/4127922662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c1fad00 con 0x7f5e3c06fde0 2024-09-10T22:09:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.801+0000 7f5e3aa00640 1 --2- 172.21.15.66:0/4127922662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e3c06fde0 0x7f5e3c157f30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00ba50 tx=0x7f5e2c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.801+0000 7f5e32a00640 1 -- 172.21.15.66:0/4127922662 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c019070 con 0x7f5e3c06fde0 2024-09-10T22:09:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.801+0000 7f5e32a00640 1 -- 172.21.15.66:0/4127922662 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e2c00cd00 con 0x7f5e3c06fde0 2024-09-10T22:09:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.801+0000 7f5e32a00640 1 -- 172.21.15.66:0/4127922662 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0146a0 con 0x7f5e3c06fde0 2024-09-10T22:09:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.805+0000 7f5e3be00640 1 -- 172.21.15.66:0/4127922662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3c1fafe0 con 0x7f5e3c06fde0 2024-09-10T22:09:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.805+0000 7f5e3be00640 1 -- 172.21.15.66:0/4127922662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3c1fb5a0 con 0x7f5e3c06fde0 2024-09-10T22:09:52.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.809+0000 7f5e3be00640 1 -- 172.21.15.66:0/4127922662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e3c06cc20 con 0x7f5e3c06fde0 2024-09-10T22:09:52.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.809+0000 7f5e32a00640 1 -- 172.21.15.66:0/4127922662 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f5e2c014840 con 0x7f5e3c06fde0 2024-09-10T22:09:52.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.809+0000 7f5e32a00640 1 --2- 172.21.15.66:0/4127922662 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5e18073ab0 0x7f5e18075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:52.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.809+0000 7f5e32a00640 1 -- 172.21.15.66:0/4127922662 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f5e2c091ec0 con 0x7f5e3c06fde0 2024-09-10T22:09:52.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.817+0000 7f5e32a00640 1 -- 172.21.15.66:0/4127922662 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5e2c05ea00 con 0x7f5e3c06fde0 2024-09-10T22:09:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.825+0000 7f5e3a000640 1 --2- 172.21.15.66:0/4127922662 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5e18073ab0 0x7f5e18075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:52.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:52.829+0000 7f5e3a000640 1 --2- 172.21.15.66:0/4127922662 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5e18073ab0 0x7f5e18075f70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c1faac0 tx=0x7f5e34006c30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:53.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:53.145+0000 7f5e3be00640 1 -- 172.21.15.66:0/4127922662 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone82612", "target": ["mon-mgr", ""]}) -- 0x7f5e3c06fbc0 con 0x7f5e18073ab0 2024-09-10T22:09:54.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.205+0000 7f5e32a00640 1 -- 172.21.15.66:0/4127922662 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5e3c06fbc0 con 0x7f5e18073ab0 2024-09-10T22:09:54.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.205+0000 7f5e3be00640 1 -- 172.21.15.66:0/4127922662 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5e18073ab0 msgr2=0x7f5e18075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:54.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.205+0000 7f5e3be00640 1 --2- 172.21.15.66:0/4127922662 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5e18073ab0 0x7f5e18075f70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c1faac0 tx=0x7f5e34006c30 comp rx=0 tx=0).stop 2024-09-10T22:09:54.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.205+0000 7f5e3be00640 1 -- 172.21.15.66:0/4127922662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e3c06fde0 msgr2=0x7f5e3c157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:54.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.205+0000 7f5e3be00640 1 --2- 172.21.15.66:0/4127922662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e3c06fde0 0x7f5e3c157f30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00ba50 tx=0x7f5e2c00bf10 comp rx=0 tx=0).stop 2024-09-10T22:09:54.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.205+0000 7f5e3be00640 1 -- 172.21.15.66:0/4127922662 shutdown_connections 2024-09-10T22:09:54.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.205+0000 7f5e3be00640 1 --2- 172.21.15.66:0/4127922662 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5e18073ab0 0x7f5e18075f70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:54.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.205+0000 7f5e3be00640 1 --2- 172.21.15.66:0/4127922662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c150df0 0x7f5e3c1433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:54.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.205+0000 7f5e3be00640 1 --2- 172.21.15.66:0/4127922662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e3c070580 0x7f5e3c142e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:54.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.205+0000 7f5e3be00640 1 --2- 172.21.15.66:0/4127922662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e3c06fde0 0x7f5e3c157f30 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:54.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.205+0000 7f5e3be00640 1 -- 172.21.15.66:0/4127922662 >> 172.21.15.66:0/4127922662 conn(0x7f5e3c0cc3c0 msgr2=0x7f5e3c0cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:54.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.205+0000 7f5e3be00640 1 -- 172.21.15.66:0/4127922662 shutdown_connections 2024-09-10T22:09:54.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.209+0000 7f5e3be00640 1 -- 172.21.15.66:0/4127922662 wait complete. 2024-09-10T22:09:54.242 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone16013 2024-09-10T22:09:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.477+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3762270028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd44160490 msgr2=0x7fbd440d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.477+0000 7fbd4ae00640 1 --2- 172.21.15.66:0/3762270028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd44160490 0x7fbd440d0300 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbd44073670 tx=0x7fbd3c01f580 comp rx=0 tx=0).stop 2024-09-10T22:09:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.477+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3762270028 shutdown_connections 2024-09-10T22:09:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.477+0000 7fbd4ae00640 1 --2- 172.21.15.66:0/3762270028 >> v2:172.21.15.66:3300/0 conn(0x7fbd440af1e0 0x7fbd44166ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.477+0000 7fbd4ae00640 1 --2- 172.21.15.66:0/3762270028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd44160490 0x7fbd440d0300 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.477+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3762270028 >> 172.21.15.66:0/3762270028 conn(0x7fbd440d1d20 msgr2=0x7fbd440d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.477+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3762270028 shutdown_connections 2024-09-10T22:09:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.477+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3762270028 wait complete. 2024-09-10T22:09:54.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.477+0000 7fbd4ae00640 1 Processor -- start 2024-09-10T22:09:54.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.477+0000 7fbd4ae00640 1 -- start start 2024-09-10T22:09:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd4ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd440af1e0 0x7fbd441539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd4ae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd440b2a20 0x7fbd440b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd4ae00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd44160490 0x7fbd440b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd4ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbd440b3580 con 0x7fbd440af1e0 2024-09-10T22:09:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd4ae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd440b3400 con 0x7fbd440b2a20 2024-09-10T22:09:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd4ae00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbd440b11f0 con 0x7fbd44160490 2024-09-10T22:09:54.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd43e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd440af1e0 0x7fbd441539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:54.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd43e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd440af1e0 0x7fbd441539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38596/0 (socket says 172.21.15.66:38596) 2024-09-10T22:09:54.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd43e00640 1 -- 172.21.15.66:0/3488357017 learned_addr learned my addr 172.21.15.66:0/3488357017 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd43400640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd440b2a20 0x7fbd440b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd43e00640 1 -- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd44160490 msgr2=0x7fbd440b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd43e00640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd44160490 0x7fbd440b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd43e00640 1 -- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd440b2a20 msgr2=0x7fbd440b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd43e00640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd440b2a20 0x7fbd440b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd43e00640 1 -- 172.21.15.66:0/3488357017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd440c6420 con 0x7fbd440af1e0 2024-09-10T22:09:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd43e00640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd440af1e0 0x7fbd441539a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0b3040 tx=0x7fbd3c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd3be00640 1 -- 172.21.15.66:0/3488357017 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0bf070 con 0x7fbd440af1e0 2024-09-10T22:09:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd3be00640 1 -- 172.21.15.66:0/3488357017 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd3c004b40 con 0x7fbd440af1e0 2024-09-10T22:09:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd3be00640 1 -- 172.21.15.66:0/3488357017 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0b5040 con 0x7fbd440af1e0 2024-09-10T22:09:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3488357017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd440c6700 con 0x7fbd440af1e0 2024-09-10T22:09:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.481+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3488357017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd440c6cc0 con 0x7fbd440af1e0 2024-09-10T22:09:54.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.485+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3488357017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd440af630 con 0x7fbd440af1e0 2024-09-10T22:09:54.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.489+0000 7fbd3be00640 1 -- 172.21.15.66:0/3488357017 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7fbd3c0b8070 con 0x7fbd440af1e0 2024-09-10T22:09:54.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.489+0000 7fbd3be00640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd24073ab0 0x7fbd24075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:54.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.489+0000 7fbd3be00640 1 -- 172.21.15.66:0/3488357017 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fbd3c12ee70 con 0x7fbd440af1e0 2024-09-10T22:09:54.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.497+0000 7fbd3be00640 1 -- 172.21.15.66:0/3488357017 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbd3c0190e0 con 0x7fbd440af1e0 2024-09-10T22:09:54.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.501+0000 7fbd43400640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd24073ab0 0x7fbd24075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:54.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.509+0000 7fbd43400640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd24073ab0 0x7fbd24075f70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbd440d1100 tx=0x7fbd34009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:54.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:54.897+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3488357017 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone16013", "target": ["mon-mgr", ""]}) -- 0x7fbd44153ee0 con 0x7fbd24073ab0 2024-09-10T22:09:56.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.549+0000 7fbd3be00640 1 -- 172.21.15.66:0/3488357017 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbd44153ee0 con 0x7fbd24073ab0 2024-09-10T22:09:56.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.553+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd24073ab0 msgr2=0x7fbd24075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:56.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.553+0000 7fbd4ae00640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd24073ab0 0x7fbd24075f70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbd440d1100 tx=0x7fbd34009040 comp rx=0 tx=0).stop 2024-09-10T22:09:56.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.553+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3488357017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd440af1e0 msgr2=0x7fbd441539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:56.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.553+0000 7fbd4ae00640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd440af1e0 0x7fbd441539a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0b3040 tx=0x7fbd3c003f60 comp rx=0 tx=0).stop 2024-09-10T22:09:56.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.553+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3488357017 shutdown_connections 2024-09-10T22:09:56.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.553+0000 7fbd4ae00640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd24073ab0 0x7fbd24075f70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:56.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.553+0000 7fbd4ae00640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd44160490 0x7fbd440b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:56.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.553+0000 7fbd4ae00640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd440b2a20 0x7fbd440b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:56.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.553+0000 7fbd4ae00640 1 --2- 172.21.15.66:0/3488357017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd440af1e0 0x7fbd441539a0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:56.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.553+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3488357017 >> 172.21.15.66:0/3488357017 conn(0x7fbd440d1d20 msgr2=0x7fbd440aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:56.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.553+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3488357017 shutdown_connections 2024-09-10T22:09:56.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.553+0000 7fbd4ae00640 1 -- 172.21.15.66:0/3488357017 wait complete. 2024-09-10T22:09:56.586 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone344114 2024-09-10T22:09:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.817+0000 7f29e0800640 1 -- 172.21.15.66:0/3206707435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d815acd0 msgr2=0x7f29d80d0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.817+0000 7f29e0800640 1 --2- 172.21.15.66:0/3206707435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d815acd0 0x7f29d80d0550 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009840 tx=0x7f29cc01c820 comp rx=0 tx=0).stop 2024-09-10T22:09:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.821+0000 7f29e0800640 1 -- 172.21.15.66:0/3206707435 shutdown_connections 2024-09-10T22:09:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.821+0000 7f29e0800640 1 --2- 172.21.15.66:0/3206707435 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29d80c5a80 0x7f29d80cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.821+0000 7f29e0800640 1 --2- 172.21.15.66:0/3206707435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d815acd0 0x7f29d80d0550 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.821+0000 7f29e0800640 1 -- 172.21.15.66:0/3206707435 >> 172.21.15.66:0/3206707435 conn(0x7f29d80cfa30 msgr2=0x7f29d80cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.821+0000 7f29e0800640 1 -- 172.21.15.66:0/3206707435 shutdown_connections 2024-09-10T22:09:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.821+0000 7f29e0800640 1 -- 172.21.15.66:0/3206707435 wait complete. 2024-09-10T22:09:56.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.825+0000 7f29e0800640 1 Processor -- start 2024-09-10T22:09:56.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.825+0000 7f29e0800640 1 -- start start 2024-09-10T22:09:56.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.825+0000 7f29e0800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29d80c5a80 0x7f29d8167f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:56.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.825+0000 7f29e0800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d815acd0 0x7f29d81465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:56.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.825+0000 7f29e0800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29d8200880 0x7f29d8153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:56.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.825+0000 7f29e0800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29d806f720 con 0x7f29d815acd0 2024-09-10T22:09:56.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.825+0000 7f29e0800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29d806f5a0 con 0x7f29d8200880 2024-09-10T22:09:56.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.825+0000 7f29e0800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f29d806f8a0 con 0x7f29d80c5a80 2024-09-10T22:09:56.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.829+0000 7f29dde00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29d80c5a80 0x7f29d8167f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:56.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.829+0000 7f29dde00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29d80c5a80 0x7f29d8167f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:35320/0 (socket says 172.21.15.66:35320) 2024-09-10T22:09:56.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.829+0000 7f29dde00640 1 -- 172.21.15.66:0/2259862224 learned_addr learned my addr 172.21.15.66:0/2259862224 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:56.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.829+0000 7f29dd400640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d815acd0 0x7f29d81465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:56.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.833+0000 7f29dde00640 1 -- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29d8200880 msgr2=0x7f29d8153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:56.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.833+0000 7f29dde00640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29d8200880 0x7f29d8153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:56.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.833+0000 7f29dde00640 1 -- 172.21.15.66:0/2259862224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d815acd0 msgr2=0x7f29d81465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:56.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.833+0000 7f29dde00640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d815acd0 0x7f29d81465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:56.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.833+0000 7f29dde00640 1 -- 172.21.15.66:0/2259862224 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d8167730 con 0x7f29d80c5a80 2024-09-10T22:09:56.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.833+0000 7f29dde00640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29d80c5a80 0x7f29d8167f10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009790 tx=0x7f29cc0a56e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:56.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.837+0000 7f29d2000640 1 -- 172.21.15.66:0/2259862224 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc0b2040 con 0x7f29d80c5a80 2024-09-10T22:09:56.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.837+0000 7f29d2000640 1 -- 172.21.15.66:0/2259862224 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29cc005030 con 0x7f29d80c5a80 2024-09-10T22:09:56.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.837+0000 7f29d2000640 1 -- 172.21.15.66:0/2259862224 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc0044f0 con 0x7f29d80c5a80 2024-09-10T22:09:56.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.837+0000 7f29e0800640 1 -- 172.21.15.66:0/2259862224 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d8151b70 con 0x7f29d80c5a80 2024-09-10T22:09:56.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.837+0000 7f29e0800640 1 -- 172.21.15.66:0/2259862224 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d81679f0 con 0x7f29d80c5a80 2024-09-10T22:09:56.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.837+0000 7f29e0800640 1 -- 172.21.15.66:0/2259862224 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d80cd930 con 0x7f29d80c5a80 2024-09-10T22:09:56.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.841+0000 7f29d2000640 1 -- 172.21.15.66:0/2259862224 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f29cc0b5070 con 0x7f29d80c5a80 2024-09-10T22:09:56.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.845+0000 7f29d2000640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29bc073ab0 0x7f29bc075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:56.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.845+0000 7f29d2000640 1 -- 172.21.15.66:0/2259862224 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f29cc12b8e0 con 0x7f29d80c5a80 2024-09-10T22:09:56.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.849+0000 7f29dd400640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29bc073ab0 0x7f29bc075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:56.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.861+0000 7f29dd400640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29bc073ab0 0x7f29bc075f70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f29d81fd7b0 tx=0x7f29c8007ea0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:56.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:56.861+0000 7f29d2000640 1 -- 172.21.15.66:0/2259862224 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f29cc016410 con 0x7f29d80c5a80 2024-09-10T22:09:57.042 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:09:57.048 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:09:57.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:57.285+0000 7f29e0800640 1 -- 172.21.15.66:0/2259862224 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone344114", "target": ["mon-mgr", ""]}) -- 0x7f29d8075130 con 0x7f29bc073ab0 2024-09-10T22:09:58.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.385+0000 7f29d2000640 1 -- 172.21.15.66:0/2259862224 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f29c80014b0 con 0x7f29bc073ab0 2024-09-10T22:09:58.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.389+0000 7f29e0800640 1 -- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29bc073ab0 msgr2=0x7f29bc075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:58.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.389+0000 7f29e0800640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29bc073ab0 0x7f29bc075f70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f29d81fd7b0 tx=0x7f29c8007ea0 comp rx=0 tx=0).stop 2024-09-10T22:09:58.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.389+0000 7f29e0800640 1 -- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29d80c5a80 msgr2=0x7f29d8167f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:58.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.389+0000 7f29e0800640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29d80c5a80 0x7f29d8167f10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009790 tx=0x7f29cc0a56e0 comp rx=0 tx=0).stop 2024-09-10T22:09:58.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.389+0000 7f29e0800640 1 -- 172.21.15.66:0/2259862224 shutdown_connections 2024-09-10T22:09:58.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.389+0000 7f29e0800640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29bc073ab0 0x7f29bc075f70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:58.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.389+0000 7f29e0800640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29d8200880 0x7f29d8153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:58.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.389+0000 7f29e0800640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d815acd0 0x7f29d81465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:58.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.389+0000 7f29e0800640 1 --2- 172.21.15.66:0/2259862224 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29d80c5a80 0x7f29d8167f10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:58.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.389+0000 7f29e0800640 1 -- 172.21.15.66:0/2259862224 >> 172.21.15.66:0/2259862224 conn(0x7f29d80cfa30 msgr2=0x7f29d8073cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:58.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.389+0000 7f29e0800640 1 -- 172.21.15.66:0/2259862224 shutdown_connections 2024-09-10T22:09:58.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.389+0000 7f29e0800640 1 -- 172.21.15.66:0/2259862224 wait complete. 2024-09-10T22:09:58.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8415 2024-09-10T22:09:58.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.661+0000 7f01daa00640 1 -- 172.21.15.66:0/720186894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d4073f40 msgr2=0x7f01d40743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.661+0000 7f01daa00640 1 --2- 172.21.15.66:0/720186894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d4073f40 0x7f01d40743f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009c80 tx=0x7f01cc01cda0 comp rx=0 tx=0).stop 2024-09-10T22:09:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.661+0000 7f01daa00640 1 -- 172.21.15.66:0/720186894 shutdown_connections 2024-09-10T22:09:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.661+0000 7f01daa00640 1 --2- 172.21.15.66:0/720186894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d4073f40 0x7f01d40743f0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.661+0000 7f01daa00640 1 --2- 172.21.15.66:0/720186894 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01d4152b90 0x7f01d406f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.661+0000 7f01daa00640 1 -- 172.21.15.66:0/720186894 >> 172.21.15.66:0/720186894 conn(0x7f01d40b26c0 msgr2=0x7f01d40b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.661+0000 7f01daa00640 1 -- 172.21.15.66:0/720186894 shutdown_connections 2024-09-10T22:09:58.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.661+0000 7f01daa00640 1 -- 172.21.15.66:0/720186894 wait complete. 2024-09-10T22:09:58.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01daa00640 1 Processor -- start 2024-09-10T22:09:58.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01daa00640 1 -- start start 2024-09-10T22:09:58.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01daa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01d40737c0 0x7f01d40cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:58.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01daa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d4073f40 0x7f01d4150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:58.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01daa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d4152b90 0x7f01d4150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:58.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01daa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41689e0 con 0x7f01d4152b90 2024-09-10T22:09:58.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01daa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01d4168b60 con 0x7f01d4073f40 2024-09-10T22:09:58.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01daa00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f01d4168ce0 con 0x7f01d40737c0 2024-09-10T22:09:58.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01cbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d4073f40 0x7f01d4150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:58.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01cbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d4073f40 0x7f01d4150410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:47464/0 (socket says 172.21.15.66:47464) 2024-09-10T22:09:58.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01cbe00640 1 -- 172.21.15.66:0/1176559718 learned_addr learned my addr 172.21.15.66:0/1176559718 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:09:58.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01d3e00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01d40737c0 0x7f01d40cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:58.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01d8a00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d4152b90 0x7f01d4150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:58.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01cbe00640 1 -- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01d40737c0 msgr2=0x7f01d40cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:58.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01cbe00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01d40737c0 0x7f01d40cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:58.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01cbe00640 1 -- 172.21.15.66:0/1176559718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d4152b90 msgr2=0x7f01d4150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:58.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01cbe00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d4152b90 0x7f01d4150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:58.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01cbe00640 1 -- 172.21.15.66:0/1176559718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d4151930 con 0x7f01d4073f40 2024-09-10T22:09:58.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.665+0000 7f01d8a00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d4152b90 0x7f01d4150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:09:58.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.669+0000 7f01cbe00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d4073f40 0x7f01d4150410 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00c400 tx=0x7f01bc00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:58.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.669+0000 7f01d0c00640 1 -- 172.21.15.66:0/1176559718 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc017070 con 0x7f01d4073f40 2024-09-10T22:09:58.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.669+0000 7f01daa00640 1 -- 172.21.15.66:0/1176559718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d40c4660 con 0x7f01d4073f40 2024-09-10T22:09:58.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.669+0000 7f01daa00640 1 -- 172.21.15.66:0/1176559718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d40c4920 con 0x7f01d4073f40 2024-09-10T22:09:58.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.669+0000 7f01daa00640 1 -- 172.21.15.66:0/1176559718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d40743f0 con 0x7f01d4073f40 2024-09-10T22:09:58.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.673+0000 7f01d0c00640 1 -- 172.21.15.66:0/1176559718 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01bc0052d0 con 0x7f01d4073f40 2024-09-10T22:09:58.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.673+0000 7f01d0c00640 1 -- 172.21.15.66:0/1176559718 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc021060 con 0x7f01d4073f40 2024-09-10T22:09:58.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.673+0000 7f01d0c00640 1 -- 172.21.15.66:0/1176559718 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f01bc024080 con 0x7f01d4073f40 2024-09-10T22:09:58.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.677+0000 7f01d0c00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f01b0073ae0 0x7f01b0075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:58.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.677+0000 7f01d0c00640 1 -- 172.21.15.66:0/1176559718 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f01bc091ac0 con 0x7f01d4073f40 2024-09-10T22:09:58.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.677+0000 7f01d3e00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f01b0073ae0 0x7f01b0075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.681+0000 7f01d3e00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f01b0073ae0 0x7f01b0075fa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f01c40098f0 tx=0x7f01c4006f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:58.681+0000 7f01d0c00640 1 -- 172.21.15.66:0/1176559718 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f01bc05e5a0 con 0x7f01d4073f40 2024-09-10T22:09:59.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:09:59.049+0000 7f01daa00640 1 -- 172.21.15.66:0/1176559718 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8415", "target": ["mon-mgr", ""]}) -- 0x7f01d40d0b80 con 0x7f01b0073ae0 2024-09-10T22:10:00.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.441+0000 7f01d0c00640 1 -- 172.21.15.66:0/1176559718 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f01d40d0b80 con 0x7f01b0073ae0 2024-09-10T22:10:00.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.441+0000 7f01daa00640 1 -- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f01b0073ae0 msgr2=0x7f01b0075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:00.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.441+0000 7f01daa00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f01b0073ae0 0x7f01b0075fa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f01c40098f0 tx=0x7f01c4006f90 comp rx=0 tx=0).stop 2024-09-10T22:10:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.441+0000 7f01daa00640 1 -- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d4073f40 msgr2=0x7f01d4150410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.441+0000 7f01daa00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d4073f40 0x7f01d4150410 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00c400 tx=0x7f01bc00c8c0 comp rx=0 tx=0).stop 2024-09-10T22:10:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.445+0000 7f01daa00640 1 -- 172.21.15.66:0/1176559718 shutdown_connections 2024-09-10T22:10:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.445+0000 7f01daa00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f01b0073ae0 0x7f01b0075fa0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.445+0000 7f01daa00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d4152b90 0x7f01d4150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.445+0000 7f01daa00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d4073f40 0x7f01d4150410 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.445+0000 7f01daa00640 1 --2- 172.21.15.66:0/1176559718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01d40737c0 0x7f01d40cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.445+0000 7f01daa00640 1 -- 172.21.15.66:0/1176559718 >> 172.21.15.66:0/1176559718 conn(0x7f01d40b26c0 msgr2=0x7f01d4070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.445+0000 7f01daa00640 1 -- 172.21.15.66:0/1176559718 shutdown_connections 2024-09-10T22:10:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.445+0000 7f01daa00640 1 -- 172.21.15.66:0/1176559718 wait complete. 2024-09-10T22:10:00.477 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone302916 2024-09-10T22:10:00.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.745+0000 7f252bc00640 1 -- 172.21.15.66:0/1572729693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252415df10 msgr2=0x7f25241620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:00.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.745+0000 7f252bc00640 1 --2- 172.21.15.66:0/1572729693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252415df10 0x7f25241620a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f25240c6d60 tx=0x7f251c01ca60 comp rx=0 tx=0).stop 2024-09-10T22:10:00.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.745+0000 7f252bc00640 1 -- 172.21.15.66:0/1572729693 shutdown_connections 2024-09-10T22:10:00.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.745+0000 7f252bc00640 1 --2- 172.21.15.66:0/1572729693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252415df10 0x7f25241620a0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:00.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.745+0000 7f252bc00640 1 --2- 172.21.15.66:0/1572729693 >> v2:172.21.15.94:3300/0 conn(0x7f25240d1670 0x7f2524071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:00.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.745+0000 7f252bc00640 1 -- 172.21.15.66:0/1572729693 >> 172.21.15.66:0/1572729693 conn(0x7f25240d0b50 msgr2=0x7f25240d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:00.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.745+0000 7f252bc00640 1 -- 172.21.15.66:0/1572729693 shutdown_connections 2024-09-10T22:10:00.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.745+0000 7f252bc00640 1 -- 172.21.15.66:0/1572729693 wait complete. 2024-09-10T22:10:00.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.749+0000 7f252bc00640 1 Processor -- start 2024-09-10T22:10:00.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.749+0000 7f252bc00640 1 -- start start 2024-09-10T22:10:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.749+0000 7f252bc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25240d1670 0x7f25241406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.749+0000 7f252bc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f252415db30 0x7f2524153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.749+0000 7f252bc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252415df10 0x7f2524153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.749+0000 7f252bc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f252416aca0 con 0x7f252415df10 2024-09-10T22:10:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.749+0000 7f252bc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f252416ab20 con 0x7f25240d1670 2024-09-10T22:10:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.749+0000 7f252bc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f252416ae20 con 0x7f252415db30 2024-09-10T22:10:00.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f2529c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252415df10 0x7f2524153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:00.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f2529c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252415df10 0x7f2524153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38652/0 (socket says 172.21.15.66:38652) 2024-09-10T22:10:00.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f2529c00640 1 -- 172.21.15.66:0/296663544 learned_addr learned my addr 172.21.15.66:0/296663544 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:00.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f251be00640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f252415db30 0x7f2524153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:00.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f2529c00640 1 -- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f252415db30 msgr2=0x7f2524153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:00.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f2529c00640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f252415db30 0x7f2524153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:00.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f2529c00640 1 -- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25240d1670 msgr2=0x7f25241406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:00.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f2529c00640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25240d1670 0x7f25241406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:00.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f2529c00640 1 -- 172.21.15.66:0/296663544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252413fb90 con 0x7f252415df10 2024-09-10T22:10:00.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f251be00640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f252415db30 0x7f2524153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:00.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f2529c00640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252415df10 0x7f2524153760 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f251c01a690 tx=0x7f251c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:00.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f2521600640 1 -- 172.21.15.66:0/296663544 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c0b2040 con 0x7f252415df10 2024-09-10T22:10:00.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f2521600640 1 -- 172.21.15.66:0/296663544 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f251c004200 con 0x7f252415df10 2024-09-10T22:10:00.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f2521600640 1 -- 172.21.15.66:0/296663544 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c004610 con 0x7f252415df10 2024-09-10T22:10:00.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f252bc00640 1 -- 172.21.15.66:0/296663544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25240b05c0 con 0x7f252415df10 2024-09-10T22:10:00.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.753+0000 7f252bc00640 1 -- 172.21.15.66:0/296663544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2524140100 con 0x7f252415df10 2024-09-10T22:10:00.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.757+0000 7f252bc00640 1 -- 172.21.15.66:0/296663544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2524071220 con 0x7f252415df10 2024-09-10T22:10:00.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.757+0000 7f2521600640 1 -- 172.21.15.66:0/296663544 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f251c0b5070 con 0x7f252415df10 2024-09-10T22:10:00.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.761+0000 7f2521600640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24f8073ab0 0x7f24f8075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:00.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.761+0000 7f2521600640 1 -- 172.21.15.66:0/296663544 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f251c12ca70 con 0x7f252415df10 2024-09-10T22:10:00.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.765+0000 7f2521600640 1 -- 172.21.15.66:0/296663544 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f251c0c4060 con 0x7f252415df10 2024-09-10T22:10:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.773+0000 7f2529200640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24f8073ab0 0x7f24f8075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:00.781+0000 7f2529200640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24f8073ab0 0x7f24f8075f70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f25240ae6f0 tx=0x7f2508007a40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:01.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.129+0000 7f252bc00640 1 -- 172.21.15.66:0/296663544 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone302916", "target": ["mon-mgr", ""]}) -- 0x7f25240ceb10 con 0x7f24f8073ab0 2024-09-10T22:10:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.965+0000 7f2521600640 1 -- 172.21.15.66:0/296663544 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f25240ceb10 con 0x7f24f8073ab0 2024-09-10T22:10:01.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.969+0000 7f252bc00640 1 -- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24f8073ab0 msgr2=0x7f24f8075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:01.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.969+0000 7f252bc00640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24f8073ab0 0x7f24f8075f70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f25240ae6f0 tx=0x7f2508007a40 comp rx=0 tx=0).stop 2024-09-10T22:10:01.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.969+0000 7f252bc00640 1 -- 172.21.15.66:0/296663544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252415df10 msgr2=0x7f2524153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:01.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.969+0000 7f252bc00640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252415df10 0x7f2524153760 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f251c01a690 tx=0x7f251c009310 comp rx=0 tx=0).stop 2024-09-10T22:10:01.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.969+0000 7f252bc00640 1 -- 172.21.15.66:0/296663544 shutdown_connections 2024-09-10T22:10:01.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.969+0000 7f252bc00640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24f8073ab0 0x7f24f8075f70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:01.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.969+0000 7f252bc00640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252415df10 0x7f2524153760 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:01.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.969+0000 7f252bc00640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f252415db30 0x7f2524153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.969+0000 7f252bc00640 1 --2- 172.21.15.66:0/296663544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25240d1670 0x7f25241406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.969+0000 7f252bc00640 1 -- 172.21.15.66:0/296663544 >> 172.21.15.66:0/296663544 conn(0x7f25240d0b50 msgr2=0x7f25240cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.969+0000 7f252bc00640 1 -- 172.21.15.66:0/296663544 shutdown_connections 2024-09-10T22:10:01.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:01.969+0000 7f252bc00640 1 -- 172.21.15.66:0/296663544 wait complete. 2024-09-10T22:10:02.003 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone885517 2024-09-10T22:10:02.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.229+0000 7fb45a400640 1 -- 172.21.15.66:0/2098472494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb454159900 msgr2=0x7fb4540ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:02.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.229+0000 7fb45a400640 1 --2- 172.21.15.66:0/2098472494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb454159900 0x7fb4540ca080 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb4540701e0 tx=0x7fb43c01c830 comp rx=0 tx=0).stop 2024-09-10T22:10:02.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.229+0000 7fb45a400640 1 -- 172.21.15.66:0/2098472494 shutdown_connections 2024-09-10T22:10:02.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.229+0000 7fb45a400640 1 --2- 172.21.15.66:0/2098472494 >> v2:172.21.15.94:3300/0 conn(0x7fb4540aee20 0x7fb4540cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:02.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.229+0000 7fb45a400640 1 --2- 172.21.15.66:0/2098472494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb454159900 0x7fb4540ca080 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:02.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.229+0000 7fb45a400640 1 -- 172.21.15.66:0/2098472494 >> 172.21.15.66:0/2098472494 conn(0x7fb4540cfa30 msgr2=0x7fb4540cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:02.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.229+0000 7fb45a400640 1 -- 172.21.15.66:0/2098472494 shutdown_connections 2024-09-10T22:10:02.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.229+0000 7fb45a400640 1 -- 172.21.15.66:0/2098472494 wait complete. 2024-09-10T22:10:02.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb45a400640 1 Processor -- start 2024-09-10T22:10:02.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb45a400640 1 -- start start 2024-09-10T22:10:02.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb45a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4540aee20 0x7fb4541561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:02.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb45a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb454159900 0x7fb45415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:02.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb45a400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb45415cc80 0x7fb4541526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:02.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb45a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb454168f70 con 0x7fb4540aee20 2024-09-10T22:10:02.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb45a400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb454168df0 con 0x7fb454159900 2024-09-10T22:10:02.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb45a400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb4541690f0 con 0x7fb45415cc80 2024-09-10T22:10:02.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb453400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4540aee20 0x7fb4541561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:02.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb453400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4540aee20 0x7fb4541561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38672/0 (socket says 172.21.15.66:38672) 2024-09-10T22:10:02.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb453e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb45415cc80 0x7fb4541526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:02.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb453400640 1 -- 172.21.15.66:0/886165574 learned_addr learned my addr 172.21.15.66:0/886165574 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:02.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb452a00640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb454159900 0x7fb45415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:02.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb453400640 1 -- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb45415cc80 msgr2=0x7fb4541526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:02.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb453400640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb45415cc80 0x7fb4541526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:02.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb453400640 1 -- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb454159900 msgr2=0x7fb45415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:02.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb453400640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb454159900 0x7fb45415fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:02.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb453400640 1 -- 172.21.15.66:0/886165574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4540c68a0 con 0x7fb4540aee20 2024-09-10T22:10:02.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb452a00640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb454159900 0x7fb45415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:02.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.233+0000 7fb453400640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4540aee20 0x7fb4541561b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0b0040 tx=0x7fb43c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:02.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.237+0000 7fb44b400640 1 -- 172.21.15.66:0/886165574 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0bb070 con 0x7fb4540aee20 2024-09-10T22:10:02.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.237+0000 7fb44b400640 1 -- 172.21.15.66:0/886165574 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb43c0042b0 con 0x7fb4540aee20 2024-09-10T22:10:02.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.237+0000 7fb44b400640 1 -- 172.21.15.66:0/886165574 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c005280 con 0x7fb4540aee20 2024-09-10T22:10:02.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.237+0000 7fb45a400640 1 -- 172.21.15.66:0/886165574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4541fb4a0 con 0x7fb4540aee20 2024-09-10T22:10:02.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.237+0000 7fb45a400640 1 -- 172.21.15.66:0/886165574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4540aea00 con 0x7fb4540aee20 2024-09-10T22:10:02.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.237+0000 7fb45a400640 1 -- 172.21.15.66:0/886165574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4540aec40 con 0x7fb4540aee20 2024-09-10T22:10:02.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.241+0000 7fb44b400640 1 -- 172.21.15.66:0/886165574 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7fb43c005420 con 0x7fb4540aee20 2024-09-10T22:10:02.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.241+0000 7fb44b400640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb430073ab0 0x7fb430075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:02.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.241+0000 7fb44b400640 1 -- 172.21.15.66:0/886165574 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fb43c12d2f0 con 0x7fb4540aee20 2024-09-10T22:10:02.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.249+0000 7fb452a00640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb430073ab0 0x7fb430075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:02.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.253+0000 7fb44b400640 1 -- 172.21.15.66:0/886165574 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb4540aec40 con 0x7fb4540aee20 2024-09-10T22:10:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.253+0000 7fb452a00640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb430073ab0 0x7fb430075f70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb4541577c0 tx=0x7fb444009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:02.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:02.645+0000 7fb45a400640 1 -- 172.21.15.66:0/886165574 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone885517", "target": ["mon-mgr", ""]}) -- 0x7fb454070470 con 0x7fb430073ab0 2024-09-10T22:10:03.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.937+0000 7fb44b400640 1 -- 172.21.15.66:0/886165574 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb454070470 con 0x7fb430073ab0 2024-09-10T22:10:03.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.941+0000 7fb45a400640 1 -- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb430073ab0 msgr2=0x7fb430075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:03.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.941+0000 7fb45a400640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb430073ab0 0x7fb430075f70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb4541577c0 tx=0x7fb444009040 comp rx=0 tx=0).stop 2024-09-10T22:10:03.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.941+0000 7fb45a400640 1 -- 172.21.15.66:0/886165574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4540aee20 msgr2=0x7fb4541561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:03.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.941+0000 7fb45a400640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4540aee20 0x7fb4541561b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0b0040 tx=0x7fb43c005760 comp rx=0 tx=0).stop 2024-09-10T22:10:03.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.941+0000 7fb45a400640 1 -- 172.21.15.66:0/886165574 shutdown_connections 2024-09-10T22:10:03.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.941+0000 7fb45a400640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb430073ab0 0x7fb430075f70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:03.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.941+0000 7fb45a400640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb45415cc80 0x7fb4541526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:03.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.941+0000 7fb45a400640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb454159900 0x7fb45415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:03.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.941+0000 7fb45a400640 1 --2- 172.21.15.66:0/886165574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4540aee20 0x7fb4541561b0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:03.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.941+0000 7fb45a400640 1 -- 172.21.15.66:0/886165574 >> 172.21.15.66:0/886165574 conn(0x7fb4540cfa30 msgr2=0x7fb4540af470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:03.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.941+0000 7fb45a400640 1 -- 172.21.15.66:0/886165574 shutdown_connections 2024-09-10T22:10:03.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:03.941+0000 7fb45a400640 1 -- 172.21.15.66:0/886165574 wait complete. 2024-09-10T22:10:03.979 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone931018 2024-09-10T22:10:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.257+0000 7f588d800640 1 -- 172.21.15.66:0/1861348080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5888071820 msgr2=0x7f588815fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.257+0000 7f588d800640 1 --2- 172.21.15.66:0/1861348080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5888071820 0x7f588815fb20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5888072950 tx=0x7f588001cac0 comp rx=0 tx=0).stop 2024-09-10T22:10:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.257+0000 7f588d800640 1 -- 172.21.15.66:0/1861348080 shutdown_connections 2024-09-10T22:10:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.257+0000 7f588d800640 1 --2- 172.21.15.66:0/1861348080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5888071820 0x7f588815fb20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.257+0000 7f588d800640 1 --2- 172.21.15.66:0/1861348080 >> v2:172.21.15.66:3300/0 conn(0x7f58880af4e0 0x7f58880712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.257+0000 7f588d800640 1 --2- 172.21.15.66:0/1861348080 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f588815f530 0x7f58880d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.257+0000 7f588d800640 1 -- 172.21.15.66:0/1861348080 >> 172.21.15.66:0/1861348080 conn(0x7f58880d1d20 msgr2=0x7f58880d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:04.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.261+0000 7f588d800640 1 -- 172.21.15.66:0/1861348080 shutdown_connections 2024-09-10T22:10:04.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.261+0000 7f588d800640 1 -- 172.21.15.66:0/1861348080 wait complete. 2024-09-10T22:10:04.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.265+0000 7f588d800640 1 Processor -- start 2024-09-10T22:10:04.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.265+0000 7f588d800640 1 -- start start 2024-09-10T22:10:04.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.265+0000 7f588d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5888071820 0x7f58880c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:04.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.265+0000 7f588d800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58880af4e0 0x7f58880c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:04.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.265+0000 7f588d800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f588815f530 0x7f58880cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:04.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.265+0000 7f588d800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f58880d0430 con 0x7f5888071820 2024-09-10T22:10:04.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.265+0000 7f588d800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58880b1020 con 0x7f58880af4e0 2024-09-10T22:10:04.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.265+0000 7f588d800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f58880d05b0 con 0x7f588815f530 2024-09-10T22:10:04.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.269+0000 7f5886a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5888071820 0x7f58880c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:04.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.269+0000 7f5886a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5888071820 0x7f58880c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36816/0 (socket says 172.21.15.66:36816) 2024-09-10T22:10:04.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.269+0000 7f5886a00640 1 -- 172.21.15.66:0/2468432093 learned_addr learned my addr 172.21.15.66:0/2468432093 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:04.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.269+0000 7f5887400640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f588815f530 0x7f58880cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:04.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.269+0000 7f5886a00640 1 -- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f588815f530 msgr2=0x7f58880cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:04.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.269+0000 7f587e000640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58880af4e0 0x7f58880c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:04.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.269+0000 7f5886a00640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f588815f530 0x7f58880cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:04.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.269+0000 7f5886a00640 1 -- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58880af4e0 msgr2=0x7f58880c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:04.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.269+0000 7f5886a00640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58880af4e0 0x7f58880c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:04.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.273+0000 7f5886a00640 1 -- 172.21.15.66:0/2468432093 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58881f7b90 con 0x7f5888071820 2024-09-10T22:10:04.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.273+0000 7f5887400640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f588815f530 0x7f58880cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:04.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.273+0000 7f5886a00640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5888071820 0x7f58880c8df0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f587800e7d0 tx=0x7f587800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:04.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.273+0000 7f587e000640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58880af4e0 0x7f58880c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:04.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.273+0000 7f587f400640 1 -- 172.21.15.66:0/2468432093 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587800ccc0 con 0x7f5888071820 2024-09-10T22:10:04.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.273+0000 7f587f400640 1 -- 172.21.15.66:0/2468432093 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5878010430 con 0x7f5888071820 2024-09-10T22:10:04.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.273+0000 7f588d800640 1 -- 172.21.15.66:0/2468432093 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58881f7e70 con 0x7f5888071820 2024-09-10T22:10:04.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.273+0000 7f588d800640 1 -- 172.21.15.66:0/2468432093 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58881f8430 con 0x7f5888071820 2024-09-10T22:10:04.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.273+0000 7f587f400640 1 -- 172.21.15.66:0/2468432093 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878010840 con 0x7f5888071820 2024-09-10T22:10:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.273+0000 7f588d800640 1 -- 172.21.15.66:0/2468432093 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58600052a0 con 0x7f5888071820 2024-09-10T22:10:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.281+0000 7f587f400640 1 -- 172.21.15.66:0/2468432093 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f5878041080 con 0x7f5888071820 2024-09-10T22:10:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.285+0000 7f587f400640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5864073ab0 0x7f5864075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.285+0000 7f587f400640 1 -- 172.21.15.66:0/2468432093 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f5878093290 con 0x7f5888071820 2024-09-10T22:10:04.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.285+0000 7f587f400640 1 -- 172.21.15.66:0/2468432093 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f587805ffe0 con 0x7f5888071820 2024-09-10T22:10:04.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.289+0000 7f587e000640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5864073ab0 0x7f5864075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:04.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.297+0000 7f587e000640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5864073ab0 0x7f5864075f70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f58880b6110 tx=0x7f5870009260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:04.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:04.669+0000 7f588d800640 1 -- 172.21.15.66:0/2468432093 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone931018", "target": ["mon-mgr", ""]}) -- 0x7f5860000bd0 con 0x7f5864073ab0 2024-09-10T22:10:05.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.225+0000 7f587f400640 1 -- 172.21.15.66:0/2468432093 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5860000bd0 con 0x7f5864073ab0 2024-09-10T22:10:05.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.233+0000 7f588d800640 1 -- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5864073ab0 msgr2=0x7f5864075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:05.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.233+0000 7f588d800640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5864073ab0 0x7f5864075f70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f58880b6110 tx=0x7f5870009260 comp rx=0 tx=0).stop 2024-09-10T22:10:05.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.233+0000 7f588d800640 1 -- 172.21.15.66:0/2468432093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5888071820 msgr2=0x7f58880c8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:05.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.233+0000 7f588d800640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5888071820 0x7f58880c8df0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f587800e7d0 tx=0x7f587800ec90 comp rx=0 tx=0).stop 2024-09-10T22:10:05.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.233+0000 7f588d800640 1 -- 172.21.15.66:0/2468432093 shutdown_connections 2024-09-10T22:10:05.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.233+0000 7f588d800640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5864073ab0 0x7f5864075f70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:05.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.233+0000 7f588d800640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f588815f530 0x7f58880cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:05.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.233+0000 7f588d800640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58880af4e0 0x7f58880c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:05.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.233+0000 7f588d800640 1 --2- 172.21.15.66:0/2468432093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5888071820 0x7f58880c8df0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:05.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.233+0000 7f588d800640 1 -- 172.21.15.66:0/2468432093 >> 172.21.15.66:0/2468432093 conn(0x7f58880d1d20 msgr2=0x7f58880b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:05.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.237+0000 7f588d800640 1 -- 172.21.15.66:0/2468432093 shutdown_connections 2024-09-10T22:10:05.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.237+0000 7f588d800640 1 -- 172.21.15.66:0/2468432093 wait complete. 2024-09-10T22:10:05.267 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone533619 2024-09-10T22:10:05.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.493+0000 7f6a7c200640 1 -- 172.21.15.66:0/4235794077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a7415bd40 msgr2=0x7f6a740cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:05.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.493+0000 7f6a7c200640 1 --2- 172.21.15.66:0/4235794077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a7415bd40 0x7f6a740cffa0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009900 tx=0x7f6a6801c850 comp rx=0 tx=0).stop 2024-09-10T22:10:05.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.493+0000 7f6a7c200640 1 -- 172.21.15.66:0/4235794077 shutdown_connections 2024-09-10T22:10:05.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.493+0000 7f6a7c200640 1 --2- 172.21.15.66:0/4235794077 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a740cf830 0x7f6a740d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:05.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.493+0000 7f6a7c200640 1 --2- 172.21.15.66:0/4235794077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a7415bd40 0x7f6a740cffa0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:05.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.493+0000 7f6a7c200640 1 -- 172.21.15.66:0/4235794077 >> 172.21.15.66:0/4235794077 conn(0x7f6a740d1c60 msgr2=0x7f6a740d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:05.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.493+0000 7f6a7c200640 1 -- 172.21.15.66:0/4235794077 shutdown_connections 2024-09-10T22:10:05.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.493+0000 7f6a7c200640 1 -- 172.21.15.66:0/4235794077 wait complete. 2024-09-10T22:10:05.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.493+0000 7f6a7c200640 1 Processor -- start 2024-09-10T22:10:05.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.493+0000 7f6a7c200640 1 -- start start 2024-09-10T22:10:05.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a7c200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a740aec10 0x7f6a741f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:05.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a7c200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a740cf830 0x7f6a74157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:05.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a7c200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a7415bd40 0x7f6a740cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:05.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a7c200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74166f20 con 0x7f6a740aec10 2024-09-10T22:10:05.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a7c200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74166da0 con 0x7f6a740cf830 2024-09-10T22:10:05.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a7c200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6a741670a0 con 0x7f6a7415bd40 2024-09-10T22:10:05.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a78e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a740cf830 0x7f6a74157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:05.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a78e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a740cf830 0x7f6a74157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:39954/0 (socket says 172.21.15.66:39954) 2024-09-10T22:10:05.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a78e00640 1 -- 172.21.15.66:0/4229079393 learned_addr learned my addr 172.21.15.66:0/4229079393 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:05.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a7a200640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a7415bd40 0x7f6a740cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:05.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a79800640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a740aec10 0x7f6a741f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:05.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a78e00640 1 -- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a7415bd40 msgr2=0x7f6a740cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:05.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a78e00640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a7415bd40 0x7f6a740cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:05.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a78e00640 1 -- 172.21.15.66:0/4229079393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a740aec10 msgr2=0x7f6a741f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:05.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a78e00640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a740aec10 0x7f6a741f8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:05.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.497+0000 7f6a78e00640 1 -- 172.21.15.66:0/4229079393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a74154f10 con 0x7f6a740cf830 2024-09-10T22:10:05.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.501+0000 7f6a78e00640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a740cf830 0x7f6a74157bd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b550 tx=0x7f6a6000ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:05.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.501+0000 7f6a71600640 1 -- 172.21.15.66:0/4229079393 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60014070 con 0x7f6a740cf830 2024-09-10T22:10:05.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.501+0000 7f6a71600640 1 -- 172.21.15.66:0/4229079393 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a60010980 con 0x7f6a740cf830 2024-09-10T22:10:05.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.501+0000 7f6a71600640 1 -- 172.21.15.66:0/4229079393 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60010dd0 con 0x7f6a740cf830 2024-09-10T22:10:05.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.501+0000 7f6a7c200640 1 -- 172.21.15.66:0/4229079393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a741f9270 con 0x7f6a740cf830 2024-09-10T22:10:05.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.501+0000 7f6a7c200640 1 -- 172.21.15.66:0/4229079393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a74155490 con 0x7f6a740cf830 2024-09-10T22:10:05.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.501+0000 7f6a7c200640 1 -- 172.21.15.66:0/4229079393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a740af010 con 0x7f6a740cf830 2024-09-10T22:10:05.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.509+0000 7f6a71600640 1 -- 172.21.15.66:0/4229079393 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f6a60023480 con 0x7f6a740cf830 2024-09-10T22:10:05.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.509+0000 7f6a71600640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a54073ab0 0x7f6a54075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:05.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.509+0000 7f6a71600640 1 -- 172.21.15.66:0/4229079393 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f6a600924e0 con 0x7f6a740cf830 2024-09-10T22:10:05.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.513+0000 7f6a79800640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a54073ab0 0x7f6a54075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:05.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.513+0000 7f6a71600640 1 -- 172.21.15.66:0/4229079393 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6a6005f000 con 0x7f6a740cf830 2024-09-10T22:10:05.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.513+0000 7f6a79800640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a54073ab0 0x7f6a54075f70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6a680af000 tx=0x7f6a6801b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:05.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:05.845+0000 7f6a7c200640 1 -- 172.21.15.66:0/4229079393 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone533619", "target": ["mon-mgr", ""]}) -- 0x7f6a740aff30 con 0x7f6a54073ab0 2024-09-10T22:10:06.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a71600640 1 -- 172.21.15.66:0/4229079393 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a740aff30 con 0x7f6a54073ab0 2024-09-10T22:10:06.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a7c200640 1 -- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a54073ab0 msgr2=0x7f6a54075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:06.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a7c200640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a54073ab0 0x7f6a54075f70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6a680af000 tx=0x7f6a6801b090 comp rx=0 tx=0).stop 2024-09-10T22:10:06.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a7c200640 1 -- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a740cf830 msgr2=0x7f6a74157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:06.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a7c200640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a740cf830 0x7f6a74157bd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b550 tx=0x7f6a6000ba10 comp rx=0 tx=0).stop 2024-09-10T22:10:06.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a7c200640 1 -- 172.21.15.66:0/4229079393 shutdown_connections 2024-09-10T22:10:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a7c200640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a54073ab0 0x7f6a54075f70 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a7c200640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a7415bd40 0x7f6a740cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a7c200640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a740cf830 0x7f6a74157bd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a7c200640 1 --2- 172.21.15.66:0/4229079393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a740aec10 0x7f6a741f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a7c200640 1 -- 172.21.15.66:0/4229079393 >> 172.21.15.66:0/4229079393 conn(0x7f6a740d1c60 msgr2=0x7f6a740cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a7c200640 1 -- 172.21.15.66:0/4229079393 shutdown_connections 2024-09-10T22:10:06.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.037+0000 7f6a7c200640 1 -- 172.21.15.66:0/4229079393 wait complete. 2024-09-10T22:10:06.062 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:10:06.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.209+0000 7f09dfa00640 1 -- 172.21.15.66:0/118468420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09d80afc20 msgr2=0x7f09d80ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:06.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.209+0000 7f09dfa00640 1 --2- 172.21.15.66:0/118468420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09d80afc20 0x7f09d80ce6b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009c80 tx=0x7f09d001cd10 comp rx=0 tx=0).stop 2024-09-10T22:10:06.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.209+0000 7f09dfa00640 1 -- 172.21.15.66:0/118468420 shutdown_connections 2024-09-10T22:10:06.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.209+0000 7f09dfa00640 1 --2- 172.21.15.66:0/118468420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09d80afc20 0x7f09d80ce6b0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.209+0000 7f09dfa00640 1 --2- 172.21.15.66:0/118468420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09d80d06b0 0x7f09d80ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.209+0000 7f09dfa00640 1 -- 172.21.15.66:0/118468420 >> 172.21.15.66:0/118468420 conn(0x7f09d80d1c60 msgr2=0x7f09d80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:06.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.209+0000 7f09dfa00640 1 -- 172.21.15.66:0/118468420 shutdown_connections 2024-09-10T22:10:06.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.209+0000 7f09dfa00640 1 -- 172.21.15.66:0/118468420 wait complete. 2024-09-10T22:10:06.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.209+0000 7f09dfa00640 1 Processor -- start 2024-09-10T22:10:06.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.209+0000 7f09dfa00640 1 -- start start 2024-09-10T22:10:06.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.209+0000 7f09dfa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09d80afc20 0x7f09d80b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:06.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dfa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09d80d06b0 0x7f09d8158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:06.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dfa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d815bd40 0x7f09d80b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:06.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dfa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09d8169250 con 0x7f09d80afc20 2024-09-10T22:10:06.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dfa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f09d81690d0 con 0x7f09d815bd40 2024-09-10T22:10:06.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dfa00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f09d81693d0 con 0x7f09d80d06b0 2024-09-10T22:10:06.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09d80afc20 0x7f09d80b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:06.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09d80afc20 0x7f09d80b6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36854/0 (socket says 172.21.15.66:36854) 2024-09-10T22:10:06.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dd000640 1 -- 172.21.15.66:0/1806393095 learned_addr learned my addr 172.21.15.66:0/1806393095 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:06.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dda00640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d815bd40 0x7f09d80b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:06.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09d7e00640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09d80d06b0 0x7f09d8158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dd000640 1 -- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09d80d06b0 msgr2=0x7f09d8158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dd000640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09d80d06b0 0x7f09d8158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dd000640 1 -- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d815bd40 msgr2=0x7f09d80b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dd000640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d815bd40 0x7f09d80b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dd000640 1 -- 172.21.15.66:0/1806393095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d8158ed0 con 0x7f09d80afc20 2024-09-10T22:10:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dd000640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09d80afc20 0x7f09d80b6f90 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00c3c0 tx=0x7f09cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09d4c00640 1 -- 172.21.15.66:0/1806393095 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc017070 con 0x7f09d80afc20 2024-09-10T22:10:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09d4c00640 1 -- 172.21.15.66:0/1806393095 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09cc0053c0 con 0x7f09d80afc20 2024-09-10T22:10:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09d4c00640 1 -- 172.21.15.66:0/1806393095 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc003860 con 0x7f09d80afc20 2024-09-10T22:10:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dfa00640 1 -- 172.21.15.66:0/1806393095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d80b75c0 con 0x7f09d80afc20 2024-09-10T22:10:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dfa00640 1 -- 172.21.15.66:0/1806393095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d80b4f10 con 0x7f09d80afc20 2024-09-10T22:10:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09dfa00640 1 -- 172.21.15.66:0/1806393095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d80cffa0 con 0x7f09d80afc20 2024-09-10T22:10:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.213+0000 7f09d4c00640 1 -- 172.21.15.66:0/1806393095 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f09cc003a00 con 0x7f09d80afc20 2024-09-10T22:10:06.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.217+0000 7f09d4c00640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09b8073ab0 0x7f09b8075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:06.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.217+0000 7f09d4c00640 1 -- 172.21.15.66:0/1806393095 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f09cc091490 con 0x7f09d80afc20 2024-09-10T22:10:06.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.217+0000 7f09d7e00640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09b8073ab0 0x7f09b8075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:06.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.217+0000 7f09d7e00640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09b8073ab0 0x7f09b8075f70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f09d815b3f0 tx=0x7f09c400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:06.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.221+0000 7f09d4c00640 1 -- 172.21.15.66:0/1806393095 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f09cc05dfb0 con 0x7f09d80afc20 2024-09-10T22:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.413+0000 7f09dfa00640 1 -- 172.21.15.66:0/1806393095 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f09d8156240 con 0x7f09b8073ab0 2024-09-10T22:10:06.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09d4c00640 1 -- 172.21.15.66:0/1806393095 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f09d8156240 con 0x7f09b8073ab0 2024-09-10T22:10:06.418 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:10:06.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.419 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:10:06.419 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:06.419 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:10:06.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09dfa00640 1 -- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09b8073ab0 msgr2=0x7f09b8075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:06.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09dfa00640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09b8073ab0 0x7f09b8075f70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f09d815b3f0 tx=0x7f09c400b390 comp rx=0 tx=0).stop 2024-09-10T22:10:06.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09dfa00640 1 -- 172.21.15.66:0/1806393095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09d80afc20 msgr2=0x7f09d80b6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:06.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09dfa00640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09d80afc20 0x7f09d80b6f90 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00c3c0 tx=0x7f09cc00c880 comp rx=0 tx=0).stop 2024-09-10T22:10:06.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09dfa00640 1 -- 172.21.15.66:0/1806393095 shutdown_connections 2024-09-10T22:10:06.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09dfa00640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f09b8073ab0 0x7f09b8075f70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09dfa00640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d815bd40 0x7f09d80b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09dfa00640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09d80d06b0 0x7f09d8158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09dfa00640 1 --2- 172.21.15.66:0/1806393095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09d80afc20 0x7f09d80b6f90 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09dfa00640 1 -- 172.21.15.66:0/1806393095 >> 172.21.15.66:0/1806393095 conn(0x7f09d80d1c60 msgr2=0x7f09d81554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:06.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09dfa00640 1 -- 172.21.15.66:0/1806393095 shutdown_connections 2024-09-10T22:10:06.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.417+0000 7f09dfa00640 1 -- 172.21.15.66:0/1806393095 wait complete. 2024-09-10T22:10:06.440 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-10T22:10:06.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 -- 172.21.15.66:0/1282428437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce0416ac10 msgr2=0x7fce040c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:06.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 --2- 172.21.15.66:0/1282428437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce0416ac10 0x7fce040c48b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009840 tx=0x7fcdf401c820 comp rx=0 tx=0).stop 2024-09-10T22:10:06.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 -- 172.21.15.66:0/1282428437 shutdown_connections 2024-09-10T22:10:06.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 --2- 172.21.15.66:0/1282428437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce0416ac10 0x7fce040c48b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 -- 172.21.15.66:0/1282428437 >> 172.21.15.66:0/1282428437 conn(0x7fce040cfa30 msgr2=0x7fce040cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:06.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 -- 172.21.15.66:0/1282428437 shutdown_connections 2024-09-10T22:10:06.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 -- 172.21.15.66:0/1282428437 wait complete. 2024-09-10T22:10:06.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 Processor -- start 2024-09-10T22:10:06.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 -- start start 2024-09-10T22:10:06.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce040aee00 0x7fce0420cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:06.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce040cf610 0x7fce040ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:06.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0416ac10 0x7fce040b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:06.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce04179370 con 0x7fce0416ac10 2024-09-10T22:10:06.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce041791f0 con 0x7fce040cf610 2024-09-10T22:10:06.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce09400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fce041794f0 con 0x7fce040aee00 2024-09-10T22:10:06.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce03200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0416ac10 0x7fce040b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce03200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0416ac10 0x7fce040b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36864/0 (socket says 172.21.15.66:36864) 2024-09-10T22:10:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce03200640 1 -- 172.21.15.66:0/1617019459 learned_addr learned my addr 172.21.15.66:0/1617019459 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fcdfbe00640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce040cf610 0x7fce040ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.593+0000 7fce02800640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce040aee00 0x7fce0420cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fce03200640 1 -- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce040aee00 msgr2=0x7fce0420cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fce03200640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce040aee00 0x7fce0420cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fce03200640 1 -- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce040cf610 msgr2=0x7fce040ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fce03200640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce040cf610 0x7fce040ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fce03200640 1 -- 172.21.15.66:0/1617019459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0416a7a0 con 0x7fce0416ac10 2024-09-10T22:10:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fce03200640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0416ac10 0x7fce040b4950 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c730 tx=0x7fcdfc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fcdfaa00640 1 -- 172.21.15.66:0/1617019459 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc003db0 con 0x7fce0416ac10 2024-09-10T22:10:06.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fcdfaa00640 1 -- 172.21.15.66:0/1617019459 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdfc00ce40 con 0x7fce0416ac10 2024-09-10T22:10:06.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fce09400640 1 -- 172.21.15.66:0/1617019459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce04167a90 con 0x7fce0416ac10 2024-09-10T22:10:06.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fcdfaa00640 1 -- 172.21.15.66:0/1617019459 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc005290 con 0x7fce0416ac10 2024-09-10T22:10:06.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fce09400640 1 -- 172.21.15.66:0/1617019459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce04168050 con 0x7fce0416ac10 2024-09-10T22:10:06.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fce09400640 1 -- 172.21.15.66:0/1617019459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce04168250 con 0x7fce0416ac10 2024-09-10T22:10:06.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.597+0000 7fcdfaa00640 1 -- 172.21.15.66:0/1617019459 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7fcdfc005430 con 0x7fce0416ac10 2024-09-10T22:10:06.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.601+0000 7fcdfaa00640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcde0073ab0 0x7fcde0075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:06.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.601+0000 7fcdfaa00640 1 -- 172.21.15.66:0/1617019459 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fcdfc092030 con 0x7fce0416ac10 2024-09-10T22:10:06.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.601+0000 7fce02800640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcde0073ab0 0x7fcde0075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:06.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.601+0000 7fce02800640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcde0073ab0 0x7fcde0075f70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40af000 tx=0x7fcdf401b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:06.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.605+0000 7fcdfaa00640 1 -- 172.21.15.66:0/1617019459 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fce04168250 con 0x7fce0416ac10 2024-09-10T22:10:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.809+0000 7fce09400640 1 -- 172.21.15.66:0/1617019459 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce040cef00 con 0x7fcde0073ab0 2024-09-10T22:10:06.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.825+0000 7fcdfaa00640 1 -- 172.21.15.66:0/1617019459 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1021 (secure 0 0 0) 0x7fce040cef00 con 0x7fcde0073ab0 2024-09-10T22:10:06.828 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:06.828 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:10:06.828 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.828 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone86889" 2024-09-10T22:10:06.828 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.828 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.829 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone37214" 2024-09-10T22:10:06.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.829 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.829 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone23716" 2024-09-10T22:10:06.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.829 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.829 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sv1" 2024-09-10T22:10:06.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.829 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.829 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone201510" 2024-09-10T22:10:06.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.829 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone723" 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone94742" 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone33350" 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone8415" 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.830 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone82612" 2024-09-10T22:10:06.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.831 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.831 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone16013" 2024-09-10T22:10:06.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.831 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.831 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone885517" 2024-09-10T22:10:06.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.831 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.831 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone344114" 2024-09-10T22:10:06.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.831 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone533619" 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone51775" 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone403911" 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone302916" 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.832 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone5127" 2024-09-10T22:10:06.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.833 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.833 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone86448" 2024-09-10T22:10:06.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.833 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.833 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone75881" 2024-09-10T22:10:06.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:06.833 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:06.833 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone931018" 2024-09-10T22:10:06.833 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:06.833 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:10:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.829+0000 7fce09400640 1 -- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcde0073ab0 msgr2=0x7fcde0075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.829+0000 7fce09400640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcde0073ab0 0x7fcde0075f70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40af000 tx=0x7fcdf401b060 comp rx=0 tx=0).stop 2024-09-10T22:10:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.829+0000 7fce09400640 1 -- 172.21.15.66:0/1617019459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0416ac10 msgr2=0x7fce040b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.829+0000 7fce09400640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0416ac10 0x7fce040b4950 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c730 tx=0x7fcdfc00cbf0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.829+0000 7fce09400640 1 -- 172.21.15.66:0/1617019459 shutdown_connections 2024-09-10T22:10:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.829+0000 7fce09400640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcde0073ab0 0x7fcde0075f70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.829+0000 7fce09400640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce0416ac10 0x7fce040b4950 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.829+0000 7fce09400640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce040cf610 0x7fce040ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.829+0000 7fce09400640 1 --2- 172.21.15.66:0/1617019459 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce040aee00 0x7fce0420cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.829+0000 7fce09400640 1 -- 172.21.15.66:0/1617019459 >> 172.21.15.66:0/1617019459 conn(0x7fce040cfa30 msgr2=0x7fce040c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.829+0000 7fce09400640 1 -- 172.21.15.66:0/1617019459 shutdown_connections 2024-09-10T22:10:06.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:06.829+0000 7fce09400640 1 -- 172.21.15.66:0/1617019459 wait complete. 2024-09-10T22:10:06.850 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone86889 2024-09-10T22:10:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 -- 172.21.15.66:0/905994889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55c0004d50 msgr2=0x7f55c00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 --2- 172.21.15.66:0/905994889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55c0004d50 0x7f55c00051d0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c670 tx=0x7f55d401d2c0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1400640 1 -- 172.21.15.66:0/905994889 reap_dead start 2024-09-10T22:10:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 -- 172.21.15.66:0/905994889 shutdown_connections 2024-09-10T22:10:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 -- 172.21.15.66:0/905994889 >> 172.21.15.66:0/905994889 conn(0x7f55dc09d0a0 msgr2=0x7f55dc0d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:07.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 -- 172.21.15.66:0/905994889 shutdown_connections 2024-09-10T22:10:07.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 -- 172.21.15.66:0/905994889 wait complete. 2024-09-10T22:10:07.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 Processor -- start 2024-09-10T22:10:07.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 -- start start 2024-09-10T22:10:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55c0004d50 0x7f55dc1542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55dc0cc4a0 0x7f55dc0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55dc0b9440 0x7f55dc0cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc169ec0 con 0x7f55dc0b9440 2024-09-10T22:10:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc169d40 con 0x7f55c0004d50 2024-09-10T22:10:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f55dc16a040 con 0x7f55dc0cc4a0 2024-09-10T22:10:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55dc0b9440 0x7f55dc0cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55c0004d50 0x7f55dc1542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55d3e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55dc0cc4a0 0x7f55dc0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55d3e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55dc0cc4a0 0x7f55dc0ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:55158/0 (socket says 172.21.15.66:55158) 2024-09-10T22:10:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55e0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55c0004d50 0x7f55dc1542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:39982/0 (socket says 172.21.15.66:39982) 2024-09-10T22:10:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.001+0000 7f55d3e00640 1 -- 172.21.15.66:0/546090161 learned_addr learned my addr 172.21.15.66:0/546090161 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55d3e00640 1 -- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55c0004d50 msgr2=0x7f55dc1542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55d3e00640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55c0004d50 0x7f55dc1542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55d3e00640 1 -- 172.21.15.66:0/546090161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55dc0b9440 msgr2=0x7f55dc0cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55d3e00640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55dc0b9440 0x7f55dc0cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55d3e00640 1 -- 172.21.15.66:0/546090161 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc0b77d0 con 0x7f55dc0cc4a0 2024-09-10T22:10:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55e1400640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55dc0b9440 0x7f55dc0cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:10:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55d3e00640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55dc0cc4a0 0x7f55dc0ccc90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f55c4003f60 tx=0x7f55c4003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55d9600640 1 -- 172.21.15.66:0/546090161 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4015070 con 0x7f55dc0cc4a0 2024-09-10T22:10:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55d9600640 1 -- 172.21.15.66:0/546090161 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55c4011840 con 0x7f55dc0cc4a0 2024-09-10T22:10:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55e1e00640 1 -- 172.21.15.66:0/546090161 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc0b7ab0 con 0x7f55dc0cc4a0 2024-09-10T22:10:07.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55d9600640 1 -- 172.21.15.66:0/546090161 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4011c90 con 0x7f55dc0cc4a0 2024-09-10T22:10:07.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55e1e00640 1 -- 172.21.15.66:0/546090161 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc1549c0 con 0x7f55dc0cc4a0 2024-09-10T22:10:07.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55e1e00640 1 -- 172.21.15.66:0/546090161 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55dc0b1ad0 con 0x7f55dc0cc4a0 2024-09-10T22:10:07.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.005+0000 7f55d9600640 1 -- 172.21.15.66:0/546090161 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f55c40103c0 con 0x7f55dc0cc4a0 2024-09-10T22:10:07.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.009+0000 7f55d9600640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55b8073ab0 0x7f55b8075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:07.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.009+0000 7f55d9600640 1 -- 172.21.15.66:0/546090161 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f55c4092e30 con 0x7f55dc0cc4a0 2024-09-10T22:10:07.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.009+0000 7f55e0a00640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55b8073ab0 0x7f55b8075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:07.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.009+0000 7f55e0a00640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55b8073ab0 0x7f55b8075f70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009c60 tx=0x7f55cc0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.013+0000 7f55d9600640 1 -- 172.21.15.66:0/546090161 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f55c405f910 con 0x7f55dc0cc4a0 2024-09-10T22:10:07.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.213+0000 7f55e1e00640 1 -- 172.21.15.66:0/546090161 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone86889", "target": ["mon-mgr", ""]}) -- 0x7f55dc09d540 con 0x7f55b8073ab0 2024-09-10T22:10:07.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55d9600640 1 -- 172.21.15.66:0/546090161 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f55dc09d540 con 0x7f55b8073ab0 2024-09-10T22:10:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55e1e00640 1 -- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55b8073ab0 msgr2=0x7f55b8075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55e1e00640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55b8073ab0 0x7f55b8075f70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009c60 tx=0x7f55cc0091e0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55e1e00640 1 -- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55dc0cc4a0 msgr2=0x7f55dc0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55e1e00640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55dc0cc4a0 0x7f55dc0ccc90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f55c4003f60 tx=0x7f55c4003f90 comp rx=0 tx=0).stop 2024-09-10T22:10:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55e1e00640 1 -- 172.21.15.66:0/546090161 shutdown_connections 2024-09-10T22:10:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55e1e00640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55b8073ab0 0x7f55b8075f70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55e1e00640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55dc0b9440 0x7f55dc0cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55e1e00640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55dc0cc4a0 0x7f55dc0ccc90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55e1e00640 1 --2- 172.21.15.66:0/546090161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55c0004d50 0x7f55dc1542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55e1e00640 1 -- 172.21.15.66:0/546090161 >> 172.21.15.66:0/546090161 conn(0x7f55dc09d0a0 msgr2=0x7f55dc15d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55e1e00640 1 -- 172.21.15.66:0/546090161 shutdown_connections 2024-09-10T22:10:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.233+0000 7f55e1e00640 1 -- 172.21.15.66:0/546090161 wait complete. 2024-09-10T22:10:07.238 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:07.234+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:07.238 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:07.234+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:07.238 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:07.234+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:07.238 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:07.234+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:07.238 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:07.234+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:07.256 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:07.254+0000 7f5feba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:07.256 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:07.254+0000 7f5feba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:07.256 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:07.254+0000 7f5feba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:07.256 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:07.254+0000 7f5feba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:07.256 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:07.254+0000 7f5feba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:07.257 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone37214 2024-09-10T22:10:07.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.405+0000 7fc1c9600640 1 -- 172.21.15.66:0/2916670398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1c4145050 msgr2=0x7fc1c40ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.405+0000 7fc1c9600640 1 --2- 172.21.15.66:0/2916670398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1c4145050 0x7fc1c40ceb40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc1c406a620 tx=0x7fc1ac01c7e0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.405+0000 7fc1c9600640 1 -- 172.21.15.66:0/2916670398 shutdown_connections 2024-09-10T22:10:07.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.405+0000 7fc1c9600640 1 --2- 172.21.15.66:0/2916670398 >> v2:172.21.15.94:3300/0 conn(0x7fc1c40cf080 0x7fc1c4072640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.405+0000 7fc1c9600640 1 --2- 172.21.15.66:0/2916670398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1c4145050 0x7fc1c40ceb40 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.405+0000 7fc1c9600640 1 --2- 172.21.15.66:0/2916670398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1c4144ac0 0x7fc1c406e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.405+0000 7fc1c9600640 1 -- 172.21.15.66:0/2916670398 >> 172.21.15.66:0/2916670398 conn(0x7fc1c40cfaa0 msgr2=0x7fc1c40cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:07.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.405+0000 7fc1c9600640 1 -- 172.21.15.66:0/2916670398 shutdown_connections 2024-09-10T22:10:07.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.405+0000 7fc1c9600640 1 -- 172.21.15.66:0/2916670398 wait complete. 2024-09-10T22:10:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c9600640 1 Processor -- start 2024-09-10T22:10:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c9600640 1 -- start start 2024-09-10T22:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c9600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1c40cf080 0x7fc1c415b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c9600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1c4144ac0 0x7fc1c41424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c9600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1c4145050 0x7fc1c4142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c9600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c415fe70 con 0x7fc1c40cf080 2024-09-10T22:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c9600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c415fcf0 con 0x7fc1c4145050 2024-09-10T22:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c9600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c415fff0 con 0x7fc1c4144ac0 2024-09-10T22:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1c40cf080 0x7fc1c415b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1c40cf080 0x7fc1c415b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36900/0 (socket says 172.21.15.66:36900) 2024-09-10T22:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c2a00640 1 -- 172.21.15.66:0/1086298717 learned_addr learned my addr 172.21.15.66:0/1086298717 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1bbe00640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1c4144ac0 0x7fc1c41424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c3400640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1c4145050 0x7fc1c4142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:07.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c2a00640 1 -- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1c4144ac0 msgr2=0x7fc1c41424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c2a00640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1c4144ac0 0x7fc1c41424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c2a00640 1 -- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1c4145050 msgr2=0x7fc1c4142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c2a00640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1c4145050 0x7fc1c4142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c2a00640 1 -- 172.21.15.66:0/1086298717 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c4143040 con 0x7fc1c40cf080 2024-09-10T22:10:07.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1bbe00640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1c4144ac0 0x7fc1c41424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:10:07.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c2a00640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1c40cf080 0x7fc1c415b570 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400e940 tx=0x7fc1b400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:07.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1baa00640 1 -- 172.21.15.66:0/1086298717 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400cd80 con 0x7fc1c40cf080 2024-09-10T22:10:07.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1baa00640 1 -- 172.21.15.66:0/1086298717 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1b4005020 con 0x7fc1c40cf080 2024-09-10T22:10:07.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c9600640 1 -- 172.21.15.66:0/1086298717 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c4143320 con 0x7fc1c40cf080 2024-09-10T22:10:07.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1baa00640 1 -- 172.21.15.66:0/1086298717 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4005450 con 0x7fc1c40cf080 2024-09-10T22:10:07.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.409+0000 7fc1c9600640 1 -- 172.21.15.66:0/1086298717 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c41fba60 con 0x7fc1c40cf080 2024-09-10T22:10:07.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.413+0000 7fc1c9600640 1 -- 172.21.15.66:0/1086298717 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c4160510 con 0x7fc1c40cf080 2024-09-10T22:10:07.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.413+0000 7fc1baa00640 1 -- 172.21.15.66:0/1086298717 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7fc1b40047b0 con 0x7fc1c40cf080 2024-09-10T22:10:07.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.413+0000 7fc1baa00640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc1a0073ab0 0x7fc1a0075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:07.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.413+0000 7fc1baa00640 1 -- 172.21.15.66:0/1086298717 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fc1b4092ce0 con 0x7fc1c40cf080 2024-09-10T22:10:07.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.413+0000 7fc1bbe00640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc1a0073ab0 0x7fc1a0075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:07.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.413+0000 7fc1bbe00640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc1a0073ab0 0x7fc1a0075f70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0af000 tx=0x7fc1ac01b020 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:07.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.417+0000 7fc1baa00640 1 -- 172.21.15.66:0/1086298717 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc1b405f3e0 con 0x7fc1c40cf080 2024-09-10T22:10:07.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.617+0000 7fc1c9600640 1 -- 172.21.15.66:0/1086298717 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone37214", "target": ["mon-mgr", ""]}) -- 0x7fc1c4138000 con 0x7fc1a0073ab0 2024-09-10T22:10:07.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.637+0000 7fc1baa00640 1 -- 172.21.15.66:0/1086298717 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc1c4138000 con 0x7fc1a0073ab0 2024-09-10T22:10:07.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.637+0000 7fc1c9600640 1 -- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc1a0073ab0 msgr2=0x7fc1a0075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.637+0000 7fc1c9600640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc1a0073ab0 0x7fc1a0075f70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0af000 tx=0x7fc1ac01b020 comp rx=0 tx=0).stop 2024-09-10T22:10:07.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.637+0000 7fc1c9600640 1 -- 172.21.15.66:0/1086298717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1c40cf080 msgr2=0x7fc1c415b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.637+0000 7fc1c9600640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1c40cf080 0x7fc1c415b570 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400e940 tx=0x7fc1b400ee00 comp rx=0 tx=0).stop 2024-09-10T22:10:07.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.637+0000 7fc1c9600640 1 -- 172.21.15.66:0/1086298717 shutdown_connections 2024-09-10T22:10:07.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.637+0000 7fc1c9600640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc1a0073ab0 0x7fc1a0075f70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.637+0000 7fc1c9600640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1c4145050 0x7fc1c4142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.637+0000 7fc1c9600640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1c4144ac0 0x7fc1c41424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.637+0000 7fc1c9600640 1 --2- 172.21.15.66:0/1086298717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1c40cf080 0x7fc1c415b570 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.637+0000 7fc1c9600640 1 -- 172.21.15.66:0/1086298717 >> 172.21.15.66:0/1086298717 conn(0x7fc1c40cfaa0 msgr2=0x7fc1c41402a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:07.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.641+0000 7fc1c9600640 1 -- 172.21.15.66:0/1086298717 shutdown_connections 2024-09-10T22:10:07.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.641+0000 7fc1c9600640 1 -- 172.21.15.66:0/1086298717 wait complete. 2024-09-10T22:10:07.661 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone23716 2024-09-10T22:10:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.809+0000 7f68af600640 1 -- 172.21.15.66:0/4202458850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a815ab00 msgr2=0x7f68a806e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.809+0000 7f68af600640 1 --2- 172.21.15.66:0/4202458850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a815ab00 0x7f68a806e6a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f68a806e340 tx=0x7f689c01c770 comp rx=0 tx=0).stop 2024-09-10T22:10:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.809+0000 7f68af600640 1 -- 172.21.15.66:0/4202458850 shutdown_connections 2024-09-10T22:10:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.809+0000 7f68af600640 1 --2- 172.21.15.66:0/4202458850 >> v2:172.21.15.94:3300/0 conn(0x7f68a8072d50 0x7f68a80cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.809+0000 7f68af600640 1 --2- 172.21.15.66:0/4202458850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a815ab00 0x7f68a806e6a0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.809+0000 7f68af600640 1 -- 172.21.15.66:0/4202458850 >> 172.21.15.66:0/4202458850 conn(0x7f68a80cfaa0 msgr2=0x7f68a80cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.809+0000 7f68af600640 1 -- 172.21.15.66:0/4202458850 shutdown_connections 2024-09-10T22:10:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.809+0000 7f68af600640 1 -- 172.21.15.66:0/4202458850 wait complete. 2024-09-10T22:10:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.809+0000 7f68af600640 1 Processor -- start 2024-09-10T22:10:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.809+0000 7f68af600640 1 -- start start 2024-09-10T22:10:07.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68af600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a8072d50 0x7f68a814da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:07.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68af600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68a80cce70 0x7f68a81507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:07.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68af600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a815ab00 0x7f68a8150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:07.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68af600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8167bc0 con 0x7f68a815ab00 2024-09-10T22:10:07.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68af600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8167a40 con 0x7f68a8072d50 2024-09-10T22:10:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68af600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f68a8167d40 con 0x7f68a80cce70 2024-09-10T22:10:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68ad600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a815ab00 0x7f68a8150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68acc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a8072d50 0x7f68a814da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68ad600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a815ab00 0x7f68a8150d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36930/0 (socket says 172.21.15.66:36930) 2024-09-10T22:10:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68ad600640 1 -- 172.21.15.66:0/2788196021 learned_addr learned my addr 172.21.15.66:0/2788196021 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68a7e00640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68a80cce70 0x7f68a81507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:07.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68ad600640 1 -- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68a80cce70 msgr2=0x7f68a81507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68ad600640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68a80cce70 0x7f68a81507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68ad600640 1 -- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a8072d50 msgr2=0x7f68a814da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:07.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68ad600640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a8072d50 0x7f68a814da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:07.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68ad600640 1 -- 172.21.15.66:0/2788196021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a815ba30 con 0x7f68a815ab00 2024-09-10T22:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68acc00640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a8072d50 0x7f68a814da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68ad600640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a815ab00 0x7f68a8150d30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f68a000eab0 tx=0x7f68a000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68a4c00640 1 -- 172.21.15.66:0/2788196021 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a000ce40 con 0x7f68a815ab00 2024-09-10T22:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68a4c00640 1 -- 172.21.15.66:0/2788196021 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68a0004c20 con 0x7f68a815ab00 2024-09-10T22:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68a4c00640 1 -- 172.21.15.66:0/2788196021 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0005020 con 0x7f68a815ab00 2024-09-10T22:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68af600640 1 -- 172.21.15.66:0/2788196021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a815bcb0 con 0x7f68a815ab00 2024-09-10T22:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68af600640 1 -- 172.21.15.66:0/2788196021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a815c270 con 0x7f68a815ab00 2024-09-10T22:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68af600640 1 -- 172.21.15.66:0/2788196021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a80cbfa0 con 0x7f68a815ab00 2024-09-10T22:10:07.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.813+0000 7f68a4c00640 1 -- 172.21.15.66:0/2788196021 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f68a00051c0 con 0x7f68a815ab00 2024-09-10T22:10:07.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.817+0000 7f68a4c00640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6888073ab0 0x7f6888075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:07.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.817+0000 7f68a4c00640 1 -- 172.21.15.66:0/2788196021 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f68a0092f40 con 0x7f68a815ab00 2024-09-10T22:10:07.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.817+0000 7f68acc00640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6888073ab0 0x7f6888075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:07.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.817+0000 7f68acc00640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6888073ab0 0x7f6888075f70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f689c0af000 tx=0x7f689c01afb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:07.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:07.821+0000 7f68a4c00640 1 -- 172.21.15.66:0/2788196021 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f68a815ba30 con 0x7f68a815ab00 2024-09-10T22:10:08.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.021+0000 7f68af600640 1 -- 172.21.15.66:0/2788196021 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone23716", "target": ["mon-mgr", ""]}) -- 0x7f68a815a060 con 0x7f6888073ab0 2024-09-10T22:10:08.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.041+0000 7f68a4c00640 1 -- 172.21.15.66:0/2788196021 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f68a815a060 con 0x7f6888073ab0 2024-09-10T22:10:08.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.045+0000 7f68af600640 1 -- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6888073ab0 msgr2=0x7f6888075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:08.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.045+0000 7f68af600640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6888073ab0 0x7f6888075f70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f689c0af000 tx=0x7f689c01afb0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.045+0000 7f68af600640 1 -- 172.21.15.66:0/2788196021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a815ab00 msgr2=0x7f68a8150d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:08.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.045+0000 7f68af600640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a815ab00 0x7f68a8150d30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f68a000eab0 tx=0x7f68a000ef70 comp rx=0 tx=0).stop 2024-09-10T22:10:08.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.045+0000 7f68af600640 1 -- 172.21.15.66:0/2788196021 shutdown_connections 2024-09-10T22:10:08.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.045+0000 7f68af600640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6888073ab0 0x7f6888075f70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.045+0000 7f68af600640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a815ab00 0x7f68a8150d30 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.045+0000 7f68af600640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68a80cce70 0x7f68a81507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.045+0000 7f68af600640 1 --2- 172.21.15.66:0/2788196021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a8072d50 0x7f68a814da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.045+0000 7f68af600640 1 -- 172.21.15.66:0/2788196021 >> 172.21.15.66:0/2788196021 conn(0x7f68a80cfaa0 msgr2=0x7f68a814fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:08.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.045+0000 7f68af600640 1 -- 172.21.15.66:0/2788196021 shutdown_connections 2024-09-10T22:10:08.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.045+0000 7f68af600640 1 -- 172.21.15.66:0/2788196021 wait complete. 2024-09-10T22:10:08.067 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-10T22:10:08.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.213+0000 7fdf34c00640 1 -- 172.21.15.66:0/3813081838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf300b1d30 msgr2=0x7fdf30163c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:08.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.213+0000 7fdf34c00640 1 --2- 172.21.15.66:0/3813081838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf300b1d30 0x7fdf30163c90 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fdf300b0c30 tx=0x7fdf1801c8c0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.213+0000 7fdf34c00640 1 -- 172.21.15.66:0/3813081838 shutdown_connections 2024-09-10T22:10:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.213+0000 7fdf34c00640 1 --2- 172.21.15.66:0/3813081838 >> v2:172.21.15.94:3300/0 conn(0x7fdf300b2b10 0x7fdf301661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.213+0000 7fdf34c00640 1 --2- 172.21.15.66:0/3813081838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf300b1d30 0x7fdf30163c90 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.213+0000 7fdf34c00640 1 -- 172.21.15.66:0/3813081838 >> 172.21.15.66:0/3813081838 conn(0x7fdf300d1ce0 msgr2=0x7fdf300d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.213+0000 7fdf34c00640 1 -- 172.21.15.66:0/3813081838 shutdown_connections 2024-09-10T22:10:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.213+0000 7fdf34c00640 1 -- 172.21.15.66:0/3813081838 wait complete. 2024-09-10T22:10:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.213+0000 7fdf34c00640 1 Processor -- start 2024-09-10T22:10:08.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.213+0000 7fdf34c00640 1 -- start start 2024-09-10T22:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf34c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf300b1d30 0x7fdf300b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf34c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdf300b2b10 0x7fdf301f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf34c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf301605c0 0x7fdf301f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf34c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdf301690c0 con 0x7fdf301605c0 2024-09-10T22:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf34c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf30168f40 con 0x7fdf300b1d30 2024-09-10T22:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf34c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdf30169240 con 0x7fdf300b2b10 2024-09-10T22:10:08.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf2e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf301605c0 0x7fdf301f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:08.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf2de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf300b1d30 0x7fdf300b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:08.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf27e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdf300b2b10 0x7fdf301f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:08.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf27e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdf300b2b10 0x7fdf301f9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:55192/0 (socket says 172.21.15.66:55192) 2024-09-10T22:10:08.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf2e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf301605c0 0x7fdf301f5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36948/0 (socket says 172.21.15.66:36948) 2024-09-10T22:10:08.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf27e00640 1 -- 172.21.15.66:0/1435237074 learned_addr learned my addr 172.21.15.66:0/1435237074 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf2e800640 1 -- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdf300b2b10 msgr2=0x7fdf301f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf2e800640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdf300b2b10 0x7fdf301f9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf2e800640 1 -- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf300b1d30 msgr2=0x7fdf300b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf2e800640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf300b1d30 0x7fdf300b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf2e800640 1 -- 172.21.15.66:0/1435237074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf301f6790 con 0x7fdf301605c0 2024-09-10T22:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf2de00640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf300b1d30 0x7fdf300b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf27e00640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdf300b2b10 0x7fdf301f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf2e800640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf301605c0 0x7fdf301f5c90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800d9a0 tx=0x7fdf2800de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf26000640 1 -- 172.21.15.66:0/1435237074 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28017030 con 0x7fdf301605c0 2024-09-10T22:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf26000640 1 -- 172.21.15.66:0/1435237074 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf28004c20 con 0x7fdf301605c0 2024-09-10T22:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf26000640 1 -- 172.21.15.66:0/1435237074 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2800bb10 con 0x7fdf301605c0 2024-09-10T22:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf34c00640 1 -- 172.21.15.66:0/1435237074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf301f6a70 con 0x7fdf301605c0 2024-09-10T22:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf34c00640 1 -- 172.21.15.66:0/1435237074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf300b58a0 con 0x7fdf301605c0 2024-09-10T22:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.217+0000 7fdf34c00640 1 -- 172.21.15.66:0/1435237074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf300b1620 con 0x7fdf301605c0 2024-09-10T22:10:08.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.221+0000 7fdf26000640 1 -- 172.21.15.66:0/1435237074 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7fdf28005000 con 0x7fdf301605c0 2024-09-10T22:10:08.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.221+0000 7fdf26000640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdf0c073ab0 0x7fdf0c075f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:08.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.221+0000 7fdf26000640 1 -- 172.21.15.66:0/1435237074 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fdf28092390 con 0x7fdf301605c0 2024-09-10T22:10:08.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.221+0000 7fdf2de00640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdf0c073ab0 0x7fdf0c075f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:08.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.221+0000 7fdf2de00640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdf0c073ab0 0x7fdf0c075f70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdf300b7c10 tx=0x7fdf20009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:08.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.225+0000 7fdf26000640 1 -- 172.21.15.66:0/1435237074 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdf28021020 con 0x7fdf301605c0 2024-09-10T22:10:08.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.429+0000 7fdf34c00640 1 -- 172.21.15.66:0/1435237074 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf301fa0f0 con 0x7fdf0c073ab0 2024-09-10T22:10:08.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.437+0000 7fdf26000640 1 -- 172.21.15.66:0/1435237074 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdf20001c80 con 0x7fdf0c073ab0 2024-09-10T22:10:08.442 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:08.442 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:10:08.442 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:08.442 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ss1" 2024-09-10T22:10:08.442 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:08.442 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:10:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.441+0000 7fdf34c00640 1 -- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdf0c073ab0 msgr2=0x7fdf0c075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.441+0000 7fdf34c00640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdf0c073ab0 0x7fdf0c075f70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdf300b7c10 tx=0x7fdf20009040 comp rx=0 tx=0).stop 2024-09-10T22:10:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.441+0000 7fdf34c00640 1 -- 172.21.15.66:0/1435237074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf301605c0 msgr2=0x7fdf301f5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.441+0000 7fdf34c00640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf301605c0 0x7fdf301f5c90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800d9a0 tx=0x7fdf2800de60 comp rx=0 tx=0).stop 2024-09-10T22:10:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.441+0000 7fdf34c00640 1 -- 172.21.15.66:0/1435237074 shutdown_connections 2024-09-10T22:10:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.441+0000 7fdf34c00640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdf0c073ab0 0x7fdf0c075f70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.441+0000 7fdf34c00640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf301605c0 0x7fdf301f5c90 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.441+0000 7fdf34c00640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdf300b2b10 0x7fdf301f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.441+0000 7fdf34c00640 1 --2- 172.21.15.66:0/1435237074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf300b1d30 0x7fdf300b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.441+0000 7fdf34c00640 1 -- 172.21.15.66:0/1435237074 >> 172.21.15.66:0/1435237074 conn(0x7fdf300d1ce0 msgr2=0x7fdf300b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.441+0000 7fdf34c00640 1 -- 172.21.15.66:0/1435237074 shutdown_connections 2024-09-10T22:10:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.441+0000 7fdf34c00640 1 -- 172.21.15.66:0/1435237074 wait complete. 2024-09-10T22:10:08.463 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-10T22:10:08.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.613+0000 7f0a91400640 1 -- 172.21.15.66:0/1252092068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a8c15ef20 msgr2=0x7f0a8c0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:08.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.613+0000 7f0a91400640 1 --2- 172.21.15.66:0/1252092068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a8c15ef20 0x7f0a8c0cecc0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009840 tx=0x7f0a7c01c7b0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.613+0000 7f0a91400640 1 -- 172.21.15.66:0/1252092068 shutdown_connections 2024-09-10T22:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.613+0000 7f0a91400640 1 --2- 172.21.15.66:0/1252092068 >> v2:172.21.15.94:3300/0 conn(0x7f0a8c15f300 0x7f0a8c0b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.613+0000 7f0a91400640 1 --2- 172.21.15.66:0/1252092068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a8c15ef20 0x7f0a8c0cecc0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.613+0000 7f0a91400640 1 -- 172.21.15.66:0/1252092068 >> 172.21.15.66:0/1252092068 conn(0x7f0a8c0d1b10 msgr2=0x7f0a8c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.613+0000 7f0a91400640 1 -- 172.21.15.66:0/1252092068 shutdown_connections 2024-09-10T22:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.613+0000 7f0a91400640 1 -- 172.21.15.66:0/1252092068 wait complete. 2024-09-10T22:10:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.613+0000 7f0a91400640 1 Processor -- start 2024-09-10T22:10:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.613+0000 7f0a91400640 1 -- start start 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a91400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a8c0b15d0 0x7f0a8c09f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a91400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a8c15ef20 0x7f0a8c14d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a91400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a8c15f300 0x7f0a8c0cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a91400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c168d60 con 0x7f0a8c15f300 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a91400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c0ce0b0 con 0x7f0a8c15ef20 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a91400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8c168ee0 con 0x7f0a8c0b15d0 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a90a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a8c15f300 0x7f0a8c0cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a90a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a8c15f300 0x7f0a8c0cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36962/0 (socket says 172.21.15.66:36962) 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a90a00640 1 -- 172.21.15.66:0/2035330909 learned_addr learned my addr 172.21.15.66:0/2035330909 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a83400640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a8c15ef20 0x7f0a8c14d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a8be00640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a8c0b15d0 0x7f0a8c09f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a90a00640 1 -- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a8c0b15d0 msgr2=0x7f0a8c09f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a90a00640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a8c0b15d0 0x7f0a8c09f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a90a00640 1 -- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a8c15ef20 msgr2=0x7f0a8c14d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:08.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a90a00640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a8c15ef20 0x7f0a8c14d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a90a00640 1 -- 172.21.15.66:0/2035330909 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c1f8e50 con 0x7f0a8c15f300 2024-09-10T22:10:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a90a00640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a8c15f300 0x7f0a8c0cb2e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400cb30 tx=0x7f0a8400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a88c00640 1 -- 172.21.15.66:0/2035330909 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84016020 con 0x7f0a8c15f300 2024-09-10T22:10:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a88c00640 1 -- 172.21.15.66:0/2035330909 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a8400ab40 con 0x7f0a8c15f300 2024-09-10T22:10:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a88c00640 1 -- 172.21.15.66:0/2035330909 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84012420 con 0x7f0a8c15f300 2024-09-10T22:10:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a91400640 1 -- 172.21.15.66:0/2035330909 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8c14e140 con 0x7f0a8c15f300 2024-09-10T22:10:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a91400640 1 -- 172.21.15.66:0/2035330909 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8c0b4f20 con 0x7f0a8c15f300 2024-09-10T22:10:08.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a91400640 1 -- 172.21.15.66:0/2035330909 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a8c0cecc0 con 0x7f0a8c15f300 2024-09-10T22:10:08.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.617+0000 7f0a88c00640 1 -- 172.21.15.66:0/2035330909 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 32) ==== 98281+0+0 (secure 0 0 0) 0x7f0a840125c0 con 0x7f0a8c15f300 2024-09-10T22:10:08.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.621+0000 7f0a88c00640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0a68073ae0 0x7f0a68075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:08.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.621+0000 7f0a88c00640 1 -- 172.21.15.66:0/2035330909 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f0a84092390 con 0x7f0a8c15f300 2024-09-10T22:10:08.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.621+0000 7f0a8be00640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0a68073ae0 0x7f0a68075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:08.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.621+0000 7f0a8be00640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0a68073ae0 0x7f0a68075fa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0af000 tx=0x7f0a7c01aff0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:08.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.625+0000 7f0a88c00640 1 -- 172.21.15.66:0/2035330909 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0a84020020 con 0x7f0a8c15f300 2024-09-10T22:10:08.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.761+0000 7f0a88c00640 1 -- 172.21.15.66:0/2035330909 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f0a8405b0e0 con 0x7f0a8c15f300 2024-09-10T22:10:08.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.833+0000 7f0a91400640 1 -- 172.21.15.66:0/2035330909 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0a8c14e660 con 0x7f0a68073ae0 2024-09-10T22:10:08.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.857+0000 7f0a88c00640 1 -- 172.21.15.66:0/2035330909 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0a7c004140 con 0x7f0a68073ae0 2024-09-10T22:10:08.861 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:08.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.861+0000 7f0a91400640 1 -- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0a68073ae0 msgr2=0x7f0a68075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:08.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.861+0000 7f0a91400640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0a68073ae0 0x7f0a68075fa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0af000 tx=0x7f0a7c01aff0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.861+0000 7f0a91400640 1 -- 172.21.15.66:0/2035330909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a8c15f300 msgr2=0x7f0a8c0cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:08.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.861+0000 7f0a91400640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a8c15f300 0x7f0a8c0cb2e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400cb30 tx=0x7f0a8400cc20 comp rx=0 tx=0).stop 2024-09-10T22:10:08.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.861+0000 7f0a91400640 1 -- 172.21.15.66:0/2035330909 shutdown_connections 2024-09-10T22:10:08.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.861+0000 7f0a91400640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0a68073ae0 0x7f0a68075fa0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.861+0000 7f0a91400640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a8c15f300 0x7f0a8c0cb2e0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.861+0000 7f0a91400640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a8c15ef20 0x7f0a8c14d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.861+0000 7f0a91400640 1 --2- 172.21.15.66:0/2035330909 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a8c0b15d0 0x7f0a8c09f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:08.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.861+0000 7f0a91400640 1 -- 172.21.15.66:0/2035330909 >> 172.21.15.66:0/2035330909 conn(0x7f0a8c0d1b10 msgr2=0x7f0a8c06f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:08.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.861+0000 7f0a91400640 1 -- 172.21.15.66:0/2035330909 shutdown_connections 2024-09-10T22:10:08.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:08.861+0000 7f0a91400640 1 -- 172.21.15.66:0/2035330909 wait complete. 2024-09-10T22:10:08.883 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-10T22:10:09.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.029+0000 7f6f66600640 1 -- 172.21.15.66:0/3475599856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f6015a310 msgr2=0x7f6f600c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:09.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.029+0000 7f6f66600640 1 --2- 172.21.15.66:0/3475599856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f6015a310 0x7f6f600c48b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6f600af550 tx=0x7f6f5001daa0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.029+0000 7f6f66600640 1 -- 172.21.15.66:0/3475599856 shutdown_connections 2024-09-10T22:10:09.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.029+0000 7f6f66600640 1 --2- 172.21.15.66:0/3475599856 >> v2:172.21.15.94:3300/0 conn(0x7f6f600aeec0 0x7f6f600c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.029+0000 7f6f66600640 1 --2- 172.21.15.66:0/3475599856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f6015a310 0x7f6f600c48b0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.029+0000 7f6f66600640 1 -- 172.21.15.66:0/3475599856 >> 172.21.15.66:0/3475599856 conn(0x7f6f600cfa30 msgr2=0x7f6f600cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:09.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.029+0000 7f6f66600640 1 -- 172.21.15.66:0/3475599856 shutdown_connections 2024-09-10T22:10:09.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.029+0000 7f6f66600640 1 -- 172.21.15.66:0/3475599856 wait complete. 2024-09-10T22:10:09.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f66600640 1 Processor -- start 2024-09-10T22:10:09.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f66600640 1 -- start start 2024-09-10T22:10:09.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f66600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f600aeec0 0x7f6f6014e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:09.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f66600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6015a310 0x7f6f600b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:09.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f66600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f6013d740 0x7f6f600b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:09.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f66600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6015edb0 con 0x7f6f600aeec0 2024-09-10T22:10:09.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f66600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6015ec30 con 0x7f6f6015a310 2024-09-10T22:10:09.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f66600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6f60167300 con 0x7f6f6013d740 2024-09-10T22:10:09.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f5f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f600aeec0 0x7f6f6014e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:09.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f5f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f600aeec0 0x7f6f6014e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36970/0 (socket says 172.21.15.66:36970) 2024-09-10T22:10:09.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f5f400640 1 -- 172.21.15.66:0/2081015841 learned_addr learned my addr 172.21.15.66:0/2081015841 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:09.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f57e00640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6015a310 0x7f6f600b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:09.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f5fe00640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f6013d740 0x7f6f600b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:09.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f5f400640 1 -- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f6013d740 msgr2=0x7f6f600b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:09.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f5f400640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f6013d740 0x7f6f600b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f5f400640 1 -- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6015a310 msgr2=0x7f6f600b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:09.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f5f400640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6015a310 0x7f6f600b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f5f400640 1 -- 172.21.15.66:0/2081015841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f601fba30 con 0x7f6f600aeec0 2024-09-10T22:10:09.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f5f400640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f600aeec0 0x7f6f6014e110 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6f50000c10 tx=0x7f6f500040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:09.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f57400640 1 -- 172.21.15.66:0/2081015841 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f500bd070 con 0x7f6f600aeec0 2024-09-10T22:10:09.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f57400640 1 -- 172.21.15.66:0/2081015841 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f50005030 con 0x7f6f600aeec0 2024-09-10T22:10:09.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f57400640 1 -- 172.21.15.66:0/2081015841 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f500b3040 con 0x7f6f600aeec0 2024-09-10T22:10:09.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f66600640 1 -- 172.21.15.66:0/2081015841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f601fbd10 con 0x7f6f600aeec0 2024-09-10T22:10:09.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.033+0000 7f6f66600640 1 -- 172.21.15.66:0/2081015841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6015bfa0 con 0x7f6f600aeec0 2024-09-10T22:10:09.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.037+0000 7f6f66600640 1 -- 172.21.15.66:0/2081015841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6015df60 con 0x7f6f600aeec0 2024-09-10T22:10:09.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.037+0000 7f6f57400640 1 -- 172.21.15.66:0/2081015841 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f6f5000b040 con 0x7f6f600aeec0 2024-09-10T22:10:09.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.037+0000 7f6f57400640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f3c073bd0 0x7f6f3c076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:09.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.037+0000 7f6f57400640 1 -- 172.21.15.66:0/2081015841 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f6f5012b760 con 0x7f6f600aeec0 2024-09-10T22:10:09.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.037+0000 7f6f57e00640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f3c073bd0 0x7f6f3c076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:09.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.037+0000 7f6f57e00640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f3c073bd0 0x7f6f3c076090 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6f600744c0 tx=0x7f6f48007ea0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:09.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.041+0000 7f6f57400640 1 -- 172.21.15.66:0/2081015841 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6f50017720 con 0x7f6f600aeec0 2024-09-10T22:10:09.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.241+0000 7f6f66600640 1 -- 172.21.15.66:0/2081015841 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f6f60070420 con 0x7f6f3c073bd0 2024-09-10T22:10:09.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.257+0000 7f6f57400640 1 -- 172.21.15.66:0/2081015841 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6f60070420 con 0x7f6f3c073bd0 2024-09-10T22:10:09.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.261+0000 7f6f66600640 1 -- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f3c073bd0 msgr2=0x7f6f3c076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.261+0000 7f6f66600640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f3c073bd0 0x7f6f3c076090 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6f600744c0 tx=0x7f6f48007ea0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.261+0000 7f6f66600640 1 -- 172.21.15.66:0/2081015841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f600aeec0 msgr2=0x7f6f6014e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.261+0000 7f6f66600640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f600aeec0 0x7f6f6014e110 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6f50000c10 tx=0x7f6f500040e0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.261+0000 7f6f66600640 1 -- 172.21.15.66:0/2081015841 shutdown_connections 2024-09-10T22:10:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.261+0000 7f6f66600640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6f3c073bd0 0x7f6f3c076090 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.261+0000 7f6f66600640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f6013d740 0x7f6f600b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.261+0000 7f6f66600640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6015a310 0x7f6f600b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.261+0000 7f6f66600640 1 --2- 172.21.15.66:0/2081015841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f600aeec0 0x7f6f6014e110 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.261+0000 7f6f66600640 1 -- 172.21.15.66:0/2081015841 >> 172.21.15.66:0/2081015841 conn(0x7f6f600cfa30 msgr2=0x7f6f600ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.261+0000 7f6f66600640 1 -- 172.21.15.66:0/2081015841 shutdown_connections 2024-09-10T22:10:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.261+0000 7f6f66600640 1 -- 172.21.15.66:0/2081015841 wait complete. 2024-09-10T22:10:09.283 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone201510 2024-09-10T22:10:09.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.433+0000 7f3e08a00640 1 -- 172.21.15.66:0/1766037907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e040cd2c0 msgr2=0x7f3e0409dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:09.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.433+0000 7f3e08a00640 1 --2- 172.21.15.66:0/1766037907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e040cd2c0 0x7f3e0409dd90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009840 tx=0x7f3df001c820 comp rx=0 tx=0).stop 2024-09-10T22:10:09.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.433+0000 7f3e08a00640 1 -- 172.21.15.66:0/1766037907 shutdown_connections 2024-09-10T22:10:09.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.433+0000 7f3e08a00640 1 --2- 172.21.15.66:0/1766037907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e040cd2c0 0x7f3e0409dd90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.433+0000 7f3e08a00640 1 -- 172.21.15.66:0/1766037907 >> 172.21.15.66:0/1766037907 conn(0x7f3e040b1cd0 msgr2=0x7f3e040b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:09.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.433+0000 7f3e08a00640 1 -- 172.21.15.66:0/1766037907 shutdown_connections 2024-09-10T22:10:09.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.433+0000 7f3e08a00640 1 -- 172.21.15.66:0/1766037907 wait complete. 2024-09-10T22:10:09.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e08a00640 1 Processor -- start 2024-09-10T22:10:09.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e08a00640 1 -- start start 2024-09-10T22:10:09.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e08a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e04068b60 0x7f3e0414dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:09.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e08a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e040cd2c0 0x7f3e04156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:09.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e08a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e0415bba0 0x7f3e04157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:09.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e08a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04169c80 con 0x7f3e04068b60 2024-09-10T22:10:09.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e08a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04169b00 con 0x7f3e0415bba0 2024-09-10T22:10:09.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e08a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3e04169e00 con 0x7f3e040cd2c0 2024-09-10T22:10:09.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e03400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e04068b60 0x7f3e0414dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:09.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e03400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e04068b60 0x7f3e0414dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36980/0 (socket says 172.21.15.66:36980) 2024-09-10T22:10:09.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e03400640 1 -- 172.21.15.66:0/2992083155 learned_addr learned my addr 172.21.15.66:0/2992083155 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:09.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e02a00640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e040cd2c0 0x7f3e04156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:09.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e03e00640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e0415bba0 0x7f3e04157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:09.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e03400640 1 -- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e040cd2c0 msgr2=0x7f3e04156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:09.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e03400640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e040cd2c0 0x7f3e04156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e03400640 1 -- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e0415bba0 msgr2=0x7f3e04157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:09.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e03400640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e0415bba0 0x7f3e04157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e03400640 1 -- 172.21.15.66:0/2992083155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e040b5c90 con 0x7f3e04068b60 2024-09-10T22:10:09.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e03400640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e04068b60 0x7f3e0414dc50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3df400c460 tx=0x7f3df400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:09.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3dfb400640 1 -- 172.21.15.66:0/2992083155 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4016020 con 0x7f3e04068b60 2024-09-10T22:10:09.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e08a00640 1 -- 172.21.15.66:0/2992083155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e040b3170 con 0x7f3e04068b60 2024-09-10T22:10:09.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3dfb400640 1 -- 172.21.15.66:0/2992083155 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3df4012960 con 0x7f3e04068b60 2024-09-10T22:10:09.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3dfb400640 1 -- 172.21.15.66:0/2992083155 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4012db0 con 0x7f3e04068b60 2024-09-10T22:10:09.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.437+0000 7f3e08a00640 1 -- 172.21.15.66:0/2992083155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0414e190 con 0x7f3e04068b60 2024-09-10T22:10:09.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.441+0000 7f3e08a00640 1 -- 172.21.15.66:0/2992083155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0409dd90 con 0x7f3e04068b60 2024-09-10T22:10:09.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.441+0000 7f3dfb400640 1 -- 172.21.15.66:0/2992083155 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f3df401a3b0 con 0x7f3e04068b60 2024-09-10T22:10:09.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.441+0000 7f3dfb400640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3de4073bd0 0x7f3de4076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:09.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.441+0000 7f3dfb400640 1 -- 172.21.15.66:0/2992083155 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f3df4092270 con 0x7f3e04068b60 2024-09-10T22:10:09.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.441+0000 7f3e02a00640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3de4073bd0 0x7f3de4076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:09.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.441+0000 7f3e02a00640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3de4073bd0 0x7f3de4076090 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3df00af000 tx=0x7f3df001b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:09.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.445+0000 7f3dfb400640 1 -- 172.21.15.66:0/2992083155 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3df4020030 con 0x7f3e04068b60 2024-09-10T22:10:09.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.665+0000 7f3e08a00640 1 -- 172.21.15.66:0/2992083155 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone201510", "target": ["mon-mgr", ""]}) -- 0x7f3e040b5a40 con 0x7f3de4073bd0 2024-09-10T22:10:09.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.709+0000 7f3dfb400640 1 -- 172.21.15.66:0/2992083155 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3e040b5a40 con 0x7f3de4073bd0 2024-09-10T22:10:09.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.709+0000 7f3e08a00640 1 -- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3de4073bd0 msgr2=0x7f3de4076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:09.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.709+0000 7f3e08a00640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3de4073bd0 0x7f3de4076090 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3df00af000 tx=0x7f3df001b060 comp rx=0 tx=0).stop 2024-09-10T22:10:09.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.709+0000 7f3e08a00640 1 -- 172.21.15.66:0/2992083155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e04068b60 msgr2=0x7f3e0414dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:09.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.709+0000 7f3e08a00640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e04068b60 0x7f3e0414dc50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3df400c460 tx=0x7f3df400c920 comp rx=0 tx=0).stop 2024-09-10T22:10:09.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.709+0000 7f3e08a00640 1 -- 172.21.15.66:0/2992083155 shutdown_connections 2024-09-10T22:10:09.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.709+0000 7f3e08a00640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3de4073bd0 0x7f3de4076090 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.709+0000 7f3e08a00640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e0415bba0 0x7f3e04157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.709+0000 7f3e08a00640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e040cd2c0 0x7f3e04156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.709+0000 7f3e08a00640 1 --2- 172.21.15.66:0/2992083155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e04068b60 0x7f3e0414dc50 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:09.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.709+0000 7f3e08a00640 1 -- 172.21.15.66:0/2992083155 >> 172.21.15.66:0/2992083155 conn(0x7f3e040b1cd0 msgr2=0x7f3e040b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:09.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.713+0000 7f3e08a00640 1 -- 172.21.15.66:0/2992083155 shutdown_connections 2024-09-10T22:10:09.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:09.713+0000 7f3e08a00640 1 -- 172.21.15.66:0/2992083155 wait complete. 2024-09-10T22:10:09.749 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone723 2024-09-10T22:10:10.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.029+0000 7fd163600640 1 -- 172.21.15.66:0/2272114650 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd15c161440 msgr2=0x7fd15c0d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:10.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.029+0000 7fd163600640 1 --2- 172.21.15.66:0/2272114650 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd15c161440 0x7fd15c0d4970 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009900 tx=0x7fd14c01c8e0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.029+0000 7fd163600640 1 -- 172.21.15.66:0/2272114650 shutdown_connections 2024-09-10T22:10:10.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.029+0000 7fd163600640 1 --2- 172.21.15.66:0/2272114650 >> v2:172.21.15.66:3300/0 conn(0x7fd15c1619f0 0x7fd15c161e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.029+0000 7fd163600640 1 --2- 172.21.15.66:0/2272114650 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd15c161440 0x7fd15c0d4970 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.029+0000 7fd163600640 1 -- 172.21.15.66:0/2272114650 >> 172.21.15.66:0/2272114650 conn(0x7fd15c0d3e50 msgr2=0x7fd15c0d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:10.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.029+0000 7fd163600640 1 -- 172.21.15.66:0/2272114650 shutdown_connections 2024-09-10T22:10:10.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.029+0000 7fd163600640 1 -- 172.21.15.66:0/2272114650 wait complete. 2024-09-10T22:10:10.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.033+0000 7fd163600640 1 Processor -- start 2024-09-10T22:10:10.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.033+0000 7fd163600640 1 -- start start 2024-09-10T22:10:10.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.033+0000 7fd163600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd15c0b3720 0x7fd15c152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:10.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.033+0000 7fd163600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd15c161440 0x7fd15c1fb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:10.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.033+0000 7fd163600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd15c1619f0 0x7fd15c1fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:10.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.033+0000 7fd163600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c0d0fb0 con 0x7fd15c161440 2024-09-10T22:10:10.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.033+0000 7fd163600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c0d0e30 con 0x7fd15c1619f0 2024-09-10T22:10:10.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.033+0000 7fd163600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd15c0d1130 con 0x7fd15c0b3720 2024-09-10T22:10:10.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd160c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd15c0b3720 0x7fd15c152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:10.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd160c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd15c0b3720 0x7fd15c152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:55232/0 (socket says 172.21.15.66:55232) 2024-09-10T22:10:10.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd160c00640 1 -- 172.21.15.66:0/3481612017 learned_addr learned my addr 172.21.15.66:0/3481612017 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:10.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd161600640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd15c1619f0 0x7fd15c1fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:10.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd15be00640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd15c161440 0x7fd15c1fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:10.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd161600640 1 -- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd15c0b3720 msgr2=0x7fd15c152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:10.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd161600640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd15c0b3720 0x7fd15c152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd161600640 1 -- 172.21.15.66:0/3481612017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd15c161440 msgr2=0x7fd15c1fb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:10.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd161600640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd15c161440 0x7fd15c1fb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd161600640 1 -- 172.21.15.66:0/3481612017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c0b8930 con 0x7fd15c1619f0 2024-09-10T22:10:10.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd161600640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd15c1619f0 0x7fd15c1fbaa0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd15400be20 tx=0x7fd15400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:10.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd158c00640 1 -- 172.21.15.66:0/3481612017 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154019070 con 0x7fd15c1619f0 2024-09-10T22:10:10.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd158c00640 1 -- 172.21.15.66:0/3481612017 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd154014440 con 0x7fd15c1619f0 2024-09-10T22:10:10.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd163600640 1 -- 172.21.15.66:0/3481612017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15c0b8c10 con 0x7fd15c1619f0 2024-09-10T22:10:10.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd158c00640 1 -- 172.21.15.66:0/3481612017 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154014890 con 0x7fd15c1619f0 2024-09-10T22:10:10.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd163600640 1 -- 172.21.15.66:0/3481612017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15c0b91d0 con 0x7fd15c1619f0 2024-09-10T22:10:10.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.037+0000 7fd163600640 1 -- 172.21.15.66:0/3481612017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd15c0b93d0 con 0x7fd15c1619f0 2024-09-10T22:10:10.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.041+0000 7fd158c00640 1 -- 172.21.15.66:0/3481612017 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fd154014a30 con 0x7fd15c1619f0 2024-09-10T22:10:10.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.045+0000 7fd158c00640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd13c073c00 0x7fd13c0760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:10.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.045+0000 7fd158c00640 1 -- 172.21.15.66:0/3481612017 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fd154092b10 con 0x7fd15c1619f0 2024-09-10T22:10:10.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.045+0000 7fd160c00640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd13c073c00 0x7fd13c0760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:10.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.045+0000 7fd160c00640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd13c073c00 0x7fd13c0760c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009850 tx=0x7fd14c019680 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:10.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.049+0000 7fd158c00640 1 -- 172.21.15.66:0/3481612017 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd15c0b93d0 con 0x7fd15c1619f0 2024-09-10T22:10:10.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.409+0000 7fd163600640 1 -- 172.21.15.66:0/3481612017 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone723", "target": ["mon-mgr", ""]}) -- 0x7fd15c0d2210 con 0x7fd13c073c00 2024-09-10T22:10:10.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.441+0000 7fd158c00640 1 -- 172.21.15.66:0/3481612017 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd15c0d2210 con 0x7fd13c073c00 2024-09-10T22:10:10.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.445+0000 7fd163600640 1 -- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd13c073c00 msgr2=0x7fd13c0760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:10.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.445+0000 7fd163600640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd13c073c00 0x7fd13c0760c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009850 tx=0x7fd14c019680 comp rx=0 tx=0).stop 2024-09-10T22:10:10.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.445+0000 7fd163600640 1 -- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd15c1619f0 msgr2=0x7fd15c1fbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:10.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.445+0000 7fd163600640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd15c1619f0 0x7fd15c1fbaa0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd15400be20 tx=0x7fd15400bf10 comp rx=0 tx=0).stop 2024-09-10T22:10:10.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.445+0000 7fd163600640 1 -- 172.21.15.66:0/3481612017 shutdown_connections 2024-09-10T22:10:10.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.445+0000 7fd163600640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd13c073c00 0x7fd13c0760c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.445+0000 7fd163600640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd15c1619f0 0x7fd15c1fbaa0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.445+0000 7fd163600640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd15c161440 0x7fd15c1fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.445+0000 7fd163600640 1 --2- 172.21.15.66:0/3481612017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd15c0b3720 0x7fd15c152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.445+0000 7fd163600640 1 -- 172.21.15.66:0/3481612017 >> 172.21.15.66:0/3481612017 conn(0x7fd15c0d3e50 msgr2=0x7fd15c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:10.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.445+0000 7fd163600640 1 -- 172.21.15.66:0/3481612017 shutdown_connections 2024-09-10T22:10:10.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.445+0000 7fd163600640 1 -- 172.21.15.66:0/3481612017 wait complete. 2024-09-10T22:10:10.481 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone94742 2024-09-10T22:10:10.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.661+0000 7f7767a00640 1 -- 172.21.15.66:0/3734721379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7760150dc0 msgr2=0x7f77600b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:10.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.661+0000 7f7767a00640 1 --2- 172.21.15.66:0/3734721379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7760150dc0 0x7f77600b2cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7754009900 tx=0x7f775401c900 comp rx=0 tx=0).stop 2024-09-10T22:10:10.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.661+0000 7f7767a00640 1 -- 172.21.15.66:0/3734721379 shutdown_connections 2024-09-10T22:10:10.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.661+0000 7f7767a00640 1 --2- 172.21.15.66:0/3734721379 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7760072d50 0x7f776006e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.661+0000 7f7767a00640 1 --2- 172.21.15.66:0/3734721379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7760150dc0 0x7f77600b2cc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.661+0000 7f7767a00640 1 -- 172.21.15.66:0/3734721379 >> 172.21.15.66:0/3734721379 conn(0x7f77600cfaa0 msgr2=0x7f77600cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:10.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.661+0000 7f7767a00640 1 -- 172.21.15.66:0/3734721379 shutdown_connections 2024-09-10T22:10:10.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.661+0000 7f7767a00640 1 -- 172.21.15.66:0/3734721379 wait complete. 2024-09-10T22:10:10.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f7767a00640 1 Processor -- start 2024-09-10T22:10:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f7767a00640 1 -- start start 2024-09-10T22:10:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f7767a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7760072d50 0x7f776013cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f7767a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7760150dc0 0x7f7760153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f7767a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f776015a3b0 0x7f7760163ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f7767a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7760166bf0 con 0x7f7760072d50 2024-09-10T22:10:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f7767a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f776006fc00 con 0x7f776015a3b0 2024-09-10T22:10:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f7767a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7760166d70 con 0x7f7760150dc0 2024-09-10T22:10:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f775fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7760150dc0 0x7f7760153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f7765a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f776015a3b0 0x7f7760163ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f775fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7760150dc0 0x7f7760153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:55238/0 (socket says 172.21.15.66:55238) 2024-09-10T22:10:10.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f775fe00640 1 -- 172.21.15.66:0/4006243528 learned_addr learned my addr 172.21.15.66:0/4006243528 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:10.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f7765000640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7760072d50 0x7f776013cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:10.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f775fe00640 1 -- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f776015a3b0 msgr2=0x7f7760163ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:10.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f775fe00640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f776015a3b0 0x7f7760163ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f775fe00640 1 -- 172.21.15.66:0/4006243528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7760072d50 msgr2=0x7f776013cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:10.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f775fe00640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7760072d50 0x7f776013cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:10.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.665+0000 7f775fe00640 1 -- 172.21.15.66:0/4006243528 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7760169ac0 con 0x7f7760150dc0 2024-09-10T22:10:10.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.669+0000 7f775fe00640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7760150dc0 0x7f7760153e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f774c00c3f0 tx=0x7f774c00c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:10.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.669+0000 7f775cc00640 1 -- 172.21.15.66:0/4006243528 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0052d0 con 0x7f7760150dc0 2024-09-10T22:10:10.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.669+0000 7f7767a00640 1 -- 172.21.15.66:0/4006243528 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7760151ee0 con 0x7f7760150dc0 2024-09-10T22:10:10.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.669+0000 7f7767a00640 1 -- 172.21.15.66:0/4006243528 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7760151640 con 0x7f7760150dc0 2024-09-10T22:10:10.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.669+0000 7f7767a00640 1 -- 172.21.15.66:0/4006243528 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77600b4a30 con 0x7f7760150dc0 2024-09-10T22:10:10.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.669+0000 7f775cc00640 1 -- 172.21.15.66:0/4006243528 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f774c005470 con 0x7f7760150dc0 2024-09-10T22:10:10.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.669+0000 7f775cc00640 1 -- 172.21.15.66:0/4006243528 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c007260 con 0x7f7760150dc0 2024-09-10T22:10:10.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.673+0000 7f775cc00640 1 -- 172.21.15.66:0/4006243528 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f774c023080 con 0x7f7760150dc0 2024-09-10T22:10:10.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.673+0000 7f775cc00640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7740073bd0 0x7f7740076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:10.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.673+0000 7f775cc00640 1 -- 172.21.15.66:0/4006243528 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f774c091da0 con 0x7f7760150dc0 2024-09-10T22:10:10.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.673+0000 7f7765000640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7740073bd0 0x7f7740076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:10.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.677+0000 7f7765000640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7740073bd0 0x7f7740076090 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f77540af000 tx=0x7f775401b140 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:10.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:10.681+0000 7f775cc00640 1 -- 172.21.15.66:0/4006243528 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f774c05e4b0 con 0x7f7760150dc0 2024-09-10T22:10:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.041+0000 7f7767a00640 1 -- 172.21.15.66:0/4006243528 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone94742", "target": ["mon-mgr", ""]}) -- 0x7f776006e8c0 con 0x7f7740073bd0 2024-09-10T22:10:11.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.057+0000 7f775cc00640 1 -- 172.21.15.66:0/4006243528 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f776006e8c0 con 0x7f7740073bd0 2024-09-10T22:10:11.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.061+0000 7f7767a00640 1 -- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7740073bd0 msgr2=0x7f7740076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:11.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.061+0000 7f7767a00640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7740073bd0 0x7f7740076090 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f77540af000 tx=0x7f775401b140 comp rx=0 tx=0).stop 2024-09-10T22:10:11.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.061+0000 7f7767a00640 1 -- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7760150dc0 msgr2=0x7f7760153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:11.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.061+0000 7f7767a00640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7760150dc0 0x7f7760153e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f774c00c3f0 tx=0x7f774c00c8b0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.061+0000 7f7767a00640 1 -- 172.21.15.66:0/4006243528 shutdown_connections 2024-09-10T22:10:11.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.061+0000 7f7767a00640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7740073bd0 0x7f7740076090 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.061+0000 7f7767a00640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f776015a3b0 0x7f7760163ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.061+0000 7f7767a00640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7760150dc0 0x7f7760153e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.061+0000 7f7767a00640 1 --2- 172.21.15.66:0/4006243528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7760072d50 0x7f776013cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.061+0000 7f7767a00640 1 -- 172.21.15.66:0/4006243528 >> 172.21.15.66:0/4006243528 conn(0x7f77600cfaa0 msgr2=0x7f776014e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:11.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.061+0000 7f7767a00640 1 -- 172.21.15.66:0/4006243528 shutdown_connections 2024-09-10T22:10:11.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.061+0000 7f7767a00640 1 -- 172.21.15.66:0/4006243528 wait complete. 2024-09-10T22:10:11.082 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone33350 2024-09-10T22:10:11.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.229+0000 7fb336200640 1 -- 172.21.15.66:0/670967483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3300cee30 msgr2=0x7fb3300cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:11.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.229+0000 7fb336200640 1 --2- 172.21.15.66:0/670967483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3300cee30 0x7fb3300cf210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb320009840 tx=0x7fb32001f010 comp rx=0 tx=0).stop 2024-09-10T22:10:11.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.229+0000 7fb336200640 1 -- 172.21.15.66:0/670967483 shutdown_connections 2024-09-10T22:10:11.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.229+0000 7fb336200640 1 --2- 172.21.15.66:0/670967483 >> v2:172.21.15.94:3300/0 conn(0x7fb33015ddd0 0x7fb33015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.229+0000 7fb336200640 1 --2- 172.21.15.66:0/670967483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3300cee30 0x7fb3300cf210 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.229+0000 7fb336200640 1 -- 172.21.15.66:0/670967483 >> 172.21.15.66:0/670967483 conn(0x7fb3300cfaa0 msgr2=0x7fb3300cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:11.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb336200640 1 -- 172.21.15.66:0/670967483 shutdown_connections 2024-09-10T22:10:11.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb336200640 1 -- 172.21.15.66:0/670967483 wait complete. 2024-09-10T22:10:11.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb336200640 1 Processor -- start 2024-09-10T22:10:11.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb336200640 1 -- start start 2024-09-10T22:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb336200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33006ef30 0x7fb3300cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb336200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3300cee30 0x7fb33014ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb336200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33015ddd0 0x7fb330159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb336200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb33015c740 con 0x7fb33006ef30 2024-09-10T22:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb336200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb33015c8c0 con 0x7fb33015ddd0 2024-09-10T22:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb336200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb33006faf0 con 0x7fb3300cee30 2024-09-10T22:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb32f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33006ef30 0x7fb3300cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb32f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33006ef30 0x7fb3300cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37016/0 (socket says 172.21.15.66:37016) 2024-09-10T22:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb32f400640 1 -- 172.21.15.66:0/1805015092 learned_addr learned my addr 172.21.15.66:0/1805015092 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb326a00640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3300cee30 0x7fb33014ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb32fe00640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33015ddd0 0x7fb330159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb32f400640 1 -- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3300cee30 msgr2=0x7fb33014ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb32f400640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3300cee30 0x7fb33014ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb32f400640 1 -- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33015ddd0 msgr2=0x7fb330159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb32f400640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33015ddd0 0x7fb330159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb32f400640 1 -- 172.21.15.66:0/1805015092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3301f8c90 con 0x7fb33006ef30 2024-09-10T22:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb326a00640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3300cee30 0x7fb33014ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb32f400640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33006ef30 0x7fb3300cd310 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb320009790 tx=0x7fb320004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.233+0000 7fb327e00640 1 -- 172.21.15.66:0/1805015092 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3200be070 con 0x7fb33006ef30 2024-09-10T22:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.237+0000 7fb327e00640 1 -- 172.21.15.66:0/1805015092 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb320005230 con 0x7fb33006ef30 2024-09-10T22:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.237+0000 7fb336200640 1 -- 172.21.15.66:0/1805015092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3300cc550 con 0x7fb33006ef30 2024-09-10T22:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.237+0000 7fb327e00640 1 -- 172.21.15.66:0/1805015092 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3200b5040 con 0x7fb33006ef30 2024-09-10T22:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.237+0000 7fb336200640 1 -- 172.21.15.66:0/1805015092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3300ccb10 con 0x7fb33006ef30 2024-09-10T22:10:11.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.237+0000 7fb336200640 1 -- 172.21.15.66:0/1805015092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb33015e1f0 con 0x7fb33006ef30 2024-09-10T22:10:11.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.237+0000 7fb327e00640 1 -- 172.21.15.66:0/1805015092 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fb3200b8030 con 0x7fb33006ef30 2024-09-10T22:10:11.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.237+0000 7fb327e00640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb30c073bd0 0x7fb30c076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:11.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.237+0000 7fb327e00640 1 -- 172.21.15.66:0/1805015092 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7fb32012eb40 con 0x7fb33006ef30 2024-09-10T22:10:11.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.237+0000 7fb326a00640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb30c073bd0 0x7fb30c076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:11.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.241+0000 7fb326a00640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb30c073bd0 0x7fb30c076090 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb330070310 tx=0x7fb318010040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.241+0000 7fb327e00640 1 -- 172.21.15.66:0/1805015092 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb320018b70 con 0x7fb33006ef30 2024-09-10T22:10:11.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.445+0000 7fb336200640 1 -- 172.21.15.66:0/1805015092 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone33350", "target": ["mon-mgr", ""]}) -- 0x7fb3300d07c0 con 0x7fb30c073bd0 2024-09-10T22:10:11.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb327e00640 1 -- 172.21.15.66:0/1805015092 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb3300d07c0 con 0x7fb30c073bd0 2024-09-10T22:10:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb336200640 1 -- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb30c073bd0 msgr2=0x7fb30c076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb336200640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb30c073bd0 0x7fb30c076090 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb330070310 tx=0x7fb318010040 comp rx=0 tx=0).stop 2024-09-10T22:10:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb336200640 1 -- 172.21.15.66:0/1805015092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33006ef30 msgr2=0x7fb3300cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb336200640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33006ef30 0x7fb3300cd310 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb320009790 tx=0x7fb320004620 comp rx=0 tx=0).stop 2024-09-10T22:10:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb336200640 1 -- 172.21.15.66:0/1805015092 shutdown_connections 2024-09-10T22:10:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb336200640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb30c073bd0 0x7fb30c076090 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb336200640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33015ddd0 0x7fb330159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb336200640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3300cee30 0x7fb33014ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb336200640 1 --2- 172.21.15.66:0/1805015092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33006ef30 0x7fb3300cd310 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb336200640 1 -- 172.21.15.66:0/1805015092 >> 172.21.15.66:0/1805015092 conn(0x7fb3300cfaa0 msgr2=0x7fb33006f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:11.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb336200640 1 -- 172.21.15.66:0/1805015092 shutdown_connections 2024-09-10T22:10:11.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.465+0000 7fb336200640 1 -- 172.21.15.66:0/1805015092 wait complete. 2024-09-10T22:10:11.491 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8415 2024-09-10T22:10:11.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.649+0000 7f997dc00640 1 -- 172.21.15.66:0/2236249357 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f997815dd70 msgr2=0x7f99780b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:11.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.649+0000 7f997dc00640 1 --2- 172.21.15.66:0/2236249357 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f997815dd70 0x7f99780b29b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9970009bf0 tx=0x7f997001cac0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.649+0000 7f997dc00640 1 -- 172.21.15.66:0/2236249357 shutdown_connections 2024-09-10T22:10:11.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.649+0000 7f997dc00640 1 --2- 172.21.15.66:0/2236249357 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f997815dd70 0x7f99780b29b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.649+0000 7f997dc00640 1 -- 172.21.15.66:0/2236249357 >> 172.21.15.66:0/2236249357 conn(0x7f99780cfaa0 msgr2=0x7f99780cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:11.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.649+0000 7f997dc00640 1 -- 172.21.15.66:0/2236249357 shutdown_connections 2024-09-10T22:10:11.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.649+0000 7f997dc00640 1 -- 172.21.15.66:0/2236249357 wait complete. 2024-09-10T22:10:11.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.653+0000 7f997dc00640 1 Processor -- start 2024-09-10T22:10:11.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.653+0000 7f997dc00640 1 -- start start 2024-09-10T22:10:11.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.653+0000 7f997dc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99780743e0 0x7f99780cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:11.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.653+0000 7f997dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99780ada70 0x7f99781fc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:11.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.653+0000 7f997dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997815dd70 0x7f99780cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:11.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.653+0000 7f997dc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f997816c020 con 0x7f99780ada70 2024-09-10T22:10:11.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.653+0000 7f997dc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f997816bea0 con 0x7f997815dd70 2024-09-10T22:10:11.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.653+0000 7f997dc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f997816c1a0 con 0x7f99780743e0 2024-09-10T22:10:11.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.653+0000 7f9976000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99780ada70 0x7f99781fc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:11.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.653+0000 7f9976000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99780ada70 0x7f99781fc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37030/0 (socket says 172.21.15.66:37030) 2024-09-10T22:10:11.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.653+0000 7f9976a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99780743e0 0x7f99780cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:11.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.653+0000 7f9976000640 1 -- 172.21.15.66:0/1369246144 learned_addr learned my addr 172.21.15.66:0/1369246144 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:11.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f9977400640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997815dd70 0x7f99780cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:11.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f9976a00640 1 -- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997815dd70 msgr2=0x7f99780cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:11.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f9976a00640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997815dd70 0x7f99780cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f9976a00640 1 -- 172.21.15.66:0/1369246144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99780ada70 msgr2=0x7f99781fc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:11.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f9976a00640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99780ada70 0x7f99781fc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:11.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f9976a00640 1 -- 172.21.15.66:0/1369246144 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99780ce190 con 0x7f99780743e0 2024-09-10T22:10:11.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f9977400640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997815dd70 0x7f99780cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:11.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f9976a00640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99780743e0 0x7f99780cec00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f99700ac000 tx=0x7f99700a58c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:11.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f996ea00640 1 -- 172.21.15.66:0/1369246144 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99700b2040 con 0x7f99780743e0 2024-09-10T22:10:11.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f996ea00640 1 -- 172.21.15.66:0/1369246144 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9970003d30 con 0x7f99780743e0 2024-09-10T22:10:11.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f996ea00640 1 -- 172.21.15.66:0/1369246144 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970004290 con 0x7f99780743e0 2024-09-10T22:10:11.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f997dc00640 1 -- 172.21.15.66:0/1369246144 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997815ac50 con 0x7f99780743e0 2024-09-10T22:10:11.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.657+0000 7f997dc00640 1 -- 172.21.15.66:0/1369246144 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f99780ce6e0 con 0x7f99780743e0 2024-09-10T22:10:11.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.661+0000 7f997dc00640 1 -- 172.21.15.66:0/1369246144 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99780adf20 con 0x7f99780743e0 2024-09-10T22:10:11.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.665+0000 7f996ea00640 1 -- 172.21.15.66:0/1369246144 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f99700b5070 con 0x7f99780743e0 2024-09-10T22:10:11.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.665+0000 7f996ea00640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9954073bd0 0x7f9954076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:11.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.665+0000 7f996ea00640 1 -- 172.21.15.66:0/1369246144 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 5823+0+0 (secure 0 0 0) 0x7f997012c670 con 0x7f99780743e0 2024-09-10T22:10:11.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.665+0000 7f9976000640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9954073bd0 0x7f9954076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:11.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.669+0000 7f9976000640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9954073bd0 0x7f9954076090 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9960007fd0 tx=0x7f9960007f60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:11.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:11.669+0000 7f996ea00640 1 -- 172.21.15.66:0/1369246144 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f99700167d0 con 0x7f99780743e0 2024-09-10T22:10:12.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.041+0000 7f997dc00640 1 -- 172.21.15.66:0/1369246144 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8415", "target": ["mon-mgr", ""]}) -- 0x7f9978153c10 con 0x7f9954073bd0 2024-09-10T22:10:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.069+0000 7f996ea00640 1 -- 172.21.15.66:0/1369246144 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9978153c10 con 0x7f9954073bd0 2024-09-10T22:10:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.073+0000 7f997dc00640 1 -- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9954073bd0 msgr2=0x7f9954076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.073+0000 7f997dc00640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9954073bd0 0x7f9954076090 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9960007fd0 tx=0x7f9960007f60 comp rx=0 tx=0).stop 2024-09-10T22:10:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.073+0000 7f997dc00640 1 -- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99780743e0 msgr2=0x7f99780cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.073+0000 7f997dc00640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99780743e0 0x7f99780cec00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f99700ac000 tx=0x7f99700a58c0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.073+0000 7f997dc00640 1 -- 172.21.15.66:0/1369246144 shutdown_connections 2024-09-10T22:10:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.073+0000 7f997dc00640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9954073bd0 0x7f9954076090 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.073+0000 7f997dc00640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997815dd70 0x7f99780cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.073+0000 7f997dc00640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99780ada70 0x7f99781fc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.073+0000 7f997dc00640 1 --2- 172.21.15.66:0/1369246144 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f99780743e0 0x7f99780cec00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.073+0000 7f997dc00640 1 -- 172.21.15.66:0/1369246144 >> 172.21.15.66:0/1369246144 conn(0x7f99780cfaa0 msgr2=0x7f9978154130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.073+0000 7f997dc00640 1 -- 172.21.15.66:0/1369246144 shutdown_connections 2024-09-10T22:10:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.073+0000 7f997dc00640 1 -- 172.21.15.66:0/1369246144 wait complete. 2024-09-10T22:10:12.094 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone82612 2024-09-10T22:10:12.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.245+0000 7f8cab600640 1 -- 172.21.15.66:0/2493068748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ca415df80 msgr2=0x7f8ca40cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:12.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.245+0000 7f8cab600640 1 --2- 172.21.15.66:0/2493068748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ca415df80 0x7f8ca40cee20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009900 tx=0x7f8c9801c8e0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.245+0000 7f8cab600640 1 -- 172.21.15.66:0/2493068748 shutdown_connections 2024-09-10T22:10:12.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.245+0000 7f8cab600640 1 --2- 172.21.15.66:0/2493068748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ca415df80 0x7f8ca40cee20 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.245+0000 7f8cab600640 1 -- 172.21.15.66:0/2493068748 >> 172.21.15.66:0/2493068748 conn(0x7f8ca40d1c60 msgr2=0x7f8ca40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:12.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.245+0000 7f8cab600640 1 -- 172.21.15.66:0/2493068748 shutdown_connections 2024-09-10T22:10:12.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.245+0000 7f8cab600640 1 -- 172.21.15.66:0/2493068748 wait complete. 2024-09-10T22:10:12.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8cab600640 1 Processor -- start 2024-09-10T22:10:12.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8cab600640 1 -- start start 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8cab600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d03f0 0x7f8ca4152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8cab600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ca40d08a0 0x7f8ca4151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8cab600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ca415df80 0x7f8ca4140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8cab600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca416c840 con 0x7f8ca415df80 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8cab600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca416c6c0 con 0x7f8ca40d03f0 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8cab600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca416c9c0 con 0x7f8ca40d08a0 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca9600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ca415df80 0x7f8ca4140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca9600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ca415df80 0x7f8ca4140d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37042/0 (socket says 172.21.15.66:37042) 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca9600640 1 -- 172.21.15.66:0/542944571 learned_addr learned my addr 172.21.15.66:0/542944571 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca8c00640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d03f0 0x7f8ca4152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca3e00640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ca40d08a0 0x7f8ca4151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca9600640 1 -- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ca40d08a0 msgr2=0x7f8ca4151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca9600640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ca40d08a0 0x7f8ca4151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca9600640 1 -- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d03f0 msgr2=0x7f8ca4152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca9600640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d03f0 0x7f8ca4152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca9600640 1 -- 172.21.15.66:0/542944571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca415b900 con 0x7f8ca415df80 2024-09-10T22:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca3e00640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ca40d08a0 0x7f8ca4151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca9600640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ca415df80 0x7f8ca4140d80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00c780 tx=0x7f8c9c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca0c00640 1 -- 172.21.15.66:0/542944571 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c003b80 con 0x7f8ca415df80 2024-09-10T22:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca0c00640 1 -- 172.21.15.66:0/542944571 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c9c003d20 con 0x7f8ca415df80 2024-09-10T22:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8ca0c00640 1 -- 172.21.15.66:0/542944571 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c00aab0 con 0x7f8ca415df80 2024-09-10T22:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8cab600640 1 -- 172.21.15.66:0/542944571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca415bbe0 con 0x7f8ca415df80 2024-09-10T22:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8cab600640 1 -- 172.21.15.66:0/542944571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca415c1a0 con 0x7f8ca415df80 2024-09-10T22:10:12.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.249+0000 7f8cab600640 1 -- 172.21.15.66:0/542944571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca415c3a0 con 0x7f8ca415df80 2024-09-10T22:10:12.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.253+0000 7f8ca0c00640 1 -- 172.21.15.66:0/542944571 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f8c9c00ac50 con 0x7f8ca415df80 2024-09-10T22:10:12.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.253+0000 7f8ca0c00640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8c84073bd0 0x7f8c84076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:12.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.253+0000 7f8ca0c00640 1 -- 172.21.15.66:0/542944571 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5879+0+0 (secure 0 0 0) 0x7f8c9c092d40 con 0x7f8ca415df80 2024-09-10T22:10:12.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.253+0000 7f8ca8c00640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8c84073bd0 0x7f8c84076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:12.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.253+0000 7f8ca8c00640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8c84073bd0 0x7f8c84076090 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8c980af000 tx=0x7f8c9801b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:12.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.257+0000 7f8ca0c00640 1 -- 172.21.15.66:0/542944571 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ca415c3a0 con 0x7f8ca415df80 2024-09-10T22:10:12.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.461+0000 7f8cab600640 1 -- 172.21.15.66:0/542944571 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone82612", "target": ["mon-mgr", ""]}) -- 0x7f8ca406fd20 con 0x7f8c84073bd0 2024-09-10T22:10:12.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.477+0000 7f8ca0c00640 1 -- 172.21.15.66:0/542944571 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ca406fd20 con 0x7f8c84073bd0 2024-09-10T22:10:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.481+0000 7f8cab600640 1 -- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8c84073bd0 msgr2=0x7f8c84076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.481+0000 7f8cab600640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8c84073bd0 0x7f8c84076090 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8c980af000 tx=0x7f8c9801b120 comp rx=0 tx=0).stop 2024-09-10T22:10:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.481+0000 7f8cab600640 1 -- 172.21.15.66:0/542944571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ca415df80 msgr2=0x7f8ca4140d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.481+0000 7f8cab600640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ca415df80 0x7f8ca4140d80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00c780 tx=0x7f8c9c00cc40 comp rx=0 tx=0).stop 2024-09-10T22:10:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.481+0000 7f8cab600640 1 -- 172.21.15.66:0/542944571 shutdown_connections 2024-09-10T22:10:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.481+0000 7f8cab600640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8c84073bd0 0x7f8c84076090 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.481+0000 7f8cab600640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ca415df80 0x7f8ca4140d80 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.481+0000 7f8cab600640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ca40d08a0 0x7f8ca4151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.481+0000 7f8cab600640 1 --2- 172.21.15.66:0/542944571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ca40d03f0 0x7f8ca4152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.481+0000 7f8cab600640 1 -- 172.21.15.66:0/542944571 >> 172.21.15.66:0/542944571 conn(0x7f8ca40d1c60 msgr2=0x7f8ca40b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.481+0000 7f8cab600640 1 -- 172.21.15.66:0/542944571 shutdown_connections 2024-09-10T22:10:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.481+0000 7f8cab600640 1 -- 172.21.15.66:0/542944571 wait complete. 2024-09-10T22:10:12.503 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone16013 2024-09-10T22:10:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.669+0000 7fb3c8400640 1 -- 172.21.15.66:0/391848219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c0070040 msgr2=0x7fb3c0072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.669+0000 7fb3c8400640 1 --2- 172.21.15.66:0/391848219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c0070040 0x7fb3c0072a20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009840 tx=0x7fb3ac01c820 comp rx=0 tx=0).stop 2024-09-10T22:10:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.669+0000 7fb3c8400640 1 -- 172.21.15.66:0/391848219 shutdown_connections 2024-09-10T22:10:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.669+0000 7fb3c8400640 1 --2- 172.21.15.66:0/391848219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c0070040 0x7fb3c0072a20 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.669+0000 7fb3c8400640 1 --2- 172.21.15.66:0/391848219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c0072640 0x7fb3c00cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.669+0000 7fb3c8400640 1 -- 172.21.15.66:0/391848219 >> 172.21.15.66:0/391848219 conn(0x7fb3c00cfaa0 msgr2=0x7fb3c00cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:12.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.669+0000 7fb3c8400640 1 -- 172.21.15.66:0/391848219 shutdown_connections 2024-09-10T22:10:12.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.669+0000 7fb3c8400640 1 -- 172.21.15.66:0/391848219 wait complete. 2024-09-10T22:10:12.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c8400640 1 Processor -- start 2024-09-10T22:10:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c8400640 1 -- start start 2024-09-10T22:10:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c8400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c0070040 0x7fb3c0154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c8400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c0072640 0x7fb3c01febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c8400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c015b740 0x7fb3c01ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c8400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01664e0 con 0x7fb3c0072640 2024-09-10T22:10:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c8400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c0166360 con 0x7fb3c015b740 2024-09-10T22:10:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c8400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c0166660 con 0x7fb3c0070040 2024-09-10T22:10:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c5a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c0070040 0x7fb3c0154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c5a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c0070040 0x7fb3c0154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:46044/0 (socket says 172.21.15.66:46044) 2024-09-10T22:10:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c5a00640 1 -- 172.21.15.66:0/4275679310 learned_addr learned my addr 172.21.15.66:0/4275679310 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c6400640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c015b740 0x7fb3c01ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c5000640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c0072640 0x7fb3c01febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c5a00640 1 -- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c015b740 msgr2=0x7fb3c01ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c5a00640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c015b740 0x7fb3c01ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.673+0000 7fb3c5a00640 1 -- 172.21.15.66:0/4275679310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c0072640 msgr2=0x7fb3c01febc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.677+0000 7fb3c5a00640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c0072640 0x7fb3c01febc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.677+0000 7fb3c5a00640 1 -- 172.21.15.66:0/4275679310 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c015a1d0 con 0x7fb3c0070040 2024-09-10T22:10:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.677+0000 7fb3c5000640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c0072640 0x7fb3c01febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.677+0000 7fb3c5a00640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c0070040 0x7fb3c0154010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400e9f0 tx=0x7fb3b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:12.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.677+0000 7fb3bd600640 1 -- 172.21.15.66:0/4275679310 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b400cd80 con 0x7fb3c0070040 2024-09-10T22:10:12.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.677+0000 7fb3bd600640 1 -- 172.21.15.66:0/4275679310 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3b4010430 con 0x7fb3c0070040 2024-09-10T22:10:12.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.677+0000 7fb3bd600640 1 -- 172.21.15.66:0/4275679310 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4010860 con 0x7fb3c0070040 2024-09-10T22:10:12.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.677+0000 7fb3c8400640 1 -- 172.21.15.66:0/4275679310 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c015a4b0 con 0x7fb3c0070040 2024-09-10T22:10:12.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.677+0000 7fb3c8400640 1 -- 172.21.15.66:0/4275679310 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c015aa70 con 0x7fb3c0070040 2024-09-10T22:10:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.677+0000 7fb3c8400640 1 -- 172.21.15.66:0/4275679310 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c0052a0 con 0x7fb3c0070040 2024-09-10T22:10:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.681+0000 7fb3bd600640 1 -- 172.21.15.66:0/4275679310 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fb3b4010a00 con 0x7fb3c0070040 2024-09-10T22:10:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.681+0000 7fb3bd600640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3a0073bd0 0x7fb3a0076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.681+0000 7fb3bd600640 1 -- 172.21.15.66:0/4275679310 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(88..88 src has 1..88) ==== 5879+0+0 (secure 0 0 0) 0x7fb3b4092e50 con 0x7fb3c0070040 2024-09-10T22:10:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.681+0000 7fb3c5000640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3a0073bd0 0x7fb3a0076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.685+0000 7fb3c5000640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3a0073bd0 0x7fb3a0076090 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0af000 tx=0x7fb3ac01b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:12.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:12.689+0000 7fb3bd600640 1 -- 172.21.15.66:0/4275679310 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb3b405fc10 con 0x7fb3c0070040 2024-09-10T22:10:13.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.065+0000 7fb3c8400640 1 -- 172.21.15.66:0/4275679310 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone16013", "target": ["mon-mgr", ""]}) -- 0x7fb39c000bd0 con 0x7fb3a0073bd0 2024-09-10T22:10:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.113+0000 7fb3bd600640 1 -- 172.21.15.66:0/4275679310 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb39c000bd0 con 0x7fb3a0073bd0 2024-09-10T22:10:13.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.117+0000 7fb3c8400640 1 -- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3a0073bd0 msgr2=0x7fb3a0076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:13.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.117+0000 7fb3c8400640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3a0073bd0 0x7fb3a0076090 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0af000 tx=0x7fb3ac01b060 comp rx=0 tx=0).stop 2024-09-10T22:10:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.117+0000 7fb3c8400640 1 -- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c0070040 msgr2=0x7fb3c0154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.117+0000 7fb3c8400640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c0070040 0x7fb3c0154010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400e9f0 tx=0x7fb3b400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.117+0000 7fb3c8400640 1 -- 172.21.15.66:0/4275679310 shutdown_connections 2024-09-10T22:10:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.117+0000 7fb3c8400640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3a0073bd0 0x7fb3a0076090 secure :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0af000 tx=0x7fb3ac01b060 comp rx=0 tx=0).stop 2024-09-10T22:10:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.117+0000 7fb3c8400640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c015b740 0x7fb3c01ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.117+0000 7fb3c8400640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c0072640 0x7fb3c01febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.117+0000 7fb3c8400640 1 --2- 172.21.15.66:0/4275679310 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c0070040 0x7fb3c0154010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.117+0000 7fb3c8400640 1 -- 172.21.15.66:0/4275679310 >> 172.21.15.66:0/4275679310 conn(0x7fb3c00cfaa0 msgr2=0x7fb3c00b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:13.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.117+0000 7fb3c8400640 1 -- 172.21.15.66:0/4275679310 shutdown_connections 2024-09-10T22:10:13.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.117+0000 7fb3c8400640 1 -- 172.21.15.66:0/4275679310 wait complete. 2024-09-10T22:10:13.156 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone885517 2024-09-10T22:10:13.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.365+0000 7f962b200640 1 -- 172.21.15.66:0/3406368444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96240b0e20 msgr2=0x7f9624160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:13.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.365+0000 7f962b200640 1 --2- 172.21.15.66:0/3406368444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96240b0e20 0x7f9624160460 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f961c009c80 tx=0x7f961c01cda0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.365+0000 7f962b200640 1 -- 172.21.15.66:0/3406368444 shutdown_connections 2024-09-10T22:10:13.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.365+0000 7f962b200640 1 --2- 172.21.15.66:0/3406368444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96240b0e20 0x7f9624160460 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.365+0000 7f962b200640 1 -- 172.21.15.66:0/3406368444 >> 172.21.15.66:0/3406368444 conn(0x7f96240d1c60 msgr2=0x7f96240d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:13.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.365+0000 7f962b200640 1 -- 172.21.15.66:0/3406368444 shutdown_connections 2024-09-10T22:10:13.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.365+0000 7f962b200640 1 -- 172.21.15.66:0/3406368444 wait complete. 2024-09-10T22:10:13.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f962b200640 1 Processor -- start 2024-09-10T22:10:13.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f962b200640 1 -- start start 2024-09-10T22:10:13.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f962b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96240b0e20 0x7f962406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:13.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f962b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96240cdf90 0x7f96240c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:13.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f962b200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f96240ce8d0 0x7f9624153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:13.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f962b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9624169e90 con 0x7f96240b0e20 2024-09-10T22:10:13.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f962b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9624169d10 con 0x7f96240cdf90 2024-09-10T22:10:13.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f962b200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f962416a010 con 0x7f96240ce8d0 2024-09-10T22:10:13.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9623e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96240b0e20 0x7f962406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:13.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9623e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96240b0e20 0x7f962406f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51312/0 (socket says 172.21.15.66:51312) 2024-09-10T22:10:13.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9623e00640 1 -- 172.21.15.66:0/4281209778 learned_addr learned my addr 172.21.15.66:0/4281209778 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:13.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f961b400640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96240cdf90 0x7f96240c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:13.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9629200640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f96240ce8d0 0x7f9624153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:13.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9623e00640 1 -- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f96240ce8d0 msgr2=0x7f9624153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:13.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9623e00640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f96240ce8d0 0x7f9624153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9623e00640 1 -- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96240cdf90 msgr2=0x7f96240c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:13.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9623e00640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96240cdf90 0x7f96240c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9623e00640 1 -- 172.21.15.66:0/4281209778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96240c66c0 con 0x7f96240b0e20 2024-09-10T22:10:13.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9623e00640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96240b0e20 0x7f962406f2e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f961400c420 tx=0x7f961400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:13.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9620c00640 1 -- 172.21.15.66:0/4281209778 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614017070 con 0x7f96240b0e20 2024-09-10T22:10:13.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9620c00640 1 -- 172.21.15.66:0/4281209778 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96140053c0 con 0x7f96240b0e20 2024-09-10T22:10:13.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f9620c00640 1 -- 172.21.15.66:0/4281209778 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614003860 con 0x7f96240b0e20 2024-09-10T22:10:13.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f962b200640 1 -- 172.21.15.66:0/4281209778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96240c5910 con 0x7f96240b0e20 2024-09-10T22:10:13.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f962b200640 1 -- 172.21.15.66:0/4281209778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96240c5ed0 con 0x7f96240b0e20 2024-09-10T22:10:13.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.369+0000 7f962b200640 1 -- 172.21.15.66:0/4281209778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96240cecb0 con 0x7f96240b0e20 2024-09-10T22:10:13.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.373+0000 7f9620c00640 1 -- 172.21.15.66:0/4281209778 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f9614003a00 con 0x7f96240b0e20 2024-09-10T22:10:13.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.373+0000 7f9620c00640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9600073bd0 0x7f9600076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:13.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.373+0000 7f9620c00640 1 -- 172.21.15.66:0/4281209778 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5879+0+0 (secure 0 0 0) 0x7f96140919b0 con 0x7f96240b0e20 2024-09-10T22:10:13.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.373+0000 7f961b400640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9600073bd0 0x7f9600076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:13.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.373+0000 7f961b400640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9600073bd0 0x7f9600076090 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f962406d200 tx=0x7f960c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:13.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.377+0000 7f9620c00640 1 -- 172.21.15.66:0/4281209778 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9614091e50 con 0x7f96240b0e20 2024-09-10T22:10:13.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.577+0000 7f962b200640 1 -- 172.21.15.66:0/4281209778 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone885517", "target": ["mon-mgr", ""]}) -- 0x7f9624070550 con 0x7f9600073bd0 2024-09-10T22:10:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f9620c00640 1 -- 172.21.15.66:0/4281209778 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9624070550 con 0x7f9600073bd0 2024-09-10T22:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f962b200640 1 -- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9600073bd0 msgr2=0x7f9600076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f962b200640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9600073bd0 0x7f9600076090 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f962406d200 tx=0x7f960c009040 comp rx=0 tx=0).stop 2024-09-10T22:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f962b200640 1 -- 172.21.15.66:0/4281209778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96240b0e20 msgr2=0x7f962406f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f962b200640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96240b0e20 0x7f962406f2e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f961400c420 tx=0x7f961400c8e0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f962b200640 1 -- 172.21.15.66:0/4281209778 shutdown_connections 2024-09-10T22:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f962b200640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9600073bd0 0x7f9600076090 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f962b200640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f96240ce8d0 0x7f9624153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f962b200640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96240cdf90 0x7f96240c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f962b200640 1 --2- 172.21.15.66:0/4281209778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96240b0e20 0x7f962406f2e0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f962b200640 1 -- 172.21.15.66:0/4281209778 >> 172.21.15.66:0/4281209778 conn(0x7f96240d1c60 msgr2=0x7f962415adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:13.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f962b200640 1 -- 172.21.15.66:0/4281209778 shutdown_connections 2024-09-10T22:10:13.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.593+0000 7f962b200640 1 -- 172.21.15.66:0/4281209778 wait complete. 2024-09-10T22:10:13.619 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone344114 2024-09-10T22:10:13.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.861+0000 7fcc89000640 1 -- 172.21.15.66:0/1506955181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc840b3720 msgr2=0x7fcc840d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:13.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.861+0000 7fcc89000640 1 --2- 172.21.15.66:0/1506955181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc840b3720 0x7fcc840d4970 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fcc840d3d30 tx=0x7fcc7401c770 comp rx=0 tx=0).stop 2024-09-10T22:10:13.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.861+0000 7fcc89000640 1 -- 172.21.15.66:0/1506955181 shutdown_connections 2024-09-10T22:10:13.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.861+0000 7fcc89000640 1 --2- 172.21.15.66:0/1506955181 >> v2:172.21.15.94:3300/0 conn(0x7fcc841619f0 0x7fcc84161e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.861+0000 7fcc89000640 1 --2- 172.21.15.66:0/1506955181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc840b3720 0x7fcc840d4970 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.861+0000 7fcc89000640 1 -- 172.21.15.66:0/1506955181 >> 172.21.15.66:0/1506955181 conn(0x7fcc840d3e50 msgr2=0x7fcc840d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:13.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.861+0000 7fcc89000640 1 -- 172.21.15.66:0/1506955181 shutdown_connections 2024-09-10T22:10:13.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.861+0000 7fcc89000640 1 -- 172.21.15.66:0/1506955181 wait complete. 2024-09-10T22:10:13.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.865+0000 7fcc89000640 1 Processor -- start 2024-09-10T22:10:13.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.865+0000 7fcc89000640 1 -- start start 2024-09-10T22:10:13.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc89000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcc840b3720 0x7fcc840c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:13.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc89000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc84161440 0x7fcc840ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:13.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc89000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc841619f0 0x7fcc84154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:13.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc89000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcc840d0fe0 con 0x7fcc841619f0 2024-09-10T22:10:13.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc89000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc840d0e60 con 0x7fcc84161440 2024-09-10T22:10:13.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc89000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcc840d1160 con 0x7fcc840b3720 2024-09-10T22:10:13.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc81400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc84161440 0x7fcc840ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:13.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc81400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc84161440 0x7fcc840ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:60514/0 (socket says 172.21.15.66:60514) 2024-09-10T22:10:13.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc82800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc841619f0 0x7fcc84154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:13.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc81400640 1 -- 172.21.15.66:0/3209425281 learned_addr learned my addr 172.21.15.66:0/3209425281 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:13.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc81e00640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcc840b3720 0x7fcc840c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:13.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc81400640 1 -- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcc840b3720 msgr2=0x7fcc840c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:13.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc81400640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcc840b3720 0x7fcc840c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc81400640 1 -- 172.21.15.66:0/3209425281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc841619f0 msgr2=0x7fcc84154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:13.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc81400640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc841619f0 0x7fcc84154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:13.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc81400640 1 -- 172.21.15.66:0/3209425281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc840c8f10 con 0x7fcc84161440 2024-09-10T22:10:13.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.869+0000 7fcc81400640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc84161440 0x7fcc840ca230 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcc74000c10 tx=0x7fcc74005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:13.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.873+0000 7fcc7a000640 1 -- 172.21.15.66:0/3209425281 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc740bb070 con 0x7fcc84161440 2024-09-10T22:10:13.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.873+0000 7fcc7a000640 1 -- 172.21.15.66:0/3209425281 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc74004320 con 0x7fcc84161440 2024-09-10T22:10:13.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.873+0000 7fcc89000640 1 -- 172.21.15.66:0/3209425281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc840ca770 con 0x7fcc84161440 2024-09-10T22:10:13.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.873+0000 7fcc7a000640 1 -- 172.21.15.66:0/3209425281 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc740b2040 con 0x7fcc84161440 2024-09-10T22:10:13.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.873+0000 7fcc89000640 1 -- 172.21.15.66:0/3209425281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc840c9470 con 0x7fcc84161440 2024-09-10T22:10:13.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.873+0000 7fcc89000640 1 -- 172.21.15.66:0/3209425281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc840d1680 con 0x7fcc84161440 2024-09-10T22:10:13.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.877+0000 7fcc7a000640 1 -- 172.21.15.66:0/3209425281 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fcc740b5030 con 0x7fcc84161440 2024-09-10T22:10:13.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.877+0000 7fcc7a000640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcc64073bd0 0x7fcc64076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:13.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.877+0000 7fcc7a000640 1 -- 172.21.15.66:0/3209425281 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5879+0+0 (secure 0 0 0) 0x7fcc7412b950 con 0x7fcc84161440 2024-09-10T22:10:13.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.881+0000 7fcc81e00640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcc64073bd0 0x7fcc64076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:13.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.881+0000 7fcc81e00640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcc64073bd0 0x7fcc64076090 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcc70001690 tx=0x7fcc70009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:13.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:13.885+0000 7fcc7a000640 1 -- 172.21.15.66:0/3209425281 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcc840c8f10 con 0x7fcc84161440 2024-09-10T22:10:14.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.237+0000 7fcc89000640 1 -- 172.21.15.66:0/3209425281 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone344114", "target": ["mon-mgr", ""]}) -- 0x7fcc840b8a70 con 0x7fcc64073bd0 2024-09-10T22:10:14.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.273+0000 7fcc7a000640 1 -- 172.21.15.66:0/3209425281 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcc70001c70 con 0x7fcc64073bd0 2024-09-10T22:10:14.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.277+0000 7fcc89000640 1 -- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcc64073bd0 msgr2=0x7fcc64076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:14.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.277+0000 7fcc89000640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcc64073bd0 0x7fcc64076090 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcc70001690 tx=0x7fcc70009040 comp rx=0 tx=0).stop 2024-09-10T22:10:14.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.277+0000 7fcc89000640 1 -- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc84161440 msgr2=0x7fcc840ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:14.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.277+0000 7fcc89000640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc84161440 0x7fcc840ca230 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcc74000c10 tx=0x7fcc74005dd0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.277+0000 7fcc89000640 1 -- 172.21.15.66:0/3209425281 shutdown_connections 2024-09-10T22:10:14.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.277+0000 7fcc89000640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcc64073bd0 0x7fcc64076090 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.277+0000 7fcc89000640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc841619f0 0x7fcc84154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.277+0000 7fcc89000640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc84161440 0x7fcc840ca230 secure :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcc74000c10 tx=0x7fcc74005dd0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.277+0000 7fcc89000640 1 --2- 172.21.15.66:0/3209425281 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcc840b3720 0x7fcc840c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.277+0000 7fcc89000640 1 -- 172.21.15.66:0/3209425281 >> 172.21.15.66:0/3209425281 conn(0x7fcc840d3e50 msgr2=0x7fcc84160a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:14.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.281+0000 7fcc89000640 1 -- 172.21.15.66:0/3209425281 shutdown_connections 2024-09-10T22:10:14.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.281+0000 7fcc89000640 1 -- 172.21.15.66:0/3209425281 wait complete. 2024-09-10T22:10:14.313 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone533619 2024-09-10T22:10:14.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.517+0000 7f5b5d800640 1 -- 172.21.15.66:0/2458470645 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b580b3720 msgr2=0x7f5b580d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:14.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.517+0000 7f5b5d800640 1 --2- 172.21.15.66:0/2458470645 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b580b3720 0x7f5b580d4970 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009870 tx=0x7f5b4001c6f0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.517+0000 7f5b5d800640 1 -- 172.21.15.66:0/2458470645 shutdown_connections 2024-09-10T22:10:14.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.517+0000 7f5b5d800640 1 --2- 172.21.15.66:0/2458470645 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b580b3720 0x7f5b580d4970 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.517+0000 7f5b5d800640 1 -- 172.21.15.66:0/2458470645 >> 172.21.15.66:0/2458470645 conn(0x7f5b580d3e50 msgr2=0x7f5b580d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:14.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.517+0000 7f5b5d800640 1 -- 172.21.15.66:0/2458470645 shutdown_connections 2024-09-10T22:10:14.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.517+0000 7f5b5d800640 1 -- 172.21.15.66:0/2458470645 wait complete. 2024-09-10T22:10:14.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b5d800640 1 Processor -- start 2024-09-10T22:10:14.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b5d800640 1 -- start start 2024-09-10T22:10:14.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b5d800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b580b3720 0x7f5b581fbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:14.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b5d800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b58161440 0x7f5b58152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:14.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b5d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b581619f0 0x7f5b58154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:14.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b5d800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5b580d15d0 con 0x7f5b581619f0 2024-09-10T22:10:14.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b5d800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b580d1450 con 0x7f5b580b3720 2024-09-10T22:10:14.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b5d800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5b580d1750 con 0x7f5b58161440 2024-09-10T22:10:14.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b4fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b58161440 0x7f5b58152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:14.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b57400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b581619f0 0x7f5b58154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:14.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b56a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b580b3720 0x7f5b581fbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:14.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b4fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b58161440 0x7f5b58152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:46086/0 (socket says 172.21.15.66:46086) 2024-09-10T22:10:14.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b4fe00640 1 -- 172.21.15.66:0/523517105 learned_addr learned my addr 172.21.15.66:0/523517105 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:14.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b4fe00640 1 -- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b580b3720 msgr2=0x7f5b581fbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:14.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b4fe00640 1 --2- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b580b3720 0x7f5b581fbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b4fe00640 1 -- 172.21.15.66:0/523517105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b581619f0 msgr2=0x7f5b58154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:14.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b4fe00640 1 --2- 172.21.15.66:0/523517105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b581619f0 0x7f5b58154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b4fe00640 1 -- 172.21.15.66:0/523517105 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b58159060 con 0x7f5b58161440 2024-09-10T22:10:14.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b4fe00640 1 --2- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b58161440 0x7f5b58152790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b40000c10 tx=0x7f5b40005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:14.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b4ea00640 1 -- 172.21.15.66:0/523517105 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b400ac070 con 0x7f5b58161440 2024-09-10T22:10:14.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b4ea00640 1 -- 172.21.15.66:0/523517105 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b40004a40 con 0x7f5b58161440 2024-09-10T22:10:14.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b4ea00640 1 -- 172.21.15.66:0/523517105 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40004e10 con 0x7f5b58161440 2024-09-10T22:10:14.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b5d800640 1 -- 172.21.15.66:0/523517105 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b581fc2c0 con 0x7f5b58161440 2024-09-10T22:10:14.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.521+0000 7f5b5d800640 1 -- 172.21.15.66:0/523517105 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b581532c0 con 0x7f5b58161440 2024-09-10T22:10:14.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.525+0000 7f5b5d800640 1 -- 172.21.15.66:0/523517105 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b580d4970 con 0x7f5b58161440 2024-09-10T22:10:14.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.525+0000 7f5b4ea00640 1 -- 172.21.15.66:0/523517105 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f5b400b1450 con 0x7f5b58161440 2024-09-10T22:10:14.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.525+0000 7f5b4ea00640 1 --2- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b34073c00 0x7f5b340760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:14.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.525+0000 7f5b4ea00640 1 -- 172.21.15.66:0/523517105 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(88..88 src has 1..88) ==== 5879+0+0 (secure 0 0 0) 0x7f5b4012c0d0 con 0x7f5b58161440 2024-09-10T22:10:14.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.525+0000 7f5b56a00640 1 --2- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b34073c00 0x7f5b340760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:14.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.525+0000 7f5b56a00640 1 --2- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b34073c00 0x7f5b340760c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5b48001230 tx=0x7f5b48001760 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:14.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.529+0000 7f5b4ea00640 1 -- 172.21.15.66:0/523517105 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5b400b6070 con 0x7f5b58161440 2024-09-10T22:10:14.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.809+0000 7f5b5d800640 1 -- 172.21.15.66:0/523517105 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone533619", "target": ["mon-mgr", ""]}) -- 0x7f5b580d2210 con 0x7f5b34073c00 2024-09-10T22:10:14.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.841+0000 7f5b4ea00640 1 -- 172.21.15.66:0/523517105 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5b580d2210 con 0x7f5b34073c00 2024-09-10T22:10:14.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.845+0000 7f5b5d800640 1 -- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b34073c00 msgr2=0x7f5b340760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:14.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.845+0000 7f5b5d800640 1 --2- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b34073c00 0x7f5b340760c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5b48001230 tx=0x7f5b48001760 comp rx=0 tx=0).stop 2024-09-10T22:10:14.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.845+0000 7f5b5d800640 1 -- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b58161440 msgr2=0x7f5b58152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:14.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.845+0000 7f5b5d800640 1 --2- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b58161440 0x7f5b58152790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b40000c10 tx=0x7f5b40005750 comp rx=0 tx=0).stop 2024-09-10T22:10:14.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.845+0000 7f5b5d800640 1 -- 172.21.15.66:0/523517105 shutdown_connections 2024-09-10T22:10:14.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.845+0000 7f5b5d800640 1 --2- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b34073c00 0x7f5b340760c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.845+0000 7f5b5d800640 1 --2- 172.21.15.66:0/523517105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b581619f0 0x7f5b58154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.845+0000 7f5b5d800640 1 --2- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b58161440 0x7f5b58152790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.845+0000 7f5b5d800640 1 --2- 172.21.15.66:0/523517105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b580b3720 0x7f5b581fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:14.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.845+0000 7f5b5d800640 1 -- 172.21.15.66:0/523517105 >> 172.21.15.66:0/523517105 conn(0x7f5b580d3e50 msgr2=0x7f5b58160680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:14.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.845+0000 7f5b5d800640 1 -- 172.21.15.66:0/523517105 shutdown_connections 2024-09-10T22:10:14.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:14.845+0000 7f5b5d800640 1 -- 172.21.15.66:0/523517105 wait complete. 2024-09-10T22:10:14.884 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone51775 2024-09-10T22:10:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.065+0000 7ff98fa00640 1 -- 172.21.15.66:0/2156849660 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9880cce70 msgr2=0x7ff98806e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.065+0000 7ff98fa00640 1 --2- 172.21.15.66:0/2156849660 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9880cce70 0x7ff98806e6a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff980009bf0 tx=0x7ff98001ca90 comp rx=0 tx=0).stop 2024-09-10T22:10:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.065+0000 7ff98fa00640 1 -- 172.21.15.66:0/2156849660 shutdown_connections 2024-09-10T22:10:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.065+0000 7ff98fa00640 1 --2- 172.21.15.66:0/2156849660 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9880cce70 0x7ff98806e6a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.065+0000 7ff98fa00640 1 -- 172.21.15.66:0/2156849660 >> 172.21.15.66:0/2156849660 conn(0x7ff9880cfaa0 msgr2=0x7ff9880cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.065+0000 7ff98fa00640 1 -- 172.21.15.66:0/2156849660 shutdown_connections 2024-09-10T22:10:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.065+0000 7ff98fa00640 1 -- 172.21.15.66:0/2156849660 wait complete. 2024-09-10T22:10:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98fa00640 1 Processor -- start 2024-09-10T22:10:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98fa00640 1 -- start start 2024-09-10T22:10:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98fa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff988072d50 0x7ff9881f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98fa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9880cce70 0x7ff988141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98fa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff98815ab00 0x7ff988144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98fa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff988169370 con 0x7ff988072d50 2024-09-10T22:10:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98fa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881691f0 con 0x7ff9880cce70 2024-09-10T22:10:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98fa00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff9881694f0 con 0x7ff98815ab00 2024-09-10T22:10:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff988072d50 0x7ff9881f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff988072d50 0x7ff9881f9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51340/0 (socket says 172.21.15.66:51340) 2024-09-10T22:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98d000640 1 -- 172.21.15.66:0/3681849227 learned_addr learned my addr 172.21.15.66:0/3681849227 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff987e00640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9880cce70 0x7ff988141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98da00640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff98815ab00 0x7ff988144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98d000640 1 -- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff98815ab00 msgr2=0x7ff988144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98d000640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff98815ab00 0x7ff988144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98d000640 1 -- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9880cce70 msgr2=0x7ff988141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98d000640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9880cce70 0x7ff988141280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98d000640 1 -- 172.21.15.66:0/3681849227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff988148a50 con 0x7ff988072d50 2024-09-10T22:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98d000640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff988072d50 0x7ff9881f9ca0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff97c001ec0 tx=0x7ff97c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff984c00640 1 -- 172.21.15.66:0/3681849227 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c017070 con 0x7ff988072d50 2024-09-10T22:10:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff984c00640 1 -- 172.21.15.66:0/3681849227 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff97c003600 con 0x7ff988072d50 2024-09-10T22:10:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff984c00640 1 -- 172.21.15.66:0/3681849227 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c003a50 con 0x7ff988072d50 2024-09-10T22:10:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.069+0000 7ff98fa00640 1 -- 172.21.15.66:0/3681849227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff988148d30 con 0x7ff988072d50 2024-09-10T22:10:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.073+0000 7ff98fa00640 1 -- 172.21.15.66:0/3681849227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff988144f60 con 0x7ff988072d50 2024-09-10T22:10:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.073+0000 7ff98fa00640 1 -- 172.21.15.66:0/3681849227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9880d07c0 con 0x7ff988072d50 2024-09-10T22:10:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.073+0000 7ff984c00640 1 -- 172.21.15.66:0/3681849227 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7ff97c003bf0 con 0x7ff988072d50 2024-09-10T22:10:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.073+0000 7ff984c00640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff968073c00 0x7ff9680760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.073+0000 7ff984c00640 1 -- 172.21.15.66:0/3681849227 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5879+0+0 (secure 0 0 0) 0x7ff97c091990 con 0x7ff988072d50 2024-09-10T22:10:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.073+0000 7ff987e00640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff968073c00 0x7ff9680760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.077+0000 7ff987e00640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff968073c00 0x7ff9680760c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff988073fe0 tx=0x7ff974001420 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:15.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.077+0000 7ff984c00640 1 -- 172.21.15.66:0/3681849227 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff97c091e30 con 0x7ff988072d50 2024-09-10T22:10:15.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.277+0000 7ff98fa00640 1 -- 172.21.15.66:0/3681849227 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone51775", "target": ["mon-mgr", ""]}) -- 0x7ff9881417c0 con 0x7ff968073c00 2024-09-10T22:10:15.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff984c00640 1 -- 172.21.15.66:0/3681849227 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff97400f680 con 0x7ff968073c00 2024-09-10T22:10:15.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff98fa00640 1 -- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff968073c00 msgr2=0x7ff9680760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:15.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff98fa00640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff968073c00 0x7ff9680760c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff988073fe0 tx=0x7ff974001420 comp rx=0 tx=0).stop 2024-09-10T22:10:15.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff98fa00640 1 -- 172.21.15.66:0/3681849227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff988072d50 msgr2=0x7ff9881f9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:15.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff98fa00640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff988072d50 0x7ff9881f9ca0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff97c001ec0 tx=0x7ff97c00c890 comp rx=0 tx=0).stop 2024-09-10T22:10:15.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff98fa00640 1 -- 172.21.15.66:0/3681849227 shutdown_connections 2024-09-10T22:10:15.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff98fa00640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff968073c00 0x7ff9680760c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff98fa00640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff98815ab00 0x7ff988144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff98fa00640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9880cce70 0x7ff988141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff98fa00640 1 --2- 172.21.15.66:0/3681849227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff988072d50 0x7ff9881f9ca0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff98fa00640 1 -- 172.21.15.66:0/3681849227 >> 172.21.15.66:0/3681849227 conn(0x7ff9880cfaa0 msgr2=0x7ff9880ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:15.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff98fa00640 1 -- 172.21.15.66:0/3681849227 shutdown_connections 2024-09-10T22:10:15.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.297+0000 7ff98fa00640 1 -- 172.21.15.66:0/3681849227 wait complete. 2024-09-10T22:10:15.321 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone403911 2024-09-10T22:10:15.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.469+0000 7ffae5600640 1 -- 172.21.15.66:0/3302437849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae00cee30 msgr2=0x7ffae00cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:15.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.469+0000 7ffae5600640 1 --2- 172.21.15.66:0/3302437849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae00cee30 0x7ffae00cf210 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ffad000aa10 tx=0x7ffad0020420 comp rx=0 tx=0).stop 2024-09-10T22:10:15.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.469+0000 7ffae5600640 1 -- 172.21.15.66:0/3302437849 shutdown_connections 2024-09-10T22:10:15.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.469+0000 7ffae5600640 1 --2- 172.21.15.66:0/3302437849 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffae015b730 0x7ffae015bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.469+0000 7ffae5600640 1 --2- 172.21.15.66:0/3302437849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae00cee30 0x7ffae00cf210 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.469+0000 7ffae5600640 1 -- 172.21.15.66:0/3302437849 >> 172.21.15.66:0/3302437849 conn(0x7ffae00cfaa0 msgr2=0x7ffae00cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:15.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.469+0000 7ffae5600640 1 -- 172.21.15.66:0/3302437849 shutdown_connections 2024-09-10T22:10:15.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.469+0000 7ffae5600640 1 -- 172.21.15.66:0/3302437849 wait complete. 2024-09-10T22:10:15.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffae5600640 1 Processor -- start 2024-09-10T22:10:15.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffae5600640 1 -- start start 2024-09-10T22:10:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffae5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffae00cee30 0x7ffae00b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffae5600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffae015b730 0x7ffae014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffae5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae015ddd0 0x7ffae00af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffae5600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffae006e820 con 0x7ffae015ddd0 2024-09-10T22:10:15.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffae5600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffae006e6a0 con 0x7ffae00cee30 2024-09-10T22:10:15.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffae5600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ffae006e9a0 con 0x7ffae015b730 2024-09-10T22:10:15.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffadf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae015ddd0 0x7ffae00af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:15.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffadf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae015ddd0 0x7ffae00af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51362/0 (socket says 172.21.15.66:51362) 2024-09-10T22:10:15.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffadf400640 1 -- 172.21.15.66:0/1453152448 learned_addr learned my addr 172.21.15.66:0/1453152448 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:15.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffad6000640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffae015b730 0x7ffae014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:15.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffadea00640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffae00cee30 0x7ffae00b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:15.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffadf400640 1 -- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffae015b730 msgr2=0x7ffae014e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:15.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffadf400640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffae015b730 0x7ffae014e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffadf400640 1 -- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffae00cee30 msgr2=0x7ffae00b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:15.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffadf400640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffae00cee30 0x7ffae00b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffadf400640 1 -- 172.21.15.66:0/1453152448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae0157750 con 0x7ffae015ddd0 2024-09-10T22:10:15.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffadea00640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffae00cee30 0x7ffae00b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:15.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffad6000640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffae015b730 0x7ffae014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffadf400640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae015ddd0 0x7ffae00af620 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ffad800e960 tx=0x7ffad800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffad7400640 1 -- 172.21.15.66:0/1453152448 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8004d30 con 0x7ffae015ddd0 2024-09-10T22:10:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffad7400640 1 -- 172.21.15.66:0/1453152448 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffad800cb20 con 0x7ffae015ddd0 2024-09-10T22:10:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffad7400640 1 -- 172.21.15.66:0/1453152448 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8005020 con 0x7ffae015ddd0 2024-09-10T22:10:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffae5600640 1 -- 172.21.15.66:0/1453152448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae0157a30 con 0x7ffae015ddd0 2024-09-10T22:10:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffae5600640 1 -- 172.21.15.66:0/1453152448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae00b0980 con 0x7ffae015ddd0 2024-09-10T22:10:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.473+0000 7ffae5600640 1 -- 172.21.15.66:0/1453152448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae00cf210 con 0x7ffae015ddd0 2024-09-10T22:10:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.477+0000 7ffad7400640 1 -- 172.21.15.66:0/1453152448 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7ffad800f040 con 0x7ffae015ddd0 2024-09-10T22:10:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.477+0000 7ffad7400640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffabc073bd0 0x7ffabc076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.477+0000 7ffad7400640 1 -- 172.21.15.66:0/1453152448 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5879+0+0 (secure 0 0 0) 0x7ffad80929f0 con 0x7ffae015ddd0 2024-09-10T22:10:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.477+0000 7ffadea00640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffabc073bd0 0x7ffabc076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:15.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.477+0000 7ffadea00640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffabc073bd0 0x7ffabc076090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffad00b2000 tx=0x7ffad001ec60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:15.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.481+0000 7ffad7400640 1 -- 172.21.15.66:0/1453152448 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffad8016030 con 0x7ffae015ddd0 2024-09-10T22:10:15.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.725+0000 7ffae5600640 1 -- 172.21.15.66:0/1453152448 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone403911", "target": ["mon-mgr", ""]}) -- 0x7ffae00d07c0 con 0x7ffabc073bd0 2024-09-10T22:10:15.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.761+0000 7ffad7400640 1 -- 172.21.15.66:0/1453152448 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffae00d07c0 con 0x7ffabc073bd0 2024-09-10T22:10:15.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.765+0000 7ffae5600640 1 -- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffabc073bd0 msgr2=0x7ffabc076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:15.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.765+0000 7ffae5600640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffabc073bd0 0x7ffabc076090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffad00b2000 tx=0x7ffad001ec60 comp rx=0 tx=0).stop 2024-09-10T22:10:15.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.765+0000 7ffae5600640 1 -- 172.21.15.66:0/1453152448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae015ddd0 msgr2=0x7ffae00af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:15.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.765+0000 7ffae5600640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae015ddd0 0x7ffae00af620 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ffad800e960 tx=0x7ffad800ee20 comp rx=0 tx=0).stop 2024-09-10T22:10:15.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.765+0000 7ffae5600640 1 -- 172.21.15.66:0/1453152448 shutdown_connections 2024-09-10T22:10:15.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.765+0000 7ffae5600640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffabc073bd0 0x7ffabc076090 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.765+0000 7ffae5600640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffae015ddd0 0x7ffae00af620 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.765+0000 7ffae5600640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffae015b730 0x7ffae014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.765+0000 7ffae5600640 1 --2- 172.21.15.66:0/1453152448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffae00cee30 0x7ffae00b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:15.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.765+0000 7ffae5600640 1 -- 172.21.15.66:0/1453152448 >> 172.21.15.66:0/1453152448 conn(0x7ffae00cfaa0 msgr2=0x7ffae0150610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:15.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.765+0000 7ffae5600640 1 -- 172.21.15.66:0/1453152448 shutdown_connections 2024-09-10T22:10:15.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:15.765+0000 7ffae5600640 1 -- 172.21.15.66:0/1453152448 wait complete. 2024-09-10T22:10:15.802 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone302916 2024-09-10T22:10:16.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 -- 172.21.15.66:0/2622639063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0cf2e0 msgr2=0x7f04bc0cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 --2- 172.21.15.66:0/2622639063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0cf2e0 0x7f04bc0cf790 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009c80 tx=0x7f04b401cd60 comp rx=0 tx=0).stop 2024-09-10T22:10:16.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 -- 172.21.15.66:0/2622639063 shutdown_connections 2024-09-10T22:10:16.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 --2- 172.21.15.66:0/2622639063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0cf2e0 0x7f04bc0cf790 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 --2- 172.21.15.66:0/2622639063 >> v2:172.21.15.94:3300/0 conn(0x7f04bc0ce990 0x7f04bc0cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 --2- 172.21.15.66:0/2622639063 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04bc15e310 0x7f04bc0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 -- 172.21.15.66:0/2622639063 >> 172.21.15.66:0/2622639063 conn(0x7f04bc0d0bb0 msgr2=0x7f04bc0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:16.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 -- 172.21.15.66:0/2622639063 shutdown_connections 2024-09-10T22:10:16.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 -- 172.21.15.66:0/2622639063 wait complete. 2024-09-10T22:10:16.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 Processor -- start 2024-09-10T22:10:16.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 -- start start 2024-09-10T22:10:16.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04bc0ce990 0x7f04bc0c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:16.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04bc0cf2e0 0x7f04bc14efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:16.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc15e310 0x7f04bc1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:16.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc15e790 con 0x7f04bc15e310 2024-09-10T22:10:16.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc070330 con 0x7f04bc0cf2e0 2024-09-10T22:10:16.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.017+0000 7f04c3800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc15e910 con 0x7f04bc0ce990 2024-09-10T22:10:16.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04bbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04bc0cf2e0 0x7f04bc14efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:16.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04c1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc15e310 0x7f04bc1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:16.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04bbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04bc0cf2e0 0x7f04bc14efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:60538/0 (socket says 172.21.15.66:60538) 2024-09-10T22:10:16.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04bbe00640 1 -- 172.21.15.66:0/1333416711 learned_addr learned my addr 172.21.15.66:0/1333416711 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:16.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04c0e00640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04bc0ce990 0x7f04bc0c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:16.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04c1800640 1 -- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04bc0ce990 msgr2=0x7f04bc0c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04c1800640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04bc0ce990 0x7f04bc0c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04c1800640 1 -- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04bc0cf2e0 msgr2=0x7f04bc14efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04c1800640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04bc0cf2e0 0x7f04bc14efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04c1800640 1 -- 172.21.15.66:0/1333416711 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc0c9d50 con 0x7f04bc15e310 2024-09-10T22:10:16.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04bbe00640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04bc0cf2e0 0x7f04bc14efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:10:16.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04c1800640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc15e310 0x7f04bc1539e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f04b40ac000 tx=0x7f04b40a5ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:16.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04b8c00640 1 -- 172.21.15.66:0/1333416711 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b40bb070 con 0x7f04bc15e310 2024-09-10T22:10:16.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04b8c00640 1 -- 172.21.15.66:0/1333416711 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04b4004d40 con 0x7f04bc15e310 2024-09-10T22:10:16.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04c3800640 1 -- 172.21.15.66:0/1333416711 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc0ca400 con 0x7f04bc15e310 2024-09-10T22:10:16.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04b8c00640 1 -- 172.21.15.66:0/1333416711 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b40b2040 con 0x7f04bc15e310 2024-09-10T22:10:16.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04c3800640 1 -- 172.21.15.66:0/1333416711 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc0ca6c0 con 0x7f04bc15e310 2024-09-10T22:10:16.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04c3800640 1 -- 172.21.15.66:0/1333416711 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04bc0cf790 con 0x7f04bc15e310 2024-09-10T22:10:16.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.021+0000 7f04b8c00640 1 -- 172.21.15.66:0/1333416711 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f04b40b5030 con 0x7f04bc15e310 2024-09-10T22:10:16.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.025+0000 7f04b8c00640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f049c073c00 0x7f049c0760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:16.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.025+0000 7f04b8c00640 1 -- 172.21.15.66:0/1333416711 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5879+0+0 (secure 0 0 0) 0x7f04b412ba50 con 0x7f04bc15e310 2024-09-10T22:10:16.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.025+0000 7f04c0e00640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f049c073c00 0x7f049c0760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:16.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.025+0000 7f04c0e00640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f049c073c00 0x7f049c0760c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f04bc154ec0 tx=0x7f04b000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:16.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.029+0000 7f04b8c00640 1 -- 172.21.15.66:0/1333416711 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f04b4016950 con 0x7f04bc15e310 2024-09-10T22:10:16.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.225+0000 7f04c3800640 1 -- 172.21.15.66:0/1333416711 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone302916", "target": ["mon-mgr", ""]}) -- 0x7f04bc070b10 con 0x7f049c073c00 2024-09-10T22:10:16.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.241+0000 7f04b8c00640 1 -- 172.21.15.66:0/1333416711 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f04bc070b10 con 0x7f049c073c00 2024-09-10T22:10:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.245+0000 7f04c3800640 1 -- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f049c073c00 msgr2=0x7f049c0760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.245+0000 7f04c3800640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f049c073c00 0x7f049c0760c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f04bc154ec0 tx=0x7f04b000b2e0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.245+0000 7f04c3800640 1 -- 172.21.15.66:0/1333416711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc15e310 msgr2=0x7f04bc1539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.245+0000 7f04c3800640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc15e310 0x7f04bc1539e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f04b40ac000 tx=0x7f04b40a5ae0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.245+0000 7f04c3800640 1 -- 172.21.15.66:0/1333416711 shutdown_connections 2024-09-10T22:10:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.245+0000 7f04c3800640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f049c073c00 0x7f049c0760c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.245+0000 7f04c3800640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc15e310 0x7f04bc1539e0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.245+0000 7f04c3800640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04bc0cf2e0 0x7f04bc14efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.245+0000 7f04c3800640 1 --2- 172.21.15.66:0/1333416711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04bc0ce990 0x7f04bc0c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.245+0000 7f04c3800640 1 -- 172.21.15.66:0/1333416711 >> 172.21.15.66:0/1333416711 conn(0x7f04bc0d0bb0 msgr2=0x7f04bc15c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:16.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.245+0000 7f04c3800640 1 -- 172.21.15.66:0/1333416711 shutdown_connections 2024-09-10T22:10:16.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.245+0000 7f04c3800640 1 -- 172.21.15.66:0/1333416711 wait complete. 2024-09-10T22:10:16.267 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone5127 2024-09-10T22:10:16.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.417+0000 7f0d06200640 1 -- 172.21.15.66:0/3297165525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8004d50 msgr2=0x7f0ce80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.417+0000 7f0d06200640 1 --2- 172.21.15.66:0/3297165525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8004d50 0x7f0ce80051d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8001f50 tx=0x7f0cf801f3b0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.417+0000 7f0d05800640 1 -- 172.21.15.66:0/3297165525 reap_dead start 2024-09-10T22:10:16.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.417+0000 7f0d06200640 1 -- 172.21.15.66:0/3297165525 shutdown_connections 2024-09-10T22:10:16.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.417+0000 7f0d06200640 1 -- 172.21.15.66:0/3297165525 >> 172.21.15.66:0/3297165525 conn(0x7f0d0009d0a0 msgr2=0x7f0d000d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:16.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.417+0000 7f0d06200640 1 -- 172.21.15.66:0/3297165525 shutdown_connections 2024-09-10T22:10:16.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.417+0000 7f0d06200640 1 -- 172.21.15.66:0/3297165525 wait complete. 2024-09-10T22:10:16.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.417+0000 7f0d06200640 1 Processor -- start 2024-09-10T22:10:16.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.417+0000 7f0d06200640 1 -- start start 2024-09-10T22:10:16.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d06200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8004d50 0x7f0d001576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:16.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d06200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d00153a50 0x7f0d000cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:16.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d06200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d000cd5b0 0x7f0d0013f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:16.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d06200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d000b3730 con 0x7f0ce8004d50 2024-09-10T22:10:16.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d06200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d000d2860 con 0x7f0d00153a50 2024-09-10T22:10:16.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d06200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0d000b38b0 con 0x7f0d000cd5b0 2024-09-10T22:10:16.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d04e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8004d50 0x7f0d001576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:16.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d04e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8004d50 0x7f0d001576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51388/0 (socket says 172.21.15.66:51388) 2024-09-10T22:10:16.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d04e00640 1 -- 172.21.15.66:0/2774393527 learned_addr learned my addr 172.21.15.66:0/2774393527 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:16.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d05800640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d000cd5b0 0x7f0d0013f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:16.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0cffe00640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d00153a50 0x7f0d000cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:16.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d04e00640 1 -- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d000cd5b0 msgr2=0x7f0d0013f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d04e00640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d000cd5b0 0x7f0d0013f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d04e00640 1 -- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d00153a50 msgr2=0x7f0d000cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d04e00640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d00153a50 0x7f0d000cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d04e00640 1 -- 172.21.15.66:0/2774393527 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d00154260 con 0x7f0ce8004d50 2024-09-10T22:10:16.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d05800640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d000cd5b0 0x7f0d0013f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:16.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0cffe00640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d00153a50 0x7f0d000cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:16.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d04e00640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8004d50 0x7f0d001576a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4011ae0 tx=0x7f0cf4009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:16.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0cfcc00640 1 -- 172.21.15.66:0/2774393527 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf401e070 con 0x7f0ce8004d50 2024-09-10T22:10:16.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0cfcc00640 1 -- 172.21.15.66:0/2774393527 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cf400fe40 con 0x7f0ce8004d50 2024-09-10T22:10:16.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d06200640 1 -- 172.21.15.66:0/2774393527 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d00157be0 con 0x7f0ce8004d50 2024-09-10T22:10:16.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0cfcc00640 1 -- 172.21.15.66:0/2774393527 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf40042b0 con 0x7f0ce8004d50 2024-09-10T22:10:16.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d06200640 1 -- 172.21.15.66:0/2774393527 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d000b6fa0 con 0x7f0ce8004d50 2024-09-10T22:10:16.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.421+0000 7f0d06200640 1 -- 172.21.15.66:0/2774393527 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d000b2690 con 0x7f0ce8004d50 2024-09-10T22:10:16.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.425+0000 7f0cfcc00640 1 -- 172.21.15.66:0/2774393527 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f0cf4003600 con 0x7f0ce8004d50 2024-09-10T22:10:16.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.425+0000 7f0cfcc00640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ce0073bd0 0x7f0ce0076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:16.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.425+0000 7f0cfcc00640 1 -- 172.21.15.66:0/2774393527 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5879+0+0 (secure 0 0 0) 0x7f0cf40931d0 con 0x7f0ce8004d50 2024-09-10T22:10:16.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.425+0000 7f0cffe00640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ce0073bd0 0x7f0ce0076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:16.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.425+0000 7f0cffe00640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ce0073bd0 0x7f0ce0076090 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0d00146050 tx=0x7f0cec000f80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:16.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.429+0000 7f0cfcc00640 1 -- 172.21.15.66:0/2774393527 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0cf405f930 con 0x7f0ce8004d50 2024-09-10T22:10:16.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.629+0000 7f0d06200640 1 -- 172.21.15.66:0/2774393527 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone5127", "target": ["mon-mgr", ""]}) -- 0x7f0d0015bdd0 con 0x7f0ce0073bd0 2024-09-10T22:10:16.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.661+0000 7f0cfcc00640 1 -- 172.21.15.66:0/2774393527 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d0015bdd0 con 0x7f0ce0073bd0 2024-09-10T22:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.665+0000 7f0d06200640 1 -- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ce0073bd0 msgr2=0x7f0ce0076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.665+0000 7f0d06200640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ce0073bd0 0x7f0ce0076090 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0d00146050 tx=0x7f0cec000f80 comp rx=0 tx=0).stop 2024-09-10T22:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.665+0000 7f0d06200640 1 -- 172.21.15.66:0/2774393527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8004d50 msgr2=0x7f0d001576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.665+0000 7f0d06200640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8004d50 0x7f0d001576a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4011ae0 tx=0x7f0cf4009ea0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.665+0000 7f0d06200640 1 -- 172.21.15.66:0/2774393527 shutdown_connections 2024-09-10T22:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.665+0000 7f0d06200640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ce0073bd0 0x7f0ce0076090 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.665+0000 7f0d06200640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d000cd5b0 0x7f0d0013f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.665+0000 7f0d06200640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d00153a50 0x7f0d000cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.665+0000 7f0d06200640 1 --2- 172.21.15.66:0/2774393527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8004d50 0x7f0d001576a0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.665+0000 7f0d06200640 1 -- 172.21.15.66:0/2774393527 >> 172.21.15.66:0/2774393527 conn(0x7f0d0009d0a0 msgr2=0x7f0d000cfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.665+0000 7f0d06200640 1 -- 172.21.15.66:0/2774393527 shutdown_connections 2024-09-10T22:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.665+0000 7f0d06200640 1 -- 172.21.15.66:0/2774393527 wait complete. 2024-09-10T22:10:16.702 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone86448 2024-09-10T22:10:16.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.897+0000 7f296f200640 1 -- 172.21.15.66:0/3387576226 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2954004810 msgr2=0x7f2954004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.897+0000 7f296f200640 1 --2- 172.21.15.66:0/3387576226 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2954004810 0x7f2954004c90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f29700d24e0 tx=0x7f295801d210 comp rx=0 tx=0).stop 2024-09-10T22:10:16.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.897+0000 7f296f200640 1 -- 172.21.15.66:0/3387576226 shutdown_connections 2024-09-10T22:10:16.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.897+0000 7f296f200640 1 --2- 172.21.15.66:0/3387576226 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2954004810 0x7f2954004c90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.897+0000 7f296f200640 1 -- 172.21.15.66:0/3387576226 >> 172.21.15.66:0/3387576226 conn(0x7f297009aa50 msgr2=0x7f297009ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:16.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.897+0000 7f296f200640 1 -- 172.21.15.66:0/3387576226 shutdown_connections 2024-09-10T22:10:16.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.897+0000 7f296f200640 1 -- 172.21.15.66:0/3387576226 wait complete. 2024-09-10T22:10:16.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.897+0000 7f296f200640 1 Processor -- start 2024-09-10T22:10:16.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.897+0000 7f296f200640 1 -- start start 2024-09-10T22:10:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2954004810 0x7f29701f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f297014f470 0x7f297014e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296f200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f297014ee00 0x7f29700b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29700b12f0 con 0x7f2954004810 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29700b1170 con 0x7f297014f470 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296f200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f29700b1470 con 0x7f297014ee00 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2954004810 0x7f29701f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2954004810 0x7f29701f8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51412/0 (socket says 172.21.15.66:51412) 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296de00640 1 -- 172.21.15.66:0/275930577 learned_addr learned my addr 172.21.15.66:0/275930577 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f2967e00640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f297014f470 0x7f297014e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296e800640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f297014ee00 0x7f29700b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296de00640 1 -- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f297014ee00 msgr2=0x7f29700b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296de00640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f297014ee00 0x7f29700b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296de00640 1 -- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f297014f470 msgr2=0x7f297014e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296de00640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f297014f470 0x7f297014e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296de00640 1 -- 172.21.15.66:0/275930577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29701f4410 con 0x7f2954004810 2024-09-10T22:10:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296e800640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f297014ee00 0x7f29700b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296de00640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2954004810 0x7f29701f8490 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2960006600 tx=0x7f29600039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f2966000640 1 -- 172.21.15.66:0/275930577 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960015070 con 0x7f2954004810 2024-09-10T22:10:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f2966000640 1 -- 172.21.15.66:0/275930577 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2960011920 con 0x7f2954004810 2024-09-10T22:10:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f2966000640 1 -- 172.21.15.66:0/275930577 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960011d70 con 0x7f2954004810 2024-09-10T22:10:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296f200640 1 -- 172.21.15.66:0/275930577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29701f89d0 con 0x7f2954004810 2024-09-10T22:10:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296f200640 1 -- 172.21.15.66:0/275930577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29701f4960 con 0x7f2954004810 2024-09-10T22:10:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f296f200640 1 -- 172.21.15.66:0/275930577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297014f210 con 0x7f2954004810 2024-09-10T22:10:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.901+0000 7f2966000640 1 -- 172.21.15.66:0/275930577 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f2960010420 con 0x7f2954004810 2024-09-10T22:10:16.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.905+0000 7f2966000640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f294c073c00 0x7f294c0760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:16.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.905+0000 7f2966000640 1 -- 172.21.15.66:0/275930577 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5879+0+0 (secure 0 0 0) 0x7f2960093430 con 0x7f2954004810 2024-09-10T22:10:16.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.905+0000 7f2967e00640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f294c073c00 0x7f294c0760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.905+0000 7f2967e00640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f294c073c00 0x7f294c0760c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f29580ae000 tx=0x7f2958019fb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:16.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:16.909+0000 7f2966000640 1 -- 172.21.15.66:0/275930577 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f29701f89d0 con 0x7f2954004810 2024-09-10T22:10:17.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:17.109+0000 7f296f200640 1 -- 172.21.15.66:0/275930577 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone86448", "target": ["mon-mgr", ""]}) -- 0x7f29700d1c30 con 0x7f294c073c00 2024-09-10T22:10:20.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f2966000640 1 -- 172.21.15.66:0/275930577 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f29700d1c30 con 0x7f294c073c00 2024-09-10T22:10:20.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f296f200640 1 -- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f294c073c00 msgr2=0x7f294c0760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:20.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f296f200640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f294c073c00 0x7f294c0760c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f29580ae000 tx=0x7f2958019fb0 comp rx=0 tx=0).stop 2024-09-10T22:10:20.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f296f200640 1 -- 172.21.15.66:0/275930577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2954004810 msgr2=0x7f29701f8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:20.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f296f200640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2954004810 0x7f29701f8490 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2960006600 tx=0x7f29600039d0 comp rx=0 tx=0).stop 2024-09-10T22:10:20.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f296f200640 1 -- 172.21.15.66:0/275930577 shutdown_connections 2024-09-10T22:10:20.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f296f200640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f294c073c00 0x7f294c0760c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:20.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f296f200640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f297014ee00 0x7f29700b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:20.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f296f200640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f297014f470 0x7f297014e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:20.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f296f200640 1 --2- 172.21.15.66:0/275930577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2954004810 0x7f29701f8490 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:20.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f296f200640 1 -- 172.21.15.66:0/275930577 >> 172.21.15.66:0/275930577 conn(0x7f297009aa50 msgr2=0x7f29700b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:20.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f296f200640 1 -- 172.21.15.66:0/275930577 shutdown_connections 2024-09-10T22:10:20.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:20.941+0000 7f296f200640 1 -- 172.21.15.66:0/275930577 wait complete. 2024-09-10T22:10:20.972 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone75881 2024-09-10T22:10:21.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 -- 172.21.15.66:0/2505917811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c15f240 msgr2=0x7f125c0ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:21.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 --2- 172.21.15.66:0/2505917811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c15f240 0x7f125c0ce480 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f124c009900 tx=0x7f124c01c8b0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 -- 172.21.15.66:0/2505917811 shutdown_connections 2024-09-10T22:10:21.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 --2- 172.21.15.66:0/2505917811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f125c0cd020 0x7f125c0ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 --2- 172.21.15.66:0/2505917811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c15f240 0x7f125c0ce480 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 -- 172.21.15.66:0/2505917811 >> 172.21.15.66:0/2505917811 conn(0x7f125c0d1c50 msgr2=0x7f125c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:21.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 -- 172.21.15.66:0/2505917811 shutdown_connections 2024-09-10T22:10:21.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 -- 172.21.15.66:0/2505917811 wait complete. 2024-09-10T22:10:21.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 Processor -- start 2024-09-10T22:10:21.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 -- start start 2024-09-10T22:10:21.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f125c0cd020 0x7f125c0c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:21.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c0cef00 0x7f125c156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:21.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c15f240 0x7f125c0ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:21.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f125c169020 con 0x7f125c15f240 2024-09-10T22:10:21.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f125c168ea0 con 0x7f125c0cef00 2024-09-10T22:10:21.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1262e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f125c1691a0 con 0x7f125c0cd020 2024-09-10T22:10:21.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.125+0000 7f1260e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c15f240 0x7f125c0ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:21.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1260e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c15f240 0x7f125c0ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51428/0 (socket says 172.21.15.66:51428) 2024-09-10T22:10:21.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1260e00640 1 -- 172.21.15.66:0/2192039704 learned_addr learned my addr 172.21.15.66:0/2192039704 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:21.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f125be00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f125c0cd020 0x7f125c0c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:21.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1253e00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c0cef00 0x7f125c156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:21.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1260e00640 1 -- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f125c0cd020 msgr2=0x7f125c0c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1260e00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f125c0cd020 0x7f125c0c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1260e00640 1 -- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c0cef00 msgr2=0x7f125c156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1260e00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c0cef00 0x7f125c156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1260e00640 1 -- 172.21.15.66:0/2192039704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c1fb540 con 0x7f125c15f240 2024-09-10T22:10:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1260e00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c15f240 0x7f125c0ae9f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f125400c730 tx=0x7f125400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1258c00640 1 -- 172.21.15.66:0/2192039704 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254003db0 con 0x7f125c15f240 2024-09-10T22:10:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1258c00640 1 -- 172.21.15.66:0/2192039704 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f125400ce40 con 0x7f125c15f240 2024-09-10T22:10:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1258c00640 1 -- 172.21.15.66:0/2192039704 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12540052d0 con 0x7f125c15f240 2024-09-10T22:10:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1262e00640 1 -- 172.21.15.66:0/2192039704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125c1fb7e0 con 0x7f125c15f240 2024-09-10T22:10:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1262e00640 1 -- 172.21.15.66:0/2192039704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125c1fbda0 con 0x7f125c15f240 2024-09-10T22:10:21.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1262e00640 1 -- 172.21.15.66:0/2192039704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125c0cfa30 con 0x7f125c15f240 2024-09-10T22:10:21.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.129+0000 7f1258c00640 1 -- 172.21.15.66:0/2192039704 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f1254005470 con 0x7f125c15f240 2024-09-10T22:10:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.133+0000 7f1258c00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1238073bd0 0x7f1238076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.133+0000 7f1258c00640 1 -- 172.21.15.66:0/2192039704 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5851+0+0 (secure 0 0 0) 0x7f12540920a0 con 0x7f125c15f240 2024-09-10T22:10:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.133+0000 7f125be00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1238073bd0 0x7f1238076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:21.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.133+0000 7f125be00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1238073bd0 0x7f1238076090 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f124c0af000 tx=0x7f124c01b0f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.137+0000 7f1258c00640 1 -- 172.21.15.66:0/2192039704 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f125405e7d0 con 0x7f125c15f240 2024-09-10T22:10:21.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.333+0000 7f1262e00640 1 -- 172.21.15.66:0/2192039704 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone75881", "target": ["mon-mgr", ""]}) -- 0x7f125c157340 con 0x7f1238073bd0 2024-09-10T22:10:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1258c00640 1 -- 172.21.15.66:0/2192039704 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f125c157340 con 0x7f1238073bd0 2024-09-10T22:10:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1262e00640 1 -- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1238073bd0 msgr2=0x7f1238076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1262e00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1238073bd0 0x7f1238076090 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f124c0af000 tx=0x7f124c01b0f0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1262e00640 1 -- 172.21.15.66:0/2192039704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c15f240 msgr2=0x7f125c0ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1262e00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c15f240 0x7f125c0ae9f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f125400c730 tx=0x7f125400cbf0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1262e00640 1 -- 172.21.15.66:0/2192039704 shutdown_connections 2024-09-10T22:10:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1262e00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1238073bd0 0x7f1238076090 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1262e00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c15f240 0x7f125c0ae9f0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1262e00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c0cef00 0x7f125c156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1262e00640 1 --2- 172.21.15.66:0/2192039704 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f125c0cd020 0x7f125c0c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1262e00640 1 -- 172.21.15.66:0/2192039704 >> 172.21.15.66:0/2192039704 conn(0x7f125c0d1c50 msgr2=0x7f125c0b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1262e00640 1 -- 172.21.15.66:0/2192039704 shutdown_connections 2024-09-10T22:10:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.357+0000 7f1262e00640 1 -- 172.21.15.66:0/2192039704 wait complete. 2024-09-10T22:10:21.382 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone931018 2024-09-10T22:10:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 -- 172.21.15.66:0/1155437249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa340b2b10 msgr2=0x7ffa340b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 --2- 172.21.15.66:0/1155437249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa340b2b10 0x7ffa340b1b60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ffa340706e0 tx=0x7ffa1c01c8c0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 -- 172.21.15.66:0/1155437249 shutdown_connections 2024-09-10T22:10:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 --2- 172.21.15.66:0/1155437249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa340b2b10 0x7ffa340b1b60 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 --2- 172.21.15.66:0/1155437249 >> v2:172.21.15.94:3300/0 conn(0x7ffa341605c0 0x7ffa340b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 -- 172.21.15.66:0/1155437249 >> 172.21.15.66:0/1155437249 conn(0x7ffa340d1ce0 msgr2=0x7ffa340d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 -- 172.21.15.66:0/1155437249 shutdown_connections 2024-09-10T22:10:21.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 -- 172.21.15.66:0/1155437249 wait complete. 2024-09-10T22:10:21.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 Processor -- start 2024-09-10T22:10:21.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 -- start start 2024-09-10T22:10:21.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340b2b10 0x7ffa340cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:21.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa341605c0 0x7ffa341f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:21.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa34162c80 0x7ffa341f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:21.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34168fb0 con 0x7ffa34162c80 2024-09-10T22:10:21.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa340b1400 con 0x7ffa340b2b10 2024-09-10T22:10:21.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa3a200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ffa34169130 con 0x7ffa341605c0 2024-09-10T22:10:21.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa33e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa34162c80 0x7ffa341f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:21.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa33e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa34162c80 0x7ffa341f9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51440/0 (socket says 172.21.15.66:51440) 2024-09-10T22:10:21.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa33e00640 1 -- 172.21.15.66:0/2257397256 learned_addr learned my addr 172.21.15.66:0/2257397256 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:21.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa2aa00640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa341605c0 0x7ffa341f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:21.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.533+0000 7ffa33400640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340b2b10 0x7ffa340cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:21.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa33e00640 1 -- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa341605c0 msgr2=0x7ffa341f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:21.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa33e00640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa341605c0 0x7ffa341f8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa33e00640 1 -- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340b2b10 msgr2=0x7ffa340cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:21.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa33e00640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340b2b10 0x7ffa340cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa33e00640 1 -- 172.21.15.66:0/2257397256 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa340b59e0 con 0x7ffa34162c80 2024-09-10T22:10:21.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa33e00640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa34162c80 0x7ffa341f9460 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00b440 tx=0x7ffa2c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:21.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa2be00640 1 -- 172.21.15.66:0/2257397256 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c015020 con 0x7ffa34162c80 2024-09-10T22:10:21.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa2be00640 1 -- 172.21.15.66:0/2257397256 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa2c011960 con 0x7ffa34162c80 2024-09-10T22:10:21.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa2be00640 1 -- 172.21.15.66:0/2257397256 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c011db0 con 0x7ffa34162c80 2024-09-10T22:10:21.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa3a200640 1 -- 172.21.15.66:0/2257397256 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa340b5cc0 con 0x7ffa34162c80 2024-09-10T22:10:21.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa3a200640 1 -- 172.21.15.66:0/2257397256 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa340b6280 con 0x7ffa34162c80 2024-09-10T22:10:21.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa3a200640 1 -- 172.21.15.66:0/2257397256 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa340b1620 con 0x7ffa34162c80 2024-09-10T22:10:21.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.537+0000 7ffa2be00640 1 -- 172.21.15.66:0/2257397256 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7ffa2c0193b0 con 0x7ffa34162c80 2024-09-10T22:10:21.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.541+0000 7ffa2be00640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa10073bd0 0x7ffa10076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:21.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.541+0000 7ffa2be00640 1 -- 172.21.15.66:0/2257397256 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5851+0+0 (secure 0 0 0) 0x7ffa2c092120 con 0x7ffa34162c80 2024-09-10T22:10:21.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.541+0000 7ffa33400640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa10073bd0 0x7ffa10076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:21.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.541+0000 7ffa33400640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa10073bd0 0x7ffa10076090 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ffa24005fd0 tx=0x7ffa24007600 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.545+0000 7ffa2be00640 1 -- 172.21.15.66:0/2257397256 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffa2c01f030 con 0x7ffa34162c80 2024-09-10T22:10:21.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.745+0000 7ffa3a200640 1 -- 172.21.15.66:0/2257397256 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone931018", "target": ["mon-mgr", ""]}) -- 0x7ffa340cc280 con 0x7ffa10073bd0 2024-09-10T22:10:21.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.765+0000 7ffa2be00640 1 -- 172.21.15.66:0/2257397256 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffa24003980 con 0x7ffa10073bd0 2024-09-10T22:10:21.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.769+0000 7ffa3a200640 1 -- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa10073bd0 msgr2=0x7ffa10076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:21.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.769+0000 7ffa3a200640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa10073bd0 0x7ffa10076090 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ffa24005fd0 tx=0x7ffa24007600 comp rx=0 tx=0).stop 2024-09-10T22:10:21.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.769+0000 7ffa3a200640 1 -- 172.21.15.66:0/2257397256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa34162c80 msgr2=0x7ffa341f9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:21.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.769+0000 7ffa3a200640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa34162c80 0x7ffa341f9460 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00b440 tx=0x7ffa2c00b900 comp rx=0 tx=0).stop 2024-09-10T22:10:21.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.769+0000 7ffa3a200640 1 -- 172.21.15.66:0/2257397256 shutdown_connections 2024-09-10T22:10:21.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.769+0000 7ffa3a200640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa10073bd0 0x7ffa10076090 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.769+0000 7ffa3a200640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa34162c80 0x7ffa341f9460 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.769+0000 7ffa3a200640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa341605c0 0x7ffa341f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.769+0000 7ffa3a200640 1 --2- 172.21.15.66:0/2257397256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340b2b10 0x7ffa340cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:21.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.769+0000 7ffa3a200640 1 -- 172.21.15.66:0/2257397256 >> 172.21.15.66:0/2257397256 conn(0x7ffa340d1ce0 msgr2=0x7ffa340712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:21.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.769+0000 7ffa3a200640 1 -- 172.21.15.66:0/2257397256 shutdown_connections 2024-09-10T22:10:21.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21.769+0000 7ffa3a200640 1 -- 172.21.15.66:0/2257397256 wait complete. 2024-09-10T22:10:21.791 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T22:10:21.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:10:21.849 INFO:teuthology.orchestra.run.smithi066.stdout:2 2024-09-10T22:10:21.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:10:21.856 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:10:22.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 -- 172.21.15.66:0/3362218220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2154160280 msgr2=0x7f21540b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:22.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 --2- 172.21.15.66:0/3362218220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2154160280 0x7f21540b3170 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f213c009840 tx=0x7f213c01d1c0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 -- 172.21.15.66:0/3362218220 shutdown_connections 2024-09-10T22:10:22.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 --2- 172.21.15.66:0/3362218220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2154160280 0x7f21540b3170 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 -- 172.21.15.66:0/3362218220 >> 172.21.15.66:0/3362218220 conn(0x7f21540d1c60 msgr2=0x7f21540d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:22.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 -- 172.21.15.66:0/3362218220 shutdown_connections 2024-09-10T22:10:22.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 -- 172.21.15.66:0/3362218220 wait complete. 2024-09-10T22:10:22.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 Processor -- start 2024-09-10T22:10:22.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 -- start start 2024-09-10T22:10:22.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21540b1e10 0x7f2154150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:22.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215415f450 0x7f2154166260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:22.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2154160280 0x7f21541677f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:22.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f215416a040 con 0x7f215415f450 2024-09-10T22:10:22.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2154169ec0 con 0x7f21540b1e10 2024-09-10T22:10:22.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f215a400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f215416a1c0 con 0x7f2154160280 2024-09-10T22:10:22.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f214be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215415f450 0x7f2154166260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:22.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f214be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215415f450 0x7f2154166260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51444/0 (socket says 172.21.15.66:51444) 2024-09-10T22:10:22.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f214be00640 1 -- 172.21.15.66:0/742701309 learned_addr learned my addr 172.21.15.66:0/742701309 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:22.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f2153e00640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2154160280 0x7f21541677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:22.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.005+0000 7f2153400640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21540b1e10 0x7f2154150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:22.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f214be00640 1 -- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2154160280 msgr2=0x7f21541677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:22.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f214be00640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2154160280 0x7f21541677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f214be00640 1 -- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21540b1e10 msgr2=0x7f2154150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:22.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f214be00640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21540b1e10 0x7f2154150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f214be00640 1 -- 172.21.15.66:0/742701309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2154151c40 con 0x7f215415f450 2024-09-10T22:10:22.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f214be00640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215415f450 0x7f2154166260 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f213c01cc40 tx=0x7f213c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:22.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f214b400640 1 -- 172.21.15.66:0/742701309 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c0b2040 con 0x7f215415f450 2024-09-10T22:10:22.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f214b400640 1 -- 172.21.15.66:0/742701309 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f213c004330 con 0x7f215415f450 2024-09-10T22:10:22.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f214b400640 1 -- 172.21.15.66:0/742701309 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c003800 con 0x7f215415f450 2024-09-10T22:10:22.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f215a400640 1 -- 172.21.15.66:0/742701309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2154166890 con 0x7f215415f450 2024-09-10T22:10:22.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f215a400640 1 -- 172.21.15.66:0/742701309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2154166e50 con 0x7f215415f450 2024-09-10T22:10:22.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f215a400640 1 -- 172.21.15.66:0/742701309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21540b3170 con 0x7f215415f450 2024-09-10T22:10:22.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.009+0000 7f214b400640 1 -- 172.21.15.66:0/742701309 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f213c0b5070 con 0x7f215415f450 2024-09-10T22:10:22.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.013+0000 7f214b400640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2130073bd0 0x7f2130076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:22.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.013+0000 7f214b400640 1 -- 172.21.15.66:0/742701309 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5851+0+0 (secure 0 0 0) 0x7f213c12b4f0 con 0x7f215415f450 2024-09-10T22:10:22.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.013+0000 7f2153400640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2130073bd0 0x7f2130076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:22.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.013+0000 7f2153400640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2130073bd0 0x7f2130076090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2144007f30 tx=0x7f2144007ec0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:22.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.017+0000 7f214b400640 1 -- 172.21.15.66:0/742701309 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f213c016410 con 0x7f215415f450 2024-09-10T22:10:22.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.189+0000 7f215a400640 1 -- 172.21.15.66:0/742701309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f2154075980 con 0x7f215415f450 2024-09-10T22:10:22.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.189+0000 7f214b400640 1 -- 172.21.15.66:0/742701309 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v43)=0 v43) ==== 131+0+0 (secure 0 0 0) 0x7f213c12b990 con 0x7f215415f450 2024-09-10T22:10:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.193+0000 7f215a400640 1 -- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2130073bd0 msgr2=0x7f2130076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.193+0000 7f215a400640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2130073bd0 0x7f2130076090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2144007f30 tx=0x7f2144007ec0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.193+0000 7f215a400640 1 -- 172.21.15.66:0/742701309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215415f450 msgr2=0x7f2154166260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.193+0000 7f215a400640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215415f450 0x7f2154166260 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f213c01cc40 tx=0x7f213c005780 comp rx=0 tx=0).stop 2024-09-10T22:10:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.193+0000 7f215a400640 1 -- 172.21.15.66:0/742701309 shutdown_connections 2024-09-10T22:10:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.193+0000 7f215a400640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2130073bd0 0x7f2130076090 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.193+0000 7f215a400640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2154160280 0x7f21541677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.193+0000 7f215a400640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215415f450 0x7f2154166260 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.193+0000 7f215a400640 1 --2- 172.21.15.66:0/742701309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21540b1e10 0x7f2154150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.193+0000 7f215a400640 1 -- 172.21.15.66:0/742701309 >> 172.21.15.66:0/742701309 conn(0x7f21540d1c60 msgr2=0x7f21540d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:22.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.193+0000 7f215a400640 1 -- 172.21.15.66:0/742701309 shutdown_connections 2024-09-10T22:10:22.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.193+0000 7f215a400640 1 -- 172.21.15.66:0/742701309 wait complete. 2024-09-10T22:10:22.215 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:10:22.216 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:10:22.292 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:10:22.365 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:10:22.365 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:10:22.365 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:10:22.366 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:10:22.375 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:10:22.375 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:10:22.430 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:10:22.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 -- 172.21.15.66:0/2893165450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7e0004810 msgr2=0x7fd7e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:22.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 --2- 172.21.15.66:0/2893165450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7e0004810 0x7fd7e0004c90 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4001060 tx=0x7fd7e401cee0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 -- 172.21.15.66:0/2893165450 shutdown_connections 2024-09-10T22:10:22.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 --2- 172.21.15.66:0/2893165450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7e0004810 0x7fd7e0004c90 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 -- 172.21.15.66:0/2893165450 >> 172.21.15.66:0/2893165450 conn(0x7fd7fc0cfa30 msgr2=0x7fd7fc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:22.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 -- 172.21.15.66:0/2893165450 shutdown_connections 2024-09-10T22:10:22.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 -- 172.21.15.66:0/2893165450 wait complete. 2024-09-10T22:10:22.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 Processor -- start 2024-09-10T22:10:22.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 -- start start 2024-09-10T22:10:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7e0004810 0x7fd7fc15a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd7fc15aee0 0x7fd7fc1eb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7fc1eb780 0x7fd7fc15ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc0c4f50 con 0x7fd7e0004810 2024-09-10T22:10:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc0c4dd0 con 0x7fd7fc1eb780 2024-09-10T22:10:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd802a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd7fc0c50d0 con 0x7fd7fc15aee0 2024-09-10T22:10:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7e0004810 0x7fd7fc15a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7e0004810 0x7fd7fc15a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50680/0 (socket says 172.21.15.66:50680) 2024-09-10T22:10:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd7fbe00640 1 -- 172.21.15.66:0/928386790 learned_addr learned my addr 172.21.15.66:0/928386790 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd7f3400640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd7fc15aee0 0x7fd7fc1eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.617+0000 7fd800a00640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7fc1eb780 0x7fd7fc15ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.621+0000 7fd7fbe00640 1 -- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd7fc15aee0 msgr2=0x7fd7fc1eb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.621+0000 7fd7fbe00640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd7fc15aee0 0x7fd7fc1eb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.621+0000 7fd7fbe00640 1 -- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7fc1eb780 msgr2=0x7fd7fc15ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.621+0000 7fd7fbe00640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7fc1eb780 0x7fd7fc15ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.621+0000 7fd7fbe00640 1 -- 172.21.15.66:0/928386790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7fc1ea1a0 con 0x7fd7e0004810 2024-09-10T22:10:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.621+0000 7fd7fbe00640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7e0004810 0x7fd7fc15a9a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00eaf0 tx=0x7fd7ec00d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.621+0000 7fd7f8c00640 1 -- 172.21.15.66:0/928386790 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec004050 con 0x7fd7e0004810 2024-09-10T22:10:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.621+0000 7fd7f8c00640 1 -- 172.21.15.66:0/928386790 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7ec0041f0 con 0x7fd7e0004810 2024-09-10T22:10:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.621+0000 7fd7f8c00640 1 -- 172.21.15.66:0/928386790 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec003600 con 0x7fd7e0004810 2024-09-10T22:10:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.621+0000 7fd802a00640 1 -- 172.21.15.66:0/928386790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7fc1ea480 con 0x7fd7e0004810 2024-09-10T22:10:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.621+0000 7fd802a00640 1 -- 172.21.15.66:0/928386790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7fc1eaa40 con 0x7fd7e0004810 2024-09-10T22:10:22.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.621+0000 7fd802a00640 1 -- 172.21.15.66:0/928386790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7fc15e1e0 con 0x7fd7e0004810 2024-09-10T22:10:22.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.625+0000 7fd7f8c00640 1 -- 172.21.15.66:0/928386790 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fd7ec029080 con 0x7fd7e0004810 2024-09-10T22:10:22.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.625+0000 7fd7f8c00640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd7d8073bd0 0x7fd7d8076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:22.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.625+0000 7fd7f8c00640 1 -- 172.21.15.66:0/928386790 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5851+0+0 (secure 0 0 0) 0x7fd7ec0934b0 con 0x7fd7e0004810 2024-09-10T22:10:22.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.625+0000 7fd7f3400640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd7d8073bd0 0x7fd7d8076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:22.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.625+0000 7fd7f3400640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd7d8073bd0 0x7fd7d8076090 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40ab000 tx=0x7fd7e4019d20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.629+0000 7fd7f8c00640 1 -- 172.21.15.66:0/928386790 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd7ec0605e0 con 0x7fd7e0004810 2024-09-10T22:10:22.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.812+0000 7fd802a00640 1 -- 172.21.15.66:0/928386790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd7fc070420 con 0x7fd7e0004810 2024-09-10T22:10:22.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.812+0000 7fd7f8c00640 1 -- 172.21.15.66:0/928386790 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) ==== 80+0+0 (secure 0 0 0) 0x7fd7ec05fee0 con 0x7fd7e0004810 2024-09-10T22:10:22.817 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:10:22.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.816+0000 7fd802a00640 1 -- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd7d8073bd0 msgr2=0x7fd7d8076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:22.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.816+0000 7fd802a00640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd7d8073bd0 0x7fd7d8076090 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40ab000 tx=0x7fd7e4019d20 comp rx=0 tx=0).stop 2024-09-10T22:10:22.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.816+0000 7fd802a00640 1 -- 172.21.15.66:0/928386790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7e0004810 msgr2=0x7fd7fc15a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:22.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.816+0000 7fd802a00640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7e0004810 0x7fd7fc15a9a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00eaf0 tx=0x7fd7ec00d3e0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.816+0000 7fd802a00640 1 -- 172.21.15.66:0/928386790 shutdown_connections 2024-09-10T22:10:22.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.816+0000 7fd802a00640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd7d8073bd0 0x7fd7d8076090 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.816+0000 7fd802a00640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7fc1eb780 0x7fd7fc15ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.816+0000 7fd802a00640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd7fc15aee0 0x7fd7fc1eb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.816+0000 7fd802a00640 1 --2- 172.21.15.66:0/928386790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7e0004810 0x7fd7fc15a9a0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:22.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.816+0000 7fd802a00640 1 -- 172.21.15.66:0/928386790 >> 172.21.15.66:0/928386790 conn(0x7fd7fc0cfa30 msgr2=0x7fd7fc06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:22.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.816+0000 7fd802a00640 1 -- 172.21.15.66:0/928386790 shutdown_connections 2024-09-10T22:10:22.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:22.816+0000 7fd802a00640 1 -- 172.21.15.66:0/928386790 wait complete. 2024-09-10T22:10:22.839 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-09-10T22:10:22.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:22.839 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T22:10:22.860 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:10:22.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:22.860 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:10:22.866 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[44770]: fuse finished with error 0 and tester_r 0 2024-09-10T22:10:22.990 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:10:22.990 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:10:22.990 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:10:22.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:10:22.999 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:10:23.000 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:10:23.000 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:10:23.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:10:23.042 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:10:23.042 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:10:23.042 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:10:23.050 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:10:23.050 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:10:23.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.228+0000 7f7d86e00640 1 -- 172.21.15.66:0/1851405020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d68004810 msgr2=0x7f7d68004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:23.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.228+0000 7f7d86e00640 1 --2- 172.21.15.66:0/1851405020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d68004810 0x7f7d68004c90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c001830 tx=0x7f7d6c01eff0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.228+0000 7f7d86e00640 1 -- 172.21.15.66:0/1851405020 shutdown_connections 2024-09-10T22:10:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.228+0000 7f7d86e00640 1 --2- 172.21.15.66:0/1851405020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d68004810 0x7f7d68004c90 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.228+0000 7f7d86e00640 1 -- 172.21.15.66:0/1851405020 >> 172.21.15.66:0/1851405020 conn(0x7f7d800d1b10 msgr2=0x7f7d800d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.228+0000 7f7d86e00640 1 -- 172.21.15.66:0/1851405020 shutdown_connections 2024-09-10T22:10:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.228+0000 7f7d86e00640 1 -- 172.21.15.66:0/1851405020 wait complete. 2024-09-10T22:10:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.228+0000 7f7d86e00640 1 Processor -- start 2024-09-10T22:10:23.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.228+0000 7f7d86e00640 1 -- start start 2024-09-10T22:10:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d86e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d68004810 0x7f7d80159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d86e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d800ceac0 0x7f7d800cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d86e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d800b6a20 0x7f7d800b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d86e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7d800d1650 con 0x7f7d800ceac0 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d86e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d800d14d0 con 0x7f7d68004810 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d86e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8006dd30 con 0x7f7d800b6a20 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d85000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d800ceac0 0x7f7d800cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d85000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d800ceac0 0x7f7d800cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50696/0 (socket says 172.21.15.66:50696) 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d85000640 1 -- 172.21.15.66:0/299340533 learned_addr learned my addr 172.21.15.66:0/299340533 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d86400640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d800b6a20 0x7f7d800b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d85a00640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d68004810 0x7f7d80159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d85000640 1 -- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d800b6a20 msgr2=0x7f7d800b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d85000640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d800b6a20 0x7f7d800b6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d85000640 1 -- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d68004810 msgr2=0x7f7d80159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d85000640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d68004810 0x7f7d80159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d85000640 1 -- 172.21.15.66:0/299340533 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d800cef40 con 0x7f7d800ceac0 2024-09-10T22:10:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d85000640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d800ceac0 0x7f7d800cd330 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c005040 tx=0x7f7d6c001540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:23.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d7d600640 1 -- 172.21.15.66:0/299340533 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c003600 con 0x7f7d800ceac0 2024-09-10T22:10:23.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d7d600640 1 -- 172.21.15.66:0/299340533 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d6c0b7030 con 0x7f7d800ceac0 2024-09-10T22:10:23.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d7d600640 1 -- 172.21.15.66:0/299340533 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c003ad0 con 0x7f7d800ceac0 2024-09-10T22:10:23.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d86e00640 1 -- 172.21.15.66:0/299340533 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d800cd870 con 0x7f7d800ceac0 2024-09-10T22:10:23.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d86e00640 1 -- 172.21.15.66:0/299340533 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d80158f40 con 0x7f7d800ceac0 2024-09-10T22:10:23.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.232+0000 7f7d86e00640 1 -- 172.21.15.66:0/299340533 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d801619a0 con 0x7f7d800ceac0 2024-09-10T22:10:23.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.236+0000 7f7d7d600640 1 -- 172.21.15.66:0/299340533 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f7d6c003c70 con 0x7f7d800ceac0 2024-09-10T22:10:23.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.236+0000 7f7d7d600640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7d60073bd0 0x7f7d60076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:23.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.236+0000 7f7d7d600640 1 -- 172.21.15.66:0/299340533 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5851+0+0 (secure 0 0 0) 0x7f7d6c12f5b0 con 0x7f7d800ceac0 2024-09-10T22:10:23.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.236+0000 7f7d85a00640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7d60073bd0 0x7f7d60076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:23.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.236+0000 7f7d85a00640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7d60073bd0 0x7f7d60076090 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7d800ce5f0 tx=0x7f7d740091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:23.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.240+0000 7f7d7d600640 1 -- 172.21.15.66:0/299340533 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7d6c018da0 con 0x7f7d800ceac0 2024-09-10T22:10:23.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 -- 172.21.15.66:0/299340533 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d80072e00 con 0x7f7d800ceac0 2024-09-10T22:10:23.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d7d600640 1 -- 172.21.15.66:0/299340533 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5222 (secure 0 0 0) 0x7f7d6c0fc850 con 0x7f7d800ceac0 2024-09-10T22:10:23.451 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:23.451 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":57,"btime":"2024-09-10T22:08:29:488488+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3360242011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3360242011},{"type":"v1","addr":"172.21.15.94:6837","nonce":3360242011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3211574496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3211574496},{"type":"v1","addr":"172.21.15.66:6835","nonce":3211574496}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/553876835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":553876835},{"type":"v1","addr":"172.21.15.66:6834","nonce":553876835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:08:19.775657+0000","modified":"2024-09-10T22:08:22.793838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6836/3187502512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3187502512},{"type":"v1","addr":"172.21.15.94:6836","nonce":3187502512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":6}]} 2024-09-10T22:10:23.451 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 57 2024-09-10T22:10:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 -- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7d60073bd0 msgr2=0x7f7d60076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7d60073bd0 0x7f7d60076090 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7d800ce5f0 tx=0x7f7d740091e0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 -- 172.21.15.66:0/299340533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d800ceac0 msgr2=0x7f7d800cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d800ceac0 0x7f7d800cd330 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c005040 tx=0x7f7d6c001540 comp rx=0 tx=0).stop 2024-09-10T22:10:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 -- 172.21.15.66:0/299340533 shutdown_connections 2024-09-10T22:10:23.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7d60073bd0 0x7f7d60076090 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d800b6a20 0x7f7d800b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d800ceac0 0x7f7d800cd330 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 --2- 172.21.15.66:0/299340533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d68004810 0x7f7d80159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 -- 172.21.15.66:0/299340533 >> 172.21.15.66:0/299340533 conn(0x7f7d800d1b10 msgr2=0x7f7d800b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:23.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 -- 172.21.15.66:0/299340533 shutdown_connections 2024-09-10T22:10:23.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.448+0000 7f7d86e00640 1 -- 172.21.15.66:0/299340533 wait complete. 2024-09-10T22:10:23.474 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:10:23.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.616+0000 7f4b9d600640 1 -- 172.21.15.66:0/1894719633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b980d03e0 msgr2=0x7f4b980d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:23.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.616+0000 7f4b9d600640 1 --2- 172.21.15.66:0/1894719633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b980d03e0 0x7f4b980d0820 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009900 tx=0x7f4b8401c850 comp rx=0 tx=0).stop 2024-09-10T22:10:23.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.616+0000 7f4b9d600640 1 -- 172.21.15.66:0/1894719633 shutdown_connections 2024-09-10T22:10:23.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.616+0000 7f4b9d600640 1 --2- 172.21.15.66:0/1894719633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b980d03e0 0x7f4b980d0820 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.616+0000 7f4b9d600640 1 --2- 172.21.15.66:0/1894719633 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b9815e010 0x7f4b980cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.616+0000 7f4b9d600640 1 -- 172.21.15.66:0/1894719633 >> 172.21.15.66:0/1894719633 conn(0x7f4b980d1c60 msgr2=0x7f4b980d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:23.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.616+0000 7f4b9d600640 1 -- 172.21.15.66:0/1894719633 shutdown_connections 2024-09-10T22:10:23.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.616+0000 7f4b9d600640 1 -- 172.21.15.66:0/1894719633 wait complete. 2024-09-10T22:10:23.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.616+0000 7f4b9d600640 1 Processor -- start 2024-09-10T22:10:23.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.616+0000 7f4b9d600640 1 -- start start 2024-09-10T22:10:23.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b9d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980d03e0 0x7f4b98158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:23.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b9d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b980d0d60 0x7f4b9814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:23.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b9d600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b9815e010 0x7f4b980b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:23.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b9d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9816aea0 con 0x7f4b980d0d60 2024-09-10T22:10:23.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b9d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9816ad20 con 0x7f4b980d03e0 2024-09-10T22:10:23.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b9d600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9816b020 con 0x7f4b9815e010 2024-09-10T22:10:23.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b96000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b980d0d60 0x7f4b9814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:23.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b96000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b980d0d60 0x7f4b9814fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50718/0 (socket says 172.21.15.66:50718) 2024-09-10T22:10:23.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b96000640 1 -- 172.21.15.66:0/2612063314 learned_addr learned my addr 172.21.15.66:0/2612063314 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:23.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b96a00640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980d03e0 0x7f4b98158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:23.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b97400640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b9815e010 0x7f4b980b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:23.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b96000640 1 -- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b9815e010 msgr2=0x7f4b980b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:23.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b96000640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b9815e010 0x7f4b980b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b96000640 1 -- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980d03e0 msgr2=0x7f4b98158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:23.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b96000640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980d03e0 0x7f4b98158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:23.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b96000640 1 -- 172.21.15.66:0/2612063314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b981592d0 con 0x7f4b980d0d60 2024-09-10T22:10:23.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b97400640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b9815e010 0x7f4b980b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:23.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b96000640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b980d0d60 0x7f4b9814fa20 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009850 tx=0x7f4b840a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:23.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b8ea00640 1 -- 172.21.15.66:0/2612063314 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b840b2040 con 0x7f4b980d0d60 2024-09-10T22:10:23.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b8ea00640 1 -- 172.21.15.66:0/2612063314 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b84005030 con 0x7f4b980d0d60 2024-09-10T22:10:23.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b8ea00640 1 -- 172.21.15.66:0/2612063314 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b840044f0 con 0x7f4b980d0d60 2024-09-10T22:10:23.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b9d600640 1 -- 172.21.15.66:0/2612063314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b981595a0 con 0x7f4b980d0d60 2024-09-10T22:10:23.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b9d600640 1 -- 172.21.15.66:0/2612063314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9815b270 con 0x7f4b980d0d60 2024-09-10T22:10:23.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.620+0000 7f4b9d600640 1 -- 172.21.15.66:0/2612063314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b980d0820 con 0x7f4b980d0d60 2024-09-10T22:10:23.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.624+0000 7f4b8ea00640 1 -- 172.21.15.66:0/2612063314 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f4b84005440 con 0x7f4b980d0d60 2024-09-10T22:10:23.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.624+0000 7f4b8ea00640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b78073bd0 0x7f4b78076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:23.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.624+0000 7f4b8ea00640 1 -- 172.21.15.66:0/2612063314 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5851+0+0 (secure 0 0 0) 0x7f4b8412b850 con 0x7f4b980d0d60 2024-09-10T22:10:23.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.624+0000 7f4b96a00640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b78073bd0 0x7f4b78076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:23.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.624+0000 7f4b96a00640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b78073bd0 0x7f4b78076090 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4b88004630 tx=0x7f4b8800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:23.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:23.628+0000 7f4b8ea00640 1 -- 172.21.15.66:0/2612063314 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4b840164d0 con 0x7f4b980d0d60 2024-09-10T22:10:24.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.024+0000 7f4b9d600640 1 -- 172.21.15.66:0/2612063314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b980cbef0 con 0x7f4b980d0d60 2024-09-10T22:10:24.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.028+0000 7f4b8ea00640 1 -- 172.21.15.66:0/2612063314 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5222 (secure 0 0 0) 0x7f4b840f8980 con 0x7f4b980d0d60 2024-09-10T22:10:24.033 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":57,"btime":"2024-09-10T22:08:29:488488+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3360242011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3360242011},{"type":"v1","addr":"172.21.15.94:6837","nonce":3360242011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3211574496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3211574496},{"type":"v1","addr":"172.21.15.66:6835","nonce":3211574496}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/553876835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":553876835},{"type":"v1","addr":"172.21.15.66:6834","nonce":553876835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:08:19.775657+0000","modified":"2024-09-10T22:08:22.793838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6836/3187502512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3187502512},{"type":"v1","addr":"172.21.15.94:6836","nonce":3187502512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":6}]} 2024-09-10T22:10:24.034 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 57 2024-09-10T22:10:24.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.032+0000 7f4b9d600640 1 -- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b78073bd0 msgr2=0x7f4b78076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:24.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.032+0000 7f4b9d600640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b78073bd0 0x7f4b78076090 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4b88004630 tx=0x7f4b8800b2e0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.032+0000 7f4b9d600640 1 -- 172.21.15.66:0/2612063314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b980d0d60 msgr2=0x7f4b9814fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:24.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.032+0000 7f4b9d600640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b980d0d60 0x7f4b9814fa20 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009850 tx=0x7f4b840a5710 comp rx=0 tx=0).stop 2024-09-10T22:10:24.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.032+0000 7f4b9d600640 1 -- 172.21.15.66:0/2612063314 shutdown_connections 2024-09-10T22:10:24.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.032+0000 7f4b9d600640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4b78073bd0 0x7f4b78076090 secure :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4b88004630 tx=0x7f4b8800b2e0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.032+0000 7f4b9d600640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4b9815e010 0x7f4b980b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.032+0000 7f4b9d600640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b980d0d60 0x7f4b9814fa20 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.032+0000 7f4b9d600640 1 --2- 172.21.15.66:0/2612063314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980d03e0 0x7f4b98158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.032+0000 7f4b9d600640 1 -- 172.21.15.66:0/2612063314 >> 172.21.15.66:0/2612063314 conn(0x7f4b980d1c60 msgr2=0x7f4b980b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:24.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.036+0000 7f4b9d600640 1 -- 172.21.15.66:0/2612063314 shutdown_connections 2024-09-10T22:10:24.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.036+0000 7f4b9d600640 1 -- 172.21.15.66:0/2612063314 wait complete. 2024-09-10T22:10:24.074 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:10:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.320+0000 7f02a8200640 1 -- 172.21.15.66:0/2459678451 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02a0152b90 msgr2=0x7f02a006f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.320+0000 7f02a8200640 1 --2- 172.21.15.66:0/2459678451 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02a0152b90 0x7f02a006f650 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f02a00d0b00 tx=0x7f029401c8c0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.320+0000 7f02a8200640 1 -- 172.21.15.66:0/2459678451 shutdown_connections 2024-09-10T22:10:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.320+0000 7f02a8200640 1 --2- 172.21.15.66:0/2459678451 >> v2:172.21.15.66:3300/0 conn(0x7f02a0073f40 0x7f02a00743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.320+0000 7f02a8200640 1 --2- 172.21.15.66:0/2459678451 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02a0152b90 0x7f02a006f650 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.320+0000 7f02a8200640 1 -- 172.21.15.66:0/2459678451 >> 172.21.15.66:0/2459678451 conn(0x7f02a00b26c0 msgr2=0x7f02a00b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:24.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.324+0000 7f02a8200640 1 -- 172.21.15.66:0/2459678451 shutdown_connections 2024-09-10T22:10:24.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.324+0000 7f02a8200640 1 -- 172.21.15.66:0/2459678451 wait complete. 2024-09-10T22:10:24.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.324+0000 7f02a8200640 1 Processor -- start 2024-09-10T22:10:24.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a8200640 1 -- start start 2024-09-10T22:10:24.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a8200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02a00737c0 0x7f02a00d0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:24.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a8200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a0073f40 0x7f02a01ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:24.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a8200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02a0152b90 0x7f02a00b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:24.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a8200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f02a01689e0 con 0x7f02a00737c0 2024-09-10T22:10:24.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a8200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f02a00ce440 con 0x7f02a0073f40 2024-09-10T22:10:24.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a8200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f02a0168b60 con 0x7f02a0152b90 2024-09-10T22:10:24.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02a00737c0 0x7f02a00d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:24.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a6200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02a0152b90 0x7f02a00b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:24.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02a00737c0 0x7f02a00d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50740/0 (socket says 172.21.15.66:50740) 2024-09-10T22:10:24.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a5800640 1 -- 172.21.15.66:0/375375840 learned_addr learned my addr 172.21.15.66:0/375375840 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:24.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a5800640 1 -- 172.21.15.66:0/375375840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02a0152b90 msgr2=0x7f02a00b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:24.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a5800640 1 --2- 172.21.15.66:0/375375840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02a0152b90 0x7f02a00b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a5800640 1 -- 172.21.15.66:0/375375840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a0073f40 msgr2=0x7f02a01ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:24.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a5800640 1 --2- 172.21.15.66:0/375375840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a0073f40 0x7f02a01ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a5800640 1 -- 172.21.15.66:0/375375840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a0154d60 con 0x7f02a00737c0 2024-09-10T22:10:24.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.328+0000 7f02a5800640 1 --2- 172.21.15.66:0/375375840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02a00737c0 0x7f02a00d0120 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f029401d160 tx=0x7f0294005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:24.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.332+0000 7f029d600640 1 -- 172.21.15.66:0/375375840 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02940b2040 con 0x7f02a00737c0 2024-09-10T22:10:24.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.332+0000 7f029d600640 1 -- 172.21.15.66:0/375375840 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02940043f0 con 0x7f02a00737c0 2024-09-10T22:10:24.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.332+0000 7f029d600640 1 -- 172.21.15.66:0/375375840 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294003860 con 0x7f02a00737c0 2024-09-10T22:10:24.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.332+0000 7f02a8200640 1 -- 172.21.15.66:0/375375840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a0155040 con 0x7f02a00737c0 2024-09-10T22:10:24.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.332+0000 7f02a8200640 1 -- 172.21.15.66:0/375375840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a0155600 con 0x7f02a00737c0 2024-09-10T22:10:24.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.332+0000 7f02a8200640 1 -- 172.21.15.66:0/375375840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a00b6a80 con 0x7f02a00737c0 2024-09-10T22:10:24.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.336+0000 7f029d600640 1 -- 172.21.15.66:0/375375840 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f0294003a00 con 0x7f02a00737c0 2024-09-10T22:10:24.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.336+0000 7f029d600640 1 --2- 172.21.15.66:0/375375840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0280073c20 0x7f02800760e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:24.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.336+0000 7f029d600640 1 -- 172.21.15.66:0/375375840 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5851+0+0 (secure 0 0 0) 0x7f029412bc20 con 0x7f02a00737c0 2024-09-10T22:10:24.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.340+0000 7f02a4e00640 1 --2- 172.21.15.66:0/375375840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0280073c20 0x7f02800760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.340+0000 7f02a4e00640 1 --2- 172.21.15.66:0/375375840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0280073c20 0x7f02800760e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f02a015b9f0 tx=0x7f028c0079c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:24.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.352+0000 7f029d600640 1 -- 172.21.15.66:0/375375840 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0294136080 con 0x7f02a00737c0 2024-09-10T22:10:24.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 -- 172.21.15.66:0/375375840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02a006c840 con 0x7f02a00737c0 2024-09-10T22:10:24.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f029d600640 1 -- 172.21.15.66:0/375375840 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16898 (secure 0 0 0) 0x7f02940164b0 con 0x7f02a00737c0 2024-09-10T22:10:24.563 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:24.564 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":89,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:10:16.942418+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:08:17.494722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T22:08:18.745603+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":88,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.7","mappings":[{"from":6,"to":3}]},{"pgid":"13.a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:10:24.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 -- 172.21.15.66:0/375375840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0280073c20 msgr2=0x7f02800760e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:24.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 --2- 172.21.15.66:0/375375840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0280073c20 0x7f02800760e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f02a015b9f0 tx=0x7f028c0079c0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 -- 172.21.15.66:0/375375840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02a00737c0 msgr2=0x7f02a00d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:24.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 --2- 172.21.15.66:0/375375840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02a00737c0 0x7f02a00d0120 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f029401d160 tx=0x7f0294005770 comp rx=0 tx=0).stop 2024-09-10T22:10:24.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 -- 172.21.15.66:0/375375840 shutdown_connections 2024-09-10T22:10:24.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 --2- 172.21.15.66:0/375375840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0280073c20 0x7f02800760e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 --2- 172.21.15.66:0/375375840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02a0152b90 0x7f02a00b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 --2- 172.21.15.66:0/375375840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a0073f40 0x7f02a01ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 --2- 172.21.15.66:0/375375840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02a00737c0 0x7f02a00d0120 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 -- 172.21.15.66:0/375375840 >> 172.21.15.66:0/375375840 conn(0x7f02a00b26c0 msgr2=0x7f02a00d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:24.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 -- 172.21.15.66:0/375375840 shutdown_connections 2024-09-10T22:10:24.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.560+0000 7f02a8200640 1 -- 172.21.15.66:0/375375840 wait complete. 2024-09-10T22:10:24.586 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:10:24.586 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:10:24.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.792+0000 7f1918400640 1 -- 172.21.15.66:0/635185250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19100d03e0 msgr2=0x7f19100d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:24.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.792+0000 7f1918400640 1 --2- 172.21.15.66:0/635185250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19100d03e0 0x7f19100d0820 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009900 tx=0x7f18fc01c7f0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.796+0000 7f1918400640 1 -- 172.21.15.66:0/635185250 shutdown_connections 2024-09-10T22:10:24.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.796+0000 7f1918400640 1 --2- 172.21.15.66:0/635185250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19100d03e0 0x7f19100d0820 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.796+0000 7f1918400640 1 --2- 172.21.15.66:0/635185250 >> v2:172.21.15.66:3300/0 conn(0x7f191015e010 0x7f19100cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.796+0000 7f1918400640 1 -- 172.21.15.66:0/635185250 >> 172.21.15.66:0/635185250 conn(0x7f19100d1c60 msgr2=0x7f19100d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:24.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.796+0000 7f1918400640 1 -- 172.21.15.66:0/635185250 shutdown_connections 2024-09-10T22:10:24.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.796+0000 7f1918400640 1 -- 172.21.15.66:0/635185250 wait complete. 2024-09-10T22:10:24.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.796+0000 7f1918400640 1 Processor -- start 2024-09-10T22:10:24.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.796+0000 7f1918400640 1 -- start start 2024-09-10T22:10:24.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1918400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19100d03e0 0x7f1910201930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:24.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1918400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19100d0d60 0x7f1910204460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:24.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1918400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f191015e010 0x7f19100b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:24.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1918400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f191016aea0 con 0x7f19100d03e0 2024-09-10T22:10:24.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1918400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f191016ad20 con 0x7f19100d0d60 2024-09-10T22:10:24.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1918400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f191016b020 con 0x7f191015e010 2024-09-10T22:10:24.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1915a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19100d03e0 0x7f1910201930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:24.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1915a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19100d03e0 0x7f1910201930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50756/0 (socket says 172.21.15.66:50756) 2024-09-10T22:10:24.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1915a00640 1 -- 172.21.15.66:0/2079992724 learned_addr learned my addr 172.21.15.66:0/2079992724 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:24.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1916400640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f191015e010 0x7f19100b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:24.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1915000640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19100d0d60 0x7f1910204460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:24.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1915a00640 1 -- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f191015e010 msgr2=0x7f19100b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:24.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1915a00640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f191015e010 0x7f19100b6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1915a00640 1 -- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19100d0d60 msgr2=0x7f1910204460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:24.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1915a00640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19100d0d60 0x7f1910204460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:24.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1915a00640 1 -- 172.21.15.66:0/2079992724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1910201e70 con 0x7f19100d03e0 2024-09-10T22:10:24.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1915a00640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19100d03e0 0x7f1910201930 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f190800ea20 tx=0x7f190800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:24.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f190d600640 1 -- 172.21.15.66:0/2079992724 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190800cde0 con 0x7f19100d03e0 2024-09-10T22:10:24.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f190d600640 1 -- 172.21.15.66:0/2079992724 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1908004c20 con 0x7f19100d03e0 2024-09-10T22:10:24.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f190d600640 1 -- 172.21.15.66:0/2079992724 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908005020 con 0x7f19100d03e0 2024-09-10T22:10:24.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.800+0000 7f1918400640 1 -- 172.21.15.66:0/2079992724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1910204cb0 con 0x7f19100d03e0 2024-09-10T22:10:24.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.804+0000 7f1918400640 1 -- 172.21.15.66:0/2079992724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1910205270 con 0x7f19100d03e0 2024-09-10T22:10:24.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.804+0000 7f1918400640 1 -- 172.21.15.66:0/2079992724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19100d0820 con 0x7f19100d03e0 2024-09-10T22:10:24.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.804+0000 7f190d600640 1 -- 172.21.15.66:0/2079992724 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f190800f040 con 0x7f19100d03e0 2024-09-10T22:10:24.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.808+0000 7f190d600640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f18f0073bd0 0x7f18f0076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:24.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.808+0000 7f190d600640 1 -- 172.21.15.66:0/2079992724 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5851+0+0 (secure 0 0 0) 0x7f19080937d0 con 0x7f19100d03e0 2024-09-10T22:10:24.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.808+0000 7f1915000640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f18f0073bd0 0x7f18f0076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:24.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.808+0000 7f1915000640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f18f0073bd0 0x7f18f0076090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0af000 tx=0x7f18fc01b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:24.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:24.812+0000 7f190d600640 1 -- 172.21.15.66:0/2079992724 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1908016030 con 0x7f19100d03e0 2024-09-10T22:10:25.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.020+0000 7f1918400640 1 -- 172.21.15.66:0/2079992724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19100d14e0 con 0x7f19100d03e0 2024-09-10T22:10:25.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.020+0000 7f190d600640 1 -- 172.21.15.66:0/2079992724 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5222 (secure 0 0 0) 0x7f190805f050 con 0x7f19100d03e0 2024-09-10T22:10:25.026 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:25.026 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":57,"btime":"2024-09-10T22:08:29:488488+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3360242011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3360242011},{"type":"v1","addr":"172.21.15.94:6837","nonce":3360242011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3211574496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3211574496},{"type":"v1","addr":"172.21.15.66:6835","nonce":3211574496}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/553876835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":553876835},{"type":"v1","addr":"172.21.15.66:6834","nonce":553876835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:08:19.775657+0000","modified":"2024-09-10T22:08:22.793838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6836/3187502512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3187502512},{"type":"v1","addr":"172.21.15.94:6836","nonce":3187502512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":6}]} 2024-09-10T22:10:25.026 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 57 2024-09-10T22:10:25.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.024+0000 7f1918400640 1 -- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f18f0073bd0 msgr2=0x7f18f0076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.024+0000 7f1918400640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f18f0073bd0 0x7f18f0076090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0af000 tx=0x7f18fc01b090 comp rx=0 tx=0).stop 2024-09-10T22:10:25.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.024+0000 7f1918400640 1 -- 172.21.15.66:0/2079992724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19100d03e0 msgr2=0x7f1910201930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.024+0000 7f1918400640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19100d03e0 0x7f1910201930 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f190800ea20 tx=0x7f190800eee0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.024+0000 7f1918400640 1 -- 172.21.15.66:0/2079992724 shutdown_connections 2024-09-10T22:10:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.024+0000 7f1918400640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f18f0073bd0 0x7f18f0076090 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.024+0000 7f1918400640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f191015e010 0x7f19100b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.024+0000 7f1918400640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19100d0d60 0x7f1910204460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.024+0000 7f1918400640 1 --2- 172.21.15.66:0/2079992724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19100d03e0 0x7f1910201930 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.024+0000 7f1918400640 1 -- 172.21.15.66:0/2079992724 >> 172.21.15.66:0/2079992724 conn(0x7f19100d1c60 msgr2=0x7f19100b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.024+0000 7f1918400640 1 -- 172.21.15.66:0/2079992724 shutdown_connections 2024-09-10T22:10:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.024+0000 7f1918400640 1 -- 172.21.15.66:0/2079992724 wait complete. 2024-09-10T22:10:25.047 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:10:25.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.192+0000 7f6b82000640 1 -- 172.21.15.66:0/1086145887 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6b7c0ae620 msgr2=0x7f6b7c0725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.192+0000 7f6b82000640 1 --2- 172.21.15.66:0/1086145887 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6b7c0ae620 0x7f6b7c0725d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009840 tx=0x7f6b6401c820 comp rx=0 tx=0).stop 2024-09-10T22:10:25.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.192+0000 7f6b82000640 1 -- 172.21.15.66:0/1086145887 shutdown_connections 2024-09-10T22:10:25.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.192+0000 7f6b82000640 1 --2- 172.21.15.66:0/1086145887 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6b7c0ae620 0x7f6b7c0725d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.192+0000 7f6b82000640 1 --2- 172.21.15.66:0/1086145887 >> v2:172.21.15.94:3300/0 conn(0x7f6b7c0c4b10 0x7f6b7c06e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.192+0000 7f6b82000640 1 -- 172.21.15.66:0/1086145887 >> 172.21.15.66:0/1086145887 conn(0x7f6b7c0cfa30 msgr2=0x7f6b7c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:25.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.192+0000 7f6b82000640 1 -- 172.21.15.66:0/1086145887 shutdown_connections 2024-09-10T22:10:25.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.192+0000 7f6b82000640 1 -- 172.21.15.66:0/1086145887 wait complete. 2024-09-10T22:10:25.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.192+0000 7f6b82000640 1 Processor -- start 2024-09-10T22:10:25.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.192+0000 7f6b82000640 1 -- start start 2024-09-10T22:10:25.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b82000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b7c0ae620 0x7f6b7c1fb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:25.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b82000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6b7c0c4b10 0x7f6b7c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:25.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b82000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b7c0c4ef0 0x7f6b7c1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:25.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b82000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1683e0 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b82000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c168260 con 0x7f6b7c0ae620 2024-09-10T22:10:25.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b82000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7c168560 con 0x7f6b7c0c4b10 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b7c0c4ef0 0x7f6b7c1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b7c0c4ef0 0x7f6b7c1448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50762/0 (socket says 172.21.15.66:50762) 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b7be00640 1 -- 172.21.15.66:0/781282943 learned_addr learned my addr 172.21.15.66:0/781282943 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b7b400640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b7c0ae620 0x7f6b7c1fb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b72a00640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6b7c0c4b10 0x7f6b7c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b7be00640 1 -- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6b7c0c4b10 msgr2=0x7f6b7c0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b7be00640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6b7c0c4b10 0x7f6b7c0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b7be00640 1 -- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b7c0ae620 msgr2=0x7f6b7c1fb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b7be00640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b7c0ae620 0x7f6b7c1fb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b7be00640 1 -- 172.21.15.66:0/781282943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c1fc4f0 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b7be00640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b7c0c4ef0 0x7f6b7c1448b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400cb10 tx=0x7f6b7400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b73e00640 1 -- 172.21.15.66:0/781282943 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74016020 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b73e00640 1 -- 172.21.15.66:0/781282943 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b7400ce50 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b73e00640 1 -- 172.21.15.66:0/781282943 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7400ae00 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b82000640 1 -- 172.21.15.66:0/781282943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c1fbc50 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b82000640 1 -- 172.21.15.66:0/781282943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c1fc210 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.196+0000 7f6b82000640 1 -- 172.21.15.66:0/781282943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b7c0725d0 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.200+0000 7f6b73e00640 1 -- 172.21.15.66:0/781282943 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f6b74012420 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.200+0000 7f6b73e00640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6b58073bd0 0x7f6b58076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:25.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.200+0000 7f6b73e00640 1 -- 172.21.15.66:0/781282943 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5851+0+0 (secure 0 0 0) 0x7f6b740927f0 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.200+0000 7f6b7b400640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6b58073bd0 0x7f6b58076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:25.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.200+0000 7f6b7b400640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6b58073bd0 0x7f6b58076090 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009600 tx=0x7f6b6c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:25.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.204+0000 7f6b73e00640 1 -- 172.21.15.66:0/781282943 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6b74020020 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.412+0000 7f6b82000640 1 -- 172.21.15.66:0/781282943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b7c0b0c00 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.412+0000 7f6b73e00640 1 -- 172.21.15.66:0/781282943 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5222 (secure 0 0 0) 0x7f6b7405eec0 con 0x7f6b7c0c4ef0 2024-09-10T22:10:25.417 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:25.417 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":57,"btime":"2024-09-10T22:08:29:488488+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3360242011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3360242011},{"type":"v1","addr":"172.21.15.94:6837","nonce":3360242011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3211574496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3211574496},{"type":"v1","addr":"172.21.15.66:6835","nonce":3211574496}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/553876835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":553876835},{"type":"v1","addr":"172.21.15.66:6834","nonce":553876835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:08:19.775657+0000","modified":"2024-09-10T22:08:22.793838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6836/3187502512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3187502512},{"type":"v1","addr":"172.21.15.94:6836","nonce":3187502512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":6}]} 2024-09-10T22:10:25.417 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 57 2024-09-10T22:10:25.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.416+0000 7f6b82000640 1 -- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6b58073bd0 msgr2=0x7f6b58076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.416+0000 7f6b82000640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6b58073bd0 0x7f6b58076090 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009600 tx=0x7f6b6c0091e0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.416+0000 7f6b82000640 1 -- 172.21.15.66:0/781282943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b7c0c4ef0 msgr2=0x7f6b7c1448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.416+0000 7f6b82000640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b7c0c4ef0 0x7f6b7c1448b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400cb10 tx=0x7f6b7400cc00 comp rx=0 tx=0).stop 2024-09-10T22:10:25.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.416+0000 7f6b82000640 1 -- 172.21.15.66:0/781282943 shutdown_connections 2024-09-10T22:10:25.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.416+0000 7f6b82000640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6b58073bd0 0x7f6b58076090 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.416+0000 7f6b82000640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b7c0c4ef0 0x7f6b7c1448b0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.416+0000 7f6b82000640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6b7c0c4b10 0x7f6b7c0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.416+0000 7f6b82000640 1 --2- 172.21.15.66:0/781282943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b7c0ae620 0x7f6b7c1fb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.416+0000 7f6b82000640 1 -- 172.21.15.66:0/781282943 >> 172.21.15.66:0/781282943 conn(0x7f6b7c0cfa30 msgr2=0x7f6b7c157c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:25.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.416+0000 7f6b82000640 1 -- 172.21.15.66:0/781282943 shutdown_connections 2024-09-10T22:10:25.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.416+0000 7f6b82000640 1 -- 172.21.15.66:0/781282943 wait complete. 2024-09-10T22:10:25.439 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:10:25.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.580+0000 7f3f86800640 1 -- 172.21.15.66:0/3416694700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f80070b20 msgr2=0x7f3f80070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.580+0000 7f3f86800640 1 --2- 172.21.15.66:0/3416694700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f80070b20 0x7f3f80070f70 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3f80069e40 tx=0x7f3f7801ccf0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.580+0000 7f3f86800640 1 -- 172.21.15.66:0/3416694700 shutdown_connections 2024-09-10T22:10:25.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.580+0000 7f3f86800640 1 --2- 172.21.15.66:0/3416694700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f80070b20 0x7f3f80070f70 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.580+0000 7f3f86800640 1 --2- 172.21.15.66:0/3416694700 >> v2:172.21.15.94:3300/0 conn(0x7f3f80160370 0x7f3f800cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.580+0000 7f3f86800640 1 --2- 172.21.15.66:0/3416694700 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f80156430 0x7f3f8006d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.580+0000 7f3f86800640 1 -- 172.21.15.66:0/3416694700 >> 172.21.15.66:0/3416694700 conn(0x7f3f800d0b30 msgr2=0x7f3f800d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:25.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f86800640 1 -- 172.21.15.66:0/3416694700 shutdown_connections 2024-09-10T22:10:25.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f86800640 1 -- 172.21.15.66:0/3416694700 wait complete. 2024-09-10T22:10:25.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f86800640 1 Processor -- start 2024-09-10T22:10:25.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f86800640 1 -- start start 2024-09-10T22:10:25.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f86800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80070b20 0x7f3f8014c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:25.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f86800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f80156430 0x7f3f8015e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:25.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f86800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f80160370 0x7f3f8015edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:25.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f86800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f800cebb0 con 0x7f3f80160370 2024-09-10T22:10:25.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f86800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f800cea30 con 0x7f3f80070b20 2024-09-10T22:10:25.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f86800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3f800ced30 con 0x7f3f80156430 2024-09-10T22:10:25.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f7fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f80160370 0x7f3f8015edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:25.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f7fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f80160370 0x7f3f8015edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50772/0 (socket says 172.21.15.66:50772) 2024-09-10T22:10:25.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f7fe00640 1 -- 172.21.15.66:0/3367895443 learned_addr learned my addr 172.21.15.66:0/3367895443 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:25.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f7f400640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80070b20 0x7f3f8014c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:25.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f77e00640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f80156430 0x7f3f8015e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f7fe00640 1 -- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f80156430 msgr2=0x7f3f8015e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f7fe00640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f80156430 0x7f3f8015e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f7fe00640 1 -- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80070b20 msgr2=0x7f3f8014c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f7fe00640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80070b20 0x7f3f8014c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f7fe00640 1 -- 172.21.15.66:0/3367895443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f801f83b0 con 0x7f3f80160370 2024-09-10T22:10:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f77e00640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f80156430 0x7f3f8015e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f7fe00640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f80160370 0x7f3f8015edb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009bd0 tx=0x7f3f78003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f77400640 1 -- 172.21.15.66:0/3367895443 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f780bb070 con 0x7f3f80160370 2024-09-10T22:10:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f77400640 1 -- 172.21.15.66:0/3367895443 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f78004c50 con 0x7f3f80160370 2024-09-10T22:10:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f77400640 1 -- 172.21.15.66:0/3367895443 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f780b2040 con 0x7f3f80160370 2024-09-10T22:10:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.584+0000 7f3f86800640 1 -- 172.21.15.66:0/3367895443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8014cc60 con 0x7f3f80160370 2024-09-10T22:10:25.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.588+0000 7f3f86800640 1 -- 172.21.15.66:0/3367895443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f800cd100 con 0x7f3f80160370 2024-09-10T22:10:25.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.588+0000 7f3f86800640 1 -- 172.21.15.66:0/3367895443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8006d630 con 0x7f3f80160370 2024-09-10T22:10:25.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.588+0000 7f3f77400640 1 -- 172.21.15.66:0/3367895443 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f3f780b5030 con 0x7f3f80160370 2024-09-10T22:10:25.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.588+0000 7f3f77400640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3f5c073bd0 0x7f3f5c076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:25.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.588+0000 7f3f77400640 1 -- 172.21.15.66:0/3367895443 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5851+0+0 (secure 0 0 0) 0x7f3f7812bbb0 con 0x7f3f80160370 2024-09-10T22:10:25.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.588+0000 7f3f7f400640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3f5c073bd0 0x7f3f5c076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:25.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.592+0000 7f3f7f400640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3f5c073bd0 0x7f3f5c076090 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009b60 tx=0x7f3f700091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:25.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.592+0000 7f3f77400640 1 -- 172.21.15.66:0/3367895443 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3f78016970 con 0x7f3f80160370 2024-09-10T22:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.768+0000 7f3f86800640 1 -- 172.21.15.66:0/3367895443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f80073780 con 0x7f3f80160370 2024-09-10T22:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.768+0000 7f3f77400640 1 -- 172.21.15.66:0/3367895443 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16898 (secure 0 0 0) 0x7f3f78101400 con 0x7f3f80160370 2024-09-10T22:10:25.773 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:25.773 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":89,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:10:16.942418+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:08:17.494722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T22:08:18.745603+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":88,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.7","mappings":[{"from":6,"to":3}]},{"pgid":"13.a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:10:25.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.772+0000 7f3f86800640 1 -- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3f5c073bd0 msgr2=0x7f3f5c076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.772+0000 7f3f86800640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3f5c073bd0 0x7f3f5c076090 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009b60 tx=0x7f3f700091e0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.772+0000 7f3f86800640 1 -- 172.21.15.66:0/3367895443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f80160370 msgr2=0x7f3f8015edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.772+0000 7f3f86800640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f80160370 0x7f3f8015edb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009bd0 tx=0x7f3f78003f60 comp rx=0 tx=0).stop 2024-09-10T22:10:25.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.772+0000 7f3f86800640 1 -- 172.21.15.66:0/3367895443 shutdown_connections 2024-09-10T22:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.772+0000 7f3f86800640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3f5c073bd0 0x7f3f5c076090 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.772+0000 7f3f86800640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f80160370 0x7f3f8015edb0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.772+0000 7f3f86800640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f80156430 0x7f3f8015e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.772+0000 7f3f86800640 1 --2- 172.21.15.66:0/3367895443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80070b20 0x7f3f8014c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.772+0000 7f3f86800640 1 -- 172.21.15.66:0/3367895443 >> 172.21.15.66:0/3367895443 conn(0x7f3f800d0b30 msgr2=0x7f3f8015a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.772+0000 7f3f86800640 1 -- 172.21.15.66:0/3367895443 shutdown_connections 2024-09-10T22:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.772+0000 7f3f86800640 1 -- 172.21.15.66:0/3367895443 wait complete. 2024-09-10T22:10:25.795 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:10:25.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.940+0000 7fa82e800640 1 -- 172.21.15.66:0/1045461956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa82815f300 msgr2=0x7fa8280b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.940+0000 7fa82e800640 1 --2- 172.21.15.66:0/1045461956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa82815f300 0x7fa8280b1090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa814009870 tx=0x7fa81401c740 comp rx=0 tx=0).stop 2024-09-10T22:10:25.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.940+0000 7fa82e800640 1 -- 172.21.15.66:0/1045461956 shutdown_connections 2024-09-10T22:10:25.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.940+0000 7fa82e800640 1 --2- 172.21.15.66:0/1045461956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa82815f300 0x7fa8280b1090 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.940+0000 7fa82e800640 1 -- 172.21.15.66:0/1045461956 >> 172.21.15.66:0/1045461956 conn(0x7fa8280d1b10 msgr2=0x7fa8280d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:25.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.940+0000 7fa82e800640 1 -- 172.21.15.66:0/1045461956 shutdown_connections 2024-09-10T22:10:25.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.940+0000 7fa82e800640 1 -- 172.21.15.66:0/1045461956 wait complete. 2024-09-10T22:10:25.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82e800640 1 Processor -- start 2024-09-10T22:10:25.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82e800640 1 -- start start 2024-09-10T22:10:25.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82e800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8280b15d0 0x7fa828159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:25.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82815ef20 0x7fa8280b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:25.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa82815f300 0x7fa8280aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:25.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa82816a0e0 con 0x7fa82815ef20 2024-09-10T22:10:25.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82e800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa828169f60 con 0x7fa82815f300 2024-09-10T22:10:25.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82e800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa82816a260 con 0x7fa8280b15d0 2024-09-10T22:10:25.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82815ef20 0x7fa8280b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:25.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82815ef20 0x7fa8280b6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50782/0 (socket says 172.21.15.66:50782) 2024-09-10T22:10:25.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82ca00640 1 -- 172.21.15.66:0/1277364101 learned_addr learned my addr 172.21.15.66:0/1277364101 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:25.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82de00640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa82815f300 0x7fa8280aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:25.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82d400640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8280b15d0 0x7fa828159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:25.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82ca00640 1 -- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8280b15d0 msgr2=0x7fa828159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82ca00640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8280b15d0 0x7fa828159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82ca00640 1 -- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa82815f300 msgr2=0x7fa8280aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:25.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82ca00640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa82815f300 0x7fa8280aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:25.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82ca00640 1 -- 172.21.15.66:0/1277364101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8280b5460 con 0x7fa82815ef20 2024-09-10T22:10:25.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82ca00640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82815ef20 0x7fa8280b6eb0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa814019350 tx=0x7fa814005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:25.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa825600640 1 -- 172.21.15.66:0/1277364101 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8140be070 con 0x7fa82815ef20 2024-09-10T22:10:25.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa825600640 1 -- 172.21.15.66:0/1277364101 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8140a6860 con 0x7fa82815ef20 2024-09-10T22:10:25.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82e800640 1 -- 172.21.15.66:0/1277364101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8280b5740 con 0x7fa82815ef20 2024-09-10T22:10:25.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa825600640 1 -- 172.21.15.66:0/1277364101 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8140a6c30 con 0x7fa82815ef20 2024-09-10T22:10:25.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.944+0000 7fa82e800640 1 -- 172.21.15.66:0/1277364101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8280b5d00 con 0x7fa82815ef20 2024-09-10T22:10:25.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.948+0000 7fa82e800640 1 -- 172.21.15.66:0/1277364101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8280b1090 con 0x7fa82815ef20 2024-09-10T22:10:25.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.948+0000 7fa825600640 1 -- 172.21.15.66:0/1277364101 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fa8280d0100 con 0x7fa82815ef20 2024-09-10T22:10:25.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.948+0000 7fa825600640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa808073bd0 0x7fa808076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:25.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.948+0000 7fa825600640 1 -- 172.21.15.66:0/1277364101 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5851+0+0 (secure 0 0 0) 0x7fa81412b9e0 con 0x7fa82815ef20 2024-09-10T22:10:25.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.948+0000 7fa82d400640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa808073bd0 0x7fa808076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:25.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.948+0000 7fa82d400640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa808073bd0 0x7fa808076090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa81c000fc0 tx=0x7fa81c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:25.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:25.952+0000 7fa825600640 1 -- 172.21.15.66:0/1277364101 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa8140ac030 con 0x7fa82815ef20 2024-09-10T22:10:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.164+0000 7fa82e800640 1 -- 172.21.15.66:0/1277364101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa8280b5ed0 con 0x7fa82815ef20 2024-09-10T22:10:26.168 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:10:26.164+0000 7fcc93a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:10:26.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.164+0000 7fa825600640 1 -- 172.21.15.66:0/1277364101 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) ==== 161+0+0 (secure 0 0 0) 0x7fa814016370 con 0x7fa82815ef20 2024-09-10T22:10:26.170 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:10:26.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.168+0000 7fa82e800640 1 -- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa808073bd0 msgr2=0x7fa808076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:26.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.168+0000 7fa82e800640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa808073bd0 0x7fa808076090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa81c000fc0 tx=0x7fa81c009040 comp rx=0 tx=0).stop 2024-09-10T22:10:26.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.168+0000 7fa82e800640 1 -- 172.21.15.66:0/1277364101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82815ef20 msgr2=0x7fa8280b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:26.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.168+0000 7fa82e800640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82815ef20 0x7fa8280b6eb0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa814019350 tx=0x7fa814005cc0 comp rx=0 tx=0).stop 2024-09-10T22:10:26.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.168+0000 7fa82e800640 1 -- 172.21.15.66:0/1277364101 shutdown_connections 2024-09-10T22:10:26.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.168+0000 7fa82e800640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa808073bd0 0x7fa808076090 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:26.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.168+0000 7fa82e800640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa82815f300 0x7fa8280aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:26.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.168+0000 7fa82e800640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82815ef20 0x7fa8280b6eb0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:26.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.168+0000 7fa82e800640 1 --2- 172.21.15.66:0/1277364101 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa8280b15d0 0x7fa828159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:26.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.168+0000 7fa82e800640 1 -- 172.21.15.66:0/1277364101 >> 172.21.15.66:0/1277364101 conn(0x7fa8280d1b10 msgr2=0x7fa82815e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:26.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.168+0000 7fa82e800640 1 -- 172.21.15.66:0/1277364101 shutdown_connections 2024-09-10T22:10:26.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.168+0000 7fa82e800640 1 -- 172.21.15.66:0/1277364101 wait complete. 2024-09-10T22:10:26.194 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:10:26.241 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:10:26.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 -- 172.21.15.66:0/800942001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8770003a10 msgr2=0x7f8770003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:26.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 --2- 172.21.15.66:0/800942001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8770003a10 0x7f8770003e30 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f877c00c340 tx=0x7f877c01cf70 comp rx=0 tx=0).stop 2024-09-10T22:10:26.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f878a800640 1 -- 172.21.15.66:0/800942001 reap_dead start 2024-09-10T22:10:26.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 -- 172.21.15.66:0/800942001 shutdown_connections 2024-09-10T22:10:26.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 -- 172.21.15.66:0/800942001 >> 172.21.15.66:0/800942001 conn(0x7f878c0cfa30 msgr2=0x7f878c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:26.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 -- 172.21.15.66:0/800942001 shutdown_connections 2024-09-10T22:10:26.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 -- 172.21.15.66:0/800942001 wait complete. 2024-09-10T22:10:26.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 Processor -- start 2024-09-10T22:10:26.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 -- start start 2024-09-10T22:10:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8770003a10 0x7f878c0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f878c0b01a0 0x7f878c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f878c1fb6f0 0x7f878c1fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f878c164aa0 con 0x7f8770003a10 2024-09-10T22:10:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f878c164920 con 0x7f878c0b01a0 2024-09-10T22:10:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8790c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f878c164c20 con 0x7f878c1fb6f0 2024-09-10T22:10:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8789e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8770003a10 0x7f878c0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8789e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8770003a10 0x7f878c0b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50800/0 (socket says 172.21.15.66:50800) 2024-09-10T22:10:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8789e00640 1 -- 172.21.15.66:0/2606892647 learned_addr learned my addr 172.21.15.66:0/2606892647 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f878a800640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f878c1fb6f0 0x7f878c1fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.344+0000 7f8783e00640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f878c0b01a0 0x7f878c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8789e00640 1 -- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f878c1fb6f0 msgr2=0x7f878c1fbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8789e00640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f878c1fb6f0 0x7f878c1fbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8789e00640 1 -- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f878c0b01a0 msgr2=0x7f878c0b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8789e00640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f878c0b01a0 0x7f878c0b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8789e00640 1 -- 172.21.15.66:0/2606892647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c1f7ea0 con 0x7f8770003a10 2024-09-10T22:10:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8789e00640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8770003a10 0x7f878c0b2ca0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f877c01ca30 tx=0x7f877c001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8782000640 1 -- 172.21.15.66:0/2606892647 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c003600 con 0x7f8770003a10 2024-09-10T22:10:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8782000640 1 -- 172.21.15.66:0/2606892647 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f877c003890 con 0x7f8770003a10 2024-09-10T22:10:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8782000640 1 -- 172.21.15.66:0/2606892647 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c003ca0 con 0x7f8770003a10 2024-09-10T22:10:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8790c00640 1 -- 172.21.15.66:0/2606892647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c0b3280 con 0x7f8770003a10 2024-09-10T22:10:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8790c00640 1 -- 172.21.15.66:0/2606892647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c1f8440 con 0x7f8770003a10 2024-09-10T22:10:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8790c00640 1 -- 172.21.15.66:0/2606892647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878c15d640 con 0x7f8770003a10 2024-09-10T22:10:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.348+0000 7f8782000640 1 -- 172.21.15.66:0/2606892647 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f877c0c2030 con 0x7f8770003a10 2024-09-10T22:10:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.352+0000 7f8782000640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8768073c00 0x7f87680760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.352+0000 7f8782000640 1 -- 172.21.15.66:0/2606892647 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5909+0+0 (secure 0 0 0) 0x7f877c12c3c0 con 0x7f8770003a10 2024-09-10T22:10:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.352+0000 7f8783e00640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8768073c00 0x7f87680760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.352+0000 7f8783e00640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8768073c00 0x7f87680760c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f877400afd0 tx=0x7f877400c040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:26.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.356+0000 7f8782000640 1 -- 172.21.15.66:0/2606892647 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f877c016d50 con 0x7f8770003a10 2024-09-10T22:10:26.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:26.564+0000 7f8790c00640 1 -- 172.21.15.66:0/2606892647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f878c0d0120 con 0x7f8770003a10 2024-09-10T22:10:27.121 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:10:27.125 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:10:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8782000640 1 -- 172.21.15.66:0/2606892647 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) ==== 104+0+0 (secure 0 0 0) 0x7f877c12c860 con 0x7f8770003a10 2024-09-10T22:10:27.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8790c00640 1 -- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8768073c00 msgr2=0x7f87680760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:27.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8790c00640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8768073c00 0x7f87680760c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f877400afd0 tx=0x7f877400c040 comp rx=0 tx=0).stop 2024-09-10T22:10:27.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8790c00640 1 -- 172.21.15.66:0/2606892647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8770003a10 msgr2=0x7f878c0b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8790c00640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8770003a10 0x7f878c0b2ca0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f877c01ca30 tx=0x7f877c001d70 comp rx=0 tx=0).stop 2024-09-10T22:10:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8790c00640 1 -- 172.21.15.66:0/2606892647 shutdown_connections 2024-09-10T22:10:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8790c00640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8768073c00 0x7f87680760c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8790c00640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f878c1fb6f0 0x7f878c1fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8790c00640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f878c0b01a0 0x7f878c0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8790c00640 1 --2- 172.21.15.66:0/2606892647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8770003a10 0x7f878c0b2ca0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8790c00640 1 -- 172.21.15.66:0/2606892647 >> 172.21.15.66:0/2606892647 conn(0x7f878c0cfa30 msgr2=0x7f878c0ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8790c00640 1 -- 172.21.15.66:0/2606892647 shutdown_connections 2024-09-10T22:10:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.172+0000 7f8790c00640 1 -- 172.21.15.66:0/2606892647 wait complete. 2024-09-10T22:10:27.197 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:10:27.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 -- 172.21.15.66:0/3426291395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c15a3b0 msgr2=0x7fb94c072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:27.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 --2- 172.21.15.66:0/3426291395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c15a3b0 0x7fb94c072640 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb94c070470 tx=0x7fb93401c8c0 comp rx=0 tx=0).stop 2024-09-10T22:10:27.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 -- 172.21.15.66:0/3426291395 shutdown_connections 2024-09-10T22:10:27.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 --2- 172.21.15.66:0/3426291395 >> v2:172.21.15.94:3300/0 conn(0x7fb94c072b80 0x7fb94c15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:27.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 --2- 172.21.15.66:0/3426291395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c15a3b0 0x7fb94c072640 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 --2- 172.21.15.66:0/3426291395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb94c150dc0 0x7fb94c0b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 -- 172.21.15.66:0/3426291395 >> 172.21.15.66:0/3426291395 conn(0x7fb94c0cfaa0 msgr2=0x7fb94c0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 -- 172.21.15.66:0/3426291395 shutdown_connections 2024-09-10T22:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 -- 172.21.15.66:0/3426291395 wait complete. 2024-09-10T22:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 Processor -- start 2024-09-10T22:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 -- start start 2024-09-10T22:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb94c072b80 0x7fb94c146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb94c150dc0 0x7fb94c143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c15a3b0 0x7fb94c1fc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c06f270 con 0x7fb94c15a3b0 2024-09-10T22:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c06f0f0 con 0x7fb94c072b80 2024-09-10T22:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb950a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb94c06f3f0 con 0x7fb94c150dc0 2024-09-10T22:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb949e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb94c072b80 0x7fb94c146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb94a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c15a3b0 0x7fb94c1fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb943e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb94c150dc0 0x7fb94c143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb949e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb94c072b80 0x7fb94c146df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:55830/0 (socket says 172.21.15.66:55830) 2024-09-10T22:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb94a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c15a3b0 0x7fb94c1fc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50820/0 (socket says 172.21.15.66:50820) 2024-09-10T22:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb943e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb94c150dc0 0x7fb94c143810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:51546/0 (socket says 172.21.15.66:51546) 2024-09-10T22:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb949e00640 1 -- 172.21.15.66:0/4025609173 learned_addr learned my addr 172.21.15.66:0/4025609173 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb94a800640 1 -- 172.21.15.66:0/4025609173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb94c150dc0 msgr2=0x7fb94c143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb94a800640 1 --2- 172.21.15.66:0/4025609173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb94c150dc0 0x7fb94c143810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb94a800640 1 -- 172.21.15.66:0/4025609173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb94c072b80 msgr2=0x7fb94c146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb94a800640 1 --2- 172.21.15.66:0/4025609173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb94c072b80 0x7fb94c146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.348+0000 7fb94a800640 1 -- 172.21.15.66:0/4025609173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c147330 con 0x7fb94c15a3b0 2024-09-10T22:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.352+0000 7fb94a800640 1 --2- 172.21.15.66:0/4025609173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c15a3b0 0x7fb94c1fc010 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb94400db90 tx=0x7fb94400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.352+0000 7fb942000640 1 -- 172.21.15.66:0/4025609173 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944004d30 con 0x7fb94c15a3b0 2024-09-10T22:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.352+0000 7fb942000640 1 -- 172.21.15.66:0/4025609173 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb944003980 con 0x7fb94c15a3b0 2024-09-10T22:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.352+0000 7fb942000640 1 -- 172.21.15.66:0/4025609173 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944003d70 con 0x7fb94c15a3b0 2024-09-10T22:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.352+0000 7fb950a00640 1 -- 172.21.15.66:0/4025609173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94c1442d0 con 0x7fb94c15a3b0 2024-09-10T22:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.352+0000 7fb950a00640 1 -- 172.21.15.66:0/4025609173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94c144890 con 0x7fb94c15a3b0 2024-09-10T22:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.352+0000 7fb950a00640 1 -- 172.21.15.66:0/4025609173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94c06f910 con 0x7fb94c15a3b0 2024-09-10T22:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.352+0000 7fb942000640 1 -- 172.21.15.66:0/4025609173 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fb94400b040 con 0x7fb94c15a3b0 2024-09-10T22:10:27.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.352+0000 7fb942000640 1 --2- 172.21.15.66:0/4025609173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb928073bd0 0x7fb928076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:27.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.352+0000 7fb942000640 1 -- 172.21.15.66:0/4025609173 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5909+0+0 (secure 0 0 0) 0x7fb944092360 con 0x7fb94c15a3b0 2024-09-10T22:10:27.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.356+0000 7fb949e00640 1 --2- 172.21.15.66:0/4025609173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb928073bd0 0x7fb928076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:27.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.356+0000 7fb949e00640 1 --2- 172.21.15.66:0/4025609173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb928073bd0 0x7fb928076090 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb94c14cc20 tx=0x7fb93c00b410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:27.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.356+0000 7fb942000640 1 -- 172.21.15.66:0/4025609173 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb94405eea0 con 0x7fb94c15a3b0 2024-09-10T22:10:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:27.536+0000 7fb950a00640 1 -- 172.21.15.66:0/4025609173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb94c0b4a30 con 0x7fb94c15a3b0 2024-09-10T22:10:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.172+0000 7fb942000640 1 -- 172.21.15.66:0/4025609173 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7fb9440638d0 con 0x7fb94c15a3b0 2024-09-10T22:10:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:10:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.176+0000 7fb950a00640 1 -- 172.21.15.66:0/4025609173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb928073bd0 msgr2=0x7fb928076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.176+0000 7fb950a00640 1 --2- 172.21.15.66:0/4025609173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb928073bd0 0x7fb928076090 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb94c14cc20 tx=0x7fb93c00b410 comp rx=0 tx=0).stop 2024-09-10T22:10:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.176+0000 7fb950a00640 1 -- 172.21.15.66:0/4025609173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c15a3b0 msgr2=0x7fb94c1fc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.176+0000 7fb950a00640 1 --2- 172.21.15.66:0/4025609173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c15a3b0 0x7fb94c1fc010 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb94400db90 tx=0x7fb94400b730 comp rx=0 tx=0).stop 2024-09-10T22:10:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.176+0000 7fb950a00640 1 -- 172.21.15.66:0/4025609173 shutdown_connections 2024-09-10T22:10:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.176+0000 7fb950a00640 1 --2- 172.21.15.66:0/4025609173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb928073bd0 0x7fb928076090 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.176+0000 7fb950a00640 1 --2- 172.21.15.66:0/4025609173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c15a3b0 0x7fb94c1fc010 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.176+0000 7fb950a00640 1 --2- 172.21.15.66:0/4025609173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb94c150dc0 0x7fb94c143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.176+0000 7fb950a00640 1 --2- 172.21.15.66:0/4025609173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb94c072b80 0x7fb94c146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.176+0000 7fb950a00640 1 -- 172.21.15.66:0/4025609173 >> 172.21.15.66:0/4025609173 conn(0x7fb94c0cfaa0 msgr2=0x7fb94c14e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.176+0000 7fb950a00640 1 -- 172.21.15.66:0/4025609173 shutdown_connections 2024-09-10T22:10:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.176+0000 7fb950a00640 1 -- 172.21.15.66:0/4025609173 wait complete. 2024-09-10T22:10:28.203 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:10:28.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 -- 172.21.15.66:0/1485972154 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe380cc340 msgr2=0x7fbe380ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:28.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 --2- 172.21.15.66:0/1485972154 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe380cc340 0x7fbe380ce560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009870 tx=0x7fbe2801c780 comp rx=0 tx=0).stop 2024-09-10T22:10:28.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 -- 172.21.15.66:0/1485972154 shutdown_connections 2024-09-10T22:10:28.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 --2- 172.21.15.66:0/1485972154 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe380cc340 0x7fbe380ce560 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:28.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 -- 172.21.15.66:0/1485972154 >> 172.21.15.66:0/1485972154 conn(0x7fbe380d0350 msgr2=0x7fbe380d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:28.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 -- 172.21.15.66:0/1485972154 shutdown_connections 2024-09-10T22:10:28.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 -- 172.21.15.66:0/1485972154 wait complete. 2024-09-10T22:10:28.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 Processor -- start 2024-09-10T22:10:28.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 -- start start 2024-09-10T22:10:28.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe380b0520 0x7fbe38156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:28.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe380cc340 0x7fbe38156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:28.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe380ce180 0x7fbe38157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:28.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381693a0 con 0x7fbe380b0520 2024-09-10T22:10:28.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38169220 con 0x7fbe380ce180 2024-09-10T22:10:28.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe40800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbe38169520 con 0x7fbe380cc340 2024-09-10T22:10:28.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe3de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe380b0520 0x7fbe38156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:28.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe3d400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe380cc340 0x7fbe38156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:28.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe3de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe380b0520 0x7fbe38156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50834/0 (socket says 172.21.15.66:50834) 2024-09-10T22:10:28.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe3de00640 1 -- 172.21.15.66:0/1349950799 learned_addr learned my addr 172.21.15.66:0/1349950799 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:28.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe3e800640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe380ce180 0x7fbe38157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:28.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe3de00640 1 -- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe380cc340 msgr2=0x7fbe38156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:28.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.396+0000 7fbe3de00640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe380cc340 0x7fbe38156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:28.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.400+0000 7fbe3de00640 1 -- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe380ce180 msgr2=0x7fbe38157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:28.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.400+0000 7fbe3de00640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe380ce180 0x7fbe38157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:28.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.400+0000 7fbe3de00640 1 -- 172.21.15.66:0/1349950799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe380708d0 con 0x7fbe380b0520 2024-09-10T22:10:28.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.400+0000 7fbe3de00640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe380b0520 0x7fbe38156780 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00c400 tx=0x7fbe2c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:28.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.400+0000 7fbe32000640 1 -- 172.21.15.66:0/1349950799 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c017070 con 0x7fbe380b0520 2024-09-10T22:10:28.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.400+0000 7fbe32000640 1 -- 172.21.15.66:0/1349950799 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe2c005400 con 0x7fbe380b0520 2024-09-10T22:10:28.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.400+0000 7fbe32000640 1 -- 172.21.15.66:0/1349950799 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c003860 con 0x7fbe380b0520 2024-09-10T22:10:28.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.400+0000 7fbe40800640 1 -- 172.21.15.66:0/1349950799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3806fd20 con 0x7fbe380b0520 2024-09-10T22:10:28.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.400+0000 7fbe40800640 1 -- 172.21.15.66:0/1349950799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe38070560 con 0x7fbe380b0520 2024-09-10T22:10:28.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.400+0000 7fbe40800640 1 -- 172.21.15.66:0/1349950799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe380ce560 con 0x7fbe380b0520 2024-09-10T22:10:28.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.400+0000 7fbe32000640 1 -- 172.21.15.66:0/1349950799 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fbe2c003a00 con 0x7fbe380b0520 2024-09-10T22:10:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.400+0000 7fbe32000640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbe1c073bd0 0x7fbe1c076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.404+0000 7fbe32000640 1 -- 172.21.15.66:0/1349950799 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5471+0+0 (secure 0 0 0) 0x7fbe2c091a80 con 0x7fbe380b0520 2024-09-10T22:10:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.404+0000 7fbe3d400640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbe1c073bd0 0x7fbe1c076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:28.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.404+0000 7fbe3d400640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbe1c073bd0 0x7fbe1c076090 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbe280ae000 tx=0x7fbe2801afc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:28.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.408+0000 7fbe32000640 1 -- 172.21.15.66:0/1349950799 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbe3806fd20 con 0x7fbe380b0520 2024-09-10T22:10:28.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:28.592+0000 7fbe40800640 1 -- 172.21.15.66:0/1349950799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbe38074af0 con 0x7fbe380b0520 2024-09-10T22:10:29.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.180+0000 7fbe32000640 1 -- 172.21.15.66:0/1349950799 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7fbe2c091f20 con 0x7fbe380b0520 2024-09-10T22:10:29.183 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:10:29.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.184+0000 7fbe40800640 1 -- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbe1c073bd0 msgr2=0x7fbe1c076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:29.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.184+0000 7fbe40800640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbe1c073bd0 0x7fbe1c076090 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbe280ae000 tx=0x7fbe2801afc0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.184+0000 7fbe40800640 1 -- 172.21.15.66:0/1349950799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe380b0520 msgr2=0x7fbe38156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.184+0000 7fbe40800640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe380b0520 0x7fbe38156780 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00c400 tx=0x7fbe2c00c8c0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.184+0000 7fbe40800640 1 -- 172.21.15.66:0/1349950799 shutdown_connections 2024-09-10T22:10:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.184+0000 7fbe40800640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbe1c073bd0 0x7fbe1c076090 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.184+0000 7fbe40800640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe380ce180 0x7fbe38157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.184+0000 7fbe40800640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe380cc340 0x7fbe38156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.184+0000 7fbe40800640 1 --2- 172.21.15.66:0/1349950799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe380b0520 0x7fbe38156780 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.184+0000 7fbe40800640 1 -- 172.21.15.66:0/1349950799 >> 172.21.15.66:0/1349950799 conn(0x7fbe380d0350 msgr2=0x7fbe380b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.184+0000 7fbe40800640 1 -- 172.21.15.66:0/1349950799 shutdown_connections 2024-09-10T22:10:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.184+0000 7fbe40800640 1 -- 172.21.15.66:0/1349950799 wait complete. 2024-09-10T22:10:29.216 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T22:10:29.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.360+0000 7f2e5fe00640 1 -- 172.21.15.66:0/152328610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6015ef20 msgr2=0x7f2e600cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:29.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.360+0000 7f2e5fe00640 1 --2- 172.21.15.66:0/152328610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6015ef20 0x7f2e600cecc0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009930 tx=0x7f2e5001c8e0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.360+0000 7f2e5fe00640 1 -- 172.21.15.66:0/152328610 shutdown_connections 2024-09-10T22:10:29.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.360+0000 7f2e5fe00640 1 --2- 172.21.15.66:0/152328610 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e600cd7a0 0x7f2e600cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.360+0000 7f2e5fe00640 1 --2- 172.21.15.66:0/152328610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6015ef20 0x7f2e600cecc0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.360+0000 7f2e5fe00640 1 -- 172.21.15.66:0/152328610 >> 172.21.15.66:0/152328610 conn(0x7f2e600d1b10 msgr2=0x7f2e600d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:29.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.360+0000 7f2e5fe00640 1 -- 172.21.15.66:0/152328610 shutdown_connections 2024-09-10T22:10:29.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.360+0000 7f2e5fe00640 1 -- 172.21.15.66:0/152328610 wait complete. 2024-09-10T22:10:29.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e5fe00640 1 Processor -- start 2024-09-10T22:10:29.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e5fe00640 1 -- start start 2024-09-10T22:10:29.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e5fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e600cd7a0 0x7f2e601647c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:29.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e5fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6015ef20 0x7f2e60164f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:29.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e5fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6015f300 0x7f2e601522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e5fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60168ee0 con 0x7f2e6015ef20 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e5fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60168d60 con 0x7f2e600cd7a0 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e5fe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2e60169060 con 0x7f2e6015f300 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e56000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6015ef20 0x7f2e60164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e56000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6015ef20 0x7f2e60164f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50850/0 (socket says 172.21.15.66:50850) 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e56000640 1 -- 172.21.15.66:0/1895193714 learned_addr learned my addr 172.21.15.66:0/1895193714 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e5f400640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6015f300 0x7f2e601522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e5ea00640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e600cd7a0 0x7f2e601647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e56000640 1 -- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6015f300 msgr2=0x7f2e601522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e56000640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6015f300 0x7f2e601522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e56000640 1 -- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e600cd7a0 msgr2=0x7f2e601647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e56000640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e600cd7a0 0x7f2e601647c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e56000640 1 -- 172.21.15.66:0/1895193714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e60165480 con 0x7f2e6015ef20 2024-09-10T22:10:29.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e56000640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6015ef20 0x7f2e60164f40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800c3c0 tx=0x7f2e4800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:29.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e57e00640 1 -- 172.21.15.66:0/1895193714 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48017070 con 0x7f2e6015ef20 2024-09-10T22:10:29.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e57e00640 1 -- 172.21.15.66:0/1895193714 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e480053c0 con 0x7f2e6015ef20 2024-09-10T22:10:29.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e57e00640 1 -- 172.21.15.66:0/1895193714 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48003860 con 0x7f2e6015ef20 2024-09-10T22:10:29.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e5fe00640 1 -- 172.21.15.66:0/1895193714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e60165750 con 0x7f2e6015ef20 2024-09-10T22:10:29.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.364+0000 7f2e5fe00640 1 -- 172.21.15.66:0/1895193714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e60163fc0 con 0x7f2e6015ef20 2024-09-10T22:10:29.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.368+0000 7f2e5fe00640 1 -- 172.21.15.66:0/1895193714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e600cdc30 con 0x7f2e6015ef20 2024-09-10T22:10:29.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.368+0000 7f2e57e00640 1 -- 172.21.15.66:0/1895193714 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f2e48003a00 con 0x7f2e6015ef20 2024-09-10T22:10:29.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.368+0000 7f2e57e00640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2e3c073bd0 0x7f2e3c076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:29.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.368+0000 7f2e57e00640 1 -- 172.21.15.66:0/1895193714 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7f2e48091970 con 0x7f2e6015ef20 2024-09-10T22:10:29.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.368+0000 7f2e5ea00640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2e3c073bd0 0x7f2e3c076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:29.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.368+0000 7f2e5ea00640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2e3c073bd0 0x7f2e3c076090 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2e500af000 tx=0x7f2e5001b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:29.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.372+0000 7f2e57e00640 1 -- 172.21.15.66:0/1895193714 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2e4805e690 con 0x7f2e6015ef20 2024-09-10T22:10:29.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.560+0000 7f2e5fe00640 1 -- 172.21.15.66:0/1895193714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f2e60072e00 con 0x7f2e6015ef20 2024-09-10T22:10:29.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.564+0000 7f2e57e00640 1 -- 172.21.15.66:0/1895193714 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v44) ==== 71+0+0 (secure 0 0 0) 0x7f2e48001090 con 0x7f2e6015ef20 2024-09-10T22:10:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.568+0000 7f2e5fe00640 1 -- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2e3c073bd0 msgr2=0x7f2e3c076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.568+0000 7f2e5fe00640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2e3c073bd0 0x7f2e3c076090 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2e500af000 tx=0x7f2e5001b120 comp rx=0 tx=0).stop 2024-09-10T22:10:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.568+0000 7f2e5fe00640 1 -- 172.21.15.66:0/1895193714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6015ef20 msgr2=0x7f2e60164f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.568+0000 7f2e5fe00640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6015ef20 0x7f2e60164f40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800c3c0 tx=0x7f2e4800c880 comp rx=0 tx=0).stop 2024-09-10T22:10:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.568+0000 7f2e5fe00640 1 -- 172.21.15.66:0/1895193714 shutdown_connections 2024-09-10T22:10:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.568+0000 7f2e5fe00640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2e3c073bd0 0x7f2e3c076090 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.568+0000 7f2e5fe00640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e6015f300 0x7f2e601522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.568+0000 7f2e5fe00640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e6015ef20 0x7f2e60164f40 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.568+0000 7f2e5fe00640 1 --2- 172.21.15.66:0/1895193714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e600cd7a0 0x7f2e601647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.568+0000 7f2e5fe00640 1 -- 172.21.15.66:0/1895193714 >> 172.21.15.66:0/1895193714 conn(0x7f2e600d1b10 msgr2=0x7f2e6006f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:29.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.568+0000 7f2e5fe00640 1 -- 172.21.15.66:0/1895193714 shutdown_connections 2024-09-10T22:10:29.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.568+0000 7f2e5fe00640 1 -- 172.21.15.66:0/1895193714 wait complete. 2024-09-10T22:10:29.591 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-10T22:10:29.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.732+0000 7f8930e00640 1 -- 172.21.15.66:0/55846510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f892c0aee20 msgr2=0x7f892c0ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:29.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.732+0000 7f8930e00640 1 --2- 172.21.15.66:0/55846510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f892c0aee20 0x7f892c0ca080 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8914009840 tx=0x7f891401c790 comp rx=0 tx=0).stop 2024-09-10T22:10:29.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.732+0000 7f8930e00640 1 -- 172.21.15.66:0/55846510 shutdown_connections 2024-09-10T22:10:29.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.732+0000 7f8930e00640 1 --2- 172.21.15.66:0/55846510 >> v2:172.21.15.94:3300/0 conn(0x7f892c15cc80 0x7f892c0cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.732+0000 7f8930e00640 1 --2- 172.21.15.66:0/55846510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f892c0aee20 0x7f892c0ca080 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.732+0000 7f8930e00640 1 -- 172.21.15.66:0/55846510 >> 172.21.15.66:0/55846510 conn(0x7f892c0cfa30 msgr2=0x7f892c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:29.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.732+0000 7f8930e00640 1 -- 172.21.15.66:0/55846510 shutdown_connections 2024-09-10T22:10:29.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.732+0000 7f8930e00640 1 -- 172.21.15.66:0/55846510 wait complete. 2024-09-10T22:10:29.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8930e00640 1 Processor -- start 2024-09-10T22:10:29.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8930e00640 1 -- start start 2024-09-10T22:10:29.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8930e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f892c0aee20 0x7f892c154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:29.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8930e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f892c159900 0x7f892c1f9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:29.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8930e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f892c15cc80 0x7f892c157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:29.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8930e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f892c168f70 con 0x7f892c159900 2024-09-10T22:10:29.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8930e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f892c168df0 con 0x7f892c0aee20 2024-09-10T22:10:29.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8930e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f892c1690f0 con 0x7f892c15cc80 2024-09-10T22:10:29.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8921400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f892c159900 0x7f892c1f9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:29.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8921400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f892c159900 0x7f892c1f9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50860/0 (socket says 172.21.15.66:50860) 2024-09-10T22:10:29.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8921400640 1 -- 172.21.15.66:0/3063198214 learned_addr learned my addr 172.21.15.66:0/3063198214 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:29.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f892a800640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f892c15cc80 0x7f892c157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:29.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8929e00640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f892c0aee20 0x7f892c154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:29.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8921400640 1 -- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f892c15cc80 msgr2=0x7f892c157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:29.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8921400640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f892c15cc80 0x7f892c157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8921400640 1 -- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f892c0aee20 msgr2=0x7f892c154840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:29.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8921400640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f892c0aee20 0x7f892c154840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:29.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8921400640 1 -- 172.21.15.66:0/3063198214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c15a010 con 0x7f892c159900 2024-09-10T22:10:29.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8929e00640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f892c0aee20 0x7f892c154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:29.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8921400640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f892c159900 0x7f892c1f9640 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f891401d030 tx=0x7f89140a6f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:29.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8922a00640 1 -- 172.21.15.66:0/3063198214 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89140ac070 con 0x7f892c159900 2024-09-10T22:10:29.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8922a00640 1 -- 172.21.15.66:0/3063198214 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8914005130 con 0x7f892c159900 2024-09-10T22:10:29.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8922a00640 1 -- 172.21.15.66:0/3063198214 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914005560 con 0x7f892c159900 2024-09-10T22:10:29.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8930e00640 1 -- 172.21.15.66:0/3063198214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892c15a2f0 con 0x7f892c159900 2024-09-10T22:10:29.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.736+0000 7f8930e00640 1 -- 172.21.15.66:0/3063198214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892c15a8b0 con 0x7f892c159900 2024-09-10T22:10:29.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.740+0000 7f8930e00640 1 -- 172.21.15.66:0/3063198214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f892c0ca080 con 0x7f892c159900 2024-09-10T22:10:29.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.740+0000 7f8922a00640 1 -- 172.21.15.66:0/3063198214 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f89140046d0 con 0x7f892c159900 2024-09-10T22:10:29.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.740+0000 7f8922a00640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8908073bd0 0x7f8908076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:29.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.740+0000 7f8922a00640 1 -- 172.21.15.66:0/3063198214 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7f891412c290 con 0x7f892c159900 2024-09-10T22:10:29.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.740+0000 7f8929e00640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8908073bd0 0x7f8908076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:29.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.744+0000 7f8929e00640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8908073bd0 0x7f8908076090 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f891c007a60 tx=0x7f891c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:29.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.744+0000 7f8922a00640 1 -- 172.21.15.66:0/3063198214 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8914016410 con 0x7f892c159900 2024-09-10T22:10:29.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:29.988+0000 7f8930e00640 1 -- 172.21.15.66:0/3063198214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f892c074230 con 0x7f892c159900 2024-09-10T22:10:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.180+0000 7f8922a00640 1 -- 172.21.15.66:0/3063198214 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v451) ==== 174+0+0 (secure 0 0 0) 0x7f89140f93c0 con 0x7f892c159900 2024-09-10T22:10:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.184+0000 7f8930e00640 1 -- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8908073bd0 msgr2=0x7f8908076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.184+0000 7f8930e00640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8908073bd0 0x7f8908076090 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f891c007a60 tx=0x7f891c009040 comp rx=0 tx=0).stop 2024-09-10T22:10:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.184+0000 7f8930e00640 1 -- 172.21.15.66:0/3063198214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f892c159900 msgr2=0x7f892c1f9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:30.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.184+0000 7f8930e00640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f892c159900 0x7f892c1f9640 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f891401d030 tx=0x7f89140a6f90 comp rx=0 tx=0).stop 2024-09-10T22:10:30.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.184+0000 7f8930e00640 1 -- 172.21.15.66:0/3063198214 shutdown_connections 2024-09-10T22:10:30.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.184+0000 7f8930e00640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8908073bd0 0x7f8908076090 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.184+0000 7f8930e00640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f892c15cc80 0x7f892c157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.184+0000 7f8930e00640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f892c159900 0x7f892c1f9640 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.184+0000 7f8930e00640 1 --2- 172.21.15.66:0/3063198214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f892c0aee20 0x7f892c154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.184+0000 7f8930e00640 1 -- 172.21.15.66:0/3063198214 >> 172.21.15.66:0/3063198214 conn(0x7f892c0cfa30 msgr2=0x7f892c06e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:30.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.184+0000 7f8930e00640 1 -- 172.21.15.66:0/3063198214 shutdown_connections 2024-09-10T22:10:30.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.184+0000 7f8930e00640 1 -- 172.21.15.66:0/3063198214 wait complete. 2024-09-10T22:10:30.208 INFO:tasks.cephfs_test_runner:Test that progress indicated by progress bar for ongoing+pending clones ... ok 2024-09-10T22:10:30.208 INFO:tasks.cephfs_test_runner:Starting test: test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the command "ceph fs clone status" prints progress stats 2024-09-10T22:10:30.208 INFO:tasks.cephfs_test_runner:test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-10T22:10:30.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:10:30.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.352+0000 7fe802400640 1 -- 172.21.15.66:0/3096714262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7e4003a10 msgr2=0x7fe7e4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:30.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.352+0000 7fe802400640 1 --2- 172.21.15.66:0/3096714262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7e4003a10 0x7fe7e4003e30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009970 tx=0x7fe7f001c730 comp rx=0 tx=0).stop 2024-09-10T22:10:30.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.352+0000 7fe801a00640 1 -- 172.21.15.66:0/3096714262 reap_dead start 2024-09-10T22:10:30.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.352+0000 7fe802400640 1 -- 172.21.15.66:0/3096714262 shutdown_connections 2024-09-10T22:10:30.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.352+0000 7fe802400640 1 -- 172.21.15.66:0/3096714262 >> 172.21.15.66:0/3096714262 conn(0x7fe7fc0d1b10 msgr2=0x7fe7fc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:30.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.352+0000 7fe802400640 1 -- 172.21.15.66:0/3096714262 shutdown_connections 2024-09-10T22:10:30.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.352+0000 7fe802400640 1 -- 172.21.15.66:0/3096714262 wait complete. 2024-09-10T22:10:30.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe802400640 1 Processor -- start 2024-09-10T22:10:30.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe802400640 1 -- start start 2024-09-10T22:10:30.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe802400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7e4003a10 0x7fe7fc15abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:30.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe802400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc15b2d0 0x7fe7fc0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:30.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe802400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc0b5870 0x7fe7fc15ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:30.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe802400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc166d50 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe802400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc166ed0 con 0x7fe7e4003a10 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe802400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe7fc167050 con 0x7fe7fc0b5870 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe7fbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc15b2d0 0x7fe7fc0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe7fbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc15b2d0 0x7fe7fc0b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50888/0 (socket says 172.21.15.66:50888) 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe7fbe00640 1 -- 172.21.15.66:0/2996227789 learned_addr learned my addr 172.21.15.66:0/2996227789 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe801a00640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc0b5870 0x7fe7fc15ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe801000640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7e4003a10 0x7fe7fc15abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe7fbe00640 1 -- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc0b5870 msgr2=0x7fe7fc15ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe7fbe00640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc0b5870 0x7fe7fc15ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe7fbe00640 1 -- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7e4003a10 msgr2=0x7fe7fc15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe7fbe00640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7e4003a10 0x7fe7fc15abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe7fbe00640 1 -- 172.21.15.66:0/2996227789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7fc1f7bd0 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe7fbe00640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc15b2d0 0x7fe7fc0b5330 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8005980 tx=0x7fe7e800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe7f8c00640 1 -- 172.21.15.66:0/2996227789 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8017070 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe7f8c00640 1 -- 172.21.15.66:0/2996227789 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7e8009a70 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe802400640 1 -- 172.21.15.66:0/2996227789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7fc1f7eb0 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe7f8c00640 1 -- 172.21.15.66:0/2996227789 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8010460 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.356+0000 7fe802400640 1 -- 172.21.15.66:0/2996227789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7fc1f8470 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.360+0000 7fe802400640 1 -- 172.21.15.66:0/2996227789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7fc0cecc0 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.360+0000 7fe7f8c00640 1 -- 172.21.15.66:0/2996227789 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fe7e8010600 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.360+0000 7fe7f8c00640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7dc073bd0 0x7fe7dc076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.360+0000 7fe7f8c00640 1 -- 172.21.15.66:0/2996227789 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7fe7e80927c0 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.360+0000 7fe801000640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7dc073bd0 0x7fe7dc076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.364+0000 7fe801000640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7dc073bd0 0x7fe7dc076090 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00ae000 tx=0x7fe7f00aa040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.364+0000 7fe7f8c00640 1 -- 172.21.15.66:0/2996227789 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe7e805f4e0 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.540+0000 7fe802400640 1 -- 172.21.15.66:0/2996227789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7fc072e00 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.540+0000 7fe7f8c00640 1 -- 172.21.15.66:0/2996227789 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+13049 (secure 0 0 0) 0x7fe7e805ede0 con 0x7fe7fc15b2d0 2024-09-10T22:10:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:30.546 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":92,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:10:29.176686+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6836/3187502512":"2024-09-11T22:10:26.166948+0000","172.21.15.94:6834/3187502512":"2024-09-11T22:10:26.166948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:10:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.544+0000 7fe802400640 1 -- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7dc073bd0 msgr2=0x7fe7dc076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.544+0000 7fe802400640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7dc073bd0 0x7fe7dc076090 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00ae000 tx=0x7fe7f00aa040 comp rx=0 tx=0).stop 2024-09-10T22:10:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.544+0000 7fe802400640 1 -- 172.21.15.66:0/2996227789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc15b2d0 msgr2=0x7fe7fc0b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.544+0000 7fe802400640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc15b2d0 0x7fe7fc0b5330 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8005980 tx=0x7fe7e800dae0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.544+0000 7fe802400640 1 -- 172.21.15.66:0/2996227789 shutdown_connections 2024-09-10T22:10:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.544+0000 7fe802400640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7dc073bd0 0x7fe7dc076090 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.544+0000 7fe802400640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7fc0b5870 0x7fe7fc15ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.544+0000 7fe802400640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc15b2d0 0x7fe7fc0b5330 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.544+0000 7fe802400640 1 --2- 172.21.15.66:0/2996227789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7e4003a10 0x7fe7fc15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.544+0000 7fe802400640 1 -- 172.21.15.66:0/2996227789 >> 172.21.15.66:0/2996227789 conn(0x7fe7fc0d1b10 msgr2=0x7fe7fc071620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:30.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.544+0000 7fe802400640 1 -- 172.21.15.66:0/2996227789 shutdown_connections 2024-09-10T22:10:30.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.544+0000 7fe802400640 1 -- 172.21.15.66:0/2996227789 wait complete. 2024-09-10T22:10:30.567 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:00:22.340318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:10:30.567 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:10:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.712+0000 7eff36c00640 1 -- 172.21.15.66:0/2687355122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff300c4b10 msgr2=0x7eff3006e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.712+0000 7eff36c00640 1 --2- 172.21.15.66:0/2687355122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff300c4b10 0x7eff3006e960 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7eff20009900 tx=0x7eff2001c880 comp rx=0 tx=0).stop 2024-09-10T22:10:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.712+0000 7eff36c00640 1 -- 172.21.15.66:0/2687355122 shutdown_connections 2024-09-10T22:10:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.712+0000 7eff36c00640 1 --2- 172.21.15.66:0/2687355122 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7eff300ae620 0x7eff300725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.712+0000 7eff36c00640 1 --2- 172.21.15.66:0/2687355122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff300c4b10 0x7eff3006e960 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.712+0000 7eff36c00640 1 -- 172.21.15.66:0/2687355122 >> 172.21.15.66:0/2687355122 conn(0x7eff300cfa30 msgr2=0x7eff300cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.712+0000 7eff36c00640 1 -- 172.21.15.66:0/2687355122 shutdown_connections 2024-09-10T22:10:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.712+0000 7eff36c00640 1 -- 172.21.15.66:0/2687355122 wait complete. 2024-09-10T22:10:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff36c00640 1 Processor -- start 2024-09-10T22:10:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff36c00640 1 -- start start 2024-09-10T22:10:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff36c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff300ae620 0x7eff301465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff36c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff300c4b10 0x7eff301531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff36c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7eff300c4ef0 0x7eff300ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff36c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7eff301684a0 con 0x7eff300ae620 2024-09-10T22:10:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff36c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff30168320 con 0x7eff300c4b10 2024-09-10T22:10:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff36c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7eff30168620 con 0x7eff300c4ef0 2024-09-10T22:10:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff2fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff300ae620 0x7eff301465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff2fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff300ae620 0x7eff301465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50906/0 (socket says 172.21.15.66:50906) 2024-09-10T22:10:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff2fe00640 1 -- 172.21.15.66:0/2743734401 learned_addr learned my addr 172.21.15.66:0/2743734401 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff27e00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff300c4b10 0x7eff301531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff2fe00640 1 -- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7eff300c4ef0 msgr2=0x7eff300ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff2fe00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7eff300c4ef0 0x7eff300ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff2fe00640 1 -- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff300c4b10 msgr2=0x7eff301531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff2fe00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff300c4b10 0x7eff301531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff2fe00640 1 -- 172.21.15.66:0/2743734401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff301592c0 con 0x7eff300ae620 2024-09-10T22:10:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff27e00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff300c4b10 0x7eff301531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff2fe00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff300ae620 0x7eff301465b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7eff2001af90 tx=0x7eff200040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff2cc00640 1 -- 172.21.15.66:0/2743734401 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff200044b0 con 0x7eff300ae620 2024-09-10T22:10:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff2cc00640 1 -- 172.21.15.66:0/2743734401 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff20005110 con 0x7eff300ae620 2024-09-10T22:10:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff2cc00640 1 -- 172.21.15.66:0/2743734401 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20005560 con 0x7eff300ae620 2024-09-10T22:10:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff36c00640 1 -- 172.21.15.66:0/2743734401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff300b0d30 con 0x7eff300ae620 2024-09-10T22:10:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff36c00640 1 -- 172.21.15.66:0/2743734401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff300b12f0 con 0x7eff300ae620 2024-09-10T22:10:30.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.716+0000 7eff36c00640 1 -- 172.21.15.66:0/2743734401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff300725d0 con 0x7eff300ae620 2024-09-10T22:10:30.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.720+0000 7eff2cc00640 1 -- 172.21.15.66:0/2743734401 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7eff200046d0 con 0x7eff300ae620 2024-09-10T22:10:30.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.720+0000 7eff2cc00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7eff0c073bd0 0x7eff0c076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:30.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.720+0000 7eff2cc00640 1 -- 172.21.15.66:0/2743734401 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7eff2012c880 con 0x7eff300ae620 2024-09-10T22:10:30.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.720+0000 7eff27e00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7eff0c073bd0 0x7eff0c076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:30.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.720+0000 7eff27e00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7eff0c073bd0 0x7eff0c076090 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7eff18005fd0 tx=0x7eff18005da0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:30.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.724+0000 7eff2cc00640 1 -- 172.21.15.66:0/2743734401 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7eff200164d0 con 0x7eff300ae620 2024-09-10T22:10:30.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.896+0000 7eff36c00640 1 -- 172.21.15.66:0/2743734401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff3006f890 con 0x7eff300ae620 2024-09-10T22:10:30.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff2cc00640 1 -- 172.21.15.66:0/2743734401 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7eff200f99b0 con 0x7eff300ae620 2024-09-10T22:10:30.902 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-09-10T22:10:30.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff36c00640 1 -- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7eff0c073bd0 msgr2=0x7eff0c076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:30.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff36c00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7eff0c073bd0 0x7eff0c076090 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7eff18005fd0 tx=0x7eff18005da0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff36c00640 1 -- 172.21.15.66:0/2743734401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff300ae620 msgr2=0x7eff301465b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:30.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff36c00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff300ae620 0x7eff301465b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7eff2001af90 tx=0x7eff200040c0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff36c00640 1 -- 172.21.15.66:0/2743734401 shutdown_connections 2024-09-10T22:10:30.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff36c00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7eff0c073bd0 0x7eff0c076090 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff36c00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7eff300c4ef0 0x7eff300ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff36c00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff300c4b10 0x7eff301531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff36c00640 1 --2- 172.21.15.66:0/2743734401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff300ae620 0x7eff301465b0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:30.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff36c00640 1 -- 172.21.15.66:0/2743734401 >> 172.21.15.66:0/2743734401 conn(0x7eff300cfa30 msgr2=0x7eff30155640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:30.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff36c00640 1 -- 172.21.15.66:0/2743734401 shutdown_connections 2024-09-10T22:10:30.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:30.900+0000 7eff36c00640 1 -- 172.21.15.66:0/2743734401 wait complete. 2024-09-10T22:10:30.923 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-10T22:10:31.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 -- 172.21.15.66:0/1454341490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f00b0f60 msgr2=0x7f38f00d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:31.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 --2- 172.21.15.66:0/1454341490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f00b0f60 0x7f38f00d0140 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f38f0069970 tx=0x7f38d801c4b0 comp rx=0 tx=0).stop 2024-09-10T22:10:31.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 -- 172.21.15.66:0/1454341490 shutdown_connections 2024-09-10T22:10:31.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 --2- 172.21.15.66:0/1454341490 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38f00b32a0 0x7f38f00b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:31.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 --2- 172.21.15.66:0/1454341490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f00b0f60 0x7f38f00d0140 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:31.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 --2- 172.21.15.66:0/1454341490 >> v2:172.21.15.94:3300/0 conn(0x7f38f015f460 0x7f38f00cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:31.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 -- 172.21.15.66:0/1454341490 >> 172.21.15.66:0/1454341490 conn(0x7f38f00d1d00 msgr2=0x7f38f00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:31.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 -- 172.21.15.66:0/1454341490 shutdown_connections 2024-09-10T22:10:31.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 -- 172.21.15.66:0/1454341490 wait complete. 2024-09-10T22:10:31.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 Processor -- start 2024-09-10T22:10:31.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 -- start start 2024-09-10T22:10:31.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f00b0f60 0x7f38f014ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38f00b32a0 0x7f38f00c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38f015f460 0x7f38f014c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f38f00af720 con 0x7f38f00b0f60 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38f00af5a0 con 0x7f38f015f460 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f7c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f38f00af8a0 con 0x7f38f00b32a0 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f00b0f60 0x7f38f014ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f00b0f60 0x7f38f014ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50922/0 (socket says 172.21.15.66:50922) 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f5200640 1 -- 172.21.15.66:0/1239101236 learned_addr learned my addr 172.21.15.66:0/1239101236 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.068+0000 7f38f5c00640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38f015f460 0x7f38f014c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38e7e00640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38f00b32a0 0x7f38f00c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38f5200640 1 -- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38f00b32a0 msgr2=0x7f38f00c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38f5200640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38f00b32a0 0x7f38f00c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:31.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38f5200640 1 -- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38f015f460 msgr2=0x7f38f014c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:31.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38f5200640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38f015f460 0x7f38f014c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:31.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38f5200640 1 -- 172.21.15.66:0/1239101236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f00c4b60 con 0x7f38f00b0f60 2024-09-10T22:10:31.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38f5c00640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38f015f460 0x7f38f014c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:31.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38f5200640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f00b0f60 0x7f38f014ce00 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f38e000ca20 tx=0x7f38e000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:31.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38ed600640 1 -- 172.21.15.66:0/1239101236 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e0007c70 con 0x7f38f00b0f60 2024-09-10T22:10:31.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38ed600640 1 -- 172.21.15.66:0/1239101236 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38e000f450 con 0x7f38f00b0f60 2024-09-10T22:10:31.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38ed600640 1 -- 172.21.15.66:0/1239101236 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e000f8a0 con 0x7f38f00b0f60 2024-09-10T22:10:31.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38f7c00640 1 -- 172.21.15.66:0/1239101236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f00c4e40 con 0x7f38f00b0f60 2024-09-10T22:10:31.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38f7c00640 1 -- 172.21.15.66:0/1239101236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f0153610 con 0x7f38f00b0f60 2024-09-10T22:10:31.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38f7c00640 1 -- 172.21.15.66:0/1239101236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f00b14b0 con 0x7f38f00b0f60 2024-09-10T22:10:31.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38ed600640 1 -- 172.21.15.66:0/1239101236 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f38e000fa40 con 0x7f38f00b0f60 2024-09-10T22:10:31.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38ed600640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f38cc073bd0 0x7f38cc076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:31.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.072+0000 7f38ed600640 1 -- 172.21.15.66:0/1239101236 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7f38e0092b80 con 0x7f38f00b0f60 2024-09-10T22:10:31.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.076+0000 7f38e7e00640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f38cc073bd0 0x7f38cc076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:31.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.076+0000 7f38e7e00640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f38cc073bd0 0x7f38cc076090 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f38d80af000 tx=0x7f38d801acf0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:31.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.076+0000 7f38ed600640 1 -- 172.21.15.66:0/1239101236 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f38e0014030 con 0x7f38f00b0f60 2024-09-10T22:10:31.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:31.312+0000 7f38f7c00640 1 -- 172.21.15.66:0/1239101236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7f38f0074aa0 con 0x7f38f00b0f60 2024-09-10T22:10:32.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.188+0000 7f38ed600640 1 -- 172.21.15.66:0/1239101236 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v453) ==== 181+0+0 (secure 0 0 0) 0x7f38e005f8a0 con 0x7f38f00b0f60 2024-09-10T22:10:32.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.192+0000 7f38f7c00640 1 -- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f38cc073bd0 msgr2=0x7f38cc076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:32.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.192+0000 7f38f7c00640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f38cc073bd0 0x7f38cc076090 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f38d80af000 tx=0x7f38d801acf0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.192+0000 7f38f7c00640 1 -- 172.21.15.66:0/1239101236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f00b0f60 msgr2=0x7f38f014ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:32.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.192+0000 7f38f7c00640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f00b0f60 0x7f38f014ce00 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f38e000ca20 tx=0x7f38e000cee0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.192+0000 7f38f7c00640 1 -- 172.21.15.66:0/1239101236 shutdown_connections 2024-09-10T22:10:32.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.192+0000 7f38f7c00640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f38cc073bd0 0x7f38cc076090 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.192+0000 7f38f7c00640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38f015f460 0x7f38f014c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.192+0000 7f38f7c00640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38f00b32a0 0x7f38f00c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.192+0000 7f38f7c00640 1 --2- 172.21.15.66:0/1239101236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f00b0f60 0x7f38f014ce00 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.192+0000 7f38f7c00640 1 -- 172.21.15.66:0/1239101236 >> 172.21.15.66:0/1239101236 conn(0x7f38f00d1d00 msgr2=0x7f38f0151b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:32.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.192+0000 7f38f7c00640 1 -- 172.21.15.66:0/1239101236 shutdown_connections 2024-09-10T22:10:32.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.192+0000 7f38f7c00640 1 -- 172.21.15.66:0/1239101236 wait complete. 2024-09-10T22:10:32.218 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T22:10:32.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.364+0000 7fced6a00640 1 -- 172.21.15.66:0/2804515779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced01614e0 msgr2=0x7fced006dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:32.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.364+0000 7fced6a00640 1 --2- 172.21.15.66:0/2804515779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced01614e0 0x7fced006dd30 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009840 tx=0x7fcebc01c820 comp rx=0 tx=0).stop 2024-09-10T22:10:32.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.364+0000 7fced6a00640 1 -- 172.21.15.66:0/2804515779 shutdown_connections 2024-09-10T22:10:32.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.364+0000 7fced6a00640 1 --2- 172.21.15.66:0/2804515779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced01614e0 0x7fced006dd30 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.364+0000 7fced6a00640 1 --2- 172.21.15.66:0/2804515779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fced0161100 0x7fced00d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.364+0000 7fced6a00640 1 -- 172.21.15.66:0/2804515779 >> 172.21.15.66:0/2804515779 conn(0x7fced00d1b10 msgr2=0x7fced00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:32.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.364+0000 7fced6a00640 1 -- 172.21.15.66:0/2804515779 shutdown_connections 2024-09-10T22:10:32.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.364+0000 7fced6a00640 1 -- 172.21.15.66:0/2804515779 wait complete. 2024-09-10T22:10:32.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.364+0000 7fced6a00640 1 Processor -- start 2024-09-10T22:10:32.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.364+0000 7fced6a00640 1 -- start start 2024-09-10T22:10:32.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced006e270 0x7fced015c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:32.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fced0161100 0x7fced00c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:32.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced01614e0 0x7fced00b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:32.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fced016b570 con 0x7fced01614e0 2024-09-10T22:10:32.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fced016b3f0 con 0x7fced006e270 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fced016b6f0 con 0x7fced0161100 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced01614e0 0x7fced00b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced01614e0 0x7fced00b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54620/0 (socket says 172.21.15.66:54620) 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6000640 1 -- 172.21.15.66:0/3131610491 learned_addr learned my addr 172.21.15.66:0/3131610491 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced5600640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced006e270 0x7fced015c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced4c00640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fced0161100 0x7fced00c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6000640 1 -- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fced0161100 msgr2=0x7fced00c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6000640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fced0161100 0x7fced00c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6000640 1 -- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced006e270 msgr2=0x7fced015c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6000640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced006e270 0x7fced015c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6000640 1 -- 172.21.15.66:0/3131610491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced00c9b30 con 0x7fced01614e0 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced4c00640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fced0161100 0x7fced00c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:32.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6000640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced01614e0 0x7fced00b35e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fcec800c730 tx=0x7fcec800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:32.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fcecd600640 1 -- 172.21.15.66:0/3131610491 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8003db0 con 0x7fced01614e0 2024-09-10T22:10:32.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fcecd600640 1 -- 172.21.15.66:0/3131610491 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcec800ce40 con 0x7fced01614e0 2024-09-10T22:10:32.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fcecd600640 1 -- 172.21.15.66:0/3131610491 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec80052d0 con 0x7fced01614e0 2024-09-10T22:10:32.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6a00640 1 -- 172.21.15.66:0/3131610491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced0153130 con 0x7fced01614e0 2024-09-10T22:10:32.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6a00640 1 -- 172.21.15.66:0/3131610491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced01536f0 con 0x7fced01614e0 2024-09-10T22:10:32.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.368+0000 7fced6a00640 1 -- 172.21.15.66:0/3131610491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced00d12c0 con 0x7fced01614e0 2024-09-10T22:10:32.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.372+0000 7fcecd600640 1 -- 172.21.15.66:0/3131610491 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fcec8005470 con 0x7fced01614e0 2024-09-10T22:10:32.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.372+0000 7fcecd600640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fceb0073bd0 0x7fceb0076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:32.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.372+0000 7fcecd600640 1 -- 172.21.15.66:0/3131610491 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7fcec8092230 con 0x7fced01614e0 2024-09-10T22:10:32.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.372+0000 7fced5600640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fceb0073bd0 0x7fceb0076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:32.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.372+0000 7fced5600640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fceb0073bd0 0x7fceb0076090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fcec40058d0 tx=0x7fcec4005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:32.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.376+0000 7fcecd600640 1 -- 172.21.15.66:0/3131610491 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcec805ef50 con 0x7fced01614e0 2024-09-10T22:10:32.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.556+0000 7fced6a00640 1 -- 172.21.15.66:0/3131610491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fced0072e00 con 0x7fced01614e0 2024-09-10T22:10:32.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.556+0000 7fcecd600640 1 -- 172.21.15.66:0/3131610491 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v44) ==== 59+0+1998 (secure 0 0 0) 0x7fcec805e850 con 0x7fced01614e0 2024-09-10T22:10:32.561 INFO:teuthology.orchestra.run.smithi066.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4673.907365 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4537.831230 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 5706.341617 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4484.013907 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3862.285837 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3776.439661 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3820.270368 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3863.602758 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds 20/20 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T22:10:32.562 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_ms 2/2 2024-09-10T22:10:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_frag true 2024-09-10T22:10:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_scatterstat true 2024-09-10T22:10:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T22:10:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_verify_scatter true 2024-09-10T22:10:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T22:10:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T22:10:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T22:10:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced client_mount_timeout 600 2024-09-10T22:10:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_client 20/20 2024-09-10T22:10:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_ms 1/1 2024-09-10T22:10:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T22:10:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T22:10:32.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.560+0000 7fced6a00640 1 -- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fceb0073bd0 msgr2=0x7fceb0076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:32.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.560+0000 7fced6a00640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fceb0073bd0 0x7fceb0076090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fcec40058d0 tx=0x7fcec4005860 comp rx=0 tx=0).stop 2024-09-10T22:10:32.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.560+0000 7fced6a00640 1 -- 172.21.15.66:0/3131610491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced01614e0 msgr2=0x7fced00b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:32.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.560+0000 7fced6a00640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced01614e0 0x7fced00b35e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fcec800c730 tx=0x7fcec800cbf0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.560+0000 7fced6a00640 1 -- 172.21.15.66:0/3131610491 shutdown_connections 2024-09-10T22:10:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.560+0000 7fced6a00640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fceb0073bd0 0x7fceb0076090 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.560+0000 7fced6a00640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced01614e0 0x7fced00b35e0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.560+0000 7fced6a00640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fced0161100 0x7fced00c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.560+0000 7fced6a00640 1 --2- 172.21.15.66:0/3131610491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced006e270 0x7fced015c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.560+0000 7fced6a00640 1 -- 172.21.15.66:0/3131610491 >> 172.21.15.66:0/3131610491 conn(0x7fced00d1b10 msgr2=0x7fced00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.560+0000 7fced6a00640 1 -- 172.21.15.66:0/3131610491 shutdown_connections 2024-09-10T22:10:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.560+0000 7fced6a00640 1 -- 172.21.15.66:0/3131610491 wait complete. 2024-09-10T22:10:32.583 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:10:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.772+0000 7fdb53000640 1 -- 172.21.15.66:0/3924618770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c16ac10 msgr2=0x7fdb4c0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.772+0000 7fdb53000640 1 --2- 172.21.15.66:0/3924618770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c16ac10 0x7fdb4c0c48b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009840 tx=0x7fdb3c01c7f0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.772+0000 7fdb53000640 1 -- 172.21.15.66:0/3924618770 shutdown_connections 2024-09-10T22:10:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.772+0000 7fdb53000640 1 --2- 172.21.15.66:0/3924618770 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb4c0cf610 0x7fdb4c06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.772+0000 7fdb53000640 1 --2- 172.21.15.66:0/3924618770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c16ac10 0x7fdb4c0c48b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.772+0000 7fdb53000640 1 -- 172.21.15.66:0/3924618770 >> 172.21.15.66:0/3924618770 conn(0x7fdb4c0cfa30 msgr2=0x7fdb4c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.772+0000 7fdb53000640 1 -- 172.21.15.66:0/3924618770 shutdown_connections 2024-09-10T22:10:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.772+0000 7fdb53000640 1 -- 172.21.15.66:0/3924618770 wait complete. 2024-09-10T22:10:32.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb53000640 1 Processor -- start 2024-09-10T22:10:32.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb53000640 1 -- start start 2024-09-10T22:10:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb53000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c0aee00 0x7fdb4c0cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb53000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb4c0cf610 0x7fdb4c16d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb53000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb4c16ac10 0x7fdb4c16da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb53000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c178090 con 0x7fdb4c0cf610 2024-09-10T22:10:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb53000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c177f10 con 0x7fdb4c0aee00 2024-09-10T22:10:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb53000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4c178210 con 0x7fdb4c16ac10 2024-09-10T22:10:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb43e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb4c0cf610 0x7fdb4c16d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb43e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb4c0cf610 0x7fdb4c16d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54624/0 (socket says 172.21.15.66:54624) 2024-09-10T22:10:32.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb43e00640 1 -- 172.21.15.66:0/4100164229 learned_addr learned my addr 172.21.15.66:0/4100164229 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:32.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb4be00640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c0aee00 0x7fdb4c0cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:32.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb51000640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb4c16ac10 0x7fdb4c16da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:32.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb43e00640 1 -- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb4c16ac10 msgr2=0x7fdb4c16da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:32.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb43e00640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb4c16ac10 0x7fdb4c16da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb43e00640 1 -- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c0aee00 msgr2=0x7fdb4c0cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:32.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb43e00640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c0aee00 0x7fdb4c0cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb43e00640 1 -- 172.21.15.66:0/4100164229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c20c910 con 0x7fdb4c0cf610 2024-09-10T22:10:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb43e00640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb4c0cf610 0x7fdb4c16d530 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400c450 tx=0x7fdb3400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb48c00640 1 -- 172.21.15.66:0/4100164229 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34017070 con 0x7fdb4c0cf610 2024-09-10T22:10:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb48c00640 1 -- 172.21.15.66:0/4100164229 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb34003600 con 0x7fdb4c0cf610 2024-09-10T22:10:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb48c00640 1 -- 172.21.15.66:0/4100164229 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34003a50 con 0x7fdb4c0cf610 2024-09-10T22:10:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.776+0000 7fdb53000640 1 -- 172.21.15.66:0/4100164229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4c20cbf0 con 0x7fdb4c0cf610 2024-09-10T22:10:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.780+0000 7fdb53000640 1 -- 172.21.15.66:0/4100164229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb4c20d1b0 con 0x7fdb4c0cf610 2024-09-10T22:10:32.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.780+0000 7fdb53000640 1 -- 172.21.15.66:0/4100164229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb4c0727e0 con 0x7fdb4c0cf610 2024-09-10T22:10:32.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.780+0000 7fdb48c00640 1 -- 172.21.15.66:0/4100164229 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fdb34003bf0 con 0x7fdb4c0cf610 2024-09-10T22:10:32.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.784+0000 7fdb48c00640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb28073bd0 0x7fdb28076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:32.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.784+0000 7fdb48c00640 1 -- 172.21.15.66:0/4100164229 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7fdb34091970 con 0x7fdb4c0cf610 2024-09-10T22:10:32.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.784+0000 7fdb4be00640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb28073bd0 0x7fdb28076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:32.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.784+0000 7fdb4be00640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb28073bd0 0x7fdb28076090 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0af000 tx=0x7fdb3c01b030 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:32.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.788+0000 7fdb48c00640 1 -- 172.21.15.66:0/4100164229 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdb3405e160 con 0x7fdb4c0cf610 2024-09-10T22:10:32.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.968+0000 7fdb53000640 1 -- 172.21.15.66:0/4100164229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdb4c0cf0e0 con 0x7fdb4c0cf610 2024-09-10T22:10:32.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb48c00640 1 -- 172.21.15.66:0/4100164229 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7fdb34062b90 con 0x7fdb4c0cf610 2024-09-10T22:10:32.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb53000640 1 -- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb28073bd0 msgr2=0x7fdb28076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb53000640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb28073bd0 0x7fdb28076090 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0af000 tx=0x7fdb3c01b030 comp rx=0 tx=0).stop 2024-09-10T22:10:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb53000640 1 -- 172.21.15.66:0/4100164229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb4c0cf610 msgr2=0x7fdb4c16d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb53000640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb4c0cf610 0x7fdb4c16d530 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400c450 tx=0x7fdb3400c910 comp rx=0 tx=0).stop 2024-09-10T22:10:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb53000640 1 -- 172.21.15.66:0/4100164229 shutdown_connections 2024-09-10T22:10:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb53000640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdb28073bd0 0x7fdb28076090 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb53000640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdb4c16ac10 0x7fdb4c16da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb53000640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb4c0cf610 0x7fdb4c16d530 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb53000640 1 --2- 172.21.15.66:0/4100164229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c0aee00 0x7fdb4c0cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb53000640 1 -- 172.21.15.66:0/4100164229 >> 172.21.15.66:0/4100164229 conn(0x7fdb4c0cfa30 msgr2=0x7fdb4c0b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb53000640 1 -- 172.21.15.66:0/4100164229 shutdown_connections 2024-09-10T22:10:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:32.972+0000 7fdb53000640 1 -- 172.21.15.66:0/4100164229 wait complete. 2024-09-10T22:10:32.996 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:10:32.997 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:10:33.120 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:10:33.205 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:10:33.205 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:10:33.206 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:10:33.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:10:33.215 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:10:33.215 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:10:33.224 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:10:33.224 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:10:33.225 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:10:33.266 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:10:33.266 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:10:33.266 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:10:33.274 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:10:33.275 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T22:10:33.275 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:10:33.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.456+0000 7f1666e00640 1 -- 172.21.15.66:0/1077523520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16600aee20 msgr2=0x7f16600ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:33.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.456+0000 7f1666e00640 1 --2- 172.21.15.66:0/1077523520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16600aee20 0x7f16600ca080 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f1648009900 tx=0x7f164801c850 comp rx=0 tx=0).stop 2024-09-10T22:10:33.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.456+0000 7f1666e00640 1 -- 172.21.15.66:0/1077523520 shutdown_connections 2024-09-10T22:10:33.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.456+0000 7f1666e00640 1 --2- 172.21.15.66:0/1077523520 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f166015cc80 0x7f16600cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.456+0000 7f1666e00640 1 --2- 172.21.15.66:0/1077523520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16600aee20 0x7f16600ca080 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.456+0000 7f1666e00640 1 -- 172.21.15.66:0/1077523520 >> 172.21.15.66:0/1077523520 conn(0x7f16600cfa30 msgr2=0x7f16600cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:33.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.456+0000 7f1666e00640 1 -- 172.21.15.66:0/1077523520 shutdown_connections 2024-09-10T22:10:33.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.456+0000 7f1666e00640 1 -- 172.21.15.66:0/1077523520 wait complete. 2024-09-10T22:10:33.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.456+0000 7f1666e00640 1 Processor -- start 2024-09-10T22:10:33.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.456+0000 7f1666e00640 1 -- start start 2024-09-10T22:10:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.456+0000 7f1666e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16600aee20 0x7f1660145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1666e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1660159900 0x7f16601f9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1666e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f166015cc80 0x7f16600b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1666e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1660168f70 con 0x7f166015cc80 2024-09-10T22:10:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1666e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1660168df0 con 0x7f16600aee20 2024-09-10T22:10:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1666e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f16601690f0 con 0x7f1660159900 2024-09-10T22:10:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1664e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f166015cc80 0x7f16600b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1664e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f166015cc80 0x7f16600b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54642/0 (socket says 172.21.15.66:54642) 2024-09-10T22:10:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1664e00640 1 -- 172.21.15.66:0/4131440324 learned_addr learned my addr 172.21.15.66:0/4131440324 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f165fe00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16600aee20 0x7f1660145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1657e00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1660159900 0x7f16601f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:33.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1664e00640 1 -- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1660159900 msgr2=0x7f16601f9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:33.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1664e00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1660159900 0x7f16601f9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1664e00640 1 -- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16600aee20 msgr2=0x7f1660145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:33.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1664e00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16600aee20 0x7f1660145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1664e00640 1 -- 172.21.15.66:0/4131440324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16601f5a30 con 0x7f166015cc80 2024-09-10T22:10:33.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1664e00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f166015cc80 0x7f16600b0710 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f165800e9a0 tx=0x7f165800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f165cc00640 1 -- 172.21.15.66:0/4131440324 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658004d30 con 0x7f166015cc80 2024-09-10T22:10:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f165cc00640 1 -- 172.21.15.66:0/4131440324 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f165800cb20 con 0x7f166015cc80 2024-09-10T22:10:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f165cc00640 1 -- 172.21.15.66:0/4131440324 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658005020 con 0x7f166015cc80 2024-09-10T22:10:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1666e00640 1 -- 172.21.15.66:0/4131440324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16601f5d10 con 0x7f166015cc80 2024-09-10T22:10:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1666e00640 1 -- 172.21.15.66:0/4131440324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16601f62d0 con 0x7f166015cc80 2024-09-10T22:10:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f1666e00640 1 -- 172.21.15.66:0/4131440324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16600ca080 con 0x7f166015cc80 2024-09-10T22:10:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.460+0000 7f165cc00640 1 -- 172.21.15.66:0/4131440324 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f165800f040 con 0x7f166015cc80 2024-09-10T22:10:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.464+0000 7f165cc00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f163c073bd0 0x7f163c076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.464+0000 7f165cc00640 1 -- 172.21.15.66:0/4131440324 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7f1658092940 con 0x7f166015cc80 2024-09-10T22:10:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.464+0000 7f165fe00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f163c073bd0 0x7f163c076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.464+0000 7f165fe00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f163c073bd0 0x7f163c076090 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1650007fd0 tx=0x7f1650009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.468+0000 7f165cc00640 1 -- 172.21.15.66:0/4131440324 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1658016030 con 0x7f166015cc80 2024-09-10T22:10:33.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.676+0000 7f1666e00640 1 -- 172.21.15.66:0/4131440324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1660070470 con 0x7f166015cc80 2024-09-10T22:10:33.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.676+0000 7f165cc00640 1 -- 172.21.15.66:0/4131440324 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+3908 (secure 0 0 0) 0x7f165805f010 con 0x7f166015cc80 2024-09-10T22:10:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:33.682 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":59,"btime":"2024-09-10T22:10:27:169381+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3360242011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3360242011},{"type":"v1","addr":"172.21.15.94:6837","nonce":3360242011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3211574496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3211574496},{"type":"v1","addr":"172.21.15.66:6835","nonce":3211574496}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/553876835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":553876835},{"type":"v1","addr":"172.21.15.66:6834","nonce":553876835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7168,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1806349513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1806349513},{"type":"v1","addr":"172.21.15.94:6836","nonce":1806349513}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[]} 2024-09-10T22:10:33.682 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 59 2024-09-10T22:10:33.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.680+0000 7f1666e00640 1 -- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f163c073bd0 msgr2=0x7f163c076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:33.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.680+0000 7f1666e00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f163c073bd0 0x7f163c076090 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1650007fd0 tx=0x7f1650009040 comp rx=0 tx=0).stop 2024-09-10T22:10:33.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.680+0000 7f1666e00640 1 -- 172.21.15.66:0/4131440324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f166015cc80 msgr2=0x7f16600b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:33.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.680+0000 7f1666e00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f166015cc80 0x7f16600b0710 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f165800e9a0 tx=0x7f165800ee60 comp rx=0 tx=0).stop 2024-09-10T22:10:33.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.680+0000 7f1666e00640 1 -- 172.21.15.66:0/4131440324 shutdown_connections 2024-09-10T22:10:33.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.680+0000 7f1666e00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f163c073bd0 0x7f163c076090 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.680+0000 7f1666e00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f166015cc80 0x7f16600b0710 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.680+0000 7f1666e00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1660159900 0x7f16601f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.680+0000 7f1666e00640 1 --2- 172.21.15.66:0/4131440324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16600aee20 0x7f1660145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.680+0000 7f1666e00640 1 -- 172.21.15.66:0/4131440324 >> 172.21.15.66:0/4131440324 conn(0x7f16600cfa30 msgr2=0x7f1660073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:33.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.680+0000 7f1666e00640 1 -- 172.21.15.66:0/4131440324 shutdown_connections 2024-09-10T22:10:33.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.680+0000 7f1666e00640 1 -- 172.21.15.66:0/4131440324 wait complete. 2024-09-10T22:10:33.708 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:10:33.708 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:10:33.709 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:10:33.709 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:10:33.709 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:10:33.709 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:10:33.709 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:10:33.709 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:10:33.710 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:10:33.710 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:10:33.710 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:10:33.710 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:10:33.710 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:10:33.710 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:10:33.710 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:10:33.710 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:10:33.720 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:10:33.720 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:10:33.722 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:10:33.722 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:10:33.726 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:10:33.727 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:10:33.728 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:10:33.728 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:10:33.772 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:10:33.772 DEBUG:teuthology.parallel:result is None 2024-09-10T22:10:33.773 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:10:33.773 DEBUG:teuthology.parallel:result is None 2024-09-10T22:10:33.774 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:10:33.774 DEBUG:teuthology.parallel:result is None 2024-09-10T22:10:33.775 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:10:33.775 DEBUG:teuthology.parallel:result is None 2024-09-10T22:10:33.776 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T22:10:33.859 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:10:33.860 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:10:33.860 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:10:33.862 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:10:33.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.924+0000 7f3334800640 1 -- 172.21.15.66:0/3692219212 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f332c0c4b10 msgr2=0x7f332c06e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:33.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.924+0000 7f3334800640 1 --2- 172.21.15.66:0/3692219212 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f332c0c4b10 0x7f332c06e960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3320009840 tx=0x7f332001c6b0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.924+0000 7f3334800640 1 -- 172.21.15.66:0/3692219212 shutdown_connections 2024-09-10T22:10:33.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.924+0000 7f3334800640 1 --2- 172.21.15.66:0/3692219212 >> v2:172.21.15.94:3300/0 conn(0x7f332c0c4ef0 0x7f332c0c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.924+0000 7f3334800640 1 --2- 172.21.15.66:0/3692219212 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f332c0c4b10 0x7f332c06e960 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.924+0000 7f3334800640 1 -- 172.21.15.66:0/3692219212 >> 172.21.15.66:0/3692219212 conn(0x7f332c0cfa30 msgr2=0x7f332c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:33.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.924+0000 7f3334800640 1 -- 172.21.15.66:0/3692219212 shutdown_connections 2024-09-10T22:10:33.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.924+0000 7f3334800640 1 -- 172.21.15.66:0/3692219212 wait complete. 2024-09-10T22:10:33.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.924+0000 7f3334800640 1 Processor -- start 2024-09-10T22:10:33.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.924+0000 7f3334800640 1 -- start start 2024-09-10T22:10:33.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3334800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f332c0ae620 0x7f332c145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:33.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3334800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f332c0c4b10 0x7f332c073830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:33.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3334800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f332c0c4ef0 0x7f332c1facc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:33.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3334800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f332c167fa0 con 0x7f332c0c4b10 2024-09-10T22:10:33.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3334800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f332c167e20 con 0x7f332c0c4ef0 2024-09-10T22:10:33.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3334800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f332c168120 con 0x7f332c0ae620 2024-09-10T22:10:33.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3331400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f332c0c4b10 0x7f332c073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:33.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3331400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f332c0c4b10 0x7f332c073830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54700/0 (socket says 172.21.15.66:54700) 2024-09-10T22:10:33.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3331e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f332c0ae620 0x7f332c145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:33.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3331400640 1 -- 172.21.15.66:0/2069899059 learned_addr learned my addr 172.21.15.66:0/2069899059 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:33.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3332800640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f332c0c4ef0 0x7f332c1facc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:33.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3331400640 1 -- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f332c0ae620 msgr2=0x7f332c145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:33.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3331400640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f332c0ae620 0x7f332c145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3331400640 1 -- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f332c0c4ef0 msgr2=0x7f332c1facc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:33.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3331400640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f332c0c4ef0 0x7f332c1facc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:33.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3331400640 1 -- 172.21.15.66:0/2069899059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332c1fb6a0 con 0x7f332c0c4b10 2024-09-10T22:10:33.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3331400640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f332c0c4b10 0x7f332c073830 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f331c001ec0 tx=0x7f331c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:33.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3326000640 1 -- 172.21.15.66:0/2069899059 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c00a7e0 con 0x7f332c0c4b10 2024-09-10T22:10:33.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3326000640 1 -- 172.21.15.66:0/2069899059 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f331c017070 con 0x7f332c0c4b10 2024-09-10T22:10:33.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3334800640 1 -- 172.21.15.66:0/2069899059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332c1fb980 con 0x7f332c0c4b10 2024-09-10T22:10:33.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3326000640 1 -- 172.21.15.66:0/2069899059 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c00ad30 con 0x7f332c0c4b10 2024-09-10T22:10:33.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3334800640 1 -- 172.21.15.66:0/2069899059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f332c073310 con 0x7f332c0c4b10 2024-09-10T22:10:33.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3334800640 1 -- 172.21.15.66:0/2069899059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f332c06e960 con 0x7f332c0c4b10 2024-09-10T22:10:33.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.928+0000 7f3326000640 1 -- 172.21.15.66:0/2069899059 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f331c01c460 con 0x7f332c0c4b10 2024-09-10T22:10:33.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.932+0000 7f3326000640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3310073bd0 0x7f3310076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:33.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.932+0000 7f3326000640 1 -- 172.21.15.66:0/2069899059 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4999+0+0 (secure 0 0 0) 0x7f331c0920c0 con 0x7f332c0c4b10 2024-09-10T22:10:33.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.932+0000 7f3331e00640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3310073bd0 0x7f3310076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:33.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.932+0000 7f3331e00640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3310073bd0 0x7f3310076090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f33200ae000 tx=0x7f332001aef0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:33.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:33.936+0000 7f3326000640 1 -- 172.21.15.66:0/2069899059 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f331c05e790 con 0x7f332c0c4b10 2024-09-10T22:10:34.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.120+0000 7f3334800640 1 -- 172.21.15.66:0/2069899059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f332c1fbb50 con 0x7f332c0c4b10 2024-09-10T22:10:34.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.204+0000 7f3326000640 1 -- 172.21.15.66:0/2069899059 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v94) ==== 97+0+0 (secure 0 0 0) 0x7f331c0631c0 con 0x7f332c0c4b10 2024-09-10T22:10:34.209 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-09-10T22:10:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.208+0000 7f3334800640 1 -- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3310073bd0 msgr2=0x7f3310076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.208+0000 7f3334800640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3310073bd0 0x7f3310076090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f33200ae000 tx=0x7f332001aef0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.208+0000 7f3334800640 1 -- 172.21.15.66:0/2069899059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f332c0c4b10 msgr2=0x7f332c073830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.208+0000 7f3334800640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f332c0c4b10 0x7f332c073830 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f331c001ec0 tx=0x7f331c00cb90 comp rx=0 tx=0).stop 2024-09-10T22:10:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.208+0000 7f3334800640 1 -- 172.21.15.66:0/2069899059 shutdown_connections 2024-09-10T22:10:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.208+0000 7f3334800640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3310073bd0 0x7f3310076090 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.208+0000 7f3334800640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f332c0c4ef0 0x7f332c1facc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.208+0000 7f3334800640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f332c0c4b10 0x7f332c073830 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.208+0000 7f3334800640 1 --2- 172.21.15.66:0/2069899059 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f332c0ae620 0x7f332c145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.208+0000 7f3334800640 1 -- 172.21.15.66:0/2069899059 >> 172.21.15.66:0/2069899059 conn(0x7f332c0cfa30 msgr2=0x7f332c152070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.208+0000 7f3334800640 1 -- 172.21.15.66:0/2069899059 shutdown_connections 2024-09-10T22:10:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.208+0000 7f3334800640 1 -- 172.21.15.66:0/2069899059 wait complete. 2024-09-10T22:10:34.233 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T22:10:34.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.376+0000 7fe887200640 1 -- 172.21.15.66:0/2628953393 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe888073530 msgr2=0x7fe888073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:34.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.376+0000 7fe887200640 1 --2- 172.21.15.66:0/2628953393 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe888073530 0x7fe888073910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe874009900 tx=0x7fe87401c7c0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.376+0000 7fe887200640 1 -- 172.21.15.66:0/2628953393 shutdown_connections 2024-09-10T22:10:34.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.376+0000 7fe887200640 1 --2- 172.21.15.66:0/2628953393 >> v2:172.21.15.94:3300/0 conn(0x7fe8880cb5b0 0x7fe8880cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.376+0000 7fe887200640 1 --2- 172.21.15.66:0/2628953393 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe888073530 0x7fe888073910 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.376+0000 7fe887200640 1 -- 172.21.15.66:0/2628953393 >> 172.21.15.66:0/2628953393 conn(0x7fe8880cd4b0 msgr2=0x7fe8880cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:34.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.376+0000 7fe887200640 1 -- 172.21.15.66:0/2628953393 shutdown_connections 2024-09-10T22:10:34.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.376+0000 7fe887200640 1 -- 172.21.15.66:0/2628953393 wait complete. 2024-09-10T22:10:34.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.376+0000 7fe887200640 1 Processor -- start 2024-09-10T22:10:34.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.376+0000 7fe887200640 1 -- start start 2024-09-10T22:10:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe887200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe888070b20 0x7fe8881510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe887200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe888073530 0x7fe888135ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe887200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8880cb5b0 0x7fe88814d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe887200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe888167c90 con 0x7fe8880cb5b0 2024-09-10T22:10:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe887200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe888167b10 con 0x7fe888073530 2024-09-10T22:10:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe887200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe888167e10 con 0x7fe888070b20 2024-09-10T22:10:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe886800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8880cb5b0 0x7fe88814d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe885e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe888070b20 0x7fe8881510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe886800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8880cb5b0 0x7fe88814d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54704/0 (socket says 172.21.15.66:54704) 2024-09-10T22:10:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe885400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe888073530 0x7fe888135ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe886800640 1 -- 172.21.15.66:0/471181488 learned_addr learned my addr 172.21.15.66:0/471181488 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:34.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe885400640 1 -- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe888070b20 msgr2=0x7fe8881510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:34.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe885400640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe888070b20 0x7fe8881510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe885400640 1 -- 172.21.15.66:0/471181488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8880cb5b0 msgr2=0x7fe88814d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:34.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe885400640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8880cb5b0 0x7fe88814d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe885400640 1 -- 172.21.15.66:0/471181488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe888152570 con 0x7fe888073530 2024-09-10T22:10:34.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe886800640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8880cb5b0 0x7fe88814d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:10:34.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe885400640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe888073530 0x7fe888135ac0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe8740ac000 tx=0x7fe874005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:34.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe87e000640 1 -- 172.21.15.66:0/471181488 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8740b2040 con 0x7fe888073530 2024-09-10T22:10:34.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe87e000640 1 -- 172.21.15.66:0/471181488 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe874004350 con 0x7fe888073530 2024-09-10T22:10:34.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe887200640 1 -- 172.21.15.66:0/471181488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8881363f0 con 0x7fe888073530 2024-09-10T22:10:34.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe87e000640 1 -- 172.21.15.66:0/471181488 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874005280 con 0x7fe888073530 2024-09-10T22:10:34.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe887200640 1 -- 172.21.15.66:0/471181488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8881369b0 con 0x7fe888073530 2024-09-10T22:10:34.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.380+0000 7fe887200640 1 -- 172.21.15.66:0/471181488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe888073910 con 0x7fe888073530 2024-09-10T22:10:34.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.384+0000 7fe87e000640 1 -- 172.21.15.66:0/471181488 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fe8740046d0 con 0x7fe888073530 2024-09-10T22:10:34.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.384+0000 7fe87e000640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe868073c00 0x7fe8680760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:34.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.384+0000 7fe87e000640 1 -- 172.21.15.66:0/471181488 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4913+0+0 (secure 0 0 0) 0x7fe8740bb070 con 0x7fe888073530 2024-09-10T22:10:34.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.384+0000 7fe885e00640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe868073c00 0x7fe8680760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:34.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.384+0000 7fe885e00640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe868073c00 0x7fe8680760c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe878007fd0 tx=0x7fe878009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:34.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.388+0000 7fe87e000640 1 -- 172.21.15.66:0/471181488 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe8740164d0 con 0x7fe888073530 2024-09-10T22:10:34.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.620+0000 7fe887200640 1 -- 172.21.15.66:0/471181488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe8880b5df0 con 0x7fe888073530 2024-09-10T22:10:34.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.620+0000 7fe87e000640 1 -- 172.21.15.66:0/471181488 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) ==== 80+0+5304 (secure 0 0 0) 0x7fe8740ffe20 con 0x7fe888073530 2024-09-10T22:10:34.625 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:34.625 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:10:34.625 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-09-10T22:10:34.625 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmxQD7CBAAttZP0KtdjTI7QWbD7CVinQ==", 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmI7ufFRAANelhvqylKyfGPVfBuY40kg==", 2024-09-10T22:10:34.626 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQnVoEBAAvsZ/TSOOP/JgJBr/aHwI/g==", 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:10:34.627 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.628 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.628 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-09-10T22:10:34.628 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmRQkTHRAAHPIdwqtkkyh31D7IO2vxPA==", 2024-09-10T22:10:34.628 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.628 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:10:34.628 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:10:34.628 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:10:34.628 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.628 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBCweBmDJkxJBAAVSQNiMAQl5bR3RUwVOuGYw==", 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBFweBmwcdsOxAAPPQyzktcfiF8pJNP8uAjwg==", 2024-09-10T22:10:34.629 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBJweBmXQhZExAAnEf8mjGX6nnYwWygKjo/2A==", 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBNweBmlauHAxAA/k9xRqbHZsqJq9OYbJKfNQ==", 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBRweBmHnFDJRAAgF1ZAPvmoNzeUftZQyL0eg==", 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.631 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBUweBm/hXpORAA6jpOI/I+QyB2mZsEoAWF0w==", 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBYweBmYu2MExAAPD9WcxrBLzwwJQm19z7YcQ==", 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbweBmXfJ9KRAA/AbWlli+Qfg7gy0SblJ7fg==", 2024-09-10T22:10:34.633 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw==", 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmbS6IKhAAaEmK9irxMipALgkAct/+OQ==", 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-09-10T22:10:34.635 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQ47KJhAAI50ZaSQIhH6FxRM48JZICA==", 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmUcLCMBAAdf4tfcpVaZoaKjS1UzrxhA==", 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:10:34.636 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA+weBmPIAzKBAAkyVV875PNhBmEoKsrAkg8w==", 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmLiy1BBAADnraML8IUet7Sw0j83TUVQ==", 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmItWpABAAlJ+6dFjvtBV3g+jYmUh0tQ==", 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:10:34.638 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:10:34.639 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:10:34.639 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:10:34.639 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.639 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.639 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T22:10:34.639 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:10:34.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.624+0000 7fe887200640 1 -- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe868073c00 msgr2=0x7fe8680760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:34.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.624+0000 7fe887200640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe868073c00 0x7fe8680760c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe878007fd0 tx=0x7fe878009040 comp rx=0 tx=0).stop 2024-09-10T22:10:34.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.624+0000 7fe887200640 1 -- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe888073530 msgr2=0x7fe888135ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:34.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.624+0000 7fe887200640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe888073530 0x7fe888135ac0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe8740ac000 tx=0x7fe874005e70 comp rx=0 tx=0).stop 2024-09-10T22:10:34.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.624+0000 7fe887200640 1 -- 172.21.15.66:0/471181488 shutdown_connections 2024-09-10T22:10:34.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.624+0000 7fe887200640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe868073c00 0x7fe8680760c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.624+0000 7fe887200640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8880cb5b0 0x7fe88814d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.624+0000 7fe887200640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe888073530 0x7fe888135ac0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.624+0000 7fe887200640 1 --2- 172.21.15.66:0/471181488 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe888070b20 0x7fe8881510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.624+0000 7fe887200640 1 -- 172.21.15.66:0/471181488 >> 172.21.15.66:0/471181488 conn(0x7fe8880cd4b0 msgr2=0x7fe88806cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:34.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.624+0000 7fe887200640 1 -- 172.21.15.66:0/471181488 shutdown_connections 2024-09-10T22:10:34.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.624+0000 7fe887200640 1 -- 172.21.15.66:0/471181488 wait complete. 2024-09-10T22:10:34.647 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:10:34.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.792+0000 7ff027200640 1 -- 172.21.15.66:0/1067008869 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff02015a310 msgr2=0x7ff0200c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:34.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.792+0000 7ff027200640 1 --2- 172.21.15.66:0/1067008869 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff02015a310 0x7ff0200c48b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff010009870 tx=0x7ff01001c7d0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.792+0000 7ff027200640 1 -- 172.21.15.66:0/1067008869 shutdown_connections 2024-09-10T22:10:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.792+0000 7ff027200640 1 --2- 172.21.15.66:0/1067008869 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff02015a310 0x7ff0200c48b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.792+0000 7ff027200640 1 -- 172.21.15.66:0/1067008869 >> 172.21.15.66:0/1067008869 conn(0x7ff0200cfa30 msgr2=0x7ff0200cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.792+0000 7ff027200640 1 -- 172.21.15.66:0/1067008869 shutdown_connections 2024-09-10T22:10:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.792+0000 7ff027200640 1 -- 172.21.15.66:0/1067008869 wait complete. 2024-09-10T22:10:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.792+0000 7ff027200640 1 Processor -- start 2024-09-10T22:10:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.792+0000 7ff027200640 1 -- start start 2024-09-10T22:10:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.792+0000 7ff027200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0200aeec0 0x7ff02014dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.792+0000 7ff027200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff02015a310 0x7ff02014c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff027200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0200c6b00 0x7ff02014c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff027200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff0201687b0 con 0x7ff0200c6b00 2024-09-10T22:10:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff027200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff020168630 con 0x7ff02015a310 2024-09-10T22:10:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff027200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff020168930 con 0x7ff0200aeec0 2024-09-10T22:10:34.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff025200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0200c6b00 0x7ff02014c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:34.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff01fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0200aeec0 0x7ff02014dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:34.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff025200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0200c6b00 0x7ff02014c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54720/0 (socket says 172.21.15.66:54720) 2024-09-10T22:10:34.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff025200640 1 -- 172.21.15.66:0/3676644625 learned_addr learned my addr 172.21.15.66:0/3676644625 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:34.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff017e00640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff02015a310 0x7ff02014c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:34.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff025200640 1 -- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0200aeec0 msgr2=0x7ff02014dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:34.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff025200640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0200aeec0 0x7ff02014dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff025200640 1 -- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff02015a310 msgr2=0x7ff02014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:34.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff025200640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff02015a310 0x7ff02014c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff025200640 1 -- 172.21.15.66:0/3676644625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0200c72e0 con 0x7ff0200c6b00 2024-09-10T22:10:34.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff025200640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0200c6b00 0x7ff02014c820 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff01800cb90 tx=0x7ff01800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:34.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff01cc00640 1 -- 172.21.15.66:0/3676644625 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018016020 con 0x7ff0200c6b00 2024-09-10T22:10:34.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff01cc00640 1 -- 172.21.15.66:0/3676644625 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff01800ab60 con 0x7ff0200c6b00 2024-09-10T22:10:34.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff01cc00640 1 -- 172.21.15.66:0/3676644625 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018012420 con 0x7ff0200c6b00 2024-09-10T22:10:34.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff027200640 1 -- 172.21.15.66:0/3676644625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0200c7000 con 0x7ff0200c6b00 2024-09-10T22:10:34.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff027200640 1 -- 172.21.15.66:0/3676644625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0200c96b0 con 0x7ff0200c6b00 2024-09-10T22:10:34.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff027200640 1 -- 172.21.15.66:0/3676644625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0200c48b0 con 0x7ff0200c6b00 2024-09-10T22:10:34.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.796+0000 7ff01cc00640 1 -- 172.21.15.66:0/3676644625 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7ff0180125c0 con 0x7ff0200c6b00 2024-09-10T22:10:34.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.800+0000 7ff01cc00640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feffc073bd0 0x7feffc076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:34.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.800+0000 7ff01cc00640 1 -- 172.21.15.66:0/3676644625 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4913+0+0 (secure 0 0 0) 0x7ff018092800 con 0x7ff0200c6b00 2024-09-10T22:10:34.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.800+0000 7ff01fe00640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feffc073bd0 0x7feffc076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:34.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.800+0000 7ff01fe00640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feffc073bd0 0x7feffc076090 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff0100af000 tx=0x7ff0100b2040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:34.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.804+0000 7ff01cc00640 1 -- 172.21.15.66:0/3676644625 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff018020020 con 0x7ff0200c6b00 2024-09-10T22:10:34.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.980+0000 7ff027200640 1 -- 172.21.15.66:0/3676644625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff0200d0190 con 0x7ff0200c6b00 2024-09-10T22:10:34.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff01cc00640 1 -- 172.21.15.66:0/3676644625 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) ==== 84+0+65 (secure 0 0 0) 0x7ff01805ef10 con 0x7ff0200c6b00 2024-09-10T22:10:34.986 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:34.986 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:10:34.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:34.987 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:10:34.987 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:10:34.987 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:34.987 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:10:34.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff027200640 1 -- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feffc073bd0 msgr2=0x7feffc076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:34.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff027200640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feffc073bd0 0x7feffc076090 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff0100af000 tx=0x7ff0100b2040 comp rx=0 tx=0).stop 2024-09-10T22:10:34.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff027200640 1 -- 172.21.15.66:0/3676644625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0200c6b00 msgr2=0x7ff02014c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:34.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff027200640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0200c6b00 0x7ff02014c820 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff01800cb90 tx=0x7ff01800cc80 comp rx=0 tx=0).stop 2024-09-10T22:10:34.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff027200640 1 -- 172.21.15.66:0/3676644625 shutdown_connections 2024-09-10T22:10:34.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff027200640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feffc073bd0 0x7feffc076090 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff027200640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0200c6b00 0x7ff02014c820 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff027200640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff02015a310 0x7ff02014c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff027200640 1 --2- 172.21.15.66:0/3676644625 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0200aeec0 0x7ff02014dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:34.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff027200640 1 -- 172.21.15.66:0/3676644625 >> 172.21.15.66:0/3676644625 conn(0x7ff0200cfa30 msgr2=0x7ff0200b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:34.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff027200640 1 -- 172.21.15.66:0/3676644625 shutdown_connections 2024-09-10T22:10:34.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:34.984+0000 7ff027200640 1 -- 172.21.15.66:0/3676644625 wait complete. 2024-09-10T22:10:35.008 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:10:35.008 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:10:35.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 -- 172.21.15.66:0/3182885784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fead40b3cd0 msgr2=0x7fead41600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:35.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 --2- 172.21.15.66:0/3182885784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fead40b3cd0 0x7fead41600b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7feabc009900 tx=0x7feabc01c850 comp rx=0 tx=0).stop 2024-09-10T22:10:35.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 -- 172.21.15.66:0/3182885784 shutdown_connections 2024-09-10T22:10:35.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 --2- 172.21.15.66:0/3182885784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fead40b3cd0 0x7fead41600b0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:35.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 --2- 172.21.15.66:0/3182885784 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fead406f610 0x7fead415b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:35.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 -- 172.21.15.66:0/3182885784 >> 172.21.15.66:0/3182885784 conn(0x7fead4068990 msgr2=0x7fead4068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:35.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 -- 172.21.15.66:0/3182885784 shutdown_connections 2024-09-10T22:10:35.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 -- 172.21.15.66:0/3182885784 wait complete. 2024-09-10T22:10:35.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 Processor -- start 2024-09-10T22:10:35.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 -- start start 2024-09-10T22:10:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead406f230 0x7fead4141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fead406f610 0x7fead4165e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fead40b3cd0 0x7fead4154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fead4168c30 con 0x7fead40b3cd0 2024-09-10T22:10:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fead4168ab0 con 0x7fead406f230 2024-09-10T22:10:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feadb000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fead4168db0 con 0x7fead406f610 2024-09-10T22:10:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7fead9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fead40b3cd0 0x7fead4154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feacbe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fead406f610 0x7fead4165e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7fead9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fead40b3cd0 0x7fead4154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54744/0 (socket says 172.21.15.66:54744) 2024-09-10T22:10:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7fead3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead406f230 0x7fead4141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7fead9000640 1 -- 172.21.15.66:0/2806524145 learned_addr learned my addr 172.21.15.66:0/2806524145 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.156+0000 7feacbe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fead406f610 0x7fead4165e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:50182/0 (socket says 172.21.15.66:50182) 2024-09-10T22:10:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7fead9000640 1 -- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fead406f610 msgr2=0x7fead4165e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7fead9000640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fead406f610 0x7fead4165e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7fead9000640 1 -- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead406f230 msgr2=0x7fead4141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7fead9000640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead406f230 0x7fead4141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7fead9000640 1 -- 172.21.15.66:0/2806524145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead4141e90 con 0x7fead40b3cd0 2024-09-10T22:10:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7feacbe00640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fead406f610 0x7fead4165e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:10:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7fead9000640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fead40b3cd0 0x7fead4154cb0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7feacc00cce0 tx=0x7feacc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7fead0c00640 1 -- 172.21.15.66:0/2806524145 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc00f450 con 0x7fead40b3cd0 2024-09-10T22:10:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7fead0c00640 1 -- 172.21.15.66:0/2806524145 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feacc00f700 con 0x7fead40b3cd0 2024-09-10T22:10:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7feadb000640 1 -- 172.21.15.66:0/2806524145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead41faba0 con 0x7fead40b3cd0 2024-09-10T22:10:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7fead0c00640 1 -- 172.21.15.66:0/2806524145 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc00fb50 con 0x7fead40b3cd0 2024-09-10T22:10:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7feadb000640 1 -- 172.21.15.66:0/2806524145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead41fb160 con 0x7fead40b3cd0 2024-09-10T22:10:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7feadb000640 1 -- 172.21.15.66:0/2806524145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead415b5d0 con 0x7fead40b3cd0 2024-09-10T22:10:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.160+0000 7fead0c00640 1 -- 172.21.15.66:0/2806524145 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7feacc00e3b0 con 0x7fead40b3cd0 2024-09-10T22:10:35.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.164+0000 7fead0c00640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feab0073bd0 0x7feab0076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:35.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.164+0000 7fead0c00640 1 -- 172.21.15.66:0/2806524145 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4913+0+0 (secure 0 0 0) 0x7feacc092ea0 con 0x7fead40b3cd0 2024-09-10T22:10:35.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.164+0000 7fead3e00640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feab0073bd0 0x7feab0076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:35.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.164+0000 7fead3e00640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feab0073bd0 0x7feab0076090 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7feac4007fd0 tx=0x7feac4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:35.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.164+0000 7fead0c00640 1 -- 172.21.15.66:0/2806524145 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feacc014030 con 0x7fead40b3cd0 2024-09-10T22:10:35.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:35.344+0000 7feadb000640 1 -- 172.21.15.66:0/2806524145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fead4160a30 con 0x7fead40b3cd0 2024-09-10T22:10:36.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.216+0000 7fead0c00640 1 -- 172.21.15.66:0/2806524145 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) ==== 138+0+0 (secure 0 0 0) 0x7feacc093340 con 0x7fead40b3cd0 2024-09-10T22:10:36.223 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:10:36.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.220+0000 7feadb000640 1 -- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feab0073bd0 msgr2=0x7feab0076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:36.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.220+0000 7feadb000640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feab0073bd0 0x7feab0076090 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7feac4007fd0 tx=0x7feac4009040 comp rx=0 tx=0).stop 2024-09-10T22:10:36.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.220+0000 7feadb000640 1 -- 172.21.15.66:0/2806524145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fead40b3cd0 msgr2=0x7fead4154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:36.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.220+0000 7feadb000640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fead40b3cd0 0x7fead4154cb0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7feacc00cce0 tx=0x7feacc00a620 comp rx=0 tx=0).stop 2024-09-10T22:10:36.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.220+0000 7feadb000640 1 -- 172.21.15.66:0/2806524145 shutdown_connections 2024-09-10T22:10:36.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.220+0000 7feadb000640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feab0073bd0 0x7feab0076090 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:36.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.220+0000 7feadb000640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fead40b3cd0 0x7fead4154cb0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:36.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.220+0000 7feadb000640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fead406f610 0x7fead4165e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:36.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.220+0000 7feadb000640 1 --2- 172.21.15.66:0/2806524145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead406f230 0x7fead4141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:36.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.220+0000 7feadb000640 1 -- 172.21.15.66:0/2806524145 >> 172.21.15.66:0/2806524145 conn(0x7fead4068990 msgr2=0x7fead415ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:36.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.224+0000 7feadb000640 1 -- 172.21.15.66:0/2806524145 shutdown_connections 2024-09-10T22:10:36.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.224+0000 7feadb000640 1 -- 172.21.15.66:0/2806524145 wait complete. 2024-09-10T22:10:36.255 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:10:36.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.408+0000 7f2004c00640 1 -- 172.21.15.66:0/4192307761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2000159900 msgr2=0x7f20000ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:36.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.408+0000 7f2004c00640 1 --2- 172.21.15.66:0/4192307761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2000159900 0x7f20000ca080 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009900 tx=0x7f1ff001c850 comp rx=0 tx=0).stop 2024-09-10T22:10:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.408+0000 7f2004c00640 1 -- 172.21.15.66:0/4192307761 shutdown_connections 2024-09-10T22:10:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.408+0000 7f2004c00640 1 --2- 172.21.15.66:0/4192307761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f200015cc80 0x7f20000ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.408+0000 7f2004c00640 1 --2- 172.21.15.66:0/4192307761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2000159900 0x7f20000ca080 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.408+0000 7f2004c00640 1 -- 172.21.15.66:0/4192307761 >> 172.21.15.66:0/4192307761 conn(0x7f20000cfa30 msgr2=0x7f20000cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.408+0000 7f2004c00640 1 -- 172.21.15.66:0/4192307761 shutdown_connections 2024-09-10T22:10:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.408+0000 7f2004c00640 1 -- 172.21.15.66:0/4192307761 wait complete. 2024-09-10T22:10:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f2004c00640 1 Processor -- start 2024-09-10T22:10:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f2004c00640 1 -- start start 2024-09-10T22:10:36.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f2004c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20000aec50 0x7f20000b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:36.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f2004c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2000159900 0x7f200006f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:36.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f2004c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f200015cc80 0x7f200006fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f2004c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2000168f70 con 0x7f20000aec50 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f2004c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2000168df0 con 0x7f2000159900 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f2004c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f20001690f0 con 0x7f200015cc80 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ffde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20000aec50 0x7f20000b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ffde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20000aec50 0x7f20000b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54768/0 (socket says 172.21.15.66:54768) 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ffde00640 1 -- 172.21.15.66:0/337291349 learned_addr learned my addr 172.21.15.66:0/337291349 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ff7e00640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2000159900 0x7f200006f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ffe800640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f200015cc80 0x7f200006fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ffde00640 1 -- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f200015cc80 msgr2=0x7f200006fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ffde00640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f200015cc80 0x7f200006fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ffde00640 1 -- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2000159900 msgr2=0x7f200006f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ffde00640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2000159900 0x7f200006f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ffde00640 1 -- 172.21.15.66:0/337291349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20001546e0 con 0x7f20000aec50 2024-09-10T22:10:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ffde00640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20000aec50 0x7f20000b4df0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1ff001a480 tx=0x7f1ff00a6ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ff6000640 1 -- 172.21.15.66:0/337291349 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00ac070 con 0x7f20000aec50 2024-09-10T22:10:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ff6000640 1 -- 172.21.15.66:0/337291349 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ff0004030 con 0x7f20000aec50 2024-09-10T22:10:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f1ff6000640 1 -- 172.21.15.66:0/337291349 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0004480 con 0x7f20000aec50 2024-09-10T22:10:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f2004c00640 1 -- 172.21.15.66:0/337291349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20001549c0 con 0x7f20000aec50 2024-09-10T22:10:36.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f2004c00640 1 -- 172.21.15.66:0/337291349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2000154f80 con 0x7f20000aec50 2024-09-10T22:10:36.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.412+0000 7f2004c00640 1 -- 172.21.15.66:0/337291349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2000070470 con 0x7f20000aec50 2024-09-10T22:10:36.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.416+0000 7f1ff6000640 1 -- 172.21.15.66:0/337291349 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f1ff0005040 con 0x7f20000aec50 2024-09-10T22:10:36.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.416+0000 7f1ff6000640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1fdc073bd0 0x7f1fdc076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:36.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.416+0000 7f1ff6000640 1 -- 172.21.15.66:0/337291349 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5283+0+0 (secure 0 0 0) 0x7f1ff012c410 con 0x7f20000aec50 2024-09-10T22:10:36.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.416+0000 7f1ff7e00640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1fdc073bd0 0x7f1fdc076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:36.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.420+0000 7f1ff7e00640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1fdc073bd0 0x7f1fdc076090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80058d0 tx=0x7f1fe8005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:36.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.420+0000 7f1ff6000640 1 -- 172.21.15.66:0/337291349 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ff00164d0 con 0x7f20000aec50 2024-09-10T22:10:36.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:36.596+0000 7f2004c00640 1 -- 172.21.15.66:0/337291349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f20000b0d90 con 0x7f20000aec50 2024-09-10T22:10:37.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.224+0000 7f1ff6000640 1 -- 172.21.15.66:0/337291349 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) ==== 170+0+0 (secure 0 0 0) 0x7f1ff00f9580 con 0x7f20000aec50 2024-09-10T22:10:37.227 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:10:37.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.228+0000 7f2004c00640 1 -- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1fdc073bd0 msgr2=0x7f1fdc076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:37.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.228+0000 7f2004c00640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1fdc073bd0 0x7f1fdc076090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80058d0 tx=0x7f1fe8005860 comp rx=0 tx=0).stop 2024-09-10T22:10:37.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.228+0000 7f2004c00640 1 -- 172.21.15.66:0/337291349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20000aec50 msgr2=0x7f20000b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:37.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.228+0000 7f2004c00640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20000aec50 0x7f20000b4df0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1ff001a480 tx=0x7f1ff00a6ac0 comp rx=0 tx=0).stop 2024-09-10T22:10:37.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.228+0000 7f2004c00640 1 -- 172.21.15.66:0/337291349 shutdown_connections 2024-09-10T22:10:37.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.228+0000 7f2004c00640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1fdc073bd0 0x7f1fdc076090 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:37.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.228+0000 7f2004c00640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f200015cc80 0x7f200006fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:37.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.228+0000 7f2004c00640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2000159900 0x7f200006f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:37.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.228+0000 7f2004c00640 1 --2- 172.21.15.66:0/337291349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20000aec50 0x7f20000b4df0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:37.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.228+0000 7f2004c00640 1 -- 172.21.15.66:0/337291349 >> 172.21.15.66:0/337291349 conn(0x7f20000cfa30 msgr2=0x7f2000072fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:37.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.228+0000 7f2004c00640 1 -- 172.21.15.66:0/337291349 shutdown_connections 2024-09-10T22:10:37.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.228+0000 7f2004c00640 1 -- 172.21.15.66:0/337291349 wait complete. 2024-09-10T22:10:37.271 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:10:37.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.416+0000 7f49ed600640 1 -- 172.21.15.66:0/3841677634 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e8156430 msgr2=0x7f49e806d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:37.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.416+0000 7f49ed600640 1 --2- 172.21.15.66:0/3841677634 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e8156430 0x7f49e806d630 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009900 tx=0x7f49d801c7f0 comp rx=0 tx=0).stop 2024-09-10T22:10:37.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.416+0000 7f49ed600640 1 -- 172.21.15.66:0/3841677634 shutdown_connections 2024-09-10T22:10:37.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.416+0000 7f49ed600640 1 --2- 172.21.15.66:0/3841677634 >> v2:172.21.15.94:3300/0 conn(0x7f49e8160370 0x7f49e80cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:37.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.416+0000 7f49ed600640 1 --2- 172.21.15.66:0/3841677634 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e8156430 0x7f49e806d630 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:37.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.416+0000 7f49ed600640 1 -- 172.21.15.66:0/3841677634 >> 172.21.15.66:0/3841677634 conn(0x7f49e80d0b30 msgr2=0x7f49e80d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:37.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.416+0000 7f49ed600640 1 -- 172.21.15.66:0/3841677634 shutdown_connections 2024-09-10T22:10:37.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.416+0000 7f49ed600640 1 -- 172.21.15.66:0/3841677634 wait complete. 2024-09-10T22:10:37.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49ed600640 1 Processor -- start 2024-09-10T22:10:37.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49ed600640 1 -- start start 2024-09-10T22:10:37.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49ed600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e8070b20 0x7f49e814d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:37.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49ed600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e8156430 0x7f49e80ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:37.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49ed600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49e8160370 0x7f49e80ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:37.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49ed600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8168d10 con 0x7f49e8160370 2024-09-10T22:10:37.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49ed600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8168b90 con 0x7f49e8156430 2024-09-10T22:10:37.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49ed600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f49e8168e90 con 0x7f49e8070b20 2024-09-10T22:10:37.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49e7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49e8160370 0x7f49e80ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:37.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49e6a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e8070b20 0x7f49e814d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:37.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49dfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e8156430 0x7f49e80ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:37.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49e6a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e8070b20 0x7f49e814d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:50230/0 (socket says 172.21.15.66:50230) 2024-09-10T22:10:37.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49e6a00640 1 -- 172.21.15.66:0/1224114505 learned_addr learned my addr 172.21.15.66:0/1224114505 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:37.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49e6a00640 1 -- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e8156430 msgr2=0x7f49e80ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:37.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49e6a00640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e8156430 0x7f49e80ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:37.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49e6a00640 1 -- 172.21.15.66:0/1224114505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49e8160370 msgr2=0x7f49e80ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:37.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49e6a00640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49e8160370 0x7f49e80ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:37.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49e6a00640 1 -- 172.21.15.66:0/1224114505 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e815b520 con 0x7f49e8070b20 2024-09-10T22:10:37.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49dfe00640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e8156430 0x7f49e80ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:10:37.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49e6a00640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e8070b20 0x7f49e814d310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009850 tx=0x7f49d80a55a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:37.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49dea00640 1 -- 172.21.15.66:0/1224114505 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d80b2040 con 0x7f49e8070b20 2024-09-10T22:10:37.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49dea00640 1 -- 172.21.15.66:0/1224114505 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49d8004470 con 0x7f49e8070b20 2024-09-10T22:10:37.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49dea00640 1 -- 172.21.15.66:0/1224114505 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8005280 con 0x7f49e8070b20 2024-09-10T22:10:37.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49ed600640 1 -- 172.21.15.66:0/1224114505 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e80c80a0 con 0x7f49e8070b20 2024-09-10T22:10:37.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.420+0000 7f49ed600640 1 -- 172.21.15.66:0/1224114505 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e80c8660 con 0x7f49e8070b20 2024-09-10T22:10:37.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.424+0000 7f49ed600640 1 -- 172.21.15.66:0/1224114505 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e806d630 con 0x7f49e8070b20 2024-09-10T22:10:37.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.424+0000 7f49dea00640 1 -- 172.21.15.66:0/1224114505 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f49d8005420 con 0x7f49e8070b20 2024-09-10T22:10:37.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.424+0000 7f49dea00640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f49c4073bd0 0x7f49c4076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:37.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.424+0000 7f49dea00640 1 -- 172.21.15.66:0/1224114505 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(98..98 src has 1..98) ==== 5665+0+0 (secure 0 0 0) 0x7f49d812ba40 con 0x7f49e8070b20 2024-09-10T22:10:37.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.424+0000 7f49dfe00640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f49c4073bd0 0x7f49c4076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:37.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.428+0000 7f49dfe00640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f49c4073bd0 0x7f49c4076090 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f49e80cad40 tx=0x7f49d00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:37.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.428+0000 7f49dea00640 1 -- 172.21.15.66:0/1224114505 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f49d80164d0 con 0x7f49e8070b20 2024-09-10T22:10:37.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:37.640+0000 7f49ed600640 1 -- 172.21.15.66:0/1224114505 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f49e8076650 con 0x7f49e8070b20 2024-09-10T22:10:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.232+0000 7f49dea00640 1 -- 172.21.15.66:0/1224114505 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v62) ==== 174+0+0 (secure 0 0 0) 0x7f49d80f8520 con 0x7f49e8070b20 2024-09-10T22:10:38.237 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 14 and data pool 15 2024-09-10T22:10:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.240+0000 7f49ed600640 1 -- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f49c4073bd0 msgr2=0x7f49c4076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.240+0000 7f49ed600640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f49c4073bd0 0x7f49c4076090 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f49e80cad40 tx=0x7f49d00091e0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.240+0000 7f49ed600640 1 -- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e8070b20 msgr2=0x7f49e814d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.240+0000 7f49ed600640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e8070b20 0x7f49e814d310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009850 tx=0x7f49d80a55a0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.244+0000 7f49ed600640 1 -- 172.21.15.66:0/1224114505 shutdown_connections 2024-09-10T22:10:38.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.244+0000 7f49ed600640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f49c4073bd0 0x7f49c4076090 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.244+0000 7f49ed600640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49e8160370 0x7f49e80ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.244+0000 7f49ed600640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e8156430 0x7f49e80ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.244+0000 7f49ed600640 1 --2- 172.21.15.66:0/1224114505 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e8070b20 0x7f49e814d310 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.244+0000 7f49ed600640 1 -- 172.21.15.66:0/1224114505 >> 172.21.15.66:0/1224114505 conn(0x7f49e80d0b30 msgr2=0x7f49e80cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:38.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.248+0000 7f49ed600640 1 -- 172.21.15.66:0/1224114505 shutdown_connections 2024-09-10T22:10:38.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.248+0000 7f49ed600640 1 -- 172.21.15.66:0/1224114505 wait complete. 2024-09-10T22:10:38.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:10:38.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.428+0000 7f8e36000640 1 -- 172.21.15.66:0/1137545192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e30071cc0 msgr2=0x7f8e300cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:38.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.428+0000 7f8e36000640 1 --2- 172.21.15.66:0/1137545192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e30071cc0 0x7f8e300cf830 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009bc0 tx=0x7f8e2801cb10 comp rx=0 tx=0).stop 2024-09-10T22:10:38.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.428+0000 7f8e36000640 1 -- 172.21.15.66:0/1137545192 shutdown_connections 2024-09-10T22:10:38.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.428+0000 7f8e36000640 1 --2- 172.21.15.66:0/1137545192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e30071cc0 0x7f8e300cf830 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.428+0000 7f8e36000640 1 --2- 172.21.15.66:0/1137545192 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e3015f1c0 0x7f8e300cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.428+0000 7f8e36000640 1 -- 172.21.15.66:0/1137545192 >> 172.21.15.66:0/1137545192 conn(0x7f8e300d1c60 msgr2=0x7f8e300d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:38.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.428+0000 7f8e36000640 1 -- 172.21.15.66:0/1137545192 shutdown_connections 2024-09-10T22:10:38.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.428+0000 7f8e36000640 1 -- 172.21.15.66:0/1137545192 wait complete. 2024-09-10T22:10:38.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e36000640 1 Processor -- start 2024-09-10T22:10:38.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e36000640 1 -- start start 2024-09-10T22:10:38.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e36000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e30071cc0 0x7f8e3014e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:38.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e36000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e300ccd20 0x7f8e300c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e36000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3015f1c0 0x7f8e300afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e36000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30168a00 con 0x7f8e300ccd20 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e36000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30168880 con 0x7f8e3015f1c0 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e36000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8e30168b80 con 0x7f8e30071cc0 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e26a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e300ccd20 0x7f8e300c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e26a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e300ccd20 0x7f8e300c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54800/0 (socket says 172.21.15.66:54800) 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e26a00640 1 -- 172.21.15.66:0/50108695 learned_addr learned my addr 172.21.15.66:0/50108695 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e2f400640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e30071cc0 0x7f8e3014e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e2fe00640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3015f1c0 0x7f8e300afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e26a00640 1 -- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e30071cc0 msgr2=0x7f8e3014e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e26a00640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e30071cc0 0x7f8e3014e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e26a00640 1 -- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3015f1c0 msgr2=0x7f8e300afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:38.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e26a00640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3015f1c0 0x7f8e300afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e26a00640 1 -- 172.21.15.66:0/50108695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e300c7190 con 0x7f8e300ccd20 2024-09-10T22:10:38.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e2fe00640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3015f1c0 0x7f8e300afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:38.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e26a00640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e300ccd20 0x7f8e300c6810 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800c460 tx=0x7f8e1800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:38.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e27e00640 1 -- 172.21.15.66:0/50108695 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18016020 con 0x7f8e300ccd20 2024-09-10T22:10:38.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e27e00640 1 -- 172.21.15.66:0/50108695 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e18012920 con 0x7f8e300ccd20 2024-09-10T22:10:38.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e36000640 1 -- 172.21.15.66:0/50108695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3014de00 con 0x7f8e300ccd20 2024-09-10T22:10:38.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e27e00640 1 -- 172.21.15.66:0/50108695 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18012d50 con 0x7f8e300ccd20 2024-09-10T22:10:38.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.432+0000 7f8e36000640 1 -- 172.21.15.66:0/50108695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3014ee20 con 0x7f8e300ccd20 2024-09-10T22:10:38.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.436+0000 7f8e36000640 1 -- 172.21.15.66:0/50108695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e300cf830 con 0x7f8e300ccd20 2024-09-10T22:10:38.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.436+0000 7f8e27e00640 1 -- 172.21.15.66:0/50108695 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f8e18025440 con 0x7f8e300ccd20 2024-09-10T22:10:38.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.436+0000 7f8e27e00640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8e0c073bd0 0x7f8e0c076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:38.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.436+0000 7f8e27e00640 1 -- 172.21.15.66:0/50108695 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5765+0+0 (secure 0 0 0) 0x7f8e180928d0 con 0x7f8e300ccd20 2024-09-10T22:10:38.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.436+0000 7f8e2f400640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8e0c073bd0 0x7f8e0c076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:38.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.436+0000 7f8e2f400640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8e0c073bd0 0x7f8e0c076090 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009ad0 tx=0x7f8e200091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:38.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.440+0000 7f8e27e00640 1 -- 172.21.15.66:0/50108695 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8e18020020 con 0x7f8e300ccd20 2024-09-10T22:10:38.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.616+0000 7f8e36000640 1 -- 172.21.15.66:0/50108695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e300c89d0 con 0x7f8e300ccd20 2024-09-10T22:10:38.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.616+0000 7f8e27e00640 1 -- 172.21.15.66:0/50108695 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16697 (secure 0 0 0) 0x7f8e1805fa00 con 0x7f8e300ccd20 2024-09-10T22:10:38.621 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:38.621 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":99,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:10:37.643980+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:10:35.358226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T22:10:36.608313+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:10:38.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.620+0000 7f8e36000640 1 -- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8e0c073bd0 msgr2=0x7f8e0c076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:38.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.620+0000 7f8e36000640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8e0c073bd0 0x7f8e0c076090 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009ad0 tx=0x7f8e200091e0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.620+0000 7f8e36000640 1 -- 172.21.15.66:0/50108695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e300ccd20 msgr2=0x7f8e300c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:38.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.620+0000 7f8e36000640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e300ccd20 0x7f8e300c6810 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800c460 tx=0x7f8e1800c920 comp rx=0 tx=0).stop 2024-09-10T22:10:38.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.620+0000 7f8e36000640 1 -- 172.21.15.66:0/50108695 shutdown_connections 2024-09-10T22:10:38.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.620+0000 7f8e36000640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8e0c073bd0 0x7f8e0c076090 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.620+0000 7f8e36000640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e3015f1c0 0x7f8e300afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.620+0000 7f8e36000640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e300ccd20 0x7f8e300c6810 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.620+0000 7f8e36000640 1 --2- 172.21.15.66:0/50108695 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e30071cc0 0x7f8e3014e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.620+0000 7f8e36000640 1 -- 172.21.15.66:0/50108695 >> 172.21.15.66:0/50108695 conn(0x7f8e300d1c60 msgr2=0x7f8e301581b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:38.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.620+0000 7f8e36000640 1 -- 172.21.15.66:0/50108695 shutdown_connections 2024-09-10T22:10:38.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.620+0000 7f8e36000640 1 -- 172.21.15.66:0/50108695 wait complete. 2024-09-10T22:10:38.642 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:10:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.784+0000 7f2197200640 1 -- 172.21.15.66:0/2204442846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21740039f0 msgr2=0x7f2174003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.784+0000 7f2197200640 1 --2- 172.21.15.66:0/2204442846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21740039f0 0x7f2174003dd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2180009b60 tx=0x7f218001d270 comp rx=0 tx=0).stop 2024-09-10T22:10:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.784+0000 7f2195200640 1 -- 172.21.15.66:0/2204442846 reap_dead start 2024-09-10T22:10:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.784+0000 7f2197200640 1 -- 172.21.15.66:0/2204442846 shutdown_connections 2024-09-10T22:10:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.784+0000 7f2197200640 1 -- 172.21.15.66:0/2204442846 >> 172.21.15.66:0/2204442846 conn(0x7f21900d1c60 msgr2=0x7f21900d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.784+0000 7f2197200640 1 -- 172.21.15.66:0/2204442846 shutdown_connections 2024-09-10T22:10:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.784+0000 7f2197200640 1 -- 172.21.15.66:0/2204442846 wait complete. 2024-09-10T22:10:38.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f2197200640 1 Processor -- start 2024-09-10T22:10:38.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f2197200640 1 -- start start 2024-09-10T22:10:38.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f2197200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21740039f0 0x7f2190158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:38.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f2197200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2190159220 0x7f219015e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f2197200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f219015a6c0 0x7f219015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f2197200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2190169e40 con 0x7f21740039f0 2024-09-10T22:10:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f2197200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2190169cc0 con 0x7f2190159220 2024-09-10T22:10:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f2197200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2190169fc0 con 0x7f219015a6c0 2024-09-10T22:10:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f218fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21740039f0 0x7f2190158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f218fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21740039f0 0x7f2190158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54816/0 (socket says 172.21.15.66:54816) 2024-09-10T22:10:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f218fe00640 1 -- 172.21.15.66:0/765175906 learned_addr learned my addr 172.21.15.66:0/765175906 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f2195200640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f219015a6c0 0x7f219015eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f2187e00640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2190159220 0x7f219015e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f218fe00640 1 -- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f219015a6c0 msgr2=0x7f219015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f218fe00640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f219015a6c0 0x7f219015eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f218fe00640 1 -- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2190159220 msgr2=0x7f219015e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f218fe00640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2190159220 0x7f219015e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f218fe00640 1 -- 172.21.15.66:0/765175906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2190161270 con 0x7f21740039f0 2024-09-10T22:10:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f218fe00640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21740039f0 0x7f2190158b80 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f218001cce0 tx=0x7f2180005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f218cc00640 1 -- 172.21.15.66:0/765175906 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21800b1040 con 0x7f21740039f0 2024-09-10T22:10:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f218cc00640 1 -- 172.21.15.66:0/765175906 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21800bb070 con 0x7f21740039f0 2024-09-10T22:10:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f218cc00640 1 -- 172.21.15.66:0/765175906 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180004290 con 0x7f21740039f0 2024-09-10T22:10:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f2197200640 1 -- 172.21.15.66:0/765175906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2190161510 con 0x7f21740039f0 2024-09-10T22:10:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.788+0000 7f2197200640 1 -- 172.21.15.66:0/765175906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21900ca150 con 0x7f21740039f0 2024-09-10T22:10:38.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.792+0000 7f2197200640 1 -- 172.21.15.66:0/765175906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f219006dd30 con 0x7f21740039f0 2024-09-10T22:10:38.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.792+0000 7f218cc00640 1 -- 172.21.15.66:0/765175906 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f21800046d0 con 0x7f21740039f0 2024-09-10T22:10:38.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.792+0000 7f218cc00640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f216c073bd0 0x7f216c076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:38.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.792+0000 7f218cc00640 1 -- 172.21.15.66:0/765175906 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5765+0+0 (secure 0 0 0) 0x7f218012cd80 con 0x7f21740039f0 2024-09-10T22:10:38.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.792+0000 7f2187e00640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f216c073bd0 0x7f216c076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:38.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.792+0000 7f2187e00640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f216c073bd0 0x7f216c076090 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f21901fad00 tx=0x7f2178001e80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:38.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.796+0000 7f218cc00640 1 -- 172.21.15.66:0/765175906 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2180016f80 con 0x7f21740039f0 2024-09-10T22:10:38.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.972+0000 7f2197200640 1 -- 172.21.15.66:0/765175906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21900cbef0 con 0x7f21740039f0 2024-09-10T22:10:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.972+0000 7f218cc00640 1 -- 172.21.15.66:0/765175906 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16697 (secure 0 0 0) 0x7f21800f9860 con 0x7f21740039f0 2024-09-10T22:10:38.977 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:38.977 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":99,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:10:37.643980+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:10:35.358226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T22:10:36.608313+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:10:38.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.976+0000 7f2197200640 1 -- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f216c073bd0 msgr2=0x7f216c076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.976+0000 7f2197200640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f216c073bd0 0x7f216c076090 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f21901fad00 tx=0x7f2178001e80 comp rx=0 tx=0).stop 2024-09-10T22:10:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.976+0000 7f2197200640 1 -- 172.21.15.66:0/765175906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21740039f0 msgr2=0x7f2190158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.976+0000 7f2197200640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21740039f0 0x7f2190158b80 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f218001cce0 tx=0x7f2180005590 comp rx=0 tx=0).stop 2024-09-10T22:10:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.976+0000 7f2197200640 1 -- 172.21.15.66:0/765175906 shutdown_connections 2024-09-10T22:10:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.976+0000 7f2197200640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f216c073bd0 0x7f216c076090 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.976+0000 7f2197200640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f219015a6c0 0x7f219015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.976+0000 7f2197200640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2190159220 0x7f219015e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.976+0000 7f2197200640 1 --2- 172.21.15.66:0/765175906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21740039f0 0x7f2190158b80 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.976+0000 7f2197200640 1 -- 172.21.15.66:0/765175906 >> 172.21.15.66:0/765175906 conn(0x7f21900d1c60 msgr2=0x7f219015d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.976+0000 7f2197200640 1 -- 172.21.15.66:0/765175906 shutdown_connections 2024-09-10T22:10:38.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:38.976+0000 7f2197200640 1 -- 172.21.15.66:0/765175906 wait complete. 2024-09-10T22:10:38.999 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:10:39.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.144+0000 7f9d28600640 1 -- 172.21.15.66:0/3790279833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d200af970 msgr2=0x7f9d200d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:39.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.144+0000 7f9d28600640 1 --2- 172.21.15.66:0/3790279833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d200af970 0x7f9d200d0090 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009c80 tx=0x7f9d1801cda0 comp rx=0 tx=0).stop 2024-09-10T22:10:39.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.144+0000 7f9d28600640 1 -- 172.21.15.66:0/3790279833 shutdown_connections 2024-09-10T22:10:39.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.144+0000 7f9d28600640 1 --2- 172.21.15.66:0/3790279833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d200af970 0x7f9d200d0090 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:39.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.144+0000 7f9d28600640 1 --2- 172.21.15.66:0/3790279833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d2015f340 0x7f9d200cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:39.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.144+0000 7f9d28600640 1 -- 172.21.15.66:0/3790279833 >> 172.21.15.66:0/3790279833 conn(0x7f9d200d1c50 msgr2=0x7f9d200d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:39.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.144+0000 7f9d28600640 1 -- 172.21.15.66:0/3790279833 shutdown_connections 2024-09-10T22:10:39.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.144+0000 7f9d28600640 1 -- 172.21.15.66:0/3790279833 wait complete. 2024-09-10T22:10:39.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d28600640 1 Processor -- start 2024-09-10T22:10:39.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d28600640 1 -- start start 2024-09-10T22:10:39.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d28600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d200af970 0x7f9d200ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:39.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d28600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d200b2bf0 0x7f9d20158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:39.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d28600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d2015f340 0x7f9d2014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:39.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d28600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d20169190 con 0x7f9d200b2bf0 2024-09-10T22:10:39.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d28600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9d20169010 con 0x7f9d2015f340 2024-09-10T22:10:39.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d28600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9d20169310 con 0x7f9d200af970 2024-09-10T22:10:39.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d25200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d200b2bf0 0x7f9d20158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:39.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d25200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d200b2bf0 0x7f9d20158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54840/0 (socket says 172.21.15.66:54840) 2024-09-10T22:10:39.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d25200640 1 -- 172.21.15.66:0/716436162 learned_addr learned my addr 172.21.15.66:0/716436162 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:39.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d26600640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d2015f340 0x7f9d2014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:39.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d25c00640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d200af970 0x7f9d200ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:39.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d25200640 1 -- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d200af970 msgr2=0x7f9d200ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:39.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d25200640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d200af970 0x7f9d200ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:39.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d25200640 1 -- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d2015f340 msgr2=0x7f9d2014fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:39.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d25200640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d2015f340 0x7f9d2014fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:39.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d25200640 1 -- 172.21.15.66:0/716436162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d20150650 con 0x7f9d200b2bf0 2024-09-10T22:10:39.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d25200640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d200b2bf0 0x7f9d20158020 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00c400 tx=0x7f9d0c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:39.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d1d600640 1 -- 172.21.15.66:0/716436162 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c017070 con 0x7f9d200b2bf0 2024-09-10T22:10:39.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d1d600640 1 -- 172.21.15.66:0/716436162 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d0c0053e0 con 0x7f9d200b2bf0 2024-09-10T22:10:39.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d1d600640 1 -- 172.21.15.66:0/716436162 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c003860 con 0x7f9d200b2bf0 2024-09-10T22:10:39.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d28600640 1 -- 172.21.15.66:0/716436162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d200cda60 con 0x7f9d200b2bf0 2024-09-10T22:10:39.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d28600640 1 -- 172.21.15.66:0/716436162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d200ce020 con 0x7f9d200b2bf0 2024-09-10T22:10:39.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.148+0000 7f9d28600640 1 -- 172.21.15.66:0/716436162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d2015fac0 con 0x7f9d200b2bf0 2024-09-10T22:10:39.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.152+0000 7f9d1d600640 1 -- 172.21.15.66:0/716436162 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f9d0c003a00 con 0x7f9d200b2bf0 2024-09-10T22:10:39.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.152+0000 7f9d1d600640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d00073bd0 0x7f9d00076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:39.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.152+0000 7f9d1d600640 1 -- 172.21.15.66:0/716436162 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5765+0+0 (secure 0 0 0) 0x7f9d0c0919f0 con 0x7f9d200b2bf0 2024-09-10T22:10:39.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.152+0000 7f9d25c00640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d00073bd0 0x7f9d00076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:39.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.152+0000 7f9d25c00640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d00073bd0 0x7f9d00076090 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009b40 tx=0x7f9d140091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:39.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.156+0000 7f9d1d600640 1 -- 172.21.15.66:0/716436162 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9d0c001130 con 0x7f9d200b2bf0 2024-09-10T22:10:39.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:39.368+0000 7f9d28600640 1 -- 172.21.15.66:0/716436162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9d2015a0c0 con 0x7f9d200b2bf0 2024-09-10T22:10:40.218 INFO:tasks.ceph.mds.d.smithi094.stderr:2024-09-10T22:10:40.218+0000 7f2996600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:10:40.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.248+0000 7f9d1d600640 1 -- 172.21.15.66:0/716436162 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) ==== 118+0+0 (secure 0 0 0) 0x7f9d0c091e90 con 0x7f9d200b2bf0 2024-09-10T22:10:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.252+0000 7f9d28600640 1 -- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d00073bd0 msgr2=0x7f9d00076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.252+0000 7f9d28600640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d00073bd0 0x7f9d00076090 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009b40 tx=0x7f9d140091e0 comp rx=0 tx=0).stop 2024-09-10T22:10:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.252+0000 7f9d28600640 1 -- 172.21.15.66:0/716436162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d200b2bf0 msgr2=0x7f9d20158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.252+0000 7f9d28600640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d200b2bf0 0x7f9d20158020 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00c400 tx=0x7f9d0c00c8c0 comp rx=0 tx=0).stop 2024-09-10T22:10:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.252+0000 7f9d28600640 1 -- 172.21.15.66:0/716436162 shutdown_connections 2024-09-10T22:10:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.252+0000 7f9d28600640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9d00073bd0 0x7f9d00076090 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.252+0000 7f9d28600640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d2015f340 0x7f9d2014fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.252+0000 7f9d28600640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d200b2bf0 0x7f9d20158020 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.252+0000 7f9d28600640 1 --2- 172.21.15.66:0/716436162 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d200af970 0x7f9d200ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.252+0000 7f9d28600640 1 -- 172.21.15.66:0/716436162 >> 172.21.15.66:0/716436162 conn(0x7f9d200d1c50 msgr2=0x7f9d200cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.252+0000 7f9d28600640 1 -- 172.21.15.66:0/716436162 shutdown_connections 2024-09-10T22:10:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.252+0000 7f9d28600640 1 -- 172.21.15.66:0/716436162 wait complete. 2024-09-10T22:10:40.277 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:10:40.277 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:10:40.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.424+0000 7fb226200640 1 -- 172.21.15.66:0/2037835843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb220071700 msgr2=0x7fb220070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:40.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.424+0000 7fb226200640 1 --2- 172.21.15.66:0/2037835843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb220071700 0x7fb220070ea0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009870 tx=0x7fb20c01c740 comp rx=0 tx=0).stop 2024-09-10T22:10:40.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.424+0000 7fb226200640 1 -- 172.21.15.66:0/2037835843 shutdown_connections 2024-09-10T22:10:40.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.424+0000 7fb226200640 1 --2- 172.21.15.66:0/2037835843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb220071700 0x7fb220070ea0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:40.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.424+0000 7fb226200640 1 -- 172.21.15.66:0/2037835843 >> 172.21.15.66:0/2037835843 conn(0x7fb2200d1d00 msgr2=0x7fb2200d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:40.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.424+0000 7fb226200640 1 -- 172.21.15.66:0/2037835843 shutdown_connections 2024-09-10T22:10:40.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.424+0000 7fb226200640 1 -- 172.21.15.66:0/2037835843 wait complete. 2024-09-10T22:10:40.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.424+0000 7fb226200640 1 Processor -- start 2024-09-10T22:10:40.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.424+0000 7fb226200640 1 -- start start 2024-09-10T22:10:40.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb226200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb220070ac0 0x7fb2200c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:40.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb226200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb220071700 0x7fb220159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:40.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb226200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2200b0ef0 0x7fb2201faa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:40.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb226200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb220169e20 con 0x7fb220071700 2024-09-10T22:10:40.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb226200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb220169ca0 con 0x7fb220070ac0 2024-09-10T22:10:40.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb226200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb220169fa0 con 0x7fb2200b0ef0 2024-09-10T22:10:40.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb21ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb220071700 0x7fb220159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb21ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb220071700 0x7fb220159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54844/0 (socket says 172.21.15.66:54844) 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb21ea00640 1 -- 172.21.15.66:0/2811216143 learned_addr learned my addr 172.21.15.66:0/2811216143 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb21fe00640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2200b0ef0 0x7fb2201faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb21f400640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb220070ac0 0x7fb2200c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb21ea00640 1 -- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2200b0ef0 msgr2=0x7fb2201faa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb21ea00640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2200b0ef0 0x7fb2201faa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb21ea00640 1 -- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb220070ac0 msgr2=0x7fb2200c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb21ea00640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb220070ac0 0x7fb2200c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb21ea00640 1 -- 172.21.15.66:0/2811216143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2200b6ad0 con 0x7fb220071700 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb21ea00640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb220071700 0x7fb220159950 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb20c019350 tx=0x7fb20c0a64a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb217400640 1 -- 172.21.15.66:0/2811216143 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c005250 con 0x7fb220071700 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb217400640 1 -- 172.21.15.66:0/2811216143 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb20c0053f0 con 0x7fb220071700 2024-09-10T22:10:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb217400640 1 -- 172.21.15.66:0/2811216143 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c0046d0 con 0x7fb220071700 2024-09-10T22:10:40.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb226200640 1 -- 172.21.15.66:0/2811216143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2200b6d70 con 0x7fb220071700 2024-09-10T22:10:40.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb226200640 1 -- 172.21.15.66:0/2811216143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2200b7330 con 0x7fb220071700 2024-09-10T22:10:40.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.428+0000 7fb226200640 1 -- 172.21.15.66:0/2811216143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb220070ea0 con 0x7fb220071700 2024-09-10T22:10:40.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.432+0000 7fb217400640 1 -- 172.21.15.66:0/2811216143 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fb20c0a6690 con 0x7fb220071700 2024-09-10T22:10:40.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.432+0000 7fb217400640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb200073bd0 0x7fb200076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:40.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.432+0000 7fb217400640 1 -- 172.21.15.66:0/2811216143 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fb20c12c510 con 0x7fb220071700 2024-09-10T22:10:40.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.432+0000 7fb21f400640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb200073bd0 0x7fb200076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:40.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.432+0000 7fb21f400640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb200073bd0 0x7fb200076090 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb210001910 tx=0x7fb210001420 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:40.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.436+0000 7fb217400640 1 -- 172.21.15.66:0/2811216143 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb20c0ac030 con 0x7fb220071700 2024-09-10T22:10:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:40.644+0000 7fb226200640 1 -- 172.21.15.66:0/2811216143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb2200b1440 con 0x7fb220071700 2024-09-10T22:10:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb217400640 1 -- 172.21.15.66:0/2811216143 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) ==== 122+0+0 (secure 0 0 0) 0x7fb20c016450 con 0x7fb220071700 2024-09-10T22:10:41.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb226200640 1 -- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb200073bd0 msgr2=0x7fb200076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:41.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb226200640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb200073bd0 0x7fb200076090 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb210001910 tx=0x7fb210001420 comp rx=0 tx=0).stop 2024-09-10T22:10:41.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb226200640 1 -- 172.21.15.66:0/2811216143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb220071700 msgr2=0x7fb220159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:41.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb226200640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb220071700 0x7fb220159950 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb20c019350 tx=0x7fb20c0a64a0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb226200640 1 -- 172.21.15.66:0/2811216143 shutdown_connections 2024-09-10T22:10:41.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb226200640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb200073bd0 0x7fb200076090 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb226200640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2200b0ef0 0x7fb2201faa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb226200640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb220071700 0x7fb220159950 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb226200640 1 --2- 172.21.15.66:0/2811216143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb220070ac0 0x7fb2200c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb226200640 1 -- 172.21.15.66:0/2811216143 >> 172.21.15.66:0/2811216143 conn(0x7fb2200d1d00 msgr2=0x7fb22015e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:41.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb226200640 1 -- 172.21.15.66:0/2811216143 shutdown_connections 2024-09-10T22:10:41.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.256+0000 7fb226200640 1 -- 172.21.15.66:0/2811216143 wait complete. 2024-09-10T22:10:41.282 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:10:41.283 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:10:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 -- 172.21.15.66:0/2464669278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad8c004d50 msgr2=0x7fad8c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 --2- 172.21.15.66:0/2464669278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad8c004d50 0x7fad8c0051d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00c640 tx=0x7fad9c01d270 comp rx=0 tx=0).stop 2024-09-10T22:10:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadaa600640 1 -- 172.21.15.66:0/2464669278 reap_dead start 2024-09-10T22:10:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 -- 172.21.15.66:0/2464669278 shutdown_connections 2024-09-10T22:10:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 -- 172.21.15.66:0/2464669278 >> 172.21.15.66:0/2464669278 conn(0x7fada40d1d00 msgr2=0x7fada40d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 -- 172.21.15.66:0/2464669278 shutdown_connections 2024-09-10T22:10:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 -- 172.21.15.66:0/2464669278 wait complete. 2024-09-10T22:10:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 Processor -- start 2024-09-10T22:10:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 -- start start 2024-09-10T22:10:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c004d50 0x7fada41f9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fada4166250 0x7fada40c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fada41585b0 0x7fada4158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fada4166dd0 con 0x7fada4166250 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fada4166c50 con 0x7fad8c004d50 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.428+0000 7fadac600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fada4166f50 con 0x7fada41585b0 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada9200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fada4166250 0x7fada40c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada9200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fada4166250 0x7fada40c59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54862/0 (socket says 172.21.15.66:54862) 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada9200640 1 -- 172.21.15.66:0/1279323400 learned_addr learned my addr 172.21.15.66:0/1279323400 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada9c00640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c004d50 0x7fada41f9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fadaa600640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fada41585b0 0x7fada4158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada9200640 1 -- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fada41585b0 msgr2=0x7fada4158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada9200640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fada41585b0 0x7fada4158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada9200640 1 -- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c004d50 msgr2=0x7fada41f9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada9200640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c004d50 0x7fada41f9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada9200640 1 -- 172.21.15.66:0/1279323400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada40b4ce0 con 0x7fada4166250 2024-09-10T22:10:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada9c00640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c004d50 0x7fada41f9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:41.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada9200640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fada4166250 0x7fada40c59e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fad90011b10 tx=0x7fad90009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:41.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada1600640 1 -- 172.21.15.66:0/1279323400 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9001e070 con 0x7fada4166250 2024-09-10T22:10:41.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada1600640 1 -- 172.21.15.66:0/1279323400 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad90004050 con 0x7fada4166250 2024-09-10T22:10:41.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fadac600640 1 -- 172.21.15.66:0/1279323400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada40c6160 con 0x7fada4166250 2024-09-10T22:10:41.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fada1600640 1 -- 172.21.15.66:0/1279323400 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9000fea0 con 0x7fada4166250 2024-09-10T22:10:41.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fadac600640 1 -- 172.21.15.66:0/1279323400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada41f94d0 con 0x7fada4166250 2024-09-10T22:10:41.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.432+0000 7fadac600640 1 -- 172.21.15.66:0/1279323400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada40d0120 con 0x7fada4166250 2024-09-10T22:10:41.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.436+0000 7fada1600640 1 -- 172.21.15.66:0/1279323400 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fad9002b080 con 0x7fada4166250 2024-09-10T22:10:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.436+0000 7fada1600640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fad84073c00 0x7fad840760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.436+0000 7fada1600640 1 -- 172.21.15.66:0/1279323400 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fad90093670 con 0x7fada4166250 2024-09-10T22:10:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.436+0000 7fada9c00640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fad84073c00 0x7fad840760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:41.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.436+0000 7fada9c00640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fad84073c00 0x7fad840760c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fada40c87a0 tx=0x7fad980091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:41.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.440+0000 7fada1600640 1 -- 172.21.15.66:0/1279323400 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fad9005fd40 con 0x7fada4166250 2024-09-10T22:10:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.648+0000 7fadac600640 1 -- 172.21.15.66:0/1279323400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fada4071050 con 0x7fada4166250 2024-09-10T22:10:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.648+0000 7fada1600640 1 -- 172.21.15.66:0/1279323400 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5222 (secure 0 0 0) 0x7fad90064770 con 0x7fada4166250 2024-09-10T22:10:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:41.653 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":66,"btime":"2024-09-10T22:10:41:252812+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3262462745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3262462745},{"type":"v1","addr":"172.21.15.66:6834","nonce":3262462745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/273414872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":273414872},{"type":"v1","addr":"172.21.15.94:6835","nonce":273414872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7246,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3486474426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3486474426},{"type":"v1","addr":"172.21.15.66:6835","nonce":3486474426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:10:37.643939+0000","modified":"2024-09-10T22:10:40.648893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7249":{"gid":7249,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/1920075377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1920075377},{"type":"v1","addr":"172.21.15.94:6837","nonce":1920075377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7249,"qdb_cluster":[7249]},"id":7}]} 2024-09-10T22:10:41.653 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 66 2024-09-10T22:10:41.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.652+0000 7fadac600640 1 -- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fad84073c00 msgr2=0x7fad840760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:41.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.652+0000 7fadac600640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fad84073c00 0x7fad840760c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fada40c87a0 tx=0x7fad980091e0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.652+0000 7fadac600640 1 -- 172.21.15.66:0/1279323400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fada4166250 msgr2=0x7fada40c59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:41.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.652+0000 7fadac600640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fada4166250 0x7fada40c59e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fad90011b10 tx=0x7fad90009e90 comp rx=0 tx=0).stop 2024-09-10T22:10:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.652+0000 7fadac600640 1 -- 172.21.15.66:0/1279323400 shutdown_connections 2024-09-10T22:10:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.652+0000 7fadac600640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fad84073c00 0x7fad840760c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.652+0000 7fadac600640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fada41585b0 0x7fada4158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.652+0000 7fadac600640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fada4166250 0x7fada40c59e0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.652+0000 7fadac600640 1 --2- 172.21.15.66:0/1279323400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c004d50 0x7fada41f9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.652+0000 7fadac600640 1 -- 172.21.15.66:0/1279323400 >> 172.21.15.66:0/1279323400 conn(0x7fada40d1d00 msgr2=0x7fada4070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.652+0000 7fadac600640 1 -- 172.21.15.66:0/1279323400 shutdown_connections 2024-09-10T22:10:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.652+0000 7fadac600640 1 -- 172.21.15.66:0/1279323400 wait complete. 2024-09-10T22:10:41.674 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:10:41.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.816+0000 7f2494600640 1 -- 172.21.15.66:0/1191685380 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f248c15cc80 msgr2=0x7f248c0cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:41.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.816+0000 7f2494600640 1 --2- 172.21.15.66:0/1191685380 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f248c15cc80 0x7f248c0cd930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2484009c80 tx=0x7f248401cd10 comp rx=0 tx=0).stop 2024-09-10T22:10:41.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.816+0000 7f2494600640 1 -- 172.21.15.66:0/1191685380 shutdown_connections 2024-09-10T22:10:41.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.816+0000 7f2494600640 1 --2- 172.21.15.66:0/1191685380 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f248c15cc80 0x7f248c0cd930 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.816+0000 7f2494600640 1 --2- 172.21.15.66:0/1191685380 >> v2:172.21.15.94:3300/0 conn(0x7f248c0aee20 0x7f248c0ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.816+0000 7f2494600640 1 -- 172.21.15.66:0/1191685380 >> 172.21.15.66:0/1191685380 conn(0x7f248c0cfa30 msgr2=0x7f248c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:41.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.816+0000 7f2494600640 1 -- 172.21.15.66:0/1191685380 shutdown_connections 2024-09-10T22:10:41.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.816+0000 7f2494600640 1 -- 172.21.15.66:0/1191685380 wait complete. 2024-09-10T22:10:41.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2494600640 1 Processor -- start 2024-09-10T22:10:41.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2494600640 1 -- start start 2024-09-10T22:10:41.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2494600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f248c0aee20 0x7f248c1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:41.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2494600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f248c159900 0x7f248c1f6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:41.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2494600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248c15cc80 0x7f248c14df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:41.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2494600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f248c168f70 con 0x7f248c0aee20 2024-09-10T22:10:41.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2494600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f248c168df0 con 0x7f248c15cc80 2024-09-10T22:10:41.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2494600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f248c1690f0 con 0x7f248c159900 2024-09-10T22:10:41.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2491c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f248c0aee20 0x7f248c1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:41.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2491c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f248c0aee20 0x7f248c1f5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54872/0 (socket says 172.21.15.66:54872) 2024-09-10T22:10:41.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2491c00640 1 -- 172.21.15.66:0/425957813 learned_addr learned my addr 172.21.15.66:0/425957813 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:41.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2491200640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f248c159900 0x7f248c1f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:41.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2492600640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248c15cc80 0x7f248c14df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2491c00640 1 -- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f248c159900 msgr2=0x7f248c1f6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2491c00640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f248c159900 0x7f248c1f6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2491c00640 1 -- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248c15cc80 msgr2=0x7f248c14df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2491c00640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248c15cc80 0x7f248c14df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2491c00640 1 -- 172.21.15.66:0/425957813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c15e890 con 0x7f248c0aee20 2024-09-10T22:10:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2491c00640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f248c0aee20 0x7f248c1f5e40 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f248000c500 tx=0x7f248000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2489600640 1 -- 172.21.15.66:0/425957813 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480017070 con 0x7f248c0aee20 2024-09-10T22:10:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2489600640 1 -- 172.21.15.66:0/425957813 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24800053c0 con 0x7f248c0aee20 2024-09-10T22:10:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2489600640 1 -- 172.21.15.66:0/425957813 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480012890 con 0x7f248c0aee20 2024-09-10T22:10:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2494600640 1 -- 172.21.15.66:0/425957813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c15eb70 con 0x7f248c0aee20 2024-09-10T22:10:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.820+0000 7f2494600640 1 -- 172.21.15.66:0/425957813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c073ee0 con 0x7f248c0aee20 2024-09-10T22:10:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.824+0000 7f2494600640 1 -- 172.21.15.66:0/425957813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248c0ca080 con 0x7f248c0aee20 2024-09-10T22:10:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.824+0000 7f2489600640 1 -- 172.21.15.66:0/425957813 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f2480003600 con 0x7f248c0aee20 2024-09-10T22:10:41.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.824+0000 7f2489600640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f246c073bd0 0x7f246c076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:41.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.824+0000 7f2489600640 1 -- 172.21.15.66:0/425957813 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f2480091990 con 0x7f248c0aee20 2024-09-10T22:10:41.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.824+0000 7f2491200640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f246c073bd0 0x7f246c076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:41.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.824+0000 7f2491200640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f246c073bd0 0x7f246c076090 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2478009600 tx=0x7f24780091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:41.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:41.828+0000 7f2489600640 1 -- 172.21.15.66:0/425957813 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2480091e30 con 0x7f248c0aee20 2024-09-10T22:10:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.004+0000 7f2494600640 1 -- 172.21.15.66:0/425957813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f248c070470 con 0x7f248c0aee20 2024-09-10T22:10:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.004+0000 7f2489600640 1 -- 172.21.15.66:0/425957813 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16686 (secure 0 0 0) 0x7f248005dfe0 con 0x7f248c0aee20 2024-09-10T22:10:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:42.008 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":100,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:10:39.240066+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:10:35.358226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T22:10:36.608313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:10:42.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.004+0000 7f2494600640 1 -- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f246c073bd0 msgr2=0x7f246c076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.004+0000 7f2494600640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f246c073bd0 0x7f246c076090 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2478009600 tx=0x7f24780091e0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.004+0000 7f2494600640 1 -- 172.21.15.66:0/425957813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f248c0aee20 msgr2=0x7f248c1f5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.004+0000 7f2494600640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f248c0aee20 0x7f248c1f5e40 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f248000c500 tx=0x7f248000c9c0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.004+0000 7f2494600640 1 -- 172.21.15.66:0/425957813 shutdown_connections 2024-09-10T22:10:42.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.004+0000 7f2494600640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f246c073bd0 0x7f246c076090 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.004+0000 7f2494600640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248c15cc80 0x7f248c14df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.004+0000 7f2494600640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f248c159900 0x7f248c1f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.004+0000 7f2494600640 1 --2- 172.21.15.66:0/425957813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f248c0aee20 0x7f248c1f5e40 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.004+0000 7f2494600640 1 -- 172.21.15.66:0/425957813 >> 172.21.15.66:0/425957813 conn(0x7f248c0cfa30 msgr2=0x7f248c0af990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:42.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.008+0000 7f2494600640 1 -- 172.21.15.66:0/425957813 shutdown_connections 2024-09-10T22:10:42.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.008+0000 7f2494600640 1 -- 172.21.15.66:0/425957813 wait complete. 2024-09-10T22:10:42.030 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:10:42.030 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:10:42.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.172+0000 7f4d7e200640 1 -- 172.21.15.66:0/687159676 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d780d06b0 msgr2=0x7f4d780ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.172+0000 7f4d7e200640 1 --2- 172.21.15.66:0/687159676 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d780d06b0 0x7f4d780ce170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009870 tx=0x7f4d6001c7d0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.172+0000 7f4d7e200640 1 -- 172.21.15.66:0/687159676 shutdown_connections 2024-09-10T22:10:42.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.172+0000 7f4d7e200640 1 --2- 172.21.15.66:0/687159676 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d780d06b0 0x7f4d780ce170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.172+0000 7f4d7e200640 1 -- 172.21.15.66:0/687159676 >> 172.21.15.66:0/687159676 conn(0x7f4d780d1c60 msgr2=0x7f4d780d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:42.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.172+0000 7f4d7e200640 1 -- 172.21.15.66:0/687159676 shutdown_connections 2024-09-10T22:10:42.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.172+0000 7f4d7e200640 1 -- 172.21.15.66:0/687159676 wait complete. 2024-09-10T22:10:42.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d7e200640 1 Processor -- start 2024-09-10T22:10:42.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d7e200640 1 -- start start 2024-09-10T22:10:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d7e200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d780afc20 0x7f4d780b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d7e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d780d06b0 0x7f4d7815b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d7e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7815bd40 0x7f4d7806eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d7e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7816a110 con 0x7f4d780d06b0 2024-09-10T22:10:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d7e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78169f90 con 0x7f4d7815bd40 2024-09-10T22:10:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d7e200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4d7816a290 con 0x7f4d780afc20 2024-09-10T22:10:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d6ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d780d06b0 0x7f4d7815b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d6ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d780d06b0 0x7f4d7815b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54878/0 (socket says 172.21.15.66:54878) 2024-09-10T22:10:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d6ea00640 1 -- 172.21.15.66:0/2397877405 learned_addr learned my addr 172.21.15.66:0/2397877405 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d77400640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d780afc20 0x7f4d780b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d77e00640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7815bd40 0x7f4d7806eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d6ea00640 1 -- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d780afc20 msgr2=0x7f4d780b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d6ea00640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d780afc20 0x7f4d780b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d6ea00640 1 -- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7815bd40 msgr2=0x7f4d7806eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d6ea00640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7815bd40 0x7f4d7806eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d6ea00640 1 -- 172.21.15.66:0/2397877405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d78152e60 con 0x7f4d780d06b0 2024-09-10T22:10:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d77e00640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7815bd40 0x7f4d7806eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d6ea00640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d780d06b0 0x7f4d7815b1d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4d600193e0 tx=0x7f4d60005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d6fe00640 1 -- 172.21.15.66:0/2397877405 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d600b2040 con 0x7f4d780d06b0 2024-09-10T22:10:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d6fe00640 1 -- 172.21.15.66:0/2397877405 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d60004040 con 0x7f4d780d06b0 2024-09-10T22:10:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d6fe00640 1 -- 172.21.15.66:0/2397877405 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60004430 con 0x7f4d780d06b0 2024-09-10T22:10:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d7e200640 1 -- 172.21.15.66:0/2397877405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7806dd30 con 0x7f4d780d06b0 2024-09-10T22:10:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d7e200640 1 -- 172.21.15.66:0/2397877405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7806e2f0 con 0x7f4d780d06b0 2024-09-10T22:10:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.176+0000 7f4d7e200640 1 -- 172.21.15.66:0/2397877405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d780cffa0 con 0x7f4d780d06b0 2024-09-10T22:10:42.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.180+0000 7f4d6fe00640 1 -- 172.21.15.66:0/2397877405 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f4d600b5070 con 0x7f4d780d06b0 2024-09-10T22:10:42.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.180+0000 7f4d6fe00640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4d54073bd0 0x7f4d54076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:42.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.180+0000 7f4d6fe00640 1 -- 172.21.15.66:0/2397877405 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f4d6012b930 con 0x7f4d780d06b0 2024-09-10T22:10:42.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.180+0000 7f4d77400640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4d54073bd0 0x7f4d54076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:42.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.180+0000 7f4d77400640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4d54073bd0 0x7f4d54076090 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4d680014b0 tx=0x7f4d68009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:42.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.184+0000 7f4d6fe00640 1 -- 172.21.15.66:0/2397877405 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4d600c4020 con 0x7f4d780d06b0 2024-09-10T22:10:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.356+0000 7f4d7e200640 1 -- 172.21.15.66:0/2397877405 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d780b5530 con 0x7f4d54073bd0 2024-09-10T22:10:42.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.360+0000 7f4d6fe00640 1 -- 172.21.15.66:0/2397877405 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243590 (secure 0 0 0) 0x7f4d780b5530 con 0x7f4d54073bd0 2024-09-10T22:10:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:42.368 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:10:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.364+0000 7f4d7e200640 1 -- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4d54073bd0 msgr2=0x7f4d54076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.364+0000 7f4d7e200640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4d54073bd0 0x7f4d54076090 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4d680014b0 tx=0x7f4d68009040 comp rx=0 tx=0).stop 2024-09-10T22:10:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.364+0000 7f4d7e200640 1 -- 172.21.15.66:0/2397877405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d780d06b0 msgr2=0x7f4d7815b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.364+0000 7f4d7e200640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d780d06b0 0x7f4d7815b1d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4d600193e0 tx=0x7f4d60005780 comp rx=0 tx=0).stop 2024-09-10T22:10:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.364+0000 7f4d7e200640 1 -- 172.21.15.66:0/2397877405 shutdown_connections 2024-09-10T22:10:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.364+0000 7f4d7e200640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4d54073bd0 0x7f4d54076090 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.364+0000 7f4d7e200640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7815bd40 0x7f4d7806eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.364+0000 7f4d7e200640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d780d06b0 0x7f4d7815b1d0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.364+0000 7f4d7e200640 1 --2- 172.21.15.66:0/2397877405 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d780afc20 0x7f4d780b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.364+0000 7f4d7e200640 1 -- 172.21.15.66:0/2397877405 >> 172.21.15.66:0/2397877405 conn(0x7f4d780d1c60 msgr2=0x7f4d78154240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.364+0000 7f4d7e200640 1 -- 172.21.15.66:0/2397877405 shutdown_connections 2024-09-10T22:10:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.364+0000 7f4d7e200640 1 -- 172.21.15.66:0/2397877405 wait complete. 2024-09-10T22:10:42.388 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-09-10T22:10:40.542312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":975860,"kb_used_data":10472,"kb_used_omap":12,"kb_used_meta":965171,"kb_avail":753998860,"statfs":{"total":773094113280,"available":772094832640,"internally_reserved":0,"allocated":10723328,"data_stored":5278202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":988335668},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.317665"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.237499+0000","last_change":"2024-09-10T22:10:37.237499+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.237499+0000","last_undegraded":"2024-09-10T22:10:37.237499+0000","last_fullsized":"2024-09-10T22:10:37.237499+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.233983+0000","last_change":"2024-09-10T22:10:37.233983+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.233983+0000","last_undegraded":"2024-09-10T22:10:37.233983+0000","last_fullsized":"2024-09-10T22:10:37.233983+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240134+0000","last_change":"2024-09-10T22:10:37.240134+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240134+0000","last_undegraded":"2024-09-10T22:10:37.240134+0000","last_fullsized":"2024-09-10T22:10:37.240134+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.241724+0000","last_change":"2024-09-10T22:10:37.241724+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.241724+0000","last_undegraded":"2024-09-10T22:10:37.241724+0000","last_fullsized":"2024-09-10T22:10:37.241724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240036+0000","last_change":"2024-09-10T22:10:37.240036+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240036+0000","last_undegraded":"2024-09-10T22:10:37.240036+0000","last_fullsized":"2024-09-10T22:10:37.240036+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.236651+0000","last_change":"2024-09-10T22:10:37.236651+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.236651+0000","last_undegraded":"2024-09-10T22:10:37.236651+0000","last_fullsized":"2024-09-10T22:10:37.236651+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.239408+0000","last_change":"2024-09-10T22:10:37.239408+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.239408+0000","last_undegraded":"2024-09-10T22:10:37.239408+0000","last_fullsized":"2024-09-10T22:10:37.239408+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.243036+0000","last_change":"2024-09-10T22:10:37.243036+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.243036+0000","last_undegraded":"2024-09-10T22:10:37.243036+0000","last_fullsized":"2024-09-10T22:10:37.243036+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.235317+0000","last_change":"2024-09-10T22:10:37.235317+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.235317+0000","last_undegraded":"2024-09-10T22:10:37.235317+0000","last_fullsized":"2024-09-10T22:10:37.235317+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240030+0000","last_change":"2024-09-10T22:10:37.240030+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240030+0000","last_undegraded":"2024-09-10T22:10:37.240030+0000","last_fullsized":"2024-09-10T22:10:37.240030+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.234877+0000","last_change":"2024-09-10T22:10:37.234877+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.234877+0000","last_undegraded":"2024-09-10T22:10:37.234877+0000","last_fullsized":"2024-09-10T22:10:37.234877+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.239464+0000","last_change":"2024-09-10T22:10:37.239464+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.239464+0000","last_undegraded":"2024-09-10T22:10:37.239464+0000","last_fullsized":"2024-09-10T22:10:37.239464+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240975+0000","last_change":"2024-09-10T22:10:37.240975+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240975+0000","last_undegraded":"2024-09-10T22:10:37.240975+0000","last_fullsized":"2024-09-10T22:10:37.240975+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.239961+0000","last_change":"2024-09-10T22:10:37.239961+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.239961+0000","last_undegraded":"2024-09-10T22:10:37.239961+0000","last_fullsized":"2024-09-10T22:10:37.239961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.238880+0000","last_change":"2024-09-10T22:10:37.238880+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.238880+0000","last_undegraded":"2024-09-10T22:10:37.238880+0000","last_fullsized":"2024-09-10T22:10:37.238880+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.233970+0000","last_change":"2024-09-10T22:10:37.233970+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.233970+0000","last_undegraded":"2024-09-10T22:10:37.233970+0000","last_fullsized":"2024-09-10T22:10:37.233970+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-10T22:10:36.225016+0000","last_change":"2024-09-10T22:10:36.225016+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.225016+0000","last_undegraded":"2024-09-10T22:10:36.225016+0000","last_fullsized":"2024-09-10T22:10:36.225016+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.251427+0000","last_change":"2024-09-10T22:10:37.248597+0000","last_active":"2024-09-10T22:10:37.251427+0000","last_peered":"2024-09-10T22:10:37.251427+0000","last_clean":"2024-09-10T22:10:37.251427+0000","last_became_active":"2024-09-10T22:10:37.248297+0000","last_became_peered":"2024-09-10T22:10:37.248297+0000","last_unstale":"2024-09-10T22:10:37.251427+0000","last_undegraded":"2024-09-10T22:10:37.251427+0000","last_fullsized":"2024-09-10T22:10:37.251427+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:56.601971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.235593+0000","last_change":"2024-09-10T22:10:37.235593+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.235593+0000","last_undegraded":"2024-09-10T22:10:37.235593+0000","last_fullsized":"2024-09-10T22:10:37.235593+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-10T22:10:36.224629+0000","last_change":"2024-09-10T22:10:36.224629+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.224629+0000","last_undegraded":"2024-09-10T22:10:36.224629+0000","last_fullsized":"2024-09-10T22:10:36.224629+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.235103+0000","last_change":"2024-09-10T22:10:37.235103+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.235103+0000","last_undegraded":"2024-09-10T22:10:37.235103+0000","last_fullsized":"2024-09-10T22:10:37.235103+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-10T22:10:36.225872+0000","last_change":"2024-09-10T22:10:36.225872+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.225872+0000","last_undegraded":"2024-09-10T22:10:36.225872+0000","last_fullsized":"2024-09-10T22:10:36.225872+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.233373+0000","last_change":"2024-09-10T22:10:37.233373+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.233373+0000","last_undegraded":"2024-09-10T22:10:37.233373+0000","last_fullsized":"2024-09-10T22:10:37.233373+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.250388+0000","last_change":"2024-09-10T22:10:37.247963+0000","last_active":"2024-09-10T22:10:37.250388+0000","last_peered":"2024-09-10T22:10:37.250388+0000","last_clean":"2024-09-10T22:10:37.250388+0000","last_became_active":"2024-09-10T22:10:37.246784+0000","last_became_peered":"2024-09-10T22:10:37.246784+0000","last_unstale":"2024-09-10T22:10:37.250388+0000","last_undegraded":"2024-09-10T22:10:37.250388+0000","last_fullsized":"2024-09-10T22:10:37.250388+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:00.178385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.241495+0000","last_change":"2024-09-10T22:10:37.241495+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.241495+0000","last_undegraded":"2024-09-10T22:10:37.241495+0000","last_fullsized":"2024-09-10T22:10:37.241495+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-10T22:10:36.224963+0000","last_change":"2024-09-10T22:10:36.224963+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.224963+0000","last_undegraded":"2024-09-10T22:10:36.224963+0000","last_fullsized":"2024-09-10T22:10:36.224963+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.258484+0000","last_change":"2024-09-10T22:10:37.258021+0000","last_active":"2024-09-10T22:10:37.258484+0000","last_peered":"2024-09-10T22:10:37.258484+0000","last_clean":"2024-09-10T22:10:37.258484+0000","last_became_active":"2024-09-10T22:10:37.257766+0000","last_became_peered":"2024-09-10T22:10:37.257766+0000","last_unstale":"2024-09-10T22:10:37.258484+0000","last_undegraded":"2024-09-10T22:10:37.258484+0000","last_fullsized":"2024-09-10T22:10:37.258484+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.310665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.233855+0000","last_change":"2024-09-10T22:10:37.233855+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.233855+0000","last_undegraded":"2024-09-10T22:10:37.233855+0000","last_fullsized":"2024-09-10T22:10:37.233855+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.250373+0000","last_change":"2024-09-10T22:10:37.247160+0000","last_active":"2024-09-10T22:10:37.250373+0000","last_peered":"2024-09-10T22:10:37.250373+0000","last_clean":"2024-09-10T22:10:37.250373+0000","last_became_active":"2024-09-10T22:10:37.246865+0000","last_became_peered":"2024-09-10T22:10:37.246865+0000","last_unstale":"2024-09-10T22:10:37.250373+0000","last_undegraded":"2024-09-10T22:10:37.250373+0000","last_fullsized":"2024-09-10T22:10:37.250373+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:58.583521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.250016+0000","last_change":"2024-09-10T22:10:37.246423+0000","last_active":"2024-09-10T22:10:37.250016+0000","last_peered":"2024-09-10T22:10:37.250016+0000","last_clean":"2024-09-10T22:10:37.250016+0000","last_became_active":"2024-09-10T22:10:37.246139+0000","last_became_peered":"2024-09-10T22:10:37.246139+0000","last_unstale":"2024-09-10T22:10:37.250016+0000","last_undegraded":"2024-09-10T22:10:37.250016+0000","last_fullsized":"2024-09-10T22:10:37.250016+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:25.467901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.235313+0000","last_change":"2024-09-10T22:10:37.235313+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.235313+0000","last_undegraded":"2024-09-10T22:10:37.235313+0000","last_fullsized":"2024-09-10T22:10:37.235313+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-10T22:10:36.227555+0000","last_change":"2024-09-10T22:10:36.227555+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.227555+0000","last_undegraded":"2024-09-10T22:10:36.227555+0000","last_fullsized":"2024-09-10T22:10:36.227555+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.241194+0000","last_change":"2024-09-10T22:10:37.241194+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.241194+0000","last_undegraded":"2024-09-10T22:10:37.241194+0000","last_fullsized":"2024-09-10T22:10:37.241194+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.253794+0000","last_change":"2024-09-10T22:10:37.252591+0000","last_active":"2024-09-10T22:10:37.253794+0000","last_peered":"2024-09-10T22:10:37.253794+0000","last_clean":"2024-09-10T22:10:37.253794+0000","last_became_active":"2024-09-10T22:10:37.252236+0000","last_became_peered":"2024-09-10T22:10:37.252236+0000","last_unstale":"2024-09-10T22:10:37.253794+0000","last_undegraded":"2024-09-10T22:10:37.253794+0000","last_fullsized":"2024-09-10T22:10:37.253794+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:25.052167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.242629+0000","last_change":"2024-09-10T22:10:37.242629+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.242629+0000","last_undegraded":"2024-09-10T22:10:37.242629+0000","last_fullsized":"2024-09-10T22:10:37.242629+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.246359+0000","last_change":"2024-09-10T22:10:37.245595+0000","last_active":"2024-09-10T22:10:37.246359+0000","last_peered":"2024-09-10T22:10:37.246359+0000","last_clean":"2024-09-10T22:10:37.246359+0000","last_became_active":"2024-09-10T22:10:37.245237+0000","last_became_peered":"2024-09-10T22:10:37.245237+0000","last_unstale":"2024-09-10T22:10:37.246359+0000","last_undegraded":"2024-09-10T22:10:37.246359+0000","last_fullsized":"2024-09-10T22:10:37.246359+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:56.773630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:36.216847+0000","last_change":"2024-09-10T22:10:36.216847+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.216847+0000","last_undegraded":"2024-09-10T22:10:36.216847+0000","last_fullsized":"2024-09-10T22:10:36.216847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.233394+0000","last_change":"2024-09-10T22:10:37.233394+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.233394+0000","last_undegraded":"2024-09-10T22:10:37.233394+0000","last_fullsized":"2024-09-10T22:10:37.233394+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:36.216847+0000","last_change":"2024-09-10T22:10:36.216847+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.216847+0000","last_undegraded":"2024-09-10T22:10:36.216847+0000","last_fullsized":"2024-09-10T22:10:36.216847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.233848+0000","last_change":"2024-09-10T22:10:37.233848+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.233848+0000","last_undegraded":"2024-09-10T22:10:37.233848+0000","last_fullsized":"2024-09-10T22:10:37.233848+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.250755+0000","last_change":"2024-09-10T22:10:37.248956+0000","last_active":"2024-09-10T22:10:37.250755+0000","last_peered":"2024-09-10T22:10:37.250755+0000","last_clean":"2024-09-10T22:10:37.250755+0000","last_became_active":"2024-09-10T22:10:37.248634+0000","last_became_peered":"2024-09-10T22:10:37.248634+0000","last_unstale":"2024-09-10T22:10:37.250755+0000","last_undegraded":"2024-09-10T22:10:37.250755+0000","last_fullsized":"2024-09-10T22:10:37.250755+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:41.093672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.243338+0000","last_change":"2024-09-10T22:10:37.241895+0000","last_active":"2024-09-10T22:10:37.243338+0000","last_peered":"2024-09-10T22:10:37.243338+0000","last_clean":"2024-09-10T22:10:37.243338+0000","last_became_active":"2024-09-10T22:10:37.241654+0000","last_became_peered":"2024-09-10T22:10:37.241654+0000","last_unstale":"2024-09-10T22:10:37.243338+0000","last_undegraded":"2024-09-10T22:10:37.243338+0000","last_fullsized":"2024-09-10T22:10:37.243338+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:59.031042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240508+0000","last_change":"2024-09-10T22:10:37.240508+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240508+0000","last_undegraded":"2024-09-10T22:10:37.240508+0000","last_fullsized":"2024-09-10T22:10:37.240508+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.255097+0000","last_change":"2024-09-10T22:10:37.254551+0000","last_active":"2024-09-10T22:10:37.255097+0000","last_peered":"2024-09-10T22:10:37.255097+0000","last_clean":"2024-09-10T22:10:37.255097+0000","last_became_active":"2024-09-10T22:10:37.254311+0000","last_became_peered":"2024-09-10T22:10:37.254311+0000","last_unstale":"2024-09-10T22:10:37.255097+0000","last_undegraded":"2024-09-10T22:10:37.255097+0000","last_fullsized":"2024-09-10T22:10:37.255097+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:41.187412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.233600+0000","last_change":"2024-09-10T22:10:37.233600+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.233600+0000","last_undegraded":"2024-09-10T22:10:37.233600+0000","last_fullsized":"2024-09-10T22:10:37.233600+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.253934+0000","last_change":"2024-09-10T22:10:37.249193+0000","last_active":"2024-09-10T22:10:37.253934+0000","last_peered":"2024-09-10T22:10:37.253934+0000","last_clean":"2024-09-10T22:10:37.253934+0000","last_became_active":"2024-09-10T22:10:37.248911+0000","last_became_peered":"2024-09-10T22:10:37.248911+0000","last_unstale":"2024-09-10T22:10:37.253934+0000","last_undegraded":"2024-09-10T22:10:37.253934+0000","last_fullsized":"2024-09-10T22:10:37.253934+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:57.484359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240468+0000","last_change":"2024-09-10T22:10:37.240468+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240468+0000","last_undegraded":"2024-09-10T22:10:37.240468+0000","last_fullsized":"2024-09-10T22:10:37.240468+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-10T22:10:36.225370+0000","last_change":"2024-09-10T22:10:36.225370+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.225370+0000","last_undegraded":"2024-09-10T22:10:36.225370+0000","last_fullsized":"2024-09-10T22:10:36.225370+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-10T22:10:36.223938+0000","last_change":"2024-09-10T22:10:36.223938+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.223938+0000","last_undegraded":"2024-09-10T22:10:36.223938+0000","last_fullsized":"2024-09-10T22:10:36.223938+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.247771+0000","last_change":"2024-09-10T22:10:37.246142+0000","last_active":"2024-09-10T22:10:37.247771+0000","last_peered":"2024-09-10T22:10:37.247771+0000","last_clean":"2024-09-10T22:10:37.247771+0000","last_became_active":"2024-09-10T22:10:37.245696+0000","last_became_peered":"2024-09-10T22:10:37.245696+0000","last_unstale":"2024-09-10T22:10:37.247771+0000","last_undegraded":"2024-09-10T22:10:37.247771+0000","last_fullsized":"2024-09-10T22:10:37.247771+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:57.484359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.253510+0000","last_change":"2024-09-10T22:10:37.249833+0000","last_active":"2024-09-10T22:10:37.253510+0000","last_peered":"2024-09-10T22:10:37.253510+0000","last_clean":"2024-09-10T22:10:37.253510+0000","last_became_active":"2024-09-10T22:10:37.248940+0000","last_became_peered":"2024-09-10T22:10:37.248940+0000","last_unstale":"2024-09-10T22:10:37.253510+0000","last_undegraded":"2024-09-10T22:10:37.253510+0000","last_fullsized":"2024-09-10T22:10:37.253510+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:55.320245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240893+0000","last_change":"2024-09-10T22:10:37.240893+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240893+0000","last_undegraded":"2024-09-10T22:10:37.240893+0000","last_fullsized":"2024-09-10T22:10:37.240893+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-10T22:10:36.223350+0000","last_change":"2024-09-10T22:10:36.223350+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.223350+0000","last_undegraded":"2024-09-10T22:10:36.223350+0000","last_fullsized":"2024-09-10T22:10:36.223350+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.253087+0000","last_change":"2024-09-10T22:10:37.248132+0000","last_active":"2024-09-10T22:10:37.253087+0000","last_peered":"2024-09-10T22:10:37.253087+0000","last_clean":"2024-09-10T22:10:37.253087+0000","last_became_active":"2024-09-10T22:10:37.247855+0000","last_became_peered":"2024-09-10T22:10:37.247855+0000","last_unstale":"2024-09-10T22:10:37.253087+0000","last_undegraded":"2024-09-10T22:10:37.253087+0000","last_fullsized":"2024-09-10T22:10:37.253087+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.310665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.237073+0000","last_change":"2024-09-10T22:10:37.237073+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.237073+0000","last_undegraded":"2024-09-10T22:10:37.237073+0000","last_fullsized":"2024-09-10T22:10:37.237073+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-10T22:10:36.223281+0000","last_change":"2024-09-10T22:10:36.223281+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.223281+0000","last_undegraded":"2024-09-10T22:10:36.223281+0000","last_fullsized":"2024-09-10T22:10:36.223281+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.235167+0000","last_change":"2024-09-10T22:10:37.235167+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.235167+0000","last_undegraded":"2024-09-10T22:10:37.235167+0000","last_fullsized":"2024-09-10T22:10:37.235167+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-10T22:10:36.223190+0000","last_change":"2024-09-10T22:10:36.223190+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.223190+0000","last_undegraded":"2024-09-10T22:10:36.223190+0000","last_fullsized":"2024-09-10T22:10:36.223190+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.239514+0000","last_change":"2024-09-10T22:10:37.239514+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.239514+0000","last_undegraded":"2024-09-10T22:10:37.239514+0000","last_fullsized":"2024-09-10T22:10:37.239514+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.251650+0000","last_change":"2024-09-10T22:10:37.249927+0000","last_active":"2024-09-10T22:10:37.251650+0000","last_peered":"2024-09-10T22:10:37.251650+0000","last_clean":"2024-09-10T22:10:37.251650+0000","last_became_active":"2024-09-10T22:10:37.249443+0000","last_became_peered":"2024-09-10T22:10:37.249443+0000","last_unstale":"2024-09-10T22:10:37.251650+0000","last_undegraded":"2024-09-10T22:10:37.251650+0000","last_fullsized":"2024-09-10T22:10:37.251650+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:09.268161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-10T22:10:36.223188+0000","last_change":"2024-09-10T22:10:36.223188+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.223188+0000","last_undegraded":"2024-09-10T22:10:36.223188+0000","last_fullsized":"2024-09-10T22:10:36.223188+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.235876+0000","last_change":"2024-09-10T22:10:37.235876+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.235876+0000","last_undegraded":"2024-09-10T22:10:37.235876+0000","last_fullsized":"2024-09-10T22:10:37.235876+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.253917+0000","last_change":"2024-09-10T22:10:37.251944+0000","last_active":"2024-09-10T22:10:37.253917+0000","last_peered":"2024-09-10T22:10:37.253917+0000","last_clean":"2024-09-10T22:10:37.253917+0000","last_became_active":"2024-09-10T22:10:37.251665+0000","last_became_peered":"2024-09-10T22:10:37.251665+0000","last_unstale":"2024-09-10T22:10:37.253917+0000","last_undegraded":"2024-09-10T22:10:37.253917+0000","last_fullsized":"2024-09-10T22:10:37.253917+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:22.695503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-10T22:10:36.226139+0000","last_change":"2024-09-10T22:10:36.226139+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.226139+0000","last_undegraded":"2024-09-10T22:10:36.226139+0000","last_fullsized":"2024-09-10T22:10:36.226139+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240490+0000","last_change":"2024-09-10T22:10:37.240490+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240490+0000","last_undegraded":"2024-09-10T22:10:37.240490+0000","last_fullsized":"2024-09-10T22:10:37.240490+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:36.216847+0000","last_change":"2024-09-10T22:10:36.216847+0000","last_active":"2024-09-10T22:10:36.216847+0000","last_peered":"2024-09-10T22:10:36.216847+0000","last_clean":"2024-09-10T22:10:36.216847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:36.216847+0000","last_undegraded":"2024-09-10T22:10:36.216847+0000","last_fullsized":"2024-09-10T22:10:36.216847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:10:37.219743+0000","last_change":"2024-09-10T22:10:37.219743+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.219743+0000","last_undegraded":"2024-09-10T22:10:37.219743+0000","last_fullsized":"2024-09-10T22:10:37.219743+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.252402+0000","last_change":"2024-09-10T22:10:37.249681+0000","last_active":"2024-09-10T22:10:37.252402+0000","last_peered":"2024-09-10T22:10:37.252402+0000","last_clean":"2024-09-10T22:10:37.252402+0000","last_became_active":"2024-09-10T22:10:37.249376+0000","last_became_peered":"2024-09-10T22:10:37.249376+0000","last_unstale":"2024-09-10T22:10:37.252402+0000","last_undegraded":"2024-09-10T22:10:37.252402+0000","last_fullsized":"2024-09-10T22:10:37.252402+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:00.548850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":256,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.244716+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:10:37.244716+0000","last_peered":"2024-09-10T22:10:37.244716+0000","last_clean":"2024-09-10T22:10:37.244716+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:10:37.244716+0000","last_undegraded":"2024-09-10T22:10:37.244716+0000","last_fullsized":"2024-09-10T22:10:37.244716+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":70,"acting":70,"num_store_stats":5},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509566,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123120,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248720,"statfs":{"total":96636764160,"available":96510689280,"internally_reserved":0,"allocated":1798144,"data_stored":1108825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509567,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130848,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240992,"statfs":{"total":96636764160,"available":96502775808,"internally_reserved":0,"allocated":1191936,"data_stored":512126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509566,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64240,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307600,"statfs":{"total":96636764160,"available":96570982400,"internally_reserved":0,"allocated":1191936,"data_stored":512126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509567,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140588,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231252,"statfs":{"total":96636764160,"available":96492802048,"internally_reserved":0,"allocated":1191936,"data_stored":512126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122528,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249312,"statfs":{"total":96636764160,"available":96511295488,"internally_reserved":0,"allocated":1191936,"data_stored":512126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509566,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126692,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245148,"statfs":{"total":96636764160,"available":96507031552,"internally_reserved":0,"allocated":1191936,"data_stored":512126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509566,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123120,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248720,"statfs":{"total":96636764160,"available":96510689280,"internally_reserved":0,"allocated":1785856,"data_stored":1102494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":124254665},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542271,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":144724,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227116,"statfs":{"total":96636764160,"available":96488566784,"internally_reserved":0,"allocated":1179648,"data_stored":506253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:10:42.393 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:10:42.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.572+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3385246571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a68004810 msgr2=0x7f2a68004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.572+0000 7f2a8ba00640 1 --2- 172.21.15.66:0/3385246571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a68004810 0x7f2a68004c90 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009b30 tx=0x7f2a6c01d230 comp rx=0 tx=0).stop 2024-09-10T22:10:42.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.572+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3385246571 shutdown_connections 2024-09-10T22:10:42.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.572+0000 7f2a8ba00640 1 --2- 172.21.15.66:0/3385246571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a68004810 0x7f2a68004c90 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.572+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3385246571 >> 172.21.15.66:0/3385246571 conn(0x7f2a840d1c60 msgr2=0x7f2a840d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:42.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.572+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3385246571 shutdown_connections 2024-09-10T22:10:42.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.572+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3385246571 wait complete. 2024-09-10T22:10:42.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a8ba00640 1 Processor -- start 2024-09-10T22:10:42.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a8ba00640 1 -- start start 2024-09-10T22:10:42.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a8ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a68004810 0x7f2a8415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:42.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a8ba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8409df90 0x7f2a8409c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:42.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a8ba00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a840c5850 0x7f2a840c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:42.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a8ba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a840cdfc0 con 0x7f2a68004810 2024-09-10T22:10:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a8ba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a840cde40 con 0x7f2a8409df90 2024-09-10T22:10:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a8ba00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2a840ce140 con 0x7f2a840c5850 2024-09-10T22:10:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a89000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a68004810 0x7f2a8415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a89000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a68004810 0x7f2a8415bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54274/0 (socket says 172.21.15.66:54274) 2024-09-10T22:10:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a89000640 1 -- 172.21.15.66:0/3604396435 learned_addr learned my addr 172.21.15.66:0/3604396435 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a89000640 1 -- 172.21.15.66:0/3604396435 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a840c5850 msgr2=0x7f2a840c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a89000640 1 --2- 172.21.15.66:0/3604396435 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a840c5850 0x7f2a840c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a89000640 1 -- 172.21.15.66:0/3604396435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8409df90 msgr2=0x7f2a8409c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a89000640 1 --2- 172.21.15.66:0/3604396435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8409df90 0x7f2a8409c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a89000640 1 -- 172.21.15.66:0/3604396435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8409e530 con 0x7f2a68004810 2024-09-10T22:10:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a89000640 1 --2- 172.21.15.66:0/3604396435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a68004810 0x7f2a8415bfd0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2a74004da0 tx=0x7f2a74001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a81600640 1 -- 172.21.15.66:0/3604396435 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74001d50 con 0x7f2a68004810 2024-09-10T22:10:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a81600640 1 -- 172.21.15.66:0/3604396435 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a74016070 con 0x7f2a68004810 2024-09-10T22:10:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a81600640 1 -- 172.21.15.66:0/3604396435 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74010b40 con 0x7f2a68004810 2024-09-10T22:10:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3604396435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8409bc10 con 0x7f2a68004810 2024-09-10T22:10:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.576+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3604396435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8409c1d0 con 0x7f2a68004810 2024-09-10T22:10:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.580+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3604396435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a840c6270 con 0x7f2a68004810 2024-09-10T22:10:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.580+0000 7f2a81600640 1 -- 172.21.15.66:0/3604396435 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f2a74010ce0 con 0x7f2a68004810 2024-09-10T22:10:42.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.580+0000 7f2a81600640 1 --2- 172.21.15.66:0/3604396435 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a60073c00 0x7f2a600760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:42.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.580+0000 7f2a81600640 1 -- 172.21.15.66:0/3604396435 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f2a74092b80 con 0x7f2a68004810 2024-09-10T22:10:42.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.580+0000 7f2a7be00640 1 --2- 172.21.15.66:0/3604396435 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a60073c00 0x7f2a600760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:42.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.584+0000 7f2a7be00640 1 --2- 172.21.15.66:0/3604396435 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a60073c00 0x7f2a600760c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0ae000 tx=0x7f2a68005840 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:42.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.584+0000 7f2a81600640 1 -- 172.21.15.66:0/3604396435 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2a7405f8a0 con 0x7f2a68004810 2024-09-10T22:10:42.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.764+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3604396435 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a840cbef0 con 0x7f2a60073c00 2024-09-10T22:10:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.772+0000 7f2a81600640 1 -- 172.21.15.66:0/3604396435 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251440 (secure 0 0 0) 0x7f2a840cbef0 con 0x7f2a60073c00 2024-09-10T22:10:42.777 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:42.779 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:10:42.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.776+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3604396435 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a60073c00 msgr2=0x7f2a600760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.776+0000 7f2a8ba00640 1 --2- 172.21.15.66:0/3604396435 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a60073c00 0x7f2a600760c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0ae000 tx=0x7f2a68005840 comp rx=0 tx=0).stop 2024-09-10T22:10:42.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.776+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3604396435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a68004810 msgr2=0x7f2a8415bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.776+0000 7f2a8ba00640 1 --2- 172.21.15.66:0/3604396435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a68004810 0x7f2a8415bfd0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2a74004da0 tx=0x7f2a74001640 comp rx=0 tx=0).stop 2024-09-10T22:10:42.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.776+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3604396435 shutdown_connections 2024-09-10T22:10:42.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.776+0000 7f2a8ba00640 1 --2- 172.21.15.66:0/3604396435 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a60073c00 0x7f2a600760c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.776+0000 7f2a8ba00640 1 --2- 172.21.15.66:0/3604396435 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a840c5850 0x7f2a840c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.776+0000 7f2a8ba00640 1 --2- 172.21.15.66:0/3604396435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8409df90 0x7f2a8409c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.776+0000 7f2a8ba00640 1 --2- 172.21.15.66:0/3604396435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a68004810 0x7f2a8415bfd0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.776+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3604396435 >> 172.21.15.66:0/3604396435 conn(0x7f2a840d1c60 msgr2=0x7f2a840d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:42.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.776+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3604396435 shutdown_connections 2024-09-10T22:10:42.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.776+0000 7f2a8ba00640 1 -- 172.21.15.66:0/3604396435 wait complete. 2024-09-10T22:10:42.798 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-10T22:10:42.542883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":980324,"kb_used_data":10792,"kb_used_omap":12,"kb_used_meta":969331,"kb_avail":753994396,"statfs":{"total":773094113280,"available":772090261504,"internally_reserved":0,"allocated":11051008,"data_stored":5425409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":992595508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001610"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381580+0000","last_change":"2024-09-10T22:10:38.251267+0000","last_active":"2024-09-10T22:10:39.381580+0000","last_peered":"2024-09-10T22:10:39.381580+0000","last_clean":"2024-09-10T22:10:39.381580+0000","last_became_active":"2024-09-10T22:10:38.250933+0000","last_became_peered":"2024-09-10T22:10:38.250933+0000","last_unstale":"2024-09-10T22:10:39.381580+0000","last_undegraded":"2024-09-10T22:10:39.381580+0000","last_fullsized":"2024-09-10T22:10:39.381580+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:41.184652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523102+0000","last_change":"2024-09-10T22:10:38.251283+0000","last_active":"2024-09-10T22:10:39.523102+0000","last_peered":"2024-09-10T22:10:39.523102+0000","last_clean":"2024-09-10T22:10:39.523102+0000","last_became_active":"2024-09-10T22:10:38.250986+0000","last_became_peered":"2024-09-10T22:10:38.250986+0000","last_unstale":"2024-09-10T22:10:39.523102+0000","last_undegraded":"2024-09-10T22:10:39.523102+0000","last_fullsized":"2024-09-10T22:10:39.523102+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:02.661632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381540+0000","last_change":"2024-09-10T22:10:38.253355+0000","last_active":"2024-09-10T22:10:39.381540+0000","last_peered":"2024-09-10T22:10:39.381540+0000","last_clean":"2024-09-10T22:10:39.381540+0000","last_became_active":"2024-09-10T22:10:38.253088+0000","last_became_peered":"2024-09-10T22:10:38.253088+0000","last_unstale":"2024-09-10T22:10:39.381540+0000","last_undegraded":"2024-09-10T22:10:39.381540+0000","last_fullsized":"2024-09-10T22:10:39.381540+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:18.638299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247441+0000","last_change":"2024-09-10T22:10:38.252302+0000","last_active":"2024-09-10T22:10:39.247441+0000","last_peered":"2024-09-10T22:10:39.247441+0000","last_clean":"2024-09-10T22:10:39.247441+0000","last_became_active":"2024-09-10T22:10:38.252084+0000","last_became_peered":"2024-09-10T22:10:38.252084+0000","last_unstale":"2024-09-10T22:10:39.247441+0000","last_undegraded":"2024-09-10T22:10:39.247441+0000","last_fullsized":"2024-09-10T22:10:39.247441+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:47.247838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528491+0000","last_change":"2024-09-10T22:10:38.263805+0000","last_active":"2024-09-10T22:10:39.528491+0000","last_peered":"2024-09-10T22:10:39.528491+0000","last_clean":"2024-09-10T22:10:39.528491+0000","last_became_active":"2024-09-10T22:10:38.263535+0000","last_became_peered":"2024-09-10T22:10:38.263535+0000","last_unstale":"2024-09-10T22:10:39.528491+0000","last_undegraded":"2024-09-10T22:10:39.528491+0000","last_fullsized":"2024-09-10T22:10:39.528491+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:26.055062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.241724+0000","last_change":"2024-09-10T22:10:37.241724+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.241724+0000","last_undegraded":"2024-09-10T22:10:37.241724+0000","last_fullsized":"2024-09-10T22:10:37.241724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240036+0000","last_change":"2024-09-10T22:10:37.240036+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240036+0000","last_undegraded":"2024-09-10T22:10:37.240036+0000","last_fullsized":"2024-09-10T22:10:37.240036+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522599+0000","last_change":"2024-09-10T22:10:38.254564+0000","last_active":"2024-09-10T22:10:39.522599+0000","last_peered":"2024-09-10T22:10:39.522599+0000","last_clean":"2024-09-10T22:10:39.522599+0000","last_became_active":"2024-09-10T22:10:38.254280+0000","last_became_peered":"2024-09-10T22:10:38.254280+0000","last_unstale":"2024-09-10T22:10:39.522599+0000","last_undegraded":"2024-09-10T22:10:39.522599+0000","last_fullsized":"2024-09-10T22:10:39.522599+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:48.733204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.255200+0000","last_change":"2024-09-10T22:10:38.249346+0000","last_active":"2024-09-10T22:10:39.255200+0000","last_peered":"2024-09-10T22:10:39.255200+0000","last_clean":"2024-09-10T22:10:39.255200+0000","last_became_active":"2024-09-10T22:10:38.249101+0000","last_became_peered":"2024-09-10T22:10:38.249101+0000","last_unstale":"2024-09-10T22:10:39.255200+0000","last_undegraded":"2024-09-10T22:10:39.255200+0000","last_fullsized":"2024-09-10T22:10:39.255200+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:38.972161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528751+0000","last_change":"2024-09-10T22:10:38.266215+0000","last_active":"2024-09-10T22:10:39.528751+0000","last_peered":"2024-09-10T22:10:39.528751+0000","last_clean":"2024-09-10T22:10:39.528751+0000","last_became_active":"2024-09-10T22:10:38.264693+0000","last_became_peered":"2024-09-10T22:10:38.264693+0000","last_unstale":"2024-09-10T22:10:39.528751+0000","last_undegraded":"2024-09-10T22:10:39.528751+0000","last_fullsized":"2024-09-10T22:10:39.528751+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:09.177869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.243036+0000","last_change":"2024-09-10T22:10:37.243036+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.243036+0000","last_undegraded":"2024-09-10T22:10:37.243036+0000","last_fullsized":"2024-09-10T22:10:37.243036+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254568+0000","last_change":"2024-09-10T22:10:38.247539+0000","last_active":"2024-09-10T22:10:39.254568+0000","last_peered":"2024-09-10T22:10:39.254568+0000","last_clean":"2024-09-10T22:10:39.254568+0000","last_became_active":"2024-09-10T22:10:38.247124+0000","last_became_peered":"2024-09-10T22:10:38.247124+0000","last_unstale":"2024-09-10T22:10:39.254568+0000","last_undegraded":"2024-09-10T22:10:39.254568+0000","last_fullsized":"2024-09-10T22:10:39.254568+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:26.055062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247126+0000","last_change":"2024-09-10T22:10:38.243927+0000","last_active":"2024-09-10T22:10:39.247126+0000","last_peered":"2024-09-10T22:10:39.247126+0000","last_clean":"2024-09-10T22:10:39.247126+0000","last_became_active":"2024-09-10T22:10:38.240651+0000","last_became_peered":"2024-09-10T22:10:38.240651+0000","last_unstale":"2024-09-10T22:10:39.247126+0000","last_undegraded":"2024-09-10T22:10:39.247126+0000","last_fullsized":"2024-09-10T22:10:39.247126+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:34.748065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523481+0000","last_change":"2024-09-10T22:10:38.264327+0000","last_active":"2024-09-10T22:10:39.523481+0000","last_peered":"2024-09-10T22:10:39.523481+0000","last_clean":"2024-09-10T22:10:39.523481+0000","last_became_active":"2024-09-10T22:10:38.264014+0000","last_became_peered":"2024-09-10T22:10:38.264014+0000","last_unstale":"2024-09-10T22:10:39.523481+0000","last_undegraded":"2024-09-10T22:10:39.523481+0000","last_fullsized":"2024-09-10T22:10:39.523481+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:24.533086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247264+0000","last_change":"2024-09-10T22:10:38.244207+0000","last_active":"2024-09-10T22:10:39.247264+0000","last_peered":"2024-09-10T22:10:39.247264+0000","last_clean":"2024-09-10T22:10:39.247264+0000","last_became_active":"2024-09-10T22:10:38.243891+0000","last_became_peered":"2024-09-10T22:10:38.243891+0000","last_unstale":"2024-09-10T22:10:39.247264+0000","last_undegraded":"2024-09-10T22:10:39.247264+0000","last_fullsized":"2024-09-10T22:10:39.247264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:39.927846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240030+0000","last_change":"2024-09-10T22:10:37.240030+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240030+0000","last_undegraded":"2024-09-10T22:10:37.240030+0000","last_fullsized":"2024-09-10T22:10:37.240030+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523341+0000","last_change":"2024-09-10T22:10:38.243673+0000","last_active":"2024-09-10T22:10:39.523341+0000","last_peered":"2024-09-10T22:10:39.523341+0000","last_clean":"2024-09-10T22:10:39.523341+0000","last_became_active":"2024-09-10T22:10:38.243416+0000","last_became_peered":"2024-09-10T22:10:38.243416+0000","last_unstale":"2024-09-10T22:10:39.523341+0000","last_undegraded":"2024-09-10T22:10:39.523341+0000","last_fullsized":"2024-09-10T22:10:39.523341+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:28.506115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247572+0000","last_change":"2024-09-10T22:10:38.255081+0000","last_active":"2024-09-10T22:10:39.247572+0000","last_peered":"2024-09-10T22:10:39.247572+0000","last_clean":"2024-09-10T22:10:39.247572+0000","last_became_active":"2024-09-10T22:10:38.254659+0000","last_became_peered":"2024-09-10T22:10:38.254659+0000","last_unstale":"2024-09-10T22:10:39.247572+0000","last_undegraded":"2024-09-10T22:10:39.247572+0000","last_fullsized":"2024-09-10T22:10:39.247572+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:23.035694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524131+0000","last_change":"2024-09-10T22:10:38.247997+0000","last_active":"2024-09-10T22:10:39.524131+0000","last_peered":"2024-09-10T22:10:39.524131+0000","last_clean":"2024-09-10T22:10:39.524131+0000","last_became_active":"2024-09-10T22:10:38.247730+0000","last_became_peered":"2024-09-10T22:10:38.247730+0000","last_unstale":"2024-09-10T22:10:39.524131+0000","last_undegraded":"2024-09-10T22:10:39.524131+0000","last_fullsized":"2024-09-10T22:10:39.524131+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:32.488690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254657+0000","last_change":"2024-09-10T22:10:38.245564+0000","last_active":"2024-09-10T22:10:39.254657+0000","last_peered":"2024-09-10T22:10:39.254657+0000","last_clean":"2024-09-10T22:10:39.254657+0000","last_became_active":"2024-09-10T22:10:38.245271+0000","last_became_peered":"2024-09-10T22:10:38.245271+0000","last_unstale":"2024-09-10T22:10:39.254657+0000","last_undegraded":"2024-09-10T22:10:39.254657+0000","last_fullsized":"2024-09-10T22:10:39.254657+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:26.491631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523720+0000","last_change":"2024-09-10T22:10:38.250433+0000","last_active":"2024-09-10T22:10:39.523720+0000","last_peered":"2024-09-10T22:10:39.523720+0000","last_clean":"2024-09-10T22:10:39.523720+0000","last_became_active":"2024-09-10T22:10:38.250154+0000","last_became_peered":"2024-09-10T22:10:38.250154+0000","last_unstale":"2024-09-10T22:10:39.523720+0000","last_undegraded":"2024-09-10T22:10:39.523720+0000","last_fullsized":"2024-09-10T22:10:39.523720+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:32.858119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522651+0000","last_change":"2024-09-10T22:10:38.245066+0000","last_active":"2024-09-10T22:10:39.522651+0000","last_peered":"2024-09-10T22:10:39.522651+0000","last_clean":"2024-09-10T22:10:39.522651+0000","last_became_active":"2024-09-10T22:10:38.244768+0000","last_became_peered":"2024-09-10T22:10:38.244768+0000","last_unstale":"2024-09-10T22:10:39.522651+0000","last_undegraded":"2024-09-10T22:10:39.522651+0000","last_fullsized":"2024-09-10T22:10:39.522651+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:44.888191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522566+0000","last_change":"2024-09-10T22:10:38.250358+0000","last_active":"2024-09-10T22:10:39.522566+0000","last_peered":"2024-09-10T22:10:39.522566+0000","last_clean":"2024-09-10T22:10:39.522566+0000","last_became_active":"2024-09-10T22:10:38.250103+0000","last_became_peered":"2024-09-10T22:10:38.250103+0000","last_unstale":"2024-09-10T22:10:39.522566+0000","last_undegraded":"2024-09-10T22:10:39.522566+0000","last_fullsized":"2024-09-10T22:10:39.522566+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:35.427777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.239464+0000","last_change":"2024-09-10T22:10:37.239464+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.239464+0000","last_undegraded":"2024-09-10T22:10:37.239464+0000","last_fullsized":"2024-09-10T22:10:37.239464+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528701+0000","last_change":"2024-09-10T22:10:38.269614+0000","last_active":"2024-09-10T22:10:39.528701+0000","last_peered":"2024-09-10T22:10:39.528701+0000","last_clean":"2024-09-10T22:10:39.528701+0000","last_became_active":"2024-09-10T22:10:38.269330+0000","last_became_peered":"2024-09-10T22:10:38.269330+0000","last_unstale":"2024-09-10T22:10:39.528701+0000","last_undegraded":"2024-09-10T22:10:39.528701+0000","last_fullsized":"2024-09-10T22:10:39.528701+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:38.972161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523101+0000","last_change":"2024-09-10T22:10:38.247355+0000","last_active":"2024-09-10T22:10:39.523101+0000","last_peered":"2024-09-10T22:10:39.523101+0000","last_clean":"2024-09-10T22:10:39.523101+0000","last_became_active":"2024-09-10T22:10:38.246782+0000","last_became_peered":"2024-09-10T22:10:38.246782+0000","last_unstale":"2024-09-10T22:10:39.523101+0000","last_undegraded":"2024-09-10T22:10:39.523101+0000","last_fullsized":"2024-09-10T22:10:39.523101+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:57.493480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.239961+0000","last_change":"2024-09-10T22:10:37.239961+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.239961+0000","last_undegraded":"2024-09-10T22:10:37.239961+0000","last_fullsized":"2024-09-10T22:10:37.239961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528635+0000","last_change":"2024-09-10T22:10:38.270949+0000","last_active":"2024-09-10T22:10:39.528635+0000","last_peered":"2024-09-10T22:10:39.528635+0000","last_clean":"2024-09-10T22:10:39.528635+0000","last_became_active":"2024-09-10T22:10:38.270580+0000","last_became_peered":"2024-09-10T22:10:38.270580+0000","last_unstale":"2024-09-10T22:10:39.528635+0000","last_undegraded":"2024-09-10T22:10:39.528635+0000","last_fullsized":"2024-09-10T22:10:39.528635+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:46.669269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522776+0000","last_change":"2024-09-10T22:10:38.244297+0000","last_active":"2024-09-10T22:10:39.522776+0000","last_peered":"2024-09-10T22:10:39.522776+0000","last_clean":"2024-09-10T22:10:39.522776+0000","last_became_active":"2024-09-10T22:10:38.243953+0000","last_became_peered":"2024-09-10T22:10:38.243953+0000","last_unstale":"2024-09-10T22:10:39.522776+0000","last_undegraded":"2024-09-10T22:10:39.522776+0000","last_fullsized":"2024-09-10T22:10:39.522776+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:08.221611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523686+0000","last_change":"2024-09-10T22:10:38.247381+0000","last_active":"2024-09-10T22:10:39.523686+0000","last_peered":"2024-09-10T22:10:39.523686+0000","last_clean":"2024-09-10T22:10:39.523686+0000","last_became_active":"2024-09-10T22:10:38.247092+0000","last_became_peered":"2024-09-10T22:10:38.247092+0000","last_unstale":"2024-09-10T22:10:39.523686+0000","last_undegraded":"2024-09-10T22:10:39.523686+0000","last_fullsized":"2024-09-10T22:10:39.523686+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:04.052881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247281+0000","last_change":"2024-09-10T22:10:38.262682+0000","last_active":"2024-09-10T22:10:39.247281+0000","last_peered":"2024-09-10T22:10:39.247281+0000","last_clean":"2024-09-10T22:10:39.247281+0000","last_became_active":"2024-09-10T22:10:38.262426+0000","last_became_peered":"2024-09-10T22:10:38.262426+0000","last_unstale":"2024-09-10T22:10:39.247281+0000","last_undegraded":"2024-09-10T22:10:39.247281+0000","last_fullsized":"2024-09-10T22:10:39.247281+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:16.836613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381139+0000","last_change":"2024-09-10T22:10:38.248324+0000","last_active":"2024-09-10T22:10:39.381139+0000","last_peered":"2024-09-10T22:10:39.381139+0000","last_clean":"2024-09-10T22:10:39.381139+0000","last_became_active":"2024-09-10T22:10:38.245453+0000","last_became_peered":"2024-09-10T22:10:38.245453+0000","last_unstale":"2024-09-10T22:10:39.381139+0000","last_undegraded":"2024-09-10T22:10:39.381139+0000","last_fullsized":"2024-09-10T22:10:39.381139+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:22.865604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524095+0000","last_change":"2024-09-10T22:10:38.255400+0000","last_active":"2024-09-10T22:10:39.524095+0000","last_peered":"2024-09-10T22:10:39.524095+0000","last_clean":"2024-09-10T22:10:39.524095+0000","last_became_active":"2024-09-10T22:10:38.255069+0000","last_became_peered":"2024-09-10T22:10:38.255069+0000","last_unstale":"2024-09-10T22:10:39.524095+0000","last_undegraded":"2024-09-10T22:10:39.524095+0000","last_fullsized":"2024-09-10T22:10:39.524095+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:32.770140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"99'4","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523313+0000","last_change":"2024-09-10T22:10:37.247494+0000","last_active":"2024-09-10T22:10:39.523313+0000","last_peered":"2024-09-10T22:10:39.523313+0000","last_clean":"2024-09-10T22:10:39.523313+0000","last_became_active":"2024-09-10T22:10:37.244676+0000","last_became_peered":"2024-09-10T22:10:37.244676+0000","last_unstale":"2024-09-10T22:10:39.523313+0000","last_undegraded":"2024-09-10T22:10:39.523313+0000","last_fullsized":"2024-09-10T22:10:39.523313+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:25.917947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247778+0000","last_change":"2024-09-10T22:10:38.263771+0000","last_active":"2024-09-10T22:10:39.247778+0000","last_peered":"2024-09-10T22:10:39.247778+0000","last_clean":"2024-09-10T22:10:39.247778+0000","last_became_active":"2024-09-10T22:10:38.263458+0000","last_became_peered":"2024-09-10T22:10:38.263458+0000","last_unstale":"2024-09-10T22:10:39.247778+0000","last_undegraded":"2024-09-10T22:10:39.247778+0000","last_fullsized":"2024-09-10T22:10:39.247778+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:48.091865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.251427+0000","last_change":"2024-09-10T22:10:37.248597+0000","last_active":"2024-09-10T22:10:37.251427+0000","last_peered":"2024-09-10T22:10:37.251427+0000","last_clean":"2024-09-10T22:10:37.251427+0000","last_became_active":"2024-09-10T22:10:37.248297+0000","last_became_peered":"2024-09-10T22:10:37.248297+0000","last_unstale":"2024-09-10T22:10:37.251427+0000","last_undegraded":"2024-09-10T22:10:37.251427+0000","last_fullsized":"2024-09-10T22:10:37.251427+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:56.601971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381440+0000","last_change":"2024-09-10T22:10:38.241783+0000","last_active":"2024-09-10T22:10:39.381440+0000","last_peered":"2024-09-10T22:10:39.381440+0000","last_clean":"2024-09-10T22:10:39.381440+0000","last_became_active":"2024-09-10T22:10:38.241478+0000","last_became_peered":"2024-09-10T22:10:38.241478+0000","last_unstale":"2024-09-10T22:10:39.381440+0000","last_undegraded":"2024-09-10T22:10:39.381440+0000","last_fullsized":"2024-09-10T22:10:39.381440+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:37.074694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522717+0000","last_change":"2024-09-10T22:10:37.249272+0000","last_active":"2024-09-10T22:10:39.522717+0000","last_peered":"2024-09-10T22:10:39.522717+0000","last_clean":"2024-09-10T22:10:39.522717+0000","last_became_active":"2024-09-10T22:10:37.248969+0000","last_became_peered":"2024-09-10T22:10:37.248969+0000","last_unstale":"2024-09-10T22:10:39.522717+0000","last_undegraded":"2024-09-10T22:10:39.522717+0000","last_fullsized":"2024-09-10T22:10:39.522717+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:00.267217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.255113+0000","last_change":"2024-09-10T22:10:38.247440+0000","last_active":"2024-09-10T22:10:39.255113+0000","last_peered":"2024-09-10T22:10:39.255113+0000","last_clean":"2024-09-10T22:10:39.255113+0000","last_became_active":"2024-09-10T22:10:38.243369+0000","last_became_peered":"2024-09-10T22:10:38.243369+0000","last_unstale":"2024-09-10T22:10:39.255113+0000","last_undegraded":"2024-09-10T22:10:39.255113+0000","last_fullsized":"2024-09-10T22:10:39.255113+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:31.048301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522825+0000","last_change":"2024-09-10T22:10:37.235026+0000","last_active":"2024-09-10T22:10:39.522825+0000","last_peered":"2024-09-10T22:10:39.522825+0000","last_clean":"2024-09-10T22:10:39.522825+0000","last_became_active":"2024-09-10T22:10:37.234835+0000","last_became_peered":"2024-09-10T22:10:37.234835+0000","last_unstale":"2024-09-10T22:10:39.522825+0000","last_undegraded":"2024-09-10T22:10:39.522825+0000","last_fullsized":"2024-09-10T22:10:39.522825+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:39.992087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252383+0000","last_change":"2024-09-10T22:10:38.250119+0000","last_active":"2024-09-10T22:10:39.252383+0000","last_peered":"2024-09-10T22:10:39.252383+0000","last_clean":"2024-09-10T22:10:39.252383+0000","last_became_active":"2024-09-10T22:10:38.249748+0000","last_became_peered":"2024-09-10T22:10:38.249748+0000","last_unstale":"2024-09-10T22:10:39.252383+0000","last_undegraded":"2024-09-10T22:10:39.252383+0000","last_fullsized":"2024-09-10T22:10:39.252383+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:33.757624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381382+0000","last_change":"2024-09-10T22:10:37.247963+0000","last_active":"2024-09-10T22:10:39.381382+0000","last_peered":"2024-09-10T22:10:39.381382+0000","last_clean":"2024-09-10T22:10:39.381382+0000","last_became_active":"2024-09-10T22:10:37.246784+0000","last_became_peered":"2024-09-10T22:10:37.246784+0000","last_unstale":"2024-09-10T22:10:39.381382+0000","last_undegraded":"2024-09-10T22:10:39.381382+0000","last_fullsized":"2024-09-10T22:10:39.381382+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:33.104121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.241495+0000","last_change":"2024-09-10T22:10:37.241495+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.241495+0000","last_undegraded":"2024-09-10T22:10:37.241495+0000","last_fullsized":"2024-09-10T22:10:37.241495+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522451+0000","last_change":"2024-09-10T22:10:37.240295+0000","last_active":"2024-09-10T22:10:39.522451+0000","last_peered":"2024-09-10T22:10:39.522451+0000","last_clean":"2024-09-10T22:10:39.522451+0000","last_became_active":"2024-09-10T22:10:37.240014+0000","last_became_peered":"2024-09-10T22:10:37.240014+0000","last_unstale":"2024-09-10T22:10:39.522451+0000","last_undegraded":"2024-09-10T22:10:39.522451+0000","last_fullsized":"2024-09-10T22:10:39.522451+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:19.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247084+0000","last_change":"2024-09-10T22:10:38.245203+0000","last_active":"2024-09-10T22:10:39.247084+0000","last_peered":"2024-09-10T22:10:39.247084+0000","last_clean":"2024-09-10T22:10:39.247084+0000","last_became_active":"2024-09-10T22:10:38.244911+0000","last_became_peered":"2024-09-10T22:10:38.244911+0000","last_unstale":"2024-09-10T22:10:39.247084+0000","last_undegraded":"2024-09-10T22:10:39.247084+0000","last_fullsized":"2024-09-10T22:10:39.247084+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:19.820527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528261+0000","last_change":"2024-09-10T22:10:37.258021+0000","last_active":"2024-09-10T22:10:39.528261+0000","last_peered":"2024-09-10T22:10:39.528261+0000","last_clean":"2024-09-10T22:10:39.528261+0000","last_became_active":"2024-09-10T22:10:37.257766+0000","last_became_peered":"2024-09-10T22:10:37.257766+0000","last_unstale":"2024-09-10T22:10:39.528261+0000","last_undegraded":"2024-09-10T22:10:39.528261+0000","last_fullsized":"2024-09-10T22:10:39.528261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:36.653416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254271+0000","last_change":"2024-09-10T22:10:38.243002+0000","last_active":"2024-09-10T22:10:39.254271+0000","last_peered":"2024-09-10T22:10:39.254271+0000","last_clean":"2024-09-10T22:10:39.254271+0000","last_became_active":"2024-09-10T22:10:38.241629+0000","last_became_peered":"2024-09-10T22:10:38.241629+0000","last_unstale":"2024-09-10T22:10:39.254271+0000","last_undegraded":"2024-09-10T22:10:39.254271+0000","last_fullsized":"2024-09-10T22:10:39.254271+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:21.178257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381135+0000","last_change":"2024-09-10T22:10:37.247160+0000","last_active":"2024-09-10T22:10:39.381135+0000","last_peered":"2024-09-10T22:10:39.381135+0000","last_clean":"2024-09-10T22:10:39.381135+0000","last_became_active":"2024-09-10T22:10:37.246865+0000","last_became_peered":"2024-09-10T22:10:37.246865+0000","last_unstale":"2024-09-10T22:10:39.381135+0000","last_undegraded":"2024-09-10T22:10:39.381135+0000","last_fullsized":"2024-09-10T22:10:39.381135+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:54.381121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523577+0000","last_change":"2024-09-10T22:10:38.257125+0000","last_active":"2024-09-10T22:10:39.523577+0000","last_peered":"2024-09-10T22:10:39.523577+0000","last_clean":"2024-09-10T22:10:39.523577+0000","last_became_active":"2024-09-10T22:10:38.256821+0000","last_became_peered":"2024-09-10T22:10:38.256821+0000","last_unstale":"2024-09-10T22:10:39.523577+0000","last_undegraded":"2024-09-10T22:10:39.523577+0000","last_fullsized":"2024-09-10T22:10:39.523577+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:13.792877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381703+0000","last_change":"2024-09-10T22:10:37.246423+0000","last_active":"2024-09-10T22:10:39.381703+0000","last_peered":"2024-09-10T22:10:39.381703+0000","last_clean":"2024-09-10T22:10:39.381703+0000","last_became_active":"2024-09-10T22:10:37.246139+0000","last_became_peered":"2024-09-10T22:10:37.246139+0000","last_unstale":"2024-09-10T22:10:39.381703+0000","last_undegraded":"2024-09-10T22:10:39.381703+0000","last_fullsized":"2024-09-10T22:10:39.381703+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:00.685355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.255201+0000","last_change":"2024-09-10T22:10:38.253930+0000","last_active":"2024-09-10T22:10:39.255201+0000","last_peered":"2024-09-10T22:10:39.255201+0000","last_clean":"2024-09-10T22:10:39.255201+0000","last_became_active":"2024-09-10T22:10:38.253264+0000","last_became_peered":"2024-09-10T22:10:38.253264+0000","last_unstale":"2024-09-10T22:10:39.255201+0000","last_undegraded":"2024-09-10T22:10:39.255201+0000","last_fullsized":"2024-09-10T22:10:39.255201+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:19.820527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247473+0000","last_change":"2024-09-10T22:10:37.254160+0000","last_active":"2024-09-10T22:10:39.247473+0000","last_peered":"2024-09-10T22:10:39.247473+0000","last_clean":"2024-09-10T22:10:39.247473+0000","last_became_active":"2024-09-10T22:10:37.253831+0000","last_became_peered":"2024-09-10T22:10:37.253831+0000","last_unstale":"2024-09-10T22:10:39.247473+0000","last_undegraded":"2024-09-10T22:10:39.247473+0000","last_fullsized":"2024-09-10T22:10:39.247473+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:00.267217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528676+0000","last_change":"2024-09-10T22:10:38.266948+0000","last_active":"2024-09-10T22:10:39.528676+0000","last_peered":"2024-09-10T22:10:39.528676+0000","last_clean":"2024-09-10T22:10:39.528676+0000","last_became_active":"2024-09-10T22:10:38.266677+0000","last_became_peered":"2024-09-10T22:10:38.266677+0000","last_unstale":"2024-09-10T22:10:39.528676+0000","last_undegraded":"2024-09-10T22:10:39.528676+0000","last_fullsized":"2024-09-10T22:10:39.528676+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:08.232358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.253794+0000","last_change":"2024-09-10T22:10:37.252591+0000","last_active":"2024-09-10T22:10:37.253794+0000","last_peered":"2024-09-10T22:10:37.253794+0000","last_clean":"2024-09-10T22:10:37.253794+0000","last_became_active":"2024-09-10T22:10:37.252236+0000","last_became_peered":"2024-09-10T22:10:37.252236+0000","last_unstale":"2024-09-10T22:10:37.253794+0000","last_undegraded":"2024-09-10T22:10:37.253794+0000","last_fullsized":"2024-09-10T22:10:37.253794+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:25.052167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.242629+0000","last_change":"2024-09-10T22:10:37.242629+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.242629+0000","last_undegraded":"2024-09-10T22:10:37.242629+0000","last_fullsized":"2024-09-10T22:10:37.242629+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"99'6","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254866+0000","last_change":"2024-09-10T22:10:37.245595+0000","last_active":"2024-09-10T22:10:39.254866+0000","last_peered":"2024-09-10T22:10:39.254866+0000","last_clean":"2024-09-10T22:10:39.254866+0000","last_became_active":"2024-09-10T22:10:37.245237+0000","last_became_peered":"2024-09-10T22:10:37.245237+0000","last_unstale":"2024-09-10T22:10:39.254866+0000","last_undegraded":"2024-09-10T22:10:39.254866+0000","last_fullsized":"2024-09-10T22:10:39.254866+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:17.957594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247188+0000","last_change":"2024-09-10T22:10:38.261146+0000","last_active":"2024-09-10T22:10:39.247188+0000","last_peered":"2024-09-10T22:10:39.247188+0000","last_clean":"2024-09-10T22:10:39.247188+0000","last_became_active":"2024-09-10T22:10:38.260713+0000","last_became_peered":"2024-09-10T22:10:38.260713+0000","last_unstale":"2024-09-10T22:10:39.247188+0000","last_undegraded":"2024-09-10T22:10:39.247188+0000","last_fullsized":"2024-09-10T22:10:39.247188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:20.279287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523954+0000","last_change":"2024-09-10T22:10:37.247682+0000","last_active":"2024-09-10T22:10:39.523954+0000","last_peered":"2024-09-10T22:10:39.523954+0000","last_clean":"2024-09-10T22:10:39.523954+0000","last_became_active":"2024-09-10T22:10:37.247416+0000","last_became_peered":"2024-09-10T22:10:37.247416+0000","last_unstale":"2024-09-10T22:10:39.523954+0000","last_undegraded":"2024-09-10T22:10:39.523954+0000","last_fullsized":"2024-09-10T22:10:39.523954+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:32.804016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381286+0000","last_change":"2024-09-10T22:10:38.248180+0000","last_active":"2024-09-10T22:10:39.381286+0000","last_peered":"2024-09-10T22:10:39.381286+0000","last_clean":"2024-09-10T22:10:39.381286+0000","last_became_active":"2024-09-10T22:10:38.247917+0000","last_became_peered":"2024-09-10T22:10:38.247917+0000","last_unstale":"2024-09-10T22:10:39.381286+0000","last_undegraded":"2024-09-10T22:10:39.381286+0000","last_fullsized":"2024-09-10T22:10:39.381286+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:45.132830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523823+0000","last_change":"2024-09-10T22:10:37.251530+0000","last_active":"2024-09-10T22:10:39.523823+0000","last_peered":"2024-09-10T22:10:39.523823+0000","last_clean":"2024-09-10T22:10:39.523823+0000","last_became_active":"2024-09-10T22:10:37.251337+0000","last_became_peered":"2024-09-10T22:10:37.251337+0000","last_unstale":"2024-09-10T22:10:39.523823+0000","last_undegraded":"2024-09-10T22:10:39.523823+0000","last_fullsized":"2024-09-10T22:10:39.523823+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:30.617743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254115+0000","last_change":"2024-09-10T22:10:38.246234+0000","last_active":"2024-09-10T22:10:39.254115+0000","last_peered":"2024-09-10T22:10:39.254115+0000","last_clean":"2024-09-10T22:10:39.254115+0000","last_became_active":"2024-09-10T22:10:38.241936+0000","last_became_peered":"2024-09-10T22:10:38.241936+0000","last_unstale":"2024-09-10T22:10:39.254115+0000","last_undegraded":"2024-09-10T22:10:39.254115+0000","last_fullsized":"2024-09-10T22:10:39.254115+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:40.994982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254261+0000","last_change":"2024-09-10T22:10:37.248956+0000","last_active":"2024-09-10T22:10:39.254261+0000","last_peered":"2024-09-10T22:10:39.254261+0000","last_clean":"2024-09-10T22:10:39.254261+0000","last_became_active":"2024-09-10T22:10:37.248634+0000","last_became_peered":"2024-09-10T22:10:37.248634+0000","last_unstale":"2024-09-10T22:10:39.254261+0000","last_undegraded":"2024-09-10T22:10:39.254261+0000","last_fullsized":"2024-09-10T22:10:39.254261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:23.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523489+0000","last_change":"2024-09-10T22:10:38.248859+0000","last_active":"2024-09-10T22:10:39.523489+0000","last_peered":"2024-09-10T22:10:39.523489+0000","last_clean":"2024-09-10T22:10:39.523489+0000","last_became_active":"2024-09-10T22:10:38.247750+0000","last_became_peered":"2024-09-10T22:10:38.247750+0000","last_unstale":"2024-09-10T22:10:39.523489+0000","last_undegraded":"2024-09-10T22:10:39.523489+0000","last_fullsized":"2024-09-10T22:10:39.523489+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:09.599932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"99'8","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381272+0000","last_change":"2024-09-10T22:10:37.241895+0000","last_active":"2024-09-10T22:10:39.381272+0000","last_peered":"2024-09-10T22:10:39.381272+0000","last_clean":"2024-09-10T22:10:39.381272+0000","last_became_active":"2024-09-10T22:10:37.241654+0000","last_became_peered":"2024-09-10T22:10:37.241654+0000","last_unstale":"2024-09-10T22:10:39.381272+0000","last_undegraded":"2024-09-10T22:10:39.381272+0000","last_fullsized":"2024-09-10T22:10:39.381272+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:10.220978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528948+0000","last_change":"2024-09-10T22:10:38.269389+0000","last_active":"2024-09-10T22:10:39.528948+0000","last_peered":"2024-09-10T22:10:39.528948+0000","last_clean":"2024-09-10T22:10:39.528948+0000","last_became_active":"2024-09-10T22:10:38.269130+0000","last_became_peered":"2024-09-10T22:10:38.269130+0000","last_unstale":"2024-09-10T22:10:39.528948+0000","last_undegraded":"2024-09-10T22:10:39.528948+0000","last_fullsized":"2024-09-10T22:10:39.528948+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:10.005594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254954+0000","last_change":"2024-09-10T22:10:37.254551+0000","last_active":"2024-09-10T22:10:39.254954+0000","last_peered":"2024-09-10T22:10:39.254954+0000","last_clean":"2024-09-10T22:10:39.254954+0000","last_became_active":"2024-09-10T22:10:37.254311+0000","last_became_peered":"2024-09-10T22:10:37.254311+0000","last_unstale":"2024-09-10T22:10:39.254954+0000","last_undegraded":"2024-09-10T22:10:39.254954+0000","last_fullsized":"2024-09-10T22:10:39.254954+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:25.574209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252554+0000","last_change":"2024-09-10T22:10:38.265801+0000","last_active":"2024-09-10T22:10:39.252554+0000","last_peered":"2024-09-10T22:10:39.252554+0000","last_clean":"2024-09-10T22:10:39.252554+0000","last_became_active":"2024-09-10T22:10:38.265549+0000","last_became_peered":"2024-09-10T22:10:38.265549+0000","last_unstale":"2024-09-10T22:10:39.252554+0000","last_undegraded":"2024-09-10T22:10:39.252554+0000","last_fullsized":"2024-09-10T22:10:39.252554+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:49.342939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528666+0000","last_change":"2024-09-10T22:10:37.249193+0000","last_active":"2024-09-10T22:10:39.528666+0000","last_peered":"2024-09-10T22:10:39.528666+0000","last_clean":"2024-09-10T22:10:39.528666+0000","last_became_active":"2024-09-10T22:10:37.248911+0000","last_became_peered":"2024-09-10T22:10:37.248911+0000","last_unstale":"2024-09-10T22:10:39.528666+0000","last_undegraded":"2024-09-10T22:10:39.528666+0000","last_fullsized":"2024-09-10T22:10:39.528666+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:02.732555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240468+0000","last_change":"2024-09-10T22:10:37.240468+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240468+0000","last_undegraded":"2024-09-10T22:10:37.240468+0000","last_fullsized":"2024-09-10T22:10:37.240468+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522890+0000","last_change":"2024-09-10T22:10:37.234553+0000","last_active":"2024-09-10T22:10:39.522890+0000","last_peered":"2024-09-10T22:10:39.522890+0000","last_clean":"2024-09-10T22:10:39.522890+0000","last_became_active":"2024-09-10T22:10:37.234278+0000","last_became_peered":"2024-09-10T22:10:37.234278+0000","last_unstale":"2024-09-10T22:10:39.522890+0000","last_undegraded":"2024-09-10T22:10:39.522890+0000","last_fullsized":"2024-09-10T22:10:39.522890+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:20.175362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523714+0000","last_change":"2024-09-10T22:10:38.256468+0000","last_active":"2024-09-10T22:10:39.523714+0000","last_peered":"2024-09-10T22:10:39.523714+0000","last_clean":"2024-09-10T22:10:39.523714+0000","last_became_active":"2024-09-10T22:10:38.256224+0000","last_became_peered":"2024-09-10T22:10:38.256224+0000","last_unstale":"2024-09-10T22:10:39.523714+0000","last_undegraded":"2024-09-10T22:10:39.523714+0000","last_fullsized":"2024-09-10T22:10:39.523714+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:23.765455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522649+0000","last_change":"2024-09-10T22:10:37.249141+0000","last_active":"2024-09-10T22:10:39.522649+0000","last_peered":"2024-09-10T22:10:39.522649+0000","last_clean":"2024-09-10T22:10:39.522649+0000","last_became_active":"2024-09-10T22:10:37.248865+0000","last_became_peered":"2024-09-10T22:10:37.248865+0000","last_unstale":"2024-09-10T22:10:39.522649+0000","last_undegraded":"2024-09-10T22:10:39.522649+0000","last_fullsized":"2024-09-10T22:10:39.522649+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:16.473063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523961+0000","last_change":"2024-09-10T22:10:38.247400+0000","last_active":"2024-09-10T22:10:39.523961+0000","last_peered":"2024-09-10T22:10:39.523961+0000","last_clean":"2024-09-10T22:10:39.523961+0000","last_became_active":"2024-09-10T22:10:38.244941+0000","last_became_peered":"2024-09-10T22:10:38.244941+0000","last_unstale":"2024-09-10T22:10:39.523961+0000","last_undegraded":"2024-09-10T22:10:39.523961+0000","last_fullsized":"2024-09-10T22:10:39.523961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:18.374331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252406+0000","last_change":"2024-09-10T22:10:37.246142+0000","last_active":"2024-09-10T22:10:39.252406+0000","last_peered":"2024-09-10T22:10:39.252406+0000","last_clean":"2024-09-10T22:10:39.252406+0000","last_became_active":"2024-09-10T22:10:37.245696+0000","last_became_peered":"2024-09-10T22:10:37.245696+0000","last_unstale":"2024-09-10T22:10:39.252406+0000","last_undegraded":"2024-09-10T22:10:39.252406+0000","last_fullsized":"2024-09-10T22:10:39.252406+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:23.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523257+0000","last_change":"2024-09-10T22:10:38.257969+0000","last_active":"2024-09-10T22:10:39.523257+0000","last_peered":"2024-09-10T22:10:39.523257+0000","last_clean":"2024-09-10T22:10:39.523257+0000","last_became_active":"2024-09-10T22:10:38.255428+0000","last_became_peered":"2024-09-10T22:10:38.255428+0000","last_unstale":"2024-09-10T22:10:39.523257+0000","last_undegraded":"2024-09-10T22:10:39.523257+0000","last_fullsized":"2024-09-10T22:10:39.523257+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:23.507403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252252+0000","last_change":"2024-09-10T22:10:37.249833+0000","last_active":"2024-09-10T22:10:39.252252+0000","last_peered":"2024-09-10T22:10:39.252252+0000","last_clean":"2024-09-10T22:10:39.252252+0000","last_became_active":"2024-09-10T22:10:37.248940+0000","last_became_peered":"2024-09-10T22:10:37.248940+0000","last_unstale":"2024-09-10T22:10:39.252252+0000","last_undegraded":"2024-09-10T22:10:39.252252+0000","last_fullsized":"2024-09-10T22:10:39.252252+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:17.957594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527472+0000","last_change":"2024-09-10T22:10:38.260839+0000","last_active":"2024-09-10T22:10:39.527472+0000","last_peered":"2024-09-10T22:10:39.527472+0000","last_clean":"2024-09-10T22:10:39.527472+0000","last_became_active":"2024-09-10T22:10:38.260199+0000","last_became_peered":"2024-09-10T22:10:38.260199+0000","last_unstale":"2024-09-10T22:10:39.527472+0000","last_undegraded":"2024-09-10T22:10:39.527472+0000","last_fullsized":"2024-09-10T22:10:39.527472+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:57.604867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522526+0000","last_change":"2024-09-10T22:10:37.248217+0000","last_active":"2024-09-10T22:10:39.522526+0000","last_peered":"2024-09-10T22:10:39.522526+0000","last_clean":"2024-09-10T22:10:39.522526+0000","last_became_active":"2024-09-10T22:10:37.247946+0000","last_became_peered":"2024-09-10T22:10:37.247946+0000","last_unstale":"2024-09-10T22:10:39.522526+0000","last_undegraded":"2024-09-10T22:10:39.522526+0000","last_fullsized":"2024-09-10T22:10:39.522526+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:25.488736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523362+0000","last_change":"2024-09-10T22:10:38.251544+0000","last_active":"2024-09-10T22:10:39.523362+0000","last_peered":"2024-09-10T22:10:39.523362+0000","last_clean":"2024-09-10T22:10:39.523362+0000","last_became_active":"2024-09-10T22:10:38.251270+0000","last_became_peered":"2024-09-10T22:10:38.251270+0000","last_unstale":"2024-09-10T22:10:39.523362+0000","last_undegraded":"2024-09-10T22:10:39.523362+0000","last_fullsized":"2024-09-10T22:10:39.523362+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:58.242589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252755+0000","last_change":"2024-09-10T22:10:37.248132+0000","last_active":"2024-09-10T22:10:39.252755+0000","last_peered":"2024-09-10T22:10:39.252755+0000","last_clean":"2024-09-10T22:10:39.252755+0000","last_became_active":"2024-09-10T22:10:37.247855+0000","last_became_peered":"2024-09-10T22:10:37.247855+0000","last_unstale":"2024-09-10T22:10:39.252755+0000","last_undegraded":"2024-09-10T22:10:39.252755+0000","last_fullsized":"2024-09-10T22:10:39.252755+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:44.140728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381589+0000","last_change":"2024-09-10T22:10:38.251921+0000","last_active":"2024-09-10T22:10:39.381589+0000","last_peered":"2024-09-10T22:10:39.381589+0000","last_clean":"2024-09-10T22:10:39.381589+0000","last_became_active":"2024-09-10T22:10:38.251620+0000","last_became_peered":"2024-09-10T22:10:38.251620+0000","last_unstale":"2024-09-10T22:10:39.381589+0000","last_undegraded":"2024-09-10T22:10:39.381589+0000","last_fullsized":"2024-09-10T22:10:39.381589+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:25.497370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"100'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523469+0000","last_change":"2024-09-10T22:10:37.253726+0000","last_active":"2024-09-10T22:10:39.523469+0000","last_peered":"2024-09-10T22:10:39.523469+0000","last_clean":"2024-09-10T22:10:39.523469+0000","last_became_active":"2024-09-10T22:10:37.252258+0000","last_became_peered":"2024-09-10T22:10:37.252258+0000","last_unstale":"2024-09-10T22:10:39.523469+0000","last_undegraded":"2024-09-10T22:10:39.523469+0000","last_fullsized":"2024-09-10T22:10:39.523469+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:30.045406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381513+0000","last_change":"2024-09-10T22:10:38.261993+0000","last_active":"2024-09-10T22:10:39.381513+0000","last_peered":"2024-09-10T22:10:39.381513+0000","last_clean":"2024-09-10T22:10:39.381513+0000","last_became_active":"2024-09-10T22:10:38.261706+0000","last_became_peered":"2024-09-10T22:10:38.261706+0000","last_unstale":"2024-09-10T22:10:39.381513+0000","last_undegraded":"2024-09-10T22:10:39.381513+0000","last_fullsized":"2024-09-10T22:10:39.381513+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:27.116503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522775+0000","last_change":"2024-09-10T22:10:37.236693+0000","last_active":"2024-09-10T22:10:39.522775+0000","last_peered":"2024-09-10T22:10:39.522775+0000","last_clean":"2024-09-10T22:10:39.522775+0000","last_became_active":"2024-09-10T22:10:37.236322+0000","last_became_peered":"2024-09-10T22:10:37.236322+0000","last_unstale":"2024-09-10T22:10:39.522775+0000","last_undegraded":"2024-09-10T22:10:39.522775+0000","last_fullsized":"2024-09-10T22:10:39.522775+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:48.453353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528630+0000","last_change":"2024-09-10T22:10:38.263553+0000","last_active":"2024-09-10T22:10:39.528630+0000","last_peered":"2024-09-10T22:10:39.528630+0000","last_clean":"2024-09-10T22:10:39.528630+0000","last_became_active":"2024-09-10T22:10:38.263302+0000","last_became_peered":"2024-09-10T22:10:38.263302+0000","last_unstale":"2024-09-10T22:10:39.528630+0000","last_undegraded":"2024-09-10T22:10:39.528630+0000","last_fullsized":"2024-09-10T22:10:39.528630+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:08.130137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528811+0000","last_change":"2024-09-10T22:10:37.249927+0000","last_active":"2024-09-10T22:10:39.528811+0000","last_peered":"2024-09-10T22:10:39.528811+0000","last_clean":"2024-09-10T22:10:39.528811+0000","last_became_active":"2024-09-10T22:10:37.249443+0000","last_became_peered":"2024-09-10T22:10:37.249443+0000","last_unstale":"2024-09-10T22:10:39.528811+0000","last_undegraded":"2024-09-10T22:10:39.528811+0000","last_fullsized":"2024-09-10T22:10:39.528811+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:20.595899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524302+0000","last_change":"2024-09-10T22:10:38.249997+0000","last_active":"2024-09-10T22:10:39.524302+0000","last_peered":"2024-09-10T22:10:39.524302+0000","last_clean":"2024-09-10T22:10:39.524302+0000","last_became_active":"2024-09-10T22:10:38.249737+0000","last_became_peered":"2024-09-10T22:10:38.249737+0000","last_unstale":"2024-09-10T22:10:39.524302+0000","last_undegraded":"2024-09-10T22:10:39.524302+0000","last_fullsized":"2024-09-10T22:10:39.524302+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:19.948050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522780+0000","last_change":"2024-09-10T22:10:37.245541+0000","last_active":"2024-09-10T22:10:39.522780+0000","last_peered":"2024-09-10T22:10:39.522780+0000","last_clean":"2024-09-10T22:10:39.522780+0000","last_became_active":"2024-09-10T22:10:37.245066+0000","last_became_peered":"2024-09-10T22:10:37.245066+0000","last_unstale":"2024-09-10T22:10:39.522780+0000","last_undegraded":"2024-09-10T22:10:39.522780+0000","last_fullsized":"2024-09-10T22:10:39.522780+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:40.695789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254591+0000","last_change":"2024-09-10T22:10:38.262616+0000","last_active":"2024-09-10T22:10:39.254591+0000","last_peered":"2024-09-10T22:10:39.254591+0000","last_clean":"2024-09-10T22:10:39.254591+0000","last_became_active":"2024-09-10T22:10:38.260645+0000","last_became_peered":"2024-09-10T22:10:38.260645+0000","last_unstale":"2024-09-10T22:10:39.254591+0000","last_undegraded":"2024-09-10T22:10:39.254591+0000","last_fullsized":"2024-09-10T22:10:39.254591+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:41.698684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528261+0000","last_change":"2024-09-10T22:10:37.251944+0000","last_active":"2024-09-10T22:10:39.528261+0000","last_peered":"2024-09-10T22:10:39.528261+0000","last_clean":"2024-09-10T22:10:39.528261+0000","last_became_active":"2024-09-10T22:10:37.251665+0000","last_became_peered":"2024-09-10T22:10:37.251665+0000","last_unstale":"2024-09-10T22:10:39.528261+0000","last_undegraded":"2024-09-10T22:10:39.528261+0000","last_fullsized":"2024-09-10T22:10:39.528261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:55.320245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523396+0000","last_change":"2024-09-10T22:10:38.250847+0000","last_active":"2024-09-10T22:10:39.523396+0000","last_peered":"2024-09-10T22:10:39.523396+0000","last_clean":"2024-09-10T22:10:39.523396+0000","last_became_active":"2024-09-10T22:10:38.250591+0000","last_became_peered":"2024-09-10T22:10:38.250591+0000","last_unstale":"2024-09-10T22:10:39.523396+0000","last_undegraded":"2024-09-10T22:10:39.523396+0000","last_fullsized":"2024-09-10T22:10:39.523396+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:57.861382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247352+0000","last_change":"2024-09-10T22:10:37.249468+0000","last_active":"2024-09-10T22:10:39.247352+0000","last_peered":"2024-09-10T22:10:39.247352+0000","last_clean":"2024-09-10T22:10:39.247352+0000","last_became_active":"2024-09-10T22:10:37.249196+0000","last_became_peered":"2024-09-10T22:10:37.249196+0000","last_unstale":"2024-09-10T22:10:39.247352+0000","last_undegraded":"2024-09-10T22:10:39.247352+0000","last_fullsized":"2024-09-10T22:10:39.247352+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:48.453353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528272+0000","last_change":"2024-09-10T22:10:38.263051+0000","last_active":"2024-09-10T22:10:39.528272+0000","last_peered":"2024-09-10T22:10:39.528272+0000","last_clean":"2024-09-10T22:10:39.528272+0000","last_became_active":"2024-09-10T22:10:38.262760+0000","last_became_peered":"2024-09-10T22:10:38.262760+0000","last_unstale":"2024-09-10T22:10:39.528272+0000","last_undegraded":"2024-09-10T22:10:39.528272+0000","last_fullsized":"2024-09-10T22:10:39.528272+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:01.551745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523361+0000","last_change":"2024-09-10T22:10:37.254152+0000","last_active":"2024-09-10T22:10:39.523361+0000","last_peered":"2024-09-10T22:10:39.523361+0000","last_clean":"2024-09-10T22:10:39.523361+0000","last_became_active":"2024-09-10T22:10:37.253879+0000","last_became_peered":"2024-09-10T22:10:37.253879+0000","last_unstale":"2024-09-10T22:10:39.523361+0000","last_undegraded":"2024-09-10T22:10:39.523361+0000","last_fullsized":"2024-09-10T22:10:39.523361+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:29.614570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524110+0000","last_change":"2024-09-10T22:10:38.259332+0000","last_active":"2024-09-10T22:10:39.524110+0000","last_peered":"2024-09-10T22:10:39.524110+0000","last_clean":"2024-09-10T22:10:39.524110+0000","last_became_active":"2024-09-10T22:10:38.255536+0000","last_became_peered":"2024-09-10T22:10:38.255536+0000","last_unstale":"2024-09-10T22:10:39.524110+0000","last_undegraded":"2024-09-10T22:10:39.524110+0000","last_fullsized":"2024-09-10T22:10:39.524110+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:52.177586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.252402+0000","last_change":"2024-09-10T22:10:37.249681+0000","last_active":"2024-09-10T22:10:37.252402+0000","last_peered":"2024-09-10T22:10:37.252402+0000","last_clean":"2024-09-10T22:10:37.252402+0000","last_became_active":"2024-09-10T22:10:37.249376+0000","last_became_peered":"2024-09-10T22:10:37.249376+0000","last_unstale":"2024-09-10T22:10:37.252402+0000","last_undegraded":"2024-09-10T22:10:37.252402+0000","last_fullsized":"2024-09-10T22:10:37.252402+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:00.548850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":256,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.244716+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:10:37.244716+0000","last_peered":"2024-09-10T22:10:37.244716+0000","last_clean":"2024-09-10T22:10:37.244716+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:10:37.244716+0000","last_undegraded":"2024-09-10T22:10:37.244716+0000","last_fullsized":"2024-09-10T22:10:37.244716+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509566,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123120,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248720,"statfs":{"total":96636764160,"available":96510689280,"internally_reserved":0,"allocated":1798144,"data_stored":1108825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509568,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130896,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240944,"statfs":{"total":96636764160,"available":96502726656,"internally_reserved":0,"allocated":1236992,"data_stored":532296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509567,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64272,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307568,"statfs":{"total":96636764160,"available":96570949632,"internally_reserved":0,"allocated":1232896,"data_stored":531714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509568,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144800,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227040,"statfs":{"total":96636764160,"available":96488488960,"internally_reserved":0,"allocated":1249280,"data_stored":533380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509566,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122580,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249260,"statfs":{"total":96636764160,"available":96511242240,"internally_reserved":0,"allocated":1245184,"data_stored":533278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509567,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126732,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245108,"statfs":{"total":96636764160,"available":96506990592,"internally_reserved":0,"allocated":1228800,"data_stored":531644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509567,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123148,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248692,"statfs":{"total":96636764160,"available":96510660608,"internally_reserved":0,"allocated":1826816,"data_stored":1122594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":124254665},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542272,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144776,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227064,"statfs":{"total":96636764160,"available":96488513536,"internally_reserved":0,"allocated":1232896,"data_stored":531678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:10:42.802 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:10:42.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.984+0000 7f36a5200640 1 -- 172.21.15.66:0/3206431332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a00afc20 msgr2=0x7f36a00ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.984+0000 7f36a5200640 1 --2- 172.21.15.66:0/3206431332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a00afc20 0x7f36a00ce880 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f3698009bc0 tx=0x7f369801cb10 comp rx=0 tx=0).stop 2024-09-10T22:10:42.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.984+0000 7f36a5200640 1 -- 172.21.15.66:0/3206431332 shutdown_connections 2024-09-10T22:10:42.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.984+0000 7f36a5200640 1 --2- 172.21.15.66:0/3206431332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a00afc20 0x7f36a00ce880 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.984+0000 7f36a5200640 1 -- 172.21.15.66:0/3206431332 >> 172.21.15.66:0/3206431332 conn(0x7f36a00d1c60 msgr2=0x7f36a00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:42.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.984+0000 7f36a5200640 1 -- 172.21.15.66:0/3206431332 shutdown_connections 2024-09-10T22:10:42.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.984+0000 7f36a5200640 1 -- 172.21.15.66:0/3206431332 wait complete. 2024-09-10T22:10:42.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f36a5200640 1 Processor -- start 2024-09-10T22:10:42.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f36a5200640 1 -- start start 2024-09-10T22:10:42.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f36a5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a00afc20 0x7f36a014e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:42.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f36a5200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36a00d06b0 0x7f36a00ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:42.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f36a5200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a015bd40 0x7f36a00caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:42.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f36a5200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f36a016a1b0 con 0x7f36a00afc20 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f36a5200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36a016a030 con 0x7f36a015bd40 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f36a5200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f36a016a330 con 0x7f36a00d06b0 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f369de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a00afc20 0x7f36a014e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f369de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a00afc20 0x7f36a014e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54296/0 (socket says 172.21.15.66:54296) 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f369de00640 1 -- 172.21.15.66:0/3007445657 learned_addr learned my addr 172.21.15.66:0/3007445657 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f369de00640 1 -- 172.21.15.66:0/3007445657 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36a00d06b0 msgr2=0x7f36a00ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f369de00640 1 --2- 172.21.15.66:0/3007445657 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36a00d06b0 0x7f36a00ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f369de00640 1 -- 172.21.15.66:0/3007445657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a015bd40 msgr2=0x7f36a00caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f369de00640 1 --2- 172.21.15.66:0/3007445657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a015bd40 0x7f36a00caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f369de00640 1 -- 172.21.15.66:0/3007445657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a006fbd0 con 0x7f36a00afc20 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f369de00640 1 --2- 172.21.15.66:0/3007445657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a00afc20 0x7f36a014e2d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f369000c3c0 tx=0x7f369000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f3696000640 1 -- 172.21.15.66:0/3007445657 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690017070 con 0x7f36a00afc20 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f3696000640 1 -- 172.21.15.66:0/3007445657 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36900053c0 con 0x7f36a00afc20 2024-09-10T22:10:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f3696000640 1 -- 172.21.15.66:0/3007445657 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690003860 con 0x7f36a00afc20 2024-09-10T22:10:42.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f36a5200640 1 -- 172.21.15.66:0/3007445657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a00c9740 con 0x7f36a00afc20 2024-09-10T22:10:42.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f36a5200640 1 -- 172.21.15.66:0/3007445657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a00c9d00 con 0x7f36a00afc20 2024-09-10T22:10:42.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.988+0000 7f36a5200640 1 -- 172.21.15.66:0/3007445657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a00cffa0 con 0x7f36a00afc20 2024-09-10T22:10:42.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.992+0000 7f3696000640 1 -- 172.21.15.66:0/3007445657 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f3690003a00 con 0x7f36a00afc20 2024-09-10T22:10:42.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.992+0000 7f3696000640 1 --2- 172.21.15.66:0/3007445657 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f367c073c00 0x7f367c0760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:42.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.992+0000 7f3696000640 1 -- 172.21.15.66:0/3007445657 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f36900919d0 con 0x7f36a00afc20 2024-09-10T22:10:42.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.992+0000 7f3697e00640 1 --2- 172.21.15.66:0/3007445657 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f367c073c00 0x7f367c0760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:42.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.992+0000 7f3697e00640 1 --2- 172.21.15.66:0/3007445657 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f367c073c00 0x7f367c0760c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f36a015b690 tx=0x7f3688007a00 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:42.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:42.996+0000 7f3696000640 1 -- 172.21.15.66:0/3007445657 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f369005e730 con 0x7f36a00afc20 2024-09-10T22:10:43.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.168+0000 7f36a5200640 1 -- 172.21.15.66:0/3007445657 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36a0076150 con 0x7f367c073c00 2024-09-10T22:10:43.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.172+0000 7f3696000640 1 -- 172.21.15.66:0/3007445657 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251440 (secure 0 0 0) 0x7f36a0076150 con 0x7f367c073c00 2024-09-10T22:10:43.178 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:43.179 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:10:43.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.176+0000 7f36a5200640 1 -- 172.21.15.66:0/3007445657 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f367c073c00 msgr2=0x7f367c0760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:43.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.176+0000 7f36a5200640 1 --2- 172.21.15.66:0/3007445657 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f367c073c00 0x7f367c0760c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f36a015b690 tx=0x7f3688007a00 comp rx=0 tx=0).stop 2024-09-10T22:10:43.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.176+0000 7f36a5200640 1 -- 172.21.15.66:0/3007445657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a00afc20 msgr2=0x7f36a014e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:43.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.176+0000 7f36a5200640 1 --2- 172.21.15.66:0/3007445657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a00afc20 0x7f36a014e2d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f369000c3c0 tx=0x7f369000c880 comp rx=0 tx=0).stop 2024-09-10T22:10:43.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.176+0000 7f36a5200640 1 -- 172.21.15.66:0/3007445657 shutdown_connections 2024-09-10T22:10:43.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.176+0000 7f36a5200640 1 --2- 172.21.15.66:0/3007445657 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f367c073c00 0x7f367c0760c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.176+0000 7f36a5200640 1 --2- 172.21.15.66:0/3007445657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a015bd40 0x7f36a00caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.176+0000 7f36a5200640 1 --2- 172.21.15.66:0/3007445657 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36a00d06b0 0x7f36a00ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.176+0000 7f36a5200640 1 --2- 172.21.15.66:0/3007445657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36a00afc20 0x7f36a014e2d0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.176+0000 7f36a5200640 1 -- 172.21.15.66:0/3007445657 >> 172.21.15.66:0/3007445657 conn(0x7f36a00d1c60 msgr2=0x7f36a00b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:43.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.176+0000 7f36a5200640 1 -- 172.21.15.66:0/3007445657 shutdown_connections 2024-09-10T22:10:43.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.176+0000 7f36a5200640 1 -- 172.21.15.66:0/3007445657 wait complete. 2024-09-10T22:10:43.199 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-10T22:10:42.542883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":980324,"kb_used_data":10792,"kb_used_omap":12,"kb_used_meta":969331,"kb_avail":753994396,"statfs":{"total":773094113280,"available":772090261504,"internally_reserved":0,"allocated":11051008,"data_stored":5425409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":992595508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001610"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381580+0000","last_change":"2024-09-10T22:10:38.251267+0000","last_active":"2024-09-10T22:10:39.381580+0000","last_peered":"2024-09-10T22:10:39.381580+0000","last_clean":"2024-09-10T22:10:39.381580+0000","last_became_active":"2024-09-10T22:10:38.250933+0000","last_became_peered":"2024-09-10T22:10:38.250933+0000","last_unstale":"2024-09-10T22:10:39.381580+0000","last_undegraded":"2024-09-10T22:10:39.381580+0000","last_fullsized":"2024-09-10T22:10:39.381580+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:41.184652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523102+0000","last_change":"2024-09-10T22:10:38.251283+0000","last_active":"2024-09-10T22:10:39.523102+0000","last_peered":"2024-09-10T22:10:39.523102+0000","last_clean":"2024-09-10T22:10:39.523102+0000","last_became_active":"2024-09-10T22:10:38.250986+0000","last_became_peered":"2024-09-10T22:10:38.250986+0000","last_unstale":"2024-09-10T22:10:39.523102+0000","last_undegraded":"2024-09-10T22:10:39.523102+0000","last_fullsized":"2024-09-10T22:10:39.523102+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:02.661632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381540+0000","last_change":"2024-09-10T22:10:38.253355+0000","last_active":"2024-09-10T22:10:39.381540+0000","last_peered":"2024-09-10T22:10:39.381540+0000","last_clean":"2024-09-10T22:10:39.381540+0000","last_became_active":"2024-09-10T22:10:38.253088+0000","last_became_peered":"2024-09-10T22:10:38.253088+0000","last_unstale":"2024-09-10T22:10:39.381540+0000","last_undegraded":"2024-09-10T22:10:39.381540+0000","last_fullsized":"2024-09-10T22:10:39.381540+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:18.638299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247441+0000","last_change":"2024-09-10T22:10:38.252302+0000","last_active":"2024-09-10T22:10:39.247441+0000","last_peered":"2024-09-10T22:10:39.247441+0000","last_clean":"2024-09-10T22:10:39.247441+0000","last_became_active":"2024-09-10T22:10:38.252084+0000","last_became_peered":"2024-09-10T22:10:38.252084+0000","last_unstale":"2024-09-10T22:10:39.247441+0000","last_undegraded":"2024-09-10T22:10:39.247441+0000","last_fullsized":"2024-09-10T22:10:39.247441+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:47.247838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528491+0000","last_change":"2024-09-10T22:10:38.263805+0000","last_active":"2024-09-10T22:10:39.528491+0000","last_peered":"2024-09-10T22:10:39.528491+0000","last_clean":"2024-09-10T22:10:39.528491+0000","last_became_active":"2024-09-10T22:10:38.263535+0000","last_became_peered":"2024-09-10T22:10:38.263535+0000","last_unstale":"2024-09-10T22:10:39.528491+0000","last_undegraded":"2024-09-10T22:10:39.528491+0000","last_fullsized":"2024-09-10T22:10:39.528491+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:26.055062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.241724+0000","last_change":"2024-09-10T22:10:37.241724+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.241724+0000","last_undegraded":"2024-09-10T22:10:37.241724+0000","last_fullsized":"2024-09-10T22:10:37.241724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240036+0000","last_change":"2024-09-10T22:10:37.240036+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240036+0000","last_undegraded":"2024-09-10T22:10:37.240036+0000","last_fullsized":"2024-09-10T22:10:37.240036+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522599+0000","last_change":"2024-09-10T22:10:38.254564+0000","last_active":"2024-09-10T22:10:39.522599+0000","last_peered":"2024-09-10T22:10:39.522599+0000","last_clean":"2024-09-10T22:10:39.522599+0000","last_became_active":"2024-09-10T22:10:38.254280+0000","last_became_peered":"2024-09-10T22:10:38.254280+0000","last_unstale":"2024-09-10T22:10:39.522599+0000","last_undegraded":"2024-09-10T22:10:39.522599+0000","last_fullsized":"2024-09-10T22:10:39.522599+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:48.733204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.255200+0000","last_change":"2024-09-10T22:10:38.249346+0000","last_active":"2024-09-10T22:10:39.255200+0000","last_peered":"2024-09-10T22:10:39.255200+0000","last_clean":"2024-09-10T22:10:39.255200+0000","last_became_active":"2024-09-10T22:10:38.249101+0000","last_became_peered":"2024-09-10T22:10:38.249101+0000","last_unstale":"2024-09-10T22:10:39.255200+0000","last_undegraded":"2024-09-10T22:10:39.255200+0000","last_fullsized":"2024-09-10T22:10:39.255200+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:38.972161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528751+0000","last_change":"2024-09-10T22:10:38.266215+0000","last_active":"2024-09-10T22:10:39.528751+0000","last_peered":"2024-09-10T22:10:39.528751+0000","last_clean":"2024-09-10T22:10:39.528751+0000","last_became_active":"2024-09-10T22:10:38.264693+0000","last_became_peered":"2024-09-10T22:10:38.264693+0000","last_unstale":"2024-09-10T22:10:39.528751+0000","last_undegraded":"2024-09-10T22:10:39.528751+0000","last_fullsized":"2024-09-10T22:10:39.528751+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:09.177869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.243036+0000","last_change":"2024-09-10T22:10:37.243036+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.243036+0000","last_undegraded":"2024-09-10T22:10:37.243036+0000","last_fullsized":"2024-09-10T22:10:37.243036+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254568+0000","last_change":"2024-09-10T22:10:38.247539+0000","last_active":"2024-09-10T22:10:39.254568+0000","last_peered":"2024-09-10T22:10:39.254568+0000","last_clean":"2024-09-10T22:10:39.254568+0000","last_became_active":"2024-09-10T22:10:38.247124+0000","last_became_peered":"2024-09-10T22:10:38.247124+0000","last_unstale":"2024-09-10T22:10:39.254568+0000","last_undegraded":"2024-09-10T22:10:39.254568+0000","last_fullsized":"2024-09-10T22:10:39.254568+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:26.055062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247126+0000","last_change":"2024-09-10T22:10:38.243927+0000","last_active":"2024-09-10T22:10:39.247126+0000","last_peered":"2024-09-10T22:10:39.247126+0000","last_clean":"2024-09-10T22:10:39.247126+0000","last_became_active":"2024-09-10T22:10:38.240651+0000","last_became_peered":"2024-09-10T22:10:38.240651+0000","last_unstale":"2024-09-10T22:10:39.247126+0000","last_undegraded":"2024-09-10T22:10:39.247126+0000","last_fullsized":"2024-09-10T22:10:39.247126+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:34.748065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523481+0000","last_change":"2024-09-10T22:10:38.264327+0000","last_active":"2024-09-10T22:10:39.523481+0000","last_peered":"2024-09-10T22:10:39.523481+0000","last_clean":"2024-09-10T22:10:39.523481+0000","last_became_active":"2024-09-10T22:10:38.264014+0000","last_became_peered":"2024-09-10T22:10:38.264014+0000","last_unstale":"2024-09-10T22:10:39.523481+0000","last_undegraded":"2024-09-10T22:10:39.523481+0000","last_fullsized":"2024-09-10T22:10:39.523481+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:24.533086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247264+0000","last_change":"2024-09-10T22:10:38.244207+0000","last_active":"2024-09-10T22:10:39.247264+0000","last_peered":"2024-09-10T22:10:39.247264+0000","last_clean":"2024-09-10T22:10:39.247264+0000","last_became_active":"2024-09-10T22:10:38.243891+0000","last_became_peered":"2024-09-10T22:10:38.243891+0000","last_unstale":"2024-09-10T22:10:39.247264+0000","last_undegraded":"2024-09-10T22:10:39.247264+0000","last_fullsized":"2024-09-10T22:10:39.247264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:39.927846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240030+0000","last_change":"2024-09-10T22:10:37.240030+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240030+0000","last_undegraded":"2024-09-10T22:10:37.240030+0000","last_fullsized":"2024-09-10T22:10:37.240030+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523341+0000","last_change":"2024-09-10T22:10:38.243673+0000","last_active":"2024-09-10T22:10:39.523341+0000","last_peered":"2024-09-10T22:10:39.523341+0000","last_clean":"2024-09-10T22:10:39.523341+0000","last_became_active":"2024-09-10T22:10:38.243416+0000","last_became_peered":"2024-09-10T22:10:38.243416+0000","last_unstale":"2024-09-10T22:10:39.523341+0000","last_undegraded":"2024-09-10T22:10:39.523341+0000","last_fullsized":"2024-09-10T22:10:39.523341+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:28.506115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247572+0000","last_change":"2024-09-10T22:10:38.255081+0000","last_active":"2024-09-10T22:10:39.247572+0000","last_peered":"2024-09-10T22:10:39.247572+0000","last_clean":"2024-09-10T22:10:39.247572+0000","last_became_active":"2024-09-10T22:10:38.254659+0000","last_became_peered":"2024-09-10T22:10:38.254659+0000","last_unstale":"2024-09-10T22:10:39.247572+0000","last_undegraded":"2024-09-10T22:10:39.247572+0000","last_fullsized":"2024-09-10T22:10:39.247572+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:23.035694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524131+0000","last_change":"2024-09-10T22:10:38.247997+0000","last_active":"2024-09-10T22:10:39.524131+0000","last_peered":"2024-09-10T22:10:39.524131+0000","last_clean":"2024-09-10T22:10:39.524131+0000","last_became_active":"2024-09-10T22:10:38.247730+0000","last_became_peered":"2024-09-10T22:10:38.247730+0000","last_unstale":"2024-09-10T22:10:39.524131+0000","last_undegraded":"2024-09-10T22:10:39.524131+0000","last_fullsized":"2024-09-10T22:10:39.524131+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:32.488690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254657+0000","last_change":"2024-09-10T22:10:38.245564+0000","last_active":"2024-09-10T22:10:39.254657+0000","last_peered":"2024-09-10T22:10:39.254657+0000","last_clean":"2024-09-10T22:10:39.254657+0000","last_became_active":"2024-09-10T22:10:38.245271+0000","last_became_peered":"2024-09-10T22:10:38.245271+0000","last_unstale":"2024-09-10T22:10:39.254657+0000","last_undegraded":"2024-09-10T22:10:39.254657+0000","last_fullsized":"2024-09-10T22:10:39.254657+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:26.491631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523720+0000","last_change":"2024-09-10T22:10:38.250433+0000","last_active":"2024-09-10T22:10:39.523720+0000","last_peered":"2024-09-10T22:10:39.523720+0000","last_clean":"2024-09-10T22:10:39.523720+0000","last_became_active":"2024-09-10T22:10:38.250154+0000","last_became_peered":"2024-09-10T22:10:38.250154+0000","last_unstale":"2024-09-10T22:10:39.523720+0000","last_undegraded":"2024-09-10T22:10:39.523720+0000","last_fullsized":"2024-09-10T22:10:39.523720+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:32.858119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522651+0000","last_change":"2024-09-10T22:10:38.245066+0000","last_active":"2024-09-10T22:10:39.522651+0000","last_peered":"2024-09-10T22:10:39.522651+0000","last_clean":"2024-09-10T22:10:39.522651+0000","last_became_active":"2024-09-10T22:10:38.244768+0000","last_became_peered":"2024-09-10T22:10:38.244768+0000","last_unstale":"2024-09-10T22:10:39.522651+0000","last_undegraded":"2024-09-10T22:10:39.522651+0000","last_fullsized":"2024-09-10T22:10:39.522651+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:44.888191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522566+0000","last_change":"2024-09-10T22:10:38.250358+0000","last_active":"2024-09-10T22:10:39.522566+0000","last_peered":"2024-09-10T22:10:39.522566+0000","last_clean":"2024-09-10T22:10:39.522566+0000","last_became_active":"2024-09-10T22:10:38.250103+0000","last_became_peered":"2024-09-10T22:10:38.250103+0000","last_unstale":"2024-09-10T22:10:39.522566+0000","last_undegraded":"2024-09-10T22:10:39.522566+0000","last_fullsized":"2024-09-10T22:10:39.522566+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:35.427777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.239464+0000","last_change":"2024-09-10T22:10:37.239464+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.239464+0000","last_undegraded":"2024-09-10T22:10:37.239464+0000","last_fullsized":"2024-09-10T22:10:37.239464+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528701+0000","last_change":"2024-09-10T22:10:38.269614+0000","last_active":"2024-09-10T22:10:39.528701+0000","last_peered":"2024-09-10T22:10:39.528701+0000","last_clean":"2024-09-10T22:10:39.528701+0000","last_became_active":"2024-09-10T22:10:38.269330+0000","last_became_peered":"2024-09-10T22:10:38.269330+0000","last_unstale":"2024-09-10T22:10:39.528701+0000","last_undegraded":"2024-09-10T22:10:39.528701+0000","last_fullsized":"2024-09-10T22:10:39.528701+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:38.972161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523101+0000","last_change":"2024-09-10T22:10:38.247355+0000","last_active":"2024-09-10T22:10:39.523101+0000","last_peered":"2024-09-10T22:10:39.523101+0000","last_clean":"2024-09-10T22:10:39.523101+0000","last_became_active":"2024-09-10T22:10:38.246782+0000","last_became_peered":"2024-09-10T22:10:38.246782+0000","last_unstale":"2024-09-10T22:10:39.523101+0000","last_undegraded":"2024-09-10T22:10:39.523101+0000","last_fullsized":"2024-09-10T22:10:39.523101+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:57.493480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.239961+0000","last_change":"2024-09-10T22:10:37.239961+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.239961+0000","last_undegraded":"2024-09-10T22:10:37.239961+0000","last_fullsized":"2024-09-10T22:10:37.239961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528635+0000","last_change":"2024-09-10T22:10:38.270949+0000","last_active":"2024-09-10T22:10:39.528635+0000","last_peered":"2024-09-10T22:10:39.528635+0000","last_clean":"2024-09-10T22:10:39.528635+0000","last_became_active":"2024-09-10T22:10:38.270580+0000","last_became_peered":"2024-09-10T22:10:38.270580+0000","last_unstale":"2024-09-10T22:10:39.528635+0000","last_undegraded":"2024-09-10T22:10:39.528635+0000","last_fullsized":"2024-09-10T22:10:39.528635+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:46.669269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522776+0000","last_change":"2024-09-10T22:10:38.244297+0000","last_active":"2024-09-10T22:10:39.522776+0000","last_peered":"2024-09-10T22:10:39.522776+0000","last_clean":"2024-09-10T22:10:39.522776+0000","last_became_active":"2024-09-10T22:10:38.243953+0000","last_became_peered":"2024-09-10T22:10:38.243953+0000","last_unstale":"2024-09-10T22:10:39.522776+0000","last_undegraded":"2024-09-10T22:10:39.522776+0000","last_fullsized":"2024-09-10T22:10:39.522776+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:08.221611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523686+0000","last_change":"2024-09-10T22:10:38.247381+0000","last_active":"2024-09-10T22:10:39.523686+0000","last_peered":"2024-09-10T22:10:39.523686+0000","last_clean":"2024-09-10T22:10:39.523686+0000","last_became_active":"2024-09-10T22:10:38.247092+0000","last_became_peered":"2024-09-10T22:10:38.247092+0000","last_unstale":"2024-09-10T22:10:39.523686+0000","last_undegraded":"2024-09-10T22:10:39.523686+0000","last_fullsized":"2024-09-10T22:10:39.523686+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:04.052881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247281+0000","last_change":"2024-09-10T22:10:38.262682+0000","last_active":"2024-09-10T22:10:39.247281+0000","last_peered":"2024-09-10T22:10:39.247281+0000","last_clean":"2024-09-10T22:10:39.247281+0000","last_became_active":"2024-09-10T22:10:38.262426+0000","last_became_peered":"2024-09-10T22:10:38.262426+0000","last_unstale":"2024-09-10T22:10:39.247281+0000","last_undegraded":"2024-09-10T22:10:39.247281+0000","last_fullsized":"2024-09-10T22:10:39.247281+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:16.836613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381139+0000","last_change":"2024-09-10T22:10:38.248324+0000","last_active":"2024-09-10T22:10:39.381139+0000","last_peered":"2024-09-10T22:10:39.381139+0000","last_clean":"2024-09-10T22:10:39.381139+0000","last_became_active":"2024-09-10T22:10:38.245453+0000","last_became_peered":"2024-09-10T22:10:38.245453+0000","last_unstale":"2024-09-10T22:10:39.381139+0000","last_undegraded":"2024-09-10T22:10:39.381139+0000","last_fullsized":"2024-09-10T22:10:39.381139+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:22.865604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524095+0000","last_change":"2024-09-10T22:10:38.255400+0000","last_active":"2024-09-10T22:10:39.524095+0000","last_peered":"2024-09-10T22:10:39.524095+0000","last_clean":"2024-09-10T22:10:39.524095+0000","last_became_active":"2024-09-10T22:10:38.255069+0000","last_became_peered":"2024-09-10T22:10:38.255069+0000","last_unstale":"2024-09-10T22:10:39.524095+0000","last_undegraded":"2024-09-10T22:10:39.524095+0000","last_fullsized":"2024-09-10T22:10:39.524095+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:32.770140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"99'4","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523313+0000","last_change":"2024-09-10T22:10:37.247494+0000","last_active":"2024-09-10T22:10:39.523313+0000","last_peered":"2024-09-10T22:10:39.523313+0000","last_clean":"2024-09-10T22:10:39.523313+0000","last_became_active":"2024-09-10T22:10:37.244676+0000","last_became_peered":"2024-09-10T22:10:37.244676+0000","last_unstale":"2024-09-10T22:10:39.523313+0000","last_undegraded":"2024-09-10T22:10:39.523313+0000","last_fullsized":"2024-09-10T22:10:39.523313+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:25.917947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247778+0000","last_change":"2024-09-10T22:10:38.263771+0000","last_active":"2024-09-10T22:10:39.247778+0000","last_peered":"2024-09-10T22:10:39.247778+0000","last_clean":"2024-09-10T22:10:39.247778+0000","last_became_active":"2024-09-10T22:10:38.263458+0000","last_became_peered":"2024-09-10T22:10:38.263458+0000","last_unstale":"2024-09-10T22:10:39.247778+0000","last_undegraded":"2024-09-10T22:10:39.247778+0000","last_fullsized":"2024-09-10T22:10:39.247778+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:48.091865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.251427+0000","last_change":"2024-09-10T22:10:37.248597+0000","last_active":"2024-09-10T22:10:37.251427+0000","last_peered":"2024-09-10T22:10:37.251427+0000","last_clean":"2024-09-10T22:10:37.251427+0000","last_became_active":"2024-09-10T22:10:37.248297+0000","last_became_peered":"2024-09-10T22:10:37.248297+0000","last_unstale":"2024-09-10T22:10:37.251427+0000","last_undegraded":"2024-09-10T22:10:37.251427+0000","last_fullsized":"2024-09-10T22:10:37.251427+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:56.601971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381440+0000","last_change":"2024-09-10T22:10:38.241783+0000","last_active":"2024-09-10T22:10:39.381440+0000","last_peered":"2024-09-10T22:10:39.381440+0000","last_clean":"2024-09-10T22:10:39.381440+0000","last_became_active":"2024-09-10T22:10:38.241478+0000","last_became_peered":"2024-09-10T22:10:38.241478+0000","last_unstale":"2024-09-10T22:10:39.381440+0000","last_undegraded":"2024-09-10T22:10:39.381440+0000","last_fullsized":"2024-09-10T22:10:39.381440+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:37.074694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522717+0000","last_change":"2024-09-10T22:10:37.249272+0000","last_active":"2024-09-10T22:10:39.522717+0000","last_peered":"2024-09-10T22:10:39.522717+0000","last_clean":"2024-09-10T22:10:39.522717+0000","last_became_active":"2024-09-10T22:10:37.248969+0000","last_became_peered":"2024-09-10T22:10:37.248969+0000","last_unstale":"2024-09-10T22:10:39.522717+0000","last_undegraded":"2024-09-10T22:10:39.522717+0000","last_fullsized":"2024-09-10T22:10:39.522717+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:00.267217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.255113+0000","last_change":"2024-09-10T22:10:38.247440+0000","last_active":"2024-09-10T22:10:39.255113+0000","last_peered":"2024-09-10T22:10:39.255113+0000","last_clean":"2024-09-10T22:10:39.255113+0000","last_became_active":"2024-09-10T22:10:38.243369+0000","last_became_peered":"2024-09-10T22:10:38.243369+0000","last_unstale":"2024-09-10T22:10:39.255113+0000","last_undegraded":"2024-09-10T22:10:39.255113+0000","last_fullsized":"2024-09-10T22:10:39.255113+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:31.048301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522825+0000","last_change":"2024-09-10T22:10:37.235026+0000","last_active":"2024-09-10T22:10:39.522825+0000","last_peered":"2024-09-10T22:10:39.522825+0000","last_clean":"2024-09-10T22:10:39.522825+0000","last_became_active":"2024-09-10T22:10:37.234835+0000","last_became_peered":"2024-09-10T22:10:37.234835+0000","last_unstale":"2024-09-10T22:10:39.522825+0000","last_undegraded":"2024-09-10T22:10:39.522825+0000","last_fullsized":"2024-09-10T22:10:39.522825+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:39.992087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252383+0000","last_change":"2024-09-10T22:10:38.250119+0000","last_active":"2024-09-10T22:10:39.252383+0000","last_peered":"2024-09-10T22:10:39.252383+0000","last_clean":"2024-09-10T22:10:39.252383+0000","last_became_active":"2024-09-10T22:10:38.249748+0000","last_became_peered":"2024-09-10T22:10:38.249748+0000","last_unstale":"2024-09-10T22:10:39.252383+0000","last_undegraded":"2024-09-10T22:10:39.252383+0000","last_fullsized":"2024-09-10T22:10:39.252383+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:33.757624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381382+0000","last_change":"2024-09-10T22:10:37.247963+0000","last_active":"2024-09-10T22:10:39.381382+0000","last_peered":"2024-09-10T22:10:39.381382+0000","last_clean":"2024-09-10T22:10:39.381382+0000","last_became_active":"2024-09-10T22:10:37.246784+0000","last_became_peered":"2024-09-10T22:10:37.246784+0000","last_unstale":"2024-09-10T22:10:39.381382+0000","last_undegraded":"2024-09-10T22:10:39.381382+0000","last_fullsized":"2024-09-10T22:10:39.381382+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:33.104121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.241495+0000","last_change":"2024-09-10T22:10:37.241495+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.241495+0000","last_undegraded":"2024-09-10T22:10:37.241495+0000","last_fullsized":"2024-09-10T22:10:37.241495+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522451+0000","last_change":"2024-09-10T22:10:37.240295+0000","last_active":"2024-09-10T22:10:39.522451+0000","last_peered":"2024-09-10T22:10:39.522451+0000","last_clean":"2024-09-10T22:10:39.522451+0000","last_became_active":"2024-09-10T22:10:37.240014+0000","last_became_peered":"2024-09-10T22:10:37.240014+0000","last_unstale":"2024-09-10T22:10:39.522451+0000","last_undegraded":"2024-09-10T22:10:39.522451+0000","last_fullsized":"2024-09-10T22:10:39.522451+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:19.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247084+0000","last_change":"2024-09-10T22:10:38.245203+0000","last_active":"2024-09-10T22:10:39.247084+0000","last_peered":"2024-09-10T22:10:39.247084+0000","last_clean":"2024-09-10T22:10:39.247084+0000","last_became_active":"2024-09-10T22:10:38.244911+0000","last_became_peered":"2024-09-10T22:10:38.244911+0000","last_unstale":"2024-09-10T22:10:39.247084+0000","last_undegraded":"2024-09-10T22:10:39.247084+0000","last_fullsized":"2024-09-10T22:10:39.247084+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:19.820527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528261+0000","last_change":"2024-09-10T22:10:37.258021+0000","last_active":"2024-09-10T22:10:39.528261+0000","last_peered":"2024-09-10T22:10:39.528261+0000","last_clean":"2024-09-10T22:10:39.528261+0000","last_became_active":"2024-09-10T22:10:37.257766+0000","last_became_peered":"2024-09-10T22:10:37.257766+0000","last_unstale":"2024-09-10T22:10:39.528261+0000","last_undegraded":"2024-09-10T22:10:39.528261+0000","last_fullsized":"2024-09-10T22:10:39.528261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:36.653416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254271+0000","last_change":"2024-09-10T22:10:38.243002+0000","last_active":"2024-09-10T22:10:39.254271+0000","last_peered":"2024-09-10T22:10:39.254271+0000","last_clean":"2024-09-10T22:10:39.254271+0000","last_became_active":"2024-09-10T22:10:38.241629+0000","last_became_peered":"2024-09-10T22:10:38.241629+0000","last_unstale":"2024-09-10T22:10:39.254271+0000","last_undegraded":"2024-09-10T22:10:39.254271+0000","last_fullsized":"2024-09-10T22:10:39.254271+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:21.178257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381135+0000","last_change":"2024-09-10T22:10:37.247160+0000","last_active":"2024-09-10T22:10:39.381135+0000","last_peered":"2024-09-10T22:10:39.381135+0000","last_clean":"2024-09-10T22:10:39.381135+0000","last_became_active":"2024-09-10T22:10:37.246865+0000","last_became_peered":"2024-09-10T22:10:37.246865+0000","last_unstale":"2024-09-10T22:10:39.381135+0000","last_undegraded":"2024-09-10T22:10:39.381135+0000","last_fullsized":"2024-09-10T22:10:39.381135+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:54.381121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523577+0000","last_change":"2024-09-10T22:10:38.257125+0000","last_active":"2024-09-10T22:10:39.523577+0000","last_peered":"2024-09-10T22:10:39.523577+0000","last_clean":"2024-09-10T22:10:39.523577+0000","last_became_active":"2024-09-10T22:10:38.256821+0000","last_became_peered":"2024-09-10T22:10:38.256821+0000","last_unstale":"2024-09-10T22:10:39.523577+0000","last_undegraded":"2024-09-10T22:10:39.523577+0000","last_fullsized":"2024-09-10T22:10:39.523577+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:13.792877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381703+0000","last_change":"2024-09-10T22:10:37.246423+0000","last_active":"2024-09-10T22:10:39.381703+0000","last_peered":"2024-09-10T22:10:39.381703+0000","last_clean":"2024-09-10T22:10:39.381703+0000","last_became_active":"2024-09-10T22:10:37.246139+0000","last_became_peered":"2024-09-10T22:10:37.246139+0000","last_unstale":"2024-09-10T22:10:39.381703+0000","last_undegraded":"2024-09-10T22:10:39.381703+0000","last_fullsized":"2024-09-10T22:10:39.381703+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:00.685355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.255201+0000","last_change":"2024-09-10T22:10:38.253930+0000","last_active":"2024-09-10T22:10:39.255201+0000","last_peered":"2024-09-10T22:10:39.255201+0000","last_clean":"2024-09-10T22:10:39.255201+0000","last_became_active":"2024-09-10T22:10:38.253264+0000","last_became_peered":"2024-09-10T22:10:38.253264+0000","last_unstale":"2024-09-10T22:10:39.255201+0000","last_undegraded":"2024-09-10T22:10:39.255201+0000","last_fullsized":"2024-09-10T22:10:39.255201+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:19.820527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247473+0000","last_change":"2024-09-10T22:10:37.254160+0000","last_active":"2024-09-10T22:10:39.247473+0000","last_peered":"2024-09-10T22:10:39.247473+0000","last_clean":"2024-09-10T22:10:39.247473+0000","last_became_active":"2024-09-10T22:10:37.253831+0000","last_became_peered":"2024-09-10T22:10:37.253831+0000","last_unstale":"2024-09-10T22:10:39.247473+0000","last_undegraded":"2024-09-10T22:10:39.247473+0000","last_fullsized":"2024-09-10T22:10:39.247473+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:00.267217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528676+0000","last_change":"2024-09-10T22:10:38.266948+0000","last_active":"2024-09-10T22:10:39.528676+0000","last_peered":"2024-09-10T22:10:39.528676+0000","last_clean":"2024-09-10T22:10:39.528676+0000","last_became_active":"2024-09-10T22:10:38.266677+0000","last_became_peered":"2024-09-10T22:10:38.266677+0000","last_unstale":"2024-09-10T22:10:39.528676+0000","last_undegraded":"2024-09-10T22:10:39.528676+0000","last_fullsized":"2024-09-10T22:10:39.528676+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:08.232358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.253794+0000","last_change":"2024-09-10T22:10:37.252591+0000","last_active":"2024-09-10T22:10:37.253794+0000","last_peered":"2024-09-10T22:10:37.253794+0000","last_clean":"2024-09-10T22:10:37.253794+0000","last_became_active":"2024-09-10T22:10:37.252236+0000","last_became_peered":"2024-09-10T22:10:37.252236+0000","last_unstale":"2024-09-10T22:10:37.253794+0000","last_undegraded":"2024-09-10T22:10:37.253794+0000","last_fullsized":"2024-09-10T22:10:37.253794+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:25.052167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.242629+0000","last_change":"2024-09-10T22:10:37.242629+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.242629+0000","last_undegraded":"2024-09-10T22:10:37.242629+0000","last_fullsized":"2024-09-10T22:10:37.242629+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"99'6","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254866+0000","last_change":"2024-09-10T22:10:37.245595+0000","last_active":"2024-09-10T22:10:39.254866+0000","last_peered":"2024-09-10T22:10:39.254866+0000","last_clean":"2024-09-10T22:10:39.254866+0000","last_became_active":"2024-09-10T22:10:37.245237+0000","last_became_peered":"2024-09-10T22:10:37.245237+0000","last_unstale":"2024-09-10T22:10:39.254866+0000","last_undegraded":"2024-09-10T22:10:39.254866+0000","last_fullsized":"2024-09-10T22:10:39.254866+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:17.957594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247188+0000","last_change":"2024-09-10T22:10:38.261146+0000","last_active":"2024-09-10T22:10:39.247188+0000","last_peered":"2024-09-10T22:10:39.247188+0000","last_clean":"2024-09-10T22:10:39.247188+0000","last_became_active":"2024-09-10T22:10:38.260713+0000","last_became_peered":"2024-09-10T22:10:38.260713+0000","last_unstale":"2024-09-10T22:10:39.247188+0000","last_undegraded":"2024-09-10T22:10:39.247188+0000","last_fullsized":"2024-09-10T22:10:39.247188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:20.279287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523954+0000","last_change":"2024-09-10T22:10:37.247682+0000","last_active":"2024-09-10T22:10:39.523954+0000","last_peered":"2024-09-10T22:10:39.523954+0000","last_clean":"2024-09-10T22:10:39.523954+0000","last_became_active":"2024-09-10T22:10:37.247416+0000","last_became_peered":"2024-09-10T22:10:37.247416+0000","last_unstale":"2024-09-10T22:10:39.523954+0000","last_undegraded":"2024-09-10T22:10:39.523954+0000","last_fullsized":"2024-09-10T22:10:39.523954+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:32.804016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381286+0000","last_change":"2024-09-10T22:10:38.248180+0000","last_active":"2024-09-10T22:10:39.381286+0000","last_peered":"2024-09-10T22:10:39.381286+0000","last_clean":"2024-09-10T22:10:39.381286+0000","last_became_active":"2024-09-10T22:10:38.247917+0000","last_became_peered":"2024-09-10T22:10:38.247917+0000","last_unstale":"2024-09-10T22:10:39.381286+0000","last_undegraded":"2024-09-10T22:10:39.381286+0000","last_fullsized":"2024-09-10T22:10:39.381286+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:45.132830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523823+0000","last_change":"2024-09-10T22:10:37.251530+0000","last_active":"2024-09-10T22:10:39.523823+0000","last_peered":"2024-09-10T22:10:39.523823+0000","last_clean":"2024-09-10T22:10:39.523823+0000","last_became_active":"2024-09-10T22:10:37.251337+0000","last_became_peered":"2024-09-10T22:10:37.251337+0000","last_unstale":"2024-09-10T22:10:39.523823+0000","last_undegraded":"2024-09-10T22:10:39.523823+0000","last_fullsized":"2024-09-10T22:10:39.523823+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:30.617743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254115+0000","last_change":"2024-09-10T22:10:38.246234+0000","last_active":"2024-09-10T22:10:39.254115+0000","last_peered":"2024-09-10T22:10:39.254115+0000","last_clean":"2024-09-10T22:10:39.254115+0000","last_became_active":"2024-09-10T22:10:38.241936+0000","last_became_peered":"2024-09-10T22:10:38.241936+0000","last_unstale":"2024-09-10T22:10:39.254115+0000","last_undegraded":"2024-09-10T22:10:39.254115+0000","last_fullsized":"2024-09-10T22:10:39.254115+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:40.994982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254261+0000","last_change":"2024-09-10T22:10:37.248956+0000","last_active":"2024-09-10T22:10:39.254261+0000","last_peered":"2024-09-10T22:10:39.254261+0000","last_clean":"2024-09-10T22:10:39.254261+0000","last_became_active":"2024-09-10T22:10:37.248634+0000","last_became_peered":"2024-09-10T22:10:37.248634+0000","last_unstale":"2024-09-10T22:10:39.254261+0000","last_undegraded":"2024-09-10T22:10:39.254261+0000","last_fullsized":"2024-09-10T22:10:39.254261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:23.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523489+0000","last_change":"2024-09-10T22:10:38.248859+0000","last_active":"2024-09-10T22:10:39.523489+0000","last_peered":"2024-09-10T22:10:39.523489+0000","last_clean":"2024-09-10T22:10:39.523489+0000","last_became_active":"2024-09-10T22:10:38.247750+0000","last_became_peered":"2024-09-10T22:10:38.247750+0000","last_unstale":"2024-09-10T22:10:39.523489+0000","last_undegraded":"2024-09-10T22:10:39.523489+0000","last_fullsized":"2024-09-10T22:10:39.523489+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:09.599932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"99'8","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381272+0000","last_change":"2024-09-10T22:10:37.241895+0000","last_active":"2024-09-10T22:10:39.381272+0000","last_peered":"2024-09-10T22:10:39.381272+0000","last_clean":"2024-09-10T22:10:39.381272+0000","last_became_active":"2024-09-10T22:10:37.241654+0000","last_became_peered":"2024-09-10T22:10:37.241654+0000","last_unstale":"2024-09-10T22:10:39.381272+0000","last_undegraded":"2024-09-10T22:10:39.381272+0000","last_fullsized":"2024-09-10T22:10:39.381272+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:10.220978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528948+0000","last_change":"2024-09-10T22:10:38.269389+0000","last_active":"2024-09-10T22:10:39.528948+0000","last_peered":"2024-09-10T22:10:39.528948+0000","last_clean":"2024-09-10T22:10:39.528948+0000","last_became_active":"2024-09-10T22:10:38.269130+0000","last_became_peered":"2024-09-10T22:10:38.269130+0000","last_unstale":"2024-09-10T22:10:39.528948+0000","last_undegraded":"2024-09-10T22:10:39.528948+0000","last_fullsized":"2024-09-10T22:10:39.528948+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:10.005594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254954+0000","last_change":"2024-09-10T22:10:37.254551+0000","last_active":"2024-09-10T22:10:39.254954+0000","last_peered":"2024-09-10T22:10:39.254954+0000","last_clean":"2024-09-10T22:10:39.254954+0000","last_became_active":"2024-09-10T22:10:37.254311+0000","last_became_peered":"2024-09-10T22:10:37.254311+0000","last_unstale":"2024-09-10T22:10:39.254954+0000","last_undegraded":"2024-09-10T22:10:39.254954+0000","last_fullsized":"2024-09-10T22:10:39.254954+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:25.574209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252554+0000","last_change":"2024-09-10T22:10:38.265801+0000","last_active":"2024-09-10T22:10:39.252554+0000","last_peered":"2024-09-10T22:10:39.252554+0000","last_clean":"2024-09-10T22:10:39.252554+0000","last_became_active":"2024-09-10T22:10:38.265549+0000","last_became_peered":"2024-09-10T22:10:38.265549+0000","last_unstale":"2024-09-10T22:10:39.252554+0000","last_undegraded":"2024-09-10T22:10:39.252554+0000","last_fullsized":"2024-09-10T22:10:39.252554+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:49.342939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528666+0000","last_change":"2024-09-10T22:10:37.249193+0000","last_active":"2024-09-10T22:10:39.528666+0000","last_peered":"2024-09-10T22:10:39.528666+0000","last_clean":"2024-09-10T22:10:39.528666+0000","last_became_active":"2024-09-10T22:10:37.248911+0000","last_became_peered":"2024-09-10T22:10:37.248911+0000","last_unstale":"2024-09-10T22:10:39.528666+0000","last_undegraded":"2024-09-10T22:10:39.528666+0000","last_fullsized":"2024-09-10T22:10:39.528666+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:02.732555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240468+0000","last_change":"2024-09-10T22:10:37.240468+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240468+0000","last_undegraded":"2024-09-10T22:10:37.240468+0000","last_fullsized":"2024-09-10T22:10:37.240468+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522890+0000","last_change":"2024-09-10T22:10:37.234553+0000","last_active":"2024-09-10T22:10:39.522890+0000","last_peered":"2024-09-10T22:10:39.522890+0000","last_clean":"2024-09-10T22:10:39.522890+0000","last_became_active":"2024-09-10T22:10:37.234278+0000","last_became_peered":"2024-09-10T22:10:37.234278+0000","last_unstale":"2024-09-10T22:10:39.522890+0000","last_undegraded":"2024-09-10T22:10:39.522890+0000","last_fullsized":"2024-09-10T22:10:39.522890+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:20.175362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523714+0000","last_change":"2024-09-10T22:10:38.256468+0000","last_active":"2024-09-10T22:10:39.523714+0000","last_peered":"2024-09-10T22:10:39.523714+0000","last_clean":"2024-09-10T22:10:39.523714+0000","last_became_active":"2024-09-10T22:10:38.256224+0000","last_became_peered":"2024-09-10T22:10:38.256224+0000","last_unstale":"2024-09-10T22:10:39.523714+0000","last_undegraded":"2024-09-10T22:10:39.523714+0000","last_fullsized":"2024-09-10T22:10:39.523714+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:23.765455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522649+0000","last_change":"2024-09-10T22:10:37.249141+0000","last_active":"2024-09-10T22:10:39.522649+0000","last_peered":"2024-09-10T22:10:39.522649+0000","last_clean":"2024-09-10T22:10:39.522649+0000","last_became_active":"2024-09-10T22:10:37.248865+0000","last_became_peered":"2024-09-10T22:10:37.248865+0000","last_unstale":"2024-09-10T22:10:39.522649+0000","last_undegraded":"2024-09-10T22:10:39.522649+0000","last_fullsized":"2024-09-10T22:10:39.522649+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:16.473063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523961+0000","last_change":"2024-09-10T22:10:38.247400+0000","last_active":"2024-09-10T22:10:39.523961+0000","last_peered":"2024-09-10T22:10:39.523961+0000","last_clean":"2024-09-10T22:10:39.523961+0000","last_became_active":"2024-09-10T22:10:38.244941+0000","last_became_peered":"2024-09-10T22:10:38.244941+0000","last_unstale":"2024-09-10T22:10:39.523961+0000","last_undegraded":"2024-09-10T22:10:39.523961+0000","last_fullsized":"2024-09-10T22:10:39.523961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:18.374331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252406+0000","last_change":"2024-09-10T22:10:37.246142+0000","last_active":"2024-09-10T22:10:39.252406+0000","last_peered":"2024-09-10T22:10:39.252406+0000","last_clean":"2024-09-10T22:10:39.252406+0000","last_became_active":"2024-09-10T22:10:37.245696+0000","last_became_peered":"2024-09-10T22:10:37.245696+0000","last_unstale":"2024-09-10T22:10:39.252406+0000","last_undegraded":"2024-09-10T22:10:39.252406+0000","last_fullsized":"2024-09-10T22:10:39.252406+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:23.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523257+0000","last_change":"2024-09-10T22:10:38.257969+0000","last_active":"2024-09-10T22:10:39.523257+0000","last_peered":"2024-09-10T22:10:39.523257+0000","last_clean":"2024-09-10T22:10:39.523257+0000","last_became_active":"2024-09-10T22:10:38.255428+0000","last_became_peered":"2024-09-10T22:10:38.255428+0000","last_unstale":"2024-09-10T22:10:39.523257+0000","last_undegraded":"2024-09-10T22:10:39.523257+0000","last_fullsized":"2024-09-10T22:10:39.523257+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:23.507403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252252+0000","last_change":"2024-09-10T22:10:37.249833+0000","last_active":"2024-09-10T22:10:39.252252+0000","last_peered":"2024-09-10T22:10:39.252252+0000","last_clean":"2024-09-10T22:10:39.252252+0000","last_became_active":"2024-09-10T22:10:37.248940+0000","last_became_peered":"2024-09-10T22:10:37.248940+0000","last_unstale":"2024-09-10T22:10:39.252252+0000","last_undegraded":"2024-09-10T22:10:39.252252+0000","last_fullsized":"2024-09-10T22:10:39.252252+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:17.957594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527472+0000","last_change":"2024-09-10T22:10:38.260839+0000","last_active":"2024-09-10T22:10:39.527472+0000","last_peered":"2024-09-10T22:10:39.527472+0000","last_clean":"2024-09-10T22:10:39.527472+0000","last_became_active":"2024-09-10T22:10:38.260199+0000","last_became_peered":"2024-09-10T22:10:38.260199+0000","last_unstale":"2024-09-10T22:10:39.527472+0000","last_undegraded":"2024-09-10T22:10:39.527472+0000","last_fullsized":"2024-09-10T22:10:39.527472+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:57.604867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522526+0000","last_change":"2024-09-10T22:10:37.248217+0000","last_active":"2024-09-10T22:10:39.522526+0000","last_peered":"2024-09-10T22:10:39.522526+0000","last_clean":"2024-09-10T22:10:39.522526+0000","last_became_active":"2024-09-10T22:10:37.247946+0000","last_became_peered":"2024-09-10T22:10:37.247946+0000","last_unstale":"2024-09-10T22:10:39.522526+0000","last_undegraded":"2024-09-10T22:10:39.522526+0000","last_fullsized":"2024-09-10T22:10:39.522526+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:25.488736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523362+0000","last_change":"2024-09-10T22:10:38.251544+0000","last_active":"2024-09-10T22:10:39.523362+0000","last_peered":"2024-09-10T22:10:39.523362+0000","last_clean":"2024-09-10T22:10:39.523362+0000","last_became_active":"2024-09-10T22:10:38.251270+0000","last_became_peered":"2024-09-10T22:10:38.251270+0000","last_unstale":"2024-09-10T22:10:39.523362+0000","last_undegraded":"2024-09-10T22:10:39.523362+0000","last_fullsized":"2024-09-10T22:10:39.523362+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:58.242589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252755+0000","last_change":"2024-09-10T22:10:37.248132+0000","last_active":"2024-09-10T22:10:39.252755+0000","last_peered":"2024-09-10T22:10:39.252755+0000","last_clean":"2024-09-10T22:10:39.252755+0000","last_became_active":"2024-09-10T22:10:37.247855+0000","last_became_peered":"2024-09-10T22:10:37.247855+0000","last_unstale":"2024-09-10T22:10:39.252755+0000","last_undegraded":"2024-09-10T22:10:39.252755+0000","last_fullsized":"2024-09-10T22:10:39.252755+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:44.140728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381589+0000","last_change":"2024-09-10T22:10:38.251921+0000","last_active":"2024-09-10T22:10:39.381589+0000","last_peered":"2024-09-10T22:10:39.381589+0000","last_clean":"2024-09-10T22:10:39.381589+0000","last_became_active":"2024-09-10T22:10:38.251620+0000","last_became_peered":"2024-09-10T22:10:38.251620+0000","last_unstale":"2024-09-10T22:10:39.381589+0000","last_undegraded":"2024-09-10T22:10:39.381589+0000","last_fullsized":"2024-09-10T22:10:39.381589+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:25.497370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"100'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523469+0000","last_change":"2024-09-10T22:10:37.253726+0000","last_active":"2024-09-10T22:10:39.523469+0000","last_peered":"2024-09-10T22:10:39.523469+0000","last_clean":"2024-09-10T22:10:39.523469+0000","last_became_active":"2024-09-10T22:10:37.252258+0000","last_became_peered":"2024-09-10T22:10:37.252258+0000","last_unstale":"2024-09-10T22:10:39.523469+0000","last_undegraded":"2024-09-10T22:10:39.523469+0000","last_fullsized":"2024-09-10T22:10:39.523469+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:30.045406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381513+0000","last_change":"2024-09-10T22:10:38.261993+0000","last_active":"2024-09-10T22:10:39.381513+0000","last_peered":"2024-09-10T22:10:39.381513+0000","last_clean":"2024-09-10T22:10:39.381513+0000","last_became_active":"2024-09-10T22:10:38.261706+0000","last_became_peered":"2024-09-10T22:10:38.261706+0000","last_unstale":"2024-09-10T22:10:39.381513+0000","last_undegraded":"2024-09-10T22:10:39.381513+0000","last_fullsized":"2024-09-10T22:10:39.381513+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:27.116503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522775+0000","last_change":"2024-09-10T22:10:37.236693+0000","last_active":"2024-09-10T22:10:39.522775+0000","last_peered":"2024-09-10T22:10:39.522775+0000","last_clean":"2024-09-10T22:10:39.522775+0000","last_became_active":"2024-09-10T22:10:37.236322+0000","last_became_peered":"2024-09-10T22:10:37.236322+0000","last_unstale":"2024-09-10T22:10:39.522775+0000","last_undegraded":"2024-09-10T22:10:39.522775+0000","last_fullsized":"2024-09-10T22:10:39.522775+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:48.453353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528630+0000","last_change":"2024-09-10T22:10:38.263553+0000","last_active":"2024-09-10T22:10:39.528630+0000","last_peered":"2024-09-10T22:10:39.528630+0000","last_clean":"2024-09-10T22:10:39.528630+0000","last_became_active":"2024-09-10T22:10:38.263302+0000","last_became_peered":"2024-09-10T22:10:38.263302+0000","last_unstale":"2024-09-10T22:10:39.528630+0000","last_undegraded":"2024-09-10T22:10:39.528630+0000","last_fullsized":"2024-09-10T22:10:39.528630+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:08.130137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528811+0000","last_change":"2024-09-10T22:10:37.249927+0000","last_active":"2024-09-10T22:10:39.528811+0000","last_peered":"2024-09-10T22:10:39.528811+0000","last_clean":"2024-09-10T22:10:39.528811+0000","last_became_active":"2024-09-10T22:10:37.249443+0000","last_became_peered":"2024-09-10T22:10:37.249443+0000","last_unstale":"2024-09-10T22:10:39.528811+0000","last_undegraded":"2024-09-10T22:10:39.528811+0000","last_fullsized":"2024-09-10T22:10:39.528811+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:20.595899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524302+0000","last_change":"2024-09-10T22:10:38.249997+0000","last_active":"2024-09-10T22:10:39.524302+0000","last_peered":"2024-09-10T22:10:39.524302+0000","last_clean":"2024-09-10T22:10:39.524302+0000","last_became_active":"2024-09-10T22:10:38.249737+0000","last_became_peered":"2024-09-10T22:10:38.249737+0000","last_unstale":"2024-09-10T22:10:39.524302+0000","last_undegraded":"2024-09-10T22:10:39.524302+0000","last_fullsized":"2024-09-10T22:10:39.524302+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:19.948050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522780+0000","last_change":"2024-09-10T22:10:37.245541+0000","last_active":"2024-09-10T22:10:39.522780+0000","last_peered":"2024-09-10T22:10:39.522780+0000","last_clean":"2024-09-10T22:10:39.522780+0000","last_became_active":"2024-09-10T22:10:37.245066+0000","last_became_peered":"2024-09-10T22:10:37.245066+0000","last_unstale":"2024-09-10T22:10:39.522780+0000","last_undegraded":"2024-09-10T22:10:39.522780+0000","last_fullsized":"2024-09-10T22:10:39.522780+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:40.695789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254591+0000","last_change":"2024-09-10T22:10:38.262616+0000","last_active":"2024-09-10T22:10:39.254591+0000","last_peered":"2024-09-10T22:10:39.254591+0000","last_clean":"2024-09-10T22:10:39.254591+0000","last_became_active":"2024-09-10T22:10:38.260645+0000","last_became_peered":"2024-09-10T22:10:38.260645+0000","last_unstale":"2024-09-10T22:10:39.254591+0000","last_undegraded":"2024-09-10T22:10:39.254591+0000","last_fullsized":"2024-09-10T22:10:39.254591+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:41.698684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528261+0000","last_change":"2024-09-10T22:10:37.251944+0000","last_active":"2024-09-10T22:10:39.528261+0000","last_peered":"2024-09-10T22:10:39.528261+0000","last_clean":"2024-09-10T22:10:39.528261+0000","last_became_active":"2024-09-10T22:10:37.251665+0000","last_became_peered":"2024-09-10T22:10:37.251665+0000","last_unstale":"2024-09-10T22:10:39.528261+0000","last_undegraded":"2024-09-10T22:10:39.528261+0000","last_fullsized":"2024-09-10T22:10:39.528261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:55.320245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523396+0000","last_change":"2024-09-10T22:10:38.250847+0000","last_active":"2024-09-10T22:10:39.523396+0000","last_peered":"2024-09-10T22:10:39.523396+0000","last_clean":"2024-09-10T22:10:39.523396+0000","last_became_active":"2024-09-10T22:10:38.250591+0000","last_became_peered":"2024-09-10T22:10:38.250591+0000","last_unstale":"2024-09-10T22:10:39.523396+0000","last_undegraded":"2024-09-10T22:10:39.523396+0000","last_fullsized":"2024-09-10T22:10:39.523396+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:57.861382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247352+0000","last_change":"2024-09-10T22:10:37.249468+0000","last_active":"2024-09-10T22:10:39.247352+0000","last_peered":"2024-09-10T22:10:39.247352+0000","last_clean":"2024-09-10T22:10:39.247352+0000","last_became_active":"2024-09-10T22:10:37.249196+0000","last_became_peered":"2024-09-10T22:10:37.249196+0000","last_unstale":"2024-09-10T22:10:39.247352+0000","last_undegraded":"2024-09-10T22:10:39.247352+0000","last_fullsized":"2024-09-10T22:10:39.247352+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:48.453353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528272+0000","last_change":"2024-09-10T22:10:38.263051+0000","last_active":"2024-09-10T22:10:39.528272+0000","last_peered":"2024-09-10T22:10:39.528272+0000","last_clean":"2024-09-10T22:10:39.528272+0000","last_became_active":"2024-09-10T22:10:38.262760+0000","last_became_peered":"2024-09-10T22:10:38.262760+0000","last_unstale":"2024-09-10T22:10:39.528272+0000","last_undegraded":"2024-09-10T22:10:39.528272+0000","last_fullsized":"2024-09-10T22:10:39.528272+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:01.551745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523361+0000","last_change":"2024-09-10T22:10:37.254152+0000","last_active":"2024-09-10T22:10:39.523361+0000","last_peered":"2024-09-10T22:10:39.523361+0000","last_clean":"2024-09-10T22:10:39.523361+0000","last_became_active":"2024-09-10T22:10:37.253879+0000","last_became_peered":"2024-09-10T22:10:37.253879+0000","last_unstale":"2024-09-10T22:10:39.523361+0000","last_undegraded":"2024-09-10T22:10:39.523361+0000","last_fullsized":"2024-09-10T22:10:39.523361+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:29.614570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524110+0000","last_change":"2024-09-10T22:10:38.259332+0000","last_active":"2024-09-10T22:10:39.524110+0000","last_peered":"2024-09-10T22:10:39.524110+0000","last_clean":"2024-09-10T22:10:39.524110+0000","last_became_active":"2024-09-10T22:10:38.255536+0000","last_became_peered":"2024-09-10T22:10:38.255536+0000","last_unstale":"2024-09-10T22:10:39.524110+0000","last_undegraded":"2024-09-10T22:10:39.524110+0000","last_fullsized":"2024-09-10T22:10:39.524110+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:52.177586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.252402+0000","last_change":"2024-09-10T22:10:37.249681+0000","last_active":"2024-09-10T22:10:37.252402+0000","last_peered":"2024-09-10T22:10:37.252402+0000","last_clean":"2024-09-10T22:10:37.252402+0000","last_became_active":"2024-09-10T22:10:37.249376+0000","last_became_peered":"2024-09-10T22:10:37.249376+0000","last_unstale":"2024-09-10T22:10:37.252402+0000","last_undegraded":"2024-09-10T22:10:37.252402+0000","last_fullsized":"2024-09-10T22:10:37.252402+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:00.548850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":256,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.244716+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:10:37.244716+0000","last_peered":"2024-09-10T22:10:37.244716+0000","last_clean":"2024-09-10T22:10:37.244716+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:10:37.244716+0000","last_undegraded":"2024-09-10T22:10:37.244716+0000","last_fullsized":"2024-09-10T22:10:37.244716+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509566,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123120,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248720,"statfs":{"total":96636764160,"available":96510689280,"internally_reserved":0,"allocated":1798144,"data_stored":1108825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509568,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130896,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240944,"statfs":{"total":96636764160,"available":96502726656,"internally_reserved":0,"allocated":1236992,"data_stored":532296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509567,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64272,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307568,"statfs":{"total":96636764160,"available":96570949632,"internally_reserved":0,"allocated":1232896,"data_stored":531714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509568,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144800,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227040,"statfs":{"total":96636764160,"available":96488488960,"internally_reserved":0,"allocated":1249280,"data_stored":533380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509566,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122580,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249260,"statfs":{"total":96636764160,"available":96511242240,"internally_reserved":0,"allocated":1245184,"data_stored":533278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509567,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126732,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245108,"statfs":{"total":96636764160,"available":96506990592,"internally_reserved":0,"allocated":1228800,"data_stored":531644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509567,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123148,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248692,"statfs":{"total":96636764160,"available":96510660608,"internally_reserved":0,"allocated":1826816,"data_stored":1122594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":124254665},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542272,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144776,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227064,"statfs":{"total":96636764160,"available":96488513536,"internally_reserved":0,"allocated":1232896,"data_stored":531678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:10:43.203 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3a is not active+clean 2024-09-10T22:10:43.203 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:10:37.241724+0000', 'last_change': '2024-09-10T22:10:37.241724+0000', 'last_active': '2024-09-10T22:10:37.219743+0000', 'last_peered': '2024-09-10T22:10:37.219743+0000', 'last_clean': '2024-09-10T22:10:37.219743+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:10:37.241724+0000', 'last_undegraded': '2024-09-10T22:10:37.241724+0000', 'last_fullsized': '2024-09-10T22:10:37.241724+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_clean_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:10:43.203 INFO:tasks.ceph.ceph_manager.ceph:PG 15.39 is not active+clean 2024-09-10T22:10:43.203 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:10:37.240036+0000', 'last_change': '2024-09-10T22:10:37.240036+0000', 'last_active': '2024-09-10T22:10:37.219743+0000', 'last_peered': '2024-09-10T22:10:37.219743+0000', 'last_clean': '2024-09-10T22:10:37.219743+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:10:37.240036+0000', 'last_undegraded': '2024-09-10T22:10:37.240036+0000', 'last_fullsized': '2024-09-10T22:10:37.240036+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_clean_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:10:43.203 INFO:tasks.ceph.ceph_manager.ceph:PG 15.35 is not active+clean 2024-09-10T22:10:43.203 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:10:37.243036+0000', 'last_change': '2024-09-10T22:10:37.243036+0000', 'last_active': '2024-09-10T22:10:37.219743+0000', 'last_peered': '2024-09-10T22:10:37.219743+0000', 'last_clean': '2024-09-10T22:10:37.219743+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:10:37.243036+0000', 'last_undegraded': '2024-09-10T22:10:37.243036+0000', 'last_fullsized': '2024-09-10T22:10:37.243036+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_clean_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:10:43.203 INFO:tasks.ceph.ceph_manager.ceph:PG 15.30 is not active+clean 2024-09-10T22:10:43.203 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:10:37.240030+0000', 'last_change': '2024-09-10T22:10:37.240030+0000', 'last_active': '2024-09-10T22:10:37.219743+0000', 'last_peered': '2024-09-10T22:10:37.219743+0000', 'last_clean': '2024-09-10T22:10:37.219743+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:10:37.240030+0000', 'last_undegraded': '2024-09-10T22:10:37.240030+0000', 'last_fullsized': '2024-09-10T22:10:37.240030+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_clean_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:10:43.203 INFO:tasks.ceph.ceph_manager.ceph:PG 15.28 is not active+clean 2024-09-10T22:10:43.204 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:10:37.239464+0000', 'last_change': '2024-09-10T22:10:37.239464+0000', 'last_active': '2024-09-10T22:10:37.219743+0000', 'last_peered': '2024-09-10T22:10:37.219743+0000', 'last_clean': '2024-09-10T22:10:37.219743+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:10:37.239464+0000', 'last_undegraded': '2024-09-10T22:10:37.239464+0000', 'last_fullsized': '2024-09-10T22:10:37.239464+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_clean_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:10:43.204 INFO:tasks.ceph.ceph_manager.ceph:PG 15.25 is not active+clean 2024-09-10T22:10:43.204 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:10:37.239961+0000', 'last_change': '2024-09-10T22:10:37.239961+0000', 'last_active': '2024-09-10T22:10:37.219743+0000', 'last_peered': '2024-09-10T22:10:37.219743+0000', 'last_clean': '2024-09-10T22:10:37.219743+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:10:37.239961+0000', 'last_undegraded': '2024-09-10T22:10:37.239961+0000', 'last_fullsized': '2024-09-10T22:10:37.239961+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_clean_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:10:43.204 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1b is not active+clean 2024-09-10T22:10:43.204 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:10:37.241495+0000', 'last_change': '2024-09-10T22:10:37.241495+0000', 'last_active': '2024-09-10T22:10:37.219743+0000', 'last_peered': '2024-09-10T22:10:37.219743+0000', 'last_clean': '2024-09-10T22:10:37.219743+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:10:37.241495+0000', 'last_undegraded': '2024-09-10T22:10:37.241495+0000', 'last_fullsized': '2024-09-10T22:10:37.241495+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_clean_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:10:43.204 INFO:tasks.ceph.ceph_manager.ceph:PG 15.15 is not active+clean 2024-09-10T22:10:43.204 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:10:37.242629+0000', 'last_change': '2024-09-10T22:10:37.242629+0000', 'last_active': '2024-09-10T22:10:37.219743+0000', 'last_peered': '2024-09-10T22:10:37.219743+0000', 'last_clean': '2024-09-10T22:10:37.219743+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:10:37.242629+0000', 'last_undegraded': '2024-09-10T22:10:37.242629+0000', 'last_fullsized': '2024-09-10T22:10:37.242629+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_clean_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:10:43.204 INFO:tasks.ceph.ceph_manager.ceph:PG 15.d is not active+clean 2024-09-10T22:10:43.204 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:10:37.240468+0000', 'last_change': '2024-09-10T22:10:37.240468+0000', 'last_active': '2024-09-10T22:10:37.219743+0000', 'last_peered': '2024-09-10T22:10:37.219743+0000', 'last_clean': '2024-09-10T22:10:37.219743+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:10:37.240468+0000', 'last_undegraded': '2024-09-10T22:10:37.240468+0000', 'last_fullsized': '2024-09-10T22:10:37.240468+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'last_clean_scrub_stamp': '2024-09-10T22:10:37.219743+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:10:43.205 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T22:10:43.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.384+0000 7f5e01600640 1 -- 172.21.15.66:0/2364623627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc15bd40 msgr2=0x7f5dfc0cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:43.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.384+0000 7f5e01600640 1 --2- 172.21.15.66:0/2364623627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc15bd40 0x7f5dfc0cffa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009840 tx=0x7f5dec01c790 comp rx=0 tx=0).stop 2024-09-10T22:10:43.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.384+0000 7f5e01600640 1 -- 172.21.15.66:0/2364623627 shutdown_connections 2024-09-10T22:10:43.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.384+0000 7f5e01600640 1 --2- 172.21.15.66:0/2364623627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc15bd40 0x7f5dfc0cffa0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.384+0000 7f5e01600640 1 -- 172.21.15.66:0/2364623627 >> 172.21.15.66:0/2364623627 conn(0x7f5dfc0d1c60 msgr2=0x7f5dfc0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:43.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.384+0000 7f5e01600640 1 -- 172.21.15.66:0/2364623627 shutdown_connections 2024-09-10T22:10:43.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.384+0000 7f5e01600640 1 -- 172.21.15.66:0/2364623627 wait complete. 2024-09-10T22:10:43.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.384+0000 7f5e01600640 1 Processor -- start 2024-09-10T22:10:43.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.384+0000 7f5e01600640 1 -- start start 2024-09-10T22:10:43.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5e01600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5dfc0cf180 0x7f5dfc14e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:43.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5e01600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc0cf830 0x7f5dfc06e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:43.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5e01600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5dfc15bd40 0x7f5dfc153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:43.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5e01600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc16a580 con 0x7f5dfc15bd40 2024-09-10T22:10:43.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5e01600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc16a400 con 0x7f5dfc0cf830 2024-09-10T22:10:43.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5e01600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5dfc16a700 con 0x7f5dfc0cf180 2024-09-10T22:10:43.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5dfb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5dfc15bd40 0x7f5dfc153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:43.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5dfb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5dfc15bd40 0x7f5dfc153210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54298/0 (socket says 172.21.15.66:54298) 2024-09-10T22:10:43.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5dfb400640 1 -- 172.21.15.66:0/3603546205 learned_addr learned my addr 172.21.15.66:0/3603546205 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:43.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5df2000640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc0cf830 0x7f5dfc06e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:43.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5dfaa00640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5dfc0cf180 0x7f5dfc14e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:43.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5dfb400640 1 -- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5dfc0cf180 msgr2=0x7f5dfc14e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:43.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5dfb400640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5dfc0cf180 0x7f5dfc14e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5dfb400640 1 -- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc0cf830 msgr2=0x7f5dfc06e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:43.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5dfb400640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc0cf830 0x7f5dfc06e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5dfb400640 1 -- 172.21.15.66:0/3603546205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc0b5db0 con 0x7f5dfc15bd40 2024-09-10T22:10:43.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5dfb400640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5dfc15bd40 0x7f5dfc153210 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5df400c790 tx=0x7f5df400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:43.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5df3400640 1 -- 172.21.15.66:0/3603546205 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4003b80 con 0x7f5dfc15bd40 2024-09-10T22:10:43.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5df3400640 1 -- 172.21.15.66:0/3603546205 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5df4003d20 con 0x7f5dfc15bd40 2024-09-10T22:10:43.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5df3400640 1 -- 172.21.15.66:0/3603546205 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df400aab0 con 0x7f5dfc15bd40 2024-09-10T22:10:43.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5e01600640 1 -- 172.21.15.66:0/3603546205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dfc0b4c30 con 0x7f5dfc15bd40 2024-09-10T22:10:43.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5e01600640 1 -- 172.21.15.66:0/3603546205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dfc0b51f0 con 0x7f5dfc15bd40 2024-09-10T22:10:43.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.388+0000 7f5e01600640 1 -- 172.21.15.66:0/3603546205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dfc0cffa0 con 0x7f5dfc15bd40 2024-09-10T22:10:43.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.392+0000 7f5df3400640 1 -- 172.21.15.66:0/3603546205 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f5df400ac50 con 0x7f5dfc15bd40 2024-09-10T22:10:43.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.392+0000 7f5df3400640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5dd8073bd0 0x7f5dd8076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:43.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.392+0000 7f5df3400640 1 -- 172.21.15.66:0/3603546205 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f5df40927f0 con 0x7f5dfc15bd40 2024-09-10T22:10:43.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.392+0000 7f5dfaa00640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5dd8073bd0 0x7f5dd8076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:43.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.396+0000 7f5dfaa00640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5dd8073bd0 0x7f5dd8076090 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0af000 tx=0x7f5dec01afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:43.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.396+0000 7f5df3400640 1 -- 172.21.15.66:0/3603546205 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5dfc0b5db0 con 0x7f5dfc15bd40 2024-09-10T22:10:43.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.628+0000 7f5e01600640 1 -- 172.21.15.66:0/3603546205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5dfc0b0df0 con 0x7f5dfc15bd40 2024-09-10T22:10:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.628+0000 7f5df3400640 1 -- 172.21.15.66:0/3603546205 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1144 (secure 0 0 0) 0x7f5df405f2d0 con 0x7f5dfc15bd40 2024-09-10T22:10:43.634 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:43.634 INFO:teuthology.orchestra.run.smithi066.stdout:{"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":636,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1726005621,"num_in_osds":8,"osd_in_since":1726005611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":88},{"state_name":"creating+peering","count":9}],"num_pgs":97,"num_pools":3,"num_objects":21,"data_bytes":592666,"bytes_used":1003851776,"bytes_avail":772090261504,"bytes_total":773094113280,"inactive_pgs_ratio":0.092783503234386444,"write_bytes_sec":2047,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":66,"btime":"2024-09-10T22:10:41:252812+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":7249}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-09-10T22:09:28.497849+0000","services":{}},"progress_events":{}} 2024-09-10T22:10:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.632+0000 7f5e01600640 1 -- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5dd8073bd0 msgr2=0x7f5dd8076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.632+0000 7f5e01600640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5dd8073bd0 0x7f5dd8076090 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0af000 tx=0x7f5dec01afd0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.632+0000 7f5e01600640 1 -- 172.21.15.66:0/3603546205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5dfc15bd40 msgr2=0x7f5dfc153210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:43.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.632+0000 7f5e01600640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5dfc15bd40 0x7f5dfc153210 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5df400c790 tx=0x7f5df400cc50 comp rx=0 tx=0).stop 2024-09-10T22:10:43.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.632+0000 7f5e01600640 1 -- 172.21.15.66:0/3603546205 shutdown_connections 2024-09-10T22:10:43.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.632+0000 7f5e01600640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5dd8073bd0 0x7f5dd8076090 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.632+0000 7f5e01600640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5dfc15bd40 0x7f5dfc153210 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.632+0000 7f5e01600640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc0cf830 0x7f5dfc06e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.632+0000 7f5e01600640 1 --2- 172.21.15.66:0/3603546205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5dfc0cf180 0x7f5dfc14e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.632+0000 7f5e01600640 1 -- 172.21.15.66:0/3603546205 >> 172.21.15.66:0/3603546205 conn(0x7f5dfc0d1c60 msgr2=0x7f5dfc154d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:43.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.632+0000 7f5e01600640 1 -- 172.21.15.66:0/3603546205 shutdown_connections 2024-09-10T22:10:43.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.632+0000 7f5e01600640 1 -- 172.21.15.66:0/3603546205 wait complete. 2024-09-10T22:10:43.656 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T22:10:43.657 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:10:43.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.800+0000 7f9a82e00640 1 -- 172.21.15.66:0/1188339845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a60003a60 msgr2=0x7f9a60003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:43.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.800+0000 7f9a82e00640 1 --2- 172.21.15.66:0/1188339845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a60003a60 0x7f9a60003e80 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c006fd0 tx=0x7f9a6c01f640 comp rx=0 tx=0).stop 2024-09-10T22:10:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.800+0000 7f9a82e00640 1 -- 172.21.15.66:0/1188339845 shutdown_connections 2024-09-10T22:10:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.800+0000 7f9a82e00640 1 --2- 172.21.15.66:0/1188339845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a60003a60 0x7f9a60003e80 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.800+0000 7f9a82e00640 1 -- 172.21.15.66:0/1188339845 >> 172.21.15.66:0/1188339845 conn(0x7f9a7c074240 msgr2=0x7f9a7c074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.800+0000 7f9a82e00640 1 -- 172.21.15.66:0/1188339845 shutdown_connections 2024-09-10T22:10:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.800+0000 7f9a82e00640 1 -- 172.21.15.66:0/1188339845 wait complete. 2024-09-10T22:10:43.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.800+0000 7f9a82e00640 1 Processor -- start 2024-09-10T22:10:43.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a82e00640 1 -- start start 2024-09-10T22:10:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a82e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a60003a60 0x7f9a7c1593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a82e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a7c0c5a90 0x7f9a7c0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a82e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a7c0c5f10 0x7f9a7c0b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a82e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c0ce230 con 0x7f9a60003a60 2024-09-10T22:10:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a82e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c0caeb0 con 0x7f9a7c0c5f10 2024-09-10T22:10:43.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a82e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9a7c0ce3b0 con 0x7f9a7c0c5a90 2024-09-10T22:10:43.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a7be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a60003a60 0x7f9a7c1593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:43.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a7be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a60003a60 0x7f9a7c1593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54310/0 (socket says 172.21.15.66:54310) 2024-09-10T22:10:43.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a7be00640 1 -- 172.21.15.66:0/4181602075 learned_addr learned my addr 172.21.15.66:0/4181602075 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:43.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a80e00640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a7c0c5f10 0x7f9a7c0b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:43.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a73400640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a7c0c5a90 0x7f9a7c0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a7be00640 1 -- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a7c0c5a90 msgr2=0x7f9a7c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a7be00640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a7c0c5a90 0x7f9a7c0b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a7be00640 1 -- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a7c0c5f10 msgr2=0x7f9a7c0b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a7be00640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a7c0c5f10 0x7f9a7c0b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a7be00640 1 -- 172.21.15.66:0/4181602075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c0c3e60 con 0x7f9a60003a60 2024-09-10T22:10:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a7be00640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a60003a60 0x7f9a7c1593d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0b1040 tx=0x7f9a6c003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a78c00640 1 -- 172.21.15.66:0/4181602075 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c006ca0 con 0x7f9a60003a60 2024-09-10T22:10:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a78c00640 1 -- 172.21.15.66:0/4181602075 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a6c0be070 con 0x7f9a60003a60 2024-09-10T22:10:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a82e00640 1 -- 172.21.15.66:0/4181602075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7c0c4140 con 0x7f9a60003a60 2024-09-10T22:10:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a78c00640 1 -- 172.21.15.66:0/4181602075 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c001d10 con 0x7f9a60003a60 2024-09-10T22:10:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a82e00640 1 -- 172.21.15.66:0/4181602075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7c0c4700 con 0x7f9a60003a60 2024-09-10T22:10:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.804+0000 7f9a82e00640 1 -- 172.21.15.66:0/4181602075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7c0ccd40 con 0x7f9a60003a60 2024-09-10T22:10:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.808+0000 7f9a78c00640 1 -- 172.21.15.66:0/4181602075 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f9a6c0c3460 con 0x7f9a60003a60 2024-09-10T22:10:43.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.808+0000 7f9a78c00640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a58073bd0 0x7f9a58076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:43.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.808+0000 7f9a78c00640 1 -- 172.21.15.66:0/4181602075 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f9a6c1300f0 con 0x7f9a60003a60 2024-09-10T22:10:43.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.808+0000 7f9a73400640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a58073bd0 0x7f9a58076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:43.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.808+0000 7f9a73400640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a58073bd0 0x7f9a58076090 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9a64000c10 tx=0x7f9a64006b80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:43.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.812+0000 7f9a78c00640 1 -- 172.21.15.66:0/4181602075 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9a6c0193f0 con 0x7f9a60003a60 2024-09-10T22:10:43.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.980+0000 7f9a82e00640 1 -- 172.21.15.66:0/4181602075 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a7c06de30 con 0x7f9a58073bd0 2024-09-10T22:10:43.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.984+0000 7f9a78c00640 1 -- 172.21.15.66:0/4181602075 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251440 (secure 0 0 0) 0x7f9a7c06de30 con 0x7f9a58073bd0 2024-09-10T22:10:43.992 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:43.993 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:10:43.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.988+0000 7f9a82e00640 1 -- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a58073bd0 msgr2=0x7f9a58076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:43.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.988+0000 7f9a82e00640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a58073bd0 0x7f9a58076090 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9a64000c10 tx=0x7f9a64006b80 comp rx=0 tx=0).stop 2024-09-10T22:10:43.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.988+0000 7f9a82e00640 1 -- 172.21.15.66:0/4181602075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a60003a60 msgr2=0x7f9a7c1593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.988+0000 7f9a82e00640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a60003a60 0x7f9a7c1593d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0b1040 tx=0x7f9a6c003c60 comp rx=0 tx=0).stop 2024-09-10T22:10:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.988+0000 7f9a82e00640 1 -- 172.21.15.66:0/4181602075 shutdown_connections 2024-09-10T22:10:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.988+0000 7f9a82e00640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a58073bd0 0x7f9a58076090 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.988+0000 7f9a82e00640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a7c0c5f10 0x7f9a7c0b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.988+0000 7f9a82e00640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a7c0c5a90 0x7f9a7c0b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.988+0000 7f9a82e00640 1 --2- 172.21.15.66:0/4181602075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a60003a60 0x7f9a7c1593d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.988+0000 7f9a82e00640 1 -- 172.21.15.66:0/4181602075 >> 172.21.15.66:0/4181602075 conn(0x7f9a7c074240 msgr2=0x7f9a7c0b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.988+0000 7f9a82e00640 1 -- 172.21.15.66:0/4181602075 shutdown_connections 2024-09-10T22:10:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:43.988+0000 7f9a82e00640 1 -- 172.21.15.66:0/4181602075 wait complete. 2024-09-10T22:10:44.014 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-10T22:10:42.542883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":980324,"kb_used_data":10792,"kb_used_omap":12,"kb_used_meta":969331,"kb_avail":753994396,"statfs":{"total":773094113280,"available":772090261504,"internally_reserved":0,"allocated":11051008,"data_stored":5425409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":992595508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001610"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381580+0000","last_change":"2024-09-10T22:10:38.251267+0000","last_active":"2024-09-10T22:10:39.381580+0000","last_peered":"2024-09-10T22:10:39.381580+0000","last_clean":"2024-09-10T22:10:39.381580+0000","last_became_active":"2024-09-10T22:10:38.250933+0000","last_became_peered":"2024-09-10T22:10:38.250933+0000","last_unstale":"2024-09-10T22:10:39.381580+0000","last_undegraded":"2024-09-10T22:10:39.381580+0000","last_fullsized":"2024-09-10T22:10:39.381580+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:41.184652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523102+0000","last_change":"2024-09-10T22:10:38.251283+0000","last_active":"2024-09-10T22:10:39.523102+0000","last_peered":"2024-09-10T22:10:39.523102+0000","last_clean":"2024-09-10T22:10:39.523102+0000","last_became_active":"2024-09-10T22:10:38.250986+0000","last_became_peered":"2024-09-10T22:10:38.250986+0000","last_unstale":"2024-09-10T22:10:39.523102+0000","last_undegraded":"2024-09-10T22:10:39.523102+0000","last_fullsized":"2024-09-10T22:10:39.523102+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:02.661632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381540+0000","last_change":"2024-09-10T22:10:38.253355+0000","last_active":"2024-09-10T22:10:39.381540+0000","last_peered":"2024-09-10T22:10:39.381540+0000","last_clean":"2024-09-10T22:10:39.381540+0000","last_became_active":"2024-09-10T22:10:38.253088+0000","last_became_peered":"2024-09-10T22:10:38.253088+0000","last_unstale":"2024-09-10T22:10:39.381540+0000","last_undegraded":"2024-09-10T22:10:39.381540+0000","last_fullsized":"2024-09-10T22:10:39.381540+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:18.638299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247441+0000","last_change":"2024-09-10T22:10:38.252302+0000","last_active":"2024-09-10T22:10:39.247441+0000","last_peered":"2024-09-10T22:10:39.247441+0000","last_clean":"2024-09-10T22:10:39.247441+0000","last_became_active":"2024-09-10T22:10:38.252084+0000","last_became_peered":"2024-09-10T22:10:38.252084+0000","last_unstale":"2024-09-10T22:10:39.247441+0000","last_undegraded":"2024-09-10T22:10:39.247441+0000","last_fullsized":"2024-09-10T22:10:39.247441+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:47.247838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528491+0000","last_change":"2024-09-10T22:10:38.263805+0000","last_active":"2024-09-10T22:10:39.528491+0000","last_peered":"2024-09-10T22:10:39.528491+0000","last_clean":"2024-09-10T22:10:39.528491+0000","last_became_active":"2024-09-10T22:10:38.263535+0000","last_became_peered":"2024-09-10T22:10:38.263535+0000","last_unstale":"2024-09-10T22:10:39.528491+0000","last_undegraded":"2024-09-10T22:10:39.528491+0000","last_fullsized":"2024-09-10T22:10:39.528491+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:26.055062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.241724+0000","last_change":"2024-09-10T22:10:37.241724+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.241724+0000","last_undegraded":"2024-09-10T22:10:37.241724+0000","last_fullsized":"2024-09-10T22:10:37.241724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240036+0000","last_change":"2024-09-10T22:10:37.240036+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240036+0000","last_undegraded":"2024-09-10T22:10:37.240036+0000","last_fullsized":"2024-09-10T22:10:37.240036+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522599+0000","last_change":"2024-09-10T22:10:38.254564+0000","last_active":"2024-09-10T22:10:39.522599+0000","last_peered":"2024-09-10T22:10:39.522599+0000","last_clean":"2024-09-10T22:10:39.522599+0000","last_became_active":"2024-09-10T22:10:38.254280+0000","last_became_peered":"2024-09-10T22:10:38.254280+0000","last_unstale":"2024-09-10T22:10:39.522599+0000","last_undegraded":"2024-09-10T22:10:39.522599+0000","last_fullsized":"2024-09-10T22:10:39.522599+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:48.733204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.255200+0000","last_change":"2024-09-10T22:10:38.249346+0000","last_active":"2024-09-10T22:10:39.255200+0000","last_peered":"2024-09-10T22:10:39.255200+0000","last_clean":"2024-09-10T22:10:39.255200+0000","last_became_active":"2024-09-10T22:10:38.249101+0000","last_became_peered":"2024-09-10T22:10:38.249101+0000","last_unstale":"2024-09-10T22:10:39.255200+0000","last_undegraded":"2024-09-10T22:10:39.255200+0000","last_fullsized":"2024-09-10T22:10:39.255200+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:38.972161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528751+0000","last_change":"2024-09-10T22:10:38.266215+0000","last_active":"2024-09-10T22:10:39.528751+0000","last_peered":"2024-09-10T22:10:39.528751+0000","last_clean":"2024-09-10T22:10:39.528751+0000","last_became_active":"2024-09-10T22:10:38.264693+0000","last_became_peered":"2024-09-10T22:10:38.264693+0000","last_unstale":"2024-09-10T22:10:39.528751+0000","last_undegraded":"2024-09-10T22:10:39.528751+0000","last_fullsized":"2024-09-10T22:10:39.528751+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:09.177869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.243036+0000","last_change":"2024-09-10T22:10:37.243036+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.243036+0000","last_undegraded":"2024-09-10T22:10:37.243036+0000","last_fullsized":"2024-09-10T22:10:37.243036+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254568+0000","last_change":"2024-09-10T22:10:38.247539+0000","last_active":"2024-09-10T22:10:39.254568+0000","last_peered":"2024-09-10T22:10:39.254568+0000","last_clean":"2024-09-10T22:10:39.254568+0000","last_became_active":"2024-09-10T22:10:38.247124+0000","last_became_peered":"2024-09-10T22:10:38.247124+0000","last_unstale":"2024-09-10T22:10:39.254568+0000","last_undegraded":"2024-09-10T22:10:39.254568+0000","last_fullsized":"2024-09-10T22:10:39.254568+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:26.055062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247126+0000","last_change":"2024-09-10T22:10:38.243927+0000","last_active":"2024-09-10T22:10:39.247126+0000","last_peered":"2024-09-10T22:10:39.247126+0000","last_clean":"2024-09-10T22:10:39.247126+0000","last_became_active":"2024-09-10T22:10:38.240651+0000","last_became_peered":"2024-09-10T22:10:38.240651+0000","last_unstale":"2024-09-10T22:10:39.247126+0000","last_undegraded":"2024-09-10T22:10:39.247126+0000","last_fullsized":"2024-09-10T22:10:39.247126+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:34.748065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523481+0000","last_change":"2024-09-10T22:10:38.264327+0000","last_active":"2024-09-10T22:10:39.523481+0000","last_peered":"2024-09-10T22:10:39.523481+0000","last_clean":"2024-09-10T22:10:39.523481+0000","last_became_active":"2024-09-10T22:10:38.264014+0000","last_became_peered":"2024-09-10T22:10:38.264014+0000","last_unstale":"2024-09-10T22:10:39.523481+0000","last_undegraded":"2024-09-10T22:10:39.523481+0000","last_fullsized":"2024-09-10T22:10:39.523481+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:24.533086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247264+0000","last_change":"2024-09-10T22:10:38.244207+0000","last_active":"2024-09-10T22:10:39.247264+0000","last_peered":"2024-09-10T22:10:39.247264+0000","last_clean":"2024-09-10T22:10:39.247264+0000","last_became_active":"2024-09-10T22:10:38.243891+0000","last_became_peered":"2024-09-10T22:10:38.243891+0000","last_unstale":"2024-09-10T22:10:39.247264+0000","last_undegraded":"2024-09-10T22:10:39.247264+0000","last_fullsized":"2024-09-10T22:10:39.247264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:39.927846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240030+0000","last_change":"2024-09-10T22:10:37.240030+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240030+0000","last_undegraded":"2024-09-10T22:10:37.240030+0000","last_fullsized":"2024-09-10T22:10:37.240030+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523341+0000","last_change":"2024-09-10T22:10:38.243673+0000","last_active":"2024-09-10T22:10:39.523341+0000","last_peered":"2024-09-10T22:10:39.523341+0000","last_clean":"2024-09-10T22:10:39.523341+0000","last_became_active":"2024-09-10T22:10:38.243416+0000","last_became_peered":"2024-09-10T22:10:38.243416+0000","last_unstale":"2024-09-10T22:10:39.523341+0000","last_undegraded":"2024-09-10T22:10:39.523341+0000","last_fullsized":"2024-09-10T22:10:39.523341+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:28.506115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247572+0000","last_change":"2024-09-10T22:10:38.255081+0000","last_active":"2024-09-10T22:10:39.247572+0000","last_peered":"2024-09-10T22:10:39.247572+0000","last_clean":"2024-09-10T22:10:39.247572+0000","last_became_active":"2024-09-10T22:10:38.254659+0000","last_became_peered":"2024-09-10T22:10:38.254659+0000","last_unstale":"2024-09-10T22:10:39.247572+0000","last_undegraded":"2024-09-10T22:10:39.247572+0000","last_fullsized":"2024-09-10T22:10:39.247572+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:23.035694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524131+0000","last_change":"2024-09-10T22:10:38.247997+0000","last_active":"2024-09-10T22:10:39.524131+0000","last_peered":"2024-09-10T22:10:39.524131+0000","last_clean":"2024-09-10T22:10:39.524131+0000","last_became_active":"2024-09-10T22:10:38.247730+0000","last_became_peered":"2024-09-10T22:10:38.247730+0000","last_unstale":"2024-09-10T22:10:39.524131+0000","last_undegraded":"2024-09-10T22:10:39.524131+0000","last_fullsized":"2024-09-10T22:10:39.524131+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:32.488690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254657+0000","last_change":"2024-09-10T22:10:38.245564+0000","last_active":"2024-09-10T22:10:39.254657+0000","last_peered":"2024-09-10T22:10:39.254657+0000","last_clean":"2024-09-10T22:10:39.254657+0000","last_became_active":"2024-09-10T22:10:38.245271+0000","last_became_peered":"2024-09-10T22:10:38.245271+0000","last_unstale":"2024-09-10T22:10:39.254657+0000","last_undegraded":"2024-09-10T22:10:39.254657+0000","last_fullsized":"2024-09-10T22:10:39.254657+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:26.491631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523720+0000","last_change":"2024-09-10T22:10:38.250433+0000","last_active":"2024-09-10T22:10:39.523720+0000","last_peered":"2024-09-10T22:10:39.523720+0000","last_clean":"2024-09-10T22:10:39.523720+0000","last_became_active":"2024-09-10T22:10:38.250154+0000","last_became_peered":"2024-09-10T22:10:38.250154+0000","last_unstale":"2024-09-10T22:10:39.523720+0000","last_undegraded":"2024-09-10T22:10:39.523720+0000","last_fullsized":"2024-09-10T22:10:39.523720+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:32.858119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522651+0000","last_change":"2024-09-10T22:10:38.245066+0000","last_active":"2024-09-10T22:10:39.522651+0000","last_peered":"2024-09-10T22:10:39.522651+0000","last_clean":"2024-09-10T22:10:39.522651+0000","last_became_active":"2024-09-10T22:10:38.244768+0000","last_became_peered":"2024-09-10T22:10:38.244768+0000","last_unstale":"2024-09-10T22:10:39.522651+0000","last_undegraded":"2024-09-10T22:10:39.522651+0000","last_fullsized":"2024-09-10T22:10:39.522651+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:44.888191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522566+0000","last_change":"2024-09-10T22:10:38.250358+0000","last_active":"2024-09-10T22:10:39.522566+0000","last_peered":"2024-09-10T22:10:39.522566+0000","last_clean":"2024-09-10T22:10:39.522566+0000","last_became_active":"2024-09-10T22:10:38.250103+0000","last_became_peered":"2024-09-10T22:10:38.250103+0000","last_unstale":"2024-09-10T22:10:39.522566+0000","last_undegraded":"2024-09-10T22:10:39.522566+0000","last_fullsized":"2024-09-10T22:10:39.522566+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:35.427777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.239464+0000","last_change":"2024-09-10T22:10:37.239464+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.239464+0000","last_undegraded":"2024-09-10T22:10:37.239464+0000","last_fullsized":"2024-09-10T22:10:37.239464+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528701+0000","last_change":"2024-09-10T22:10:38.269614+0000","last_active":"2024-09-10T22:10:39.528701+0000","last_peered":"2024-09-10T22:10:39.528701+0000","last_clean":"2024-09-10T22:10:39.528701+0000","last_became_active":"2024-09-10T22:10:38.269330+0000","last_became_peered":"2024-09-10T22:10:38.269330+0000","last_unstale":"2024-09-10T22:10:39.528701+0000","last_undegraded":"2024-09-10T22:10:39.528701+0000","last_fullsized":"2024-09-10T22:10:39.528701+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:38.972161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523101+0000","last_change":"2024-09-10T22:10:38.247355+0000","last_active":"2024-09-10T22:10:39.523101+0000","last_peered":"2024-09-10T22:10:39.523101+0000","last_clean":"2024-09-10T22:10:39.523101+0000","last_became_active":"2024-09-10T22:10:38.246782+0000","last_became_peered":"2024-09-10T22:10:38.246782+0000","last_unstale":"2024-09-10T22:10:39.523101+0000","last_undegraded":"2024-09-10T22:10:39.523101+0000","last_fullsized":"2024-09-10T22:10:39.523101+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:57.493480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.239961+0000","last_change":"2024-09-10T22:10:37.239961+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.239961+0000","last_undegraded":"2024-09-10T22:10:37.239961+0000","last_fullsized":"2024-09-10T22:10:37.239961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528635+0000","last_change":"2024-09-10T22:10:38.270949+0000","last_active":"2024-09-10T22:10:39.528635+0000","last_peered":"2024-09-10T22:10:39.528635+0000","last_clean":"2024-09-10T22:10:39.528635+0000","last_became_active":"2024-09-10T22:10:38.270580+0000","last_became_peered":"2024-09-10T22:10:38.270580+0000","last_unstale":"2024-09-10T22:10:39.528635+0000","last_undegraded":"2024-09-10T22:10:39.528635+0000","last_fullsized":"2024-09-10T22:10:39.528635+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:46.669269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522776+0000","last_change":"2024-09-10T22:10:38.244297+0000","last_active":"2024-09-10T22:10:39.522776+0000","last_peered":"2024-09-10T22:10:39.522776+0000","last_clean":"2024-09-10T22:10:39.522776+0000","last_became_active":"2024-09-10T22:10:38.243953+0000","last_became_peered":"2024-09-10T22:10:38.243953+0000","last_unstale":"2024-09-10T22:10:39.522776+0000","last_undegraded":"2024-09-10T22:10:39.522776+0000","last_fullsized":"2024-09-10T22:10:39.522776+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:08.221611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523686+0000","last_change":"2024-09-10T22:10:38.247381+0000","last_active":"2024-09-10T22:10:39.523686+0000","last_peered":"2024-09-10T22:10:39.523686+0000","last_clean":"2024-09-10T22:10:39.523686+0000","last_became_active":"2024-09-10T22:10:38.247092+0000","last_became_peered":"2024-09-10T22:10:38.247092+0000","last_unstale":"2024-09-10T22:10:39.523686+0000","last_undegraded":"2024-09-10T22:10:39.523686+0000","last_fullsized":"2024-09-10T22:10:39.523686+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:04.052881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247281+0000","last_change":"2024-09-10T22:10:38.262682+0000","last_active":"2024-09-10T22:10:39.247281+0000","last_peered":"2024-09-10T22:10:39.247281+0000","last_clean":"2024-09-10T22:10:39.247281+0000","last_became_active":"2024-09-10T22:10:38.262426+0000","last_became_peered":"2024-09-10T22:10:38.262426+0000","last_unstale":"2024-09-10T22:10:39.247281+0000","last_undegraded":"2024-09-10T22:10:39.247281+0000","last_fullsized":"2024-09-10T22:10:39.247281+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:16.836613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381139+0000","last_change":"2024-09-10T22:10:38.248324+0000","last_active":"2024-09-10T22:10:39.381139+0000","last_peered":"2024-09-10T22:10:39.381139+0000","last_clean":"2024-09-10T22:10:39.381139+0000","last_became_active":"2024-09-10T22:10:38.245453+0000","last_became_peered":"2024-09-10T22:10:38.245453+0000","last_unstale":"2024-09-10T22:10:39.381139+0000","last_undegraded":"2024-09-10T22:10:39.381139+0000","last_fullsized":"2024-09-10T22:10:39.381139+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:22.865604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524095+0000","last_change":"2024-09-10T22:10:38.255400+0000","last_active":"2024-09-10T22:10:39.524095+0000","last_peered":"2024-09-10T22:10:39.524095+0000","last_clean":"2024-09-10T22:10:39.524095+0000","last_became_active":"2024-09-10T22:10:38.255069+0000","last_became_peered":"2024-09-10T22:10:38.255069+0000","last_unstale":"2024-09-10T22:10:39.524095+0000","last_undegraded":"2024-09-10T22:10:39.524095+0000","last_fullsized":"2024-09-10T22:10:39.524095+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:32.770140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"99'4","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523313+0000","last_change":"2024-09-10T22:10:37.247494+0000","last_active":"2024-09-10T22:10:39.523313+0000","last_peered":"2024-09-10T22:10:39.523313+0000","last_clean":"2024-09-10T22:10:39.523313+0000","last_became_active":"2024-09-10T22:10:37.244676+0000","last_became_peered":"2024-09-10T22:10:37.244676+0000","last_unstale":"2024-09-10T22:10:39.523313+0000","last_undegraded":"2024-09-10T22:10:39.523313+0000","last_fullsized":"2024-09-10T22:10:39.523313+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:25.917947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247778+0000","last_change":"2024-09-10T22:10:38.263771+0000","last_active":"2024-09-10T22:10:39.247778+0000","last_peered":"2024-09-10T22:10:39.247778+0000","last_clean":"2024-09-10T22:10:39.247778+0000","last_became_active":"2024-09-10T22:10:38.263458+0000","last_became_peered":"2024-09-10T22:10:38.263458+0000","last_unstale":"2024-09-10T22:10:39.247778+0000","last_undegraded":"2024-09-10T22:10:39.247778+0000","last_fullsized":"2024-09-10T22:10:39.247778+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:48.091865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.251427+0000","last_change":"2024-09-10T22:10:37.248597+0000","last_active":"2024-09-10T22:10:37.251427+0000","last_peered":"2024-09-10T22:10:37.251427+0000","last_clean":"2024-09-10T22:10:37.251427+0000","last_became_active":"2024-09-10T22:10:37.248297+0000","last_became_peered":"2024-09-10T22:10:37.248297+0000","last_unstale":"2024-09-10T22:10:37.251427+0000","last_undegraded":"2024-09-10T22:10:37.251427+0000","last_fullsized":"2024-09-10T22:10:37.251427+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:56.601971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381440+0000","last_change":"2024-09-10T22:10:38.241783+0000","last_active":"2024-09-10T22:10:39.381440+0000","last_peered":"2024-09-10T22:10:39.381440+0000","last_clean":"2024-09-10T22:10:39.381440+0000","last_became_active":"2024-09-10T22:10:38.241478+0000","last_became_peered":"2024-09-10T22:10:38.241478+0000","last_unstale":"2024-09-10T22:10:39.381440+0000","last_undegraded":"2024-09-10T22:10:39.381440+0000","last_fullsized":"2024-09-10T22:10:39.381440+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:37.074694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522717+0000","last_change":"2024-09-10T22:10:37.249272+0000","last_active":"2024-09-10T22:10:39.522717+0000","last_peered":"2024-09-10T22:10:39.522717+0000","last_clean":"2024-09-10T22:10:39.522717+0000","last_became_active":"2024-09-10T22:10:37.248969+0000","last_became_peered":"2024-09-10T22:10:37.248969+0000","last_unstale":"2024-09-10T22:10:39.522717+0000","last_undegraded":"2024-09-10T22:10:39.522717+0000","last_fullsized":"2024-09-10T22:10:39.522717+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:00.267217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.255113+0000","last_change":"2024-09-10T22:10:38.247440+0000","last_active":"2024-09-10T22:10:39.255113+0000","last_peered":"2024-09-10T22:10:39.255113+0000","last_clean":"2024-09-10T22:10:39.255113+0000","last_became_active":"2024-09-10T22:10:38.243369+0000","last_became_peered":"2024-09-10T22:10:38.243369+0000","last_unstale":"2024-09-10T22:10:39.255113+0000","last_undegraded":"2024-09-10T22:10:39.255113+0000","last_fullsized":"2024-09-10T22:10:39.255113+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:31.048301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522825+0000","last_change":"2024-09-10T22:10:37.235026+0000","last_active":"2024-09-10T22:10:39.522825+0000","last_peered":"2024-09-10T22:10:39.522825+0000","last_clean":"2024-09-10T22:10:39.522825+0000","last_became_active":"2024-09-10T22:10:37.234835+0000","last_became_peered":"2024-09-10T22:10:37.234835+0000","last_unstale":"2024-09-10T22:10:39.522825+0000","last_undegraded":"2024-09-10T22:10:39.522825+0000","last_fullsized":"2024-09-10T22:10:39.522825+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:39.992087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252383+0000","last_change":"2024-09-10T22:10:38.250119+0000","last_active":"2024-09-10T22:10:39.252383+0000","last_peered":"2024-09-10T22:10:39.252383+0000","last_clean":"2024-09-10T22:10:39.252383+0000","last_became_active":"2024-09-10T22:10:38.249748+0000","last_became_peered":"2024-09-10T22:10:38.249748+0000","last_unstale":"2024-09-10T22:10:39.252383+0000","last_undegraded":"2024-09-10T22:10:39.252383+0000","last_fullsized":"2024-09-10T22:10:39.252383+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:33.757624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381382+0000","last_change":"2024-09-10T22:10:37.247963+0000","last_active":"2024-09-10T22:10:39.381382+0000","last_peered":"2024-09-10T22:10:39.381382+0000","last_clean":"2024-09-10T22:10:39.381382+0000","last_became_active":"2024-09-10T22:10:37.246784+0000","last_became_peered":"2024-09-10T22:10:37.246784+0000","last_unstale":"2024-09-10T22:10:39.381382+0000","last_undegraded":"2024-09-10T22:10:39.381382+0000","last_fullsized":"2024-09-10T22:10:39.381382+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:33.104121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.241495+0000","last_change":"2024-09-10T22:10:37.241495+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.241495+0000","last_undegraded":"2024-09-10T22:10:37.241495+0000","last_fullsized":"2024-09-10T22:10:37.241495+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522451+0000","last_change":"2024-09-10T22:10:37.240295+0000","last_active":"2024-09-10T22:10:39.522451+0000","last_peered":"2024-09-10T22:10:39.522451+0000","last_clean":"2024-09-10T22:10:39.522451+0000","last_became_active":"2024-09-10T22:10:37.240014+0000","last_became_peered":"2024-09-10T22:10:37.240014+0000","last_unstale":"2024-09-10T22:10:39.522451+0000","last_undegraded":"2024-09-10T22:10:39.522451+0000","last_fullsized":"2024-09-10T22:10:39.522451+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:19.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247084+0000","last_change":"2024-09-10T22:10:38.245203+0000","last_active":"2024-09-10T22:10:39.247084+0000","last_peered":"2024-09-10T22:10:39.247084+0000","last_clean":"2024-09-10T22:10:39.247084+0000","last_became_active":"2024-09-10T22:10:38.244911+0000","last_became_peered":"2024-09-10T22:10:38.244911+0000","last_unstale":"2024-09-10T22:10:39.247084+0000","last_undegraded":"2024-09-10T22:10:39.247084+0000","last_fullsized":"2024-09-10T22:10:39.247084+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:19.820527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528261+0000","last_change":"2024-09-10T22:10:37.258021+0000","last_active":"2024-09-10T22:10:39.528261+0000","last_peered":"2024-09-10T22:10:39.528261+0000","last_clean":"2024-09-10T22:10:39.528261+0000","last_became_active":"2024-09-10T22:10:37.257766+0000","last_became_peered":"2024-09-10T22:10:37.257766+0000","last_unstale":"2024-09-10T22:10:39.528261+0000","last_undegraded":"2024-09-10T22:10:39.528261+0000","last_fullsized":"2024-09-10T22:10:39.528261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:36.653416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254271+0000","last_change":"2024-09-10T22:10:38.243002+0000","last_active":"2024-09-10T22:10:39.254271+0000","last_peered":"2024-09-10T22:10:39.254271+0000","last_clean":"2024-09-10T22:10:39.254271+0000","last_became_active":"2024-09-10T22:10:38.241629+0000","last_became_peered":"2024-09-10T22:10:38.241629+0000","last_unstale":"2024-09-10T22:10:39.254271+0000","last_undegraded":"2024-09-10T22:10:39.254271+0000","last_fullsized":"2024-09-10T22:10:39.254271+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:21.178257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381135+0000","last_change":"2024-09-10T22:10:37.247160+0000","last_active":"2024-09-10T22:10:39.381135+0000","last_peered":"2024-09-10T22:10:39.381135+0000","last_clean":"2024-09-10T22:10:39.381135+0000","last_became_active":"2024-09-10T22:10:37.246865+0000","last_became_peered":"2024-09-10T22:10:37.246865+0000","last_unstale":"2024-09-10T22:10:39.381135+0000","last_undegraded":"2024-09-10T22:10:39.381135+0000","last_fullsized":"2024-09-10T22:10:39.381135+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:54.381121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523577+0000","last_change":"2024-09-10T22:10:38.257125+0000","last_active":"2024-09-10T22:10:39.523577+0000","last_peered":"2024-09-10T22:10:39.523577+0000","last_clean":"2024-09-10T22:10:39.523577+0000","last_became_active":"2024-09-10T22:10:38.256821+0000","last_became_peered":"2024-09-10T22:10:38.256821+0000","last_unstale":"2024-09-10T22:10:39.523577+0000","last_undegraded":"2024-09-10T22:10:39.523577+0000","last_fullsized":"2024-09-10T22:10:39.523577+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:13.792877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381703+0000","last_change":"2024-09-10T22:10:37.246423+0000","last_active":"2024-09-10T22:10:39.381703+0000","last_peered":"2024-09-10T22:10:39.381703+0000","last_clean":"2024-09-10T22:10:39.381703+0000","last_became_active":"2024-09-10T22:10:37.246139+0000","last_became_peered":"2024-09-10T22:10:37.246139+0000","last_unstale":"2024-09-10T22:10:39.381703+0000","last_undegraded":"2024-09-10T22:10:39.381703+0000","last_fullsized":"2024-09-10T22:10:39.381703+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:00.685355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.255201+0000","last_change":"2024-09-10T22:10:38.253930+0000","last_active":"2024-09-10T22:10:39.255201+0000","last_peered":"2024-09-10T22:10:39.255201+0000","last_clean":"2024-09-10T22:10:39.255201+0000","last_became_active":"2024-09-10T22:10:38.253264+0000","last_became_peered":"2024-09-10T22:10:38.253264+0000","last_unstale":"2024-09-10T22:10:39.255201+0000","last_undegraded":"2024-09-10T22:10:39.255201+0000","last_fullsized":"2024-09-10T22:10:39.255201+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:19.820527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247473+0000","last_change":"2024-09-10T22:10:37.254160+0000","last_active":"2024-09-10T22:10:39.247473+0000","last_peered":"2024-09-10T22:10:39.247473+0000","last_clean":"2024-09-10T22:10:39.247473+0000","last_became_active":"2024-09-10T22:10:37.253831+0000","last_became_peered":"2024-09-10T22:10:37.253831+0000","last_unstale":"2024-09-10T22:10:39.247473+0000","last_undegraded":"2024-09-10T22:10:39.247473+0000","last_fullsized":"2024-09-10T22:10:39.247473+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:00.267217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528676+0000","last_change":"2024-09-10T22:10:38.266948+0000","last_active":"2024-09-10T22:10:39.528676+0000","last_peered":"2024-09-10T22:10:39.528676+0000","last_clean":"2024-09-10T22:10:39.528676+0000","last_became_active":"2024-09-10T22:10:38.266677+0000","last_became_peered":"2024-09-10T22:10:38.266677+0000","last_unstale":"2024-09-10T22:10:39.528676+0000","last_undegraded":"2024-09-10T22:10:39.528676+0000","last_fullsized":"2024-09-10T22:10:39.528676+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:08.232358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.253794+0000","last_change":"2024-09-10T22:10:37.252591+0000","last_active":"2024-09-10T22:10:37.253794+0000","last_peered":"2024-09-10T22:10:37.253794+0000","last_clean":"2024-09-10T22:10:37.253794+0000","last_became_active":"2024-09-10T22:10:37.252236+0000","last_became_peered":"2024-09-10T22:10:37.252236+0000","last_unstale":"2024-09-10T22:10:37.253794+0000","last_undegraded":"2024-09-10T22:10:37.253794+0000","last_fullsized":"2024-09-10T22:10:37.253794+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:25.052167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.242629+0000","last_change":"2024-09-10T22:10:37.242629+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.242629+0000","last_undegraded":"2024-09-10T22:10:37.242629+0000","last_fullsized":"2024-09-10T22:10:37.242629+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"99'6","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254866+0000","last_change":"2024-09-10T22:10:37.245595+0000","last_active":"2024-09-10T22:10:39.254866+0000","last_peered":"2024-09-10T22:10:39.254866+0000","last_clean":"2024-09-10T22:10:39.254866+0000","last_became_active":"2024-09-10T22:10:37.245237+0000","last_became_peered":"2024-09-10T22:10:37.245237+0000","last_unstale":"2024-09-10T22:10:39.254866+0000","last_undegraded":"2024-09-10T22:10:39.254866+0000","last_fullsized":"2024-09-10T22:10:39.254866+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:17.957594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247188+0000","last_change":"2024-09-10T22:10:38.261146+0000","last_active":"2024-09-10T22:10:39.247188+0000","last_peered":"2024-09-10T22:10:39.247188+0000","last_clean":"2024-09-10T22:10:39.247188+0000","last_became_active":"2024-09-10T22:10:38.260713+0000","last_became_peered":"2024-09-10T22:10:38.260713+0000","last_unstale":"2024-09-10T22:10:39.247188+0000","last_undegraded":"2024-09-10T22:10:39.247188+0000","last_fullsized":"2024-09-10T22:10:39.247188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:20.279287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523954+0000","last_change":"2024-09-10T22:10:37.247682+0000","last_active":"2024-09-10T22:10:39.523954+0000","last_peered":"2024-09-10T22:10:39.523954+0000","last_clean":"2024-09-10T22:10:39.523954+0000","last_became_active":"2024-09-10T22:10:37.247416+0000","last_became_peered":"2024-09-10T22:10:37.247416+0000","last_unstale":"2024-09-10T22:10:39.523954+0000","last_undegraded":"2024-09-10T22:10:39.523954+0000","last_fullsized":"2024-09-10T22:10:39.523954+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:32.804016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381286+0000","last_change":"2024-09-10T22:10:38.248180+0000","last_active":"2024-09-10T22:10:39.381286+0000","last_peered":"2024-09-10T22:10:39.381286+0000","last_clean":"2024-09-10T22:10:39.381286+0000","last_became_active":"2024-09-10T22:10:38.247917+0000","last_became_peered":"2024-09-10T22:10:38.247917+0000","last_unstale":"2024-09-10T22:10:39.381286+0000","last_undegraded":"2024-09-10T22:10:39.381286+0000","last_fullsized":"2024-09-10T22:10:39.381286+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:45.132830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523823+0000","last_change":"2024-09-10T22:10:37.251530+0000","last_active":"2024-09-10T22:10:39.523823+0000","last_peered":"2024-09-10T22:10:39.523823+0000","last_clean":"2024-09-10T22:10:39.523823+0000","last_became_active":"2024-09-10T22:10:37.251337+0000","last_became_peered":"2024-09-10T22:10:37.251337+0000","last_unstale":"2024-09-10T22:10:39.523823+0000","last_undegraded":"2024-09-10T22:10:39.523823+0000","last_fullsized":"2024-09-10T22:10:39.523823+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:30.617743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254115+0000","last_change":"2024-09-10T22:10:38.246234+0000","last_active":"2024-09-10T22:10:39.254115+0000","last_peered":"2024-09-10T22:10:39.254115+0000","last_clean":"2024-09-10T22:10:39.254115+0000","last_became_active":"2024-09-10T22:10:38.241936+0000","last_became_peered":"2024-09-10T22:10:38.241936+0000","last_unstale":"2024-09-10T22:10:39.254115+0000","last_undegraded":"2024-09-10T22:10:39.254115+0000","last_fullsized":"2024-09-10T22:10:39.254115+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:40.994982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254261+0000","last_change":"2024-09-10T22:10:37.248956+0000","last_active":"2024-09-10T22:10:39.254261+0000","last_peered":"2024-09-10T22:10:39.254261+0000","last_clean":"2024-09-10T22:10:39.254261+0000","last_became_active":"2024-09-10T22:10:37.248634+0000","last_became_peered":"2024-09-10T22:10:37.248634+0000","last_unstale":"2024-09-10T22:10:39.254261+0000","last_undegraded":"2024-09-10T22:10:39.254261+0000","last_fullsized":"2024-09-10T22:10:39.254261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:23.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523489+0000","last_change":"2024-09-10T22:10:38.248859+0000","last_active":"2024-09-10T22:10:39.523489+0000","last_peered":"2024-09-10T22:10:39.523489+0000","last_clean":"2024-09-10T22:10:39.523489+0000","last_became_active":"2024-09-10T22:10:38.247750+0000","last_became_peered":"2024-09-10T22:10:38.247750+0000","last_unstale":"2024-09-10T22:10:39.523489+0000","last_undegraded":"2024-09-10T22:10:39.523489+0000","last_fullsized":"2024-09-10T22:10:39.523489+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:09.599932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"99'8","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381272+0000","last_change":"2024-09-10T22:10:37.241895+0000","last_active":"2024-09-10T22:10:39.381272+0000","last_peered":"2024-09-10T22:10:39.381272+0000","last_clean":"2024-09-10T22:10:39.381272+0000","last_became_active":"2024-09-10T22:10:37.241654+0000","last_became_peered":"2024-09-10T22:10:37.241654+0000","last_unstale":"2024-09-10T22:10:39.381272+0000","last_undegraded":"2024-09-10T22:10:39.381272+0000","last_fullsized":"2024-09-10T22:10:39.381272+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:10.220978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528948+0000","last_change":"2024-09-10T22:10:38.269389+0000","last_active":"2024-09-10T22:10:39.528948+0000","last_peered":"2024-09-10T22:10:39.528948+0000","last_clean":"2024-09-10T22:10:39.528948+0000","last_became_active":"2024-09-10T22:10:38.269130+0000","last_became_peered":"2024-09-10T22:10:38.269130+0000","last_unstale":"2024-09-10T22:10:39.528948+0000","last_undegraded":"2024-09-10T22:10:39.528948+0000","last_fullsized":"2024-09-10T22:10:39.528948+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:10.005594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254954+0000","last_change":"2024-09-10T22:10:37.254551+0000","last_active":"2024-09-10T22:10:39.254954+0000","last_peered":"2024-09-10T22:10:39.254954+0000","last_clean":"2024-09-10T22:10:39.254954+0000","last_became_active":"2024-09-10T22:10:37.254311+0000","last_became_peered":"2024-09-10T22:10:37.254311+0000","last_unstale":"2024-09-10T22:10:39.254954+0000","last_undegraded":"2024-09-10T22:10:39.254954+0000","last_fullsized":"2024-09-10T22:10:39.254954+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:25.574209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252554+0000","last_change":"2024-09-10T22:10:38.265801+0000","last_active":"2024-09-10T22:10:39.252554+0000","last_peered":"2024-09-10T22:10:39.252554+0000","last_clean":"2024-09-10T22:10:39.252554+0000","last_became_active":"2024-09-10T22:10:38.265549+0000","last_became_peered":"2024-09-10T22:10:38.265549+0000","last_unstale":"2024-09-10T22:10:39.252554+0000","last_undegraded":"2024-09-10T22:10:39.252554+0000","last_fullsized":"2024-09-10T22:10:39.252554+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:49.342939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528666+0000","last_change":"2024-09-10T22:10:37.249193+0000","last_active":"2024-09-10T22:10:39.528666+0000","last_peered":"2024-09-10T22:10:39.528666+0000","last_clean":"2024-09-10T22:10:39.528666+0000","last_became_active":"2024-09-10T22:10:37.248911+0000","last_became_peered":"2024-09-10T22:10:37.248911+0000","last_unstale":"2024-09-10T22:10:39.528666+0000","last_undegraded":"2024-09-10T22:10:39.528666+0000","last_fullsized":"2024-09-10T22:10:39.528666+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:02.732555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T22:10:37.240468+0000","last_change":"2024-09-10T22:10:37.240468+0000","last_active":"2024-09-10T22:10:37.219743+0000","last_peered":"2024-09-10T22:10:37.219743+0000","last_clean":"2024-09-10T22:10:37.219743+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:10:37.240468+0000","last_undegraded":"2024-09-10T22:10:37.240468+0000","last_fullsized":"2024-09-10T22:10:37.240468+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522890+0000","last_change":"2024-09-10T22:10:37.234553+0000","last_active":"2024-09-10T22:10:39.522890+0000","last_peered":"2024-09-10T22:10:39.522890+0000","last_clean":"2024-09-10T22:10:39.522890+0000","last_became_active":"2024-09-10T22:10:37.234278+0000","last_became_peered":"2024-09-10T22:10:37.234278+0000","last_unstale":"2024-09-10T22:10:39.522890+0000","last_undegraded":"2024-09-10T22:10:39.522890+0000","last_fullsized":"2024-09-10T22:10:39.522890+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:20.175362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523714+0000","last_change":"2024-09-10T22:10:38.256468+0000","last_active":"2024-09-10T22:10:39.523714+0000","last_peered":"2024-09-10T22:10:39.523714+0000","last_clean":"2024-09-10T22:10:39.523714+0000","last_became_active":"2024-09-10T22:10:38.256224+0000","last_became_peered":"2024-09-10T22:10:38.256224+0000","last_unstale":"2024-09-10T22:10:39.523714+0000","last_undegraded":"2024-09-10T22:10:39.523714+0000","last_fullsized":"2024-09-10T22:10:39.523714+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:23.765455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522649+0000","last_change":"2024-09-10T22:10:37.249141+0000","last_active":"2024-09-10T22:10:39.522649+0000","last_peered":"2024-09-10T22:10:39.522649+0000","last_clean":"2024-09-10T22:10:39.522649+0000","last_became_active":"2024-09-10T22:10:37.248865+0000","last_became_peered":"2024-09-10T22:10:37.248865+0000","last_unstale":"2024-09-10T22:10:39.522649+0000","last_undegraded":"2024-09-10T22:10:39.522649+0000","last_fullsized":"2024-09-10T22:10:39.522649+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:16.473063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523961+0000","last_change":"2024-09-10T22:10:38.247400+0000","last_active":"2024-09-10T22:10:39.523961+0000","last_peered":"2024-09-10T22:10:39.523961+0000","last_clean":"2024-09-10T22:10:39.523961+0000","last_became_active":"2024-09-10T22:10:38.244941+0000","last_became_peered":"2024-09-10T22:10:38.244941+0000","last_unstale":"2024-09-10T22:10:39.523961+0000","last_undegraded":"2024-09-10T22:10:39.523961+0000","last_fullsized":"2024-09-10T22:10:39.523961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:18.374331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252406+0000","last_change":"2024-09-10T22:10:37.246142+0000","last_active":"2024-09-10T22:10:39.252406+0000","last_peered":"2024-09-10T22:10:39.252406+0000","last_clean":"2024-09-10T22:10:39.252406+0000","last_became_active":"2024-09-10T22:10:37.245696+0000","last_became_peered":"2024-09-10T22:10:37.245696+0000","last_unstale":"2024-09-10T22:10:39.252406+0000","last_undegraded":"2024-09-10T22:10:39.252406+0000","last_fullsized":"2024-09-10T22:10:39.252406+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:23.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523257+0000","last_change":"2024-09-10T22:10:38.257969+0000","last_active":"2024-09-10T22:10:39.523257+0000","last_peered":"2024-09-10T22:10:39.523257+0000","last_clean":"2024-09-10T22:10:39.523257+0000","last_became_active":"2024-09-10T22:10:38.255428+0000","last_became_peered":"2024-09-10T22:10:38.255428+0000","last_unstale":"2024-09-10T22:10:39.523257+0000","last_undegraded":"2024-09-10T22:10:39.523257+0000","last_fullsized":"2024-09-10T22:10:39.523257+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:23.507403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252252+0000","last_change":"2024-09-10T22:10:37.249833+0000","last_active":"2024-09-10T22:10:39.252252+0000","last_peered":"2024-09-10T22:10:39.252252+0000","last_clean":"2024-09-10T22:10:39.252252+0000","last_became_active":"2024-09-10T22:10:37.248940+0000","last_became_peered":"2024-09-10T22:10:37.248940+0000","last_unstale":"2024-09-10T22:10:39.252252+0000","last_undegraded":"2024-09-10T22:10:39.252252+0000","last_fullsized":"2024-09-10T22:10:39.252252+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:17.957594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527472+0000","last_change":"2024-09-10T22:10:38.260839+0000","last_active":"2024-09-10T22:10:39.527472+0000","last_peered":"2024-09-10T22:10:39.527472+0000","last_clean":"2024-09-10T22:10:39.527472+0000","last_became_active":"2024-09-10T22:10:38.260199+0000","last_became_peered":"2024-09-10T22:10:38.260199+0000","last_unstale":"2024-09-10T22:10:39.527472+0000","last_undegraded":"2024-09-10T22:10:39.527472+0000","last_fullsized":"2024-09-10T22:10:39.527472+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:57.604867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522526+0000","last_change":"2024-09-10T22:10:37.248217+0000","last_active":"2024-09-10T22:10:39.522526+0000","last_peered":"2024-09-10T22:10:39.522526+0000","last_clean":"2024-09-10T22:10:39.522526+0000","last_became_active":"2024-09-10T22:10:37.247946+0000","last_became_peered":"2024-09-10T22:10:37.247946+0000","last_unstale":"2024-09-10T22:10:39.522526+0000","last_undegraded":"2024-09-10T22:10:39.522526+0000","last_fullsized":"2024-09-10T22:10:39.522526+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:25.488736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523362+0000","last_change":"2024-09-10T22:10:38.251544+0000","last_active":"2024-09-10T22:10:39.523362+0000","last_peered":"2024-09-10T22:10:39.523362+0000","last_clean":"2024-09-10T22:10:39.523362+0000","last_became_active":"2024-09-10T22:10:38.251270+0000","last_became_peered":"2024-09-10T22:10:38.251270+0000","last_unstale":"2024-09-10T22:10:39.523362+0000","last_undegraded":"2024-09-10T22:10:39.523362+0000","last_fullsized":"2024-09-10T22:10:39.523362+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:58.242589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252755+0000","last_change":"2024-09-10T22:10:37.248132+0000","last_active":"2024-09-10T22:10:39.252755+0000","last_peered":"2024-09-10T22:10:39.252755+0000","last_clean":"2024-09-10T22:10:39.252755+0000","last_became_active":"2024-09-10T22:10:37.247855+0000","last_became_peered":"2024-09-10T22:10:37.247855+0000","last_unstale":"2024-09-10T22:10:39.252755+0000","last_undegraded":"2024-09-10T22:10:39.252755+0000","last_fullsized":"2024-09-10T22:10:39.252755+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:44.140728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381589+0000","last_change":"2024-09-10T22:10:38.251921+0000","last_active":"2024-09-10T22:10:39.381589+0000","last_peered":"2024-09-10T22:10:39.381589+0000","last_clean":"2024-09-10T22:10:39.381589+0000","last_became_active":"2024-09-10T22:10:38.251620+0000","last_became_peered":"2024-09-10T22:10:38.251620+0000","last_unstale":"2024-09-10T22:10:39.381589+0000","last_undegraded":"2024-09-10T22:10:39.381589+0000","last_fullsized":"2024-09-10T22:10:39.381589+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:25.497370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"100'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523469+0000","last_change":"2024-09-10T22:10:37.253726+0000","last_active":"2024-09-10T22:10:39.523469+0000","last_peered":"2024-09-10T22:10:39.523469+0000","last_clean":"2024-09-10T22:10:39.523469+0000","last_became_active":"2024-09-10T22:10:37.252258+0000","last_became_peered":"2024-09-10T22:10:37.252258+0000","last_unstale":"2024-09-10T22:10:39.523469+0000","last_undegraded":"2024-09-10T22:10:39.523469+0000","last_fullsized":"2024-09-10T22:10:39.523469+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:30.045406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381513+0000","last_change":"2024-09-10T22:10:38.261993+0000","last_active":"2024-09-10T22:10:39.381513+0000","last_peered":"2024-09-10T22:10:39.381513+0000","last_clean":"2024-09-10T22:10:39.381513+0000","last_became_active":"2024-09-10T22:10:38.261706+0000","last_became_peered":"2024-09-10T22:10:38.261706+0000","last_unstale":"2024-09-10T22:10:39.381513+0000","last_undegraded":"2024-09-10T22:10:39.381513+0000","last_fullsized":"2024-09-10T22:10:39.381513+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:27.116503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522775+0000","last_change":"2024-09-10T22:10:37.236693+0000","last_active":"2024-09-10T22:10:39.522775+0000","last_peered":"2024-09-10T22:10:39.522775+0000","last_clean":"2024-09-10T22:10:39.522775+0000","last_became_active":"2024-09-10T22:10:37.236322+0000","last_became_peered":"2024-09-10T22:10:37.236322+0000","last_unstale":"2024-09-10T22:10:39.522775+0000","last_undegraded":"2024-09-10T22:10:39.522775+0000","last_fullsized":"2024-09-10T22:10:39.522775+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:48.453353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528630+0000","last_change":"2024-09-10T22:10:38.263553+0000","last_active":"2024-09-10T22:10:39.528630+0000","last_peered":"2024-09-10T22:10:39.528630+0000","last_clean":"2024-09-10T22:10:39.528630+0000","last_became_active":"2024-09-10T22:10:38.263302+0000","last_became_peered":"2024-09-10T22:10:38.263302+0000","last_unstale":"2024-09-10T22:10:39.528630+0000","last_undegraded":"2024-09-10T22:10:39.528630+0000","last_fullsized":"2024-09-10T22:10:39.528630+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:08.130137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528811+0000","last_change":"2024-09-10T22:10:37.249927+0000","last_active":"2024-09-10T22:10:39.528811+0000","last_peered":"2024-09-10T22:10:39.528811+0000","last_clean":"2024-09-10T22:10:39.528811+0000","last_became_active":"2024-09-10T22:10:37.249443+0000","last_became_peered":"2024-09-10T22:10:37.249443+0000","last_unstale":"2024-09-10T22:10:39.528811+0000","last_undegraded":"2024-09-10T22:10:39.528811+0000","last_fullsized":"2024-09-10T22:10:39.528811+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:20.595899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524302+0000","last_change":"2024-09-10T22:10:38.249997+0000","last_active":"2024-09-10T22:10:39.524302+0000","last_peered":"2024-09-10T22:10:39.524302+0000","last_clean":"2024-09-10T22:10:39.524302+0000","last_became_active":"2024-09-10T22:10:38.249737+0000","last_became_peered":"2024-09-10T22:10:38.249737+0000","last_unstale":"2024-09-10T22:10:39.524302+0000","last_undegraded":"2024-09-10T22:10:39.524302+0000","last_fullsized":"2024-09-10T22:10:39.524302+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:19.948050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522780+0000","last_change":"2024-09-10T22:10:37.245541+0000","last_active":"2024-09-10T22:10:39.522780+0000","last_peered":"2024-09-10T22:10:39.522780+0000","last_clean":"2024-09-10T22:10:39.522780+0000","last_became_active":"2024-09-10T22:10:37.245066+0000","last_became_peered":"2024-09-10T22:10:37.245066+0000","last_unstale":"2024-09-10T22:10:39.522780+0000","last_undegraded":"2024-09-10T22:10:39.522780+0000","last_fullsized":"2024-09-10T22:10:39.522780+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:40.695789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254591+0000","last_change":"2024-09-10T22:10:38.262616+0000","last_active":"2024-09-10T22:10:39.254591+0000","last_peered":"2024-09-10T22:10:39.254591+0000","last_clean":"2024-09-10T22:10:39.254591+0000","last_became_active":"2024-09-10T22:10:38.260645+0000","last_became_peered":"2024-09-10T22:10:38.260645+0000","last_unstale":"2024-09-10T22:10:39.254591+0000","last_undegraded":"2024-09-10T22:10:39.254591+0000","last_fullsized":"2024-09-10T22:10:39.254591+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:41.698684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528261+0000","last_change":"2024-09-10T22:10:37.251944+0000","last_active":"2024-09-10T22:10:39.528261+0000","last_peered":"2024-09-10T22:10:39.528261+0000","last_clean":"2024-09-10T22:10:39.528261+0000","last_became_active":"2024-09-10T22:10:37.251665+0000","last_became_peered":"2024-09-10T22:10:37.251665+0000","last_unstale":"2024-09-10T22:10:39.528261+0000","last_undegraded":"2024-09-10T22:10:39.528261+0000","last_fullsized":"2024-09-10T22:10:39.528261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:55.320245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523396+0000","last_change":"2024-09-10T22:10:38.250847+0000","last_active":"2024-09-10T22:10:39.523396+0000","last_peered":"2024-09-10T22:10:39.523396+0000","last_clean":"2024-09-10T22:10:39.523396+0000","last_became_active":"2024-09-10T22:10:38.250591+0000","last_became_peered":"2024-09-10T22:10:38.250591+0000","last_unstale":"2024-09-10T22:10:39.523396+0000","last_undegraded":"2024-09-10T22:10:39.523396+0000","last_fullsized":"2024-09-10T22:10:39.523396+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:57.861382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247352+0000","last_change":"2024-09-10T22:10:37.249468+0000","last_active":"2024-09-10T22:10:39.247352+0000","last_peered":"2024-09-10T22:10:39.247352+0000","last_clean":"2024-09-10T22:10:39.247352+0000","last_became_active":"2024-09-10T22:10:37.249196+0000","last_became_peered":"2024-09-10T22:10:37.249196+0000","last_unstale":"2024-09-10T22:10:39.247352+0000","last_undegraded":"2024-09-10T22:10:39.247352+0000","last_fullsized":"2024-09-10T22:10:39.247352+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:48.453353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528272+0000","last_change":"2024-09-10T22:10:38.263051+0000","last_active":"2024-09-10T22:10:39.528272+0000","last_peered":"2024-09-10T22:10:39.528272+0000","last_clean":"2024-09-10T22:10:39.528272+0000","last_became_active":"2024-09-10T22:10:38.262760+0000","last_became_peered":"2024-09-10T22:10:38.262760+0000","last_unstale":"2024-09-10T22:10:39.528272+0000","last_undegraded":"2024-09-10T22:10:39.528272+0000","last_fullsized":"2024-09-10T22:10:39.528272+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:01.551745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523361+0000","last_change":"2024-09-10T22:10:37.254152+0000","last_active":"2024-09-10T22:10:39.523361+0000","last_peered":"2024-09-10T22:10:39.523361+0000","last_clean":"2024-09-10T22:10:39.523361+0000","last_became_active":"2024-09-10T22:10:37.253879+0000","last_became_peered":"2024-09-10T22:10:37.253879+0000","last_unstale":"2024-09-10T22:10:39.523361+0000","last_undegraded":"2024-09-10T22:10:39.523361+0000","last_fullsized":"2024-09-10T22:10:39.523361+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:29.614570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524110+0000","last_change":"2024-09-10T22:10:38.259332+0000","last_active":"2024-09-10T22:10:39.524110+0000","last_peered":"2024-09-10T22:10:39.524110+0000","last_clean":"2024-09-10T22:10:39.524110+0000","last_became_active":"2024-09-10T22:10:38.255536+0000","last_became_peered":"2024-09-10T22:10:38.255536+0000","last_unstale":"2024-09-10T22:10:39.524110+0000","last_undegraded":"2024-09-10T22:10:39.524110+0000","last_fullsized":"2024-09-10T22:10:39.524110+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:52.177586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.252402+0000","last_change":"2024-09-10T22:10:37.249681+0000","last_active":"2024-09-10T22:10:37.252402+0000","last_peered":"2024-09-10T22:10:37.252402+0000","last_clean":"2024-09-10T22:10:37.252402+0000","last_became_active":"2024-09-10T22:10:37.249376+0000","last_became_peered":"2024-09-10T22:10:37.249376+0000","last_unstale":"2024-09-10T22:10:37.252402+0000","last_undegraded":"2024-09-10T22:10:37.252402+0000","last_fullsized":"2024-09-10T22:10:37.252402+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:00.548850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":256,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T22:10:37.244716+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:10:37.244716+0000","last_peered":"2024-09-10T22:10:37.244716+0000","last_clean":"2024-09-10T22:10:37.244716+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:10:37.244716+0000","last_undegraded":"2024-09-10T22:10:37.244716+0000","last_fullsized":"2024-09-10T22:10:37.244716+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509566,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123120,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248720,"statfs":{"total":96636764160,"available":96510689280,"internally_reserved":0,"allocated":1798144,"data_stored":1108825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509568,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130896,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240944,"statfs":{"total":96636764160,"available":96502726656,"internally_reserved":0,"allocated":1236992,"data_stored":532296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509567,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64272,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307568,"statfs":{"total":96636764160,"available":96570949632,"internally_reserved":0,"allocated":1232896,"data_stored":531714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509568,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144800,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227040,"statfs":{"total":96636764160,"available":96488488960,"internally_reserved":0,"allocated":1249280,"data_stored":533380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509566,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122580,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249260,"statfs":{"total":96636764160,"available":96511242240,"internally_reserved":0,"allocated":1245184,"data_stored":533278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509567,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126732,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245108,"statfs":{"total":96636764160,"available":96506990592,"internally_reserved":0,"allocated":1228800,"data_stored":531644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509567,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123148,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248692,"statfs":{"total":96636764160,"available":96510660608,"internally_reserved":0,"allocated":1826816,"data_stored":1122594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":124254665},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542272,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144776,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227064,"statfs":{"total":96636764160,"available":96488513536,"internally_reserved":0,"allocated":1232896,"data_stored":531678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:10:47.019 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:10:47.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 -- 172.21.15.66:0/2197912939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46040ae620 msgr2=0x7f46040725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:47.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 --2- 172.21.15.66:0/2197912939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46040ae620 0x7f46040725d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009900 tx=0x7f45ec01c880 comp rx=0 tx=0).stop 2024-09-10T22:10:47.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 -- 172.21.15.66:0/2197912939 shutdown_connections 2024-09-10T22:10:47.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 --2- 172.21.15.66:0/2197912939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46040ae620 0x7f46040725d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 --2- 172.21.15.66:0/2197912939 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46040c4b10 0x7f460406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 -- 172.21.15.66:0/2197912939 >> 172.21.15.66:0/2197912939 conn(0x7f46040cfa30 msgr2=0x7f46040cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:47.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 -- 172.21.15.66:0/2197912939 shutdown_connections 2024-09-10T22:10:47.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 -- 172.21.15.66:0/2197912939 wait complete. 2024-09-10T22:10:47.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 Processor -- start 2024-09-10T22:10:47.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 -- start start 2024-09-10T22:10:47.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46040ae620 0x7f46041545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46040c4b10 0x7f4604073170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46040c4ef0 0x7f46040736b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f46041684a0 con 0x7f46040ae620 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4604168320 con 0x7f46040c4ef0 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4609000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4604168620 con 0x7f46040c4b10 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4601e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46040ae620 0x7f46041545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4601e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46040ae620 0x7f46041545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54324/0 (socket says 172.21.15.66:54324) 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4601e00640 1 -- 172.21.15.66:0/1972499747 learned_addr learned my addr 172.21.15.66:0/1972499747 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4601e00640 1 -- 172.21.15.66:0/1972499747 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46040c4b10 msgr2=0x7f4604073170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4601e00640 1 --2- 172.21.15.66:0/1972499747 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46040c4b10 0x7f4604073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4601e00640 1 -- 172.21.15.66:0/1972499747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46040c4ef0 msgr2=0x7f46040736b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4601e00640 1 --2- 172.21.15.66:0/1972499747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46040c4ef0 0x7f46040736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4601e00640 1 -- 172.21.15.66:0/1972499747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4604156c90 con 0x7f46040ae620 2024-09-10T22:10:47.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.176+0000 7f4601e00640 1 --2- 172.21.15.66:0/1972499747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46040ae620 0x7f46041545d0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f45f400d980 tx=0x7f45f400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:47.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.180+0000 7f45fa000640 1 -- 172.21.15.66:0/1972499747 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4017070 con 0x7f46040ae620 2024-09-10T22:10:47.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.180+0000 7f45fa000640 1 -- 172.21.15.66:0/1972499747 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45f4009d20 con 0x7f46040ae620 2024-09-10T22:10:47.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.180+0000 7f45fa000640 1 -- 172.21.15.66:0/1972499747 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f40106c0 con 0x7f46040ae620 2024-09-10T22:10:47.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.180+0000 7f4609000640 1 -- 172.21.15.66:0/1972499747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46041f7f20 con 0x7f46040ae620 2024-09-10T22:10:47.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.180+0000 7f4609000640 1 -- 172.21.15.66:0/1972499747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46041f7bd0 con 0x7f46040ae620 2024-09-10T22:10:47.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.180+0000 7f4609000640 1 -- 172.21.15.66:0/1972499747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46040725d0 con 0x7f46040ae620 2024-09-10T22:10:47.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.180+0000 7f45fa000640 1 -- 172.21.15.66:0/1972499747 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f45f4010860 con 0x7f46040ae620 2024-09-10T22:10:47.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.180+0000 7f45fa000640 1 --2- 172.21.15.66:0/1972499747 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45e0073c00 0x7f45e00760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:47.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.184+0000 7f45fa000640 1 -- 172.21.15.66:0/1972499747 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f45f40927a0 con 0x7f46040ae620 2024-09-10T22:10:47.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.184+0000 7f45fbe00640 1 --2- 172.21.15.66:0/1972499747 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45e0073c00 0x7f45e00760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:47.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.184+0000 7f45fbe00640 1 --2- 172.21.15.66:0/1972499747 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45e0073c00 0x7f45e00760c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0af000 tx=0x7f45ec019680 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:47.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.188+0000 7f45fa000640 1 -- 172.21.15.66:0/1972499747 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f45f405f4c0 con 0x7f46040ae620 2024-09-10T22:10:47.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.356+0000 7f4609000640 1 -- 172.21.15.66:0/1972499747 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46041579e0 con 0x7f45e0073c00 2024-09-10T22:10:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.360+0000 7f45fa000640 1 -- 172.21.15.66:0/1972499747 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252181 (secure 0 0 0) 0x7f46041579e0 con 0x7f45e0073c00 2024-09-10T22:10:47.367 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:47.368 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:10:47.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.364+0000 7f4609000640 1 -- 172.21.15.66:0/1972499747 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45e0073c00 msgr2=0x7f45e00760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:47.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.364+0000 7f4609000640 1 --2- 172.21.15.66:0/1972499747 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45e0073c00 0x7f45e00760c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0af000 tx=0x7f45ec019680 comp rx=0 tx=0).stop 2024-09-10T22:10:47.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.364+0000 7f4609000640 1 -- 172.21.15.66:0/1972499747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46040ae620 msgr2=0x7f46041545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:47.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.364+0000 7f4609000640 1 --2- 172.21.15.66:0/1972499747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46040ae620 0x7f46041545d0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f45f400d980 tx=0x7f45f400de40 comp rx=0 tx=0).stop 2024-09-10T22:10:47.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.364+0000 7f4609000640 1 -- 172.21.15.66:0/1972499747 shutdown_connections 2024-09-10T22:10:47.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.364+0000 7f4609000640 1 --2- 172.21.15.66:0/1972499747 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45e0073c00 0x7f45e00760c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.364+0000 7f4609000640 1 --2- 172.21.15.66:0/1972499747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46040c4ef0 0x7f46040736b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.364+0000 7f4609000640 1 --2- 172.21.15.66:0/1972499747 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46040c4b10 0x7f4604073170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.364+0000 7f4609000640 1 --2- 172.21.15.66:0/1972499747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46040ae620 0x7f46041545d0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.364+0000 7f4609000640 1 -- 172.21.15.66:0/1972499747 >> 172.21.15.66:0/1972499747 conn(0x7f46040cfa30 msgr2=0x7f46040b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:47.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.364+0000 7f4609000640 1 -- 172.21.15.66:0/1972499747 shutdown_connections 2024-09-10T22:10:47.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.364+0000 7f4609000640 1 -- 172.21.15.66:0/1972499747 wait complete. 2024-09-10T22:10:47.387 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-10T22:10:46.544178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":980364,"kb_used_data":10824,"kb_used_omap":12,"kb_used_meta":969331,"kb_avail":753994356,"statfs":{"total":773094113280,"available":772090220544,"internally_reserved":0,"allocated":11083776,"data_stored":5438708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":992595508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.310456"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381580+0000","last_change":"2024-09-10T22:10:38.251267+0000","last_active":"2024-09-10T22:10:39.381580+0000","last_peered":"2024-09-10T22:10:39.381580+0000","last_clean":"2024-09-10T22:10:39.381580+0000","last_became_active":"2024-09-10T22:10:38.250933+0000","last_became_peered":"2024-09-10T22:10:38.250933+0000","last_unstale":"2024-09-10T22:10:39.381580+0000","last_undegraded":"2024-09-10T22:10:39.381580+0000","last_fullsized":"2024-09-10T22:10:39.381580+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:41.184652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523102+0000","last_change":"2024-09-10T22:10:38.251283+0000","last_active":"2024-09-10T22:10:39.523102+0000","last_peered":"2024-09-10T22:10:39.523102+0000","last_clean":"2024-09-10T22:10:39.523102+0000","last_became_active":"2024-09-10T22:10:38.250986+0000","last_became_peered":"2024-09-10T22:10:38.250986+0000","last_unstale":"2024-09-10T22:10:39.523102+0000","last_undegraded":"2024-09-10T22:10:39.523102+0000","last_fullsized":"2024-09-10T22:10:39.523102+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:02.661632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381540+0000","last_change":"2024-09-10T22:10:38.253355+0000","last_active":"2024-09-10T22:10:39.381540+0000","last_peered":"2024-09-10T22:10:39.381540+0000","last_clean":"2024-09-10T22:10:39.381540+0000","last_became_active":"2024-09-10T22:10:38.253088+0000","last_became_peered":"2024-09-10T22:10:38.253088+0000","last_unstale":"2024-09-10T22:10:39.381540+0000","last_undegraded":"2024-09-10T22:10:39.381540+0000","last_fullsized":"2024-09-10T22:10:39.381540+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:18.638299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247441+0000","last_change":"2024-09-10T22:10:38.252302+0000","last_active":"2024-09-10T22:10:39.247441+0000","last_peered":"2024-09-10T22:10:39.247441+0000","last_clean":"2024-09-10T22:10:39.247441+0000","last_became_active":"2024-09-10T22:10:38.252084+0000","last_became_peered":"2024-09-10T22:10:38.252084+0000","last_unstale":"2024-09-10T22:10:39.247441+0000","last_undegraded":"2024-09-10T22:10:39.247441+0000","last_fullsized":"2024-09-10T22:10:39.247441+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:47.247838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528491+0000","last_change":"2024-09-10T22:10:38.263805+0000","last_active":"2024-09-10T22:10:39.528491+0000","last_peered":"2024-09-10T22:10:39.528491+0000","last_clean":"2024-09-10T22:10:39.528491+0000","last_became_active":"2024-09-10T22:10:38.263535+0000","last_became_peered":"2024-09-10T22:10:38.263535+0000","last_unstale":"2024-09-10T22:10:39.528491+0000","last_undegraded":"2024-09-10T22:10:39.528491+0000","last_fullsized":"2024-09-10T22:10:39.528491+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:26.055062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527294+0000","last_change":"2024-09-10T22:10:38.261438+0000","last_active":"2024-09-10T22:10:39.527294+0000","last_peered":"2024-09-10T22:10:39.527294+0000","last_clean":"2024-09-10T22:10:39.527294+0000","last_became_active":"2024-09-10T22:10:38.261121+0000","last_became_peered":"2024-09-10T22:10:38.261121+0000","last_unstale":"2024-09-10T22:10:39.527294+0000","last_undegraded":"2024-09-10T22:10:39.527294+0000","last_fullsized":"2024-09-10T22:10:39.527294+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:44.888191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527393+0000","last_change":"2024-09-10T22:10:38.260363+0000","last_active":"2024-09-10T22:10:39.527393+0000","last_peered":"2024-09-10T22:10:39.527393+0000","last_clean":"2024-09-10T22:10:39.527393+0000","last_became_active":"2024-09-10T22:10:38.260087+0000","last_became_peered":"2024-09-10T22:10:38.260087+0000","last_unstale":"2024-09-10T22:10:39.527393+0000","last_undegraded":"2024-09-10T22:10:39.527393+0000","last_fullsized":"2024-09-10T22:10:39.527393+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:58.820289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522599+0000","last_change":"2024-09-10T22:10:38.254564+0000","last_active":"2024-09-10T22:10:39.522599+0000","last_peered":"2024-09-10T22:10:39.522599+0000","last_clean":"2024-09-10T22:10:39.522599+0000","last_became_active":"2024-09-10T22:10:38.254280+0000","last_became_peered":"2024-09-10T22:10:38.254280+0000","last_unstale":"2024-09-10T22:10:39.522599+0000","last_undegraded":"2024-09-10T22:10:39.522599+0000","last_fullsized":"2024-09-10T22:10:39.522599+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:48.733204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.255200+0000","last_change":"2024-09-10T22:10:38.249346+0000","last_active":"2024-09-10T22:10:39.255200+0000","last_peered":"2024-09-10T22:10:39.255200+0000","last_clean":"2024-09-10T22:10:39.255200+0000","last_became_active":"2024-09-10T22:10:38.249101+0000","last_became_peered":"2024-09-10T22:10:38.249101+0000","last_unstale":"2024-09-10T22:10:39.255200+0000","last_undegraded":"2024-09-10T22:10:39.255200+0000","last_fullsized":"2024-09-10T22:10:39.255200+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:38.972161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528751+0000","last_change":"2024-09-10T22:10:38.266215+0000","last_active":"2024-09-10T22:10:39.528751+0000","last_peered":"2024-09-10T22:10:39.528751+0000","last_clean":"2024-09-10T22:10:39.528751+0000","last_became_active":"2024-09-10T22:10:38.264693+0000","last_became_peered":"2024-09-10T22:10:38.264693+0000","last_unstale":"2024-09-10T22:10:39.528751+0000","last_undegraded":"2024-09-10T22:10:39.528751+0000","last_fullsized":"2024-09-10T22:10:39.528751+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:09.177869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527890+0000","last_change":"2024-09-10T22:10:38.259143+0000","last_active":"2024-09-10T22:10:39.527890+0000","last_peered":"2024-09-10T22:10:39.527890+0000","last_clean":"2024-09-10T22:10:39.527890+0000","last_became_active":"2024-09-10T22:10:38.258607+0000","last_became_peered":"2024-09-10T22:10:38.258607+0000","last_unstale":"2024-09-10T22:10:39.527890+0000","last_undegraded":"2024-09-10T22:10:39.527890+0000","last_fullsized":"2024-09-10T22:10:39.527890+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:58.242589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254568+0000","last_change":"2024-09-10T22:10:38.247539+0000","last_active":"2024-09-10T22:10:39.254568+0000","last_peered":"2024-09-10T22:10:39.254568+0000","last_clean":"2024-09-10T22:10:39.254568+0000","last_became_active":"2024-09-10T22:10:38.247124+0000","last_became_peered":"2024-09-10T22:10:38.247124+0000","last_unstale":"2024-09-10T22:10:39.254568+0000","last_undegraded":"2024-09-10T22:10:39.254568+0000","last_fullsized":"2024-09-10T22:10:39.254568+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:26.055062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247126+0000","last_change":"2024-09-10T22:10:38.243927+0000","last_active":"2024-09-10T22:10:39.247126+0000","last_peered":"2024-09-10T22:10:39.247126+0000","last_clean":"2024-09-10T22:10:39.247126+0000","last_became_active":"2024-09-10T22:10:38.240651+0000","last_became_peered":"2024-09-10T22:10:38.240651+0000","last_unstale":"2024-09-10T22:10:39.247126+0000","last_undegraded":"2024-09-10T22:10:39.247126+0000","last_fullsized":"2024-09-10T22:10:39.247126+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:34.748065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523481+0000","last_change":"2024-09-10T22:10:38.264327+0000","last_active":"2024-09-10T22:10:39.523481+0000","last_peered":"2024-09-10T22:10:39.523481+0000","last_clean":"2024-09-10T22:10:39.523481+0000","last_became_active":"2024-09-10T22:10:38.264014+0000","last_became_peered":"2024-09-10T22:10:38.264014+0000","last_unstale":"2024-09-10T22:10:39.523481+0000","last_undegraded":"2024-09-10T22:10:39.523481+0000","last_fullsized":"2024-09-10T22:10:39.523481+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:24.533086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247264+0000","last_change":"2024-09-10T22:10:38.244207+0000","last_active":"2024-09-10T22:10:39.247264+0000","last_peered":"2024-09-10T22:10:39.247264+0000","last_clean":"2024-09-10T22:10:39.247264+0000","last_became_active":"2024-09-10T22:10:38.243891+0000","last_became_peered":"2024-09-10T22:10:38.243891+0000","last_unstale":"2024-09-10T22:10:39.247264+0000","last_undegraded":"2024-09-10T22:10:39.247264+0000","last_fullsized":"2024-09-10T22:10:39.247264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:39.927846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527325+0000","last_change":"2024-09-10T22:10:38.256900+0000","last_active":"2024-09-10T22:10:39.527325+0000","last_peered":"2024-09-10T22:10:39.527325+0000","last_clean":"2024-09-10T22:10:39.527325+0000","last_became_active":"2024-09-10T22:10:38.256340+0000","last_became_peered":"2024-09-10T22:10:38.256340+0000","last_unstale":"2024-09-10T22:10:39.527325+0000","last_undegraded":"2024-09-10T22:10:39.527325+0000","last_fullsized":"2024-09-10T22:10:39.527325+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:53.212126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523341+0000","last_change":"2024-09-10T22:10:38.243673+0000","last_active":"2024-09-10T22:10:39.523341+0000","last_peered":"2024-09-10T22:10:39.523341+0000","last_clean":"2024-09-10T22:10:39.523341+0000","last_became_active":"2024-09-10T22:10:38.243416+0000","last_became_peered":"2024-09-10T22:10:38.243416+0000","last_unstale":"2024-09-10T22:10:39.523341+0000","last_undegraded":"2024-09-10T22:10:39.523341+0000","last_fullsized":"2024-09-10T22:10:39.523341+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:28.506115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247572+0000","last_change":"2024-09-10T22:10:38.255081+0000","last_active":"2024-09-10T22:10:39.247572+0000","last_peered":"2024-09-10T22:10:39.247572+0000","last_clean":"2024-09-10T22:10:39.247572+0000","last_became_active":"2024-09-10T22:10:38.254659+0000","last_became_peered":"2024-09-10T22:10:38.254659+0000","last_unstale":"2024-09-10T22:10:39.247572+0000","last_undegraded":"2024-09-10T22:10:39.247572+0000","last_fullsized":"2024-09-10T22:10:39.247572+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:23.035694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524131+0000","last_change":"2024-09-10T22:10:38.247997+0000","last_active":"2024-09-10T22:10:39.524131+0000","last_peered":"2024-09-10T22:10:39.524131+0000","last_clean":"2024-09-10T22:10:39.524131+0000","last_became_active":"2024-09-10T22:10:38.247730+0000","last_became_peered":"2024-09-10T22:10:38.247730+0000","last_unstale":"2024-09-10T22:10:39.524131+0000","last_undegraded":"2024-09-10T22:10:39.524131+0000","last_fullsized":"2024-09-10T22:10:39.524131+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:32.488690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254657+0000","last_change":"2024-09-10T22:10:38.245564+0000","last_active":"2024-09-10T22:10:39.254657+0000","last_peered":"2024-09-10T22:10:39.254657+0000","last_clean":"2024-09-10T22:10:39.254657+0000","last_became_active":"2024-09-10T22:10:38.245271+0000","last_became_peered":"2024-09-10T22:10:38.245271+0000","last_unstale":"2024-09-10T22:10:39.254657+0000","last_undegraded":"2024-09-10T22:10:39.254657+0000","last_fullsized":"2024-09-10T22:10:39.254657+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:26.491631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523720+0000","last_change":"2024-09-10T22:10:38.250433+0000","last_active":"2024-09-10T22:10:39.523720+0000","last_peered":"2024-09-10T22:10:39.523720+0000","last_clean":"2024-09-10T22:10:39.523720+0000","last_became_active":"2024-09-10T22:10:38.250154+0000","last_became_peered":"2024-09-10T22:10:38.250154+0000","last_unstale":"2024-09-10T22:10:39.523720+0000","last_undegraded":"2024-09-10T22:10:39.523720+0000","last_fullsized":"2024-09-10T22:10:39.523720+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:32.858119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522651+0000","last_change":"2024-09-10T22:10:38.245066+0000","last_active":"2024-09-10T22:10:39.522651+0000","last_peered":"2024-09-10T22:10:39.522651+0000","last_clean":"2024-09-10T22:10:39.522651+0000","last_became_active":"2024-09-10T22:10:38.244768+0000","last_became_peered":"2024-09-10T22:10:38.244768+0000","last_unstale":"2024-09-10T22:10:39.522651+0000","last_undegraded":"2024-09-10T22:10:39.522651+0000","last_fullsized":"2024-09-10T22:10:39.522651+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:44.888191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522566+0000","last_change":"2024-09-10T22:10:38.250358+0000","last_active":"2024-09-10T22:10:39.522566+0000","last_peered":"2024-09-10T22:10:39.522566+0000","last_clean":"2024-09-10T22:10:39.522566+0000","last_became_active":"2024-09-10T22:10:38.250103+0000","last_became_peered":"2024-09-10T22:10:38.250103+0000","last_unstale":"2024-09-10T22:10:39.522566+0000","last_undegraded":"2024-09-10T22:10:39.522566+0000","last_fullsized":"2024-09-10T22:10:39.522566+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:35.427777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527295+0000","last_change":"2024-09-10T22:10:38.261087+0000","last_active":"2024-09-10T22:10:39.527295+0000","last_peered":"2024-09-10T22:10:39.527295+0000","last_clean":"2024-09-10T22:10:39.527295+0000","last_became_active":"2024-09-10T22:10:38.256649+0000","last_became_peered":"2024-09-10T22:10:38.256649+0000","last_unstale":"2024-09-10T22:10:39.527295+0000","last_undegraded":"2024-09-10T22:10:39.527295+0000","last_fullsized":"2024-09-10T22:10:39.527295+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:02.661632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528701+0000","last_change":"2024-09-10T22:10:38.269614+0000","last_active":"2024-09-10T22:10:39.528701+0000","last_peered":"2024-09-10T22:10:39.528701+0000","last_clean":"2024-09-10T22:10:39.528701+0000","last_became_active":"2024-09-10T22:10:38.269330+0000","last_became_peered":"2024-09-10T22:10:38.269330+0000","last_unstale":"2024-09-10T22:10:39.528701+0000","last_undegraded":"2024-09-10T22:10:39.528701+0000","last_fullsized":"2024-09-10T22:10:39.528701+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:38.972161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523101+0000","last_change":"2024-09-10T22:10:38.247355+0000","last_active":"2024-09-10T22:10:39.523101+0000","last_peered":"2024-09-10T22:10:39.523101+0000","last_clean":"2024-09-10T22:10:39.523101+0000","last_became_active":"2024-09-10T22:10:38.246782+0000","last_became_peered":"2024-09-10T22:10:38.246782+0000","last_unstale":"2024-09-10T22:10:39.523101+0000","last_undegraded":"2024-09-10T22:10:39.523101+0000","last_fullsized":"2024-09-10T22:10:39.523101+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:57.493480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527932+0000","last_change":"2024-09-10T22:10:38.260055+0000","last_active":"2024-09-10T22:10:39.527932+0000","last_peered":"2024-09-10T22:10:39.527932+0000","last_clean":"2024-09-10T22:10:39.527932+0000","last_became_active":"2024-09-10T22:10:38.259791+0000","last_became_peered":"2024-09-10T22:10:38.259791+0000","last_unstale":"2024-09-10T22:10:39.527932+0000","last_undegraded":"2024-09-10T22:10:39.527932+0000","last_fullsized":"2024-09-10T22:10:39.527932+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:26.034437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528635+0000","last_change":"2024-09-10T22:10:38.270949+0000","last_active":"2024-09-10T22:10:39.528635+0000","last_peered":"2024-09-10T22:10:39.528635+0000","last_clean":"2024-09-10T22:10:39.528635+0000","last_became_active":"2024-09-10T22:10:38.270580+0000","last_became_peered":"2024-09-10T22:10:38.270580+0000","last_unstale":"2024-09-10T22:10:39.528635+0000","last_undegraded":"2024-09-10T22:10:39.528635+0000","last_fullsized":"2024-09-10T22:10:39.528635+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:46.669269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522776+0000","last_change":"2024-09-10T22:10:38.244297+0000","last_active":"2024-09-10T22:10:39.522776+0000","last_peered":"2024-09-10T22:10:39.522776+0000","last_clean":"2024-09-10T22:10:39.522776+0000","last_became_active":"2024-09-10T22:10:38.243953+0000","last_became_peered":"2024-09-10T22:10:38.243953+0000","last_unstale":"2024-09-10T22:10:39.522776+0000","last_undegraded":"2024-09-10T22:10:39.522776+0000","last_fullsized":"2024-09-10T22:10:39.522776+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:08.221611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523686+0000","last_change":"2024-09-10T22:10:38.247381+0000","last_active":"2024-09-10T22:10:39.523686+0000","last_peered":"2024-09-10T22:10:39.523686+0000","last_clean":"2024-09-10T22:10:39.523686+0000","last_became_active":"2024-09-10T22:10:38.247092+0000","last_became_peered":"2024-09-10T22:10:38.247092+0000","last_unstale":"2024-09-10T22:10:39.523686+0000","last_undegraded":"2024-09-10T22:10:39.523686+0000","last_fullsized":"2024-09-10T22:10:39.523686+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:04.052881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247281+0000","last_change":"2024-09-10T22:10:38.262682+0000","last_active":"2024-09-10T22:10:39.247281+0000","last_peered":"2024-09-10T22:10:39.247281+0000","last_clean":"2024-09-10T22:10:39.247281+0000","last_became_active":"2024-09-10T22:10:38.262426+0000","last_became_peered":"2024-09-10T22:10:38.262426+0000","last_unstale":"2024-09-10T22:10:39.247281+0000","last_undegraded":"2024-09-10T22:10:39.247281+0000","last_fullsized":"2024-09-10T22:10:39.247281+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:16.836613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381139+0000","last_change":"2024-09-10T22:10:38.248324+0000","last_active":"2024-09-10T22:10:39.381139+0000","last_peered":"2024-09-10T22:10:39.381139+0000","last_clean":"2024-09-10T22:10:39.381139+0000","last_became_active":"2024-09-10T22:10:38.245453+0000","last_became_peered":"2024-09-10T22:10:38.245453+0000","last_unstale":"2024-09-10T22:10:39.381139+0000","last_undegraded":"2024-09-10T22:10:39.381139+0000","last_fullsized":"2024-09-10T22:10:39.381139+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:22.865604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524095+0000","last_change":"2024-09-10T22:10:38.255400+0000","last_active":"2024-09-10T22:10:39.524095+0000","last_peered":"2024-09-10T22:10:39.524095+0000","last_clean":"2024-09-10T22:10:39.524095+0000","last_became_active":"2024-09-10T22:10:38.255069+0000","last_became_peered":"2024-09-10T22:10:38.255069+0000","last_unstale":"2024-09-10T22:10:39.524095+0000","last_undegraded":"2024-09-10T22:10:39.524095+0000","last_fullsized":"2024-09-10T22:10:39.524095+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:32.770140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"99'4","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523313+0000","last_change":"2024-09-10T22:10:37.247494+0000","last_active":"2024-09-10T22:10:39.523313+0000","last_peered":"2024-09-10T22:10:39.523313+0000","last_clean":"2024-09-10T22:10:39.523313+0000","last_became_active":"2024-09-10T22:10:37.244676+0000","last_became_peered":"2024-09-10T22:10:37.244676+0000","last_unstale":"2024-09-10T22:10:39.523313+0000","last_undegraded":"2024-09-10T22:10:39.523313+0000","last_fullsized":"2024-09-10T22:10:39.523313+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:25.917947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247778+0000","last_change":"2024-09-10T22:10:38.263771+0000","last_active":"2024-09-10T22:10:39.247778+0000","last_peered":"2024-09-10T22:10:39.247778+0000","last_clean":"2024-09-10T22:10:39.247778+0000","last_became_active":"2024-09-10T22:10:38.263458+0000","last_became_peered":"2024-09-10T22:10:38.263458+0000","last_unstale":"2024-09-10T22:10:39.247778+0000","last_undegraded":"2024-09-10T22:10:39.247778+0000","last_fullsized":"2024-09-10T22:10:39.247778+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:48.091865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527747+0000","last_change":"2024-09-10T22:10:37.248597+0000","last_active":"2024-09-10T22:10:39.527747+0000","last_peered":"2024-09-10T22:10:39.527747+0000","last_clean":"2024-09-10T22:10:39.527747+0000","last_became_active":"2024-09-10T22:10:37.248297+0000","last_became_peered":"2024-09-10T22:10:37.248297+0000","last_unstale":"2024-09-10T22:10:39.527747+0000","last_undegraded":"2024-09-10T22:10:39.527747+0000","last_fullsized":"2024-09-10T22:10:39.527747+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:45.666374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381440+0000","last_change":"2024-09-10T22:10:38.241783+0000","last_active":"2024-09-10T22:10:39.381440+0000","last_peered":"2024-09-10T22:10:39.381440+0000","last_clean":"2024-09-10T22:10:39.381440+0000","last_became_active":"2024-09-10T22:10:38.241478+0000","last_became_peered":"2024-09-10T22:10:38.241478+0000","last_unstale":"2024-09-10T22:10:39.381440+0000","last_undegraded":"2024-09-10T22:10:39.381440+0000","last_fullsized":"2024-09-10T22:10:39.381440+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:37.074694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522717+0000","last_change":"2024-09-10T22:10:37.249272+0000","last_active":"2024-09-10T22:10:39.522717+0000","last_peered":"2024-09-10T22:10:39.522717+0000","last_clean":"2024-09-10T22:10:39.522717+0000","last_became_active":"2024-09-10T22:10:37.248969+0000","last_became_peered":"2024-09-10T22:10:37.248969+0000","last_unstale":"2024-09-10T22:10:39.522717+0000","last_undegraded":"2024-09-10T22:10:39.522717+0000","last_fullsized":"2024-09-10T22:10:39.522717+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:00.267217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.255113+0000","last_change":"2024-09-10T22:10:38.247440+0000","last_active":"2024-09-10T22:10:39.255113+0000","last_peered":"2024-09-10T22:10:39.255113+0000","last_clean":"2024-09-10T22:10:39.255113+0000","last_became_active":"2024-09-10T22:10:38.243369+0000","last_became_peered":"2024-09-10T22:10:38.243369+0000","last_unstale":"2024-09-10T22:10:39.255113+0000","last_undegraded":"2024-09-10T22:10:39.255113+0000","last_fullsized":"2024-09-10T22:10:39.255113+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:31.048301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522825+0000","last_change":"2024-09-10T22:10:37.235026+0000","last_active":"2024-09-10T22:10:39.522825+0000","last_peered":"2024-09-10T22:10:39.522825+0000","last_clean":"2024-09-10T22:10:39.522825+0000","last_became_active":"2024-09-10T22:10:37.234835+0000","last_became_peered":"2024-09-10T22:10:37.234835+0000","last_unstale":"2024-09-10T22:10:39.522825+0000","last_undegraded":"2024-09-10T22:10:39.522825+0000","last_fullsized":"2024-09-10T22:10:39.522825+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:39.992087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252383+0000","last_change":"2024-09-10T22:10:38.250119+0000","last_active":"2024-09-10T22:10:39.252383+0000","last_peered":"2024-09-10T22:10:39.252383+0000","last_clean":"2024-09-10T22:10:39.252383+0000","last_became_active":"2024-09-10T22:10:38.249748+0000","last_became_peered":"2024-09-10T22:10:38.249748+0000","last_unstale":"2024-09-10T22:10:39.252383+0000","last_undegraded":"2024-09-10T22:10:39.252383+0000","last_fullsized":"2024-09-10T22:10:39.252383+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:33.757624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381382+0000","last_change":"2024-09-10T22:10:37.247963+0000","last_active":"2024-09-10T22:10:39.381382+0000","last_peered":"2024-09-10T22:10:39.381382+0000","last_clean":"2024-09-10T22:10:39.381382+0000","last_became_active":"2024-09-10T22:10:37.246784+0000","last_became_peered":"2024-09-10T22:10:37.246784+0000","last_unstale":"2024-09-10T22:10:39.381382+0000","last_undegraded":"2024-09-10T22:10:39.381382+0000","last_fullsized":"2024-09-10T22:10:39.381382+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:33.104121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527295+0000","last_change":"2024-09-10T22:10:38.264154+0000","last_active":"2024-09-10T22:10:39.527295+0000","last_peered":"2024-09-10T22:10:39.527295+0000","last_clean":"2024-09-10T22:10:39.527295+0000","last_became_active":"2024-09-10T22:10:38.262946+0000","last_became_peered":"2024-09-10T22:10:38.262946+0000","last_unstale":"2024-09-10T22:10:39.527295+0000","last_undegraded":"2024-09-10T22:10:39.527295+0000","last_fullsized":"2024-09-10T22:10:39.527295+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:48.733204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522451+0000","last_change":"2024-09-10T22:10:37.240295+0000","last_active":"2024-09-10T22:10:39.522451+0000","last_peered":"2024-09-10T22:10:39.522451+0000","last_clean":"2024-09-10T22:10:39.522451+0000","last_became_active":"2024-09-10T22:10:37.240014+0000","last_became_peered":"2024-09-10T22:10:37.240014+0000","last_unstale":"2024-09-10T22:10:39.522451+0000","last_undegraded":"2024-09-10T22:10:39.522451+0000","last_fullsized":"2024-09-10T22:10:39.522451+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:19.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247084+0000","last_change":"2024-09-10T22:10:38.245203+0000","last_active":"2024-09-10T22:10:39.247084+0000","last_peered":"2024-09-10T22:10:39.247084+0000","last_clean":"2024-09-10T22:10:39.247084+0000","last_became_active":"2024-09-10T22:10:38.244911+0000","last_became_peered":"2024-09-10T22:10:38.244911+0000","last_unstale":"2024-09-10T22:10:39.247084+0000","last_undegraded":"2024-09-10T22:10:39.247084+0000","last_fullsized":"2024-09-10T22:10:39.247084+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:19.820527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528261+0000","last_change":"2024-09-10T22:10:37.258021+0000","last_active":"2024-09-10T22:10:39.528261+0000","last_peered":"2024-09-10T22:10:39.528261+0000","last_clean":"2024-09-10T22:10:39.528261+0000","last_became_active":"2024-09-10T22:10:37.257766+0000","last_became_peered":"2024-09-10T22:10:37.257766+0000","last_unstale":"2024-09-10T22:10:39.528261+0000","last_undegraded":"2024-09-10T22:10:39.528261+0000","last_fullsized":"2024-09-10T22:10:39.528261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:36.653416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254271+0000","last_change":"2024-09-10T22:10:38.243002+0000","last_active":"2024-09-10T22:10:39.254271+0000","last_peered":"2024-09-10T22:10:39.254271+0000","last_clean":"2024-09-10T22:10:39.254271+0000","last_became_active":"2024-09-10T22:10:38.241629+0000","last_became_peered":"2024-09-10T22:10:38.241629+0000","last_unstale":"2024-09-10T22:10:39.254271+0000","last_undegraded":"2024-09-10T22:10:39.254271+0000","last_fullsized":"2024-09-10T22:10:39.254271+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:21.178257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381135+0000","last_change":"2024-09-10T22:10:37.247160+0000","last_active":"2024-09-10T22:10:39.381135+0000","last_peered":"2024-09-10T22:10:39.381135+0000","last_clean":"2024-09-10T22:10:39.381135+0000","last_became_active":"2024-09-10T22:10:37.246865+0000","last_became_peered":"2024-09-10T22:10:37.246865+0000","last_unstale":"2024-09-10T22:10:39.381135+0000","last_undegraded":"2024-09-10T22:10:39.381135+0000","last_fullsized":"2024-09-10T22:10:39.381135+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:54.381121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523577+0000","last_change":"2024-09-10T22:10:38.257125+0000","last_active":"2024-09-10T22:10:39.523577+0000","last_peered":"2024-09-10T22:10:39.523577+0000","last_clean":"2024-09-10T22:10:39.523577+0000","last_became_active":"2024-09-10T22:10:38.256821+0000","last_became_peered":"2024-09-10T22:10:38.256821+0000","last_unstale":"2024-09-10T22:10:39.523577+0000","last_undegraded":"2024-09-10T22:10:39.523577+0000","last_fullsized":"2024-09-10T22:10:39.523577+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:13.792877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381703+0000","last_change":"2024-09-10T22:10:37.246423+0000","last_active":"2024-09-10T22:10:39.381703+0000","last_peered":"2024-09-10T22:10:39.381703+0000","last_clean":"2024-09-10T22:10:39.381703+0000","last_became_active":"2024-09-10T22:10:37.246139+0000","last_became_peered":"2024-09-10T22:10:37.246139+0000","last_unstale":"2024-09-10T22:10:39.381703+0000","last_undegraded":"2024-09-10T22:10:39.381703+0000","last_fullsized":"2024-09-10T22:10:39.381703+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:00.685355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.255201+0000","last_change":"2024-09-10T22:10:38.253930+0000","last_active":"2024-09-10T22:10:39.255201+0000","last_peered":"2024-09-10T22:10:39.255201+0000","last_clean":"2024-09-10T22:10:39.255201+0000","last_became_active":"2024-09-10T22:10:38.253264+0000","last_became_peered":"2024-09-10T22:10:38.253264+0000","last_unstale":"2024-09-10T22:10:39.255201+0000","last_undegraded":"2024-09-10T22:10:39.255201+0000","last_fullsized":"2024-09-10T22:10:39.255201+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:19.820527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247473+0000","last_change":"2024-09-10T22:10:37.254160+0000","last_active":"2024-09-10T22:10:39.247473+0000","last_peered":"2024-09-10T22:10:39.247473+0000","last_clean":"2024-09-10T22:10:39.247473+0000","last_became_active":"2024-09-10T22:10:37.253831+0000","last_became_peered":"2024-09-10T22:10:37.253831+0000","last_unstale":"2024-09-10T22:10:39.247473+0000","last_undegraded":"2024-09-10T22:10:39.247473+0000","last_fullsized":"2024-09-10T22:10:39.247473+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:00.267217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528676+0000","last_change":"2024-09-10T22:10:38.266948+0000","last_active":"2024-09-10T22:10:39.528676+0000","last_peered":"2024-09-10T22:10:39.528676+0000","last_clean":"2024-09-10T22:10:39.528676+0000","last_became_active":"2024-09-10T22:10:38.266677+0000","last_became_peered":"2024-09-10T22:10:38.266677+0000","last_unstale":"2024-09-10T22:10:39.528676+0000","last_undegraded":"2024-09-10T22:10:39.528676+0000","last_fullsized":"2024-09-10T22:10:39.528676+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:08.232358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"99'3","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528078+0000","last_change":"2024-09-10T22:10:37.252591+0000","last_active":"2024-09-10T22:10:39.528078+0000","last_peered":"2024-09-10T22:10:39.528078+0000","last_clean":"2024-09-10T22:10:39.528078+0000","last_became_active":"2024-09-10T22:10:37.252236+0000","last_became_peered":"2024-09-10T22:10:37.252236+0000","last_unstale":"2024-09-10T22:10:39.528078+0000","last_undegraded":"2024-09-10T22:10:39.528078+0000","last_fullsized":"2024-09-10T22:10:39.528078+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:07.229462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527662+0000","last_change":"2024-09-10T22:10:38.258945+0000","last_active":"2024-09-10T22:10:39.527662+0000","last_peered":"2024-09-10T22:10:39.527662+0000","last_clean":"2024-09-10T22:10:39.527662+0000","last_became_active":"2024-09-10T22:10:38.258552+0000","last_became_peered":"2024-09-10T22:10:38.258552+0000","last_unstale":"2024-09-10T22:10:39.527662+0000","last_undegraded":"2024-09-10T22:10:39.527662+0000","last_fullsized":"2024-09-10T22:10:39.527662+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:59.233986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"99'6","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254866+0000","last_change":"2024-09-10T22:10:37.245595+0000","last_active":"2024-09-10T22:10:39.254866+0000","last_peered":"2024-09-10T22:10:39.254866+0000","last_clean":"2024-09-10T22:10:39.254866+0000","last_became_active":"2024-09-10T22:10:37.245237+0000","last_became_peered":"2024-09-10T22:10:37.245237+0000","last_unstale":"2024-09-10T22:10:39.254866+0000","last_undegraded":"2024-09-10T22:10:39.254866+0000","last_fullsized":"2024-09-10T22:10:39.254866+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:17.957594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247188+0000","last_change":"2024-09-10T22:10:38.261146+0000","last_active":"2024-09-10T22:10:39.247188+0000","last_peered":"2024-09-10T22:10:39.247188+0000","last_clean":"2024-09-10T22:10:39.247188+0000","last_became_active":"2024-09-10T22:10:38.260713+0000","last_became_peered":"2024-09-10T22:10:38.260713+0000","last_unstale":"2024-09-10T22:10:39.247188+0000","last_undegraded":"2024-09-10T22:10:39.247188+0000","last_fullsized":"2024-09-10T22:10:39.247188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:20.279287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523954+0000","last_change":"2024-09-10T22:10:37.247682+0000","last_active":"2024-09-10T22:10:39.523954+0000","last_peered":"2024-09-10T22:10:39.523954+0000","last_clean":"2024-09-10T22:10:39.523954+0000","last_became_active":"2024-09-10T22:10:37.247416+0000","last_became_peered":"2024-09-10T22:10:37.247416+0000","last_unstale":"2024-09-10T22:10:39.523954+0000","last_undegraded":"2024-09-10T22:10:39.523954+0000","last_fullsized":"2024-09-10T22:10:39.523954+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:32.804016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381286+0000","last_change":"2024-09-10T22:10:38.248180+0000","last_active":"2024-09-10T22:10:39.381286+0000","last_peered":"2024-09-10T22:10:39.381286+0000","last_clean":"2024-09-10T22:10:39.381286+0000","last_became_active":"2024-09-10T22:10:38.247917+0000","last_became_peered":"2024-09-10T22:10:38.247917+0000","last_unstale":"2024-09-10T22:10:39.381286+0000","last_undegraded":"2024-09-10T22:10:39.381286+0000","last_fullsized":"2024-09-10T22:10:39.381286+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:45.132830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523823+0000","last_change":"2024-09-10T22:10:37.251530+0000","last_active":"2024-09-10T22:10:39.523823+0000","last_peered":"2024-09-10T22:10:39.523823+0000","last_clean":"2024-09-10T22:10:39.523823+0000","last_became_active":"2024-09-10T22:10:37.251337+0000","last_became_peered":"2024-09-10T22:10:37.251337+0000","last_unstale":"2024-09-10T22:10:39.523823+0000","last_undegraded":"2024-09-10T22:10:39.523823+0000","last_fullsized":"2024-09-10T22:10:39.523823+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:30.617743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254115+0000","last_change":"2024-09-10T22:10:38.246234+0000","last_active":"2024-09-10T22:10:39.254115+0000","last_peered":"2024-09-10T22:10:39.254115+0000","last_clean":"2024-09-10T22:10:39.254115+0000","last_became_active":"2024-09-10T22:10:38.241936+0000","last_became_peered":"2024-09-10T22:10:38.241936+0000","last_unstale":"2024-09-10T22:10:39.254115+0000","last_undegraded":"2024-09-10T22:10:39.254115+0000","last_fullsized":"2024-09-10T22:10:39.254115+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:40.994982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254261+0000","last_change":"2024-09-10T22:10:37.248956+0000","last_active":"2024-09-10T22:10:39.254261+0000","last_peered":"2024-09-10T22:10:39.254261+0000","last_clean":"2024-09-10T22:10:39.254261+0000","last_became_active":"2024-09-10T22:10:37.248634+0000","last_became_peered":"2024-09-10T22:10:37.248634+0000","last_unstale":"2024-09-10T22:10:39.254261+0000","last_undegraded":"2024-09-10T22:10:39.254261+0000","last_fullsized":"2024-09-10T22:10:39.254261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:23.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523489+0000","last_change":"2024-09-10T22:10:38.248859+0000","last_active":"2024-09-10T22:10:39.523489+0000","last_peered":"2024-09-10T22:10:39.523489+0000","last_clean":"2024-09-10T22:10:39.523489+0000","last_became_active":"2024-09-10T22:10:38.247750+0000","last_became_peered":"2024-09-10T22:10:38.247750+0000","last_unstale":"2024-09-10T22:10:39.523489+0000","last_undegraded":"2024-09-10T22:10:39.523489+0000","last_fullsized":"2024-09-10T22:10:39.523489+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:09.599932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"99'8","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381272+0000","last_change":"2024-09-10T22:10:37.241895+0000","last_active":"2024-09-10T22:10:39.381272+0000","last_peered":"2024-09-10T22:10:39.381272+0000","last_clean":"2024-09-10T22:10:39.381272+0000","last_became_active":"2024-09-10T22:10:37.241654+0000","last_became_peered":"2024-09-10T22:10:37.241654+0000","last_unstale":"2024-09-10T22:10:39.381272+0000","last_undegraded":"2024-09-10T22:10:39.381272+0000","last_fullsized":"2024-09-10T22:10:39.381272+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:10.220978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528948+0000","last_change":"2024-09-10T22:10:38.269389+0000","last_active":"2024-09-10T22:10:39.528948+0000","last_peered":"2024-09-10T22:10:39.528948+0000","last_clean":"2024-09-10T22:10:39.528948+0000","last_became_active":"2024-09-10T22:10:38.269130+0000","last_became_peered":"2024-09-10T22:10:38.269130+0000","last_unstale":"2024-09-10T22:10:39.528948+0000","last_undegraded":"2024-09-10T22:10:39.528948+0000","last_fullsized":"2024-09-10T22:10:39.528948+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:10.005594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254954+0000","last_change":"2024-09-10T22:10:37.254551+0000","last_active":"2024-09-10T22:10:39.254954+0000","last_peered":"2024-09-10T22:10:39.254954+0000","last_clean":"2024-09-10T22:10:39.254954+0000","last_became_active":"2024-09-10T22:10:37.254311+0000","last_became_peered":"2024-09-10T22:10:37.254311+0000","last_unstale":"2024-09-10T22:10:39.254954+0000","last_undegraded":"2024-09-10T22:10:39.254954+0000","last_fullsized":"2024-09-10T22:10:39.254954+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:25.574209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252554+0000","last_change":"2024-09-10T22:10:38.265801+0000","last_active":"2024-09-10T22:10:39.252554+0000","last_peered":"2024-09-10T22:10:39.252554+0000","last_clean":"2024-09-10T22:10:39.252554+0000","last_became_active":"2024-09-10T22:10:38.265549+0000","last_became_peered":"2024-09-10T22:10:38.265549+0000","last_unstale":"2024-09-10T22:10:39.252554+0000","last_undegraded":"2024-09-10T22:10:39.252554+0000","last_fullsized":"2024-09-10T22:10:39.252554+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:49.342939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528666+0000","last_change":"2024-09-10T22:10:37.249193+0000","last_active":"2024-09-10T22:10:39.528666+0000","last_peered":"2024-09-10T22:10:39.528666+0000","last_clean":"2024-09-10T22:10:39.528666+0000","last_became_active":"2024-09-10T22:10:37.248911+0000","last_became_peered":"2024-09-10T22:10:37.248911+0000","last_unstale":"2024-09-10T22:10:39.528666+0000","last_undegraded":"2024-09-10T22:10:39.528666+0000","last_fullsized":"2024-09-10T22:10:39.528666+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:02.732555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527811+0000","last_change":"2024-09-10T22:10:38.259272+0000","last_active":"2024-09-10T22:10:39.527811+0000","last_peered":"2024-09-10T22:10:39.527811+0000","last_clean":"2024-09-10T22:10:39.527811+0000","last_became_active":"2024-09-10T22:10:38.258856+0000","last_became_peered":"2024-09-10T22:10:38.258856+0000","last_unstale":"2024-09-10T22:10:39.527811+0000","last_undegraded":"2024-09-10T22:10:39.527811+0000","last_fullsized":"2024-09-10T22:10:39.527811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:28.506115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522890+0000","last_change":"2024-09-10T22:10:37.234553+0000","last_active":"2024-09-10T22:10:39.522890+0000","last_peered":"2024-09-10T22:10:39.522890+0000","last_clean":"2024-09-10T22:10:39.522890+0000","last_became_active":"2024-09-10T22:10:37.234278+0000","last_became_peered":"2024-09-10T22:10:37.234278+0000","last_unstale":"2024-09-10T22:10:39.522890+0000","last_undegraded":"2024-09-10T22:10:39.522890+0000","last_fullsized":"2024-09-10T22:10:39.522890+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:20.175362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523714+0000","last_change":"2024-09-10T22:10:38.256468+0000","last_active":"2024-09-10T22:10:39.523714+0000","last_peered":"2024-09-10T22:10:39.523714+0000","last_clean":"2024-09-10T22:10:39.523714+0000","last_became_active":"2024-09-10T22:10:38.256224+0000","last_became_peered":"2024-09-10T22:10:38.256224+0000","last_unstale":"2024-09-10T22:10:39.523714+0000","last_undegraded":"2024-09-10T22:10:39.523714+0000","last_fullsized":"2024-09-10T22:10:39.523714+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:23.765455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522649+0000","last_change":"2024-09-10T22:10:37.249141+0000","last_active":"2024-09-10T22:10:39.522649+0000","last_peered":"2024-09-10T22:10:39.522649+0000","last_clean":"2024-09-10T22:10:39.522649+0000","last_became_active":"2024-09-10T22:10:37.248865+0000","last_became_peered":"2024-09-10T22:10:37.248865+0000","last_unstale":"2024-09-10T22:10:39.522649+0000","last_undegraded":"2024-09-10T22:10:39.522649+0000","last_fullsized":"2024-09-10T22:10:39.522649+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:16.473063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523961+0000","last_change":"2024-09-10T22:10:38.247400+0000","last_active":"2024-09-10T22:10:39.523961+0000","last_peered":"2024-09-10T22:10:39.523961+0000","last_clean":"2024-09-10T22:10:39.523961+0000","last_became_active":"2024-09-10T22:10:38.244941+0000","last_became_peered":"2024-09-10T22:10:38.244941+0000","last_unstale":"2024-09-10T22:10:39.523961+0000","last_undegraded":"2024-09-10T22:10:39.523961+0000","last_fullsized":"2024-09-10T22:10:39.523961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:18.374331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252406+0000","last_change":"2024-09-10T22:10:37.246142+0000","last_active":"2024-09-10T22:10:39.252406+0000","last_peered":"2024-09-10T22:10:39.252406+0000","last_clean":"2024-09-10T22:10:39.252406+0000","last_became_active":"2024-09-10T22:10:37.245696+0000","last_became_peered":"2024-09-10T22:10:37.245696+0000","last_unstale":"2024-09-10T22:10:39.252406+0000","last_undegraded":"2024-09-10T22:10:39.252406+0000","last_fullsized":"2024-09-10T22:10:39.252406+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:23.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523257+0000","last_change":"2024-09-10T22:10:38.257969+0000","last_active":"2024-09-10T22:10:39.523257+0000","last_peered":"2024-09-10T22:10:39.523257+0000","last_clean":"2024-09-10T22:10:39.523257+0000","last_became_active":"2024-09-10T22:10:38.255428+0000","last_became_peered":"2024-09-10T22:10:38.255428+0000","last_unstale":"2024-09-10T22:10:39.523257+0000","last_undegraded":"2024-09-10T22:10:39.523257+0000","last_fullsized":"2024-09-10T22:10:39.523257+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:23.507403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252252+0000","last_change":"2024-09-10T22:10:37.249833+0000","last_active":"2024-09-10T22:10:39.252252+0000","last_peered":"2024-09-10T22:10:39.252252+0000","last_clean":"2024-09-10T22:10:39.252252+0000","last_became_active":"2024-09-10T22:10:37.248940+0000","last_became_peered":"2024-09-10T22:10:37.248940+0000","last_unstale":"2024-09-10T22:10:39.252252+0000","last_undegraded":"2024-09-10T22:10:39.252252+0000","last_fullsized":"2024-09-10T22:10:39.252252+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:17.957594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527472+0000","last_change":"2024-09-10T22:10:38.260839+0000","last_active":"2024-09-10T22:10:39.527472+0000","last_peered":"2024-09-10T22:10:39.527472+0000","last_clean":"2024-09-10T22:10:39.527472+0000","last_became_active":"2024-09-10T22:10:38.260199+0000","last_became_peered":"2024-09-10T22:10:38.260199+0000","last_unstale":"2024-09-10T22:10:39.527472+0000","last_undegraded":"2024-09-10T22:10:39.527472+0000","last_fullsized":"2024-09-10T22:10:39.527472+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:57.604867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522526+0000","last_change":"2024-09-10T22:10:37.248217+0000","last_active":"2024-09-10T22:10:39.522526+0000","last_peered":"2024-09-10T22:10:39.522526+0000","last_clean":"2024-09-10T22:10:39.522526+0000","last_became_active":"2024-09-10T22:10:37.247946+0000","last_became_peered":"2024-09-10T22:10:37.247946+0000","last_unstale":"2024-09-10T22:10:39.522526+0000","last_undegraded":"2024-09-10T22:10:39.522526+0000","last_fullsized":"2024-09-10T22:10:39.522526+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:25.488736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523362+0000","last_change":"2024-09-10T22:10:38.251544+0000","last_active":"2024-09-10T22:10:39.523362+0000","last_peered":"2024-09-10T22:10:39.523362+0000","last_clean":"2024-09-10T22:10:39.523362+0000","last_became_active":"2024-09-10T22:10:38.251270+0000","last_became_peered":"2024-09-10T22:10:38.251270+0000","last_unstale":"2024-09-10T22:10:39.523362+0000","last_undegraded":"2024-09-10T22:10:39.523362+0000","last_fullsized":"2024-09-10T22:10:39.523362+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:58.242589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.252755+0000","last_change":"2024-09-10T22:10:37.248132+0000","last_active":"2024-09-10T22:10:39.252755+0000","last_peered":"2024-09-10T22:10:39.252755+0000","last_clean":"2024-09-10T22:10:39.252755+0000","last_became_active":"2024-09-10T22:10:37.247855+0000","last_became_peered":"2024-09-10T22:10:37.247855+0000","last_unstale":"2024-09-10T22:10:39.252755+0000","last_undegraded":"2024-09-10T22:10:39.252755+0000","last_fullsized":"2024-09-10T22:10:39.252755+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:44.140728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381589+0000","last_change":"2024-09-10T22:10:38.251921+0000","last_active":"2024-09-10T22:10:39.381589+0000","last_peered":"2024-09-10T22:10:39.381589+0000","last_clean":"2024-09-10T22:10:39.381589+0000","last_became_active":"2024-09-10T22:10:38.251620+0000","last_became_peered":"2024-09-10T22:10:38.251620+0000","last_unstale":"2024-09-10T22:10:39.381589+0000","last_undegraded":"2024-09-10T22:10:39.381589+0000","last_fullsized":"2024-09-10T22:10:39.381589+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:25.497370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"100'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523469+0000","last_change":"2024-09-10T22:10:37.253726+0000","last_active":"2024-09-10T22:10:39.523469+0000","last_peered":"2024-09-10T22:10:39.523469+0000","last_clean":"2024-09-10T22:10:39.523469+0000","last_became_active":"2024-09-10T22:10:37.252258+0000","last_became_peered":"2024-09-10T22:10:37.252258+0000","last_unstale":"2024-09-10T22:10:39.523469+0000","last_undegraded":"2024-09-10T22:10:39.523469+0000","last_fullsized":"2024-09-10T22:10:39.523469+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:30.045406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.381513+0000","last_change":"2024-09-10T22:10:38.261993+0000","last_active":"2024-09-10T22:10:39.381513+0000","last_peered":"2024-09-10T22:10:39.381513+0000","last_clean":"2024-09-10T22:10:39.381513+0000","last_became_active":"2024-09-10T22:10:38.261706+0000","last_became_peered":"2024-09-10T22:10:38.261706+0000","last_unstale":"2024-09-10T22:10:39.381513+0000","last_undegraded":"2024-09-10T22:10:39.381513+0000","last_fullsized":"2024-09-10T22:10:39.381513+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:27.116503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522775+0000","last_change":"2024-09-10T22:10:37.236693+0000","last_active":"2024-09-10T22:10:39.522775+0000","last_peered":"2024-09-10T22:10:39.522775+0000","last_clean":"2024-09-10T22:10:39.522775+0000","last_became_active":"2024-09-10T22:10:37.236322+0000","last_became_peered":"2024-09-10T22:10:37.236322+0000","last_unstale":"2024-09-10T22:10:39.522775+0000","last_undegraded":"2024-09-10T22:10:39.522775+0000","last_fullsized":"2024-09-10T22:10:39.522775+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:48.453353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528630+0000","last_change":"2024-09-10T22:10:38.263553+0000","last_active":"2024-09-10T22:10:39.528630+0000","last_peered":"2024-09-10T22:10:39.528630+0000","last_clean":"2024-09-10T22:10:39.528630+0000","last_became_active":"2024-09-10T22:10:38.263302+0000","last_became_peered":"2024-09-10T22:10:38.263302+0000","last_unstale":"2024-09-10T22:10:39.528630+0000","last_undegraded":"2024-09-10T22:10:39.528630+0000","last_fullsized":"2024-09-10T22:10:39.528630+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:08.130137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528811+0000","last_change":"2024-09-10T22:10:37.249927+0000","last_active":"2024-09-10T22:10:39.528811+0000","last_peered":"2024-09-10T22:10:39.528811+0000","last_clean":"2024-09-10T22:10:39.528811+0000","last_became_active":"2024-09-10T22:10:37.249443+0000","last_became_peered":"2024-09-10T22:10:37.249443+0000","last_unstale":"2024-09-10T22:10:39.528811+0000","last_undegraded":"2024-09-10T22:10:39.528811+0000","last_fullsized":"2024-09-10T22:10:39.528811+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:20.595899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524302+0000","last_change":"2024-09-10T22:10:38.249997+0000","last_active":"2024-09-10T22:10:39.524302+0000","last_peered":"2024-09-10T22:10:39.524302+0000","last_clean":"2024-09-10T22:10:39.524302+0000","last_became_active":"2024-09-10T22:10:38.249737+0000","last_became_peered":"2024-09-10T22:10:38.249737+0000","last_unstale":"2024-09-10T22:10:39.524302+0000","last_undegraded":"2024-09-10T22:10:39.524302+0000","last_fullsized":"2024-09-10T22:10:39.524302+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:19.948050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.522780+0000","last_change":"2024-09-10T22:10:37.245541+0000","last_active":"2024-09-10T22:10:39.522780+0000","last_peered":"2024-09-10T22:10:39.522780+0000","last_clean":"2024-09-10T22:10:39.522780+0000","last_became_active":"2024-09-10T22:10:37.245066+0000","last_became_peered":"2024-09-10T22:10:37.245066+0000","last_unstale":"2024-09-10T22:10:39.522780+0000","last_undegraded":"2024-09-10T22:10:39.522780+0000","last_fullsized":"2024-09-10T22:10:39.522780+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:40.695789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.254591+0000","last_change":"2024-09-10T22:10:38.262616+0000","last_active":"2024-09-10T22:10:39.254591+0000","last_peered":"2024-09-10T22:10:39.254591+0000","last_clean":"2024-09-10T22:10:39.254591+0000","last_became_active":"2024-09-10T22:10:38.260645+0000","last_became_peered":"2024-09-10T22:10:38.260645+0000","last_unstale":"2024-09-10T22:10:39.254591+0000","last_undegraded":"2024-09-10T22:10:39.254591+0000","last_fullsized":"2024-09-10T22:10:39.254591+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:41.698684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528261+0000","last_change":"2024-09-10T22:10:37.251944+0000","last_active":"2024-09-10T22:10:39.528261+0000","last_peered":"2024-09-10T22:10:39.528261+0000","last_clean":"2024-09-10T22:10:39.528261+0000","last_became_active":"2024-09-10T22:10:37.251665+0000","last_became_peered":"2024-09-10T22:10:37.251665+0000","last_unstale":"2024-09-10T22:10:39.528261+0000","last_undegraded":"2024-09-10T22:10:39.528261+0000","last_fullsized":"2024-09-10T22:10:39.528261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:55.320245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523396+0000","last_change":"2024-09-10T22:10:38.250847+0000","last_active":"2024-09-10T22:10:39.523396+0000","last_peered":"2024-09-10T22:10:39.523396+0000","last_clean":"2024-09-10T22:10:39.523396+0000","last_became_active":"2024-09-10T22:10:38.250591+0000","last_became_peered":"2024-09-10T22:10:38.250591+0000","last_unstale":"2024-09-10T22:10:39.523396+0000","last_undegraded":"2024-09-10T22:10:39.523396+0000","last_fullsized":"2024-09-10T22:10:39.523396+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:57.861382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.247352+0000","last_change":"2024-09-10T22:10:37.249468+0000","last_active":"2024-09-10T22:10:39.247352+0000","last_peered":"2024-09-10T22:10:39.247352+0000","last_clean":"2024-09-10T22:10:39.247352+0000","last_became_active":"2024-09-10T22:10:37.249196+0000","last_became_peered":"2024-09-10T22:10:37.249196+0000","last_unstale":"2024-09-10T22:10:39.247352+0000","last_undegraded":"2024-09-10T22:10:39.247352+0000","last_fullsized":"2024-09-10T22:10:39.247352+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:48.453353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.528272+0000","last_change":"2024-09-10T22:10:38.263051+0000","last_active":"2024-09-10T22:10:39.528272+0000","last_peered":"2024-09-10T22:10:39.528272+0000","last_clean":"2024-09-10T22:10:39.528272+0000","last_became_active":"2024-09-10T22:10:38.262760+0000","last_became_peered":"2024-09-10T22:10:38.262760+0000","last_unstale":"2024-09-10T22:10:39.528272+0000","last_undegraded":"2024-09-10T22:10:39.528272+0000","last_fullsized":"2024-09-10T22:10:39.528272+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:01.551745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.523361+0000","last_change":"2024-09-10T22:10:37.254152+0000","last_active":"2024-09-10T22:10:39.523361+0000","last_peered":"2024-09-10T22:10:39.523361+0000","last_clean":"2024-09-10T22:10:39.523361+0000","last_became_active":"2024-09-10T22:10:37.253879+0000","last_became_peered":"2024-09-10T22:10:37.253879+0000","last_unstale":"2024-09-10T22:10:39.523361+0000","last_undegraded":"2024-09-10T22:10:39.523361+0000","last_fullsized":"2024-09-10T22:10:39.523361+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:29.614570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.524110+0000","last_change":"2024-09-10T22:10:38.259332+0000","last_active":"2024-09-10T22:10:39.524110+0000","last_peered":"2024-09-10T22:10:39.524110+0000","last_clean":"2024-09-10T22:10:39.524110+0000","last_became_active":"2024-09-10T22:10:38.255536+0000","last_became_peered":"2024-09-10T22:10:38.255536+0000","last_unstale":"2024-09-10T22:10:39.524110+0000","last_undegraded":"2024-09-10T22:10:39.524110+0000","last_fullsized":"2024-09-10T22:10:39.524110+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:37.219743+0000","last_clean_scrub_stamp":"2024-09-10T22:10:37.219743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:52.177586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527667+0000","last_change":"2024-09-10T22:10:37.249681+0000","last_active":"2024-09-10T22:10:39.527667+0000","last_peered":"2024-09-10T22:10:39.527667+0000","last_clean":"2024-09-10T22:10:39.527667+0000","last_became_active":"2024-09-10T22:10:37.249376+0000","last_became_peered":"2024-09-10T22:10:37.249376+0000","last_unstale":"2024-09-10T22:10:39.527667+0000","last_undegraded":"2024-09-10T22:10:39.527667+0000","last_fullsized":"2024-09-10T22:10:39.527667+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:10:36.216847+0000","last_clean_scrub_stamp":"2024-09-10T22:10:36.216847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:07.127242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":260,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T22:10:39.527584+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:10:39.527584+0000","last_peered":"2024-09-10T22:10:39.527584+0000","last_clean":"2024-09-10T22:10:39.527584+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:10:39.527584+0000","last_undegraded":"2024-09-10T22:10:39.527584+0000","last_fullsized":"2024-09-10T22:10:39.527584+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123160,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248680,"statfs":{"total":96636764160,"available":96510648320,"internally_reserved":0,"allocated":1830912,"data_stored":1122124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509568,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130896,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240944,"statfs":{"total":96636764160,"available":96502726656,"internally_reserved":0,"allocated":1236992,"data_stored":532296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509567,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64272,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307568,"statfs":{"total":96636764160,"available":96570949632,"internally_reserved":0,"allocated":1232896,"data_stored":531714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509568,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144800,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227040,"statfs":{"total":96636764160,"available":96488488960,"internally_reserved":0,"allocated":1249280,"data_stored":533380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509566,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122580,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249260,"statfs":{"total":96636764160,"available":96511242240,"internally_reserved":0,"allocated":1245184,"data_stored":533278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509567,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126732,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245108,"statfs":{"total":96636764160,"available":96506990592,"internally_reserved":0,"allocated":1228800,"data_stored":531644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509567,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123148,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248692,"statfs":{"total":96636764160,"available":96510660608,"internally_reserved":0,"allocated":1826816,"data_stored":1122594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":124254665},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542273,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144776,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227064,"statfs":{"total":96636764160,"available":96488513536,"internally_reserved":0,"allocated":1232896,"data_stored":531678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:10:47.391 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:10:47.391 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T22:10:47.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 -- 172.21.15.66:0/3396561629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c340aee20 msgr2=0x7f9c340cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:47.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 --2- 172.21.15.66:0/3396561629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c340aee20 0x7f9c340cd930 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009c80 tx=0x7f9c2c01cd10 comp rx=0 tx=0).stop 2024-09-10T22:10:47.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 -- 172.21.15.66:0/3396561629 shutdown_connections 2024-09-10T22:10:47.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 --2- 172.21.15.66:0/3396561629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c340aee20 0x7f9c340cd930 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 --2- 172.21.15.66:0/3396561629 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c34159900 0x7f9c340ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 -- 172.21.15.66:0/3396561629 >> 172.21.15.66:0/3396561629 conn(0x7f9c340cfa30 msgr2=0x7f9c340cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:47.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 -- 172.21.15.66:0/3396561629 shutdown_connections 2024-09-10T22:10:47.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 -- 172.21.15.66:0/3396561629 wait complete. 2024-09-10T22:10:47.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 Processor -- start 2024-09-10T22:10:47.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 -- start start 2024-09-10T22:10:47.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c340aee20 0x7f9c34157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:47.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c34159900 0x7f9c341579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:47.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c3415cc80 0x7f9c34157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:47.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9c34168f70 con 0x7f9c3415cc80 2024-09-10T22:10:47.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c34168df0 con 0x7f9c340aee20 2024-09-10T22:10:47.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c3aa00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9c341690f0 con 0x7f9c34159900 2024-09-10T22:10:47.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c38a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c3415cc80 0x7f9c34157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:47.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c38a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c3415cc80 0x7f9c34157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54334/0 (socket says 172.21.15.66:54334) 2024-09-10T22:10:47.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c38a00640 1 -- 172.21.15.66:0/2079984053 learned_addr learned my addr 172.21.15.66:0/2079984053 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:47.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c2be00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c34159900 0x7f9c341579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:47.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c33e00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c340aee20 0x7f9c34157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:47.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c38a00640 1 -- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c34159900 msgr2=0x7f9c341579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:47.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c38a00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c34159900 0x7f9c341579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c38a00640 1 -- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c340aee20 msgr2=0x7f9c34157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:47.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c38a00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c340aee20 0x7f9c34157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c38a00640 1 -- 172.21.15.66:0/2079984053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3415df30 con 0x7f9c3415cc80 2024-09-10T22:10:47.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c33e00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c340aee20 0x7f9c34157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:47.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c38a00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c3415cc80 0x7f9c34157f10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c01b3c0 tx=0x7f9c2c0a5a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:47.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.584+0000 7f9c30c00640 1 -- 172.21.15.66:0/2079984053 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0bb070 con 0x7f9c3415cc80 2024-09-10T22:10:47.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.588+0000 7f9c30c00640 1 -- 172.21.15.66:0/2079984053 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c2c004d60 con 0x7f9c3415cc80 2024-09-10T22:10:47.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.588+0000 7f9c30c00640 1 -- 172.21.15.66:0/2079984053 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0b2040 con 0x7f9c3415cc80 2024-09-10T22:10:47.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.588+0000 7f9c3aa00640 1 -- 172.21.15.66:0/2079984053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c3415e210 con 0x7f9c3415cc80 2024-09-10T22:10:47.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.588+0000 7f9c3aa00640 1 -- 172.21.15.66:0/2079984053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c3415e7d0 con 0x7f9c3415cc80 2024-09-10T22:10:47.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.588+0000 7f9c3aa00640 1 -- 172.21.15.66:0/2079984053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c340cd930 con 0x7f9c3415cc80 2024-09-10T22:10:47.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.588+0000 7f9c30c00640 1 -- 172.21.15.66:0/2079984053 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f9c2c0b5030 con 0x7f9c3415cc80 2024-09-10T22:10:47.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.588+0000 7f9c30c00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9c10073bd0 0x7f9c10076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:47.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.588+0000 7f9c30c00640 1 -- 172.21.15.66:0/2079984053 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f9c2c12bb70 con 0x7f9c3415cc80 2024-09-10T22:10:47.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.588+0000 7f9c33e00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9c10073bd0 0x7f9c10076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:47.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.592+0000 7f9c33e00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9c10073bd0 0x7f9c10076090 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9c3415f0c0 tx=0x7f9c24008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:47.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.592+0000 7f9c30c00640 1 -- 172.21.15.66:0/2079984053 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9c2c016990 con 0x7f9c3415cc80 2024-09-10T22:10:47.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.828+0000 7f9c3aa00640 1 -- 172.21.15.66:0/2079984053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9c34070470 con 0x7f9c3415cc80 2024-09-10T22:10:47.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.832+0000 7f9c30c00640 1 -- 172.21.15.66:0/2079984053 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v31) ==== 190+0+147 (secure 0 0 0) 0x7f9c2c0f8ca0 con 0x7f9c3415cc80 2024-09-10T22:10:47.836 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T22:10:47.836 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw== 2024-09-10T22:10:47.837 INFO:teuthology.orchestra.run.smithi066.stdout: caps mds = "allow" 2024-09-10T22:10:47.837 INFO:teuthology.orchestra.run.smithi066.stdout: caps mon = "allow r" 2024-09-10T22:10:47.837 INFO:teuthology.orchestra.run.smithi066.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T22:10:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-09-10T22:10:47.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.836+0000 7f9c3aa00640 1 -- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9c10073bd0 msgr2=0x7f9c10076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.836+0000 7f9c3aa00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9c10073bd0 0x7f9c10076090 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9c3415f0c0 tx=0x7f9c24008040 comp rx=0 tx=0).stop 2024-09-10T22:10:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.836+0000 7f9c3aa00640 1 -- 172.21.15.66:0/2079984053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c3415cc80 msgr2=0x7f9c34157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.836+0000 7f9c3aa00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c3415cc80 0x7f9c34157f10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c01b3c0 tx=0x7f9c2c0a5a00 comp rx=0 tx=0).stop 2024-09-10T22:10:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.836+0000 7f9c3aa00640 1 -- 172.21.15.66:0/2079984053 shutdown_connections 2024-09-10T22:10:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.836+0000 7f9c3aa00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9c10073bd0 0x7f9c10076090 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.836+0000 7f9c3aa00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c3415cc80 0x7f9c34157f10 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.836+0000 7f9c3aa00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c34159900 0x7f9c341579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.836+0000 7f9c3aa00640 1 --2- 172.21.15.66:0/2079984053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c340aee20 0x7f9c34157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.836+0000 7f9c3aa00640 1 -- 172.21.15.66:0/2079984053 >> 172.21.15.66:0/2079984053 conn(0x7f9c340cfa30 msgr2=0x7f9c3415f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.836+0000 7f9c3aa00640 1 -- 172.21.15.66:0/2079984053 shutdown_connections 2024-09-10T22:10:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:47.836+0000 7f9c3aa00640 1 -- 172.21.15.66:0/2079984053 wait complete. 2024-09-10T22:10:47.858 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:10:48.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.000+0000 7f6667200640 1 -- 172.21.15.66:0/2235023549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66600cf180 msgr2=0x7f66600d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.000+0000 7f6667200640 1 --2- 172.21.15.66:0/2235023549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66600cf180 0x7f66600d04e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6660073840 tx=0x7f665801ccf0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.000+0000 7f6667200640 1 -- 172.21.15.66:0/2235023549 shutdown_connections 2024-09-10T22:10:48.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.000+0000 7f6667200640 1 --2- 172.21.15.66:0/2235023549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66600cf180 0x7f66600d04e0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.000+0000 7f6667200640 1 --2- 172.21.15.66:0/2235023549 >> v2:172.21.15.94:3300/0 conn(0x7f666015bd40 0x7f66600cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.000+0000 7f6667200640 1 -- 172.21.15.66:0/2235023549 >> 172.21.15.66:0/2235023549 conn(0x7f66600d1c60 msgr2=0x7f66600d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:48.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.000+0000 7f6667200640 1 -- 172.21.15.66:0/2235023549 shutdown_connections 2024-09-10T22:10:48.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.000+0000 7f6667200640 1 -- 172.21.15.66:0/2235023549 wait complete. 2024-09-10T22:10:48.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6667200640 1 Processor -- start 2024-09-10T22:10:48.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6667200640 1 -- start start 2024-09-10T22:10:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6667200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66600cf180 0x7f66600ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6667200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66600cf830 0x7f66601fa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6667200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f666015bd40 0x7f66601faf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6667200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6660168ba0 con 0x7f66600cf180 2024-09-10T22:10:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6667200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6660168a20 con 0x7f66600cf830 2024-09-10T22:10:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6667200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6660168d20 con 0x7f666015bd40 2024-09-10T22:10:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f665fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66600cf180 0x7f66600ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f665fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66600cf180 0x7f66600ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54350/0 (socket says 172.21.15.66:54350) 2024-09-10T22:10:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f665fe00640 1 -- 172.21.15.66:0/3363806322 learned_addr learned my addr 172.21.15.66:0/3363806322 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6657400640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66600cf830 0x7f66601fa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6665200640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f666015bd40 0x7f66601faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f665fe00640 1 -- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f666015bd40 msgr2=0x7f66601faf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f665fe00640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f666015bd40 0x7f66601faf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f665fe00640 1 -- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66600cf830 msgr2=0x7f66601fa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f665fe00640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66600cf830 0x7f66601fa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f665fe00640 1 -- 172.21.15.66:0/3363806322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66600b53a0 con 0x7f66600cf180 2024-09-10T22:10:48.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f665fe00640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66600cf180 0x7f66600ca720 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f66500058d0 tx=0x7f665000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:48.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f665cc00640 1 -- 172.21.15.66:0/3363806322 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650017070 con 0x7f66600cf180 2024-09-10T22:10:48.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f665cc00640 1 -- 172.21.15.66:0/3363806322 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6650009d20 con 0x7f66600cf180 2024-09-10T22:10:48.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6667200640 1 -- 172.21.15.66:0/3363806322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66600b5620 con 0x7f66600cf180 2024-09-10T22:10:48.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f665cc00640 1 -- 172.21.15.66:0/3363806322 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500106c0 con 0x7f66600cf180 2024-09-10T22:10:48.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6667200640 1 -- 172.21.15.66:0/3363806322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66600b5be0 con 0x7f66600cf180 2024-09-10T22:10:48.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.004+0000 7f6667200640 1 -- 172.21.15.66:0/3363806322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66600aec10 con 0x7f66600cf180 2024-09-10T22:10:48.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.008+0000 7f665cc00640 1 -- 172.21.15.66:0/3363806322 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f6650010860 con 0x7f66600cf180 2024-09-10T22:10:48.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.008+0000 7f665cc00640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f663c073bd0 0x7f663c076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.008+0000 7f665cc00640 1 -- 172.21.15.66:0/3363806322 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f66500927e0 con 0x7f66600cf180 2024-09-10T22:10:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.008+0000 7f6657400640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f663c073bd0 0x7f663c076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.008+0000 7f6657400640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f663c073bd0 0x7f663c076090 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f66600b6650 tx=0x7f6648009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:48.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.012+0000 7f665cc00640 1 -- 172.21.15.66:0/3363806322 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f66600b53a0 con 0x7f66600cf180 2024-09-10T22:10:48.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.220+0000 7f6667200640 1 -- 172.21.15.66:0/3363806322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66600b1350 con 0x7f66600cf180 2024-09-10T22:10:48.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f665cc00640 1 -- 172.21.15.66:0/3363806322 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5222 (secure 0 0 0) 0x7f665005eeb0 con 0x7f66600cf180 2024-09-10T22:10:48.226 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:10:48.227 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":66,"btime":"2024-09-10T22:10:41:252812+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3262462745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3262462745},{"type":"v1","addr":"172.21.15.66:6834","nonce":3262462745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/273414872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":273414872},{"type":"v1","addr":"172.21.15.94:6835","nonce":273414872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7246,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3486474426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3486474426},{"type":"v1","addr":"172.21.15.66:6835","nonce":3486474426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:10:37.643939+0000","modified":"2024-09-10T22:10:40.648893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7249":{"gid":7249,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/1920075377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1920075377},{"type":"v1","addr":"172.21.15.94:6837","nonce":1920075377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7249,"qdb_cluster":[7249]},"id":7}]} 2024-09-10T22:10:48.227 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 66 2024-09-10T22:10:48.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f6667200640 1 -- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f663c073bd0 msgr2=0x7f663c076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f6667200640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f663c073bd0 0x7f663c076090 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f66600b6650 tx=0x7f6648009040 comp rx=0 tx=0).stop 2024-09-10T22:10:48.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f6667200640 1 -- 172.21.15.66:0/3363806322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66600cf180 msgr2=0x7f66600ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f6667200640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66600cf180 0x7f66600ca720 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f66500058d0 tx=0x7f665000dde0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f6667200640 1 -- 172.21.15.66:0/3363806322 shutdown_connections 2024-09-10T22:10:48.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f6667200640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f663c073bd0 0x7f663c076090 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f6667200640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f666015bd40 0x7f66601faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f6667200640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66600cf830 0x7f66601fa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f6667200640 1 --2- 172.21.15.66:0/3363806322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66600cf180 0x7f66600ca720 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f6667200640 1 -- 172.21.15.66:0/3363806322 >> 172.21.15.66:0/3363806322 conn(0x7f66600d1c60 msgr2=0x7f666006dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:48.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f6667200640 1 -- 172.21.15.66:0/3363806322 shutdown_connections 2024-09-10T22:10:48.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.224+0000 7f6667200640 1 -- 172.21.15.66:0/3363806322 wait complete. 2024-09-10T22:10:48.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:10:37.643939+0000', 'modified': '2024-09-10T22:10:40.648893+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7249}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7249': {'gid': 7249, 'name': 'd', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6837/1920075377', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6836', 'nonce': 1920075377}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 1920075377}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7249, 'qdb_cluster': [7249]} 2024-09-10T22:10:48.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:10:48.652 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T22:10:48.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.796+0000 7ffa7ca00640 1 -- 172.21.15.66:0/4244192402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa5c004d50 msgr2=0x7ffa5c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.796+0000 7ffa7ca00640 1 --2- 172.21.15.66:0/4244192402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa5c004d50 0x7ffa5c0051d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000c640 tx=0x7ffa7001d320 comp rx=0 tx=0).stop 2024-09-10T22:10:48.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.796+0000 7ffa76800640 1 -- 172.21.15.66:0/4244192402 reap_dead start 2024-09-10T22:10:48.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.796+0000 7ffa7ca00640 1 -- 172.21.15.66:0/4244192402 shutdown_connections 2024-09-10T22:10:48.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.796+0000 7ffa7ca00640 1 -- 172.21.15.66:0/4244192402 >> 172.21.15.66:0/4244192402 conn(0x7ffa780cfa30 msgr2=0x7ffa780cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:48.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.796+0000 7ffa7ca00640 1 -- 172.21.15.66:0/4244192402 shutdown_connections 2024-09-10T22:10:48.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.796+0000 7ffa7ca00640 1 -- 172.21.15.66:0/4244192402 wait complete. 2024-09-10T22:10:48.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa7ca00640 1 Processor -- start 2024-09-10T22:10:48.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa7ca00640 1 -- start start 2024-09-10T22:10:48.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa7ca00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa780b54d0 0x7ffa780ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa7ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa781f8370 0x7ffa780ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa7ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa781f4110 0x7ffa781f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa7ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffa780cc130 con 0x7ffa781f4110 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa7ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa780cbfb0 con 0x7ffa781f8370 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa7ca00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ffa780cc2b0 con 0x7ffa780b54d0 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa76800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa781f4110 0x7ffa781f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa76800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa781f4110 0x7ffa781f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54370/0 (socket says 172.21.15.66:54370) 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa76800640 1 -- 172.21.15.66:0/1430335572 learned_addr learned my addr 172.21.15.66:0/1430335572 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa75e00640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa780b54d0 0x7ffa780ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa75400640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa781f8370 0x7ffa780ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa76800640 1 -- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa780b54d0 msgr2=0x7ffa780ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa76800640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa780b54d0 0x7ffa780ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa76800640 1 -- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa781f8370 msgr2=0x7ffa780ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa76800640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa781f8370 0x7ffa780ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa76800640 1 -- 172.21.15.66:0/1430335572 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa781f78b0 con 0x7ffa781f4110 2024-09-10T22:10:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa75400640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa781f8370 0x7ffa780ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa76800640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa781f4110 0x7ffa781f4580 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000c640 tx=0x7ffa700ad8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa6e000640 1 -- 172.21.15.66:0/1430335572 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa700011d0 con 0x7ffa781f4110 2024-09-10T22:10:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa6e000640 1 -- 172.21.15.66:0/1430335572 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa700bb070 con 0x7ffa781f4110 2024-09-10T22:10:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa6e000640 1 -- 172.21.15.66:0/1430335572 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70003a60 con 0x7ffa781f4110 2024-09-10T22:10:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa7ca00640 1 -- 172.21.15.66:0/1430335572 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa781f4e50 con 0x7ffa781f4110 2024-09-10T22:10:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.800+0000 7ffa7ca00640 1 -- 172.21.15.66:0/1430335572 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa781f7e50 con 0x7ffa781f4110 2024-09-10T22:10:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.804+0000 7ffa6e000640 1 -- 172.21.15.66:0/1430335572 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7ffa70003c00 con 0x7ffa781f4110 2024-09-10T22:10:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.804+0000 7ffa6e000640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa58073bd0 0x7ffa58076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.804+0000 7ffa6e000640 1 -- 172.21.15.66:0/1430335572 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7ffa7012cb10 con 0x7ffa781f4110 2024-09-10T22:10:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 Processor -- start 2024-09-10T22:10:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa75e00640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa58073bd0 0x7ffa58076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 -- start start 2024-09-10T22:10:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa75e00640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa58073bd0 0x7ffa58076090 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ffa780c8230 tx=0x7ffa680091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a769d0 0x5587f4a76db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a772f0 0x5587f4a2a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a2ae10 0x5587f4a2b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5587f494b3f0 con 0x5587f4a769d0 2024-09-10T22:10:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5587f4888b20 con 0x5587f4a772f0 2024-09-10T22:10:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x5587f4877f50 con 0x5587f4a2ae10 2024-09-10T22:10:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa75e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a769d0 0x5587f4a76db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa75e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a769d0 0x5587f4a76db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54382/0 (socket says 172.21.15.66:54382) 2024-09-10T22:10:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa75e00640 1 -- 172.21.15.66:0/1659330631 learned_addr learned my addr 172.21.15.66:0/1659330631 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa76800640 1 --2- 172.21.15.66:0/1659330631 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a2ae10 0x5587f4a2b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa75400640 1 --2- 172.21.15.66:0/1659330631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a772f0 0x5587f4a2a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa75e00640 1 -- 172.21.15.66:0/1659330631 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a2ae10 msgr2=0x5587f4a2b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:10:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa75e00640 1 --2- 172.21.15.66:0/1659330631 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a2ae10 0x5587f4a2b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa75e00640 1 -- 172.21.15.66:0/1659330631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a772f0 msgr2=0x5587f4a2a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:10:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa75e00640 1 --2- 172.21.15.66:0/1659330631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a772f0 0x5587f4a2a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa75e00640 1 -- 172.21.15.66:0/1659330631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587f483f590 con 0x5587f4a769d0 2024-09-10T22:10:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa75e00640 1 --2- 172.21.15.66:0/1659330631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a769d0 0x5587f4a76db0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ffa68004140 tx=0x7ffa68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b191bda5682f45ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa62a00640 1 -- 172.21.15.66:0/1659330631 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68003bc0 con 0x5587f4a769d0 2024-09-10T22:10:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa62a00640 1 -- 172.21.15.66:0/1659330631 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa68003d60 con 0x5587f4a769d0 2024-09-10T22:10:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa62a00640 1 -- 172.21.15.66:0/1659330631 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa680296e0 con 0x5587f4a769d0 2024-09-10T22:10:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 -- 172.21.15.66:0/1659330631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a769d0 msgr2=0x5587f4a76db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/1659330631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a769d0 0x5587f4a76db0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ffa68004140 tx=0x7ffa68017000 comp rx=0 tx=0).stop 2024-09-10T22:10:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 -- 172.21.15.66:0/1659330631 shutdown_connections 2024-09-10T22:10:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/1659330631 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a2ae10 0x5587f4a2b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/1659330631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a772f0 0x5587f4a2a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/1659330631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a769d0 0x5587f4a76db0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 -- 172.21.15.66:0/1659330631 >> 172.21.15.66:0/1659330631 conn(0x5587f4914130 msgr2=0x5587f4a59e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 -- 172.21.15.66:0/1659330631 shutdown_connections 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 -- 172.21.15.66:0/1659330631 wait complete. 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.808+0000 7ffa7fd81000 1 -- start start 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 1 Processor -- start 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a2ae10 0x5587f4a576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a769d0 0x5587f4a57e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a772f0 0x5587f4a583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5587f48125d0 con 0x5587f4a2ae10 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5587f487bd60 con 0x5587f4a769d0 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x5587f481e500 con 0x5587f4a772f0 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 10 client.0 _ll_register_callbacks cb 0x5587f4822810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa75e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a2ae10 0x5587f4a576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa75e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a2ae10 0x5587f4a576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54390/0 (socket says 172.21.15.66:54390) 2024-09-10T22:10:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa75e00640 1 -- 172.21.15.66:0/2943817723 learned_addr learned my addr 172.21.15.66:0/2943817723 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa75400640 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a769d0 0x5587f4a57e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa76800640 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a772f0 0x5587f4a583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa75e00640 1 -- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a772f0 msgr2=0x5587f4a583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa75e00640 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a772f0 0x5587f4a583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa75e00640 1 -- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a769d0 msgr2=0x5587f4a57e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa75e00640 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a769d0 0x5587f4a57e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa75e00640 1 -- 172.21.15.66:0/2943817723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587f4816170 con 0x5587f4a2ae10 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa76800640 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a772f0 0x5587f4a583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa75400640 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a769d0 0x5587f4a57e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa75e00640 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a2ae10 0x5587f4a576a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ffa68012cb0 tx=0x7ffa680098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3838e73e5ec89089 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa57400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa57400640 1 -- 172.21.15.66:0/2943817723 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6802aa60 con 0x5587f4a2ae10 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa57400640 1 -- 172.21.15.66:0/2943817723 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7ffa6802ac00 con 0x5587f4a2ae10 2024-09-10T22:10:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 1 -- 172.21.15.66:0/2943817723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5587f48100f0 con 0x5587f4a2ae10 2024-09-10T22:10:48.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa57400640 1 -- 172.21.15.66:0/2943817723 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7ffa6803b050 con 0x5587f4a2ae10 2024-09-10T22:10:48.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 10 client.7965 fetch_fsmap learned FSMap version 66 2024-09-10T22:10:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 1 -- 172.21.15.66:0/2943817723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5587f48fcef0 con 0x5587f4a2ae10 2024-09-10T22:10:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa57400640 1 -- 172.21.15.66:0/2943817723 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5587f48100f0 con 0x5587f4a2ae10 2024-09-10T22:10:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa57400640 1 -- 172.21.15.66:0/2943817723 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 66) ==== 3502+0+0 (secure 0 0 0) 0x7ffa6803e020 con 0x5587f4a2ae10 2024-09-10T22:10:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 10 client.7965 fetch_fsmap finished waiting for FSMap version 66 2024-09-10T22:10:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 10 client.7965 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T22:10:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 20 client.7965 populate_metadata read hostname 'smithi066' 2024-09-10T22:10:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:6836/1920075377,v1:172.21.15.94:6837/1920075377] conn(0x5587f4a590f0 0x5587f4a88c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 4 client.7965 mds_command: new command op to 7249 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:10:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa7fd81000 1 -- 172.21.15.66:0/2943817723 --> [v2:172.21.15.94:6836/1920075377,v1:172.21.15.94:6837/1920075377] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587f488aa70 con 0x5587f4a590f0 2024-09-10T22:10:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.812+0000 7ffa75400640 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:6836/1920075377,v1:172.21.15.94:6837/1920075377] conn(0x5587f4a590f0 0x5587f4a88c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa75400640 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:6836/1920075377,v1:172.21.15.94:6837/1920075377] conn(0x5587f4a590f0 0x5587f4a88c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30e19aa31358d18e server_cookie=41d07617b2b17366 in_seq=0 out_seq=0 2024-09-10T22:10:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa57400640 10 client.7965 ms_handle_connect on v2:172.21.15.94:6836/1920075377 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa57400640 1 -- 172.21.15.66:0/2943817723 <== mds.0 v2:172.21.15.94:6836/1920075377 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5587f488aa70 con 0x5587f4a590f0 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa57400640 10 client.7965 handle_command_reply: tid=0 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa7fd81000 1 client.7965 shutdown 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa7fd81000 1 -- 172.21.15.66:0/2943817723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a2ae10 msgr2=0x5587f4a576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a2ae10 0x5587f4a576a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ffa68012cb0 tx=0x7ffa680098f0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa7fd81000 1 -- 172.21.15.66:0/2943817723 shutdown_connections 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:6836/1920075377,v1:172.21.15.94:6837/1920075377] conn(0x5587f4a590f0 0x5587f4a88c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a772f0 0x5587f4a583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a769d0 0x5587f4a57e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/2943817723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a2ae10 0x5587f4a576a0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa57400640 0 client.7965 ms_handle_reset on v2:172.21.15.94:6836/1920075377 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa7fd81000 1 -- 172.21.15.66:0/2943817723 >> 172.21.15.66:0/2943817723 conn(0x5587f4914130 msgr2=0x5587f4a29fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa7fd81000 1 -- 172.21.15.66:0/2943817723 shutdown_connections 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa7fd81000 1 -- 172.21.15.66:0/2943817723 wait complete. 2024-09-10T22:10:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.816+0000 7ffa7fd81000 20 client.7965 trim_cache size 0 max 16384 2024-09-10T22:10:48.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa7fd81000 1 Processor -- start 2024-09-10T22:10:48.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa7fd81000 1 -- start start 2024-09-10T22:10:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa7fd81000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a2ae10 0x5587f4ae8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa7fd81000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a769d0 0x5587f4ae9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa7fd81000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 0x5587f4ae9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa7fd81000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffa782017d0 con 0x5587f4a772f0 2024-09-10T22:10:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa7fd81000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa782007d0 con 0x5587f4a769d0 2024-09-10T22:10:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa7fd81000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ffa781ffb60 con 0x5587f4a2ae10 2024-09-10T22:10:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa76800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 0x5587f4ae9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa76800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 0x5587f4ae9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54396/0 (socket says 172.21.15.66:54396) 2024-09-10T22:10:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa76800640 1 -- 172.21.15.66:0/3301180898 learned_addr learned my addr 172.21.15.66:0/3301180898 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa75400640 1 --2- 172.21.15.66:0/3301180898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a769d0 0x5587f4ae9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa75e00640 1 --2- 172.21.15.66:0/3301180898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a2ae10 0x5587f4ae8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa76800640 1 -- 172.21.15.66:0/3301180898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a2ae10 msgr2=0x5587f4ae8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:10:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa76800640 1 --2- 172.21.15.66:0/3301180898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a2ae10 0x5587f4ae8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa76800640 1 -- 172.21.15.66:0/3301180898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a769d0 msgr2=0x5587f4ae9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:10:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa76800640 1 --2- 172.21.15.66:0/3301180898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a769d0 0x5587f4ae9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa76800640 1 -- 172.21.15.66:0/3301180898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587f48d3d90 con 0x5587f4a772f0 2024-09-10T22:10:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.836+0000 7ffa76800640 1 --2- 172.21.15.66:0/3301180898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 0x5587f4ae9640 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ffa700f6870 tx=0x7ffa700f8e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76226097bd5f755c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa62000640 1 -- 172.21.15.66:0/3301180898 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70102450 con 0x5587f4a772f0 2024-09-10T22:10:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa62000640 1 -- 172.21.15.66:0/3301180898 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa700f9600 con 0x5587f4a772f0 2024-09-10T22:10:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa62000640 1 -- 172.21.15.66:0/3301180898 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7012d000 con 0x5587f4a772f0 2024-09-10T22:10:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 -- 172.21.15.66:0/3301180898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 msgr2=0x5587f4ae9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/3301180898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 0x5587f4ae9640 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ffa700f6870 tx=0x7ffa700f8e80 comp rx=0 tx=0).stop 2024-09-10T22:10:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 -- 172.21.15.66:0/3301180898 shutdown_connections 2024-09-10T22:10:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/3301180898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 0x5587f4ae9640 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/3301180898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a769d0 0x5587f4ae9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/3301180898 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a2ae10 0x5587f4ae8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 -- 172.21.15.66:0/3301180898 >> 172.21.15.66:0/3301180898 conn(0x5587f4914130 msgr2=0x5587f4a83fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 -- 172.21.15.66:0/3301180898 shutdown_connections 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 -- 172.21.15.66:0/3301180898 wait complete. 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 -- start start 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 Processor -- start 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a2ae10 0x5587f4aecc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a769d0 0x5587f4aed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 0x5587f4aed680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5587f48878f0 con 0x5587f4a772f0 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5587f487b070 con 0x5587f4a2ae10 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x5587f4937730 con 0x5587f4a769d0 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 10 client.0 _ll_register_callbacks cb 0x7ffa64009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa76800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 0x5587f4aed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa76800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 0x5587f4aed680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54400/0 (socket says 172.21.15.66:54400) 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa76800640 1 -- 172.21.15.66:0/26922035 learned_addr learned my addr 172.21.15.66:0/26922035 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa75e00640 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a2ae10 0x5587f4aecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa75400640 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a769d0 0x5587f4aed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa76800640 1 -- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a769d0 msgr2=0x5587f4aed140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa76800640 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a769d0 0x5587f4aed140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa76800640 1 -- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a2ae10 msgr2=0x5587f4aecc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa76800640 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a2ae10 0x5587f4aecc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa76800640 1 -- 172.21.15.66:0/26922035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587f483ea90 con 0x5587f4a772f0 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa75400640 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a769d0 0x5587f4aed140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa76800640 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 0x5587f4aed680 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ffa7001dbc0 tx=0x7ffa7012d090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8436c95a451c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa57400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa57400640 1 -- 172.21.15.66:0/26922035 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70102450 con 0x5587f4a772f0 2024-09-10T22:10:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa57400640 1 -- 172.21.15.66:0/26922035 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7ffa700f8860 con 0x5587f4a772f0 2024-09-10T22:10:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.840+0000 7ffa7fd81000 1 -- 172.21.15.66:0/26922035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa700f9600 con 0x5587f4a772f0 2024-09-10T22:10:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa57400640 1 -- 172.21.15.66:0/26922035 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7ffa700fe0e0 con 0x5587f4a772f0 2024-09-10T22:10:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 10 client.7971 fetch_fsmap learned FSMap version 66 2024-09-10T22:10:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 -- 172.21.15.66:0/26922035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5587f481d050 con 0x5587f4a772f0 2024-09-10T22:10:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa57400640 1 -- 172.21.15.66:0/26922035 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa700f9600 con 0x5587f4a772f0 2024-09-10T22:10:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa57400640 1 -- 172.21.15.66:0/26922035 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 66) ==== 3502+0+0 (secure 0 0 0) 0x7ffa700f4f50 con 0x5587f4a772f0 2024-09-10T22:10:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 10 client.7971 fetch_fsmap finished waiting for FSMap version 66 2024-09-10T22:10:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 10 client.7971 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T22:10:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 20 client.7971 populate_metadata read hostname 'smithi066' 2024-09-10T22:10:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:6836/1920075377,v1:172.21.15.94:6837/1920075377] conn(0x5587f4a8abe0 0x5587f4a8afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 4 client.7971 mds_command: new command op to 7249 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:10:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 -- 172.21.15.66:0/26922035 --> [v2:172.21.15.94:6836/1920075377,v1:172.21.15.94:6837/1920075377] -- command(tid 0: {"prefix": "status"}) -- 0x5587f4886520 con 0x5587f4a8abe0 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa75e00640 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:6836/1920075377,v1:172.21.15.94:6837/1920075377] conn(0x5587f4a8abe0 0x5587f4a8afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa75e00640 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:6836/1920075377,v1:172.21.15.94:6837/1920075377] conn(0x5587f4a8abe0 0x5587f4a8afc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f8f02104d656c4b server_cookie=702871951dd2111 in_seq=0 out_seq=0 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa57400640 10 client.7971 ms_handle_connect on v2:172.21.15.94:6836/1920075377 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa57400640 1 -- 172.21.15.66:0/26922035 <== mds.0 v2:172.21.15.94:6836/1920075377 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5587f4886520 con 0x5587f4a8abe0 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa57400640 10 client.7971 handle_command_reply: tid=0 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 client.7971 shutdown 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 -- 172.21.15.66:0/26922035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 msgr2=0x5587f4aed680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 0x5587f4aed680 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ffa7001dbc0 tx=0x7ffa7012d090 comp rx=0 tx=0).stop 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 -- 172.21.15.66:0/26922035 shutdown_connections 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:6836/1920075377,v1:172.21.15.94:6837/1920075377] conn(0x5587f4a8abe0 0x5587f4a8afc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587f4a772f0 0x5587f4aed680 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5587f4a769d0 0x5587f4aed140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa57400640 0 client.7971 ms_handle_reset on v2:172.21.15.94:6836/1920075377 2024-09-10T22:10:48.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 --2- 172.21.15.66:0/26922035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587f4a2ae10 0x5587f4aecc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 -- 172.21.15.66:0/26922035 >> 172.21.15.66:0/26922035 conn(0x5587f4914130 msgr2=0x5587f4a32b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:48.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 -- 172.21.15.66:0/26922035 shutdown_connections 2024-09-10T22:10:48.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 1 -- 172.21.15.66:0/26922035 wait complete. 2024-09-10T22:10:48.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7fd81000 20 client.7971 trim_cache size 0 max 16384 2024-09-10T22:10:48.852 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:10:48.852 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:10:48.852 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:10:48.852 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 7249, 2024-09-10T22:10:48.852 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:10:48.852 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:10:48.853 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:10:48.853 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 10.616988556000001, 2024-09-10T22:10:48.853 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 66, 2024-09-10T22:10:48.853 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 100, 2024-09-10T22:10:48.853 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 100, 2024-09-10T22:10:48.853 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 14.980134494 2024-09-10T22:10:48.853 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:10:48.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7ca00640 1 -- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa58073bd0 msgr2=0x7ffa58076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7ca00640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa58073bd0 0x7ffa58076090 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ffa780c8230 tx=0x7ffa680091e0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7ca00640 1 -- 172.21.15.66:0/1430335572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa781f4110 msgr2=0x7ffa781f4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7ca00640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa781f4110 0x7ffa781f4580 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000c640 tx=0x7ffa700ad8f0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7ca00640 1 -- 172.21.15.66:0/1430335572 shutdown_connections 2024-09-10T22:10:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7ca00640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffa58073bd0 0x7ffa58076090 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7ca00640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa781f4110 0x7ffa781f4580 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7ca00640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa781f8370 0x7ffa780ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7ca00640 1 --2- 172.21.15.66:0/1430335572 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa780b54d0 0x7ffa780ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7ca00640 1 -- 172.21.15.66:0/1430335572 >> 172.21.15.66:0/1430335572 conn(0x7ffa780cfa30 msgr2=0x7ffa780af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7ca00640 1 -- 172.21.15.66:0/1430335572 shutdown_connections 2024-09-10T22:10:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:48.844+0000 7ffa7ca00640 1 -- 172.21.15.66:0/1430335572 wait complete. 2024-09-10T22:10:48.869 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:10:49.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.012+0000 7f1848e00640 1 -- 172.21.15.66:0/2994926561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18440aee20 msgr2=0x7f18440ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:49.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.012+0000 7f1848e00640 1 --2- 172.21.15.66:0/2994926561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18440aee20 0x7f18440ca080 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f182c009870 tx=0x7f182c01c6f0 comp rx=0 tx=0).stop 2024-09-10T22:10:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.012+0000 7f1848e00640 1 -- 172.21.15.66:0/2994926561 shutdown_connections 2024-09-10T22:10:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.012+0000 7f1848e00640 1 --2- 172.21.15.66:0/2994926561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18440aee20 0x7f18440ca080 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.012+0000 7f1848e00640 1 -- 172.21.15.66:0/2994926561 >> 172.21.15.66:0/2994926561 conn(0x7f18440cfa30 msgr2=0x7f18440cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.012+0000 7f1848e00640 1 -- 172.21.15.66:0/2994926561 shutdown_connections 2024-09-10T22:10:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.012+0000 7f1848e00640 1 -- 172.21.15.66:0/2994926561 wait complete. 2024-09-10T22:10:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.012+0000 7f1848e00640 1 Processor -- start 2024-09-10T22:10:49.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.012+0000 7f1848e00640 1 -- start start 2024-09-10T22:10:49.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1848e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18440aee20 0x7f184415e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:49.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1848e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1844159900 0x7f18440af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:49.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1848e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f184415cc80 0x7f18440b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:49.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1848e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f184416a360 con 0x7f1844159900 2024-09-10T22:10:49.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1848e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f184416a1e0 con 0x7f184415cc80 2024-09-10T22:10:49.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1848e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f184416a4e0 con 0x7f18440aee20 2024-09-10T22:10:49.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f183be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1844159900 0x7f18440af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:49.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f183be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1844159900 0x7f18440af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54416/0 (socket says 172.21.15.66:54416) 2024-09-10T22:10:49.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f183be00640 1 -- 172.21.15.66:0/4263147351 learned_addr learned my addr 172.21.15.66:0/4263147351 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:49.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1842800640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f184415cc80 0x7f18440b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:49.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1841e00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18440aee20 0x7f184415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:49.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f183be00640 1 -- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18440aee20 msgr2=0x7f184415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:49.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f183be00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18440aee20 0x7f184415e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:49.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f183be00640 1 -- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f184415cc80 msgr2=0x7f18440b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:49.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f183be00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f184415cc80 0x7f18440b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:49.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f183be00640 1 -- 172.21.15.66:0/4263147351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18441544f0 con 0x7f1844159900 2024-09-10T22:10:49.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1842800640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f184415cc80 0x7f18440b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:49.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1841e00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18440aee20 0x7f184415e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:10:49.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f183be00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1844159900 0x7f18440af7c0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f182c000c10 tx=0x7f182c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:49.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f183a000640 1 -- 172.21.15.66:0/4263147351 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c0ad070 con 0x7f1844159900 2024-09-10T22:10:49.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f183a000640 1 -- 172.21.15.66:0/4263147351 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f182c0040f0 con 0x7f1844159900 2024-09-10T22:10:49.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f183a000640 1 -- 172.21.15.66:0/4263147351 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c0044c0 con 0x7f1844159900 2024-09-10T22:10:49.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1848e00640 1 -- 172.21.15.66:0/4263147351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18441547d0 con 0x7f1844159900 2024-09-10T22:10:49.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1848e00640 1 -- 172.21.15.66:0/4263147351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18440b03f0 con 0x7f1844159900 2024-09-10T22:10:49.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.016+0000 7f1848e00640 1 -- 172.21.15.66:0/4263147351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18440ca080 con 0x7f1844159900 2024-09-10T22:10:49.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.020+0000 7f183a000640 1 -- 172.21.15.66:0/4263147351 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f18440ce850 con 0x7f1844159900 2024-09-10T22:10:49.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.020+0000 7f183a000640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1820073bd0 0x7f1820076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:49.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.020+0000 7f183a000640 1 -- 172.21.15.66:0/4263147351 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f182c12c470 con 0x7f1844159900 2024-09-10T22:10:49.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.020+0000 7f1841e00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1820073bd0 0x7f1820076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:49.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.020+0000 7f1841e00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1820073bd0 0x7f1820076090 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1834001230 tx=0x7f1834009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:49.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.024+0000 7f183a000640 1 -- 172.21.15.66:0/4263147351 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f182c016370 con 0x7f1844159900 2024-09-10T22:10:49.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.232+0000 7f1848e00640 1 -- 172.21.15.66:0/4263147351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1844070470 con 0x7f1844159900 2024-09-10T22:10:49.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.232+0000 7f183a000640 1 -- 172.21.15.66:0/4263147351 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) ==== 53+0+73 (secure 0 0 0) 0x7f182c0f95a0 con 0x7f1844159900 2024-09-10T22:10:49.237 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:10:49.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.236+0000 7f1848e00640 1 -- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1820073bd0 msgr2=0x7f1820076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:49.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.236+0000 7f1848e00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1820073bd0 0x7f1820076090 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1834001230 tx=0x7f1834009040 comp rx=0 tx=0).stop 2024-09-10T22:10:49.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.236+0000 7f1848e00640 1 -- 172.21.15.66:0/4263147351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1844159900 msgr2=0x7f18440af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:49.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.236+0000 7f1848e00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1844159900 0x7f18440af7c0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f182c000c10 tx=0x7f182c0056d0 comp rx=0 tx=0).stop 2024-09-10T22:10:49.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.236+0000 7f1848e00640 1 -- 172.21.15.66:0/4263147351 shutdown_connections 2024-09-10T22:10:49.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.236+0000 7f1848e00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1820073bd0 0x7f1820076090 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:49.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.236+0000 7f1848e00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f184415cc80 0x7f18440b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:49.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.236+0000 7f1848e00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1844159900 0x7f18440af7c0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:49.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.236+0000 7f1848e00640 1 --2- 172.21.15.66:0/4263147351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18440aee20 0x7f184415e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:49.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.236+0000 7f1848e00640 1 -- 172.21.15.66:0/4263147351 >> 172.21.15.66:0/4263147351 conn(0x7f18440cfa30 msgr2=0x7f18440cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:49.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.236+0000 7f1848e00640 1 -- 172.21.15.66:0/4263147351 shutdown_connections 2024-09-10T22:10:49.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49.236+0000 7f1848e00640 1 -- 172.21.15.66:0/4263147351 wait complete. 2024-09-10T22:10:49.258 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:10:49.258 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:10:49.258 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:10:49.258 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:10:49.258 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:10:49.258 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:10:49.259 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:10:49.259 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:10:49.259 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:10:49.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:49.259 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:10:49.268 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:10:49.268 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:10:49.269 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41825sec preferred_lft 41825sec 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:81:fc:4c:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:10:49.270 INFO:teuthology.orchestra.run.smithi066.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:10:49.271 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:10:49.271 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T22:10:49.271 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:10:49.271 INFO:teuthology.orchestra.run.smithi066.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:10:49.271 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:10:49.271 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T22:10:49.271 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:10:49.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:49.272 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:10:49.335 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:10:49.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:49.337 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:10:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:10:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:10:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:10:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:10:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:10:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:10:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:10:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:10:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:10:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:10:49.398 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:10:49.398 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:10:49.398 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:10:49.398 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:10:49.398 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:10:49.398 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:10:49.398 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:10:49.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:10:49.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:10:49.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:49.534 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:10:49.543 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:10:49.543 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:10:49.543 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:10:49.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:49.544 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:10:49.605 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T22:10:49.606 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:10:49.606 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:10:49.661 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:10:49.662 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:10:49.662 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:10:49.717 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:10:49.778 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:10:49.831 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:10:49.831 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:10:49.831 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:10:49.831 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:10:49.831 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:10:49.832 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:10:49.833 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:10:49.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:49.834 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:10:49.890 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:10:49.890 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T22:10:49.940 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:10:49.961 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:10:50.011 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:10:50.011 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:10:50.011 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:10:50.011 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:10:50.011 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:10:50.011 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:10:50.011 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:10:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:10:50.013 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:10:50.013 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:10:50.013 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:10:50.013 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:10:50.013 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:10:50.013 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:10:50.013 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:10:50.013 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:10:50.013 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:10:50.013 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:10:50.013 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:10:50.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:50.014 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:10:50.037 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-09-10T22:10:50.032+0000 7f7f27f46580 -1 init, newargv = 0x5573149f7c50 newargc=15 2024-09-10T22:10:50.037 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[48229]: starting ceph client 2024-09-10T22:10:50.045 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[48229]: starting fuse 2024-09-10T22:10:50.046 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-09-10T22:10:50.046 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T22:10:50.047 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:10:50.047 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-09-10T22:10:50.047 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-09-10T22:10:50.047 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:10:50.047 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-09-10T22:10:50.047 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:10:50.047 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-09-10T22:10:50.047 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T22:10:50.047 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-09-10T22:10:50.047 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-09-10T22:10:50.048 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-09-10T22:10:50.049 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:10:50.049 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:10:50.049 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-09-10T22:10:50.049 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:10:50.173 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.48229.asok 2024-09-10T22:10:50.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:10:50.185 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48229.asok 2024-09-10T22:10:50.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:50.185 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48229.asok status 2024-09-10T22:10:50.358 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "48229", 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 7980, 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-09-10T22:10:50.359 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 7980 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2906850933 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2906850933 2024-09-10T22:10:50.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:10:50.361 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.7980 192.168.0.1:0/2906850933", 2024-09-10T22:10:50.361 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/2906850933", 2024-09-10T22:10:50.361 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-09-10T22:10:50.361 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 66, 2024-09-10T22:10:50.361 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 100, 2024-09-10T22:10:50.361 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:10:50.361 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-09-10T22:10:50.361 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-09-10T22:10:50.361 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:10:50.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:50.374 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-09-10T22:10:50.374 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:10:50.384 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-09-10T22:10:50.384 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T22:10:50.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:50.384 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:10:50.446 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T22:10:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.640+0000 7fce28200640 1 -- 172.21.15.66:0/2465557649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce2015ab00 msgr2=0x7fce2006e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.640+0000 7fce28200640 1 --2- 172.21.15.66:0/2465557649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce2015ab00 0x7fce2006e6a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fce14009840 tx=0x7fce1401c6d0 comp rx=0 tx=0).stop 2024-09-10T22:10:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.640+0000 7fce28200640 1 -- 172.21.15.66:0/2465557649 shutdown_connections 2024-09-10T22:10:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.640+0000 7fce28200640 1 --2- 172.21.15.66:0/2465557649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce2015ab00 0x7fce2006e6a0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.640+0000 7fce28200640 1 -- 172.21.15.66:0/2465557649 >> 172.21.15.66:0/2465557649 conn(0x7fce200cfaa0 msgr2=0x7fce200cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.640+0000 7fce28200640 1 -- 172.21.15.66:0/2465557649 shutdown_connections 2024-09-10T22:10:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.640+0000 7fce28200640 1 -- 172.21.15.66:0/2465557649 wait complete. 2024-09-10T22:10:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.640+0000 7fce28200640 1 Processor -- start 2024-09-10T22:10:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.640+0000 7fce28200640 1 -- start start 2024-09-10T22:10:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce28200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce20072d50 0x7fce20158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce28200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce200cce70 0x7fce20142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce28200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce2015ab00 0x7fce201f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce28200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce201692b0 con 0x7fce20072d50 2024-09-10T22:10:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce28200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce20169130 con 0x7fce200cce70 2024-09-10T22:10:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce28200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fce20169430 con 0x7fce2015ab00 2024-09-10T22:10:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce25800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce20072d50 0x7fce20158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce25800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce20072d50 0x7fce20158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54430/0 (socket says 172.21.15.66:54430) 2024-09-10T22:10:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce25800640 1 -- 172.21.15.66:0/3116890198 learned_addr learned my addr 172.21.15.66:0/3116890198 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce24e00640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce200cce70 0x7fce20142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce26200640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce2015ab00 0x7fce201f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce25800640 1 -- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce2015ab00 msgr2=0x7fce201f97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce25800640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce2015ab00 0x7fce201f97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce25800640 1 -- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce200cce70 msgr2=0x7fce20142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce25800640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce200cce70 0x7fce20142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce25800640 1 -- 172.21.15.66:0/3116890198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce20157ce0 con 0x7fce20072d50 2024-09-10T22:10:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce26200640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce2015ab00 0x7fce201f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:10:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce25800640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce20072d50 0x7fce20158780 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fce14009790 tx=0x7fce14004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce1d600640 1 -- 172.21.15.66:0/3116890198 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce140bb020 con 0x7fce20072d50 2024-09-10T22:10:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce1d600640 1 -- 172.21.15.66:0/3116890198 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce140043e0 con 0x7fce20072d50 2024-09-10T22:10:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce1d600640 1 -- 172.21.15.66:0/3116890198 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14005290 con 0x7fce20072d50 2024-09-10T22:10:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce28200640 1 -- 172.21.15.66:0/3116890198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce201f9d20 con 0x7fce20072d50 2024-09-10T22:10:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce28200640 1 -- 172.21.15.66:0/3116890198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce20157f80 con 0x7fce20072d50 2024-09-10T22:10:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.644+0000 7fce28200640 1 -- 172.21.15.66:0/3116890198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2013fac0 con 0x7fce20072d50 2024-09-10T22:10:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.648+0000 7fce1d600640 1 -- 172.21.15.66:0/3116890198 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fce14005430 con 0x7fce20072d50 2024-09-10T22:10:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.648+0000 7fce1d600640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce00073bd0 0x7fce00076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.648+0000 7fce1d600640 1 -- 172.21.15.66:0/3116890198 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fce1412c800 con 0x7fce20072d50 2024-09-10T22:10:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.648+0000 7fce24e00640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce00073bd0 0x7fce00076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:50.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.648+0000 7fce24e00640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce00073bd0 0x7fce00076090 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fce20166620 tx=0x7fce0c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:50.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.652+0000 7fce1d600640 1 -- 172.21.15.66:0/3116890198 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fce14016410 con 0x7fce20072d50 2024-09-10T22:10:50.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:50.860+0000 7fce28200640 1 -- 172.21.15.66:0/3116890198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fce200d07c0 con 0x7fce20072d50 2024-09-10T22:10:51.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.844+0000 7fce1d600640 1 -- 172.21.15.66:0/3116890198 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v67) ==== 136+0+0 (secure 0 0 0) 0x7fce140f9990 con 0x7fce20072d50 2024-09-10T22:10:51.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.848+0000 7fce28200640 1 -- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce00073bd0 msgr2=0x7fce00076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:51.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.848+0000 7fce28200640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce00073bd0 0x7fce00076090 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fce20166620 tx=0x7fce0c009040 comp rx=0 tx=0).stop 2024-09-10T22:10:51.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.848+0000 7fce28200640 1 -- 172.21.15.66:0/3116890198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce20072d50 msgr2=0x7fce20158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:51.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.848+0000 7fce28200640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce20072d50 0x7fce20158780 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fce14009790 tx=0x7fce14004040 comp rx=0 tx=0).stop 2024-09-10T22:10:51.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.848+0000 7fce28200640 1 -- 172.21.15.66:0/3116890198 shutdown_connections 2024-09-10T22:10:51.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.848+0000 7fce28200640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce00073bd0 0x7fce00076090 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:51.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.848+0000 7fce28200640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce2015ab00 0x7fce201f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:51.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.848+0000 7fce28200640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce200cce70 0x7fce20142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:51.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.848+0000 7fce28200640 1 --2- 172.21.15.66:0/3116890198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce20072d50 0x7fce20158780 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:51.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.848+0000 7fce28200640 1 -- 172.21.15.66:0/3116890198 >> 172.21.15.66:0/3116890198 conn(0x7fce200cfaa0 msgr2=0x7fce200732d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:51.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.848+0000 7fce28200640 1 -- 172.21.15.66:0/3116890198 shutdown_connections 2024-09-10T22:10:51.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:51.848+0000 7fce28200640 1 -- 172.21.15.66:0/3116890198 wait complete. 2024-09-10T22:10:51.876 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:10:52.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 -- 172.21.15.66:0/422263059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d78004dc0 msgr2=0x7f5d78005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 --2- 172.21.15.66:0/422263059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d78004dc0 0x7f5d78005240 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00c600 tx=0x7f5d8c01cee0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d99000640 1 -- 172.21.15.66:0/422263059 reap_dead start 2024-09-10T22:10:52.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 -- 172.21.15.66:0/422263059 shutdown_connections 2024-09-10T22:10:52.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 -- 172.21.15.66:0/422263059 >> 172.21.15.66:0/422263059 conn(0x7f5d940d1c60 msgr2=0x7f5d940d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:52.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 -- 172.21.15.66:0/422263059 shutdown_connections 2024-09-10T22:10:52.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 -- 172.21.15.66:0/422263059 wait complete. 2024-09-10T22:10:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 Processor -- start 2024-09-10T22:10:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 -- start start 2024-09-10T22:10:52.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d78004dc0 0x7f5d940c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:52.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d9415a300 0x7f5d940b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:52.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d9415a780 0x7f5d940b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5d940b3500 con 0x7f5d9415a780 2024-09-10T22:10:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d940b3380 con 0x7f5d9415a300 2024-09-10T22:10:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.028+0000 7f5d9b000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5d940af320 con 0x7f5d78004dc0 2024-09-10T22:10:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d99000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d9415a780 0x7f5d940b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d93e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d78004dc0 0x7f5d940c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d99000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d9415a780 0x7f5d940b7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54446/0 (socket says 172.21.15.66:54446) 2024-09-10T22:10:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d99000640 1 -- 172.21.15.66:0/2605162653 learned_addr learned my addr 172.21.15.66:0/2605162653 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d8be00640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d9415a300 0x7f5d940b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d99000640 1 -- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d78004dc0 msgr2=0x7f5d940c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d99000640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d78004dc0 0x7f5d940c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d99000640 1 -- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d9415a300 msgr2=0x7f5d940b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d99000640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d9415a300 0x7f5d940b6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d99000640 1 -- 172.21.15.66:0/2605162653 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d940c9a40 con 0x7f5d9415a780 2024-09-10T22:10:52.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d99000640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d9415a780 0x7f5d940b7370 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c098a70 tx=0x7f5d8c0065e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d90c00640 1 -- 172.21.15.66:0/2605162653 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c004e60 con 0x7f5d9415a780 2024-09-10T22:10:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d90c00640 1 -- 172.21.15.66:0/2605162653 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d8c001880 con 0x7f5d9415a780 2024-09-10T22:10:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d9b000640 1 -- 172.21.15.66:0/2605162653 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d940b4c30 con 0x7f5d9415a780 2024-09-10T22:10:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d90c00640 1 -- 172.21.15.66:0/2605162653 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c006d30 con 0x7f5d9415a780 2024-09-10T22:10:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d9b000640 1 -- 172.21.15.66:0/2605162653 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d940b51f0 con 0x7f5d9415a780 2024-09-10T22:10:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d9b000640 1 -- 172.21.15.66:0/2605162653 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d940aec10 con 0x7f5d9415a780 2024-09-10T22:10:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.032+0000 7f5d90c00640 1 -- 172.21.15.66:0/2605162653 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f5d8c003980 con 0x7f5d9415a780 2024-09-10T22:10:52.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.036+0000 7f5d90c00640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5d70073bd0 0x7f5d70076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:52.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.036+0000 7f5d90c00640 1 -- 172.21.15.66:0/2605162653 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f5d8c12c760 con 0x7f5d9415a780 2024-09-10T22:10:52.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.036+0000 7f5d93e00640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5d70073bd0 0x7f5d70076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:52.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.036+0000 7f5d93e00640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5d70073bd0 0x7f5d70076090 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5d940c4d20 tx=0x7f5d84006b20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.040+0000 7f5d90c00640 1 -- 172.21.15.66:0/2605162653 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5d8c016f50 con 0x7f5d9415a780 2024-09-10T22:10:52.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.236+0000 7f5d9b000640 1 -- 172.21.15.66:0/2605162653 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f5d940d2a40 con 0x7f5d70073bd0 2024-09-10T22:10:52.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.236+0000 7f5d90c00640 1 -- 172.21.15.66:0/2605162653 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f5d940d2a40 con 0x7f5d70073bd0 2024-09-10T22:10:52.242 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:10:52.242 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:10:52.242 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:10:52.242 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:10:52.242 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:10:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.240+0000 7f5d9b000640 1 -- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5d70073bd0 msgr2=0x7f5d70076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.240+0000 7f5d9b000640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5d70073bd0 0x7f5d70076090 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5d940c4d20 tx=0x7f5d84006b20 comp rx=0 tx=0).stop 2024-09-10T22:10:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.240+0000 7f5d9b000640 1 -- 172.21.15.66:0/2605162653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d9415a780 msgr2=0x7f5d940b7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.240+0000 7f5d9b000640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d9415a780 0x7f5d940b7370 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c098a70 tx=0x7f5d8c0065e0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.240+0000 7f5d9b000640 1 -- 172.21.15.66:0/2605162653 shutdown_connections 2024-09-10T22:10:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.240+0000 7f5d9b000640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5d70073bd0 0x7f5d70076090 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.240+0000 7f5d9b000640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d9415a780 0x7f5d940b7370 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.240+0000 7f5d9b000640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d9415a300 0x7f5d940b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.240+0000 7f5d9b000640 1 --2- 172.21.15.66:0/2605162653 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d78004dc0 0x7f5d940c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.240+0000 7f5d9b000640 1 -- 172.21.15.66:0/2605162653 >> 172.21.15.66:0/2605162653 conn(0x7f5d940d1c60 msgr2=0x7f5d94070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.240+0000 7f5d9b000640 1 -- 172.21.15.66:0/2605162653 shutdown_connections 2024-09-10T22:10:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.240+0000 7f5d9b000640 1 -- 172.21.15.66:0/2605162653 wait complete. 2024-09-10T22:10:52.265 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:10:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.412+0000 7f2aec600640 1 -- 172.21.15.66:0/1806785031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2acc004810 msgr2=0x7f2acc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.412+0000 7f2aec600640 1 --2- 172.21.15.66:0/1806785031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2acc004810 0x7f2acc004c90 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000adf0 tx=0x7f2ad001e230 comp rx=0 tx=0).stop 2024-09-10T22:10:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.412+0000 7f2aea600640 1 -- 172.21.15.66:0/1806785031 reap_dead start 2024-09-10T22:10:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.412+0000 7f2aec600640 1 -- 172.21.15.66:0/1806785031 shutdown_connections 2024-09-10T22:10:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.412+0000 7f2aec600640 1 -- 172.21.15.66:0/1806785031 >> 172.21.15.66:0/1806785031 conn(0x7f2ae40cfa30 msgr2=0x7f2ae40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.412+0000 7f2aec600640 1 -- 172.21.15.66:0/1806785031 shutdown_connections 2024-09-10T22:10:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.412+0000 7f2aec600640 1 -- 172.21.15.66:0/1806785031 wait complete. 2024-09-10T22:10:52.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.412+0000 7f2aec600640 1 Processor -- start 2024-09-10T22:10:52.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.412+0000 7f2aec600640 1 -- start start 2024-09-10T22:10:52.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aec600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2acc004810 0x7f2ae41594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:52.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aec600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ae40adbb0 0x7f2ae40ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:52.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aec600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ae41f8760 0x7f2ae41f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:52.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aec600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae40caf70 con 0x7f2ae41f8760 2024-09-10T22:10:52.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aec600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae40cadf0 con 0x7f2acc004810 2024-09-10T22:10:52.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aec600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae40cb0f0 con 0x7f2ae40adbb0 2024-09-10T22:10:52.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aea600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ae41f8760 0x7f2ae41f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:52.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aea600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ae41f8760 0x7f2ae41f8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59464/0 (socket says 172.21.15.66:59464) 2024-09-10T22:10:52.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aea600640 1 -- 172.21.15.66:0/2521703198 learned_addr learned my addr 172.21.15.66:0/2521703198 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:52.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2ae9c00640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2acc004810 0x7f2ae41594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:52.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2ae9200640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ae40adbb0 0x7f2ae40ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:52.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aea600640 1 -- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ae40adbb0 msgr2=0x7f2ae40ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aea600640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ae40adbb0 0x7f2ae40ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aea600640 1 -- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2acc004810 msgr2=0x7f2ae41594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aea600640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2acc004810 0x7f2ae41594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aea600640 1 -- 172.21.15.66:0/2521703198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae40c9c90 con 0x7f2ae41f8760 2024-09-10T22:10:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aea600640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ae41f8760 0x7f2ae41f8bf0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f2ae406deb0 tx=0x7f2adc00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2ae1600640 1 -- 172.21.15.66:0/2521703198 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc003980 con 0x7f2ae41f8760 2024-09-10T22:10:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2ae1600640 1 -- 172.21.15.66:0/2521703198 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2adc003b20 con 0x7f2ae41f8760 2024-09-10T22:10:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2ae1600640 1 -- 172.21.15.66:0/2521703198 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc004030 con 0x7f2ae41f8760 2024-09-10T22:10:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aec600640 1 -- 172.21.15.66:0/2521703198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae40ca4b0 con 0x7f2ae41f8760 2024-09-10T22:10:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aec600640 1 -- 172.21.15.66:0/2521703198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41f5f90 con 0x7f2ae41f8760 2024-09-10T22:10:52.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.416+0000 7f2aec600640 1 -- 172.21.15.66:0/2521703198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae40ca1a0 con 0x7f2ae41f8760 2024-09-10T22:10:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.420+0000 7f2ae1600640 1 -- 172.21.15.66:0/2521703198 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f2adc018070 con 0x7f2ae41f8760 2024-09-10T22:10:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.420+0000 7f2ae1600640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2ac4073bd0 0x7f2ac4076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.420+0000 7f2ae1600640 1 -- 172.21.15.66:0/2521703198 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f2adc093200 con 0x7f2ae41f8760 2024-09-10T22:10:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.420+0000 7f2ae9c00640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2ac4073bd0 0x7f2ac4076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.420+0000 7f2ae9c00640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2ac4073bd0 0x7f2ac4076090 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40b5a00 tx=0x7f2ad8006730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.420+0000 7f2ae1600640 1 -- 172.21.15.66:0/2521703198 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2adc03e020 con 0x7f2ae41f8760 2024-09-10T22:10:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 -- 172.21.15.66:0/2521703198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2ae40c4bc0 con 0x7f2ae41f8760 2024-09-10T22:10:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2ae1600640 1 -- 172.21.15.66:0/2521703198 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f2adc05f8d0 con 0x7f2ae41f8760 2024-09-10T22:10:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 -- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2ac4073bd0 msgr2=0x7f2ac4076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2ac4073bd0 0x7f2ac4076090 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40b5a00 tx=0x7f2ad8006730 comp rx=0 tx=0).stop 2024-09-10T22:10:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 -- 172.21.15.66:0/2521703198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ae41f8760 msgr2=0x7f2ae41f8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ae41f8760 0x7f2ae41f8bf0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f2ae406deb0 tx=0x7f2adc00f3c0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 -- 172.21.15.66:0/2521703198 shutdown_connections 2024-09-10T22:10:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2ac4073bd0 0x7f2ac4076090 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ae41f8760 0x7f2ae41f8bf0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ae40adbb0 0x7f2ae40ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 --2- 172.21.15.66:0/2521703198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2acc004810 0x7f2ae41594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 -- 172.21.15.66:0/2521703198 >> 172.21.15.66:0/2521703198 conn(0x7f2ae40cfa30 msgr2=0x7f2ae4150680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:52.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 -- 172.21.15.66:0/2521703198 shutdown_connections 2024-09-10T22:10:52.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.596+0000 7f2aec600640 1 -- 172.21.15.66:0/2521703198 wait complete. 2024-09-10T22:10:52.623 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-10T22:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 -- 172.21.15.66:0/3512318109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1318159900 msgr2=0x7f13180ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 --2- 172.21.15.66:0/3512318109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1318159900 0x7f13180ca080 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f1308009840 tx=0x7f130801c790 comp rx=0 tx=0).stop 2024-09-10T22:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 -- 172.21.15.66:0/3512318109 shutdown_connections 2024-09-10T22:10:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 --2- 172.21.15.66:0/3512318109 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13180aee20 0x7f13180cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 --2- 172.21.15.66:0/3512318109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1318159900 0x7f13180ca080 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 -- 172.21.15.66:0/3512318109 >> 172.21.15.66:0/3512318109 conn(0x7f13180cfa30 msgr2=0x7f13180cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 -- 172.21.15.66:0/3512318109 shutdown_connections 2024-09-10T22:10:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 -- 172.21.15.66:0/3512318109 wait complete. 2024-09-10T22:10:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 Processor -- start 2024-09-10T22:10:52.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 -- start start 2024-09-10T22:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13180aee20 0x7f1318159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1318159900 0x7f1318152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f131815cc80 0x7f13180afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1318168f70 con 0x7f13180aee20 2024-09-10T22:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1318168df0 con 0x7f131815cc80 2024-09-10T22:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f131d600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f13181690f0 con 0x7f1318159900 2024-09-10T22:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.768+0000 7f1316a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13180aee20 0x7f1318159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f1316a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13180aee20 0x7f1318159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59482/0 (socket says 172.21.15.66:59482) 2024-09-10T22:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f1316a00640 1 -- 172.21.15.66:0/243004054 learned_addr learned my addr 172.21.15.66:0/243004054 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f1317400640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f131815cc80 0x7f13180afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f130fe00640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1318159900 0x7f1318152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f1316a00640 1 -- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1318159900 msgr2=0x7f1318152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f1316a00640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1318159900 0x7f1318152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f1316a00640 1 -- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f131815cc80 msgr2=0x7f13180afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f1316a00640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f131815cc80 0x7f13180afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f1316a00640 1 -- 172.21.15.66:0/243004054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13180ced90 con 0x7f13180aee20 2024-09-10T22:10:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f1316a00640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13180aee20 0x7f1318159e70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f1308009790 tx=0x7f13080a6900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f130ea00640 1 -- 172.21.15.66:0/243004054 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13080ac070 con 0x7f13180aee20 2024-09-10T22:10:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f130ea00640 1 -- 172.21.15.66:0/243004054 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13080a6de0 con 0x7f13180aee20 2024-09-10T22:10:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f130ea00640 1 -- 172.21.15.66:0/243004054 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308003840 con 0x7f13180aee20 2024-09-10T22:10:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f131d600640 1 -- 172.21.15.66:0/243004054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131815a3b0 con 0x7f13180aee20 2024-09-10T22:10:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f131d600640 1 -- 172.21.15.66:0/243004054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131815a630 con 0x7f13180aee20 2024-09-10T22:10:52.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f131d600640 1 -- 172.21.15.66:0/243004054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13180cd930 con 0x7f13180aee20 2024-09-10T22:10:52.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.772+0000 7f130ea00640 1 -- 172.21.15.66:0/243004054 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f13080039e0 con 0x7f13180aee20 2024-09-10T22:10:52.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.776+0000 7f130ea00640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12f4073bd0 0x7f12f4076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:52.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.776+0000 7f130ea00640 1 -- 172.21.15.66:0/243004054 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f130812c6f0 con 0x7f13180aee20 2024-09-10T22:10:52.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.776+0000 7f130fe00640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12f4073bd0 0x7f12f4076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.776+0000 7f130fe00640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12f4073bd0 0x7f12f4076090 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f13180b5a40 tx=0x7f1300009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:52.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.780+0000 7f130ea00640 1 -- 172.21.15.66:0/243004054 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f131815a3b0 con 0x7f13180aee20 2024-09-10T22:10:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.952+0000 7f131d600640 1 -- 172.21.15.66:0/243004054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f1318070470 con 0x7f13180aee20 2024-09-10T22:10:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.952+0000 7f130ea00640 1 -- 172.21.15.66:0/243004054 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v45) ==== 116+0+2 (secure 0 0 0) 0x7f13080b6070 con 0x7f13180aee20 2024-09-10T22:10:52.958 INFO:teuthology.orchestra.run.smithi066.stdout:4 2024-09-10T22:10:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.956+0000 7f131d600640 1 -- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12f4073bd0 msgr2=0x7f12f4076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.956+0000 7f131d600640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12f4073bd0 0x7f12f4076090 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f13180b5a40 tx=0x7f1300009040 comp rx=0 tx=0).stop 2024-09-10T22:10:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.956+0000 7f131d600640 1 -- 172.21.15.66:0/243004054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13180aee20 msgr2=0x7f1318159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.956+0000 7f131d600640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13180aee20 0x7f1318159e70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f1308009790 tx=0x7f13080a6900 comp rx=0 tx=0).stop 2024-09-10T22:10:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.956+0000 7f131d600640 1 -- 172.21.15.66:0/243004054 shutdown_connections 2024-09-10T22:10:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.956+0000 7f131d600640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12f4073bd0 0x7f12f4076090 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.956+0000 7f131d600640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f131815cc80 0x7f13180afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.956+0000 7f131d600640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1318159900 0x7f1318152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.956+0000 7f131d600640 1 --2- 172.21.15.66:0/243004054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13180aee20 0x7f1318159e70 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.956+0000 7f131d600640 1 -- 172.21.15.66:0/243004054 >> 172.21.15.66:0/243004054 conn(0x7f13180cfa30 msgr2=0x7f1318155430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.956+0000 7f131d600640 1 -- 172.21.15.66:0/243004054 shutdown_connections 2024-09-10T22:10:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:52.956+0000 7f131d600640 1 -- 172.21.15.66:0/243004054 wait complete. 2024-09-10T22:10:52.980 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:10:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.124+0000 7f0953c00640 1 -- 172.21.15.66:0/803251155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f094c15f340 msgr2=0x7f094c0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.124+0000 7f0953c00640 1 --2- 172.21.15.66:0/803251155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f094c15f340 0x7f094c0cfb50 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f093c009840 tx=0x7f093c01c790 comp rx=0 tx=0).stop 2024-09-10T22:10:53.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 -- 172.21.15.66:0/803251155 shutdown_connections 2024-09-10T22:10:53.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 --2- 172.21.15.66:0/803251155 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f094c0b2bf0 0x7f094c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 --2- 172.21.15.66:0/803251155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f094c15f340 0x7f094c0cfb50 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 -- 172.21.15.66:0/803251155 >> 172.21.15.66:0/803251155 conn(0x7f094c0d1c50 msgr2=0x7f094c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:53.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 -- 172.21.15.66:0/803251155 shutdown_connections 2024-09-10T22:10:53.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 -- 172.21.15.66:0/803251155 wait complete. 2024-09-10T22:10:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 Processor -- start 2024-09-10T22:10:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 -- start start 2024-09-10T22:10:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f094c0b2bf0 0x7f094c1fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f094c0d2770 0x7f094c165f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f094c15f340 0x7f094c0b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f094c169190 con 0x7f094c0d2770 2024-09-10T22:10:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f094c169010 con 0x7f094c15f340 2024-09-10T22:10:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f094c169310 con 0x7f094c0b2bf0 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0943e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f094c0d2770 0x7f094c165f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0943e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f094c0d2770 0x7f094c165f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59494/0 (socket says 172.21.15.66:59494) 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0943e00640 1 -- 172.21.15.66:0/2434490450 learned_addr learned my addr 172.21.15.66:0/2434490450 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0951200640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f094c0b2bf0 0x7f094c1fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0951c00640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f094c15f340 0x7f094c0b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0943e00640 1 -- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f094c0b2bf0 msgr2=0x7f094c1fbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0943e00640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f094c0b2bf0 0x7f094c1fbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0943e00640 1 -- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f094c15f340 msgr2=0x7f094c0b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0943e00640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f094c15f340 0x7f094c0b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0943e00640 1 -- 172.21.15.66:0/2434490450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094c1fc330 con 0x7f094c0d2770 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0943e00640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f094c0d2770 0x7f094c165f70 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f093400d950 tx=0x7f093400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0949600640 1 -- 172.21.15.66:0/2434490450 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934017070 con 0x7f094c0d2770 2024-09-10T22:10:53.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0949600640 1 -- 172.21.15.66:0/2434490450 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0934009d20 con 0x7f094c0d2770 2024-09-10T22:10:53.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 -- 172.21.15.66:0/2434490450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094c0ce400 con 0x7f094c0d2770 2024-09-10T22:10:53.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0949600640 1 -- 172.21.15.66:0/2434490450 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09340106c0 con 0x7f094c0d2770 2024-09-10T22:10:53.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.128+0000 7f0953c00640 1 -- 172.21.15.66:0/2434490450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094c0ceca0 con 0x7f094c0d2770 2024-09-10T22:10:53.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.132+0000 7f0953c00640 1 -- 172.21.15.66:0/2434490450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f094c15fac0 con 0x7f094c0d2770 2024-09-10T22:10:53.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.132+0000 7f0949600640 1 -- 172.21.15.66:0/2434490450 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f0934010860 con 0x7f094c0d2770 2024-09-10T22:10:53.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.132+0000 7f0949600640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0928073bd0 0x7f0928076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:53.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.132+0000 7f0949600640 1 -- 172.21.15.66:0/2434490450 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f0934092830 con 0x7f094c0d2770 2024-09-10T22:10:53.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.132+0000 7f0951200640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0928073bd0 0x7f0928076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:53.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.136+0000 7f0951200640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0928073bd0 0x7f0928076090 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f093c0af000 tx=0x7f093c01afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:53.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.136+0000 7f0949600640 1 -- 172.21.15.66:0/2434490450 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f094c1fc330 con 0x7f094c0d2770 2024-09-10T22:10:53.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.312+0000 7f0953c00640 1 -- 172.21.15.66:0/2434490450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f094c165860 con 0x7f094c0d2770 2024-09-10T22:10:53.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.316+0000 7f0949600640 1 -- 172.21.15.66:0/2434490450 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7f093405f4d0 con 0x7f094c0d2770 2024-09-10T22:10:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.320+0000 7f0953c00640 1 -- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0928073bd0 msgr2=0x7f0928076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.320+0000 7f0953c00640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0928073bd0 0x7f0928076090 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f093c0af000 tx=0x7f093c01afd0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.320+0000 7f0953c00640 1 -- 172.21.15.66:0/2434490450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f094c0d2770 msgr2=0x7f094c165f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.320+0000 7f0953c00640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f094c0d2770 0x7f094c165f70 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f093400d950 tx=0x7f093400de10 comp rx=0 tx=0).stop 2024-09-10T22:10:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.320+0000 7f0953c00640 1 -- 172.21.15.66:0/2434490450 shutdown_connections 2024-09-10T22:10:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.320+0000 7f0953c00640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0928073bd0 0x7f0928076090 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.320+0000 7f0953c00640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f094c15f340 0x7f094c0b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.320+0000 7f0953c00640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f094c0d2770 0x7f094c165f70 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.320+0000 7f0953c00640 1 --2- 172.21.15.66:0/2434490450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f094c0b2bf0 0x7f094c1fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.320+0000 7f0953c00640 1 -- 172.21.15.66:0/2434490450 >> 172.21.15.66:0/2434490450 conn(0x7f094c0d1c50 msgr2=0x7f094c15e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:53.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.320+0000 7f0953c00640 1 -- 172.21.15.66:0/2434490450 shutdown_connections 2024-09-10T22:10:53.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.320+0000 7f0953c00640 1 -- 172.21.15.66:0/2434490450 wait complete. 2024-09-10T22:10:53.339 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:53.338+0000 7f5fea600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:53.339 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:53.338+0000 7f5fea600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:53.340 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:53.338+0000 7f5fea600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:53.340 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:53.338+0000 7f5fea600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:53.340 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:53.338+0000 7f5fea600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:53.344 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-10T22:10:53.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.488+0000 7fbc19a00640 1 -- 172.21.15.66:0/2517464441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc14152a90 msgr2=0x7fbc14070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.488+0000 7fbc19a00640 1 --2- 172.21.15.66:0/2517464441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc14152a90 0x7fbc14070f90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009870 tx=0x7fbc0401c6d0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 -- 172.21.15.66:0/2517464441 shutdown_connections 2024-09-10T22:10:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 --2- 172.21.15.66:0/2517464441 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc1415fd00 0x7fbc14073850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 --2- 172.21.15.66:0/2517464441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc14152a90 0x7fbc14070f90 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 -- 172.21.15.66:0/2517464441 >> 172.21.15.66:0/2517464441 conn(0x7fbc140d0bb0 msgr2=0x7fbc140d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 -- 172.21.15.66:0/2517464441 shutdown_connections 2024-09-10T22:10:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 -- 172.21.15.66:0/2517464441 wait complete. 2024-09-10T22:10:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 Processor -- start 2024-09-10T22:10:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 -- start start 2024-09-10T22:10:53.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc14152a90 0x7fbc141652a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:53.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc1415fd00 0x7fbc141fbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:53.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc14160270 0x7fbc140b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:53.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc14169310 con 0x7fbc14160270 2024-09-10T22:10:53.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc14169190 con 0x7fbc1415fd00 2024-09-10T22:10:53.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc19a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbc14169490 con 0x7fbc14152a90 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc13400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc14160270 0x7fbc140b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc12a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc14152a90 0x7fbc141652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc13400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc14160270 0x7fbc140b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59518/0 (socket says 172.21.15.66:59518) 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc13400640 1 -- 172.21.15.66:0/1686999194 learned_addr learned my addr 172.21.15.66:0/1686999194 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc12000640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc1415fd00 0x7fbc141fbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc13400640 1 -- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc14152a90 msgr2=0x7fbc141652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc13400640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc14152a90 0x7fbc141652a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc13400640 1 -- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc1415fd00 msgr2=0x7fbc141fbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc13400640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc1415fd00 0x7fbc141fbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc13400640 1 -- 172.21.15.66:0/1686999194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc140b5b10 con 0x7fbc14160270 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.492+0000 7fbc13400640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc14160270 0x7fbc140b55d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00e960 tx=0x7fbc0c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.496+0000 7fbc0aa00640 1 -- 172.21.15.66:0/1686999194 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c004d30 con 0x7fbc14160270 2024-09-10T22:10:53.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.496+0000 7fbc0aa00640 1 -- 172.21.15.66:0/1686999194 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc0c00cb20 con 0x7fbc14160270 2024-09-10T22:10:53.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.496+0000 7fbc0aa00640 1 -- 172.21.15.66:0/1686999194 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c005020 con 0x7fbc14160270 2024-09-10T22:10:53.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.496+0000 7fbc19a00640 1 -- 172.21.15.66:0/1686999194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc14165880 con 0x7fbc14160270 2024-09-10T22:10:53.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.496+0000 7fbc19a00640 1 -- 172.21.15.66:0/1686999194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc141fb510 con 0x7fbc14160270 2024-09-10T22:10:53.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.496+0000 7fbc19a00640 1 -- 172.21.15.66:0/1686999194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc14070f90 con 0x7fbc14160270 2024-09-10T22:10:53.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.496+0000 7fbc0aa00640 1 -- 172.21.15.66:0/1686999194 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7fbc0c00f040 con 0x7fbc14160270 2024-09-10T22:10:53.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.496+0000 7fbc0aa00640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbbf4073bd0 0x7fbbf4076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:53.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.496+0000 7fbc0aa00640 1 -- 172.21.15.66:0/1686999194 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fbc0c0928c0 con 0x7fbc14160270 2024-09-10T22:10:53.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.500+0000 7fbc12a00640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbbf4073bd0 0x7fbbf4076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:53.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.500+0000 7fbc12a00640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbbf4073bd0 0x7fbbf4076090 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbc040af000 tx=0x7fbc0401af10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:53.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.500+0000 7fbc0aa00640 1 -- 172.21.15.66:0/1686999194 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbc0c016030 con 0x7fbc14160270 2024-09-10T22:10:53.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.696+0000 7fbc19a00640 1 -- 172.21.15.66:0/1686999194 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fbc141fba10 con 0x7fbbf4073bd0 2024-09-10T22:10:53.713 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:53.710+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:53.713 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:53.710+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:53.713 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:53.710+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:53.713 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:53.710+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:53.713 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:10:53.710+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:10:53.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc0aa00640 1 -- 172.21.15.66:0/1686999194 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbc040042a0 con 0x7fbbf4073bd0 2024-09-10T22:10:53.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc19a00640 1 -- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbbf4073bd0 msgr2=0x7fbbf4076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc19a00640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbbf4073bd0 0x7fbbf4076090 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbc040af000 tx=0x7fbc0401af10 comp rx=0 tx=0).stop 2024-09-10T22:10:53.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc19a00640 1 -- 172.21.15.66:0/1686999194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc14160270 msgr2=0x7fbc140b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc19a00640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc14160270 0x7fbc140b55d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00e960 tx=0x7fbc0c00ee20 comp rx=0 tx=0).stop 2024-09-10T22:10:53.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc19a00640 1 -- 172.21.15.66:0/1686999194 shutdown_connections 2024-09-10T22:10:53.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc19a00640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbbf4073bd0 0x7fbbf4076090 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc19a00640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc14160270 0x7fbc140b55d0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc19a00640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc1415fd00 0x7fbc141fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc19a00640 1 --2- 172.21.15.66:0/1686999194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc14152a90 0x7fbc141652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc19a00640 1 -- 172.21.15.66:0/1686999194 >> 172.21.15.66:0/1686999194 conn(0x7fbc140d0bb0 msgr2=0x7fbc1415d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:53.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc19a00640 1 -- 172.21.15.66:0/1686999194 shutdown_connections 2024-09-10T22:10:53.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.744+0000 7fbc19a00640 1 -- 172.21.15.66:0/1686999194 wait complete. 2024-09-10T22:10:53.769 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:10:53.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.916+0000 7f9f3c400640 1 -- 172.21.15.66:0/1695642611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f34160280 msgr2=0x7f9f340aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.916+0000 7f9f3c400640 1 --2- 172.21.15.66:0/1695642611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f34160280 0x7f9f340aec10 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009bc0 tx=0x7f9f2c01d540 comp rx=0 tx=0).stop 2024-09-10T22:10:53.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.916+0000 7f9f3c400640 1 -- 172.21.15.66:0/1695642611 shutdown_connections 2024-09-10T22:10:53.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.916+0000 7f9f3c400640 1 --2- 172.21.15.66:0/1695642611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f34160280 0x7f9f340aec10 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.916+0000 7f9f3c400640 1 --2- 172.21.15.66:0/1695642611 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f3415f450 0x7f9f340b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.916+0000 7f9f3c400640 1 -- 172.21.15.66:0/1695642611 >> 172.21.15.66:0/1695642611 conn(0x7f9f340d1c60 msgr2=0x7f9f340d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:53.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.916+0000 7f9f3c400640 1 -- 172.21.15.66:0/1695642611 shutdown_connections 2024-09-10T22:10:53.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.916+0000 7f9f3c400640 1 -- 172.21.15.66:0/1695642611 wait complete. 2024-09-10T22:10:53.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.916+0000 7f9f3c400640 1 Processor -- start 2024-09-10T22:10:53.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.916+0000 7f9f3c400640 1 -- start start 2024-09-10T22:10:53.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f3c400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f340b1e10 0x7f9f340ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:53.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f3c400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f3415f450 0x7f9f340ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:53.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f3c400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f34160280 0x7f9f34156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:53.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f3c400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9f34168eb0 con 0x7f9f3415f450 2024-09-10T22:10:53.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f3c400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f340afe40 con 0x7f9f340b1e10 2024-09-10T22:10:53.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f3c400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9f34169030 con 0x7f9f34160280 2024-09-10T22:10:53.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f39000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f3415f450 0x7f9f340ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:53.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f39a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f340b1e10 0x7f9f340ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:53.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f39a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f340b1e10 0x7f9f340ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:38182/0 (socket says 172.21.15.66:38182) 2024-09-10T22:10:53.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f39a00640 1 -- 172.21.15.66:0/2411721070 learned_addr learned my addr 172.21.15.66:0/2411721070 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:10:53.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f39000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f3415f450 0x7f9f340ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59536/0 (socket says 172.21.15.66:59536) 2024-09-10T22:10:53.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f3a400640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f34160280 0x7f9f34156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:53.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f39000640 1 -- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f34160280 msgr2=0x7f9f34156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f39000640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f34160280 0x7f9f34156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f39000640 1 -- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f340b1e10 msgr2=0x7f9f340ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:53.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f39000640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f340b1e10 0x7f9f340ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:53.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f39000640 1 -- 172.21.15.66:0/2411721070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f34154450 con 0x7f9f3415f450 2024-09-10T22:10:53.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f39a00640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f340b1e10 0x7f9f340ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:10:53.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f39000640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f3415f450 0x7f9f340ce170 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000c3e0 tx=0x7f9f2000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:53.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f31600640 1 -- 172.21.15.66:0/2411721070 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20017070 con 0x7f9f3415f450 2024-09-10T22:10:53.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f31600640 1 -- 172.21.15.66:0/2411721070 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f20003600 con 0x7f9f3415f450 2024-09-10T22:10:53.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f31600640 1 -- 172.21.15.66:0/2411721070 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20003a90 con 0x7f9f3415f450 2024-09-10T22:10:53.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f3c400640 1 -- 172.21.15.66:0/2411721070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f340c90f0 con 0x7f9f3415f450 2024-09-10T22:10:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f3c400640 1 -- 172.21.15.66:0/2411721070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f340c96b0 con 0x7f9f3415f450 2024-09-10T22:10:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.920+0000 7f9f3c400640 1 -- 172.21.15.66:0/2411721070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f340b3170 con 0x7f9f3415f450 2024-09-10T22:10:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.924+0000 7f9f31600640 1 -- 172.21.15.66:0/2411721070 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 33) ==== 98383+0+0 (secure 0 0 0) 0x7f9f20003c30 con 0x7f9f3415f450 2024-09-10T22:10:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.924+0000 7f9f31600640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9f14073bd0 0x7f9f14076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:10:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.924+0000 7f9f31600640 1 -- 172.21.15.66:0/2411721070 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f9f200918d0 con 0x7f9f3415f450 2024-09-10T22:10:53.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.924+0000 7f9f39a00640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9f14073bd0 0x7f9f14076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:10:53.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.924+0000 7f9f39a00640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9f14073bd0 0x7f9f14076090 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800f4d0 tx=0x7f9f28005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:10:53.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:53.928+0000 7f9f31600640 1 -- 172.21.15.66:0/2411721070 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9f20007e20 con 0x7f9f3415f450 2024-09-10T22:10:54.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.128+0000 7f9f3c400640 1 -- 172.21.15.66:0/2411721070 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f9f340d2a40 con 0x7f9f14073bd0 2024-09-10T22:10:54.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.132+0000 7f9f31600640 1 -- 172.21.15.66:0/2411721070 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f9f340d2a40 con 0x7f9f14073bd0 2024-09-10T22:10:54.136 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/7bff86cd-997b-4eb3-ae59-8d3db9546164 2024-09-10T22:10:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.132+0000 7f9f3c400640 1 -- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9f14073bd0 msgr2=0x7f9f14076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.132+0000 7f9f3c400640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9f14073bd0 0x7f9f14076090 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800f4d0 tx=0x7f9f28005f90 comp rx=0 tx=0).stop 2024-09-10T22:10:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.132+0000 7f9f3c400640 1 -- 172.21.15.66:0/2411721070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f3415f450 msgr2=0x7f9f340ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:10:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.132+0000 7f9f3c400640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f3415f450 0x7f9f340ce170 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000c3e0 tx=0x7f9f2000c8a0 comp rx=0 tx=0).stop 2024-09-10T22:10:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.132+0000 7f9f3c400640 1 -- 172.21.15.66:0/2411721070 shutdown_connections 2024-09-10T22:10:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.132+0000 7f9f3c400640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9f14073bd0 0x7f9f14076090 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.132+0000 7f9f3c400640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f34160280 0x7f9f34156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:54.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.132+0000 7f9f3c400640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f3415f450 0x7f9f340ce170 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:54.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.132+0000 7f9f3c400640 1 --2- 172.21.15.66:0/2411721070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f340b1e10 0x7f9f340ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:10:54.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.132+0000 7f9f3c400640 1 -- 172.21.15.66:0/2411721070 >> 172.21.15.66:0/2411721070 conn(0x7f9f340d1c60 msgr2=0x7f9f34166160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:10:54.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.136+0000 7f9f3c400640 1 -- 172.21.15.66:0/2411721070 shutdown_connections 2024-09-10T22:10:54.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:10:54.136+0000 7f9f3c400640 1 -- 172.21.15.66:0/2411721070 wait complete. 2024-09-10T22:10:54.159 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/7bff86cd-997b-4eb3-ae59-8d3db9546164 2024-09-10T22:10:54.159 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:10:54.159 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:10:54.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:10:54.168 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/7bff86cd-997b-4eb3-ae59-8d3db9546164/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:10:57.515 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:10:57.519 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:11:01.272 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:11:01.272 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:11:01.272 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.03684 s, 153 MB/s 2024-09-10T22:11:01.273 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:11:01.274 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:11:01.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:01.282 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/7bff86cd-997b-4eb3-ae59-8d3db9546164/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:11:08.255 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:11:08.255 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:11:08.255 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.91792 s, 155 MB/s 2024-09-10T22:11:08.256 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:11:08.256 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:11:08.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:08.265 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/7bff86cd-997b-4eb3-ae59-8d3db9546164/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:11:15.130 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:11:15.131 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:11:15.131 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.80809 s, 158 MB/s 2024-09-10T22:11:15.132 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-10T22:11:15.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.280+0000 7f19ebe00640 1 -- 172.21.15.66:0/4252870263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e415f470 msgr2=0x7f19e40cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:15.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.280+0000 7f19ebe00640 1 --2- 172.21.15.66:0/4252870263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e415f470 0x7f19e40cd0a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009840 tx=0x7f19d801c7f0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.280+0000 7f19ebe00640 1 -- 172.21.15.66:0/4252870263 shutdown_connections 2024-09-10T22:11:15.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.280+0000 7f19ebe00640 1 --2- 172.21.15.66:0/4252870263 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e40d05c0 0x7f19e40aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.280+0000 7f19ebe00640 1 --2- 172.21.15.66:0/4252870263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e415f470 0x7f19e40cd0a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.280+0000 7f19ebe00640 1 -- 172.21.15.66:0/4252870263 >> 172.21.15.66:0/4252870263 conn(0x7f19e40d1c60 msgr2=0x7f19e40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:15.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.280+0000 7f19ebe00640 1 -- 172.21.15.66:0/4252870263 shutdown_connections 2024-09-10T22:11:15.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.280+0000 7f19ebe00640 1 -- 172.21.15.66:0/4252870263 wait complete. 2024-09-10T22:11:15.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19ebe00640 1 Processor -- start 2024-09-10T22:11:15.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19ebe00640 1 -- start start 2024-09-10T22:11:15.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19ebe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e40cfa40 0x7f19e415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:15.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19ebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40d05c0 0x7f19e41f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:15.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19ebe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e415f470 0x7f19e40ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:15.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19ebe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19e41689b0 con 0x7f19e40d05c0 2024-09-10T22:11:15.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19ebe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19e40afe40 con 0x7f19e415f470 2024-09-10T22:11:15.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19ebe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f19e4168b30 con 0x7f19e40cfa40 2024-09-10T22:11:15.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40d05c0 0x7f19e41f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:15.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40d05c0 0x7f19e41f7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42626/0 (socket says 172.21.15.66:42626) 2024-09-10T22:11:15.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e8a00640 1 -- 172.21.15.66:0/2230276542 learned_addr learned my addr 172.21.15.66:0/2230276542 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:15.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e9400640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e40cfa40 0x7f19e415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:15.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e9e00640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e415f470 0x7f19e40ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:15.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e8a00640 1 -- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e40cfa40 msgr2=0x7f19e415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:15.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e8a00640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e40cfa40 0x7f19e415afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e8a00640 1 -- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e415f470 msgr2=0x7f19e40ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:15.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e8a00640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e415f470 0x7f19e40ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e8a00640 1 -- 172.21.15.66:0/2230276542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e40c89f0 con 0x7f19e40d05c0 2024-09-10T22:11:15.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e8a00640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40d05c0 0x7f19e41f7fd0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f19d000c490 tx=0x7f19d000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:15.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e1600640 1 -- 172.21.15.66:0/2230276542 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0016020 con 0x7f19e40d05c0 2024-09-10T22:11:15.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19ebe00640 1 -- 172.21.15.66:0/2230276542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e40ca820 con 0x7f19e40d05c0 2024-09-10T22:11:15.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e1600640 1 -- 172.21.15.66:0/2230276542 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d0012940 con 0x7f19e40d05c0 2024-09-10T22:11:15.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19e1600640 1 -- 172.21.15.66:0/2230276542 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0012d90 con 0x7f19e40d05c0 2024-09-10T22:11:15.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19ebe00640 1 -- 172.21.15.66:0/2230276542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e40b5dc0 con 0x7f19e40d05c0 2024-09-10T22:11:15.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.284+0000 7f19ebe00640 1 -- 172.21.15.66:0/2230276542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e40aebd0 con 0x7f19e40d05c0 2024-09-10T22:11:15.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.288+0000 7f19e1600640 1 -- 172.21.15.66:0/2230276542 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 34) ==== 98485+0+0 (secure 0 0 0) 0x7f19d001b400 con 0x7f19e40d05c0 2024-09-10T22:11:15.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.288+0000 7f19e1600640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19c4073d20 0x7f19c40761e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:15.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.288+0000 7f19e1600640 1 -- 172.21.15.66:0/2230276542 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f19d0091f90 con 0x7f19e40d05c0 2024-09-10T22:11:15.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.288+0000 7f19e9400640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19c4073d20 0x7f19c40761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:15.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.288+0000 7f19e9400640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19c4073d20 0x7f19c40761e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f19d80af000 tx=0x7f19d801b030 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:15.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.292+0000 7f19e1600640 1 -- 172.21.15.66:0/2230276542 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f19d0020030 con 0x7f19e40d05c0 2024-09-10T22:11:15.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.496+0000 7f19ebe00640 1 -- 172.21.15.66:0/2230276542 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f19e40723e0 con 0x7f19c4073d20 2024-09-10T22:11:15.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.508+0000 7f19e1600640 1 -- 172.21.15.66:0/2230276542 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19e40723e0 con 0x7f19c4073d20 2024-09-10T22:11:15.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.512+0000 7f19ebe00640 1 -- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19c4073d20 msgr2=0x7f19c40761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:15.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.512+0000 7f19ebe00640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19c4073d20 0x7f19c40761e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f19d80af000 tx=0x7f19d801b030 comp rx=0 tx=0).stop 2024-09-10T22:11:15.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.512+0000 7f19ebe00640 1 -- 172.21.15.66:0/2230276542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40d05c0 msgr2=0x7f19e41f7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:15.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.512+0000 7f19ebe00640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40d05c0 0x7f19e41f7fd0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f19d000c490 tx=0x7f19d000c950 comp rx=0 tx=0).stop 2024-09-10T22:11:15.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.512+0000 7f19ebe00640 1 -- 172.21.15.66:0/2230276542 shutdown_connections 2024-09-10T22:11:15.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.512+0000 7f19ebe00640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f19c4073d20 0x7f19c40761e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.512+0000 7f19ebe00640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e415f470 0x7f19e40ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.512+0000 7f19ebe00640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40d05c0 0x7f19e41f7fd0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.512+0000 7f19ebe00640 1 --2- 172.21.15.66:0/2230276542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e40cfa40 0x7f19e415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.512+0000 7f19ebe00640 1 -- 172.21.15.66:0/2230276542 >> 172.21.15.66:0/2230276542 conn(0x7f19e40d1c60 msgr2=0x7f19e406f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:15.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.512+0000 7f19ebe00640 1 -- 172.21.15.66:0/2230276542 shutdown_connections 2024-09-10T22:11:15.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.512+0000 7f19ebe00640 1 -- 172.21.15.66:0/2230276542 wait complete. 2024-09-10T22:11:15.533 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:11:15.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 -- 172.21.15.66:0/3418126332 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17180d0550 msgr2=0x7f17180ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:15.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 --2- 172.21.15.66:0/3418126332 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17180d0550 0x7f17180ca1a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f170000a950 tx=0x7f170001dae0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 -- 172.21.15.66:0/3418126332 shutdown_connections 2024-09-10T22:11:15.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 --2- 172.21.15.66:0/3418126332 >> v2:172.21.15.94:3300/0 conn(0x7f17180c4fc0 0x7f17180ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 --2- 172.21.15.66:0/3418126332 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17180d0550 0x7f17180ca1a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 -- 172.21.15.66:0/3418126332 >> 172.21.15.66:0/3418126332 conn(0x7f17180cfa30 msgr2=0x7f17180cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:15.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 -- 172.21.15.66:0/3418126332 shutdown_connections 2024-09-10T22:11:15.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 -- 172.21.15.66:0/3418126332 wait complete. 2024-09-10T22:11:15.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 Processor -- start 2024-09-10T22:11:15.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 -- start start 2024-09-10T22:11:15.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17180c4fc0 0x7f171814eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:15.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171815a310 0x7f171814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:15.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17181fbbf0 0x7f17180ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:15.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1718167480 con 0x7f17181fbbf0 2024-09-10T22:11:15.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1718167300 con 0x7f17180c4fc0 2024-09-10T22:11:15.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f171e200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1718167600 con 0x7f171815a310 2024-09-10T22:11:15.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f1717e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17181fbbf0 0x7f17180ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:15.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f1717e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17181fbbf0 0x7f17180ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42636/0 (socket says 172.21.15.66:42636) 2024-09-10T22:11:15.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f1717400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17180c4fc0 0x7f171814eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:15.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f1717e00640 1 -- 172.21.15.66:0/1498425563 learned_addr learned my addr 172.21.15.66:0/1498425563 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:15.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.680+0000 7f170fe00640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171815a310 0x7f171814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:15.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f1717e00640 1 -- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171815a310 msgr2=0x7f171814f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:15.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f1717e00640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171815a310 0x7f171814f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f1717e00640 1 -- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17180c4fc0 msgr2=0x7f171814eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:15.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f1717e00640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17180c4fc0 0x7f171814eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f1717e00640 1 -- 172.21.15.66:0/1498425563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17181f7b90 con 0x7f17181fbbf0 2024-09-10T22:11:15.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f1717e00640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17181fbbf0 0x7f17180ad8c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f171000ec80 tx=0x7f171000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:15.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f170f400640 1 -- 172.21.15.66:0/1498425563 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710003b80 con 0x7f17181fbbf0 2024-09-10T22:11:15.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f170f400640 1 -- 172.21.15.66:0/1498425563 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1710003d20 con 0x7f17181fbbf0 2024-09-10T22:11:15.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f171e200640 1 -- 172.21.15.66:0/1498425563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17181f7e00 con 0x7f17181fbbf0 2024-09-10T22:11:15.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f170f400640 1 -- 172.21.15.66:0/1498425563 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710004280 con 0x7f17181fbbf0 2024-09-10T22:11:15.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f171e200640 1 -- 172.21.15.66:0/1498425563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17181f83c0 con 0x7f17181fbbf0 2024-09-10T22:11:15.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f171e200640 1 -- 172.21.15.66:0/1498425563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17180c48b0 con 0x7f17181fbbf0 2024-09-10T22:11:15.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.684+0000 7f170f400640 1 -- 172.21.15.66:0/1498425563 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 34) ==== 98485+0+0 (secure 0 0 0) 0x7f1710004420 con 0x7f17181fbbf0 2024-09-10T22:11:15.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.688+0000 7f170f400640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f16f4073d20 0x7f16f40761e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:15.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.688+0000 7f170f400640 1 -- 172.21.15.66:0/1498425563 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f1710092bb0 con 0x7f17181fbbf0 2024-09-10T22:11:15.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.688+0000 7f1717400640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f16f4073d20 0x7f16f40761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:15.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.688+0000 7f1717400640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f16f4073d20 0x7f16f40761e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1718153ad0 tx=0x7f17080013a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:15.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.688+0000 7f170f400640 1 -- 172.21.15.66:0/1498425563 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f171005f6a0 con 0x7f17181fbbf0 2024-09-10T22:11:15.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.892+0000 7f171e200640 1 -- 172.21.15.66:0/1498425563 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f1718070420 con 0x7f16f4073d20 2024-09-10T22:11:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.896+0000 7f170f400640 1 -- 172.21.15.66:0/1498425563 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f1718070420 con 0x7f16f4073d20 2024-09-10T22:11:15.901 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/7bff86cd-997b-4eb3-ae59-8d3db9546164 2024-09-10T22:11:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.900+0000 7f171e200640 1 -- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f16f4073d20 msgr2=0x7f16f40761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.900+0000 7f171e200640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f16f4073d20 0x7f16f40761e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1718153ad0 tx=0x7f17080013a0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.900+0000 7f171e200640 1 -- 172.21.15.66:0/1498425563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17181fbbf0 msgr2=0x7f17180ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.900+0000 7f171e200640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17181fbbf0 0x7f17180ad8c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f171000ec80 tx=0x7f171000c6a0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.900+0000 7f171e200640 1 -- 172.21.15.66:0/1498425563 shutdown_connections 2024-09-10T22:11:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.900+0000 7f171e200640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f16f4073d20 0x7f16f40761e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.900+0000 7f171e200640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17181fbbf0 0x7f17180ad8c0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.900+0000 7f171e200640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171815a310 0x7f171814f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.900+0000 7f171e200640 1 --2- 172.21.15.66:0/1498425563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17180c4fc0 0x7f171814eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.900+0000 7f171e200640 1 -- 172.21.15.66:0/1498425563 >> 172.21.15.66:0/1498425563 conn(0x7f17180cfa30 msgr2=0x7f17180725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.900+0000 7f171e200640 1 -- 172.21.15.66:0/1498425563 shutdown_connections 2024-09-10T22:11:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:15.900+0000 7f171e200640 1 -- 172.21.15.66:0/1498425563 wait complete. 2024-09-10T22:11:15.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:15.922 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/7bff86cd-997b-4eb3-ae59-8d3db9546164 2024-09-10T22:11:15.923 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:11:15.937 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/7bff86cd-997b-4eb3-ae59-8d3db9546164 2024-09-10T22:11:15.937 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3138256896" 2024-09-10T22:11:15.937 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:15.938 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-09-10T22:11:17.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:17.939 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/7bff86cd-997b-4eb3-ae59-8d3db9546164 2024-09-10T22:11:17.939 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:11:17.954 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/7bff86cd-997b-4eb3-ae59-8d3db9546164 2024-09-10T22:11:17.954 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3138256896" 2024-09-10T22:11:17.954 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:17.955 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-09-10T22:11:19.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:19.956 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/7bff86cd-997b-4eb3-ae59-8d3db9546164 2024-09-10T22:11:19.956 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:11:19.972 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/7bff86cd-997b-4eb3-ae59-8d3db9546164 2024-09-10T22:11:19.972 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3221225472" 2024-09-10T22:11:19.972 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:19.973 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-10T22:11:19.974 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-10T22:11:20.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.176+0000 7f9a60800640 1 -- 172.21.15.66:0/133726646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58071700 msgr2=0x7f9a580b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:20.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.176+0000 7f9a60800640 1 --2- 172.21.15.66:0/133726646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58071700 0x7f9a580b0410 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009900 tx=0x7f9a4801c7c0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.176+0000 7f9a60800640 1 -- 172.21.15.66:0/133726646 shutdown_connections 2024-09-10T22:11:20.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.176+0000 7f9a60800640 1 --2- 172.21.15.66:0/133726646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58071700 0x7f9a580b0410 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.176+0000 7f9a60800640 1 --2- 172.21.15.66:0/133726646 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a58070ac0 0x7f9a58070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.176+0000 7f9a60800640 1 -- 172.21.15.66:0/133726646 >> 172.21.15.66:0/133726646 conn(0x7f9a580d1d00 msgr2=0x7f9a580d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.176+0000 7f9a60800640 1 -- 172.21.15.66:0/133726646 shutdown_connections 2024-09-10T22:11:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.176+0000 7f9a60800640 1 -- 172.21.15.66:0/133726646 wait complete. 2024-09-10T22:11:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a60800640 1 Processor -- start 2024-09-10T22:11:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a60800640 1 -- start start 2024-09-10T22:11:20.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a60800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a58070ac0 0x7f9a580c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:20.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a60800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a58071700 0x7f9a580c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:20.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a60800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a580b0ef0 0x7f9a581669d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:20.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a60800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58160b50 con 0x7f9a580b0ef0 2024-09-10T22:11:20.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a60800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581609d0 con 0x7f9a58071700 2024-09-10T22:11:20.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a60800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9a58160cd0 con 0x7f9a58070ac0 2024-09-10T22:11:20.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a5e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a580b0ef0 0x7f9a581669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:20.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a5de00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a58070ac0 0x7f9a580c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:20.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a5de00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a58070ac0 0x7f9a580c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:56014/0 (socket says 172.21.15.66:56014) 2024-09-10T22:11:20.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a5de00640 1 -- 172.21.15.66:0/1987883761 learned_addr learned my addr 172.21.15.66:0/1987883761 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:20.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a5d400640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a58071700 0x7f9a580c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:20.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a5e800640 1 -- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a58070ac0 msgr2=0x7f9a580c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a5e800640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a58070ac0 0x7f9a580c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a5e800640 1 -- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a58071700 msgr2=0x7f9a580c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a5e800640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a58071700 0x7f9a580c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a5e800640 1 -- 172.21.15.66:0/1987883761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a58071ae0 con 0x7f9a580b0ef0 2024-09-10T22:11:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a5e800640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a580b0ef0 0x7f9a581669d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400c730 tx=0x7f9a5400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a52000640 1 -- 172.21.15.66:0/1987883761 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54003b80 con 0x7f9a580b0ef0 2024-09-10T22:11:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a52000640 1 -- 172.21.15.66:0/1987883761 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a54003d20 con 0x7f9a580b0ef0 2024-09-10T22:11:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a52000640 1 -- 172.21.15.66:0/1987883761 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54005330 con 0x7f9a580b0ef0 2024-09-10T22:11:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a60800640 1 -- 172.21.15.66:0/1987883761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a581f7fc0 con 0x7f9a580b0ef0 2024-09-10T22:11:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.180+0000 7f9a60800640 1 -- 172.21.15.66:0/1987883761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a581f8580 con 0x7f9a580b0ef0 2024-09-10T22:11:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.184+0000 7f9a60800640 1 -- 172.21.15.66:0/1987883761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a58070ea0 con 0x7f9a580b0ef0 2024-09-10T22:11:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.184+0000 7f9a52000640 1 -- 172.21.15.66:0/1987883761 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 34) ==== 98485+0+0 (secure 0 0 0) 0x7f9a540054d0 con 0x7f9a580b0ef0 2024-09-10T22:11:20.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.184+0000 7f9a52000640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a3c073cf0 0x7f9a3c0761b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:20.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.184+0000 7f9a52000640 1 -- 172.21.15.66:0/1987883761 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f9a54091ba0 con 0x7f9a580b0ef0 2024-09-10T22:11:20.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.184+0000 7f9a5de00640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a3c073cf0 0x7f9a3c0761b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:20.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.188+0000 7f9a5de00640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a3c073cf0 0x7f9a3c0761b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9a580b5f10 tx=0x7f9a4c00b410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:20.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.188+0000 7f9a52000640 1 -- 172.21.15.66:0/1987883761 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9a5405e690 con 0x7f9a580b0ef0 2024-09-10T22:11:20.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.396+0000 7f9a60800640 1 -- 172.21.15.66:0/1987883761 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f9a580b76d0 con 0x7f9a3c073cf0 2024-09-10T22:11:20.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.464+0000 7f9a52000640 1 -- 172.21.15.66:0/1987883761 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9a580b76d0 con 0x7f9a3c073cf0 2024-09-10T22:11:20.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.468+0000 7f9a60800640 1 -- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a3c073cf0 msgr2=0x7f9a3c0761b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:20.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.468+0000 7f9a60800640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a3c073cf0 0x7f9a3c0761b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9a580b5f10 tx=0x7f9a4c00b410 comp rx=0 tx=0).stop 2024-09-10T22:11:20.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.468+0000 7f9a60800640 1 -- 172.21.15.66:0/1987883761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a580b0ef0 msgr2=0x7f9a581669d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:20.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.468+0000 7f9a60800640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a580b0ef0 0x7f9a581669d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400c730 tx=0x7f9a5400cbf0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.468+0000 7f9a60800640 1 -- 172.21.15.66:0/1987883761 shutdown_connections 2024-09-10T22:11:20.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.468+0000 7f9a60800640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a3c073cf0 0x7f9a3c0761b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.468+0000 7f9a60800640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a580b0ef0 0x7f9a581669d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.468+0000 7f9a60800640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a58071700 0x7f9a580c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.468+0000 7f9a60800640 1 --2- 172.21.15.66:0/1987883761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a58070ac0 0x7f9a580c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.468+0000 7f9a60800640 1 -- 172.21.15.66:0/1987883761 >> 172.21.15.66:0/1987883761 conn(0x7f9a580d1d00 msgr2=0x7f9a580b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:20.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.468+0000 7f9a60800640 1 -- 172.21.15.66:0/1987883761 shutdown_connections 2024-09-10T22:11:20.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.468+0000 7f9a60800640 1 -- 172.21.15.66:0/1987883761 wait complete. 2024-09-10T22:11:20.472 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:20.469+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:20.473 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:20.469+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:20.473 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:20.469+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:20.473 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:20.469+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:20.473 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:20.469+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:20.490 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:20.485+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:20.490 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:20.485+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:20.490 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:20.485+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:20.490 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:20.485+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:20.490 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:20.485+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:20.491 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-10T22:11:20.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.636+0000 7f3577000640 1 -- 172.21.15.66:0/3220367317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f357015f300 msgr2=0x7f35700b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.636+0000 7f3577000640 1 --2- 172.21.15.66:0/3220367317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f357015f300 0x7f35700b1090 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f355c009930 tx=0x7f355c01c8e0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.636+0000 7f3577000640 1 -- 172.21.15.66:0/3220367317 shutdown_connections 2024-09-10T22:11:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.636+0000 7f3577000640 1 --2- 172.21.15.66:0/3220367317 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35700b15d0 0x7f35700cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.636+0000 7f3577000640 1 --2- 172.21.15.66:0/3220367317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f357015f300 0x7f35700b1090 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.636+0000 7f3577000640 1 -- 172.21.15.66:0/3220367317 >> 172.21.15.66:0/3220367317 conn(0x7f35700d1b10 msgr2=0x7f35700d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.636+0000 7f3577000640 1 -- 172.21.15.66:0/3220367317 shutdown_connections 2024-09-10T22:11:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.636+0000 7f3577000640 1 -- 172.21.15.66:0/3220367317 wait complete. 2024-09-10T22:11:20.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3577000640 1 Processor -- start 2024-09-10T22:11:20.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3577000640 1 -- start start 2024-09-10T22:11:20.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3577000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35700b15d0 0x7f357014aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:20.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3577000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f357015ef20 0x7f35700b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:20.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3577000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f357015f300 0x7f357015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:20.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3577000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3570169360 con 0x7f35700b15d0 2024-09-10T22:11:20.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3577000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f35701691e0 con 0x7f357015ef20 2024-09-10T22:11:20.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3577000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f35701694e0 con 0x7f357015f300 2024-09-10T22:11:20.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3575c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35700b15d0 0x7f357014aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:20.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3575c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35700b15d0 0x7f357014aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42680/0 (socket says 172.21.15.66:42680) 2024-09-10T22:11:20.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3575c00640 1 -- 172.21.15.66:0/1748180268 learned_addr learned my addr 172.21.15.66:0/1748180268 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:20.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3575200640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f357015ef20 0x7f35700b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:20.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3576600640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f357015f300 0x7f357015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:20.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3575c00640 1 -- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f357015f300 msgr2=0x7f357015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:20.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3575c00640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f357015f300 0x7f357015d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3575c00640 1 -- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f357015ef20 msgr2=0x7f35700b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:20.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3575c00640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f357015ef20 0x7f35700b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:20.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3575c00640 1 -- 172.21.15.66:0/1748180268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35700b49e0 con 0x7f35700b15d0 2024-09-10T22:11:20.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3575c00640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35700b15d0 0x7f357014aea0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f356400c500 tx=0x7f356400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:20.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f356d600640 1 -- 172.21.15.66:0/1748180268 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564017070 con 0x7f35700b15d0 2024-09-10T22:11:20.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3577000640 1 -- 172.21.15.66:0/1748180268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35700b4c80 con 0x7f35700b15d0 2024-09-10T22:11:20.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f356d600640 1 -- 172.21.15.66:0/1748180268 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35640053c0 con 0x7f35700b15d0 2024-09-10T22:11:20.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f356d600640 1 -- 172.21.15.66:0/1748180268 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564003860 con 0x7f35700b15d0 2024-09-10T22:11:20.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.640+0000 7f3577000640 1 -- 172.21.15.66:0/1748180268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35700b5520 con 0x7f35700b15d0 2024-09-10T22:11:20.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.644+0000 7f3577000640 1 -- 172.21.15.66:0/1748180268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35700cd7a0 con 0x7f35700b15d0 2024-09-10T22:11:20.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.644+0000 7f356d600640 1 -- 172.21.15.66:0/1748180268 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 34) ==== 98485+0+0 (secure 0 0 0) 0x7f3564003a00 con 0x7f35700b15d0 2024-09-10T22:11:20.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.644+0000 7f356d600640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3550073cf0 0x7f35500761b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:20.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.644+0000 7f356d600640 1 -- 172.21.15.66:0/1748180268 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f3564091700 con 0x7f35700b15d0 2024-09-10T22:11:20.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.644+0000 7f3575200640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3550073cf0 0x7f35500761b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:20.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.648+0000 7f3575200640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3550073cf0 0x7f35500761b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f355c0af000 tx=0x7f355c01b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.648+0000 7f356d600640 1 -- 172.21.15.66:0/1748180268 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f356405e170 con 0x7f35700b15d0 2024-09-10T22:11:20.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.860+0000 7f3577000640 1 -- 172.21.15.66:0/1748180268 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f3570072e00 con 0x7f3550073cf0 2024-09-10T22:11:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:20.912+0000 7f356d600640 1 -- 172.21.15.66:0/1748180268 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 35) ==== 98587+0+0 (secure 0 0 0) 0x7f356405a3e0 con 0x7f35700b15d0 2024-09-10T22:11:21.473 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:21.469+0000 7f5f93800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:21.474 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:21.469+0000 7f5f93800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:21.474 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:21.469+0000 7f5f93800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:21.474 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:21.469+0000 7f5f93800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:21.474 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:21.469+0000 7f5f93800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:21.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f356d600640 1 -- 172.21.15.66:0/1748180268 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f3570072e00 con 0x7f3550073cf0 2024-09-10T22:11:21.506 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:11:21.506 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "in-progress", 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_report": { 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout: "percentage cloned": "0.0%", 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout: "amount cloned": "184/3.0G", 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout: "files cloned": "3/4" 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:21.507 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:11:21.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f3577000640 1 -- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3550073cf0 msgr2=0x7f35500761b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:21.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f3577000640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3550073cf0 0x7f35500761b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f355c0af000 tx=0x7f355c01b120 comp rx=0 tx=0).stop 2024-09-10T22:11:21.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f3577000640 1 -- 172.21.15.66:0/1748180268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35700b15d0 msgr2=0x7f357014aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:21.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f3577000640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35700b15d0 0x7f357014aea0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f356400c500 tx=0x7f356400c9c0 comp rx=0 tx=0).stop 2024-09-10T22:11:21.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f3577000640 1 -- 172.21.15.66:0/1748180268 shutdown_connections 2024-09-10T22:11:21.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f3577000640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3550073cf0 0x7f35500761b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:21.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f3577000640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f357015f300 0x7f357015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:21.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f3577000640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f357015ef20 0x7f35700b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:21.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f3577000640 1 --2- 172.21.15.66:0/1748180268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35700b15d0 0x7f357014aea0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:21.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f3577000640 1 -- 172.21.15.66:0/1748180268 >> 172.21.15.66:0/1748180268 conn(0x7f35700d1b10 msgr2=0x7f357015e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:21.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f3577000640 1 -- 172.21.15.66:0/1748180268 shutdown_connections 2024-09-10T22:11:21.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.504+0000 7f3577000640 1 -- 172.21.15.66:0/1748180268 wait complete. 2024-09-10T22:11:21.540 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-10T22:11:21.541 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-10T22:11:21.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.812+0000 7f3341e00640 1 -- 172.21.15.66:0/3503887476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3320003a10 msgr2=0x7f3320003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:21.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.812+0000 7f3341e00640 1 --2- 172.21.15.66:0/3503887476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3320003a10 0x7f3320003e10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3334009f60 tx=0x7f333401c930 comp rx=0 tx=0).stop 2024-09-10T22:11:21.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.816+0000 7f3341e00640 1 -- 172.21.15.66:0/3503887476 shutdown_connections 2024-09-10T22:11:21.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.816+0000 7f3341e00640 1 --2- 172.21.15.66:0/3503887476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3320003a10 0x7f3320003e10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:21.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.816+0000 7f3341e00640 1 -- 172.21.15.66:0/3503887476 >> 172.21.15.66:0/3503887476 conn(0x7f333c0b2730 msgr2=0x7f333c0b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:21.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.816+0000 7f3341e00640 1 -- 172.21.15.66:0/3503887476 shutdown_connections 2024-09-10T22:11:21.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.816+0000 7f3341e00640 1 -- 172.21.15.66:0/3503887476 wait complete. 2024-09-10T22:11:21.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.820+0000 7f3341e00640 1 Processor -- start 2024-09-10T22:11:21.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.820+0000 7f3341e00640 1 -- start start 2024-09-10T22:11:21.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.820+0000 7f3341e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3320003a10 0x7f333c0c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:21.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3341e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c1f50d0 0x7f333c0ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:21.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3340a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3320003a10 0x7f333c0c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:21.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3340a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3320003a10 0x7f333c0c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:56052/0 (socket says 172.21.15.66:56052) 2024-09-10T22:11:21.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3340a00640 1 -- 172.21.15.66:0/2466248831 learned_addr learned my addr 172.21.15.66:0/2466248831 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:21.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3341e00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f333c159650 0x7f333c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:21.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f333c071010 con 0x7f333c159650 2024-09-10T22:11:21.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f333c070e90 con 0x7f333c1f50d0 2024-09-10T22:11:21.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f333c071190 con 0x7f3320003a10 2024-09-10T22:11:21.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f333be00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c1f50d0 0x7f333c0ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:21.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3341400640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f333c159650 0x7f333c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:21.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3340a00640 1 -- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c1f50d0 msgr2=0x7f333c0ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:21.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3340a00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c1f50d0 0x7f333c0ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:21.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3340a00640 1 -- 172.21.15.66:0/2466248831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f333c159650 msgr2=0x7f333c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:21.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3340a00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f333c159650 0x7f333c159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:21.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3340a00640 1 -- 172.21.15.66:0/2466248831 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c0c9e90 con 0x7f3320003a10 2024-09-10T22:11:21.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.824+0000 7f3340a00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3320003a10 0x7f333c0c9950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f333401d1d0 tx=0x7f33340a5730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:21.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.828+0000 7f3338c00640 1 -- 172.21.15.66:0/2466248831 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33340b2040 con 0x7f3320003a10 2024-09-10T22:11:21.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.828+0000 7f3338c00640 1 -- 172.21.15.66:0/2466248831 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33340bd070 con 0x7f3320003a10 2024-09-10T22:11:21.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.828+0000 7f3338c00640 1 -- 172.21.15.66:0/2466248831 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33340ad6a0 con 0x7f3320003a10 2024-09-10T22:11:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.828+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c0cac90 con 0x7f3320003a10 2024-09-10T22:11:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.828+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c0cb250 con 0x7f3320003a10 2024-09-10T22:11:21.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.836+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333c06cf90 con 0x7f3320003a10 2024-09-10T22:11:21.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.836+0000 7f3338c00640 1 -- 172.21.15.66:0/2466248831 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 35) ==== 98587+0+0 (secure 0 0 0) 0x7f33340ad840 con 0x7f3320003a10 2024-09-10T22:11:21.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.840+0000 7f3338c00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f331c073e60 0x7f331c076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:21.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.840+0000 7f3338c00640 1 -- 172.21.15.66:0/2466248831 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f333412c560 con 0x7f3320003a10 2024-09-10T22:11:21.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.844+0000 7f333be00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f331c073e60 0x7f331c076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:21.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.848+0000 7f333be00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f331c073e60 0x7f331c076320 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f333c158b10 tx=0x7f332c006d90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:21.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:21.852+0000 7f3338c00640 1 -- 172.21.15.66:0/2466248831 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f333c0c9e90 con 0x7f3320003a10 2024-09-10T22:11:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.196+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f333c0b17d0 con 0x7f331c073e60 2024-09-10T22:11:22.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.256+0000 7f3338c00640 1 -- 172.21.15.66:0/2466248831 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f333c0b17d0 con 0x7f331c073e60 2024-09-10T22:11:22.261 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "in-progress", 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_report": { 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: "percentage cloned": "0.0%", 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: "amount cloned": "184/3.0G", 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: "files cloned": "4/4" 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:22.262 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:22.263 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:11:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.260+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f331c073e60 msgr2=0x7f331c076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.260+0000 7f3341e00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f331c073e60 0x7f331c076320 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f333c158b10 tx=0x7f332c006d90 comp rx=0 tx=0).stop 2024-09-10T22:11:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.260+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3320003a10 msgr2=0x7f333c0c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.260+0000 7f3341e00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3320003a10 0x7f333c0c9950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f333401d1d0 tx=0x7f33340a5730 comp rx=0 tx=0).stop 2024-09-10T22:11:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.260+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 shutdown_connections 2024-09-10T22:11:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.260+0000 7f3341e00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f331c073e60 0x7f331c076320 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.260+0000 7f3341e00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f333c159650 0x7f333c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.260+0000 7f3341e00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c1f50d0 0x7f333c0ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.260+0000 7f3341e00640 1 --2- 172.21.15.66:0/2466248831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3320003a10 0x7f333c0c9950 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.260+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 >> 172.21.15.66:0/2466248831 conn(0x7f333c0b2730 msgr2=0x7f333c0b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.260+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 shutdown_connections 2024-09-10T22:11:22.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:22.260+0000 7f3341e00640 1 -- 172.21.15.66:0/2466248831 wait complete. 2024-09-10T22:11:22.315 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 0.0% 2024-09-10T22:11:23.316 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-10T22:11:23.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.524+0000 7f4cf2200640 1 -- 172.21.15.66:0/2274871190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cec15bd40 msgr2=0x7f4cec0cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:23.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.524+0000 7f4cf2200640 1 --2- 172.21.15.66:0/2274871190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cec15bd40 0x7f4cec0cffa0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0747e0 tx=0x7f4cdc01c790 comp rx=0 tx=0).stop 2024-09-10T22:11:23.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.524+0000 7f4cf2200640 1 -- 172.21.15.66:0/2274871190 shutdown_connections 2024-09-10T22:11:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.524+0000 7f4cf2200640 1 --2- 172.21.15.66:0/2274871190 >> v2:172.21.15.94:3300/0 conn(0x7f4cec0cf830 0x7f4cec0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.524+0000 7f4cf2200640 1 --2- 172.21.15.66:0/2274871190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cec15bd40 0x7f4cec0cffa0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.524+0000 7f4cf2200640 1 -- 172.21.15.66:0/2274871190 >> 172.21.15.66:0/2274871190 conn(0x7f4cec0d1c60 msgr2=0x7f4cec0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.524+0000 7f4cf2200640 1 -- 172.21.15.66:0/2274871190 shutdown_connections 2024-09-10T22:11:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.524+0000 7f4cf2200640 1 -- 172.21.15.66:0/2274871190 wait complete. 2024-09-10T22:11:23.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.528+0000 7f4cf2200640 1 Processor -- start 2024-09-10T22:11:23.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.528+0000 7f4cf2200640 1 -- start start 2024-09-10T22:11:23.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.528+0000 7f4cf2200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cec0aec10 0x7f4cec0b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:23.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cf2200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec0cf830 0x7f4cec14bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:23.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cf2200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cec15bd40 0x7f4cec149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:23.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cf2200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec06e520 con 0x7f4cec15bd40 2024-09-10T22:11:23.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cf2200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec06e3a0 con 0x7f4cec0cf830 2024-09-10T22:11:23.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cf2200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4cec06e6a0 con 0x7f4cec0aec10 2024-09-10T22:11:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cec15bd40 0x7f4cec149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cec15bd40 0x7f4cec149e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47632/0 (socket says 172.21.15.66:47632) 2024-09-10T22:11:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cebe00640 1 -- 172.21.15.66:0/2540282886 learned_addr learned my addr 172.21.15.66:0/2540282886 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4ceaa00640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec0cf830 0x7f4cec14bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cebe00640 1 -- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cec0aec10 msgr2=0x7f4cec0b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cebe00640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cec0aec10 0x7f4cec0b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cebe00640 1 -- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec0cf830 msgr2=0x7f4cec14bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cebe00640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec0cf830 0x7f4cec14bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cebe00640 1 -- 172.21.15.66:0/2540282886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cec1f7e10 con 0x7f4cec15bd40 2024-09-10T22:11:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4ceaa00640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec0cf830 0x7f4cec14bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:11:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.532+0000 7f4cebe00640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cec15bd40 0x7f4cec149e60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4002200 tx=0x7f4ce400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.536+0000 7f4ce3400640 1 -- 172.21.15.66:0/2540282886 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4022070 con 0x7f4cec15bd40 2024-09-10T22:11:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.536+0000 7f4ce3400640 1 -- 172.21.15.66:0/2540282886 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ce400a640 con 0x7f4cec15bd40 2024-09-10T22:11:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.536+0000 7f4ce3400640 1 -- 172.21.15.66:0/2540282886 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4015e60 con 0x7f4cec15bd40 2024-09-10T22:11:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.536+0000 7f4cf2200640 1 -- 172.21.15.66:0/2540282886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cec15a250 con 0x7f4cec15bd40 2024-09-10T22:11:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.536+0000 7f4cf2200640 1 -- 172.21.15.66:0/2540282886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cec15a810 con 0x7f4cec15bd40 2024-09-10T22:11:23.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.540+0000 7f4cf2200640 1 -- 172.21.15.66:0/2540282886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc40052a0 con 0x7f4cec15bd40 2024-09-10T22:11:23.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.540+0000 7f4ce3400640 1 -- 172.21.15.66:0/2540282886 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7f4ce4012060 con 0x7f4cec15bd40 2024-09-10T22:11:23.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.540+0000 7f4ce3400640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4ccc073ea0 0x7f4ccc076360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:23.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.540+0000 7f4ce3400640 1 -- 172.21.15.66:0/2540282886 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f4ce4091950 con 0x7f4cec15bd40 2024-09-10T22:11:23.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.540+0000 7f4ceb400640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4ccc073ea0 0x7f4ccc076360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:23.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.548+0000 7f4ce3400640 1 -- 172.21.15.66:0/2540282886 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4ce405ee60 con 0x7f4cec15bd40 2024-09-10T22:11:23.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.556+0000 7f4ceb400640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4ccc073ea0 0x7f4ccc076360 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0af000 tx=0x7f4cdc019530 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:23.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:23.936+0000 7f4cf2200640 1 -- 172.21.15.66:0/2540282886 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f4cc4000bd0 con 0x7f4ccc073ea0 2024-09-10T22:11:24.002 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "in-progress", 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_report": { 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: "percentage cloned": "42.188%", 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: "amount cloned": "1.2G/3.0G", 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: "files cloned": "5/4" 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:24.003 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:11:24.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.000+0000 7f4ce3400640 1 -- 172.21.15.66:0/2540282886 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7f4cc4000bd0 con 0x7f4ccc073ea0 2024-09-10T22:11:24.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.004+0000 7f4cf2200640 1 -- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4ccc073ea0 msgr2=0x7f4ccc076360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:24.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.004+0000 7f4cf2200640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4ccc073ea0 0x7f4ccc076360 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0af000 tx=0x7f4cdc019530 comp rx=0 tx=0).stop 2024-09-10T22:11:24.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.004+0000 7f4cf2200640 1 -- 172.21.15.66:0/2540282886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cec15bd40 msgr2=0x7f4cec149e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.004+0000 7f4cf2200640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cec15bd40 0x7f4cec149e60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4002200 tx=0x7f4ce400cc50 comp rx=0 tx=0).stop 2024-09-10T22:11:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.004+0000 7f4cf2200640 1 -- 172.21.15.66:0/2540282886 shutdown_connections 2024-09-10T22:11:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.004+0000 7f4cf2200640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4ccc073ea0 0x7f4ccc076360 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.004+0000 7f4cf2200640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cec15bd40 0x7f4cec149e60 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.004+0000 7f4cf2200640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec0cf830 0x7f4cec14bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.004+0000 7f4cf2200640 1 --2- 172.21.15.66:0/2540282886 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4cec0aec10 0x7f4cec0b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.004+0000 7f4cf2200640 1 -- 172.21.15.66:0/2540282886 >> 172.21.15.66:0/2540282886 conn(0x7f4cec0d1c60 msgr2=0x7f4cec072110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.004+0000 7f4cf2200640 1 -- 172.21.15.66:0/2540282886 shutdown_connections 2024-09-10T22:11:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:24.004+0000 7f4cf2200640 1 -- 172.21.15.66:0/2540282886 wait complete. 2024-09-10T22:11:24.037 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 42.188% 2024-09-10T22:11:25.039 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-10T22:11:25.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.240+0000 7fdcb8000640 1 -- 172.21.15.66:0/4205813376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc98003a10 msgr2=0x7fdc98003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:25.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.240+0000 7fdcb8000640 1 --2- 172.21.15.66:0/4205813376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc98003a10 0x7fdc98003e10 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fdca4005fd0 tx=0x7fdca401cd90 comp rx=0 tx=0).stop 2024-09-10T22:11:25.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.240+0000 7fdcb8000640 1 -- 172.21.15.66:0/4205813376 shutdown_connections 2024-09-10T22:11:25.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.240+0000 7fdcb8000640 1 --2- 172.21.15.66:0/4205813376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc98003a10 0x7fdc98003e10 secure :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fdca4005fd0 tx=0x7fdca401cd90 comp rx=0 tx=0).stop 2024-09-10T22:11:25.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.240+0000 7fdcb8000640 1 -- 172.21.15.66:0/4205813376 >> 172.21.15.66:0/4205813376 conn(0x7fdcb00b26c0 msgr2=0x7fdcb00b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:25.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.240+0000 7fdcb8000640 1 -- 172.21.15.66:0/4205813376 shutdown_connections 2024-09-10T22:11:25.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.240+0000 7fdcb8000640 1 -- 172.21.15.66:0/4205813376 wait complete. 2024-09-10T22:11:25.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.244+0000 7fdcb8000640 1 Processor -- start 2024-09-10T22:11:25.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.244+0000 7fdcb8000640 1 -- start start 2024-09-10T22:11:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb8000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcb01fba80 0x7fdcb00c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb8000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcb00c8140 0x7fdcb00ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb8000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00ca860 0x7fdcb00c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb8000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb00cd960 con 0x7fdcb00c8140 2024-09-10T22:11:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb8000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb006d680 con 0x7fdcb00ca860 2024-09-10T22:11:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb8000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb00cdae0 con 0x7fdcb01fba80 2024-09-10T22:11:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcb00c8140 0x7fdcb00ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcb00c8140 0x7fdcb00ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47646/0 (socket says 172.21.15.66:47646) 2024-09-10T22:11:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb4c00640 1 -- 172.21.15.66:0/2987060365 learned_addr learned my addr 172.21.15.66:0/2987060365 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb6000640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00ca860 0x7fdcb00c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb4c00640 1 -- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcb01fba80 msgr2=0x7fdcb00c7c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:11:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb4c00640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcb01fba80 0x7fdcb00c7c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb4c00640 1 -- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00ca860 msgr2=0x7fdcb00c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb4c00640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00ca860 0x7fdcb00c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:25.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb4c00640 1 -- 172.21.15.66:0/2987060365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb015cf10 con 0x7fdcb00c8140 2024-09-10T22:11:25.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcb4c00640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcb00c8140 0x7fdcb00ca320 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00c310 tx=0x7fdc9c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:25.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.248+0000 7fdcad600640 1 -- 172.21.15.66:0/2987060365 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c011790 con 0x7fdcb00c8140 2024-09-10T22:11:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.252+0000 7fdcad600640 1 -- 172.21.15.66:0/2987060365 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc9c017070 con 0x7fdcb00c8140 2024-09-10T22:11:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.252+0000 7fdcad600640 1 -- 172.21.15.66:0/2987060365 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c011ce0 con 0x7fdcb00c8140 2024-09-10T22:11:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.252+0000 7fdcb8000640 1 -- 172.21.15.66:0/2987060365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb00c5900 con 0x7fdcb00c8140 2024-09-10T22:11:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.252+0000 7fdcb8000640 1 -- 172.21.15.66:0/2987060365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb00c5ec0 con 0x7fdcb00c8140 2024-09-10T22:11:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.252+0000 7fdcb8000640 1 -- 172.21.15.66:0/2987060365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb00ce040 con 0x7fdcb00c8140 2024-09-10T22:11:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.256+0000 7fdcad600640 1 -- 172.21.15.66:0/2987060365 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7fdc9c01c460 con 0x7fdcb00c8140 2024-09-10T22:11:25.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.256+0000 7fdcad600640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc90073ef0 0x7fdc900763b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:25.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.256+0000 7fdcad600640 1 -- 172.21.15.66:0/2987060365 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fdc9c091d40 con 0x7fdcb00c8140 2024-09-10T22:11:25.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.256+0000 7fdcb5600640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc90073ef0 0x7fdc900763b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:25.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.260+0000 7fdcad600640 1 -- 172.21.15.66:0/2987060365 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdc9c05f000 con 0x7fdcb00c8140 2024-09-10T22:11:25.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.264+0000 7fdcb5600640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc90073ef0 0x7fdc900763b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdca40b0000 tx=0x7fdca4019b30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.576+0000 7fdcb8000640 1 -- 172.21.15.66:0/2987060365 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fdcb00c56b0 con 0x7fdc90073ef0 2024-09-10T22:11:25.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.612+0000 7fdcad600640 1 -- 172.21.15.66:0/2987060365 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7fdcb00c56b0 con 0x7fdc90073ef0 2024-09-10T22:11:25.617 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:11:25.617 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:11:25.617 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "in-progress", 2024-09-10T22:11:25.617 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:11:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:11:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:11:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:11:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_report": { 2024-09-10T22:11:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "percentage cloned": "64.323%", 2024-09-10T22:11:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "amount cloned": "1.9G/3.0G", 2024-09-10T22:11:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "files cloned": "5/4" 2024-09-10T22:11:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:25.618 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:11:25.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.616+0000 7fdcb8000640 1 -- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc90073ef0 msgr2=0x7fdc900763b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:25.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.616+0000 7fdcb8000640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc90073ef0 0x7fdc900763b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdca40b0000 tx=0x7fdca4019b30 comp rx=0 tx=0).stop 2024-09-10T22:11:25.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.616+0000 7fdcb8000640 1 -- 172.21.15.66:0/2987060365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcb00c8140 msgr2=0x7fdcb00ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:25.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.616+0000 7fdcb8000640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcb00c8140 0x7fdcb00ca320 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00c310 tx=0x7fdc9c00c7d0 comp rx=0 tx=0).stop 2024-09-10T22:11:25.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.616+0000 7fdcb8000640 1 -- 172.21.15.66:0/2987060365 shutdown_connections 2024-09-10T22:11:25.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.616+0000 7fdcb8000640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc90073ef0 0x7fdc900763b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:25.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.616+0000 7fdcb8000640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00ca860 0x7fdcb00c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.616+0000 7fdcb8000640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcb00c8140 0x7fdcb00ca320 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.616+0000 7fdcb8000640 1 --2- 172.21.15.66:0/2987060365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcb01fba80 0x7fdcb00c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.616+0000 7fdcb8000640 1 -- 172.21.15.66:0/2987060365 >> 172.21.15.66:0/2987060365 conn(0x7fdcb00b26c0 msgr2=0x7fdcb015d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.616+0000 7fdcb8000640 1 -- 172.21.15.66:0/2987060365 shutdown_connections 2024-09-10T22:11:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:25.616+0000 7fdcb8000640 1 -- 172.21.15.66:0/2987060365 wait complete. 2024-09-10T22:11:25.654 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 64.323% 2024-09-10T22:11:26.655 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-10T22:11:26.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.900+0000 7f1621600640 1 -- 172.21.15.66:0/158070718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f161c0cc970 msgr2=0x7f161c0affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:26.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.900+0000 7f1621600640 1 --2- 172.21.15.66:0/158070718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f161c0cc970 0x7f161c0affa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f160c009900 tx=0x7f160c01c8e0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.900+0000 7f1621600640 1 -- 172.21.15.66:0/158070718 shutdown_connections 2024-09-10T22:11:26.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.900+0000 7f1621600640 1 --2- 172.21.15.66:0/158070718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f161c0737c0 0x7f161c073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.900+0000 7f1621600640 1 --2- 172.21.15.66:0/158070718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f161c0cc970 0x7f161c0affa0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.900+0000 7f1621600640 1 -- 172.21.15.66:0/158070718 >> 172.21.15.66:0/158070718 conn(0x7f161c0b26c0 msgr2=0x7f161c0b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:26.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.900+0000 7f1621600640 1 -- 172.21.15.66:0/158070718 shutdown_connections 2024-09-10T22:11:26.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.900+0000 7f1621600640 1 -- 172.21.15.66:0/158070718 wait complete. 2024-09-10T22:11:26.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f1621600640 1 Processor -- start 2024-09-10T22:11:26.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f1621600640 1 -- start start 2024-09-10T22:11:26.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f1621600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f161c0737c0 0x7f161c15b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:26.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f1621600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f161c0cc970 0x7f161c1fab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:26.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f1621600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c161800 0x7f161c0b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:26.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f1621600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f161c16b020 con 0x7f161c161800 2024-09-10T22:11:26.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f1621600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f161c16aea0 con 0x7f161c0737c0 2024-09-10T22:11:26.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f1621600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f161c16b1a0 con 0x7f161c0cc970 2024-09-10T22:11:26.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f161b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c161800 0x7f161c0b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:26.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f161b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c161800 0x7f161c0b60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47650/0 (socket says 172.21.15.66:47650) 2024-09-10T22:11:26.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f161b400640 1 -- 172.21.15.66:0/2203240119 learned_addr learned my addr 172.21.15.66:0/2203240119 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:26.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f161aa00640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f161c0737c0 0x7f161c15b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:26.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f161a000640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f161c0cc970 0x7f161c1fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:26.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f161b400640 1 -- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f161c0cc970 msgr2=0x7f161c1fab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:26.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f161b400640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f161c0cc970 0x7f161c1fab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f161b400640 1 -- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f161c0737c0 msgr2=0x7f161c15b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:26.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f161b400640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f161c0737c0 0x7f161c15b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.904+0000 7f161b400640 1 -- 172.21.15.66:0/2203240119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161c1fb450 con 0x7f161c161800 2024-09-10T22:11:26.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.908+0000 7f161b400640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c161800 0x7f161c0b60f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f161400eb50 tx=0x7f161400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:26.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.908+0000 7f1612a00640 1 -- 172.21.15.66:0/2203240119 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614004d30 con 0x7f161c161800 2024-09-10T22:11:26.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.908+0000 7f1612a00640 1 -- 172.21.15.66:0/2203240119 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f161400ce70 con 0x7f161c161800 2024-09-10T22:11:26.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.908+0000 7f1612a00640 1 -- 172.21.15.66:0/2203240119 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614005240 con 0x7f161c161800 2024-09-10T22:11:26.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.908+0000 7f1621600640 1 -- 172.21.15.66:0/2203240119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161c1fb730 con 0x7f161c161800 2024-09-10T22:11:26.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.908+0000 7f1621600640 1 -- 172.21.15.66:0/2203240119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f161c0b58f0 con 0x7f161c161800 2024-09-10T22:11:26.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.908+0000 7f1621600640 1 -- 172.21.15.66:0/2203240119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161c073bf0 con 0x7f161c161800 2024-09-10T22:11:26.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.912+0000 7f1612a00640 1 -- 172.21.15.66:0/2203240119 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7f16140053e0 con 0x7f161c161800 2024-09-10T22:11:26.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.912+0000 7f1612a00640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f15fc073f70 0x7f15fc076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:26.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.912+0000 7f1612a00640 1 -- 172.21.15.66:0/2203240119 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f1614092ca0 con 0x7f161c161800 2024-09-10T22:11:26.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.916+0000 7f161aa00640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f15fc073f70 0x7f15fc076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:26.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.920+0000 7f161aa00640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f15fc073f70 0x7f15fc076430 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f160c0af000 tx=0x7f160c01b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:26.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:26.920+0000 7f1612a00640 1 -- 172.21.15.66:0/2203240119 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f161405fe60 con 0x7f161c161800 2024-09-10T22:11:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:27.148+0000 7f1621600640 1 -- 172.21.15.66:0/2203240119 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f161c0d00a0 con 0x7f15fc073f70 2024-09-10T22:11:27.576 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:11:27.579 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:11:28.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.880+0000 7f1612a00640 1 -- 172.21.15.66:0/2203240119 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f161c0d00a0 con 0x7f15fc073f70 2024-09-10T22:11:28.883 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:11:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:11:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "complete" 2024-09-10T22:11:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:28.884 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:11:28.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.884+0000 7f1621600640 1 -- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f15fc073f70 msgr2=0x7f15fc076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:28.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.884+0000 7f1621600640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f15fc073f70 0x7f15fc076430 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f160c0af000 tx=0x7f160c01b120 comp rx=0 tx=0).stop 2024-09-10T22:11:28.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.884+0000 7f1621600640 1 -- 172.21.15.66:0/2203240119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c161800 msgr2=0x7f161c0b60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:28.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.884+0000 7f1621600640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c161800 0x7f161c0b60f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f161400eb50 tx=0x7f161400c6a0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.884+0000 7f1621600640 1 -- 172.21.15.66:0/2203240119 shutdown_connections 2024-09-10T22:11:28.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.884+0000 7f1621600640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f15fc073f70 0x7f15fc076430 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.884+0000 7f1621600640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161c161800 0x7f161c0b60f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.884+0000 7f1621600640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f161c0cc970 0x7f161c1fab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.884+0000 7f1621600640 1 --2- 172.21.15.66:0/2203240119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f161c0737c0 0x7f161c15b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.884+0000 7f1621600640 1 -- 172.21.15.66:0/2203240119 >> 172.21.15.66:0/2203240119 conn(0x7f161c0b26c0 msgr2=0x7f161c167db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:28.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.884+0000 7f1621600640 1 -- 172.21.15.66:0/2203240119 shutdown_connections 2024-09-10T22:11:28.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:28.884+0000 7f1621600640 1 -- 172.21.15.66:0/2203240119 wait complete. 2024-09-10T22:11:28.911 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-10T22:11:29.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.064+0000 7fa433e00640 1 -- 172.21.15.66:0/3594182497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa42c0d0550 msgr2=0x7fa42c0c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.064+0000 7fa433e00640 1 --2- 172.21.15.66:0/3594182497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa42c0d0550 0x7fa42c0c4df0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa42c06c4f0 tx=0x7fa41801c830 comp rx=0 tx=0).stop 2024-09-10T22:11:29.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.064+0000 7fa433e00640 1 -- 172.21.15.66:0/3594182497 shutdown_connections 2024-09-10T22:11:29.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.064+0000 7fa433e00640 1 --2- 172.21.15.66:0/3594182497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa42c0d0550 0x7fa42c0c4df0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.064+0000 7fa433e00640 1 --2- 172.21.15.66:0/3594182497 >> v2:172.21.15.94:3300/0 conn(0x7fa42c15a310 0x7fa42c0c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.064+0000 7fa433e00640 1 -- 172.21.15.66:0/3594182497 >> 172.21.15.66:0/3594182497 conn(0x7fa42c0cfa30 msgr2=0x7fa42c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:29.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.064+0000 7fa433e00640 1 -- 172.21.15.66:0/3594182497 shutdown_connections 2024-09-10T22:11:29.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.064+0000 7fa433e00640 1 -- 172.21.15.66:0/3594182497 wait complete. 2024-09-10T22:11:29.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa433e00640 1 Processor -- start 2024-09-10T22:11:29.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa433e00640 1 -- start start 2024-09-10T22:11:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa433e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa42c0aeec0 0x7fa42c157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa433e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa42c15a310 0x7fa42c146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa433e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa42c152ba0 0x7fa42c0b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa433e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c15ed60 con 0x7fa42c15a310 2024-09-10T22:11:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa433e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c15ebe0 con 0x7fa42c0aeec0 2024-09-10T22:11:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa433e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa42c167300 con 0x7fa42c152ba0 2024-09-10T22:11:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa430a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa42c15a310 0x7fa42c146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa430a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa42c15a310 0x7fa42c146060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47664/0 (socket says 172.21.15.66:47664) 2024-09-10T22:11:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa430a00640 1 -- 172.21.15.66:0/3401074857 learned_addr learned my addr 172.21.15.66:0/3401074857 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa431400640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa42c0aeec0 0x7fa42c157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa431e00640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa42c152ba0 0x7fa42c0b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa430a00640 1 -- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa42c152ba0 msgr2=0x7fa42c0b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa430a00640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa42c152ba0 0x7fa42c0b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa430a00640 1 -- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa42c0aeec0 msgr2=0x7fa42c157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa430a00640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa42c0aeec0 0x7fa42c157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa430a00640 1 -- 172.21.15.66:0/3401074857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c146910 con 0x7fa42c15a310 2024-09-10T22:11:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa430a00640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa42c15a310 0x7fa42c146060 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa418009850 tx=0x7fa4180a6b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa429600640 1 -- 172.21.15.66:0/3401074857 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4180ac070 con 0x7fa42c15a310 2024-09-10T22:11:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa429600640 1 -- 172.21.15.66:0/3401074857 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa418004030 con 0x7fa42c15a310 2024-09-10T22:11:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa433e00640 1 -- 172.21.15.66:0/3401074857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42c157ef0 con 0x7fa42c15a310 2024-09-10T22:11:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa429600640 1 -- 172.21.15.66:0/3401074857 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4180044a0 con 0x7fa42c15a310 2024-09-10T22:11:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa433e00640 1 -- 172.21.15.66:0/3401074857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42c1584b0 con 0x7fa42c15a310 2024-09-10T22:11:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.068+0000 7fa433e00640 1 -- 172.21.15.66:0/3401074857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42c152fb0 con 0x7fa42c15a310 2024-09-10T22:11:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.072+0000 7fa429600640 1 -- 172.21.15.66:0/3401074857 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7fa418005040 con 0x7fa42c15a310 2024-09-10T22:11:29.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.072+0000 7fa429600640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa40c073ed0 0x7fa40c076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:29.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.072+0000 7fa429600640 1 -- 172.21.15.66:0/3401074857 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fa41812c370 con 0x7fa42c15a310 2024-09-10T22:11:29.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.072+0000 7fa431400640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa40c073ed0 0x7fa40c076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.072+0000 7fa431400640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa40c073ed0 0x7fa40c076390 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa42000bc50 tx=0x7fa42000b3e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.076+0000 7fa429600640 1 -- 172.21.15.66:0/3401074857 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa42c152fb0 con 0x7fa42c15a310 2024-09-10T22:11:29.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.284+0000 7fa433e00640 1 -- 172.21.15.66:0/3401074857 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fa42c070420 con 0x7fa40c073ed0 2024-09-10T22:11:29.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.292+0000 7fa429600640 1 -- 172.21.15.66:0/3401074857 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fa42c070420 con 0x7fa40c073ed0 2024-09-10T22:11:29.297 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:11:29.297 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:11:29.297 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "complete" 2024-09-10T22:11:29.297 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:29.298 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:11:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.296+0000 7fa433e00640 1 -- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa40c073ed0 msgr2=0x7fa40c076390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.296+0000 7fa433e00640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa40c073ed0 0x7fa40c076390 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa42000bc50 tx=0x7fa42000b3e0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.296+0000 7fa433e00640 1 -- 172.21.15.66:0/3401074857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa42c15a310 msgr2=0x7fa42c146060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.296+0000 7fa433e00640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa42c15a310 0x7fa42c146060 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa418009850 tx=0x7fa4180a6b50 comp rx=0 tx=0).stop 2024-09-10T22:11:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.296+0000 7fa433e00640 1 -- 172.21.15.66:0/3401074857 shutdown_connections 2024-09-10T22:11:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.296+0000 7fa433e00640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa40c073ed0 0x7fa40c076390 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.296+0000 7fa433e00640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa42c152ba0 0x7fa42c0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.296+0000 7fa433e00640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa42c15a310 0x7fa42c146060 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.296+0000 7fa433e00640 1 --2- 172.21.15.66:0/3401074857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa42c0aeec0 0x7fa42c157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.296+0000 7fa433e00640 1 -- 172.21.15.66:0/3401074857 >> 172.21.15.66:0/3401074857 conn(0x7fa42c0cfa30 msgr2=0x7fa42c0ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:29.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.296+0000 7fa433e00640 1 -- 172.21.15.66:0/3401074857 shutdown_connections 2024-09-10T22:11:29.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.296+0000 7fa433e00640 1 -- 172.21.15.66:0/3401074857 wait complete. 2024-09-10T22:11:29.318 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-10T22:11:29.318 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:11:29.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 -- 172.21.15.66:0/3442075470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54800051d0 msgr2=0x7f548000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 --2- 172.21.15.66:0/3442075470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54800051d0 0x7f548000b9d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f5490009ee0 tx=0x7f549001d5f0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549d800640 1 -- 172.21.15.66:0/3442075470 reap_dead start 2024-09-10T22:11:29.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 -- 172.21.15.66:0/3442075470 shutdown_connections 2024-09-10T22:11:29.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 -- 172.21.15.66:0/3442075470 >> 172.21.15.66:0/3442075470 conn(0x7f54980d1c50 msgr2=0x7f54980d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:29.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 -- 172.21.15.66:0/3442075470 shutdown_connections 2024-09-10T22:11:29.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 -- 172.21.15.66:0/3442075470 wait complete. 2024-09-10T22:11:29.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 Processor -- start 2024-09-10T22:11:29.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 -- start start 2024-09-10T22:11:29.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54800051d0 0x7f54981f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:29.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54980cab90 0x7f54980c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:29.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54980caff0 0x7f54980c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:29.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f54980d28f0 con 0x7f54980caff0 2024-09-10T22:11:29.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54980d2770 con 0x7f54800051d0 2024-09-10T22:11:29.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549f800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f54980d2a70 con 0x7f54980cab90 2024-09-10T22:11:29.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54980caff0 0x7f54980c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:29.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54980caff0 0x7f54980c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47690/0 (socket says 172.21.15.66:47690) 2024-09-10T22:11:29.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549d800640 1 -- 172.21.15.66:0/3783388831 learned_addr learned my addr 172.21.15.66:0/3783388831 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:29.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549ce00640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54800051d0 0x7f54981f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:29.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f5497e00640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54980cab90 0x7f54980c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:29.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549d800640 1 -- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54980cab90 msgr2=0x7f54980c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549d800640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54980cab90 0x7f54980c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549d800640 1 -- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54800051d0 msgr2=0x7f54981f4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549d800640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54800051d0 0x7f54981f4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.464+0000 7f549d800640 1 -- 172.21.15.66:0/3783388831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54980cb4c0 con 0x7f54980caff0 2024-09-10T22:11:29.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.468+0000 7f549d800640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54980caff0 0x7f54980c8d20 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5490002240 tx=0x7f54900a7d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:29.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.468+0000 7f5494c00640 1 -- 172.21.15.66:0/3783388831 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54900ad050 con 0x7f54980caff0 2024-09-10T22:11:29.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.468+0000 7f5494c00640 1 -- 172.21.15.66:0/3783388831 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54900a74d0 con 0x7f54980caff0 2024-09-10T22:11:29.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.468+0000 7f549f800640 1 -- 172.21.15.66:0/3783388831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54980cb7a0 con 0x7f54980caff0 2024-09-10T22:11:29.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.468+0000 7f5494c00640 1 -- 172.21.15.66:0/3783388831 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490004c60 con 0x7f54980caff0 2024-09-10T22:11:29.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.468+0000 7f549f800640 1 -- 172.21.15.66:0/3783388831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549806df80 con 0x7f54980caff0 2024-09-10T22:11:29.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.468+0000 7f549f800640 1 -- 172.21.15.66:0/3783388831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f549806dd90 con 0x7f54980caff0 2024-09-10T22:11:29.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.468+0000 7f5494c00640 1 -- 172.21.15.66:0/3783388831 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7f54900b2450 con 0x7f54980caff0 2024-09-10T22:11:29.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.468+0000 7f5494c00640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5478073ed0 0x7f5478076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:29.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.468+0000 7f5494c00640 1 -- 172.21.15.66:0/3783388831 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f549012cf20 con 0x7f54980caff0 2024-09-10T22:11:29.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.472+0000 7f549ce00640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5478073ed0 0x7f5478076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:29.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.472+0000 7f549ce00640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5478073ed0 0x7f5478076390 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f548c000fc0 tx=0x7f548c001cc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:29.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.472+0000 7f5494c00640 1 -- 172.21.15.66:0/3783388831 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f54900b8070 con 0x7f54980caff0 2024-09-10T22:11:29.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.676+0000 7f549f800640 1 -- 172.21.15.66:0/3783388831 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f54980cfb50 con 0x7f5478073ed0 2024-09-10T22:11:29.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.676+0000 7f5494c00640 1 -- 172.21.15.66:0/3783388831 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f54980cfb50 con 0x7f5478073ed0 2024-09-10T22:11:29.682 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:11:29.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:29.682 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:11:29.682 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:29.682 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:11:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.680+0000 7f549f800640 1 -- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5478073ed0 msgr2=0x7f5478076390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.680+0000 7f549f800640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5478073ed0 0x7f5478076390 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f548c000fc0 tx=0x7f548c001cc0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.680+0000 7f549f800640 1 -- 172.21.15.66:0/3783388831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54980caff0 msgr2=0x7f54980c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.680+0000 7f549f800640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54980caff0 0x7f54980c8d20 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5490002240 tx=0x7f54900a7d80 comp rx=0 tx=0).stop 2024-09-10T22:11:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.680+0000 7f549f800640 1 -- 172.21.15.66:0/3783388831 shutdown_connections 2024-09-10T22:11:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.680+0000 7f549f800640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5478073ed0 0x7f5478076390 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.680+0000 7f549f800640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54980caff0 0x7f54980c8d20 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.680+0000 7f549f800640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54980cab90 0x7f54980c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.680+0000 7f549f800640 1 --2- 172.21.15.66:0/3783388831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54800051d0 0x7f54981f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.680+0000 7f549f800640 1 -- 172.21.15.66:0/3783388831 >> 172.21.15.66:0/3783388831 conn(0x7f54980d1c50 msgr2=0x7f54980b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:29.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.680+0000 7f549f800640 1 -- 172.21.15.66:0/3783388831 shutdown_connections 2024-09-10T22:11:29.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.680+0000 7f549f800640 1 -- 172.21.15.66:0/3783388831 wait complete. 2024-09-10T22:11:29.703 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-10T22:11:29.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.848+0000 7f5350600640 1 -- 172.21.15.66:0/2587362175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f534815f450 msgr2=0x7f53480b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.848+0000 7f5350600640 1 --2- 172.21.15.66:0/2587362175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f534815f450 0x7f53480b3170 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f533c009840 tx=0x7f533c01d1c0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.848+0000 7f5350600640 1 -- 172.21.15.66:0/2587362175 shutdown_connections 2024-09-10T22:11:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.848+0000 7f5350600640 1 --2- 172.21.15.66:0/2587362175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5348160280 0x7f53480aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.848+0000 7f5350600640 1 --2- 172.21.15.66:0/2587362175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f534815f450 0x7f53480b3170 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.848+0000 7f5350600640 1 -- 172.21.15.66:0/2587362175 >> 172.21.15.66:0/2587362175 conn(0x7f53480d1c60 msgr2=0x7f53480d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.848+0000 7f5350600640 1 -- 172.21.15.66:0/2587362175 shutdown_connections 2024-09-10T22:11:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.848+0000 7f5350600640 1 -- 172.21.15.66:0/2587362175 wait complete. 2024-09-10T22:11:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5350600640 1 Processor -- start 2024-09-10T22:11:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5350600640 1 -- start start 2024-09-10T22:11:29.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5350600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53480b1e10 0x7f5348164fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:29.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5350600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f534815f450 0x7f5348159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:29.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5350600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5348160280 0x7f53480c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:29.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5350600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53480afe80 con 0x7f53480b1e10 2024-09-10T22:11:29.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5350600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53480afd00 con 0x7f5348160280 2024-09-10T22:11:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5350600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5348168eb0 con 0x7f534815f450 2024-09-10T22:11:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f534dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53480b1e10 0x7f5348164fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f534dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53480b1e10 0x7f5348164fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47698/0 (socket says 172.21.15.66:47698) 2024-09-10T22:11:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f534dc00640 1 -- 172.21.15.66:0/3652333254 learned_addr learned my addr 172.21.15.66:0/3652333254 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f534d200640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f534815f450 0x7f5348159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f534e600640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5348160280 0x7f53480c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f534dc00640 1 -- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f534815f450 msgr2=0x7f5348159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f534dc00640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f534815f450 0x7f5348159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f534dc00640 1 -- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5348160280 msgr2=0x7f53480c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f534dc00640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5348160280 0x7f53480c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f534dc00640 1 -- 172.21.15.66:0/3652333254 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53481f81a0 con 0x7f53480b1e10 2024-09-10T22:11:29.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f534dc00640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53480b1e10 0x7f5348164fc0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f533c009790 tx=0x7f533c005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:29.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5345600640 1 -- 172.21.15.66:0/3652333254 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c0b2040 con 0x7f53480b1e10 2024-09-10T22:11:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5345600640 1 -- 172.21.15.66:0/3652333254 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f533c004330 con 0x7f53480b1e10 2024-09-10T22:11:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5350600640 1 -- 172.21.15.66:0/3652333254 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53481f8480 con 0x7f53480b1e10 2024-09-10T22:11:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5345600640 1 -- 172.21.15.66:0/3652333254 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c003800 con 0x7f53480b1e10 2024-09-10T22:11:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.852+0000 7f5350600640 1 -- 172.21.15.66:0/3652333254 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53481f8a40 con 0x7f53480b1e10 2024-09-10T22:11:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.856+0000 7f5350600640 1 -- 172.21.15.66:0/3652333254 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53480b3170 con 0x7f53480b1e10 2024-09-10T22:11:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.856+0000 7f5345600640 1 -- 172.21.15.66:0/3652333254 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7f533c0b5070 con 0x7f53480b1e10 2024-09-10T22:11:29.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.856+0000 7f5345600640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5328073ea0 0x7f5328076360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.856+0000 7f5345600640 1 -- 172.21.15.66:0/3652333254 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f533c12b6a0 con 0x7f53480b1e10 2024-09-10T22:11:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.856+0000 7f534d200640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5328073ea0 0x7f5328076360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:29.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.860+0000 7f534d200640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5328073ea0 0x7f5328076360 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f53480b1de0 tx=0x7f5334001420 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:29.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:29.860+0000 7f5345600640 1 -- 172.21.15.66:0/3652333254 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f533c016410 con 0x7f53480b1e10 2024-09-10T22:11:30.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.060+0000 7f5350600640 1 -- 172.21.15.66:0/3652333254 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53480d2a40 con 0x7f5328073ea0 2024-09-10T22:11:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.064+0000 7f5345600640 1 -- 172.21.15.66:0/3652333254 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f53480d2a40 con 0x7f5328073ea0 2024-09-10T22:11:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:30.070 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:11:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sv1" 2024-09-10T22:11:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ss1clone1" 2024-09-10T22:11:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:30.071 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:11:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.068+0000 7f5350600640 1 -- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5328073ea0 msgr2=0x7f5328076360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.068+0000 7f5350600640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5328073ea0 0x7f5328076360 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f53480b1de0 tx=0x7f5334001420 comp rx=0 tx=0).stop 2024-09-10T22:11:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.068+0000 7f5350600640 1 -- 172.21.15.66:0/3652333254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53480b1e10 msgr2=0x7f5348164fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.068+0000 7f5350600640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53480b1e10 0x7f5348164fc0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f533c009790 tx=0x7f533c005790 comp rx=0 tx=0).stop 2024-09-10T22:11:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.068+0000 7f5350600640 1 -- 172.21.15.66:0/3652333254 shutdown_connections 2024-09-10T22:11:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.068+0000 7f5350600640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5328073ea0 0x7f5328076360 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.068+0000 7f5350600640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5348160280 0x7f53480c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.068+0000 7f5350600640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f534815f450 0x7f5348159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.068+0000 7f5350600640 1 --2- 172.21.15.66:0/3652333254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53480b1e10 0x7f5348164fc0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.068+0000 7f5350600640 1 -- 172.21.15.66:0/3652333254 >> 172.21.15.66:0/3652333254 conn(0x7f53480d1c60 msgr2=0x7f53480cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.068+0000 7f5350600640 1 -- 172.21.15.66:0/3652333254 shutdown_connections 2024-09-10T22:11:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.068+0000 7f5350600640 1 -- 172.21.15.66:0/3652333254 wait complete. 2024-09-10T22:11:30.091 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-10T22:11:30.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 -- 172.21.15.66:0/1890172766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f8003fe0 msgr2=0x7f57f8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:30.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 --2- 172.21.15.66:0/1890172766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f8003fe0 0x7f57f8004810 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009b60 tx=0x7f57fc01d270 comp rx=0 tx=0).stop 2024-09-10T22:11:30.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f5819200640 1 -- 172.21.15.66:0/1890172766 reap_dead start 2024-09-10T22:11:30.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 -- 172.21.15.66:0/1890172766 shutdown_connections 2024-09-10T22:11:30.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 -- 172.21.15.66:0/1890172766 >> 172.21.15.66:0/1890172766 conn(0x7f58140d1c60 msgr2=0x7f58140d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:30.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 -- 172.21.15.66:0/1890172766 shutdown_connections 2024-09-10T22:11:30.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 -- 172.21.15.66:0/1890172766 wait complete. 2024-09-10T22:11:30.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 Processor -- start 2024-09-10T22:11:30.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 -- start start 2024-09-10T22:11:30.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f8003fe0 0x7f58140c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:30.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58140b4a90 0x7f58140b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:30.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58140b5d40 0x7f581415e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:30.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f58140afd60 con 0x7f58140b5d40 2024-09-10T22:11:30.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58140afbe0 con 0x7f57f8003fe0 2024-09-10T22:11:30.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f581b200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f58140afee0 con 0x7f58140b4a90 2024-09-10T22:11:30.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f5819200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58140b5d40 0x7f581415e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:30.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f5819200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58140b5d40 0x7f581415e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47712/0 (socket says 172.21.15.66:47712) 2024-09-10T22:11:30.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f5819200640 1 -- 172.21.15.66:0/2891579230 learned_addr learned my addr 172.21.15.66:0/2891579230 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:30.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f5813e00640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f8003fe0 0x7f58140c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:30.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.240+0000 7f580be00640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58140b4a90 0x7f58140b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:30.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f5819200640 1 -- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58140b4a90 msgr2=0x7f58140b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:30.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f5819200640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58140b4a90 0x7f58140b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f5819200640 1 -- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f8003fe0 msgr2=0x7f58140c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:30.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f5819200640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f8003fe0 0x7f58140c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f5819200640 1 -- 172.21.15.66:0/2891579230 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58140c8350 con 0x7f58140b5d40 2024-09-10T22:11:30.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f5819200640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58140b5d40 0x7f581415e270 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f580c002240 tx=0x7f580c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:30.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f5810c00640 1 -- 172.21.15.66:0/2891579230 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c0137e0 con 0x7f58140b5d40 2024-09-10T22:11:30.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f5810c00640 1 -- 172.21.15.66:0/2891579230 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f580c018020 con 0x7f58140b5d40 2024-09-10T22:11:30.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f581b200640 1 -- 172.21.15.66:0/2891579230 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58140b5000 con 0x7f58140b5d40 2024-09-10T22:11:30.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f5810c00640 1 -- 172.21.15.66:0/2891579230 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c013d10 con 0x7f58140b5d40 2024-09-10T22:11:30.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f581b200640 1 -- 172.21.15.66:0/2891579230 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58140c47c0 con 0x7f58140b5d40 2024-09-10T22:11:30.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f581b200640 1 -- 172.21.15.66:0/2891579230 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58140cfe80 con 0x7f58140b5d40 2024-09-10T22:11:30.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.244+0000 7f5810c00640 1 -- 172.21.15.66:0/2891579230 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7f580c010040 con 0x7f58140b5d40 2024-09-10T22:11:30.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.248+0000 7f5810c00640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f57f0073ea0 0x7f57f0076360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:30.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.248+0000 7f5810c00640 1 -- 172.21.15.66:0/2891579230 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f580c095b70 con 0x7f58140b5d40 2024-09-10T22:11:30.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.248+0000 7f5813e00640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f57f0073ea0 0x7f57f0076360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:30.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.248+0000 7f5813e00640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f57f0073ea0 0x7f57f0076360 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f58141f7d40 tx=0x7f58040091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:30.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.252+0000 7f5810c00640 1 -- 172.21.15.66:0/2891579230 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f580c022020 con 0x7f58140b5d40 2024-09-10T22:11:30.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.452+0000 7f581b200640 1 -- 172.21.15.66:0/2891579230 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5814072090 con 0x7f57f0073ea0 2024-09-10T22:11:30.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f5810c00640 1 -- 172.21.15.66:0/2891579230 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5814072090 con 0x7f57f0073ea0 2024-09-10T22:11:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:30.472 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:11:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ss1" 2024-09-10T22:11:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:30.472 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:11:30.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f581b200640 1 -- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f57f0073ea0 msgr2=0x7f57f0076360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:30.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f581b200640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f57f0073ea0 0x7f57f0076360 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f58141f7d40 tx=0x7f58040091e0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f581b200640 1 -- 172.21.15.66:0/2891579230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58140b5d40 msgr2=0x7f581415e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:30.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f581b200640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58140b5d40 0x7f581415e270 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f580c002240 tx=0x7f580c011970 comp rx=0 tx=0).stop 2024-09-10T22:11:30.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f581b200640 1 -- 172.21.15.66:0/2891579230 shutdown_connections 2024-09-10T22:11:30.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f581b200640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f57f0073ea0 0x7f57f0076360 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f581b200640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58140b5d40 0x7f581415e270 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f581b200640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58140b4a90 0x7f58140b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f581b200640 1 --2- 172.21.15.66:0/2891579230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f8003fe0 0x7f58140c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f581b200640 1 -- 172.21.15.66:0/2891579230 >> 172.21.15.66:0/2891579230 conn(0x7f58140d1c60 msgr2=0x7f58140b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:30.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f581b200640 1 -- 172.21.15.66:0/2891579230 shutdown_connections 2024-09-10T22:11:30.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.468+0000 7f581b200640 1 -- 172.21.15.66:0/2891579230 wait complete. 2024-09-10T22:11:30.493 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-10T22:11:30.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.640+0000 7f9837e00640 1 -- 172.21.15.66:0/2122383572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f983806e440 msgr2=0x7f98380b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:30.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.640+0000 7f9837e00640 1 --2- 172.21.15.66:0/2122383572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f983806e440 0x7f98380b16c0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f9838074a50 tx=0x7f983001cd20 comp rx=0 tx=0).stop 2024-09-10T22:11:30.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.640+0000 7f9837e00640 1 -- 172.21.15.66:0/2122383572 shutdown_connections 2024-09-10T22:11:30.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.640+0000 7f9837e00640 1 --2- 172.21.15.66:0/2122383572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f983806e440 0x7f98380b16c0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.640+0000 7f9837e00640 1 --2- 172.21.15.66:0/2122383572 >> v2:172.21.15.94:3300/0 conn(0x7f98381614e0 0x7f98380d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.640+0000 7f9837e00640 1 -- 172.21.15.66:0/2122383572 >> 172.21.15.66:0/2122383572 conn(0x7f98380d1b10 msgr2=0x7f98380d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:30.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.640+0000 7f9837e00640 1 -- 172.21.15.66:0/2122383572 shutdown_connections 2024-09-10T22:11:30.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.640+0000 7f9837e00640 1 -- 172.21.15.66:0/2122383572 wait complete. 2024-09-10T22:11:30.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f9837e00640 1 Processor -- start 2024-09-10T22:11:30.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f9837e00640 1 -- start start 2024-09-10T22:11:30.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f9837e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f983806e440 0x7f98381523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:30.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f9837e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9838161100 0x7f983815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:30.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f9837e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98381614e0 0x7f98381596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:30.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f9837e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f983816aef0 con 0x7f9838161100 2024-09-10T22:11:30.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f9837e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f98380b29a0 con 0x7f98381614e0 2024-09-10T22:11:30.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f9837e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f983816b070 con 0x7f983806e440 2024-09-10T22:11:30.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f982fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9838161100 0x7f983815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:30.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f982fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9838161100 0x7f983815aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47738/0 (socket says 172.21.15.66:47738) 2024-09-10T22:11:30.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f982fe00640 1 -- 172.21.15.66:0/2644185578 learned_addr learned my addr 172.21.15.66:0/2644185578 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:30.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f9837400640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98381614e0 0x7f98381596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:30.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f982fe00640 1 -- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f983806e440 msgr2=0x7f98381523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:30.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f982fe00640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f983806e440 0x7f98381523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f982fe00640 1 -- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98381614e0 msgr2=0x7f98381596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:30.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f982fe00640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98381614e0 0x7f98381596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f982fe00640 1 -- 172.21.15.66:0/2644185578 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98380cacd0 con 0x7f9838161100 2024-09-10T22:11:30.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f9837400640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98381614e0 0x7f98381596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:11:30.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f982fe00640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9838161100 0x7f983815aa40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f982000d720 tx=0x7f982000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:30.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f982ea00640 1 -- 172.21.15.66:0/2644185578 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820017070 con 0x7f9838161100 2024-09-10T22:11:30.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f982ea00640 1 -- 172.21.15.66:0/2644185578 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9820009d20 con 0x7f9838161100 2024-09-10T22:11:30.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f982ea00640 1 -- 172.21.15.66:0/2644185578 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98200106c0 con 0x7f9838161100 2024-09-10T22:11:30.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f9837e00640 1 -- 172.21.15.66:0/2644185578 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98380cafb0 con 0x7f9838161100 2024-09-10T22:11:30.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.644+0000 7f9837e00640 1 -- 172.21.15.66:0/2644185578 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98380cb570 con 0x7f9838161100 2024-09-10T22:11:30.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.648+0000 7f9837e00640 1 -- 172.21.15.66:0/2644185578 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98380cb7b0 con 0x7f9838161100 2024-09-10T22:11:30.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.648+0000 7f982ea00640 1 -- 172.21.15.66:0/2644185578 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7f9820010860 con 0x7f9838161100 2024-09-10T22:11:30.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.648+0000 7f982ea00640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9814073ea0 0x7f9814076360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:30.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.648+0000 7f982ea00640 1 -- 172.21.15.66:0/2644185578 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f9820092500 con 0x7f9838161100 2024-09-10T22:11:30.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.648+0000 7f9836a00640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9814073ea0 0x7f9814076360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:30.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.648+0000 7f9836a00640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9814073ea0 0x7f9814076360 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9828000f70 tx=0x7f98280079c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:30.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.652+0000 7f982ea00640 1 -- 172.21.15.66:0/2644185578 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f98380cb7b0 con 0x7f9838161100 2024-09-10T22:11:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.856+0000 7f9837e00640 1 -- 172.21.15.66:0/2644185578 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f98380b16c0 con 0x7f9814073ea0 2024-09-10T22:11:30.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.884+0000 7f982ea00640 1 -- 172.21.15.66:0/2644185578 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f98380b16c0 con 0x7f9814073ea0 2024-09-10T22:11:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:30.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.888+0000 7f9837e00640 1 -- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9814073ea0 msgr2=0x7f9814076360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:30.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.888+0000 7f9837e00640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9814073ea0 0x7f9814076360 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9828000f70 tx=0x7f98280079c0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.888+0000 7f9837e00640 1 -- 172.21.15.66:0/2644185578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9838161100 msgr2=0x7f983815aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:30.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.888+0000 7f9837e00640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9838161100 0x7f983815aa40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f982000d720 tx=0x7f982000dbe0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.888+0000 7f9837e00640 1 -- 172.21.15.66:0/2644185578 shutdown_connections 2024-09-10T22:11:30.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.888+0000 7f9837e00640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9814073ea0 0x7f9814076360 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.888+0000 7f9837e00640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98381614e0 0x7f98381596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.888+0000 7f9837e00640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9838161100 0x7f983815aa40 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.888+0000 7f9837e00640 1 --2- 172.21.15.66:0/2644185578 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f983806e440 0x7f98381523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:30.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.888+0000 7f9837e00640 1 -- 172.21.15.66:0/2644185578 >> 172.21.15.66:0/2644185578 conn(0x7f98380d1b10 msgr2=0x7f98380b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:30.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.888+0000 7f9837e00640 1 -- 172.21.15.66:0/2644185578 shutdown_connections 2024-09-10T22:11:30.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:30.888+0000 7f9837e00640 1 -- 172.21.15.66:0/2644185578 wait complete. 2024-09-10T22:11:30.912 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-10T22:11:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.060+0000 7f5376600640 1 -- 172.21.15.66:0/2070286629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53700b0f60 msgr2=0x7f53700d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.060+0000 7f5376600640 1 --2- 172.21.15.66:0/2070286629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53700b0f60 0x7f53700d0310 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5358009840 tx=0x7f535801c620 comp rx=0 tx=0).stop 2024-09-10T22:11:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.060+0000 7f5376600640 1 -- 172.21.15.66:0/2070286629 shutdown_connections 2024-09-10T22:11:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.060+0000 7f5376600640 1 --2- 172.21.15.66:0/2070286629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53700b0f60 0x7f53700d0310 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.060+0000 7f5376600640 1 -- 172.21.15.66:0/2070286629 >> 172.21.15.66:0/2070286629 conn(0x7f53700d1d00 msgr2=0x7f53700d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.060+0000 7f5376600640 1 -- 172.21.15.66:0/2070286629 shutdown_connections 2024-09-10T22:11:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.060+0000 7f5376600640 1 -- 172.21.15.66:0/2070286629 wait complete. 2024-09-10T22:11:31.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.060+0000 7f5376600640 1 Processor -- start 2024-09-10T22:11:31.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.060+0000 7f5376600640 1 -- start start 2024-09-10T22:11:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5376600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53700b0f60 0x7f53700ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5376600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700b32a0 0x7f53701f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5376600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f537015f460 0x7f53701fb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5376600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5370169df0 con 0x7f53700b32a0 2024-09-10T22:11:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5376600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5370169c70 con 0x7f53700b0f60 2024-09-10T22:11:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5376600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5370169f70 con 0x7f537015f460 2024-09-10T22:11:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700b32a0 0x7f53701f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700b32a0 0x7f53701f82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47748/0 (socket says 172.21.15.66:47748) 2024-09-10T22:11:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367e00640 1 -- 172.21.15.66:0/1638830061 learned_addr learned my addr 172.21.15.66:0/1638830061 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f536f400640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53700b0f60 0x7f53700ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f536fe00640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f537015f460 0x7f53701fb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367e00640 1 -- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f537015f460 msgr2=0x7f53701fb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367e00640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f537015f460 0x7f53701fb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367e00640 1 -- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53700b0f60 msgr2=0x7f53700ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367e00640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53700b0f60 0x7f53700ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367e00640 1 -- 172.21.15.66:0/1638830061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53700cec60 con 0x7f53700b32a0 2024-09-10T22:11:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367e00640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700b32a0 0x7f53701f82e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f5358009790 tx=0x7f5358005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367400640 1 -- 172.21.15.66:0/1638830061 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53580ad070 con 0x7f53700b32a0 2024-09-10T22:11:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367400640 1 -- 172.21.15.66:0/1638830061 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53580047d0 con 0x7f53700b32a0 2024-09-10T22:11:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367400640 1 -- 172.21.15.66:0/1638830061 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358004c00 con 0x7f53700b32a0 2024-09-10T22:11:31.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5376600640 1 -- 172.21.15.66:0/1638830061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53700cef40 con 0x7f53700b32a0 2024-09-10T22:11:31.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5376600640 1 -- 172.21.15.66:0/1638830061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53700cde70 con 0x7f53700b32a0 2024-09-10T22:11:31.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5376600640 1 -- 172.21.15.66:0/1638830061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53700af390 con 0x7f53700b32a0 2024-09-10T22:11:31.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.064+0000 7f5367400640 1 -- 172.21.15.66:0/1638830061 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7f53580a9480 con 0x7f53700b32a0 2024-09-10T22:11:31.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.068+0000 7f5367400640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f534c073ed0 0x7f534c076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:31.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.068+0000 7f5367400640 1 -- 172.21.15.66:0/1638830061 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f535812c930 con 0x7f53700b32a0 2024-09-10T22:11:31.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.068+0000 7f536f400640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f534c073ed0 0x7f534c076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:31.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.068+0000 7f536f400640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f534c073ed0 0x7f534c076390 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5360001ec0 tx=0x7f53600019e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:31.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.072+0000 7f5367400640 1 -- 172.21.15.66:0/1638830061 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5358016330 con 0x7f53700b32a0 2024-09-10T22:11:31.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.272+0000 7f5376600640 1 -- 172.21.15.66:0/1638830061 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f5370074aa0 con 0x7f534c073ed0 2024-09-10T22:11:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.292+0000 7f5367400640 1 -- 172.21.15.66:0/1638830061 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5370074aa0 con 0x7f534c073ed0 2024-09-10T22:11:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.296+0000 7f5376600640 1 -- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f534c073ed0 msgr2=0x7f534c076390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.296+0000 7f5376600640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f534c073ed0 0x7f534c076390 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5360001ec0 tx=0x7f53600019e0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.296+0000 7f5376600640 1 -- 172.21.15.66:0/1638830061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700b32a0 msgr2=0x7f53701f82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.296+0000 7f5376600640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700b32a0 0x7f53701f82e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f5358009790 tx=0x7f5358005780 comp rx=0 tx=0).stop 2024-09-10T22:11:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.296+0000 7f5376600640 1 -- 172.21.15.66:0/1638830061 shutdown_connections 2024-09-10T22:11:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.296+0000 7f5376600640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f534c073ed0 0x7f534c076390 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.296+0000 7f5376600640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f537015f460 0x7f53701fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.296+0000 7f5376600640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700b32a0 0x7f53701f82e0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.296+0000 7f5376600640 1 --2- 172.21.15.66:0/1638830061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53700b0f60 0x7f53700ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.296+0000 7f5376600640 1 -- 172.21.15.66:0/1638830061 >> 172.21.15.66:0/1638830061 conn(0x7f53700d1d00 msgr2=0x7f53700b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.296+0000 7f5376600640 1 -- 172.21.15.66:0/1638830061 shutdown_connections 2024-09-10T22:11:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.296+0000 7f5376600640 1 -- 172.21.15.66:0/1638830061 wait complete. 2024-09-10T22:11:31.311 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:31.309+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:31.311 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:31.309+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:31.311 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:31.309+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:31.311 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:31.309+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:31.311 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:11:31.309+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:11:31.319 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-10T22:11:31.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.468+0000 7f86ff600640 1 -- 172.21.15.66:0/3589248558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f80aef10 msgr2=0x7f86f80b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.468+0000 7f86ff600640 1 --2- 172.21.15.66:0/3589248558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f80aef10 0x7f86f80b25d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009840 tx=0x7f86e401c820 comp rx=0 tx=0).stop 2024-09-10T22:11:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.468+0000 7f86ff600640 1 -- 172.21.15.66:0/3589248558 shutdown_connections 2024-09-10T22:11:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.468+0000 7f86ff600640 1 --2- 172.21.15.66:0/3589248558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f80aef10 0x7f86f80b25d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.468+0000 7f86ff600640 1 --2- 172.21.15.66:0/3589248558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86f8161730 0x7f86f80ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.468+0000 7f86ff600640 1 -- 172.21.15.66:0/3589248558 >> 172.21.15.66:0/3589248558 conn(0x7f86f80d1c60 msgr2=0x7f86f80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.468+0000 7f86ff600640 1 -- 172.21.15.66:0/3589248558 shutdown_connections 2024-09-10T22:11:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.468+0000 7f86ff600640 1 -- 172.21.15.66:0/3589248558 wait complete. 2024-09-10T22:11:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.468+0000 7f86ff600640 1 Processor -- start 2024-09-10T22:11:31.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.468+0000 7f86ff600640 1 -- start start 2024-09-10T22:11:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86ff600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f80aef10 0x7f86f8160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86ff600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f80b2b10 0x7f86f80b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86ff600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86f8161730 0x7f86f81f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86ff600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f86f816b300 con 0x7f86f80aef10 2024-09-10T22:11:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86ff600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86f816b180 con 0x7f86f80b2b10 2024-09-10T22:11:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86ff600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f86f816b480 con 0x7f86f8161730 2024-09-10T22:11:31.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86fcc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f80aef10 0x7f86f8160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:31.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86fcc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f80aef10 0x7f86f8160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47754/0 (socket says 172.21.15.66:47754) 2024-09-10T22:11:31.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86fcc00640 1 -- 172.21.15.66:0/3852563905 learned_addr learned my addr 172.21.15.66:0/3852563905 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:31.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86f7e00640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f80b2b10 0x7f86f80b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:31.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86fd600640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86f8161730 0x7f86f81f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:31.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86fcc00640 1 -- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86f8161730 msgr2=0x7f86f81f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86fcc00640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86f8161730 0x7f86f81f7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86fcc00640 1 -- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f80b2b10 msgr2=0x7f86f80b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86fcc00640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f80b2b10 0x7f86f80b6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86fcc00640 1 -- 172.21.15.66:0/3852563905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f81f8550 con 0x7f86f80aef10 2024-09-10T22:11:31.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86fcc00640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f80aef10 0x7f86f8160670 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e7a0 tx=0x7f86ec00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:31.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86f4c00640 1 -- 172.21.15.66:0/3852563905 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec00cc60 con 0x7f86f80aef10 2024-09-10T22:11:31.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86f4c00640 1 -- 172.21.15.66:0/3852563905 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86ec010430 con 0x7f86f80aef10 2024-09-10T22:11:31.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86ff600640 1 -- 172.21.15.66:0/3852563905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f81f8830 con 0x7f86f80aef10 2024-09-10T22:11:31.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86f4c00640 1 -- 172.21.15.66:0/3852563905 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec010880 con 0x7f86f80aef10 2024-09-10T22:11:31.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86ff600640 1 -- 172.21.15.66:0/3852563905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f80b58d0 con 0x7f86f80aef10 2024-09-10T22:11:31.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.472+0000 7f86ff600640 1 -- 172.21.15.66:0/3852563905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f80b25d0 con 0x7f86f80aef10 2024-09-10T22:11:31.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.476+0000 7f86f4c00640 1 -- 172.21.15.66:0/3852563905 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7f86ec010a20 con 0x7f86f80aef10 2024-09-10T22:11:31.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.476+0000 7f86f4c00640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86d8073ed0 0x7f86d8076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:31.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.476+0000 7f86f4c00640 1 -- 172.21.15.66:0/3852563905 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f86ec0929e0 con 0x7f86f80aef10 2024-09-10T22:11:31.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.476+0000 7f86f7e00640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86d8073ed0 0x7f86d8076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:31.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.476+0000 7f86f7e00640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86d8073ed0 0x7f86d8076390 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f86e40af000 tx=0x7f86e401b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:31.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.480+0000 7f86f4c00640 1 -- 172.21.15.66:0/3852563905 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f86ec05f4d0 con 0x7f86f80aef10 2024-09-10T22:11:31.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.680+0000 7f86ff600640 1 -- 172.21.15.66:0/3852563905 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f86f815bfb0 con 0x7f86d8073ed0 2024-09-10T22:11:31.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.696+0000 7f86f4c00640 1 -- 172.21.15.66:0/3852563905 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86f815bfb0 con 0x7f86d8073ed0 2024-09-10T22:11:31.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.700+0000 7f86ff600640 1 -- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86d8073ed0 msgr2=0x7f86d8076390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.700+0000 7f86ff600640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86d8073ed0 0x7f86d8076390 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f86e40af000 tx=0x7f86e401b060 comp rx=0 tx=0).stop 2024-09-10T22:11:31.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.700+0000 7f86ff600640 1 -- 172.21.15.66:0/3852563905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f80aef10 msgr2=0x7f86f8160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.700+0000 7f86ff600640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f80aef10 0x7f86f8160670 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e7a0 tx=0x7f86ec00ec60 comp rx=0 tx=0).stop 2024-09-10T22:11:31.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.700+0000 7f86ff600640 1 -- 172.21.15.66:0/3852563905 shutdown_connections 2024-09-10T22:11:31.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.700+0000 7f86ff600640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86d8073ed0 0x7f86d8076390 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.700+0000 7f86ff600640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86f8161730 0x7f86f81f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.700+0000 7f86ff600640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f80b2b10 0x7f86f80b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.700+0000 7f86ff600640 1 --2- 172.21.15.66:0/3852563905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f80aef10 0x7f86f8160670 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.700+0000 7f86ff600640 1 -- 172.21.15.66:0/3852563905 >> 172.21.15.66:0/3852563905 conn(0x7f86f80d1c60 msgr2=0x7f86f80b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:31.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.700+0000 7f86ff600640 1 -- 172.21.15.66:0/3852563905 shutdown_connections 2024-09-10T22:11:31.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.700+0000 7f86ff600640 1 -- 172.21.15.66:0/3852563905 wait complete. 2024-09-10T22:11:31.726 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T22:11:31.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:11:31.783 INFO:teuthology.orchestra.run.smithi066.stdout:2 2024-09-10T22:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:11:31.791 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:11:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.936+0000 7fd6fa600640 1 -- 172.21.15.66:0/3662296497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f4159900 msgr2=0x7fd6f40ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.936+0000 7fd6fa600640 1 --2- 172.21.15.66:0/3662296497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f4159900 0x7fd6f40ca080 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009900 tx=0x7fd6e401c850 comp rx=0 tx=0).stop 2024-09-10T22:11:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.936+0000 7fd6fa600640 1 -- 172.21.15.66:0/3662296497 shutdown_connections 2024-09-10T22:11:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.936+0000 7fd6fa600640 1 --2- 172.21.15.66:0/3662296497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd6f40aee20 0x7fd6f40cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.936+0000 7fd6fa600640 1 --2- 172.21.15.66:0/3662296497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f4159900 0x7fd6f40ca080 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.936+0000 7fd6fa600640 1 -- 172.21.15.66:0/3662296497 >> 172.21.15.66:0/3662296497 conn(0x7fd6f40cfa30 msgr2=0x7fd6f40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.936+0000 7fd6fa600640 1 -- 172.21.15.66:0/3662296497 shutdown_connections 2024-09-10T22:11:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.936+0000 7fd6fa600640 1 -- 172.21.15.66:0/3662296497 wait complete. 2024-09-10T22:11:31.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6fa600640 1 Processor -- start 2024-09-10T22:11:31.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6fa600640 1 -- start start 2024-09-10T22:11:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6fa600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f40aee20 0x7fd6f4150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6fa600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd6f4159900 0x7fd6f406ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6fa600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f415cc80 0x7fd6f415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:31.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6fa600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f4168f70 con 0x7fd6f415cc80 2024-09-10T22:11:31.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6fa600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f4168df0 con 0x7fd6f40aee20 2024-09-10T22:11:31.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6fa600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f41690f0 con 0x7fd6f4159900 2024-09-10T22:11:31.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6f3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f415cc80 0x7fd6f415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:31.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6f3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f415cc80 0x7fd6f415aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47770/0 (socket says 172.21.15.66:47770) 2024-09-10T22:11:31.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6f3e00640 1 -- 172.21.15.66:0/2695836529 learned_addr learned my addr 172.21.15.66:0/2695836529 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6f3400640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f40aee20 0x7fd6f4150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6ebe00640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd6f4159900 0x7fd6f406ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6f3e00640 1 -- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd6f4159900 msgr2=0x7fd6f406ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6f3e00640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd6f4159900 0x7fd6f406ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6f3e00640 1 -- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f40aee20 msgr2=0x7fd6f4150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:31.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6f3e00640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f40aee20 0x7fd6f4150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:31.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6f3e00640 1 -- 172.21.15.66:0/2695836529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f40ada60 con 0x7fd6f415cc80 2024-09-10T22:11:31.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6ebe00640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd6f4159900 0x7fd6f406ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:31.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6f3e00640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f415cc80 0x7fd6f415aff0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00eb20 tx=0x7fd6ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:31.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6eb400640 1 -- 172.21.15.66:0/2695836529 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec004b30 con 0x7fd6f415cc80 2024-09-10T22:11:31.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6eb400640 1 -- 172.21.15.66:0/2695836529 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6ec004cf0 con 0x7fd6f415cc80 2024-09-10T22:11:31.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6eb400640 1 -- 172.21.15.66:0/2695836529 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec005280 con 0x7fd6f415cc80 2024-09-10T22:11:31.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.940+0000 7fd6fa600640 1 -- 172.21.15.66:0/2695836529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f40add40 con 0x7fd6f415cc80 2024-09-10T22:11:31.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.944+0000 7fd6fa600640 1 -- 172.21.15.66:0/2695836529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f40ae300 con 0x7fd6f415cc80 2024-09-10T22:11:31.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.944+0000 7fd6fa600640 1 -- 172.21.15.66:0/2695836529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f40cd930 con 0x7fd6f415cc80 2024-09-10T22:11:31.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.944+0000 7fd6eb400640 1 -- 172.21.15.66:0/2695836529 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7fd6ec005420 con 0x7fd6f415cc80 2024-09-10T22:11:31.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.944+0000 7fd6eb400640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd6d0073ea0 0x7fd6d0076360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:31.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.944+0000 7fd6eb400640 1 -- 172.21.15.66:0/2695836529 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fd6ec092d60 con 0x7fd6f415cc80 2024-09-10T22:11:31.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.944+0000 7fd6f3400640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd6d0073ea0 0x7fd6d0076360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:31.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.948+0000 7fd6f3400640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd6d0073ea0 0x7fd6d0076360 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40af000 tx=0x7fd6e401b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:31.948+0000 7fd6eb400640 1 -- 172.21.15.66:0/2695836529 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd6ec05fea0 con 0x7fd6f415cc80 2024-09-10T22:11:32.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.124+0000 7fd6fa600640 1 -- 172.21.15.66:0/2695836529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fd6f4070470 con 0x7fd6f415cc80 2024-09-10T22:11:32.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.124+0000 7fd6eb400640 1 -- 172.21.15.66:0/2695836529 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7fd6ec05f7a0 con 0x7fd6f415cc80 2024-09-10T22:11:32.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.128+0000 7fd6fa600640 1 -- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd6d0073ea0 msgr2=0x7fd6d0076360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:32.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.128+0000 7fd6fa600640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd6d0073ea0 0x7fd6d0076360 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40af000 tx=0x7fd6e401b090 comp rx=0 tx=0).stop 2024-09-10T22:11:32.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.128+0000 7fd6fa600640 1 -- 172.21.15.66:0/2695836529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f415cc80 msgr2=0x7fd6f415aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:32.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.128+0000 7fd6fa600640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f415cc80 0x7fd6f415aff0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00eb20 tx=0x7fd6ec00c2d0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.128+0000 7fd6fa600640 1 -- 172.21.15.66:0/2695836529 shutdown_connections 2024-09-10T22:11:32.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.128+0000 7fd6fa600640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd6d0073ea0 0x7fd6d0076360 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.128+0000 7fd6fa600640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6f415cc80 0x7fd6f415aff0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.128+0000 7fd6fa600640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd6f4159900 0x7fd6f406ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.128+0000 7fd6fa600640 1 --2- 172.21.15.66:0/2695836529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f40aee20 0x7fd6f4150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.128+0000 7fd6fa600640 1 -- 172.21.15.66:0/2695836529 >> 172.21.15.66:0/2695836529 conn(0x7fd6f40cfa30 msgr2=0x7fd6f4073e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:32.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.128+0000 7fd6fa600640 1 -- 172.21.15.66:0/2695836529 shutdown_connections 2024-09-10T22:11:32.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.128+0000 7fd6fa600640 1 -- 172.21.15.66:0/2695836529 wait complete. 2024-09-10T22:11:32.150 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:11:32.150 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:11:32.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:11:32.313 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:11:32.313 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:11:32.313 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:11:32.314 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:11:32.322 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:11:32.322 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:11:32.374 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:11:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.560+0000 7f0f2c400640 1 -- 172.21.15.66:0/2276500397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f2415f460 msgr2=0x7f0f240cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.560+0000 7f0f2c400640 1 --2- 172.21.15.66:0/2276500397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f2415f460 0x7f0f240cfc00 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009900 tx=0x7f0f1801c790 comp rx=0 tx=0).stop 2024-09-10T22:11:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.560+0000 7f0f2c400640 1 -- 172.21.15.66:0/2276500397 shutdown_connections 2024-09-10T22:11:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.560+0000 7f0f2c400640 1 --2- 172.21.15.66:0/2276500397 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f240b0f60 0x7f0f240d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.560+0000 7f0f2c400640 1 --2- 172.21.15.66:0/2276500397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f2415f460 0x7f0f240cfc00 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.560+0000 7f0f2c400640 1 -- 172.21.15.66:0/2276500397 >> 172.21.15.66:0/2276500397 conn(0x7f0f240d1d00 msgr2=0x7f0f240d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.560+0000 7f0f2c400640 1 -- 172.21.15.66:0/2276500397 shutdown_connections 2024-09-10T22:11:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.560+0000 7f0f2c400640 1 -- 172.21.15.66:0/2276500397 wait complete. 2024-09-10T22:11:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f2c400640 1 Processor -- start 2024-09-10T22:11:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f2c400640 1 -- start start 2024-09-10T22:11:32.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f2c400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f240b0f60 0x7f0f240ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:32.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f2c400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f240b32a0 0x7f0f240b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:32.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f2c400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f2415f460 0x7f0f240b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:32.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f2c400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0f24168ff0 con 0x7f0f240b0f60 2024-09-10T22:11:32.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f2c400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f24168e70 con 0x7f0f2415f460 2024-09-10T22:11:32.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f2c400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0f24169170 con 0x7f0f240b32a0 2024-09-10T22:11:32.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f29a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f240b0f60 0x7f0f240ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:32.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f29a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f240b0f60 0x7f0f240ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44990/0 (socket says 172.21.15.66:44990) 2024-09-10T22:11:32.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f29a00640 1 -- 172.21.15.66:0/2998430129 learned_addr learned my addr 172.21.15.66:0/2998430129 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:32.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f2a400640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f2415f460 0x7f0f240b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:32.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f29000640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f240b32a0 0x7f0f240b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:32.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f29a00640 1 -- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f240b32a0 msgr2=0x7f0f240b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:32.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f29a00640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f240b32a0 0x7f0f240b6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f29a00640 1 -- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f2415f460 msgr2=0x7f0f240b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:32.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f29a00640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f2415f460 0x7f0f240b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f29a00640 1 -- 172.21.15.66:0/2998430129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f240cee30 con 0x7f0f240b0f60 2024-09-10T22:11:32.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f2a400640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f2415f460 0x7f0f240b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:32.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f29a00640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f240b0f60 0x7f0f240ce6b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f0f1801d030 tx=0x7f0f18005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:32.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f21600640 1 -- 172.21.15.66:0/2998430129 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f180b2040 con 0x7f0f240b0f60 2024-09-10T22:11:32.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f21600640 1 -- 172.21.15.66:0/2998430129 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f18004240 con 0x7f0f240b0f60 2024-09-10T22:11:32.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f21600640 1 -- 172.21.15.66:0/2998430129 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18005030 con 0x7f0f240b0f60 2024-09-10T22:11:32.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f2c400640 1 -- 172.21.15.66:0/2998430129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f240cf110 con 0x7f0f240b0f60 2024-09-10T22:11:32.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.564+0000 7f0f2c400640 1 -- 172.21.15.66:0/2998430129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f240b4f40 con 0x7f0f240b0f60 2024-09-10T22:11:32.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.568+0000 7f0f2c400640 1 -- 172.21.15.66:0/2998430129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f240b14b0 con 0x7f0f240b0f60 2024-09-10T22:11:32.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.568+0000 7f0f21600640 1 -- 172.21.15.66:0/2998430129 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7f0f180051d0 con 0x7f0f240b0f60 2024-09-10T22:11:32.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.568+0000 7f0f21600640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0f04073ea0 0x7f0f04076360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:32.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.568+0000 7f0f21600640 1 -- 172.21.15.66:0/2998430129 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f0f1812bc50 con 0x7f0f240b0f60 2024-09-10T22:11:32.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.568+0000 7f0f29000640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0f04073ea0 0x7f0f04076360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:32.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.572+0000 7f0f29000640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0f04073ea0 0x7f0f04076360 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0f2415c310 tx=0x7f0f1000b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:32.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.572+0000 7f0f21600640 1 -- 172.21.15.66:0/2998430129 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0f180164d0 con 0x7f0f240b0f60 2024-09-10T22:11:32.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.756+0000 7f0f2c400640 1 -- 172.21.15.66:0/2998430129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0f24074aa0 con 0x7f0f240b0f60 2024-09-10T22:11:32.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.756+0000 7f0f21600640 1 -- 172.21.15.66:0/2998430129 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) ==== 80+0+0 (secure 0 0 0) 0x7f0f180f8d90 con 0x7f0f240b0f60 2024-09-10T22:11:32.762 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:11:32.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.760+0000 7f0f2c400640 1 -- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0f04073ea0 msgr2=0x7f0f04076360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:32.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.760+0000 7f0f2c400640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0f04073ea0 0x7f0f04076360 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0f2415c310 tx=0x7f0f1000b390 comp rx=0 tx=0).stop 2024-09-10T22:11:32.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.760+0000 7f0f2c400640 1 -- 172.21.15.66:0/2998430129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f240b0f60 msgr2=0x7f0f240ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:32.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.760+0000 7f0f2c400640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f240b0f60 0x7f0f240ce6b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f0f1801d030 tx=0x7f0f18005760 comp rx=0 tx=0).stop 2024-09-10T22:11:32.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.760+0000 7f0f2c400640 1 -- 172.21.15.66:0/2998430129 shutdown_connections 2024-09-10T22:11:32.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.760+0000 7f0f2c400640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0f04073ea0 0x7f0f04076360 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.760+0000 7f0f2c400640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f2415f460 0x7f0f240b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.760+0000 7f0f2c400640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f240b32a0 0x7f0f240b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.760+0000 7f0f2c400640 1 --2- 172.21.15.66:0/2998430129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f240b0f60 0x7f0f240ce6b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:32.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.760+0000 7f0f2c400640 1 -- 172.21.15.66:0/2998430129 >> 172.21.15.66:0/2998430129 conn(0x7f0f240d1d00 msgr2=0x7f0f24150a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:32.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.760+0000 7f0f2c400640 1 -- 172.21.15.66:0/2998430129 shutdown_connections 2024-09-10T22:11:32.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:32.760+0000 7f0f2c400640 1 -- 172.21.15.66:0/2998430129 wait complete. 2024-09-10T22:11:32.784 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-09-10T22:11:32.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:32.784 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T22:11:32.805 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:11:32.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:32.806 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:11:32.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[48229]: fuse finished with error 0 and tester_r 0 2024-09-10T22:11:32.936 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:11:32.936 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:11:32.936 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:11:32.945 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:11:32.945 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:11:32.945 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:11:32.946 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:11:32.990 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:11:32.990 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:11:32.990 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:11:32.990 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:11:32.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:11:32.999 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:11:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.176+0000 7f0449e00640 1 -- 172.21.15.66:0/423540001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0428004d50 msgr2=0x7f04280051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.176+0000 7f0449e00640 1 --2- 172.21.15.66:0/423540001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0428004d50 0x7f04280051d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f043c00c640 tx=0x7f043c01fa80 comp rx=0 tx=0).stop 2024-09-10T22:11:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.176+0000 7f0443400640 1 -- 172.21.15.66:0/423540001 reap_dead start 2024-09-10T22:11:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.176+0000 7f0449e00640 1 -- 172.21.15.66:0/423540001 shutdown_connections 2024-09-10T22:11:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.176+0000 7f0449e00640 1 -- 172.21.15.66:0/423540001 >> 172.21.15.66:0/423540001 conn(0x7f04440d1c60 msgr2=0x7f04440d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.176+0000 7f0449e00640 1 -- 172.21.15.66:0/423540001 shutdown_connections 2024-09-10T22:11:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.176+0000 7f0449e00640 1 -- 172.21.15.66:0/423540001 wait complete. 2024-09-10T22:11:33.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0449e00640 1 Processor -- start 2024-09-10T22:11:33.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0449e00640 1 -- start start 2024-09-10T22:11:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0449e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0428004d50 0x7f04440b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0449e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04440b6820 0x7f044415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0449e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04441e6d30 0x7f04441e7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0449e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04440d04e0 con 0x7f0428004d50 2024-09-10T22:11:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0449e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04440cf4a0 con 0x7f04441e6d30 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0449e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f04440d0660 con 0x7f04440b6820 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0442a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0428004d50 0x7f04440b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0442a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0428004d50 0x7f04440b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45012/0 (socket says 172.21.15.66:45012) 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0442a00640 1 -- 172.21.15.66:0/2351968138 learned_addr learned my addr 172.21.15.66:0/2351968138 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0442a00640 1 -- 172.21.15.66:0/2351968138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04440b6820 msgr2=0x7f044415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0442a00640 1 --2- 172.21.15.66:0/2351968138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04440b6820 0x7f044415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0442a00640 1 -- 172.21.15.66:0/2351968138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04441e6d30 msgr2=0x7f04441e7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0442a00640 1 --2- 172.21.15.66:0/2351968138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04441e6d30 0x7f04441e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0442a00640 1 -- 172.21.15.66:0/2351968138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04440b6d70 con 0x7f0428004d50 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0442a00640 1 --2- 172.21.15.66:0/2351968138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0428004d50 0x7f04440b2f80 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f0434011ae0 tx=0x7f0434009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f043b400640 1 -- 172.21.15.66:0/2351968138 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043400fc50 con 0x7f0428004d50 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f043b400640 1 -- 172.21.15.66:0/2351968138 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f043400fdf0 con 0x7f0428004d50 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0449e00640 1 -- 172.21.15.66:0/2351968138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04440b7050 con 0x7f0428004d50 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f043b400640 1 -- 172.21.15.66:0/2351968138 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434004280 con 0x7f0428004d50 2024-09-10T22:11:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.180+0000 7f0449e00640 1 -- 172.21.15.66:0/2351968138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04440c90b0 con 0x7f0428004d50 2024-09-10T22:11:33.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.184+0000 7f0449e00640 1 -- 172.21.15.66:0/2351968138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04441e76d0 con 0x7f0428004d50 2024-09-10T22:11:33.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.184+0000 7f043b400640 1 -- 172.21.15.66:0/2351968138 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 36) ==== 98638+0+0 (secure 0 0 0) 0x7f0434003600 con 0x7f0428004d50 2024-09-10T22:11:33.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.184+0000 7f043b400640 1 --2- 172.21.15.66:0/2351968138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0420073ea0 0x7f0420076360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:33.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.184+0000 7f043b400640 1 -- 172.21.15.66:0/2351968138 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f0434092b80 con 0x7f0428004d50 2024-09-10T22:11:33.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.184+0000 7f043be00640 1 --2- 172.21.15.66:0/2351968138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0420073ea0 0x7f0420076360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:33.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.188+0000 7f043be00640 1 --2- 172.21.15.66:0/2351968138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0420073ea0 0x7f0420076360 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f04440b7990 tx=0x7f042c009670 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:33.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.188+0000 7f043b400640 1 -- 172.21.15.66:0/2351968138 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f043405fcc0 con 0x7f0428004d50 2024-09-10T22:11:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.500+0000 7f043b400640 1 -- 172.21.15.66:0/2351968138 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f043405b830 con 0x7f0428004d50 2024-09-10T22:11:33.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.536+0000 7f0449e00640 1 -- 172.21.15.66:0/2351968138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04440cffa0 con 0x7f0428004d50 2024-09-10T22:11:33.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.536+0000 7f043b400640 1 -- 172.21.15.66:0/2351968138 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5222 (secure 0 0 0) 0x7f0434095580 con 0x7f0428004d50 2024-09-10T22:11:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:33.542 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":67,"btime":"2024-09-10T22:10:51:841262+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3262462745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3262462745},{"type":"v1","addr":"172.21.15.66:6834","nonce":3262462745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/273414872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":273414872},{"type":"v1","addr":"172.21.15.94:6835","nonce":273414872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7246,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3486474426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3486474426},{"type":"v1","addr":"172.21.15.66:6835","nonce":3486474426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:10:37.643939+0000","modified":"2024-09-10T22:10:40.648893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7249":{"gid":7249,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/1920075377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1920075377},{"type":"v1","addr":"172.21.15.94:6837","nonce":1920075377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7249,"qdb_cluster":[7249]},"id":7}]} 2024-09-10T22:11:33.542 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 67 2024-09-10T22:11:33.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.540+0000 7f0449e00640 1 -- 172.21.15.66:0/2351968138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0420073ea0 msgr2=0x7f0420076360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:33.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.540+0000 7f0449e00640 1 --2- 172.21.15.66:0/2351968138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0420073ea0 0x7f0420076360 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f04440b7990 tx=0x7f042c009670 comp rx=0 tx=0).stop 2024-09-10T22:11:33.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.540+0000 7f0449e00640 1 -- 172.21.15.66:0/2351968138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0428004d50 msgr2=0x7f04440b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:33.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.540+0000 7f0449e00640 1 --2- 172.21.15.66:0/2351968138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0428004d50 0x7f04440b2f80 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f0434011ae0 tx=0x7f0434009f20 comp rx=0 tx=0).stop 2024-09-10T22:11:33.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.540+0000 7f0449e00640 1 -- 172.21.15.66:0/2351968138 shutdown_connections 2024-09-10T22:11:33.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.540+0000 7f0449e00640 1 --2- 172.21.15.66:0/2351968138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0420073ea0 0x7f0420076360 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.540+0000 7f0449e00640 1 --2- 172.21.15.66:0/2351968138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04441e6d30 0x7f04441e7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.544+0000 7f0449e00640 1 --2- 172.21.15.66:0/2351968138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04440b6820 0x7f044415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.544+0000 7f0449e00640 1 --2- 172.21.15.66:0/2351968138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0428004d50 0x7f04440b2f80 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.544+0000 7f0449e00640 1 -- 172.21.15.66:0/2351968138 >> 172.21.15.66:0/2351968138 conn(0x7f04440d1c60 msgr2=0x7f0444071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:33.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.544+0000 7f0449e00640 1 -- 172.21.15.66:0/2351968138 shutdown_connections 2024-09-10T22:11:33.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.544+0000 7f0449e00640 1 -- 172.21.15.66:0/2351968138 wait complete. 2024-09-10T22:11:33.585 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:11:33.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.764+0000 7fb6caa00640 1 -- 172.21.15.66:0/1912929603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6c40b2e60 msgr2=0x7fb6c4160850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:33.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.764+0000 7fb6caa00640 1 --2- 172.21.15.66:0/1912929603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6c40b2e60 0x7fb6c4160850 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb6c408c5e0 tx=0x7fb6bc01cd80 comp rx=0 tx=0).stop 2024-09-10T22:11:33.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.764+0000 7fb6caa00640 1 -- 172.21.15.66:0/1912929603 shutdown_connections 2024-09-10T22:11:33.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.764+0000 7fb6caa00640 1 --2- 172.21.15.66:0/1912929603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6c40b2e60 0x7fb6c4160850 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.764+0000 7fb6caa00640 1 --2- 172.21.15.66:0/1912929603 >> v2:172.21.15.94:3300/0 conn(0x7fb6c40d49b0 0x7fb6c4160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.764+0000 7fb6caa00640 1 --2- 172.21.15.66:0/1912929603 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6c41611d0 0x7fb6c41615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.764+0000 7fb6caa00640 1 -- 172.21.15.66:0/1912929603 >> 172.21.15.66:0/1912929603 conn(0x7fb6c40d3e90 msgr2=0x7fb6c40d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:33.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.764+0000 7fb6caa00640 1 -- 172.21.15.66:0/1912929603 shutdown_connections 2024-09-10T22:11:33.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.764+0000 7fb6caa00640 1 -- 172.21.15.66:0/1912929603 wait complete. 2024-09-10T22:11:33.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6caa00640 1 Processor -- start 2024-09-10T22:11:33.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6caa00640 1 -- start start 2024-09-10T22:11:33.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6caa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c40b2e60 0x7fb6c4159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:33.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6caa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6c40d49b0 0x7fb6c4157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:33.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6caa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6c41611d0 0x7fb6c4150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6caa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c40d2be0 con 0x7fb6c40d49b0 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6caa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c40d2a60 con 0x7fb6c40b2e60 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6caa00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c40ceae0 con 0x7fb6c41611d0 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6bb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6c40d49b0 0x7fb6c4157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6c3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c40b2e60 0x7fb6c4159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6c8a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6c41611d0 0x7fb6c4150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6bb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6c40d49b0 0x7fb6c4157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45038/0 (socket says 172.21.15.66:45038) 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6bb400640 1 -- 172.21.15.66:0/3746104916 learned_addr learned my addr 172.21.15.66:0/3746104916 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6c8a00640 1 -- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c40b2e60 msgr2=0x7fb6c4159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6c8a00640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c40b2e60 0x7fb6c4159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6c8a00640 1 -- 172.21.15.66:0/3746104916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6c40d49b0 msgr2=0x7fb6c4157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6c8a00640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6c40d49b0 0x7fb6c4157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6c8a00640 1 -- 172.21.15.66:0/3746104916 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c41f7950 con 0x7fb6c41611d0 2024-09-10T22:11:33.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6bb400640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6c40d49b0 0x7fb6c4157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:11:33.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6c8a00640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6c41611d0 0x7fb6c4150a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40b6a30 tx=0x7fb6bc0092c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:33.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6c0c00640 1 -- 172.21.15.66:0/3746104916 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0bb070 con 0x7fb6c41611d0 2024-09-10T22:11:33.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6c0c00640 1 -- 172.21.15.66:0/3746104916 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6bc004d40 con 0x7fb6c41611d0 2024-09-10T22:11:33.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6c0c00640 1 -- 172.21.15.66:0/3746104916 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0b2040 con 0x7fb6c41611d0 2024-09-10T22:11:33.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6caa00640 1 -- 172.21.15.66:0/3746104916 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c41f7c30 con 0x7fb6c41611d0 2024-09-10T22:11:33.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.768+0000 7fb6caa00640 1 -- 172.21.15.66:0/3746104916 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c41f81f0 con 0x7fb6c41611d0 2024-09-10T22:11:33.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.772+0000 7fb6caa00640 1 -- 172.21.15.66:0/3746104916 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c41615b0 con 0x7fb6c41611d0 2024-09-10T22:11:33.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.772+0000 7fb6c0c00640 1 -- 172.21.15.66:0/3746104916 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fb6bc0b5030 con 0x7fb6c41611d0 2024-09-10T22:11:33.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.772+0000 7fb6c0c00640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6a0073f30 0x7fb6a00763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:33.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.772+0000 7fb6c0c00640 1 -- 172.21.15.66:0/3746104916 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fb6bc12ba90 con 0x7fb6c41611d0 2024-09-10T22:11:33.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.772+0000 7fb6c3e00640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6a0073f30 0x7fb6a00763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:33.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.772+0000 7fb6c3e00640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6a0073f30 0x7fb6a00763f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb6c408e100 tx=0x7fb6b40091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:33.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.776+0000 7fb6c0c00640 1 -- 172.21.15.66:0/3746104916 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb6bc016970 con 0x7fb6c41611d0 2024-09-10T22:11:33.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6caa00640 1 -- 172.21.15.66:0/3746104916 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6c41f8560 con 0x7fb6c41611d0 2024-09-10T22:11:33.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6c0c00640 1 -- 172.21.15.66:0/3746104916 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5222 (secure 0 0 0) 0x7fb6bc1012f0 con 0x7fb6c41611d0 2024-09-10T22:11:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:33.992 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":67,"btime":"2024-09-10T22:10:51:841262+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3262462745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3262462745},{"type":"v1","addr":"172.21.15.66:6834","nonce":3262462745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/273414872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":273414872},{"type":"v1","addr":"172.21.15.94:6835","nonce":273414872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7246,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3486474426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3486474426},{"type":"v1","addr":"172.21.15.66:6835","nonce":3486474426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:10:37.643939+0000","modified":"2024-09-10T22:10:40.648893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7249":{"gid":7249,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/1920075377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1920075377},{"type":"v1","addr":"172.21.15.94:6837","nonce":1920075377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7249,"qdb_cluster":[7249]},"id":7}]} 2024-09-10T22:11:33.992 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 67 2024-09-10T22:11:33.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6caa00640 1 -- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6a0073f30 msgr2=0x7fb6a00763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:33.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6caa00640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6a0073f30 0x7fb6a00763f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb6c408e100 tx=0x7fb6b40091e0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6caa00640 1 -- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6c41611d0 msgr2=0x7fb6c4150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:33.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6caa00640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6c41611d0 0x7fb6c4150a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40b6a30 tx=0x7fb6bc0092c0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6caa00640 1 -- 172.21.15.66:0/3746104916 shutdown_connections 2024-09-10T22:11:33.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6caa00640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb6a0073f30 0x7fb6a00763f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6caa00640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb6c41611d0 0x7fb6c4150a10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6caa00640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6c40d49b0 0x7fb6c4157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6caa00640 1 --2- 172.21.15.66:0/3746104916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6c40b2e60 0x7fb6c4159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:33.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6caa00640 1 -- 172.21.15.66:0/3746104916 >> 172.21.15.66:0/3746104916 conn(0x7fb6c40d3e90 msgr2=0x7fb6c40b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:33.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.988+0000 7fb6caa00640 1 -- 172.21.15.66:0/3746104916 shutdown_connections 2024-09-10T22:11:33.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:33.992+0000 7fb6caa00640 1 -- 172.21.15.66:0/3746104916 wait complete. 2024-09-10T22:11:34.014 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:11:34.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 -- 172.21.15.66:0/745460342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc49c1618b0 msgr2=0x7fc49c0d2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:34.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 --2- 172.21.15.66:0/745460342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc49c1618b0 0x7fc49c0d2420 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fc49c08e130 tx=0x7fc48401c830 comp rx=0 tx=0).stop 2024-09-10T22:11:34.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 -- 172.21.15.66:0/745460342 shutdown_connections 2024-09-10T22:11:34.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 --2- 172.21.15.66:0/745460342 >> v2:172.21.15.94:3300/0 conn(0x7fc49c0d4990 0x7fc49c0d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 --2- 172.21.15.66:0/745460342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc49c1618b0 0x7fc49c0d2420 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 --2- 172.21.15.66:0/745460342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc49c1614d0 0x7fc49c0d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 -- 172.21.15.66:0/745460342 >> 172.21.15.66:0/745460342 conn(0x7fc49c0d3e70 msgr2=0x7fc49c0d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 -- 172.21.15.66:0/745460342 shutdown_connections 2024-09-10T22:11:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 -- 172.21.15.66:0/745460342 wait complete. 2024-09-10T22:11:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 Processor -- start 2024-09-10T22:11:34.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 -- start start 2024-09-10T22:11:34.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc49c0d4990 0x7fc49c15e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:34.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc49c1614d0 0x7fc49c0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:34.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc49c1618b0 0x7fc49c1502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:34.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c0b1490 con 0x7fc49c0d4990 2024-09-10T22:11:34.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c0b4430 con 0x7fc49c1614d0 2024-09-10T22:11:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a2a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc49c0b1610 con 0x7fc49c1618b0 2024-09-10T22:11:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc49be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc49c0d4990 0x7fc49c15e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc49be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc49c0d4990 0x7fc49c15e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45050/0 (socket says 172.21.15.66:45050) 2024-09-10T22:11:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc49be00640 1 -- 172.21.15.66:0/2182191931 learned_addr learned my addr 172.21.15.66:0/2182191931 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc493400640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc49c1614d0 0x7fc49c0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.160+0000 7fc4a0a00640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc49c1618b0 0x7fc49c1502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc49be00640 1 -- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc49c1618b0 msgr2=0x7fc49c1502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc49be00640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc49c1618b0 0x7fc49c1502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc49be00640 1 -- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc49c1614d0 msgr2=0x7fc49c0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc49be00640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc49c1614d0 0x7fc49c0cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc49be00640 1 -- 172.21.15.66:0/2182191931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c0c9910 con 0x7fc49c0d4990 2024-09-10T22:11:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc493400640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc49c1614d0 0x7fc49c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc49be00640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc49c0d4990 0x7fc49c15e2d0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00e9f0 tx=0x7fc48c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc498c00640 1 -- 172.21.15.66:0/2182191931 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c00cd80 con 0x7fc49c0d4990 2024-09-10T22:11:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc498c00640 1 -- 172.21.15.66:0/2182191931 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc48c010430 con 0x7fc49c0d4990 2024-09-10T22:11:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc4a2a00640 1 -- 172.21.15.66:0/2182191931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c0c9bb0 con 0x7fc49c0d4990 2024-09-10T22:11:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc498c00640 1 -- 172.21.15.66:0/2182191931 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c010860 con 0x7fc49c0d4990 2024-09-10T22:11:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc4a2a00640 1 -- 172.21.15.66:0/2182191931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c0ca170 con 0x7fc49c0d4990 2024-09-10T22:11:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc4a2a00640 1 -- 172.21.15.66:0/2182191931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49c0ca3b0 con 0x7fc49c0d4990 2024-09-10T22:11:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc498c00640 1 -- 172.21.15.66:0/2182191931 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fc48c010a00 con 0x7fc49c0d4990 2024-09-10T22:11:34.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.164+0000 7fc498c00640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc478073f30 0x7fc4780763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:34.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.168+0000 7fc498c00640 1 -- 172.21.15.66:0/2182191931 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7fc48c092c10 con 0x7fc49c0d4990 2024-09-10T22:11:34.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.168+0000 7fc493400640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc478073f30 0x7fc4780763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:34.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.168+0000 7fc493400640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc478073f30 0x7fc4780763f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc4840af000 tx=0x7fc48401b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:34.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.168+0000 7fc498c00640 1 -- 172.21.15.66:0/2182191931 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc48c05f7c0 con 0x7fc49c0d4990 2024-09-10T22:11:34.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.456+0000 7fc4a2a00640 1 -- 172.21.15.66:0/2182191931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc49c0b1ac0 con 0x7fc49c0d4990 2024-09-10T22:11:34.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.456+0000 7fc498c00640 1 -- 172.21.15.66:0/2182191931 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16686 (secure 0 0 0) 0x7fc48c0641f0 con 0x7fc49c0d4990 2024-09-10T22:11:34.459 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:34.459 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":100,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:10:39.240066+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:10:35.358226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T22:10:36.608313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:11:34.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.460+0000 7fc4a2a00640 1 -- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc478073f30 msgr2=0x7fc4780763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:34.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.460+0000 7fc4a2a00640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc478073f30 0x7fc4780763f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc4840af000 tx=0x7fc48401b070 comp rx=0 tx=0).stop 2024-09-10T22:11:34.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.460+0000 7fc4a2a00640 1 -- 172.21.15.66:0/2182191931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc49c0d4990 msgr2=0x7fc49c15e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:34.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.460+0000 7fc4a2a00640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc49c0d4990 0x7fc49c15e2d0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00e9f0 tx=0x7fc48c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.460+0000 7fc4a2a00640 1 -- 172.21.15.66:0/2182191931 shutdown_connections 2024-09-10T22:11:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.460+0000 7fc4a2a00640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc478073f30 0x7fc4780763f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.460+0000 7fc4a2a00640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc49c1618b0 0x7fc49c1502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.460+0000 7fc4a2a00640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc49c1614d0 0x7fc49c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.460+0000 7fc4a2a00640 1 --2- 172.21.15.66:0/2182191931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc49c0d4990 0x7fc49c15e2d0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.460+0000 7fc4a2a00640 1 -- 172.21.15.66:0/2182191931 >> 172.21.15.66:0/2182191931 conn(0x7fc49c0d3e70 msgr2=0x7fc49c1604d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.460+0000 7fc4a2a00640 1 -- 172.21.15.66:0/2182191931 shutdown_connections 2024-09-10T22:11:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.460+0000 7fc4a2a00640 1 -- 172.21.15.66:0/2182191931 wait complete. 2024-09-10T22:11:34.501 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:11:34.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:11:34.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.720+0000 7f669e800640 1 -- 172.21.15.66:0/1594966647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66980d06b0 msgr2=0x7f66980ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:34.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.720+0000 7f669e800640 1 --2- 172.21.15.66:0/1594966647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66980d06b0 0x7f66980ce170 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f66840097c0 tx=0x7f668401c4e0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.720+0000 7f669e800640 1 -- 172.21.15.66:0/1594966647 shutdown_connections 2024-09-10T22:11:34.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.720+0000 7f669e800640 1 --2- 172.21.15.66:0/1594966647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66980d06b0 0x7f66980ce170 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.720+0000 7f669e800640 1 -- 172.21.15.66:0/1594966647 >> 172.21.15.66:0/1594966647 conn(0x7f66980d1c60 msgr2=0x7f66980d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:34.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.720+0000 7f669e800640 1 -- 172.21.15.66:0/1594966647 shutdown_connections 2024-09-10T22:11:34.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.720+0000 7f669e800640 1 -- 172.21.15.66:0/1594966647 wait complete. 2024-09-10T22:11:34.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f669e800640 1 Processor -- start 2024-09-10T22:11:34.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f669e800640 1 -- start start 2024-09-10T22:11:34.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f669e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980afc20 0x7f66980b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:34.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f669e800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66980d06b0 0x7f669806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:34.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f669e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f669815bd40 0x7f669815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:34.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f669e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f669816a160 con 0x7f66980afc20 2024-09-10T22:11:34.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f669e800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6698169fe0 con 0x7f669815bd40 2024-09-10T22:11:34.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f669e800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f669816a2e0 con 0x7f66980d06b0 2024-09-10T22:11:34.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f6697400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980afc20 0x7f66980b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:34.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f6697400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980afc20 0x7f66980b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45064/0 (socket says 172.21.15.66:45064) 2024-09-10T22:11:34.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f6697400640 1 -- 172.21.15.66:0/1852606225 learned_addr learned my addr 172.21.15.66:0/1852606225 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:34.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f6696a00640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66980d06b0 0x7f669806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:34.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f6697e00640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f669815bd40 0x7f669815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:34.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f6697400640 1 -- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66980d06b0 msgr2=0x7f669806f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:34.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f6697400640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66980d06b0 0x7f669806f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f6697400640 1 -- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f669815bd40 msgr2=0x7f669815ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:34.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f6697400640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f669815bd40 0x7f669815ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f6697400640 1 -- 172.21.15.66:0/1852606225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669814eaa0 con 0x7f66980afc20 2024-09-10T22:11:34.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f6697400640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980afc20 0x7f66980b3eb0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f668800c6e0 tx=0x7f668800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:34.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f668f400640 1 -- 172.21.15.66:0/1852606225 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688017070 con 0x7f66980afc20 2024-09-10T22:11:34.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f668f400640 1 -- 172.21.15.66:0/1852606225 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6688005380 con 0x7f66980afc20 2024-09-10T22:11:34.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f669e800640 1 -- 172.21.15.66:0/1852606225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669814ed80 con 0x7f66980afc20 2024-09-10T22:11:34.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f668f400640 1 -- 172.21.15.66:0/1852606225 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668800a8a0 con 0x7f66980afc20 2024-09-10T22:11:34.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f669e800640 1 -- 172.21.15.66:0/1852606225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f669814f340 con 0x7f66980afc20 2024-09-10T22:11:34.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.724+0000 7f669e800640 1 -- 172.21.15.66:0/1852606225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66980cffa0 con 0x7f66980afc20 2024-09-10T22:11:34.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.728+0000 7f668f400640 1 -- 172.21.15.66:0/1852606225 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f668800aa40 con 0x7f66980afc20 2024-09-10T22:11:34.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.728+0000 7f668f400640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6678073f30 0x7f66780763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:34.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.728+0000 7f668f400640 1 -- 172.21.15.66:0/1852606225 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5821+0+0 (secure 0 0 0) 0x7f66880917e0 con 0x7f66980afc20 2024-09-10T22:11:34.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.728+0000 7f6696a00640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6678073f30 0x7f66780763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:34.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.728+0000 7f6696a00640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6678073f30 0x7f66780763f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f66840ae000 tx=0x7f668401ad20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:34.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.732+0000 7f668f400640 1 -- 172.21.15.66:0/1852606225 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f668805e920 con 0x7f66980afc20 2024-09-10T22:11:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.944+0000 7f669e800640 1 -- 172.21.15.66:0/1852606225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66980b0df0 con 0x7f66980afc20 2024-09-10T22:11:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.944+0000 7f668f400640 1 -- 172.21.15.66:0/1852606225 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5222 (secure 0 0 0) 0x7f6688067040 con 0x7f66980afc20 2024-09-10T22:11:34.948 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:34.949 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":67,"btime":"2024-09-10T22:10:51:841262+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3262462745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3262462745},{"type":"v1","addr":"172.21.15.66:6834","nonce":3262462745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/273414872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":273414872},{"type":"v1","addr":"172.21.15.94:6835","nonce":273414872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7246,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3486474426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3486474426},{"type":"v1","addr":"172.21.15.66:6835","nonce":3486474426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:10:37.643939+0000","modified":"2024-09-10T22:10:40.648893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7249":{"gid":7249,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/1920075377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1920075377},{"type":"v1","addr":"172.21.15.94:6837","nonce":1920075377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7249,"qdb_cluster":[7249]},"id":7}]} 2024-09-10T22:11:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 67 2024-09-10T22:11:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.948+0000 7f669e800640 1 -- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6678073f30 msgr2=0x7f66780763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.948+0000 7f669e800640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6678073f30 0x7f66780763f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f66840ae000 tx=0x7f668401ad20 comp rx=0 tx=0).stop 2024-09-10T22:11:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.948+0000 7f669e800640 1 -- 172.21.15.66:0/1852606225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980afc20 msgr2=0x7f66980b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.948+0000 7f669e800640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980afc20 0x7f66980b3eb0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f668800c6e0 tx=0x7f668800cba0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.948+0000 7f669e800640 1 -- 172.21.15.66:0/1852606225 shutdown_connections 2024-09-10T22:11:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.948+0000 7f669e800640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6678073f30 0x7f66780763f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.948+0000 7f669e800640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f669815bd40 0x7f669815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.948+0000 7f669e800640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66980d06b0 0x7f669806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.948+0000 7f669e800640 1 --2- 172.21.15.66:0/1852606225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980afc20 0x7f66980b3eb0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.948+0000 7f669e800640 1 -- 172.21.15.66:0/1852606225 >> 172.21.15.66:0/1852606225 conn(0x7f66980d1c60 msgr2=0x7f66980b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.948+0000 7f669e800640 1 -- 172.21.15.66:0/1852606225 shutdown_connections 2024-09-10T22:11:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:34.948+0000 7f669e800640 1 -- 172.21.15.66:0/1852606225 wait complete. 2024-09-10T22:11:34.971 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:11:35.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.112+0000 7f9507e00640 1 -- 172.21.15.66:0/2914599095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95000cf830 msgr2=0x7f95000d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.112+0000 7f9507e00640 1 --2- 172.21.15.66:0/2914599095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95000cf830 0x7f95000d04e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f9500073840 tx=0x7f94ec01c830 comp rx=0 tx=0).stop 2024-09-10T22:11:35.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.112+0000 7f9507e00640 1 -- 172.21.15.66:0/2914599095 shutdown_connections 2024-09-10T22:11:35.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.112+0000 7f9507e00640 1 --2- 172.21.15.66:0/2914599095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95000cf830 0x7f95000d04e0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.112+0000 7f9507e00640 1 --2- 172.21.15.66:0/2914599095 >> v2:172.21.15.94:3300/0 conn(0x7f950015bd40 0x7f95000cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.112+0000 7f9507e00640 1 -- 172.21.15.66:0/2914599095 >> 172.21.15.66:0/2914599095 conn(0x7f95000d1c60 msgr2=0x7f95000d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:35.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.112+0000 7f9507e00640 1 -- 172.21.15.66:0/2914599095 shutdown_connections 2024-09-10T22:11:35.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.112+0000 7f9507e00640 1 -- 172.21.15.66:0/2914599095 wait complete. 2024-09-10T22:11:35.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9507e00640 1 Processor -- start 2024-09-10T22:11:35.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9507e00640 1 -- start start 2024-09-10T22:11:35.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9507e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95000aec10 0x7f950014f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:35.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9507e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95000cf830 0x7f9500070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9507e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f950015bd40 0x7f9500071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9507e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9500168f20 con 0x7f95000aec10 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9507e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9500168da0 con 0x7f95000cf830 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9507e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f95001690a0 con 0x7f950015bd40 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9505400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95000aec10 0x7f950014f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9505400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95000aec10 0x7f950014f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45080/0 (socket says 172.21.15.66:45080) 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9505400640 1 -- 172.21.15.66:0/2022122323 learned_addr learned my addr 172.21.15.66:0/2022122323 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9504a00640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95000cf830 0x7f9500070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9505e00640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f950015bd40 0x7f9500071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9505400640 1 -- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f950015bd40 msgr2=0x7f9500071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9505400640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f950015bd40 0x7f9500071310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9505400640 1 -- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95000cf830 msgr2=0x7f9500070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9505400640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95000cf830 0x7f9500070dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9505400640 1 -- 172.21.15.66:0/2022122323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95000b6f90 con 0x7f95000aec10 2024-09-10T22:11:35.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9505400640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95000aec10 0x7f950014f050 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f94f400d780 tx=0x7f94f400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:35.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f94fd600640 1 -- 172.21.15.66:0/2022122323 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4017070 con 0x7f95000aec10 2024-09-10T22:11:35.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f94fd600640 1 -- 172.21.15.66:0/2022122323 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94f400de90 con 0x7f95000aec10 2024-09-10T22:11:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9507e00640 1 -- 172.21.15.66:0/2022122323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95000b6cb0 con 0x7f95000aec10 2024-09-10T22:11:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f94fd600640 1 -- 172.21.15.66:0/2022122323 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4004020 con 0x7f95000aec10 2024-09-10T22:11:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.116+0000 7f9507e00640 1 -- 172.21.15.66:0/2022122323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95001fb500 con 0x7f95000aec10 2024-09-10T22:11:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.120+0000 7f9507e00640 1 -- 172.21.15.66:0/2022122323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95000af0a0 con 0x7f95000aec10 2024-09-10T22:11:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.120+0000 7f94fd600640 1 -- 172.21.15.66:0/2022122323 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f94f40041c0 con 0x7f95000aec10 2024-09-10T22:11:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.120+0000 7f94fd600640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f94e0073f30 0x7f94e00763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.120+0000 7f94fd600640 1 -- 172.21.15.66:0/2022122323 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5821+0+0 (secure 0 0 0) 0x7f94f40923b0 con 0x7f95000aec10 2024-09-10T22:11:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.120+0000 7f9504a00640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f94e0073f30 0x7f94e00763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:35.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.120+0000 7f9504a00640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f94e0073f30 0x7f94e00763f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0af000 tx=0x7f94ec01b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:35.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.124+0000 7f94fd600640 1 -- 172.21.15.66:0/2022122323 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f94f4020020 con 0x7f95000aec10 2024-09-10T22:11:35.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 -- 172.21.15.66:0/2022122323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95000b0df0 con 0x7f95000aec10 2024-09-10T22:11:35.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f94fd600640 1 -- 172.21.15.66:0/2022122323 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5222 (secure 0 0 0) 0x7f94f4063690 con 0x7f95000aec10 2024-09-10T22:11:35.339 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:35.339 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":67,"btime":"2024-09-10T22:10:51:841262+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3262462745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3262462745},{"type":"v1","addr":"172.21.15.66:6834","nonce":3262462745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/273414872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":273414872},{"type":"v1","addr":"172.21.15.94:6835","nonce":273414872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7246,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3486474426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3486474426},{"type":"v1","addr":"172.21.15.66:6835","nonce":3486474426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:10:37.643939+0000","modified":"2024-09-10T22:10:40.648893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7249":{"gid":7249,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/1920075377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1920075377},{"type":"v1","addr":"172.21.15.94:6837","nonce":1920075377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7249,"qdb_cluster":[7249]},"id":7}]} 2024-09-10T22:11:35.339 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 67 2024-09-10T22:11:35.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 -- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f94e0073f30 msgr2=0x7f94e00763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f94e0073f30 0x7f94e00763f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0af000 tx=0x7f94ec01b070 comp rx=0 tx=0).stop 2024-09-10T22:11:35.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 -- 172.21.15.66:0/2022122323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95000aec10 msgr2=0x7f950014f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95000aec10 0x7f950014f050 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f94f400d780 tx=0x7f94f400dc40 comp rx=0 tx=0).stop 2024-09-10T22:11:35.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 -- 172.21.15.66:0/2022122323 shutdown_connections 2024-09-10T22:11:35.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f94e0073f30 0x7f94e00763f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f950015bd40 0x7f9500071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95000cf830 0x7f9500070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 --2- 172.21.15.66:0/2022122323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95000aec10 0x7f950014f050 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 -- 172.21.15.66:0/2022122323 >> 172.21.15.66:0/2022122323 conn(0x7f95000d1c60 msgr2=0x7f95000b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:35.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 -- 172.21.15.66:0/2022122323 shutdown_connections 2024-09-10T22:11:35.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.336+0000 7f9507e00640 1 -- 172.21.15.66:0/2022122323 wait complete. 2024-09-10T22:11:35.361 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:11:35.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 -- 172.21.15.66:0/2706124367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11280afc20 msgr2=0x7f11280ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 --2- 172.21.15.66:0/2706124367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11280afc20 0x7f11280ce880 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f1120009c80 tx=0x7f112001cd10 comp rx=0 tx=0).stop 2024-09-10T22:11:35.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 -- 172.21.15.66:0/2706124367 shutdown_connections 2024-09-10T22:11:35.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 --2- 172.21.15.66:0/2706124367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11280afc20 0x7f11280ce880 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 -- 172.21.15.66:0/2706124367 >> 172.21.15.66:0/2706124367 conn(0x7f11280d1c60 msgr2=0x7f11280d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:35.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 -- 172.21.15.66:0/2706124367 shutdown_connections 2024-09-10T22:11:35.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 -- 172.21.15.66:0/2706124367 wait complete. 2024-09-10T22:11:35.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 Processor -- start 2024-09-10T22:11:35.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 -- start start 2024-09-10T22:11:35.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11280afc20 0x7f11281f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:35.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11280d06b0 0x7f1128163b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:35.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f112815bd40 0x7f11281640a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:35.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f112816a1b0 con 0x7f11280d06b0 2024-09-10T22:11:35.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f112816a030 con 0x7f11280afc20 2024-09-10T22:11:35.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.504+0000 7f112d800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f112816a330 con 0x7f112815bd40 2024-09-10T22:11:35.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11280d06b0 0x7f1128163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11280d06b0 0x7f1128163b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45092/0 (socket says 172.21.15.66:45092) 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111fe00640 1 -- 172.21.15.66:0/3525317378 learned_addr learned my addr 172.21.15.66:0/3525317378 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f1126a00640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11280afc20 0x7f11281f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f1127400640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f112815bd40 0x7f11281640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111fe00640 1 -- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f112815bd40 msgr2=0x7f11281640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111fe00640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f112815bd40 0x7f11281640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111fe00640 1 -- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11280afc20 msgr2=0x7f11281f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111fe00640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11280afc20 0x7f11281f84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111fe00640 1 -- 172.21.15.66:0/3525317378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11281f8a30 con 0x7f11280d06b0 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111fe00640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11280d06b0 0x7f1128163b60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f111000c4b0 tx=0x7f111000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111ea00640 1 -- 172.21.15.66:0/3525317378 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11100052d0 con 0x7f11280d06b0 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111ea00640 1 -- 172.21.15.66:0/3525317378 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1110005470 con 0x7f11280d06b0 2024-09-10T22:11:35.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111ea00640 1 -- 172.21.15.66:0/3525317378 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110012890 con 0x7f11280d06b0 2024-09-10T22:11:35.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f112d800640 1 -- 172.21.15.66:0/3525317378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11281659d0 con 0x7f11280d06b0 2024-09-10T22:11:35.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f112d800640 1 -- 172.21.15.66:0/3525317378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1128165f50 con 0x7f11280d06b0 2024-09-10T22:11:35.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f112d800640 1 -- 172.21.15.66:0/3525317378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11280cffa0 con 0x7f11280d06b0 2024-09-10T22:11:35.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.508+0000 7f111ea00640 1 -- 172.21.15.66:0/3525317378 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f1110003600 con 0x7f11280d06b0 2024-09-10T22:11:35.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.512+0000 7f111ea00640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1104073f30 0x7f11040763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:35.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.512+0000 7f111ea00640 1 -- 172.21.15.66:0/3525317378 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5821+0+0 (secure 0 0 0) 0x7f1110091b20 con 0x7f11280d06b0 2024-09-10T22:11:35.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.512+0000 7f1126a00640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1104073f30 0x7f11040763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:35.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.512+0000 7f1126a00640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1104073f30 0x7f11040763f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1118007fd0 tx=0x7f1118009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:35.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.512+0000 7f111ea00640 1 -- 172.21.15.66:0/3525317378 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f11281f8a30 con 0x7f11280d06b0 2024-09-10T22:11:35.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.688+0000 7f112d800640 1 -- 172.21.15.66:0/3525317378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11280b0df0 con 0x7f11280d06b0 2024-09-10T22:11:35.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f111ea00640 1 -- 172.21.15.66:0/3525317378 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16798 (secure 0 0 0) 0x7f1110067380 con 0x7f11280d06b0 2024-09-10T22:11:35.694 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:35.695 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":101,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:11:34.494041+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:10:35.358226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T22:10:36.608313+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:11:35.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f112d800640 1 -- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1104073f30 msgr2=0x7f11040763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f112d800640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1104073f30 0x7f11040763f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1118007fd0 tx=0x7f1118009040 comp rx=0 tx=0).stop 2024-09-10T22:11:35.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f112d800640 1 -- 172.21.15.66:0/3525317378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11280d06b0 msgr2=0x7f1128163b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f112d800640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11280d06b0 0x7f1128163b60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f111000c4b0 tx=0x7f111000c970 comp rx=0 tx=0).stop 2024-09-10T22:11:35.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f112d800640 1 -- 172.21.15.66:0/3525317378 shutdown_connections 2024-09-10T22:11:35.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f112d800640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1104073f30 0x7f11040763f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f112d800640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f112815bd40 0x7f11281640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f112d800640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11280d06b0 0x7f1128163b60 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f112d800640 1 --2- 172.21.15.66:0/3525317378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11280afc20 0x7f11281f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f112d800640 1 -- 172.21.15.66:0/3525317378 >> 172.21.15.66:0/3525317378 conn(0x7f11280d1c60 msgr2=0x7f112806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:35.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f112d800640 1 -- 172.21.15.66:0/3525317378 shutdown_connections 2024-09-10T22:11:35.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.692+0000 7f112d800640 1 -- 172.21.15.66:0/3525317378 wait complete. 2024-09-10T22:11:35.716 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:11:35.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 -- 172.21.15.66:0/419352338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c80aeec0 msgr2=0x7f24c815df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 --2- 172.21.15.66:0/419352338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c80aeec0 0x7f24c815df60 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f24c8069da0 tx=0x7f24c001ccf0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 -- 172.21.15.66:0/419352338 shutdown_connections 2024-09-10T22:11:35.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 --2- 172.21.15.66:0/419352338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c80aeec0 0x7f24c815df60 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 --2- 172.21.15.66:0/419352338 >> v2:172.21.15.94:3300/0 conn(0x7f24c80d0550 0x7f24c80c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 --2- 172.21.15.66:0/419352338 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f24c815a310 0x7f24c80c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 -- 172.21.15.66:0/419352338 >> 172.21.15.66:0/419352338 conn(0x7f24c80cfa30 msgr2=0x7f24c80cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:35.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 -- 172.21.15.66:0/419352338 shutdown_connections 2024-09-10T22:11:35.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 -- 172.21.15.66:0/419352338 wait complete. 2024-09-10T22:11:35.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 Processor -- start 2024-09-10T22:11:35.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 -- start start 2024-09-10T22:11:35.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f24c80aeec0 0x7f24c8156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:35.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24c815a310 0x7f24c814ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:35.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c815ae50 0x7f24c815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:35.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80ca8c0 con 0x7f24c815ae50 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80ca740 con 0x7f24c815a310 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.860+0000 7f24d0000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f24c80caa40 con 0x7f24c80aeec0 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24ce000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c815ae50 0x7f24c815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24ce000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c815ae50 0x7f24c815b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45104/0 (socket says 172.21.15.66:45104) 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24ce000640 1 -- 172.21.15.66:0/4105295656 learned_addr learned my addr 172.21.15.66:0/4105295656 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24cd600640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f24c80aeec0 0x7f24c8156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24ccc00640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24c815a310 0x7f24c814ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24ce000640 1 -- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f24c80aeec0 msgr2=0x7f24c8156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24ce000640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f24c80aeec0 0x7f24c8156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24ce000640 1 -- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24c815a310 msgr2=0x7f24c814ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24ce000640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24c815a310 0x7f24c814ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24ce000640 1 -- 172.21.15.66:0/4105295656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c815cb80 con 0x7f24c815ae50 2024-09-10T22:11:35.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24ce000640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c815ae50 0x7f24c815b2a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f24c001b3a0 tx=0x7f24c00042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:35.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24c5600640 1 -- 172.21.15.66:0/4105295656 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00bb070 con 0x7f24c815ae50 2024-09-10T22:11:35.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24c5600640 1 -- 172.21.15.66:0/4105295656 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24c00045b0 con 0x7f24c815ae50 2024-09-10T22:11:35.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24d0000640 1 -- 172.21.15.66:0/4105295656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c815ce60 con 0x7f24c815ae50 2024-09-10T22:11:35.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24c5600640 1 -- 172.21.15.66:0/4105295656 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00b2040 con 0x7f24c815ae50 2024-09-10T22:11:35.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24d0000640 1 -- 172.21.15.66:0/4105295656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c814d350 con 0x7f24c815ae50 2024-09-10T22:11:35.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24d0000640 1 -- 172.21.15.66:0/4105295656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c80caf80 con 0x7f24c815ae50 2024-09-10T22:11:35.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.864+0000 7f24c5600640 1 -- 172.21.15.66:0/4105295656 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f24c00b5030 con 0x7f24c815ae50 2024-09-10T22:11:35.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.868+0000 7f24c5600640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24a8073f30 0x7f24a80763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:35.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.868+0000 7f24c5600640 1 -- 172.21.15.66:0/4105295656 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5821+0+0 (secure 0 0 0) 0x7f24c012c840 con 0x7f24c815ae50 2024-09-10T22:11:35.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.868+0000 7f24cd600640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24a8073f30 0x7f24a80763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:35.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.868+0000 7f24cd600640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24a8073f30 0x7f24a80763f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f24bc006fd0 tx=0x7f24bc008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:35.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:35.868+0000 7f24c5600640 1 -- 172.21.15.66:0/4105295656 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f24c00a5e20 con 0x7f24c815ae50 2024-09-10T22:11:36.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.080+0000 7f24d0000640 1 -- 172.21.15.66:0/4105295656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f24c80d00f0 con 0x7f24c815ae50 2024-09-10T22:11:36.086 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:11:36.080+0000 7fcc93a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:11:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.084+0000 7f24c5600640 1 -- 172.21.15.66:0/4105295656 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7f24c0016970 con 0x7f24c815ae50 2024-09-10T22:11:36.088 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:11:36.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.088+0000 7f24d0000640 1 -- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24a8073f30 msgr2=0x7f24a80763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:36.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.088+0000 7f24d0000640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24a8073f30 0x7f24a80763f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f24bc006fd0 tx=0x7f24bc008040 comp rx=0 tx=0).stop 2024-09-10T22:11:36.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.088+0000 7f24d0000640 1 -- 172.21.15.66:0/4105295656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c815ae50 msgr2=0x7f24c815b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:36.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.088+0000 7f24d0000640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c815ae50 0x7f24c815b2a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f24c001b3a0 tx=0x7f24c00042d0 comp rx=0 tx=0).stop 2024-09-10T22:11:36.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.088+0000 7f24d0000640 1 -- 172.21.15.66:0/4105295656 shutdown_connections 2024-09-10T22:11:36.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.088+0000 7f24d0000640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f24a8073f30 0x7f24a80763f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:36.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.088+0000 7f24d0000640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c815ae50 0x7f24c815b2a0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:36.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.088+0000 7f24d0000640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24c815a310 0x7f24c814ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:36.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.088+0000 7f24d0000640 1 --2- 172.21.15.66:0/4105295656 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f24c80aeec0 0x7f24c8156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:36.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.088+0000 7f24d0000640 1 -- 172.21.15.66:0/4105295656 >> 172.21.15.66:0/4105295656 conn(0x7f24c80cfa30 msgr2=0x7f24c806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:36.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.088+0000 7f24d0000640 1 -- 172.21.15.66:0/4105295656 shutdown_connections 2024-09-10T22:11:36.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.088+0000 7f24d0000640 1 -- 172.21.15.66:0/4105295656 wait complete. 2024-09-10T22:11:36.110 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:11:36.158 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:11:36.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 -- 172.21.15.66:0/2010695774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a00cd7a0 msgr2=0x7f49a00cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:36.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 --2- 172.21.15.66:0/2010695774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a00cd7a0 0x7f49a00cdc30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4998009bc0 tx=0x7f499801cba0 comp rx=0 tx=0).stop 2024-09-10T22:11:36.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 -- 172.21.15.66:0/2010695774 shutdown_connections 2024-09-10T22:11:36.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 --2- 172.21.15.66:0/2010695774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a00cd7a0 0x7f49a00cdc30 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:36.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 --2- 172.21.15.66:0/2010695774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49a015ef20 0x7f49a00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:36.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 -- 172.21.15.66:0/2010695774 >> 172.21.15.66:0/2010695774 conn(0x7f49a00d1b10 msgr2=0x7f49a00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:36.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 -- 172.21.15.66:0/2010695774 shutdown_connections 2024-09-10T22:11:36.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 -- 172.21.15.66:0/2010695774 wait complete. 2024-09-10T22:11:36.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 Processor -- start 2024-09-10T22:11:36.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 -- start start 2024-09-10T22:11:36.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a00cd7a0 0x7f49a00b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:36.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49a015ef20 0x7f49a00cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:36.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49a015f300 0x7f49a009efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:36.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0168ee0 con 0x7f49a00cd7a0 2024-09-10T22:11:36.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0168d60 con 0x7f49a015f300 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499fe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f49a0169060 con 0x7f49a015ef20 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a00cd7a0 0x7f49a00b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a00cd7a0 0x7f49a00b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45126/0 (socket says 172.21.15.66:45126) 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499ea00640 1 -- 172.21.15.66:0/1274960884 learned_addr learned my addr 172.21.15.66:0/1274960884 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f4997e00640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49a015ef20 0x7f49a00cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499f400640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49a015f300 0x7f49a009efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499ea00640 1 -- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49a015ef20 msgr2=0x7f49a00cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499ea00640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49a015ef20 0x7f49a00cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499ea00640 1 -- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49a015f300 msgr2=0x7f49a009efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499ea00640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49a015f300 0x7f49a009efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499ea00640 1 -- 172.21.15.66:0/1274960884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a00b7280 con 0x7f49a00cd7a0 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f499ea00640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a00cd7a0 0x7f49a00b5990 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f499000d950 tx=0x7f499000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.260+0000 7f4997400640 1 -- 172.21.15.66:0/1274960884 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990017070 con 0x7f49a00cd7a0 2024-09-10T22:11:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.264+0000 7f4997400640 1 -- 172.21.15.66:0/1274960884 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4990009d20 con 0x7f49a00cd7a0 2024-09-10T22:11:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.264+0000 7f499fe00640 1 -- 172.21.15.66:0/1274960884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a00b6240 con 0x7f49a00cd7a0 2024-09-10T22:11:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.264+0000 7f4997400640 1 -- 172.21.15.66:0/1274960884 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49900106c0 con 0x7f49a00cd7a0 2024-09-10T22:11:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.264+0000 7f499fe00640 1 -- 172.21.15.66:0/1274960884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a00b6a80 con 0x7f49a00cd7a0 2024-09-10T22:11:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.264+0000 7f499fe00640 1 -- 172.21.15.66:0/1274960884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a00cdc30 con 0x7f49a00cd7a0 2024-09-10T22:11:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.264+0000 7f4997400640 1 -- 172.21.15.66:0/1274960884 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f4990010860 con 0x7f49a00cd7a0 2024-09-10T22:11:36.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.264+0000 7f4997400640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f497c073f30 0x7f497c0763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:36.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.264+0000 7f4997400640 1 -- 172.21.15.66:0/1274960884 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5879+0+0 (secure 0 0 0) 0x7f4990092400 con 0x7f49a00cd7a0 2024-09-10T22:11:36.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.264+0000 7f4997e00640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f497c073f30 0x7f497c0763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:36.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.268+0000 7f4997e00640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f497c073f30 0x7f497c0763f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f49a00cb470 tx=0x7f4988007ea0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.268+0000 7f4997400640 1 -- 172.21.15.66:0/1274960884 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f49a00b6240 con 0x7f49a00cd7a0 2024-09-10T22:11:36.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:36.476+0000 7f499fe00640 1 -- 172.21.15.66:0/1274960884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f49a0072e00 con 0x7f49a00cd7a0 2024-09-10T22:11:37.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.088+0000 7f4997400640 1 -- 172.21.15.66:0/1274960884 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7f499005ef90 con 0x7f49a00cd7a0 2024-09-10T22:11:37.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.092+0000 7f499fe00640 1 -- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f497c073f30 msgr2=0x7f497c0763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:37.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.092+0000 7f499fe00640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f497c073f30 0x7f497c0763f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f49a00cb470 tx=0x7f4988007ea0 comp rx=0 tx=0).stop 2024-09-10T22:11:37.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.092+0000 7f499fe00640 1 -- 172.21.15.66:0/1274960884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a00cd7a0 msgr2=0x7f49a00b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:37.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.092+0000 7f499fe00640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a00cd7a0 0x7f49a00b5990 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f499000d950 tx=0x7f499000de10 comp rx=0 tx=0).stop 2024-09-10T22:11:37.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.092+0000 7f499fe00640 1 -- 172.21.15.66:0/1274960884 shutdown_connections 2024-09-10T22:11:37.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.092+0000 7f499fe00640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f497c073f30 0x7f497c0763f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:37.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.092+0000 7f499fe00640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49a015f300 0x7f49a009efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:37.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.092+0000 7f499fe00640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49a015ef20 0x7f49a00cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:37.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.092+0000 7f499fe00640 1 --2- 172.21.15.66:0/1274960884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a00cd7a0 0x7f49a00b5990 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:37.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.092+0000 7f499fe00640 1 -- 172.21.15.66:0/1274960884 >> 172.21.15.66:0/1274960884 conn(0x7f49a00d1b10 msgr2=0x7f49a0075880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:37.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.092+0000 7f499fe00640 1 -- 172.21.15.66:0/1274960884 shutdown_connections 2024-09-10T22:11:37.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.092+0000 7f499fe00640 1 -- 172.21.15.66:0/1274960884 wait complete. 2024-09-10T22:11:37.118 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:11:37.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.268+0000 7f3accc00640 1 -- 172.21.15.66:0/1128827731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac815f470 msgr2=0x7f3ac80cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:37.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.268+0000 7f3accc00640 1 --2- 172.21.15.66:0/1128827731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac815f470 0x7f3ac80cd0a0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8073840 tx=0x7f3ab801c890 comp rx=0 tx=0).stop 2024-09-10T22:11:37.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.268+0000 7f3accc00640 1 -- 172.21.15.66:0/1128827731 shutdown_connections 2024-09-10T22:11:37.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.268+0000 7f3accc00640 1 --2- 172.21.15.66:0/1128827731 >> v2:172.21.15.94:3300/0 conn(0x7f3ac80d05c0 0x7f3ac80aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:37.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.268+0000 7f3accc00640 1 --2- 172.21.15.66:0/1128827731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac815f470 0x7f3ac80cd0a0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:37.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.268+0000 7f3accc00640 1 -- 172.21.15.66:0/1128827731 >> 172.21.15.66:0/1128827731 conn(0x7f3ac80d1c60 msgr2=0x7f3ac80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:37.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.268+0000 7f3accc00640 1 -- 172.21.15.66:0/1128827731 shutdown_connections 2024-09-10T22:11:37.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.268+0000 7f3accc00640 1 -- 172.21.15.66:0/1128827731 wait complete. 2024-09-10T22:11:37.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.268+0000 7f3accc00640 1 Processor -- start 2024-09-10T22:11:37.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.268+0000 7f3accc00640 1 -- start start 2024-09-10T22:11:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3accc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac80cfa40 0x7f3ac8152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3accc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac80d05c0 0x7f3ac80b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3accc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ac815f470 0x7f3ac80b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3accc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac81689b0 con 0x7f3ac80cfa40 2024-09-10T22:11:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3accc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac80aff20 con 0x7f3ac80d05c0 2024-09-10T22:11:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3accc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac8168b30 con 0x7f3ac815f470 2024-09-10T22:11:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3ac5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac80cfa40 0x7f3ac8152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:37.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3ac5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac80cfa40 0x7f3ac8152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45148/0 (socket says 172.21.15.66:45148) 2024-09-10T22:11:37.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3ac5e00640 1 -- 172.21.15.66:0/157377822 learned_addr learned my addr 172.21.15.66:0/157377822 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:37.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3ac6800640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ac815f470 0x7f3ac80b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:37.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3abd400640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac80d05c0 0x7f3ac80b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:37.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3ac5e00640 1 -- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ac815f470 msgr2=0x7f3ac80b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:37.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3ac5e00640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ac815f470 0x7f3ac80b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:37.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3ac5e00640 1 -- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac80d05c0 msgr2=0x7f3ac80b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:37.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3ac5e00640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac80d05c0 0x7f3ac80b63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:37.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3ac5e00640 1 -- 172.21.15.66:0/157377822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac81520e0 con 0x7f3ac80cfa40 2024-09-10T22:11:37.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3abd400640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac80d05c0 0x7f3ac80b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:11:37.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3ac5e00640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac80cfa40 0x7f3ac8152ea0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80b0040 tx=0x7f3ab80a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:37.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3abea00640 1 -- 172.21.15.66:0/157377822 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80b2040 con 0x7f3ac80cfa40 2024-09-10T22:11:37.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3abea00640 1 -- 172.21.15.66:0/157377822 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ab80043b0 con 0x7f3ac80cfa40 2024-09-10T22:11:37.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3abea00640 1 -- 172.21.15.66:0/157377822 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8005280 con 0x7f3ac80cfa40 2024-09-10T22:11:37.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3accc00640 1 -- 172.21.15.66:0/157377822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac81523c0 con 0x7f3ac80cfa40 2024-09-10T22:11:37.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3accc00640 1 -- 172.21.15.66:0/157377822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac80b57c0 con 0x7f3ac80cfa40 2024-09-10T22:11:37.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3accc00640 1 -- 172.21.15.66:0/157377822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac80aebd0 con 0x7f3ac80cfa40 2024-09-10T22:11:37.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.272+0000 7f3abea00640 1 -- 172.21.15.66:0/157377822 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f3ab8005420 con 0x7f3ac80cfa40 2024-09-10T22:11:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.276+0000 7f3abea00640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3aa4073f30 0x7f3aa40763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.276+0000 7f3abea00640 1 -- 172.21.15.66:0/157377822 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5879+0+0 (secure 0 0 0) 0x7f3ab812bbd0 con 0x7f3ac80cfa40 2024-09-10T22:11:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.276+0000 7f3abd400640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3aa4073f30 0x7f3aa40763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.276+0000 7f3abd400640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3aa4073f30 0x7f3aa40763f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80cbff0 tx=0x7f3ab0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:37.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.280+0000 7f3abea00640 1 -- 172.21.15.66:0/157377822 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3ab80164b0 con 0x7f3ac80cfa40 2024-09-10T22:11:37.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:37.456+0000 7f3accc00640 1 -- 172.21.15.66:0/157377822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3ac815cf90 con 0x7f3ac80cfa40 2024-09-10T22:11:38.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.092+0000 7f3abea00640 1 -- 172.21.15.66:0/157377822 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7f3ab80f8d30 con 0x7f3ac80cfa40 2024-09-10T22:11:38.095 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:11:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.096+0000 7f3accc00640 1 -- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3aa4073f30 msgr2=0x7f3aa40763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.096+0000 7f3accc00640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3aa4073f30 0x7f3aa40763f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80cbff0 tx=0x7f3ab0009040 comp rx=0 tx=0).stop 2024-09-10T22:11:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.096+0000 7f3accc00640 1 -- 172.21.15.66:0/157377822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac80cfa40 msgr2=0x7f3ac8152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.096+0000 7f3accc00640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac80cfa40 0x7f3ac8152ea0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80b0040 tx=0x7f3ab80a5700 comp rx=0 tx=0).stop 2024-09-10T22:11:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.096+0000 7f3accc00640 1 -- 172.21.15.66:0/157377822 shutdown_connections 2024-09-10T22:11:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.096+0000 7f3accc00640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3aa4073f30 0x7f3aa40763f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.096+0000 7f3accc00640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ac815f470 0x7f3ac80b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.096+0000 7f3accc00640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac80d05c0 0x7f3ac80b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.096+0000 7f3accc00640 1 --2- 172.21.15.66:0/157377822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac80cfa40 0x7f3ac8152ea0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.096+0000 7f3accc00640 1 -- 172.21.15.66:0/157377822 >> 172.21.15.66:0/157377822 conn(0x7f3ac80d1c60 msgr2=0x7f3ac80cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.096+0000 7f3accc00640 1 -- 172.21.15.66:0/157377822 shutdown_connections 2024-09-10T22:11:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.096+0000 7f3accc00640 1 -- 172.21.15.66:0/157377822 wait complete. 2024-09-10T22:11:38.122 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:11:38.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 -- 172.21.15.66:0/2089149678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc080b2b10 msgr2=0x7efc080b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:38.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 --2- 172.21.15.66:0/2089149678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc080b2b10 0x7efc080b1b60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7efc080706e0 tx=0x7efbf001c8c0 comp rx=0 tx=0).stop 2024-09-10T22:11:38.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 -- 172.21.15.66:0/2089149678 shutdown_connections 2024-09-10T22:11:38.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 --2- 172.21.15.66:0/2089149678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc080b2b10 0x7efc080b1b60 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:38.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 --2- 172.21.15.66:0/2089149678 >> v2:172.21.15.94:3300/0 conn(0x7efc081605c0 0x7efc080b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:38.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 -- 172.21.15.66:0/2089149678 >> 172.21.15.66:0/2089149678 conn(0x7efc080d1ce0 msgr2=0x7efc080d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:38.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 -- 172.21.15.66:0/2089149678 shutdown_connections 2024-09-10T22:11:38.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 -- 172.21.15.66:0/2089149678 wait complete. 2024-09-10T22:11:38.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 Processor -- start 2024-09-10T22:11:38.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 -- start start 2024-09-10T22:11:38.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc080b2b10 0x7efc08153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:38.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc081605c0 0x7efc081fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:38.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc08162c80 0x7efc0815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:38.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efc08168fb0 con 0x7efc08162c80 2024-09-10T22:11:38.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc080b1400 con 0x7efc081605c0 2024-09-10T22:11:38.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc0d400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7efc08169130 con 0x7efc080b2b10 2024-09-10T22:11:38.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc07200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc08162c80 0x7efc0815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc07200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc08162c80 0x7efc0815b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45154/0 (socket says 172.21.15.66:45154) 2024-09-10T22:11:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc07200640 1 -- 172.21.15.66:0/2774600326 learned_addr learned my addr 172.21.15.66:0/2774600326 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efbfde00640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc081605c0 0x7efc081fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc06800640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc080b2b10 0x7efc08153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc07200640 1 -- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc080b2b10 msgr2=0x7efc08153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc07200640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc080b2b10 0x7efc08153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc07200640 1 -- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc081605c0 msgr2=0x7efc081fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc07200640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc081605c0 0x7efc081fb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc07200640 1 -- 172.21.15.66:0/2774600326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc081fbe80 con 0x7efc08162c80 2024-09-10T22:11:38.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc06800640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc080b2b10 0x7efc08153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:38.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.276+0000 7efc07200640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc08162c80 0x7efc0815b4d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7efc0000c760 tx=0x7efc0000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:38.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.280+0000 7efbff400640 1 -- 172.21.15.66:0/2774600326 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00003db0 con 0x7efc08162c80 2024-09-10T22:11:38.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.280+0000 7efbff400640 1 -- 172.21.15.66:0/2774600326 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc0000ce10 con 0x7efc08162c80 2024-09-10T22:11:38.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.280+0000 7efbff400640 1 -- 172.21.15.66:0/2774600326 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc000052d0 con 0x7efc08162c80 2024-09-10T22:11:38.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.280+0000 7efc0d400640 1 -- 172.21.15.66:0/2774600326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc08156bc0 con 0x7efc08162c80 2024-09-10T22:11:38.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.280+0000 7efc0d400640 1 -- 172.21.15.66:0/2774600326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc080b59c0 con 0x7efc08162c80 2024-09-10T22:11:38.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.280+0000 7efc0d400640 1 -- 172.21.15.66:0/2774600326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc081facf0 con 0x7efc08162c80 2024-09-10T22:11:38.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.280+0000 7efbff400640 1 -- 172.21.15.66:0/2774600326 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7efc00005470 con 0x7efc08162c80 2024-09-10T22:11:38.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.280+0000 7efbff400640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efbe4073f60 0x7efbe4076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:38.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.280+0000 7efbff400640 1 -- 172.21.15.66:0/2774600326 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5441+0+0 (secure 0 0 0) 0x7efc00091da0 con 0x7efc08162c80 2024-09-10T22:11:38.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.280+0000 7efc06800640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efbe4073f60 0x7efbe4076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:38.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.284+0000 7efc06800640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efbe4073f60 0x7efbe4076420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7efbf80058d0 tx=0x7efbf8005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.284+0000 7efbff400640 1 -- 172.21.15.66:0/2774600326 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efc081fbe80 con 0x7efc08162c80 2024-09-10T22:11:38.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:38.472+0000 7efc0d400640 1 -- 172.21.15.66:0/2774600326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efc080b1620 con 0x7efc08162c80 2024-09-10T22:11:39.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.096+0000 7efbff400640 1 -- 172.21.15.66:0/2774600326 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7efc0005eee0 con 0x7efc08162c80 2024-09-10T22:11:39.101 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:11:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.100+0000 7efc0d400640 1 -- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efbe4073f60 msgr2=0x7efbe4076420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.100+0000 7efc0d400640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efbe4073f60 0x7efbe4076420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7efbf80058d0 tx=0x7efbf8005860 comp rx=0 tx=0).stop 2024-09-10T22:11:39.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.100+0000 7efc0d400640 1 -- 172.21.15.66:0/2774600326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc08162c80 msgr2=0x7efc0815b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:39.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.100+0000 7efc0d400640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc08162c80 0x7efc0815b4d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7efc0000c760 tx=0x7efc0000cc20 comp rx=0 tx=0).stop 2024-09-10T22:11:39.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.100+0000 7efc0d400640 1 -- 172.21.15.66:0/2774600326 shutdown_connections 2024-09-10T22:11:39.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.100+0000 7efc0d400640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7efbe4073f60 0x7efbe4076420 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.100+0000 7efc0d400640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc08162c80 0x7efc0815b4d0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.100+0000 7efc0d400640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc081605c0 0x7efc081fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.100+0000 7efc0d400640 1 --2- 172.21.15.66:0/2774600326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc080b2b10 0x7efc08153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.100+0000 7efc0d400640 1 -- 172.21.15.66:0/2774600326 >> 172.21.15.66:0/2774600326 conn(0x7efc080d1ce0 msgr2=0x7efc080aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:39.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.100+0000 7efc0d400640 1 -- 172.21.15.66:0/2774600326 shutdown_connections 2024-09-10T22:11:39.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.100+0000 7efc0d400640 1 -- 172.21.15.66:0/2774600326 wait complete. 2024-09-10T22:11:39.131 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T22:11:39.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 -- 172.21.15.66:0/1273281451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f108c0b0f60 msgr2=0x7f108c0d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:39.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 --2- 172.21.15.66:0/1273281451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f108c0b0f60 0x7f108c0d0310 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1074009900 tx=0x7f107401c730 comp rx=0 tx=0).stop 2024-09-10T22:11:39.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 -- 172.21.15.66:0/1273281451 shutdown_connections 2024-09-10T22:11:39.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 --2- 172.21.15.66:0/1273281451 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f108c0b32a0 0x7f108c0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 --2- 172.21.15.66:0/1273281451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f108c0b0f60 0x7f108c0d0310 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 -- 172.21.15.66:0/1273281451 >> 172.21.15.66:0/1273281451 conn(0x7f108c0d1d00 msgr2=0x7f108c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 -- 172.21.15.66:0/1273281451 shutdown_connections 2024-09-10T22:11:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 -- 172.21.15.66:0/1273281451 wait complete. 2024-09-10T22:11:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 Processor -- start 2024-09-10T22:11:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 -- start start 2024-09-10T22:11:39.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f108c0b0f60 0x7f108c152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:39.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f108c0b32a0 0x7f108c148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:39.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f108c15f460 0x7f108c06ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:39.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f108c0b0600 con 0x7f108c0b32a0 2024-09-10T22:11:39.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f108c0b0480 con 0x7f108c0b0f60 2024-09-10T22:11:39.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1091000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f108c168c60 con 0x7f108c15f460 2024-09-10T22:11:39.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1083e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f108c0b32a0 0x7f108c148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:39.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1083e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f108c0b32a0 0x7f108c148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45164/0 (socket says 172.21.15.66:45164) 2024-09-10T22:11:39.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1083e00640 1 -- 172.21.15.66:0/3637267314 learned_addr learned my addr 172.21.15.66:0/3637267314 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:39.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f1089e00640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f108c0b0f60 0x7f108c152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:39.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.276+0000 7f108a800640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f108c15f460 0x7f108c06ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:39.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1083e00640 1 -- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f108c15f460 msgr2=0x7f108c06ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:39.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1083e00640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f108c15f460 0x7f108c06ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1083e00640 1 -- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f108c0b0f60 msgr2=0x7f108c152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:39.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1083e00640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f108c0b0f60 0x7f108c152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1083e00640 1 -- 172.21.15.66:0/3637267314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108c149280 con 0x7f108c0b32a0 2024-09-10T22:11:39.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1083e00640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f108c0b32a0 0x7f108c148d40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f1074009850 tx=0x7f10740a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:39.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1082000640 1 -- 172.21.15.66:0/3637267314 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10740b2040 con 0x7f108c0b32a0 2024-09-10T22:11:39.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1082000640 1 -- 172.21.15.66:0/3637267314 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10740043b0 con 0x7f108c0b32a0 2024-09-10T22:11:39.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1082000640 1 -- 172.21.15.66:0/3637267314 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074005280 con 0x7f108c0b32a0 2024-09-10T22:11:39.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1091000640 1 -- 172.21.15.66:0/3637267314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108c152620 con 0x7f108c0b32a0 2024-09-10T22:11:39.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1091000640 1 -- 172.21.15.66:0/3637267314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f108c150a70 con 0x7f108c0b32a0 2024-09-10T22:11:39.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1091000640 1 -- 172.21.15.66:0/3637267314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108c0af390 con 0x7f108c0b32a0 2024-09-10T22:11:39.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1082000640 1 -- 172.21.15.66:0/3637267314 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f1074005420 con 0x7f108c0b32a0 2024-09-10T22:11:39.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1082000640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1068073f30 0x7f10680763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:39.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.280+0000 7f1082000640 1 -- 172.21.15.66:0/3637267314 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7f107412bab0 con 0x7f108c0b32a0 2024-09-10T22:11:39.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.284+0000 7f1089e00640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1068073f30 0x7f10680763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:39.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.284+0000 7f1089e00640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1068073f30 0x7f10680763f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f107c007fd0 tx=0x7f107c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:39.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.284+0000 7f1082000640 1 -- 172.21.15.66:0/3637267314 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f10740164d0 con 0x7f108c0b32a0 2024-09-10T22:11:39.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.472+0000 7f1091000640 1 -- 172.21.15.66:0/3637267314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f108c074aa0 con 0x7f108c0b32a0 2024-09-10T22:11:39.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1082000640 1 -- 172.21.15.66:0/3637267314 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v47) ==== 71+0+0 (secure 0 0 0) 0x7f10740f8b70 con 0x7f108c0b32a0 2024-09-10T22:11:39.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1091000640 1 -- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1068073f30 msgr2=0x7f10680763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:39.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1091000640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1068073f30 0x7f10680763f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f107c007fd0 tx=0x7f107c009040 comp rx=0 tx=0).stop 2024-09-10T22:11:39.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1091000640 1 -- 172.21.15.66:0/3637267314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f108c0b32a0 msgr2=0x7f108c148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:39.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1091000640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f108c0b32a0 0x7f108c148d40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f1074009850 tx=0x7f10740a5670 comp rx=0 tx=0).stop 2024-09-10T22:11:39.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1091000640 1 -- 172.21.15.66:0/3637267314 shutdown_connections 2024-09-10T22:11:39.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1091000640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1068073f30 0x7f10680763f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1091000640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f108c15f460 0x7f108c06ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1091000640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f108c0b32a0 0x7f108c148d40 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1091000640 1 --2- 172.21.15.66:0/3637267314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f108c0b0f60 0x7f108c152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1091000640 1 -- 172.21.15.66:0/3637267314 >> 172.21.15.66:0/3637267314 conn(0x7f108c0d1d00 msgr2=0x7f108c0d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1091000640 1 -- 172.21.15.66:0/3637267314 shutdown_connections 2024-09-10T22:11:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.476+0000 7f1091000640 1 -- 172.21.15.66:0/3637267314 wait complete. 2024-09-10T22:11:39.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-10T22:11:39.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 -- 172.21.15.66:0/3891257984 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac2015f460 msgr2=0x7fac200cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:39.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 --2- 172.21.15.66:0/3891257984 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac2015f460 0x7fac200cfc00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac10009840 tx=0x7fac1001c6d0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 -- 172.21.15.66:0/3891257984 shutdown_connections 2024-09-10T22:11:39.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 --2- 172.21.15.66:0/3891257984 >> v2:172.21.15.94:3300/0 conn(0x7fac200b0f60 0x7fac200d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 --2- 172.21.15.66:0/3891257984 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac2015f460 0x7fac200cfc00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 -- 172.21.15.66:0/3891257984 >> 172.21.15.66:0/3891257984 conn(0x7fac200d1d00 msgr2=0x7fac200d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:39.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 -- 172.21.15.66:0/3891257984 shutdown_connections 2024-09-10T22:11:39.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 -- 172.21.15.66:0/3891257984 wait complete. 2024-09-10T22:11:39.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 Processor -- start 2024-09-10T22:11:39.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 -- start start 2024-09-10T22:11:39.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac200b0f60 0x7fac200b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac200b32a0 0x7fac2015cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac2015f460 0x7fac2015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fac20168ff0 con 0x7fac200b32a0 2024-09-10T22:11:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac20168e70 con 0x7fac200b0f60 2024-09-10T22:11:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac25000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fac20169170 con 0x7fac2015f460 2024-09-10T22:11:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac17e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac200b32a0 0x7fac2015cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac17e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac200b32a0 0x7fac2015cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45168/0 (socket says 172.21.15.66:45168) 2024-09-10T22:11:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac17e00640 1 -- 172.21.15.66:0/977272997 learned_addr learned my addr 172.21.15.66:0/977272997 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac1e800640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac2015f460 0x7fac2015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac1de00640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac200b0f60 0x7fac200b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:39.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac17e00640 1 -- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac2015f460 msgr2=0x7fac2015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:39.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac17e00640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac2015f460 0x7fac2015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac17e00640 1 -- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac200b0f60 msgr2=0x7fac200b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:39.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac17e00640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac200b0f60 0x7fac200b65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:39.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.648+0000 7fac17e00640 1 -- 172.21.15.66:0/977272997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac201f8b60 con 0x7fac200b32a0 2024-09-10T22:11:39.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.652+0000 7fac17e00640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac200b32a0 0x7fac2015cfe0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fac0800ea20 tx=0x7fac0800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:39.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.652+0000 7fac16000640 1 -- 172.21.15.66:0/977272997 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0800cd20 con 0x7fac200b32a0 2024-09-10T22:11:39.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.652+0000 7fac16000640 1 -- 172.21.15.66:0/977272997 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac08010430 con 0x7fac200b32a0 2024-09-10T22:11:39.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.652+0000 7fac25000640 1 -- 172.21.15.66:0/977272997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac200b4bd0 con 0x7fac200b32a0 2024-09-10T22:11:39.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.652+0000 7fac16000640 1 -- 172.21.15.66:0/977272997 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08010860 con 0x7fac200b32a0 2024-09-10T22:11:39.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.652+0000 7fac25000640 1 -- 172.21.15.66:0/977272997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2015cac0 con 0x7fac200b32a0 2024-09-10T22:11:39.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.652+0000 7fac25000640 1 -- 172.21.15.66:0/977272997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac200b14b0 con 0x7fac200b32a0 2024-09-10T22:11:39.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.652+0000 7fac16000640 1 -- 172.21.15.66:0/977272997 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fac08010a00 con 0x7fac200b32a0 2024-09-10T22:11:39.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.652+0000 7fac16000640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fabfc073f30 0x7fabfc0763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:39.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.652+0000 7fac16000640 1 -- 172.21.15.66:0/977272997 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7fac08092ae0 con 0x7fac200b32a0 2024-09-10T22:11:39.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.656+0000 7fac1de00640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fabfc073f30 0x7fabfc0763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:39.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.656+0000 7fac1de00640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fabfc073f30 0x7fabfc0763f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fac100af000 tx=0x7fac1001af10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:39.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.656+0000 7fac16000640 1 -- 172.21.15.66:0/977272997 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fac0805f5d0 con 0x7fac200b32a0 2024-09-10T22:11:39.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:39.896+0000 7fac25000640 1 -- 172.21.15.66:0/977272997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7fac2015c6b0 con 0x7fac200b32a0 2024-09-10T22:11:40.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac16000640 1 -- 172.21.15.66:0/977272997 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v505) ==== 178+0+0 (secure 0 0 0) 0x7fac08064000 con 0x7fac200b32a0 2024-09-10T22:11:40.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac25000640 1 -- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fabfc073f30 msgr2=0x7fabfc0763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac25000640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fabfc073f30 0x7fabfc0763f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fac100af000 tx=0x7fac1001af10 comp rx=0 tx=0).stop 2024-09-10T22:11:40.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac25000640 1 -- 172.21.15.66:0/977272997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac200b32a0 msgr2=0x7fac2015cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac25000640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac200b32a0 0x7fac2015cfe0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fac0800ea20 tx=0x7fac0800eee0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac25000640 1 -- 172.21.15.66:0/977272997 shutdown_connections 2024-09-10T22:11:40.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac25000640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fabfc073f30 0x7fabfc0763f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac25000640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac2015f460 0x7fac2015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac25000640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac200b32a0 0x7fac2015cfe0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac25000640 1 --2- 172.21.15.66:0/977272997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac200b0f60 0x7fac200b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac25000640 1 -- 172.21.15.66:0/977272997 >> 172.21.15.66:0/977272997 conn(0x7fac200d1d00 msgr2=0x7fac2006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:40.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac25000640 1 -- 172.21.15.66:0/977272997 shutdown_connections 2024-09-10T22:11:40.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.100+0000 7fac25000640 1 -- 172.21.15.66:0/977272997 wait complete. 2024-09-10T22:11:40.125 INFO:tasks.cephfs_test_runner:Test that the command "ceph fs clone status" prints progress stats ... ok 2024-09-10T22:11:40.126 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress bars, that printed for 3 ongoing clone jobs, are 2024-09-10T22:11:40.126 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-10T22:11:40.126 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:11:40.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.268+0000 7f0d6fc00640 1 -- 172.21.15.66:0/4086136619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d6815dfe0 msgr2=0x7f0d680c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.268+0000 7f0d6fc00640 1 --2- 172.21.15.66:0/4086136619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d6815dfe0 0x7f0d680c79c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f0d680c7560 tx=0x7f0d5c01c8c0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.268+0000 7f0d6fc00640 1 -- 172.21.15.66:0/4086136619 shutdown_connections 2024-09-10T22:11:40.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.268+0000 7f0d6fc00640 1 --2- 172.21.15.66:0/4086136619 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d680d4990 0x7f0d680d2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.268+0000 7f0d6fc00640 1 --2- 172.21.15.66:0/4086136619 >> v2:172.21.15.94:3300/0 conn(0x7f0d680c7f00 0x7f0d680d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.268+0000 7f0d6fc00640 1 --2- 172.21.15.66:0/4086136619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d6815dfe0 0x7f0d680c79c0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.268+0000 7f0d6fc00640 1 -- 172.21.15.66:0/4086136619 >> 172.21.15.66:0/4086136619 conn(0x7f0d680d3e70 msgr2=0x7f0d680d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:40.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.268+0000 7f0d6fc00640 1 -- 172.21.15.66:0/4086136619 shutdown_connections 2024-09-10T22:11:40.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.268+0000 7f0d6fc00640 1 -- 172.21.15.66:0/4086136619 wait complete. 2024-09-10T22:11:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6fc00640 1 Processor -- start 2024-09-10T22:11:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6fc00640 1 -- start start 2024-09-10T22:11:40.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6fc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d680c7f00 0x7f0d6815e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6fc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d680d4990 0x7f0d68154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6fc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d6815dfe0 0x7f0d6806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6fc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d68161bc0 con 0x7f0d680c7f00 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6fc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d68161a40 con 0x7f0d680d4990 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6fc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0d68161d40 con 0x7f0d6815dfe0 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d680c7f00 0x7f0d6815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d680c7f00 0x7f0d6815e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45176/0 (socket says 172.21.15.66:45176) 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6d200640 1 -- 172.21.15.66:0/474078902 learned_addr learned my addr 172.21.15.66:0/474078902 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6dc00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d6815dfe0 0x7f0d6806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d67e00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d680d4990 0x7f0d68154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6d200640 1 -- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d6815dfe0 msgr2=0x7f0d6806dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6d200640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d6815dfe0 0x7f0d6806dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6d200640 1 -- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d680d4990 msgr2=0x7f0d68154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6d200640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d680d4990 0x7f0d68154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6d200640 1 -- 172.21.15.66:0/474078902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d68154670 con 0x7f0d680c7f00 2024-09-10T22:11:40.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6dc00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d6815dfe0 0x7f0d6806dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:40.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d67e00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d680d4990 0x7f0d68154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:40.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6d200640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d680c7f00 0x7f0d6815e540 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c01af70 tx=0x7f0d5c0a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d64c00640 1 -- 172.21.15.66:0/474078902 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c0b2040 con 0x7f0d680c7f00 2024-09-10T22:11:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d64c00640 1 -- 172.21.15.66:0/474078902 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d5c004430 con 0x7f0d680c7f00 2024-09-10T22:11:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6fc00640 1 -- 172.21.15.66:0/474078902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d68160b50 con 0x7f0d680c7f00 2024-09-10T22:11:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d64c00640 1 -- 172.21.15.66:0/474078902 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c003860 con 0x7f0d680c7f00 2024-09-10T22:11:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.272+0000 7f0d6fc00640 1 -- 172.21.15.66:0/474078902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d68161110 con 0x7f0d680c7f00 2024-09-10T22:11:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.276+0000 7f0d6fc00640 1 -- 172.21.15.66:0/474078902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d680c79c0 con 0x7f0d680c7f00 2024-09-10T22:11:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.276+0000 7f0d64c00640 1 -- 172.21.15.66:0/474078902 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f0d5c003a00 con 0x7f0d680c7f00 2024-09-10T22:11:40.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.276+0000 7f0d64c00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d48073f60 0x7f0d48076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:40.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.276+0000 7f0d64c00640 1 -- 172.21.15.66:0/474078902 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7f0d5c12bed0 con 0x7f0d680c7f00 2024-09-10T22:11:40.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.276+0000 7f0d67e00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d48073f60 0x7f0d48076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:40.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.276+0000 7f0d67e00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d48073f60 0x7f0d48076420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0d54005fd0 tx=0x7f0d54005f20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:40.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.280+0000 7f0d64c00640 1 -- 172.21.15.66:0/474078902 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0d5c0164b0 con 0x7f0d680c7f00 2024-09-10T22:11:40.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 -- 172.21.15.66:0/474078902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d680d3c50 con 0x7f0d680c7f00 2024-09-10T22:11:40.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d64c00640 1 -- 172.21.15.66:0/474078902 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+13095 (secure 0 0 0) 0x7f0d5c0f9010 con 0x7f0d680c7f00 2024-09-10T22:11:40.459 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:40.460 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":104,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:11:39.093998+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6837/1920075377":"2024-09-11T22:11:36.084776+0000","172.21.15.94:6836/1920075377":"2024-09-11T22:11:36.084776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:11:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 -- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d48073f60 msgr2=0x7f0d48076420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d48073f60 0x7f0d48076420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0d54005fd0 tx=0x7f0d54005f20 comp rx=0 tx=0).stop 2024-09-10T22:11:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 -- 172.21.15.66:0/474078902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d680c7f00 msgr2=0x7f0d6815e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d680c7f00 0x7f0d6815e540 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c01af70 tx=0x7f0d5c0a5630 comp rx=0 tx=0).stop 2024-09-10T22:11:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 -- 172.21.15.66:0/474078902 shutdown_connections 2024-09-10T22:11:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d48073f60 0x7f0d48076420 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d6815dfe0 0x7f0d6806dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d680d4990 0x7f0d68154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 --2- 172.21.15.66:0/474078902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d680c7f00 0x7f0d6815e540 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 -- 172.21.15.66:0/474078902 >> 172.21.15.66:0/474078902 conn(0x7f0d680d3e70 msgr2=0x7f0d680c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 -- 172.21.15.66:0/474078902 shutdown_connections 2024-09-10T22:11:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.456+0000 7f0d6fc00640 1 -- 172.21.15.66:0/474078902 wait complete. 2024-09-10T22:11:40.482 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:00:22.340318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:11:40.483 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:11:40.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.628+0000 7f847d600640 1 -- 172.21.15.66:0/4100538520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815e460 msgr2=0x7f84780ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.628+0000 7f847d600640 1 --2- 172.21.15.66:0/4100538520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815e460 0x7f84780ccab0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f84780ce020 tx=0x7f846001c770 comp rx=0 tx=0).stop 2024-09-10T22:11:40.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.628+0000 7f847d600640 1 -- 172.21.15.66:0/4100538520 shutdown_connections 2024-09-10T22:11:40.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.628+0000 7f847d600640 1 --2- 172.21.15.66:0/4100538520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815e460 0x7f84780ccab0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.628+0000 7f847d600640 1 -- 172.21.15.66:0/4100538520 >> 172.21.15.66:0/4100538520 conn(0x7f84780d1c50 msgr2=0x7f84780d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:40.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.628+0000 7f847d600640 1 -- 172.21.15.66:0/4100538520 shutdown_connections 2024-09-10T22:11:40.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.628+0000 7f847d600640 1 -- 172.21.15.66:0/4100538520 wait complete. 2024-09-10T22:11:40.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f847d600640 1 Processor -- start 2024-09-10T22:11:40.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f847d600640 1 -- start start 2024-09-10T22:11:40.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f847d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f847806ebc0 0x7f84780c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:40.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f847d600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84780ce660 0x7f84780b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:40.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f847d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815e460 0x7f84781531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:40.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f847d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f847816ca00 con 0x7f847815e460 2024-09-10T22:11:40.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f847d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f847806f930 con 0x7f847806ebc0 2024-09-10T22:11:40.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f847d600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f847816cb80 con 0x7f84780ce660 2024-09-10T22:11:40.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f8477400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815e460 0x7f84781531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:40.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f8477400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815e460 0x7f84781531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45188/0 (socket says 172.21.15.66:45188) 2024-09-10T22:11:40.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f8477400640 1 -- 172.21.15.66:0/3291882244 learned_addr learned my addr 172.21.15.66:0/3291882244 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f846fe00640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84780ce660 0x7f84780b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f8476a00640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f847806ebc0 0x7f84780c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f8477400640 1 -- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84780ce660 msgr2=0x7f84780b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f8477400640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84780ce660 0x7f84780b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f8477400640 1 -- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f847806ebc0 msgr2=0x7f84780c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f8477400640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f847806ebc0 0x7f84780c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f8477400640 1 -- 172.21.15.66:0/3291882244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84780c9260 con 0x7f847815e460 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f8477400640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815e460 0x7f84781531f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f847000cbc0 tx=0x7f847000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f846ea00640 1 -- 172.21.15.66:0/3291882244 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470016020 con 0x7f847815e460 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f846ea00640 1 -- 172.21.15.66:0/3291882244 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f847000aae0 con 0x7f847815e460 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f846ea00640 1 -- 172.21.15.66:0/3291882244 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470012420 con 0x7f847815e460 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f847d600640 1 -- 172.21.15.66:0/3291882244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84780b1bc0 con 0x7f847815e460 2024-09-10T22:11:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f847d600640 1 -- 172.21.15.66:0/3291882244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84780b2460 con 0x7f847815e460 2024-09-10T22:11:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.632+0000 7f847d600640 1 -- 172.21.15.66:0/3291882244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84780ccab0 con 0x7f847815e460 2024-09-10T22:11:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.636+0000 7f846ea00640 1 -- 172.21.15.66:0/3291882244 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f847001a3b0 con 0x7f847815e460 2024-09-10T22:11:40.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.636+0000 7f846ea00640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8454073f30 0x7f84540763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:40.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.636+0000 7f846ea00640 1 -- 172.21.15.66:0/3291882244 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7f8470092220 con 0x7f847815e460 2024-09-10T22:11:40.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.636+0000 7f8476a00640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8454073f30 0x7f84540763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:40.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.636+0000 7f8476a00640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8454073f30 0x7f84540763f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8468007fd0 tx=0x7f8468009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:40.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.640+0000 7f846ea00640 1 -- 172.21.15.66:0/3291882244 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8470020030 con 0x7f847815e460 2024-09-10T22:11:40.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.816+0000 7f847d600640 1 -- 172.21.15.66:0/3291882244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f847806fd20 con 0x7f847815e460 2024-09-10T22:11:40.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.816+0000 7f846ea00640 1 -- 172.21.15.66:0/3291882244 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7f847005ed10 con 0x7f847815e460 2024-09-10T22:11:40.821 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-09-10T22:11:40.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.820+0000 7f847d600640 1 -- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8454073f30 msgr2=0x7f84540763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.820+0000 7f847d600640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8454073f30 0x7f84540763f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8468007fd0 tx=0x7f8468009040 comp rx=0 tx=0).stop 2024-09-10T22:11:40.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.820+0000 7f847d600640 1 -- 172.21.15.66:0/3291882244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815e460 msgr2=0x7f84781531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.820+0000 7f847d600640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815e460 0x7f84781531f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f847000cbc0 tx=0x7f847000ccb0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.820+0000 7f847d600640 1 -- 172.21.15.66:0/3291882244 shutdown_connections 2024-09-10T22:11:40.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.820+0000 7f847d600640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8454073f30 0x7f84540763f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.820+0000 7f847d600640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815e460 0x7f84781531f0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.820+0000 7f847d600640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84780ce660 0x7f84780b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.820+0000 7f847d600640 1 --2- 172.21.15.66:0/3291882244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f847806ebc0 0x7f84780c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.820+0000 7f847d600640 1 -- 172.21.15.66:0/3291882244 >> 172.21.15.66:0/3291882244 conn(0x7f84780d1c50 msgr2=0x7f847806f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:40.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.820+0000 7f847d600640 1 -- 172.21.15.66:0/3291882244 shutdown_connections 2024-09-10T22:11:40.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.820+0000 7f847d600640 1 -- 172.21.15.66:0/3291882244 wait complete. 2024-09-10T22:11:40.842 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-10T22:11:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.984+0000 7fe2a3800640 1 -- 172.21.15.66:0/1992117865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c15a310 msgr2=0x7fe29c0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.984+0000 7fe2a3800640 1 --2- 172.21.15.66:0/1992117865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c15a310 0x7fe29c0c48b0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00aa10 tx=0x7fe28c01dba0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.984+0000 7fe2a3800640 1 -- 172.21.15.66:0/1992117865 shutdown_connections 2024-09-10T22:11:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.984+0000 7fe2a3800640 1 --2- 172.21.15.66:0/1992117865 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c0aeec0 0x7fe29c0c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.984+0000 7fe2a3800640 1 --2- 172.21.15.66:0/1992117865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c15a310 0x7fe29c0c48b0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.984+0000 7fe2a3800640 1 -- 172.21.15.66:0/1992117865 >> 172.21.15.66:0/1992117865 conn(0x7fe29c0cfa30 msgr2=0x7fe29c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.984+0000 7fe2a3800640 1 -- 172.21.15.66:0/1992117865 shutdown_connections 2024-09-10T22:11:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.984+0000 7fe2a3800640 1 -- 172.21.15.66:0/1992117865 wait complete. 2024-09-10T22:11:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a3800640 1 Processor -- start 2024-09-10T22:11:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a3800640 1 -- start start 2024-09-10T22:11:40.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a3800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c0aeec0 0x7fe29c148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:40.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a3800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c15a310 0x7fe29c147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:40.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a3800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe29c0b3630 0x7fe29c0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:40.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a3800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c167300 con 0x7fe29c0aeec0 2024-09-10T22:11:40.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a3800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c15ec90 con 0x7fe29c0b3630 2024-09-10T22:11:40.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a3800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe29c167480 con 0x7fe29c15a310 2024-09-10T22:11:40.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c0aeec0 0x7fe29c148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:40.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c0aeec0 0x7fe29c148420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45210/0 (socket says 172.21.15.66:45210) 2024-09-10T22:11:40.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a0e00640 1 -- 172.21.15.66:0/4224080463 learned_addr learned my addr 172.21.15.66:0/4224080463 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:40.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a1800640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe29c0b3630 0x7fe29c0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:40.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe293e00640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c15a310 0x7fe29c147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:40.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a0e00640 1 -- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c15a310 msgr2=0x7fe29c147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a0e00640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c15a310 0x7fe29c147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a0e00640 1 -- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe29c0b3630 msgr2=0x7fe29c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:40.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a0e00640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe29c0b3630 0x7fe29c0ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:40.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a0e00640 1 -- 172.21.15.66:0/4224080463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c073600 con 0x7fe29c0aeec0 2024-09-10T22:11:40.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe293e00640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c15a310 0x7fe29c147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a0e00640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c0aeec0 0x7fe29c148420 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0b1040 tx=0x7fe28c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe299600640 1 -- 172.21.15.66:0/4224080463 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0b3070 con 0x7fe29c0aeec0 2024-09-10T22:11:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe299600640 1 -- 172.21.15.66:0/4224080463 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe28c005530 con 0x7fe29c0aeec0 2024-09-10T22:11:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe299600640 1 -- 172.21.15.66:0/4224080463 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c004930 con 0x7fe29c0aeec0 2024-09-10T22:11:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.988+0000 7fe2a3800640 1 -- 172.21.15.66:0/4224080463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29c0738e0 con 0x7fe29c0aeec0 2024-09-10T22:11:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.992+0000 7fe2a3800640 1 -- 172.21.15.66:0/4224080463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29c073ea0 con 0x7fe29c0aeec0 2024-09-10T22:11:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.992+0000 7fe2a3800640 1 -- 172.21.15.66:0/4224080463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29c15df60 con 0x7fe29c0aeec0 2024-09-10T22:11:40.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.992+0000 7fe299600640 1 -- 172.21.15.66:0/4224080463 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fe28c00b040 con 0x7fe29c0aeec0 2024-09-10T22:11:40.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.992+0000 7fe299600640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe278073f30 0x7fe2780763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:40.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.992+0000 7fe299600640 1 -- 172.21.15.66:0/4224080463 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7fe28c12c270 con 0x7fe29c0aeec0 2024-09-10T22:11:40.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.992+0000 7fe293e00640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe278073f30 0x7fe2780763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:40.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.996+0000 7fe293e00640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe278073f30 0x7fe2780763f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0c9430 tx=0x7fe284009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:41.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:40.996+0000 7fe299600640 1 -- 172.21.15.66:0/4224080463 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe28c017820 con 0x7fe29c0aeec0 2024-09-10T22:11:41.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:41.232+0000 7fe2a3800640 1 -- 172.21.15.66:0/4224080463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7fe29c070420 con 0x7fe29c0aeec0 2024-09-10T22:11:42.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe299600640 1 -- 172.21.15.66:0/4224080463 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v507) ==== 188+0+0 (secure 0 0 0) 0x7fe28c0f9430 con 0x7fe29c0aeec0 2024-09-10T22:11:42.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe2a3800640 1 -- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe278073f30 msgr2=0x7fe2780763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:42.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe2a3800640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe278073f30 0x7fe2780763f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0c9430 tx=0x7fe284009040 comp rx=0 tx=0).stop 2024-09-10T22:11:42.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe2a3800640 1 -- 172.21.15.66:0/4224080463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c0aeec0 msgr2=0x7fe29c148420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:42.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe2a3800640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c0aeec0 0x7fe29c148420 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0b1040 tx=0x7fe28c005760 comp rx=0 tx=0).stop 2024-09-10T22:11:42.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe2a3800640 1 -- 172.21.15.66:0/4224080463 shutdown_connections 2024-09-10T22:11:42.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe2a3800640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe278073f30 0x7fe2780763f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe2a3800640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe29c0b3630 0x7fe29c0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe2a3800640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe29c15a310 0x7fe29c147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe2a3800640 1 --2- 172.21.15.66:0/4224080463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c0aeec0 0x7fe29c148420 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe2a3800640 1 -- 172.21.15.66:0/4224080463 >> 172.21.15.66:0/4224080463 conn(0x7fe29c0cfa30 msgr2=0x7fe29c1593e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:42.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe2a3800640 1 -- 172.21.15.66:0/4224080463 shutdown_connections 2024-09-10T22:11:42.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.108+0000 7fe2a3800640 1 -- 172.21.15.66:0/4224080463 wait complete. 2024-09-10T22:11:42.139 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T22:11:42.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 -- 172.21.15.66:0/3143471897 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb73c0d0260 msgr2=0x7fb73c0ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:42.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 --2- 172.21.15.66:0/3143471897 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb73c0d0260 0x7fb73c0ae960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb724009870 tx=0x7fb72401c860 comp rx=0 tx=0).stop 2024-09-10T22:11:42.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 -- 172.21.15.66:0/3143471897 shutdown_connections 2024-09-10T22:11:42.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 --2- 172.21.15.66:0/3143471897 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb73c0d0260 0x7fb73c0ae960 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 -- 172.21.15.66:0/3143471897 >> 172.21.15.66:0/3143471897 conn(0x7fb73c0d1c50 msgr2=0x7fb73c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:42.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 -- 172.21.15.66:0/3143471897 shutdown_connections 2024-09-10T22:11:42.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 -- 172.21.15.66:0/3143471897 wait complete. 2024-09-10T22:11:42.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 Processor -- start 2024-09-10T22:11:42.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 -- start start 2024-09-10T22:11:42.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb73c0b2bf0 0x7fb73c14f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:42.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb73c0d0260 0x7fb73c1f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:42.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c15f340 0x7fb73c153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:42.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c16a310 con 0x7fb73c15f340 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c16a190 con 0x7fb73c0b2bf0 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb741600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb73c16a490 con 0x7fb73c0d0260 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb73b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c15f340 0x7fb73c153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb73b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c15f340 0x7fb73c153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45212/0 (socket says 172.21.15.66:45212) 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb73b400640 1 -- 172.21.15.66:0/2693975390 learned_addr learned my addr 172.21.15.66:0/2693975390 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.288+0000 7fb73aa00640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb73c0b2bf0 0x7fb73c14f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb732000640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb73c0d0260 0x7fb73c1f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb73b400640 1 -- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb73c0d0260 msgr2=0x7fb73c1f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb73b400640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb73c0d0260 0x7fb73c1f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb73b400640 1 -- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb73c0b2bf0 msgr2=0x7fb73c14f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb73b400640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb73c0b2bf0 0x7fb73c14f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb73b400640 1 -- 172.21.15.66:0/2693975390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c0ce2f0 con 0x7fb73c15f340 2024-09-10T22:11:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb73b400640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c15f340 0x7fb73c153590 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fb73400cb70 tx=0x7fb73400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:42.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb733400640 1 -- 172.21.15.66:0/2693975390 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734016020 con 0x7fb73c15f340 2024-09-10T22:11:42.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb733400640 1 -- 172.21.15.66:0/2693975390 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb73400ab60 con 0x7fb73c15f340 2024-09-10T22:11:42.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb733400640 1 -- 172.21.15.66:0/2693975390 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734012420 con 0x7fb73c15f340 2024-09-10T22:11:42.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb741600640 1 -- 172.21.15.66:0/2693975390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73c0ce5d0 con 0x7fb73c15f340 2024-09-10T22:11:42.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb741600640 1 -- 172.21.15.66:0/2693975390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73c0ceb90 con 0x7fb73c15f340 2024-09-10T22:11:42.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb741600640 1 -- 172.21.15.66:0/2693975390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb73c158f10 con 0x7fb73c15f340 2024-09-10T22:11:42.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.292+0000 7fb733400640 1 -- 172.21.15.66:0/2693975390 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fb7340125c0 con 0x7fb73c15f340 2024-09-10T22:11:42.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.296+0000 7fb733400640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb718073f30 0x7fb7180763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:42.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.296+0000 7fb733400640 1 -- 172.21.15.66:0/2693975390 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7fb734092570 con 0x7fb73c15f340 2024-09-10T22:11:42.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.296+0000 7fb73aa00640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb718073f30 0x7fb7180763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:42.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.296+0000 7fb73aa00640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb718073f30 0x7fb7180763f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb72c007fd0 tx=0x7fb72c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:42.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.300+0000 7fb733400640 1 -- 172.21.15.66:0/2693975390 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb734020020 con 0x7fb73c15f340 2024-09-10T22:11:42.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.476+0000 7fb741600640 1 -- 172.21.15.66:0/2693975390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fb73c158c30 con 0x7fb73c15f340 2024-09-10T22:11:42.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.476+0000 7fb733400640 1 -- 172.21.15.66:0/2693975390 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v47) ==== 59+0+1998 (secure 0 0 0) 0x7fb73405f060 con 0x7fb73c15f340 2024-09-10T22:11:42.481 INFO:teuthology.orchestra.run.smithi066.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T22:11:42.481 INFO:teuthology.orchestra.run.smithi066.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T22:11:42.481 INFO:teuthology.orchestra.run.smithi066.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T22:11:42.481 INFO:teuthology.orchestra.run.smithi066.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T22:11:42.481 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4673.907365 2024-09-10T22:11:42.481 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4537.831230 2024-09-10T22:11:42.482 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 5706.341617 2024-09-10T22:11:42.482 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4484.013907 2024-09-10T22:11:42.482 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3862.285837 2024-09-10T22:11:42.482 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3776.439661 2024-09-10T22:11:42.482 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3820.270368 2024-09-10T22:11:42.482 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3863.602758 2024-09-10T22:11:42.482 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds 20/20 2024-09-10T22:11:42.482 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T22:11:42.482 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_ms 2/2 2024-09-10T22:11:42.482 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_frag true 2024-09-10T22:11:42.482 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_scatterstat true 2024-09-10T22:11:42.482 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T22:11:42.483 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_verify_scatter true 2024-09-10T22:11:42.483 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T22:11:42.483 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T22:11:42.483 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T22:11:42.483 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced client_mount_timeout 600 2024-09-10T22:11:42.483 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_client 20/20 2024-09-10T22:11:42.483 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_ms 1/1 2024-09-10T22:11:42.483 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T22:11:42.483 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T22:11:42.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.480+0000 7fb741600640 1 -- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb718073f30 msgr2=0x7fb7180763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:42.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.480+0000 7fb741600640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb718073f30 0x7fb7180763f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb72c007fd0 tx=0x7fb72c009040 comp rx=0 tx=0).stop 2024-09-10T22:11:42.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.480+0000 7fb741600640 1 -- 172.21.15.66:0/2693975390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c15f340 msgr2=0x7fb73c153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:42.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.480+0000 7fb741600640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c15f340 0x7fb73c153590 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fb73400cb70 tx=0x7fb73400cc60 comp rx=0 tx=0).stop 2024-09-10T22:11:42.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.480+0000 7fb741600640 1 -- 172.21.15.66:0/2693975390 shutdown_connections 2024-09-10T22:11:42.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.480+0000 7fb741600640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb718073f30 0x7fb7180763f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.480+0000 7fb741600640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c15f340 0x7fb73c153590 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.480+0000 7fb741600640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb73c0d0260 0x7fb73c1f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.480+0000 7fb741600640 1 --2- 172.21.15.66:0/2693975390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb73c0b2bf0 0x7fb73c14f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.480+0000 7fb741600640 1 -- 172.21.15.66:0/2693975390 >> 172.21.15.66:0/2693975390 conn(0x7fb73c0d1c50 msgr2=0x7fb73c0d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:42.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.480+0000 7fb741600640 1 -- 172.21.15.66:0/2693975390 shutdown_connections 2024-09-10T22:11:42.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.480+0000 7fb741600640 1 -- 172.21.15.66:0/2693975390 wait complete. 2024-09-10T22:11:42.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:11:42.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.648+0000 7f0bc3a00640 1 -- 172.21.15.66:0/3849592255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba4003c20 msgr2=0x7f0ba4004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:42.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.648+0000 7f0bc3a00640 1 --2- 172.21.15.66:0/3849592255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba4003c20 0x7f0ba4004020 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0001860 tx=0x7f0bb001d310 comp rx=0 tx=0).stop 2024-09-10T22:11:42.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.648+0000 7f0bc1a00640 1 -- 172.21.15.66:0/3849592255 reap_dead start 2024-09-10T22:11:42.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.648+0000 7f0bc3a00640 1 -- 172.21.15.66:0/3849592255 shutdown_connections 2024-09-10T22:11:42.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.648+0000 7f0bc3a00640 1 -- 172.21.15.66:0/3849592255 >> 172.21.15.66:0/3849592255 conn(0x7f0bbc0cfa30 msgr2=0x7f0bbc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:42.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.648+0000 7f0bc3a00640 1 -- 172.21.15.66:0/3849592255 shutdown_connections 2024-09-10T22:11:42.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.648+0000 7f0bc3a00640 1 -- 172.21.15.66:0/3849592255 wait complete. 2024-09-10T22:11:42.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc3a00640 1 Processor -- start 2024-09-10T22:11:42.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc3a00640 1 -- start start 2024-09-10T22:11:42.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc3a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ba4003c20 0x7f0bbc1fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:42.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc3a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bbc1417b0 0x7f0bbc13eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:42.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc3a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bbc13f0a0 0x7f0bbc160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:42.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc3a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc166fe0 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc3a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc166e60 con 0x7f0bbc1417b0 2024-09-10T22:11:42.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc3a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0bbc167160 con 0x7f0ba4003c20 2024-09-10T22:11:42.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bbc13f0a0 0x7f0bbc160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:42.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bbc13f0a0 0x7f0bbc160f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55200/0 (socket says 172.21.15.66:55200) 2024-09-10T22:11:42.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc1a00640 1 -- 172.21.15.66:0/1915477140 learned_addr learned my addr 172.21.15.66:0/1915477140 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:42.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc1000640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ba4003c20 0x7f0bbc1fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:42.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bbbe00640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bbc1417b0 0x7f0bbc13eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:42.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc1a00640 1 -- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ba4003c20 msgr2=0x7f0bbc1fbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:42.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc1a00640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ba4003c20 0x7f0bbc1fbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc1a00640 1 -- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bbc1417b0 msgr2=0x7f0bbc13eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:42.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc1a00640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bbc1417b0 0x7f0bbc13eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc1a00640 1 -- 172.21.15.66:0/1915477140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc0b5790 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bbbe00640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bbc1417b0 0x7f0bbc13eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:42.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc1000640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ba4003c20 0x7f0bbc1fbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:42.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc1a00640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bbc13f0a0 0x7f0bbc160f70 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc151560 tx=0x7f0bb400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:42.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bb8c00640 1 -- 172.21.15.66:0/1915477140 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4004b30 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bb8c00640 1 -- 172.21.15.66:0/1915477140 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bb4004cd0 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bb8c00640 1 -- 172.21.15.66:0/1915477140 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4003be0 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc3a00640 1 -- 172.21.15.66:0/1915477140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc141cb0 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.652+0000 7f0bc3a00640 1 -- 172.21.15.66:0/1915477140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc142270 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.656+0000 7f0bc3a00640 1 -- 172.21.15.66:0/1915477140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bbc0cd0c0 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.656+0000 7f0bb8c00640 1 -- 172.21.15.66:0/1915477140 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f0bb4021060 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.656+0000 7f0bb8c00640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b9c073f60 0x7f0b9c076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:42.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.656+0000 7f0bb8c00640 1 -- 172.21.15.66:0/1915477140 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7f0bb40921a0 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.656+0000 7f0bc1000640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b9c073f60 0x7f0b9c076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:42.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.656+0000 7f0bc1000640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b9c073f60 0x7f0b9c076420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00a9000 tx=0x7f0bb00196f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:42.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.660+0000 7f0bb8c00640 1 -- 172.21.15.66:0/1915477140 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0bb405f380 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.832+0000 7f0bc3a00640 1 -- 172.21.15.66:0/1915477140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0bbc076650 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.836+0000 7f0bb8c00640 1 -- 172.21.15.66:0/1915477140 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f0bb405ec80 con 0x7f0bbc13f0a0 2024-09-10T22:11:42.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.840+0000 7f0bc3a00640 1 -- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b9c073f60 msgr2=0x7f0b9c076420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:42.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.840+0000 7f0bc3a00640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b9c073f60 0x7f0b9c076420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00a9000 tx=0x7f0bb00196f0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.840+0000 7f0bc3a00640 1 -- 172.21.15.66:0/1915477140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bbc13f0a0 msgr2=0x7f0bbc160f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:42.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.840+0000 7f0bc3a00640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bbc13f0a0 0x7f0bbc160f70 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc151560 tx=0x7f0bb400f3c0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.840+0000 7f0bc3a00640 1 -- 172.21.15.66:0/1915477140 shutdown_connections 2024-09-10T22:11:42.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.840+0000 7f0bc3a00640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b9c073f60 0x7f0b9c076420 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.840+0000 7f0bc3a00640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bbc13f0a0 0x7f0bbc160f70 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.840+0000 7f0bc3a00640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bbc1417b0 0x7f0bbc13eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.840+0000 7f0bc3a00640 1 --2- 172.21.15.66:0/1915477140 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ba4003c20 0x7f0bbc1fbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:42.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.840+0000 7f0bc3a00640 1 -- 172.21.15.66:0/1915477140 >> 172.21.15.66:0/1915477140 conn(0x7f0bbc0cfa30 msgr2=0x7f0bbc0c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:42.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.840+0000 7f0bc3a00640 1 -- 172.21.15.66:0/1915477140 shutdown_connections 2024-09-10T22:11:42.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:42.840+0000 7f0bc3a00640 1 -- 172.21.15.66:0/1915477140 wait complete. 2024-09-10T22:11:42.863 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:11:42.863 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:11:42.964 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:11:43.049 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:11:43.049 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:11:43.049 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:11:43.058 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:11:43.058 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:11:43.058 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:11:43.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:11:43.067 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:11:43.067 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:11:43.109 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:11:43.109 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:11:43.110 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:11:43.119 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:11:43.119 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T22:11:43.119 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:11:43.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 -- 172.21.15.66:0/1889452664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94700d0260 msgr2=0x7f94700ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:43.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 --2- 172.21.15.66:0/1889452664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94700d0260 0x7f94700ae960 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f9458009840 tx=0x7f945801c820 comp rx=0 tx=0).stop 2024-09-10T22:11:43.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 -- 172.21.15.66:0/1889452664 shutdown_connections 2024-09-10T22:11:43.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 --2- 172.21.15.66:0/1889452664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94700d0260 0x7f94700ae960 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:43.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 --2- 172.21.15.66:0/1889452664 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94700b2bf0 0x7f94700aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 -- 172.21.15.66:0/1889452664 >> 172.21.15.66:0/1889452664 conn(0x7f94700d1c50 msgr2=0x7f94700d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 -- 172.21.15.66:0/1889452664 shutdown_connections 2024-09-10T22:11:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 -- 172.21.15.66:0/1889452664 wait complete. 2024-09-10T22:11:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 Processor -- start 2024-09-10T22:11:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 -- start start 2024-09-10T22:11:43.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94700b2bf0 0x7f9470151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:43.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94700d0260 0x7f947006f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947015f340 0x7f94701545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9470169240 con 0x7f94700b2bf0 2024-09-10T22:11:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94701690c0 con 0x7f947015f340 2024-09-10T22:11:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f9474a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f94701693c0 con 0x7f94700d0260 2024-09-10T22:11:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f946de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94700b2bf0 0x7f9470151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f946de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94700b2bf0 0x7f9470151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55208/0 (socket says 172.21.15.66:55208) 2024-09-10T22:11:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.300+0000 7f946de00640 1 -- 172.21.15.66:0/854549549 learned_addr learned my addr 172.21.15.66:0/854549549 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f9465400640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94700d0260 0x7f947006f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f946e800640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947015f340 0x7f94701545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f946de00640 1 -- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94700d0260 msgr2=0x7f947006f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f946de00640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94700d0260 0x7f947006f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f946de00640 1 -- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947015f340 msgr2=0x7f94701545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f946de00640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947015f340 0x7f94701545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f946de00640 1 -- 172.21.15.66:0/854549549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947006e920 con 0x7f94700b2bf0 2024-09-10T22:11:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f946de00640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94700b2bf0 0x7f9470151060 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f9460001ec0 tx=0x7f946000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f9466a00640 1 -- 172.21.15.66:0/854549549 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460017070 con 0x7f94700b2bf0 2024-09-10T22:11:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f9466a00640 1 -- 172.21.15.66:0/854549549 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9460003600 con 0x7f94700b2bf0 2024-09-10T22:11:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f9466a00640 1 -- 172.21.15.66:0/854549549 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460003a50 con 0x7f94700b2bf0 2024-09-10T22:11:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f9474a00640 1 -- 172.21.15.66:0/854549549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947006ebf0 con 0x7f94700b2bf0 2024-09-10T22:11:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f9474a00640 1 -- 172.21.15.66:0/854549549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947006f1b0 con 0x7f94700b2bf0 2024-09-10T22:11:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f9474a00640 1 -- 172.21.15.66:0/854549549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94700cfb50 con 0x7f94700b2bf0 2024-09-10T22:11:43.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.304+0000 7f9466a00640 1 -- 172.21.15.66:0/854549549 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f9460003bf0 con 0x7f94700b2bf0 2024-09-10T22:11:43.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.308+0000 7f9466a00640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f944c073f30 0x7f944c0763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:43.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.308+0000 7f9466a00640 1 -- 172.21.15.66:0/854549549 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7f9460091690 con 0x7f94700b2bf0 2024-09-10T22:11:43.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.308+0000 7f9465400640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f944c073f30 0x7f944c0763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:43.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.308+0000 7f9465400640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f944c073f30 0x7f944c0763f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f94580af000 tx=0x7f945801b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.312+0000 7f9466a00640 1 -- 172.21.15.66:0/854549549 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f946005e850 con 0x7f94700b2bf0 2024-09-10T22:11:43.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.516+0000 7f9474a00640 1 -- 172.21.15.66:0/854549549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94700b3f60 con 0x7f94700b2bf0 2024-09-10T22:11:43.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.516+0000 7f9466a00640 1 -- 172.21.15.66:0/854549549 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+3908 (secure 0 0 0) 0x7f946005e150 con 0x7f94700b2bf0 2024-09-10T22:11:43.521 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:43.521 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":69,"btime":"2024-09-10T22:11:37:087240+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3262462745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3262462745},{"type":"v1","addr":"172.21.15.66:6834","nonce":3262462745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/273414872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":273414872},{"type":"v1","addr":"172.21.15.94:6835","nonce":273414872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7246,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3486474426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3486474426},{"type":"v1","addr":"172.21.15.66:6835","nonce":3486474426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3940833195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3940833195},{"type":"v1","addr":"172.21.15.94:6837","nonce":3940833195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-09-10T22:11:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 69 2024-09-10T22:11:43.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.520+0000 7f9474a00640 1 -- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f944c073f30 msgr2=0x7f944c0763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:43.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.520+0000 7f9474a00640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f944c073f30 0x7f944c0763f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f94580af000 tx=0x7f945801b060 comp rx=0 tx=0).stop 2024-09-10T22:11:43.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.520+0000 7f9474a00640 1 -- 172.21.15.66:0/854549549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94700b2bf0 msgr2=0x7f9470151060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:43.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.520+0000 7f9474a00640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94700b2bf0 0x7f9470151060 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f9460001ec0 tx=0x7f946000c890 comp rx=0 tx=0).stop 2024-09-10T22:11:43.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.520+0000 7f9474a00640 1 -- 172.21.15.66:0/854549549 shutdown_connections 2024-09-10T22:11:43.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.520+0000 7f9474a00640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f944c073f30 0x7f944c0763f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:43.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.520+0000 7f9474a00640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947015f340 0x7f94701545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:43.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.520+0000 7f9474a00640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94700d0260 0x7f947006f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:43.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.520+0000 7f9474a00640 1 --2- 172.21.15.66:0/854549549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94700b2bf0 0x7f9470151060 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:43.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.520+0000 7f9474a00640 1 -- 172.21.15.66:0/854549549 >> 172.21.15.66:0/854549549 conn(0x7f94700d1c50 msgr2=0x7f9470152d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:43.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.520+0000 7f9474a00640 1 -- 172.21.15.66:0/854549549 shutdown_connections 2024-09-10T22:11:43.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.520+0000 7f9474a00640 1 -- 172.21.15.66:0/854549549 wait complete. 2024-09-10T22:11:43.543 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:11:43.543 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:11:43.543 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:11:43.543 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:11:43.544 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:11:43.544 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:11:43.544 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:11:43.544 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:11:43.544 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:11:43.544 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:11:43.544 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:11:43.545 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:11:43.545 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:11:43.545 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:11:43.545 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:11:43.545 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:11:43.557 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:11:43.558 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:11:43.558 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:11:43.559 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:11:43.559 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:11:43.560 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:11:43.560 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:11:43.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:11:43.608 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:11:43.609 DEBUG:teuthology.parallel:result is None 2024-09-10T22:11:43.609 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:11:43.609 DEBUG:teuthology.parallel:result is None 2024-09-10T22:11:43.610 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:11:43.610 DEBUG:teuthology.parallel:result is None 2024-09-10T22:11:43.610 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:11:43.611 DEBUG:teuthology.parallel:result is None 2024-09-10T22:11:43.611 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T22:11:43.697 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:11:43.698 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:11:43.701 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:11:43.702 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:11:43.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 -- 172.21.15.66:0/1402098194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34406d630 msgr2=0x7fc344160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:43.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 --2- 172.21.15.66:0/1402098194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34406d630 0x7fc344160370 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009bc0 tx=0x7fc34415d120 comp rx=0 tx=0).stop 2024-09-10T22:11:43.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 -- 172.21.15.66:0/1402098194 shutdown_connections 2024-09-10T22:11:43.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 --2- 172.21.15.66:0/1402098194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34406d630 0x7fc344160370 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:43.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 -- 172.21.15.66:0/1402098194 >> 172.21.15.66:0/1402098194 conn(0x7fc3440d0b30 msgr2=0x7fc3440d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:43.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 -- 172.21.15.66:0/1402098194 shutdown_connections 2024-09-10T22:11:43.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 -- 172.21.15.66:0/1402098194 wait complete. 2024-09-10T22:11:43.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 Processor -- start 2024-09-10T22:11:43.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 -- start start 2024-09-10T22:11:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34406d630 0x7fc3441f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc344071230 0x7fc34414c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc344156430 0x7fc34415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc34416a240 con 0x7fc34406d630 2024-09-10T22:11:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc34416a0c0 con 0x7fc344071230 2024-09-10T22:11:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc348e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc34416a3c0 con 0x7fc344156430 2024-09-10T22:11:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc341e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34406d630 0x7fc3441f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc341e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34406d630 0x7fc3441f9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55254/0 (socket says 172.21.15.66:55254) 2024-09-10T22:11:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc341e00640 1 -- 172.21.15.66:0/772295768 learned_addr learned my addr 172.21.15.66:0/772295768 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc342800640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc344156430 0x7fc34415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc33be00640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc344071230 0x7fc34414c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc341e00640 1 -- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc344156430 msgr2=0x7fc34415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc341e00640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc344156430 0x7fc34415d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc341e00640 1 -- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc344071230 msgr2=0x7fc34414c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc341e00640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc344071230 0x7fc34414c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.760+0000 7fc341e00640 1 -- 172.21.15.66:0/772295768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34414c840 con 0x7fc34406d630 2024-09-10T22:11:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.764+0000 7fc341e00640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34406d630 0x7fc3441f9430 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fc33400c430 tx=0x7fc33400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:43.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.764+0000 7fc33a000640 1 -- 172.21.15.66:0/772295768 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334011790 con 0x7fc34406d630 2024-09-10T22:11:43.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.764+0000 7fc33a000640 1 -- 172.21.15.66:0/772295768 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc334017070 con 0x7fc34406d630 2024-09-10T22:11:43.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.764+0000 7fc348e00640 1 -- 172.21.15.66:0/772295768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc344152690 con 0x7fc34406d630 2024-09-10T22:11:43.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.764+0000 7fc33a000640 1 -- 172.21.15.66:0/772295768 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334011ce0 con 0x7fc34406d630 2024-09-10T22:11:43.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.764+0000 7fc348e00640 1 -- 172.21.15.66:0/772295768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3441517a0 con 0x7fc34406d630 2024-09-10T22:11:43.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.764+0000 7fc348e00640 1 -- 172.21.15.66:0/772295768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc344160370 con 0x7fc34406d630 2024-09-10T22:11:43.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.764+0000 7fc33a000640 1 -- 172.21.15.66:0/772295768 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fc33401c460 con 0x7fc34406d630 2024-09-10T22:11:43.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.764+0000 7fc33a000640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc320073f30 0x7fc3200763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:43.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.764+0000 7fc33a000640 1 -- 172.21.15.66:0/772295768 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5027+0+0 (secure 0 0 0) 0x7fc334091e20 con 0x7fc34406d630 2024-09-10T22:11:43.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.764+0000 7fc33be00640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc320073f30 0x7fc3200763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:43.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.768+0000 7fc33be00640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc320073f30 0x7fc3200763f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc3440c7650 tx=0x7fc32c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:43.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.768+0000 7fc33a000640 1 -- 172.21.15.66:0/772295768 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc34414c840 con 0x7fc34406d630 2024-09-10T22:11:43.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:43.952+0000 7fc348e00640 1 -- 172.21.15.66:0/772295768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc344076650 con 0x7fc34406d630 2024-09-10T22:11:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc33a000640 1 -- 172.21.15.66:0/772295768 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v106) ==== 97+0+0 (secure 0 0 0) 0x7fc33405f020 con 0x7fc34406d630 2024-09-10T22:11:44.135 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-09-10T22:11:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc348e00640 1 -- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc320073f30 msgr2=0x7fc3200763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc348e00640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc320073f30 0x7fc3200763f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc3440c7650 tx=0x7fc32c009040 comp rx=0 tx=0).stop 2024-09-10T22:11:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc348e00640 1 -- 172.21.15.66:0/772295768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34406d630 msgr2=0x7fc3441f9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc348e00640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34406d630 0x7fc3441f9430 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fc33400c430 tx=0x7fc33400c8f0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc348e00640 1 -- 172.21.15.66:0/772295768 shutdown_connections 2024-09-10T22:11:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc348e00640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc320073f30 0x7fc3200763f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc348e00640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc344156430 0x7fc34415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc348e00640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc344071230 0x7fc34414c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc348e00640 1 --2- 172.21.15.66:0/772295768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34406d630 0x7fc3441f9430 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc348e00640 1 -- 172.21.15.66:0/772295768 >> 172.21.15.66:0/772295768 conn(0x7fc3440d0b30 msgr2=0x7fc34415edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc348e00640 1 -- 172.21.15.66:0/772295768 shutdown_connections 2024-09-10T22:11:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.132+0000 7fc348e00640 1 -- 172.21.15.66:0/772295768 wait complete. 2024-09-10T22:11:44.160 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T22:11:44.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.304+0000 7f018fc00640 1 -- 172.21.15.66:0/2629290576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0170004e00 msgr2=0x7f0170005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:44.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.304+0000 7f018fc00640 1 --2- 172.21.15.66:0/2629290576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0170004e00 0x7f0170005280 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f018000c640 tx=0x7f018001d1e0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.304+0000 7f018dc00640 1 -- 172.21.15.66:0/2629290576 reap_dead start 2024-09-10T22:11:44.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.304+0000 7f018fc00640 1 -- 172.21.15.66:0/2629290576 shutdown_connections 2024-09-10T22:11:44.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.304+0000 7f018fc00640 1 -- 172.21.15.66:0/2629290576 >> 172.21.15.66:0/2629290576 conn(0x7f01880cc900 msgr2=0x7f01880ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:44.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.304+0000 7f018fc00640 1 -- 172.21.15.66:0/2629290576 shutdown_connections 2024-09-10T22:11:44.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.304+0000 7f018fc00640 1 -- 172.21.15.66:0/2629290576 wait complete. 2024-09-10T22:11:44.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018fc00640 1 Processor -- start 2024-09-10T22:11:44.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018fc00640 1 -- start start 2024-09-10T22:11:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018fc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0170004e00 0x7f0188156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018fc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0188168cd0 0x7f01881fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018fc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01881fbd00 0x7f0188157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018fc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f018816a470 con 0x7f01881fbd00 2024-09-10T22:11:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018fc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f018816a2f0 con 0x7f0170004e00 2024-09-10T22:11:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018fc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f018816a5f0 con 0x7f0188168cd0 2024-09-10T22:11:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01881fbd00 0x7f0188157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01881fbd00 0x7f0188157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55282/0 (socket says 172.21.15.66:55282) 2024-09-10T22:11:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018dc00640 1 -- 172.21.15.66:0/2862096920 learned_addr learned my addr 172.21.15.66:0/2862096920 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018d200640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0170004e00 0x7f0188156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f0187e00640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0188168cd0 0x7f01881fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018dc00640 1 -- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0188168cd0 msgr2=0x7f01881fb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018dc00640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0188168cd0 0x7f01881fb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018dc00640 1 -- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0170004e00 msgr2=0x7f0188156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018dc00640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0170004e00 0x7f0188156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018dc00640 1 -- 172.21.15.66:0/2862096920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0188156270 con 0x7f01881fbd00 2024-09-10T22:11:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018dc00640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01881fbd00 0x7f0188157ac0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f018001d6c0 tx=0x7f01800065e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f0184c00640 1 -- 172.21.15.66:0/2862096920 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01800011d0 con 0x7f01881fbd00 2024-09-10T22:11:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f0184c00640 1 -- 172.21.15.66:0/2862096920 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0180004e40 con 0x7f01881fbd00 2024-09-10T22:11:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f0184c00640 1 -- 172.21.15.66:0/2862096920 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180006700 con 0x7f01881fbd00 2024-09-10T22:11:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.308+0000 7f018fc00640 1 -- 172.21.15.66:0/2862096920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018815e660 con 0x7f01881fbd00 2024-09-10T22:11:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.312+0000 7f018fc00640 1 -- 172.21.15.66:0/2862096920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01881fc5d0 con 0x7f01881fbd00 2024-09-10T22:11:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.312+0000 7f018fc00640 1 -- 172.21.15.66:0/2862096920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01880add90 con 0x7f01881fbd00 2024-09-10T22:11:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.312+0000 7f0184c00640 1 -- 172.21.15.66:0/2862096920 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f0180003980 con 0x7f01881fbd00 2024-09-10T22:11:44.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.312+0000 7f0184c00640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0168073f60 0x7f0168076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:44.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.312+0000 7f0184c00640 1 -- 172.21.15.66:0/2862096920 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4941+0+0 (secure 0 0 0) 0x7f018012c420 con 0x7f01881fbd00 2024-09-10T22:11:44.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.312+0000 7f018d200640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0168073f60 0x7f0168076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:44.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.316+0000 7f018d200640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0168073f60 0x7f0168076420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f017c0064f0 tx=0x7f017c006c70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:44.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.316+0000 7f0184c00640 1 -- 172.21.15.66:0/2862096920 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0180016f90 con 0x7f01881fbd00 2024-09-10T22:11:44.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.552+0000 7f018fc00640 1 -- 172.21.15.66:0/2862096920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f01880cc150 con 0x7f01881fbd00 2024-09-10T22:11:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f0184c00640 1 -- 172.21.15.66:0/2862096920 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) ==== 80+0+5304 (secure 0 0 0) 0x7f01800f9510 con 0x7f01881fbd00 2024-09-10T22:11:44.561 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:44.561 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:11:44.561 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-09-10T22:11:44.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.561 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmxQD7CBAAttZP0KtdjTI7QWbD7CVinQ==", 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmI7ufFRAANelhvqylKyfGPVfBuY40kg==", 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.562 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQnVoEBAAvsZ/TSOOP/JgJBr/aHwI/g==", 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.563 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmRQkTHRAAHPIdwqtkkyh31D7IO2vxPA==", 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.564 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-09-10T22:11:44.565 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBCweBmDJkxJBAAVSQNiMAQl5bR3RUwVOuGYw==", 2024-09-10T22:11:44.565 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.565 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:11:44.565 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:11:44.565 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.565 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.565 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-09-10T22:11:44.565 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBFweBmwcdsOxAAPPQyzktcfiF8pJNP8uAjwg==", 2024-09-10T22:11:44.565 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.566 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:11:44.566 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:11:44.566 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.566 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.566 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-09-10T22:11:44.567 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBJweBmXQhZExAAnEf8mjGX6nnYwWygKjo/2A==", 2024-09-10T22:11:44.567 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.567 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:11:44.567 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:11:44.567 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.567 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.567 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBNweBmlauHAxAA/k9xRqbHZsqJq9OYbJKfNQ==", 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBRweBmHnFDJRAAgF1ZAPvmoNzeUftZQyL0eg==", 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:11:44.568 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBUweBm/hXpORAA6jpOI/I+QyB2mZsEoAWF0w==", 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.569 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBYweBmYu2MExAAPD9WcxrBLzwwJQm19z7YcQ==", 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbweBmXfJ9KRAA/AbWlli+Qfg7gy0SblJ7fg==", 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.570 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw==", 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmbS6IKhAAaEmK9irxMipALgkAct/+OQ==", 2024-09-10T22:11:44.571 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQ47KJhAAI50ZaSQIhH6FxRM48JZICA==", 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:11:44.572 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmUcLCMBAAdf4tfcpVaZoaKjS1UzrxhA==", 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.573 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA+weBmPIAzKBAAkyVV875PNhBmEoKsrAkg8w==", 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmLiy1BBAADnraML8IUet7Sw0j83TUVQ==", 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.574 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmItWpABAAlJ+6dFjvtBV3g+jYmUh0tQ==", 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.575 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T22:11:44.576 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:11:44.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f018fc00640 1 -- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0168073f60 msgr2=0x7f0168076420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:44.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f018fc00640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0168073f60 0x7f0168076420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f017c0064f0 tx=0x7f017c006c70 comp rx=0 tx=0).stop 2024-09-10T22:11:44.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f018fc00640 1 -- 172.21.15.66:0/2862096920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01881fbd00 msgr2=0x7f0188157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:44.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f018fc00640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01881fbd00 0x7f0188157ac0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f018001d6c0 tx=0x7f01800065e0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f018fc00640 1 -- 172.21.15.66:0/2862096920 shutdown_connections 2024-09-10T22:11:44.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f018fc00640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0168073f60 0x7f0168076420 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f018fc00640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01881fbd00 0x7f0188157ac0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f018fc00640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0188168cd0 0x7f01881fb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f018fc00640 1 --2- 172.21.15.66:0/2862096920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0170004e00 0x7f0188156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f018fc00640 1 -- 172.21.15.66:0/2862096920 >> 172.21.15.66:0/2862096920 conn(0x7f01880cc900 msgr2=0x7f018815cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:44.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f018fc00640 1 -- 172.21.15.66:0/2862096920 shutdown_connections 2024-09-10T22:11:44.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.556+0000 7f018fc00640 1 -- 172.21.15.66:0/2862096920 wait complete. 2024-09-10T22:11:44.584 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:11:44.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 -- 172.21.15.66:0/1303812543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f924c0c4ef0 msgr2=0x7f924c0c5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:44.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 --2- 172.21.15.66:0/1303812543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f924c0c4ef0 0x7f924c0c5320 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f924c0cd550 tx=0x7f924401cd80 comp rx=0 tx=0).stop 2024-09-10T22:11:44.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 -- 172.21.15.66:0/1303812543 shutdown_connections 2024-09-10T22:11:44.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 --2- 172.21.15.66:0/1303812543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f924c0c4ef0 0x7f924c0c5320 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 --2- 172.21.15.66:0/1303812543 >> v2:172.21.15.94:3300/0 conn(0x7f924c0c4b10 0x7f924c06e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 -- 172.21.15.66:0/1303812543 >> 172.21.15.66:0/1303812543 conn(0x7f924c0cfa30 msgr2=0x7f924c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:44.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 -- 172.21.15.66:0/1303812543 shutdown_connections 2024-09-10T22:11:44.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 -- 172.21.15.66:0/1303812543 wait complete. 2024-09-10T22:11:44.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 Processor -- start 2024-09-10T22:11:44.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 -- start start 2024-09-10T22:11:44.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f924c0ae620 0x7f924c0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:44.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f924c0c4b10 0x7f924c0ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:44.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f924c0c4ef0 0x7f924c0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:44.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f924c167b80 con 0x7f924c0ae620 2024-09-10T22:11:44.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f924c167a00 con 0x7f924c0c4b10 2024-09-10T22:11:44.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9253e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f924c167d00 con 0x7f924c0c4ef0 2024-09-10T22:11:44.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9251400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f924c0ae620 0x7f924c0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9251400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f924c0ae620 0x7f924c0cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55300/0 (socket says 172.21.15.66:55300) 2024-09-10T22:11:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9251400640 1 -- 172.21.15.66:0/1950074992 learned_addr learned my addr 172.21.15.66:0/1950074992 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9250a00640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f924c0c4b10 0x7f924c0ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.728+0000 7f9251e00640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f924c0c4ef0 0x7f924c0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9251400640 1 -- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f924c0c4ef0 msgr2=0x7f924c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9251400640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f924c0c4ef0 0x7f924c0adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9251400640 1 -- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f924c0c4b10 msgr2=0x7f924c0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9251400640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f924c0c4b10 0x7f924c0ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9251400640 1 -- 172.21.15.66:0/1950074992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c06e790 con 0x7f924c0ae620 2024-09-10T22:11:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9251400640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f924c0ae620 0x7f924c0cdc40 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f924000e9f0 tx=0x7f924000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:44.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9249600640 1 -- 172.21.15.66:0/1950074992 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924000cd80 con 0x7f924c0ae620 2024-09-10T22:11:44.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9249600640 1 -- 172.21.15.66:0/1950074992 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9240010430 con 0x7f924c0ae620 2024-09-10T22:11:44.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9253e00640 1 -- 172.21.15.66:0/1950074992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c0ce270 con 0x7f924c0ae620 2024-09-10T22:11:44.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9249600640 1 -- 172.21.15.66:0/1950074992 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240010860 con 0x7f924c0ae620 2024-09-10T22:11:44.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9253e00640 1 -- 172.21.15.66:0/1950074992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c1fb320 con 0x7f924c0ae620 2024-09-10T22:11:44.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9253e00640 1 -- 172.21.15.66:0/1950074992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924c1fb760 con 0x7f924c0ae620 2024-09-10T22:11:44.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9249600640 1 -- 172.21.15.66:0/1950074992 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f9240010a00 con 0x7f924c0ae620 2024-09-10T22:11:44.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9249600640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f922c073f30 0x7f922c0763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:44.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.732+0000 7f9249600640 1 -- 172.21.15.66:0/1950074992 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4941+0+0 (secure 0 0 0) 0x7f9240092c30 con 0x7f924c0ae620 2024-09-10T22:11:44.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.736+0000 7f9250a00640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f922c073f30 0x7f922c0763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:44.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.736+0000 7f9250a00640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f922c073f30 0x7f922c0763f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f924c0b5840 tx=0x7f9238009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:44.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.736+0000 7f9249600640 1 -- 172.21.15.66:0/1950074992 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f924005f7e0 con 0x7f924c0ae620 2024-09-10T22:11:44.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 -- 172.21.15.66:0/1950074992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f924c0734e0 con 0x7f924c0ae620 2024-09-10T22:11:44.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9249600640 1 -- 172.21.15.66:0/1950074992 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) ==== 84+0+65 (secure 0 0 0) 0x7f9240064210 con 0x7f924c0ae620 2024-09-10T22:11:44.919 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:44.919 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:11:44.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:11:44.919 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:11:44.919 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:11:44.919 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:11:44.919 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:11:44.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 -- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f922c073f30 msgr2=0x7f922c0763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:44.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f922c073f30 0x7f922c0763f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f924c0b5840 tx=0x7f9238009040 comp rx=0 tx=0).stop 2024-09-10T22:11:44.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 -- 172.21.15.66:0/1950074992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f924c0ae620 msgr2=0x7f924c0cdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:44.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f924c0ae620 0x7f924c0cdc40 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f924000e9f0 tx=0x7f924000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 -- 172.21.15.66:0/1950074992 shutdown_connections 2024-09-10T22:11:44.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f922c073f30 0x7f922c0763f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f924c0c4ef0 0x7f924c0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f924c0c4b10 0x7f924c0ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 --2- 172.21.15.66:0/1950074992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f924c0ae620 0x7f924c0cdc40 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:44.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 -- 172.21.15.66:0/1950074992 >> 172.21.15.66:0/1950074992 conn(0x7f924c0cfa30 msgr2=0x7f924c0afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:44.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 -- 172.21.15.66:0/1950074992 shutdown_connections 2024-09-10T22:11:44.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:44.916+0000 7f9253e00640 1 -- 172.21.15.66:0/1950074992 wait complete. 2024-09-10T22:11:44.941 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:11:44.941 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:11:45.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.088+0000 7ff392600640 1 -- 172.21.15.66:0/2409046678 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff38c161440 msgr2=0x7ff38c0d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:45.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.088+0000 7ff392600640 1 --2- 172.21.15.66:0/2409046678 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff38c161440 0x7ff38c0d4970 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009900 tx=0x7ff37c01c8e0 comp rx=0 tx=0).stop 2024-09-10T22:11:45.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.088+0000 7ff392600640 1 -- 172.21.15.66:0/2409046678 shutdown_connections 2024-09-10T22:11:45.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.088+0000 7ff392600640 1 --2- 172.21.15.66:0/2409046678 >> v2:172.21.15.94:3300/0 conn(0x7ff38c1619f0 0x7ff38c161e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:45.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.088+0000 7ff392600640 1 --2- 172.21.15.66:0/2409046678 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff38c161440 0x7ff38c0d4970 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:45.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.088+0000 7ff392600640 1 -- 172.21.15.66:0/2409046678 >> 172.21.15.66:0/2409046678 conn(0x7ff38c0d3e50 msgr2=0x7ff38c0d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:45.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.088+0000 7ff392600640 1 -- 172.21.15.66:0/2409046678 shutdown_connections 2024-09-10T22:11:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.088+0000 7ff392600640 1 -- 172.21.15.66:0/2409046678 wait complete. 2024-09-10T22:11:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff392600640 1 Processor -- start 2024-09-10T22:11:45.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff392600640 1 -- start start 2024-09-10T22:11:45.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff392600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff38c0b3720 0x7ff38c150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:45.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff392600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff38c161440 0x7ff38c0ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:45.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff392600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c1619f0 0x7ff38c0cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:45.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff392600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c0d0fb0 con 0x7ff38c161440 2024-09-10T22:11:45.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff392600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c0d0e30 con 0x7ff38c1619f0 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff392600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff38c0d1130 con 0x7ff38c0b3720 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff382a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff38c161440 0x7ff38c0ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff382a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff38c161440 0x7ff38c0ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55302/0 (socket says 172.21.15.66:55302) 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff382a00640 1 -- 172.21.15.66:0/1625669265 learned_addr learned my addr 172.21.15.66:0/1625669265 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff38b400640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff38c0b3720 0x7ff38c150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff38be00640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c1619f0 0x7ff38c0cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff38b400640 1 -- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c1619f0 msgr2=0x7ff38c0cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff38b400640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c1619f0 0x7ff38c0cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff38b400640 1 -- 172.21.15.66:0/1625669265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff38c161440 msgr2=0x7ff38c0ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff38b400640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff38c161440 0x7ff38c0ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff38b400640 1 -- 172.21.15.66:0/1625669265 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c1f81f0 con 0x7ff38c0b3720 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff382a00640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff38c161440 0x7ff38c0ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:11:45.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff38b400640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff38c0b3720 0x7ff38c150c30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0ac000 tx=0x7ff37c0a55a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:45.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff383e00640 1 -- 172.21.15.66:0/1625669265 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c0b2040 con 0x7ff38c0b3720 2024-09-10T22:11:45.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff383e00640 1 -- 172.21.15.66:0/1625669265 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff37c0043f0 con 0x7ff38c0b3720 2024-09-10T22:11:45.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff392600640 1 -- 172.21.15.66:0/1625669265 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38c1f84d0 con 0x7ff38c0b3720 2024-09-10T22:11:45.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff383e00640 1 -- 172.21.15.66:0/1625669265 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c005290 con 0x7ff38c0b3720 2024-09-10T22:11:45.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.092+0000 7ff392600640 1 -- 172.21.15.66:0/1625669265 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38c1f7ef0 con 0x7ff38c0b3720 2024-09-10T22:11:45.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.096+0000 7ff392600640 1 -- 172.21.15.66:0/1625669265 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff38c0d1620 con 0x7ff38c0b3720 2024-09-10T22:11:45.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.096+0000 7ff383e00640 1 -- 172.21.15.66:0/1625669265 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7ff37c005430 con 0x7ff38c0b3720 2024-09-10T22:11:45.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.096+0000 7ff383e00640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff368073f30 0x7ff3680763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:45.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.096+0000 7ff383e00640 1 -- 172.21.15.66:0/1625669265 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(108..108 src has 1..108) ==== 4941+0+0 (secure 0 0 0) 0x7ff37c12b8b0 con 0x7ff38c0b3720 2024-09-10T22:11:45.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.096+0000 7ff382a00640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff368073f30 0x7ff3680763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:45.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.100+0000 7ff382a00640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff368073f30 0x7ff3680763f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0d1c20 tx=0x7ff374009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:45.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.100+0000 7ff383e00640 1 -- 172.21.15.66:0/1625669265 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff37c0bc060 con 0x7ff38c0b3720 2024-09-10T22:11:45.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:45.276+0000 7ff392600640 1 -- 172.21.15.66:0/1625669265 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff38c0d2210 con 0x7ff38c0b3720 2024-09-10T22:11:46.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.152+0000 7ff383e00640 1 -- 172.21.15.66:0/1625669265 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v109) ==== 138+0+0 (secure 0 0 0) 0x7ff37c0164d0 con 0x7ff38c0b3720 2024-09-10T22:11:46.157 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:11:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.160+0000 7ff392600640 1 -- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff368073f30 msgr2=0x7ff3680763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.160+0000 7ff392600640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff368073f30 0x7ff3680763f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0d1c20 tx=0x7ff374009040 comp rx=0 tx=0).stop 2024-09-10T22:11:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.160+0000 7ff392600640 1 -- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff38c0b3720 msgr2=0x7ff38c150c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.160+0000 7ff392600640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff38c0b3720 0x7ff38c150c30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0ac000 tx=0x7ff37c0a55a0 comp rx=0 tx=0).stop 2024-09-10T22:11:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.160+0000 7ff392600640 1 -- 172.21.15.66:0/1625669265 shutdown_connections 2024-09-10T22:11:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.160+0000 7ff392600640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff368073f30 0x7ff3680763f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.160+0000 7ff392600640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c1619f0 0x7ff38c0cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.160+0000 7ff392600640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff38c161440 0x7ff38c0ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.160+0000 7ff392600640 1 --2- 172.21.15.66:0/1625669265 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff38c0b3720 0x7ff38c150c30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.160+0000 7ff392600640 1 -- 172.21.15.66:0/1625669265 >> 172.21.15.66:0/1625669265 conn(0x7ff38c0d3e50 msgr2=0x7ff38c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.160+0000 7ff392600640 1 -- 172.21.15.66:0/1625669265 shutdown_connections 2024-09-10T22:11:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.160+0000 7ff392600640 1 -- 172.21.15.66:0/1625669265 wait complete. 2024-09-10T22:11:46.189 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:11:46.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.344+0000 7ff589200640 1 -- 172.21.15.66:0/2397258642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5840b0590 msgr2=0x7ff5840cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:46.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.344+0000 7ff589200640 1 --2- 172.21.15.66:0/2397258642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5840b0590 0x7ff5840cd280 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff570009840 tx=0x7ff57001c790 comp rx=0 tx=0).stop 2024-09-10T22:11:46.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.344+0000 7ff589200640 1 -- 172.21.15.66:0/2397258642 shutdown_connections 2024-09-10T22:11:46.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.344+0000 7ff589200640 1 --2- 172.21.15.66:0/2397258642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5840b0590 0x7ff5840cd280 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:46.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.344+0000 7ff589200640 1 --2- 172.21.15.66:0/2397258642 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58415bfb0 0x7ff5840ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:46.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.344+0000 7ff589200640 1 -- 172.21.15.66:0/2397258642 >> 172.21.15.66:0/2397258642 conn(0x7ff584074240 msgr2=0x7ff584074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:46.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.344+0000 7ff589200640 1 -- 172.21.15.66:0/2397258642 shutdown_connections 2024-09-10T22:11:46.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.344+0000 7ff589200640 1 -- 172.21.15.66:0/2397258642 wait complete. 2024-09-10T22:11:46.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff589200640 1 Processor -- start 2024-09-10T22:11:46.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff589200640 1 -- start start 2024-09-10T22:11:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff589200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5840b0590 0x7ff5840cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff589200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5840cd7c0 0x7ff5840d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff589200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58415bfb0 0x7ff5840adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff589200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff584168e20 con 0x7ff5840cd7c0 2024-09-10T22:11:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff589200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff584168ca0 con 0x7ff58415bfb0 2024-09-10T22:11:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff589200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff584168fa0 con 0x7ff5840b0590 2024-09-10T22:11:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff581400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5840cd7c0 0x7ff5840d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff581400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5840cd7c0 0x7ff5840d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55304/0 (socket says 172.21.15.66:55304) 2024-09-10T22:11:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff581400640 1 -- 172.21.15.66:0/513587857 learned_addr learned my addr 172.21.15.66:0/513587857 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff581e00640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5840b0590 0x7ff5840cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff582800640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58415bfb0 0x7ff5840adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:46.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff581400640 1 -- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5840b0590 msgr2=0x7ff5840cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:46.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff581400640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5840b0590 0x7ff5840cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:46.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff581400640 1 -- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58415bfb0 msgr2=0x7ff5840adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:46.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff581400640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58415bfb0 0x7ff5840adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:46.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff581400640 1 -- 172.21.15.66:0/513587857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5840caf80 con 0x7ff5840cd7c0 2024-09-10T22:11:46.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff581400640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5840cd7c0 0x7ff5840d03c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff570009790 tx=0x7ff570005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff57a000640 1 -- 172.21.15.66:0/513587857 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5700b2040 con 0x7ff5840cd7c0 2024-09-10T22:11:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff57a000640 1 -- 172.21.15.66:0/513587857 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff570004330 con 0x7ff5840cd7c0 2024-09-10T22:11:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff57a000640 1 -- 172.21.15.66:0/513587857 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570005220 con 0x7ff5840cd7c0 2024-09-10T22:11:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff589200640 1 -- 172.21.15.66:0/513587857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5840cb260 con 0x7ff5840cd7c0 2024-09-10T22:11:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff589200640 1 -- 172.21.15.66:0/513587857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5840cb820 con 0x7ff5840cd7c0 2024-09-10T22:11:46.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.348+0000 7ff589200640 1 -- 172.21.15.66:0/513587857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5840b0ae0 con 0x7ff5840cd7c0 2024-09-10T22:11:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.352+0000 7ff57a000640 1 -- 172.21.15.66:0/513587857 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7ff5700b5070 con 0x7ff5840cd7c0 2024-09-10T22:11:46.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.352+0000 7ff57a000640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff564073f30 0x7ff5640763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:46.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.352+0000 7ff57a000640 1 -- 172.21.15.66:0/513587857 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5311+0+0 (secure 0 0 0) 0x7ff57012b8e0 con 0x7ff5840cd7c0 2024-09-10T22:11:46.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.352+0000 7ff581e00640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff564073f30 0x7ff5640763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:46.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.352+0000 7ff581e00640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff564073f30 0x7ff5640763f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff574009b40 tx=0x7ff5740091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.356+0000 7ff57a000640 1 -- 172.21.15.66:0/513587857 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff570016410 con 0x7ff5840cd7c0 2024-09-10T22:11:46.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:46.532+0000 7ff589200640 1 -- 172.21.15.66:0/513587857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff5840cc970 con 0x7ff5840cd7c0 2024-09-10T22:11:47.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.156+0000 7ff57a000640 1 -- 172.21.15.66:0/513587857 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v110) ==== 170+0+0 (secure 0 0 0) 0x7ff5701011c0 con 0x7ff5840cd7c0 2024-09-10T22:11:47.161 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:11:47.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.160+0000 7ff589200640 1 -- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff564073f30 msgr2=0x7ff5640763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:47.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.160+0000 7ff589200640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff564073f30 0x7ff5640763f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff574009b40 tx=0x7ff5740091e0 comp rx=0 tx=0).stop 2024-09-10T22:11:47.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.160+0000 7ff589200640 1 -- 172.21.15.66:0/513587857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5840cd7c0 msgr2=0x7ff5840d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:47.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.160+0000 7ff589200640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5840cd7c0 0x7ff5840d03c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff570009790 tx=0x7ff570005ca0 comp rx=0 tx=0).stop 2024-09-10T22:11:47.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.160+0000 7ff589200640 1 -- 172.21.15.66:0/513587857 shutdown_connections 2024-09-10T22:11:47.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.160+0000 7ff589200640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff564073f30 0x7ff5640763f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:47.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.160+0000 7ff589200640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58415bfb0 0x7ff5840adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:47.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.160+0000 7ff589200640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5840cd7c0 0x7ff5840d03c0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:47.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.160+0000 7ff589200640 1 --2- 172.21.15.66:0/513587857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5840b0590 0x7ff5840cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:47.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.160+0000 7ff589200640 1 -- 172.21.15.66:0/513587857 >> 172.21.15.66:0/513587857 conn(0x7ff584074240 msgr2=0x7ff58414d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:47.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.160+0000 7ff589200640 1 -- 172.21.15.66:0/513587857 shutdown_connections 2024-09-10T22:11:47.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.160+0000 7ff589200640 1 -- 172.21.15.66:0/513587857 wait complete. 2024-09-10T22:11:47.208 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:11:47.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.356+0000 7f9078e00640 1 -- 172.21.15.66:0/3789623452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740cf180 msgr2=0x7f90740d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:47.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.356+0000 7f9078e00640 1 --2- 172.21.15.66:0/3789623452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740cf180 0x7f90740d04e0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9074073840 tx=0x7f906c01ccf0 comp rx=0 tx=0).stop 2024-09-10T22:11:47.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.356+0000 7f9078e00640 1 -- 172.21.15.66:0/3789623452 shutdown_connections 2024-09-10T22:11:47.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.356+0000 7f9078e00640 1 --2- 172.21.15.66:0/3789623452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740cf180 0x7f90740d04e0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:47.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.356+0000 7f9078e00640 1 --2- 172.21.15.66:0/3789623452 >> v2:172.21.15.94:3300/0 conn(0x7f907415bd40 0x7f90740cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:47.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.356+0000 7f9078e00640 1 -- 172.21.15.66:0/3789623452 >> 172.21.15.66:0/3789623452 conn(0x7f90740d1c60 msgr2=0x7f90740d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:47.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.356+0000 7f9078e00640 1 -- 172.21.15.66:0/3789623452 shutdown_connections 2024-09-10T22:11:47.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.356+0000 7f9078e00640 1 -- 172.21.15.66:0/3789623452 wait complete. 2024-09-10T22:11:47.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.356+0000 7f9078e00640 1 Processor -- start 2024-09-10T22:11:47.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.356+0000 7f9078e00640 1 -- start start 2024-09-10T22:11:47.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9078e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740cf180 0x7f90740b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9078e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90740cf830 0x7f9074154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9078e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907415bd40 0x7f90741548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9078e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9074168ba0 con 0x7f90740cf180 2024-09-10T22:11:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9078e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9074168a20 con 0x7f90740cf830 2024-09-10T22:11:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9078e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9074168d20 con 0x7f907415bd40 2024-09-10T22:11:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9071e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740cf180 0x7f90740b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9071e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740cf180 0x7f90740b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55314/0 (socket says 172.21.15.66:55314) 2024-09-10T22:11:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9071e00640 1 -- 172.21.15.66:0/241021121 learned_addr learned my addr 172.21.15.66:0/241021121 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9071400640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90740cf830 0x7f9074154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9072800640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907415bd40 0x7f90741548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9071e00640 1 -- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907415bd40 msgr2=0x7f90741548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9071e00640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907415bd40 0x7f90741548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:47.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9071e00640 1 -- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90740cf830 msgr2=0x7f9074154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:47.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9071e00640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90740cf830 0x7f9074154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:47.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9071e00640 1 -- 172.21.15.66:0/241021121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907406fc10 con 0x7f90740cf180 2024-09-10T22:11:47.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9071400640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90740cf830 0x7f9074154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:47.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9071e00640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740cf180 0x7f90740b7300 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f906400e7d0 tx=0x7f906400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:47.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f906a000640 1 -- 172.21.15.66:0/241021121 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906400ccc0 con 0x7f90740cf180 2024-09-10T22:11:47.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f906a000640 1 -- 172.21.15.66:0/241021121 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9064010430 con 0x7f90740cf180 2024-09-10T22:11:47.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f906a000640 1 -- 172.21.15.66:0/241021121 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064010860 con 0x7f90740cf180 2024-09-10T22:11:47.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9078e00640 1 -- 172.21.15.66:0/241021121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907406ea80 con 0x7f90740cf180 2024-09-10T22:11:47.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9078e00640 1 -- 172.21.15.66:0/241021121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907414f110 con 0x7f90740cf180 2024-09-10T22:11:47.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f9078e00640 1 -- 172.21.15.66:0/241021121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907406ef30 con 0x7f90740cf180 2024-09-10T22:11:47.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.360+0000 7f906a000640 1 -- 172.21.15.66:0/241021121 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f9064010a00 con 0x7f90740cf180 2024-09-10T22:11:47.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.364+0000 7f906a000640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9054073f30 0x7f90540763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:47.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.364+0000 7f906a000640 1 -- 172.21.15.66:0/241021121 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5693+0+0 (secure 0 0 0) 0x7f9064092a50 con 0x7f90740cf180 2024-09-10T22:11:47.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.364+0000 7f9071400640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9054073f30 0x7f90540763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:47.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.364+0000 7f9071400640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9054073f30 0x7f90540763f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f90740cc100 tx=0x7f9060009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:47.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.368+0000 7f906a000640 1 -- 172.21.15.66:0/241021121 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f906405fc10 con 0x7f90740cf180 2024-09-10T22:11:47.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:47.576+0000 7f9078e00640 1 -- 172.21.15.66:0/241021121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f90740aec10 con 0x7f90740cf180 2024-09-10T22:11:48.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.160+0000 7f906a000640 1 -- 172.21.15.66:0/241021121 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v72) ==== 174+0+0 (secure 0 0 0) 0x7f906405f510 con 0x7f90740cf180 2024-09-10T22:11:48.162 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 16 and data pool 17 2024-09-10T22:11:48.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.160+0000 7f9078e00640 1 -- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9054073f30 msgr2=0x7f90540763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:48.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.160+0000 7f9078e00640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9054073f30 0x7f90540763f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f90740cc100 tx=0x7f9060009040 comp rx=0 tx=0).stop 2024-09-10T22:11:48.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.160+0000 7f9078e00640 1 -- 172.21.15.66:0/241021121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740cf180 msgr2=0x7f90740b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:48.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.160+0000 7f9078e00640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740cf180 0x7f90740b7300 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f906400e7d0 tx=0x7f906400ec90 comp rx=0 tx=0).stop 2024-09-10T22:11:48.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.160+0000 7f9078e00640 1 -- 172.21.15.66:0/241021121 shutdown_connections 2024-09-10T22:11:48.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.160+0000 7f9078e00640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9054073f30 0x7f90540763f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.160+0000 7f9078e00640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907415bd40 0x7f90741548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.160+0000 7f9078e00640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90740cf830 0x7f9074154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.160+0000 7f9078e00640 1 --2- 172.21.15.66:0/241021121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90740cf180 0x7f90740b7300 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.160+0000 7f9078e00640 1 -- 172.21.15.66:0/241021121 >> 172.21.15.66:0/241021121 conn(0x7f90740d1c60 msgr2=0x7f90740b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:48.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.164+0000 7f9078e00640 1 -- 172.21.15.66:0/241021121 shutdown_connections 2024-09-10T22:11:48.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.164+0000 7f9078e00640 1 -- 172.21.15.66:0/241021121 wait complete. 2024-09-10T22:11:48.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:11:48.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.352+0000 7fe2dd400640 1 -- 172.21.15.66:0/387440154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2d80cd7a0 msgr2=0x7fe2d80cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:48.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.352+0000 7fe2dd400640 1 --2- 172.21.15.66:0/387440154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2d80cd7a0 0x7fe2d80cdc30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009c80 tx=0x7fe2d001cda0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.352+0000 7fe2dd400640 1 -- 172.21.15.66:0/387440154 shutdown_connections 2024-09-10T22:11:48.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.352+0000 7fe2dd400640 1 --2- 172.21.15.66:0/387440154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2d80cd7a0 0x7fe2d80cdc30 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.352+0000 7fe2dd400640 1 --2- 172.21.15.66:0/387440154 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2d815ef20 0x7fe2d80cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.352+0000 7fe2dd400640 1 -- 172.21.15.66:0/387440154 >> 172.21.15.66:0/387440154 conn(0x7fe2d80d1b10 msgr2=0x7fe2d80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:48.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.352+0000 7fe2dd400640 1 -- 172.21.15.66:0/387440154 shutdown_connections 2024-09-10T22:11:48.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.352+0000 7fe2dd400640 1 -- 172.21.15.66:0/387440154 wait complete. 2024-09-10T22:11:48.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.352+0000 7fe2dd400640 1 Processor -- start 2024-09-10T22:11:48.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.352+0000 7fe2dd400640 1 -- start start 2024-09-10T22:11:48.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2dd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2d80cd7a0 0x7fe2d80b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:48.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2dd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2d815ef20 0x7fe2d80b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:48.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2dd400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2d815f300 0x7fe2d80b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:48.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2dd400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d8168ee0 con 0x7fe2d815ef20 2024-09-10T22:11:48.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2dd400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d8168d60 con 0x7fe2d80cd7a0 2024-09-10T22:11:48.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2dd400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d8169060 con 0x7fe2d815f300 2024-09-10T22:11:48.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2d815ef20 0x7fe2d80b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:48.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2d815ef20 0x7fe2d80b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55340/0 (socket says 172.21.15.66:55340) 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2d7400640 1 -- 172.21.15.66:0/177821846 learned_addr learned my addr 172.21.15.66:0/177821846 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2d7e00640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2d80cd7a0 0x7fe2d80b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2dca00640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2d815f300 0x7fe2d80b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2d7400640 1 -- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2d815f300 msgr2=0x7fe2d80b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2d7400640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2d815f300 0x7fe2d80b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2d7400640 1 -- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2d80cd7a0 msgr2=0x7fe2d80b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2d7400640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2d80cd7a0 0x7fe2d80b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2d7400640 1 -- 172.21.15.66:0/177821846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d81fb020 con 0x7fe2d815ef20 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2d7400640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2d815ef20 0x7fe2d80b4940 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400c3f0 tx=0x7fe2c400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2cfe00640 1 -- 172.21.15.66:0/177821846 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4017070 con 0x7fe2d815ef20 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2cfe00640 1 -- 172.21.15.66:0/177821846 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2c40053c0 con 0x7fe2d815ef20 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2cfe00640 1 -- 172.21.15.66:0/177821846 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4003860 con 0x7fe2d815ef20 2024-09-10T22:11:48.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2dd400640 1 -- 172.21.15.66:0/177821846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d81fb2c0 con 0x7fe2d815ef20 2024-09-10T22:11:48.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2dd400640 1 -- 172.21.15.66:0/177821846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d81fb880 con 0x7fe2d815ef20 2024-09-10T22:11:48.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.356+0000 7fe2dd400640 1 -- 172.21.15.66:0/177821846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d80cdc30 con 0x7fe2d815ef20 2024-09-10T22:11:48.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.360+0000 7fe2cfe00640 1 -- 172.21.15.66:0/177821846 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fe2c4003a00 con 0x7fe2d815ef20 2024-09-10T22:11:48.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.360+0000 7fe2cfe00640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe2b8073f30 0x7fe2b80763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:48.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.360+0000 7fe2cfe00640 1 -- 172.21.15.66:0/177821846 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5793+0+0 (secure 0 0 0) 0x7fe2c4091610 con 0x7fe2d815ef20 2024-09-10T22:11:48.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.360+0000 7fe2d7e00640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe2b8073f30 0x7fe2b80763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:48.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.360+0000 7fe2d7e00640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe2b8073f30 0x7fe2b80763f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8004630 tx=0x7fe2c800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:48.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.364+0000 7fe2cfe00640 1 -- 172.21.15.66:0/177821846 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe2c405e100 con 0x7fe2d815ef20 2024-09-10T22:11:48.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.540+0000 7fe2dd400640 1 -- 172.21.15.66:0/177821846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2d8166310 con 0x7fe2d815ef20 2024-09-10T22:11:48.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.540+0000 7fe2cfe00640 1 -- 172.21.15.66:0/177821846 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16761 (secure 0 0 0) 0x7fe2c4065e20 con 0x7fe2d815ef20 2024-09-10T22:11:48.546 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:48.546 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":111,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:11:47.579878+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:11:45.291701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T22:11:46.544807+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:11:48.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.544+0000 7fe2dd400640 1 -- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe2b8073f30 msgr2=0x7fe2b80763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:48.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.544+0000 7fe2dd400640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe2b8073f30 0x7fe2b80763f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8004630 tx=0x7fe2c800b390 comp rx=0 tx=0).stop 2024-09-10T22:11:48.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.544+0000 7fe2dd400640 1 -- 172.21.15.66:0/177821846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2d815ef20 msgr2=0x7fe2d80b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:48.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.544+0000 7fe2dd400640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2d815ef20 0x7fe2d80b4940 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400c3f0 tx=0x7fe2c400c8b0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.544+0000 7fe2dd400640 1 -- 172.21.15.66:0/177821846 shutdown_connections 2024-09-10T22:11:48.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.544+0000 7fe2dd400640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe2b8073f30 0x7fe2b80763f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.544+0000 7fe2dd400640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2d815f300 0x7fe2d80b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.544+0000 7fe2dd400640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2d815ef20 0x7fe2d80b4940 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.544+0000 7fe2dd400640 1 --2- 172.21.15.66:0/177821846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2d80cd7a0 0x7fe2d80b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.544+0000 7fe2dd400640 1 -- 172.21.15.66:0/177821846 >> 172.21.15.66:0/177821846 conn(0x7fe2d80d1b10 msgr2=0x7fe2d8075880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:48.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.544+0000 7fe2dd400640 1 -- 172.21.15.66:0/177821846 shutdown_connections 2024-09-10T22:11:48.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.544+0000 7fe2dd400640 1 -- 172.21.15.66:0/177821846 wait complete. 2024-09-10T22:11:48.567 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:11:48.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 -- 172.21.15.66:0/2600111530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b0160280 msgr2=0x7f91b00b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:48.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 --2- 172.21.15.66:0/2600111530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b0160280 0x7f91b00b3170 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009840 tx=0x7f91a001d1c0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 -- 172.21.15.66:0/2600111530 shutdown_connections 2024-09-10T22:11:48.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 --2- 172.21.15.66:0/2600111530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b00b1e10 0x7f91b00aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 --2- 172.21.15.66:0/2600111530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b0160280 0x7f91b00b3170 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 -- 172.21.15.66:0/2600111530 >> 172.21.15.66:0/2600111530 conn(0x7f91b00d1c60 msgr2=0x7f91b00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:48.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 -- 172.21.15.66:0/2600111530 shutdown_connections 2024-09-10T22:11:48.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 -- 172.21.15.66:0/2600111530 wait complete. 2024-09-10T22:11:48.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 Processor -- start 2024-09-10T22:11:48.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 -- start start 2024-09-10T22:11:48.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b00b1e10 0x7f91b014e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:48.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b015f450 0x7f91b00c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:48.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b0160280 0x7f91b00cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:48.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f91b00afe80 con 0x7f91b015f450 2024-09-10T22:11:48.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91b00afd00 con 0x7f91b0160280 2024-09-10T22:11:48.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b8800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f91b0168eb0 con 0x7f91b00b1e10 2024-09-10T22:11:48.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b015f450 0x7f91b00c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:48.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b5e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b00b1e10 0x7f91b014e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:48.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b015f450 0x7f91b00c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55354/0 (socket says 172.21.15.66:55354) 2024-09-10T22:11:48.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b5400640 1 -- 172.21.15.66:0/1841381675 learned_addr learned my addr 172.21.15.66:0/1841381675 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:48.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b6800640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b0160280 0x7f91b00cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:48.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b5400640 1 -- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b00b1e10 msgr2=0x7f91b014e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:48.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b5400640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b00b1e10 0x7f91b014e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b5400640 1 -- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b0160280 msgr2=0x7f91b00cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:48.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b5400640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b0160280 0x7f91b00cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b5400640 1 -- 172.21.15.66:0/1841381675 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b00c8570 con 0x7f91b015f450 2024-09-10T22:11:48.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b5e00640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b00b1e10 0x7f91b014e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:11:48.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.712+0000 7f91b6800640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b0160280 0x7f91b00cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:11:48.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.716+0000 7f91b5400640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b015f450 0x7f91b00c8030 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009790 tx=0x7f91a0005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:48.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.716+0000 7f91aa000640 1 -- 172.21.15.66:0/1841381675 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a00bb070 con 0x7f91b015f450 2024-09-10T22:11:48.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.716+0000 7f91aa000640 1 -- 172.21.15.66:0/1841381675 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91a0004330 con 0x7f91b015f450 2024-09-10T22:11:48.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.716+0000 7f91aa000640 1 -- 172.21.15.66:0/1841381675 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a00b2040 con 0x7f91b015f450 2024-09-10T22:11:48.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.716+0000 7f91b8800640 1 -- 172.21.15.66:0/1841381675 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b014ec70 con 0x7f91b015f450 2024-09-10T22:11:48.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.716+0000 7f91b8800640 1 -- 172.21.15.66:0/1841381675 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b014f230 con 0x7f91b015f450 2024-09-10T22:11:48.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.716+0000 7f91b8800640 1 -- 172.21.15.66:0/1841381675 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b00aec10 con 0x7f91b015f450 2024-09-10T22:11:48.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.716+0000 7f91aa000640 1 -- 172.21.15.66:0/1841381675 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f91a00b5070 con 0x7f91b015f450 2024-09-10T22:11:48.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.720+0000 7f91aa000640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9194073f30 0x7f91940763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:48.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.720+0000 7f91aa000640 1 -- 172.21.15.66:0/1841381675 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5793+0+0 (secure 0 0 0) 0x7f91a012b5f0 con 0x7f91b015f450 2024-09-10T22:11:48.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.720+0000 7f91b5e00640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9194073f30 0x7f91940763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:48.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.720+0000 7f91b5e00640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9194073f30 0x7f91940763f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f91a4000f70 tx=0x7f91a4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:48.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.724+0000 7f91aa000640 1 -- 172.21.15.66:0/1841381675 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f91a0016410 con 0x7f91b015f450 2024-09-10T22:11:48.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.900+0000 7f91b8800640 1 -- 172.21.15.66:0/1841381675 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91b00d2a40 con 0x7f91b015f450 2024-09-10T22:11:48.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.900+0000 7f91aa000640 1 -- 172.21.15.66:0/1841381675 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16761 (secure 0 0 0) 0x7f91a00f80e0 con 0x7f91b015f450 2024-09-10T22:11:48.904 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:48.904 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":111,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:11:47.579878+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:11:45.291701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T22:11:46.544807+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:11:48.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.900+0000 7f91b8800640 1 -- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9194073f30 msgr2=0x7f91940763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:48.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.900+0000 7f91b8800640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9194073f30 0x7f91940763f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f91a4000f70 tx=0x7f91a4009040 comp rx=0 tx=0).stop 2024-09-10T22:11:48.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.900+0000 7f91b8800640 1 -- 172.21.15.66:0/1841381675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b015f450 msgr2=0x7f91b00c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:48.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.900+0000 7f91b8800640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b015f450 0x7f91b00c8030 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009790 tx=0x7f91a0005d20 comp rx=0 tx=0).stop 2024-09-10T22:11:48.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.900+0000 7f91b8800640 1 -- 172.21.15.66:0/1841381675 shutdown_connections 2024-09-10T22:11:48.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.900+0000 7f91b8800640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9194073f30 0x7f91940763f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.900+0000 7f91b8800640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b0160280 0x7f91b00cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.900+0000 7f91b8800640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b015f450 0x7f91b00c8030 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.900+0000 7f91b8800640 1 --2- 172.21.15.66:0/1841381675 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b00b1e10 0x7f91b014e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:48.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.900+0000 7f91b8800640 1 -- 172.21.15.66:0/1841381675 >> 172.21.15.66:0/1841381675 conn(0x7f91b00d1c60 msgr2=0x7f91b0154560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:48.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.904+0000 7f91b8800640 1 -- 172.21.15.66:0/1841381675 shutdown_connections 2024-09-10T22:11:48.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:48.904+0000 7f91b8800640 1 -- 172.21.15.66:0/1841381675 wait complete. 2024-09-10T22:11:48.926 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:11:49.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.072+0000 7f3c99e00640 1 -- 172.21.15.66:0/3459383524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c940aeec0 msgr2=0x7f3c940c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:49.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.072+0000 7f3c99e00640 1 --2- 172.21.15.66:0/3459383524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c940aeec0 0x7f3c940c4df0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c009c80 tx=0x7f3c8c01cd10 comp rx=0 tx=0).stop 2024-09-10T22:11:49.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.072+0000 7f3c99e00640 1 -- 172.21.15.66:0/3459383524 shutdown_connections 2024-09-10T22:11:49.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.072+0000 7f3c99e00640 1 --2- 172.21.15.66:0/3459383524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c940aeec0 0x7f3c940c4df0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:49.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.072+0000 7f3c99e00640 1 --2- 172.21.15.66:0/3459383524 >> v2:172.21.15.94:3300/0 conn(0x7f3c9415a310 0x7f3c940c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:49.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.072+0000 7f3c99e00640 1 -- 172.21.15.66:0/3459383524 >> 172.21.15.66:0/3459383524 conn(0x7f3c940cfa30 msgr2=0x7f3c940cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:49.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.072+0000 7f3c99e00640 1 -- 172.21.15.66:0/3459383524 shutdown_connections 2024-09-10T22:11:49.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.072+0000 7f3c99e00640 1 -- 172.21.15.66:0/3459383524 wait complete. 2024-09-10T22:11:49.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c99e00640 1 Processor -- start 2024-09-10T22:11:49.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c99e00640 1 -- start start 2024-09-10T22:11:49.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c99e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c940aeec0 0x7f3c9413fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:49.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c99e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c9415a310 0x7f3c9414fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:49.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c99e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c941faed0 0x7f3c941fb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:49.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c99e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9415ed60 con 0x7f3c940aeec0 2024-09-10T22:11:49.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c99e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9415ebe0 con 0x7f3c941faed0 2024-09-10T22:11:49.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c99e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3c94167300 con 0x7f3c9415a310 2024-09-10T22:11:49.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c92a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c940aeec0 0x7f3c9413fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:49.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c92a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c940aeec0 0x7f3c9413fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55368/0 (socket says 172.21.15.66:55368) 2024-09-10T22:11:49.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c92a00640 1 -- 172.21.15.66:0/213858566 learned_addr learned my addr 172.21.15.66:0/213858566 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:49.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c8be00640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c9415a310 0x7f3c9414fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:49.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c93400640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c941faed0 0x7f3c941fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:49.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c92a00640 1 -- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c9415a310 msgr2=0x7f3c9414fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:49.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c92a00640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c9415a310 0x7f3c9414fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:49.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c92a00640 1 -- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c941faed0 msgr2=0x7f3c941fb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:49.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c92a00640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c941faed0 0x7f3c941fb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:49.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c92a00640 1 -- 172.21.15.66:0/213858566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c941fbd20 con 0x7f3c940aeec0 2024-09-10T22:11:49.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c93400640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c941faed0 0x7f3c941fb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:11:49.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c92a00640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c940aeec0 0x7f3c9413fbf0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400ea90 tx=0x7f3c8400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:49.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c8b400640 1 -- 172.21.15.66:0/213858566 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8401b070 con 0x7f3c940aeec0 2024-09-10T22:11:49.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c8b400640 1 -- 172.21.15.66:0/213858566 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c8400ccb0 con 0x7f3c940aeec0 2024-09-10T22:11:49.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c99e00640 1 -- 172.21.15.66:0/213858566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c94151430 con 0x7f3c940aeec0 2024-09-10T22:11:49.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c8b400640 1 -- 172.21.15.66:0/213858566 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84016670 con 0x7f3c940aeec0 2024-09-10T22:11:49.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c99e00640 1 -- 172.21.15.66:0/213858566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9413f110 con 0x7f3c940aeec0 2024-09-10T22:11:49.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.076+0000 7f3c99e00640 1 -- 172.21.15.66:0/213858566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c94070420 con 0x7f3c940aeec0 2024-09-10T22:11:49.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.080+0000 7f3c8b400640 1 -- 172.21.15.66:0/213858566 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f3c8400b040 con 0x7f3c940aeec0 2024-09-10T22:11:49.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.080+0000 7f3c8b400640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c70073f60 0x7f3c70076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:49.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.080+0000 7f3c8b400640 1 -- 172.21.15.66:0/213858566 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5793+0+0 (secure 0 0 0) 0x7f3c840921c0 con 0x7f3c940aeec0 2024-09-10T22:11:49.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.080+0000 7f3c8be00640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c70073f60 0x7f3c70076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:49.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.080+0000 7f3c8be00640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c70073f60 0x7f3c70076420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3c94150a40 tx=0x7f3c7c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:49.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.084+0000 7f3c8b400640 1 -- 172.21.15.66:0/213858566 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3c8405ecb0 con 0x7f3c940aeec0 2024-09-10T22:11:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:49.296+0000 7f3c99e00640 1 -- 172.21.15.66:0/213858566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3c9406ef60 con 0x7f3c940aeec0 2024-09-10T22:11:50.142 INFO:tasks.ceph.mds.b.smithi094.stderr:2024-09-10T22:11:50.141+0000 7f6a9f000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:11:50.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.184+0000 7f3c8b400640 1 -- 172.21.15.66:0/213858566 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) ==== 118+0+0 (secure 0 0 0) 0x7f3c840636e0 con 0x7f3c940aeec0 2024-09-10T22:11:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.188+0000 7f3c99e00640 1 -- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c70073f60 msgr2=0x7f3c70076420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.188+0000 7f3c99e00640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c70073f60 0x7f3c70076420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3c94150a40 tx=0x7f3c7c009040 comp rx=0 tx=0).stop 2024-09-10T22:11:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.188+0000 7f3c99e00640 1 -- 172.21.15.66:0/213858566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c940aeec0 msgr2=0x7f3c9413fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.188+0000 7f3c99e00640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c940aeec0 0x7f3c9413fbf0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400ea90 tx=0x7f3c8400ef50 comp rx=0 tx=0).stop 2024-09-10T22:11:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.188+0000 7f3c99e00640 1 -- 172.21.15.66:0/213858566 shutdown_connections 2024-09-10T22:11:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.188+0000 7f3c99e00640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c70073f60 0x7f3c70076420 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.188+0000 7f3c99e00640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c941faed0 0x7f3c941fb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.188+0000 7f3c99e00640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c9415a310 0x7f3c9414fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.188+0000 7f3c99e00640 1 --2- 172.21.15.66:0/213858566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c940aeec0 0x7f3c9413fbf0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.188+0000 7f3c99e00640 1 -- 172.21.15.66:0/213858566 >> 172.21.15.66:0/213858566 conn(0x7f3c940cfa30 msgr2=0x7f3c940cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.188+0000 7f3c99e00640 1 -- 172.21.15.66:0/213858566 shutdown_connections 2024-09-10T22:11:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.188+0000 7f3c99e00640 1 -- 172.21.15.66:0/213858566 wait complete. 2024-09-10T22:11:50.213 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:11:50.213 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:11:50.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.360+0000 7f011d400640 1 -- 172.21.15.66:0/3376250803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01180aebf0 msgr2=0x7f01180cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:50.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.360+0000 7f011d400640 1 --2- 172.21.15.66:0/3376250803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01180aebf0 0x7f01180cde70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0110009c80 tx=0x7f011001cd70 comp rx=0 tx=0).stop 2024-09-10T22:11:50.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.360+0000 7f011d400640 1 -- 172.21.15.66:0/3376250803 shutdown_connections 2024-09-10T22:11:50.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.360+0000 7f011d400640 1 --2- 172.21.15.66:0/3376250803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01180aebf0 0x7f01180cde70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.360+0000 7f011d400640 1 --2- 172.21.15.66:0/3376250803 >> v2:172.21.15.94:3300/0 conn(0x7f01180ae7b0 0x7f01180cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.360+0000 7f011d400640 1 -- 172.21.15.66:0/3376250803 >> 172.21.15.66:0/3376250803 conn(0x7f01180cfa30 msgr2=0x7f01180cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.360+0000 7f011d400640 1 -- 172.21.15.66:0/3376250803 shutdown_connections 2024-09-10T22:11:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.360+0000 7f011d400640 1 -- 172.21.15.66:0/3376250803 wait complete. 2024-09-10T22:11:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.360+0000 7f011d400640 1 Processor -- start 2024-09-10T22:11:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.360+0000 7f011d400640 1 -- start start 2024-09-10T22:11:50.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f011d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01180ae7b0 0x7f01180b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:50.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f011d400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01180aebf0 0x7f01180b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:50.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f011d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01181447b0 0x7f011813e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:50.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f011d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f011816ad90 con 0x7f01181447b0 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f011d400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f011816ac10 con 0x7f01180ae7b0 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f011d400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f011816af10 con 0x7f01180aebf0 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f0117200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01181447b0 0x7f011813e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f0117200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01181447b0 0x7f011813e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55370/0 (socket says 172.21.15.66:55370) 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f0117200640 1 -- 172.21.15.66:0/2715468085 learned_addr learned my addr 172.21.15.66:0/2715468085 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f0116800640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01180ae7b0 0x7f01180b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f010de00640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01180aebf0 0x7f01180b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f0117200640 1 -- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01180aebf0 msgr2=0x7f01180b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f0117200640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01180aebf0 0x7f01180b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f0117200640 1 -- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01180ae7b0 msgr2=0x7f01180b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f0117200640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01180ae7b0 0x7f01180b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f0117200640 1 -- 172.21.15.66:0/2715468085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01180b2f90 con 0x7f01181447b0 2024-09-10T22:11:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f0117200640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01181447b0 0x7f011813e300 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f0110098980 tx=0x7f0110005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:50.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f010f400640 1 -- 172.21.15.66:0/2715468085 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01100b2040 con 0x7f01181447b0 2024-09-10T22:11:50.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f010f400640 1 -- 172.21.15.66:0/2715468085 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01100bb070 con 0x7f01181447b0 2024-09-10T22:11:50.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f010f400640 1 -- 172.21.15.66:0/2715468085 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110004240 con 0x7f01181447b0 2024-09-10T22:11:50.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f011d400640 1 -- 172.21.15.66:0/2715468085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0118073600 con 0x7f01181447b0 2024-09-10T22:11:50.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f011d400640 1 -- 172.21.15.66:0/2715468085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011813e840 con 0x7f01181447b0 2024-09-10T22:11:50.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.364+0000 7f011d400640 1 -- 172.21.15.66:0/2715468085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011815b6c0 con 0x7f01181447b0 2024-09-10T22:11:50.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.368+0000 7f010f400640 1 -- 172.21.15.66:0/2715468085 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f01100b5070 con 0x7f01181447b0 2024-09-10T22:11:50.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.368+0000 7f010f400640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f00f4073f30 0x7f00f40763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:50.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.368+0000 7f010f400640 1 -- 172.21.15.66:0/2715468085 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f011012c750 con 0x7f01181447b0 2024-09-10T22:11:50.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.368+0000 7f0116800640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f00f4073f30 0x7f00f40763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:50.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.368+0000 7f0116800640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f00f4073f30 0x7f00f40763f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0108001640 tx=0x7f0108009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:50.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.372+0000 7f010f400640 1 -- 172.21.15.66:0/2715468085 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0110016990 con 0x7f01181447b0 2024-09-10T22:11:50.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:50.580+0000 7f011d400640 1 -- 172.21.15.66:0/2715468085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f011815b3e0 con 0x7f01181447b0 2024-09-10T22:11:51.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.188+0000 7f010f400640 1 -- 172.21.15.66:0/2715468085 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) ==== 122+0+0 (secure 0 0 0) 0x7f01100f9890 con 0x7f01181447b0 2024-09-10T22:11:51.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.192+0000 7f011d400640 1 -- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f00f4073f30 msgr2=0x7f00f40763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:51.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.192+0000 7f011d400640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f00f4073f30 0x7f00f40763f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0108001640 tx=0x7f0108009040 comp rx=0 tx=0).stop 2024-09-10T22:11:51.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.192+0000 7f011d400640 1 -- 172.21.15.66:0/2715468085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01181447b0 msgr2=0x7f011813e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:51.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.192+0000 7f011d400640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01181447b0 0x7f011813e300 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f0110098980 tx=0x7f0110005c40 comp rx=0 tx=0).stop 2024-09-10T22:11:51.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.192+0000 7f011d400640 1 -- 172.21.15.66:0/2715468085 shutdown_connections 2024-09-10T22:11:51.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.192+0000 7f011d400640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f00f4073f30 0x7f00f40763f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.192+0000 7f011d400640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01181447b0 0x7f011813e300 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.192+0000 7f011d400640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01180aebf0 0x7f01180b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.192+0000 7f011d400640 1 --2- 172.21.15.66:0/2715468085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01180ae7b0 0x7f01180b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.192+0000 7f011d400640 1 -- 172.21.15.66:0/2715468085 >> 172.21.15.66:0/2715468085 conn(0x7f01180cfa30 msgr2=0x7f0118161a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:51.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.192+0000 7f011d400640 1 -- 172.21.15.66:0/2715468085 shutdown_connections 2024-09-10T22:11:51.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.192+0000 7f011d400640 1 -- 172.21.15.66:0/2715468085 wait complete. 2024-09-10T22:11:51.218 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:11:51.218 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:11:51.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.364+0000 7fdcdcc00640 1 -- 172.21.15.66:0/3172977124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcd80c4fc0 msgr2=0x7fdcd806e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:51.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.364+0000 7fdcdcc00640 1 --2- 172.21.15.66:0/3172977124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcd80c4fc0 0x7fdcd806e630 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fdcd815b510 tx=0x7fdcc001dbc0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.364+0000 7fdcdcc00640 1 -- 172.21.15.66:0/3172977124 shutdown_connections 2024-09-10T22:11:51.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.364+0000 7fdcdcc00640 1 --2- 172.21.15.66:0/3172977124 >> v2:172.21.15.94:3300/0 conn(0x7fdcd815e130 0x7fdcd806eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.364+0000 7fdcdcc00640 1 --2- 172.21.15.66:0/3172977124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcd80c4fc0 0x7fdcd806e630 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.364+0000 7fdcdcc00640 1 -- 172.21.15.66:0/3172977124 >> 172.21.15.66:0/3172977124 conn(0x7fdcd80cfa30 msgr2=0x7fdcd80cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:51.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.364+0000 7fdcdcc00640 1 -- 172.21.15.66:0/3172977124 shutdown_connections 2024-09-10T22:11:51.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.364+0000 7fdcdcc00640 1 -- 172.21.15.66:0/3172977124 wait complete. 2024-09-10T22:11:51.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.364+0000 7fdcdcc00640 1 Processor -- start 2024-09-10T22:11:51.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.364+0000 7fdcdcc00640 1 -- start start 2024-09-10T22:11:51.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcdcc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcd80c4fc0 0x7fdcd80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:51.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcdcc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcd815dd50 0x7fdcd80b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:51.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcdcc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcd815e130 0x7fdcd80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:51.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcdcc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd8166d10 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcdcc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd8166b90 con 0x7fdcd815dd50 2024-09-10T22:11:51.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcdcc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd8166e90 con 0x7fdcd815e130 2024-09-10T22:11:51.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcd5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcd80c4fc0 0x7fdcd80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:51.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcd5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcd80c4fc0 0x7fdcd80cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55388/0 (socket says 172.21.15.66:55388) 2024-09-10T22:11:51.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcd5e00640 1 -- 172.21.15.66:0/2212393559 learned_addr learned my addr 172.21.15.66:0/2212393559 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:51.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdccfe00640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcd815dd50 0x7fdcd80b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:51.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcd6800640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcd815e130 0x7fdcd80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:51.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcd5e00640 1 -- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcd815e130 msgr2=0x7fdcd80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:51.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcd5e00640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcd815e130 0x7fdcd80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcd5e00640 1 -- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcd815dd50 msgr2=0x7fdcd80b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:51.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcd5e00640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcd815dd50 0x7fdcd80b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcd5e00640 1 -- 172.21.15.66:0/2212393559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd8152140 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcd6800640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcd815e130 0x7fdcd80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:51.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcd5e00640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcd80c4fc0 0x7fdcd80cf280 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800c500 tx=0x7fdcc800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:51.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcce000640 1 -- 172.21.15.66:0/2212393559 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8017070 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcce000640 1 -- 172.21.15.66:0/2212393559 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcc80053c0 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcce000640 1 -- 172.21.15.66:0/2212393559 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8003860 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcdcc00640 1 -- 172.21.15.66:0/2212393559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd8152420 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcdcc00640 1 -- 172.21.15.66:0/2212393559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd81529e0 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcdcc00640 1 -- 172.21.15.66:0/2212393559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd80c48b0 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.368+0000 7fdcce000640 1 -- 172.21.15.66:0/2212393559 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fdcc8003a00 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.372+0000 7fdcce000640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdcb4073f30 0x7fdcb40763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:51.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.372+0000 7fdcce000640 1 -- 172.21.15.66:0/2212393559 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fdcc80917b0 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.372+0000 7fdccfe00640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdcb4073f30 0x7fdcb40763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:51.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.372+0000 7fdccfe00640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdcb4073f30 0x7fdcb40763f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00b0000 tx=0x7fdcc001c400 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:51.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.376+0000 7fdcce000640 1 -- 172.21.15.66:0/2212393559 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdcc805e220 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 -- 172.21.15.66:0/2212393559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcd80ae2e0 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcce000640 1 -- 172.21.15.66:0/2212393559 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5222 (secure 0 0 0) 0x7fdcc8062c50 con 0x7fdcd80c4fc0 2024-09-10T22:11:51.587 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:51.587 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":76,"btime":"2024-09-10T22:11:51:187007+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1769669365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1769669365},{"type":"v1","addr":"172.21.15.94:6836","nonce":1769669365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8301,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/298680158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":298680158},{"type":"v1","addr":"172.21.15.66:6835","nonce":298680158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1808370701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1808370701},{"type":"v1","addr":"172.21.15.66:6834","nonce":1808370701}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:11:47.579843+0000","modified":"2024-09-10T22:11:50.583212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8307":{"gid":8307,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/3818206155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3818206155},{"type":"v1","addr":"172.21.15.94:6837","nonce":3818206155}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8307,"qdb_cluster":[8307]},"id":8}]} 2024-09-10T22:11:51.588 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 76 2024-09-10T22:11:51.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 -- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdcb4073f30 msgr2=0x7fdcb40763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:51.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdcb4073f30 0x7fdcb40763f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00b0000 tx=0x7fdcc001c400 comp rx=0 tx=0).stop 2024-09-10T22:11:51.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 -- 172.21.15.66:0/2212393559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcd80c4fc0 msgr2=0x7fdcd80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:51.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcd80c4fc0 0x7fdcd80cf280 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800c500 tx=0x7fdcc800c9c0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 -- 172.21.15.66:0/2212393559 shutdown_connections 2024-09-10T22:11:51.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdcb4073f30 0x7fdcb40763f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcd815e130 0x7fdcd80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcd815dd50 0x7fdcd80b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 --2- 172.21.15.66:0/2212393559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcd80c4fc0 0x7fdcd80cf280 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 -- 172.21.15.66:0/2212393559 >> 172.21.15.66:0/2212393559 conn(0x7fdcd80cfa30 msgr2=0x7fdcd8073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:51.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 -- 172.21.15.66:0/2212393559 shutdown_connections 2024-09-10T22:11:51.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.584+0000 7fdcdcc00640 1 -- 172.21.15.66:0/2212393559 wait complete. 2024-09-10T22:11:51.609 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:11:51.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.752+0000 7f1bb6a00640 1 -- 172.21.15.66:0/3045020638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1bb00aee20 msgr2=0x7f1bb00cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:51.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.752+0000 7f1bb6a00640 1 --2- 172.21.15.66:0/3045020638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1bb00aee20 0x7f1bb00cd930 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0071740 tx=0x7f1ba801ccf0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.752+0000 7f1bb6a00640 1 -- 172.21.15.66:0/3045020638 shutdown_connections 2024-09-10T22:11:51.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.752+0000 7f1bb6a00640 1 --2- 172.21.15.66:0/3045020638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1bb00aee20 0x7f1bb00cd930 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.752+0000 7f1bb6a00640 1 --2- 172.21.15.66:0/3045020638 >> v2:172.21.15.94:3300/0 conn(0x7f1bb0159900 0x7f1bb00ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.752+0000 7f1bb6a00640 1 -- 172.21.15.66:0/3045020638 >> 172.21.15.66:0/3045020638 conn(0x7f1bb00cfa30 msgr2=0x7f1bb00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:51.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.752+0000 7f1bb6a00640 1 -- 172.21.15.66:0/3045020638 shutdown_connections 2024-09-10T22:11:51.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.752+0000 7f1bb6a00640 1 -- 172.21.15.66:0/3045020638 wait complete. 2024-09-10T22:11:51.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.752+0000 7f1bb6a00640 1 Processor -- start 2024-09-10T22:11:51.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.752+0000 7f1bb6a00640 1 -- start start 2024-09-10T22:11:51.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bb6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1bb00aee20 0x7f1bb00ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:51.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bb6a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1bb0159900 0x7f1bb014e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:51.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bb6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bb015cc80 0x7f1bb0153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:51.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bb6a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0168f70 con 0x7f1bb00aee20 2024-09-10T22:11:51.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bb6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0168df0 con 0x7f1bb015cc80 2024-09-10T22:11:51.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bb6a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb01690f0 con 0x7f1bb0159900 2024-09-10T22:11:51.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bafe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1bb00aee20 0x7f1bb00ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:51.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bafe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1bb00aee20 0x7f1bb00ae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55398/0 (socket says 172.21.15.66:55398) 2024-09-10T22:11:51.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bafe00640 1 -- 172.21.15.66:0/665422250 learned_addr learned my addr 172.21.15.66:0/665422250 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:51.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1ba7400640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1bb0159900 0x7f1bb014e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:51.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bb4a00640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bb015cc80 0x7f1bb0153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:51.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bafe00640 1 -- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1bb0159900 msgr2=0x7f1bb014e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:51.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bafe00640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1bb0159900 0x7f1bb014e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bafe00640 1 -- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bb015cc80 msgr2=0x7f1bb0153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:51.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bafe00640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bb015cc80 0x7f1bb0153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bafe00640 1 -- 172.21.15.66:0/665422250 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb014eab0 con 0x7f1bb00aee20 2024-09-10T22:11:51.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bafe00640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1bb00aee20 0x7f1bb00ae710 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000d950 tx=0x7f1ba000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:51.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bacc00640 1 -- 172.21.15.66:0/665422250 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0017070 con 0x7f1bb00aee20 2024-09-10T22:11:51.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bacc00640 1 -- 172.21.15.66:0/665422250 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ba0009d20 con 0x7f1bb00aee20 2024-09-10T22:11:51.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bacc00640 1 -- 172.21.15.66:0/665422250 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba00106c0 con 0x7f1bb00aee20 2024-09-10T22:11:51.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bb6a00640 1 -- 172.21.15.66:0/665422250 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb0154e30 con 0x7f1bb00aee20 2024-09-10T22:11:51.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bb6a00640 1 -- 172.21.15.66:0/665422250 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb01553f0 con 0x7f1bb00aee20 2024-09-10T22:11:51.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.756+0000 7f1bb6a00640 1 -- 172.21.15.66:0/665422250 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb00cd930 con 0x7f1bb00aee20 2024-09-10T22:11:51.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.760+0000 7f1bacc00640 1 -- 172.21.15.66:0/665422250 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f1ba0010860 con 0x7f1bb00aee20 2024-09-10T22:11:51.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.760+0000 7f1bacc00640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b8c073f30 0x7f1b8c0763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:51.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.760+0000 7f1bacc00640 1 -- 172.21.15.66:0/665422250 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f1ba0092380 con 0x7f1bb00aee20 2024-09-10T22:11:51.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.760+0000 7f1ba7400640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b8c073f30 0x7f1b8c0763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:51.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.760+0000 7f1ba7400640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b8c073f30 0x7f1b8c0763f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1bb014eea0 tx=0x7f1b98009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:51.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.764+0000 7f1bacc00640 1 -- 172.21.15.66:0/665422250 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ba005ef10 con 0x7f1bb00aee20 2024-09-10T22:11:51.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.940+0000 7f1bb6a00640 1 -- 172.21.15.66:0/665422250 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bb014ca90 con 0x7f1bb00aee20 2024-09-10T22:11:51.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.940+0000 7f1bacc00640 1 -- 172.21.15.66:0/665422250 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16748 (secure 0 0 0) 0x7f1ba0063940 con 0x7f1bb00aee20 2024-09-10T22:11:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:51.945 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":112,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:11:49.173366+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:11:45.291701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T22:11:46.544807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:11:51.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.944+0000 7f1bb6a00640 1 -- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b8c073f30 msgr2=0x7f1b8c0763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:51.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.944+0000 7f1bb6a00640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b8c073f30 0x7f1b8c0763f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1bb014eea0 tx=0x7f1b98009060 comp rx=0 tx=0).stop 2024-09-10T22:11:51.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.944+0000 7f1bb6a00640 1 -- 172.21.15.66:0/665422250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1bb00aee20 msgr2=0x7f1bb00ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:51.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.944+0000 7f1bb6a00640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1bb00aee20 0x7f1bb00ae710 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000d950 tx=0x7f1ba000de10 comp rx=0 tx=0).stop 2024-09-10T22:11:51.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.944+0000 7f1bb6a00640 1 -- 172.21.15.66:0/665422250 shutdown_connections 2024-09-10T22:11:51.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.944+0000 7f1bb6a00640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1b8c073f30 0x7f1b8c0763f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.944+0000 7f1bb6a00640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bb015cc80 0x7f1bb0153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.944+0000 7f1bb6a00640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1bb0159900 0x7f1bb014e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.944+0000 7f1bb6a00640 1 --2- 172.21.15.66:0/665422250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1bb00aee20 0x7f1bb00ae710 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:51.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.944+0000 7f1bb6a00640 1 -- 172.21.15.66:0/665422250 >> 172.21.15.66:0/665422250 conn(0x7f1bb00cfa30 msgr2=0x7f1bb006fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:51.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.944+0000 7f1bb6a00640 1 -- 172.21.15.66:0/665422250 shutdown_connections 2024-09-10T22:11:51.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:51.944+0000 7f1bb6a00640 1 -- 172.21.15.66:0/665422250 wait complete. 2024-09-10T22:11:51.966 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:11:51.966 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:11:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.108+0000 7f3a7c000640 1 -- 172.21.15.66:0/386595740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a740d06b0 msgr2=0x7f3a740ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.108+0000 7f3a7c000640 1 --2- 172.21.15.66:0/386595740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a740d06b0 0x7f3a740ce170 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f3a740d1940 tx=0x7f3a6001c830 comp rx=0 tx=0).stop 2024-09-10T22:11:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.108+0000 7f3a7c000640 1 -- 172.21.15.66:0/386595740 shutdown_connections 2024-09-10T22:11:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.108+0000 7f3a7c000640 1 --2- 172.21.15.66:0/386595740 >> v2:172.21.15.94:3300/0 conn(0x7f3a740afc20 0x7f3a740ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.108+0000 7f3a7c000640 1 --2- 172.21.15.66:0/386595740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a740d06b0 0x7f3a740ce170 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.108+0000 7f3a7c000640 1 -- 172.21.15.66:0/386595740 >> 172.21.15.66:0/386595740 conn(0x7f3a740d1c60 msgr2=0x7f3a740d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:52.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.108+0000 7f3a7c000640 1 -- 172.21.15.66:0/386595740 shutdown_connections 2024-09-10T22:11:52.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.108+0000 7f3a7c000640 1 -- 172.21.15.66:0/386595740 wait complete. 2024-09-10T22:11:52.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7c000640 1 Processor -- start 2024-09-10T22:11:52.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7c000640 1 -- start start 2024-09-10T22:11:52.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7c000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a740afc20 0x7f3a740b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:52.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7c000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a740d06b0 0x7f3a741f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:52.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7c000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a7415bd40 0x7f3a7406f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:52.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7c000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74169250 con 0x7f3a740d06b0 2024-09-10T22:11:52.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7c000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a741690d0 con 0x7f3a740afc20 2024-09-10T22:11:52.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7c000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3a741693d0 con 0x7f3a7415bd40 2024-09-10T22:11:52.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a78c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a740d06b0 0x7f3a741f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:52.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a78c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a740d06b0 0x7f3a741f7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55404/0 (socket says 172.21.15.66:55404) 2024-09-10T22:11:52.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a78c00640 1 -- 172.21.15.66:0/1892962601 learned_addr learned my addr 172.21.15.66:0/1892962601 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:52.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7a000640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a7415bd40 0x7f3a7406f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:52.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a79600640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a740afc20 0x7f3a740b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:52.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a78c00640 1 -- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a7415bd40 msgr2=0x7f3a7406f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a78c00640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a7415bd40 0x7f3a7406f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a78c00640 1 -- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a740afc20 msgr2=0x7f3a740b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a78c00640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a740afc20 0x7f3a740b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a78c00640 1 -- 172.21.15.66:0/1892962601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a741f85d0 con 0x7f3a740d06b0 2024-09-10T22:11:52.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7a000640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a7415bd40 0x7f3a7406f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:11:52.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a79600640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a740afc20 0x7f3a740b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:11:52.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a78c00640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a740d06b0 0x7f3a741f7e30 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f3a600b0040 tx=0x7f3a600a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:52.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a71600640 1 -- 172.21.15.66:0/1892962601 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a600b2040 con 0x7f3a740d06b0 2024-09-10T22:11:52.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a71600640 1 -- 172.21.15.66:0/1892962601 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a60005020 con 0x7f3a740d06b0 2024-09-10T22:11:52.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a71600640 1 -- 172.21.15.66:0/1892962601 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60005470 con 0x7f3a740d06b0 2024-09-10T22:11:52.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7c000640 1 -- 172.21.15.66:0/1892962601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a741f88b0 con 0x7f3a740d06b0 2024-09-10T22:11:52.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7c000640 1 -- 172.21.15.66:0/1892962601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7415a950 con 0x7f3a740d06b0 2024-09-10T22:11:52.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.112+0000 7f3a7c000640 1 -- 172.21.15.66:0/1892962601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a741f8dd0 con 0x7f3a740d06b0 2024-09-10T22:11:52.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.116+0000 7f3a71600640 1 -- 172.21.15.66:0/1892962601 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f3a600046d0 con 0x7f3a740d06b0 2024-09-10T22:11:52.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.116+0000 7f3a71600640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3a54073f60 0x7f3a54076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:52.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.116+0000 7f3a71600640 1 -- 172.21.15.66:0/1892962601 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f3a6012bb40 con 0x7f3a740d06b0 2024-09-10T22:11:52.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.116+0000 7f3a79600640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3a54073f60 0x7f3a54076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:52.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.116+0000 7f3a79600640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3a54073f60 0x7f3a54076420 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3a74166720 tx=0x7f3a68009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:52.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.120+0000 7f3a71600640 1 -- 172.21.15.66:0/1892962601 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3a741f8dd0 con 0x7f3a740d06b0 2024-09-10T22:11:52.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.292+0000 7f3a7c000640 1 -- 172.21.15.66:0/1892962601 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a741f8370 con 0x7f3a54073f60 2024-09-10T22:11:52.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.296+0000 7f3a71600640 1 -- 172.21.15.66:0/1892962601 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246265 (secure 0 0 0) 0x7f3a68001c60 con 0x7f3a54073f60 2024-09-10T22:11:52.302 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:52.303 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:11:52.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.300+0000 7f3a7c000640 1 -- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3a54073f60 msgr2=0x7f3a54076420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.300+0000 7f3a7c000640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3a54073f60 0x7f3a54076420 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3a74166720 tx=0x7f3a68009040 comp rx=0 tx=0).stop 2024-09-10T22:11:52.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.300+0000 7f3a7c000640 1 -- 172.21.15.66:0/1892962601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a740d06b0 msgr2=0x7f3a741f7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.300+0000 7f3a7c000640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a740d06b0 0x7f3a741f7e30 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f3a600b0040 tx=0x7f3a600a5710 comp rx=0 tx=0).stop 2024-09-10T22:11:52.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.300+0000 7f3a7c000640 1 -- 172.21.15.66:0/1892962601 shutdown_connections 2024-09-10T22:11:52.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.300+0000 7f3a7c000640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3a54073f60 0x7f3a54076420 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.300+0000 7f3a7c000640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a7415bd40 0x7f3a7406f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.300+0000 7f3a7c000640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a740d06b0 0x7f3a741f7e30 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.300+0000 7f3a7c000640 1 --2- 172.21.15.66:0/1892962601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a740afc20 0x7f3a740b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.300+0000 7f3a7c000640 1 -- 172.21.15.66:0/1892962601 >> 172.21.15.66:0/1892962601 conn(0x7f3a740d1c60 msgr2=0x7f3a740b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:52.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.300+0000 7f3a7c000640 1 -- 172.21.15.66:0/1892962601 shutdown_connections 2024-09-10T22:11:52.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.300+0000 7f3a7c000640 1 -- 172.21.15.66:0/1892962601 wait complete. 2024-09-10T22:11:52.323 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-09-10T22:11:50.561121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":190,"acting":190,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1047876,"kb_used_data":11712,"kb_used_omap":12,"kb_used_meta":1035955,"kb_avail":753926844,"statfs":{"total":773094113280,"available":772021088256,"internally_reserved":0,"allocated":11993088,"data_stored":5854760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1060818484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.404141"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.175610+0000","last_change":"2024-09-10T22:11:47.175610+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.175610+0000","last_undegraded":"2024-09-10T22:11:47.175610+0000","last_fullsized":"2024-09-10T22:11:47.175610+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.168181+0000","last_change":"2024-09-10T22:11:47.168181+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.168181+0000","last_undegraded":"2024-09-10T22:11:47.168181+0000","last_fullsized":"2024-09-10T22:11:47.168181+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.177096+0000","last_change":"2024-09-10T22:11:47.177096+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.177096+0000","last_undegraded":"2024-09-10T22:11:47.177096+0000","last_fullsized":"2024-09-10T22:11:47.177096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173208+0000","last_change":"2024-09-10T22:11:47.173208+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173208+0000","last_undegraded":"2024-09-10T22:11:47.173208+0000","last_fullsized":"2024-09-10T22:11:47.173208+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:11:47.151613+0000","last_change":"2024-09-10T22:11:47.151613+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.151613+0000","last_undegraded":"2024-09-10T22:11:47.151613+0000","last_fullsized":"2024-09-10T22:11:47.151613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.175411+0000","last_change":"2024-09-10T22:11:47.175411+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.175411+0000","last_undegraded":"2024-09-10T22:11:47.175411+0000","last_fullsized":"2024-09-10T22:11:47.175411+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.171416+0000","last_change":"2024-09-10T22:11:47.171416+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.171416+0000","last_undegraded":"2024-09-10T22:11:47.171416+0000","last_fullsized":"2024-09-10T22:11:47.171416+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174226+0000","last_change":"2024-09-10T22:11:47.174226+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174226+0000","last_undegraded":"2024-09-10T22:11:47.174226+0000","last_fullsized":"2024-09-10T22:11:47.174226+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.171760+0000","last_change":"2024-09-10T22:11:47.171760+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.171760+0000","last_undegraded":"2024-09-10T22:11:47.171760+0000","last_fullsized":"2024-09-10T22:11:47.171760+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.171631+0000","last_change":"2024-09-10T22:11:47.171631+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.171631+0000","last_undegraded":"2024-09-10T22:11:47.171631+0000","last_fullsized":"2024-09-10T22:11:47.171631+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.167706+0000","last_change":"2024-09-10T22:11:47.167706+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.167706+0000","last_undegraded":"2024-09-10T22:11:47.167706+0000","last_fullsized":"2024-09-10T22:11:47.167706+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.172709+0000","last_change":"2024-09-10T22:11:47.172709+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.172709+0000","last_undegraded":"2024-09-10T22:11:47.172709+0000","last_fullsized":"2024-09-10T22:11:47.172709+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.175144+0000","last_change":"2024-09-10T22:11:47.175144+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.175144+0000","last_undegraded":"2024-09-10T22:11:47.175144+0000","last_fullsized":"2024-09-10T22:11:47.175144+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174758+0000","last_change":"2024-09-10T22:11:47.174758+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174758+0000","last_undegraded":"2024-09-10T22:11:47.174758+0000","last_fullsized":"2024-09-10T22:11:47.174758+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.176173+0000","last_change":"2024-09-10T22:11:47.176173+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.176173+0000","last_undegraded":"2024-09-10T22:11:47.176173+0000","last_fullsized":"2024-09-10T22:11:47.176173+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173368+0000","last_change":"2024-09-10T22:11:47.173368+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173368+0000","last_undegraded":"2024-09-10T22:11:47.173368+0000","last_fullsized":"2024-09-10T22:11:47.173368+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.169164+0000","last_change":"2024-09-10T22:11:47.169164+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.169164+0000","last_undegraded":"2024-09-10T22:11:47.169164+0000","last_fullsized":"2024-09-10T22:11:47.169164+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.168691+0000","last_change":"2024-09-10T22:11:47.168691+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.168691+0000","last_undegraded":"2024-09-10T22:11:47.168691+0000","last_fullsized":"2024-09-10T22:11:47.168691+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.176608+0000","last_change":"2024-09-10T22:11:47.176608+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.176608+0000","last_undegraded":"2024-09-10T22:11:47.176608+0000","last_fullsized":"2024-09-10T22:11:47.176608+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.166993+0000","last_change":"2024-09-10T22:11:47.166993+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.166993+0000","last_undegraded":"2024-09-10T22:11:47.166993+0000","last_fullsized":"2024-09-10T22:11:47.166993+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.169572+0000","last_change":"2024-09-10T22:11:47.169572+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.169572+0000","last_undegraded":"2024-09-10T22:11:47.169572+0000","last_fullsized":"2024-09-10T22:11:47.169572+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.178773+0000","last_change":"2024-09-10T22:11:47.178773+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.178773+0000","last_undegraded":"2024-09-10T22:11:47.178773+0000","last_fullsized":"2024-09-10T22:11:47.178773+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.167011+0000","last_change":"2024-09-10T22:11:47.167011+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.167011+0000","last_undegraded":"2024-09-10T22:11:47.167011+0000","last_fullsized":"2024-09-10T22:11:47.167011+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.177612+0000","last_change":"2024-09-10T22:11:47.177612+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.177612+0000","last_undegraded":"2024-09-10T22:11:47.177612+0000","last_fullsized":"2024-09-10T22:11:47.177612+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.168721+0000","last_change":"2024-09-10T22:11:47.168721+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.168721+0000","last_undegraded":"2024-09-10T22:11:47.168721+0000","last_fullsized":"2024-09-10T22:11:47.168721+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.171243+0000","last_change":"2024-09-10T22:11:47.171243+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.171243+0000","last_undegraded":"2024-09-10T22:11:47.171243+0000","last_fullsized":"2024-09-10T22:11:47.171243+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.175911+0000","last_change":"2024-09-10T22:11:47.175911+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.175911+0000","last_undegraded":"2024-09-10T22:11:47.175911+0000","last_fullsized":"2024-09-10T22:11:47.175911+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.177895+0000","last_change":"2024-09-10T22:11:47.177895+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.177895+0000","last_undegraded":"2024-09-10T22:11:47.177895+0000","last_fullsized":"2024-09-10T22:11:47.177895+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.170983+0000","last_change":"2024-09-10T22:11:47.170983+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.170983+0000","last_undegraded":"2024-09-10T22:11:47.170983+0000","last_fullsized":"2024-09-10T22:11:47.170983+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.171600+0000","last_change":"2024-09-10T22:11:47.171600+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.171600+0000","last_undegraded":"2024-09-10T22:11:47.171600+0000","last_fullsized":"2024-09-10T22:11:47.171600+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173576+0000","last_change":"2024-09-10T22:11:47.173576+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173576+0000","last_undegraded":"2024-09-10T22:11:47.173576+0000","last_fullsized":"2024-09-10T22:11:47.173576+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.176468+0000","last_change":"2024-09-10T22:11:47.176468+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.176468+0000","last_undegraded":"2024-09-10T22:11:47.176468+0000","last_fullsized":"2024-09-10T22:11:47.176468+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.175560+0000","last_change":"2024-09-10T22:11:47.175560+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.175560+0000","last_undegraded":"2024-09-10T22:11:47.175560+0000","last_fullsized":"2024-09-10T22:11:47.175560+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.180743+0000","last_change":"2024-09-10T22:11:47.179953+0000","last_active":"2024-09-10T22:11:47.180743+0000","last_peered":"2024-09-10T22:11:47.180743+0000","last_clean":"2024-09-10T22:11:47.180743+0000","last_became_active":"2024-09-10T22:11:47.179653+0000","last_became_peered":"2024-09-10T22:11:47.179653+0000","last_unstale":"2024-09-10T22:11:47.180743+0000","last_undegraded":"2024-09-10T22:11:47.180743+0000","last_fullsized":"2024-09-10T22:11:47.180743+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:57.020788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.169807+0000","last_change":"2024-09-10T22:11:47.169807+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.169807+0000","last_undegraded":"2024-09-10T22:11:47.169807+0000","last_fullsized":"2024-09-10T22:11:47.169807+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.185431+0000","last_change":"2024-09-10T22:11:47.184730+0000","last_active":"2024-09-10T22:11:47.185431+0000","last_peered":"2024-09-10T22:11:47.185431+0000","last_clean":"2024-09-10T22:11:47.185431+0000","last_became_active":"2024-09-10T22:11:47.184361+0000","last_became_peered":"2024-09-10T22:11:47.184361+0000","last_unstale":"2024-09-10T22:11:47.185431+0000","last_undegraded":"2024-09-10T22:11:47.185431+0000","last_fullsized":"2024-09-10T22:11:47.185431+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:19.251040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.176091+0000","last_change":"2024-09-10T22:11:47.176091+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.176091+0000","last_undegraded":"2024-09-10T22:11:47.176091+0000","last_fullsized":"2024-09-10T22:11:47.176091+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-09-10T22:11:46.154376+0000","last_change":"2024-09-10T22:11:46.154376+0000","last_active":"2024-09-10T22:11:46.148666+0000","last_peered":"2024-09-10T22:11:46.148666+0000","last_clean":"2024-09-10T22:11:46.148666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:46.154376+0000","last_undegraded":"2024-09-10T22:11:46.154376+0000","last_fullsized":"2024-09-10T22:11:46.154376+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174368+0000","last_change":"2024-09-10T22:11:47.174368+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174368+0000","last_undegraded":"2024-09-10T22:11:47.174368+0000","last_fullsized":"2024-09-10T22:11:47.174368+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-09-10T22:11:46.154789+0000","last_change":"2024-09-10T22:11:46.154789+0000","last_active":"2024-09-10T22:11:46.148666+0000","last_peered":"2024-09-10T22:11:46.148666+0000","last_clean":"2024-09-10T22:11:46.148666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:46.154789+0000","last_undegraded":"2024-09-10T22:11:46.154789+0000","last_fullsized":"2024-09-10T22:11:46.154789+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.169845+0000","last_change":"2024-09-10T22:11:47.169845+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.169845+0000","last_undegraded":"2024-09-10T22:11:47.169845+0000","last_fullsized":"2024-09-10T22:11:47.169845+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.178254+0000","last_change":"2024-09-10T22:11:47.177297+0000","last_active":"2024-09-10T22:11:47.178254+0000","last_peered":"2024-09-10T22:11:47.178254+0000","last_clean":"2024-09-10T22:11:47.178254+0000","last_became_active":"2024-09-10T22:11:47.176973+0000","last_became_peered":"2024-09-10T22:11:47.176973+0000","last_unstale":"2024-09-10T22:11:47.178254+0000","last_undegraded":"2024-09-10T22:11:47.178254+0000","last_fullsized":"2024-09-10T22:11:47.178254+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:59:31.964617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.170528+0000","last_change":"2024-09-10T22:11:47.170528+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.170528+0000","last_undegraded":"2024-09-10T22:11:47.170528+0000","last_fullsized":"2024-09-10T22:11:47.170528+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.186918+0000","last_change":"2024-09-10T22:11:47.184025+0000","last_active":"2024-09-10T22:11:47.186918+0000","last_peered":"2024-09-10T22:11:47.186918+0000","last_clean":"2024-09-10T22:11:47.186918+0000","last_became_active":"2024-09-10T22:11:47.183724+0000","last_became_peered":"2024-09-10T22:11:47.183724+0000","last_unstale":"2024-09-10T22:11:47.186918+0000","last_undegraded":"2024-09-10T22:11:47.186918+0000","last_fullsized":"2024-09-10T22:11:47.186918+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:12.970576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.172897+0000","last_change":"2024-09-10T22:11:47.172897+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.172897+0000","last_undegraded":"2024-09-10T22:11:47.172897+0000","last_fullsized":"2024-09-10T22:11:47.172897+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.183660+0000","last_change":"2024-09-10T22:11:47.181785+0000","last_active":"2024-09-10T22:11:47.183660+0000","last_peered":"2024-09-10T22:11:47.183660+0000","last_clean":"2024-09-10T22:11:47.183660+0000","last_became_active":"2024-09-10T22:11:47.181505+0000","last_became_peered":"2024-09-10T22:11:47.181505+0000","last_unstale":"2024-09-10T22:11:47.183660+0000","last_undegraded":"2024-09-10T22:11:47.183660+0000","last_fullsized":"2024-09-10T22:11:47.183660+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:02.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.171893+0000","last_change":"2024-09-10T22:11:47.171893+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.171893+0000","last_undegraded":"2024-09-10T22:11:47.171893+0000","last_fullsized":"2024-09-10T22:11:47.171893+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.189353+0000","last_change":"2024-09-10T22:11:47.187068+0000","last_active":"2024-09-10T22:11:47.189353+0000","last_peered":"2024-09-10T22:11:47.189353+0000","last_clean":"2024-09-10T22:11:47.189353+0000","last_became_active":"2024-09-10T22:11:47.186777+0000","last_became_peered":"2024-09-10T22:11:47.186777+0000","last_unstale":"2024-09-10T22:11:47.189353+0000","last_undegraded":"2024-09-10T22:11:47.189353+0000","last_fullsized":"2024-09-10T22:11:47.189353+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:32.056519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.172651+0000","last_change":"2024-09-10T22:11:47.172651+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.172651+0000","last_undegraded":"2024-09-10T22:11:47.172651+0000","last_fullsized":"2024-09-10T22:11:47.172651+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-09-10T22:11:46.154507+0000","last_change":"2024-09-10T22:11:46.154507+0000","last_active":"2024-09-10T22:11:46.148666+0000","last_peered":"2024-09-10T22:11:46.148666+0000","last_clean":"2024-09-10T22:11:46.148666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:46.154507+0000","last_undegraded":"2024-09-10T22:11:46.154507+0000","last_fullsized":"2024-09-10T22:11:46.154507+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.167762+0000","last_change":"2024-09-10T22:11:47.167762+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.167762+0000","last_undegraded":"2024-09-10T22:11:47.167762+0000","last_fullsized":"2024-09-10T22:11:47.167762+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.186511+0000","last_change":"2024-09-10T22:11:47.183552+0000","last_active":"2024-09-10T22:11:47.186511+0000","last_peered":"2024-09-10T22:11:47.186511+0000","last_clean":"2024-09-10T22:11:47.186511+0000","last_became_active":"2024-09-10T22:11:47.183260+0000","last_became_peered":"2024-09-10T22:11:47.183260+0000","last_unstale":"2024-09-10T22:11:47.186511+0000","last_undegraded":"2024-09-10T22:11:47.186511+0000","last_fullsized":"2024-09-10T22:11:47.186511+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:32.687217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.177412+0000","last_change":"2024-09-10T22:11:47.177412+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.177412+0000","last_undegraded":"2024-09-10T22:11:47.177412+0000","last_fullsized":"2024-09-10T22:11:47.177412+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.183285+0000","last_change":"2024-09-10T22:11:47.177926+0000","last_active":"2024-09-10T22:11:47.183285+0000","last_peered":"2024-09-10T22:11:47.183285+0000","last_clean":"2024-09-10T22:11:47.183285+0000","last_became_active":"2024-09-10T22:11:47.177598+0000","last_became_peered":"2024-09-10T22:11:47.177598+0000","last_unstale":"2024-09-10T22:11:47.183285+0000","last_undegraded":"2024-09-10T22:11:47.183285+0000","last_fullsized":"2024-09-10T22:11:47.183285+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:09.414566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174435+0000","last_change":"2024-09-10T22:11:47.174435+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174435+0000","last_undegraded":"2024-09-10T22:11:47.174435+0000","last_fullsized":"2024-09-10T22:11:47.174435+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.189461+0000","last_change":"2024-09-10T22:11:47.187146+0000","last_active":"2024-09-10T22:11:47.189461+0000","last_peered":"2024-09-10T22:11:47.189461+0000","last_clean":"2024-09-10T22:11:47.189461+0000","last_became_active":"2024-09-10T22:11:47.186886+0000","last_became_peered":"2024-09-10T22:11:47.186886+0000","last_unstale":"2024-09-10T22:11:47.189461+0000","last_undegraded":"2024-09-10T22:11:47.189461+0000","last_fullsized":"2024-09-10T22:11:47.189461+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:56.311845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174727+0000","last_change":"2024-09-10T22:11:47.174727+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174727+0000","last_undegraded":"2024-09-10T22:11:47.174727+0000","last_fullsized":"2024-09-10T22:11:47.174727+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.188469+0000","last_change":"2024-09-10T22:11:47.181080+0000","last_active":"2024-09-10T22:11:47.188469+0000","last_peered":"2024-09-10T22:11:47.188469+0000","last_clean":"2024-09-10T22:11:47.188469+0000","last_became_active":"2024-09-10T22:11:47.180823+0000","last_became_peered":"2024-09-10T22:11:47.180823+0000","last_unstale":"2024-09-10T22:11:47.188469+0000","last_undegraded":"2024-09-10T22:11:47.188469+0000","last_fullsized":"2024-09-10T22:11:47.188469+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:46.585234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.177126+0000","last_change":"2024-09-10T22:11:47.177126+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.177126+0000","last_undegraded":"2024-09-10T22:11:47.177126+0000","last_fullsized":"2024-09-10T22:11:47.177126+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.181480+0000","last_change":"2024-09-10T22:11:47.176434+0000","last_active":"2024-09-10T22:11:47.181480+0000","last_peered":"2024-09-10T22:11:47.181480+0000","last_clean":"2024-09-10T22:11:47.181480+0000","last_became_active":"2024-09-10T22:11:47.175932+0000","last_became_peered":"2024-09-10T22:11:47.175932+0000","last_unstale":"2024-09-10T22:11:47.181480+0000","last_undegraded":"2024-09-10T22:11:47.181480+0000","last_fullsized":"2024-09-10T22:11:47.181480+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:01.594145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174656+0000","last_change":"2024-09-10T22:11:47.174656+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174656+0000","last_undegraded":"2024-09-10T22:11:47.174656+0000","last_fullsized":"2024-09-10T22:11:47.174656+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-09-10T22:11:46.154774+0000","last_change":"2024-09-10T22:11:46.154774+0000","last_active":"2024-09-10T22:11:46.148666+0000","last_peered":"2024-09-10T22:11:46.148666+0000","last_clean":"2024-09-10T22:11:46.148666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:46.154774+0000","last_undegraded":"2024-09-10T22:11:46.154774+0000","last_fullsized":"2024-09-10T22:11:46.154774+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:11:47.151613+0000","last_change":"2024-09-10T22:11:47.151613+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.151613+0000","last_undegraded":"2024-09-10T22:11:47.151613+0000","last_fullsized":"2024-09-10T22:11:47.151613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.184835+0000","last_change":"2024-09-10T22:11:47.182799+0000","last_active":"2024-09-10T22:11:47.184835+0000","last_peered":"2024-09-10T22:11:47.184835+0000","last_clean":"2024-09-10T22:11:47.184835+0000","last_became_active":"2024-09-10T22:11:47.180852+0000","last_became_peered":"2024-09-10T22:11:47.180852+0000","last_unstale":"2024-09-10T22:11:47.184835+0000","last_undegraded":"2024-09-10T22:11:47.184835+0000","last_fullsized":"2024-09-10T22:11:47.184835+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:44.659694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174590+0000","last_change":"2024-09-10T22:11:47.174590+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174590+0000","last_undegraded":"2024-09-10T22:11:47.174590+0000","last_fullsized":"2024-09-10T22:11:47.174590+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-09-10T22:11:46.154479+0000","last_change":"2024-09-10T22:11:46.154479+0000","last_active":"2024-09-10T22:11:46.148666+0000","last_peered":"2024-09-10T22:11:46.148666+0000","last_clean":"2024-09-10T22:11:46.148666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:46.154479+0000","last_undegraded":"2024-09-10T22:11:46.154479+0000","last_fullsized":"2024-09-10T22:11:46.154479+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174644+0000","last_change":"2024-09-10T22:11:47.174644+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174644+0000","last_undegraded":"2024-09-10T22:11:47.174644+0000","last_fullsized":"2024-09-10T22:11:47.174644+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.191647+0000","last_change":"2024-09-10T22:11:47.191138+0000","last_active":"2024-09-10T22:11:47.191647+0000","last_peered":"2024-09-10T22:11:47.191647+0000","last_clean":"2024-09-10T22:11:47.191647+0000","last_became_active":"2024-09-10T22:11:47.190913+0000","last_became_peered":"2024-09-10T22:11:47.190913+0000","last_unstale":"2024-09-10T22:11:47.191647+0000","last_undegraded":"2024-09-10T22:11:47.191647+0000","last_fullsized":"2024-09-10T22:11:47.191647+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:25.018768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173918+0000","last_change":"2024-09-10T22:11:47.173918+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173918+0000","last_undegraded":"2024-09-10T22:11:47.173918+0000","last_fullsized":"2024-09-10T22:11:47.173918+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.184822+0000","last_change":"2024-09-10T22:11:47.182589+0000","last_active":"2024-09-10T22:11:47.184822+0000","last_peered":"2024-09-10T22:11:47.184822+0000","last_clean":"2024-09-10T22:11:47.184822+0000","last_became_active":"2024-09-10T22:11:47.182316+0000","last_became_peered":"2024-09-10T22:11:47.182316+0000","last_unstale":"2024-09-10T22:11:47.184822+0000","last_undegraded":"2024-09-10T22:11:47.184822+0000","last_fullsized":"2024-09-10T22:11:47.184822+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:44.356700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174289+0000","last_change":"2024-09-10T22:11:47.174289+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174289+0000","last_undegraded":"2024-09-10T22:11:47.174289+0000","last_fullsized":"2024-09-10T22:11:47.174289+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.190349+0000","last_change":"2024-09-10T22:11:47.187951+0000","last_active":"2024-09-10T22:11:47.190349+0000","last_peered":"2024-09-10T22:11:47.190349+0000","last_clean":"2024-09-10T22:11:47.190349+0000","last_became_active":"2024-09-10T22:11:47.187697+0000","last_became_peered":"2024-09-10T22:11:47.187697+0000","last_unstale":"2024-09-10T22:11:47.190349+0000","last_undegraded":"2024-09-10T22:11:47.190349+0000","last_fullsized":"2024-09-10T22:11:47.190349+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:30.527718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.172311+0000","last_change":"2024-09-10T22:11:47.172311+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.172311+0000","last_undegraded":"2024-09-10T22:11:47.172311+0000","last_fullsized":"2024-09-10T22:11:47.172311+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-09-10T22:11:46.154267+0000","last_change":"2024-09-10T22:11:46.154267+0000","last_active":"2024-09-10T22:11:46.148666+0000","last_peered":"2024-09-10T22:11:46.148666+0000","last_clean":"2024-09-10T22:11:46.148666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:46.154267+0000","last_undegraded":"2024-09-10T22:11:46.154267+0000","last_fullsized":"2024-09-10T22:11:46.154267+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173811+0000","last_change":"2024-09-10T22:11:47.173811+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173811+0000","last_undegraded":"2024-09-10T22:11:47.173811+0000","last_fullsized":"2024-09-10T22:11:47.173811+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.190372+0000","last_change":"2024-09-10T22:11:47.188145+0000","last_active":"2024-09-10T22:11:47.190372+0000","last_peered":"2024-09-10T22:11:47.190372+0000","last_clean":"2024-09-10T22:11:47.190372+0000","last_became_active":"2024-09-10T22:11:47.187866+0000","last_became_peered":"2024-09-10T22:11:47.187866+0000","last_unstale":"2024-09-10T22:11:47.190372+0000","last_undegraded":"2024-09-10T22:11:47.190372+0000","last_fullsized":"2024-09-10T22:11:47.190372+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:57.839056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174152+0000","last_change":"2024-09-10T22:11:47.174152+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174152+0000","last_undegraded":"2024-09-10T22:11:47.174152+0000","last_fullsized":"2024-09-10T22:11:47.174152+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.180254+0000","last_change":"2024-09-10T22:11:47.174439+0000","last_active":"2024-09-10T22:11:47.180254+0000","last_peered":"2024-09-10T22:11:47.180254+0000","last_clean":"2024-09-10T22:11:47.180254+0000","last_became_active":"2024-09-10T22:11:47.174130+0000","last_became_peered":"2024-09-10T22:11:47.174130+0000","last_unstale":"2024-09-10T22:11:47.180254+0000","last_undegraded":"2024-09-10T22:11:47.180254+0000","last_fullsized":"2024-09-10T22:11:47.180254+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:56.804619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.168178+0000","last_change":"2024-09-10T22:11:47.168178+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.168178+0000","last_undegraded":"2024-09-10T22:11:47.168178+0000","last_fullsized":"2024-09-10T22:11:47.168178+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.187026+0000","last_change":"2024-09-10T22:11:47.185574+0000","last_active":"2024-09-10T22:11:47.187026+0000","last_peered":"2024-09-10T22:11:47.187026+0000","last_clean":"2024-09-10T22:11:47.187026+0000","last_became_active":"2024-09-10T22:11:47.185262+0000","last_became_peered":"2024-09-10T22:11:47.185262+0000","last_unstale":"2024-09-10T22:11:47.187026+0000","last_undegraded":"2024-09-10T22:11:47.187026+0000","last_fullsized":"2024-09-10T22:11:47.187026+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:45.981456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.169852+0000","last_change":"2024-09-10T22:11:47.169852+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.169852+0000","last_undegraded":"2024-09-10T22:11:47.169852+0000","last_fullsized":"2024-09-10T22:11:47.169852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.182335+0000","last_change":"2024-09-10T22:11:47.178850+0000","last_active":"2024-09-10T22:11:47.182335+0000","last_peered":"2024-09-10T22:11:47.182335+0000","last_clean":"2024-09-10T22:11:47.182335+0000","last_became_active":"2024-09-10T22:11:47.178587+0000","last_became_peered":"2024-09-10T22:11:47.178587+0000","last_unstale":"2024-09-10T22:11:47.182335+0000","last_undegraded":"2024-09-10T22:11:47.182335+0000","last_fullsized":"2024-09-10T22:11:47.182335+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:53.033722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.172212+0000","last_change":"2024-09-10T22:11:47.172212+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.172212+0000","last_undegraded":"2024-09-10T22:11:47.172212+0000","last_fullsized":"2024-09-10T22:11:47.172212+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.183259+0000","last_change":"2024-09-10T22:11:47.175153+0000","last_active":"2024-09-10T22:11:47.183259+0000","last_peered":"2024-09-10T22:11:47.183259+0000","last_clean":"2024-09-10T22:11:47.183259+0000","last_became_active":"2024-09-10T22:11:47.174910+0000","last_became_peered":"2024-09-10T22:11:47.174910+0000","last_unstale":"2024-09-10T22:11:47.183259+0000","last_undegraded":"2024-09-10T22:11:47.183259+0000","last_fullsized":"2024-09-10T22:11:47.183259+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:19.251040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.167707+0000","last_change":"2024-09-10T22:11:47.167707+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.167707+0000","last_undegraded":"2024-09-10T22:11:47.167707+0000","last_fullsized":"2024-09-10T22:11:47.167707+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.190681+0000","last_change":"2024-09-10T22:11:47.190009+0000","last_active":"2024-09-10T22:11:47.190681+0000","last_peered":"2024-09-10T22:11:47.190681+0000","last_clean":"2024-09-10T22:11:47.190681+0000","last_became_active":"2024-09-10T22:11:47.189727+0000","last_became_peered":"2024-09-10T22:11:47.189727+0000","last_unstale":"2024-09-10T22:11:47.190681+0000","last_undegraded":"2024-09-10T22:11:47.190681+0000","last_fullsized":"2024-09-10T22:11:47.190681+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:24:45.805288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.167006+0000","last_change":"2024-09-10T22:11:47.167006+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.167006+0000","last_undegraded":"2024-09-10T22:11:47.167006+0000","last_fullsized":"2024-09-10T22:11:47.167006+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.184760+0000","last_change":"2024-09-10T22:11:47.182353+0000","last_active":"2024-09-10T22:11:47.184760+0000","last_peered":"2024-09-10T22:11:47.184760+0000","last_clean":"2024-09-10T22:11:47.184760+0000","last_became_active":"2024-09-10T22:11:47.182082+0000","last_became_peered":"2024-09-10T22:11:47.182082+0000","last_unstale":"2024-09-10T22:11:47.184760+0000","last_undegraded":"2024-09-10T22:11:47.184760+0000","last_fullsized":"2024-09-10T22:11:47.184760+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:25.709851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.170038+0000","last_change":"2024-09-10T22:11:47.170038+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.170038+0000","last_undegraded":"2024-09-10T22:11:47.170038+0000","last_fullsized":"2024-09-10T22:11:47.170038+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.180034+0000","last_change":"2024-09-10T22:11:47.177554+0000","last_active":"2024-09-10T22:11:47.180034+0000","last_peered":"2024-09-10T22:11:47.180034+0000","last_clean":"2024-09-10T22:11:47.180034+0000","last_became_active":"2024-09-10T22:11:47.177260+0000","last_became_peered":"2024-09-10T22:11:47.177260+0000","last_unstale":"2024-09-10T22:11:47.180034+0000","last_undegraded":"2024-09-10T22:11:47.180034+0000","last_fullsized":"2024-09-10T22:11:47.180034+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:51.033282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173263+0000","last_change":"2024-09-10T22:11:47.173263+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173263+0000","last_undegraded":"2024-09-10T22:11:47.173263+0000","last_fullsized":"2024-09-10T22:11:47.173263+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-09-10T22:11:46.154320+0000","last_change":"2024-09-10T22:11:46.154320+0000","last_active":"2024-09-10T22:11:46.148666+0000","last_peered":"2024-09-10T22:11:46.148666+0000","last_clean":"2024-09-10T22:11:46.148666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:46.154320+0000","last_undegraded":"2024-09-10T22:11:46.154320+0000","last_fullsized":"2024-09-10T22:11:46.154320+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.170431+0000","last_change":"2024-09-10T22:11:47.170431+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.170431+0000","last_undegraded":"2024-09-10T22:11:47.170431+0000","last_fullsized":"2024-09-10T22:11:47.170431+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.184556+0000","last_change":"2024-09-10T22:11:47.180051+0000","last_active":"2024-09-10T22:11:47.184556+0000","last_peered":"2024-09-10T22:11:47.184556+0000","last_clean":"2024-09-10T22:11:47.184556+0000","last_became_active":"2024-09-10T22:11:47.179772+0000","last_became_peered":"2024-09-10T22:11:47.179772+0000","last_unstale":"2024-09-10T22:11:47.184556+0000","last_undegraded":"2024-09-10T22:11:47.184556+0000","last_fullsized":"2024-09-10T22:11:47.184556+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:22.088173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174725+0000","last_change":"2024-09-10T22:11:47.174725+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174725+0000","last_undegraded":"2024-09-10T22:11:47.174725+0000","last_fullsized":"2024-09-10T22:11:47.174725+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.184850+0000","last_change":"2024-09-10T22:11:47.183264+0000","last_active":"2024-09-10T22:11:47.184850+0000","last_peered":"2024-09-10T22:11:47.184850+0000","last_clean":"2024-09-10T22:11:47.184850+0000","last_became_active":"2024-09-10T22:11:47.182945+0000","last_became_peered":"2024-09-10T22:11:47.182945+0000","last_unstale":"2024-09-10T22:11:47.184850+0000","last_undegraded":"2024-09-10T22:11:47.184850+0000","last_fullsized":"2024-09-10T22:11:47.184850+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:57.662127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":279,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.178911+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:11:47.178911+0000","last_peered":"2024-09-10T22:11:47.178911+0000","last_clean":"2024-09-10T22:11:47.178911+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:11:47.178911+0000","last_undegraded":"2024-09-10T22:11:47.178911+0000","last_fullsized":"2024-09-10T22:11:47.178911+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":124,"acting":124,"num_store_stats":7},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509580,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127436,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244404,"statfs":{"total":96636764160,"available":96506269696,"internally_reserved":0,"allocated":1953792,"data_stored":1179333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118532,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253308,"statfs":{"total":96636764160,"available":96515387392,"internally_reserved":0,"allocated":1347584,"data_stored":582606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509580,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104648,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267192,"statfs":{"total":96636764160,"available":96529604608,"internally_reserved":0,"allocated":1359872,"data_stored":588965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149064,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222776,"statfs":{"total":96636764160,"available":96484122624,"internally_reserved":0,"allocated":1359872,"data_stored":588965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509579,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144900,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226940,"statfs":{"total":96636764160,"available":96488386560,"internally_reserved":0,"allocated":1347584,"data_stored":582606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509580,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131000,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240840,"statfs":{"total":96636764160,"available":96502620160,"internally_reserved":0,"allocated":1347584,"data_stored":582606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509580,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127420,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244420,"statfs":{"total":96636764160,"available":96506286080,"internally_reserved":0,"allocated":1941504,"data_stored":1172974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":128514505},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144876,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226964,"statfs":{"total":96636764160,"available":96488411136,"internally_reserved":0,"allocated":1335296,"data_stored":576705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:11:52.327 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:11:52.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.512+0000 7f561ba00640 1 -- 172.21.15.66:0/3389175711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56140b2bf0 msgr2=0x7f56140d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.512+0000 7f561ba00640 1 --2- 172.21.15.66:0/3389175711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56140b2bf0 0x7f56140d0090 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5600009900 tx=0x7f560001c8e0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.512+0000 7f561ba00640 1 -- 172.21.15.66:0/3389175711 shutdown_connections 2024-09-10T22:11:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.512+0000 7f561ba00640 1 --2- 172.21.15.66:0/3389175711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56140b2bf0 0x7f56140d0090 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.512+0000 7f561ba00640 1 --2- 172.21.15.66:0/3389175711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f561415f340 0x7f56140cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.512+0000 7f561ba00640 1 -- 172.21.15.66:0/3389175711 >> 172.21.15.66:0/3389175711 conn(0x7f56140d1c50 msgr2=0x7f56140d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.512+0000 7f561ba00640 1 -- 172.21.15.66:0/3389175711 shutdown_connections 2024-09-10T22:11:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.512+0000 7f561ba00640 1 -- 172.21.15.66:0/3389175711 wait complete. 2024-09-10T22:11:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.512+0000 7f561ba00640 1 Processor -- start 2024-09-10T22:11:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.512+0000 7f561ba00640 1 -- start start 2024-09-10T22:11:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f561ba00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56140b2bf0 0x7f5614152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f561ba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56140d2770 0x7f56141558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f561ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561415f340 0x7f5614157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f561ba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5614169190 con 0x7f561415f340 2024-09-10T22:11:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f561ba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5614169010 con 0x7f56140d2770 2024-09-10T22:11:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f561ba00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5614169310 con 0x7f56140b2bf0 2024-09-10T22:11:52.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5619a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561415f340 0x7f5614157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:52.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5619000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56140b2bf0 0x7f5614152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:52.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5619a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561415f340 0x7f5614157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33070/0 (socket says 172.21.15.66:33070) 2024-09-10T22:11:52.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5619a00640 1 -- 172.21.15.66:0/3857951298 learned_addr learned my addr 172.21.15.66:0/3857951298 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:52.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5613e00640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56140d2770 0x7f56141558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:52.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5619a00640 1 -- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56140b2bf0 msgr2=0x7f5614152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5619a00640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56140b2bf0 0x7f5614152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5619a00640 1 -- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56140d2770 msgr2=0x7f56141558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5619a00640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56140d2770 0x7f56141558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5619a00640 1 -- 172.21.15.66:0/3857951298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5614156370 con 0x7f561415f340 2024-09-10T22:11:52.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5619000640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56140b2bf0 0x7f5614152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:11:52.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5619a00640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561415f340 0x7f5614157040 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f560c00c7a0 tx=0x7f560c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:52.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5610c00640 1 -- 172.21.15.66:0/3857951298 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c003b80 con 0x7f561415f340 2024-09-10T22:11:52.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5610c00640 1 -- 172.21.15.66:0/3857951298 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f560c003d40 con 0x7f561415f340 2024-09-10T22:11:52.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f5610c00640 1 -- 172.21.15.66:0/3857951298 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c00aab0 con 0x7f561415f340 2024-09-10T22:11:52.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f561ba00640 1 -- 172.21.15.66:0/3857951298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5614156650 con 0x7f561415f340 2024-09-10T22:11:52.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f561ba00640 1 -- 172.21.15.66:0/3857951298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5614156c10 con 0x7f561415f340 2024-09-10T22:11:52.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.516+0000 7f561ba00640 1 -- 172.21.15.66:0/3857951298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f561415fac0 con 0x7f561415f340 2024-09-10T22:11:52.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.520+0000 7f5610c00640 1 -- 172.21.15.66:0/3857951298 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f560c00ac50 con 0x7f561415f340 2024-09-10T22:11:52.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.520+0000 7f5610c00640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55f4073f30 0x7f55f40763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:52.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.520+0000 7f5610c00640 1 -- 172.21.15.66:0/3857951298 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f560c091db0 con 0x7f561415f340 2024-09-10T22:11:52.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.520+0000 7f5619000640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55f4073f30 0x7f55f40763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:52.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.520+0000 7f5619000640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55f4073f30 0x7f55f40763f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f560800bf40 tx=0x7f560800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:52.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.524+0000 7f5610c00640 1 -- 172.21.15.66:0/3857951298 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f560c05e8a0 con 0x7f561415f340 2024-09-10T22:11:52.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.696+0000 7f561ba00640 1 -- 172.21.15.66:0/3857951298 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f561406e1c0 con 0x7f55f4073f30 2024-09-10T22:11:52.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.700+0000 7f5610c00640 1 -- 172.21.15.66:0/3857951298 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251590 (secure 0 0 0) 0x7f561406e1c0 con 0x7f55f4073f30 2024-09-10T22:11:52.709 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:52.711 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:11:52.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.704+0000 7f561ba00640 1 -- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55f4073f30 msgr2=0x7f55f40763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.704+0000 7f561ba00640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55f4073f30 0x7f55f40763f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f560800bf40 tx=0x7f560800b2e0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.704+0000 7f561ba00640 1 -- 172.21.15.66:0/3857951298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561415f340 msgr2=0x7f5614157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.704+0000 7f561ba00640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561415f340 0x7f5614157040 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f560c00c7a0 tx=0x7f560c00cc60 comp rx=0 tx=0).stop 2024-09-10T22:11:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.704+0000 7f561ba00640 1 -- 172.21.15.66:0/3857951298 shutdown_connections 2024-09-10T22:11:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.704+0000 7f561ba00640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55f4073f30 0x7f55f40763f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.704+0000 7f561ba00640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561415f340 0x7f5614157040 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.704+0000 7f561ba00640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56140d2770 0x7f56141558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.704+0000 7f561ba00640 1 --2- 172.21.15.66:0/3857951298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56140b2bf0 0x7f5614152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.704+0000 7f561ba00640 1 -- 172.21.15.66:0/3857951298 >> 172.21.15.66:0/3857951298 conn(0x7f56140d1c50 msgr2=0x7f56140c5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:52.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.704+0000 7f561ba00640 1 -- 172.21.15.66:0/3857951298 shutdown_connections 2024-09-10T22:11:52.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.704+0000 7f561ba00640 1 -- 172.21.15.66:0/3857951298 wait complete. 2024-09-10T22:11:52.728 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-10T22:11:52.561491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1048164,"kb_used_data":12008,"kb_used_omap":12,"kb_used_meta":1035955,"kb_avail":753926556,"statfs":{"total":773094113280,"available":772020793344,"internally_reserved":0,"allocated":12296192,"data_stored":5989361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1060818484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001201"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265042+0000","last_change":"2024-09-10T22:11:48.180766+0000","last_active":"2024-09-10T22:11:49.265042+0000","last_peered":"2024-09-10T22:11:49.265042+0000","last_clean":"2024-09-10T22:11:49.265042+0000","last_became_active":"2024-09-10T22:11:48.179819+0000","last_became_peered":"2024-09-10T22:11:48.179819+0000","last_unstale":"2024-09-10T22:11:49.265042+0000","last_undegraded":"2024-09-10T22:11:49.265042+0000","last_fullsized":"2024-09-10T22:11:49.265042+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:21.885630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186814+0000","last_change":"2024-09-10T22:11:48.203979+0000","last_active":"2024-09-10T22:11:49.186814+0000","last_peered":"2024-09-10T22:11:49.186814+0000","last_clean":"2024-09-10T22:11:49.186814+0000","last_became_active":"2024-09-10T22:11:48.203764+0000","last_became_peered":"2024-09-10T22:11:48.203764+0000","last_unstale":"2024-09-10T22:11:49.186814+0000","last_undegraded":"2024-09-10T22:11:49.186814+0000","last_fullsized":"2024-09-10T22:11:49.186814+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:41.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.177096+0000","last_change":"2024-09-10T22:11:47.177096+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.177096+0000","last_undegraded":"2024-09-10T22:11:47.177096+0000","last_fullsized":"2024-09-10T22:11:47.177096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264205+0000","last_change":"2024-09-10T22:11:48.191506+0000","last_active":"2024-09-10T22:11:49.264205+0000","last_peered":"2024-09-10T22:11:49.264205+0000","last_clean":"2024-09-10T22:11:49.264205+0000","last_became_active":"2024-09-10T22:11:48.191100+0000","last_became_peered":"2024-09-10T22:11:48.191100+0000","last_unstale":"2024-09-10T22:11:49.264205+0000","last_undegraded":"2024-09-10T22:11:49.264205+0000","last_fullsized":"2024-09-10T22:11:49.264205+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:54.388719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263748+0000","last_change":"2024-09-10T22:11:48.178358+0000","last_active":"2024-09-10T22:11:49.263748+0000","last_peered":"2024-09-10T22:11:49.263748+0000","last_clean":"2024-09-10T22:11:49.263748+0000","last_became_active":"2024-09-10T22:11:48.177965+0000","last_became_peered":"2024-09-10T22:11:48.177965+0000","last_unstale":"2024-09-10T22:11:49.263748+0000","last_undegraded":"2024-09-10T22:11:49.263748+0000","last_fullsized":"2024-09-10T22:11:49.263748+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:16.479719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264893+0000","last_change":"2024-09-10T22:11:48.182705+0000","last_active":"2024-09-10T22:11:49.264893+0000","last_peered":"2024-09-10T22:11:49.264893+0000","last_clean":"2024-09-10T22:11:49.264893+0000","last_became_active":"2024-09-10T22:11:48.182428+0000","last_became_peered":"2024-09-10T22:11:48.182428+0000","last_unstale":"2024-09-10T22:11:49.264893+0000","last_undegraded":"2024-09-10T22:11:49.264893+0000","last_fullsized":"2024-09-10T22:11:49.264893+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:56.601140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186457+0000","last_change":"2024-09-10T22:11:48.181122+0000","last_active":"2024-09-10T22:11:49.186457+0000","last_peered":"2024-09-10T22:11:49.186457+0000","last_clean":"2024-09-10T22:11:49.186457+0000","last_became_active":"2024-09-10T22:11:48.178614+0000","last_became_peered":"2024-09-10T22:11:48.178614+0000","last_unstale":"2024-09-10T22:11:49.186457+0000","last_undegraded":"2024-09-10T22:11:49.186457+0000","last_fullsized":"2024-09-10T22:11:49.186457+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:54.036670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174226+0000","last_change":"2024-09-10T22:11:47.174226+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174226+0000","last_undegraded":"2024-09-10T22:11:47.174226+0000","last_fullsized":"2024-09-10T22:11:47.174226+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265249+0000","last_change":"2024-09-10T22:11:48.180116+0000","last_active":"2024-09-10T22:11:49.265249+0000","last_peered":"2024-09-10T22:11:49.265249+0000","last_clean":"2024-09-10T22:11:49.265249+0000","last_became_active":"2024-09-10T22:11:48.179490+0000","last_became_peered":"2024-09-10T22:11:48.179490+0000","last_unstale":"2024-09-10T22:11:49.265249+0000","last_undegraded":"2024-09-10T22:11:49.265249+0000","last_fullsized":"2024-09-10T22:11:49.265249+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:19.843998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264589+0000","last_change":"2024-09-10T22:11:48.183940+0000","last_active":"2024-09-10T22:11:49.264589+0000","last_peered":"2024-09-10T22:11:49.264589+0000","last_clean":"2024-09-10T22:11:49.264589+0000","last_became_active":"2024-09-10T22:11:48.183694+0000","last_became_peered":"2024-09-10T22:11:48.183694+0000","last_unstale":"2024-09-10T22:11:49.264589+0000","last_undegraded":"2024-09-10T22:11:49.264589+0000","last_fullsized":"2024-09-10T22:11:49.264589+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:00.322112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187457+0000","last_change":"2024-09-10T22:11:48.197453+0000","last_active":"2024-09-10T22:11:49.187457+0000","last_peered":"2024-09-10T22:11:49.187457+0000","last_clean":"2024-09-10T22:11:49.187457+0000","last_became_active":"2024-09-10T22:11:48.197138+0000","last_became_peered":"2024-09-10T22:11:48.197138+0000","last_unstale":"2024-09-10T22:11:49.187457+0000","last_undegraded":"2024-09-10T22:11:49.187457+0000","last_fullsized":"2024-09-10T22:11:49.187457+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:33.059467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264403+0000","last_change":"2024-09-10T22:11:48.189077+0000","last_active":"2024-09-10T22:11:49.264403+0000","last_peered":"2024-09-10T22:11:49.264403+0000","last_clean":"2024-09-10T22:11:49.264403+0000","last_became_active":"2024-09-10T22:11:48.185421+0000","last_became_peered":"2024-09-10T22:11:48.185421+0000","last_unstale":"2024-09-10T22:11:49.264403+0000","last_undegraded":"2024-09-10T22:11:49.264403+0000","last_fullsized":"2024-09-10T22:11:49.264403+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:25.288843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.175144+0000","last_change":"2024-09-10T22:11:47.175144+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.175144+0000","last_undegraded":"2024-09-10T22:11:47.175144+0000","last_fullsized":"2024-09-10T22:11:47.175144+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174758+0000","last_change":"2024-09-10T22:11:47.174758+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174758+0000","last_undegraded":"2024-09-10T22:11:47.174758+0000","last_fullsized":"2024-09-10T22:11:47.174758+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264482+0000","last_change":"2024-09-10T22:11:48.183435+0000","last_active":"2024-09-10T22:11:49.264482+0000","last_peered":"2024-09-10T22:11:49.264482+0000","last_clean":"2024-09-10T22:11:49.264482+0000","last_became_active":"2024-09-10T22:11:48.183243+0000","last_became_peered":"2024-09-10T22:11:48.183243+0000","last_unstale":"2024-09-10T22:11:49.264482+0000","last_undegraded":"2024-09-10T22:11:49.264482+0000","last_fullsized":"2024-09-10T22:11:49.264482+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:11.483616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263496+0000","last_change":"2024-09-10T22:11:48.186035+0000","last_active":"2024-09-10T22:11:49.263496+0000","last_peered":"2024-09-10T22:11:49.263496+0000","last_clean":"2024-09-10T22:11:49.263496+0000","last_became_active":"2024-09-10T22:11:48.185768+0000","last_became_peered":"2024-09-10T22:11:48.185768+0000","last_unstale":"2024-09-10T22:11:49.263496+0000","last_undegraded":"2024-09-10T22:11:49.263496+0000","last_fullsized":"2024-09-10T22:11:49.263496+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:22.226210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187717+0000","last_change":"2024-09-10T22:11:48.200285+0000","last_active":"2024-09-10T22:11:49.187717+0000","last_peered":"2024-09-10T22:11:49.187717+0000","last_clean":"2024-09-10T22:11:49.187717+0000","last_became_active":"2024-09-10T22:11:48.199965+0000","last_became_peered":"2024-09-10T22:11:48.199965+0000","last_unstale":"2024-09-10T22:11:49.187717+0000","last_undegraded":"2024-09-10T22:11:49.187717+0000","last_fullsized":"2024-09-10T22:11:49.187717+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:22.961644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268722+0000","last_change":"2024-09-10T22:11:48.194873+0000","last_active":"2024-09-10T22:11:49.268722+0000","last_peered":"2024-09-10T22:11:49.268722+0000","last_clean":"2024-09-10T22:11:49.268722+0000","last_became_active":"2024-09-10T22:11:48.194469+0000","last_became_peered":"2024-09-10T22:11:48.194469+0000","last_unstale":"2024-09-10T22:11:49.268722+0000","last_undegraded":"2024-09-10T22:11:49.268722+0000","last_fullsized":"2024-09-10T22:11:49.268722+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:37.166232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.176608+0000","last_change":"2024-09-10T22:11:47.176608+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.176608+0000","last_undegraded":"2024-09-10T22:11:47.176608+0000","last_fullsized":"2024-09-10T22:11:47.176608+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186908+0000","last_change":"2024-09-10T22:11:48.199337+0000","last_active":"2024-09-10T22:11:49.186908+0000","last_peered":"2024-09-10T22:11:49.186908+0000","last_clean":"2024-09-10T22:11:49.186908+0000","last_became_active":"2024-09-10T22:11:48.199028+0000","last_became_peered":"2024-09-10T22:11:48.199028+0000","last_unstale":"2024-09-10T22:11:49.186908+0000","last_undegraded":"2024-09-10T22:11:49.186908+0000","last_fullsized":"2024-09-10T22:11:49.186908+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:57.807567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187347+0000","last_change":"2024-09-10T22:11:48.200296+0000","last_active":"2024-09-10T22:11:49.187347+0000","last_peered":"2024-09-10T22:11:49.187347+0000","last_clean":"2024-09-10T22:11:49.187347+0000","last_became_active":"2024-09-10T22:11:48.198375+0000","last_became_peered":"2024-09-10T22:11:48.198375+0000","last_unstale":"2024-09-10T22:11:49.187347+0000","last_undegraded":"2024-09-10T22:11:49.187347+0000","last_fullsized":"2024-09-10T22:11:49.187347+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:45.662642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180916+0000","last_change":"2024-09-10T22:11:48.180779+0000","last_active":"2024-09-10T22:11:49.180916+0000","last_peered":"2024-09-10T22:11:49.180916+0000","last_clean":"2024-09-10T22:11:49.180916+0000","last_became_active":"2024-09-10T22:11:48.179454+0000","last_became_peered":"2024-09-10T22:11:48.179454+0000","last_unstale":"2024-09-10T22:11:49.180916+0000","last_undegraded":"2024-09-10T22:11:49.180916+0000","last_fullsized":"2024-09-10T22:11:49.180916+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:28:27.233254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186778+0000","last_change":"2024-09-10T22:11:48.198947+0000","last_active":"2024-09-10T22:11:49.186778+0000","last_peered":"2024-09-10T22:11:49.186778+0000","last_clean":"2024-09-10T22:11:49.186778+0000","last_became_active":"2024-09-10T22:11:48.198672+0000","last_became_peered":"2024-09-10T22:11:48.198672+0000","last_unstale":"2024-09-10T22:11:49.186778+0000","last_undegraded":"2024-09-10T22:11:49.186778+0000","last_fullsized":"2024-09-10T22:11:49.186778+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:08.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180712+0000","last_change":"2024-09-10T22:11:48.177447+0000","last_active":"2024-09-10T22:11:49.180712+0000","last_peered":"2024-09-10T22:11:49.180712+0000","last_clean":"2024-09-10T22:11:49.180712+0000","last_became_active":"2024-09-10T22:11:48.177207+0000","last_became_peered":"2024-09-10T22:11:48.177207+0000","last_unstale":"2024-09-10T22:11:49.180712+0000","last_undegraded":"2024-09-10T22:11:49.180712+0000","last_fullsized":"2024-09-10T22:11:49.180712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:57.314793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187731+0000","last_change":"2024-09-10T22:11:48.201225+0000","last_active":"2024-09-10T22:11:49.187731+0000","last_peered":"2024-09-10T22:11:49.187731+0000","last_clean":"2024-09-10T22:11:49.187731+0000","last_became_active":"2024-09-10T22:11:48.200931+0000","last_became_peered":"2024-09-10T22:11:48.200931+0000","last_unstale":"2024-09-10T22:11:49.187731+0000","last_undegraded":"2024-09-10T22:11:49.187731+0000","last_fullsized":"2024-09-10T22:11:49.187731+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:16.417992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264799+0000","last_change":"2024-09-10T22:11:48.185741+0000","last_active":"2024-09-10T22:11:49.264799+0000","last_peered":"2024-09-10T22:11:49.264799+0000","last_clean":"2024-09-10T22:11:49.264799+0000","last_became_active":"2024-09-10T22:11:48.185438+0000","last_became_peered":"2024-09-10T22:11:48.185438+0000","last_unstale":"2024-09-10T22:11:49.264799+0000","last_undegraded":"2024-09-10T22:11:49.264799+0000","last_fullsized":"2024-09-10T22:11:49.264799+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:58.072179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265352+0000","last_change":"2024-09-10T22:11:48.195500+0000","last_active":"2024-09-10T22:11:49.265352+0000","last_peered":"2024-09-10T22:11:49.265352+0000","last_clean":"2024-09-10T22:11:49.265352+0000","last_became_active":"2024-09-10T22:11:48.195019+0000","last_became_peered":"2024-09-10T22:11:48.195019+0000","last_unstale":"2024-09-10T22:11:49.265352+0000","last_undegraded":"2024-09-10T22:11:49.265352+0000","last_fullsized":"2024-09-10T22:11:49.265352+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:18.164228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181343+0000","last_change":"2024-09-10T22:11:48.189755+0000","last_active":"2024-09-10T22:11:49.181343+0000","last_peered":"2024-09-10T22:11:49.181343+0000","last_clean":"2024-09-10T22:11:49.181343+0000","last_became_active":"2024-09-10T22:11:48.189436+0000","last_became_peered":"2024-09-10T22:11:48.189436+0000","last_unstale":"2024-09-10T22:11:49.181343+0000","last_undegraded":"2024-09-10T22:11:49.181343+0000","last_fullsized":"2024-09-10T22:11:49.181343+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:42.442483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187041+0000","last_change":"2024-09-10T22:11:48.199608+0000","last_active":"2024-09-10T22:11:49.187041+0000","last_peered":"2024-09-10T22:11:49.187041+0000","last_clean":"2024-09-10T22:11:49.187041+0000","last_became_active":"2024-09-10T22:11:48.199331+0000","last_became_peered":"2024-09-10T22:11:48.199331+0000","last_unstale":"2024-09-10T22:11:49.187041+0000","last_undegraded":"2024-09-10T22:11:49.187041+0000","last_fullsized":"2024-09-10T22:11:49.187041+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:20.253988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186852+0000","last_change":"2024-09-10T22:11:48.184404+0000","last_active":"2024-09-10T22:11:49.186852+0000","last_peered":"2024-09-10T22:11:49.186852+0000","last_clean":"2024-09-10T22:11:49.186852+0000","last_became_active":"2024-09-10T22:11:48.180465+0000","last_became_peered":"2024-09-10T22:11:48.180465+0000","last_unstale":"2024-09-10T22:11:49.186852+0000","last_undegraded":"2024-09-10T22:11:49.186852+0000","last_fullsized":"2024-09-10T22:11:49.186852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:02.597092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263547+0000","last_change":"2024-09-10T22:11:48.180454+0000","last_active":"2024-09-10T22:11:49.263547+0000","last_peered":"2024-09-10T22:11:49.263547+0000","last_clean":"2024-09-10T22:11:49.263547+0000","last_became_active":"2024-09-10T22:11:48.179907+0000","last_became_peered":"2024-09-10T22:11:48.179907+0000","last_unstale":"2024-09-10T22:11:49.263547+0000","last_undegraded":"2024-09-10T22:11:49.263547+0000","last_fullsized":"2024-09-10T22:11:49.263547+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:15.319909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264477+0000","last_change":"2024-09-10T22:11:48.182798+0000","last_active":"2024-09-10T22:11:49.264477+0000","last_peered":"2024-09-10T22:11:49.264477+0000","last_clean":"2024-09-10T22:11:49.264477+0000","last_became_active":"2024-09-10T22:11:48.182533+0000","last_became_peered":"2024-09-10T22:11:48.182533+0000","last_unstale":"2024-09-10T22:11:49.264477+0000","last_undegraded":"2024-09-10T22:11:49.264477+0000","last_fullsized":"2024-09-10T22:11:49.264477+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:35.986933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265665+0000","last_change":"2024-09-10T22:11:48.184978+0000","last_active":"2024-09-10T22:11:49.265665+0000","last_peered":"2024-09-10T22:11:49.265665+0000","last_clean":"2024-09-10T22:11:49.265665+0000","last_became_active":"2024-09-10T22:11:48.184722+0000","last_became_peered":"2024-09-10T22:11:48.184722+0000","last_unstale":"2024-09-10T22:11:49.265665+0000","last_undegraded":"2024-09-10T22:11:49.265665+0000","last_fullsized":"2024-09-10T22:11:49.265665+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:48.904032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187304+0000","last_change":"2024-09-10T22:11:47.179953+0000","last_active":"2024-09-10T22:11:49.187304+0000","last_peered":"2024-09-10T22:11:49.187304+0000","last_clean":"2024-09-10T22:11:49.187304+0000","last_became_active":"2024-09-10T22:11:47.179653+0000","last_became_peered":"2024-09-10T22:11:47.179653+0000","last_unstale":"2024-09-10T22:11:49.187304+0000","last_undegraded":"2024-09-10T22:11:49.187304+0000","last_fullsized":"2024-09-10T22:11:49.187304+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:17.150534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268977+0000","last_change":"2024-09-10T22:11:48.186184+0000","last_active":"2024-09-10T22:11:49.268977+0000","last_peered":"2024-09-10T22:11:49.268977+0000","last_clean":"2024-09-10T22:11:49.268977+0000","last_became_active":"2024-09-10T22:11:48.185822+0000","last_became_peered":"2024-09-10T22:11:48.185822+0000","last_unstale":"2024-09-10T22:11:49.268977+0000","last_undegraded":"2024-09-10T22:11:49.268977+0000","last_fullsized":"2024-09-10T22:11:49.268977+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:41.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.185431+0000","last_change":"2024-09-10T22:11:47.184730+0000","last_active":"2024-09-10T22:11:47.185431+0000","last_peered":"2024-09-10T22:11:47.185431+0000","last_clean":"2024-09-10T22:11:47.185431+0000","last_became_active":"2024-09-10T22:11:47.184361+0000","last_became_peered":"2024-09-10T22:11:47.184361+0000","last_unstale":"2024-09-10T22:11:47.185431+0000","last_undegraded":"2024-09-10T22:11:47.185431+0000","last_fullsized":"2024-09-10T22:11:47.185431+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:19.251040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265333+0000","last_change":"2024-09-10T22:11:48.182402+0000","last_active":"2024-09-10T22:11:49.265333+0000","last_peered":"2024-09-10T22:11:49.265333+0000","last_clean":"2024-09-10T22:11:49.265333+0000","last_became_active":"2024-09-10T22:11:48.182057+0000","last_became_peered":"2024-09-10T22:11:48.182057+0000","last_unstale":"2024-09-10T22:11:49.265333+0000","last_undegraded":"2024-09-10T22:11:49.265333+0000","last_fullsized":"2024-09-10T22:11:49.265333+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:18.062008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265043+0000","last_change":"2024-09-10T22:11:47.181206+0000","last_active":"2024-09-10T22:11:49.265043+0000","last_peered":"2024-09-10T22:11:49.265043+0000","last_clean":"2024-09-10T22:11:49.265043+0000","last_became_active":"2024-09-10T22:11:47.179544+0000","last_became_peered":"2024-09-10T22:11:47.179544+0000","last_unstale":"2024-09-10T22:11:49.265043+0000","last_undegraded":"2024-09-10T22:11:49.265043+0000","last_fullsized":"2024-09-10T22:11:49.265043+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:26.154879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174368+0000","last_change":"2024-09-10T22:11:47.174368+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174368+0000","last_undegraded":"2024-09-10T22:11:47.174368+0000","last_fullsized":"2024-09-10T22:11:47.174368+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263584+0000","last_change":"2024-09-10T22:11:47.182277+0000","last_active":"2024-09-10T22:11:49.263584+0000","last_peered":"2024-09-10T22:11:49.263584+0000","last_clean":"2024-09-10T22:11:49.263584+0000","last_became_active":"2024-09-10T22:11:47.181435+0000","last_became_peered":"2024-09-10T22:11:47.181435+0000","last_unstale":"2024-09-10T22:11:49.263584+0000","last_undegraded":"2024-09-10T22:11:49.263584+0000","last_fullsized":"2024-09-10T22:11:49.263584+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:47.604910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263891+0000","last_change":"2024-09-10T22:11:48.181857+0000","last_active":"2024-09-10T22:11:49.263891+0000","last_peered":"2024-09-10T22:11:49.263891+0000","last_clean":"2024-09-10T22:11:49.263891+0000","last_became_active":"2024-09-10T22:11:48.179210+0000","last_became_peered":"2024-09-10T22:11:48.179210+0000","last_unstale":"2024-09-10T22:11:49.263891+0000","last_undegraded":"2024-09-10T22:11:49.263891+0000","last_fullsized":"2024-09-10T22:11:49.263891+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:51.570280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186717+0000","last_change":"2024-09-10T22:11:47.177297+0000","last_active":"2024-09-10T22:11:49.186717+0000","last_peered":"2024-09-10T22:11:49.186717+0000","last_clean":"2024-09-10T22:11:49.186717+0000","last_became_active":"2024-09-10T22:11:47.176973+0000","last_became_peered":"2024-09-10T22:11:47.176973+0000","last_unstale":"2024-09-10T22:11:49.186717+0000","last_undegraded":"2024-09-10T22:11:49.186717+0000","last_fullsized":"2024-09-10T22:11:49.186717+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:08.162909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187075+0000","last_change":"2024-09-10T22:11:48.204412+0000","last_active":"2024-09-10T22:11:49.187075+0000","last_peered":"2024-09-10T22:11:49.187075+0000","last_clean":"2024-09-10T22:11:49.187075+0000","last_became_active":"2024-09-10T22:11:48.204105+0000","last_became_peered":"2024-09-10T22:11:48.204105+0000","last_unstale":"2024-09-10T22:11:49.187075+0000","last_undegraded":"2024-09-10T22:11:49.187075+0000","last_fullsized":"2024-09-10T22:11:49.187075+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:04.692623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181125+0000","last_change":"2024-09-10T22:11:47.184025+0000","last_active":"2024-09-10T22:11:49.181125+0000","last_peered":"2024-09-10T22:11:49.181125+0000","last_clean":"2024-09-10T22:11:49.181125+0000","last_became_active":"2024-09-10T22:11:47.183724+0000","last_became_peered":"2024-09-10T22:11:47.183724+0000","last_unstale":"2024-09-10T22:11:49.181125+0000","last_undegraded":"2024-09-10T22:11:49.181125+0000","last_fullsized":"2024-09-10T22:11:49.181125+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:50.420296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263722+0000","last_change":"2024-09-10T22:11:48.186204+0000","last_active":"2024-09-10T22:11:49.263722+0000","last_peered":"2024-09-10T22:11:49.263722+0000","last_clean":"2024-09-10T22:11:49.263722+0000","last_became_active":"2024-09-10T22:11:48.185455+0000","last_became_peered":"2024-09-10T22:11:48.185455+0000","last_unstale":"2024-09-10T22:11:49.263722+0000","last_undegraded":"2024-09-10T22:11:49.263722+0000","last_fullsized":"2024-09-10T22:11:49.263722+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:57.468704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186931+0000","last_change":"2024-09-10T22:11:47.181785+0000","last_active":"2024-09-10T22:11:49.186931+0000","last_peered":"2024-09-10T22:11:49.186931+0000","last_clean":"2024-09-10T22:11:49.186931+0000","last_became_active":"2024-09-10T22:11:47.181505+0000","last_became_peered":"2024-09-10T22:11:47.181505+0000","last_unstale":"2024-09-10T22:11:49.186931+0000","last_undegraded":"2024-09-10T22:11:49.186931+0000","last_fullsized":"2024-09-10T22:11:49.186931+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:01.594145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186384+0000","last_change":"2024-09-10T22:11:48.181429+0000","last_active":"2024-09-10T22:11:49.186384+0000","last_peered":"2024-09-10T22:11:49.186384+0000","last_clean":"2024-09-10T22:11:49.186384+0000","last_became_active":"2024-09-10T22:11:48.178978+0000","last_became_peered":"2024-09-10T22:11:48.178978+0000","last_unstale":"2024-09-10T22:11:49.186384+0000","last_undegraded":"2024-09-10T22:11:49.186384+0000","last_fullsized":"2024-09-10T22:11:49.186384+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:52.036230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263948+0000","last_change":"2024-09-10T22:11:47.187068+0000","last_active":"2024-09-10T22:11:49.263948+0000","last_peered":"2024-09-10T22:11:49.263948+0000","last_clean":"2024-09-10T22:11:49.263948+0000","last_became_active":"2024-09-10T22:11:47.186777+0000","last_became_peered":"2024-09-10T22:11:47.186777+0000","last_unstale":"2024-09-10T22:11:49.263948+0000","last_undegraded":"2024-09-10T22:11:49.263948+0000","last_fullsized":"2024-09-10T22:11:49.263948+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:21.958696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187129+0000","last_change":"2024-09-10T22:11:48.181451+0000","last_active":"2024-09-10T22:11:49.187129+0000","last_peered":"2024-09-10T22:11:49.187129+0000","last_clean":"2024-09-10T22:11:49.187129+0000","last_became_active":"2024-09-10T22:11:48.181161+0000","last_became_peered":"2024-09-10T22:11:48.181161+0000","last_unstale":"2024-09-10T22:11:49.187129+0000","last_undegraded":"2024-09-10T22:11:49.187129+0000","last_fullsized":"2024-09-10T22:11:49.187129+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:26.021716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265100+0000","last_change":"2024-09-10T22:11:47.179876+0000","last_active":"2024-09-10T22:11:49.265100+0000","last_peered":"2024-09-10T22:11:49.265100+0000","last_clean":"2024-09-10T22:11:49.265100+0000","last_became_active":"2024-09-10T22:11:47.179525+0000","last_became_peered":"2024-09-10T22:11:47.179525+0000","last_unstale":"2024-09-10T22:11:49.265100+0000","last_undegraded":"2024-09-10T22:11:49.265100+0000","last_fullsized":"2024-09-10T22:11:49.265100+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:59.073260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.269023+0000","last_change":"2024-09-10T22:11:48.187132+0000","last_active":"2024-09-10T22:11:49.269023+0000","last_peered":"2024-09-10T22:11:49.269023+0000","last_clean":"2024-09-10T22:11:49.269023+0000","last_became_active":"2024-09-10T22:11:48.186733+0000","last_became_peered":"2024-09-10T22:11:48.186733+0000","last_unstale":"2024-09-10T22:11:49.269023+0000","last_undegraded":"2024-09-10T22:11:49.269023+0000","last_fullsized":"2024-09-10T22:11:49.269023+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:57.807567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181341+0000","last_change":"2024-09-10T22:11:47.183552+0000","last_active":"2024-09-10T22:11:49.181341+0000","last_peered":"2024-09-10T22:11:49.181341+0000","last_clean":"2024-09-10T22:11:49.181341+0000","last_became_active":"2024-09-10T22:11:47.183260+0000","last_became_peered":"2024-09-10T22:11:47.183260+0000","last_unstale":"2024-09-10T22:11:49.181341+0000","last_undegraded":"2024-09-10T22:11:49.181341+0000","last_fullsized":"2024-09-10T22:11:49.181341+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:13.345360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181355+0000","last_change":"2024-09-10T22:11:48.194176+0000","last_active":"2024-09-10T22:11:49.181355+0000","last_peered":"2024-09-10T22:11:49.181355+0000","last_clean":"2024-09-10T22:11:49.181355+0000","last_became_active":"2024-09-10T22:11:48.192478+0000","last_became_peered":"2024-09-10T22:11:48.192478+0000","last_unstale":"2024-09-10T22:11:49.181355+0000","last_undegraded":"2024-09-10T22:11:49.181355+0000","last_fullsized":"2024-09-10T22:11:49.181355+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:59.444904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264160+0000","last_change":"2024-09-10T22:11:47.177926+0000","last_active":"2024-09-10T22:11:49.264160+0000","last_peered":"2024-09-10T22:11:49.264160+0000","last_clean":"2024-09-10T22:11:49.264160+0000","last_became_active":"2024-09-10T22:11:47.177598+0000","last_became_peered":"2024-09-10T22:11:47.177598+0000","last_unstale":"2024-09-10T22:11:49.264160+0000","last_undegraded":"2024-09-10T22:11:49.264160+0000","last_fullsized":"2024-09-10T22:11:49.264160+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:07.075072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186536+0000","last_change":"2024-09-10T22:11:48.178827+0000","last_active":"2024-09-10T22:11:49.186536+0000","last_peered":"2024-09-10T22:11:49.186536+0000","last_clean":"2024-09-10T22:11:49.186536+0000","last_became_active":"2024-09-10T22:11:48.177493+0000","last_became_peered":"2024-09-10T22:11:48.177493+0000","last_unstale":"2024-09-10T22:11:49.186536+0000","last_undegraded":"2024-09-10T22:11:49.186536+0000","last_fullsized":"2024-09-10T22:11:49.186536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:03.124106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":21,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264888+0000","last_change":"2024-09-10T22:11:47.187146+0000","last_active":"2024-09-10T22:11:49.264888+0000","last_peered":"2024-09-10T22:11:49.264888+0000","last_clean":"2024-09-10T22:11:49.264888+0000","last_became_active":"2024-09-10T22:11:47.186886+0000","last_became_peered":"2024-09-10T22:11:47.186886+0000","last_unstale":"2024-09-10T22:11:49.264888+0000","last_undegraded":"2024-09-10T22:11:49.264888+0000","last_fullsized":"2024-09-10T22:11:49.264888+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:13.528123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180957+0000","last_change":"2024-09-10T22:11:48.174022+0000","last_active":"2024-09-10T22:11:49.180957+0000","last_peered":"2024-09-10T22:11:49.180957+0000","last_clean":"2024-09-10T22:11:49.180957+0000","last_became_active":"2024-09-10T22:11:48.173765+0000","last_became_peered":"2024-09-10T22:11:48.173765+0000","last_unstale":"2024-09-10T22:11:49.180957+0000","last_undegraded":"2024-09-10T22:11:49.180957+0000","last_fullsized":"2024-09-10T22:11:49.180957+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:08.078020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265642+0000","last_change":"2024-09-10T22:11:47.181080+0000","last_active":"2024-09-10T22:11:49.265642+0000","last_peered":"2024-09-10T22:11:49.265642+0000","last_clean":"2024-09-10T22:11:49.265642+0000","last_became_active":"2024-09-10T22:11:47.180823+0000","last_became_peered":"2024-09-10T22:11:47.180823+0000","last_unstale":"2024-09-10T22:11:49.265642+0000","last_undegraded":"2024-09-10T22:11:49.265642+0000","last_fullsized":"2024-09-10T22:11:49.265642+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:27.455791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181216+0000","last_change":"2024-09-10T22:11:48.181817+0000","last_active":"2024-09-10T22:11:49.181216+0000","last_peered":"2024-09-10T22:11:49.181216+0000","last_clean":"2024-09-10T22:11:49.181216+0000","last_became_active":"2024-09-10T22:11:48.181514+0000","last_became_peered":"2024-09-10T22:11:48.181514+0000","last_unstale":"2024-09-10T22:11:49.181216+0000","last_undegraded":"2024-09-10T22:11:49.181216+0000","last_fullsized":"2024-09-10T22:11:49.181216+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:08.640825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268274+0000","last_change":"2024-09-10T22:11:47.176434+0000","last_active":"2024-09-10T22:11:49.268274+0000","last_peered":"2024-09-10T22:11:49.268274+0000","last_clean":"2024-09-10T22:11:49.268274+0000","last_became_active":"2024-09-10T22:11:47.175932+0000","last_became_peered":"2024-09-10T22:11:47.175932+0000","last_unstale":"2024-09-10T22:11:49.268274+0000","last_undegraded":"2024-09-10T22:11:49.268274+0000","last_fullsized":"2024-09-10T22:11:49.268274+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:41.968728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180789+0000","last_change":"2024-09-10T22:11:48.193087+0000","last_active":"2024-09-10T22:11:49.180789+0000","last_peered":"2024-09-10T22:11:49.180789+0000","last_clean":"2024-09-10T22:11:49.180789+0000","last_became_active":"2024-09-10T22:11:48.192299+0000","last_became_peered":"2024-09-10T22:11:48.192299+0000","last_unstale":"2024-09-10T22:11:49.180789+0000","last_undegraded":"2024-09-10T22:11:49.180789+0000","last_fullsized":"2024-09-10T22:11:49.180789+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:22.961644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":23,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263811+0000","last_change":"2024-09-10T22:11:47.184300+0000","last_active":"2024-09-10T22:11:49.263811+0000","last_peered":"2024-09-10T22:11:49.263811+0000","last_clean":"2024-09-10T22:11:49.263811+0000","last_became_active":"2024-09-10T22:11:47.184039+0000","last_became_peered":"2024-09-10T22:11:47.184039+0000","last_unstale":"2024-09-10T22:11:49.263811+0000","last_undegraded":"2024-09-10T22:11:49.263811+0000","last_fullsized":"2024-09-10T22:11:49.263811+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:39.714146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265318+0000","last_change":"2024-09-10T22:11:48.189515+0000","last_active":"2024-09-10T22:11:49.265318+0000","last_peered":"2024-09-10T22:11:49.265318+0000","last_clean":"2024-09-10T22:11:49.265318+0000","last_became_active":"2024-09-10T22:11:48.189102+0000","last_became_peered":"2024-09-10T22:11:48.189102+0000","last_unstale":"2024-09-10T22:11:49.265318+0000","last_undegraded":"2024-09-10T22:11:49.265318+0000","last_fullsized":"2024-09-10T22:11:49.265318+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:57.585521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265585+0000","last_change":"2024-09-10T22:11:47.182799+0000","last_active":"2024-09-10T22:11:49.265585+0000","last_peered":"2024-09-10T22:11:49.265585+0000","last_clean":"2024-09-10T22:11:49.265585+0000","last_became_active":"2024-09-10T22:11:47.180852+0000","last_became_peered":"2024-09-10T22:11:47.180852+0000","last_unstale":"2024-09-10T22:11:49.265585+0000","last_undegraded":"2024-09-10T22:11:49.265585+0000","last_fullsized":"2024-09-10T22:11:49.265585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:26.638919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265757+0000","last_change":"2024-09-10T22:11:48.195154+0000","last_active":"2024-09-10T22:11:49.265757+0000","last_peered":"2024-09-10T22:11:49.265757+0000","last_clean":"2024-09-10T22:11:49.265757+0000","last_became_active":"2024-09-10T22:11:48.194886+0000","last_became_peered":"2024-09-10T22:11:48.194886+0000","last_unstale":"2024-09-10T22:11:49.265757+0000","last_undegraded":"2024-09-10T22:11:49.265757+0000","last_fullsized":"2024-09-10T22:11:49.265757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:19.109740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265379+0000","last_change":"2024-09-10T22:11:47.174576+0000","last_active":"2024-09-10T22:11:49.265379+0000","last_peered":"2024-09-10T22:11:49.265379+0000","last_clean":"2024-09-10T22:11:49.265379+0000","last_became_active":"2024-09-10T22:11:47.174204+0000","last_became_peered":"2024-09-10T22:11:47.174204+0000","last_unstale":"2024-09-10T22:11:49.265379+0000","last_undegraded":"2024-09-10T22:11:49.265379+0000","last_fullsized":"2024-09-10T22:11:49.265379+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:55.279368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174644+0000","last_change":"2024-09-10T22:11:47.174644+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174644+0000","last_undegraded":"2024-09-10T22:11:47.174644+0000","last_fullsized":"2024-09-10T22:11:47.174644+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268594+0000","last_change":"2024-09-10T22:11:47.191138+0000","last_active":"2024-09-10T22:11:49.268594+0000","last_peered":"2024-09-10T22:11:49.268594+0000","last_clean":"2024-09-10T22:11:49.268594+0000","last_became_active":"2024-09-10T22:11:47.190913+0000","last_became_peered":"2024-09-10T22:11:47.190913+0000","last_unstale":"2024-09-10T22:11:49.268594+0000","last_undegraded":"2024-09-10T22:11:49.268594+0000","last_fullsized":"2024-09-10T22:11:49.268594+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:20.539547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173918+0000","last_change":"2024-09-10T22:11:47.173918+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173918+0000","last_undegraded":"2024-09-10T22:11:47.173918+0000","last_fullsized":"2024-09-10T22:11:47.173918+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186356+0000","last_change":"2024-09-10T22:11:47.182589+0000","last_active":"2024-09-10T22:11:49.186356+0000","last_peered":"2024-09-10T22:11:49.186356+0000","last_clean":"2024-09-10T22:11:49.186356+0000","last_became_active":"2024-09-10T22:11:47.182316+0000","last_became_peered":"2024-09-10T22:11:47.182316+0000","last_unstale":"2024-09-10T22:11:49.186356+0000","last_undegraded":"2024-09-10T22:11:49.186356+0000","last_fullsized":"2024-09-10T22:11:49.186356+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:53.817114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174289+0000","last_change":"2024-09-10T22:11:47.174289+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174289+0000","last_undegraded":"2024-09-10T22:11:47.174289+0000","last_fullsized":"2024-09-10T22:11:47.174289+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265637+0000","last_change":"2024-09-10T22:11:47.187951+0000","last_active":"2024-09-10T22:11:49.265637+0000","last_peered":"2024-09-10T22:11:49.265637+0000","last_clean":"2024-09-10T22:11:49.265637+0000","last_became_active":"2024-09-10T22:11:47.187697+0000","last_became_peered":"2024-09-10T22:11:47.187697+0000","last_unstale":"2024-09-10T22:11:49.265637+0000","last_undegraded":"2024-09-10T22:11:49.265637+0000","last_fullsized":"2024-09-10T22:11:49.265637+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:16.898662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263621+0000","last_change":"2024-09-10T22:11:48.179679+0000","last_active":"2024-09-10T22:11:49.263621+0000","last_peered":"2024-09-10T22:11:49.263621+0000","last_clean":"2024-09-10T22:11:49.263621+0000","last_became_active":"2024-09-10T22:11:48.179414+0000","last_became_peered":"2024-09-10T22:11:48.179414+0000","last_unstale":"2024-09-10T22:11:49.263621+0000","last_undegraded":"2024-09-10T22:11:49.263621+0000","last_fullsized":"2024-09-10T22:11:49.263621+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:00.909311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263535+0000","last_change":"2024-09-10T22:11:47.181464+0000","last_active":"2024-09-10T22:11:49.263535+0000","last_peered":"2024-09-10T22:11:49.263535+0000","last_clean":"2024-09-10T22:11:49.263535+0000","last_became_active":"2024-09-10T22:11:47.180230+0000","last_became_peered":"2024-09-10T22:11:47.180230+0000","last_unstale":"2024-09-10T22:11:49.263535+0000","last_undegraded":"2024-09-10T22:11:49.263535+0000","last_fullsized":"2024-09-10T22:11:49.263535+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:31.441376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173811+0000","last_change":"2024-09-10T22:11:47.173811+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173811+0000","last_undegraded":"2024-09-10T22:11:47.173811+0000","last_fullsized":"2024-09-10T22:11:47.173811+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264725+0000","last_change":"2024-09-10T22:11:47.188145+0000","last_active":"2024-09-10T22:11:49.264725+0000","last_peered":"2024-09-10T22:11:49.264725+0000","last_clean":"2024-09-10T22:11:49.264725+0000","last_became_active":"2024-09-10T22:11:47.187866+0000","last_became_peered":"2024-09-10T22:11:47.187866+0000","last_unstale":"2024-09-10T22:11:49.264725+0000","last_undegraded":"2024-09-10T22:11:49.264725+0000","last_fullsized":"2024-09-10T22:11:49.264725+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:12.664373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180993+0000","last_change":"2024-09-10T22:11:48.190644+0000","last_active":"2024-09-10T22:11:49.180993+0000","last_peered":"2024-09-10T22:11:49.180993+0000","last_clean":"2024-09-10T22:11:49.180993+0000","last_became_active":"2024-09-10T22:11:48.190081+0000","last_became_peered":"2024-09-10T22:11:48.190081+0000","last_unstale":"2024-09-10T22:11:49.180993+0000","last_undegraded":"2024-09-10T22:11:49.180993+0000","last_fullsized":"2024-09-10T22:11:49.180993+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:14.531071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264069+0000","last_change":"2024-09-10T22:11:47.174439+0000","last_active":"2024-09-10T22:11:49.264069+0000","last_peered":"2024-09-10T22:11:49.264069+0000","last_clean":"2024-09-10T22:11:49.264069+0000","last_became_active":"2024-09-10T22:11:47.174130+0000","last_became_peered":"2024-09-10T22:11:47.174130+0000","last_unstale":"2024-09-10T22:11:49.264069+0000","last_undegraded":"2024-09-10T22:11:49.264069+0000","last_fullsized":"2024-09-10T22:11:49.264069+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:57.534207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268768+0000","last_change":"2024-09-10T22:11:48.187621+0000","last_active":"2024-09-10T22:11:49.268768+0000","last_peered":"2024-09-10T22:11:49.268768+0000","last_clean":"2024-09-10T22:11:49.268768+0000","last_became_active":"2024-09-10T22:11:48.187340+0000","last_became_peered":"2024-09-10T22:11:48.187340+0000","last_unstale":"2024-09-10T22:11:49.268768+0000","last_undegraded":"2024-09-10T22:11:49.268768+0000","last_fullsized":"2024-09-10T22:11:49.268768+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:08.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181466+0000","last_change":"2024-09-10T22:11:47.185574+0000","last_active":"2024-09-10T22:11:49.181466+0000","last_peered":"2024-09-10T22:11:49.181466+0000","last_clean":"2024-09-10T22:11:49.181466+0000","last_became_active":"2024-09-10T22:11:47.185262+0000","last_became_peered":"2024-09-10T22:11:47.185262+0000","last_unstale":"2024-09-10T22:11:49.181466+0000","last_undegraded":"2024-09-10T22:11:49.181466+0000","last_fullsized":"2024-09-10T22:11:49.181466+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:14.036978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265445+0000","last_change":"2024-09-10T22:11:48.191766+0000","last_active":"2024-09-10T22:11:49.265445+0000","last_peered":"2024-09-10T22:11:49.265445+0000","last_clean":"2024-09-10T22:11:49.265445+0000","last_became_active":"2024-09-10T22:11:48.191476+0000","last_became_peered":"2024-09-10T22:11:48.191476+0000","last_unstale":"2024-09-10T22:11:49.265445+0000","last_undegraded":"2024-09-10T22:11:49.265445+0000","last_fullsized":"2024-09-10T22:11:49.265445+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:35.382116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268957+0000","last_change":"2024-09-10T22:11:47.178850+0000","last_active":"2024-09-10T22:11:49.268957+0000","last_peered":"2024-09-10T22:11:49.268957+0000","last_clean":"2024-09-10T22:11:49.268957+0000","last_became_active":"2024-09-10T22:11:47.178587+0000","last_became_peered":"2024-09-10T22:11:47.178587+0000","last_unstale":"2024-09-10T22:11:49.268957+0000","last_undegraded":"2024-09-10T22:11:49.268957+0000","last_fullsized":"2024-09-10T22:11:49.268957+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:23.791045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264595+0000","last_change":"2024-09-10T22:11:48.185926+0000","last_active":"2024-09-10T22:11:49.264595+0000","last_peered":"2024-09-10T22:11:49.264595+0000","last_clean":"2024-09-10T22:11:49.264595+0000","last_became_active":"2024-09-10T22:11:48.185651+0000","last_became_peered":"2024-09-10T22:11:48.185651+0000","last_unstale":"2024-09-10T22:11:49.264595+0000","last_undegraded":"2024-09-10T22:11:49.264595+0000","last_fullsized":"2024-09-10T22:11:49.264595+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:12.374323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264243+0000","last_change":"2024-09-10T22:11:47.175153+0000","last_active":"2024-09-10T22:11:49.264243+0000","last_peered":"2024-09-10T22:11:49.264243+0000","last_clean":"2024-09-10T22:11:49.264243+0000","last_became_active":"2024-09-10T22:11:47.174910+0000","last_became_peered":"2024-09-10T22:11:47.174910+0000","last_unstale":"2024-09-10T22:11:49.264243+0000","last_undegraded":"2024-09-10T22:11:49.264243+0000","last_fullsized":"2024-09-10T22:11:49.264243+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:07.637877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268469+0000","last_change":"2024-09-10T22:11:48.186801+0000","last_active":"2024-09-10T22:11:49.268469+0000","last_peered":"2024-09-10T22:11:49.268469+0000","last_clean":"2024-09-10T22:11:49.268469+0000","last_became_active":"2024-09-10T22:11:48.186510+0000","last_became_peered":"2024-09-10T22:11:48.186510+0000","last_unstale":"2024-09-10T22:11:49.268469+0000","last_undegraded":"2024-09-10T22:11:49.268469+0000","last_fullsized":"2024-09-10T22:11:49.268469+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:07.277597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181554+0000","last_change":"2024-09-10T22:11:47.190009+0000","last_active":"2024-09-10T22:11:49.181554+0000","last_peered":"2024-09-10T22:11:49.181554+0000","last_clean":"2024-09-10T22:11:49.181554+0000","last_became_active":"2024-09-10T22:11:47.189727+0000","last_became_peered":"2024-09-10T22:11:47.189727+0000","last_unstale":"2024-09-10T22:11:49.181554+0000","last_undegraded":"2024-09-10T22:11:49.181554+0000","last_fullsized":"2024-09-10T22:11:49.181554+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:30.357142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187529+0000","last_change":"2024-09-10T22:11:48.195733+0000","last_active":"2024-09-10T22:11:49.187529+0000","last_peered":"2024-09-10T22:11:49.187529+0000","last_clean":"2024-09-10T22:11:49.187529+0000","last_became_active":"2024-09-10T22:11:48.195182+0000","last_became_peered":"2024-09-10T22:11:48.195182+0000","last_unstale":"2024-09-10T22:11:49.187529+0000","last_undegraded":"2024-09-10T22:11:49.187529+0000","last_fullsized":"2024-09-10T22:11:49.187529+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:57.314793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186601+0000","last_change":"2024-09-10T22:11:47.182353+0000","last_active":"2024-09-10T22:11:49.186601+0000","last_peered":"2024-09-10T22:11:49.186601+0000","last_clean":"2024-09-10T22:11:49.186601+0000","last_became_active":"2024-09-10T22:11:47.182082+0000","last_became_peered":"2024-09-10T22:11:47.182082+0000","last_unstale":"2024-09-10T22:11:49.186601+0000","last_undegraded":"2024-09-10T22:11:49.186601+0000","last_fullsized":"2024-09-10T22:11:49.186601+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:37.435038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187229+0000","last_change":"2024-09-10T22:11:48.200370+0000","last_active":"2024-09-10T22:11:49.187229+0000","last_peered":"2024-09-10T22:11:49.187229+0000","last_clean":"2024-09-10T22:11:49.187229+0000","last_became_active":"2024-09-10T22:11:48.200102+0000","last_became_peered":"2024-09-10T22:11:48.200102+0000","last_unstale":"2024-09-10T22:11:49.187229+0000","last_undegraded":"2024-09-10T22:11:49.187229+0000","last_fullsized":"2024-09-10T22:11:49.187229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:23.091121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187581+0000","last_change":"2024-09-10T22:11:47.177554+0000","last_active":"2024-09-10T22:11:49.187581+0000","last_peered":"2024-09-10T22:11:49.187581+0000","last_clean":"2024-09-10T22:11:49.187581+0000","last_became_active":"2024-09-10T22:11:47.177260+0000","last_became_peered":"2024-09-10T22:11:47.177260+0000","last_unstale":"2024-09-10T22:11:49.187581+0000","last_undegraded":"2024-09-10T22:11:49.187581+0000","last_fullsized":"2024-09-10T22:11:49.187581+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:53.033722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173263+0000","last_change":"2024-09-10T22:11:47.173263+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173263+0000","last_undegraded":"2024-09-10T22:11:47.173263+0000","last_fullsized":"2024-09-10T22:11:47.173263+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263945+0000","last_change":"2024-09-10T22:11:47.181053+0000","last_active":"2024-09-10T22:11:49.263945+0000","last_peered":"2024-09-10T22:11:49.263945+0000","last_clean":"2024-09-10T22:11:49.263945+0000","last_became_active":"2024-09-10T22:11:47.180799+0000","last_became_peered":"2024-09-10T22:11:47.180799+0000","last_unstale":"2024-09-10T22:11:49.263945+0000","last_undegraded":"2024-09-10T22:11:49.263945+0000","last_fullsized":"2024-09-10T22:11:49.263945+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:16.715511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263656+0000","last_change":"2024-09-10T22:11:48.184225+0000","last_active":"2024-09-10T22:11:49.263656+0000","last_peered":"2024-09-10T22:11:49.263656+0000","last_clean":"2024-09-10T22:11:49.263656+0000","last_became_active":"2024-09-10T22:11:48.183190+0000","last_became_peered":"2024-09-10T22:11:48.183190+0000","last_unstale":"2024-09-10T22:11:49.263656+0000","last_undegraded":"2024-09-10T22:11:49.263656+0000","last_fullsized":"2024-09-10T22:11:49.263656+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:30.164067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264055+0000","last_change":"2024-09-10T22:11:47.180051+0000","last_active":"2024-09-10T22:11:49.264055+0000","last_peered":"2024-09-10T22:11:49.264055+0000","last_clean":"2024-09-10T22:11:49.264055+0000","last_became_active":"2024-09-10T22:11:47.179772+0000","last_became_peered":"2024-09-10T22:11:47.179772+0000","last_unstale":"2024-09-10T22:11:49.264055+0000","last_undegraded":"2024-09-10T22:11:49.264055+0000","last_fullsized":"2024-09-10T22:11:49.264055+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:55.541109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180927+0000","last_change":"2024-09-10T22:11:48.177392+0000","last_active":"2024-09-10T22:11:49.180927+0000","last_peered":"2024-09-10T22:11:49.180927+0000","last_clean":"2024-09-10T22:11:49.180927+0000","last_became_active":"2024-09-10T22:11:48.176877+0000","last_became_peered":"2024-09-10T22:11:48.176877+0000","last_unstale":"2024-09-10T22:11:49.180927+0000","last_undegraded":"2024-09-10T22:11:49.180927+0000","last_fullsized":"2024-09-10T22:11:49.180927+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:58.537155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186582+0000","last_change":"2024-09-10T22:11:47.183264+0000","last_active":"2024-09-10T22:11:49.186582+0000","last_peered":"2024-09-10T22:11:49.186582+0000","last_clean":"2024-09-10T22:11:49.186582+0000","last_became_active":"2024-09-10T22:11:47.182945+0000","last_became_peered":"2024-09-10T22:11:47.182945+0000","last_unstale":"2024-09-10T22:11:49.186582+0000","last_undegraded":"2024-09-10T22:11:49.186582+0000","last_fullsized":"2024-09-10T22:11:49.186582+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:11.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":279,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.178911+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:11:47.178911+0000","last_peered":"2024-09-10T22:11:47.178911+0000","last_clean":"2024-09-10T22:11:47.178911+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:11:47.178911+0000","last_undegraded":"2024-09-10T22:11:47.178911+0000","last_fullsized":"2024-09-10T22:11:47.178911+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509580,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127436,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244404,"statfs":{"total":96636764160,"available":96506269696,"internally_reserved":0,"allocated":1953792,"data_stored":1179333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118568,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253272,"statfs":{"total":96636764160,"available":96515350528,"internally_reserved":0,"allocated":1388544,"data_stored":602278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104676,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267164,"statfs":{"total":96636764160,"available":96529575936,"internally_reserved":0,"allocated":1392640,"data_stored":602812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149088,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222752,"statfs":{"total":96636764160,"available":96484098048,"internally_reserved":0,"allocated":1384448,"data_stored":602208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144948,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226892,"statfs":{"total":96636764160,"available":96488337408,"internally_reserved":0,"allocated":1400832,"data_stored":603890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509581,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131044,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240796,"statfs":{"total":96636764160,"available":96502575104,"internally_reserved":0,"allocated":1392640,"data_stored":602332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509581,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127476,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244364,"statfs":{"total":96636764160,"available":96506228736,"internally_reserved":0,"allocated":1998848,"data_stored":1194300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":128514505},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542286,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144928,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226912,"statfs":{"total":96636764160,"available":96488357888,"internally_reserved":0,"allocated":1384448,"data_stored":602208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:11:52.733 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:11:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.916+0000 7f0e11800640 1 -- 172.21.15.66:0/1499693580 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e0c06e900 msgr2=0x7f0e0c0b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.916+0000 7f0e11800640 1 --2- 172.21.15.66:0/1499693580 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e0c06e900 0x7f0e0c0b3060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009870 tx=0x7f0df801f050 comp rx=0 tx=0).stop 2024-09-10T22:11:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.916+0000 7f0e11800640 1 -- 172.21.15.66:0/1499693580 shutdown_connections 2024-09-10T22:11:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.916+0000 7f0e11800640 1 --2- 172.21.15.66:0/1499693580 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e0c06e900 0x7f0e0c0b3060 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.916+0000 7f0e11800640 1 -- 172.21.15.66:0/1499693580 >> 172.21.15.66:0/1499693580 conn(0x7f0e0c0d1c60 msgr2=0x7f0e0c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.916+0000 7f0e11800640 1 -- 172.21.15.66:0/1499693580 shutdown_connections 2024-09-10T22:11:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.916+0000 7f0e11800640 1 -- 172.21.15.66:0/1499693580 wait complete. 2024-09-10T22:11:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.916+0000 7f0e11800640 1 Processor -- start 2024-09-10T22:11:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e11800640 1 -- start start 2024-09-10T22:11:52.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e11800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c06dcd0 0x7f0e0c151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e11800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e0c06e900 0x7f0e0c0c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e11800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e0c161570 0x7f0e0c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e11800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c0d0eb0 con 0x7f0e0c161570 2024-09-10T22:11:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e11800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c0d0d30 con 0x7f0e0c06dcd0 2024-09-10T22:11:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e11800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c0d1030 con 0x7f0e0c06e900 2024-09-10T22:11:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e0b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e0c161570 0x7f0e0c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e0b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e0c161570 0x7f0e0c0b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33086/0 (socket says 172.21.15.66:33086) 2024-09-10T22:11:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e0b400640 1 -- 172.21.15.66:0/3098618919 learned_addr learned my addr 172.21.15.66:0/3098618919 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e0a000640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e0c06e900 0x7f0e0c0c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e0aa00640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c06dcd0 0x7f0e0c151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e0b400640 1 -- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e0c06e900 msgr2=0x7f0e0c0c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e0b400640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e0c06e900 0x7f0e0c0c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e0b400640 1 -- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c06dcd0 msgr2=0x7f0e0c151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e0b400640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c06dcd0 0x7f0e0c151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e0b400640 1 -- 172.21.15.66:0/3098618919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c1f8050 con 0x7f0e0c161570 2024-09-10T22:11:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e0aa00640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c06dcd0 0x7f0e0c151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e0b400640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e0c161570 0x7f0e0c0b5ff0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400cbc0 tx=0x7f0e0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e02a00640 1 -- 172.21.15.66:0/3098618919 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04016020 con 0x7f0e0c161570 2024-09-10T22:11:52.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e02a00640 1 -- 172.21.15.66:0/3098618919 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e0400ab00 con 0x7f0e0c161570 2024-09-10T22:11:52.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e02a00640 1 -- 172.21.15.66:0/3098618919 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04012420 con 0x7f0e0c161570 2024-09-10T22:11:52.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e11800640 1 -- 172.21.15.66:0/3098618919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c1f8330 con 0x7f0e0c161570 2024-09-10T22:11:52.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e11800640 1 -- 172.21.15.66:0/3098618919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c1f88f0 con 0x7f0e0c161570 2024-09-10T22:11:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.920+0000 7f0e11800640 1 -- 172.21.15.66:0/3098618919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0c1f8b30 con 0x7f0e0c161570 2024-09-10T22:11:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.924+0000 7f0e02a00640 1 -- 172.21.15.66:0/3098618919 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f0e040125c0 con 0x7f0e0c161570 2024-09-10T22:11:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.924+0000 7f0e02a00640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0dec073f30 0x7f0dec0763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.924+0000 7f0e02a00640 1 -- 172.21.15.66:0/3098618919 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f0e04092580 con 0x7f0e0c161570 2024-09-10T22:11:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.924+0000 7f0e0aa00640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0dec073f30 0x7f0dec0763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.924+0000 7f0e0aa00640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0dec073f30 0x7f0dec0763f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc000c10 tx=0x7f0dfc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:52.928+0000 7f0e02a00640 1 -- 172.21.15.66:0/3098618919 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0e0c1f8b30 con 0x7f0e0c161570 2024-09-10T22:11:53.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.096+0000 7f0e11800640 1 -- 172.21.15.66:0/3098618919 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e0c0af420 con 0x7f0dec073f30 2024-09-10T22:11:53.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.100+0000 7f0e02a00640 1 -- 172.21.15.66:0/3098618919 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251590 (secure 0 0 0) 0x7f0e0c0af420 con 0x7f0dec073f30 2024-09-10T22:11:53.108 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:53.110 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:11:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.104+0000 7f0e11800640 1 -- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0dec073f30 msgr2=0x7f0dec0763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.104+0000 7f0e11800640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0dec073f30 0x7f0dec0763f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc000c10 tx=0x7f0dfc009040 comp rx=0 tx=0).stop 2024-09-10T22:11:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.104+0000 7f0e11800640 1 -- 172.21.15.66:0/3098618919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e0c161570 msgr2=0x7f0e0c0b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.104+0000 7f0e11800640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e0c161570 0x7f0e0c0b5ff0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400cbc0 tx=0x7f0e0400ccb0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.104+0000 7f0e11800640 1 -- 172.21.15.66:0/3098618919 shutdown_connections 2024-09-10T22:11:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.104+0000 7f0e11800640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0dec073f30 0x7f0dec0763f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.104+0000 7f0e11800640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e0c161570 0x7f0e0c0b5ff0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.104+0000 7f0e11800640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e0c06e900 0x7f0e0c0c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.104+0000 7f0e11800640 1 --2- 172.21.15.66:0/3098618919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c06dcd0 0x7f0e0c151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.104+0000 7f0e11800640 1 -- 172.21.15.66:0/3098618919 >> 172.21.15.66:0/3098618919 conn(0x7f0e0c0d1c60 msgr2=0x7f0e0c0b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:53.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.104+0000 7f0e11800640 1 -- 172.21.15.66:0/3098618919 shutdown_connections 2024-09-10T22:11:53.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.104+0000 7f0e11800640 1 -- 172.21.15.66:0/3098618919 wait complete. 2024-09-10T22:11:53.129 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-10T22:11:52.561491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1048164,"kb_used_data":12008,"kb_used_omap":12,"kb_used_meta":1035955,"kb_avail":753926556,"statfs":{"total":773094113280,"available":772020793344,"internally_reserved":0,"allocated":12296192,"data_stored":5989361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1060818484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001201"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265042+0000","last_change":"2024-09-10T22:11:48.180766+0000","last_active":"2024-09-10T22:11:49.265042+0000","last_peered":"2024-09-10T22:11:49.265042+0000","last_clean":"2024-09-10T22:11:49.265042+0000","last_became_active":"2024-09-10T22:11:48.179819+0000","last_became_peered":"2024-09-10T22:11:48.179819+0000","last_unstale":"2024-09-10T22:11:49.265042+0000","last_undegraded":"2024-09-10T22:11:49.265042+0000","last_fullsized":"2024-09-10T22:11:49.265042+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:21.885630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186814+0000","last_change":"2024-09-10T22:11:48.203979+0000","last_active":"2024-09-10T22:11:49.186814+0000","last_peered":"2024-09-10T22:11:49.186814+0000","last_clean":"2024-09-10T22:11:49.186814+0000","last_became_active":"2024-09-10T22:11:48.203764+0000","last_became_peered":"2024-09-10T22:11:48.203764+0000","last_unstale":"2024-09-10T22:11:49.186814+0000","last_undegraded":"2024-09-10T22:11:49.186814+0000","last_fullsized":"2024-09-10T22:11:49.186814+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:41.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.177096+0000","last_change":"2024-09-10T22:11:47.177096+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.177096+0000","last_undegraded":"2024-09-10T22:11:47.177096+0000","last_fullsized":"2024-09-10T22:11:47.177096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264205+0000","last_change":"2024-09-10T22:11:48.191506+0000","last_active":"2024-09-10T22:11:49.264205+0000","last_peered":"2024-09-10T22:11:49.264205+0000","last_clean":"2024-09-10T22:11:49.264205+0000","last_became_active":"2024-09-10T22:11:48.191100+0000","last_became_peered":"2024-09-10T22:11:48.191100+0000","last_unstale":"2024-09-10T22:11:49.264205+0000","last_undegraded":"2024-09-10T22:11:49.264205+0000","last_fullsized":"2024-09-10T22:11:49.264205+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:54.388719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263748+0000","last_change":"2024-09-10T22:11:48.178358+0000","last_active":"2024-09-10T22:11:49.263748+0000","last_peered":"2024-09-10T22:11:49.263748+0000","last_clean":"2024-09-10T22:11:49.263748+0000","last_became_active":"2024-09-10T22:11:48.177965+0000","last_became_peered":"2024-09-10T22:11:48.177965+0000","last_unstale":"2024-09-10T22:11:49.263748+0000","last_undegraded":"2024-09-10T22:11:49.263748+0000","last_fullsized":"2024-09-10T22:11:49.263748+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:16.479719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264893+0000","last_change":"2024-09-10T22:11:48.182705+0000","last_active":"2024-09-10T22:11:49.264893+0000","last_peered":"2024-09-10T22:11:49.264893+0000","last_clean":"2024-09-10T22:11:49.264893+0000","last_became_active":"2024-09-10T22:11:48.182428+0000","last_became_peered":"2024-09-10T22:11:48.182428+0000","last_unstale":"2024-09-10T22:11:49.264893+0000","last_undegraded":"2024-09-10T22:11:49.264893+0000","last_fullsized":"2024-09-10T22:11:49.264893+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:56.601140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186457+0000","last_change":"2024-09-10T22:11:48.181122+0000","last_active":"2024-09-10T22:11:49.186457+0000","last_peered":"2024-09-10T22:11:49.186457+0000","last_clean":"2024-09-10T22:11:49.186457+0000","last_became_active":"2024-09-10T22:11:48.178614+0000","last_became_peered":"2024-09-10T22:11:48.178614+0000","last_unstale":"2024-09-10T22:11:49.186457+0000","last_undegraded":"2024-09-10T22:11:49.186457+0000","last_fullsized":"2024-09-10T22:11:49.186457+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:54.036670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174226+0000","last_change":"2024-09-10T22:11:47.174226+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174226+0000","last_undegraded":"2024-09-10T22:11:47.174226+0000","last_fullsized":"2024-09-10T22:11:47.174226+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265249+0000","last_change":"2024-09-10T22:11:48.180116+0000","last_active":"2024-09-10T22:11:49.265249+0000","last_peered":"2024-09-10T22:11:49.265249+0000","last_clean":"2024-09-10T22:11:49.265249+0000","last_became_active":"2024-09-10T22:11:48.179490+0000","last_became_peered":"2024-09-10T22:11:48.179490+0000","last_unstale":"2024-09-10T22:11:49.265249+0000","last_undegraded":"2024-09-10T22:11:49.265249+0000","last_fullsized":"2024-09-10T22:11:49.265249+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:19.843998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264589+0000","last_change":"2024-09-10T22:11:48.183940+0000","last_active":"2024-09-10T22:11:49.264589+0000","last_peered":"2024-09-10T22:11:49.264589+0000","last_clean":"2024-09-10T22:11:49.264589+0000","last_became_active":"2024-09-10T22:11:48.183694+0000","last_became_peered":"2024-09-10T22:11:48.183694+0000","last_unstale":"2024-09-10T22:11:49.264589+0000","last_undegraded":"2024-09-10T22:11:49.264589+0000","last_fullsized":"2024-09-10T22:11:49.264589+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:00.322112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187457+0000","last_change":"2024-09-10T22:11:48.197453+0000","last_active":"2024-09-10T22:11:49.187457+0000","last_peered":"2024-09-10T22:11:49.187457+0000","last_clean":"2024-09-10T22:11:49.187457+0000","last_became_active":"2024-09-10T22:11:48.197138+0000","last_became_peered":"2024-09-10T22:11:48.197138+0000","last_unstale":"2024-09-10T22:11:49.187457+0000","last_undegraded":"2024-09-10T22:11:49.187457+0000","last_fullsized":"2024-09-10T22:11:49.187457+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:33.059467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264403+0000","last_change":"2024-09-10T22:11:48.189077+0000","last_active":"2024-09-10T22:11:49.264403+0000","last_peered":"2024-09-10T22:11:49.264403+0000","last_clean":"2024-09-10T22:11:49.264403+0000","last_became_active":"2024-09-10T22:11:48.185421+0000","last_became_peered":"2024-09-10T22:11:48.185421+0000","last_unstale":"2024-09-10T22:11:49.264403+0000","last_undegraded":"2024-09-10T22:11:49.264403+0000","last_fullsized":"2024-09-10T22:11:49.264403+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:25.288843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.175144+0000","last_change":"2024-09-10T22:11:47.175144+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.175144+0000","last_undegraded":"2024-09-10T22:11:47.175144+0000","last_fullsized":"2024-09-10T22:11:47.175144+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174758+0000","last_change":"2024-09-10T22:11:47.174758+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174758+0000","last_undegraded":"2024-09-10T22:11:47.174758+0000","last_fullsized":"2024-09-10T22:11:47.174758+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264482+0000","last_change":"2024-09-10T22:11:48.183435+0000","last_active":"2024-09-10T22:11:49.264482+0000","last_peered":"2024-09-10T22:11:49.264482+0000","last_clean":"2024-09-10T22:11:49.264482+0000","last_became_active":"2024-09-10T22:11:48.183243+0000","last_became_peered":"2024-09-10T22:11:48.183243+0000","last_unstale":"2024-09-10T22:11:49.264482+0000","last_undegraded":"2024-09-10T22:11:49.264482+0000","last_fullsized":"2024-09-10T22:11:49.264482+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:11.483616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263496+0000","last_change":"2024-09-10T22:11:48.186035+0000","last_active":"2024-09-10T22:11:49.263496+0000","last_peered":"2024-09-10T22:11:49.263496+0000","last_clean":"2024-09-10T22:11:49.263496+0000","last_became_active":"2024-09-10T22:11:48.185768+0000","last_became_peered":"2024-09-10T22:11:48.185768+0000","last_unstale":"2024-09-10T22:11:49.263496+0000","last_undegraded":"2024-09-10T22:11:49.263496+0000","last_fullsized":"2024-09-10T22:11:49.263496+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:22.226210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187717+0000","last_change":"2024-09-10T22:11:48.200285+0000","last_active":"2024-09-10T22:11:49.187717+0000","last_peered":"2024-09-10T22:11:49.187717+0000","last_clean":"2024-09-10T22:11:49.187717+0000","last_became_active":"2024-09-10T22:11:48.199965+0000","last_became_peered":"2024-09-10T22:11:48.199965+0000","last_unstale":"2024-09-10T22:11:49.187717+0000","last_undegraded":"2024-09-10T22:11:49.187717+0000","last_fullsized":"2024-09-10T22:11:49.187717+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:22.961644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268722+0000","last_change":"2024-09-10T22:11:48.194873+0000","last_active":"2024-09-10T22:11:49.268722+0000","last_peered":"2024-09-10T22:11:49.268722+0000","last_clean":"2024-09-10T22:11:49.268722+0000","last_became_active":"2024-09-10T22:11:48.194469+0000","last_became_peered":"2024-09-10T22:11:48.194469+0000","last_unstale":"2024-09-10T22:11:49.268722+0000","last_undegraded":"2024-09-10T22:11:49.268722+0000","last_fullsized":"2024-09-10T22:11:49.268722+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:37.166232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.176608+0000","last_change":"2024-09-10T22:11:47.176608+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.176608+0000","last_undegraded":"2024-09-10T22:11:47.176608+0000","last_fullsized":"2024-09-10T22:11:47.176608+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186908+0000","last_change":"2024-09-10T22:11:48.199337+0000","last_active":"2024-09-10T22:11:49.186908+0000","last_peered":"2024-09-10T22:11:49.186908+0000","last_clean":"2024-09-10T22:11:49.186908+0000","last_became_active":"2024-09-10T22:11:48.199028+0000","last_became_peered":"2024-09-10T22:11:48.199028+0000","last_unstale":"2024-09-10T22:11:49.186908+0000","last_undegraded":"2024-09-10T22:11:49.186908+0000","last_fullsized":"2024-09-10T22:11:49.186908+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:57.807567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187347+0000","last_change":"2024-09-10T22:11:48.200296+0000","last_active":"2024-09-10T22:11:49.187347+0000","last_peered":"2024-09-10T22:11:49.187347+0000","last_clean":"2024-09-10T22:11:49.187347+0000","last_became_active":"2024-09-10T22:11:48.198375+0000","last_became_peered":"2024-09-10T22:11:48.198375+0000","last_unstale":"2024-09-10T22:11:49.187347+0000","last_undegraded":"2024-09-10T22:11:49.187347+0000","last_fullsized":"2024-09-10T22:11:49.187347+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:45.662642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180916+0000","last_change":"2024-09-10T22:11:48.180779+0000","last_active":"2024-09-10T22:11:49.180916+0000","last_peered":"2024-09-10T22:11:49.180916+0000","last_clean":"2024-09-10T22:11:49.180916+0000","last_became_active":"2024-09-10T22:11:48.179454+0000","last_became_peered":"2024-09-10T22:11:48.179454+0000","last_unstale":"2024-09-10T22:11:49.180916+0000","last_undegraded":"2024-09-10T22:11:49.180916+0000","last_fullsized":"2024-09-10T22:11:49.180916+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:28:27.233254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186778+0000","last_change":"2024-09-10T22:11:48.198947+0000","last_active":"2024-09-10T22:11:49.186778+0000","last_peered":"2024-09-10T22:11:49.186778+0000","last_clean":"2024-09-10T22:11:49.186778+0000","last_became_active":"2024-09-10T22:11:48.198672+0000","last_became_peered":"2024-09-10T22:11:48.198672+0000","last_unstale":"2024-09-10T22:11:49.186778+0000","last_undegraded":"2024-09-10T22:11:49.186778+0000","last_fullsized":"2024-09-10T22:11:49.186778+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:08.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180712+0000","last_change":"2024-09-10T22:11:48.177447+0000","last_active":"2024-09-10T22:11:49.180712+0000","last_peered":"2024-09-10T22:11:49.180712+0000","last_clean":"2024-09-10T22:11:49.180712+0000","last_became_active":"2024-09-10T22:11:48.177207+0000","last_became_peered":"2024-09-10T22:11:48.177207+0000","last_unstale":"2024-09-10T22:11:49.180712+0000","last_undegraded":"2024-09-10T22:11:49.180712+0000","last_fullsized":"2024-09-10T22:11:49.180712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:57.314793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187731+0000","last_change":"2024-09-10T22:11:48.201225+0000","last_active":"2024-09-10T22:11:49.187731+0000","last_peered":"2024-09-10T22:11:49.187731+0000","last_clean":"2024-09-10T22:11:49.187731+0000","last_became_active":"2024-09-10T22:11:48.200931+0000","last_became_peered":"2024-09-10T22:11:48.200931+0000","last_unstale":"2024-09-10T22:11:49.187731+0000","last_undegraded":"2024-09-10T22:11:49.187731+0000","last_fullsized":"2024-09-10T22:11:49.187731+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:16.417992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264799+0000","last_change":"2024-09-10T22:11:48.185741+0000","last_active":"2024-09-10T22:11:49.264799+0000","last_peered":"2024-09-10T22:11:49.264799+0000","last_clean":"2024-09-10T22:11:49.264799+0000","last_became_active":"2024-09-10T22:11:48.185438+0000","last_became_peered":"2024-09-10T22:11:48.185438+0000","last_unstale":"2024-09-10T22:11:49.264799+0000","last_undegraded":"2024-09-10T22:11:49.264799+0000","last_fullsized":"2024-09-10T22:11:49.264799+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:58.072179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265352+0000","last_change":"2024-09-10T22:11:48.195500+0000","last_active":"2024-09-10T22:11:49.265352+0000","last_peered":"2024-09-10T22:11:49.265352+0000","last_clean":"2024-09-10T22:11:49.265352+0000","last_became_active":"2024-09-10T22:11:48.195019+0000","last_became_peered":"2024-09-10T22:11:48.195019+0000","last_unstale":"2024-09-10T22:11:49.265352+0000","last_undegraded":"2024-09-10T22:11:49.265352+0000","last_fullsized":"2024-09-10T22:11:49.265352+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:18.164228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181343+0000","last_change":"2024-09-10T22:11:48.189755+0000","last_active":"2024-09-10T22:11:49.181343+0000","last_peered":"2024-09-10T22:11:49.181343+0000","last_clean":"2024-09-10T22:11:49.181343+0000","last_became_active":"2024-09-10T22:11:48.189436+0000","last_became_peered":"2024-09-10T22:11:48.189436+0000","last_unstale":"2024-09-10T22:11:49.181343+0000","last_undegraded":"2024-09-10T22:11:49.181343+0000","last_fullsized":"2024-09-10T22:11:49.181343+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:42.442483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187041+0000","last_change":"2024-09-10T22:11:48.199608+0000","last_active":"2024-09-10T22:11:49.187041+0000","last_peered":"2024-09-10T22:11:49.187041+0000","last_clean":"2024-09-10T22:11:49.187041+0000","last_became_active":"2024-09-10T22:11:48.199331+0000","last_became_peered":"2024-09-10T22:11:48.199331+0000","last_unstale":"2024-09-10T22:11:49.187041+0000","last_undegraded":"2024-09-10T22:11:49.187041+0000","last_fullsized":"2024-09-10T22:11:49.187041+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:20.253988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186852+0000","last_change":"2024-09-10T22:11:48.184404+0000","last_active":"2024-09-10T22:11:49.186852+0000","last_peered":"2024-09-10T22:11:49.186852+0000","last_clean":"2024-09-10T22:11:49.186852+0000","last_became_active":"2024-09-10T22:11:48.180465+0000","last_became_peered":"2024-09-10T22:11:48.180465+0000","last_unstale":"2024-09-10T22:11:49.186852+0000","last_undegraded":"2024-09-10T22:11:49.186852+0000","last_fullsized":"2024-09-10T22:11:49.186852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:02.597092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263547+0000","last_change":"2024-09-10T22:11:48.180454+0000","last_active":"2024-09-10T22:11:49.263547+0000","last_peered":"2024-09-10T22:11:49.263547+0000","last_clean":"2024-09-10T22:11:49.263547+0000","last_became_active":"2024-09-10T22:11:48.179907+0000","last_became_peered":"2024-09-10T22:11:48.179907+0000","last_unstale":"2024-09-10T22:11:49.263547+0000","last_undegraded":"2024-09-10T22:11:49.263547+0000","last_fullsized":"2024-09-10T22:11:49.263547+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:15.319909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264477+0000","last_change":"2024-09-10T22:11:48.182798+0000","last_active":"2024-09-10T22:11:49.264477+0000","last_peered":"2024-09-10T22:11:49.264477+0000","last_clean":"2024-09-10T22:11:49.264477+0000","last_became_active":"2024-09-10T22:11:48.182533+0000","last_became_peered":"2024-09-10T22:11:48.182533+0000","last_unstale":"2024-09-10T22:11:49.264477+0000","last_undegraded":"2024-09-10T22:11:49.264477+0000","last_fullsized":"2024-09-10T22:11:49.264477+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:35.986933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265665+0000","last_change":"2024-09-10T22:11:48.184978+0000","last_active":"2024-09-10T22:11:49.265665+0000","last_peered":"2024-09-10T22:11:49.265665+0000","last_clean":"2024-09-10T22:11:49.265665+0000","last_became_active":"2024-09-10T22:11:48.184722+0000","last_became_peered":"2024-09-10T22:11:48.184722+0000","last_unstale":"2024-09-10T22:11:49.265665+0000","last_undegraded":"2024-09-10T22:11:49.265665+0000","last_fullsized":"2024-09-10T22:11:49.265665+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:48.904032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187304+0000","last_change":"2024-09-10T22:11:47.179953+0000","last_active":"2024-09-10T22:11:49.187304+0000","last_peered":"2024-09-10T22:11:49.187304+0000","last_clean":"2024-09-10T22:11:49.187304+0000","last_became_active":"2024-09-10T22:11:47.179653+0000","last_became_peered":"2024-09-10T22:11:47.179653+0000","last_unstale":"2024-09-10T22:11:49.187304+0000","last_undegraded":"2024-09-10T22:11:49.187304+0000","last_fullsized":"2024-09-10T22:11:49.187304+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:17.150534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268977+0000","last_change":"2024-09-10T22:11:48.186184+0000","last_active":"2024-09-10T22:11:49.268977+0000","last_peered":"2024-09-10T22:11:49.268977+0000","last_clean":"2024-09-10T22:11:49.268977+0000","last_became_active":"2024-09-10T22:11:48.185822+0000","last_became_peered":"2024-09-10T22:11:48.185822+0000","last_unstale":"2024-09-10T22:11:49.268977+0000","last_undegraded":"2024-09-10T22:11:49.268977+0000","last_fullsized":"2024-09-10T22:11:49.268977+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:41.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.185431+0000","last_change":"2024-09-10T22:11:47.184730+0000","last_active":"2024-09-10T22:11:47.185431+0000","last_peered":"2024-09-10T22:11:47.185431+0000","last_clean":"2024-09-10T22:11:47.185431+0000","last_became_active":"2024-09-10T22:11:47.184361+0000","last_became_peered":"2024-09-10T22:11:47.184361+0000","last_unstale":"2024-09-10T22:11:47.185431+0000","last_undegraded":"2024-09-10T22:11:47.185431+0000","last_fullsized":"2024-09-10T22:11:47.185431+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:19.251040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265333+0000","last_change":"2024-09-10T22:11:48.182402+0000","last_active":"2024-09-10T22:11:49.265333+0000","last_peered":"2024-09-10T22:11:49.265333+0000","last_clean":"2024-09-10T22:11:49.265333+0000","last_became_active":"2024-09-10T22:11:48.182057+0000","last_became_peered":"2024-09-10T22:11:48.182057+0000","last_unstale":"2024-09-10T22:11:49.265333+0000","last_undegraded":"2024-09-10T22:11:49.265333+0000","last_fullsized":"2024-09-10T22:11:49.265333+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:18.062008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265043+0000","last_change":"2024-09-10T22:11:47.181206+0000","last_active":"2024-09-10T22:11:49.265043+0000","last_peered":"2024-09-10T22:11:49.265043+0000","last_clean":"2024-09-10T22:11:49.265043+0000","last_became_active":"2024-09-10T22:11:47.179544+0000","last_became_peered":"2024-09-10T22:11:47.179544+0000","last_unstale":"2024-09-10T22:11:49.265043+0000","last_undegraded":"2024-09-10T22:11:49.265043+0000","last_fullsized":"2024-09-10T22:11:49.265043+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:26.154879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174368+0000","last_change":"2024-09-10T22:11:47.174368+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174368+0000","last_undegraded":"2024-09-10T22:11:47.174368+0000","last_fullsized":"2024-09-10T22:11:47.174368+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263584+0000","last_change":"2024-09-10T22:11:47.182277+0000","last_active":"2024-09-10T22:11:49.263584+0000","last_peered":"2024-09-10T22:11:49.263584+0000","last_clean":"2024-09-10T22:11:49.263584+0000","last_became_active":"2024-09-10T22:11:47.181435+0000","last_became_peered":"2024-09-10T22:11:47.181435+0000","last_unstale":"2024-09-10T22:11:49.263584+0000","last_undegraded":"2024-09-10T22:11:49.263584+0000","last_fullsized":"2024-09-10T22:11:49.263584+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:47.604910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263891+0000","last_change":"2024-09-10T22:11:48.181857+0000","last_active":"2024-09-10T22:11:49.263891+0000","last_peered":"2024-09-10T22:11:49.263891+0000","last_clean":"2024-09-10T22:11:49.263891+0000","last_became_active":"2024-09-10T22:11:48.179210+0000","last_became_peered":"2024-09-10T22:11:48.179210+0000","last_unstale":"2024-09-10T22:11:49.263891+0000","last_undegraded":"2024-09-10T22:11:49.263891+0000","last_fullsized":"2024-09-10T22:11:49.263891+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:51.570280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186717+0000","last_change":"2024-09-10T22:11:47.177297+0000","last_active":"2024-09-10T22:11:49.186717+0000","last_peered":"2024-09-10T22:11:49.186717+0000","last_clean":"2024-09-10T22:11:49.186717+0000","last_became_active":"2024-09-10T22:11:47.176973+0000","last_became_peered":"2024-09-10T22:11:47.176973+0000","last_unstale":"2024-09-10T22:11:49.186717+0000","last_undegraded":"2024-09-10T22:11:49.186717+0000","last_fullsized":"2024-09-10T22:11:49.186717+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:08.162909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187075+0000","last_change":"2024-09-10T22:11:48.204412+0000","last_active":"2024-09-10T22:11:49.187075+0000","last_peered":"2024-09-10T22:11:49.187075+0000","last_clean":"2024-09-10T22:11:49.187075+0000","last_became_active":"2024-09-10T22:11:48.204105+0000","last_became_peered":"2024-09-10T22:11:48.204105+0000","last_unstale":"2024-09-10T22:11:49.187075+0000","last_undegraded":"2024-09-10T22:11:49.187075+0000","last_fullsized":"2024-09-10T22:11:49.187075+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:04.692623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181125+0000","last_change":"2024-09-10T22:11:47.184025+0000","last_active":"2024-09-10T22:11:49.181125+0000","last_peered":"2024-09-10T22:11:49.181125+0000","last_clean":"2024-09-10T22:11:49.181125+0000","last_became_active":"2024-09-10T22:11:47.183724+0000","last_became_peered":"2024-09-10T22:11:47.183724+0000","last_unstale":"2024-09-10T22:11:49.181125+0000","last_undegraded":"2024-09-10T22:11:49.181125+0000","last_fullsized":"2024-09-10T22:11:49.181125+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:50.420296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263722+0000","last_change":"2024-09-10T22:11:48.186204+0000","last_active":"2024-09-10T22:11:49.263722+0000","last_peered":"2024-09-10T22:11:49.263722+0000","last_clean":"2024-09-10T22:11:49.263722+0000","last_became_active":"2024-09-10T22:11:48.185455+0000","last_became_peered":"2024-09-10T22:11:48.185455+0000","last_unstale":"2024-09-10T22:11:49.263722+0000","last_undegraded":"2024-09-10T22:11:49.263722+0000","last_fullsized":"2024-09-10T22:11:49.263722+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:57.468704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186931+0000","last_change":"2024-09-10T22:11:47.181785+0000","last_active":"2024-09-10T22:11:49.186931+0000","last_peered":"2024-09-10T22:11:49.186931+0000","last_clean":"2024-09-10T22:11:49.186931+0000","last_became_active":"2024-09-10T22:11:47.181505+0000","last_became_peered":"2024-09-10T22:11:47.181505+0000","last_unstale":"2024-09-10T22:11:49.186931+0000","last_undegraded":"2024-09-10T22:11:49.186931+0000","last_fullsized":"2024-09-10T22:11:49.186931+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:01.594145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186384+0000","last_change":"2024-09-10T22:11:48.181429+0000","last_active":"2024-09-10T22:11:49.186384+0000","last_peered":"2024-09-10T22:11:49.186384+0000","last_clean":"2024-09-10T22:11:49.186384+0000","last_became_active":"2024-09-10T22:11:48.178978+0000","last_became_peered":"2024-09-10T22:11:48.178978+0000","last_unstale":"2024-09-10T22:11:49.186384+0000","last_undegraded":"2024-09-10T22:11:49.186384+0000","last_fullsized":"2024-09-10T22:11:49.186384+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:52.036230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263948+0000","last_change":"2024-09-10T22:11:47.187068+0000","last_active":"2024-09-10T22:11:49.263948+0000","last_peered":"2024-09-10T22:11:49.263948+0000","last_clean":"2024-09-10T22:11:49.263948+0000","last_became_active":"2024-09-10T22:11:47.186777+0000","last_became_peered":"2024-09-10T22:11:47.186777+0000","last_unstale":"2024-09-10T22:11:49.263948+0000","last_undegraded":"2024-09-10T22:11:49.263948+0000","last_fullsized":"2024-09-10T22:11:49.263948+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:21.958696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187129+0000","last_change":"2024-09-10T22:11:48.181451+0000","last_active":"2024-09-10T22:11:49.187129+0000","last_peered":"2024-09-10T22:11:49.187129+0000","last_clean":"2024-09-10T22:11:49.187129+0000","last_became_active":"2024-09-10T22:11:48.181161+0000","last_became_peered":"2024-09-10T22:11:48.181161+0000","last_unstale":"2024-09-10T22:11:49.187129+0000","last_undegraded":"2024-09-10T22:11:49.187129+0000","last_fullsized":"2024-09-10T22:11:49.187129+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:26.021716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265100+0000","last_change":"2024-09-10T22:11:47.179876+0000","last_active":"2024-09-10T22:11:49.265100+0000","last_peered":"2024-09-10T22:11:49.265100+0000","last_clean":"2024-09-10T22:11:49.265100+0000","last_became_active":"2024-09-10T22:11:47.179525+0000","last_became_peered":"2024-09-10T22:11:47.179525+0000","last_unstale":"2024-09-10T22:11:49.265100+0000","last_undegraded":"2024-09-10T22:11:49.265100+0000","last_fullsized":"2024-09-10T22:11:49.265100+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:59.073260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.269023+0000","last_change":"2024-09-10T22:11:48.187132+0000","last_active":"2024-09-10T22:11:49.269023+0000","last_peered":"2024-09-10T22:11:49.269023+0000","last_clean":"2024-09-10T22:11:49.269023+0000","last_became_active":"2024-09-10T22:11:48.186733+0000","last_became_peered":"2024-09-10T22:11:48.186733+0000","last_unstale":"2024-09-10T22:11:49.269023+0000","last_undegraded":"2024-09-10T22:11:49.269023+0000","last_fullsized":"2024-09-10T22:11:49.269023+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:57.807567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181341+0000","last_change":"2024-09-10T22:11:47.183552+0000","last_active":"2024-09-10T22:11:49.181341+0000","last_peered":"2024-09-10T22:11:49.181341+0000","last_clean":"2024-09-10T22:11:49.181341+0000","last_became_active":"2024-09-10T22:11:47.183260+0000","last_became_peered":"2024-09-10T22:11:47.183260+0000","last_unstale":"2024-09-10T22:11:49.181341+0000","last_undegraded":"2024-09-10T22:11:49.181341+0000","last_fullsized":"2024-09-10T22:11:49.181341+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:13.345360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181355+0000","last_change":"2024-09-10T22:11:48.194176+0000","last_active":"2024-09-10T22:11:49.181355+0000","last_peered":"2024-09-10T22:11:49.181355+0000","last_clean":"2024-09-10T22:11:49.181355+0000","last_became_active":"2024-09-10T22:11:48.192478+0000","last_became_peered":"2024-09-10T22:11:48.192478+0000","last_unstale":"2024-09-10T22:11:49.181355+0000","last_undegraded":"2024-09-10T22:11:49.181355+0000","last_fullsized":"2024-09-10T22:11:49.181355+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:59.444904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264160+0000","last_change":"2024-09-10T22:11:47.177926+0000","last_active":"2024-09-10T22:11:49.264160+0000","last_peered":"2024-09-10T22:11:49.264160+0000","last_clean":"2024-09-10T22:11:49.264160+0000","last_became_active":"2024-09-10T22:11:47.177598+0000","last_became_peered":"2024-09-10T22:11:47.177598+0000","last_unstale":"2024-09-10T22:11:49.264160+0000","last_undegraded":"2024-09-10T22:11:49.264160+0000","last_fullsized":"2024-09-10T22:11:49.264160+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:07.075072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186536+0000","last_change":"2024-09-10T22:11:48.178827+0000","last_active":"2024-09-10T22:11:49.186536+0000","last_peered":"2024-09-10T22:11:49.186536+0000","last_clean":"2024-09-10T22:11:49.186536+0000","last_became_active":"2024-09-10T22:11:48.177493+0000","last_became_peered":"2024-09-10T22:11:48.177493+0000","last_unstale":"2024-09-10T22:11:49.186536+0000","last_undegraded":"2024-09-10T22:11:49.186536+0000","last_fullsized":"2024-09-10T22:11:49.186536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:03.124106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":21,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264888+0000","last_change":"2024-09-10T22:11:47.187146+0000","last_active":"2024-09-10T22:11:49.264888+0000","last_peered":"2024-09-10T22:11:49.264888+0000","last_clean":"2024-09-10T22:11:49.264888+0000","last_became_active":"2024-09-10T22:11:47.186886+0000","last_became_peered":"2024-09-10T22:11:47.186886+0000","last_unstale":"2024-09-10T22:11:49.264888+0000","last_undegraded":"2024-09-10T22:11:49.264888+0000","last_fullsized":"2024-09-10T22:11:49.264888+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:13.528123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180957+0000","last_change":"2024-09-10T22:11:48.174022+0000","last_active":"2024-09-10T22:11:49.180957+0000","last_peered":"2024-09-10T22:11:49.180957+0000","last_clean":"2024-09-10T22:11:49.180957+0000","last_became_active":"2024-09-10T22:11:48.173765+0000","last_became_peered":"2024-09-10T22:11:48.173765+0000","last_unstale":"2024-09-10T22:11:49.180957+0000","last_undegraded":"2024-09-10T22:11:49.180957+0000","last_fullsized":"2024-09-10T22:11:49.180957+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:08.078020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265642+0000","last_change":"2024-09-10T22:11:47.181080+0000","last_active":"2024-09-10T22:11:49.265642+0000","last_peered":"2024-09-10T22:11:49.265642+0000","last_clean":"2024-09-10T22:11:49.265642+0000","last_became_active":"2024-09-10T22:11:47.180823+0000","last_became_peered":"2024-09-10T22:11:47.180823+0000","last_unstale":"2024-09-10T22:11:49.265642+0000","last_undegraded":"2024-09-10T22:11:49.265642+0000","last_fullsized":"2024-09-10T22:11:49.265642+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:27.455791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181216+0000","last_change":"2024-09-10T22:11:48.181817+0000","last_active":"2024-09-10T22:11:49.181216+0000","last_peered":"2024-09-10T22:11:49.181216+0000","last_clean":"2024-09-10T22:11:49.181216+0000","last_became_active":"2024-09-10T22:11:48.181514+0000","last_became_peered":"2024-09-10T22:11:48.181514+0000","last_unstale":"2024-09-10T22:11:49.181216+0000","last_undegraded":"2024-09-10T22:11:49.181216+0000","last_fullsized":"2024-09-10T22:11:49.181216+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:08.640825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268274+0000","last_change":"2024-09-10T22:11:47.176434+0000","last_active":"2024-09-10T22:11:49.268274+0000","last_peered":"2024-09-10T22:11:49.268274+0000","last_clean":"2024-09-10T22:11:49.268274+0000","last_became_active":"2024-09-10T22:11:47.175932+0000","last_became_peered":"2024-09-10T22:11:47.175932+0000","last_unstale":"2024-09-10T22:11:49.268274+0000","last_undegraded":"2024-09-10T22:11:49.268274+0000","last_fullsized":"2024-09-10T22:11:49.268274+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:41.968728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180789+0000","last_change":"2024-09-10T22:11:48.193087+0000","last_active":"2024-09-10T22:11:49.180789+0000","last_peered":"2024-09-10T22:11:49.180789+0000","last_clean":"2024-09-10T22:11:49.180789+0000","last_became_active":"2024-09-10T22:11:48.192299+0000","last_became_peered":"2024-09-10T22:11:48.192299+0000","last_unstale":"2024-09-10T22:11:49.180789+0000","last_undegraded":"2024-09-10T22:11:49.180789+0000","last_fullsized":"2024-09-10T22:11:49.180789+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:22.961644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":23,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263811+0000","last_change":"2024-09-10T22:11:47.184300+0000","last_active":"2024-09-10T22:11:49.263811+0000","last_peered":"2024-09-10T22:11:49.263811+0000","last_clean":"2024-09-10T22:11:49.263811+0000","last_became_active":"2024-09-10T22:11:47.184039+0000","last_became_peered":"2024-09-10T22:11:47.184039+0000","last_unstale":"2024-09-10T22:11:49.263811+0000","last_undegraded":"2024-09-10T22:11:49.263811+0000","last_fullsized":"2024-09-10T22:11:49.263811+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:39.714146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265318+0000","last_change":"2024-09-10T22:11:48.189515+0000","last_active":"2024-09-10T22:11:49.265318+0000","last_peered":"2024-09-10T22:11:49.265318+0000","last_clean":"2024-09-10T22:11:49.265318+0000","last_became_active":"2024-09-10T22:11:48.189102+0000","last_became_peered":"2024-09-10T22:11:48.189102+0000","last_unstale":"2024-09-10T22:11:49.265318+0000","last_undegraded":"2024-09-10T22:11:49.265318+0000","last_fullsized":"2024-09-10T22:11:49.265318+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:57.585521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265585+0000","last_change":"2024-09-10T22:11:47.182799+0000","last_active":"2024-09-10T22:11:49.265585+0000","last_peered":"2024-09-10T22:11:49.265585+0000","last_clean":"2024-09-10T22:11:49.265585+0000","last_became_active":"2024-09-10T22:11:47.180852+0000","last_became_peered":"2024-09-10T22:11:47.180852+0000","last_unstale":"2024-09-10T22:11:49.265585+0000","last_undegraded":"2024-09-10T22:11:49.265585+0000","last_fullsized":"2024-09-10T22:11:49.265585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:26.638919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265757+0000","last_change":"2024-09-10T22:11:48.195154+0000","last_active":"2024-09-10T22:11:49.265757+0000","last_peered":"2024-09-10T22:11:49.265757+0000","last_clean":"2024-09-10T22:11:49.265757+0000","last_became_active":"2024-09-10T22:11:48.194886+0000","last_became_peered":"2024-09-10T22:11:48.194886+0000","last_unstale":"2024-09-10T22:11:49.265757+0000","last_undegraded":"2024-09-10T22:11:49.265757+0000","last_fullsized":"2024-09-10T22:11:49.265757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:19.109740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265379+0000","last_change":"2024-09-10T22:11:47.174576+0000","last_active":"2024-09-10T22:11:49.265379+0000","last_peered":"2024-09-10T22:11:49.265379+0000","last_clean":"2024-09-10T22:11:49.265379+0000","last_became_active":"2024-09-10T22:11:47.174204+0000","last_became_peered":"2024-09-10T22:11:47.174204+0000","last_unstale":"2024-09-10T22:11:49.265379+0000","last_undegraded":"2024-09-10T22:11:49.265379+0000","last_fullsized":"2024-09-10T22:11:49.265379+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:55.279368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174644+0000","last_change":"2024-09-10T22:11:47.174644+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174644+0000","last_undegraded":"2024-09-10T22:11:47.174644+0000","last_fullsized":"2024-09-10T22:11:47.174644+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268594+0000","last_change":"2024-09-10T22:11:47.191138+0000","last_active":"2024-09-10T22:11:49.268594+0000","last_peered":"2024-09-10T22:11:49.268594+0000","last_clean":"2024-09-10T22:11:49.268594+0000","last_became_active":"2024-09-10T22:11:47.190913+0000","last_became_peered":"2024-09-10T22:11:47.190913+0000","last_unstale":"2024-09-10T22:11:49.268594+0000","last_undegraded":"2024-09-10T22:11:49.268594+0000","last_fullsized":"2024-09-10T22:11:49.268594+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:20.539547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173918+0000","last_change":"2024-09-10T22:11:47.173918+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173918+0000","last_undegraded":"2024-09-10T22:11:47.173918+0000","last_fullsized":"2024-09-10T22:11:47.173918+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186356+0000","last_change":"2024-09-10T22:11:47.182589+0000","last_active":"2024-09-10T22:11:49.186356+0000","last_peered":"2024-09-10T22:11:49.186356+0000","last_clean":"2024-09-10T22:11:49.186356+0000","last_became_active":"2024-09-10T22:11:47.182316+0000","last_became_peered":"2024-09-10T22:11:47.182316+0000","last_unstale":"2024-09-10T22:11:49.186356+0000","last_undegraded":"2024-09-10T22:11:49.186356+0000","last_fullsized":"2024-09-10T22:11:49.186356+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:53.817114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174289+0000","last_change":"2024-09-10T22:11:47.174289+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174289+0000","last_undegraded":"2024-09-10T22:11:47.174289+0000","last_fullsized":"2024-09-10T22:11:47.174289+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265637+0000","last_change":"2024-09-10T22:11:47.187951+0000","last_active":"2024-09-10T22:11:49.265637+0000","last_peered":"2024-09-10T22:11:49.265637+0000","last_clean":"2024-09-10T22:11:49.265637+0000","last_became_active":"2024-09-10T22:11:47.187697+0000","last_became_peered":"2024-09-10T22:11:47.187697+0000","last_unstale":"2024-09-10T22:11:49.265637+0000","last_undegraded":"2024-09-10T22:11:49.265637+0000","last_fullsized":"2024-09-10T22:11:49.265637+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:16.898662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263621+0000","last_change":"2024-09-10T22:11:48.179679+0000","last_active":"2024-09-10T22:11:49.263621+0000","last_peered":"2024-09-10T22:11:49.263621+0000","last_clean":"2024-09-10T22:11:49.263621+0000","last_became_active":"2024-09-10T22:11:48.179414+0000","last_became_peered":"2024-09-10T22:11:48.179414+0000","last_unstale":"2024-09-10T22:11:49.263621+0000","last_undegraded":"2024-09-10T22:11:49.263621+0000","last_fullsized":"2024-09-10T22:11:49.263621+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:00.909311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263535+0000","last_change":"2024-09-10T22:11:47.181464+0000","last_active":"2024-09-10T22:11:49.263535+0000","last_peered":"2024-09-10T22:11:49.263535+0000","last_clean":"2024-09-10T22:11:49.263535+0000","last_became_active":"2024-09-10T22:11:47.180230+0000","last_became_peered":"2024-09-10T22:11:47.180230+0000","last_unstale":"2024-09-10T22:11:49.263535+0000","last_undegraded":"2024-09-10T22:11:49.263535+0000","last_fullsized":"2024-09-10T22:11:49.263535+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:31.441376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173811+0000","last_change":"2024-09-10T22:11:47.173811+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173811+0000","last_undegraded":"2024-09-10T22:11:47.173811+0000","last_fullsized":"2024-09-10T22:11:47.173811+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264725+0000","last_change":"2024-09-10T22:11:47.188145+0000","last_active":"2024-09-10T22:11:49.264725+0000","last_peered":"2024-09-10T22:11:49.264725+0000","last_clean":"2024-09-10T22:11:49.264725+0000","last_became_active":"2024-09-10T22:11:47.187866+0000","last_became_peered":"2024-09-10T22:11:47.187866+0000","last_unstale":"2024-09-10T22:11:49.264725+0000","last_undegraded":"2024-09-10T22:11:49.264725+0000","last_fullsized":"2024-09-10T22:11:49.264725+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:12.664373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180993+0000","last_change":"2024-09-10T22:11:48.190644+0000","last_active":"2024-09-10T22:11:49.180993+0000","last_peered":"2024-09-10T22:11:49.180993+0000","last_clean":"2024-09-10T22:11:49.180993+0000","last_became_active":"2024-09-10T22:11:48.190081+0000","last_became_peered":"2024-09-10T22:11:48.190081+0000","last_unstale":"2024-09-10T22:11:49.180993+0000","last_undegraded":"2024-09-10T22:11:49.180993+0000","last_fullsized":"2024-09-10T22:11:49.180993+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:14.531071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264069+0000","last_change":"2024-09-10T22:11:47.174439+0000","last_active":"2024-09-10T22:11:49.264069+0000","last_peered":"2024-09-10T22:11:49.264069+0000","last_clean":"2024-09-10T22:11:49.264069+0000","last_became_active":"2024-09-10T22:11:47.174130+0000","last_became_peered":"2024-09-10T22:11:47.174130+0000","last_unstale":"2024-09-10T22:11:49.264069+0000","last_undegraded":"2024-09-10T22:11:49.264069+0000","last_fullsized":"2024-09-10T22:11:49.264069+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:57.534207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268768+0000","last_change":"2024-09-10T22:11:48.187621+0000","last_active":"2024-09-10T22:11:49.268768+0000","last_peered":"2024-09-10T22:11:49.268768+0000","last_clean":"2024-09-10T22:11:49.268768+0000","last_became_active":"2024-09-10T22:11:48.187340+0000","last_became_peered":"2024-09-10T22:11:48.187340+0000","last_unstale":"2024-09-10T22:11:49.268768+0000","last_undegraded":"2024-09-10T22:11:49.268768+0000","last_fullsized":"2024-09-10T22:11:49.268768+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:08.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181466+0000","last_change":"2024-09-10T22:11:47.185574+0000","last_active":"2024-09-10T22:11:49.181466+0000","last_peered":"2024-09-10T22:11:49.181466+0000","last_clean":"2024-09-10T22:11:49.181466+0000","last_became_active":"2024-09-10T22:11:47.185262+0000","last_became_peered":"2024-09-10T22:11:47.185262+0000","last_unstale":"2024-09-10T22:11:49.181466+0000","last_undegraded":"2024-09-10T22:11:49.181466+0000","last_fullsized":"2024-09-10T22:11:49.181466+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:14.036978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265445+0000","last_change":"2024-09-10T22:11:48.191766+0000","last_active":"2024-09-10T22:11:49.265445+0000","last_peered":"2024-09-10T22:11:49.265445+0000","last_clean":"2024-09-10T22:11:49.265445+0000","last_became_active":"2024-09-10T22:11:48.191476+0000","last_became_peered":"2024-09-10T22:11:48.191476+0000","last_unstale":"2024-09-10T22:11:49.265445+0000","last_undegraded":"2024-09-10T22:11:49.265445+0000","last_fullsized":"2024-09-10T22:11:49.265445+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:35.382116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268957+0000","last_change":"2024-09-10T22:11:47.178850+0000","last_active":"2024-09-10T22:11:49.268957+0000","last_peered":"2024-09-10T22:11:49.268957+0000","last_clean":"2024-09-10T22:11:49.268957+0000","last_became_active":"2024-09-10T22:11:47.178587+0000","last_became_peered":"2024-09-10T22:11:47.178587+0000","last_unstale":"2024-09-10T22:11:49.268957+0000","last_undegraded":"2024-09-10T22:11:49.268957+0000","last_fullsized":"2024-09-10T22:11:49.268957+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:23.791045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264595+0000","last_change":"2024-09-10T22:11:48.185926+0000","last_active":"2024-09-10T22:11:49.264595+0000","last_peered":"2024-09-10T22:11:49.264595+0000","last_clean":"2024-09-10T22:11:49.264595+0000","last_became_active":"2024-09-10T22:11:48.185651+0000","last_became_peered":"2024-09-10T22:11:48.185651+0000","last_unstale":"2024-09-10T22:11:49.264595+0000","last_undegraded":"2024-09-10T22:11:49.264595+0000","last_fullsized":"2024-09-10T22:11:49.264595+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:12.374323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264243+0000","last_change":"2024-09-10T22:11:47.175153+0000","last_active":"2024-09-10T22:11:49.264243+0000","last_peered":"2024-09-10T22:11:49.264243+0000","last_clean":"2024-09-10T22:11:49.264243+0000","last_became_active":"2024-09-10T22:11:47.174910+0000","last_became_peered":"2024-09-10T22:11:47.174910+0000","last_unstale":"2024-09-10T22:11:49.264243+0000","last_undegraded":"2024-09-10T22:11:49.264243+0000","last_fullsized":"2024-09-10T22:11:49.264243+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:07.637877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268469+0000","last_change":"2024-09-10T22:11:48.186801+0000","last_active":"2024-09-10T22:11:49.268469+0000","last_peered":"2024-09-10T22:11:49.268469+0000","last_clean":"2024-09-10T22:11:49.268469+0000","last_became_active":"2024-09-10T22:11:48.186510+0000","last_became_peered":"2024-09-10T22:11:48.186510+0000","last_unstale":"2024-09-10T22:11:49.268469+0000","last_undegraded":"2024-09-10T22:11:49.268469+0000","last_fullsized":"2024-09-10T22:11:49.268469+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:07.277597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181554+0000","last_change":"2024-09-10T22:11:47.190009+0000","last_active":"2024-09-10T22:11:49.181554+0000","last_peered":"2024-09-10T22:11:49.181554+0000","last_clean":"2024-09-10T22:11:49.181554+0000","last_became_active":"2024-09-10T22:11:47.189727+0000","last_became_peered":"2024-09-10T22:11:47.189727+0000","last_unstale":"2024-09-10T22:11:49.181554+0000","last_undegraded":"2024-09-10T22:11:49.181554+0000","last_fullsized":"2024-09-10T22:11:49.181554+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:30.357142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187529+0000","last_change":"2024-09-10T22:11:48.195733+0000","last_active":"2024-09-10T22:11:49.187529+0000","last_peered":"2024-09-10T22:11:49.187529+0000","last_clean":"2024-09-10T22:11:49.187529+0000","last_became_active":"2024-09-10T22:11:48.195182+0000","last_became_peered":"2024-09-10T22:11:48.195182+0000","last_unstale":"2024-09-10T22:11:49.187529+0000","last_undegraded":"2024-09-10T22:11:49.187529+0000","last_fullsized":"2024-09-10T22:11:49.187529+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:57.314793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186601+0000","last_change":"2024-09-10T22:11:47.182353+0000","last_active":"2024-09-10T22:11:49.186601+0000","last_peered":"2024-09-10T22:11:49.186601+0000","last_clean":"2024-09-10T22:11:49.186601+0000","last_became_active":"2024-09-10T22:11:47.182082+0000","last_became_peered":"2024-09-10T22:11:47.182082+0000","last_unstale":"2024-09-10T22:11:49.186601+0000","last_undegraded":"2024-09-10T22:11:49.186601+0000","last_fullsized":"2024-09-10T22:11:49.186601+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:37.435038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187229+0000","last_change":"2024-09-10T22:11:48.200370+0000","last_active":"2024-09-10T22:11:49.187229+0000","last_peered":"2024-09-10T22:11:49.187229+0000","last_clean":"2024-09-10T22:11:49.187229+0000","last_became_active":"2024-09-10T22:11:48.200102+0000","last_became_peered":"2024-09-10T22:11:48.200102+0000","last_unstale":"2024-09-10T22:11:49.187229+0000","last_undegraded":"2024-09-10T22:11:49.187229+0000","last_fullsized":"2024-09-10T22:11:49.187229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:23.091121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187581+0000","last_change":"2024-09-10T22:11:47.177554+0000","last_active":"2024-09-10T22:11:49.187581+0000","last_peered":"2024-09-10T22:11:49.187581+0000","last_clean":"2024-09-10T22:11:49.187581+0000","last_became_active":"2024-09-10T22:11:47.177260+0000","last_became_peered":"2024-09-10T22:11:47.177260+0000","last_unstale":"2024-09-10T22:11:49.187581+0000","last_undegraded":"2024-09-10T22:11:49.187581+0000","last_fullsized":"2024-09-10T22:11:49.187581+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:53.033722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173263+0000","last_change":"2024-09-10T22:11:47.173263+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173263+0000","last_undegraded":"2024-09-10T22:11:47.173263+0000","last_fullsized":"2024-09-10T22:11:47.173263+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263945+0000","last_change":"2024-09-10T22:11:47.181053+0000","last_active":"2024-09-10T22:11:49.263945+0000","last_peered":"2024-09-10T22:11:49.263945+0000","last_clean":"2024-09-10T22:11:49.263945+0000","last_became_active":"2024-09-10T22:11:47.180799+0000","last_became_peered":"2024-09-10T22:11:47.180799+0000","last_unstale":"2024-09-10T22:11:49.263945+0000","last_undegraded":"2024-09-10T22:11:49.263945+0000","last_fullsized":"2024-09-10T22:11:49.263945+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:16.715511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263656+0000","last_change":"2024-09-10T22:11:48.184225+0000","last_active":"2024-09-10T22:11:49.263656+0000","last_peered":"2024-09-10T22:11:49.263656+0000","last_clean":"2024-09-10T22:11:49.263656+0000","last_became_active":"2024-09-10T22:11:48.183190+0000","last_became_peered":"2024-09-10T22:11:48.183190+0000","last_unstale":"2024-09-10T22:11:49.263656+0000","last_undegraded":"2024-09-10T22:11:49.263656+0000","last_fullsized":"2024-09-10T22:11:49.263656+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:30.164067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264055+0000","last_change":"2024-09-10T22:11:47.180051+0000","last_active":"2024-09-10T22:11:49.264055+0000","last_peered":"2024-09-10T22:11:49.264055+0000","last_clean":"2024-09-10T22:11:49.264055+0000","last_became_active":"2024-09-10T22:11:47.179772+0000","last_became_peered":"2024-09-10T22:11:47.179772+0000","last_unstale":"2024-09-10T22:11:49.264055+0000","last_undegraded":"2024-09-10T22:11:49.264055+0000","last_fullsized":"2024-09-10T22:11:49.264055+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:55.541109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180927+0000","last_change":"2024-09-10T22:11:48.177392+0000","last_active":"2024-09-10T22:11:49.180927+0000","last_peered":"2024-09-10T22:11:49.180927+0000","last_clean":"2024-09-10T22:11:49.180927+0000","last_became_active":"2024-09-10T22:11:48.176877+0000","last_became_peered":"2024-09-10T22:11:48.176877+0000","last_unstale":"2024-09-10T22:11:49.180927+0000","last_undegraded":"2024-09-10T22:11:49.180927+0000","last_fullsized":"2024-09-10T22:11:49.180927+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:58.537155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186582+0000","last_change":"2024-09-10T22:11:47.183264+0000","last_active":"2024-09-10T22:11:49.186582+0000","last_peered":"2024-09-10T22:11:49.186582+0000","last_clean":"2024-09-10T22:11:49.186582+0000","last_became_active":"2024-09-10T22:11:47.182945+0000","last_became_peered":"2024-09-10T22:11:47.182945+0000","last_unstale":"2024-09-10T22:11:49.186582+0000","last_undegraded":"2024-09-10T22:11:49.186582+0000","last_fullsized":"2024-09-10T22:11:49.186582+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:11.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":279,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.178911+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:11:47.178911+0000","last_peered":"2024-09-10T22:11:47.178911+0000","last_clean":"2024-09-10T22:11:47.178911+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:11:47.178911+0000","last_undegraded":"2024-09-10T22:11:47.178911+0000","last_fullsized":"2024-09-10T22:11:47.178911+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509580,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127436,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244404,"statfs":{"total":96636764160,"available":96506269696,"internally_reserved":0,"allocated":1953792,"data_stored":1179333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118568,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253272,"statfs":{"total":96636764160,"available":96515350528,"internally_reserved":0,"allocated":1388544,"data_stored":602278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104676,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267164,"statfs":{"total":96636764160,"available":96529575936,"internally_reserved":0,"allocated":1392640,"data_stored":602812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149088,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222752,"statfs":{"total":96636764160,"available":96484098048,"internally_reserved":0,"allocated":1384448,"data_stored":602208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144948,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226892,"statfs":{"total":96636764160,"available":96488337408,"internally_reserved":0,"allocated":1400832,"data_stored":603890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509581,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131044,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240796,"statfs":{"total":96636764160,"available":96502575104,"internally_reserved":0,"allocated":1392640,"data_stored":602332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509581,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127476,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244364,"statfs":{"total":96636764160,"available":96506228736,"internally_reserved":0,"allocated":1998848,"data_stored":1194300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":128514505},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542286,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144928,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226912,"statfs":{"total":96636764160,"available":96488357888,"internally_reserved":0,"allocated":1384448,"data_stored":602208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:11:53.134 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3d is not active+clean 2024-09-10T22:11:53.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:11:47.177096+0000', 'last_change': '2024-09-10T22:11:47.177096+0000', 'last_active': '2024-09-10T22:11:47.151613+0000', 'last_peered': '2024-09-10T22:11:47.151613+0000', 'last_clean': '2024-09-10T22:11:47.151613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:11:47.177096+0000', 'last_undegraded': '2024-09-10T22:11:47.177096+0000', 'last_fullsized': '2024-09-10T22:11:47.177096+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_clean_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:11:53.134 INFO:tasks.ceph.ceph_manager.ceph:PG 17.38 is not active+clean 2024-09-10T22:11:53.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:11:47.174226+0000', 'last_change': '2024-09-10T22:11:47.174226+0000', 'last_active': '2024-09-10T22:11:47.151613+0000', 'last_peered': '2024-09-10T22:11:47.151613+0000', 'last_clean': '2024-09-10T22:11:47.151613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:11:47.174226+0000', 'last_undegraded': '2024-09-10T22:11:47.174226+0000', 'last_fullsized': '2024-09-10T22:11:47.174226+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_clean_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:11:53.134 INFO:tasks.ceph.ceph_manager.ceph:PG 17.33 is not active+clean 2024-09-10T22:11:53.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:11:47.175144+0000', 'last_change': '2024-09-10T22:11:47.175144+0000', 'last_active': '2024-09-10T22:11:47.151613+0000', 'last_peered': '2024-09-10T22:11:47.151613+0000', 'last_clean': '2024-09-10T22:11:47.151613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:11:47.175144+0000', 'last_undegraded': '2024-09-10T22:11:47.175144+0000', 'last_fullsized': '2024-09-10T22:11:47.175144+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_clean_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:11:53.134 INFO:tasks.ceph.ceph_manager.ceph:PG 17.32 is not active+clean 2024-09-10T22:11:53.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:11:47.174758+0000', 'last_change': '2024-09-10T22:11:47.174758+0000', 'last_active': '2024-09-10T22:11:47.151613+0000', 'last_peered': '2024-09-10T22:11:47.151613+0000', 'last_clean': '2024-09-10T22:11:47.151613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:11:47.174758+0000', 'last_undegraded': '2024-09-10T22:11:47.174758+0000', 'last_fullsized': '2024-09-10T22:11:47.174758+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_clean_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:11:53.135 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2d is not active+clean 2024-09-10T22:11:53.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:11:47.176608+0000', 'last_change': '2024-09-10T22:11:47.176608+0000', 'last_active': '2024-09-10T22:11:47.151613+0000', 'last_peered': '2024-09-10T22:11:47.151613+0000', 'last_clean': '2024-09-10T22:11:47.151613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:11:47.176608+0000', 'last_undegraded': '2024-09-10T22:11:47.176608+0000', 'last_fullsized': '2024-09-10T22:11:47.176608+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_clean_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:11:53.135 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1d is not active+clean 2024-09-10T22:11:53.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:11:47.174368+0000', 'last_change': '2024-09-10T22:11:47.174368+0000', 'last_active': '2024-09-10T22:11:47.151613+0000', 'last_peered': '2024-09-10T22:11:47.151613+0000', 'last_clean': '2024-09-10T22:11:47.151613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:11:47.174368+0000', 'last_undegraded': '2024-09-10T22:11:47.174368+0000', 'last_fullsized': '2024-09-10T22:11:47.174368+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_clean_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:11:53.135 INFO:tasks.ceph.ceph_manager.ceph:PG 17.c is not active+clean 2024-09-10T22:11:53.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:11:47.174644+0000', 'last_change': '2024-09-10T22:11:47.174644+0000', 'last_active': '2024-09-10T22:11:47.151613+0000', 'last_peered': '2024-09-10T22:11:47.151613+0000', 'last_clean': '2024-09-10T22:11:47.151613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:11:47.174644+0000', 'last_undegraded': '2024-09-10T22:11:47.174644+0000', 'last_fullsized': '2024-09-10T22:11:47.174644+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_clean_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:11:53.135 INFO:tasks.ceph.ceph_manager.ceph:PG 17.d is not active+clean 2024-09-10T22:11:53.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:11:47.173918+0000', 'last_change': '2024-09-10T22:11:47.173918+0000', 'last_active': '2024-09-10T22:11:47.151613+0000', 'last_peered': '2024-09-10T22:11:47.151613+0000', 'last_clean': '2024-09-10T22:11:47.151613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:11:47.173918+0000', 'last_undegraded': '2024-09-10T22:11:47.173918+0000', 'last_fullsized': '2024-09-10T22:11:47.173918+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_clean_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:11:53.135 INFO:tasks.ceph.ceph_manager.ceph:PG 17.a is not active+clean 2024-09-10T22:11:53.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:11:47.174289+0000', 'last_change': '2024-09-10T22:11:47.174289+0000', 'last_active': '2024-09-10T22:11:47.151613+0000', 'last_peered': '2024-09-10T22:11:47.151613+0000', 'last_clean': '2024-09-10T22:11:47.151613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:11:47.174289+0000', 'last_undegraded': '2024-09-10T22:11:47.174289+0000', 'last_fullsized': '2024-09-10T22:11:47.174289+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_clean_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:11:53.135 INFO:tasks.ceph.ceph_manager.ceph:PG 17.8 is not active+clean 2024-09-10T22:11:53.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:11:47.173811+0000', 'last_change': '2024-09-10T22:11:47.173811+0000', 'last_active': '2024-09-10T22:11:47.151613+0000', 'last_peered': '2024-09-10T22:11:47.151613+0000', 'last_clean': '2024-09-10T22:11:47.151613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:11:47.173811+0000', 'last_undegraded': '2024-09-10T22:11:47.173811+0000', 'last_fullsized': '2024-09-10T22:11:47.173811+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_clean_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:11:53.136 INFO:tasks.ceph.ceph_manager.ceph:PG 17.0 is not active+clean 2024-09-10T22:11:53.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 110, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:11:47.173263+0000', 'last_change': '2024-09-10T22:11:47.173263+0000', 'last_active': '2024-09-10T22:11:47.151613+0000', 'last_peered': '2024-09-10T22:11:47.151613+0000', 'last_clean': '2024-09-10T22:11:47.151613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:11:47.173263+0000', 'last_undegraded': '2024-09-10T22:11:47.173263+0000', 'last_fullsized': '2024-09-10T22:11:47.173263+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'last_clean_scrub_stamp': '2024-09-10T22:11:47.151613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:11:53.137 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T22:11:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.312+0000 7f01af200640 1 -- 172.21.15.66:0/292569830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f018c003c20 msgr2=0x7f018c004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.312+0000 7f01af200640 1 --2- 172.21.15.66:0/292569830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f018c003c20 0x7f018c004020 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f0198006fa0 tx=0x7f019801ced0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.312+0000 7f01ad200640 1 -- 172.21.15.66:0/292569830 reap_dead start 2024-09-10T22:11:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.312+0000 7f01af200640 1 -- 172.21.15.66:0/292569830 shutdown_connections 2024-09-10T22:11:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.312+0000 7f01af200640 1 -- 172.21.15.66:0/292569830 >> 172.21.15.66:0/292569830 conn(0x7f01a80b26c0 msgr2=0x7f01a80b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.312+0000 7f01af200640 1 -- 172.21.15.66:0/292569830 shutdown_connections 2024-09-10T22:11:53.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.312+0000 7f01af200640 1 -- 172.21.15.66:0/292569830 wait complete. 2024-09-10T22:11:53.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01af200640 1 Processor -- start 2024-09-10T22:11:53.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01af200640 1 -- start start 2024-09-10T22:11:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01af200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f018c003c20 0x7f01a8155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01af200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01a815b750 0x7f01a8156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01af200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01a815bbd0 0x7f01a81572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01af200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f01a8070c50 con 0x7f01a815bbd0 2024-09-10T22:11:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01af200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01a8070ad0 con 0x7f018c003c20 2024-09-10T22:11:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01af200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f01a8166770 con 0x7f01a815b750 2024-09-10T22:11:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01ad200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01a815bbd0 0x7f01a81572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01ad200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01a815bbd0 0x7f01a81572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33104/0 (socket says 172.21.15.66:33104) 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01ad200640 1 -- 172.21.15.66:0/3321042138 learned_addr learned my addr 172.21.15.66:0/3321042138 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f019fe00640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01a815b750 0x7f01a8156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01a7e00640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f018c003c20 0x7f01a8155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01ad200640 1 -- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01a815b750 msgr2=0x7f01a8156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01ad200640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01a815b750 0x7f01a8156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01ad200640 1 -- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f018c003c20 msgr2=0x7f01a8155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01ad200640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f018c003c20 0x7f01a8155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01ad200640 1 -- 172.21.15.66:0/3321042138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a815c130 con 0x7f01a815bbd0 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01ad200640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01a815bbd0 0x7f01a81572b0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f01a0006fd0 tx=0x7f01a0006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01a4c00640 1 -- 172.21.15.66:0/3321042138 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190009730 con 0x7f01a815bbd0 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01a4c00640 1 -- 172.21.15.66:0/3321042138 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01a0018070 con 0x7f01a815bbd0 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01af200640 1 -- 172.21.15.66:0/3321042138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a815db10 con 0x7f01a815bbd0 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01a4c00640 1 -- 172.21.15.66:0/3321042138 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0011be0 con 0x7f01a815bbd0 2024-09-10T22:11:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.316+0000 7f01af200640 1 -- 172.21.15.66:0/3321042138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a815e0d0 con 0x7f01a815bbd0 2024-09-10T22:11:53.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.320+0000 7f01af200640 1 -- 172.21.15.66:0/3321042138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a815e2d0 con 0x7f01a815bbd0 2024-09-10T22:11:53.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.320+0000 7f01a4c00640 1 -- 172.21.15.66:0/3321042138 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f01a001d460 con 0x7f01a815bbd0 2024-09-10T22:11:53.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.320+0000 7f01a4c00640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0184073f60 0x7f0184076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:53.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.320+0000 7f01a4c00640 1 -- 172.21.15.66:0/3321042138 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f01a0092790 con 0x7f01a815bbd0 2024-09-10T22:11:53.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.320+0000 7f01a7e00640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0184073f60 0x7f0184076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:53.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.320+0000 7f01a7e00640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0184073f60 0x7f0184076420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0198006fa0 tx=0x7f0198019d10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.324+0000 7f01a4c00640 1 -- 172.21.15.66:0/3321042138 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f01a815e2d0 con 0x7f01a815bbd0 2024-09-10T22:11:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.560+0000 7f01af200640 1 -- 172.21.15.66:0/3321042138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f01a80cc170 con 0x7f01a815bbd0 2024-09-10T22:11:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.560+0000 7f01a4c00640 1 -- 172.21.15.66:0/3321042138 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1144 (secure 0 0 0) 0x7f01a005f8d0 con 0x7f01a815bbd0 2024-09-10T22:11:53.565 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:53.566 INFO:teuthology.orchestra.run.smithi066.stdout:{"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":706,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":112,"num_osds":8,"num_up_osds":8,"osd_up_since":1726005621,"num_in_osds":8,"osd_in_since":1726005611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":86},{"state_name":"creating+peering","count":11}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":1073319936,"bytes_avail":772020793344,"bytes_total":773094113280,"inactive_pgs_ratio":0.11340206116437912,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":76,"btime":"2024-09-10T22:11:51:187007+0000","id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":8307}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-09-10T22:11:36.558334+0000","services":{}},"progress_events":{}} 2024-09-10T22:11:53.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.564+0000 7f01af200640 1 -- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0184073f60 msgr2=0x7f0184076420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:53.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.564+0000 7f01af200640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0184073f60 0x7f0184076420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0198006fa0 tx=0x7f0198019d10 comp rx=0 tx=0).stop 2024-09-10T22:11:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.564+0000 7f01af200640 1 -- 172.21.15.66:0/3321042138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01a815bbd0 msgr2=0x7f01a81572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.564+0000 7f01af200640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01a815bbd0 0x7f01a81572b0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f01a0006fd0 tx=0x7f01a0006430 comp rx=0 tx=0).stop 2024-09-10T22:11:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.564+0000 7f01af200640 1 -- 172.21.15.66:0/3321042138 shutdown_connections 2024-09-10T22:11:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.564+0000 7f01af200640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0184073f60 0x7f0184076420 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.564+0000 7f01af200640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01a815bbd0 0x7f01a81572b0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.564+0000 7f01af200640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01a815b750 0x7f01a8156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.564+0000 7f01af200640 1 --2- 172.21.15.66:0/3321042138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f018c003c20 0x7f01a8155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.564+0000 7f01af200640 1 -- 172.21.15.66:0/3321042138 >> 172.21.15.66:0/3321042138 conn(0x7f01a80b26c0 msgr2=0x7f01a8165de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.564+0000 7f01af200640 1 -- 172.21.15.66:0/3321042138 shutdown_connections 2024-09-10T22:11:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.564+0000 7f01af200640 1 -- 172.21.15.66:0/3321042138 wait complete. 2024-09-10T22:11:53.588 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T22:11:53.588 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:11:53.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.728+0000 7fd05c200640 1 -- 172.21.15.66:0/2080734237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0540aee20 msgr2=0x7fd0540ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:53.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.728+0000 7fd05c200640 1 --2- 172.21.15.66:0/2080734237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0540aee20 0x7fd0540ca080 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fd040009900 tx=0x7fd04001c850 comp rx=0 tx=0).stop 2024-09-10T22:11:53.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.728+0000 7fd05c200640 1 -- 172.21.15.66:0/2080734237 shutdown_connections 2024-09-10T22:11:53.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.728+0000 7fd05c200640 1 --2- 172.21.15.66:0/2080734237 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd05415cc80 0x7fd0540cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.728+0000 7fd05c200640 1 --2- 172.21.15.66:0/2080734237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0540aee20 0x7fd0540ca080 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.728+0000 7fd05c200640 1 -- 172.21.15.66:0/2080734237 >> 172.21.15.66:0/2080734237 conn(0x7fd0540cfa30 msgr2=0x7fd0540cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:53.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.728+0000 7fd05c200640 1 -- 172.21.15.66:0/2080734237 shutdown_connections 2024-09-10T22:11:53.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.728+0000 7fd05c200640 1 -- 172.21.15.66:0/2080734237 wait complete. 2024-09-10T22:11:53.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05c200640 1 Processor -- start 2024-09-10T22:11:53.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05c200640 1 -- start start 2024-09-10T22:11:53.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05c200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0540aee20 0x7fd05414a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:53.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05c200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd054159900 0x7fd054159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:53.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05c200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05415cc80 0x7fd05414b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:53.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05c200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd054168f70 con 0x7fd05415cc80 2024-09-10T22:11:53.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05c200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd054168df0 con 0x7fd054159900 2024-09-10T22:11:53.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05c200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd0541690f0 con 0x7fd0540aee20 2024-09-10T22:11:53.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05415cc80 0x7fd05414b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:53.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd058e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd054159900 0x7fd054159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:53.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd059800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0540aee20 0x7fd05414a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:53.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd058e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd054159900 0x7fd054159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:39772/0 (socket says 172.21.15.66:39772) 2024-09-10T22:11:53.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd058e00640 1 -- 172.21.15.66:0/4235153689 learned_addr learned my addr 172.21.15.66:0/4235153689 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:53.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05a200640 1 -- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0540aee20 msgr2=0x7fd05414a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:53.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05a200640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0540aee20 0x7fd05414a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05a200640 1 -- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd054159900 msgr2=0x7fd054159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:53.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05a200640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd054159900 0x7fd054159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05a200640 1 -- 172.21.15.66:0/4235153689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd054151200 con 0x7fd05415cc80 2024-09-10T22:11:53.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd058e00640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd054159900 0x7fd054159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:11:53.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05a200640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05415cc80 0x7fd05414b270 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00e990 tx=0x7fd04c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:53.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd051600640 1 -- 172.21.15.66:0/4235153689 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c004d30 con 0x7fd05415cc80 2024-09-10T22:11:53.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd051600640 1 -- 172.21.15.66:0/4235153689 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd04c00caa0 con 0x7fd05415cc80 2024-09-10T22:11:53.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd051600640 1 -- 172.21.15.66:0/4235153689 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c005020 con 0x7fd05415cc80 2024-09-10T22:11:53.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05c200640 1 -- 172.21.15.66:0/4235153689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0540afc10 con 0x7fd05415cc80 2024-09-10T22:11:53.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.732+0000 7fd05c200640 1 -- 172.21.15.66:0/4235153689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0540b01d0 con 0x7fd05415cc80 2024-09-10T22:11:53.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.736+0000 7fd05c200640 1 -- 172.21.15.66:0/4235153689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0540ca080 con 0x7fd05415cc80 2024-09-10T22:11:53.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.736+0000 7fd051600640 1 -- 172.21.15.66:0/4235153689 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fd04c00f040 con 0x7fd05415cc80 2024-09-10T22:11:53.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.736+0000 7fd051600640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd034073f30 0x7fd0340763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:53.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.736+0000 7fd051600640 1 -- 172.21.15.66:0/4235153689 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fd04c092580 con 0x7fd05415cc80 2024-09-10T22:11:53.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.736+0000 7fd059800640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd034073f30 0x7fd0340763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:53.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.740+0000 7fd059800640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd034073f30 0x7fd0340763f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd048007fd0 tx=0x7fd048009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:53.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.740+0000 7fd051600640 1 -- 172.21.15.66:0/4235153689 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd04c016030 con 0x7fd05415cc80 2024-09-10T22:11:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.912+0000 7fd05c200640 1 -- 172.21.15.66:0/4235153689 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd0540cee20 con 0x7fd034073f30 2024-09-10T22:11:53.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.916+0000 7fd051600640 1 -- 172.21.15.66:0/4235153689 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251590 (secure 0 0 0) 0x7fd0540cee20 con 0x7fd034073f30 2024-09-10T22:11:53.924 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:53.925 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:11:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.920+0000 7fd05c200640 1 -- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd034073f30 msgr2=0x7fd0340763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.920+0000 7fd05c200640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd034073f30 0x7fd0340763f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd048007fd0 tx=0x7fd048009040 comp rx=0 tx=0).stop 2024-09-10T22:11:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.920+0000 7fd05c200640 1 -- 172.21.15.66:0/4235153689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05415cc80 msgr2=0x7fd05414b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.920+0000 7fd05c200640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05415cc80 0x7fd05414b270 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00e990 tx=0x7fd04c00ee50 comp rx=0 tx=0).stop 2024-09-10T22:11:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.920+0000 7fd05c200640 1 -- 172.21.15.66:0/4235153689 shutdown_connections 2024-09-10T22:11:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.920+0000 7fd05c200640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd034073f30 0x7fd0340763f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.920+0000 7fd05c200640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05415cc80 0x7fd05414b270 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.920+0000 7fd05c200640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd054159900 0x7fd054159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.920+0000 7fd05c200640 1 --2- 172.21.15.66:0/4235153689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0540aee20 0x7fd05414a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:53.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.920+0000 7fd05c200640 1 -- 172.21.15.66:0/4235153689 >> 172.21.15.66:0/4235153689 conn(0x7fd0540cfa30 msgr2=0x7fd05406f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:53.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.920+0000 7fd05c200640 1 -- 172.21.15.66:0/4235153689 shutdown_connections 2024-09-10T22:11:53.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:53.920+0000 7fd05c200640 1 -- 172.21.15.66:0/4235153689 wait complete. 2024-09-10T22:11:53.945 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-10T22:11:52.561491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1048164,"kb_used_data":12008,"kb_used_omap":12,"kb_used_meta":1035955,"kb_avail":753926556,"statfs":{"total":773094113280,"available":772020793344,"internally_reserved":0,"allocated":12296192,"data_stored":5989361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1060818484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001201"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265042+0000","last_change":"2024-09-10T22:11:48.180766+0000","last_active":"2024-09-10T22:11:49.265042+0000","last_peered":"2024-09-10T22:11:49.265042+0000","last_clean":"2024-09-10T22:11:49.265042+0000","last_became_active":"2024-09-10T22:11:48.179819+0000","last_became_peered":"2024-09-10T22:11:48.179819+0000","last_unstale":"2024-09-10T22:11:49.265042+0000","last_undegraded":"2024-09-10T22:11:49.265042+0000","last_fullsized":"2024-09-10T22:11:49.265042+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:21.885630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186814+0000","last_change":"2024-09-10T22:11:48.203979+0000","last_active":"2024-09-10T22:11:49.186814+0000","last_peered":"2024-09-10T22:11:49.186814+0000","last_clean":"2024-09-10T22:11:49.186814+0000","last_became_active":"2024-09-10T22:11:48.203764+0000","last_became_peered":"2024-09-10T22:11:48.203764+0000","last_unstale":"2024-09-10T22:11:49.186814+0000","last_undegraded":"2024-09-10T22:11:49.186814+0000","last_fullsized":"2024-09-10T22:11:49.186814+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:41.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.177096+0000","last_change":"2024-09-10T22:11:47.177096+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.177096+0000","last_undegraded":"2024-09-10T22:11:47.177096+0000","last_fullsized":"2024-09-10T22:11:47.177096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264205+0000","last_change":"2024-09-10T22:11:48.191506+0000","last_active":"2024-09-10T22:11:49.264205+0000","last_peered":"2024-09-10T22:11:49.264205+0000","last_clean":"2024-09-10T22:11:49.264205+0000","last_became_active":"2024-09-10T22:11:48.191100+0000","last_became_peered":"2024-09-10T22:11:48.191100+0000","last_unstale":"2024-09-10T22:11:49.264205+0000","last_undegraded":"2024-09-10T22:11:49.264205+0000","last_fullsized":"2024-09-10T22:11:49.264205+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:54.388719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263748+0000","last_change":"2024-09-10T22:11:48.178358+0000","last_active":"2024-09-10T22:11:49.263748+0000","last_peered":"2024-09-10T22:11:49.263748+0000","last_clean":"2024-09-10T22:11:49.263748+0000","last_became_active":"2024-09-10T22:11:48.177965+0000","last_became_peered":"2024-09-10T22:11:48.177965+0000","last_unstale":"2024-09-10T22:11:49.263748+0000","last_undegraded":"2024-09-10T22:11:49.263748+0000","last_fullsized":"2024-09-10T22:11:49.263748+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:16.479719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264893+0000","last_change":"2024-09-10T22:11:48.182705+0000","last_active":"2024-09-10T22:11:49.264893+0000","last_peered":"2024-09-10T22:11:49.264893+0000","last_clean":"2024-09-10T22:11:49.264893+0000","last_became_active":"2024-09-10T22:11:48.182428+0000","last_became_peered":"2024-09-10T22:11:48.182428+0000","last_unstale":"2024-09-10T22:11:49.264893+0000","last_undegraded":"2024-09-10T22:11:49.264893+0000","last_fullsized":"2024-09-10T22:11:49.264893+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:56.601140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186457+0000","last_change":"2024-09-10T22:11:48.181122+0000","last_active":"2024-09-10T22:11:49.186457+0000","last_peered":"2024-09-10T22:11:49.186457+0000","last_clean":"2024-09-10T22:11:49.186457+0000","last_became_active":"2024-09-10T22:11:48.178614+0000","last_became_peered":"2024-09-10T22:11:48.178614+0000","last_unstale":"2024-09-10T22:11:49.186457+0000","last_undegraded":"2024-09-10T22:11:49.186457+0000","last_fullsized":"2024-09-10T22:11:49.186457+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:54.036670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174226+0000","last_change":"2024-09-10T22:11:47.174226+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174226+0000","last_undegraded":"2024-09-10T22:11:47.174226+0000","last_fullsized":"2024-09-10T22:11:47.174226+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265249+0000","last_change":"2024-09-10T22:11:48.180116+0000","last_active":"2024-09-10T22:11:49.265249+0000","last_peered":"2024-09-10T22:11:49.265249+0000","last_clean":"2024-09-10T22:11:49.265249+0000","last_became_active":"2024-09-10T22:11:48.179490+0000","last_became_peered":"2024-09-10T22:11:48.179490+0000","last_unstale":"2024-09-10T22:11:49.265249+0000","last_undegraded":"2024-09-10T22:11:49.265249+0000","last_fullsized":"2024-09-10T22:11:49.265249+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:19.843998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264589+0000","last_change":"2024-09-10T22:11:48.183940+0000","last_active":"2024-09-10T22:11:49.264589+0000","last_peered":"2024-09-10T22:11:49.264589+0000","last_clean":"2024-09-10T22:11:49.264589+0000","last_became_active":"2024-09-10T22:11:48.183694+0000","last_became_peered":"2024-09-10T22:11:48.183694+0000","last_unstale":"2024-09-10T22:11:49.264589+0000","last_undegraded":"2024-09-10T22:11:49.264589+0000","last_fullsized":"2024-09-10T22:11:49.264589+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:00.322112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187457+0000","last_change":"2024-09-10T22:11:48.197453+0000","last_active":"2024-09-10T22:11:49.187457+0000","last_peered":"2024-09-10T22:11:49.187457+0000","last_clean":"2024-09-10T22:11:49.187457+0000","last_became_active":"2024-09-10T22:11:48.197138+0000","last_became_peered":"2024-09-10T22:11:48.197138+0000","last_unstale":"2024-09-10T22:11:49.187457+0000","last_undegraded":"2024-09-10T22:11:49.187457+0000","last_fullsized":"2024-09-10T22:11:49.187457+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:33.059467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264403+0000","last_change":"2024-09-10T22:11:48.189077+0000","last_active":"2024-09-10T22:11:49.264403+0000","last_peered":"2024-09-10T22:11:49.264403+0000","last_clean":"2024-09-10T22:11:49.264403+0000","last_became_active":"2024-09-10T22:11:48.185421+0000","last_became_peered":"2024-09-10T22:11:48.185421+0000","last_unstale":"2024-09-10T22:11:49.264403+0000","last_undegraded":"2024-09-10T22:11:49.264403+0000","last_fullsized":"2024-09-10T22:11:49.264403+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:25.288843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.175144+0000","last_change":"2024-09-10T22:11:47.175144+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.175144+0000","last_undegraded":"2024-09-10T22:11:47.175144+0000","last_fullsized":"2024-09-10T22:11:47.175144+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174758+0000","last_change":"2024-09-10T22:11:47.174758+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174758+0000","last_undegraded":"2024-09-10T22:11:47.174758+0000","last_fullsized":"2024-09-10T22:11:47.174758+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264482+0000","last_change":"2024-09-10T22:11:48.183435+0000","last_active":"2024-09-10T22:11:49.264482+0000","last_peered":"2024-09-10T22:11:49.264482+0000","last_clean":"2024-09-10T22:11:49.264482+0000","last_became_active":"2024-09-10T22:11:48.183243+0000","last_became_peered":"2024-09-10T22:11:48.183243+0000","last_unstale":"2024-09-10T22:11:49.264482+0000","last_undegraded":"2024-09-10T22:11:49.264482+0000","last_fullsized":"2024-09-10T22:11:49.264482+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:11.483616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263496+0000","last_change":"2024-09-10T22:11:48.186035+0000","last_active":"2024-09-10T22:11:49.263496+0000","last_peered":"2024-09-10T22:11:49.263496+0000","last_clean":"2024-09-10T22:11:49.263496+0000","last_became_active":"2024-09-10T22:11:48.185768+0000","last_became_peered":"2024-09-10T22:11:48.185768+0000","last_unstale":"2024-09-10T22:11:49.263496+0000","last_undegraded":"2024-09-10T22:11:49.263496+0000","last_fullsized":"2024-09-10T22:11:49.263496+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:22.226210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187717+0000","last_change":"2024-09-10T22:11:48.200285+0000","last_active":"2024-09-10T22:11:49.187717+0000","last_peered":"2024-09-10T22:11:49.187717+0000","last_clean":"2024-09-10T22:11:49.187717+0000","last_became_active":"2024-09-10T22:11:48.199965+0000","last_became_peered":"2024-09-10T22:11:48.199965+0000","last_unstale":"2024-09-10T22:11:49.187717+0000","last_undegraded":"2024-09-10T22:11:49.187717+0000","last_fullsized":"2024-09-10T22:11:49.187717+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:22.961644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268722+0000","last_change":"2024-09-10T22:11:48.194873+0000","last_active":"2024-09-10T22:11:49.268722+0000","last_peered":"2024-09-10T22:11:49.268722+0000","last_clean":"2024-09-10T22:11:49.268722+0000","last_became_active":"2024-09-10T22:11:48.194469+0000","last_became_peered":"2024-09-10T22:11:48.194469+0000","last_unstale":"2024-09-10T22:11:49.268722+0000","last_undegraded":"2024-09-10T22:11:49.268722+0000","last_fullsized":"2024-09-10T22:11:49.268722+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:37.166232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.176608+0000","last_change":"2024-09-10T22:11:47.176608+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.176608+0000","last_undegraded":"2024-09-10T22:11:47.176608+0000","last_fullsized":"2024-09-10T22:11:47.176608+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186908+0000","last_change":"2024-09-10T22:11:48.199337+0000","last_active":"2024-09-10T22:11:49.186908+0000","last_peered":"2024-09-10T22:11:49.186908+0000","last_clean":"2024-09-10T22:11:49.186908+0000","last_became_active":"2024-09-10T22:11:48.199028+0000","last_became_peered":"2024-09-10T22:11:48.199028+0000","last_unstale":"2024-09-10T22:11:49.186908+0000","last_undegraded":"2024-09-10T22:11:49.186908+0000","last_fullsized":"2024-09-10T22:11:49.186908+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:57.807567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187347+0000","last_change":"2024-09-10T22:11:48.200296+0000","last_active":"2024-09-10T22:11:49.187347+0000","last_peered":"2024-09-10T22:11:49.187347+0000","last_clean":"2024-09-10T22:11:49.187347+0000","last_became_active":"2024-09-10T22:11:48.198375+0000","last_became_peered":"2024-09-10T22:11:48.198375+0000","last_unstale":"2024-09-10T22:11:49.187347+0000","last_undegraded":"2024-09-10T22:11:49.187347+0000","last_fullsized":"2024-09-10T22:11:49.187347+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:45.662642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180916+0000","last_change":"2024-09-10T22:11:48.180779+0000","last_active":"2024-09-10T22:11:49.180916+0000","last_peered":"2024-09-10T22:11:49.180916+0000","last_clean":"2024-09-10T22:11:49.180916+0000","last_became_active":"2024-09-10T22:11:48.179454+0000","last_became_peered":"2024-09-10T22:11:48.179454+0000","last_unstale":"2024-09-10T22:11:49.180916+0000","last_undegraded":"2024-09-10T22:11:49.180916+0000","last_fullsized":"2024-09-10T22:11:49.180916+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:28:27.233254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186778+0000","last_change":"2024-09-10T22:11:48.198947+0000","last_active":"2024-09-10T22:11:49.186778+0000","last_peered":"2024-09-10T22:11:49.186778+0000","last_clean":"2024-09-10T22:11:49.186778+0000","last_became_active":"2024-09-10T22:11:48.198672+0000","last_became_peered":"2024-09-10T22:11:48.198672+0000","last_unstale":"2024-09-10T22:11:49.186778+0000","last_undegraded":"2024-09-10T22:11:49.186778+0000","last_fullsized":"2024-09-10T22:11:49.186778+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:08.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180712+0000","last_change":"2024-09-10T22:11:48.177447+0000","last_active":"2024-09-10T22:11:49.180712+0000","last_peered":"2024-09-10T22:11:49.180712+0000","last_clean":"2024-09-10T22:11:49.180712+0000","last_became_active":"2024-09-10T22:11:48.177207+0000","last_became_peered":"2024-09-10T22:11:48.177207+0000","last_unstale":"2024-09-10T22:11:49.180712+0000","last_undegraded":"2024-09-10T22:11:49.180712+0000","last_fullsized":"2024-09-10T22:11:49.180712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:57.314793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187731+0000","last_change":"2024-09-10T22:11:48.201225+0000","last_active":"2024-09-10T22:11:49.187731+0000","last_peered":"2024-09-10T22:11:49.187731+0000","last_clean":"2024-09-10T22:11:49.187731+0000","last_became_active":"2024-09-10T22:11:48.200931+0000","last_became_peered":"2024-09-10T22:11:48.200931+0000","last_unstale":"2024-09-10T22:11:49.187731+0000","last_undegraded":"2024-09-10T22:11:49.187731+0000","last_fullsized":"2024-09-10T22:11:49.187731+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:16.417992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264799+0000","last_change":"2024-09-10T22:11:48.185741+0000","last_active":"2024-09-10T22:11:49.264799+0000","last_peered":"2024-09-10T22:11:49.264799+0000","last_clean":"2024-09-10T22:11:49.264799+0000","last_became_active":"2024-09-10T22:11:48.185438+0000","last_became_peered":"2024-09-10T22:11:48.185438+0000","last_unstale":"2024-09-10T22:11:49.264799+0000","last_undegraded":"2024-09-10T22:11:49.264799+0000","last_fullsized":"2024-09-10T22:11:49.264799+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:58.072179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265352+0000","last_change":"2024-09-10T22:11:48.195500+0000","last_active":"2024-09-10T22:11:49.265352+0000","last_peered":"2024-09-10T22:11:49.265352+0000","last_clean":"2024-09-10T22:11:49.265352+0000","last_became_active":"2024-09-10T22:11:48.195019+0000","last_became_peered":"2024-09-10T22:11:48.195019+0000","last_unstale":"2024-09-10T22:11:49.265352+0000","last_undegraded":"2024-09-10T22:11:49.265352+0000","last_fullsized":"2024-09-10T22:11:49.265352+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:18.164228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181343+0000","last_change":"2024-09-10T22:11:48.189755+0000","last_active":"2024-09-10T22:11:49.181343+0000","last_peered":"2024-09-10T22:11:49.181343+0000","last_clean":"2024-09-10T22:11:49.181343+0000","last_became_active":"2024-09-10T22:11:48.189436+0000","last_became_peered":"2024-09-10T22:11:48.189436+0000","last_unstale":"2024-09-10T22:11:49.181343+0000","last_undegraded":"2024-09-10T22:11:49.181343+0000","last_fullsized":"2024-09-10T22:11:49.181343+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:42.442483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187041+0000","last_change":"2024-09-10T22:11:48.199608+0000","last_active":"2024-09-10T22:11:49.187041+0000","last_peered":"2024-09-10T22:11:49.187041+0000","last_clean":"2024-09-10T22:11:49.187041+0000","last_became_active":"2024-09-10T22:11:48.199331+0000","last_became_peered":"2024-09-10T22:11:48.199331+0000","last_unstale":"2024-09-10T22:11:49.187041+0000","last_undegraded":"2024-09-10T22:11:49.187041+0000","last_fullsized":"2024-09-10T22:11:49.187041+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:20.253988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186852+0000","last_change":"2024-09-10T22:11:48.184404+0000","last_active":"2024-09-10T22:11:49.186852+0000","last_peered":"2024-09-10T22:11:49.186852+0000","last_clean":"2024-09-10T22:11:49.186852+0000","last_became_active":"2024-09-10T22:11:48.180465+0000","last_became_peered":"2024-09-10T22:11:48.180465+0000","last_unstale":"2024-09-10T22:11:49.186852+0000","last_undegraded":"2024-09-10T22:11:49.186852+0000","last_fullsized":"2024-09-10T22:11:49.186852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:02.597092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263547+0000","last_change":"2024-09-10T22:11:48.180454+0000","last_active":"2024-09-10T22:11:49.263547+0000","last_peered":"2024-09-10T22:11:49.263547+0000","last_clean":"2024-09-10T22:11:49.263547+0000","last_became_active":"2024-09-10T22:11:48.179907+0000","last_became_peered":"2024-09-10T22:11:48.179907+0000","last_unstale":"2024-09-10T22:11:49.263547+0000","last_undegraded":"2024-09-10T22:11:49.263547+0000","last_fullsized":"2024-09-10T22:11:49.263547+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:15.319909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264477+0000","last_change":"2024-09-10T22:11:48.182798+0000","last_active":"2024-09-10T22:11:49.264477+0000","last_peered":"2024-09-10T22:11:49.264477+0000","last_clean":"2024-09-10T22:11:49.264477+0000","last_became_active":"2024-09-10T22:11:48.182533+0000","last_became_peered":"2024-09-10T22:11:48.182533+0000","last_unstale":"2024-09-10T22:11:49.264477+0000","last_undegraded":"2024-09-10T22:11:49.264477+0000","last_fullsized":"2024-09-10T22:11:49.264477+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:35.986933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265665+0000","last_change":"2024-09-10T22:11:48.184978+0000","last_active":"2024-09-10T22:11:49.265665+0000","last_peered":"2024-09-10T22:11:49.265665+0000","last_clean":"2024-09-10T22:11:49.265665+0000","last_became_active":"2024-09-10T22:11:48.184722+0000","last_became_peered":"2024-09-10T22:11:48.184722+0000","last_unstale":"2024-09-10T22:11:49.265665+0000","last_undegraded":"2024-09-10T22:11:49.265665+0000","last_fullsized":"2024-09-10T22:11:49.265665+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:48.904032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187304+0000","last_change":"2024-09-10T22:11:47.179953+0000","last_active":"2024-09-10T22:11:49.187304+0000","last_peered":"2024-09-10T22:11:49.187304+0000","last_clean":"2024-09-10T22:11:49.187304+0000","last_became_active":"2024-09-10T22:11:47.179653+0000","last_became_peered":"2024-09-10T22:11:47.179653+0000","last_unstale":"2024-09-10T22:11:49.187304+0000","last_undegraded":"2024-09-10T22:11:49.187304+0000","last_fullsized":"2024-09-10T22:11:49.187304+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:17.150534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268977+0000","last_change":"2024-09-10T22:11:48.186184+0000","last_active":"2024-09-10T22:11:49.268977+0000","last_peered":"2024-09-10T22:11:49.268977+0000","last_clean":"2024-09-10T22:11:49.268977+0000","last_became_active":"2024-09-10T22:11:48.185822+0000","last_became_peered":"2024-09-10T22:11:48.185822+0000","last_unstale":"2024-09-10T22:11:49.268977+0000","last_undegraded":"2024-09-10T22:11:49.268977+0000","last_fullsized":"2024-09-10T22:11:49.268977+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:41.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.185431+0000","last_change":"2024-09-10T22:11:47.184730+0000","last_active":"2024-09-10T22:11:47.185431+0000","last_peered":"2024-09-10T22:11:47.185431+0000","last_clean":"2024-09-10T22:11:47.185431+0000","last_became_active":"2024-09-10T22:11:47.184361+0000","last_became_peered":"2024-09-10T22:11:47.184361+0000","last_unstale":"2024-09-10T22:11:47.185431+0000","last_undegraded":"2024-09-10T22:11:47.185431+0000","last_fullsized":"2024-09-10T22:11:47.185431+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:19.251040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265333+0000","last_change":"2024-09-10T22:11:48.182402+0000","last_active":"2024-09-10T22:11:49.265333+0000","last_peered":"2024-09-10T22:11:49.265333+0000","last_clean":"2024-09-10T22:11:49.265333+0000","last_became_active":"2024-09-10T22:11:48.182057+0000","last_became_peered":"2024-09-10T22:11:48.182057+0000","last_unstale":"2024-09-10T22:11:49.265333+0000","last_undegraded":"2024-09-10T22:11:49.265333+0000","last_fullsized":"2024-09-10T22:11:49.265333+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:18.062008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265043+0000","last_change":"2024-09-10T22:11:47.181206+0000","last_active":"2024-09-10T22:11:49.265043+0000","last_peered":"2024-09-10T22:11:49.265043+0000","last_clean":"2024-09-10T22:11:49.265043+0000","last_became_active":"2024-09-10T22:11:47.179544+0000","last_became_peered":"2024-09-10T22:11:47.179544+0000","last_unstale":"2024-09-10T22:11:49.265043+0000","last_undegraded":"2024-09-10T22:11:49.265043+0000","last_fullsized":"2024-09-10T22:11:49.265043+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:26.154879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174368+0000","last_change":"2024-09-10T22:11:47.174368+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174368+0000","last_undegraded":"2024-09-10T22:11:47.174368+0000","last_fullsized":"2024-09-10T22:11:47.174368+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263584+0000","last_change":"2024-09-10T22:11:47.182277+0000","last_active":"2024-09-10T22:11:49.263584+0000","last_peered":"2024-09-10T22:11:49.263584+0000","last_clean":"2024-09-10T22:11:49.263584+0000","last_became_active":"2024-09-10T22:11:47.181435+0000","last_became_peered":"2024-09-10T22:11:47.181435+0000","last_unstale":"2024-09-10T22:11:49.263584+0000","last_undegraded":"2024-09-10T22:11:49.263584+0000","last_fullsized":"2024-09-10T22:11:49.263584+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:47.604910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263891+0000","last_change":"2024-09-10T22:11:48.181857+0000","last_active":"2024-09-10T22:11:49.263891+0000","last_peered":"2024-09-10T22:11:49.263891+0000","last_clean":"2024-09-10T22:11:49.263891+0000","last_became_active":"2024-09-10T22:11:48.179210+0000","last_became_peered":"2024-09-10T22:11:48.179210+0000","last_unstale":"2024-09-10T22:11:49.263891+0000","last_undegraded":"2024-09-10T22:11:49.263891+0000","last_fullsized":"2024-09-10T22:11:49.263891+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:51.570280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186717+0000","last_change":"2024-09-10T22:11:47.177297+0000","last_active":"2024-09-10T22:11:49.186717+0000","last_peered":"2024-09-10T22:11:49.186717+0000","last_clean":"2024-09-10T22:11:49.186717+0000","last_became_active":"2024-09-10T22:11:47.176973+0000","last_became_peered":"2024-09-10T22:11:47.176973+0000","last_unstale":"2024-09-10T22:11:49.186717+0000","last_undegraded":"2024-09-10T22:11:49.186717+0000","last_fullsized":"2024-09-10T22:11:49.186717+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:08.162909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187075+0000","last_change":"2024-09-10T22:11:48.204412+0000","last_active":"2024-09-10T22:11:49.187075+0000","last_peered":"2024-09-10T22:11:49.187075+0000","last_clean":"2024-09-10T22:11:49.187075+0000","last_became_active":"2024-09-10T22:11:48.204105+0000","last_became_peered":"2024-09-10T22:11:48.204105+0000","last_unstale":"2024-09-10T22:11:49.187075+0000","last_undegraded":"2024-09-10T22:11:49.187075+0000","last_fullsized":"2024-09-10T22:11:49.187075+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:04.692623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181125+0000","last_change":"2024-09-10T22:11:47.184025+0000","last_active":"2024-09-10T22:11:49.181125+0000","last_peered":"2024-09-10T22:11:49.181125+0000","last_clean":"2024-09-10T22:11:49.181125+0000","last_became_active":"2024-09-10T22:11:47.183724+0000","last_became_peered":"2024-09-10T22:11:47.183724+0000","last_unstale":"2024-09-10T22:11:49.181125+0000","last_undegraded":"2024-09-10T22:11:49.181125+0000","last_fullsized":"2024-09-10T22:11:49.181125+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:50.420296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263722+0000","last_change":"2024-09-10T22:11:48.186204+0000","last_active":"2024-09-10T22:11:49.263722+0000","last_peered":"2024-09-10T22:11:49.263722+0000","last_clean":"2024-09-10T22:11:49.263722+0000","last_became_active":"2024-09-10T22:11:48.185455+0000","last_became_peered":"2024-09-10T22:11:48.185455+0000","last_unstale":"2024-09-10T22:11:49.263722+0000","last_undegraded":"2024-09-10T22:11:49.263722+0000","last_fullsized":"2024-09-10T22:11:49.263722+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:57.468704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186931+0000","last_change":"2024-09-10T22:11:47.181785+0000","last_active":"2024-09-10T22:11:49.186931+0000","last_peered":"2024-09-10T22:11:49.186931+0000","last_clean":"2024-09-10T22:11:49.186931+0000","last_became_active":"2024-09-10T22:11:47.181505+0000","last_became_peered":"2024-09-10T22:11:47.181505+0000","last_unstale":"2024-09-10T22:11:49.186931+0000","last_undegraded":"2024-09-10T22:11:49.186931+0000","last_fullsized":"2024-09-10T22:11:49.186931+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:01.594145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186384+0000","last_change":"2024-09-10T22:11:48.181429+0000","last_active":"2024-09-10T22:11:49.186384+0000","last_peered":"2024-09-10T22:11:49.186384+0000","last_clean":"2024-09-10T22:11:49.186384+0000","last_became_active":"2024-09-10T22:11:48.178978+0000","last_became_peered":"2024-09-10T22:11:48.178978+0000","last_unstale":"2024-09-10T22:11:49.186384+0000","last_undegraded":"2024-09-10T22:11:49.186384+0000","last_fullsized":"2024-09-10T22:11:49.186384+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:52.036230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263948+0000","last_change":"2024-09-10T22:11:47.187068+0000","last_active":"2024-09-10T22:11:49.263948+0000","last_peered":"2024-09-10T22:11:49.263948+0000","last_clean":"2024-09-10T22:11:49.263948+0000","last_became_active":"2024-09-10T22:11:47.186777+0000","last_became_peered":"2024-09-10T22:11:47.186777+0000","last_unstale":"2024-09-10T22:11:49.263948+0000","last_undegraded":"2024-09-10T22:11:49.263948+0000","last_fullsized":"2024-09-10T22:11:49.263948+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:21.958696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187129+0000","last_change":"2024-09-10T22:11:48.181451+0000","last_active":"2024-09-10T22:11:49.187129+0000","last_peered":"2024-09-10T22:11:49.187129+0000","last_clean":"2024-09-10T22:11:49.187129+0000","last_became_active":"2024-09-10T22:11:48.181161+0000","last_became_peered":"2024-09-10T22:11:48.181161+0000","last_unstale":"2024-09-10T22:11:49.187129+0000","last_undegraded":"2024-09-10T22:11:49.187129+0000","last_fullsized":"2024-09-10T22:11:49.187129+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:26.021716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265100+0000","last_change":"2024-09-10T22:11:47.179876+0000","last_active":"2024-09-10T22:11:49.265100+0000","last_peered":"2024-09-10T22:11:49.265100+0000","last_clean":"2024-09-10T22:11:49.265100+0000","last_became_active":"2024-09-10T22:11:47.179525+0000","last_became_peered":"2024-09-10T22:11:47.179525+0000","last_unstale":"2024-09-10T22:11:49.265100+0000","last_undegraded":"2024-09-10T22:11:49.265100+0000","last_fullsized":"2024-09-10T22:11:49.265100+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:59.073260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.269023+0000","last_change":"2024-09-10T22:11:48.187132+0000","last_active":"2024-09-10T22:11:49.269023+0000","last_peered":"2024-09-10T22:11:49.269023+0000","last_clean":"2024-09-10T22:11:49.269023+0000","last_became_active":"2024-09-10T22:11:48.186733+0000","last_became_peered":"2024-09-10T22:11:48.186733+0000","last_unstale":"2024-09-10T22:11:49.269023+0000","last_undegraded":"2024-09-10T22:11:49.269023+0000","last_fullsized":"2024-09-10T22:11:49.269023+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:57.807567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181341+0000","last_change":"2024-09-10T22:11:47.183552+0000","last_active":"2024-09-10T22:11:49.181341+0000","last_peered":"2024-09-10T22:11:49.181341+0000","last_clean":"2024-09-10T22:11:49.181341+0000","last_became_active":"2024-09-10T22:11:47.183260+0000","last_became_peered":"2024-09-10T22:11:47.183260+0000","last_unstale":"2024-09-10T22:11:49.181341+0000","last_undegraded":"2024-09-10T22:11:49.181341+0000","last_fullsized":"2024-09-10T22:11:49.181341+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:13.345360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181355+0000","last_change":"2024-09-10T22:11:48.194176+0000","last_active":"2024-09-10T22:11:49.181355+0000","last_peered":"2024-09-10T22:11:49.181355+0000","last_clean":"2024-09-10T22:11:49.181355+0000","last_became_active":"2024-09-10T22:11:48.192478+0000","last_became_peered":"2024-09-10T22:11:48.192478+0000","last_unstale":"2024-09-10T22:11:49.181355+0000","last_undegraded":"2024-09-10T22:11:49.181355+0000","last_fullsized":"2024-09-10T22:11:49.181355+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:59.444904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264160+0000","last_change":"2024-09-10T22:11:47.177926+0000","last_active":"2024-09-10T22:11:49.264160+0000","last_peered":"2024-09-10T22:11:49.264160+0000","last_clean":"2024-09-10T22:11:49.264160+0000","last_became_active":"2024-09-10T22:11:47.177598+0000","last_became_peered":"2024-09-10T22:11:47.177598+0000","last_unstale":"2024-09-10T22:11:49.264160+0000","last_undegraded":"2024-09-10T22:11:49.264160+0000","last_fullsized":"2024-09-10T22:11:49.264160+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:07.075072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186536+0000","last_change":"2024-09-10T22:11:48.178827+0000","last_active":"2024-09-10T22:11:49.186536+0000","last_peered":"2024-09-10T22:11:49.186536+0000","last_clean":"2024-09-10T22:11:49.186536+0000","last_became_active":"2024-09-10T22:11:48.177493+0000","last_became_peered":"2024-09-10T22:11:48.177493+0000","last_unstale":"2024-09-10T22:11:49.186536+0000","last_undegraded":"2024-09-10T22:11:49.186536+0000","last_fullsized":"2024-09-10T22:11:49.186536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:03.124106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":21,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264888+0000","last_change":"2024-09-10T22:11:47.187146+0000","last_active":"2024-09-10T22:11:49.264888+0000","last_peered":"2024-09-10T22:11:49.264888+0000","last_clean":"2024-09-10T22:11:49.264888+0000","last_became_active":"2024-09-10T22:11:47.186886+0000","last_became_peered":"2024-09-10T22:11:47.186886+0000","last_unstale":"2024-09-10T22:11:49.264888+0000","last_undegraded":"2024-09-10T22:11:49.264888+0000","last_fullsized":"2024-09-10T22:11:49.264888+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:13.528123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180957+0000","last_change":"2024-09-10T22:11:48.174022+0000","last_active":"2024-09-10T22:11:49.180957+0000","last_peered":"2024-09-10T22:11:49.180957+0000","last_clean":"2024-09-10T22:11:49.180957+0000","last_became_active":"2024-09-10T22:11:48.173765+0000","last_became_peered":"2024-09-10T22:11:48.173765+0000","last_unstale":"2024-09-10T22:11:49.180957+0000","last_undegraded":"2024-09-10T22:11:49.180957+0000","last_fullsized":"2024-09-10T22:11:49.180957+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:08.078020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265642+0000","last_change":"2024-09-10T22:11:47.181080+0000","last_active":"2024-09-10T22:11:49.265642+0000","last_peered":"2024-09-10T22:11:49.265642+0000","last_clean":"2024-09-10T22:11:49.265642+0000","last_became_active":"2024-09-10T22:11:47.180823+0000","last_became_peered":"2024-09-10T22:11:47.180823+0000","last_unstale":"2024-09-10T22:11:49.265642+0000","last_undegraded":"2024-09-10T22:11:49.265642+0000","last_fullsized":"2024-09-10T22:11:49.265642+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:27.455791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181216+0000","last_change":"2024-09-10T22:11:48.181817+0000","last_active":"2024-09-10T22:11:49.181216+0000","last_peered":"2024-09-10T22:11:49.181216+0000","last_clean":"2024-09-10T22:11:49.181216+0000","last_became_active":"2024-09-10T22:11:48.181514+0000","last_became_peered":"2024-09-10T22:11:48.181514+0000","last_unstale":"2024-09-10T22:11:49.181216+0000","last_undegraded":"2024-09-10T22:11:49.181216+0000","last_fullsized":"2024-09-10T22:11:49.181216+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:08.640825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268274+0000","last_change":"2024-09-10T22:11:47.176434+0000","last_active":"2024-09-10T22:11:49.268274+0000","last_peered":"2024-09-10T22:11:49.268274+0000","last_clean":"2024-09-10T22:11:49.268274+0000","last_became_active":"2024-09-10T22:11:47.175932+0000","last_became_peered":"2024-09-10T22:11:47.175932+0000","last_unstale":"2024-09-10T22:11:49.268274+0000","last_undegraded":"2024-09-10T22:11:49.268274+0000","last_fullsized":"2024-09-10T22:11:49.268274+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:41.968728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180789+0000","last_change":"2024-09-10T22:11:48.193087+0000","last_active":"2024-09-10T22:11:49.180789+0000","last_peered":"2024-09-10T22:11:49.180789+0000","last_clean":"2024-09-10T22:11:49.180789+0000","last_became_active":"2024-09-10T22:11:48.192299+0000","last_became_peered":"2024-09-10T22:11:48.192299+0000","last_unstale":"2024-09-10T22:11:49.180789+0000","last_undegraded":"2024-09-10T22:11:49.180789+0000","last_fullsized":"2024-09-10T22:11:49.180789+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:22.961644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":23,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263811+0000","last_change":"2024-09-10T22:11:47.184300+0000","last_active":"2024-09-10T22:11:49.263811+0000","last_peered":"2024-09-10T22:11:49.263811+0000","last_clean":"2024-09-10T22:11:49.263811+0000","last_became_active":"2024-09-10T22:11:47.184039+0000","last_became_peered":"2024-09-10T22:11:47.184039+0000","last_unstale":"2024-09-10T22:11:49.263811+0000","last_undegraded":"2024-09-10T22:11:49.263811+0000","last_fullsized":"2024-09-10T22:11:49.263811+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:39.714146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265318+0000","last_change":"2024-09-10T22:11:48.189515+0000","last_active":"2024-09-10T22:11:49.265318+0000","last_peered":"2024-09-10T22:11:49.265318+0000","last_clean":"2024-09-10T22:11:49.265318+0000","last_became_active":"2024-09-10T22:11:48.189102+0000","last_became_peered":"2024-09-10T22:11:48.189102+0000","last_unstale":"2024-09-10T22:11:49.265318+0000","last_undegraded":"2024-09-10T22:11:49.265318+0000","last_fullsized":"2024-09-10T22:11:49.265318+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:57.585521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265585+0000","last_change":"2024-09-10T22:11:47.182799+0000","last_active":"2024-09-10T22:11:49.265585+0000","last_peered":"2024-09-10T22:11:49.265585+0000","last_clean":"2024-09-10T22:11:49.265585+0000","last_became_active":"2024-09-10T22:11:47.180852+0000","last_became_peered":"2024-09-10T22:11:47.180852+0000","last_unstale":"2024-09-10T22:11:49.265585+0000","last_undegraded":"2024-09-10T22:11:49.265585+0000","last_fullsized":"2024-09-10T22:11:49.265585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:26.638919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265757+0000","last_change":"2024-09-10T22:11:48.195154+0000","last_active":"2024-09-10T22:11:49.265757+0000","last_peered":"2024-09-10T22:11:49.265757+0000","last_clean":"2024-09-10T22:11:49.265757+0000","last_became_active":"2024-09-10T22:11:48.194886+0000","last_became_peered":"2024-09-10T22:11:48.194886+0000","last_unstale":"2024-09-10T22:11:49.265757+0000","last_undegraded":"2024-09-10T22:11:49.265757+0000","last_fullsized":"2024-09-10T22:11:49.265757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:19.109740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265379+0000","last_change":"2024-09-10T22:11:47.174576+0000","last_active":"2024-09-10T22:11:49.265379+0000","last_peered":"2024-09-10T22:11:49.265379+0000","last_clean":"2024-09-10T22:11:49.265379+0000","last_became_active":"2024-09-10T22:11:47.174204+0000","last_became_peered":"2024-09-10T22:11:47.174204+0000","last_unstale":"2024-09-10T22:11:49.265379+0000","last_undegraded":"2024-09-10T22:11:49.265379+0000","last_fullsized":"2024-09-10T22:11:49.265379+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:55.279368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174644+0000","last_change":"2024-09-10T22:11:47.174644+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174644+0000","last_undegraded":"2024-09-10T22:11:47.174644+0000","last_fullsized":"2024-09-10T22:11:47.174644+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268594+0000","last_change":"2024-09-10T22:11:47.191138+0000","last_active":"2024-09-10T22:11:49.268594+0000","last_peered":"2024-09-10T22:11:49.268594+0000","last_clean":"2024-09-10T22:11:49.268594+0000","last_became_active":"2024-09-10T22:11:47.190913+0000","last_became_peered":"2024-09-10T22:11:47.190913+0000","last_unstale":"2024-09-10T22:11:49.268594+0000","last_undegraded":"2024-09-10T22:11:49.268594+0000","last_fullsized":"2024-09-10T22:11:49.268594+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:20.539547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173918+0000","last_change":"2024-09-10T22:11:47.173918+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173918+0000","last_undegraded":"2024-09-10T22:11:47.173918+0000","last_fullsized":"2024-09-10T22:11:47.173918+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186356+0000","last_change":"2024-09-10T22:11:47.182589+0000","last_active":"2024-09-10T22:11:49.186356+0000","last_peered":"2024-09-10T22:11:49.186356+0000","last_clean":"2024-09-10T22:11:49.186356+0000","last_became_active":"2024-09-10T22:11:47.182316+0000","last_became_peered":"2024-09-10T22:11:47.182316+0000","last_unstale":"2024-09-10T22:11:49.186356+0000","last_undegraded":"2024-09-10T22:11:49.186356+0000","last_fullsized":"2024-09-10T22:11:49.186356+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:53.817114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.174289+0000","last_change":"2024-09-10T22:11:47.174289+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.174289+0000","last_undegraded":"2024-09-10T22:11:47.174289+0000","last_fullsized":"2024-09-10T22:11:47.174289+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265637+0000","last_change":"2024-09-10T22:11:47.187951+0000","last_active":"2024-09-10T22:11:49.265637+0000","last_peered":"2024-09-10T22:11:49.265637+0000","last_clean":"2024-09-10T22:11:49.265637+0000","last_became_active":"2024-09-10T22:11:47.187697+0000","last_became_peered":"2024-09-10T22:11:47.187697+0000","last_unstale":"2024-09-10T22:11:49.265637+0000","last_undegraded":"2024-09-10T22:11:49.265637+0000","last_fullsized":"2024-09-10T22:11:49.265637+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:16.898662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263621+0000","last_change":"2024-09-10T22:11:48.179679+0000","last_active":"2024-09-10T22:11:49.263621+0000","last_peered":"2024-09-10T22:11:49.263621+0000","last_clean":"2024-09-10T22:11:49.263621+0000","last_became_active":"2024-09-10T22:11:48.179414+0000","last_became_peered":"2024-09-10T22:11:48.179414+0000","last_unstale":"2024-09-10T22:11:49.263621+0000","last_undegraded":"2024-09-10T22:11:49.263621+0000","last_fullsized":"2024-09-10T22:11:49.263621+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:00.909311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263535+0000","last_change":"2024-09-10T22:11:47.181464+0000","last_active":"2024-09-10T22:11:49.263535+0000","last_peered":"2024-09-10T22:11:49.263535+0000","last_clean":"2024-09-10T22:11:49.263535+0000","last_became_active":"2024-09-10T22:11:47.180230+0000","last_became_peered":"2024-09-10T22:11:47.180230+0000","last_unstale":"2024-09-10T22:11:49.263535+0000","last_undegraded":"2024-09-10T22:11:49.263535+0000","last_fullsized":"2024-09-10T22:11:49.263535+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:31.441376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173811+0000","last_change":"2024-09-10T22:11:47.173811+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173811+0000","last_undegraded":"2024-09-10T22:11:47.173811+0000","last_fullsized":"2024-09-10T22:11:47.173811+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264725+0000","last_change":"2024-09-10T22:11:47.188145+0000","last_active":"2024-09-10T22:11:49.264725+0000","last_peered":"2024-09-10T22:11:49.264725+0000","last_clean":"2024-09-10T22:11:49.264725+0000","last_became_active":"2024-09-10T22:11:47.187866+0000","last_became_peered":"2024-09-10T22:11:47.187866+0000","last_unstale":"2024-09-10T22:11:49.264725+0000","last_undegraded":"2024-09-10T22:11:49.264725+0000","last_fullsized":"2024-09-10T22:11:49.264725+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:12.664373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180993+0000","last_change":"2024-09-10T22:11:48.190644+0000","last_active":"2024-09-10T22:11:49.180993+0000","last_peered":"2024-09-10T22:11:49.180993+0000","last_clean":"2024-09-10T22:11:49.180993+0000","last_became_active":"2024-09-10T22:11:48.190081+0000","last_became_peered":"2024-09-10T22:11:48.190081+0000","last_unstale":"2024-09-10T22:11:49.180993+0000","last_undegraded":"2024-09-10T22:11:49.180993+0000","last_fullsized":"2024-09-10T22:11:49.180993+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:14.531071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264069+0000","last_change":"2024-09-10T22:11:47.174439+0000","last_active":"2024-09-10T22:11:49.264069+0000","last_peered":"2024-09-10T22:11:49.264069+0000","last_clean":"2024-09-10T22:11:49.264069+0000","last_became_active":"2024-09-10T22:11:47.174130+0000","last_became_peered":"2024-09-10T22:11:47.174130+0000","last_unstale":"2024-09-10T22:11:49.264069+0000","last_undegraded":"2024-09-10T22:11:49.264069+0000","last_fullsized":"2024-09-10T22:11:49.264069+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:57.534207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268768+0000","last_change":"2024-09-10T22:11:48.187621+0000","last_active":"2024-09-10T22:11:49.268768+0000","last_peered":"2024-09-10T22:11:49.268768+0000","last_clean":"2024-09-10T22:11:49.268768+0000","last_became_active":"2024-09-10T22:11:48.187340+0000","last_became_peered":"2024-09-10T22:11:48.187340+0000","last_unstale":"2024-09-10T22:11:49.268768+0000","last_undegraded":"2024-09-10T22:11:49.268768+0000","last_fullsized":"2024-09-10T22:11:49.268768+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:08.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181466+0000","last_change":"2024-09-10T22:11:47.185574+0000","last_active":"2024-09-10T22:11:49.181466+0000","last_peered":"2024-09-10T22:11:49.181466+0000","last_clean":"2024-09-10T22:11:49.181466+0000","last_became_active":"2024-09-10T22:11:47.185262+0000","last_became_peered":"2024-09-10T22:11:47.185262+0000","last_unstale":"2024-09-10T22:11:49.181466+0000","last_undegraded":"2024-09-10T22:11:49.181466+0000","last_fullsized":"2024-09-10T22:11:49.181466+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:14.036978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265445+0000","last_change":"2024-09-10T22:11:48.191766+0000","last_active":"2024-09-10T22:11:49.265445+0000","last_peered":"2024-09-10T22:11:49.265445+0000","last_clean":"2024-09-10T22:11:49.265445+0000","last_became_active":"2024-09-10T22:11:48.191476+0000","last_became_peered":"2024-09-10T22:11:48.191476+0000","last_unstale":"2024-09-10T22:11:49.265445+0000","last_undegraded":"2024-09-10T22:11:49.265445+0000","last_fullsized":"2024-09-10T22:11:49.265445+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:35.382116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268957+0000","last_change":"2024-09-10T22:11:47.178850+0000","last_active":"2024-09-10T22:11:49.268957+0000","last_peered":"2024-09-10T22:11:49.268957+0000","last_clean":"2024-09-10T22:11:49.268957+0000","last_became_active":"2024-09-10T22:11:47.178587+0000","last_became_peered":"2024-09-10T22:11:47.178587+0000","last_unstale":"2024-09-10T22:11:49.268957+0000","last_undegraded":"2024-09-10T22:11:49.268957+0000","last_fullsized":"2024-09-10T22:11:49.268957+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:23.791045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264595+0000","last_change":"2024-09-10T22:11:48.185926+0000","last_active":"2024-09-10T22:11:49.264595+0000","last_peered":"2024-09-10T22:11:49.264595+0000","last_clean":"2024-09-10T22:11:49.264595+0000","last_became_active":"2024-09-10T22:11:48.185651+0000","last_became_peered":"2024-09-10T22:11:48.185651+0000","last_unstale":"2024-09-10T22:11:49.264595+0000","last_undegraded":"2024-09-10T22:11:49.264595+0000","last_fullsized":"2024-09-10T22:11:49.264595+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:12.374323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264243+0000","last_change":"2024-09-10T22:11:47.175153+0000","last_active":"2024-09-10T22:11:49.264243+0000","last_peered":"2024-09-10T22:11:49.264243+0000","last_clean":"2024-09-10T22:11:49.264243+0000","last_became_active":"2024-09-10T22:11:47.174910+0000","last_became_peered":"2024-09-10T22:11:47.174910+0000","last_unstale":"2024-09-10T22:11:49.264243+0000","last_undegraded":"2024-09-10T22:11:49.264243+0000","last_fullsized":"2024-09-10T22:11:49.264243+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:07.637877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268469+0000","last_change":"2024-09-10T22:11:48.186801+0000","last_active":"2024-09-10T22:11:49.268469+0000","last_peered":"2024-09-10T22:11:49.268469+0000","last_clean":"2024-09-10T22:11:49.268469+0000","last_became_active":"2024-09-10T22:11:48.186510+0000","last_became_peered":"2024-09-10T22:11:48.186510+0000","last_unstale":"2024-09-10T22:11:49.268469+0000","last_undegraded":"2024-09-10T22:11:49.268469+0000","last_fullsized":"2024-09-10T22:11:49.268469+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:07.277597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181554+0000","last_change":"2024-09-10T22:11:47.190009+0000","last_active":"2024-09-10T22:11:49.181554+0000","last_peered":"2024-09-10T22:11:49.181554+0000","last_clean":"2024-09-10T22:11:49.181554+0000","last_became_active":"2024-09-10T22:11:47.189727+0000","last_became_peered":"2024-09-10T22:11:47.189727+0000","last_unstale":"2024-09-10T22:11:49.181554+0000","last_undegraded":"2024-09-10T22:11:49.181554+0000","last_fullsized":"2024-09-10T22:11:49.181554+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:30.357142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187529+0000","last_change":"2024-09-10T22:11:48.195733+0000","last_active":"2024-09-10T22:11:49.187529+0000","last_peered":"2024-09-10T22:11:49.187529+0000","last_clean":"2024-09-10T22:11:49.187529+0000","last_became_active":"2024-09-10T22:11:48.195182+0000","last_became_peered":"2024-09-10T22:11:48.195182+0000","last_unstale":"2024-09-10T22:11:49.187529+0000","last_undegraded":"2024-09-10T22:11:49.187529+0000","last_fullsized":"2024-09-10T22:11:49.187529+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:57.314793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186601+0000","last_change":"2024-09-10T22:11:47.182353+0000","last_active":"2024-09-10T22:11:49.186601+0000","last_peered":"2024-09-10T22:11:49.186601+0000","last_clean":"2024-09-10T22:11:49.186601+0000","last_became_active":"2024-09-10T22:11:47.182082+0000","last_became_peered":"2024-09-10T22:11:47.182082+0000","last_unstale":"2024-09-10T22:11:49.186601+0000","last_undegraded":"2024-09-10T22:11:49.186601+0000","last_fullsized":"2024-09-10T22:11:49.186601+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:37.435038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187229+0000","last_change":"2024-09-10T22:11:48.200370+0000","last_active":"2024-09-10T22:11:49.187229+0000","last_peered":"2024-09-10T22:11:49.187229+0000","last_clean":"2024-09-10T22:11:49.187229+0000","last_became_active":"2024-09-10T22:11:48.200102+0000","last_became_peered":"2024-09-10T22:11:48.200102+0000","last_unstale":"2024-09-10T22:11:49.187229+0000","last_undegraded":"2024-09-10T22:11:49.187229+0000","last_fullsized":"2024-09-10T22:11:49.187229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:23.091121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187581+0000","last_change":"2024-09-10T22:11:47.177554+0000","last_active":"2024-09-10T22:11:49.187581+0000","last_peered":"2024-09-10T22:11:49.187581+0000","last_clean":"2024-09-10T22:11:49.187581+0000","last_became_active":"2024-09-10T22:11:47.177260+0000","last_became_peered":"2024-09-10T22:11:47.177260+0000","last_unstale":"2024-09-10T22:11:49.187581+0000","last_undegraded":"2024-09-10T22:11:49.187581+0000","last_fullsized":"2024-09-10T22:11:49.187581+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:53.033722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":110,"state":"creating+peering","last_fresh":"2024-09-10T22:11:47.173263+0000","last_change":"2024-09-10T22:11:47.173263+0000","last_active":"2024-09-10T22:11:47.151613+0000","last_peered":"2024-09-10T22:11:47.151613+0000","last_clean":"2024-09-10T22:11:47.151613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:11:47.173263+0000","last_undegraded":"2024-09-10T22:11:47.173263+0000","last_fullsized":"2024-09-10T22:11:47.173263+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263945+0000","last_change":"2024-09-10T22:11:47.181053+0000","last_active":"2024-09-10T22:11:49.263945+0000","last_peered":"2024-09-10T22:11:49.263945+0000","last_clean":"2024-09-10T22:11:49.263945+0000","last_became_active":"2024-09-10T22:11:47.180799+0000","last_became_peered":"2024-09-10T22:11:47.180799+0000","last_unstale":"2024-09-10T22:11:49.263945+0000","last_undegraded":"2024-09-10T22:11:49.263945+0000","last_fullsized":"2024-09-10T22:11:49.263945+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:16.715511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263656+0000","last_change":"2024-09-10T22:11:48.184225+0000","last_active":"2024-09-10T22:11:49.263656+0000","last_peered":"2024-09-10T22:11:49.263656+0000","last_clean":"2024-09-10T22:11:49.263656+0000","last_became_active":"2024-09-10T22:11:48.183190+0000","last_became_peered":"2024-09-10T22:11:48.183190+0000","last_unstale":"2024-09-10T22:11:49.263656+0000","last_undegraded":"2024-09-10T22:11:49.263656+0000","last_fullsized":"2024-09-10T22:11:49.263656+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:30.164067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264055+0000","last_change":"2024-09-10T22:11:47.180051+0000","last_active":"2024-09-10T22:11:49.264055+0000","last_peered":"2024-09-10T22:11:49.264055+0000","last_clean":"2024-09-10T22:11:49.264055+0000","last_became_active":"2024-09-10T22:11:47.179772+0000","last_became_peered":"2024-09-10T22:11:47.179772+0000","last_unstale":"2024-09-10T22:11:49.264055+0000","last_undegraded":"2024-09-10T22:11:49.264055+0000","last_fullsized":"2024-09-10T22:11:49.264055+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:55.541109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180927+0000","last_change":"2024-09-10T22:11:48.177392+0000","last_active":"2024-09-10T22:11:49.180927+0000","last_peered":"2024-09-10T22:11:49.180927+0000","last_clean":"2024-09-10T22:11:49.180927+0000","last_became_active":"2024-09-10T22:11:48.176877+0000","last_became_peered":"2024-09-10T22:11:48.176877+0000","last_unstale":"2024-09-10T22:11:49.180927+0000","last_undegraded":"2024-09-10T22:11:49.180927+0000","last_fullsized":"2024-09-10T22:11:49.180927+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:58.537155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186582+0000","last_change":"2024-09-10T22:11:47.183264+0000","last_active":"2024-09-10T22:11:49.186582+0000","last_peered":"2024-09-10T22:11:49.186582+0000","last_clean":"2024-09-10T22:11:49.186582+0000","last_became_active":"2024-09-10T22:11:47.182945+0000","last_became_peered":"2024-09-10T22:11:47.182945+0000","last_unstale":"2024-09-10T22:11:49.186582+0000","last_undegraded":"2024-09-10T22:11:49.186582+0000","last_fullsized":"2024-09-10T22:11:49.186582+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:11.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":279,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T22:11:47.178911+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:11:47.178911+0000","last_peered":"2024-09-10T22:11:47.178911+0000","last_clean":"2024-09-10T22:11:47.178911+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:11:47.178911+0000","last_undegraded":"2024-09-10T22:11:47.178911+0000","last_fullsized":"2024-09-10T22:11:47.178911+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509580,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127436,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244404,"statfs":{"total":96636764160,"available":96506269696,"internally_reserved":0,"allocated":1953792,"data_stored":1179333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118568,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253272,"statfs":{"total":96636764160,"available":96515350528,"internally_reserved":0,"allocated":1388544,"data_stored":602278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104676,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267164,"statfs":{"total":96636764160,"available":96529575936,"internally_reserved":0,"allocated":1392640,"data_stored":602812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149088,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222752,"statfs":{"total":96636764160,"available":96484098048,"internally_reserved":0,"allocated":1384448,"data_stored":602208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144948,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226892,"statfs":{"total":96636764160,"available":96488337408,"internally_reserved":0,"allocated":1400832,"data_stored":603890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509581,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131044,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240796,"statfs":{"total":96636764160,"available":96502575104,"internally_reserved":0,"allocated":1392640,"data_stored":602332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509581,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127476,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244364,"statfs":{"total":96636764160,"available":96506228736,"internally_reserved":0,"allocated":1998848,"data_stored":1194300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":128514505},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542286,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144928,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226912,"statfs":{"total":96636764160,"available":96488357888,"internally_reserved":0,"allocated":1384448,"data_stored":602208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:11:56.951 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:11:57.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.104+0000 7f0763600640 1 -- 172.21.15.66:0/1867725351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0744003ab0 msgr2=0x7f0744003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.104+0000 7f0763600640 1 --2- 172.21.15.66:0/1867725351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0744003ab0 0x7f0744003ed0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f0750009b60 tx=0x7f075001d270 comp rx=0 tx=0).stop 2024-09-10T22:11:57.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.104+0000 7f0763600640 1 -- 172.21.15.66:0/1867725351 shutdown_connections 2024-09-10T22:11:57.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.104+0000 7f0763600640 1 --2- 172.21.15.66:0/1867725351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0744003ab0 0x7f0744003ed0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.104+0000 7f0763600640 1 -- 172.21.15.66:0/1867725351 >> 172.21.15.66:0/1867725351 conn(0x7f075c0d1c60 msgr2=0x7f075c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:57.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.104+0000 7f0763600640 1 -- 172.21.15.66:0/1867725351 shutdown_connections 2024-09-10T22:11:57.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.104+0000 7f0763600640 1 -- 172.21.15.66:0/1867725351 wait complete. 2024-09-10T22:11:57.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0763600640 1 Processor -- start 2024-09-10T22:11:57.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0763600640 1 -- start start 2024-09-10T22:11:57.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0763600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0744003ab0 0x7f075c0c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:57.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0763600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f075c1fc240 0x7f075c1fb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:57.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0763600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f075c1fbbc0 0x7f075c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:57.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0763600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f075c0cee50 con 0x7f075c1fc240 2024-09-10T22:11:57.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0763600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f075c0cecd0 con 0x7f0744003ab0 2024-09-10T22:11:57.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0763600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f075c0cefd0 con 0x7f075c1fbbc0 2024-09-10T22:11:57.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f075be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f075c1fc240 0x7f075c1fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:57.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f075be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f075c1fc240 0x7f075c1fb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33158/0 (socket says 172.21.15.66:33158) 2024-09-10T22:11:57.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f075be00640 1 -- 172.21.15.66:0/2118139802 learned_addr learned my addr 172.21.15.66:0/2118139802 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:57.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0760c00640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0744003ab0 0x7f075c0c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:57.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0761600640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f075c1fbbc0 0x7f075c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:57.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f075be00640 1 -- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f075c1fbbc0 msgr2=0x7f075c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f075be00640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f075c1fbbc0 0x7f075c06dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f075be00640 1 -- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0744003ab0 msgr2=0x7f075c0c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f075be00640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0744003ab0 0x7f075c0c4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f075be00640 1 -- 172.21.15.66:0/2118139802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c0c8120 con 0x7f075c1fc240 2024-09-10T22:11:57.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f075be00640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f075c1fc240 0x7f075c1fb680 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f0748004da0 tx=0x7f0748001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:57.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0758c00640 1 -- 172.21.15.66:0/2118139802 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748001d50 con 0x7f075c1fc240 2024-09-10T22:11:57.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0758c00640 1 -- 172.21.15.66:0/2118139802 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0748016070 con 0x7f075c1fc240 2024-09-10T22:11:57.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0763600640 1 -- 172.21.15.66:0/2118139802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c0b7750 con 0x7f075c1fc240 2024-09-10T22:11:57.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0758c00640 1 -- 172.21.15.66:0/2118139802 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748010b40 con 0x7f075c1fc240 2024-09-10T22:11:57.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.108+0000 7f0763600640 1 -- 172.21.15.66:0/2118139802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c0c7660 con 0x7f075c1fc240 2024-09-10T22:11:57.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.112+0000 7f0763600640 1 -- 172.21.15.66:0/2118139802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075c1fbfd0 con 0x7f075c1fc240 2024-09-10T22:11:57.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.112+0000 7f0758c00640 1 -- 172.21.15.66:0/2118139802 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f0748010ce0 con 0x7f075c1fc240 2024-09-10T22:11:57.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.112+0000 7f0758c00640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f073c073f30 0x7f073c0763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:57.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.112+0000 7f0758c00640 1 -- 172.21.15.66:0/2118139802 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f0748092730 con 0x7f075c1fc240 2024-09-10T22:11:57.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.112+0000 7f0760c00640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f073c073f30 0x7f073c0763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:57.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.112+0000 7f0760c00640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f073c073f30 0x7f073c0763f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f07500ae000 tx=0x7f075001a010 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:57.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.116+0000 7f0758c00640 1 -- 172.21.15.66:0/2118139802 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f075c1fbfd0 con 0x7f075c1fc240 2024-09-10T22:11:57.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.288+0000 7f0763600640 1 -- 172.21.15.66:0/2118139802 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f075c0b0df0 con 0x7f073c073f30 2024-09-10T22:11:57.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.292+0000 7f0758c00640 1 -- 172.21.15.66:0/2118139802 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252487 (secure 0 0 0) 0x7f075c0b0df0 con 0x7f073c073f30 2024-09-10T22:11:57.299 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:57.301 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:11:57.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.296+0000 7f0763600640 1 -- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f073c073f30 msgr2=0x7f073c0763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.296+0000 7f0763600640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f073c073f30 0x7f073c0763f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f07500ae000 tx=0x7f075001a010 comp rx=0 tx=0).stop 2024-09-10T22:11:57.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.296+0000 7f0763600640 1 -- 172.21.15.66:0/2118139802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f075c1fc240 msgr2=0x7f075c1fb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.296+0000 7f0763600640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f075c1fc240 0x7f075c1fb680 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f0748004da0 tx=0x7f0748001640 comp rx=0 tx=0).stop 2024-09-10T22:11:57.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.296+0000 7f0763600640 1 -- 172.21.15.66:0/2118139802 shutdown_connections 2024-09-10T22:11:57.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.296+0000 7f0763600640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f073c073f30 0x7f073c0763f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.296+0000 7f0763600640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f075c1fbbc0 0x7f075c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.296+0000 7f0763600640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f075c1fc240 0x7f075c1fb680 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.296+0000 7f0763600640 1 --2- 172.21.15.66:0/2118139802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0744003ab0 0x7f075c0c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.296+0000 7f0763600640 1 -- 172.21.15.66:0/2118139802 >> 172.21.15.66:0/2118139802 conn(0x7f075c0d1c60 msgr2=0x7f075c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:57.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.296+0000 7f0763600640 1 -- 172.21.15.66:0/2118139802 shutdown_connections 2024-09-10T22:11:57.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.296+0000 7f0763600640 1 -- 172.21.15.66:0/2118139802 wait complete. 2024-09-10T22:11:57.321 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-10T22:11:56.562710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1048192,"kb_used_data":12040,"kb_used_omap":12,"kb_used_meta":1035955,"kb_avail":753926528,"statfs":{"total":773094113280,"available":772020764672,"internally_reserved":0,"allocated":12328960,"data_stored":6003220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1060818484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.394355"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265042+0000","last_change":"2024-09-10T22:11:48.180766+0000","last_active":"2024-09-10T22:11:49.265042+0000","last_peered":"2024-09-10T22:11:49.265042+0000","last_clean":"2024-09-10T22:11:49.265042+0000","last_became_active":"2024-09-10T22:11:48.179819+0000","last_became_peered":"2024-09-10T22:11:48.179819+0000","last_unstale":"2024-09-10T22:11:49.265042+0000","last_undegraded":"2024-09-10T22:11:49.265042+0000","last_fullsized":"2024-09-10T22:11:49.265042+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:21.885630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186814+0000","last_change":"2024-09-10T22:11:48.203979+0000","last_active":"2024-09-10T22:11:49.186814+0000","last_peered":"2024-09-10T22:11:49.186814+0000","last_clean":"2024-09-10T22:11:49.186814+0000","last_became_active":"2024-09-10T22:11:48.203764+0000","last_became_peered":"2024-09-10T22:11:48.203764+0000","last_unstale":"2024-09-10T22:11:49.186814+0000","last_undegraded":"2024-09-10T22:11:49.186814+0000","last_fullsized":"2024-09-10T22:11:49.186814+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:41.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268239+0000","last_change":"2024-09-10T22:11:48.196047+0000","last_active":"2024-09-10T22:11:49.268239+0000","last_peered":"2024-09-10T22:11:49.268239+0000","last_clean":"2024-09-10T22:11:49.268239+0000","last_became_active":"2024-09-10T22:11:48.194354+0000","last_became_peered":"2024-09-10T22:11:48.194354+0000","last_unstale":"2024-09-10T22:11:49.268239+0000","last_undegraded":"2024-09-10T22:11:49.268239+0000","last_fullsized":"2024-09-10T22:11:49.268239+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:51.391605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264205+0000","last_change":"2024-09-10T22:11:48.191506+0000","last_active":"2024-09-10T22:11:49.264205+0000","last_peered":"2024-09-10T22:11:49.264205+0000","last_clean":"2024-09-10T22:11:49.264205+0000","last_became_active":"2024-09-10T22:11:48.191100+0000","last_became_peered":"2024-09-10T22:11:48.191100+0000","last_unstale":"2024-09-10T22:11:49.264205+0000","last_undegraded":"2024-09-10T22:11:49.264205+0000","last_fullsized":"2024-09-10T22:11:49.264205+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:54.388719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263748+0000","last_change":"2024-09-10T22:11:48.178358+0000","last_active":"2024-09-10T22:11:49.263748+0000","last_peered":"2024-09-10T22:11:49.263748+0000","last_clean":"2024-09-10T22:11:49.263748+0000","last_became_active":"2024-09-10T22:11:48.177965+0000","last_became_peered":"2024-09-10T22:11:48.177965+0000","last_unstale":"2024-09-10T22:11:49.263748+0000","last_undegraded":"2024-09-10T22:11:49.263748+0000","last_fullsized":"2024-09-10T22:11:49.263748+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:16.479719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264893+0000","last_change":"2024-09-10T22:11:48.182705+0000","last_active":"2024-09-10T22:11:49.264893+0000","last_peered":"2024-09-10T22:11:49.264893+0000","last_clean":"2024-09-10T22:11:49.264893+0000","last_became_active":"2024-09-10T22:11:48.182428+0000","last_became_peered":"2024-09-10T22:11:48.182428+0000","last_unstale":"2024-09-10T22:11:49.264893+0000","last_undegraded":"2024-09-10T22:11:49.264893+0000","last_fullsized":"2024-09-10T22:11:49.264893+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:56.601140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186457+0000","last_change":"2024-09-10T22:11:48.181122+0000","last_active":"2024-09-10T22:11:49.186457+0000","last_peered":"2024-09-10T22:11:49.186457+0000","last_clean":"2024-09-10T22:11:49.186457+0000","last_became_active":"2024-09-10T22:11:48.178614+0000","last_became_peered":"2024-09-10T22:11:48.178614+0000","last_unstale":"2024-09-10T22:11:49.186457+0000","last_undegraded":"2024-09-10T22:11:49.186457+0000","last_fullsized":"2024-09-10T22:11:49.186457+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:54.036670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.267748+0000","last_change":"2024-09-10T22:11:48.187279+0000","last_active":"2024-09-10T22:11:49.267748+0000","last_peered":"2024-09-10T22:11:49.267748+0000","last_clean":"2024-09-10T22:11:49.267748+0000","last_became_active":"2024-09-10T22:11:48.185130+0000","last_became_peered":"2024-09-10T22:11:48.185130+0000","last_unstale":"2024-09-10T22:11:49.267748+0000","last_undegraded":"2024-09-10T22:11:49.267748+0000","last_fullsized":"2024-09-10T22:11:49.267748+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:36.609392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265249+0000","last_change":"2024-09-10T22:11:48.180116+0000","last_active":"2024-09-10T22:11:49.265249+0000","last_peered":"2024-09-10T22:11:49.265249+0000","last_clean":"2024-09-10T22:11:49.265249+0000","last_became_active":"2024-09-10T22:11:48.179490+0000","last_became_peered":"2024-09-10T22:11:48.179490+0000","last_unstale":"2024-09-10T22:11:49.265249+0000","last_undegraded":"2024-09-10T22:11:49.265249+0000","last_fullsized":"2024-09-10T22:11:49.265249+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:19.843998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264589+0000","last_change":"2024-09-10T22:11:48.183940+0000","last_active":"2024-09-10T22:11:49.264589+0000","last_peered":"2024-09-10T22:11:49.264589+0000","last_clean":"2024-09-10T22:11:49.264589+0000","last_became_active":"2024-09-10T22:11:48.183694+0000","last_became_peered":"2024-09-10T22:11:48.183694+0000","last_unstale":"2024-09-10T22:11:49.264589+0000","last_undegraded":"2024-09-10T22:11:49.264589+0000","last_fullsized":"2024-09-10T22:11:49.264589+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:00.322112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187457+0000","last_change":"2024-09-10T22:11:48.197453+0000","last_active":"2024-09-10T22:11:49.187457+0000","last_peered":"2024-09-10T22:11:49.187457+0000","last_clean":"2024-09-10T22:11:49.187457+0000","last_became_active":"2024-09-10T22:11:48.197138+0000","last_became_peered":"2024-09-10T22:11:48.197138+0000","last_unstale":"2024-09-10T22:11:49.187457+0000","last_undegraded":"2024-09-10T22:11:49.187457+0000","last_fullsized":"2024-09-10T22:11:49.187457+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:33.059467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264403+0000","last_change":"2024-09-10T22:11:48.189077+0000","last_active":"2024-09-10T22:11:49.264403+0000","last_peered":"2024-09-10T22:11:49.264403+0000","last_clean":"2024-09-10T22:11:49.264403+0000","last_became_active":"2024-09-10T22:11:48.185421+0000","last_became_peered":"2024-09-10T22:11:48.185421+0000","last_unstale":"2024-09-10T22:11:49.264403+0000","last_undegraded":"2024-09-10T22:11:49.264403+0000","last_fullsized":"2024-09-10T22:11:49.264403+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:25.288843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268600+0000","last_change":"2024-09-10T22:11:48.183387+0000","last_active":"2024-09-10T22:11:49.268600+0000","last_peered":"2024-09-10T22:11:49.268600+0000","last_clean":"2024-09-10T22:11:49.268600+0000","last_became_active":"2024-09-10T22:11:48.183102+0000","last_became_peered":"2024-09-10T22:11:48.183102+0000","last_unstale":"2024-09-10T22:11:49.268600+0000","last_undegraded":"2024-09-10T22:11:49.268600+0000","last_fullsized":"2024-09-10T22:11:49.268600+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:58.870500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.267725+0000","last_change":"2024-09-10T22:11:48.179050+0000","last_active":"2024-09-10T22:11:49.267725+0000","last_peered":"2024-09-10T22:11:49.267725+0000","last_clean":"2024-09-10T22:11:49.267725+0000","last_became_active":"2024-09-10T22:11:48.178760+0000","last_became_peered":"2024-09-10T22:11:48.178760+0000","last_unstale":"2024-09-10T22:11:49.267725+0000","last_undegraded":"2024-09-10T22:11:49.267725+0000","last_fullsized":"2024-09-10T22:11:49.267725+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:07.805745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264482+0000","last_change":"2024-09-10T22:11:48.183435+0000","last_active":"2024-09-10T22:11:49.264482+0000","last_peered":"2024-09-10T22:11:49.264482+0000","last_clean":"2024-09-10T22:11:49.264482+0000","last_became_active":"2024-09-10T22:11:48.183243+0000","last_became_peered":"2024-09-10T22:11:48.183243+0000","last_unstale":"2024-09-10T22:11:49.264482+0000","last_undegraded":"2024-09-10T22:11:49.264482+0000","last_fullsized":"2024-09-10T22:11:49.264482+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:11.483616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263496+0000","last_change":"2024-09-10T22:11:48.186035+0000","last_active":"2024-09-10T22:11:49.263496+0000","last_peered":"2024-09-10T22:11:49.263496+0000","last_clean":"2024-09-10T22:11:49.263496+0000","last_became_active":"2024-09-10T22:11:48.185768+0000","last_became_peered":"2024-09-10T22:11:48.185768+0000","last_unstale":"2024-09-10T22:11:49.263496+0000","last_undegraded":"2024-09-10T22:11:49.263496+0000","last_fullsized":"2024-09-10T22:11:49.263496+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:22.226210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187717+0000","last_change":"2024-09-10T22:11:48.200285+0000","last_active":"2024-09-10T22:11:49.187717+0000","last_peered":"2024-09-10T22:11:49.187717+0000","last_clean":"2024-09-10T22:11:49.187717+0000","last_became_active":"2024-09-10T22:11:48.199965+0000","last_became_peered":"2024-09-10T22:11:48.199965+0000","last_unstale":"2024-09-10T22:11:49.187717+0000","last_undegraded":"2024-09-10T22:11:49.187717+0000","last_fullsized":"2024-09-10T22:11:49.187717+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:22.961644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268722+0000","last_change":"2024-09-10T22:11:48.194873+0000","last_active":"2024-09-10T22:11:49.268722+0000","last_peered":"2024-09-10T22:11:49.268722+0000","last_clean":"2024-09-10T22:11:49.268722+0000","last_became_active":"2024-09-10T22:11:48.194469+0000","last_became_peered":"2024-09-10T22:11:48.194469+0000","last_unstale":"2024-09-10T22:11:49.268722+0000","last_undegraded":"2024-09-10T22:11:49.268722+0000","last_fullsized":"2024-09-10T22:11:49.268722+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:37.166232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268290+0000","last_change":"2024-09-10T22:11:48.184827+0000","last_active":"2024-09-10T22:11:49.268290+0000","last_peered":"2024-09-10T22:11:49.268290+0000","last_clean":"2024-09-10T22:11:49.268290+0000","last_became_active":"2024-09-10T22:11:48.184539+0000","last_became_peered":"2024-09-10T22:11:48.184539+0000","last_unstale":"2024-09-10T22:11:49.268290+0000","last_undegraded":"2024-09-10T22:11:49.268290+0000","last_fullsized":"2024-09-10T22:11:49.268290+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:24.074923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186908+0000","last_change":"2024-09-10T22:11:48.199337+0000","last_active":"2024-09-10T22:11:49.186908+0000","last_peered":"2024-09-10T22:11:49.186908+0000","last_clean":"2024-09-10T22:11:49.186908+0000","last_became_active":"2024-09-10T22:11:48.199028+0000","last_became_peered":"2024-09-10T22:11:48.199028+0000","last_unstale":"2024-09-10T22:11:49.186908+0000","last_undegraded":"2024-09-10T22:11:49.186908+0000","last_fullsized":"2024-09-10T22:11:49.186908+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:57.807567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187347+0000","last_change":"2024-09-10T22:11:48.200296+0000","last_active":"2024-09-10T22:11:49.187347+0000","last_peered":"2024-09-10T22:11:49.187347+0000","last_clean":"2024-09-10T22:11:49.187347+0000","last_became_active":"2024-09-10T22:11:48.198375+0000","last_became_peered":"2024-09-10T22:11:48.198375+0000","last_unstale":"2024-09-10T22:11:49.187347+0000","last_undegraded":"2024-09-10T22:11:49.187347+0000","last_fullsized":"2024-09-10T22:11:49.187347+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:45.662642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180916+0000","last_change":"2024-09-10T22:11:48.180779+0000","last_active":"2024-09-10T22:11:49.180916+0000","last_peered":"2024-09-10T22:11:49.180916+0000","last_clean":"2024-09-10T22:11:49.180916+0000","last_became_active":"2024-09-10T22:11:48.179454+0000","last_became_peered":"2024-09-10T22:11:48.179454+0000","last_unstale":"2024-09-10T22:11:49.180916+0000","last_undegraded":"2024-09-10T22:11:49.180916+0000","last_fullsized":"2024-09-10T22:11:49.180916+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:28:27.233254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186778+0000","last_change":"2024-09-10T22:11:48.198947+0000","last_active":"2024-09-10T22:11:49.186778+0000","last_peered":"2024-09-10T22:11:49.186778+0000","last_clean":"2024-09-10T22:11:49.186778+0000","last_became_active":"2024-09-10T22:11:48.198672+0000","last_became_peered":"2024-09-10T22:11:48.198672+0000","last_unstale":"2024-09-10T22:11:49.186778+0000","last_undegraded":"2024-09-10T22:11:49.186778+0000","last_fullsized":"2024-09-10T22:11:49.186778+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:08.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180712+0000","last_change":"2024-09-10T22:11:48.177447+0000","last_active":"2024-09-10T22:11:49.180712+0000","last_peered":"2024-09-10T22:11:49.180712+0000","last_clean":"2024-09-10T22:11:49.180712+0000","last_became_active":"2024-09-10T22:11:48.177207+0000","last_became_peered":"2024-09-10T22:11:48.177207+0000","last_unstale":"2024-09-10T22:11:49.180712+0000","last_undegraded":"2024-09-10T22:11:49.180712+0000","last_fullsized":"2024-09-10T22:11:49.180712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:57.314793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187731+0000","last_change":"2024-09-10T22:11:48.201225+0000","last_active":"2024-09-10T22:11:49.187731+0000","last_peered":"2024-09-10T22:11:49.187731+0000","last_clean":"2024-09-10T22:11:49.187731+0000","last_became_active":"2024-09-10T22:11:48.200931+0000","last_became_peered":"2024-09-10T22:11:48.200931+0000","last_unstale":"2024-09-10T22:11:49.187731+0000","last_undegraded":"2024-09-10T22:11:49.187731+0000","last_fullsized":"2024-09-10T22:11:49.187731+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:16.417992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264799+0000","last_change":"2024-09-10T22:11:48.185741+0000","last_active":"2024-09-10T22:11:49.264799+0000","last_peered":"2024-09-10T22:11:49.264799+0000","last_clean":"2024-09-10T22:11:49.264799+0000","last_became_active":"2024-09-10T22:11:48.185438+0000","last_became_peered":"2024-09-10T22:11:48.185438+0000","last_unstale":"2024-09-10T22:11:49.264799+0000","last_undegraded":"2024-09-10T22:11:49.264799+0000","last_fullsized":"2024-09-10T22:11:49.264799+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:58.072179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265352+0000","last_change":"2024-09-10T22:11:48.195500+0000","last_active":"2024-09-10T22:11:49.265352+0000","last_peered":"2024-09-10T22:11:49.265352+0000","last_clean":"2024-09-10T22:11:49.265352+0000","last_became_active":"2024-09-10T22:11:48.195019+0000","last_became_peered":"2024-09-10T22:11:48.195019+0000","last_unstale":"2024-09-10T22:11:49.265352+0000","last_undegraded":"2024-09-10T22:11:49.265352+0000","last_fullsized":"2024-09-10T22:11:49.265352+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:18.164228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181343+0000","last_change":"2024-09-10T22:11:48.189755+0000","last_active":"2024-09-10T22:11:49.181343+0000","last_peered":"2024-09-10T22:11:49.181343+0000","last_clean":"2024-09-10T22:11:49.181343+0000","last_became_active":"2024-09-10T22:11:48.189436+0000","last_became_peered":"2024-09-10T22:11:48.189436+0000","last_unstale":"2024-09-10T22:11:49.181343+0000","last_undegraded":"2024-09-10T22:11:49.181343+0000","last_fullsized":"2024-09-10T22:11:49.181343+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:42.442483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187041+0000","last_change":"2024-09-10T22:11:48.199608+0000","last_active":"2024-09-10T22:11:49.187041+0000","last_peered":"2024-09-10T22:11:49.187041+0000","last_clean":"2024-09-10T22:11:49.187041+0000","last_became_active":"2024-09-10T22:11:48.199331+0000","last_became_peered":"2024-09-10T22:11:48.199331+0000","last_unstale":"2024-09-10T22:11:49.187041+0000","last_undegraded":"2024-09-10T22:11:49.187041+0000","last_fullsized":"2024-09-10T22:11:49.187041+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:20.253988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186852+0000","last_change":"2024-09-10T22:11:48.184404+0000","last_active":"2024-09-10T22:11:49.186852+0000","last_peered":"2024-09-10T22:11:49.186852+0000","last_clean":"2024-09-10T22:11:49.186852+0000","last_became_active":"2024-09-10T22:11:48.180465+0000","last_became_peered":"2024-09-10T22:11:48.180465+0000","last_unstale":"2024-09-10T22:11:49.186852+0000","last_undegraded":"2024-09-10T22:11:49.186852+0000","last_fullsized":"2024-09-10T22:11:49.186852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:02.597092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263547+0000","last_change":"2024-09-10T22:11:48.180454+0000","last_active":"2024-09-10T22:11:49.263547+0000","last_peered":"2024-09-10T22:11:49.263547+0000","last_clean":"2024-09-10T22:11:49.263547+0000","last_became_active":"2024-09-10T22:11:48.179907+0000","last_became_peered":"2024-09-10T22:11:48.179907+0000","last_unstale":"2024-09-10T22:11:49.263547+0000","last_undegraded":"2024-09-10T22:11:49.263547+0000","last_fullsized":"2024-09-10T22:11:49.263547+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:15.319909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264477+0000","last_change":"2024-09-10T22:11:48.182798+0000","last_active":"2024-09-10T22:11:49.264477+0000","last_peered":"2024-09-10T22:11:49.264477+0000","last_clean":"2024-09-10T22:11:49.264477+0000","last_became_active":"2024-09-10T22:11:48.182533+0000","last_became_peered":"2024-09-10T22:11:48.182533+0000","last_unstale":"2024-09-10T22:11:49.264477+0000","last_undegraded":"2024-09-10T22:11:49.264477+0000","last_fullsized":"2024-09-10T22:11:49.264477+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:35.986933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265665+0000","last_change":"2024-09-10T22:11:48.184978+0000","last_active":"2024-09-10T22:11:49.265665+0000","last_peered":"2024-09-10T22:11:49.265665+0000","last_clean":"2024-09-10T22:11:49.265665+0000","last_became_active":"2024-09-10T22:11:48.184722+0000","last_became_peered":"2024-09-10T22:11:48.184722+0000","last_unstale":"2024-09-10T22:11:49.265665+0000","last_undegraded":"2024-09-10T22:11:49.265665+0000","last_fullsized":"2024-09-10T22:11:49.265665+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:48.904032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187304+0000","last_change":"2024-09-10T22:11:47.179953+0000","last_active":"2024-09-10T22:11:49.187304+0000","last_peered":"2024-09-10T22:11:49.187304+0000","last_clean":"2024-09-10T22:11:49.187304+0000","last_became_active":"2024-09-10T22:11:47.179653+0000","last_became_peered":"2024-09-10T22:11:47.179653+0000","last_unstale":"2024-09-10T22:11:49.187304+0000","last_undegraded":"2024-09-10T22:11:49.187304+0000","last_fullsized":"2024-09-10T22:11:49.187304+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:17.150534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268977+0000","last_change":"2024-09-10T22:11:48.186184+0000","last_active":"2024-09-10T22:11:49.268977+0000","last_peered":"2024-09-10T22:11:49.268977+0000","last_clean":"2024-09-10T22:11:49.268977+0000","last_became_active":"2024-09-10T22:11:48.185822+0000","last_became_peered":"2024-09-10T22:11:48.185822+0000","last_unstale":"2024-09-10T22:11:49.268977+0000","last_undegraded":"2024-09-10T22:11:49.268977+0000","last_fullsized":"2024-09-10T22:11:49.268977+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:41.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268102+0000","last_change":"2024-09-10T22:11:47.184730+0000","last_active":"2024-09-10T22:11:49.268102+0000","last_peered":"2024-09-10T22:11:49.268102+0000","last_clean":"2024-09-10T22:11:49.268102+0000","last_became_active":"2024-09-10T22:11:47.184361+0000","last_became_peered":"2024-09-10T22:11:47.184361+0000","last_unstale":"2024-09-10T22:11:49.268102+0000","last_undegraded":"2024-09-10T22:11:49.268102+0000","last_fullsized":"2024-09-10T22:11:49.268102+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:58.121143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265333+0000","last_change":"2024-09-10T22:11:48.182402+0000","last_active":"2024-09-10T22:11:49.265333+0000","last_peered":"2024-09-10T22:11:49.265333+0000","last_clean":"2024-09-10T22:11:49.265333+0000","last_became_active":"2024-09-10T22:11:48.182057+0000","last_became_peered":"2024-09-10T22:11:48.182057+0000","last_unstale":"2024-09-10T22:11:49.265333+0000","last_undegraded":"2024-09-10T22:11:49.265333+0000","last_fullsized":"2024-09-10T22:11:49.265333+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:18.062008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265043+0000","last_change":"2024-09-10T22:11:47.181206+0000","last_active":"2024-09-10T22:11:49.265043+0000","last_peered":"2024-09-10T22:11:49.265043+0000","last_clean":"2024-09-10T22:11:49.265043+0000","last_became_active":"2024-09-10T22:11:47.179544+0000","last_became_peered":"2024-09-10T22:11:47.179544+0000","last_unstale":"2024-09-10T22:11:49.265043+0000","last_undegraded":"2024-09-10T22:11:49.265043+0000","last_fullsized":"2024-09-10T22:11:49.265043+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:26.154879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268446+0000","last_change":"2024-09-10T22:11:48.184144+0000","last_active":"2024-09-10T22:11:49.268446+0000","last_peered":"2024-09-10T22:11:49.268446+0000","last_clean":"2024-09-10T22:11:49.268446+0000","last_became_active":"2024-09-10T22:11:48.183877+0000","last_became_peered":"2024-09-10T22:11:48.183877+0000","last_unstale":"2024-09-10T22:11:49.268446+0000","last_undegraded":"2024-09-10T22:11:49.268446+0000","last_fullsized":"2024-09-10T22:11:49.268446+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:03.355002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263584+0000","last_change":"2024-09-10T22:11:47.182277+0000","last_active":"2024-09-10T22:11:49.263584+0000","last_peered":"2024-09-10T22:11:49.263584+0000","last_clean":"2024-09-10T22:11:49.263584+0000","last_became_active":"2024-09-10T22:11:47.181435+0000","last_became_peered":"2024-09-10T22:11:47.181435+0000","last_unstale":"2024-09-10T22:11:49.263584+0000","last_undegraded":"2024-09-10T22:11:49.263584+0000","last_fullsized":"2024-09-10T22:11:49.263584+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:47.604910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263891+0000","last_change":"2024-09-10T22:11:48.181857+0000","last_active":"2024-09-10T22:11:49.263891+0000","last_peered":"2024-09-10T22:11:49.263891+0000","last_clean":"2024-09-10T22:11:49.263891+0000","last_became_active":"2024-09-10T22:11:48.179210+0000","last_became_peered":"2024-09-10T22:11:48.179210+0000","last_unstale":"2024-09-10T22:11:49.263891+0000","last_undegraded":"2024-09-10T22:11:49.263891+0000","last_fullsized":"2024-09-10T22:11:49.263891+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:51.570280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186717+0000","last_change":"2024-09-10T22:11:47.177297+0000","last_active":"2024-09-10T22:11:49.186717+0000","last_peered":"2024-09-10T22:11:49.186717+0000","last_clean":"2024-09-10T22:11:49.186717+0000","last_became_active":"2024-09-10T22:11:47.176973+0000","last_became_peered":"2024-09-10T22:11:47.176973+0000","last_unstale":"2024-09-10T22:11:49.186717+0000","last_undegraded":"2024-09-10T22:11:49.186717+0000","last_fullsized":"2024-09-10T22:11:49.186717+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:08.162909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187075+0000","last_change":"2024-09-10T22:11:48.204412+0000","last_active":"2024-09-10T22:11:49.187075+0000","last_peered":"2024-09-10T22:11:49.187075+0000","last_clean":"2024-09-10T22:11:49.187075+0000","last_became_active":"2024-09-10T22:11:48.204105+0000","last_became_peered":"2024-09-10T22:11:48.204105+0000","last_unstale":"2024-09-10T22:11:49.187075+0000","last_undegraded":"2024-09-10T22:11:49.187075+0000","last_fullsized":"2024-09-10T22:11:49.187075+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:04.692623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181125+0000","last_change":"2024-09-10T22:11:47.184025+0000","last_active":"2024-09-10T22:11:49.181125+0000","last_peered":"2024-09-10T22:11:49.181125+0000","last_clean":"2024-09-10T22:11:49.181125+0000","last_became_active":"2024-09-10T22:11:47.183724+0000","last_became_peered":"2024-09-10T22:11:47.183724+0000","last_unstale":"2024-09-10T22:11:49.181125+0000","last_undegraded":"2024-09-10T22:11:49.181125+0000","last_fullsized":"2024-09-10T22:11:49.181125+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:50.420296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263722+0000","last_change":"2024-09-10T22:11:48.186204+0000","last_active":"2024-09-10T22:11:49.263722+0000","last_peered":"2024-09-10T22:11:49.263722+0000","last_clean":"2024-09-10T22:11:49.263722+0000","last_became_active":"2024-09-10T22:11:48.185455+0000","last_became_peered":"2024-09-10T22:11:48.185455+0000","last_unstale":"2024-09-10T22:11:49.263722+0000","last_undegraded":"2024-09-10T22:11:49.263722+0000","last_fullsized":"2024-09-10T22:11:49.263722+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:57.468704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186931+0000","last_change":"2024-09-10T22:11:47.181785+0000","last_active":"2024-09-10T22:11:49.186931+0000","last_peered":"2024-09-10T22:11:49.186931+0000","last_clean":"2024-09-10T22:11:49.186931+0000","last_became_active":"2024-09-10T22:11:47.181505+0000","last_became_peered":"2024-09-10T22:11:47.181505+0000","last_unstale":"2024-09-10T22:11:49.186931+0000","last_undegraded":"2024-09-10T22:11:49.186931+0000","last_fullsized":"2024-09-10T22:11:49.186931+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:01.594145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186384+0000","last_change":"2024-09-10T22:11:48.181429+0000","last_active":"2024-09-10T22:11:49.186384+0000","last_peered":"2024-09-10T22:11:49.186384+0000","last_clean":"2024-09-10T22:11:49.186384+0000","last_became_active":"2024-09-10T22:11:48.178978+0000","last_became_peered":"2024-09-10T22:11:48.178978+0000","last_unstale":"2024-09-10T22:11:49.186384+0000","last_undegraded":"2024-09-10T22:11:49.186384+0000","last_fullsized":"2024-09-10T22:11:49.186384+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:52.036230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263948+0000","last_change":"2024-09-10T22:11:47.187068+0000","last_active":"2024-09-10T22:11:49.263948+0000","last_peered":"2024-09-10T22:11:49.263948+0000","last_clean":"2024-09-10T22:11:49.263948+0000","last_became_active":"2024-09-10T22:11:47.186777+0000","last_became_peered":"2024-09-10T22:11:47.186777+0000","last_unstale":"2024-09-10T22:11:49.263948+0000","last_undegraded":"2024-09-10T22:11:49.263948+0000","last_fullsized":"2024-09-10T22:11:49.263948+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:21.958696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187129+0000","last_change":"2024-09-10T22:11:48.181451+0000","last_active":"2024-09-10T22:11:49.187129+0000","last_peered":"2024-09-10T22:11:49.187129+0000","last_clean":"2024-09-10T22:11:49.187129+0000","last_became_active":"2024-09-10T22:11:48.181161+0000","last_became_peered":"2024-09-10T22:11:48.181161+0000","last_unstale":"2024-09-10T22:11:49.187129+0000","last_undegraded":"2024-09-10T22:11:49.187129+0000","last_fullsized":"2024-09-10T22:11:49.187129+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:26.021716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265100+0000","last_change":"2024-09-10T22:11:47.179876+0000","last_active":"2024-09-10T22:11:49.265100+0000","last_peered":"2024-09-10T22:11:49.265100+0000","last_clean":"2024-09-10T22:11:49.265100+0000","last_became_active":"2024-09-10T22:11:47.179525+0000","last_became_peered":"2024-09-10T22:11:47.179525+0000","last_unstale":"2024-09-10T22:11:49.265100+0000","last_undegraded":"2024-09-10T22:11:49.265100+0000","last_fullsized":"2024-09-10T22:11:49.265100+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:59.073260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.269023+0000","last_change":"2024-09-10T22:11:48.187132+0000","last_active":"2024-09-10T22:11:49.269023+0000","last_peered":"2024-09-10T22:11:49.269023+0000","last_clean":"2024-09-10T22:11:49.269023+0000","last_became_active":"2024-09-10T22:11:48.186733+0000","last_became_peered":"2024-09-10T22:11:48.186733+0000","last_unstale":"2024-09-10T22:11:49.269023+0000","last_undegraded":"2024-09-10T22:11:49.269023+0000","last_fullsized":"2024-09-10T22:11:49.269023+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:57.807567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181341+0000","last_change":"2024-09-10T22:11:47.183552+0000","last_active":"2024-09-10T22:11:49.181341+0000","last_peered":"2024-09-10T22:11:49.181341+0000","last_clean":"2024-09-10T22:11:49.181341+0000","last_became_active":"2024-09-10T22:11:47.183260+0000","last_became_peered":"2024-09-10T22:11:47.183260+0000","last_unstale":"2024-09-10T22:11:49.181341+0000","last_undegraded":"2024-09-10T22:11:49.181341+0000","last_fullsized":"2024-09-10T22:11:49.181341+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:13.345360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181355+0000","last_change":"2024-09-10T22:11:48.194176+0000","last_active":"2024-09-10T22:11:49.181355+0000","last_peered":"2024-09-10T22:11:49.181355+0000","last_clean":"2024-09-10T22:11:49.181355+0000","last_became_active":"2024-09-10T22:11:48.192478+0000","last_became_peered":"2024-09-10T22:11:48.192478+0000","last_unstale":"2024-09-10T22:11:49.181355+0000","last_undegraded":"2024-09-10T22:11:49.181355+0000","last_fullsized":"2024-09-10T22:11:49.181355+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:59.444904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264160+0000","last_change":"2024-09-10T22:11:47.177926+0000","last_active":"2024-09-10T22:11:49.264160+0000","last_peered":"2024-09-10T22:11:49.264160+0000","last_clean":"2024-09-10T22:11:49.264160+0000","last_became_active":"2024-09-10T22:11:47.177598+0000","last_became_peered":"2024-09-10T22:11:47.177598+0000","last_unstale":"2024-09-10T22:11:49.264160+0000","last_undegraded":"2024-09-10T22:11:49.264160+0000","last_fullsized":"2024-09-10T22:11:49.264160+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:07.075072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186536+0000","last_change":"2024-09-10T22:11:48.178827+0000","last_active":"2024-09-10T22:11:49.186536+0000","last_peered":"2024-09-10T22:11:49.186536+0000","last_clean":"2024-09-10T22:11:49.186536+0000","last_became_active":"2024-09-10T22:11:48.177493+0000","last_became_peered":"2024-09-10T22:11:48.177493+0000","last_unstale":"2024-09-10T22:11:49.186536+0000","last_undegraded":"2024-09-10T22:11:49.186536+0000","last_fullsized":"2024-09-10T22:11:49.186536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:03.124106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":21,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264888+0000","last_change":"2024-09-10T22:11:47.187146+0000","last_active":"2024-09-10T22:11:49.264888+0000","last_peered":"2024-09-10T22:11:49.264888+0000","last_clean":"2024-09-10T22:11:49.264888+0000","last_became_active":"2024-09-10T22:11:47.186886+0000","last_became_peered":"2024-09-10T22:11:47.186886+0000","last_unstale":"2024-09-10T22:11:49.264888+0000","last_undegraded":"2024-09-10T22:11:49.264888+0000","last_fullsized":"2024-09-10T22:11:49.264888+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:13.528123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180957+0000","last_change":"2024-09-10T22:11:48.174022+0000","last_active":"2024-09-10T22:11:49.180957+0000","last_peered":"2024-09-10T22:11:49.180957+0000","last_clean":"2024-09-10T22:11:49.180957+0000","last_became_active":"2024-09-10T22:11:48.173765+0000","last_became_peered":"2024-09-10T22:11:48.173765+0000","last_unstale":"2024-09-10T22:11:49.180957+0000","last_undegraded":"2024-09-10T22:11:49.180957+0000","last_fullsized":"2024-09-10T22:11:49.180957+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:08.078020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265642+0000","last_change":"2024-09-10T22:11:47.181080+0000","last_active":"2024-09-10T22:11:49.265642+0000","last_peered":"2024-09-10T22:11:49.265642+0000","last_clean":"2024-09-10T22:11:49.265642+0000","last_became_active":"2024-09-10T22:11:47.180823+0000","last_became_peered":"2024-09-10T22:11:47.180823+0000","last_unstale":"2024-09-10T22:11:49.265642+0000","last_undegraded":"2024-09-10T22:11:49.265642+0000","last_fullsized":"2024-09-10T22:11:49.265642+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:27.455791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181216+0000","last_change":"2024-09-10T22:11:48.181817+0000","last_active":"2024-09-10T22:11:49.181216+0000","last_peered":"2024-09-10T22:11:49.181216+0000","last_clean":"2024-09-10T22:11:49.181216+0000","last_became_active":"2024-09-10T22:11:48.181514+0000","last_became_peered":"2024-09-10T22:11:48.181514+0000","last_unstale":"2024-09-10T22:11:49.181216+0000","last_undegraded":"2024-09-10T22:11:49.181216+0000","last_fullsized":"2024-09-10T22:11:49.181216+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:08.640825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268274+0000","last_change":"2024-09-10T22:11:47.176434+0000","last_active":"2024-09-10T22:11:49.268274+0000","last_peered":"2024-09-10T22:11:49.268274+0000","last_clean":"2024-09-10T22:11:49.268274+0000","last_became_active":"2024-09-10T22:11:47.175932+0000","last_became_peered":"2024-09-10T22:11:47.175932+0000","last_unstale":"2024-09-10T22:11:49.268274+0000","last_undegraded":"2024-09-10T22:11:49.268274+0000","last_fullsized":"2024-09-10T22:11:49.268274+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:41.968728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180789+0000","last_change":"2024-09-10T22:11:48.193087+0000","last_active":"2024-09-10T22:11:49.180789+0000","last_peered":"2024-09-10T22:11:49.180789+0000","last_clean":"2024-09-10T22:11:49.180789+0000","last_became_active":"2024-09-10T22:11:48.192299+0000","last_became_peered":"2024-09-10T22:11:48.192299+0000","last_unstale":"2024-09-10T22:11:49.180789+0000","last_undegraded":"2024-09-10T22:11:49.180789+0000","last_fullsized":"2024-09-10T22:11:49.180789+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:22.961644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":23,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263811+0000","last_change":"2024-09-10T22:11:47.184300+0000","last_active":"2024-09-10T22:11:49.263811+0000","last_peered":"2024-09-10T22:11:49.263811+0000","last_clean":"2024-09-10T22:11:49.263811+0000","last_became_active":"2024-09-10T22:11:47.184039+0000","last_became_peered":"2024-09-10T22:11:47.184039+0000","last_unstale":"2024-09-10T22:11:49.263811+0000","last_undegraded":"2024-09-10T22:11:49.263811+0000","last_fullsized":"2024-09-10T22:11:49.263811+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:39.714146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265318+0000","last_change":"2024-09-10T22:11:48.189515+0000","last_active":"2024-09-10T22:11:49.265318+0000","last_peered":"2024-09-10T22:11:49.265318+0000","last_clean":"2024-09-10T22:11:49.265318+0000","last_became_active":"2024-09-10T22:11:48.189102+0000","last_became_peered":"2024-09-10T22:11:48.189102+0000","last_unstale":"2024-09-10T22:11:49.265318+0000","last_undegraded":"2024-09-10T22:11:49.265318+0000","last_fullsized":"2024-09-10T22:11:49.265318+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:57.585521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265585+0000","last_change":"2024-09-10T22:11:47.182799+0000","last_active":"2024-09-10T22:11:49.265585+0000","last_peered":"2024-09-10T22:11:49.265585+0000","last_clean":"2024-09-10T22:11:49.265585+0000","last_became_active":"2024-09-10T22:11:47.180852+0000","last_became_peered":"2024-09-10T22:11:47.180852+0000","last_unstale":"2024-09-10T22:11:49.265585+0000","last_undegraded":"2024-09-10T22:11:49.265585+0000","last_fullsized":"2024-09-10T22:11:49.265585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:26.638919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265757+0000","last_change":"2024-09-10T22:11:48.195154+0000","last_active":"2024-09-10T22:11:49.265757+0000","last_peered":"2024-09-10T22:11:49.265757+0000","last_clean":"2024-09-10T22:11:49.265757+0000","last_became_active":"2024-09-10T22:11:48.194886+0000","last_became_peered":"2024-09-10T22:11:48.194886+0000","last_unstale":"2024-09-10T22:11:49.265757+0000","last_undegraded":"2024-09-10T22:11:49.265757+0000","last_fullsized":"2024-09-10T22:11:49.265757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:19.109740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265379+0000","last_change":"2024-09-10T22:11:47.174576+0000","last_active":"2024-09-10T22:11:49.265379+0000","last_peered":"2024-09-10T22:11:49.265379+0000","last_clean":"2024-09-10T22:11:49.265379+0000","last_became_active":"2024-09-10T22:11:47.174204+0000","last_became_peered":"2024-09-10T22:11:47.174204+0000","last_unstale":"2024-09-10T22:11:49.265379+0000","last_undegraded":"2024-09-10T22:11:49.265379+0000","last_fullsized":"2024-09-10T22:11:49.265379+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:55.279368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268047+0000","last_change":"2024-09-10T22:11:48.192583+0000","last_active":"2024-09-10T22:11:49.268047+0000","last_peered":"2024-09-10T22:11:49.268047+0000","last_clean":"2024-09-10T22:11:49.268047+0000","last_became_active":"2024-09-10T22:11:48.192228+0000","last_became_peered":"2024-09-10T22:11:48.192228+0000","last_unstale":"2024-09-10T22:11:49.268047+0000","last_undegraded":"2024-09-10T22:11:49.268047+0000","last_fullsized":"2024-09-10T22:11:49.268047+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:03.581375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268594+0000","last_change":"2024-09-10T22:11:47.191138+0000","last_active":"2024-09-10T22:11:49.268594+0000","last_peered":"2024-09-10T22:11:49.268594+0000","last_clean":"2024-09-10T22:11:49.268594+0000","last_became_active":"2024-09-10T22:11:47.190913+0000","last_became_peered":"2024-09-10T22:11:47.190913+0000","last_unstale":"2024-09-10T22:11:49.268594+0000","last_undegraded":"2024-09-10T22:11:49.268594+0000","last_fullsized":"2024-09-10T22:11:49.268594+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:20.539547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268707+0000","last_change":"2024-09-10T22:11:48.184152+0000","last_active":"2024-09-10T22:11:49.268707+0000","last_peered":"2024-09-10T22:11:49.268707+0000","last_clean":"2024-09-10T22:11:49.268707+0000","last_became_active":"2024-09-10T22:11:48.183865+0000","last_became_peered":"2024-09-10T22:11:48.183865+0000","last_unstale":"2024-09-10T22:11:49.268707+0000","last_undegraded":"2024-09-10T22:11:49.268707+0000","last_fullsized":"2024-09-10T22:11:49.268707+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:02.733745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186356+0000","last_change":"2024-09-10T22:11:47.182589+0000","last_active":"2024-09-10T22:11:49.186356+0000","last_peered":"2024-09-10T22:11:49.186356+0000","last_clean":"2024-09-10T22:11:49.186356+0000","last_became_active":"2024-09-10T22:11:47.182316+0000","last_became_peered":"2024-09-10T22:11:47.182316+0000","last_unstale":"2024-09-10T22:11:49.186356+0000","last_undegraded":"2024-09-10T22:11:49.186356+0000","last_fullsized":"2024-09-10T22:11:49.186356+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:53.817114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.267809+0000","last_change":"2024-09-10T22:11:48.197738+0000","last_active":"2024-09-10T22:11:49.267809+0000","last_peered":"2024-09-10T22:11:49.267809+0000","last_clean":"2024-09-10T22:11:49.267809+0000","last_became_active":"2024-09-10T22:11:48.197327+0000","last_became_peered":"2024-09-10T22:11:48.197327+0000","last_unstale":"2024-09-10T22:11:49.267809+0000","last_undegraded":"2024-09-10T22:11:49.267809+0000","last_fullsized":"2024-09-10T22:11:49.267809+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:46.221780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265637+0000","last_change":"2024-09-10T22:11:47.187951+0000","last_active":"2024-09-10T22:11:49.265637+0000","last_peered":"2024-09-10T22:11:49.265637+0000","last_clean":"2024-09-10T22:11:49.265637+0000","last_became_active":"2024-09-10T22:11:47.187697+0000","last_became_peered":"2024-09-10T22:11:47.187697+0000","last_unstale":"2024-09-10T22:11:49.265637+0000","last_undegraded":"2024-09-10T22:11:49.265637+0000","last_fullsized":"2024-09-10T22:11:49.265637+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:16.898662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263621+0000","last_change":"2024-09-10T22:11:48.179679+0000","last_active":"2024-09-10T22:11:49.263621+0000","last_peered":"2024-09-10T22:11:49.263621+0000","last_clean":"2024-09-10T22:11:49.263621+0000","last_became_active":"2024-09-10T22:11:48.179414+0000","last_became_peered":"2024-09-10T22:11:48.179414+0000","last_unstale":"2024-09-10T22:11:49.263621+0000","last_undegraded":"2024-09-10T22:11:49.263621+0000","last_fullsized":"2024-09-10T22:11:49.263621+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:00.909311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263535+0000","last_change":"2024-09-10T22:11:47.181464+0000","last_active":"2024-09-10T22:11:49.263535+0000","last_peered":"2024-09-10T22:11:49.263535+0000","last_clean":"2024-09-10T22:11:49.263535+0000","last_became_active":"2024-09-10T22:11:47.180230+0000","last_became_peered":"2024-09-10T22:11:47.180230+0000","last_unstale":"2024-09-10T22:11:49.263535+0000","last_undegraded":"2024-09-10T22:11:49.263535+0000","last_fullsized":"2024-09-10T22:11:49.263535+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:31.441376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268026+0000","last_change":"2024-09-10T22:11:48.183459+0000","last_active":"2024-09-10T22:11:49.268026+0000","last_peered":"2024-09-10T22:11:49.268026+0000","last_clean":"2024-09-10T22:11:49.268026+0000","last_became_active":"2024-09-10T22:11:48.182919+0000","last_became_peered":"2024-09-10T22:11:48.182919+0000","last_unstale":"2024-09-10T22:11:49.268026+0000","last_undegraded":"2024-09-10T22:11:49.268026+0000","last_fullsized":"2024-09-10T22:11:49.268026+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:09.487660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264725+0000","last_change":"2024-09-10T22:11:47.188145+0000","last_active":"2024-09-10T22:11:49.264725+0000","last_peered":"2024-09-10T22:11:49.264725+0000","last_clean":"2024-09-10T22:11:49.264725+0000","last_became_active":"2024-09-10T22:11:47.187866+0000","last_became_peered":"2024-09-10T22:11:47.187866+0000","last_unstale":"2024-09-10T22:11:49.264725+0000","last_undegraded":"2024-09-10T22:11:49.264725+0000","last_fullsized":"2024-09-10T22:11:49.264725+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:12.664373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180993+0000","last_change":"2024-09-10T22:11:48.190644+0000","last_active":"2024-09-10T22:11:49.180993+0000","last_peered":"2024-09-10T22:11:49.180993+0000","last_clean":"2024-09-10T22:11:49.180993+0000","last_became_active":"2024-09-10T22:11:48.190081+0000","last_became_peered":"2024-09-10T22:11:48.190081+0000","last_unstale":"2024-09-10T22:11:49.180993+0000","last_undegraded":"2024-09-10T22:11:49.180993+0000","last_fullsized":"2024-09-10T22:11:49.180993+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:14.531071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264069+0000","last_change":"2024-09-10T22:11:47.174439+0000","last_active":"2024-09-10T22:11:49.264069+0000","last_peered":"2024-09-10T22:11:49.264069+0000","last_clean":"2024-09-10T22:11:49.264069+0000","last_became_active":"2024-09-10T22:11:47.174130+0000","last_became_peered":"2024-09-10T22:11:47.174130+0000","last_unstale":"2024-09-10T22:11:49.264069+0000","last_undegraded":"2024-09-10T22:11:49.264069+0000","last_fullsized":"2024-09-10T22:11:49.264069+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:57.534207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268768+0000","last_change":"2024-09-10T22:11:48.187621+0000","last_active":"2024-09-10T22:11:49.268768+0000","last_peered":"2024-09-10T22:11:49.268768+0000","last_clean":"2024-09-10T22:11:49.268768+0000","last_became_active":"2024-09-10T22:11:48.187340+0000","last_became_peered":"2024-09-10T22:11:48.187340+0000","last_unstale":"2024-09-10T22:11:49.268768+0000","last_undegraded":"2024-09-10T22:11:49.268768+0000","last_fullsized":"2024-09-10T22:11:49.268768+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:08.747837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181466+0000","last_change":"2024-09-10T22:11:47.185574+0000","last_active":"2024-09-10T22:11:49.181466+0000","last_peered":"2024-09-10T22:11:49.181466+0000","last_clean":"2024-09-10T22:11:49.181466+0000","last_became_active":"2024-09-10T22:11:47.185262+0000","last_became_peered":"2024-09-10T22:11:47.185262+0000","last_unstale":"2024-09-10T22:11:49.181466+0000","last_undegraded":"2024-09-10T22:11:49.181466+0000","last_fullsized":"2024-09-10T22:11:49.181466+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:14.036978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.265445+0000","last_change":"2024-09-10T22:11:48.191766+0000","last_active":"2024-09-10T22:11:49.265445+0000","last_peered":"2024-09-10T22:11:49.265445+0000","last_clean":"2024-09-10T22:11:49.265445+0000","last_became_active":"2024-09-10T22:11:48.191476+0000","last_became_peered":"2024-09-10T22:11:48.191476+0000","last_unstale":"2024-09-10T22:11:49.265445+0000","last_undegraded":"2024-09-10T22:11:49.265445+0000","last_fullsized":"2024-09-10T22:11:49.265445+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:35.382116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268957+0000","last_change":"2024-09-10T22:11:47.178850+0000","last_active":"2024-09-10T22:11:49.268957+0000","last_peered":"2024-09-10T22:11:49.268957+0000","last_clean":"2024-09-10T22:11:49.268957+0000","last_became_active":"2024-09-10T22:11:47.178587+0000","last_became_peered":"2024-09-10T22:11:47.178587+0000","last_unstale":"2024-09-10T22:11:49.268957+0000","last_undegraded":"2024-09-10T22:11:49.268957+0000","last_fullsized":"2024-09-10T22:11:49.268957+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:23.791045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264595+0000","last_change":"2024-09-10T22:11:48.185926+0000","last_active":"2024-09-10T22:11:49.264595+0000","last_peered":"2024-09-10T22:11:49.264595+0000","last_clean":"2024-09-10T22:11:49.264595+0000","last_became_active":"2024-09-10T22:11:48.185651+0000","last_became_peered":"2024-09-10T22:11:48.185651+0000","last_unstale":"2024-09-10T22:11:49.264595+0000","last_undegraded":"2024-09-10T22:11:49.264595+0000","last_fullsized":"2024-09-10T22:11:49.264595+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:12.374323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264243+0000","last_change":"2024-09-10T22:11:47.175153+0000","last_active":"2024-09-10T22:11:49.264243+0000","last_peered":"2024-09-10T22:11:49.264243+0000","last_clean":"2024-09-10T22:11:49.264243+0000","last_became_active":"2024-09-10T22:11:47.174910+0000","last_became_peered":"2024-09-10T22:11:47.174910+0000","last_unstale":"2024-09-10T22:11:49.264243+0000","last_undegraded":"2024-09-10T22:11:49.264243+0000","last_fullsized":"2024-09-10T22:11:49.264243+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:07.637877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268469+0000","last_change":"2024-09-10T22:11:48.186801+0000","last_active":"2024-09-10T22:11:49.268469+0000","last_peered":"2024-09-10T22:11:49.268469+0000","last_clean":"2024-09-10T22:11:49.268469+0000","last_became_active":"2024-09-10T22:11:48.186510+0000","last_became_peered":"2024-09-10T22:11:48.186510+0000","last_unstale":"2024-09-10T22:11:49.268469+0000","last_undegraded":"2024-09-10T22:11:49.268469+0000","last_fullsized":"2024-09-10T22:11:49.268469+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:07.277597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.181554+0000","last_change":"2024-09-10T22:11:47.190009+0000","last_active":"2024-09-10T22:11:49.181554+0000","last_peered":"2024-09-10T22:11:49.181554+0000","last_clean":"2024-09-10T22:11:49.181554+0000","last_became_active":"2024-09-10T22:11:47.189727+0000","last_became_peered":"2024-09-10T22:11:47.189727+0000","last_unstale":"2024-09-10T22:11:49.181554+0000","last_undegraded":"2024-09-10T22:11:49.181554+0000","last_fullsized":"2024-09-10T22:11:49.181554+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:30.357142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187529+0000","last_change":"2024-09-10T22:11:48.195733+0000","last_active":"2024-09-10T22:11:49.187529+0000","last_peered":"2024-09-10T22:11:49.187529+0000","last_clean":"2024-09-10T22:11:49.187529+0000","last_became_active":"2024-09-10T22:11:48.195182+0000","last_became_peered":"2024-09-10T22:11:48.195182+0000","last_unstale":"2024-09-10T22:11:49.187529+0000","last_undegraded":"2024-09-10T22:11:49.187529+0000","last_fullsized":"2024-09-10T22:11:49.187529+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:57.314793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186601+0000","last_change":"2024-09-10T22:11:47.182353+0000","last_active":"2024-09-10T22:11:49.186601+0000","last_peered":"2024-09-10T22:11:49.186601+0000","last_clean":"2024-09-10T22:11:49.186601+0000","last_became_active":"2024-09-10T22:11:47.182082+0000","last_became_peered":"2024-09-10T22:11:47.182082+0000","last_unstale":"2024-09-10T22:11:49.186601+0000","last_undegraded":"2024-09-10T22:11:49.186601+0000","last_fullsized":"2024-09-10T22:11:49.186601+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:37.435038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187229+0000","last_change":"2024-09-10T22:11:48.200370+0000","last_active":"2024-09-10T22:11:49.187229+0000","last_peered":"2024-09-10T22:11:49.187229+0000","last_clean":"2024-09-10T22:11:49.187229+0000","last_became_active":"2024-09-10T22:11:48.200102+0000","last_became_peered":"2024-09-10T22:11:48.200102+0000","last_unstale":"2024-09-10T22:11:49.187229+0000","last_undegraded":"2024-09-10T22:11:49.187229+0000","last_fullsized":"2024-09-10T22:11:49.187229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:23.091121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.187581+0000","last_change":"2024-09-10T22:11:47.177554+0000","last_active":"2024-09-10T22:11:49.187581+0000","last_peered":"2024-09-10T22:11:49.187581+0000","last_clean":"2024-09-10T22:11:49.187581+0000","last_became_active":"2024-09-10T22:11:47.177260+0000","last_became_peered":"2024-09-10T22:11:47.177260+0000","last_unstale":"2024-09-10T22:11:49.187581+0000","last_undegraded":"2024-09-10T22:11:49.187581+0000","last_fullsized":"2024-09-10T22:11:49.187581+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:53.033722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268266+0000","last_change":"2024-09-10T22:11:48.178949+0000","last_active":"2024-09-10T22:11:49.268266+0000","last_peered":"2024-09-10T22:11:49.268266+0000","last_clean":"2024-09-10T22:11:49.268266+0000","last_became_active":"2024-09-10T22:11:48.178662+0000","last_became_peered":"2024-09-10T22:11:48.178662+0000","last_unstale":"2024-09-10T22:11:49.268266+0000","last_undegraded":"2024-09-10T22:11:49.268266+0000","last_fullsized":"2024-09-10T22:11:49.268266+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:14.481457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263945+0000","last_change":"2024-09-10T22:11:47.181053+0000","last_active":"2024-09-10T22:11:49.263945+0000","last_peered":"2024-09-10T22:11:49.263945+0000","last_clean":"2024-09-10T22:11:49.263945+0000","last_became_active":"2024-09-10T22:11:47.180799+0000","last_became_peered":"2024-09-10T22:11:47.180799+0000","last_unstale":"2024-09-10T22:11:49.263945+0000","last_undegraded":"2024-09-10T22:11:49.263945+0000","last_fullsized":"2024-09-10T22:11:49.263945+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:16.715511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.263656+0000","last_change":"2024-09-10T22:11:48.184225+0000","last_active":"2024-09-10T22:11:49.263656+0000","last_peered":"2024-09-10T22:11:49.263656+0000","last_clean":"2024-09-10T22:11:49.263656+0000","last_became_active":"2024-09-10T22:11:48.183190+0000","last_became_peered":"2024-09-10T22:11:48.183190+0000","last_unstale":"2024-09-10T22:11:49.263656+0000","last_undegraded":"2024-09-10T22:11:49.263656+0000","last_fullsized":"2024-09-10T22:11:49.263656+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:30.164067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.264055+0000","last_change":"2024-09-10T22:11:47.180051+0000","last_active":"2024-09-10T22:11:49.264055+0000","last_peered":"2024-09-10T22:11:49.264055+0000","last_clean":"2024-09-10T22:11:49.264055+0000","last_became_active":"2024-09-10T22:11:47.179772+0000","last_became_peered":"2024-09-10T22:11:47.179772+0000","last_unstale":"2024-09-10T22:11:49.264055+0000","last_undegraded":"2024-09-10T22:11:49.264055+0000","last_fullsized":"2024-09-10T22:11:49.264055+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:55.541109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.180927+0000","last_change":"2024-09-10T22:11:48.177392+0000","last_active":"2024-09-10T22:11:49.180927+0000","last_peered":"2024-09-10T22:11:49.180927+0000","last_clean":"2024-09-10T22:11:49.180927+0000","last_became_active":"2024-09-10T22:11:48.176877+0000","last_became_peered":"2024-09-10T22:11:48.176877+0000","last_unstale":"2024-09-10T22:11:49.180927+0000","last_undegraded":"2024-09-10T22:11:49.180927+0000","last_fullsized":"2024-09-10T22:11:49.180927+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:47.151613+0000","last_clean_scrub_stamp":"2024-09-10T22:11:47.151613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:58.537155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.186582+0000","last_change":"2024-09-10T22:11:47.183264+0000","last_active":"2024-09-10T22:11:49.186582+0000","last_peered":"2024-09-10T22:11:49.186582+0000","last_clean":"2024-09-10T22:11:49.186582+0000","last_became_active":"2024-09-10T22:11:47.182945+0000","last_became_peered":"2024-09-10T22:11:47.182945+0000","last_unstale":"2024-09-10T22:11:49.186582+0000","last_undegraded":"2024-09-10T22:11:49.186582+0000","last_fullsized":"2024-09-10T22:11:49.186582+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:11:46.148666+0000","last_clean_scrub_stamp":"2024-09-10T22:11:46.148666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:11.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":283,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T22:11:49.268454+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:11:49.268454+0000","last_peered":"2024-09-10T22:11:49.268454+0000","last_clean":"2024-09-10T22:11:49.268454+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:11:49.268454+0000","last_undegraded":"2024-09-10T22:11:49.268454+0000","last_fullsized":"2024-09-10T22:11:49.268454+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509581,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127464,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244376,"statfs":{"total":96636764160,"available":96506241024,"internally_reserved":0,"allocated":1986560,"data_stored":1193192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118568,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253272,"statfs":{"total":96636764160,"available":96515350528,"internally_reserved":0,"allocated":1388544,"data_stored":602278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104676,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267164,"statfs":{"total":96636764160,"available":96529575936,"internally_reserved":0,"allocated":1392640,"data_stored":602812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149088,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222752,"statfs":{"total":96636764160,"available":96484098048,"internally_reserved":0,"allocated":1384448,"data_stored":602208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144948,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226892,"statfs":{"total":96636764160,"available":96488337408,"internally_reserved":0,"allocated":1400832,"data_stored":603890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509581,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131044,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240796,"statfs":{"total":96636764160,"available":96502575104,"internally_reserved":0,"allocated":1392640,"data_stored":602332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509581,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127476,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244364,"statfs":{"total":96636764160,"available":96506228736,"internally_reserved":0,"allocated":1998848,"data_stored":1194300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":128514505},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144928,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226912,"statfs":{"total":96636764160,"available":96488357888,"internally_reserved":0,"allocated":1384448,"data_stored":602208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:11:57.325 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:11:57.325 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T22:11:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.516+0000 7ff1d3e00640 1 -- 172.21.15.66:0/189318186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1d40b1ba0 msgr2=0x7ff1d40cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.516+0000 7ff1d3e00640 1 --2- 172.21.15.66:0/189318186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1d40b1ba0 0x7ff1d40cc630 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40c7120 tx=0x7ff1c401c800 comp rx=0 tx=0).stop 2024-09-10T22:11:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.516+0000 7ff1d3e00640 1 -- 172.21.15.66:0/189318186 shutdown_connections 2024-09-10T22:11:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.516+0000 7ff1d3e00640 1 --2- 172.21.15.66:0/189318186 >> v2:172.21.15.94:3300/0 conn(0x7ff1d40cf360 0x7ff1d40ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.516+0000 7ff1d3e00640 1 --2- 172.21.15.66:0/189318186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1d40b1ba0 0x7ff1d40cc630 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.516+0000 7ff1d3e00640 1 -- 172.21.15.66:0/189318186 >> 172.21.15.66:0/189318186 conn(0x7ff1d40ccba0 msgr2=0x7ff1d40ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.516+0000 7ff1d3e00640 1 -- 172.21.15.66:0/189318186 shutdown_connections 2024-09-10T22:11:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.516+0000 7ff1d3e00640 1 -- 172.21.15.66:0/189318186 wait complete. 2024-09-10T22:11:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.516+0000 7ff1d3e00640 1 Processor -- start 2024-09-10T22:11:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.516+0000 7ff1d3e00640 1 -- start start 2024-09-10T22:11:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1d40b1ba0 0x7ff1d414c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1d40cf360 0x7ff1d409cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1d415e010 0x7ff1d415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d416b2a0 con 0x7ff1d415e010 2024-09-10T22:11:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d416b120 con 0x7ff1d40b1ba0 2024-09-10T22:11:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d416b420 con 0x7ff1d40cf360 2024-09-10T22:11:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1d415e010 0x7ff1d415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1d415e010 0x7ff1d415b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33170/0 (socket says 172.21.15.66:33170) 2024-09-10T22:11:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3400640 1 -- 172.21.15.66:0/229866741 learned_addr learned my addr 172.21.15.66:0/229866741 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d2a00640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1d40b1ba0 0x7ff1d414c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d2000640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1d40cf360 0x7ff1d409cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3400640 1 -- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1d40cf360 msgr2=0x7ff1d409cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3400640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1d40cf360 0x7ff1d409cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3400640 1 -- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1d40b1ba0 msgr2=0x7ff1d414c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3400640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1d40b1ba0 0x7ff1d414c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3400640 1 -- 172.21.15.66:0/229866741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d409bc70 con 0x7ff1d415e010 2024-09-10T22:11:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d2000640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1d40cf360 0x7ff1d409cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3400640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1d415e010 0x7ff1d415b4d0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00eb50 tx=0x7ff1cc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:57.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1caa00640 1 -- 172.21.15.66:0/229866741 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc004d30 con 0x7ff1d415e010 2024-09-10T22:11:57.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1caa00640 1 -- 172.21.15.66:0/229866741 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1cc00ce70 con 0x7ff1d415e010 2024-09-10T22:11:57.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3e00640 1 -- 172.21.15.66:0/229866741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d409bf50 con 0x7ff1d415e010 2024-09-10T22:11:57.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1caa00640 1 -- 172.21.15.66:0/229866741 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc005240 con 0x7ff1d415e010 2024-09-10T22:11:57.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3e00640 1 -- 172.21.15.66:0/229866741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d409c510 con 0x7ff1d415e010 2024-09-10T22:11:57.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.520+0000 7ff1d3e00640 1 -- 172.21.15.66:0/229866741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d40cc630 con 0x7ff1d415e010 2024-09-10T22:11:57.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.524+0000 7ff1caa00640 1 -- 172.21.15.66:0/229866741 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7ff1cc0053e0 con 0x7ff1d415e010 2024-09-10T22:11:57.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.524+0000 7ff1caa00640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff1b4073f30 0x7ff1b40763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:57.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.524+0000 7ff1caa00640 1 -- 172.21.15.66:0/229866741 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7ff1cc092d60 con 0x7ff1d415e010 2024-09-10T22:11:57.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.524+0000 7ff1d2a00640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff1b4073f30 0x7ff1b40763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:57.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.524+0000 7ff1d2a00640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff1b4073f30 0x7ff1b40763f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40af000 tx=0x7ff1c401b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:57.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.528+0000 7ff1caa00640 1 -- 172.21.15.66:0/229866741 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff1cc05fea0 con 0x7ff1d415e010 2024-09-10T22:11:57.632 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:11:57.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:11:57.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.764+0000 7ff1d3e00640 1 -- 172.21.15.66:0/229866741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff1d4074bd0 con 0x7ff1d415e010 2024-09-10T22:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.768+0000 7ff1caa00640 1 -- 172.21.15.66:0/229866741 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v32) ==== 190+0+147 (secure 0 0 0) 0x7ff1cc05f7a0 con 0x7ff1d415e010 2024-09-10T22:11:57.771 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T22:11:57.771 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw== 2024-09-10T22:11:57.772 INFO:teuthology.orchestra.run.smithi066.stdout: caps mds = "allow" 2024-09-10T22:11:57.772 INFO:teuthology.orchestra.run.smithi066.stdout: caps mon = "allow r" 2024-09-10T22:11:57.772 INFO:teuthology.orchestra.run.smithi066.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T22:11:57.772 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-09-10T22:11:57.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.768+0000 7ff1d3e00640 1 -- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff1b4073f30 msgr2=0x7ff1b40763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.768+0000 7ff1d3e00640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff1b4073f30 0x7ff1b40763f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40af000 tx=0x7ff1c401b040 comp rx=0 tx=0).stop 2024-09-10T22:11:57.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.768+0000 7ff1d3e00640 1 -- 172.21.15.66:0/229866741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1d415e010 msgr2=0x7ff1d415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.768+0000 7ff1d3e00640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1d415e010 0x7ff1d415b4d0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00eb50 tx=0x7ff1cc00c6a0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.768+0000 7ff1d3e00640 1 -- 172.21.15.66:0/229866741 shutdown_connections 2024-09-10T22:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.768+0000 7ff1d3e00640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff1b4073f30 0x7ff1b40763f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.768+0000 7ff1d3e00640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1d415e010 0x7ff1d415b4d0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.768+0000 7ff1d3e00640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1d40cf360 0x7ff1d409cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.768+0000 7ff1d3e00640 1 --2- 172.21.15.66:0/229866741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1d40b1ba0 0x7ff1d414c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.768+0000 7ff1d3e00640 1 -- 172.21.15.66:0/229866741 >> 172.21.15.66:0/229866741 conn(0x7ff1d40ccba0 msgr2=0x7ff1d40b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.768+0000 7ff1d3e00640 1 -- 172.21.15.66:0/229866741 shutdown_connections 2024-09-10T22:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.772+0000 7ff1d3e00640 1 -- 172.21.15.66:0/229866741 wait complete. 2024-09-10T22:11:57.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.936+0000 7fcb4b400640 1 -- 172.21.15.66:0/2697257037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb440cf180 msgr2=0x7fcb440d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.936+0000 7fcb4b400640 1 --2- 172.21.15.66:0/2697257037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb440cf180 0x7fcb440d04e0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fcb44073840 tx=0x7fcb3c01ccf0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.936+0000 7fcb4b400640 1 -- 172.21.15.66:0/2697257037 shutdown_connections 2024-09-10T22:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.936+0000 7fcb4b400640 1 --2- 172.21.15.66:0/2697257037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb440cf180 0x7fcb440d04e0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.936+0000 7fcb4b400640 1 --2- 172.21.15.66:0/2697257037 >> v2:172.21.15.94:3300/0 conn(0x7fcb4415bd40 0x7fcb440cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.936+0000 7fcb4b400640 1 -- 172.21.15.66:0/2697257037 >> 172.21.15.66:0/2697257037 conn(0x7fcb440d1c60 msgr2=0x7fcb440d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.936+0000 7fcb4b400640 1 -- 172.21.15.66:0/2697257037 shutdown_connections 2024-09-10T22:11:57.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.936+0000 7fcb4b400640 1 -- 172.21.15.66:0/2697257037 wait complete. 2024-09-10T22:11:57.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.936+0000 7fcb4b400640 1 Processor -- start 2024-09-10T22:11:57.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.936+0000 7fcb4b400640 1 -- start start 2024-09-10T22:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb4b400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cf180 0x7fcb4415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb4b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb440cf830 0x7fcb4406f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb4b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4415bd40 0x7fcb4406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb4b400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44168ba0 con 0x7fcb4415bd40 2024-09-10T22:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb4b400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44168a20 con 0x7fcb440cf830 2024-09-10T22:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb4b400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcb44168d20 con 0x7fcb440cf180 2024-09-10T22:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb49400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4415bd40 0x7fcb4406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb49400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4415bd40 0x7fcb4406e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33188/0 (socket says 172.21.15.66:33188) 2024-09-10T22:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb49400640 1 -- 172.21.15.66:0/2616471954 learned_addr learned my addr 172.21.15.66:0/2616471954 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb49400640 1 -- 172.21.15.66:0/2616471954 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cf180 msgr2=0x7fcb4415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb49400640 1 --2- 172.21.15.66:0/2616471954 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cf180 0x7fcb4415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb49400640 1 -- 172.21.15.66:0/2616471954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb440cf830 msgr2=0x7fcb4406f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:57.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb49400640 1 --2- 172.21.15.66:0/2616471954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb440cf830 0x7fcb4406f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:57.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb49400640 1 -- 172.21.15.66:0/2616471954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4406fd60 con 0x7fcb4415bd40 2024-09-10T22:11:57.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb49400640 1 --2- 172.21.15.66:0/2616471954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4415bd40 0x7fcb4406e3e0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c019900 tx=0x7fcb3c009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:57.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb40c00640 1 -- 172.21.15.66:0/2616471954 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0bb070 con 0x7fcb4415bd40 2024-09-10T22:11:57.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb40c00640 1 -- 172.21.15.66:0/2616471954 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb3c004d40 con 0x7fcb4415bd40 2024-09-10T22:11:57.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb40c00640 1 -- 172.21.15.66:0/2616471954 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0b2040 con 0x7fcb4415bd40 2024-09-10T22:11:57.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb4b400640 1 -- 172.21.15.66:0/2616471954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4415f110 con 0x7fcb4415bd40 2024-09-10T22:11:57.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb4b400640 1 -- 172.21.15.66:0/2616471954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4415a920 con 0x7fcb4415bd40 2024-09-10T22:11:57.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb4b400640 1 -- 172.21.15.66:0/2616471954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb4415ac50 con 0x7fcb4415bd40 2024-09-10T22:11:57.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.940+0000 7fcb40c00640 1 -- 172.21.15.66:0/2616471954 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fcb3c0b5030 con 0x7fcb4415bd40 2024-09-10T22:11:57.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.944+0000 7fcb40c00640 1 --2- 172.21.15.66:0/2616471954 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcb24073f30 0x7fcb240763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:57.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.944+0000 7fcb40c00640 1 -- 172.21.15.66:0/2616471954 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fcb3c12bad0 con 0x7fcb4415bd40 2024-09-10T22:11:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.944+0000 7fcb48a00640 1 --2- 172.21.15.66:0/2616471954 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcb24073f30 0x7fcb240763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.944+0000 7fcb48a00640 1 --2- 172.21.15.66:0/2616471954 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcb24073f30 0x7fcb240763f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fcb380058d0 tx=0x7fcb38005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:57.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:57.948+0000 7fcb40c00640 1 -- 172.21.15.66:0/2616471954 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcb3c0f1de0 con 0x7fcb4415bd40 2024-09-10T22:11:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.152+0000 7fcb4b400640 1 -- 172.21.15.66:0/2616471954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb440aec10 con 0x7fcb4415bd40 2024-09-10T22:11:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.152+0000 7fcb40c00640 1 -- 172.21.15.66:0/2616471954 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5222 (secure 0 0 0) 0x7fcb4406fd60 con 0x7fcb4415bd40 2024-09-10T22:11:58.158 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:11:58.158 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":76,"btime":"2024-09-10T22:11:51:187007+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1769669365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1769669365},{"type":"v1","addr":"172.21.15.94:6836","nonce":1769669365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8301,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/298680158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":298680158},{"type":"v1","addr":"172.21.15.66:6835","nonce":298680158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1808370701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1808370701},{"type":"v1","addr":"172.21.15.66:6834","nonce":1808370701}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:11:47.579843+0000","modified":"2024-09-10T22:11:50.583212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8307":{"gid":8307,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/3818206155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3818206155},{"type":"v1","addr":"172.21.15.94:6837","nonce":3818206155}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8307,"qdb_cluster":[8307]},"id":8}]} 2024-09-10T22:11:58.158 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 76 2024-09-10T22:11:58.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.156+0000 7fcb4b400640 1 -- 172.21.15.66:0/2616471954 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcb24073f30 msgr2=0x7fcb240763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.156+0000 7fcb4b400640 1 --2- 172.21.15.66:0/2616471954 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcb24073f30 0x7fcb240763f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fcb380058d0 tx=0x7fcb38005860 comp rx=0 tx=0).stop 2024-09-10T22:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.156+0000 7fcb4b400640 1 -- 172.21.15.66:0/2616471954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4415bd40 msgr2=0x7fcb4406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.156+0000 7fcb4b400640 1 --2- 172.21.15.66:0/2616471954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4415bd40 0x7fcb4406e3e0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c019900 tx=0x7fcb3c009370 comp rx=0 tx=0).stop 2024-09-10T22:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.156+0000 7fcb4b400640 1 -- 172.21.15.66:0/2616471954 shutdown_connections 2024-09-10T22:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.156+0000 7fcb4b400640 1 --2- 172.21.15.66:0/2616471954 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcb24073f30 0x7fcb240763f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.156+0000 7fcb4b400640 1 --2- 172.21.15.66:0/2616471954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4415bd40 0x7fcb4406e3e0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.156+0000 7fcb4b400640 1 --2- 172.21.15.66:0/2616471954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb440cf830 0x7fcb4406f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.156+0000 7fcb4b400640 1 --2- 172.21.15.66:0/2616471954 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cf180 0x7fcb4415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.156+0000 7fcb4b400640 1 -- 172.21.15.66:0/2616471954 >> 172.21.15.66:0/2616471954 conn(0x7fcb440d1c60 msgr2=0x7fcb44154b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.156+0000 7fcb4b400640 1 -- 172.21.15.66:0/2616471954 shutdown_connections 2024-09-10T22:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.156+0000 7fcb4b400640 1 -- 172.21.15.66:0/2616471954 wait complete. 2024-09-10T22:11:58.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:11:47.579843+0000', 'modified': '2024-09-10T22:11:50.583212+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8307}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8307': {'gid': 8307, 'name': 'b', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6837/3818206155', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6835', 'nonce': 3818206155}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 3818206155}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8307, 'qdb_cluster': [8307]} 2024-09-10T22:11:58.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:11:58.582 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T22:11:58.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 -- 172.21.15.66:0/1849518258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda080b0f60 msgr2=0x7fda080d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 --2- 172.21.15.66:0/1849518258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda080b0f60 0x7fda080d0140 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009900 tx=0x7fd9f001c790 comp rx=0 tx=0).stop 2024-09-10T22:11:58.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 -- 172.21.15.66:0/1849518258 shutdown_connections 2024-09-10T22:11:58.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 --2- 172.21.15.66:0/1849518258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda080b0f60 0x7fda080d0140 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 --2- 172.21.15.66:0/1849518258 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda0815f460 0x7fda080cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 -- 172.21.15.66:0/1849518258 >> 172.21.15.66:0/1849518258 conn(0x7fda080d1d00 msgr2=0x7fda080d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:58.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 -- 172.21.15.66:0/1849518258 shutdown_connections 2024-09-10T22:11:58.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 -- 172.21.15.66:0/1849518258 wait complete. 2024-09-10T22:11:58.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 Processor -- start 2024-09-10T22:11:58.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 -- start start 2024-09-10T22:11:58.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda080b0f60 0x7fda0814c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda080b32a0 0x7fda081f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda0815f460 0x7fda081f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fda08168ff0 con 0x7fda0815f460 2024-09-10T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda08168e70 con 0x7fda080b0f60 2024-09-10T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda0cc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fda08169170 con 0x7fda080b32a0 2024-09-10T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda06800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda0815f460 0x7fda081f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda06800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda0815f460 0x7fda081f9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33198/0 (socket says 172.21.15.66:33198) 2024-09-10T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda06800640 1 -- 172.21.15.66:0/849604508 learned_addr learned my addr 172.21.15.66:0/849604508 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fd9fd400640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda080b32a0 0x7fda081f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda05e00640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda080b0f60 0x7fda0814c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda06800640 1 -- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda080b32a0 msgr2=0x7fda081f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda06800640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda080b32a0 0x7fda081f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda06800640 1 -- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda080b0f60 msgr2=0x7fda0814c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda06800640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda080b0f60 0x7fda0814c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.728+0000 7fda06800640 1 -- 172.21.15.66:0/849604508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda080c54d0 con 0x7fda0815f460 2024-09-10T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.732+0000 7fda06800640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda0815f460 0x7fda081f9c00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c7c0 tx=0x7fda0000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.732+0000 7fd9fea00640 1 -- 172.21.15.66:0/849604508 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00003b80 con 0x7fda0815f460 2024-09-10T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.732+0000 7fd9fea00640 1 -- 172.21.15.66:0/849604508 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda00003d20 con 0x7fda0815f460 2024-09-10T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.732+0000 7fd9fea00640 1 -- 172.21.15.66:0/849604508 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0000aab0 con 0x7fda0815f460 2024-09-10T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.732+0000 7fda0cc00640 1 -- 172.21.15.66:0/849604508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda080cf100 con 0x7fda0815f460 2024-09-10T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.732+0000 7fda0cc00640 1 -- 172.21.15.66:0/849604508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda080b43c0 con 0x7fda0815f460 2024-09-10T22:11:58.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.732+0000 7fd9fea00640 1 -- 172.21.15.66:0/849604508 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fda0000ac50 con 0x7fda0815f460 2024-09-10T22:11:58.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.732+0000 7fd9fea00640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd9e4073f30 0x7fd9e40763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.732+0000 7fd9fea00640 1 -- 172.21.15.66:0/849604508 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fda000920c0 con 0x7fda0815f460 2024-09-10T22:11:58.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.732+0000 7fda1005d000 1 Processor -- start 2024-09-10T22:11:58.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.732+0000 7fda1005d000 1 -- start start 2024-09-10T22:11:58.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.732+0000 7fda05e00640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd9e4073f30 0x7fd9e40763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 0x56297f32ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f32e2f0 0x56297f2e18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f2e1e10 0x56297f2e21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56297f2023f0 con 0x56297f32d9d0 2024-09-10T22:11:58.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56297f13fb20 con 0x56297f2e1e10 2024-09-10T22:11:58.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x56297f12ef50 con 0x56297f32e2f0 2024-09-10T22:11:58.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda05e00640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd9e4073f30 0x7fd9e40763f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8005fd0 tx=0x7fd9f8005e40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:58.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda05e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 0x56297f32ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda05e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 0x56297f32ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33204/0 (socket says 172.21.15.66:33204) 2024-09-10T22:11:58.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda05e00640 1 -- 172.21.15.66:0/3320749745 learned_addr learned my addr 172.21.15.66:0/3320749745 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:58.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda06800640 1 --2- 172.21.15.66:0/3320749745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f2e1e10 0x56297f2e21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fd9fd400640 1 --2- 172.21.15.66:0/3320749745 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f32e2f0 0x56297f2e18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda05e00640 1 -- 172.21.15.66:0/3320749745 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f32e2f0 msgr2=0x56297f2e18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:58.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda05e00640 1 --2- 172.21.15.66:0/3320749745 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f32e2f0 0x56297f2e18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda05e00640 1 -- 172.21.15.66:0/3320749745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f2e1e10 msgr2=0x56297f2e21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:58.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda05e00640 1 --2- 172.21.15.66:0/3320749745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f2e1e10 0x56297f2e21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda05e00640 1 -- 172.21.15.66:0/3320749745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56297f0f6590 con 0x56297f32d9d0 2024-09-10T22:11:58.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda06800640 1 --2- 172.21.15.66:0/3320749745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f2e1e10 0x56297f2e21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:58.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda05e00640 1 --2- 172.21.15.66:0/3320749745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 0x56297f32ddb0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800fdf0 tx=0x7fd9f8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c210ad5621887469 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:58.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fd9f6a00640 1 -- 172.21.15.66:0/3320749745 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8022070 con 0x56297f32d9d0 2024-09-10T22:11:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fd9f6a00640 1 -- 172.21.15.66:0/3320749745 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9f801e450 con 0x56297f32d9d0 2024-09-10T22:11:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fd9f6a00640 1 -- 172.21.15.66:0/3320749745 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f801e800 con 0x56297f32d9d0 2024-09-10T22:11:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 -- 172.21.15.66:0/3320749745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 msgr2=0x56297f32ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 --2- 172.21.15.66:0/3320749745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 0x56297f32ddb0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800fdf0 tx=0x7fd9f8012000 comp rx=0 tx=0).stop 2024-09-10T22:11:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 -- 172.21.15.66:0/3320749745 shutdown_connections 2024-09-10T22:11:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 --2- 172.21.15.66:0/3320749745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f2e1e10 0x56297f2e21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 --2- 172.21.15.66:0/3320749745 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f32e2f0 0x56297f2e18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 --2- 172.21.15.66:0/3320749745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 0x56297f32ddb0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 -- 172.21.15.66:0/3320749745 >> 172.21.15.66:0/3320749745 conn(0x56297f1cb130 msgr2=0x56297f310e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 -- 172.21.15.66:0/3320749745 shutdown_connections 2024-09-10T22:11:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 -- 172.21.15.66:0/3320749745 wait complete. 2024-09-10T22:11:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 -- start start 2024-09-10T22:11:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.736+0000 7fda1005d000 1 Processor -- start 2024-09-10T22:11:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f2e1e10 0x56297f2e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 0x56297f2e3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32e2f0 0x56297f2e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56297f0c95d0 con 0x56297f32d9d0 2024-09-10T22:11:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56297f132d60 con 0x56297f32e2f0 2024-09-10T22:11:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x56297f0d5500 con 0x56297f2e1e10 2024-09-10T22:11:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 10 client.0 _ll_register_callbacks cb 0x56297f0d9810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:11:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9fd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 0x56297f2e3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9fd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 0x56297f2e3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33210/0 (socket says 172.21.15.66:33210) 2024-09-10T22:11:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9fd400640 1 -- 172.21.15.66:0/904788376 learned_addr learned my addr 172.21.15.66:0/904788376 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda05e00640 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f2e1e10 0x56297f2e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda06800640 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32e2f0 0x56297f2e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9fd400640 1 -- 172.21.15.66:0/904788376 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f2e1e10 msgr2=0x56297f2e36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9fd400640 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f2e1e10 0x56297f2e36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9fd400640 1 -- 172.21.15.66:0/904788376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32e2f0 msgr2=0x56297f2e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9fd400640 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32e2f0 0x56297f2e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9fd400640 1 -- 172.21.15.66:0/904788376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56297f0cd170 con 0x56297f32d9d0 2024-09-10T22:11:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9fd400640 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 0x56297f2e3be0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00b0040 tx=0x7fd9f0004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45079982d7faa44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9eb400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:11:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9eb400640 1 -- 172.21.15.66:0/904788376 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00b2040 con 0x56297f32d9d0 2024-09-10T22:11:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9eb400640 1 -- 172.21.15.66:0/904788376 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fd9f00b7b40 con 0x56297f32d9d0 2024-09-10T22:11:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 1 -- 172.21.15.66:0/904788376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56297f0c70f0 con 0x56297f32d9d0 2024-09-10T22:11:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9eb400640 1 -- 172.21.15.66:0/904788376 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd9f00b7d40 con 0x56297f32d9d0 2024-09-10T22:11:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 10 client.8451 fetch_fsmap learned FSMap version 76 2024-09-10T22:11:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 1 -- 172.21.15.66:0/904788376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x56297f1b3ef0 con 0x56297f32d9d0 2024-09-10T22:11:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9eb400640 1 -- 172.21.15.66:0/904788376 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56297f0c70f0 con 0x56297f32d9d0 2024-09-10T22:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9eb400640 1 -- 172.21.15.66:0/904788376 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7fd9f00b5070 con 0x56297f32d9d0 2024-09-10T22:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 10 client.8451 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T22:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 10 client.8451 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 20 client.8451 populate_metadata read hostname 'smithi066' 2024-09-10T22:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.94:6835/3818206155,v1:172.21.15.94:6837/3818206155] conn(0x56297f2e4b30 0x56297f343370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 4 client.8451 mds_command: new command op to 8307 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda1005d000 1 -- 172.21.15.66:0/904788376 --> [v2:172.21.15.94:6835/3818206155,v1:172.21.15.94:6837/3818206155] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56297f141a70 con 0x56297f2e4b30 2024-09-10T22:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda05e00640 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.94:6835/3818206155,v1:172.21.15.94:6837/3818206155] conn(0x56297f2e4b30 0x56297f343370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fda05e00640 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.94:6835/3818206155,v1:172.21.15.94:6837/3818206155] conn(0x56297f2e4b30 0x56297f343370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9a8c04fa55437 server_cookie=631e1ff291100183 in_seq=0 out_seq=0 2024-09-10T22:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.740+0000 7fd9eb400640 10 client.8451 ms_handle_connect on v2:172.21.15.94:6835/3818206155 2024-09-10T22:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fd9eb400640 1 -- 172.21.15.66:0/904788376 <== mds.0 v2:172.21.15.94:6835/3818206155 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56297f141a70 con 0x56297f2e4b30 2024-09-10T22:11:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fd9eb400640 10 client.8451 handle_command_reply: tid=0 2024-09-10T22:11:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fda1005d000 1 client.8451 shutdown 2024-09-10T22:11:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fda1005d000 1 -- 172.21.15.66:0/904788376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 msgr2=0x56297f2e3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fda1005d000 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 0x56297f2e3be0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00b0040 tx=0x7fd9f0004580 comp rx=0 tx=0).stop 2024-09-10T22:11:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fda1005d000 1 -- 172.21.15.66:0/904788376 shutdown_connections 2024-09-10T22:11:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fda1005d000 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.94:6835/3818206155,v1:172.21.15.94:6837/3818206155] conn(0x56297f2e4b30 0x56297f343370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fda1005d000 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32e2f0 0x56297f2e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fda1005d000 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32d9d0 0x56297f2e3be0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fda1005d000 1 --2- 172.21.15.66:0/904788376 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f2e1e10 0x56297f2e36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fd9eb400640 0 client.8451 ms_handle_reset on v2:172.21.15.94:6835/3818206155 2024-09-10T22:11:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fda1005d000 1 -- 172.21.15.66:0/904788376 >> 172.21.15.66:0/904788376 conn(0x56297f1cb130 msgr2=0x56297f2e0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fda1005d000 1 -- 172.21.15.66:0/904788376 shutdown_connections 2024-09-10T22:11:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fda1005d000 1 -- 172.21.15.66:0/904788376 wait complete. 2024-09-10T22:11:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.744+0000 7fda1005d000 20 client.8451 trim_cache size 0 max 16384 2024-09-10T22:11:58.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 Processor -- start 2024-09-10T22:11:58.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 -- start start 2024-09-10T22:11:58.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f2e1e10 0x56297f2e10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32d9d0 0x56297f2e15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32e2f0 0x56297f339030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fda0806f730 con 0x56297f32e2f0 2024-09-10T22:11:58.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda0814fba0 con 0x56297f32d9d0 2024-09-10T22:11:58.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fda0806f0e0 con 0x56297f2e1e10 2024-09-10T22:11:58.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda06800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32e2f0 0x56297f339030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda06800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32e2f0 0x56297f339030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33218/0 (socket says 172.21.15.66:33218) 2024-09-10T22:11:58.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda06800640 1 -- 172.21.15.66:0/212568465 learned_addr learned my addr 172.21.15.66:0/212568465 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:58.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fd9fd400640 1 --2- 172.21.15.66:0/212568465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32d9d0 0x56297f2e15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda05e00640 1 --2- 172.21.15.66:0/212568465 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f2e1e10 0x56297f2e10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda06800640 1 -- 172.21.15.66:0/212568465 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f2e1e10 msgr2=0x56297f2e10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:58.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda06800640 1 --2- 172.21.15.66:0/212568465 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f2e1e10 0x56297f2e10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda06800640 1 -- 172.21.15.66:0/212568465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32d9d0 msgr2=0x56297f2e15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:58.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda06800640 1 --2- 172.21.15.66:0/212568465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32d9d0 0x56297f2e15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda06800640 1 -- 172.21.15.66:0/212568465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56297f18ad90 con 0x56297f32e2f0 2024-09-10T22:11:58.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fd9fd400640 1 --2- 172.21.15.66:0/212568465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32d9d0 0x56297f2e15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:58.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda06800640 1 --2- 172.21.15.66:0/212568465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32e2f0 0x56297f339030 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fda00092390 tx=0x7fda00005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391555126ab3ccb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:58.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fd9f6000640 1 -- 172.21.15.66:0/212568465 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00067d70 con 0x56297f32e2f0 2024-09-10T22:11:58.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fd9f6000640 1 -- 172.21.15.66:0/212568465 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda0005eb20 con 0x56297f32e2f0 2024-09-10T22:11:58.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fd9f6000640 1 -- 172.21.15.66:0/212568465 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0005dfe0 con 0x56297f32e2f0 2024-09-10T22:11:58.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 -- 172.21.15.66:0/212568465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32e2f0 msgr2=0x56297f339030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 --2- 172.21.15.66:0/212568465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32e2f0 0x56297f339030 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fda00092390 tx=0x7fda00005040 comp rx=0 tx=0).stop 2024-09-10T22:11:58.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 -- 172.21.15.66:0/212568465 shutdown_connections 2024-09-10T22:11:58.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 --2- 172.21.15.66:0/212568465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f32e2f0 0x56297f339030 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 --2- 172.21.15.66:0/212568465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32d9d0 0x56297f2e15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 --2- 172.21.15.66:0/212568465 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f2e1e10 0x56297f2e10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 -- 172.21.15.66:0/212568465 >> 172.21.15.66:0/212568465 conn(0x56297f1cb130 msgr2=0x56297f30df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:58.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 -- 172.21.15.66:0/212568465 shutdown_connections 2024-09-10T22:11:58.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 -- 172.21.15.66:0/212568465 wait complete. 2024-09-10T22:11:58.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 -- start start 2024-09-10T22:11:58.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.764+0000 7fda1005d000 1 Processor -- start 2024-09-10T22:11:58.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f2e1e10 0x56297f3a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f32d9d0 0x56297f3a8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32e2f0 0x56297f3a85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56297f1b3a60 con 0x56297f2e1e10 2024-09-10T22:11:58.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56297f0faec0 con 0x56297f32e2f0 2024-09-10T22:11:58.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x56297f132070 con 0x56297f32d9d0 2024-09-10T22:11:58.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 10 client.0 _ll_register_callbacks cb 0x7fd9f00b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:11:58.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda05e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f2e1e10 0x56297f3a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda05e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f2e1e10 0x56297f3a7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33224/0 (socket says 172.21.15.66:33224) 2024-09-10T22:11:58.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda05e00640 1 -- 172.21.15.66:0/1422475913 learned_addr learned my addr 172.21.15.66:0/1422475913 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:58.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9fd400640 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f32d9d0 0x56297f3a8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda06800640 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32e2f0 0x56297f3a85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda05e00640 1 -- 172.21.15.66:0/1422475913 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f32d9d0 msgr2=0x56297f3a8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:58.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda05e00640 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f32d9d0 0x56297f3a8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda05e00640 1 -- 172.21.15.66:0/1422475913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32e2f0 msgr2=0x56297f3a85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:58.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda05e00640 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32e2f0 0x56297f3a85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda05e00640 1 -- 172.21.15.66:0/1422475913 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56297f0f5a90 con 0x56297f2e1e10 2024-09-10T22:11:58.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda05e00640 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f2e1e10 0x56297f3a7b20 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800c010 tx=0x7fd9f800ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1f742ed98d59bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:58.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9eb400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:11:58.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9eb400640 1 -- 172.21.15.66:0/1422475913 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8015040 con 0x56297f2e1e10 2024-09-10T22:11:58.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9eb400640 1 -- 172.21.15.66:0/1422475913 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fd9f8004240 con 0x56297f2e1e10 2024-09-10T22:11:58.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 -- 172.21.15.66:0/1422475913 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda0005eb20 con 0x56297f2e1e10 2024-09-10T22:11:58.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9eb400640 1 -- 172.21.15.66:0/1422475913 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd9f8043050 con 0x56297f2e1e10 2024-09-10T22:11:58.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 10 client.8457 fetch_fsmap learned FSMap version 76 2024-09-10T22:11:58.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 -- 172.21.15.66:0/1422475913 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x56297f0d4050 con 0x56297f2e1e10 2024-09-10T22:11:58.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9eb400640 1 -- 172.21.15.66:0/1422475913 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda0005eb20 con 0x56297f2e1e10 2024-09-10T22:11:58.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9eb400640 1 -- 172.21.15.66:0/1422475913 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7fd9f8046020 con 0x56297f2e1e10 2024-09-10T22:11:58.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 10 client.8457 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T22:11:58.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 10 client.8457 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:11:58.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 20 client.8457 populate_metadata read hostname 'smithi066' 2024-09-10T22:11:58.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.94:6835/3818206155,v1:172.21.15.94:6837/3818206155] conn(0x56297f33d8b0 0x56297f33c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 4 client.8457 mds_command: new command op to 8307 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:11:58.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 -- 172.21.15.66:0/1422475913 --> [v2:172.21.15.94:6835/3818206155,v1:172.21.15.94:6837/3818206155] -- command(tid 0: {"prefix": "status"}) -- 0x56297f13d520 con 0x56297f33d8b0 2024-09-10T22:11:58.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9fd400640 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.94:6835/3818206155,v1:172.21.15.94:6837/3818206155] conn(0x56297f33d8b0 0x56297f33c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9fd400640 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.94:6835/3818206155,v1:172.21.15.94:6837/3818206155] conn(0x56297f33d8b0 0x56297f33c9a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c1df6a715e2786b server_cookie=5fa3e55e83c1532a in_seq=0 out_seq=0 2024-09-10T22:11:58.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9eb400640 10 client.8457 ms_handle_connect on v2:172.21.15.94:6835/3818206155 2024-09-10T22:11:58.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9eb400640 1 -- 172.21.15.66:0/1422475913 <== mds.0 v2:172.21.15.94:6835/3818206155 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x56297f13d520 con 0x56297f33d8b0 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9eb400640 10 client.8457 handle_command_reply: tid=0 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 client.8457 shutdown 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 -- 172.21.15.66:0/1422475913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f2e1e10 msgr2=0x56297f3a7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f2e1e10 0x56297f3a7b20 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800c010 tx=0x7fd9f800ff10 comp rx=0 tx=0).stop 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 -- 172.21.15.66:0/1422475913 shutdown_connections 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.94:6835/3818206155,v1:172.21.15.94:6837/3818206155] conn(0x56297f33d8b0 0x56297f33c9a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297f32e2f0 0x56297f3a85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x56297f32d9d0 0x56297f3a8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fd9eb400640 0 client.8457 ms_handle_reset on v2:172.21.15.94:6835/3818206155 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 --2- 172.21.15.66:0/1422475913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56297f2e1e10 0x56297f3a7b20 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.768+0000 7fda1005d000 1 -- 172.21.15.66:0/1422475913 >> 172.21.15.66:0/1422475913 conn(0x56297f1cb130 msgr2=0x56297f2e00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda1005d000 1 -- 172.21.15.66:0/1422475913 shutdown_connections 2024-09-10T22:11:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda1005d000 1 -- 172.21.15.66:0/1422475913 wait complete. 2024-09-10T22:11:58.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda1005d000 20 client.8457 trim_cache size 0 max 16384 2024-09-10T22:11:58.780 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:11:58.780 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:11:58.780 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:11:58.780 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 8307, 2024-09-10T22:11:58.780 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:11:58.780 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:11:58.780 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:11:58.780 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 10.610572981000001, 2024-09-10T22:11:58.781 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 76, 2024-09-10T22:11:58.781 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 112, 2024-09-10T22:11:58.781 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 112, 2024-09-10T22:11:58.781 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 15.068012297999999 2024-09-10T22:11:58.781 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:11:58.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda0cc00640 1 -- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd9e4073f30 msgr2=0x7fd9e40763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda0cc00640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd9e4073f30 0x7fd9e40763f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8005fd0 tx=0x7fd9f8005e40 comp rx=0 tx=0).stop 2024-09-10T22:11:58.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda0cc00640 1 -- 172.21.15.66:0/849604508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda0815f460 msgr2=0x7fda081f9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda0cc00640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda0815f460 0x7fda081f9c00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c7c0 tx=0x7fda0000cc80 comp rx=0 tx=0).stop 2024-09-10T22:11:58.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda0cc00640 1 -- 172.21.15.66:0/849604508 shutdown_connections 2024-09-10T22:11:58.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda0cc00640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd9e4073f30 0x7fd9e40763f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda0cc00640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda0815f460 0x7fda081f9c00 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda0cc00640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda080b32a0 0x7fda081f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda0cc00640 1 --2- 172.21.15.66:0/849604508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda080b0f60 0x7fda0814c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda0cc00640 1 -- 172.21.15.66:0/849604508 >> 172.21.15.66:0/849604508 conn(0x7fda080d1d00 msgr2=0x7fda0815e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda0cc00640 1 -- 172.21.15.66:0/849604508 shutdown_connections 2024-09-10T22:11:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.772+0000 7fda0cc00640 1 -- 172.21.15.66:0/849604508 wait complete. 2024-09-10T22:11:58.795 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:11:58.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 -- 172.21.15.66:0/3500899070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f709015a310 msgr2=0x7f70900c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 --2- 172.21.15.66:0/3500899070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f709015a310 0x7f70900c48b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f708000a980 tx=0x7f708001db00 comp rx=0 tx=0).stop 2024-09-10T22:11:58.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 -- 172.21.15.66:0/3500899070 shutdown_connections 2024-09-10T22:11:58.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 --2- 172.21.15.66:0/3500899070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f709015a310 0x7f70900c48b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 -- 172.21.15.66:0/3500899070 >> 172.21.15.66:0/3500899070 conn(0x7f70900cfa30 msgr2=0x7f70900cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:58.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 -- 172.21.15.66:0/3500899070 shutdown_connections 2024-09-10T22:11:58.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 -- 172.21.15.66:0/3500899070 wait complete. 2024-09-10T22:11:58.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 Processor -- start 2024-09-10T22:11:58.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 -- start start 2024-09-10T22:11:58.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70900aeec0 0x7f70901f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f709015a310 0x7f7090147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f709014a990 0x7f709014c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f70901687b0 con 0x7f709015a310 2024-09-10T22:11:58.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7090168630 con 0x7f70900aeec0 2024-09-10T22:11:58.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7096200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7090168930 con 0x7f709014a990 2024-09-10T22:11:58.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.940+0000 7f7087e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f709015a310 0x7f7090147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7087e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f709015a310 0x7f7090147070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33226/0 (socket says 172.21.15.66:33226) 2024-09-10T22:11:58.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7087e00640 1 -- 172.21.15.66:0/437203967 learned_addr learned my addr 172.21.15.66:0/437203967 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:11:58.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f708f400640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70900aeec0 0x7f70901f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f708fe00640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f709014a990 0x7f709014c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7087e00640 1 -- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f709014a990 msgr2=0x7f709014c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7087e00640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f709014a990 0x7f709014c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7087e00640 1 -- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70900aeec0 msgr2=0x7f70901f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:58.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7087e00640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70900aeec0 0x7f70901f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:58.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7087e00640 1 -- 172.21.15.66:0/437203967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70901fbad0 con 0x7f709015a310 2024-09-10T22:11:58.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f708fe00640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f709014a990 0x7f709014c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:58.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7087e00640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f709015a310 0x7f7090147070 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f707800c490 tx=0x7f707800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:58.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7087400640 1 -- 172.21.15.66:0/437203967 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078016020 con 0x7f709015a310 2024-09-10T22:11:58.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7087400640 1 -- 172.21.15.66:0/437203967 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7078012a00 con 0x7f709015a310 2024-09-10T22:11:58.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7096200640 1 -- 172.21.15.66:0/437203967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70901fbdb0 con 0x7f709015a310 2024-09-10T22:11:58.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7087400640 1 -- 172.21.15.66:0/437203967 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078011380 con 0x7f709015a310 2024-09-10T22:11:58.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7096200640 1 -- 172.21.15.66:0/437203967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70901fc370 con 0x7f709015a310 2024-09-10T22:11:58.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7096200640 1 -- 172.21.15.66:0/437203967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70900c48b0 con 0x7f709015a310 2024-09-10T22:11:58.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.944+0000 7f7087400640 1 -- 172.21.15.66:0/437203967 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f7078011520 con 0x7f709015a310 2024-09-10T22:11:58.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.948+0000 7f7087400640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f706c073f30 0x7f706c0763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:58.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.948+0000 7f7087400640 1 -- 172.21.15.66:0/437203967 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f7078092390 con 0x7f709015a310 2024-09-10T22:11:58.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.948+0000 7f708f400640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f706c073f30 0x7f706c0763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:58.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.948+0000 7f708f400640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f706c073f30 0x7f706c0763f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f70800b0000 tx=0x7f708001c340 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:58.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:58.952+0000 7f7087400640 1 -- 172.21.15.66:0/437203967 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7078020020 con 0x7f709015a310 2024-09-10T22:11:59.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.164+0000 7f7096200640 1 -- 172.21.15.66:0/437203967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7090072a50 con 0x7f709015a310 2024-09-10T22:11:59.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.164+0000 7f7087400640 1 -- 172.21.15.66:0/437203967 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7f707805f4d0 con 0x7f709015a310 2024-09-10T22:11:59.170 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:11:59.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.168+0000 7f7096200640 1 -- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f706c073f30 msgr2=0x7f706c0763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:59.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.168+0000 7f7096200640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f706c073f30 0x7f706c0763f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f70800b0000 tx=0x7f708001c340 comp rx=0 tx=0).stop 2024-09-10T22:11:59.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.168+0000 7f7096200640 1 -- 172.21.15.66:0/437203967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f709015a310 msgr2=0x7f7090147070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:59.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.168+0000 7f7096200640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f709015a310 0x7f7090147070 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f707800c490 tx=0x7f707800c950 comp rx=0 tx=0).stop 2024-09-10T22:11:59.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.168+0000 7f7096200640 1 -- 172.21.15.66:0/437203967 shutdown_connections 2024-09-10T22:11:59.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.168+0000 7f7096200640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f706c073f30 0x7f706c0763f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:59.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.168+0000 7f7096200640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f709014a990 0x7f709014c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:59.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.168+0000 7f7096200640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f709015a310 0x7f7090147070 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:59.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.168+0000 7f7096200640 1 --2- 172.21.15.66:0/437203967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70900aeec0 0x7f70901f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:59.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.168+0000 7f7096200640 1 -- 172.21.15.66:0/437203967 >> 172.21.15.66:0/437203967 conn(0x7f70900cfa30 msgr2=0x7f70900b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:59.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.168+0000 7f7096200640 1 -- 172.21.15.66:0/437203967 shutdown_connections 2024-09-10T22:11:59.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59.168+0000 7f7096200640 1 -- 172.21.15.66:0/437203967 wait complete. 2024-09-10T22:11:59.191 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:11:59.191 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:11:59.191 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:11:59.191 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:11:59.191 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:11:59.192 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:11:59.192 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:11:59.192 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:11:59.192 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:11:59.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:59.192 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:11:59.201 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:11:59.201 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:11:59.201 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:11:59.201 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:11:59.201 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:11:59.201 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41756sec preferred_lft 41756sec 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-09-10T22:11:59.202 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:81:fc:4c:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T22:11:59.203 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:11:59.204 INFO:teuthology.orchestra.run.smithi066.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:11:59.204 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:11:59.204 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T22:11:59.204 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:11:59.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:59.208 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:11:59.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:59.266 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:11:59.321 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:11:59.321 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:11:59.321 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:11:59.321 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:11:59.321 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:11:59.321 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:11:59.321 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:11:59.321 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:11:59.321 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:11:59.321 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:11:59.322 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:11:59.322 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:11:59.322 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:11:59.322 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:11:59.322 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:11:59.322 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:11:59.322 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:11:59.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:11:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:11:59.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:59.462 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:11:59.518 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:11:59.518 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:11:59.519 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:11:59.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:59.519 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:11:59.585 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T22:11:59.585 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:11:59.585 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:11:59.637 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:11:59.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:11:59.638 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:11:59.689 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:11:59.750 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:11:59.807 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:11:59.807 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:11:59.807 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:11:59.808 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:11:59.808 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:11:59.808 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:11:59.808 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:11:59.808 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:11:59.808 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:11:59.808 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:11:59.808 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:11:59.808 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:11:59.808 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:11:59.808 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:11:59.808 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:11:59.809 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:11:59.810 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:11:59.810 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:11:59.810 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:11:59.810 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:11:59.810 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:11:59.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:59.811 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:11:59.862 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:11:59.862 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T22:11:59.908 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:11:59.928 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:11:59.979 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:11:59.979 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:11:59.979 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:11:59.979 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:11:59.979 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:11:59.979 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:11:59.980 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:11:59.980 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:11:59.980 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:11:59.980 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:11:59.980 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:11:59.980 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:11:59.980 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:11:59.980 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:11:59.980 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:11:59.980 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:11:59.980 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:11:59.980 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:11:59.981 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:11:59.981 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:11:59.981 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:11:59.981 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:11:59.981 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:11:59.981 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:11:59.981 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:11:59.981 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:11:59.981 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:11:59.981 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:11:59.981 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:11:59.982 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:11:59.982 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:11:59.982 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:11:59.982 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:11:59.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:11:59.983 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:12:00.004 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-09-10T22:12:00.000+0000 7fe4de5db580 -1 init, newargv = 0x563c04e30b80 newargc=15 2024-09-10T22:12:00.004 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[50091]: starting ceph client 2024-09-10T22:12:00.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[50091]: starting fuse 2024-09-10T22:12:00.013 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-09-10T22:12:00.013 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T22:12:00.013 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:12:00.013 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:12:00.014 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-09-10T22:12:00.015 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:12:00.015 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:12:00.015 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:12:00.015 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-09-10T22:12:00.015 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-09-10T22:12:00.015 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-09-10T22:12:00.015 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:12:00.015 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:12:00.015 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-09-10T22:12:00.015 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:12:00.136 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.50091.asok 2024-09-10T22:12:00.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:12:00.147 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50091.asok 2024-09-10T22:12:00.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:12:00.148 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50091.asok status 2024-09-10T22:12:00.332 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:12:00.332 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-09-10T22:12:00.332 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:12:00.332 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:12:00.332 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-09-10T22:12:00.332 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-09-10T22:12:00.332 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T22:12:00.332 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "50091", 2024-09-10T22:12:00.332 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 8469, 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 8469 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 215280914 2024-09-10T22:12:00.333 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 215280914 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.8469 192.168.0.1:0/215280914", 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/215280914", 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 76, 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 112, 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-09-10T22:12:00.334 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:12:00.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:12:00.347 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-09-10T22:12:00.347 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:12:00.356 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-09-10T22:12:00.357 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T22:12:00.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:12:00.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:12:00.418 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T22:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.608+0000 7fc2e6200640 1 -- 172.21.15.66:0/3017959183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2c4004e00 msgr2=0x7fc2c4005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.608+0000 7fc2e6200640 1 --2- 172.21.15.66:0/3017959183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2c4004e00 0x7fc2c4005280 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00cd2e0 tx=0x7fc2d801cbf0 comp rx=0 tx=0).stop 2024-09-10T22:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.608+0000 7fc2dfe00640 1 -- 172.21.15.66:0/3017959183 reap_dead start 2024-09-10T22:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.608+0000 7fc2e6200640 1 -- 172.21.15.66:0/3017959183 shutdown_connections 2024-09-10T22:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.608+0000 7fc2e6200640 1 -- 172.21.15.66:0/3017959183 >> 172.21.15.66:0/3017959183 conn(0x7fc2e00cfaa0 msgr2=0x7fc2e00cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.608+0000 7fc2e6200640 1 -- 172.21.15.66:0/3017959183 shutdown_connections 2024-09-10T22:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.608+0000 7fc2e6200640 1 -- 172.21.15.66:0/3017959183 wait complete. 2024-09-10T22:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2e6200640 1 Processor -- start 2024-09-10T22:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2e6200640 1 -- start start 2024-09-10T22:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2e6200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2c4004e00 0x7fc2e01fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2e6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015dac0 0x7fc2e014b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2e6200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2e015e420 0x7fc2e00732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2e6200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0167090 con 0x7fc2e015dac0 2024-09-10T22:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2e6200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0166f10 con 0x7fc2e015e420 2024-09-10T22:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2e6200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e0167210 con 0x7fc2c4004e00 2024-09-10T22:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015dac0 0x7fc2e014b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015dac0 0x7fc2e014b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33254/0 (socket says 172.21.15.66:33254) 2024-09-10T22:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2d7e00640 1 -- 172.21.15.66:0/2810073218 learned_addr learned my addr 172.21.15.66:0/2810073218 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2df400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2c4004e00 0x7fc2e01fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2dfe00640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2e015e420 0x7fc2e00732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2d7e00640 1 -- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2c4004e00 msgr2=0x7fc2e01fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2d7e00640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2c4004e00 0x7fc2e01fc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2d7e00640 1 -- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2e015e420 msgr2=0x7fc2e00732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2d7e00640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2e015e420 0x7fc2e00732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2d7e00640 1 -- 172.21.15.66:0/2810073218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e015ed90 con 0x7fc2e015dac0 2024-09-10T22:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2d7e00640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015dac0 0x7fc2e014b5e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8011ae0 tx=0x7fc2c8009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2d7400640 1 -- 172.21.15.66:0/2810073218 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c801e070 con 0x7fc2e015dac0 2024-09-10T22:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2d7400640 1 -- 172.21.15.66:0/2810073218 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2c800fe40 con 0x7fc2e015dac0 2024-09-10T22:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2d7400640 1 -- 172.21.15.66:0/2810073218 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80042b0 con 0x7fc2e015dac0 2024-09-10T22:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2e6200640 1 -- 172.21.15.66:0/2810073218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e015f070 con 0x7fc2e015dac0 2024-09-10T22:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2e6200640 1 -- 172.21.15.66:0/2810073218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e015e8d0 con 0x7fc2e015dac0 2024-09-10T22:12:00.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.612+0000 7fc2e6200640 1 -- 172.21.15.66:0/2810073218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e015f2b0 con 0x7fc2e015dac0 2024-09-10T22:12:00.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.616+0000 7fc2d7400640 1 -- 172.21.15.66:0/2810073218 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fc2c8003600 con 0x7fc2e015dac0 2024-09-10T22:12:00.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.616+0000 7fc2d7400640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc2bc073f60 0x7fc2bc076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:00.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.616+0000 7fc2d7400640 1 -- 172.21.15.66:0/2810073218 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fc2c8092e50 con 0x7fc2e015dac0 2024-09-10T22:12:00.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.616+0000 7fc2df400640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc2bc073f60 0x7fc2bc076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:00.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.616+0000 7fc2df400640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc2bc073f60 0x7fc2bc076420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc2e015d230 tx=0x7fc2d0001070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:00.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.620+0000 7fc2d7400640 1 -- 172.21.15.66:0/2810073218 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc2c805fa00 con 0x7fc2e015dac0 2024-09-10T22:12:00.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:00.828+0000 7fc2e6200640 1 -- 172.21.15.66:0/2810073218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fc2e01fb6e0 con 0x7fc2e015dac0 2024-09-10T22:12:01.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.224+0000 7fc2d7400640 1 -- 172.21.15.66:0/2810073218 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v77) ==== 136+0+0 (secure 0 0 0) 0x7fc2c8064430 con 0x7fc2e015dac0 2024-09-10T22:12:01.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.224+0000 7fc2e6200640 1 -- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc2bc073f60 msgr2=0x7fc2bc076420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:01.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.224+0000 7fc2e6200640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc2bc073f60 0x7fc2bc076420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc2e015d230 tx=0x7fc2d0001070 comp rx=0 tx=0).stop 2024-09-10T22:12:01.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.224+0000 7fc2e6200640 1 -- 172.21.15.66:0/2810073218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015dac0 msgr2=0x7fc2e014b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:01.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.224+0000 7fc2e6200640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015dac0 0x7fc2e014b5e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8011ae0 tx=0x7fc2c8009ea0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.224+0000 7fc2e6200640 1 -- 172.21.15.66:0/2810073218 shutdown_connections 2024-09-10T22:12:01.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.224+0000 7fc2e6200640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc2bc073f60 0x7fc2bc076420 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.224+0000 7fc2e6200640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2e015e420 0x7fc2e00732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.224+0000 7fc2e6200640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015dac0 0x7fc2e014b5e0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.224+0000 7fc2e6200640 1 --2- 172.21.15.66:0/2810073218 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2c4004e00 0x7fc2e01fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.228+0000 7fc2e6200640 1 -- 172.21.15.66:0/2810073218 >> 172.21.15.66:0/2810073218 conn(0x7fc2e00cfaa0 msgr2=0x7fc2e00ceac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.228+0000 7fc2e6200640 1 -- 172.21.15.66:0/2810073218 shutdown_connections 2024-09-10T22:12:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.228+0000 7fc2e6200640 1 -- 172.21.15.66:0/2810073218 wait complete. 2024-09-10T22:12:01.251 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:12:01.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.396+0000 7fd06ea00640 1 -- 172.21.15.66:0/2357535881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0680d0550 msgr2=0x7fd0680c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:01.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.396+0000 7fd06ea00640 1 --2- 172.21.15.66:0/2357535881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0680d0550 0x7fd0680c4df0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd05000aa10 tx=0x7fd05001dba0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.396+0000 7fd06ea00640 1 -- 172.21.15.66:0/2357535881 shutdown_connections 2024-09-10T22:12:01.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.396+0000 7fd06ea00640 1 --2- 172.21.15.66:0/2357535881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0680d0550 0x7fd0680c4df0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.396+0000 7fd06ea00640 1 --2- 172.21.15.66:0/2357535881 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd06815a310 0x7fd0680c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.396+0000 7fd06ea00640 1 -- 172.21.15.66:0/2357535881 >> 172.21.15.66:0/2357535881 conn(0x7fd0680cfa30 msgr2=0x7fd0680cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:01.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.396+0000 7fd06ea00640 1 -- 172.21.15.66:0/2357535881 shutdown_connections 2024-09-10T22:12:01.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.396+0000 7fd06ea00640 1 -- 172.21.15.66:0/2357535881 wait complete. 2024-09-10T22:12:01.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.396+0000 7fd06ea00640 1 Processor -- start 2024-09-10T22:12:01.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd06ea00640 1 -- start start 2024-09-10T22:12:01.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd06ea00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0680aeec0 0x7fd0680b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:01.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd06ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06815a310 0x7fd068147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:01.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd06ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd068072c50 0x7fd0680b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:01.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd06ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd06815ed60 con 0x7fd06815a310 2024-09-10T22:12:01.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd06ea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd06815ebe0 con 0x7fd068072c50 2024-09-10T22:12:01.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd06ea00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd068167300 con 0x7fd0680aeec0 2024-09-10T22:12:01.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd05fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06815a310 0x7fd068147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:01.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd05fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06815a310 0x7fd068147d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33272/0 (socket says 172.21.15.66:33272) 2024-09-10T22:12:01.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd05fe00640 1 -- 172.21.15.66:0/940911094 learned_addr learned my addr 172.21.15.66:0/940911094 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:01.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd067e00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0680aeec0 0x7fd0680b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:01.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd05fe00640 1 -- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0680aeec0 msgr2=0x7fd0680b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:01.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd06ca00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd068072c50 0x7fd0680b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:01.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd05fe00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0680aeec0 0x7fd0680b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd05fe00640 1 -- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd068072c50 msgr2=0x7fd0680b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:01.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd05fe00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd068072c50 0x7fd0680b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd05fe00640 1 -- 172.21.15.66:0/940911094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0681fb170 con 0x7fd06815a310 2024-09-10T22:12:01.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd067e00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0680aeec0 0x7fd0680b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:12:01.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd06ca00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd068072c50 0x7fd0680b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:12:01.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd05fe00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06815a310 0x7fd068147d30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd05000a960 tx=0x7fd0500a67e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:01.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd064c00640 1 -- 172.21.15.66:0/940911094 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500be070 con 0x7fd06815a310 2024-09-10T22:12:01.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd064c00640 1 -- 172.21.15.66:0/940911094 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd050003b80 con 0x7fd06815a310 2024-09-10T22:12:01.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd064c00640 1 -- 172.21.15.66:0/940911094 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500b3070 con 0x7fd06815a310 2024-09-10T22:12:01.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd06ea00640 1 -- 172.21.15.66:0/940911094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0681fb450 con 0x7fd06815a310 2024-09-10T22:12:01.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd06ea00640 1 -- 172.21.15.66:0/940911094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0681fba10 con 0x7fd06815a310 2024-09-10T22:12:01.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.400+0000 7fd06ea00640 1 -- 172.21.15.66:0/940911094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06815df60 con 0x7fd06815a310 2024-09-10T22:12:01.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.404+0000 7fd064c00640 1 -- 172.21.15.66:0/940911094 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fd05000b040 con 0x7fd06815a310 2024-09-10T22:12:01.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.404+0000 7fd064c00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd044073f60 0x7fd044076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:01.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.404+0000 7fd064c00640 1 -- 172.21.15.66:0/940911094 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fd05012caa0 con 0x7fd06815a310 2024-09-10T22:12:01.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.404+0000 7fd067e00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd044073f60 0x7fd044076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:01.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.404+0000 7fd067e00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd044073f60 0x7fd044076420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd058005fd0 tx=0x7fd058005d00 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:01.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.408+0000 7fd064c00640 1 -- 172.21.15.66:0/940911094 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd050017820 con 0x7fd06815a310 2024-09-10T22:12:01.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.604+0000 7fd06ea00640 1 -- 172.21.15.66:0/940911094 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fd0680d0190 con 0x7fd044073f60 2024-09-10T22:12:01.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.604+0000 7fd064c00640 1 -- 172.21.15.66:0/940911094 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fd0680d0190 con 0x7fd044073f60 2024-09-10T22:12:01.609 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:12:01.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:12:01.609 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:12:01.609 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:01.609 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:12:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.608+0000 7fd06ea00640 1 -- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd044073f60 msgr2=0x7fd044076420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.608+0000 7fd06ea00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd044073f60 0x7fd044076420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd058005fd0 tx=0x7fd058005d00 comp rx=0 tx=0).stop 2024-09-10T22:12:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.608+0000 7fd06ea00640 1 -- 172.21.15.66:0/940911094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06815a310 msgr2=0x7fd068147d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.608+0000 7fd06ea00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06815a310 0x7fd068147d30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd05000a960 tx=0x7fd0500a67e0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.608+0000 7fd06ea00640 1 -- 172.21.15.66:0/940911094 shutdown_connections 2024-09-10T22:12:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.608+0000 7fd06ea00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd044073f60 0x7fd044076420 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.608+0000 7fd06ea00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd068072c50 0x7fd0680b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.608+0000 7fd06ea00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06815a310 0x7fd068147d30 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.608+0000 7fd06ea00640 1 --2- 172.21.15.66:0/940911094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0680aeec0 0x7fd0680b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.608+0000 7fd06ea00640 1 -- 172.21.15.66:0/940911094 >> 172.21.15.66:0/940911094 conn(0x7fd0680cfa30 msgr2=0x7fd0680b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:01.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.608+0000 7fd06ea00640 1 -- 172.21.15.66:0/940911094 shutdown_connections 2024-09-10T22:12:01.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.608+0000 7fd06ea00640 1 -- 172.21.15.66:0/940911094 wait complete. 2024-09-10T22:12:01.630 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:12:01.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.776+0000 7f008a000640 1 -- 172.21.15.66:0/4100522780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0068003fe0 msgr2=0x7f0068004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:01.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.776+0000 7f008a000640 1 --2- 172.21.15.66:0/4100522780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0068003fe0 0x7f0068004810 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f007000bb10 tx=0x7f007001cda0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.776+0000 7f0083e00640 1 -- 172.21.15.66:0/4100522780 reap_dead start 2024-09-10T22:12:01.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.776+0000 7f008a000640 1 -- 172.21.15.66:0/4100522780 shutdown_connections 2024-09-10T22:12:01.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.776+0000 7f008a000640 1 -- 172.21.15.66:0/4100522780 >> 172.21.15.66:0/4100522780 conn(0x7f00840d1c50 msgr2=0x7f00840d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:01.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.776+0000 7f008a000640 1 -- 172.21.15.66:0/4100522780 shutdown_connections 2024-09-10T22:12:01.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.776+0000 7f008a000640 1 -- 172.21.15.66:0/4100522780 wait complete. 2024-09-10T22:12:01.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f008a000640 1 Processor -- start 2024-09-10T22:12:01.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f008a000640 1 -- start start 2024-09-10T22:12:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f008a000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0068003fe0 0x7f0084158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f008a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008415a590 0x7f0084165040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f008a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00841f5ec0 0x7f00840ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:01.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f008a000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0084166f70 con 0x7f00841f5ec0 2024-09-10T22:12:01.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f008a000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0084166df0 con 0x7f008415a590 2024-09-10T22:12:01.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f008a000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f00841670f0 con 0x7f0068003fe0 2024-09-10T22:12:01.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f0083e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00841f5ec0 0x7f00840ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:01.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f0083400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0068003fe0 0x7f0084158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:01.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f0083e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00841f5ec0 0x7f00840ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33296/0 (socket says 172.21.15.66:33296) 2024-09-10T22:12:01.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f0083e00640 1 -- 172.21.15.66:0/558964659 learned_addr learned my addr 172.21.15.66:0/558964659 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:01.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f0082a00640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008415a590 0x7f0084165040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:01.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f0083400640 1 -- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008415a590 msgr2=0x7f0084165040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:01.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f0083400640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008415a590 0x7f0084165040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f0083400640 1 -- 172.21.15.66:0/558964659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00841f5ec0 msgr2=0x7f00840ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:01.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f0083400640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00841f5ec0 0x7f00840ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f0083400640 1 -- 172.21.15.66:0/558964659 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00841f6860 con 0x7f0068003fe0 2024-09-10T22:12:01.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f0083e00640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00841f5ec0 0x7f00840ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:12:01.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f0083400640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0068003fe0 0x7f0084158ff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0074009cc0 tx=0x7f007400e780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:01.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f007b400640 1 -- 172.21.15.66:0/558964659 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074004050 con 0x7f0068003fe0 2024-09-10T22:12:01.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f007b400640 1 -- 172.21.15.66:0/558964659 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00740041f0 con 0x7f0068003fe0 2024-09-10T22:12:01.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f007b400640 1 -- 172.21.15.66:0/558964659 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074003600 con 0x7f0068003fe0 2024-09-10T22:12:01.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f008a000640 1 -- 172.21.15.66:0/558964659 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00841f6b40 con 0x7f0068003fe0 2024-09-10T22:12:01.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.780+0000 7f008a000640 1 -- 172.21.15.66:0/558964659 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0084159530 con 0x7f0068003fe0 2024-09-10T22:12:01.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.784+0000 7f008a000640 1 -- 172.21.15.66:0/558964659 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00840cfb50 con 0x7f0068003fe0 2024-09-10T22:12:01.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.784+0000 7f007b400640 1 -- 172.21.15.66:0/558964659 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f00740037a0 con 0x7f0068003fe0 2024-09-10T22:12:01.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.784+0000 7f007b400640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0064073f60 0x7f0064076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:01.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.784+0000 7f007b400640 1 -- 172.21.15.66:0/558964659 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f00740971d0 con 0x7f0068003fe0 2024-09-10T22:12:01.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.784+0000 7f0082a00640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0064073f60 0x7f0064076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:01.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.788+0000 7f0082a00640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0064073f60 0x7f0064076420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f00700b0000 tx=0x7f00700a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:01.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.788+0000 7f007b400640 1 -- 172.21.15.66:0/558964659 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0074064390 con 0x7f0068003fe0 2024-09-10T22:12:01.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.964+0000 7f008a000640 1 -- 172.21.15.66:0/558964659 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f008415abb0 con 0x7f0068003fe0 2024-09-10T22:12:01.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.964+0000 7f007b400640 1 -- 172.21.15.66:0/558964659 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f0074063c90 con 0x7f0068003fe0 2024-09-10T22:12:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.968+0000 7f008a000640 1 -- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0064073f60 msgr2=0x7f0064076420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.968+0000 7f008a000640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0064073f60 0x7f0064076420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f00700b0000 tx=0x7f00700a7040 comp rx=0 tx=0).stop 2024-09-10T22:12:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.968+0000 7f008a000640 1 -- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0068003fe0 msgr2=0x7f0084158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.968+0000 7f008a000640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0068003fe0 0x7f0084158ff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0074009cc0 tx=0x7f007400e780 comp rx=0 tx=0).stop 2024-09-10T22:12:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.968+0000 7f008a000640 1 -- 172.21.15.66:0/558964659 shutdown_connections 2024-09-10T22:12:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.968+0000 7f008a000640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0064073f60 0x7f0064076420 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.968+0000 7f008a000640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00841f5ec0 0x7f00840ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.968+0000 7f008a000640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008415a590 0x7f0084165040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.968+0000 7f008a000640 1 --2- 172.21.15.66:0/558964659 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0068003fe0 0x7f0084158ff0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.968+0000 7f008a000640 1 -- 172.21.15.66:0/558964659 >> 172.21.15.66:0/558964659 conn(0x7f00840d1c50 msgr2=0x7f0084071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.968+0000 7f008a000640 1 -- 172.21.15.66:0/558964659 shutdown_connections 2024-09-10T22:12:01.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:01.968+0000 7f008a000640 1 -- 172.21.15.66:0/558964659 wait complete. 2024-09-10T22:12:01.991 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-10T22:12:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.136+0000 7fe811c00640 1 -- 172.21.15.66:0/673437573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe80c0ae620 msgr2=0x7fe80c06e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.136+0000 7fe811c00640 1 --2- 172.21.15.66:0/673437573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe80c0ae620 0x7fe80c06e960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009870 tx=0x7fe7f401c780 comp rx=0 tx=0).stop 2024-09-10T22:12:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.136+0000 7fe811c00640 1 -- 172.21.15.66:0/673437573 shutdown_connections 2024-09-10T22:12:02.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.136+0000 7fe811c00640 1 --2- 172.21.15.66:0/673437573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe80c0ae620 0x7fe80c06e960 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.136+0000 7fe811c00640 1 -- 172.21.15.66:0/673437573 >> 172.21.15.66:0/673437573 conn(0x7fe80c0cfa30 msgr2=0x7fe80c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:02.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.136+0000 7fe811c00640 1 -- 172.21.15.66:0/673437573 shutdown_connections 2024-09-10T22:12:02.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.136+0000 7fe811c00640 1 -- 172.21.15.66:0/673437573 wait complete. 2024-09-10T22:12:02.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe811c00640 1 Processor -- start 2024-09-10T22:12:02.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe811c00640 1 -- start start 2024-09-10T22:12:02.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe811c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe80c0ae620 0x7fe80c0ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:02.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe811c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe80c0c4b10 0x7fe80c1fa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:02.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe811c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe80c0c4ef0 0x7fe80c147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:02.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe811c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c169610 con 0x7fe80c0c4b10 2024-09-10T22:12:02.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe811c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c169490 con 0x7fe80c0ae620 2024-09-10T22:12:02.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe811c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c169790 con 0x7fe80c0c4ef0 2024-09-10T22:12:02.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe803e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe80c0c4b10 0x7fe80c1fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:02.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe803e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe80c0c4b10 0x7fe80c1fa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33304/0 (socket says 172.21.15.66:33304) 2024-09-10T22:12:02.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe803e00640 1 -- 172.21.15.66:0/1222564356 learned_addr learned my addr 172.21.15.66:0/1222564356 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:02.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe80b400640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe80c0c4ef0 0x7fe80c147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:02.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe80aa00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe80c0ae620 0x7fe80c0ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:02.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe803e00640 1 -- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe80c0c4ef0 msgr2=0x7fe80c147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe803e00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe80c0c4ef0 0x7fe80c147bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe803e00640 1 -- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe80c0ae620 msgr2=0x7fe80c0ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe803e00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe80c0ae620 0x7fe80c0ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe803e00640 1 -- 172.21.15.66:0/1222564356 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c14d670 con 0x7fe80c0c4b10 2024-09-10T22:12:02.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe80aa00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe80c0ae620 0x7fe80c0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:02.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe803e00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe80c0c4b10 0x7fe80c1fa150 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe7f401a3b0 tx=0x7fe7f40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:02.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe802a00640 1 -- 172.21.15.66:0/1222564356 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4004960 con 0x7fe80c0c4b10 2024-09-10T22:12:02.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe802a00640 1 -- 172.21.15.66:0/1222564356 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7f4004b00 con 0x7fe80c0c4b10 2024-09-10T22:12:02.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe802a00640 1 -- 172.21.15.66:0/1222564356 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40a86d0 con 0x7fe80c0c4b10 2024-09-10T22:12:02.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe811c00640 1 -- 172.21.15.66:0/1222564356 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c0cda60 con 0x7fe80c0c4b10 2024-09-10T22:12:02.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe811c00640 1 -- 172.21.15.66:0/1222564356 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c14d390 con 0x7fe80c0c4b10 2024-09-10T22:12:02.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.140+0000 7fe811c00640 1 -- 172.21.15.66:0/1222564356 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80c06e960 con 0x7fe80c0c4b10 2024-09-10T22:12:02.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.144+0000 7fe802a00640 1 -- 172.21.15.66:0/1222564356 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fe7f40c3030 con 0x7fe80c0c4b10 2024-09-10T22:12:02.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.144+0000 7fe802a00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7e8073f30 0x7fe7e80763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:02.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.144+0000 7fe802a00640 1 -- 172.21.15.66:0/1222564356 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fe7f412c2a0 con 0x7fe80c0c4b10 2024-09-10T22:12:02.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.144+0000 7fe80aa00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7e8073f30 0x7fe7e80763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:02.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.144+0000 7fe80aa00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7e8073f30 0x7fe7e80763f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc001230 tx=0x7fe7fc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:02.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.148+0000 7fe802a00640 1 -- 172.21.15.66:0/1222564356 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe7f40ac030 con 0x7fe80c0c4b10 2024-09-10T22:12:02.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 -- 172.21.15.66:0/1222564356 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fe80c15a960 con 0x7fe80c0c4b10 2024-09-10T22:12:02.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe802a00640 1 -- 172.21.15.66:0/1222564356 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v48) ==== 116+0+2 (secure 0 0 0) 0x7fe7f4016370 con 0x7fe80c0c4b10 2024-09-10T22:12:02.331 INFO:teuthology.orchestra.run.smithi066.stdout:4 2024-09-10T22:12:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 -- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7e8073f30 msgr2=0x7fe7e80763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7e8073f30 0x7fe7e80763f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc001230 tx=0x7fe7fc009040 comp rx=0 tx=0).stop 2024-09-10T22:12:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 -- 172.21.15.66:0/1222564356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe80c0c4b10 msgr2=0x7fe80c1fa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe80c0c4b10 0x7fe80c1fa150 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe7f401a3b0 tx=0x7fe7f40056d0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 -- 172.21.15.66:0/1222564356 shutdown_connections 2024-09-10T22:12:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe7e8073f30 0x7fe7e80763f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe80c0c4ef0 0x7fe80c147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe80c0c4b10 0x7fe80c1fa150 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 --2- 172.21.15.66:0/1222564356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe80c0ae620 0x7fe80c0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 -- 172.21.15.66:0/1222564356 >> 172.21.15.66:0/1222564356 conn(0x7fe80c0cfa30 msgr2=0x7fe80c1539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:02.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 -- 172.21.15.66:0/1222564356 shutdown_connections 2024-09-10T22:12:02.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.328+0000 7fe811c00640 1 -- 172.21.15.66:0/1222564356 wait complete. 2024-09-10T22:12:02.354 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:12:02.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.500+0000 7ff2b9400640 1 -- 172.21.15.66:0/3143204593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b415f300 msgr2=0x7ff2b40b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.500+0000 7ff2b9400640 1 --2- 172.21.15.66:0/3143204593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b415f300 0x7ff2b40b1090 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009870 tx=0x7ff29c01c820 comp rx=0 tx=0).stop 2024-09-10T22:12:02.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.500+0000 7ff2b9400640 1 -- 172.21.15.66:0/3143204593 shutdown_connections 2024-09-10T22:12:02.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.500+0000 7ff2b9400640 1 --2- 172.21.15.66:0/3143204593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b415f300 0x7ff2b40b1090 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.500+0000 7ff2b9400640 1 --2- 172.21.15.66:0/3143204593 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff2b40b15d0 0x7ff2b40cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.500+0000 7ff2b9400640 1 -- 172.21.15.66:0/3143204593 >> 172.21.15.66:0/3143204593 conn(0x7ff2b40d1b10 msgr2=0x7ff2b40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:02.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.500+0000 7ff2b9400640 1 -- 172.21.15.66:0/3143204593 shutdown_connections 2024-09-10T22:12:02.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.500+0000 7ff2b9400640 1 -- 172.21.15.66:0/3143204593 wait complete. 2024-09-10T22:12:02.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.500+0000 7ff2b9400640 1 Processor -- start 2024-09-10T22:12:02.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.500+0000 7ff2b9400640 1 -- start start 2024-09-10T22:12:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.500+0000 7ff2b9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b40b15d0 0x7ff2b409f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.500+0000 7ff2b9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2b415ef20 0x7ff2b40ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b9400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff2b415f300 0x7ff2b415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b9400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4169310 con 0x7ff2b40b15d0 2024-09-10T22:12:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b9400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4169190 con 0x7ff2b415ef20 2024-09-10T22:12:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b9400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b4169490 con 0x7ff2b415f300 2024-09-10T22:12:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b40b15d0 0x7ff2b409f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b40b15d0 0x7ff2b409f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36320/0 (socket says 172.21.15.66:36320) 2024-09-10T22:12:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b3e00640 1 -- 172.21.15.66:0/3618598755 learned_addr learned my addr 172.21.15.66:0/3618598755 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b8a00640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff2b415f300 0x7ff2b415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2abe00640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2b415ef20 0x7ff2b40ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b3e00640 1 -- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff2b415f300 msgr2=0x7ff2b415dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b3e00640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff2b415f300 0x7ff2b415dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b3e00640 1 -- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2b415ef20 msgr2=0x7ff2b40ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b3e00640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2b415ef20 0x7ff2b40ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b3e00640 1 -- 172.21.15.66:0/3618598755 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b40b63a0 con 0x7ff2b40b15d0 2024-09-10T22:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2abe00640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2b415ef20 0x7ff2b40ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b3e00640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b40b15d0 0x7ff2b409f360 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400c4c0 tx=0x7ff2a400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b0c00640 1 -- 172.21.15.66:0/3618598755 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4016020 con 0x7ff2b40b15d0 2024-09-10T22:12:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b0c00640 1 -- 172.21.15.66:0/3618598755 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2a4012940 con 0x7ff2b40b15d0 2024-09-10T22:12:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b0c00640 1 -- 172.21.15.66:0/3618598755 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4012d90 con 0x7ff2b40b15d0 2024-09-10T22:12:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b9400640 1 -- 172.21.15.66:0/3618598755 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b40b6680 con 0x7ff2b40b15d0 2024-09-10T22:12:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b9400640 1 -- 172.21.15.66:0/3618598755 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b40b6c40 con 0x7ff2b40b15d0 2024-09-10T22:12:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b9400640 1 -- 172.21.15.66:0/3618598755 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b40cd7a0 con 0x7ff2b40b15d0 2024-09-10T22:12:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.504+0000 7ff2b0c00640 1 -- 172.21.15.66:0/3618598755 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7ff2a401b400 con 0x7ff2b40b15d0 2024-09-10T22:12:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.508+0000 7ff2b0c00640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff290073f30 0x7ff2900763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.508+0000 7ff2b0c00640 1 -- 172.21.15.66:0/3618598755 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7ff2a40922f0 con 0x7ff2b40b15d0 2024-09-10T22:12:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.508+0000 7ff2abe00640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff290073f30 0x7ff2900763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.508+0000 7ff2abe00640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff290073f30 0x7ff2900763f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0af000 tx=0x7ff29c01b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.512+0000 7ff2b0c00640 1 -- 172.21.15.66:0/3618598755 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff2a4020020 con 0x7ff2b40b15d0 2024-09-10T22:12:02.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.688+0000 7ff2b9400640 1 -- 172.21.15.66:0/3618598755 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7ff2b4072e00 con 0x7ff2b40b15d0 2024-09-10T22:12:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b0c00640 1 -- 172.21.15.66:0/3618598755 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v49)=0 v49) ==== 131+0+0 (secure 0 0 0) 0x7ff2a405f4f0 con 0x7ff2b40b15d0 2024-09-10T22:12:02.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b9400640 1 -- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff290073f30 msgr2=0x7ff2900763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b9400640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff290073f30 0x7ff2900763f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0af000 tx=0x7ff29c01b060 comp rx=0 tx=0).stop 2024-09-10T22:12:02.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b9400640 1 -- 172.21.15.66:0/3618598755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b40b15d0 msgr2=0x7ff2b409f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b9400640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b40b15d0 0x7ff2b409f360 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400c4c0 tx=0x7ff2a400c980 comp rx=0 tx=0).stop 2024-09-10T22:12:02.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b9400640 1 -- 172.21.15.66:0/3618598755 shutdown_connections 2024-09-10T22:12:02.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b9400640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff290073f30 0x7ff2900763f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b9400640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff2b415f300 0x7ff2b415dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b9400640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2b415ef20 0x7ff2b40ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b9400640 1 --2- 172.21.15.66:0/3618598755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b40b15d0 0x7ff2b409f360 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b9400640 1 -- 172.21.15.66:0/3618598755 >> 172.21.15.66:0/3618598755 conn(0x7ff2b40d1b10 msgr2=0x7ff2b4166260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:02.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b9400640 1 -- 172.21.15.66:0/3618598755 shutdown_connections 2024-09-10T22:12:02.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.692+0000 7ff2b9400640 1 -- 172.21.15.66:0/3618598755 wait complete. 2024-09-10T22:12:02.715 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:02.717+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:02.715 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:02.717+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:02.715 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:02.717+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:02.715 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:02.717+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:02.716 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:02.717+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:02.718 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-10T22:12:02.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.864+0000 7fb3cd200640 1 -- 172.21.15.66:0/3688900393 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c815ddd0 msgr2=0x7fb3c815e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.864+0000 7fb3cd200640 1 --2- 172.21.15.66:0/3688900393 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c815ddd0 0x7fb3c815e1f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009900 tx=0x7fb3b001f0d0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.864+0000 7fb3cd200640 1 -- 172.21.15.66:0/3688900393 shutdown_connections 2024-09-10T22:12:02.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.864+0000 7fb3cd200640 1 --2- 172.21.15.66:0/3688900393 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c815ddd0 0x7fb3c815e1f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.864+0000 7fb3cd200640 1 --2- 172.21.15.66:0/3688900393 >> v2:172.21.15.94:3300/0 conn(0x7fb3c80cee30 0x7fb3c80cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.864+0000 7fb3cd200640 1 -- 172.21.15.66:0/3688900393 >> 172.21.15.66:0/3688900393 conn(0x7fb3c80cfaa0 msgr2=0x7fb3c80cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:02.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.864+0000 7fb3cd200640 1 -- 172.21.15.66:0/3688900393 shutdown_connections 2024-09-10T22:12:02.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.864+0000 7fb3cd200640 1 -- 172.21.15.66:0/3688900393 wait complete. 2024-09-10T22:12:02.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.864+0000 7fb3cd200640 1 Processor -- start 2024-09-10T22:12:02.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3cd200640 1 -- start start 2024-09-10T22:12:02.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3cd200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c806ef30 0x7fb3c81493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3cd200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c80cee30 0x7fb3c814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3cd200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c815ddd0 0x7fb3c80aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3cd200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c815c740 con 0x7fb3c806ef30 2024-09-10T22:12:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3cd200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c815c8c0 con 0x7fb3c815ddd0 2024-09-10T22:12:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3cd200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c806fad0 con 0x7fb3c80cee30 2024-09-10T22:12:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3c5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c806ef30 0x7fb3c81493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3c5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c806ef30 0x7fb3c81493b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36336/0 (socket says 172.21.15.66:36336) 2024-09-10T22:12:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3c5e00640 1 -- 172.21.15.66:0/159060877 learned_addr learned my addr 172.21.15.66:0/159060877 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3bfe00640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c80cee30 0x7fb3c814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3c6800640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c815ddd0 0x7fb3c80aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3c5e00640 1 -- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c80cee30 msgr2=0x7fb3c814e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3c5e00640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c80cee30 0x7fb3c814e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3c5e00640 1 -- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c815ddd0 msgr2=0x7fb3c80aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:02.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3c5e00640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c815ddd0 0x7fb3c80aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:02.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3c5e00640 1 -- 172.21.15.66:0/159060877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c80ce610 con 0x7fb3c806ef30 2024-09-10T22:12:02.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3c5e00640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c806ef30 0x7fb3c81493b0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800ea20 tx=0x7fb3b800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:02.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3be000640 1 -- 172.21.15.66:0/159060877 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800cd80 con 0x7fb3c806ef30 2024-09-10T22:12:02.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3be000640 1 -- 172.21.15.66:0/159060877 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3b8010430 con 0x7fb3c806ef30 2024-09-10T22:12:02.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3cd200640 1 -- 172.21.15.66:0/159060877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c80ce090 con 0x7fb3c806ef30 2024-09-10T22:12:02.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3be000640 1 -- 172.21.15.66:0/159060877 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8010860 con 0x7fb3c806ef30 2024-09-10T22:12:02.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3cd200640 1 -- 172.21.15.66:0/159060877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c80ce330 con 0x7fb3c806ef30 2024-09-10T22:12:02.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.868+0000 7fb3cd200640 1 -- 172.21.15.66:0/159060877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c80cf210 con 0x7fb3c806ef30 2024-09-10T22:12:02.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.872+0000 7fb3be000640 1 -- 172.21.15.66:0/159060877 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fb3b8010a00 con 0x7fb3c806ef30 2024-09-10T22:12:02.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.872+0000 7fb3be000640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3a4073f30 0x7fb3a40763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:02.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.872+0000 7fb3be000640 1 -- 172.21.15.66:0/159060877 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fb3b8092be0 con 0x7fb3c806ef30 2024-09-10T22:12:02.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.872+0000 7fb3bfe00640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3a4073f30 0x7fb3a40763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:02.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.872+0000 7fb3bfe00640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3a4073f30 0x7fb3a40763f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00b2000 tx=0x7fb3b00aa040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:02.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:02.876+0000 7fb3be000640 1 -- 172.21.15.66:0/159060877 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb3b805f770 con 0x7fb3c806ef30 2024-09-10T22:12:03.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.076+0000 7fb3cd200640 1 -- 172.21.15.66:0/159060877 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fb3c80d07c0 con 0x7fb3a4073f30 2024-09-10T22:12:03.090 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:03.089+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:03.090 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:03.089+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:03.090 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:03.089+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:03.090 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:03.089+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:03.090 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:03.089+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:03.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3be000640 1 -- 172.21.15.66:0/159060877 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb3c80d07c0 con 0x7fb3a4073f30 2024-09-10T22:12:03.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3cd200640 1 -- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3a4073f30 msgr2=0x7fb3a40763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:03.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3cd200640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3a4073f30 0x7fb3a40763f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00b2000 tx=0x7fb3b00aa040 comp rx=0 tx=0).stop 2024-09-10T22:12:03.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3cd200640 1 -- 172.21.15.66:0/159060877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c806ef30 msgr2=0x7fb3c81493b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:03.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3cd200640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c806ef30 0x7fb3c81493b0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800ea20 tx=0x7fb3b800eee0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3cd200640 1 -- 172.21.15.66:0/159060877 shutdown_connections 2024-09-10T22:12:03.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3cd200640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3a4073f30 0x7fb3a40763f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3cd200640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c815ddd0 0x7fb3c80aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3cd200640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3c80cee30 0x7fb3c814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3cd200640 1 --2- 172.21.15.66:0/159060877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3c806ef30 0x7fb3c81493b0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3cd200640 1 -- 172.21.15.66:0/159060877 >> 172.21.15.66:0/159060877 conn(0x7fb3c80cfaa0 msgr2=0x7fb3c814d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:03.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3cd200640 1 -- 172.21.15.66:0/159060877 shutdown_connections 2024-09-10T22:12:03.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.124+0000 7fb3cd200640 1 -- 172.21.15.66:0/159060877 wait complete. 2024-09-10T22:12:03.149 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:12:03.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.296+0000 7f91ba800640 1 -- 172.21.15.66:0/1773119854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b40d0550 msgr2=0x7f91b40c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.296+0000 7f91ba800640 1 --2- 172.21.15.66:0/1773119854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b40d0550 0x7f91b40c4df0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f919c00aa10 tx=0x7f919c01dba0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.296+0000 7f91ba800640 1 -- 172.21.15.66:0/1773119854 shutdown_connections 2024-09-10T22:12:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.296+0000 7f91ba800640 1 --2- 172.21.15.66:0/1773119854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b40d0550 0x7f91b40c4df0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.296+0000 7f91ba800640 1 --2- 172.21.15.66:0/1773119854 >> v2:172.21.15.94:3300/0 conn(0x7f91b415a310 0x7f91b40c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.296+0000 7f91ba800640 1 -- 172.21.15.66:0/1773119854 >> 172.21.15.66:0/1773119854 conn(0x7f91b40cfa30 msgr2=0x7f91b40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.296+0000 7f91ba800640 1 -- 172.21.15.66:0/1773119854 shutdown_connections 2024-09-10T22:12:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.296+0000 7f91ba800640 1 -- 172.21.15.66:0/1773119854 wait complete. 2024-09-10T22:12:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91ba800640 1 Processor -- start 2024-09-10T22:12:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91ba800640 1 -- start start 2024-09-10T22:12:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91ba800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b40aeec0 0x7f91b4074330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91ba800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b415a310 0x7f91b414ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91ba800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b414f6c0 0x7f91b40b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91ba800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f91b415ed60 con 0x7f91b40aeec0 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91ba800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91b415ebe0 con 0x7f91b415a310 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91ba800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f91b4167300 con 0x7f91b414f6c0 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91b3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b40aeec0 0x7f91b4074330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91b3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b40aeec0 0x7f91b4074330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36348/0 (socket says 172.21.15.66:36348) 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91b3400640 1 -- 172.21.15.66:0/607400773 learned_addr learned my addr 172.21.15.66:0/607400773 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91aaa00640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b415a310 0x7f91b414ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91b3e00640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b414f6c0 0x7f91b40b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91b3400640 1 -- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b414f6c0 msgr2=0x7f91b40b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91b3400640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b414f6c0 0x7f91b40b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91b3400640 1 -- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b415a310 msgr2=0x7f91b414ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91b3400640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b415a310 0x7f91b414ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91b3400640 1 -- 172.21.15.66:0/607400773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b4074870 con 0x7f91b40aeec0 2024-09-10T22:12:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91b3400640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b40aeec0 0x7f91b4074330 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f91a400e970 tx=0x7f91a400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91abe00640 1 -- 172.21.15.66:0/607400773 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a400cd80 con 0x7f91b40aeec0 2024-09-10T22:12:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91abe00640 1 -- 172.21.15.66:0/607400773 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91a4004020 con 0x7f91b40aeec0 2024-09-10T22:12:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91abe00640 1 -- 172.21.15.66:0/607400773 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4004470 con 0x7f91b40aeec0 2024-09-10T22:12:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91ba800640 1 -- 172.21.15.66:0/607400773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b414f370 con 0x7f91b40aeec0 2024-09-10T22:12:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.300+0000 7f91ba800640 1 -- 172.21.15.66:0/607400773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b415c0f0 con 0x7f91b40aeec0 2024-09-10T22:12:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.304+0000 7f91ba800640 1 -- 172.21.15.66:0/607400773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b415df60 con 0x7f91b40aeec0 2024-09-10T22:12:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.304+0000 7f91abe00640 1 -- 172.21.15.66:0/607400773 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7f91a4003600 con 0x7f91b40aeec0 2024-09-10T22:12:03.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.304+0000 7f91abe00640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9190073f30 0x7f91900763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:03.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.304+0000 7f91abe00640 1 -- 172.21.15.66:0/607400773 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7f91a4092a90 con 0x7f91b40aeec0 2024-09-10T22:12:03.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.304+0000 7f91aaa00640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9190073f30 0x7f91900763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:03.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.304+0000 7f91aaa00640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9190073f30 0x7f91900763f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f919c0b0000 tx=0x7f919c0a8040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:03.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.308+0000 7f91abe00640 1 -- 172.21.15.66:0/607400773 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f91b414f370 con 0x7f91b40aeec0 2024-09-10T22:12:03.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.512+0000 7f91ba800640 1 -- 172.21.15.66:0/607400773 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f91b406c7c0 con 0x7f9190073f30 2024-09-10T22:12:03.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91abe00640 1 -- 172.21.15.66:0/607400773 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f91b406c7c0 con 0x7f9190073f30 2024-09-10T22:12:03.523 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599 2024-09-10T22:12:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91ba800640 1 -- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9190073f30 msgr2=0x7f91900763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91ba800640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9190073f30 0x7f91900763f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f919c0b0000 tx=0x7f919c0a8040 comp rx=0 tx=0).stop 2024-09-10T22:12:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91ba800640 1 -- 172.21.15.66:0/607400773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b40aeec0 msgr2=0x7f91b4074330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91ba800640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b40aeec0 0x7f91b4074330 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f91a400e970 tx=0x7f91a400ee30 comp rx=0 tx=0).stop 2024-09-10T22:12:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91ba800640 1 -- 172.21.15.66:0/607400773 shutdown_connections 2024-09-10T22:12:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91ba800640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9190073f30 0x7f91900763f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91ba800640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91b414f6c0 0x7f91b40b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91ba800640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91b415a310 0x7f91b414ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91ba800640 1 --2- 172.21.15.66:0/607400773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91b40aeec0 0x7f91b4074330 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91ba800640 1 -- 172.21.15.66:0/607400773 >> 172.21.15.66:0/607400773 conn(0x7f91b40cfa30 msgr2=0x7f91b40cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:03.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91ba800640 1 -- 172.21.15.66:0/607400773 shutdown_connections 2024-09-10T22:12:03.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.520+0000 7f91ba800640 1 -- 172.21.15.66:0/607400773 wait complete. 2024-09-10T22:12:03.545 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:12:03.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 -- 172.21.15.66:0/2705476492 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb608159900 msgr2=0x7fb6080ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:03.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 --2- 172.21.15.66:0/2705476492 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb608159900 0x7fb6080ca080 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009870 tx=0x7fb5fc01c7d0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 -- 172.21.15.66:0/2705476492 shutdown_connections 2024-09-10T22:12:03.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 --2- 172.21.15.66:0/2705476492 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb608159900 0x7fb6080ca080 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 -- 172.21.15.66:0/2705476492 >> 172.21.15.66:0/2705476492 conn(0x7fb6080cfa30 msgr2=0x7fb6080cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:03.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 -- 172.21.15.66:0/2705476492 shutdown_connections 2024-09-10T22:12:03.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 -- 172.21.15.66:0/2705476492 wait complete. 2024-09-10T22:12:03.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 Processor -- start 2024-09-10T22:12:03.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 -- start start 2024-09-10T22:12:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6080aee20 0x7fb608159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb608159900 0x7fb6080c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb60815cc80 0x7fb60814b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb60816a360 con 0x7fb608159900 2024-09-10T22:12:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb60816a1e0 con 0x7fb6080aee20 2024-09-10T22:12:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60f600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb60816a4e0 con 0x7fb60815cc80 2024-09-10T22:12:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb607e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb608159900 0x7fb6080c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb607e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb608159900 0x7fb6080c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36352/0 (socket says 172.21.15.66:36352) 2024-09-10T22:12:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb607e00640 1 -- 172.21.15.66:0/299597558 learned_addr learned my addr 172.21.15.66:0/299597558 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60cc00640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6080aee20 0x7fb608159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.692+0000 7fb60d600640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb60815cc80 0x7fb60814b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb607e00640 1 -- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb60815cc80 msgr2=0x7fb60814b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb607e00640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb60815cc80 0x7fb60814b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb607e00640 1 -- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6080aee20 msgr2=0x7fb608159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb607e00640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6080aee20 0x7fb608159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb607e00640 1 -- 172.21.15.66:0/299597558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6081511f0 con 0x7fb608159900 2024-09-10T22:12:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb60d600640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb60815cc80 0x7fb60814b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:12:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb607e00640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb608159900 0x7fb6080c6290 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400c400 tx=0x7fb5f400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb604c00640 1 -- 172.21.15.66:0/299597558 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4017070 con 0x7fb608159900 2024-09-10T22:12:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb604c00640 1 -- 172.21.15.66:0/299597558 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5f4005400 con 0x7fb608159900 2024-09-10T22:12:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb604c00640 1 -- 172.21.15.66:0/299597558 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4003860 con 0x7fb608159900 2024-09-10T22:12:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb60f600640 1 -- 172.21.15.66:0/299597558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb608151470 con 0x7fb608159900 2024-09-10T22:12:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb60f600640 1 -- 172.21.15.66:0/299597558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60814f690 con 0x7fb608159900 2024-09-10T22:12:03.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb60f600640 1 -- 172.21.15.66:0/299597558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6080ca080 con 0x7fb608159900 2024-09-10T22:12:03.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb604c00640 1 -- 172.21.15.66:0/299597558 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 37) ==== 98689+0+0 (secure 0 0 0) 0x7fb5f4003a00 con 0x7fb608159900 2024-09-10T22:12:03.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb604c00640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb5e8073f30 0x7fb5e80763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:03.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.696+0000 7fb604c00640 1 -- 172.21.15.66:0/299597558 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5793+0+0 (secure 0 0 0) 0x7fb5f4091700 con 0x7fb608159900 2024-09-10T22:12:03.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.700+0000 7fb60cc00640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb5e8073f30 0x7fb5e80763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:03.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.700+0000 7fb60cc00640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb5e8073f30 0x7fb5e80763f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0af000 tx=0x7fb5fc0b2040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.700+0000 7fb604c00640 1 -- 172.21.15.66:0/299597558 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb6081511f0 con 0x7fb608159900 2024-09-10T22:12:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.904+0000 7fb60f600640 1 -- 172.21.15.66:0/299597558 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fb608070470 con 0x7fb5e8073f30 2024-09-10T22:12:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.908+0000 7fb604c00640 1 -- 172.21.15.66:0/299597558 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fb608070470 con 0x7fb5e8073f30 2024-09-10T22:12:03.912 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599 2024-09-10T22:12:03.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.912+0000 7fb60f600640 1 -- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb5e8073f30 msgr2=0x7fb5e80763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:03.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.912+0000 7fb60f600640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb5e8073f30 0x7fb5e80763f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0af000 tx=0x7fb5fc0b2040 comp rx=0 tx=0).stop 2024-09-10T22:12:03.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.912+0000 7fb60f600640 1 -- 172.21.15.66:0/299597558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb608159900 msgr2=0x7fb6080c6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:03.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.912+0000 7fb60f600640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb608159900 0x7fb6080c6290 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400c400 tx=0x7fb5f400c8c0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.912+0000 7fb60f600640 1 -- 172.21.15.66:0/299597558 shutdown_connections 2024-09-10T22:12:03.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.912+0000 7fb60f600640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb5e8073f30 0x7fb5e80763f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.912+0000 7fb60f600640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb60815cc80 0x7fb60814b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.912+0000 7fb60f600640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb608159900 0x7fb6080c6290 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.912+0000 7fb60f600640 1 --2- 172.21.15.66:0/299597558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6080aee20 0x7fb608159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.912+0000 7fb60f600640 1 -- 172.21.15.66:0/299597558 >> 172.21.15.66:0/299597558 conn(0x7fb6080cfa30 msgr2=0x7fb6080af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.912+0000 7fb60f600640 1 -- 172.21.15.66:0/299597558 shutdown_connections 2024-09-10T22:12:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:03.912+0000 7fb60f600640 1 -- 172.21.15.66:0/299597558 wait complete. 2024-09-10T22:12:03.934 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599 2024-09-10T22:12:03.934 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:12:03.934 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:12:03.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:12:03.943 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:12:11.054 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:12:11.054 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:12:11.054 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.04232 s, 152 MB/s 2024-09-10T22:12:11.055 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:12:11.055 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:12:11.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:12:11.064 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:12:18.019 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:12:18.020 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:12:18.020 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.89759 s, 156 MB/s 2024-09-10T22:12:18.021 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:12:18.021 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:12:18.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:12:18.030 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:12:24.934 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:12:24.934 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:12:24.935 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.84773 s, 157 MB/s 2024-09-10T22:12:24.936 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-10T22:12:25.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.083+0000 7f5738400640 1 -- 172.21.15.66:0/3667010033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730072b80 msgr2=0x7f573006e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:25.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.083+0000 7f5738400640 1 --2- 172.21.15.66:0/3667010033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730072b80 0x7f573006e8e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5728009c80 tx=0x7f572801cda0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.083+0000 7f5738400640 1 -- 172.21.15.66:0/3667010033 shutdown_connections 2024-09-10T22:12:25.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.083+0000 7f5738400640 1 --2- 172.21.15.66:0/3667010033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730072b80 0x7f573006e8e0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.083+0000 7f5738400640 1 --2- 172.21.15.66:0/3667010033 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f573015a3b0 0x7f5730072640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.083+0000 7f5738400640 1 -- 172.21.15.66:0/3667010033 >> 172.21.15.66:0/3667010033 conn(0x7f57300cfaa0 msgr2=0x7f57300cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:25.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.083+0000 7f5738400640 1 -- 172.21.15.66:0/3667010033 shutdown_connections 2024-09-10T22:12:25.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.083+0000 7f5738400640 1 -- 172.21.15.66:0/3667010033 wait complete. 2024-09-10T22:12:25.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.083+0000 7f5738400640 1 Processor -- start 2024-09-10T22:12:25.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.083+0000 7f5738400640 1 -- start start 2024-09-10T22:12:25.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5738400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5730072b80 0x7f5730145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:25.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5738400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5730150dc0 0x7f573013ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:25.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5738400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f573015a3b0 0x7f573013f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:25.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5738400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5730167090 con 0x7f573015a3b0 2024-09-10T22:12:25.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5738400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5730166f10 con 0x7f5730072b80 2024-09-10T22:12:25.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5738400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5730167210 con 0x7f5730150dc0 2024-09-10T22:12:25.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5736400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f573015a3b0 0x7f573013f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:25.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5736400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f573015a3b0 0x7f573013f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39844/0 (socket says 172.21.15.66:39844) 2024-09-10T22:12:25.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5736400640 1 -- 172.21.15.66:0/2503377277 learned_addr learned my addr 172.21.15.66:0/2503377277 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:25.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5735000640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5730150dc0 0x7f573013ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:25.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5735a00640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5730072b80 0x7f5730145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:25.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5736400640 1 -- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5730150dc0 msgr2=0x7f573013ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:25.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5736400640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5730150dc0 0x7f573013ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5736400640 1 -- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5730072b80 msgr2=0x7f5730145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:25.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5736400640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5730072b80 0x7f5730145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5736400640 1 -- 172.21.15.66:0/2503377277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5730145f60 con 0x7f573015a3b0 2024-09-10T22:12:25.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5736400640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f573015a3b0 0x7f573013f170 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5728009bd0 tx=0x7f5728003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:25.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f572d600640 1 -- 172.21.15.66:0/2503377277 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57280bb070 con 0x7f573015a3b0 2024-09-10T22:12:25.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f572d600640 1 -- 172.21.15.66:0/2503377277 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5728004c50 con 0x7f573015a3b0 2024-09-10T22:12:25.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f572d600640 1 -- 172.21.15.66:0/2503377277 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57280b2040 con 0x7f573015a3b0 2024-09-10T22:12:25.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5738400640 1 -- 172.21.15.66:0/2503377277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5730146240 con 0x7f573015a3b0 2024-09-10T22:12:25.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5738400640 1 -- 172.21.15.66:0/2503377277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5730146b30 con 0x7f573015a3b0 2024-09-10T22:12:25.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f5738400640 1 -- 172.21.15.66:0/2503377277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5730072640 con 0x7f573015a3b0 2024-09-10T22:12:25.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.087+0000 7f572d600640 1 -- 172.21.15.66:0/2503377277 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 38) ==== 98791+0+0 (secure 0 0 0) 0x7f57280b5030 con 0x7f573015a3b0 2024-09-10T22:12:25.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.091+0000 7f572d600640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5710074050 0x7f5710076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:25.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.091+0000 7f572d600640 1 -- 172.21.15.66:0/2503377277 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f572812bc40 con 0x7f573015a3b0 2024-09-10T22:12:25.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.091+0000 7f5735a00640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5710074050 0x7f5710076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:25.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.091+0000 7f5735a00640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5710074050 0x7f5710076510 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5724007fd0 tx=0x7f5724009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:25.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.095+0000 7f572d600640 1 -- 172.21.15.66:0/2503377277 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5728016990 con 0x7f573015a3b0 2024-09-10T22:12:25.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.295+0000 7f5738400640 1 -- 172.21.15.66:0/2503377277 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f57300b4a30 con 0x7f5710074050 2024-09-10T22:12:25.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f572d600640 1 -- 172.21.15.66:0/2503377277 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f57300b4a30 con 0x7f5710074050 2024-09-10T22:12:25.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f5738400640 1 -- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5710074050 msgr2=0x7f5710076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:25.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f5738400640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5710074050 0x7f5710076510 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5724007fd0 tx=0x7f5724009040 comp rx=0 tx=0).stop 2024-09-10T22:12:25.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f5738400640 1 -- 172.21.15.66:0/2503377277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f573015a3b0 msgr2=0x7f573013f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:25.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f5738400640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f573015a3b0 0x7f573013f170 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5728009bd0 tx=0x7f5728003f40 comp rx=0 tx=0).stop 2024-09-10T22:12:25.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f5738400640 1 -- 172.21.15.66:0/2503377277 shutdown_connections 2024-09-10T22:12:25.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f5738400640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5710074050 0x7f5710076510 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f5738400640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f573015a3b0 0x7f573013f170 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f5738400640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5730150dc0 0x7f573013ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f5738400640 1 --2- 172.21.15.66:0/2503377277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5730072b80 0x7f5730145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f5738400640 1 -- 172.21.15.66:0/2503377277 >> 172.21.15.66:0/2503377277 conn(0x7f57300cfaa0 msgr2=0x7f573014fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:25.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f5738400640 1 -- 172.21.15.66:0/2503377277 shutdown_connections 2024-09-10T22:12:25.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.311+0000 7f5738400640 1 -- 172.21.15.66:0/2503377277 wait complete. 2024-09-10T22:12:25.338 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:12:25.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828878124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa815f300 msgr2=0x7f8fa80b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:25.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 --2- 172.21.15.66:0/1828878124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa815f300 0x7f8fa80b1090 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009930 tx=0x7f8f9001c8e0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828878124 shutdown_connections 2024-09-10T22:12:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 --2- 172.21.15.66:0/1828878124 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fa80b15d0 0x7f8fa80cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 --2- 172.21.15.66:0/1828878124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa815f300 0x7f8fa80b1090 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828878124 >> 172.21.15.66:0/1828878124 conn(0x7f8fa80d1b10 msgr2=0x7f8fa80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828878124 shutdown_connections 2024-09-10T22:12:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828878124 wait complete. 2024-09-10T22:12:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 Processor -- start 2024-09-10T22:12:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 -- start start 2024-09-10T22:12:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa80b15d0 0x7f8fa814aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fa815ef20 0x7f8fa80b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa815f300 0x7f8fa815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa8169360 con 0x7f8fa815f300 2024-09-10T22:12:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa81691e0 con 0x7f8fa80b15d0 2024-09-10T22:12:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.483+0000 7f8fa7200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa81694e0 con 0x7f8fa815ef20 2024-09-10T22:12:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa815f300 0x7f8fa815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa80b15d0 0x7f8fa814aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa815f300 0x7f8fa815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39868/0 (socket says 172.21.15.66:39868) 2024-09-10T22:12:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa6800640 1 -- 172.21.15.66:0/1828003965 learned_addr learned my addr 172.21.15.66:0/1828003965 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8f9d400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fa815ef20 0x7f8fa80b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa6800640 1 -- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fa815ef20 msgr2=0x7f8fa80b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa6800640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fa815ef20 0x7f8fa80b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa6800640 1 -- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa80b15d0 msgr2=0x7f8fa814aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa6800640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa80b15d0 0x7f8fa814aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa6800640 1 -- 172.21.15.66:0/1828003965 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa80b49e0 con 0x7f8fa815f300 2024-09-10T22:12:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa5e00640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa80b15d0 0x7f8fa814aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:12:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa6800640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa815f300 0x7f8fa815d4a0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000eaf0 tx=0x7f8fa000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8f9ea00640 1 -- 172.21.15.66:0/1828003965 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0004b30 con 0x7f8fa815f300 2024-09-10T22:12:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8f9ea00640 1 -- 172.21.15.66:0/1828003965 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fa0004cf0 con 0x7f8fa815f300 2024-09-10T22:12:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828003965 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa80b4c80 con 0x7f8fa815f300 2024-09-10T22:12:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8f9ea00640 1 -- 172.21.15.66:0/1828003965 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0005280 con 0x7f8fa815f300 2024-09-10T22:12:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828003965 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa80b5520 con 0x7f8fa815f300 2024-09-10T22:12:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828003965 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa80cd7a0 con 0x7f8fa815f300 2024-09-10T22:12:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.487+0000 7f8f9ea00640 1 -- 172.21.15.66:0/1828003965 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 38) ==== 98791+0+0 (secure 0 0 0) 0x7f8fa0005420 con 0x7f8fa815f300 2024-09-10T22:12:25.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.491+0000 7f8f9ea00640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f84074050 0x7f8f84076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:25.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.491+0000 7f8f9ea00640 1 -- 172.21.15.66:0/1828003965 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f8fa0092cf0 con 0x7f8fa815f300 2024-09-10T22:12:25.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.491+0000 7f8fa5e00640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f84074050 0x7f8f84076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:25.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.491+0000 7f8fa5e00640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f84074050 0x7f8f84076510 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8fa815d3d0 tx=0x7f8f98009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:25.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.495+0000 7f8f9ea00640 1 -- 172.21.15.66:0/1828003965 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8fa005f7e0 con 0x7f8fa815f300 2024-09-10T22:12:25.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.695+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828003965 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f8fa80b4e50 con 0x7f8f84074050 2024-09-10T22:12:25.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.699+0000 7f8f9ea00640 1 -- 172.21.15.66:0/1828003965 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f8fa80b4e50 con 0x7f8f84074050 2024-09-10T22:12:25.705 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599 2024-09-10T22:12:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.703+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f84074050 msgr2=0x7f8f84076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.703+0000 7f8fa7200640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f84074050 0x7f8f84076510 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8fa815d3d0 tx=0x7f8f98009040 comp rx=0 tx=0).stop 2024-09-10T22:12:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.703+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828003965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa815f300 msgr2=0x7f8fa815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.703+0000 7f8fa7200640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa815f300 0x7f8fa815d4a0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000eaf0 tx=0x7f8fa000c2d0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.703+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828003965 shutdown_connections 2024-09-10T22:12:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.703+0000 7f8fa7200640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f84074050 0x7f8f84076510 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.703+0000 7f8fa7200640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa815f300 0x7f8fa815d4a0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.703+0000 7f8fa7200640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fa815ef20 0x7f8fa80b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.703+0000 7f8fa7200640 1 --2- 172.21.15.66:0/1828003965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa80b15d0 0x7f8fa814aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.703+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828003965 >> 172.21.15.66:0/1828003965 conn(0x7f8fa80d1b10 msgr2=0x7f8fa815e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.703+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828003965 shutdown_connections 2024-09-10T22:12:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:25.703+0000 7f8fa7200640 1 -- 172.21.15.66:0/1828003965 wait complete. 2024-09-10T22:12:25.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:12:25.726 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599 2024-09-10T22:12:25.726 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:12:25.740 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599 2024-09-10T22:12:25.740 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3138256896" 2024-09-10T22:12:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:25.741 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-09-10T22:12:27.696 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:12:27.700 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:12:27.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:12:27.742 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599 2024-09-10T22:12:27.742 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:12:27.756 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599 2024-09-10T22:12:27.757 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3138256896" 2024-09-10T22:12:27.757 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:27.758 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-09-10T22:12:29.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:12:29.758 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599 2024-09-10T22:12:29.758 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:12:29.773 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/153e2f09-00c3-44e1-9d76-ec67bbc64599 2024-09-10T22:12:29.773 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3221225472" 2024-09-10T22:12:29.773 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:29.774 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-10T22:12:29.774 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8660 2024-09-10T22:12:29.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 -- 172.21.15.66:0/1266828105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c300b15d0 msgr2=0x7f3c300cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:29.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 --2- 172.21.15.66:0/1266828105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c300b15d0 0x7f3c300cd7a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009c80 tx=0x7f3c2801cda0 comp rx=0 tx=0).stop 2024-09-10T22:12:29.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 -- 172.21.15.66:0/1266828105 shutdown_connections 2024-09-10T22:12:29.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 --2- 172.21.15.66:0/1266828105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c300b15d0 0x7f3c300cd7a0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:29.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 --2- 172.21.15.66:0/1266828105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c3015f300 0x7f3c300b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:29.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 -- 172.21.15.66:0/1266828105 >> 172.21.15.66:0/1266828105 conn(0x7f3c300d1b10 msgr2=0x7f3c300d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:29.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 -- 172.21.15.66:0/1266828105 shutdown_connections 2024-09-10T22:12:29.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 -- 172.21.15.66:0/1266828105 wait complete. 2024-09-10T22:12:29.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 Processor -- start 2024-09-10T22:12:29.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 -- start start 2024-09-10T22:12:29.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c300b15d0 0x7f3c3006f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c3015ef20 0x7f3c300b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3015f300 0x7f3c3015bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301693b0 con 0x7f3c3015f300 2024-09-10T22:12:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30169230 con 0x7f3c3015ef20 2024-09-10T22:12:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c35200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3c30169530 con 0x7f3c300b15d0 2024-09-10T22:12:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c2fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3015f300 0x7f3c3015bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c2fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3015f300 0x7f3c3015bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39886/0 (socket says 172.21.15.66:39886) 2024-09-10T22:12:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c2fe00640 1 -- 172.21.15.66:0/4103017599 learned_addr learned my addr 172.21.15.66:0/4103017599 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c27e00640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c3015ef20 0x7f3c300b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.979+0000 7f3c2f400640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c300b15d0 0x7f3c3006f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c2fe00640 1 -- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c300b15d0 msgr2=0x7f3c3006f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c2fe00640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c300b15d0 0x7f3c3006f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c2fe00640 1 -- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c3015ef20 msgr2=0x7f3c300b66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c2fe00640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c3015ef20 0x7f3c300b66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c2fe00640 1 -- 172.21.15.66:0/4103017599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c300b6bf0 con 0x7f3c3015f300 2024-09-10T22:12:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c2fe00640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3015f300 0x7f3c3015bbe0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3c2801b450 tx=0x7f3c28009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c27400640 1 -- 172.21.15.66:0/4103017599 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c280bb070 con 0x7f3c3015f300 2024-09-10T22:12:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c27400640 1 -- 172.21.15.66:0/4103017599 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c28004d10 con 0x7f3c3015f300 2024-09-10T22:12:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c27400640 1 -- 172.21.15.66:0/4103017599 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c280b2040 con 0x7f3c3015f300 2024-09-10T22:12:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c35200640 1 -- 172.21.15.66:0/4103017599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c30159110 con 0x7f3c3015f300 2024-09-10T22:12:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c35200640 1 -- 172.21.15.66:0/4103017599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c301596d0 con 0x7f3c3015f300 2024-09-10T22:12:29.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c35200640 1 -- 172.21.15.66:0/4103017599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c300cd7a0 con 0x7f3c3015f300 2024-09-10T22:12:29.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c27400640 1 -- 172.21.15.66:0/4103017599 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 38) ==== 98791+0+0 (secure 0 0 0) 0x7f3c280b5030 con 0x7f3c3015f300 2024-09-10T22:12:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c27400640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c0c074050 0x7f3c0c076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.983+0000 7f3c27400640 1 -- 172.21.15.66:0/4103017599 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f3c2812bc10 con 0x7f3c3015f300 2024-09-10T22:12:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.987+0000 7f3c2f400640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c0c074050 0x7f3c0c076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:29.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.987+0000 7f3c2f400640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c0c074050 0x7f3c0c076510 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3c20007fd0 tx=0x7f3c20009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:29.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:29.991+0000 7f3c27400640 1 -- 172.21.15.66:0/4103017599 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3c28016990 con 0x7f3c3015f300 2024-09-10T22:12:30.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.199+0000 7f3c35200640 1 -- 172.21.15.66:0/4103017599 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8660", "target": ["mon-mgr", ""]}) -- 0x7f3c30072e00 con 0x7f3c0c074050 2024-09-10T22:12:30.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.259+0000 7f3c27400640 1 -- 172.21.15.66:0/4103017599 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3c30072e00 con 0x7f3c0c074050 2024-09-10T22:12:30.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.263+0000 7f3c35200640 1 -- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c0c074050 msgr2=0x7f3c0c076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:30.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.263+0000 7f3c35200640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c0c074050 0x7f3c0c076510 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3c20007fd0 tx=0x7f3c20009040 comp rx=0 tx=0).stop 2024-09-10T22:12:30.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.263+0000 7f3c35200640 1 -- 172.21.15.66:0/4103017599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3015f300 msgr2=0x7f3c3015bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:30.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.263+0000 7f3c35200640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3015f300 0x7f3c3015bbe0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3c2801b450 tx=0x7f3c28009390 comp rx=0 tx=0).stop 2024-09-10T22:12:30.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.263+0000 7f3c35200640 1 -- 172.21.15.66:0/4103017599 shutdown_connections 2024-09-10T22:12:30.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.263+0000 7f3c35200640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3c0c074050 0x7f3c0c076510 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:30.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.263+0000 7f3c35200640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3015f300 0x7f3c3015bbe0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:30.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.263+0000 7f3c35200640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c3015ef20 0x7f3c300b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:30.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.263+0000 7f3c35200640 1 --2- 172.21.15.66:0/4103017599 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c300b15d0 0x7f3c3006f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:30.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.263+0000 7f3c35200640 1 -- 172.21.15.66:0/4103017599 >> 172.21.15.66:0/4103017599 conn(0x7f3c300d1b10 msgr2=0x7f3c300b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:30.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.263+0000 7f3c35200640 1 -- 172.21.15.66:0/4103017599 shutdown_connections 2024-09-10T22:12:30.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.263+0000 7f3c35200640 1 -- 172.21.15.66:0/4103017599 wait complete. 2024-09-10T22:12:30.271 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:30.273+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:30.271 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:30.273+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:30.271 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:30.273+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:30.271 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:30.273+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:30.271 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:30.273+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:30.287 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone9531 2024-09-10T22:12:30.287 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:30.289+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:30.288 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:30.289+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:30.288 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:30.289+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:30.288 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:30.289+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:30.288 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:30.289+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 -- 172.21.15.66:0/1759275876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8e40063a0 msgr2=0x7ff8e400cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 --2- 172.21.15.66:0/1759275876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8e40063a0 0x7ff8e400cae0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800c640 tx=0x7ff8f801d260 comp rx=0 tx=0).stop 2024-09-10T22:12:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 -- 172.21.15.66:0/1759275876 shutdown_connections 2024-09-10T22:12:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 --2- 172.21.15.66:0/1759275876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8e40063a0 0x7ff8e400cae0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 -- 172.21.15.66:0/1759275876 >> 172.21.15.66:0/1759275876 conn(0x7ff9000cfa30 msgr2=0x7ff9000cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 -- 172.21.15.66:0/1759275876 shutdown_connections 2024-09-10T22:12:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 -- 172.21.15.66:0/1759275876 wait complete. 2024-09-10T22:12:30.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 Processor -- start 2024-09-10T22:12:30.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 -- start start 2024-09-10T22:12:30.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8e40063a0 0x7ff9000a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:30.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9000b0e00 0x7ff9000af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:30.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9000b0090 0x7ff90006e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:30.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff9000ae8e0 con 0x7ff9000b0090 2024-09-10T22:12:30.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9000af0d0 con 0x7ff9000b0e00 2024-09-10T22:12:30.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff904c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff9000aea60 con 0x7ff8e40063a0 2024-09-10T22:12:30.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff8fe800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9000b0090 0x7ff90006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:30.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff8f7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9000b0e00 0x7ff9000af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:30.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff8fe800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9000b0090 0x7ff90006e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39916/0 (socket says 172.21.15.66:39916) 2024-09-10T22:12:30.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff8fe800640 1 -- 172.21.15.66:0/3299511601 learned_addr learned my addr 172.21.15.66:0/3299511601 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:30.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.439+0000 7ff8fde00640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8e40063a0 0x7ff9000a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:30.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff8fe800640 1 -- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8e40063a0 msgr2=0x7ff9000a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:30.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff8fe800640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8e40063a0 0x7ff9000a9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:30.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff8fe800640 1 -- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9000b0e00 msgr2=0x7ff9000af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:30.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff8fe800640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9000b0e00 0x7ff9000af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:30.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff8fe800640 1 -- 172.21.15.66:0/3299511601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9001e45d0 con 0x7ff9000b0090 2024-09-10T22:12:30.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff8fe800640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9000b0090 0x7ff90006e560 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff8f801db00 tx=0x7ff8f8004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:30.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff8f6000640 1 -- 172.21.15.66:0/3299511601 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8001880 con 0x7ff9000b0090 2024-09-10T22:12:30.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff8f6000640 1 -- 172.21.15.66:0/3299511601 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8f8001b10 con 0x7ff9000b0090 2024-09-10T22:12:30.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff8f6000640 1 -- 172.21.15.66:0/3299511601 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8003be0 con 0x7ff9000b0090 2024-09-10T22:12:30.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff904c00640 1 -- 172.21.15.66:0/3299511601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9001f4970 con 0x7ff9000b0090 2024-09-10T22:12:30.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff904c00640 1 -- 172.21.15.66:0/3299511601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9001f4f30 con 0x7ff9000b0090 2024-09-10T22:12:30.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff904c00640 1 -- 172.21.15.66:0/3299511601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9000c4a90 con 0x7ff9000b0090 2024-09-10T22:12:30.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff8f6000640 1 -- 172.21.15.66:0/3299511601 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 38) ==== 98791+0+0 (secure 0 0 0) 0x7ff8f8006070 con 0x7ff9000b0090 2024-09-10T22:12:30.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff8f6000640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8dc074080 0x7ff8dc076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:30.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.443+0000 7ff8f6000640 1 -- 172.21.15.66:0/3299511601 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7ff8f812bb40 con 0x7ff9000b0090 2024-09-10T22:12:30.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.447+0000 7ff8fde00640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8dc074080 0x7ff8dc076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:30.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.447+0000 7ff8fde00640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8dc074080 0x7ff8dc076540 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff9000b0ad0 tx=0x7ff8f0006b20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:30.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.447+0000 7ff8f6000640 1 -- 172.21.15.66:0/3299511601 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff8f80c0030 con 0x7ff9000b0090 2024-09-10T22:12:30.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:30.655+0000 7ff904c00640 1 -- 172.21.15.66:0/3299511601 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone9531", "target": ["mon-mgr", ""]}) -- 0x7ff900070420 con 0x7ff8dc074080 2024-09-10T22:12:31.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.035+0000 7ff8f6000640 1 -- 172.21.15.66:0/3299511601 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 39) ==== 98893+0+0 (secure 0 0 0) 0x7ff8f80f49c0 con 0x7ff9000b0090 2024-09-10T22:12:31.270 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:31.269+0000 7f5f82a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:31.270 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:31.269+0000 7f5f82a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:31.270 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:31.269+0000 7f5f82a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:31.270 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:31.273+0000 7f5f82a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:31.270 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:31.273+0000 7f5f82a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:31.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.347+0000 7ff8f6000640 1 -- 172.21.15.66:0/3299511601 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff900070420 con 0x7ff8dc074080 2024-09-10T22:12:31.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.351+0000 7ff904c00640 1 -- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8dc074080 msgr2=0x7ff8dc076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:31.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.351+0000 7ff904c00640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8dc074080 0x7ff8dc076540 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff9000b0ad0 tx=0x7ff8f0006b20 comp rx=0 tx=0).stop 2024-09-10T22:12:31.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.351+0000 7ff904c00640 1 -- 172.21.15.66:0/3299511601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9000b0090 msgr2=0x7ff90006e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:31.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.351+0000 7ff904c00640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9000b0090 0x7ff90006e560 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff8f801db00 tx=0x7ff8f8004e60 comp rx=0 tx=0).stop 2024-09-10T22:12:31.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.351+0000 7ff904c00640 1 -- 172.21.15.66:0/3299511601 shutdown_connections 2024-09-10T22:12:31.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.351+0000 7ff904c00640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff8dc074080 0x7ff8dc076540 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:31.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.351+0000 7ff904c00640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9000b0090 0x7ff90006e560 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:31.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.351+0000 7ff904c00640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9000b0e00 0x7ff9000af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:31.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.351+0000 7ff904c00640 1 --2- 172.21.15.66:0/3299511601 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8e40063a0 0x7ff9000a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:31.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.351+0000 7ff904c00640 1 -- 172.21.15.66:0/3299511601 >> 172.21.15.66:0/3299511601 conn(0x7ff9000cfa30 msgr2=0x7ff9000b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:31.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.351+0000 7ff904c00640 1 -- 172.21.15.66:0/3299511601 shutdown_connections 2024-09-10T22:12:31.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.351+0000 7ff904c00640 1 -- 172.21.15.66:0/3299511601 wait complete. 2024-09-10T22:12:31.398 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone37332 2024-09-10T22:12:31.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.631+0000 7fad0a400640 1 -- 172.21.15.66:0/2430351232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad041612d0 msgr2=0x7fad040ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:31.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.631+0000 7fad0a400640 1 --2- 172.21.15.66:0/2430351232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad041612d0 0x7fad040ae9c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7facf4009840 tx=0x7facf401c820 comp rx=0 tx=0).stop 2024-09-10T22:12:31.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.631+0000 7fad0a400640 1 -- 172.21.15.66:0/2430351232 shutdown_connections 2024-09-10T22:12:31.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.631+0000 7fad0a400640 1 --2- 172.21.15.66:0/2430351232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad040b3200 0x7fad0406dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:31.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.631+0000 7fad0a400640 1 --2- 172.21.15.66:0/2430351232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad041612d0 0x7fad040ae9c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:31.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.631+0000 7fad0a400640 1 -- 172.21.15.66:0/2430351232 >> 172.21.15.66:0/2430351232 conn(0x7fad040d1c50 msgr2=0x7fad040d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:31.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.631+0000 7fad0a400640 1 -- 172.21.15.66:0/2430351232 shutdown_connections 2024-09-10T22:12:31.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.631+0000 7fad0a400640 1 -- 172.21.15.66:0/2430351232 wait complete. 2024-09-10T22:12:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.635+0000 7fad0a400640 1 Processor -- start 2024-09-10T22:12:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.635+0000 7fad0a400640 1 -- start start 2024-09-10T22:12:31.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad0a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad040b0e30 0x7fad04158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:31.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad0a400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad040b3200 0x7fad040c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:31.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad0a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad041612d0 0x7fad040c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:31.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad0a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad0416b5b0 con 0x7fad041612d0 2024-09-10T22:12:31.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad0a400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad0416b430 con 0x7fad040b0e30 2024-09-10T22:12:31.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad0a400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fad0416b730 con 0x7fad040b3200 2024-09-10T22:12:31.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad09a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad041612d0 0x7fad040c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad09a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad041612d0 0x7fad040c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39928/0 (socket says 172.21.15.66:39928) 2024-09-10T22:12:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad09a00640 1 -- 172.21.15.66:0/362546813 learned_addr learned my addr 172.21.15.66:0/362546813 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad09a00640 1 -- 172.21.15.66:0/362546813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad040b3200 msgr2=0x7fad040c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad09a00640 1 --2- 172.21.15.66:0/362546813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad040b3200 0x7fad040c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad09a00640 1 -- 172.21.15.66:0/362546813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad040b0e30 msgr2=0x7fad04158d00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:12:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad09a00640 1 --2- 172.21.15.66:0/362546813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad040b0e30 0x7fad04158d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad09a00640 1 -- 172.21.15.66:0/362546813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad04159240 con 0x7fad041612d0 2024-09-10T22:12:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad09a00640 1 --2- 172.21.15.66:0/362546813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad041612d0 0x7fad040c6190 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7facfc00c760 tx=0x7facfc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad00c00640 1 -- 172.21.15.66:0/362546813 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc003db0 con 0x7fad041612d0 2024-09-10T22:12:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad00c00640 1 -- 172.21.15.66:0/362546813 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facfc005070 con 0x7fad041612d0 2024-09-10T22:12:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad0a400640 1 -- 172.21.15.66:0/362546813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad040c73a0 con 0x7fad041612d0 2024-09-10T22:12:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad00c00640 1 -- 172.21.15.66:0/362546813 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0054a0 con 0x7fad041612d0 2024-09-10T22:12:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad0a400640 1 -- 172.21.15.66:0/362546813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad040c7960 con 0x7fad041612d0 2024-09-10T22:12:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.639+0000 7fad0a400640 1 -- 172.21.15.66:0/362546813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0406dd30 con 0x7fad041612d0 2024-09-10T22:12:31.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.643+0000 7fad00c00640 1 -- 172.21.15.66:0/362546813 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 39) ==== 98893+0+0 (secure 0 0 0) 0x7facfc005640 con 0x7fad041612d0 2024-09-10T22:12:31.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.643+0000 7fad00c00640 1 --2- 172.21.15.66:0/362546813 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7face40741a0 0x7face4076660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:31.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.643+0000 7fad00c00640 1 -- 172.21.15.66:0/362546813 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7facfc092070 con 0x7fad041612d0 2024-09-10T22:12:31.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.647+0000 7fad00c00640 1 -- 172.21.15.66:0/362546813 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fad04159240 con 0x7fad041612d0 2024-09-10T22:12:31.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.659+0000 7fad09000640 1 --2- 172.21.15.66:0/362546813 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7face40741a0 0x7face4076660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:31.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:31.667+0000 7fad09000640 1 --2- 172.21.15.66:0/362546813 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7face40741a0 0x7face4076660 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7facf40af000 tx=0x7facf401b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:32.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.003+0000 7fad0a400640 1 -- 172.21.15.66:0/362546813 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone37332", "target": ["mon-mgr", ""]}) -- 0x7fad040b3bc0 con 0x7face40741a0 2024-09-10T22:12:32.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.439+0000 7fad00c00640 1 -- 172.21.15.66:0/362546813 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fad040b3bc0 con 0x7face40741a0 2024-09-10T22:12:32.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.447+0000 7fad0a400640 1 -- 172.21.15.66:0/362546813 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7face40741a0 msgr2=0x7face4076660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:32.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.447+0000 7fad0a400640 1 --2- 172.21.15.66:0/362546813 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7face40741a0 0x7face4076660 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7facf40af000 tx=0x7facf401b060 comp rx=0 tx=0).stop 2024-09-10T22:12:32.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.447+0000 7fad0a400640 1 -- 172.21.15.66:0/362546813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad041612d0 msgr2=0x7fad040c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:32.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.447+0000 7fad0a400640 1 --2- 172.21.15.66:0/362546813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad041612d0 0x7fad040c6190 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7facfc00c760 tx=0x7facfc00cc20 comp rx=0 tx=0).stop 2024-09-10T22:12:32.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.447+0000 7fad0a400640 1 -- 172.21.15.66:0/362546813 shutdown_connections 2024-09-10T22:12:32.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.447+0000 7fad0a400640 1 --2- 172.21.15.66:0/362546813 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7face40741a0 0x7face4076660 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:32.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.447+0000 7fad0a400640 1 --2- 172.21.15.66:0/362546813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad041612d0 0x7fad040c6190 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:32.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.447+0000 7fad0a400640 1 --2- 172.21.15.66:0/362546813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad040b3200 0x7fad040c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:32.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.447+0000 7fad0a400640 1 --2- 172.21.15.66:0/362546813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad040b0e30 0x7fad04158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:32.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.447+0000 7fad0a400640 1 -- 172.21.15.66:0/362546813 >> 172.21.15.66:0/362546813 conn(0x7fad040d1c50 msgr2=0x7fad040b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:32.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.447+0000 7fad0a400640 1 -- 172.21.15.66:0/362546813 shutdown_connections 2024-09-10T22:12:32.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:32.447+0000 7fad0a400640 1 -- 172.21.15.66:0/362546813 wait complete. 2024-09-10T22:12:32.540 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:32.541+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:32.540 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:32.541+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:32.541 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:32.541+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:32.541 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:32.541+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:32.541 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:32.541+0000 7f5fee200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:33.487 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8660 2024-09-10T22:12:33.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.703+0000 7fecd9400640 1 -- 172.21.15.66:0/3704977517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecd40cb620 msgr2=0x7fecd40ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:33.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.703+0000 7fecd9400640 1 --2- 172.21.15.66:0/3704977517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecd40cb620 0x7fecd40ceff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feccc009bc0 tx=0x7feccc01cb70 comp rx=0 tx=0).stop 2024-09-10T22:12:33.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.703+0000 7fecd9400640 1 -- 172.21.15.66:0/3704977517 shutdown_connections 2024-09-10T22:12:33.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.703+0000 7fecd9400640 1 --2- 172.21.15.66:0/3704977517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecd40cb620 0x7fecd40ceff0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:33.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.703+0000 7fecd9400640 1 --2- 172.21.15.66:0/3704977517 >> v2:172.21.15.94:3300/0 conn(0x7fecd415a8a0 0x7fecd40cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:33.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.703+0000 7fecd9400640 1 -- 172.21.15.66:0/3704977517 >> 172.21.15.66:0/3704977517 conn(0x7fecd40b26c0 msgr2=0x7fecd40b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:33.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.703+0000 7fecd9400640 1 -- 172.21.15.66:0/3704977517 shutdown_connections 2024-09-10T22:12:33.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.703+0000 7fecd9400640 1 -- 172.21.15.66:0/3704977517 wait complete. 2024-09-10T22:12:33.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.707+0000 7fecd9400640 1 Processor -- start 2024-09-10T22:12:33.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.707+0000 7fecd9400640 1 -- start start 2024-09-10T22:12:33.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecd40cb620 0x7fecd4151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecd415a8a0 0x7fecd415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd9400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecd415ac80 0x7fecd40736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd9400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fecd4167d10 con 0x7fecd415a8a0 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd9400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fecd4167b90 con 0x7fecd40cb620 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd9400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fecd4167e90 con 0x7fecd415ac80 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecd415a8a0 0x7fecd415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecd415a8a0 0x7fecd415ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59342/0 (socket says 172.21.15.66:59342) 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd1e00640 1 -- 172.21.15.66:0/362714157 learned_addr learned my addr 172.21.15.66:0/362714157 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd2800640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecd40cb620 0x7fecd4151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd3200640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecd415ac80 0x7fecd40736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd1e00640 1 -- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecd415ac80 msgr2=0x7fecd40736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd1e00640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecd415ac80 0x7fecd40736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd1e00640 1 -- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecd40cb620 msgr2=0x7fecd4151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd1e00640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecd40cb620 0x7fecd4151e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd1e00640 1 -- 172.21.15.66:0/362714157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd4200360 con 0x7fecd415a8a0 2024-09-10T22:12:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd1e00640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecd415a8a0 0x7fecd415ba20 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fecc000c3f0 tx=0x7fecc000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7feccaa00640 1 -- 172.21.15.66:0/362714157 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0017070 con 0x7fecd415a8a0 2024-09-10T22:12:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7feccaa00640 1 -- 172.21.15.66:0/362714157 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecc00053c0 con 0x7fecd415a8a0 2024-09-10T22:12:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd9400640 1 -- 172.21.15.66:0/362714157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd41ffac0 con 0x7fecd415a8a0 2024-09-10T22:12:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7feccaa00640 1 -- 172.21.15.66:0/362714157 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0003860 con 0x7fecd415a8a0 2024-09-10T22:12:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd9400640 1 -- 172.21.15.66:0/362714157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd4200080 con 0x7fecd415a8a0 2024-09-10T22:12:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.711+0000 7fecd9400640 1 -- 172.21.15.66:0/362714157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd40ceff0 con 0x7fecd415a8a0 2024-09-10T22:12:33.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.715+0000 7feccaa00640 1 -- 172.21.15.66:0/362714157 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 40) ==== 98944+0+0 (secure 0 0 0) 0x7fecc0003a00 con 0x7fecd415a8a0 2024-09-10T22:12:33.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.715+0000 7feccaa00640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fecb4074200 0x7fecb40766c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:33.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.715+0000 7feccaa00640 1 -- 172.21.15.66:0/362714157 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fecc0091870 con 0x7fecd415a8a0 2024-09-10T22:12:33.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.719+0000 7feccaa00640 1 -- 172.21.15.66:0/362714157 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fecc0021080 con 0x7fecd415a8a0 2024-09-10T22:12:33.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.735+0000 7fecd2800640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fecb4074200 0x7fecb40766c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:33.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:33.739+0000 7fecd2800640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fecb4074200 0x7fecb40766c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fecc40096c0 tx=0x7fecc4009260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:34.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.059+0000 7fecd9400640 1 -- 172.21.15.66:0/362714157 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8660", "target": ["mon-mgr", ""]}) -- 0x7fecd4159eb0 con 0x7fecb4074200 2024-09-10T22:12:34.157 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:34.157+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:34.157 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:34.157+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:34.157 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:34.157+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:34.157 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:34.157+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:34.157 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:34.157+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:34.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.731+0000 7feccaa00640 1 -- 172.21.15.66:0/362714157 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fecd4159eb0 con 0x7fecb4074200 2024-09-10T22:12:34.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.735+0000 7fecd9400640 1 -- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fecb4074200 msgr2=0x7fecb40766c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:34.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.735+0000 7fecd9400640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fecb4074200 0x7fecb40766c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fecc40096c0 tx=0x7fecc4009260 comp rx=0 tx=0).stop 2024-09-10T22:12:34.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.735+0000 7fecd9400640 1 -- 172.21.15.66:0/362714157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecd415a8a0 msgr2=0x7fecd415ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:34.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.735+0000 7fecd9400640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecd415a8a0 0x7fecd415ba20 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fecc000c3f0 tx=0x7fecc000c8b0 comp rx=0 tx=0).stop 2024-09-10T22:12:34.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.735+0000 7fecd9400640 1 -- 172.21.15.66:0/362714157 shutdown_connections 2024-09-10T22:12:34.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.735+0000 7fecd9400640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fecb4074200 0x7fecb40766c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:34.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.735+0000 7fecd9400640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecd415ac80 0x7fecd40736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:34.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.735+0000 7fecd9400640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecd415a8a0 0x7fecd415ba20 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:34.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.735+0000 7fecd9400640 1 --2- 172.21.15.66:0/362714157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecd40cb620 0x7fecd4151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:34.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.735+0000 7fecd9400640 1 -- 172.21.15.66:0/362714157 >> 172.21.15.66:0/362714157 conn(0x7fecd40b26c0 msgr2=0x7fecd4070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:34.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.735+0000 7fecd9400640 1 -- 172.21.15.66:0/362714157 shutdown_connections 2024-09-10T22:12:34.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:34.735+0000 7fecd9400640 1 -- 172.21.15.66:0/362714157 wait complete. 2024-09-10T22:12:34.774 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone9531 2024-09-10T22:12:35.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 -- 172.21.15.66:0/2726607228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f4071310 msgr2=0x7fa6f415c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:35.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 --2- 172.21.15.66:0/2726607228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f4071310 0x7fa6f415c920 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40d0b90 tx=0x7fa6e001c800 comp rx=0 tx=0).stop 2024-09-10T22:12:35.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 -- 172.21.15.66:0/2726607228 shutdown_connections 2024-09-10T22:12:35.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 --2- 172.21.15.66:0/2726607228 >> v2:172.21.15.94:3300/0 conn(0x7fa6f40737c0 0x7fa6f4073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:35.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 --2- 172.21.15.66:0/2726607228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f4071310 0x7fa6f415c920 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:35.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 --2- 172.21.15.66:0/2726607228 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f41560a0 0x7fa6f415ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:35.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 -- 172.21.15.66:0/2726607228 >> 172.21.15.66:0/2726607228 conn(0x7fa6f40b2810 msgr2=0x7fa6f40b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:35.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 -- 172.21.15.66:0/2726607228 shutdown_connections 2024-09-10T22:12:35.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 -- 172.21.15.66:0/2726607228 wait complete. 2024-09-10T22:12:35.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 Processor -- start 2024-09-10T22:12:35.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 -- start start 2024-09-10T22:12:35.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f4071310 0x7fa6f41f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:35.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f40737c0 0x7fa6f4149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:35.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f41560a0 0x7fa6f41f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:35.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f406cfb0 con 0x7fa6f40737c0 2024-09-10T22:12:35.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f406ce30 con 0x7fa6f4071310 2024-09-10T22:12:35.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6fc000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f406d130 con 0x7fa6f41560a0 2024-09-10T22:12:35.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6f8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f40737c0 0x7fa6f4149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:35.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6f8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f40737c0 0x7fa6f4149db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59364/0 (socket says 172.21.15.66:59364) 2024-09-10T22:12:35.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.035+0000 7fa6f8c00640 1 -- 172.21.15.66:0/3127151100 learned_addr learned my addr 172.21.15.66:0/3127151100 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:35.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6fa000640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f41560a0 0x7fa6f41f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6f9600640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f4071310 0x7fa6f41f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6f8c00640 1 -- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f41560a0 msgr2=0x7fa6f41f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6f8c00640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f41560a0 0x7fa6f41f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6f8c00640 1 -- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f4071310 msgr2=0x7fa6f41f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6f8c00640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f4071310 0x7fa6f41f7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6f8c00640 1 -- 172.21.15.66:0/3127151100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f41f7fc0 con 0x7fa6f40737c0 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6f9600640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f4071310 0x7fa6f41f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6f8c00640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f40737c0 0x7fa6f4149db0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00b0040 tx=0x7fa6e0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6f1600640 1 -- 172.21.15.66:0/3127151100 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00b2040 con 0x7fa6f40737c0 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6f1600640 1 -- 172.21.15.66:0/3127151100 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6e0003600 con 0x7fa6f40737c0 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6fc000640 1 -- 172.21.15.66:0/3127151100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f41f82a0 con 0x7fa6f40737c0 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6f1600640 1 -- 172.21.15.66:0/3127151100 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0003a30 con 0x7fa6f40737c0 2024-09-10T22:12:35.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6fc000640 1 -- 172.21.15.66:0/3127151100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f414a640 con 0x7fa6f40737c0 2024-09-10T22:12:35.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.039+0000 7fa6fc000640 1 -- 172.21.15.66:0/3127151100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f4073c10 con 0x7fa6f40737c0 2024-09-10T22:12:35.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.043+0000 7fa6f1600640 1 -- 172.21.15.66:0/3127151100 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 40) ==== 98944+0+0 (secure 0 0 0) 0x7fa6e0003bd0 con 0x7fa6f40737c0 2024-09-10T22:12:35.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.043+0000 7fa6f1600640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa6d4074200 0x7fa6d40766c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:35.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.043+0000 7fa6f1600640 1 -- 172.21.15.66:0/3127151100 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fa6e012c000 con 0x7fa6f40737c0 2024-09-10T22:12:35.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.047+0000 7fa6f1600640 1 -- 172.21.15.66:0/3127151100 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa6e00bd060 con 0x7fa6f40737c0 2024-09-10T22:12:35.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.051+0000 7fa6f9600640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa6d4074200 0x7fa6d40766c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:35.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.055+0000 7fa6f9600640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa6d4074200 0x7fa6d40766c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800a8b0 tx=0x7fa6e8008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:35.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.167+0000 7fa6f1600640 1 -- 172.21.15.66:0/3127151100 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7fa6e00f4cb0 con 0x7fa6f40737c0 2024-09-10T22:12:35.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.399+0000 7fa6fc000640 1 -- 172.21.15.66:0/3127151100 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone9531", "target": ["mon-mgr", ""]}) -- 0x7fa6f40ccfb0 con 0x7fa6d4074200 2024-09-10T22:12:35.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.839+0000 7fa6f1600640 1 -- 172.21.15.66:0/3127151100 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa6f40ccfb0 con 0x7fa6d4074200 2024-09-10T22:12:35.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.847+0000 7fa6fc000640 1 -- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa6d4074200 msgr2=0x7fa6d40766c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:35.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.847+0000 7fa6fc000640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa6d4074200 0x7fa6d40766c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800a8b0 tx=0x7fa6e8008040 comp rx=0 tx=0).stop 2024-09-10T22:12:35.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.847+0000 7fa6fc000640 1 -- 172.21.15.66:0/3127151100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f40737c0 msgr2=0x7fa6f4149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:35.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.847+0000 7fa6fc000640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f40737c0 0x7fa6f4149db0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00b0040 tx=0x7fa6e0004040 comp rx=0 tx=0).stop 2024-09-10T22:12:35.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.847+0000 7fa6fc000640 1 -- 172.21.15.66:0/3127151100 shutdown_connections 2024-09-10T22:12:35.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.847+0000 7fa6fc000640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa6d4074200 0x7fa6d40766c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:35.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.847+0000 7fa6fc000640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f41560a0 0x7fa6f41f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:35.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.847+0000 7fa6fc000640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f40737c0 0x7fa6f4149db0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:35.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.847+0000 7fa6fc000640 1 --2- 172.21.15.66:0/3127151100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f4071310 0x7fa6f41f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:35.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.847+0000 7fa6fc000640 1 -- 172.21.15.66:0/3127151100 >> 172.21.15.66:0/3127151100 conn(0x7fa6f40b2810 msgr2=0x7fa6f40b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:35.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.847+0000 7fa6fc000640 1 -- 172.21.15.66:0/3127151100 shutdown_connections 2024-09-10T22:12:35.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:35.847+0000 7fa6fc000640 1 -- 172.21.15.66:0/3127151100 wait complete. 2024-09-10T22:12:35.895 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone37332 2024-09-10T22:12:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.111+0000 7f8fd3200640 1 -- 172.21.15.66:0/3980263981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fcc160490 msgr2=0x7f8fcc0d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.111+0000 7f8fd3200640 1 --2- 172.21.15.66:0/3980263981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fcc160490 0x7f8fcc0d0300 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc073670 tx=0x7f8fbc01f0d0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.111+0000 7f8fd3200640 1 -- 172.21.15.66:0/3980263981 shutdown_connections 2024-09-10T22:12:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.111+0000 7f8fd3200640 1 --2- 172.21.15.66:0/3980263981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fcc0b2a20 0x7f8fcc0aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.111+0000 7f8fd3200640 1 --2- 172.21.15.66:0/3980263981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fcc160490 0x7f8fcc0d0300 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.111+0000 7f8fd3200640 1 -- 172.21.15.66:0/3980263981 >> 172.21.15.66:0/3980263981 conn(0x7f8fcc0d1d20 msgr2=0x7f8fcc0d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.111+0000 7f8fd3200640 1 -- 172.21.15.66:0/3980263981 shutdown_connections 2024-09-10T22:12:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.111+0000 7f8fd3200640 1 -- 172.21.15.66:0/3980263981 wait complete. 2024-09-10T22:12:36.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.111+0000 7f8fd3200640 1 Processor -- start 2024-09-10T22:12:36.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.111+0000 7f8fd3200640 1 -- start start 2024-09-10T22:12:36.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fd3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fcc0af010 0x7f8fcc0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:36.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fd3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fcc0b2a20 0x7f8fcc15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:36.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fd3200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fcc160490 0x7f8fcc15e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:36.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fd3200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc0b3580 con 0x7f8fcc0af010 2024-09-10T22:12:36.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fd3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc0b3400 con 0x7f8fcc0b2a20 2024-09-10T22:12:36.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fd3200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8fcc0b11f0 con 0x7f8fcc160490 2024-09-10T22:12:36.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fcc0af010 0x7f8fcc0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:36.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fcc0af010 0x7f8fcc0cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59382/0 (socket says 172.21.15.66:59382) 2024-09-10T22:12:36.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fcbe00640 1 -- 172.21.15.66:0/1112342085 learned_addr learned my addr 172.21.15.66:0/1112342085 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:36.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fcb400640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fcc0b2a20 0x7f8fcc15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:36.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fd1200640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fcc160490 0x7f8fcc15e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:36.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fcbe00640 1 -- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fcc160490 msgr2=0x7f8fcc15e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:36.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fcbe00640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fcc160490 0x7f8fcc15e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fcbe00640 1 -- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fcc0b2a20 msgr2=0x7f8fcc15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:36.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fcbe00640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fcc0b2a20 0x7f8fcc15c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fcbe00640 1 -- 172.21.15.66:0/1112342085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fcc155be0 con 0x7f8fcc0af010 2024-09-10T22:12:36.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fcb400640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fcc0b2a20 0x7f8fcc15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:12:36.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fcbe00640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fcc0af010 0x7f8fcc0cb8b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0b3040 tx=0x7f8fbc0a8ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:36.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fc3e00640 1 -- 172.21.15.66:0/1112342085 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0b5040 con 0x7f8fcc0af010 2024-09-10T22:12:36.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fc3e00640 1 -- 172.21.15.66:0/1112342085 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fbc003970 con 0x7f8fcc0af010 2024-09-10T22:12:36.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fc3e00640 1 -- 172.21.15.66:0/1112342085 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc003dc0 con 0x7f8fcc0af010 2024-09-10T22:12:36.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.115+0000 7f8fd3200640 1 -- 172.21.15.66:0/1112342085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fcc155340 con 0x7f8fcc0af010 2024-09-10T22:12:36.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.119+0000 7f8fd3200640 1 -- 172.21.15.66:0/1112342085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fcc155900 con 0x7f8fcc0af010 2024-09-10T22:12:36.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.119+0000 7f8fd3200640 1 -- 172.21.15.66:0/1112342085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fcc0aead0 con 0x7f8fcc0af010 2024-09-10T22:12:36.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.123+0000 7f8fc3e00640 1 -- 172.21.15.66:0/1112342085 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7f8fbc0bb470 con 0x7f8fcc0af010 2024-09-10T22:12:36.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.123+0000 7f8fc3e00640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8fac074320 0x7f8fac0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:36.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.123+0000 7f8fc3e00640 1 -- 172.21.15.66:0/1112342085 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f8fbc12f090 con 0x7f8fcc0af010 2024-09-10T22:12:36.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.127+0000 7f8fcb400640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8fac074320 0x7f8fac0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:36.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.127+0000 7f8fc3e00640 1 -- 172.21.15.66:0/1112342085 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8fbc018c30 con 0x7f8fcc0af010 2024-09-10T22:12:36.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.131+0000 7f8fcb400640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8fac074320 0x7f8fac0767e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc1f6820 tx=0x7f8fb8008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:36.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.451+0000 7f8fd3200640 1 -- 172.21.15.66:0/1112342085 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone37332", "target": ["mon-mgr", ""]}) -- 0x7f8fcc0d00e0 con 0x7f8fac074320 2024-09-10T22:12:36.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.543+0000 7f8fc3e00640 1 -- 172.21.15.66:0/1112342085 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8fcc0d00e0 con 0x7f8fac074320 2024-09-10T22:12:36.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.547+0000 7f8fd3200640 1 -- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8fac074320 msgr2=0x7f8fac0767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:36.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.547+0000 7f8fd3200640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8fac074320 0x7f8fac0767e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc1f6820 tx=0x7f8fb8008040 comp rx=0 tx=0).stop 2024-09-10T22:12:36.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.547+0000 7f8fd3200640 1 -- 172.21.15.66:0/1112342085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fcc0af010 msgr2=0x7f8fcc0cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:36.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.547+0000 7f8fd3200640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fcc0af010 0x7f8fcc0cb8b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0b3040 tx=0x7f8fbc0a8ce0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.547+0000 7f8fd3200640 1 -- 172.21.15.66:0/1112342085 shutdown_connections 2024-09-10T22:12:36.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.547+0000 7f8fd3200640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8fac074320 0x7f8fac0767e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.547+0000 7f8fd3200640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fcc160490 0x7f8fcc15e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.547+0000 7f8fd3200640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fcc0b2a20 0x7f8fcc15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.547+0000 7f8fd3200640 1 --2- 172.21.15.66:0/1112342085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fcc0af010 0x7f8fcc0cb8b0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.547+0000 7f8fd3200640 1 -- 172.21.15.66:0/1112342085 >> 172.21.15.66:0/1112342085 conn(0x7f8fcc0d1d20 msgr2=0x7f8fcc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:36.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.547+0000 7f8fd3200640 1 -- 172.21.15.66:0/1112342085 shutdown_connections 2024-09-10T22:12:36.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.547+0000 7f8fd3200640 1 -- 172.21.15.66:0/1112342085 wait complete. 2024-09-10T22:12:36.570 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone8660 2024-09-10T22:12:36.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.715+0000 7fbb08000640 1 -- 172.21.15.66:0/926050344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb000d0550 msgr2=0x7fbb000ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:36.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.715+0000 7fbb08000640 1 --2- 172.21.15.66:0/926050344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb000d0550 0x7fbb000ca1a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00a980 tx=0x7fbaec01db00 comp rx=0 tx=0).stop 2024-09-10T22:12:36.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.715+0000 7fbb08000640 1 -- 172.21.15.66:0/926050344 shutdown_connections 2024-09-10T22:12:36.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.715+0000 7fbb08000640 1 --2- 172.21.15.66:0/926050344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb000d0550 0x7fbb000ca1a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.715+0000 7fbb08000640 1 -- 172.21.15.66:0/926050344 >> 172.21.15.66:0/926050344 conn(0x7fbb000cfa30 msgr2=0x7fbb000cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:36.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.715+0000 7fbb08000640 1 -- 172.21.15.66:0/926050344 shutdown_connections 2024-09-10T22:12:36.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.715+0000 7fbb08000640 1 -- 172.21.15.66:0/926050344 wait complete. 2024-09-10T22:12:36.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.715+0000 7fbb08000640 1 Processor -- start 2024-09-10T22:12:36.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.715+0000 7fbb08000640 1 -- start start 2024-09-10T22:12:36.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb08000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb000c4fc0 0x7fbb000c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:36.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb08000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0015a310 0x7fbb000ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:36.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb08000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb000ade00 0x7fbb000b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:36.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb08000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001686a0 con 0x7fbb000c4fc0 2024-09-10T22:12:36.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb08000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbb00168520 con 0x7fbb0015a310 2024-09-10T22:12:36.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb08000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbb00168820 con 0x7fbb000ade00 2024-09-10T22:12:36.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb05600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb000c4fc0 0x7fbb000c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:36.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb05600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb000c4fc0 0x7fbb000c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59390/0 (socket says 172.21.15.66:59390) 2024-09-10T22:12:36.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb05600640 1 -- 172.21.15.66:0/2652749639 learned_addr learned my addr 172.21.15.66:0/2652749639 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:36.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb04c00640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0015a310 0x7fbb000ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:36.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb06000640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb000ade00 0x7fbb000b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:36.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb05600640 1 -- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb000ade00 msgr2=0x7fbb000b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:36.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb05600640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb000ade00 0x7fbb000b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb05600640 1 -- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0015a310 msgr2=0x7fbb000ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:36.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb05600640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0015a310 0x7fbb000ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:36.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb05600640 1 -- 172.21.15.66:0/2652749639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb000b3ec0 con 0x7fbb000c4fc0 2024-09-10T22:12:36.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb05600640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb000c4fc0 0x7fbb000c9390 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400c430 tx=0x7fbaf400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:36.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbafd600640 1 -- 172.21.15.66:0/2652749639 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4017070 con 0x7fbb000c4fc0 2024-09-10T22:12:36.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbafd600640 1 -- 172.21.15.66:0/2652749639 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaf40053c0 con 0x7fbb000c4fc0 2024-09-10T22:12:36.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbafd600640 1 -- 172.21.15.66:0/2652749639 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4003860 con 0x7fbb000c4fc0 2024-09-10T22:12:36.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb08000640 1 -- 172.21.15.66:0/2652749639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb000b41a0 con 0x7fbb000c4fc0 2024-09-10T22:12:36.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb08000640 1 -- 172.21.15.66:0/2652749639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb000b4760 con 0x7fbb000c4fc0 2024-09-10T22:12:36.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.719+0000 7fbb08000640 1 -- 172.21.15.66:0/2652749639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb000c48b0 con 0x7fbb000c4fc0 2024-09-10T22:12:36.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.723+0000 7fbafd600640 1 -- 172.21.15.66:0/2652749639 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7fbaf4003a00 con 0x7fbb000c4fc0 2024-09-10T22:12:36.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.723+0000 7fbafd600640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbae0074320 0x7fbae00767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:36.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.723+0000 7fbafd600640 1 -- 172.21.15.66:0/2652749639 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fbaf4091850 con 0x7fbb000c4fc0 2024-09-10T22:12:36.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.723+0000 7fbb04c00640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbae0074320 0x7fbae00767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:36.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.723+0000 7fbb04c00640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbae0074320 0x7fbae00767e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0b0000 tx=0x7fbaec01c340 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:36.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.727+0000 7fbafd600640 1 -- 172.21.15.66:0/2652749639 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbaf405e340 con 0x7fbb000c4fc0 2024-09-10T22:12:36.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:36.939+0000 7fbb08000640 1 -- 172.21.15.66:0/2652749639 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone8660", "target": ["mon-mgr", ""]}) -- 0x7fbb00070420 con 0x7fbae0074320 2024-09-10T22:12:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.719+0000 7fbafd600640 1 -- 172.21.15.66:0/2652749639 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fbb00070420 con 0x7fbae0074320 2024-09-10T22:12:38.729 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:12:38.729 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:12:38.729 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "canceled", 2024-09-10T22:12:38.729 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:12:38.729 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:12:38.729 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:12:38.729 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:12:38.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:38.730 INFO:teuthology.orchestra.run.smithi066.stdout: "failure": { 2024-09-10T22:12:38.730 INFO:teuthology.orchestra.run.smithi066.stdout: "errno": "4", 2024-09-10T22:12:38.730 INFO:teuthology.orchestra.run.smithi066.stdout: "error_msg": "user interrupted clone operation" 2024-09-10T22:12:38.730 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:38.730 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:38.730 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:12:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.723+0000 7fbb08000640 1 -- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbae0074320 msgr2=0x7fbae00767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.723+0000 7fbb08000640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbae0074320 0x7fbae00767e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0b0000 tx=0x7fbaec01c340 comp rx=0 tx=0).stop 2024-09-10T22:12:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.723+0000 7fbb08000640 1 -- 172.21.15.66:0/2652749639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb000c4fc0 msgr2=0x7fbb000c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.723+0000 7fbb08000640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb000c4fc0 0x7fbb000c9390 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400c430 tx=0x7fbaf400c8f0 comp rx=0 tx=0).stop 2024-09-10T22:12:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.723+0000 7fbb08000640 1 -- 172.21.15.66:0/2652749639 shutdown_connections 2024-09-10T22:12:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.723+0000 7fbb08000640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbae0074320 0x7fbae00767e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.723+0000 7fbb08000640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb000ade00 0x7fbb000b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.723+0000 7fbb08000640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0015a310 0x7fbb000ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.723+0000 7fbb08000640 1 --2- 172.21.15.66:0/2652749639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb000c4fc0 0x7fbb000c9390 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.723+0000 7fbb08000640 1 -- 172.21.15.66:0/2652749639 >> 172.21.15.66:0/2652749639 conn(0x7fbb000cfa30 msgr2=0x7fbb0014fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:38.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.723+0000 7fbb08000640 1 -- 172.21.15.66:0/2652749639 shutdown_connections 2024-09-10T22:12:38.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.723+0000 7fbb08000640 1 -- 172.21.15.66:0/2652749639 wait complete. 2024-09-10T22:12:38.751 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-10T22:12:38.751 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone9531 2024-09-10T22:12:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.899+0000 7f421a000640 1 -- 172.21.15.66:0/3651282977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f421415ef20 msgr2=0x7f42140cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.899+0000 7f421a000640 1 --2- 172.21.15.66:0/3651282977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f421415ef20 0x7f42140cecc0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f4204009900 tx=0x7f420401c870 comp rx=0 tx=0).stop 2024-09-10T22:12:38.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 -- 172.21.15.66:0/3651282977 shutdown_connections 2024-09-10T22:12:38.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 --2- 172.21.15.66:0/3651282977 >> v2:172.21.15.94:3300/0 conn(0x7f421415f300 0x7f42140b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:38.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 --2- 172.21.15.66:0/3651282977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f421415ef20 0x7f42140cecc0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:38.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 -- 172.21.15.66:0/3651282977 >> 172.21.15.66:0/3651282977 conn(0x7f42140d1b10 msgr2=0x7f42140d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:38.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 -- 172.21.15.66:0/3651282977 shutdown_connections 2024-09-10T22:12:38.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 -- 172.21.15.66:0/3651282977 wait complete. 2024-09-10T22:12:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 Processor -- start 2024-09-10T22:12:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 -- start start 2024-09-10T22:12:38.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42140b15d0 0x7f42140b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:38.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f421415ef20 0x7f42141fbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:38.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f421415f300 0x7f421414a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:38.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4214168d60 con 0x7f421415f300 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4214168ee0 con 0x7f42140b15d0 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f421a000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4214169060 con 0x7f421415ef20 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f4219600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f421415f300 0x7f421414a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f4219600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f421415f300 0x7f421414a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59404/0 (socket says 172.21.15.66:59404) 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f4219600640 1 -- 172.21.15.66:0/2087542748 learned_addr learned my addr 172.21.15.66:0/2087542748 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f4218c00640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42140b15d0 0x7f42140b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f420be00640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f421415ef20 0x7f42141fbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f4219600640 1 -- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f421415ef20 msgr2=0x7f42141fbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f4219600640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f421415ef20 0x7f42141fbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f4219600640 1 -- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42140b15d0 msgr2=0x7f42140b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f4219600640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42140b15d0 0x7f42140b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f4219600640 1 -- 172.21.15.66:0/2087542748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42141fc4c0 con 0x7f421415f300 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.903+0000 7f4219600640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f421415f300 0x7f421414a9f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f420c00c780 tx=0x7f420c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.907+0000 7f4211600640 1 -- 172.21.15.66:0/2087542748 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c003b80 con 0x7f421415f300 2024-09-10T22:12:38.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.907+0000 7f4211600640 1 -- 172.21.15.66:0/2087542748 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f420c003d20 con 0x7f421415f300 2024-09-10T22:12:38.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.907+0000 7f4211600640 1 -- 172.21.15.66:0/2087542748 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c00aab0 con 0x7f421415f300 2024-09-10T22:12:38.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.907+0000 7f421a000640 1 -- 172.21.15.66:0/2087542748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421414b3d0 con 0x7f421415f300 2024-09-10T22:12:38.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.907+0000 7f421a000640 1 -- 172.21.15.66:0/2087542748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42141fb780 con 0x7f421415f300 2024-09-10T22:12:38.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.907+0000 7f421a000640 1 -- 172.21.15.66:0/2087542748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421414b8f0 con 0x7f421415f300 2024-09-10T22:12:38.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.907+0000 7f4211600640 1 -- 172.21.15.66:0/2087542748 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7f420c00ac50 con 0x7f421415f300 2024-09-10T22:12:38.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.907+0000 7f4211600640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41f0074320 0x7f41f00767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:38.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.907+0000 7f4211600640 1 -- 172.21.15.66:0/2087542748 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f420c091fd0 con 0x7f421415f300 2024-09-10T22:12:38.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.907+0000 7f4218c00640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41f0074320 0x7f41f00767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:38.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.911+0000 7f4218c00640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41f0074320 0x7f41f00767e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f42040af000 tx=0x7f420401b0b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:38.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:38.915+0000 7f4211600640 1 -- 172.21.15.66:0/2087542748 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f421414b8f0 con 0x7f421415f300 2024-09-10T22:12:39.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:39.123+0000 7f421a000640 1 -- 172.21.15.66:0/2087542748 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone9531", "target": ["mon-mgr", ""]}) -- 0x7f42140ce0d0 con 0x7f41f0074320 2024-09-10T22:12:43.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.715+0000 7f4211600640 1 -- 172.21.15.66:0/2087542748 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f42040b3040 con 0x7f41f0074320 2024-09-10T22:12:43.723 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:12:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:12:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "canceled", 2024-09-10T22:12:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:12:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:12:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:12:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:12:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "failure": { 2024-09-10T22:12:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: "errno": "4", 2024-09-10T22:12:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: "error_msg": "user interrupted clone operation" 2024-09-10T22:12:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:43.724 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:12:43.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.719+0000 7f421a000640 1 -- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41f0074320 msgr2=0x7f41f00767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:43.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.719+0000 7f421a000640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41f0074320 0x7f41f00767e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f42040af000 tx=0x7f420401b0b0 comp rx=0 tx=0).stop 2024-09-10T22:12:43.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.719+0000 7f421a000640 1 -- 172.21.15.66:0/2087542748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f421415f300 msgr2=0x7f421414a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.719+0000 7f421a000640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f421415f300 0x7f421414a9f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f420c00c780 tx=0x7f420c00cc40 comp rx=0 tx=0).stop 2024-09-10T22:12:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.719+0000 7f421a000640 1 -- 172.21.15.66:0/2087542748 shutdown_connections 2024-09-10T22:12:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.719+0000 7f421a000640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41f0074320 0x7f41f00767e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.719+0000 7f421a000640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f421415f300 0x7f421414a9f0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.719+0000 7f421a000640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f421415ef20 0x7f42141fbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.719+0000 7f421a000640 1 --2- 172.21.15.66:0/2087542748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42140b15d0 0x7f42140b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.719+0000 7f421a000640 1 -- 172.21.15.66:0/2087542748 >> 172.21.15.66:0/2087542748 conn(0x7f42140d1b10 msgr2=0x7f4214071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.719+0000 7f421a000640 1 -- 172.21.15.66:0/2087542748 shutdown_connections 2024-09-10T22:12:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.719+0000 7f421a000640 1 -- 172.21.15.66:0/2087542748 wait complete. 2024-09-10T22:12:43.749 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-10T22:12:43.749 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone37332 2024-09-10T22:12:43.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.899+0000 7f9bae400640 1 -- 172.21.15.66:0/722980432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ba80b15d0 msgr2=0x7f9ba80cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:43.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.899+0000 7f9bae400640 1 --2- 172.21.15.66:0/722980432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ba80b15d0 0x7f9ba80cd7a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009bc0 tx=0x7f9ba001cba0 comp rx=0 tx=0).stop 2024-09-10T22:12:43.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.899+0000 7f9bae400640 1 -- 172.21.15.66:0/722980432 shutdown_connections 2024-09-10T22:12:43.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.899+0000 7f9bae400640 1 --2- 172.21.15.66:0/722980432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ba80b15d0 0x7f9ba80cd7a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:43.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.899+0000 7f9bae400640 1 --2- 172.21.15.66:0/722980432 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ba815f300 0x7f9ba80b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:43.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.899+0000 7f9bae400640 1 -- 172.21.15.66:0/722980432 >> 172.21.15.66:0/722980432 conn(0x7f9ba80d1b10 msgr2=0x7f9ba80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:43.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.899+0000 7f9bae400640 1 -- 172.21.15.66:0/722980432 shutdown_connections 2024-09-10T22:12:43.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.899+0000 7f9bae400640 1 -- 172.21.15.66:0/722980432 wait complete. 2024-09-10T22:12:43.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.899+0000 7f9bae400640 1 Processor -- start 2024-09-10T22:12:43.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.899+0000 7f9bae400640 1 -- start start 2024-09-10T22:12:43.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9bae400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ba80b15d0 0x7f9ba80b7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:43.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9bae400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ba815ef20 0x7f9ba80cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:43.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9bae400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ba815f300 0x7f9ba80af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:43.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9bae400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba81693b0 con 0x7f9ba815ef20 2024-09-10T22:12:43.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9bae400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8169230 con 0x7f9ba815f300 2024-09-10T22:12:43.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9bae400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba8169530 con 0x7f9ba80b15d0 2024-09-10T22:12:43.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9ba7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ba815ef20 0x7f9ba80cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:43.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9ba7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ba815ef20 0x7f9ba80cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38508/0 (socket says 172.21.15.66:38508) 2024-09-10T22:12:43.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9ba7e00640 1 -- 172.21.15.66:0/991313705 learned_addr learned my addr 172.21.15.66:0/991313705 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:43.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9bad000640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ba80b15d0 0x7f9ba80b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:43.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9bada00640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ba815f300 0x7f9ba80af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:43.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9ba7e00640 1 -- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ba80b15d0 msgr2=0x7f9ba80b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:43.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9ba7e00640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ba80b15d0 0x7f9ba80b7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:43.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9ba7e00640 1 -- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ba815f300 msgr2=0x7f9ba80af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:43.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9ba7e00640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ba815f300 0x7f9ba80af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:43.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9ba7e00640 1 -- 172.21.15.66:0/991313705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba80cb8f0 con 0x7f9ba815ef20 2024-09-10T22:12:43.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9ba7e00640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ba815ef20 0x7f9ba80cb3b0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400e9c0 tx=0x7f9b9400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:43.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9ba4c00640 1 -- 172.21.15.66:0/991313705 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400cd80 con 0x7f9ba815ef20 2024-09-10T22:12:43.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9ba4c00640 1 -- 172.21.15.66:0/991313705 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b94010520 con 0x7f9ba815ef20 2024-09-10T22:12:43.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9ba4c00640 1 -- 172.21.15.66:0/991313705 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94010970 con 0x7f9ba815ef20 2024-09-10T22:12:43.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9bae400640 1 -- 172.21.15.66:0/991313705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba81fbe70 con 0x7f9ba815ef20 2024-09-10T22:12:43.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9bae400640 1 -- 172.21.15.66:0/991313705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba81f8150 con 0x7f9ba815ef20 2024-09-10T22:12:43.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.903+0000 7f9bae400640 1 -- 172.21.15.66:0/991313705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba81f7c90 con 0x7f9ba815ef20 2024-09-10T22:12:43.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.907+0000 7f9ba4c00640 1 -- 172.21.15.66:0/991313705 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7f9b94010b10 con 0x7f9ba815ef20 2024-09-10T22:12:43.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.907+0000 7f9ba4c00640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b88074320 0x7f9b880767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:43.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.907+0000 7f9ba4c00640 1 -- 172.21.15.66:0/991313705 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f9b94092c50 con 0x7f9ba815ef20 2024-09-10T22:12:43.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.907+0000 7f9bad000640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b88074320 0x7f9b880767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:43.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.907+0000 7f9bad000640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b88074320 0x7f9b880767e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c001910 tx=0x7f9b9c001420 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:43.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:43.911+0000 7f9ba4c00640 1 -- 172.21.15.66:0/991313705 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9ba81f7c90 con 0x7f9ba815ef20 2024-09-10T22:12:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.119+0000 7f9bae400640 1 -- 172.21.15.66:0/991313705 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone37332", "target": ["mon-mgr", ""]}) -- 0x7f9ba81fc150 con 0x7f9b88074320 2024-09-10T22:12:44.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.127+0000 7f9ba4c00640 1 -- 172.21.15.66:0/991313705 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f9ba81fc150 con 0x7f9b88074320 2024-09-10T22:12:44.133 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:12:44.133 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:12:44.133 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "canceled", 2024-09-10T22:12:44.133 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:12:44.133 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:12:44.133 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:12:44.134 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:12:44.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:44.134 INFO:teuthology.orchestra.run.smithi066.stdout: "failure": { 2024-09-10T22:12:44.134 INFO:teuthology.orchestra.run.smithi066.stdout: "errno": "4", 2024-09-10T22:12:44.134 INFO:teuthology.orchestra.run.smithi066.stdout: "error_msg": "user interrupted clone operation" 2024-09-10T22:12:44.134 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:44.134 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:44.134 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.131+0000 7f9bae400640 1 -- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b88074320 msgr2=0x7f9b880767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.131+0000 7f9bae400640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b88074320 0x7f9b880767e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c001910 tx=0x7f9b9c001420 comp rx=0 tx=0).stop 2024-09-10T22:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.131+0000 7f9bae400640 1 -- 172.21.15.66:0/991313705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ba815ef20 msgr2=0x7f9ba80cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.131+0000 7f9bae400640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ba815ef20 0x7f9ba80cb3b0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400e9c0 tx=0x7f9b9400ee80 comp rx=0 tx=0).stop 2024-09-10T22:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.131+0000 7f9bae400640 1 -- 172.21.15.66:0/991313705 shutdown_connections 2024-09-10T22:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.131+0000 7f9bae400640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b88074320 0x7f9b880767e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.131+0000 7f9bae400640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ba815f300 0x7f9ba80af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.131+0000 7f9bae400640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ba815ef20 0x7f9ba80cb3b0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.131+0000 7f9bae400640 1 --2- 172.21.15.66:0/991313705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ba80b15d0 0x7f9ba80b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.131+0000 7f9bae400640 1 -- 172.21.15.66:0/991313705 >> 172.21.15.66:0/991313705 conn(0x7f9ba80d1b10 msgr2=0x7f9ba8166020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.131+0000 7f9bae400640 1 -- 172.21.15.66:0/991313705 shutdown_connections 2024-09-10T22:12:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.131+0000 7f9bae400640 1 -- 172.21.15.66:0/991313705 wait complete. 2024-09-10T22:12:44.155 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-10T22:12:44.155 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:12:44.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.299+0000 7feae6e00640 1 -- 172.21.15.66:0/1660433950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae00d06b0 msgr2=0x7feae00ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:44.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.299+0000 7feae6e00640 1 --2- 172.21.15.66:0/1660433950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae00d06b0 0x7feae00ce170 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7feacc009900 tx=0x7feacc01c850 comp rx=0 tx=0).stop 2024-09-10T22:12:44.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.299+0000 7feae6e00640 1 -- 172.21.15.66:0/1660433950 shutdown_connections 2024-09-10T22:12:44.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.299+0000 7feae6e00640 1 --2- 172.21.15.66:0/1660433950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae00d06b0 0x7feae00ce170 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.299+0000 7feae6e00640 1 -- 172.21.15.66:0/1660433950 >> 172.21.15.66:0/1660433950 conn(0x7feae00d1c60 msgr2=0x7feae00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:44.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.299+0000 7feae6e00640 1 -- 172.21.15.66:0/1660433950 shutdown_connections 2024-09-10T22:12:44.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.299+0000 7feae6e00640 1 -- 172.21.15.66:0/1660433950 wait complete. 2024-09-10T22:12:44.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.299+0000 7feae6e00640 1 Processor -- start 2024-09-10T22:12:44.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.299+0000 7feae6e00640 1 -- start start 2024-09-10T22:12:44.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feae6e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feae00afc20 0x7feae00b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:44.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feae6e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae00d06b0 0x7feae014fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:44.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feae6e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae015bd40 0x7feae00b7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:44.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feae6e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feae016a160 con 0x7feae00d06b0 2024-09-10T22:12:44.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feae6e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feae0169fe0 con 0x7feae015bd40 2024-09-10T22:12:44.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feae6e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7feae016a2e0 con 0x7feae00afc20 2024-09-10T22:12:44.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feadf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae00d06b0 0x7feae014fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:44.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feadf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae00d06b0 0x7feae014fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38520/0 (socket says 172.21.15.66:38520) 2024-09-10T22:12:44.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feadf400640 1 -- 172.21.15.66:0/3266111825 learned_addr learned my addr 172.21.15.66:0/3266111825 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:44.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feadfe00640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feae00afc20 0x7feae00b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:44.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feae4e00640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae015bd40 0x7feae00b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:44.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feadf400640 1 -- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feae00afc20 msgr2=0x7feae00b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:44.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feadf400640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feae00afc20 0x7feae00b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feadf400640 1 -- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae015bd40 msgr2=0x7feae00b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:44.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feadf400640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae015bd40 0x7feae00b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feadf400640 1 -- 172.21.15.66:0/3266111825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae00b50b0 con 0x7feae00d06b0 2024-09-10T22:12:44.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feae4e00640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae015bd40 0x7feae00b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:44.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feadf400640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae00d06b0 0x7feae014fea0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7feacc0b0040 tx=0x7feacc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:44.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7fead7e00640 1 -- 172.21.15.66:0/3266111825 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc0b2040 con 0x7feae00d06b0 2024-09-10T22:12:44.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7fead7e00640 1 -- 172.21.15.66:0/3266111825 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feacc004430 con 0x7feae00d06b0 2024-09-10T22:12:44.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7fead7e00640 1 -- 172.21.15.66:0/3266111825 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc005290 con 0x7feae00d06b0 2024-09-10T22:12:44.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feae6e00640 1 -- 172.21.15.66:0/3266111825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae01fb650 con 0x7feae00d06b0 2024-09-10T22:12:44.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feae6e00640 1 -- 172.21.15.66:0/3266111825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae01fbc10 con 0x7feae00d06b0 2024-09-10T22:12:44.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.303+0000 7feae6e00640 1 -- 172.21.15.66:0/3266111825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae01fb440 con 0x7feae00d06b0 2024-09-10T22:12:44.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.307+0000 7fead7e00640 1 -- 172.21.15.66:0/3266111825 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7feacc0046d0 con 0x7feae00d06b0 2024-09-10T22:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.307+0000 7fead7e00640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feac0074350 0x7feac0076810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.307+0000 7fead7e00640 1 -- 172.21.15.66:0/3266111825 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7feacc12c2c0 con 0x7feae00d06b0 2024-09-10T22:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.307+0000 7feadfe00640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feac0074350 0x7feac0076810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.307+0000 7feadfe00640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feac0074350 0x7feac0076810 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7feae00c8460 tx=0x7fead0008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.311+0000 7fead7e00640 1 -- 172.21.15.66:0/3266111825 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feacc0164d0 con 0x7feae00d06b0 2024-09-10T22:12:44.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 -- 172.21.15.66:0/3266111825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7feae00cffa0 con 0x7feae00d06b0 2024-09-10T22:12:44.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7fead7e00640 1 -- 172.21.15.66:0/3266111825 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1921 (secure 0 0 0) 0x7feacc0f9400 con 0x7feae00d06b0 2024-09-10T22:12:44.551 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:44.552 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:12:44.552 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:12:44.552 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:12:44.552 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:12:44.552 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:12:44.552 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:12:44.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:44.552 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:12:44.552 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:12:44.552 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 757, 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:12:44.553 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:12:44.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:44.554 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:12:44.554 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 114, 2024-09-10T22:12:44.554 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:12:44.554 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:12:44.554 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:12:44.554 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:12:44.554 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:12:44.554 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:12:44.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:44.554 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:12:44.554 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:12:44.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:12:44.555 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:12:44.555 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:12:44.555 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:44.555 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:12:44.555 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 1683, 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 6921856601, 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 14933110784, 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 758161002496, 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: "read_bytes_sec": 310227696, 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 308175678, 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 76, 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 88 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:12:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 77, 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:12:01:218274+0000", 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 8, 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 8, 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "b", 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 8307 2024-09-10T22:12:44.557 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:44.558 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:12:44.559 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 19, 2024-09-10T22:12:44.559 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:11:36.558334+0000", 2024-09-10T22:12:44.559 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:12:44.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:44.560 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": {} 2024-09-10T22:12:44.560 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:12:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 -- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feac0074350 msgr2=0x7feac0076810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feac0074350 0x7feac0076810 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7feae00c8460 tx=0x7fead0008040 comp rx=0 tx=0).stop 2024-09-10T22:12:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 -- 172.21.15.66:0/3266111825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae00d06b0 msgr2=0x7feae014fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae00d06b0 0x7feae014fea0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7feacc0b0040 tx=0x7feacc004040 comp rx=0 tx=0).stop 2024-09-10T22:12:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 -- 172.21.15.66:0/3266111825 shutdown_connections 2024-09-10T22:12:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7feac0074350 0x7feac0076810 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae015bd40 0x7feae00b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae00d06b0 0x7feae014fea0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 --2- 172.21.15.66:0/3266111825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feae00afc20 0x7feae00b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 -- 172.21.15.66:0/3266111825 >> 172.21.15.66:0/3266111825 conn(0x7feae00d1c60 msgr2=0x7feae0156960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 -- 172.21.15.66:0/3266111825 shutdown_connections 2024-09-10T22:12:44.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.547+0000 7feae6e00640 1 -- 172.21.15.66:0/3266111825 wait complete. 2024-09-10T22:12:44.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:12:44.574 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone8660/153e2f09-00c3-44e1-9d76-ec67bbc64599 2024-09-10T22:12:44.574 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:12:44.587 INFO:teuthology.orchestra.run.smithi066.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone8660/153e2f09-00c3-44e1-9d76-ec67bbc64599': No such file or directory 2024-09-10T22:12:44.587 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T22:12:44.588 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:12:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.771+0000 7f2977600640 1 -- 172.21.15.66:0/385668272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700b2bf0 msgr2=0x7f29700af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.771+0000 7f2977600640 1 --2- 172.21.15.66:0/385668272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700b2bf0 0x7f29700af070 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2968009bc0 tx=0x7f296801cac0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.771+0000 7f2977600640 1 -- 172.21.15.66:0/385668272 shutdown_connections 2024-09-10T22:12:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.771+0000 7f2977600640 1 --2- 172.21.15.66:0/385668272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700b2bf0 0x7f29700af070 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.771+0000 7f2977600640 1 -- 172.21.15.66:0/385668272 >> 172.21.15.66:0/385668272 conn(0x7f29700d1c50 msgr2=0x7f29700d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.771+0000 7f2977600640 1 -- 172.21.15.66:0/385668272 shutdown_connections 2024-09-10T22:12:44.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.771+0000 7f2977600640 1 -- 172.21.15.66:0/385668272 wait complete. 2024-09-10T22:12:44.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2977600640 1 Processor -- start 2024-09-10T22:12:44.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2977600640 1 -- start start 2024-09-10T22:12:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2977600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700b2bf0 0x7f29700c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2977600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29700d0260 0x7f297006df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2977600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f297015f340 0x7f297014fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2977600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f297016a2e0 con 0x7f297015f340 2024-09-10T22:12:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2977600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f297016a160 con 0x7f29700b2bf0 2024-09-10T22:12:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2977600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f297016a460 con 0x7f29700d0260 2024-09-10T22:12:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2975600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f297015f340 0x7f297014fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2975600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f297015f340 0x7f297014fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38522/0 (socket says 172.21.15.66:38522) 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2975600640 1 -- 172.21.15.66:0/2939708611 learned_addr learned my addr 172.21.15.66:0/2939708611 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2974c00640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700b2bf0 0x7f29700c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f296fe00640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29700d0260 0x7f297006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2975600640 1 -- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29700d0260 msgr2=0x7f297006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2975600640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29700d0260 0x7f297006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2975600640 1 -- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700b2bf0 msgr2=0x7f29700c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2975600640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700b2bf0 0x7f29700c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2975600640 1 -- 172.21.15.66:0/2939708611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2970152810 con 0x7f297015f340 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2974c00640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700b2bf0 0x7f29700c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2975600640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f297015f340 0x7f297014fd10 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f29680b0040 tx=0x7f2968009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f296cc00640 1 -- 172.21.15.66:0/2939708611 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29680bb070 con 0x7f297015f340 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f296cc00640 1 -- 172.21.15.66:0/2939708611 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2968004e00 con 0x7f297015f340 2024-09-10T22:12:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f296cc00640 1 -- 172.21.15.66:0/2939708611 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29680b2040 con 0x7f297015f340 2024-09-10T22:12:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2977600640 1 -- 172.21.15.66:0/2939708611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2970151690 con 0x7f297015f340 2024-09-10T22:12:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2977600640 1 -- 172.21.15.66:0/2939708611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2970151310 con 0x7f297015f340 2024-09-10T22:12:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.775+0000 7f2977600640 1 -- 172.21.15.66:0/2939708611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29700cfb50 con 0x7f297015f340 2024-09-10T22:12:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.779+0000 7f296cc00640 1 -- 172.21.15.66:0/2939708611 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7f29680b5070 con 0x7f297015f340 2024-09-10T22:12:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.779+0000 7f296cc00640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2950074320 0x7f29500767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.779+0000 7f296cc00640 1 -- 172.21.15.66:0/2939708611 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f296812b980 con 0x7f297015f340 2024-09-10T22:12:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.779+0000 7f2974c00640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2950074320 0x7f29500767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.779+0000 7f2974c00640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2950074320 0x7f29500767e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2964001130 tx=0x7f2964009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:44.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.783+0000 7f296cc00640 1 -- 172.21.15.66:0/2939708611 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f29680166b0 con 0x7f297015f340 2024-09-10T22:12:44.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.979+0000 7f2977600640 1 -- 172.21.15.66:0/2939708611 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f297015c550 con 0x7f2950074320 2024-09-10T22:12:44.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.979+0000 7f296cc00640 1 -- 172.21.15.66:0/2939708611 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f297015c550 con 0x7f2950074320 2024-09-10T22:12:44.985 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:12:44.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:12:44.985 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:12:44.985 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:44.985 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:12:44.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.983+0000 7f2977600640 1 -- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2950074320 msgr2=0x7f29500767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:44.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.983+0000 7f2977600640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2950074320 0x7f29500767e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2964001130 tx=0x7f2964009040 comp rx=0 tx=0).stop 2024-09-10T22:12:44.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.983+0000 7f2977600640 1 -- 172.21.15.66:0/2939708611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f297015f340 msgr2=0x7f297014fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:44.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.983+0000 7f2977600640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f297015f340 0x7f297014fd10 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f29680b0040 tx=0x7f2968009200 comp rx=0 tx=0).stop 2024-09-10T22:12:44.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.983+0000 7f2977600640 1 -- 172.21.15.66:0/2939708611 shutdown_connections 2024-09-10T22:12:44.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.983+0000 7f2977600640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2950074320 0x7f29500767e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.983+0000 7f2977600640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f297015f340 0x7f297014fd10 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.983+0000 7f2977600640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29700d0260 0x7f297006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.983+0000 7f2977600640 1 --2- 172.21.15.66:0/2939708611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700b2bf0 0x7f29700c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:44.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.983+0000 7f2977600640 1 -- 172.21.15.66:0/2939708611 >> 172.21.15.66:0/2939708611 conn(0x7f29700d1c50 msgr2=0x7f29701567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:44.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.983+0000 7f2977600640 1 -- 172.21.15.66:0/2939708611 shutdown_connections 2024-09-10T22:12:44.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:44.983+0000 7f2977600640 1 -- 172.21.15.66:0/2939708611 wait complete. 2024-09-10T22:12:45.007 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-10T22:12:45.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.151+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1352401636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0b015cc80 msgr2=0x7fa0b00ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.151+0000 7fa0b5e00640 1 --2- 172.21.15.66:0/1352401636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0b015cc80 0x7fa0b00ae710 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa098009840 tx=0x7fa09801c790 comp rx=0 tx=0).stop 2024-09-10T22:12:45.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.151+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1352401636 shutdown_connections 2024-09-10T22:12:45.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.151+0000 7fa0b5e00640 1 --2- 172.21.15.66:0/1352401636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0b015cc80 0x7fa0b00ae710 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.151+0000 7fa0b5e00640 1 --2- 172.21.15.66:0/1352401636 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0b0159900 0x7fa0b00ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.151+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1352401636 >> 172.21.15.66:0/1352401636 conn(0x7fa0b00cfa30 msgr2=0x7fa0b00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:45.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.151+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1352401636 shutdown_connections 2024-09-10T22:12:45.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.151+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1352401636 wait complete. 2024-09-10T22:12:45.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0b5e00640 1 Processor -- start 2024-09-10T22:12:45.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0b5e00640 1 -- start start 2024-09-10T22:12:45.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0b5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b00aec50 0x7fa0b0141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0b5e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0b0159900 0x7fa0b00c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0b5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0b015cc80 0x7fa0b0156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0b5e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0168fc0 con 0x7fa0b00aec50 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0b5e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0168e40 con 0x7fa0b015cc80 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0b5e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b0169140 con 0x7fa0b0159900 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0aea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b00aec50 0x7fa0b0141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0aea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b00aec50 0x7fa0b0141010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38530/0 (socket says 172.21.15.66:38530) 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0aea00640 1 -- 172.21.15.66:0/1297404086 learned_addr learned my addr 172.21.15.66:0/1297404086 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0af400640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0b015cc80 0x7fa0b0156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0a7e00640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0b0159900 0x7fa0b00c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0aea00640 1 -- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0b0159900 msgr2=0x7fa0b00c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0aea00640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0b0159900 0x7fa0b00c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0aea00640 1 -- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0b015cc80 msgr2=0x7fa0b0156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0aea00640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0b015cc80 0x7fa0b0156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0aea00640 1 -- 172.21.15.66:0/1297404086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b00aea50 con 0x7fa0b00aec50 2024-09-10T22:12:45.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0af400640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0b015cc80 0x7fa0b0156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:45.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0aea00640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b00aec50 0x7fa0b0141010 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000e9f0 tx=0x7fa0a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0a7400640 1 -- 172.21.15.66:0/1297404086 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000cd80 con 0x7fa0b00aec50 2024-09-10T22:12:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0a7400640 1 -- 172.21.15.66:0/1297404086 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0a0010430 con 0x7fa0b00aec50 2024-09-10T22:12:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1297404086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b00c4db0 con 0x7fa0b00aec50 2024-09-10T22:12:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0a7400640 1 -- 172.21.15.66:0/1297404086 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0010860 con 0x7fa0b00aec50 2024-09-10T22:12:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.155+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1297404086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b00c5370 con 0x7fa0b00aec50 2024-09-10T22:12:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.159+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1297404086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b0070470 con 0x7fa0b00aec50 2024-09-10T22:12:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.159+0000 7fa0a7400640 1 -- 172.21.15.66:0/1297404086 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7fa0a0010a00 con 0x7fa0b00aec50 2024-09-10T22:12:45.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.159+0000 7fa0a7400640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa08c074320 0x7fa08c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:45.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.159+0000 7fa0a7400640 1 -- 172.21.15.66:0/1297404086 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fa0a0092d70 con 0x7fa0b00aec50 2024-09-10T22:12:45.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.159+0000 7fa0a7e00640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa08c074320 0x7fa08c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:45.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.159+0000 7fa0a7e00640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa08c074320 0x7fa08c0767e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa0980af000 tx=0x7fa0980b5040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:45.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.163+0000 7fa0a7400640 1 -- 172.21.15.66:0/1297404086 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa0a005f920 con 0x7fa0b00aec50 2024-09-10T22:12:45.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.363+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1297404086 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0b00741f0 con 0x7fa08c074320 2024-09-10T22:12:45.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0a7400640 1 -- 172.21.15.66:0/1297404086 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+183 (secure 0 0 0) 0x7fa0b00741f0 con 0x7fa08c074320 2024-09-10T22:12:45.375 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:45.375 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:12:45.375 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:12:45.376 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone8660" 2024-09-10T22:12:45.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:45.376 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:12:45.376 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone37332" 2024-09-10T22:12:45.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:45.376 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:12:45.376 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sv1" 2024-09-10T22:12:45.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:12:45.376 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:12:45.376 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone9531" 2024-09-10T22:12:45.376 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:45.376 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:12:45.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa08c074320 msgr2=0x7fa08c0767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0b5e00640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa08c074320 0x7fa08c0767e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa0980af000 tx=0x7fa0980b5040 comp rx=0 tx=0).stop 2024-09-10T22:12:45.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1297404086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b00aec50 msgr2=0x7fa0b0141010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0b5e00640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b00aec50 0x7fa0b0141010 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000e9f0 tx=0x7fa0a000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1297404086 shutdown_connections 2024-09-10T22:12:45.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0b5e00640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fa08c074320 0x7fa08c0767e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0b5e00640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0b015cc80 0x7fa0b0156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0b5e00640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0b0159900 0x7fa0b00c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0b5e00640 1 --2- 172.21.15.66:0/1297404086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b00aec50 0x7fa0b0141010 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1297404086 >> 172.21.15.66:0/1297404086 conn(0x7fa0b00cfa30 msgr2=0x7fa0b0155dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:45.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1297404086 shutdown_connections 2024-09-10T22:12:45.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.371+0000 7fa0b5e00640 1 -- 172.21.15.66:0/1297404086 wait complete. 2024-09-10T22:12:45.397 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8660 2024-09-10T22:12:45.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.543+0000 7fc812c00640 1 -- 172.21.15.66:0/3047797569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc80c0d0950 msgr2=0x7fc80c06dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.543+0000 7fc812c00640 1 --2- 172.21.15.66:0/3047797569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc80c0d0950 0x7fc80c06dad0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc804009c80 tx=0x7fc80401cda0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.543+0000 7fc812c00640 1 -- 172.21.15.66:0/3047797569 shutdown_connections 2024-09-10T22:12:45.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.543+0000 7fc812c00640 1 --2- 172.21.15.66:0/3047797569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc80c0d0950 0x7fc80c06dad0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.543+0000 7fc812c00640 1 --2- 172.21.15.66:0/3047797569 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc80c15def0 0x7fc80c0cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.543+0000 7fc812c00640 1 -- 172.21.15.66:0/3047797569 >> 172.21.15.66:0/3047797569 conn(0x7fc80c0d1c60 msgr2=0x7fc80c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:45.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.543+0000 7fc812c00640 1 -- 172.21.15.66:0/3047797569 shutdown_connections 2024-09-10T22:12:45.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.543+0000 7fc812c00640 1 -- 172.21.15.66:0/3047797569 wait complete. 2024-09-10T22:12:45.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc812c00640 1 Processor -- start 2024-09-10T22:12:45.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc812c00640 1 -- start start 2024-09-10T22:12:45.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc812c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc80c0cf030 0x7fc80c1fb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc812c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc80c0d0950 0x7fc80c14b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc812c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc80c15def0 0x7fc80c1502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc812c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c16b360 con 0x7fc80c0d0950 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc812c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c16b1e0 con 0x7fc80c0cf030 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc812c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c16b4e0 con 0x7fc80c15def0 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc803e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc80c0d0950 0x7fc80c14b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc803e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc80c0d0950 0x7fc80c14b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38538/0 (socket says 172.21.15.66:38538) 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc803e00640 1 -- 172.21.15.66:0/2225107968 learned_addr learned my addr 172.21.15.66:0/2225107968 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc80be00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc80c0cf030 0x7fc80c1fb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc810c00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc80c15def0 0x7fc80c1502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc803e00640 1 -- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc80c15def0 msgr2=0x7fc80c1502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc803e00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc80c15def0 0x7fc80c1502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc803e00640 1 -- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc80c0cf030 msgr2=0x7fc80c1fb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc803e00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc80c0cf030 0x7fc80c1fb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc803e00640 1 -- 172.21.15.66:0/2225107968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c14c130 con 0x7fc80c0d0950 2024-09-10T22:12:45.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc80be00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc80c0cf030 0x7fc80c1fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:45.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc803e00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc80c0d0950 0x7fc80c14b7b0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c3c0 tx=0x7fc7f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:45.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc808c00640 1 -- 172.21.15.66:0/2225107968 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4017070 con 0x7fc80c0d0950 2024-09-10T22:12:45.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc808c00640 1 -- 172.21.15.66:0/2225107968 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7f40053c0 con 0x7fc80c0d0950 2024-09-10T22:12:45.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc808c00640 1 -- 172.21.15.66:0/2225107968 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4003860 con 0x7fc80c0d0950 2024-09-10T22:12:45.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc812c00640 1 -- 172.21.15.66:0/2225107968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c1fbb40 con 0x7fc80c0d0950 2024-09-10T22:12:45.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc812c00640 1 -- 172.21.15.66:0/2225107968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c14c3d0 con 0x7fc80c0d0950 2024-09-10T22:12:45.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.547+0000 7fc812c00640 1 -- 172.21.15.66:0/2225107968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc80c0d03e0 con 0x7fc80c0d0950 2024-09-10T22:12:45.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.551+0000 7fc808c00640 1 -- 172.21.15.66:0/2225107968 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7fc7f4003a00 con 0x7fc80c0d0950 2024-09-10T22:12:45.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.551+0000 7fc808c00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc7e8074350 0x7fc7e8076810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:45.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.551+0000 7fc808c00640 1 -- 172.21.15.66:0/2225107968 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fc7f40918e0 con 0x7fc80c0d0950 2024-09-10T22:12:45.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.551+0000 7fc80be00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc7e8074350 0x7fc7e8076810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:45.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.551+0000 7fc80be00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc7e8074350 0x7fc7e8076810 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00bbd0 tx=0x7fc7fc00b5b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:45.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.555+0000 7fc808c00640 1 -- 172.21.15.66:0/2225107968 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc80c1fbb40 con 0x7fc80c0d0950 2024-09-10T22:12:45.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.759+0000 7fc812c00640 1 -- 172.21.15.66:0/2225107968 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8660", "target": ["mon-mgr", ""]}) -- 0x7fc80c06fb20 con 0x7fc7e8074350 2024-09-10T22:12:45.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.775+0000 7fc808c00640 1 -- 172.21.15.66:0/2225107968 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc80c06fb20 con 0x7fc7e8074350 2024-09-10T22:12:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.779+0000 7fc812c00640 1 -- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc7e8074350 msgr2=0x7fc7e8076810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.779+0000 7fc812c00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc7e8074350 0x7fc7e8076810 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00bbd0 tx=0x7fc7fc00b5b0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.779+0000 7fc812c00640 1 -- 172.21.15.66:0/2225107968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc80c0d0950 msgr2=0x7fc80c14b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.779+0000 7fc812c00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc80c0d0950 0x7fc80c14b7b0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c3c0 tx=0x7fc7f400c880 comp rx=0 tx=0).stop 2024-09-10T22:12:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.779+0000 7fc812c00640 1 -- 172.21.15.66:0/2225107968 shutdown_connections 2024-09-10T22:12:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.779+0000 7fc812c00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc7e8074350 0x7fc7e8076810 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.779+0000 7fc812c00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc80c15def0 0x7fc80c1502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.779+0000 7fc812c00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc80c0d0950 0x7fc80c14b7b0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.779+0000 7fc812c00640 1 --2- 172.21.15.66:0/2225107968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc80c0cf030 0x7fc80c1fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.779+0000 7fc812c00640 1 -- 172.21.15.66:0/2225107968 >> 172.21.15.66:0/2225107968 conn(0x7fc80c0d1c60 msgr2=0x7fc80c0b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:45.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.779+0000 7fc812c00640 1 -- 172.21.15.66:0/2225107968 shutdown_connections 2024-09-10T22:12:45.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.779+0000 7fc812c00640 1 -- 172.21.15.66:0/2225107968 wait complete. 2024-09-10T22:12:45.794 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:45.796+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:45.794 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:45.796+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:45.794 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:45.796+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:45.794 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:45.796+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:45.794 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:12:45.796+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:12:45.803 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone37332 2024-09-10T22:12:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.947+0000 7fc4afe00640 1 -- 172.21.15.66:0/3841663864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a815ddd0 msgr2=0x7fc4a815e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.947+0000 7fc4afe00640 1 --2- 172.21.15.66:0/3841663864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a815ddd0 0x7fc4a815e1f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc49400aa10 tx=0x7fc494020420 comp rx=0 tx=0).stop 2024-09-10T22:12:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.947+0000 7fc4afe00640 1 -- 172.21.15.66:0/3841663864 shutdown_connections 2024-09-10T22:12:45.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.947+0000 7fc4afe00640 1 --2- 172.21.15.66:0/3841663864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a815ddd0 0x7fc4a815e1f0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.947+0000 7fc4afe00640 1 --2- 172.21.15.66:0/3841663864 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4a80cee30 0x7fc4a80cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.947+0000 7fc4afe00640 1 -- 172.21.15.66:0/3841663864 >> 172.21.15.66:0/3841663864 conn(0x7fc4a80cfaa0 msgr2=0x7fc4a80cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:45.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.947+0000 7fc4afe00640 1 -- 172.21.15.66:0/3841663864 shutdown_connections 2024-09-10T22:12:45.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.947+0000 7fc4afe00640 1 -- 172.21.15.66:0/3841663864 wait complete. 2024-09-10T22:12:45.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4afe00640 1 Processor -- start 2024-09-10T22:12:45.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4afe00640 1 -- start start 2024-09-10T22:12:45.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4afe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4a806ef30 0x7fc4a81523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:45.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4afe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a80cee30 0x7fc4a81529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:45.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4afe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4a815ddd0 0x7fc4a80aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:45.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4afe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a815c740 con 0x7fc4a80cee30 2024-09-10T22:12:45.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4afe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a815c8c0 con 0x7fc4a806ef30 2024-09-10T22:12:45.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4afe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a806faf0 con 0x7fc4a815ddd0 2024-09-10T22:12:45.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4aca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a80cee30 0x7fc4a81529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:45.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4aca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a80cee30 0x7fc4a81529b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38554/0 (socket says 172.21.15.66:38554) 2024-09-10T22:12:45.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4aca00640 1 -- 172.21.15.66:0/3716398479 learned_addr learned my addr 172.21.15.66:0/3716398479 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:45.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4ad400640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4a806ef30 0x7fc4a81523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:45.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4ade00640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4a815ddd0 0x7fc4a80aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:45.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4aca00640 1 -- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4a815ddd0 msgr2=0x7fc4a80aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4aca00640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4a815ddd0 0x7fc4a80aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4aca00640 1 -- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4a806ef30 msgr2=0x7fc4a81523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:45.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4aca00640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4a806ef30 0x7fc4a81523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:45.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4aca00640 1 -- 172.21.15.66:0/3716398479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a8153300 con 0x7fc4a80cee30 2024-09-10T22:12:45.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4ade00640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4a815ddd0 0x7fc4a80aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:45.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4aca00640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a80cee30 0x7fc4a81529b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fc49400a960 tx=0x7fc4940b0400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:45.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4a5600640 1 -- 172.21.15.66:0/3716398479 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940b5040 con 0x7fc4a80cee30 2024-09-10T22:12:45.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4a5600640 1 -- 172.21.15.66:0/3716398479 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4940b0cd0 con 0x7fc4a80cee30 2024-09-10T22:12:45.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4afe00640 1 -- 172.21.15.66:0/3716398479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a8151900 con 0x7fc4a80cee30 2024-09-10T22:12:45.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.951+0000 7fc4a5600640 1 -- 172.21.15.66:0/3716398479 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494005280 con 0x7fc4a80cee30 2024-09-10T22:12:45.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.955+0000 7fc4afe00640 1 -- 172.21.15.66:0/3716398479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a8151ec0 con 0x7fc4a80cee30 2024-09-10T22:12:45.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.955+0000 7fc4afe00640 1 -- 172.21.15.66:0/3716398479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a80cf210 con 0x7fc4a80cee30 2024-09-10T22:12:45.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.955+0000 7fc4a5600640 1 -- 172.21.15.66:0/3716398479 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7fc4940b8070 con 0x7fc4a80cee30 2024-09-10T22:12:45.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.955+0000 7fc4a5600640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc488074350 0x7fc488076810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:45.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.955+0000 7fc4a5600640 1 -- 172.21.15.66:0/3716398479 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fc49412f680 con 0x7fc4a80cee30 2024-09-10T22:12:45.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.955+0000 7fc4ad400640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc488074350 0x7fc488076810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:45.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.959+0000 7fc4ad400640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc488074350 0x7fc488076810 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80d0640 tx=0x7fc49c00b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:45.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:45.959+0000 7fc4a5600640 1 -- 172.21.15.66:0/3716398479 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc494019f80 con 0x7fc4a80cee30 2024-09-10T22:12:46.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.159+0000 7fc4afe00640 1 -- 172.21.15.66:0/3716398479 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone37332", "target": ["mon-mgr", ""]}) -- 0x7fc4a80d07c0 con 0x7fc488074350 2024-09-10T22:12:46.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4a5600640 1 -- 172.21.15.66:0/3716398479 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc4a80d07c0 con 0x7fc488074350 2024-09-10T22:12:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4afe00640 1 -- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc488074350 msgr2=0x7fc488076810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4afe00640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc488074350 0x7fc488076810 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80d0640 tx=0x7fc49c00b2e0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4afe00640 1 -- 172.21.15.66:0/3716398479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a80cee30 msgr2=0x7fc4a81529b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4afe00640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a80cee30 0x7fc4a81529b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fc49400a960 tx=0x7fc4940b0400 comp rx=0 tx=0).stop 2024-09-10T22:12:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4afe00640 1 -- 172.21.15.66:0/3716398479 shutdown_connections 2024-09-10T22:12:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4afe00640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc488074350 0x7fc488076810 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4afe00640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4a815ddd0 0x7fc4a80aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4afe00640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4a80cee30 0x7fc4a81529b0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4afe00640 1 --2- 172.21.15.66:0/3716398479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4a806ef30 0x7fc4a81523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4afe00640 1 -- 172.21.15.66:0/3716398479 >> 172.21.15.66:0/3716398479 conn(0x7fc4a80cfaa0 msgr2=0x7fc4a815bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4afe00640 1 -- 172.21.15.66:0/3716398479 shutdown_connections 2024-09-10T22:12:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.179+0000 7fc4afe00640 1 -- 172.21.15.66:0/3716398479 wait complete. 2024-09-10T22:12:46.207 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-10T22:12:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.351+0000 7f8977200640 1 -- 172.21.15.66:0/4182303560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970072d50 msgr2=0x7f89700cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.351+0000 7f8977200640 1 --2- 172.21.15.66:0/4182303560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970072d50 0x7f89700cbfa0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8970069e40 tx=0x7f896801cc30 comp rx=0 tx=0).stop 2024-09-10T22:12:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.351+0000 7f8977200640 1 -- 172.21.15.66:0/4182303560 shutdown_connections 2024-09-10T22:12:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.351+0000 7f8977200640 1 --2- 172.21.15.66:0/4182303560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970072d50 0x7f89700cbfa0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.351+0000 7f8977200640 1 --2- 172.21.15.66:0/4182303560 >> v2:172.21.15.94:3300/0 conn(0x7f897015ab00 0x7f897006e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.351+0000 7f8977200640 1 -- 172.21.15.66:0/4182303560 >> 172.21.15.66:0/4182303560 conn(0x7f89700cfaa0 msgr2=0x7f89700cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.351+0000 7f8977200640 1 -- 172.21.15.66:0/4182303560 shutdown_connections 2024-09-10T22:12:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.351+0000 7f8977200640 1 -- 172.21.15.66:0/4182303560 wait complete. 2024-09-10T22:12:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8977200640 1 Processor -- start 2024-09-10T22:12:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8977200640 1 -- start start 2024-09-10T22:12:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8977200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8970072d50 0x7f8970148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8977200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89700cce70 0x7f8970158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8977200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f897015ab00 0x7f8970159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8977200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8970167bc0 con 0x7f89700cce70 2024-09-10T22:12:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8977200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8970167a40 con 0x7f897015ab00 2024-09-10T22:12:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8977200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8970167d40 con 0x7f8970072d50 2024-09-10T22:12:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8967e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89700cce70 0x7f8970158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8967e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89700cce70 0x7f8970158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38580/0 (socket says 172.21.15.66:38580) 2024-09-10T22:12:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8967e00640 1 -- 172.21.15.66:0/3563912194 learned_addr learned my addr 172.21.15.66:0/3563912194 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f896fe00640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8970072d50 0x7f8970148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8975200640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f897015ab00 0x7f8970159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8967e00640 1 -- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8970072d50 msgr2=0x7f8970148180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8967e00640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8970072d50 0x7f8970148180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8967e00640 1 -- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f897015ab00 msgr2=0x7f8970159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8967e00640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f897015ab00 0x7f8970159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8967e00640 1 -- 172.21.15.66:0/3563912194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897013efb0 con 0x7f89700cce70 2024-09-10T22:12:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8967e00640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89700cce70 0x7f8970158bb0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f895800c410 tx=0x7f895800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f896cc00640 1 -- 172.21.15.66:0/3563912194 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958017070 con 0x7f89700cce70 2024-09-10T22:12:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f896cc00640 1 -- 172.21.15.66:0/3563912194 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8958003600 con 0x7f89700cce70 2024-09-10T22:12:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8977200640 1 -- 172.21.15.66:0/3563912194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897013f290 con 0x7f89700cce70 2024-09-10T22:12:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f896cc00640 1 -- 172.21.15.66:0/3563912194 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958003a50 con 0x7f89700cce70 2024-09-10T22:12:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.355+0000 7f8977200640 1 -- 172.21.15.66:0/3563912194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8970155550 con 0x7f89700cce70 2024-09-10T22:12:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.359+0000 7f8977200640 1 -- 172.21.15.66:0/3563912194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89700cbfa0 con 0x7f89700cce70 2024-09-10T22:12:46.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.359+0000 7f896cc00640 1 -- 172.21.15.66:0/3563912194 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7f8958003bf0 con 0x7f89700cce70 2024-09-10T22:12:46.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.359+0000 7f896cc00640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f894c074320 0x7f894c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:46.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.359+0000 7f896cc00640 1 -- 172.21.15.66:0/3563912194 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f8958091750 con 0x7f89700cce70 2024-09-10T22:12:46.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.359+0000 7f896fe00640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f894c074320 0x7f894c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:46.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.363+0000 7f896fe00640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f894c074320 0x7f894c0767e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f89600058d0 tx=0x7f8960005d90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:46.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.363+0000 7f896cc00640 1 -- 172.21.15.66:0/3563912194 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f895805e240 con 0x7f89700cce70 2024-09-10T22:12:46.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.571+0000 7f8977200640 1 -- 172.21.15.66:0/3563912194 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f897014c800 con 0x7f894c074320 2024-09-10T22:12:46.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f896cc00640 1 -- 172.21.15.66:0/3563912194 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f897014c800 con 0x7f894c074320 2024-09-10T22:12:46.590 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:46.590 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:12:46.590 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:12:46.590 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ss1" 2024-09-10T22:12:46.590 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:12:46.590 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:12:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f8977200640 1 -- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f894c074320 msgr2=0x7f894c0767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f8977200640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f894c074320 0x7f894c0767e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f89600058d0 tx=0x7f8960005d90 comp rx=0 tx=0).stop 2024-09-10T22:12:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f8977200640 1 -- 172.21.15.66:0/3563912194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89700cce70 msgr2=0x7f8970158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f8977200640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89700cce70 0x7f8970158bb0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f895800c410 tx=0x7f895800c8d0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f8977200640 1 -- 172.21.15.66:0/3563912194 shutdown_connections 2024-09-10T22:12:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f8977200640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f894c074320 0x7f894c0767e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f8977200640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f897015ab00 0x7f8970159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f8977200640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89700cce70 0x7f8970158bb0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f8977200640 1 --2- 172.21.15.66:0/3563912194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8970072d50 0x7f8970148180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f8977200640 1 -- 172.21.15.66:0/3563912194 >> 172.21.15.66:0/3563912194 conn(0x7f89700cfaa0 msgr2=0x7f89700732d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f8977200640 1 -- 172.21.15.66:0/3563912194 shutdown_connections 2024-09-10T22:12:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.587+0000 7f8977200640 1 -- 172.21.15.66:0/3563912194 wait complete. 2024-09-10T22:12:46.611 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-10T22:12:46.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.759+0000 7f9434400640 1 -- 172.21.15.66:0/2953506915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9414003ab0 msgr2=0x7f9414003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:46.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.759+0000 7f9434400640 1 --2- 172.21.15.66:0/2953506915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9414003ab0 0x7f9414003ed0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9420006fc0 tx=0x7f942001cf10 comp rx=0 tx=0).stop 2024-09-10T22:12:46.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.759+0000 7f9432400640 1 -- 172.21.15.66:0/2953506915 reap_dead start 2024-09-10T22:12:46.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.759+0000 7f9434400640 1 -- 172.21.15.66:0/2953506915 shutdown_connections 2024-09-10T22:12:46.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.759+0000 7f9434400640 1 -- 172.21.15.66:0/2953506915 >> 172.21.15.66:0/2953506915 conn(0x7f942c0d1d00 msgr2=0x7f942c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:46.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.759+0000 7f9434400640 1 -- 172.21.15.66:0/2953506915 shutdown_connections 2024-09-10T22:12:46.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.759+0000 7f9434400640 1 -- 172.21.15.66:0/2953506915 wait complete. 2024-09-10T22:12:46.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9434400640 1 Processor -- start 2024-09-10T22:12:46.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9434400640 1 -- start start 2024-09-10T22:12:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9434400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9414003ab0 0x7f942c153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9434400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c1547b0 0x7f942c155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9434400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c1f7090 0x7f942c1f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9434400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f942c0b04c0 con 0x7f942c1f7090 2024-09-10T22:12:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9434400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f942c0b0340 con 0x7f942c1547b0 2024-09-10T22:12:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9434400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f942c0b0640 con 0x7f9414003ab0 2024-09-10T22:12:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9432400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c1f7090 0x7f942c1f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9432400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c1f7090 0x7f942c1f7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38592/0 (socket says 172.21.15.66:38592) 2024-09-10T22:12:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9432400640 1 -- 172.21.15.66:0/1245973638 learned_addr learned my addr 172.21.15.66:0/1245973638 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9431a00640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9414003ab0 0x7f942c153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9431000640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c1547b0 0x7f942c155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:46.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9432400640 1 -- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9414003ab0 msgr2=0x7f942c153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:46.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9432400640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9414003ab0 0x7f942c153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9432400640 1 -- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c1547b0 msgr2=0x7f942c155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:46.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9432400640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c1547b0 0x7f942c155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:46.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9432400640 1 -- 172.21.15.66:0/1245973638 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c0cb170 con 0x7f942c1f7090 2024-09-10T22:12:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9432400640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c1f7090 0x7f942c1f7510 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f942c0b3730 tx=0x7f942400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9429600640 1 -- 172.21.15.66:0/1245973638 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424003d90 con 0x7f942c1f7090 2024-09-10T22:12:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9429600640 1 -- 172.21.15.66:0/1245973638 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9424004b30 con 0x7f942c1f7090 2024-09-10T22:12:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9429600640 1 -- 172.21.15.66:0/1245973638 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424004030 con 0x7f942c1f7090 2024-09-10T22:12:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9434400640 1 -- 172.21.15.66:0/1245973638 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942c1f7a50 con 0x7f942c1f7090 2024-09-10T22:12:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.763+0000 7f9434400640 1 -- 172.21.15.66:0/1245973638 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942c152c40 con 0x7f942c1f7090 2024-09-10T22:12:46.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.767+0000 7f9434400640 1 -- 172.21.15.66:0/1245973638 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942c0af390 con 0x7f942c1f7090 2024-09-10T22:12:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.767+0000 7f9429600640 1 -- 172.21.15.66:0/1245973638 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7f9424018030 con 0x7f942c1f7090 2024-09-10T22:12:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.767+0000 7f9429600640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f940c074320 0x7f940c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.767+0000 7f9429600640 1 -- 172.21.15.66:0/1245973638 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f9424092cc0 con 0x7f942c1f7090 2024-09-10T22:12:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.767+0000 7f9431a00640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f940c074320 0x7f940c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:46.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.767+0000 7f9431a00640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f940c074320 0x7f940c0767e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f94200ab000 tx=0x7f9420019d50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:46.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.771+0000 7f9429600640 1 -- 172.21.15.66:0/1245973638 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f942c1f7a50 con 0x7f942c1f7090 2024-09-10T22:12:46.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:46.975+0000 7f9434400640 1 -- 172.21.15.66:0/1245973638 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f942c165a60 con 0x7f940c074320 2024-09-10T22:12:47.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9429600640 1 -- 172.21.15.66:0/1245973638 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f942c165a60 con 0x7f940c074320 2024-09-10T22:12:47.014 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9434400640 1 -- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f940c074320 msgr2=0x7f940c0767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9434400640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f940c074320 0x7f940c0767e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f94200ab000 tx=0x7f9420019d50 comp rx=0 tx=0).stop 2024-09-10T22:12:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9434400640 1 -- 172.21.15.66:0/1245973638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c1f7090 msgr2=0x7f942c1f7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9434400640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c1f7090 0x7f942c1f7510 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f942c0b3730 tx=0x7f942400f790 comp rx=0 tx=0).stop 2024-09-10T22:12:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9434400640 1 -- 172.21.15.66:0/1245973638 shutdown_connections 2024-09-10T22:12:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9434400640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f940c074320 0x7f940c0767e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9434400640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c1f7090 0x7f942c1f7510 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9434400640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c1547b0 0x7f942c155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9434400640 1 --2- 172.21.15.66:0/1245973638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9414003ab0 0x7f942c153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9434400640 1 -- 172.21.15.66:0/1245973638 >> 172.21.15.66:0/1245973638 conn(0x7f942c0d1d00 msgr2=0x7f942c164e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9434400640 1 -- 172.21.15.66:0/1245973638 shutdown_connections 2024-09-10T22:12:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.011+0000 7f9434400640 1 -- 172.21.15.66:0/1245973638 wait complete. 2024-09-10T22:12:47.035 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-10T22:12:47.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.179+0000 7f45e8200640 1 -- 172.21.15.66:0/841219106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8004220 msgr2=0x7f45c8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:47.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.179+0000 7f45e8200640 1 --2- 172.21.15.66:0/841219106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8004220 0x7f45c8004810 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009b30 tx=0x7f45cc01d200 comp rx=0 tx=0).stop 2024-09-10T22:12:47.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.179+0000 7f45e6200640 1 -- 172.21.15.66:0/841219106 reap_dead start 2024-09-10T22:12:47.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.179+0000 7f45e8200640 1 -- 172.21.15.66:0/841219106 shutdown_connections 2024-09-10T22:12:47.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.179+0000 7f45e8200640 1 -- 172.21.15.66:0/841219106 >> 172.21.15.66:0/841219106 conn(0x7f45e00d1c60 msgr2=0x7f45e00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:47.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.179+0000 7f45e8200640 1 -- 172.21.15.66:0/841219106 shutdown_connections 2024-09-10T22:12:47.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.179+0000 7f45e8200640 1 -- 172.21.15.66:0/841219106 wait complete. 2024-09-10T22:12:47.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e8200640 1 Processor -- start 2024-09-10T22:12:47.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e8200640 1 -- start start 2024-09-10T22:12:47.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e8200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8004220 0x7f45e00c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e8200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45e00ce190 0x7f45e00ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e8200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e01599f0 0x7f45e0159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e8200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0168030 con 0x7f45c8004220 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e8200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0167eb0 con 0x7f45e01599f0 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e8200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f45e01681b0 con 0x7f45e00ce190 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8004220 0x7f45e00c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8004220 0x7f45e00c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38612/0 (socket says 172.21.15.66:38612) 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e5800640 1 -- 172.21.15.66:0/770800263 learned_addr learned my addr 172.21.15.66:0/770800263 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e6200640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e01599f0 0x7f45e0159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e4e00640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45e00ce190 0x7f45e00ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e5800640 1 -- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45e00ce190 msgr2=0x7f45e00ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e5800640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45e00ce190 0x7f45e00ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e5800640 1 -- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e01599f0 msgr2=0x7f45e0159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:47.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e5800640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e01599f0 0x7f45e0159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e5800640 1 -- 172.21.15.66:0/770800263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e00cd2f0 con 0x7f45c8004220 2024-09-10T22:12:47.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e4e00640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45e00ce190 0x7f45e00ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:47.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e5800640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8004220 0x7f45e00c7d20 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f45d4003f50 tx=0x7f45d4001870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:47.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45dd600640 1 -- 172.21.15.66:0/770800263 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4011700 con 0x7f45c8004220 2024-09-10T22:12:47.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45dd600640 1 -- 172.21.15.66:0/770800263 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45d4016020 con 0x7f45c8004220 2024-09-10T22:12:47.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e8200640 1 -- 172.21.15.66:0/770800263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e00cd5d0 con 0x7f45c8004220 2024-09-10T22:12:47.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45dd600640 1 -- 172.21.15.66:0/770800263 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4011b40 con 0x7f45c8004220 2024-09-10T22:12:47.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.183+0000 7f45e8200640 1 -- 172.21.15.66:0/770800263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e01f92a0 con 0x7f45c8004220 2024-09-10T22:12:47.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.187+0000 7f45e8200640 1 -- 172.21.15.66:0/770800263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e00cda80 con 0x7f45c8004220 2024-09-10T22:12:47.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.187+0000 7f45dd600640 1 -- 172.21.15.66:0/770800263 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 41) ==== 99046+0+0 (secure 0 0 0) 0x7f45d4011ce0 con 0x7f45c8004220 2024-09-10T22:12:47.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.187+0000 7f45dd600640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45c0074320 0x7f45c00767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:47.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.187+0000 7f45dd600640 1 -- 172.21.15.66:0/770800263 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f45d4092d90 con 0x7f45c8004220 2024-09-10T22:12:47.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.187+0000 7f45e4e00640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45c0074320 0x7f45c00767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:47.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.191+0000 7f45e4e00640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45c0074320 0x7f45c00767e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0ae000 tx=0x7f45cc019fa0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:47.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.195+0000 7f45dd600640 1 -- 172.21.15.66:0/770800263 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f45e00cda80 con 0x7f45c8004220 2024-09-10T22:12:47.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.199+0000 7f45dd600640 1 -- 172.21.15.66:0/770800263 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f45d405ba40 con 0x7f45c8004220 2024-09-10T22:12:47.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.395+0000 7f45e8200640 1 -- 172.21.15.66:0/770800263 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f45e015fb70 con 0x7f45c0074320 2024-09-10T22:12:47.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.411+0000 7f45dd600640 1 -- 172.21.15.66:0/770800263 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f45e015fb70 con 0x7f45c0074320 2024-09-10T22:12:47.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.415+0000 7f45e8200640 1 -- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45c0074320 msgr2=0x7f45c00767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.415+0000 7f45e8200640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45c0074320 0x7f45c00767e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0ae000 tx=0x7f45cc019fa0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.415+0000 7f45e8200640 1 -- 172.21.15.66:0/770800263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8004220 msgr2=0x7f45e00c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.415+0000 7f45e8200640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8004220 0x7f45e00c7d20 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f45d4003f50 tx=0x7f45d4001870 comp rx=0 tx=0).stop 2024-09-10T22:12:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.415+0000 7f45e8200640 1 -- 172.21.15.66:0/770800263 shutdown_connections 2024-09-10T22:12:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.415+0000 7f45e8200640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45c0074320 0x7f45c00767e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.415+0000 7f45e8200640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e01599f0 0x7f45e0159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.415+0000 7f45e8200640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45e00ce190 0x7f45e00ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.415+0000 7f45e8200640 1 --2- 172.21.15.66:0/770800263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8004220 0x7f45e00c7d20 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.415+0000 7f45e8200640 1 -- 172.21.15.66:0/770800263 >> 172.21.15.66:0/770800263 conn(0x7f45e00d1c60 msgr2=0x7f45e015af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.415+0000 7f45e8200640 1 -- 172.21.15.66:0/770800263 shutdown_connections 2024-09-10T22:12:47.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.415+0000 7f45e8200640 1 -- 172.21.15.66:0/770800263 wait complete. 2024-09-10T22:12:47.440 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone9531 2024-09-10T22:12:47.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.587+0000 7fcad8e00640 1 -- 172.21.15.66:0/821081822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40cf1f0 msgr2=0x7fcad40cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:47.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.587+0000 7fcad8e00640 1 --2- 172.21.15.66:0/821081822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40cf1f0 0x7fcad40cff40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fcad40d0ab0 tx=0x7fcacc01cd80 comp rx=0 tx=0).stop 2024-09-10T22:12:47.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.587+0000 7fcad8e00640 1 -- 172.21.15.66:0/821081822 shutdown_connections 2024-09-10T22:12:47.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.587+0000 7fcad8e00640 1 --2- 172.21.15.66:0/821081822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40cf1f0 0x7fcad40cff40 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.587+0000 7fcad8e00640 1 --2- 172.21.15.66:0/821081822 >> v2:172.21.15.94:3300/0 conn(0x7fcad40ce8d0 0x7fcad40cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.587+0000 7fcad8e00640 1 -- 172.21.15.66:0/821081822 >> 172.21.15.66:0/821081822 conn(0x7fcad40d1c60 msgr2=0x7fcad40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:47.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.587+0000 7fcad8e00640 1 -- 172.21.15.66:0/821081822 shutdown_connections 2024-09-10T22:12:47.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.587+0000 7fcad8e00640 1 -- 172.21.15.66:0/821081822 wait complete. 2024-09-10T22:12:47.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad8e00640 1 Processor -- start 2024-09-10T22:12:47.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad8e00640 1 -- start start 2024-09-10T22:12:47.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad8e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40cdf90 0x7fcad41500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:47.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad8e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcad40ce8d0 0x7fcad40c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:47.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad40cf1f0 0x7fcad40c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:47.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad8e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41691a0 con 0x7fcad40cdf90 2024-09-10T22:12:47.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad8e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4169020 con 0x7fcad40cf1f0 2024-09-10T22:12:47.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad8e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcad4169320 con 0x7fcad40ce8d0 2024-09-10T22:12:47.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40cdf90 0x7fcad41500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:47.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40cdf90 0x7fcad41500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38620/0 (socket says 172.21.15.66:38620) 2024-09-10T22:12:47.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad1e00640 1 -- 172.21.15.66:0/1847581171 learned_addr learned my addr 172.21.15.66:0/1847581171 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:47.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad2800640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad40cf1f0 0x7fcad40c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:47.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad1e00640 1 -- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcad40ce8d0 msgr2=0x7fcad40c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:47.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcacbe00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcad40ce8d0 0x7fcad40c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad1e00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcad40ce8d0 0x7fcad40c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad1e00640 1 -- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad40cf1f0 msgr2=0x7fcad40c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad1e00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad40cf1f0 0x7fcad40c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad1e00640 1 -- 172.21.15.66:0/1847581171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad40cb060 con 0x7fcad40cdf90 2024-09-10T22:12:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad2800640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad40cf1f0 0x7fcad40c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:12:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcacbe00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcad40ce8d0 0x7fcad40c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:12:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad1e00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40cdf90 0x7fcad41500e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fcac400e9f0 tx=0x7fcac400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcaca000640 1 -- 172.21.15.66:0/1847581171 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac400cd80 con 0x7fcad40cdf90 2024-09-10T22:12:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcaca000640 1 -- 172.21.15.66:0/1847581171 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcac4010430 con 0x7fcad40cdf90 2024-09-10T22:12:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcaca000640 1 -- 172.21.15.66:0/1847581171 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4010860 con 0x7fcad40cdf90 2024-09-10T22:12:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad8e00640 1 -- 172.21.15.66:0/1847581171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad41513b0 con 0x7fcad40cdf90 2024-09-10T22:12:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.591+0000 7fcad8e00640 1 -- 172.21.15.66:0/1847581171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad4151970 con 0x7fcad40cdf90 2024-09-10T22:12:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.595+0000 7fcad8e00640 1 -- 172.21.15.66:0/1847581171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad40cecb0 con 0x7fcad40cdf90 2024-09-10T22:12:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.595+0000 7fcaca000640 1 -- 172.21.15.66:0/1847581171 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7fcac4010a00 con 0x7fcad40cdf90 2024-09-10T22:12:47.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.595+0000 7fcaca000640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcab00743e0 0x7fcab00768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:47.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.595+0000 7fcaca000640 1 -- 172.21.15.66:0/1847581171 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fcac4092e10 con 0x7fcad40cdf90 2024-09-10T22:12:47.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.595+0000 7fcacbe00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcab00743e0 0x7fcab00768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:47.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.599+0000 7fcacbe00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcab00743e0 0x7fcab00768a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcad40c8ce0 tx=0x7fcabc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:47.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.599+0000 7fcaca000640 1 -- 172.21.15.66:0/1847581171 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcac4060010 con 0x7fcad40cdf90 2024-09-10T22:12:47.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.811+0000 7fcad8e00640 1 -- 172.21.15.66:0/1847581171 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone9531", "target": ["mon-mgr", ""]}) -- 0x7fcad4073080 con 0x7fcab00743e0 2024-09-10T22:12:47.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.827+0000 7fcaca000640 1 -- 172.21.15.66:0/1847581171 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcad4073080 con 0x7fcab00743e0 2024-09-10T22:12:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.831+0000 7fcad8e00640 1 -- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcab00743e0 msgr2=0x7fcab00768a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.831+0000 7fcad8e00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcab00743e0 0x7fcab00768a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcad40c8ce0 tx=0x7fcabc009040 comp rx=0 tx=0).stop 2024-09-10T22:12:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.831+0000 7fcad8e00640 1 -- 172.21.15.66:0/1847581171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40cdf90 msgr2=0x7fcad41500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.831+0000 7fcad8e00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40cdf90 0x7fcad41500e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fcac400e9f0 tx=0x7fcac400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.831+0000 7fcad8e00640 1 -- 172.21.15.66:0/1847581171 shutdown_connections 2024-09-10T22:12:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.831+0000 7fcad8e00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcab00743e0 0x7fcab00768a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.831+0000 7fcad8e00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad40cf1f0 0x7fcad40c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.831+0000 7fcad8e00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcad40ce8d0 0x7fcad40c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.831+0000 7fcad8e00640 1 --2- 172.21.15.66:0/1847581171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40cdf90 0x7fcad41500e0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.831+0000 7fcad8e00640 1 -- 172.21.15.66:0/1847581171 >> 172.21.15.66:0/1847581171 conn(0x7fcad40d1c60 msgr2=0x7fcad40cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.831+0000 7fcad8e00640 1 -- 172.21.15.66:0/1847581171 shutdown_connections 2024-09-10T22:12:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47.831+0000 7fcad8e00640 1 -- 172.21.15.66:0/1847581171 wait complete. 2024-09-10T22:12:47.856 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T22:12:47.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:12:47.915 INFO:teuthology.orchestra.run.smithi066.stdout:2 2024-09-10T22:12:47.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:12:47.922 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:12:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.067+0000 7f6481400640 1 -- 172.21.15.66:0/3136072124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f647c0d06b0 msgr2=0x7f647c0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.067+0000 7f6481400640 1 --2- 172.21.15.66:0/3136072124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f647c0d06b0 0x7f647c0ce170 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6468009840 tx=0x7f646801c6b0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.067+0000 7f6481400640 1 -- 172.21.15.66:0/3136072124 shutdown_connections 2024-09-10T22:12:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.067+0000 7f6481400640 1 --2- 172.21.15.66:0/3136072124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f647c0d06b0 0x7f647c0ce170 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.067+0000 7f6481400640 1 -- 172.21.15.66:0/3136072124 >> 172.21.15.66:0/3136072124 conn(0x7f647c0d1c60 msgr2=0x7f647c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.067+0000 7f6481400640 1 -- 172.21.15.66:0/3136072124 shutdown_connections 2024-09-10T22:12:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.067+0000 7f6481400640 1 -- 172.21.15.66:0/3136072124 wait complete. 2024-09-10T22:12:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6481400640 1 Processor -- start 2024-09-10T22:12:48.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6481400640 1 -- start start 2024-09-10T22:12:48.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6481400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f647c0afc20 0x7f647c14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:48.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6481400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f647c0d06b0 0x7f647c0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:48.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6481400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f647c15bd40 0x7f647c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:48.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6481400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f647c16a160 con 0x7f647c0d06b0 2024-09-10T22:12:48.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6481400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f647c169fe0 con 0x7f647c0afc20 2024-09-10T22:12:48.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6481400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f647c16a2e0 con 0x7f647c15bd40 2024-09-10T22:12:48.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6479e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f647c0d06b0 0x7f647c0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:48.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6479e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f647c0d06b0 0x7f647c0b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38622/0 (socket says 172.21.15.66:38622) 2024-09-10T22:12:48.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6479e00640 1 -- 172.21.15.66:0/366514373 learned_addr learned my addr 172.21.15.66:0/366514373 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:48.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f647b200640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f647c15bd40 0x7f647c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:48.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f647a800640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f647c0afc20 0x7f647c14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:48.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6479e00640 1 -- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f647c15bd40 msgr2=0x7f647c0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:48.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6479e00640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f647c15bd40 0x7f647c0b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6479e00640 1 -- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f647c0afc20 msgr2=0x7f647c14ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:48.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6479e00640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f647c0afc20 0x7f647c14ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6479e00640 1 -- 172.21.15.66:0/366514373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647c15b860 con 0x7f647c0d06b0 2024-09-10T22:12:48.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6479e00640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f647c0d06b0 0x7f647c0b3750 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f646801cb90 tx=0x7f6468005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:48.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.071+0000 7f6472a00640 1 -- 172.21.15.66:0/366514373 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64680be070 con 0x7f647c0d06b0 2024-09-10T22:12:48.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.075+0000 7f6472a00640 1 -- 172.21.15.66:0/366514373 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6468004300 con 0x7f647c0d06b0 2024-09-10T22:12:48.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.075+0000 7f6481400640 1 -- 172.21.15.66:0/366514373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647c15a7c0 con 0x7f647c0d06b0 2024-09-10T22:12:48.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.075+0000 7f6472a00640 1 -- 172.21.15.66:0/366514373 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468004910 con 0x7f647c0d06b0 2024-09-10T22:12:48.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.075+0000 7f6481400640 1 -- 172.21.15.66:0/366514373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647c15ad80 con 0x7f647c0d06b0 2024-09-10T22:12:48.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.075+0000 7f6481400640 1 -- 172.21.15.66:0/366514373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f647c0cffa0 con 0x7f647c0d06b0 2024-09-10T22:12:48.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.075+0000 7f6472a00640 1 -- 172.21.15.66:0/366514373 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f6468004ab0 con 0x7f647c0d06b0 2024-09-10T22:12:48.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.075+0000 7f6472a00640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f645c0743b0 0x7f645c076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:48.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.079+0000 7f647a800640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f645c0743b0 0x7f645c076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:48.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.075+0000 7f6472a00640 1 -- 172.21.15.66:0/366514373 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7f646812c4f0 con 0x7f647c0d06b0 2024-09-10T22:12:48.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.079+0000 7f647a800640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f645c0743b0 0x7f645c076870 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f646c001870 tx=0x7f646c0016a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.083+0000 7f6472a00640 1 -- 172.21.15.66:0/366514373 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f64680ac030 con 0x7f647c0d06b0 2024-09-10T22:12:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.303+0000 7f6481400640 1 -- 172.21.15.66:0/366514373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f647c0b2760 con 0x7f647c0d06b0 2024-09-10T22:12:48.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.307+0000 7f6472a00640 1 -- 172.21.15.66:0/366514373 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v49)=0 v49) ==== 131+0+0 (secure 0 0 0) 0x7f6468016330 con 0x7f647c0d06b0 2024-09-10T22:12:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.311+0000 7f6481400640 1 -- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f645c0743b0 msgr2=0x7f645c076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.311+0000 7f6481400640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f645c0743b0 0x7f645c076870 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f646c001870 tx=0x7f646c0016a0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.311+0000 7f6481400640 1 -- 172.21.15.66:0/366514373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f647c0d06b0 msgr2=0x7f647c0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:48.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.311+0000 7f6481400640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f647c0d06b0 0x7f647c0b3750 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f646801cb90 tx=0x7f6468005760 comp rx=0 tx=0).stop 2024-09-10T22:12:48.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.311+0000 7f6481400640 1 -- 172.21.15.66:0/366514373 shutdown_connections 2024-09-10T22:12:48.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.311+0000 7f6481400640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f645c0743b0 0x7f645c076870 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.311+0000 7f6481400640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f647c15bd40 0x7f647c0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.311+0000 7f6481400640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f647c0d06b0 0x7f647c0b3750 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.311+0000 7f6481400640 1 --2- 172.21.15.66:0/366514373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f647c0afc20 0x7f647c14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.311+0000 7f6481400640 1 -- 172.21.15.66:0/366514373 >> 172.21.15.66:0/366514373 conn(0x7f647c0d1c60 msgr2=0x7f647c0b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:48.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.311+0000 7f6481400640 1 -- 172.21.15.66:0/366514373 shutdown_connections 2024-09-10T22:12:48.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.311+0000 7f6481400640 1 -- 172.21.15.66:0/366514373 wait complete. 2024-09-10T22:12:48.349 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:12:48.349 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:12:48.469 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:12:48.556 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:12:48.557 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:12:48.557 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:12:48.557 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:12:48.566 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:12:48.566 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:12:48.618 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:12:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 -- 172.21.15.66:0/1647865200 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf140afc20 msgr2=0x7fcf140ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 --2- 172.21.15.66:0/1647865200 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf140afc20 0x7fcf140ce6b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009c80 tx=0x7fcf0c01cd10 comp rx=0 tx=0).stop 2024-09-10T22:12:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 -- 172.21.15.66:0/1647865200 shutdown_connections 2024-09-10T22:12:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 --2- 172.21.15.66:0/1647865200 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf140afc20 0x7fcf140ce6b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 --2- 172.21.15.66:0/1647865200 >> v2:172.21.15.94:3300/0 conn(0x7fcf140d06b0 0x7fcf140ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 -- 172.21.15.66:0/1647865200 >> 172.21.15.66:0/1647865200 conn(0x7fcf140d1c60 msgr2=0x7fcf140d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 -- 172.21.15.66:0/1647865200 shutdown_connections 2024-09-10T22:12:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 -- 172.21.15.66:0/1647865200 wait complete. 2024-09-10T22:12:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 Processor -- start 2024-09-10T22:12:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 -- start start 2024-09-10T22:12:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf140afc20 0x7fcf1415ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf140d06b0 0x7fcf1414d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf1415bd40 0x7fcf140b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141692d0 con 0x7fcf140afc20 2024-09-10T22:12:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14169150 con 0x7fcf140d06b0 2024-09-10T22:12:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf19600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcf14169450 con 0x7fcf1415bd40 2024-09-10T22:12:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf0a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf140d06b0 0x7fcf1414d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf12a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf140afc20 0x7fcf1415ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf0a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf140d06b0 0x7fcf1414d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:53416/0 (socket says 172.21.15.66:53416) 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf13400640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf1415bd40 0x7fcf140b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.807+0000 7fcf0a000640 1 -- 172.21.15.66:0/3916859096 learned_addr learned my addr 172.21.15.66:0/3916859096 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf0a000640 1 -- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf1415bd40 msgr2=0x7fcf140b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf0a000640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf1415bd40 0x7fcf140b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf0a000640 1 -- 172.21.15.66:0/3916859096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf140afc20 msgr2=0x7fcf1415ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf0a000640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf140afc20 0x7fcf1415ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf0a000640 1 -- 172.21.15.66:0/3916859096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf140b5200 con 0x7fcf140d06b0 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf13400640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf1415bd40 0x7fcf140b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf0a000640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf140d06b0 0x7fcf1414d130 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00e9c0 tx=0x7fcefc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf0b400640 1 -- 172.21.15.66:0/3916859096 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc00cd80 con 0x7fcf140d06b0 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf0b400640 1 -- 172.21.15.66:0/3916859096 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcefc005020 con 0x7fcf140d06b0 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf19600640 1 -- 172.21.15.66:0/3916859096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf140b54e0 con 0x7fcf140d06b0 2024-09-10T22:12:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf0b400640 1 -- 172.21.15.66:0/3916859096 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc005470 con 0x7fcf140d06b0 2024-09-10T22:12:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf19600640 1 -- 172.21.15.66:0/3916859096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf140b5aa0 con 0x7fcf140d06b0 2024-09-10T22:12:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf19600640 1 -- 172.21.15.66:0/3916859096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf140cffa0 con 0x7fcf140d06b0 2024-09-10T22:12:48.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf0b400640 1 -- 172.21.15.66:0/3916859096 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7fcefc0047b0 con 0x7fcf140d06b0 2024-09-10T22:12:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.811+0000 7fcf0b400640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcef00743b0 0x7fcef0076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.815+0000 7fcf0b400640 1 -- 172.21.15.66:0/3916859096 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5851+0+0 (secure 0 0 0) 0x7fcefc092c50 con 0x7fcf140d06b0 2024-09-10T22:12:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.815+0000 7fcf12a00640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcef00743b0 0x7fcef0076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.815+0000 7fcf12a00640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcef00743b0 0x7fcef0076870 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcf1415a970 tx=0x7fcf04009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:48.819+0000 7fcf0b400640 1 -- 172.21.15.66:0/3916859096 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcefc05f6c0 con 0x7fcf140d06b0 2024-09-10T22:12:49.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.003+0000 7fcf19600640 1 -- 172.21.15.66:0/3916859096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcf140b0df0 con 0x7fcf140d06b0 2024-09-10T22:12:49.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.003+0000 7fcf0b400640 1 -- 172.21.15.66:0/3916859096 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v114) ==== 80+0+0 (secure 0 0 0) 0x7fcefc068430 con 0x7fcf140d06b0 2024-09-10T22:12:49.009 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:12:49.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.007+0000 7fcf19600640 1 -- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcef00743b0 msgr2=0x7fcef0076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:49.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.007+0000 7fcf19600640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcef00743b0 0x7fcef0076870 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcf1415a970 tx=0x7fcf04009040 comp rx=0 tx=0).stop 2024-09-10T22:12:49.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.007+0000 7fcf19600640 1 -- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf140d06b0 msgr2=0x7fcf1414d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:49.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.007+0000 7fcf19600640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf140d06b0 0x7fcf1414d130 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00e9c0 tx=0x7fcefc00ee80 comp rx=0 tx=0).stop 2024-09-10T22:12:49.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.007+0000 7fcf19600640 1 -- 172.21.15.66:0/3916859096 shutdown_connections 2024-09-10T22:12:49.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.007+0000 7fcf19600640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcef00743b0 0x7fcef0076870 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.007+0000 7fcf19600640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf1415bd40 0x7fcf140b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.007+0000 7fcf19600640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf140d06b0 0x7fcf1414d130 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.007+0000 7fcf19600640 1 --2- 172.21.15.66:0/3916859096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf140afc20 0x7fcf1415ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.007+0000 7fcf19600640 1 -- 172.21.15.66:0/3916859096 >> 172.21.15.66:0/3916859096 conn(0x7fcf140d1c60 msgr2=0x7fcf14165b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:49.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.007+0000 7fcf19600640 1 -- 172.21.15.66:0/3916859096 shutdown_connections 2024-09-10T22:12:49.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.007+0000 7fcf19600640 1 -- 172.21.15.66:0/3916859096 wait complete. 2024-09-10T22:12:49.033 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-09-10T22:12:49.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:12:49.034 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T22:12:49.054 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:12:49.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:12:49.055 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:12:49.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[50091]: fuse finished with error 0 and tester_r 0 2024-09-10T22:12:49.188 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:12:49.188 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:12:49.188 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:12:49.197 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:12:49.197 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:12:49.197 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:12:49.197 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:12:49.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:12:49.248 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:12:49.248 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:12:49.248 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:12:49.263 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:12:49.263 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:12:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.535+0000 7f1e1b200640 1 -- 172.21.15.66:0/32774367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1df80053d0 msgr2=0x7f1df800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.535+0000 7f1e1b200640 1 --2- 172.21.15.66:0/32774367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1df80053d0 0x7f1df800b9d0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0099c0 tx=0x7f1e0c01f3c0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.535+0000 7f1e1b200640 1 -- 172.21.15.66:0/32774367 shutdown_connections 2024-09-10T22:12:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.535+0000 7f1e1b200640 1 --2- 172.21.15.66:0/32774367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1df80053d0 0x7f1df800b9d0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.535+0000 7f1e1b200640 1 -- 172.21.15.66:0/32774367 >> 172.21.15.66:0/32774367 conn(0x7f1e140cfa30 msgr2=0x7f1e140cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.539+0000 7f1e1b200640 1 -- 172.21.15.66:0/32774367 shutdown_connections 2024-09-10T22:12:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.539+0000 7f1e1b200640 1 -- 172.21.15.66:0/32774367 wait complete. 2024-09-10T22:12:49.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.539+0000 7f1e1b200640 1 Processor -- start 2024-09-10T22:12:49.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.539+0000 7f1e1b200640 1 -- start start 2024-09-10T22:12:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.539+0000 7f1e1b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1df80053d0 0x7f1e14148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.539+0000 7f1e1b200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e140b43a0 0x7f1e141fc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.539+0000 7f1e1b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140b4800 0x7f1e141fc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.539+0000 7f1e1b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1e140ae8e0 con 0x7f1df80053d0 2024-09-10T22:12:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.539+0000 7f1e1b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e140af0d0 con 0x7f1e140b4800 2024-09-10T22:12:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.539+0000 7f1e1b200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1e140aea60 con 0x7f1e140b43a0 2024-09-10T22:12:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e19200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140b4800 0x7f1e141fc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e13e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1df80053d0 0x7f1e14148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e19200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140b4800 0x7f1e141fc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:53436/0 (socket says 172.21.15.66:53436) 2024-09-10T22:12:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e19200640 1 -- 172.21.15.66:0/2917886222 learned_addr learned my addr 172.21.15.66:0/2917886222 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e13400640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e140b43a0 0x7f1e141fc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e13e00640 1 -- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e140b43a0 msgr2=0x7f1e141fc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:49.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e13e00640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e140b43a0 0x7f1e141fc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e13e00640 1 -- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140b4800 msgr2=0x7f1e141fc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:49.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e13e00640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140b4800 0x7f1e141fc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e13e00640 1 -- 172.21.15.66:0/2917886222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e140b4d60 con 0x7f1df80053d0 2024-09-10T22:12:49.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e13e00640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1df80053d0 0x7f1e14148b30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f1e04003e50 tx=0x7f1e04001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:49.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e0be00640 1 -- 172.21.15.66:0/2917886222 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04015070 con 0x7f1df80053d0 2024-09-10T22:12:49.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e1b200640 1 -- 172.21.15.66:0/2917886222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e140b5040 con 0x7f1df80053d0 2024-09-10T22:12:49.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e0be00640 1 -- 172.21.15.66:0/2917886222 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e04011840 con 0x7f1df80053d0 2024-09-10T22:12:49.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e1b200640 1 -- 172.21.15.66:0/2917886222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e140c85f0 con 0x7f1df80053d0 2024-09-10T22:12:49.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.543+0000 7f1e0be00640 1 -- 172.21.15.66:0/2917886222 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04011c90 con 0x7f1df80053d0 2024-09-10T22:12:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.547+0000 7f1e1b200640 1 -- 172.21.15.66:0/2917886222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e140c8140 con 0x7f1df80053d0 2024-09-10T22:12:49.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.547+0000 7f1e0be00640 1 -- 172.21.15.66:0/2917886222 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f1e040233a0 con 0x7f1df80053d0 2024-09-10T22:12:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.551+0000 7f1e0be00640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1df40743b0 0x7f1df4076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.551+0000 7f1e0be00640 1 -- 172.21.15.66:0/2917886222 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f1e040932a0 con 0x7f1df80053d0 2024-09-10T22:12:49.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.551+0000 7f1e13400640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1df40743b0 0x7f1df4076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:49.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.551+0000 7f1e0be00640 1 -- 172.21.15.66:0/2917886222 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1e0405fd90 con 0x7f1df80053d0 2024-09-10T22:12:49.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.555+0000 7f1e13400640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1df40743b0 0x7f1df4076870 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1e00001ec0 tx=0x7f1e000016f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 -- 172.21.15.66:0/2917886222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e14070420 con 0x7f1df80053d0 2024-09-10T22:12:49.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e0be00640 1 -- 172.21.15.66:0/2917886222 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5222 (secure 0 0 0) 0x7f1e040647c0 con 0x7f1df80053d0 2024-09-10T22:12:49.784 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:49.784 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":77,"btime":"2024-09-10T22:12:01:218274+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1769669365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1769669365},{"type":"v1","addr":"172.21.15.94:6836","nonce":1769669365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8301,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/298680158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":298680158},{"type":"v1","addr":"172.21.15.66:6835","nonce":298680158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1808370701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1808370701},{"type":"v1","addr":"172.21.15.66:6834","nonce":1808370701}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:11:47.579843+0000","modified":"2024-09-10T22:11:50.583212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8307":{"gid":8307,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/3818206155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3818206155},{"type":"v1","addr":"172.21.15.94:6837","nonce":3818206155}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8307,"qdb_cluster":[8307]},"id":8}]} 2024-09-10T22:12:49.785 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 77 2024-09-10T22:12:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 -- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1df40743b0 msgr2=0x7f1df4076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1df40743b0 0x7f1df4076870 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1e00001ec0 tx=0x7f1e000016f0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 -- 172.21.15.66:0/2917886222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1df80053d0 msgr2=0x7f1e14148b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1df80053d0 0x7f1e14148b30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f1e04003e50 tx=0x7f1e04001640 comp rx=0 tx=0).stop 2024-09-10T22:12:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 -- 172.21.15.66:0/2917886222 shutdown_connections 2024-09-10T22:12:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1df40743b0 0x7f1df4076870 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140b4800 0x7f1e141fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e140b43a0 0x7f1e141fc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 --2- 172.21.15.66:0/2917886222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1df80053d0 0x7f1e14148b30 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 -- 172.21.15.66:0/2917886222 >> 172.21.15.66:0/2917886222 conn(0x7f1e140cfa30 msgr2=0x7f1e141578a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 -- 172.21.15.66:0/2917886222 shutdown_connections 2024-09-10T22:12:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.779+0000 7f1e1b200640 1 -- 172.21.15.66:0/2917886222 wait complete. 2024-09-10T22:12:49.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:12:49.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.951+0000 7f642d200640 1 -- 172.21.15.66:0/2428071079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c004810 msgr2=0x7f640c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:49.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.951+0000 7f642d200640 1 --2- 172.21.15.66:0/2428071079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c004810 0x7f640c004c90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6410006f90 tx=0x7f641001cee0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.951+0000 7f6426800640 1 -- 172.21.15.66:0/2428071079 reap_dead start 2024-09-10T22:12:49.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.951+0000 7f642d200640 1 -- 172.21.15.66:0/2428071079 shutdown_connections 2024-09-10T22:12:49.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.951+0000 7f642d200640 1 -- 172.21.15.66:0/2428071079 >> 172.21.15.66:0/2428071079 conn(0x7f64280cfa30 msgr2=0x7f64280cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:49.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.951+0000 7f642d200640 1 -- 172.21.15.66:0/2428071079 shutdown_connections 2024-09-10T22:12:49.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.951+0000 7f642d200640 1 -- 172.21.15.66:0/2428071079 wait complete. 2024-09-10T22:12:49.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f642d200640 1 Processor -- start 2024-09-10T22:12:49.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f642d200640 1 -- start start 2024-09-10T22:12:49.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f642d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c004810 0x7f64281fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:49.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f642d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64280c9730 0x7f6428145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f642d200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64281491f0 0x7f6428073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f642d200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6428167180 con 0x7f64280c9730 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f642d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f642806ed40 con 0x7f640c004810 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f642d200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6428167300 con 0x7f64281491f0 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f641fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64280c9730 0x7f6428145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f6425e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c004810 0x7f64281fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f6425e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c004810 0x7f64281fbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:53448/0 (socket says 172.21.15.66:53448) 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f641fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64280c9730 0x7f6428145bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38658/0 (socket says 172.21.15.66:38658) 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f6425e00640 1 -- 172.21.15.66:0/2967312307 learned_addr learned my addr 172.21.15.66:0/2967312307 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f6426800640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64281491f0 0x7f6428073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f6425e00640 1 -- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64281491f0 msgr2=0x7f6428073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f6425e00640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64281491f0 0x7f6428073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f6425e00640 1 -- 172.21.15.66:0/2967312307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64280c9730 msgr2=0x7f6428145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:49.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f6425e00640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64280c9730 0x7f6428145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f6425e00640 1 -- 172.21.15.66:0/2967312307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64280c9c80 con 0x7f640c004810 2024-09-10T22:12:49.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f6425e00640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c004810 0x7f64281fbb80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6418003f20 tx=0x7f6418001700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:49.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f641e000640 1 -- 172.21.15.66:0/2967312307 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418011640 con 0x7f640c004810 2024-09-10T22:12:49.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f641e000640 1 -- 172.21.15.66:0/2967312307 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6418016020 con 0x7f640c004810 2024-09-10T22:12:49.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f641e000640 1 -- 172.21.15.66:0/2967312307 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418011a80 con 0x7f640c004810 2024-09-10T22:12:49.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f642d200640 1 -- 172.21.15.66:0/2967312307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6428148710 con 0x7f640c004810 2024-09-10T22:12:49.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f642d200640 1 -- 172.21.15.66:0/2967312307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6428148cd0 con 0x7f640c004810 2024-09-10T22:12:49.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.955+0000 7f642d200640 1 -- 172.21.15.66:0/2967312307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64280c48b0 con 0x7f640c004810 2024-09-10T22:12:49.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.959+0000 7f641e000640 1 -- 172.21.15.66:0/2967312307 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f6418011c20 con 0x7f640c004810 2024-09-10T22:12:49.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.959+0000 7f641e000640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f64040743b0 0x7f6404076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:49.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.959+0000 7f641e000640 1 -- 172.21.15.66:0/2967312307 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f6418092de0 con 0x7f640c004810 2024-09-10T22:12:49.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.959+0000 7f641fe00640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f64040743b0 0x7f6404076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:49.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.959+0000 7f641fe00640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f64040743b0 0x7f6404076870 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f64100ab000 tx=0x7f6410019d20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:49.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:49.963+0000 7f641e000640 1 -- 172.21.15.66:0/2967312307 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6418020020 con 0x7f640c004810 2024-09-10T22:12:50.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 -- 172.21.15.66:0/2967312307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6428070420 con 0x7f640c004810 2024-09-10T22:12:50.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f641e000640 1 -- 172.21.15.66:0/2967312307 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5222 (secure 0 0 0) 0x7f641805f8d0 con 0x7f640c004810 2024-09-10T22:12:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:50.208 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":77,"btime":"2024-09-10T22:12:01:218274+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1769669365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1769669365},{"type":"v1","addr":"172.21.15.94:6836","nonce":1769669365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8301,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/298680158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":298680158},{"type":"v1","addr":"172.21.15.66:6835","nonce":298680158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1808370701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1808370701},{"type":"v1","addr":"172.21.15.66:6834","nonce":1808370701}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:11:47.579843+0000","modified":"2024-09-10T22:11:50.583212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8307":{"gid":8307,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/3818206155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3818206155},{"type":"v1","addr":"172.21.15.94:6837","nonce":3818206155}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8307,"qdb_cluster":[8307]},"id":8}]} 2024-09-10T22:12:50.208 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 77 2024-09-10T22:12:50.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 -- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f64040743b0 msgr2=0x7f6404076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f64040743b0 0x7f6404076870 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f64100ab000 tx=0x7f6410019d20 comp rx=0 tx=0).stop 2024-09-10T22:12:50.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 -- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c004810 msgr2=0x7f64281fbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c004810 0x7f64281fbb80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6418003f20 tx=0x7f6418001700 comp rx=0 tx=0).stop 2024-09-10T22:12:50.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 -- 172.21.15.66:0/2967312307 shutdown_connections 2024-09-10T22:12:50.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f64040743b0 0x7f6404076870 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64281491f0 0x7f6428073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64280c9730 0x7f6428145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 --2- 172.21.15.66:0/2967312307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c004810 0x7f64281fbb80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 -- 172.21.15.66:0/2967312307 >> 172.21.15.66:0/2967312307 conn(0x7f64280cfa30 msgr2=0x7f64281584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:50.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 -- 172.21.15.66:0/2967312307 shutdown_connections 2024-09-10T22:12:50.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.203+0000 7f642d200640 1 -- 172.21.15.66:0/2967312307 wait complete. 2024-09-10T22:12:50.235 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:12:50.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.451+0000 7f07e6200640 1 -- 172.21.15.66:0/2533111235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07c4004810 msgr2=0x7f07c4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.451+0000 7f07e6200640 1 --2- 172.21.15.66:0/2533111235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07c4004810 0x7f07c4004c90 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f07c80017d0 tx=0x7f07c801c890 comp rx=0 tx=0).stop 2024-09-10T22:12:50.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.451+0000 7f07dfe00640 1 -- 172.21.15.66:0/2533111235 reap_dead start 2024-09-10T22:12:50.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.451+0000 7f07e6200640 1 -- 172.21.15.66:0/2533111235 shutdown_connections 2024-09-10T22:12:50.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.451+0000 7f07e6200640 1 -- 172.21.15.66:0/2533111235 >> 172.21.15.66:0/2533111235 conn(0x7f07e00d1c60 msgr2=0x7f07e00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:50.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.451+0000 7f07e6200640 1 -- 172.21.15.66:0/2533111235 shutdown_connections 2024-09-10T22:12:50.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.451+0000 7f07e6200640 1 -- 172.21.15.66:0/2533111235 wait complete. 2024-09-10T22:12:50.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07e6200640 1 Processor -- start 2024-09-10T22:12:50.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07e6200640 1 -- start start 2024-09-10T22:12:50.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07e6200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f07c4004810 0x7f07e01fbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07e6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e00ca9d0 0x7f07e015f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07e6200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07e015a290 0x7f07e0158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07e6200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f07e0169e40 con 0x7f07e00ca9d0 2024-09-10T22:12:50.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07e6200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f07e0169cc0 con 0x7f07e015a290 2024-09-10T22:12:50.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07e6200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f07e0169fc0 con 0x7f07c4004810 2024-09-10T22:12:50.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e00ca9d0 0x7f07e015f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07df400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f07c4004810 0x7f07e01fbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07df400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f07c4004810 0x7f07e01fbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:33978/0 (socket says 172.21.15.66:33978) 2024-09-10T22:12:50.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e00ca9d0 0x7f07e015f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38682/0 (socket says 172.21.15.66:38682) 2024-09-10T22:12:50.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07df400640 1 -- 172.21.15.66:0/2145996564 learned_addr learned my addr 172.21.15.66:0/2145996564 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:50.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07dfe00640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07e015a290 0x7f07e0158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07df400640 1 -- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07e015a290 msgr2=0x7f07e0158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07df400640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07e015a290 0x7f07e0158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07df400640 1 -- 172.21.15.66:0/2145996564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e00ca9d0 msgr2=0x7f07e015f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07df400640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e00ca9d0 0x7f07e015f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07df400640 1 -- 172.21.15.66:0/2145996564 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e00cae50 con 0x7f07c4004810 2024-09-10T22:12:50.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07df400640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f07c4004810 0x7f07e01fbd20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f07d0011a10 tx=0x7f07d0011ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:50.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07d7400640 1 -- 172.21.15.66:0/2145996564 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d001e070 con 0x7f07c4004810 2024-09-10T22:12:50.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07d7400640 1 -- 172.21.15.66:0/2145996564 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07d0004050 con 0x7f07c4004810 2024-09-10T22:12:50.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07d7400640 1 -- 172.21.15.66:0/2145996564 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0004440 con 0x7f07c4004810 2024-09-10T22:12:50.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.455+0000 7f07e6200640 1 -- 172.21.15.66:0/2145996564 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e015ff80 con 0x7f07c4004810 2024-09-10T22:12:50.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.459+0000 7f07e6200640 1 -- 172.21.15.66:0/2145996564 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e01fb240 con 0x7f07c4004810 2024-09-10T22:12:50.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.459+0000 7f07e6200640 1 -- 172.21.15.66:0/2145996564 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e006dd30 con 0x7f07c4004810 2024-09-10T22:12:50.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.459+0000 7f07d7400640 1 -- 172.21.15.66:0/2145996564 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f07d0005020 con 0x7f07c4004810 2024-09-10T22:12:50.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.459+0000 7f07d7400640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f07bc0743e0 0x7f07bc0768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.459+0000 7f07d7400640 1 -- 172.21.15.66:0/2145996564 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f07d00931f0 con 0x7f07c4004810 2024-09-10T22:12:50.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.459+0000 7f07d7e00640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f07bc0743e0 0x7f07bc0768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.463+0000 7f07d7e00640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f07bc0743e0 0x7f07bc0768a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f07e006fa80 tx=0x7f07c80b1040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:50.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.463+0000 7f07d7400640 1 -- 172.21.15.66:0/2145996564 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f07d005fd60 con 0x7f07c4004810 2024-09-10T22:12:50.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.643+0000 7f07e6200640 1 -- 172.21.15.66:0/2145996564 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07e0070a90 con 0x7f07c4004810 2024-09-10T22:12:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.643+0000 7f07d7400640 1 -- 172.21.15.66:0/2145996564 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17013 (secure 0 0 0) 0x7f07d0064790 con 0x7f07c4004810 2024-09-10T22:12:50.648 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:50.648 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":115,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:12:49.200878+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:11:45.291701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T22:11:46.544807+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":2}]},{"pgid":"17.34","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:12:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.643+0000 7f07e6200640 1 -- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f07bc0743e0 msgr2=0x7f07bc0768a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.643+0000 7f07e6200640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f07bc0743e0 0x7f07bc0768a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f07e006fa80 tx=0x7f07c80b1040 comp rx=0 tx=0).stop 2024-09-10T22:12:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.643+0000 7f07e6200640 1 -- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f07c4004810 msgr2=0x7f07e01fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.643+0000 7f07e6200640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f07c4004810 0x7f07e01fbd20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f07d0011a10 tx=0x7f07d0011ed0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.643+0000 7f07e6200640 1 -- 172.21.15.66:0/2145996564 shutdown_connections 2024-09-10T22:12:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.643+0000 7f07e6200640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f07bc0743e0 0x7f07bc0768a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.643+0000 7f07e6200640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07e015a290 0x7f07e0158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.643+0000 7f07e6200640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e00ca9d0 0x7f07e015f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.643+0000 7f07e6200640 1 --2- 172.21.15.66:0/2145996564 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f07c4004810 0x7f07e01fbd20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.643+0000 7f07e6200640 1 -- 172.21.15.66:0/2145996564 >> 172.21.15.66:0/2145996564 conn(0x7f07e00d1c60 msgr2=0x7f07e015d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.647+0000 7f07e6200640 1 -- 172.21.15.66:0/2145996564 shutdown_connections 2024-09-10T22:12:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.647+0000 7f07e6200640 1 -- 172.21.15.66:0/2145996564 wait complete. 2024-09-10T22:12:50.670 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:12:50.670 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:12:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.811+0000 7f9ef9000640 1 -- 172.21.15.66:0/3238247380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ef40afc20 msgr2=0x7f9ef40ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.811+0000 7f9ef9000640 1 --2- 172.21.15.66:0/3238247380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ef40afc20 0x7f9ef40ce880 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009bc0 tx=0x7f9eec01cb10 comp rx=0 tx=0).stop 2024-09-10T22:12:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.811+0000 7f9ef9000640 1 -- 172.21.15.66:0/3238247380 shutdown_connections 2024-09-10T22:12:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.811+0000 7f9ef9000640 1 --2- 172.21.15.66:0/3238247380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ef40afc20 0x7f9ef40ce880 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.811+0000 7f9ef9000640 1 -- 172.21.15.66:0/3238247380 >> 172.21.15.66:0/3238247380 conn(0x7f9ef40d1c60 msgr2=0x7f9ef40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.811+0000 7f9ef9000640 1 -- 172.21.15.66:0/3238247380 shutdown_connections 2024-09-10T22:12:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.811+0000 7f9ef9000640 1 -- 172.21.15.66:0/3238247380 wait complete. 2024-09-10T22:12:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef9000640 1 Processor -- start 2024-09-10T22:12:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef9000640 1 -- start start 2024-09-10T22:12:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef9000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ef40afc20 0x7f9ef41f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ef40d06b0 0x7f9ef4163b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef415bd40 0x7f9ef41640a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef9000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef416a1b0 con 0x7f9ef40d06b0 2024-09-10T22:12:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef9000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef416a030 con 0x7f9ef415bd40 2024-09-10T22:12:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef9000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef416a330 con 0x7f9ef40afc20 2024-09-10T22:12:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ee9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ef40d06b0 0x7f9ef4163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ee9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ef40d06b0 0x7f9ef4163b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38714/0 (socket says 172.21.15.66:38714) 2024-09-10T22:12:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ee9400640 1 -- 172.21.15.66:0/3436969538 learned_addr learned my addr 172.21.15.66:0/3436969538 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef2800640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef415bd40 0x7f9ef41640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef1e00640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ef40afc20 0x7f9ef41f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ee9400640 1 -- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ef40afc20 msgr2=0x7f9ef41f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ee9400640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ef40afc20 0x7f9ef41f84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ee9400640 1 -- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef415bd40 msgr2=0x7f9ef41640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ee9400640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef415bd40 0x7f9ef41640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ee9400640 1 -- 172.21.15.66:0/3436969538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef41f8a30 con 0x7f9ef40d06b0 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef2800640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef415bd40 0x7f9ef41640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ee9400640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ef40d06b0 0x7f9ef4163b60 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00c4b0 tx=0x7f9edc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9eeaa00640 1 -- 172.21.15.66:0/3436969538 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc016020 con 0x7f9ef40d06b0 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9eeaa00640 1 -- 172.21.15.66:0/3436969538 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9edc012920 con 0x7f9ef40d06b0 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9eeaa00640 1 -- 172.21.15.66:0/3436969538 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc012d70 con 0x7f9ef40d06b0 2024-09-10T22:12:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef9000640 1 -- 172.21.15.66:0/3436969538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef41659d0 con 0x7f9ef40d06b0 2024-09-10T22:12:50.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef9000640 1 -- 172.21.15.66:0/3436969538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef4165f50 con 0x7f9ef40d06b0 2024-09-10T22:12:50.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.815+0000 7f9ef9000640 1 -- 172.21.15.66:0/3436969538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef40cffa0 con 0x7f9ef40d06b0 2024-09-10T22:12:50.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.819+0000 7f9eeaa00640 1 -- 172.21.15.66:0/3436969538 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f9edc01a3b0 con 0x7f9ef40d06b0 2024-09-10T22:12:50.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.819+0000 7f9eeaa00640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ed00743b0 0x7f9ed0076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.819+0000 7f9eeaa00640 1 -- 172.21.15.66:0/3436969538 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f9edc0922b0 con 0x7f9ef40d06b0 2024-09-10T22:12:50.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.819+0000 7f9ef1e00640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ed00743b0 0x7f9ed0076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.819+0000 7f9ef1e00640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ed00743b0 0x7f9ed0076870 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4001650 tx=0x7f9ee4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:50.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:50.823+0000 7f9eeaa00640 1 -- 172.21.15.66:0/3436969538 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9edc020030 con 0x7f9ef40d06b0 2024-09-10T22:12:51.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.031+0000 7f9ef9000640 1 -- 172.21.15.66:0/3436969538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ef40b0df0 con 0x7f9ef40d06b0 2024-09-10T22:12:51.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.031+0000 7f9eeaa00640 1 -- 172.21.15.66:0/3436969538 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5222 (secure 0 0 0) 0x7f9ef41f8a30 con 0x7f9ef40d06b0 2024-09-10T22:12:51.038 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:51.038 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":77,"btime":"2024-09-10T22:12:01:218274+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1769669365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1769669365},{"type":"v1","addr":"172.21.15.94:6836","nonce":1769669365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8301,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/298680158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":298680158},{"type":"v1","addr":"172.21.15.66:6835","nonce":298680158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1808370701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1808370701},{"type":"v1","addr":"172.21.15.66:6834","nonce":1808370701}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:11:47.579843+0000","modified":"2024-09-10T22:11:50.583212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8307":{"gid":8307,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/3818206155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3818206155},{"type":"v1","addr":"172.21.15.94:6837","nonce":3818206155}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8307,"qdb_cluster":[8307]},"id":8}]} 2024-09-10T22:12:51.038 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 77 2024-09-10T22:12:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.035+0000 7f9ef9000640 1 -- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ed00743b0 msgr2=0x7f9ed0076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.035+0000 7f9ef9000640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ed00743b0 0x7f9ed0076870 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4001650 tx=0x7f9ee4009040 comp rx=0 tx=0).stop 2024-09-10T22:12:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.035+0000 7f9ef9000640 1 -- 172.21.15.66:0/3436969538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ef40d06b0 msgr2=0x7f9ef4163b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.035+0000 7f9ef9000640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ef40d06b0 0x7f9ef4163b60 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00c4b0 tx=0x7f9edc00c970 comp rx=0 tx=0).stop 2024-09-10T22:12:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.035+0000 7f9ef9000640 1 -- 172.21.15.66:0/3436969538 shutdown_connections 2024-09-10T22:12:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.035+0000 7f9ef9000640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9ed00743b0 0x7f9ed0076870 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.035+0000 7f9ef9000640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef415bd40 0x7f9ef41640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.035+0000 7f9ef9000640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ef40d06b0 0x7f9ef4163b60 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.035+0000 7f9ef9000640 1 --2- 172.21.15.66:0/3436969538 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ef40afc20 0x7f9ef41f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.035+0000 7f9ef9000640 1 -- 172.21.15.66:0/3436969538 >> 172.21.15.66:0/3436969538 conn(0x7f9ef40d1c60 msgr2=0x7f9ef406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.035+0000 7f9ef9000640 1 -- 172.21.15.66:0/3436969538 shutdown_connections 2024-09-10T22:12:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.035+0000 7f9ef9000640 1 -- 172.21.15.66:0/3436969538 wait complete. 2024-09-10T22:12:51.060 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:12:51.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.203+0000 7f6611000640 1 -- 172.21.15.66:0/1121644782 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f660c15e010 msgr2=0x7f660c0cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.203+0000 7f6611000640 1 --2- 172.21.15.66:0/1121644782 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f660c15e010 0x7f660c0cfea0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009840 tx=0x7f65fc01c730 comp rx=0 tx=0).stop 2024-09-10T22:12:51.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.203+0000 7f6611000640 1 -- 172.21.15.66:0/1121644782 shutdown_connections 2024-09-10T22:12:51.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.203+0000 7f6611000640 1 --2- 172.21.15.66:0/1121644782 >> v2:172.21.15.94:3300/0 conn(0x7f660c0cebc0 0x7f660c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.203+0000 7f6611000640 1 --2- 172.21.15.66:0/1121644782 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f660c15e010 0x7f660c0cfea0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.203+0000 7f6611000640 1 -- 172.21.15.66:0/1121644782 >> 172.21.15.66:0/1121644782 conn(0x7f660c0d1c60 msgr2=0x7f660c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:51.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.203+0000 7f6611000640 1 -- 172.21.15.66:0/1121644782 shutdown_connections 2024-09-10T22:12:51.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.203+0000 7f6611000640 1 -- 172.21.15.66:0/1121644782 wait complete. 2024-09-10T22:12:51.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6611000640 1 Processor -- start 2024-09-10T22:12:51.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6611000640 1 -- start start 2024-09-10T22:12:51.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6611000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f660c0cebc0 0x7f660c154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6611000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f660c0d03e0 0x7f660c09bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6611000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f660c15e010 0x7f660c155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6611000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f660c16acc0 con 0x7f660c15e010 2024-09-10T22:12:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6611000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f660c06f070 con 0x7f660c0cebc0 2024-09-10T22:12:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6611000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f660c16ae40 con 0x7f660c0d03e0 2024-09-10T22:12:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f660a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f660c15e010 0x7f660c155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f660a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f660c15e010 0x7f660c155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38720/0 (socket says 172.21.15.66:38720) 2024-09-10T22:12:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f660a800640 1 -- 172.21.15.66:0/2295680762 learned_addr learned my addr 172.21.15.66:0/2295680762 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6609e00640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f660c0cebc0 0x7f660c154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6603e00640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f660c0d03e0 0x7f660c09bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f660a800640 1 -- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f660c0d03e0 msgr2=0x7f660c09bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f660a800640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f660c0d03e0 0x7f660c09bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f660a800640 1 -- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f660c0cebc0 msgr2=0x7f660c154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f660a800640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f660c0cebc0 0x7f660c154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f660a800640 1 -- 172.21.15.66:0/2295680762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c156210 con 0x7f660c15e010 2024-09-10T22:12:51.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f660a800640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f660c15e010 0x7f660c155490 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6604009f90 tx=0x7f660400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:51.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6602000640 1 -- 172.21.15.66:0/2295680762 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604003980 con 0x7f660c15e010 2024-09-10T22:12:51.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6602000640 1 -- 172.21.15.66:0/2295680762 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6604003c30 con 0x7f660c15e010 2024-09-10T22:12:51.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6602000640 1 -- 172.21.15.66:0/2295680762 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604004280 con 0x7f660c15e010 2024-09-10T22:12:51.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6611000640 1 -- 172.21.15.66:0/2295680762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660c1564f0 con 0x7f660c15e010 2024-09-10T22:12:51.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.207+0000 7f6611000640 1 -- 172.21.15.66:0/2295680762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f660c0b61e0 con 0x7f660c15e010 2024-09-10T22:12:51.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.211+0000 7f6611000640 1 -- 172.21.15.66:0/2295680762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f660c0b63e0 con 0x7f660c15e010 2024-09-10T22:12:51.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.211+0000 7f6602000640 1 -- 172.21.15.66:0/2295680762 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f660400b040 con 0x7f660c15e010 2024-09-10T22:12:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.215+0000 7f6602000640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65e80743b0 0x7f65e8076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.215+0000 7f6602000640 1 -- 172.21.15.66:0/2295680762 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f6604092b30 con 0x7f660c15e010 2024-09-10T22:12:51.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.215+0000 7f6609e00640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65e80743b0 0x7f65e8076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.215+0000 7f6609e00640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65e80743b0 0x7f65e8076870 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0af000 tx=0x7f65fc01afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:51.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.215+0000 7f6602000640 1 -- 172.21.15.66:0/2295680762 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f660c0b63e0 con 0x7f660c15e010 2024-09-10T22:12:51.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.423+0000 7f6611000640 1 -- 172.21.15.66:0/2295680762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f660c06dd30 con 0x7f660c15e010 2024-09-10T22:12:51.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.423+0000 7f6602000640 1 -- 172.21.15.66:0/2295680762 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5222 (secure 0 0 0) 0x7f6604024020 con 0x7f660c15e010 2024-09-10T22:12:51.430 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:51.430 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":77,"btime":"2024-09-10T22:12:01:218274+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1769669365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1769669365},{"type":"v1","addr":"172.21.15.94:6836","nonce":1769669365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8301,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/298680158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":298680158},{"type":"v1","addr":"172.21.15.66:6835","nonce":298680158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1808370701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1808370701},{"type":"v1","addr":"172.21.15.66:6834","nonce":1808370701}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:11:47.579843+0000","modified":"2024-09-10T22:11:50.583212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8307":{"gid":8307,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/3818206155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3818206155},{"type":"v1","addr":"172.21.15.94:6837","nonce":3818206155}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8307,"qdb_cluster":[8307]},"id":8}]} 2024-09-10T22:12:51.430 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 77 2024-09-10T22:12:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.427+0000 7f6611000640 1 -- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65e80743b0 msgr2=0x7f65e8076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.427+0000 7f6611000640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65e80743b0 0x7f65e8076870 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0af000 tx=0x7f65fc01afd0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.427+0000 7f6611000640 1 -- 172.21.15.66:0/2295680762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f660c15e010 msgr2=0x7f660c155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.427+0000 7f6611000640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f660c15e010 0x7f660c155490 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6604009f90 tx=0x7f660400c800 comp rx=0 tx=0).stop 2024-09-10T22:12:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.427+0000 7f6611000640 1 -- 172.21.15.66:0/2295680762 shutdown_connections 2024-09-10T22:12:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.427+0000 7f6611000640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65e80743b0 0x7f65e8076870 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.427+0000 7f6611000640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f660c15e010 0x7f660c155490 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.427+0000 7f6611000640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f660c0d03e0 0x7f660c09bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.427+0000 7f6611000640 1 --2- 172.21.15.66:0/2295680762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f660c0cebc0 0x7f660c154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.427+0000 7f6611000640 1 -- 172.21.15.66:0/2295680762 >> 172.21.15.66:0/2295680762 conn(0x7f660c0d1c60 msgr2=0x7f660c15b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.427+0000 7f6611000640 1 -- 172.21.15.66:0/2295680762 shutdown_connections 2024-09-10T22:12:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.427+0000 7f6611000640 1 -- 172.21.15.66:0/2295680762 wait complete. 2024-09-10T22:12:51.452 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:12:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 -- 172.21.15.66:0/260635390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53ec003a10 msgr2=0x7f53ec003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 --2- 172.21.15.66:0/260635390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53ec003a10 0x7f53ec003e30 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009b90 tx=0x7f53f801d270 comp rx=0 tx=0).stop 2024-09-10T22:12:51.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f5407e00640 1 -- 172.21.15.66:0/260635390 reap_dead start 2024-09-10T22:12:51.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 -- 172.21.15.66:0/260635390 shutdown_connections 2024-09-10T22:12:51.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 -- 172.21.15.66:0/260635390 >> 172.21.15.66:0/260635390 conn(0x7f54080d3da0 msgr2=0x7f54080d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:51.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 -- 172.21.15.66:0/260635390 shutdown_connections 2024-09-10T22:12:51.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 -- 172.21.15.66:0/260635390 wait complete. 2024-09-10T22:12:51.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 Processor -- start 2024-09-10T22:12:51.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 -- start start 2024-09-10T22:12:51.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53ec003a10 0x7f54081f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540815f280 0x7f540815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f540815b2a0 0x7f54080cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f54080d2710 con 0x7f540815b2a0 2024-09-10T22:12:51.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54080d2590 con 0x7f540815f280 2024-09-10T22:12:51.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.595+0000 7f540d200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5408169570 con 0x7f53ec003a10 2024-09-10T22:12:51.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f5407e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f540815b2a0 0x7f54080cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f5407e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f540815b2a0 0x7f54080cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38734/0 (socket says 172.21.15.66:38734) 2024-09-10T22:12:51.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f5407e00640 1 -- 172.21.15.66:0/29010261 learned_addr learned my addr 172.21.15.66:0/29010261 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:51.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f5407400640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53ec003a10 0x7f54081f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f5406a00640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540815f280 0x7f540815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f5407e00640 1 -- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53ec003a10 msgr2=0x7f54081f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f5407e00640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53ec003a10 0x7f54081f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f5407e00640 1 -- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540815f280 msgr2=0x7f540815be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f5407e00640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540815f280 0x7f540815be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f5407e00640 1 -- 172.21.15.66:0/29010261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54081f8080 con 0x7f540815b2a0 2024-09-10T22:12:51.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f5407e00640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f540815b2a0 0x7f54080cd930 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f540808d170 tx=0x7f5400010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:51.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f53ff400640 1 -- 172.21.15.66:0/29010261 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400003980 con 0x7f540815b2a0 2024-09-10T22:12:51.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f53ff400640 1 -- 172.21.15.66:0/29010261 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5400015070 con 0x7f540815b2a0 2024-09-10T22:12:51.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f53ff400640 1 -- 172.21.15.66:0/29010261 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400004030 con 0x7f540815b2a0 2024-09-10T22:12:51.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f540d200640 1 -- 172.21.15.66:0/29010261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540815f780 con 0x7f540815b2a0 2024-09-10T22:12:51.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f540d200640 1 -- 172.21.15.66:0/29010261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54081f82f0 con 0x7f540815b2a0 2024-09-10T22:12:51.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f540d200640 1 -- 172.21.15.66:0/29010261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54080b0a10 con 0x7f540815b2a0 2024-09-10T22:12:51.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.599+0000 7f53ff400640 1 -- 172.21.15.66:0/29010261 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f54000041d0 con 0x7f540815b2a0 2024-09-10T22:12:51.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.603+0000 7f53ff400640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53e80743b0 0x7f53e8076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.603+0000 7f53ff400640 1 -- 172.21.15.66:0/29010261 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f5400092be0 con 0x7f540815b2a0 2024-09-10T22:12:51.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.603+0000 7f5407400640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53e80743b0 0x7f53e8076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.603+0000 7f5407400640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53e80743b0 0x7f53e8076870 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f53f80ae000 tx=0x7f53f801a010 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:51.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.603+0000 7f53ff400640 1 -- 172.21.15.66:0/29010261 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f540005f6d0 con 0x7f540815b2a0 2024-09-10T22:12:51.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.779+0000 7f540d200640 1 -- 172.21.15.66:0/29010261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54080d2160 con 0x7f540815b2a0 2024-09-10T22:12:51.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.779+0000 7f53ff400640 1 -- 172.21.15.66:0/29010261 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17013 (secure 0 0 0) 0x7f5400064100 con 0x7f540815b2a0 2024-09-10T22:12:51.786 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:51.786 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":115,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:12:49.200878+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:11:45.291701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T22:11:46.544807+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":2}]},{"pgid":"17.34","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:12:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.783+0000 7f540d200640 1 -- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53e80743b0 msgr2=0x7f53e8076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.783+0000 7f540d200640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53e80743b0 0x7f53e8076870 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f53f80ae000 tx=0x7f53f801a010 comp rx=0 tx=0).stop 2024-09-10T22:12:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.783+0000 7f540d200640 1 -- 172.21.15.66:0/29010261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f540815b2a0 msgr2=0x7f54080cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.783+0000 7f540d200640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f540815b2a0 0x7f54080cd930 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f540808d170 tx=0x7f5400010f40 comp rx=0 tx=0).stop 2024-09-10T22:12:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.783+0000 7f540d200640 1 -- 172.21.15.66:0/29010261 shutdown_connections 2024-09-10T22:12:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.783+0000 7f540d200640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f53e80743b0 0x7f53e8076870 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.783+0000 7f540d200640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f540815b2a0 0x7f54080cd930 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.783+0000 7f540d200640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540815f280 0x7f540815be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.783+0000 7f540d200640 1 --2- 172.21.15.66:0/29010261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53ec003a10 0x7f54081f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.783+0000 7f540d200640 1 -- 172.21.15.66:0/29010261 >> 172.21.15.66:0/29010261 conn(0x7f54080d3da0 msgr2=0x7f540815e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.783+0000 7f540d200640 1 -- 172.21.15.66:0/29010261 shutdown_connections 2024-09-10T22:12:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.783+0000 7f540d200640 1 -- 172.21.15.66:0/29010261 wait complete. 2024-09-10T22:12:51.808 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:12:51.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.951+0000 7f7559400640 1 -- 172.21.15.66:0/1160987289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75540b1e10 msgr2=0x7f75540aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.951+0000 7f7559400640 1 --2- 172.21.15.66:0/1160987289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75540b1e10 0x7f75540aec10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f754c009bc0 tx=0x7f754c01d540 comp rx=0 tx=0).stop 2024-09-10T22:12:51.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.951+0000 7f7559400640 1 -- 172.21.15.66:0/1160987289 shutdown_connections 2024-09-10T22:12:51.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.951+0000 7f7559400640 1 --2- 172.21.15.66:0/1160987289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75540b1e10 0x7f75540aec10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.951+0000 7f7559400640 1 --2- 172.21.15.66:0/1160987289 >> v2:172.21.15.94:3300/0 conn(0x7f7554160280 0x7f75540b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.951+0000 7f7559400640 1 -- 172.21.15.66:0/1160987289 >> 172.21.15.66:0/1160987289 conn(0x7f75540d1c60 msgr2=0x7f75540d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:51.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.951+0000 7f7559400640 1 -- 172.21.15.66:0/1160987289 shutdown_connections 2024-09-10T22:12:51.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.951+0000 7f7559400640 1 -- 172.21.15.66:0/1160987289 wait complete. 2024-09-10T22:12:51.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.951+0000 7f7559400640 1 Processor -- start 2024-09-10T22:12:51.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7559400640 1 -- start start 2024-09-10T22:12:51.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7559400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75540b1e10 0x7f75540cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7559400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f755415f450 0x7f7554157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7559400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7554160280 0x7f7554157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7559400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f75540aff30 con 0x7f75540b1e10 2024-09-10T22:12:51.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7559400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f75540afdb0 con 0x7f7554160280 2024-09-10T22:12:51.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7559400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7554168eb0 con 0x7f755415f450 2024-09-10T22:12:51.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7552800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75540b1e10 0x7f75540cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7552800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75540b1e10 0x7f75540cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38744/0 (socket says 172.21.15.66:38744) 2024-09-10T22:12:51.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7552800640 1 -- 172.21.15.66:0/504880550 learned_addr learned my addr 172.21.15.66:0/504880550 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:51.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7553200640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7554160280 0x7f7554157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7551e00640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f755415f450 0x7f7554157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7552800640 1 -- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f755415f450 msgr2=0x7f7554157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7552800640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f755415f450 0x7f7554157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7552800640 1 -- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7554160280 msgr2=0x7f7554157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7552800640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7554160280 0x7f7554157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7552800640 1 -- 172.21.15.66:0/504880550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75540cdeb0 con 0x7f75540b1e10 2024-09-10T22:12:51.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7552800640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75540b1e10 0x7f75540cd840 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f754400c490 tx=0x7f754400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:51.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f754aa00640 1 -- 172.21.15.66:0/504880550 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544016020 con 0x7f75540b1e10 2024-09-10T22:12:51.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f754aa00640 1 -- 172.21.15.66:0/504880550 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7544012940 con 0x7f75540b1e10 2024-09-10T22:12:51.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f754aa00640 1 -- 172.21.15.66:0/504880550 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544012d90 con 0x7f75540b1e10 2024-09-10T22:12:51.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7559400640 1 -- 172.21.15.66:0/504880550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75540ce190 con 0x7f75540b1e10 2024-09-10T22:12:51.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7559400640 1 -- 172.21.15.66:0/504880550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75540ce750 con 0x7f75540b1e10 2024-09-10T22:12:51.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.955+0000 7f7559400640 1 -- 172.21.15.66:0/504880550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75540aec10 con 0x7f75540b1e10 2024-09-10T22:12:51.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.959+0000 7f754aa00640 1 -- 172.21.15.66:0/504880550 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f754401b400 con 0x7f75540b1e10 2024-09-10T22:12:51.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.959+0000 7f754aa00640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75340743b0 0x7f7534076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.959+0000 7f754aa00640 1 -- 172.21.15.66:0/504880550 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5907+0+0 (secure 0 0 0) 0x7f7544092380 con 0x7f75540b1e10 2024-09-10T22:12:51.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.959+0000 7f7551e00640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75340743b0 0x7f7534076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.959+0000 7f7551e00640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75340743b0 0x7f7534076870 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7554166c50 tx=0x7f75400091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:51.963+0000 7f754aa00640 1 -- 172.21.15.66:0/504880550 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7544020020 con 0x7f75540b1e10 2024-09-10T22:12:52.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.171+0000 7f7559400640 1 -- 172.21.15.66:0/504880550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f75540d2a40 con 0x7f75540b1e10 2024-09-10T22:12:52.175 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:12:52.171+0000 7fcc93a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:12:52.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.171+0000 7f754aa00640 1 -- 172.21.15.66:0/504880550 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) ==== 161+0+0 (secure 0 0 0) 0x7f754405ef30 con 0x7f75540b1e10 2024-09-10T22:12:52.178 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:12:52.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.175+0000 7f7559400640 1 -- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75340743b0 msgr2=0x7f7534076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:52.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.175+0000 7f7559400640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75340743b0 0x7f7534076870 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7554166c50 tx=0x7f75400091e0 comp rx=0 tx=0).stop 2024-09-10T22:12:52.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.175+0000 7f7559400640 1 -- 172.21.15.66:0/504880550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75540b1e10 msgr2=0x7f75540cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:52.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.175+0000 7f7559400640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75540b1e10 0x7f75540cd840 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f754400c490 tx=0x7f754400c950 comp rx=0 tx=0).stop 2024-09-10T22:12:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.175+0000 7f7559400640 1 -- 172.21.15.66:0/504880550 shutdown_connections 2024-09-10T22:12:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.175+0000 7f7559400640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75340743b0 0x7f7534076870 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.175+0000 7f7559400640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7554160280 0x7f7554157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.175+0000 7f7559400640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f755415f450 0x7f7554157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.175+0000 7f7559400640 1 --2- 172.21.15.66:0/504880550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75540b1e10 0x7f75540cd840 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.175+0000 7f7559400640 1 -- 172.21.15.66:0/504880550 >> 172.21.15.66:0/504880550 conn(0x7f75540d1c60 msgr2=0x7f7554071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.175+0000 7f7559400640 1 -- 172.21.15.66:0/504880550 shutdown_connections 2024-09-10T22:12:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.175+0000 7f7559400640 1 -- 172.21.15.66:0/504880550 wait complete. 2024-09-10T22:12:52.203 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:12:52.252 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:12:52.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 -- 172.21.15.66:0/1125302916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7134003c70 msgr2=0x7f7134004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:52.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 --2- 172.21.15.66:0/1125302916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7134003c70 0x7f7134004070 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f7140006f90 tx=0x7f714001cef0 comp rx=0 tx=0).stop 2024-09-10T22:12:52.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7155000640 1 -- 172.21.15.66:0/1125302916 reap_dead start 2024-09-10T22:12:52.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 -- 172.21.15.66:0/1125302916 shutdown_connections 2024-09-10T22:12:52.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 -- 172.21.15.66:0/1125302916 >> 172.21.15.66:0/1125302916 conn(0x7f71500d1c60 msgr2=0x7f71500d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:52.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 -- 172.21.15.66:0/1125302916 shutdown_connections 2024-09-10T22:12:52.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 -- 172.21.15.66:0/1125302916 wait complete. 2024-09-10T22:12:52.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 Processor -- start 2024-09-10T22:12:52.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 -- start start 2024-09-10T22:12:52.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7134003c70 0x7f7150071d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:52.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71500c6980 0x7f7150156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:52.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f71500c6e00 0x7f71500c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:52.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7150167eb0 con 0x7f71500c6980 2024-09-10T22:12:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71500aff30 con 0x7f7134003c70 2024-09-10T22:12:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7157000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7150168030 con 0x7f71500c6e00 2024-09-10T22:12:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7147e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71500c6980 0x7f7150156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7147e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71500c6980 0x7f7150156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38770/0 (socket says 172.21.15.66:38770) 2024-09-10T22:12:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.351+0000 7f7147e00640 1 -- 172.21.15.66:0/3555341911 learned_addr learned my addr 172.21.15.66:0/3555341911 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f714fe00640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7134003c70 0x7f7150071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f7155000640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f71500c6e00 0x7f71500c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f7147e00640 1 -- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f71500c6e00 msgr2=0x7f71500c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f7147e00640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f71500c6e00 0x7f71500c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f7147e00640 1 -- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7134003c70 msgr2=0x7f7150071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f7147e00640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7134003c70 0x7f7150071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:52.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f7147e00640 1 -- 172.21.15.66:0/3555341911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71500cb2a0 con 0x7f71500c6980 2024-09-10T22:12:52.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f714fe00640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7134003c70 0x7f7150071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:52.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f7147e00640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71500c6980 0x7f7150156f10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f7138003e30 tx=0x7f7138001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:52.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f714cc00640 1 -- 172.21.15.66:0/3555341911 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138015070 con 0x7f71500c6980 2024-09-10T22:12:52.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f714cc00640 1 -- 172.21.15.66:0/3555341911 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7138011900 con 0x7f71500c6980 2024-09-10T22:12:52.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f714cc00640 1 -- 172.21.15.66:0/3555341911 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138011d50 con 0x7f71500c6980 2024-09-10T22:12:52.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f7157000640 1 -- 172.21.15.66:0/3555341911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71500cb580 con 0x7f71500c6980 2024-09-10T22:12:52.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f7157000640 1 -- 172.21.15.66:0/3555341911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7150157450 con 0x7f71500c6980 2024-09-10T22:12:52.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f7157000640 1 -- 172.21.15.66:0/3555341911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f715015fb70 con 0x7f71500c6980 2024-09-10T22:12:52.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.355+0000 7f714cc00640 1 -- 172.21.15.66:0/3555341911 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f71380103c0 con 0x7f71500c6980 2024-09-10T22:12:52.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.359+0000 7f714cc00640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f712c0743b0 0x7f712c076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:52.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.359+0000 7f714cc00640 1 -- 172.21.15.66:0/3555341911 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5965+0+0 (secure 0 0 0) 0x7f7138093300 con 0x7f71500c6980 2024-09-10T22:12:52.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.359+0000 7f714fe00640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f712c0743b0 0x7f712c076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:52.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.359+0000 7f714fe00640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f712c0743b0 0x7f712c076870 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f71400ac000 tx=0x7f71400b0040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:52.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.363+0000 7f714cc00640 1 -- 172.21.15.66:0/3555341911 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7138060440 con 0x7f71500c6980 2024-09-10T22:12:52.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:52.575+0000 7f7157000640 1 -- 172.21.15.66:0/3555341911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f71500d2a40 con 0x7f71500c6980 2024-09-10T22:12:53.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.215+0000 7f714cc00640 1 -- 172.21.15.66:0/3555341911 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) ==== 104+0+0 (secure 0 0 0) 0x7f713805fd40 con 0x7f71500c6980 2024-09-10T22:12:53.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.219+0000 7f7157000640 1 -- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f712c0743b0 msgr2=0x7f712c076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:53.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.219+0000 7f7157000640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f712c0743b0 0x7f712c076870 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f71400ac000 tx=0x7f71400b0040 comp rx=0 tx=0).stop 2024-09-10T22:12:53.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.219+0000 7f7157000640 1 -- 172.21.15.66:0/3555341911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71500c6980 msgr2=0x7f7150156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:53.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.219+0000 7f7157000640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71500c6980 0x7f7150156f10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f7138003e30 tx=0x7f7138001810 comp rx=0 tx=0).stop 2024-09-10T22:12:53.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.219+0000 7f7157000640 1 -- 172.21.15.66:0/3555341911 shutdown_connections 2024-09-10T22:12:53.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.219+0000 7f7157000640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f712c0743b0 0x7f712c076870 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:53.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.219+0000 7f7157000640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f71500c6e00 0x7f71500c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:53.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.219+0000 7f7157000640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71500c6980 0x7f7150156f10 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:53.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.219+0000 7f7157000640 1 --2- 172.21.15.66:0/3555341911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7134003c70 0x7f7150071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:53.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.219+0000 7f7157000640 1 -- 172.21.15.66:0/3555341911 >> 172.21.15.66:0/3555341911 conn(0x7f71500d1c60 msgr2=0x7f7150165710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:53.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.219+0000 7f7157000640 1 -- 172.21.15.66:0/3555341911 shutdown_connections 2024-09-10T22:12:53.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.219+0000 7f7157000640 1 -- 172.21.15.66:0/3555341911 wait complete. 2024-09-10T22:12:53.245 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:12:53.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.391+0000 7f845da00640 1 -- 172.21.15.66:0/1562993303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84580ce8d0 msgr2=0x7f84580cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:53.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.391+0000 7f845da00640 1 --2- 172.21.15.66:0/1562993303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84580ce8d0 0x7f84580cecb0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8448009840 tx=0x7f844801c820 comp rx=0 tx=0).stop 2024-09-10T22:12:53.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.391+0000 7f845da00640 1 -- 172.21.15.66:0/1562993303 shutdown_connections 2024-09-10T22:12:53.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.391+0000 7f845da00640 1 --2- 172.21.15.66:0/1562993303 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84580cdf90 0x7f84580cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:53.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.391+0000 7f845da00640 1 --2- 172.21.15.66:0/1562993303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84580ce8d0 0x7f84580cecb0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:53.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.391+0000 7f845da00640 1 -- 172.21.15.66:0/1562993303 >> 172.21.15.66:0/1562993303 conn(0x7f84580d1c60 msgr2=0x7f84580d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:53.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.391+0000 7f845da00640 1 -- 172.21.15.66:0/1562993303 shutdown_connections 2024-09-10T22:12:53.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.391+0000 7f845da00640 1 -- 172.21.15.66:0/1562993303 wait complete. 2024-09-10T22:12:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f845da00640 1 Processor -- start 2024-09-10T22:12:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f845da00640 1 -- start start 2024-09-10T22:12:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f845da00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84580cdf90 0x7f84581f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f845da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84580ce8d0 0x7f84580caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f845da00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84580cf1f0 0x7f845809f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f845da00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8458169020 con 0x7f84580ce8d0 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f845da00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84580b2350 con 0x7f84580cf1f0 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f845da00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f84581691a0 con 0x7f84580cdf90 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f844e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84580ce8d0 0x7f84580caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f844e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84580ce8d0 0x7f84580caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59344/0 (socket says 172.21.15.66:59344) 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f844e000640 1 -- 172.21.15.66:0/75544681 learned_addr learned my addr 172.21.15.66:0/75544681 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f8457400640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84580cf1f0 0x7f845809f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f8456a00640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84580cdf90 0x7f84581f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f844e000640 1 -- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84580cdf90 msgr2=0x7f84581f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f844e000640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84580cdf90 0x7f84581f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f844e000640 1 -- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84580cf1f0 msgr2=0x7f845809f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f844e000640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84580cf1f0 0x7f845809f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f844e000640 1 -- 172.21.15.66:0/75544681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84580cb010 con 0x7f84580ce8d0 2024-09-10T22:12:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f844e000640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84580ce8d0 0x7f84580caad0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f844000c970 tx=0x7f844000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f844f400640 1 -- 172.21.15.66:0/75544681 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440007c70 con 0x7f84580ce8d0 2024-09-10T22:12:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f844f400640 1 -- 172.21.15.66:0/75544681 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f844000f450 con 0x7f84580ce8d0 2024-09-10T22:12:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f844f400640 1 -- 172.21.15.66:0/75544681 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844000f8a0 con 0x7f84580ce8d0 2024-09-10T22:12:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f845da00640 1 -- 172.21.15.66:0/75544681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84580cb2b0 con 0x7f84580ce8d0 2024-09-10T22:12:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.395+0000 7f845da00640 1 -- 172.21.15.66:0/75544681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84580cb870 con 0x7f84580ce8d0 2024-09-10T22:12:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.399+0000 7f845da00640 1 -- 172.21.15.66:0/75544681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84580cecb0 con 0x7f84580ce8d0 2024-09-10T22:12:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.399+0000 7f844f400640 1 -- 172.21.15.66:0/75544681 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f844000fa40 con 0x7f84580ce8d0 2024-09-10T22:12:53.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.399+0000 7f844f400640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84340743b0 0x7f8434076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:53.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.399+0000 7f844f400640 1 -- 172.21.15.66:0/75544681 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5965+0+0 (secure 0 0 0) 0x7f84400929b0 con 0x7f84580ce8d0 2024-09-10T22:12:53.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.399+0000 7f8456a00640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84340743b0 0x7f8434076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:53.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.399+0000 7f8456a00640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84340743b0 0x7f8434076870 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f84480af000 tx=0x7f844801b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:53.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.403+0000 7f844f400640 1 -- 172.21.15.66:0/75544681 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8440014030 con 0x7f84580ce8d0 2024-09-10T22:12:53.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:53.579+0000 7f845da00640 1 -- 172.21.15.66:0/75544681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8458070550 con 0x7f84580ce8d0 2024-09-10T22:12:54.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.223+0000 7f844f400640 1 -- 172.21.15.66:0/75544681 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) ==== 181+0+0 (secure 0 0 0) 0x7f844005fb70 con 0x7f84580ce8d0 2024-09-10T22:12:54.228 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:12:54.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.227+0000 7f845da00640 1 -- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84340743b0 msgr2=0x7f8434076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:54.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.227+0000 7f845da00640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84340743b0 0x7f8434076870 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f84480af000 tx=0x7f844801b060 comp rx=0 tx=0).stop 2024-09-10T22:12:54.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.227+0000 7f845da00640 1 -- 172.21.15.66:0/75544681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84580ce8d0 msgr2=0x7f84580caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:54.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.227+0000 7f845da00640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84580ce8d0 0x7f84580caad0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f844000c970 tx=0x7f844000ce30 comp rx=0 tx=0).stop 2024-09-10T22:12:54.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.227+0000 7f845da00640 1 -- 172.21.15.66:0/75544681 shutdown_connections 2024-09-10T22:12:54.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.227+0000 7f845da00640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84340743b0 0x7f8434076870 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:54.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.227+0000 7f845da00640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84580cf1f0 0x7f845809f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:54.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.227+0000 7f845da00640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84580ce8d0 0x7f84580caad0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:54.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.227+0000 7f845da00640 1 --2- 172.21.15.66:0/75544681 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84580cdf90 0x7f84581f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:54.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.227+0000 7f845da00640 1 -- 172.21.15.66:0/75544681 >> 172.21.15.66:0/75544681 conn(0x7f84580d1c60 msgr2=0x7f84580afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:54.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.227+0000 7f845da00640 1 -- 172.21.15.66:0/75544681 shutdown_connections 2024-09-10T22:12:54.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.227+0000 7f845da00640 1 -- 172.21.15.66:0/75544681 wait complete. 2024-09-10T22:12:54.256 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:12:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.415+0000 7f3591c00640 1 -- 172.21.15.66:0/545092118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570004d50 msgr2=0x7f35700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.415+0000 7f3591c00640 1 --2- 172.21.15.66:0/545092118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570004d50 0x7f35700051d0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f3584001dc0 tx=0x7f358401cb30 comp rx=0 tx=0).stop 2024-09-10T22:12:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.415+0000 7f358b400640 1 -- 172.21.15.66:0/545092118 reap_dead start 2024-09-10T22:12:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.415+0000 7f3591c00640 1 -- 172.21.15.66:0/545092118 shutdown_connections 2024-09-10T22:12:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.415+0000 7f3591c00640 1 -- 172.21.15.66:0/545092118 >> 172.21.15.66:0/545092118 conn(0x7f358c0cfaa0 msgr2=0x7f358c0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.415+0000 7f3591c00640 1 -- 172.21.15.66:0/545092118 shutdown_connections 2024-09-10T22:12:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.415+0000 7f3591c00640 1 -- 172.21.15.66:0/545092118 wait complete. 2024-09-10T22:12:54.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.415+0000 7f3591c00640 1 Processor -- start 2024-09-10T22:12:54.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.415+0000 7f3591c00640 1 -- start start 2024-09-10T22:12:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3591c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3570004d50 0x7f358c1e3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3591c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358c0cd9b0 0x7f358c1e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3591c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c0cde30 0x7f358c1e5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3591c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f358c15c770 con 0x7f358c0cd9b0 2024-09-10T22:12:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3591c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f358c15c5f0 con 0x7f358c0cde30 2024-09-10T22:12:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3591c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f358c15c8f0 con 0x7f3570004d50 2024-09-10T22:12:54.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3583e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358c0cd9b0 0x7f358c1e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:54.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3583e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358c0cd9b0 0x7f358c1e4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59360/0 (socket says 172.21.15.66:59360) 2024-09-10T22:12:54.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3583e00640 1 -- 172.21.15.66:0/1865923977 learned_addr learned my addr 172.21.15.66:0/1865923977 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:54.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f358aa00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3570004d50 0x7f358c1e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:54.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f358b400640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c0cde30 0x7f358c1e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:54.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3583e00640 1 -- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3570004d50 msgr2=0x7f358c1e3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:54.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3583e00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3570004d50 0x7f358c1e3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:54.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3583e00640 1 -- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c0cde30 msgr2=0x7f358c1e5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:54.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3583e00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c0cde30 0x7f358c1e5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:54.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3583e00640 1 -- 172.21.15.66:0/1865923977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c1f4c10 con 0x7f358c0cd9b0 2024-09-10T22:12:54.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f358b400640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c0cde30 0x7f358c1e5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:54.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3583e00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358c0cd9b0 0x7f358c1e4ad0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f357400ead0 tx=0x7f357400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:54.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3582a00640 1 -- 172.21.15.66:0/1865923977 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357400dcf0 con 0x7f358c0cd9b0 2024-09-10T22:12:54.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3582a00640 1 -- 172.21.15.66:0/1865923977 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f357400de90 con 0x7f358c0cd9b0 2024-09-10T22:12:54.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3582a00640 1 -- 172.21.15.66:0/1865923977 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574004270 con 0x7f358c0cd9b0 2024-09-10T22:12:54.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3591c00640 1 -- 172.21.15.66:0/1865923977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358c1f4ef0 con 0x7f358c0cd9b0 2024-09-10T22:12:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3591c00640 1 -- 172.21.15.66:0/1865923977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358c1f54b0 con 0x7f358c0cd9b0 2024-09-10T22:12:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3591c00640 1 -- 172.21.15.66:0/1865923977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358c0cf210 con 0x7f358c0cd9b0 2024-09-10T22:12:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.419+0000 7f3582a00640 1 -- 172.21.15.66:0/1865923977 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f357400b040 con 0x7f358c0cd9b0 2024-09-10T22:12:54.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.423+0000 7f3582a00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35680743b0 0x7f3568076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:54.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.423+0000 7f3582a00640 1 -- 172.21.15.66:0/1865923977 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5527+0+0 (secure 0 0 0) 0x7f3574092b20 con 0x7f358c0cd9b0 2024-09-10T22:12:54.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.423+0000 7f358aa00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35680743b0 0x7f3568076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:54.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.423+0000 7f358aa00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35680743b0 0x7f3568076870 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f358c1e96e0 tx=0x7f357c0067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:54.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.427+0000 7f3582a00640 1 -- 172.21.15.66:0/1865923977 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f357405fc60 con 0x7f358c0cd9b0 2024-09-10T22:12:54.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:54.599+0000 7f3591c00640 1 -- 172.21.15.66:0/1865923977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f358c0d07c0 con 0x7f358c0cd9b0 2024-09-10T22:12:55.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.227+0000 7f3582a00640 1 -- 172.21.15.66:0/1865923977 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) ==== 169+0+0 (secure 0 0 0) 0x7f3574068380 con 0x7f358c0cd9b0 2024-09-10T22:12:55.234 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:12:55.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.231+0000 7f3591c00640 1 -- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35680743b0 msgr2=0x7f3568076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:55.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.231+0000 7f3591c00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35680743b0 0x7f3568076870 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f358c1e96e0 tx=0x7f357c0067b0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.231+0000 7f3591c00640 1 -- 172.21.15.66:0/1865923977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358c0cd9b0 msgr2=0x7f358c1e4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:55.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.231+0000 7f3591c00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358c0cd9b0 0x7f358c1e4ad0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f357400ead0 tx=0x7f357400ef90 comp rx=0 tx=0).stop 2024-09-10T22:12:55.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.231+0000 7f3591c00640 1 -- 172.21.15.66:0/1865923977 shutdown_connections 2024-09-10T22:12:55.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.231+0000 7f3591c00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35680743b0 0x7f3568076870 secure :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f358c1e96e0 tx=0x7f357c0067b0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.231+0000 7f3591c00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c0cde30 0x7f358c1e5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.231+0000 7f3591c00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f358c0cd9b0 0x7f358c1e4ad0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.231+0000 7f3591c00640 1 --2- 172.21.15.66:0/1865923977 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3570004d50 0x7f358c1e3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.231+0000 7f3591c00640 1 -- 172.21.15.66:0/1865923977 >> 172.21.15.66:0/1865923977 conn(0x7f358c0cfaa0 msgr2=0x7f358c0ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:55.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.231+0000 7f3591c00640 1 -- 172.21.15.66:0/1865923977 shutdown_connections 2024-09-10T22:12:55.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.231+0000 7f3591c00640 1 -- 172.21.15.66:0/1865923977 wait complete. 2024-09-10T22:12:55.263 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T22:12:55.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.407+0000 7f344c200640 1 -- 172.21.15.66:0/3468646466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34440713b0 msgr2=0x7f344406eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:55.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.407+0000 7f344c200640 1 --2- 172.21.15.66:0/3468646466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34440713b0 0x7f344406eea0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f3430009870 tx=0x7f343001f010 comp rx=0 tx=0).stop 2024-09-10T22:12:55.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.407+0000 7f344c200640 1 -- 172.21.15.66:0/3468646466 shutdown_connections 2024-09-10T22:12:55.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.407+0000 7f344c200640 1 --2- 172.21.15.66:0/3468646466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34440713b0 0x7f344406eea0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.407+0000 7f344c200640 1 --2- 172.21.15.66:0/3468646466 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3444070fd0 0x7f34440b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.407+0000 7f344c200640 1 -- 172.21.15.66:0/3468646466 >> 172.21.15.66:0/3468646466 conn(0x7f34440d1d00 msgr2=0x7f34440d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:55.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.407+0000 7f344c200640 1 -- 172.21.15.66:0/3468646466 shutdown_connections 2024-09-10T22:12:55.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.407+0000 7f344c200640 1 -- 172.21.15.66:0/3468646466 wait complete. 2024-09-10T22:12:55.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.407+0000 7f344c200640 1 Processor -- start 2024-09-10T22:12:55.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344c200640 1 -- start start 2024-09-10T22:12:55.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344c200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f344406dd90 0x7f344414ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:55.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344c200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3444070fd0 0x7f34440cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:55.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344c200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34440713b0 0x7f34441f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:55.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344c200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34440d03b0 con 0x7f34440713b0 2024-09-10T22:12:55.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344c200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34440d0230 con 0x7f3444070fd0 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344c200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f34440d0530 con 0x7f344406dd90 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34440713b0 0x7f34441f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34440713b0 0x7f34441f82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59384/0 (socket says 172.21.15.66:59384) 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344a200640 1 -- 172.21.15.66:0/2552313387 learned_addr learned my addr 172.21.15.66:0/2552313387 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f3449800640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f344406dd90 0x7f344414ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f3448e00640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3444070fd0 0x7f34440cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344a200640 1 -- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f344406dd90 msgr2=0x7f344414ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344a200640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f344406dd90 0x7f344414ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344a200640 1 -- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3444070fd0 msgr2=0x7f34440cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344a200640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3444070fd0 0x7f34440cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344a200640 1 -- 172.21.15.66:0/2552313387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34441f5c50 con 0x7f34440713b0 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f3449800640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f344406dd90 0x7f344414ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344a200640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34440713b0 0x7f34441f82a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f343c00cbf0 tx=0x7f343c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f3441600640 1 -- 172.21.15.66:0/2552313387 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c016020 con 0x7f34440713b0 2024-09-10T22:12:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f3441600640 1 -- 172.21.15.66:0/2552313387 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f343c00ab40 con 0x7f34440713b0 2024-09-10T22:12:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f3441600640 1 -- 172.21.15.66:0/2552313387 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c012420 con 0x7f34440713b0 2024-09-10T22:12:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344c200640 1 -- 172.21.15.66:0/2552313387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34440b28e0 con 0x7f34440713b0 2024-09-10T22:12:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344c200640 1 -- 172.21.15.66:0/2552313387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34440b2ea0 con 0x7f34440713b0 2024-09-10T22:12:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.411+0000 7f344c200640 1 -- 172.21.15.66:0/2552313387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34440b1180 con 0x7f34440713b0 2024-09-10T22:12:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.415+0000 7f3441600640 1 -- 172.21.15.66:0/2552313387 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f343c01b400 con 0x7f34440713b0 2024-09-10T22:12:55.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.415+0000 7f3441600640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f34240743b0 0x7f3424076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:55.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.415+0000 7f3441600640 1 -- 172.21.15.66:0/2552313387 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5055+0+0 (secure 0 0 0) 0x7f343c092820 con 0x7f34440713b0 2024-09-10T22:12:55.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.415+0000 7f3449800640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f34240743b0 0x7f3424076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:55.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.415+0000 7f3449800640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f34240743b0 0x7f3424076870 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3438009c30 tx=0x7f34380091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:55.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.419+0000 7f3441600640 1 -- 172.21.15.66:0/2552313387 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f343c020020 con 0x7f34440713b0 2024-09-10T22:12:55.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.599+0000 7f344c200640 1 -- 172.21.15.66:0/2552313387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f34440ae9f0 con 0x7f34440713b0 2024-09-10T22:12:55.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f3441600640 1 -- 172.21.15.66:0/2552313387 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v50) ==== 71+0+0 (secure 0 0 0) 0x7f343c05f960 con 0x7f34440713b0 2024-09-10T22:12:55.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f344c200640 1 -- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f34240743b0 msgr2=0x7f3424076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:55.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f344c200640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f34240743b0 0x7f3424076870 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3438009c30 tx=0x7f34380091e0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f344c200640 1 -- 172.21.15.66:0/2552313387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34440713b0 msgr2=0x7f34441f82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:55.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f344c200640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34440713b0 0x7f34441f82a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f343c00cbf0 tx=0x7f343c00cce0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f344c200640 1 -- 172.21.15.66:0/2552313387 shutdown_connections 2024-09-10T22:12:55.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f344c200640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f34240743b0 0x7f3424076870 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f344c200640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34440713b0 0x7f34441f82a0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f344c200640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3444070fd0 0x7f34440cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f344c200640 1 --2- 172.21.15.66:0/2552313387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f344406dd90 0x7f344414ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f344c200640 1 -- 172.21.15.66:0/2552313387 >> 172.21.15.66:0/2552313387 conn(0x7f34440d1d00 msgr2=0x7f34440d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:55.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f344c200640 1 -- 172.21.15.66:0/2552313387 shutdown_connections 2024-09-10T22:12:55.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.603+0000 7f344c200640 1 -- 172.21.15.66:0/2552313387 wait complete. 2024-09-10T22:12:55.628 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-10T22:12:55.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 -- 172.21.15.66:0/967034265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040ca980 msgr2=0x7f87040afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:55.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 --2- 172.21.15.66:0/967034265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040ca980 0x7f87040afff0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00a950 tx=0x7f86ec01db70 comp rx=0 tx=0).stop 2024-09-10T22:12:55.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 -- 172.21.15.66:0/967034265 shutdown_connections 2024-09-10T22:12:55.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 --2- 172.21.15.66:0/967034265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040ca980 0x7f87040afff0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 -- 172.21.15.66:0/967034265 >> 172.21.15.66:0/967034265 conn(0x7f87040d0330 msgr2=0x7f87040d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:55.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 -- 172.21.15.66:0/967034265 shutdown_connections 2024-09-10T22:12:55.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 -- 172.21.15.66:0/967034265 wait complete. 2024-09-10T22:12:55.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 Processor -- start 2024-09-10T22:12:55.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 -- start start 2024-09-10T22:12:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8704074530 0x7f8704155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87040ca980 0x7f87040c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040cfc90 0x7f87040b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87040cca30 con 0x7f87040cfc90 2024-09-10T22:12:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87040cc8b0 con 0x7f87040ca980 2024-09-10T22:12:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8709a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f87040ccbb0 con 0x7f8704074530 2024-09-10T22:12:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8703400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040cfc90 0x7f87040b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8703400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040cfc90 0x7f87040b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59402/0 (socket says 172.21.15.66:59402) 2024-09-10T22:12:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8703400640 1 -- 172.21.15.66:0/833688039 learned_addr learned my addr 172.21.15.66:0/833688039 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f8702a00640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8704074530 0x7f8704155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.779+0000 7f86fbe00640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87040ca980 0x7f87040c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f8703400640 1 -- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8704074530 msgr2=0x7f8704155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f8703400640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8704074530 0x7f8704155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f8703400640 1 -- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87040ca980 msgr2=0x7f87040c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f8703400640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87040ca980 0x7f87040c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f8703400640 1 -- 172.21.15.66:0/833688039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870415a790 con 0x7f87040cfc90 2024-09-10T22:12:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f8703400640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040cfc90 0x7f87040b31e0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00c710 tx=0x7f86fc00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f86faa00640 1 -- 172.21.15.66:0/833688039 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc003b80 con 0x7f87040cfc90 2024-09-10T22:12:55.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f86faa00640 1 -- 172.21.15.66:0/833688039 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86fc003d20 con 0x7f87040cfc90 2024-09-10T22:12:55.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f86faa00640 1 -- 172.21.15.66:0/833688039 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc00aab0 con 0x7f87040cfc90 2024-09-10T22:12:55.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f8709a00640 1 -- 172.21.15.66:0/833688039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8704155190 con 0x7f87040cfc90 2024-09-10T22:12:55.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f8709a00640 1 -- 172.21.15.66:0/833688039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870415a4b0 con 0x7f87040cfc90 2024-09-10T22:12:55.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f8709a00640 1 -- 172.21.15.66:0/833688039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87040b1340 con 0x7f87040cfc90 2024-09-10T22:12:55.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.783+0000 7f86faa00640 1 -- 172.21.15.66:0/833688039 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f86fc00ac50 con 0x7f87040cfc90 2024-09-10T22:12:55.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.787+0000 7f86faa00640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86e00743b0 0x7f86e0076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:55.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.787+0000 7f86faa00640 1 -- 172.21.15.66:0/833688039 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5055+0+0 (secure 0 0 0) 0x7f86fc0920c0 con 0x7f87040cfc90 2024-09-10T22:12:55.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.787+0000 7f8702a00640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86e00743b0 0x7f86e0076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:55.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.787+0000 7f8702a00640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86e00743b0 0x7f86e0076870 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f86f40046c0 tx=0x7f86f4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:55.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:55.791+0000 7f86faa00640 1 -- 172.21.15.66:0/833688039 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f86fc00ce20 con 0x7f87040cfc90 2024-09-10T22:12:56.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.023+0000 7f8709a00640 1 -- 172.21.15.66:0/833688039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7f8704073d50 con 0x7f87040cfc90 2024-09-10T22:12:56.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f86faa00640 1 -- 172.21.15.66:0/833688039 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v566) ==== 185+0+0 (secure 0 0 0) 0x7f86fc05f200 con 0x7f87040cfc90 2024-09-10T22:12:56.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f8709a00640 1 -- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86e00743b0 msgr2=0x7f86e0076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:56.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f8709a00640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86e00743b0 0x7f86e0076870 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f86f40046c0 tx=0x7f86f4009040 comp rx=0 tx=0).stop 2024-09-10T22:12:56.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f8709a00640 1 -- 172.21.15.66:0/833688039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040cfc90 msgr2=0x7f87040b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:56.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f8709a00640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040cfc90 0x7f87040b31e0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00c710 tx=0x7f86fc00cbd0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f8709a00640 1 -- 172.21.15.66:0/833688039 shutdown_connections 2024-09-10T22:12:56.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f8709a00640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f86e00743b0 0x7f86e0076870 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f8709a00640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87040cfc90 0x7f87040b31e0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f8709a00640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87040ca980 0x7f87040c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f8709a00640 1 --2- 172.21.15.66:0/833688039 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8704074530 0x7f8704155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f8709a00640 1 -- 172.21.15.66:0/833688039 >> 172.21.15.66:0/833688039 conn(0x7f87040d0330 msgr2=0x7f87040ad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:56.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f8709a00640 1 -- 172.21.15.66:0/833688039 shutdown_connections 2024-09-10T22:12:56.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.231+0000 7f8709a00640 1 -- 172.21.15.66:0/833688039 wait complete. 2024-09-10T22:12:56.259 INFO:tasks.cephfs_test_runner:Test that progress bars, that printed for 3 ongoing clone jobs, are ... ok 2024-09-10T22:12:56.259 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the progress bar that is printed for 1 ongoing clone job is 2024-09-10T22:12:56.260 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-10T22:12:56.260 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:12:56.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.403+0000 7fbd6a600640 1 -- 172.21.15.66:0/792644713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48004010 msgr2=0x7fbd48004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:56.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.403+0000 7fbd6a600640 1 --2- 172.21.15.66:0/792644713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48004010 0x7fbd48004810 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c001690 tx=0x7fbd4c01c7a0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.403+0000 7fbd63e00640 1 -- 172.21.15.66:0/792644713 reap_dead start 2024-09-10T22:12:56.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.403+0000 7fbd6a600640 1 -- 172.21.15.66:0/792644713 shutdown_connections 2024-09-10T22:12:56.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.403+0000 7fbd6a600640 1 -- 172.21.15.66:0/792644713 >> 172.21.15.66:0/792644713 conn(0x7fbd640d1c50 msgr2=0x7fbd640d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:56.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.403+0000 7fbd6a600640 1 -- 172.21.15.66:0/792644713 shutdown_connections 2024-09-10T22:12:56.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.403+0000 7fbd6a600640 1 -- 172.21.15.66:0/792644713 wait complete. 2024-09-10T22:12:56.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.403+0000 7fbd6a600640 1 Processor -- start 2024-09-10T22:12:56.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.403+0000 7fbd6a600640 1 -- start start 2024-09-10T22:12:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd6a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48004010 0x7fbd641f8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd6a600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd6414c2b0 0x7fbd64070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd6a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd6414ed90 0x7fbd6414f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd6a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbd640d28f0 con 0x7fbd48004010 2024-09-10T22:12:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd6a600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd640d2770 con 0x7fbd6414ed90 2024-09-10T22:12:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd6a600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbd640d2a70 con 0x7fbd6414c2b0 2024-09-10T22:12:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd63400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48004010 0x7fbd641f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd63400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48004010 0x7fbd641f8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59424/0 (socket says 172.21.15.66:59424) 2024-09-10T22:12:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd63400640 1 -- 172.21.15.66:0/2122173811 learned_addr learned my addr 172.21.15.66:0/2122173811 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd5be00640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd6414c2b0 0x7fbd64070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd63e00640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd6414ed90 0x7fbd6414f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd63400640 1 -- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd6414c2b0 msgr2=0x7fbd64070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd63400640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd6414c2b0 0x7fbd64070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd63400640 1 -- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd6414ed90 msgr2=0x7fbd6414f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd63400640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd6414ed90 0x7fbd6414f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd63400640 1 -- 172.21.15.66:0/2122173811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd641f7cb0 con 0x7fbd48004010 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd63e00640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd6414ed90 0x7fbd6414f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd63400640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48004010 0x7fbd641f8770 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fbd54012480 tx=0x7fbd54012940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd5b400640 1 -- 172.21.15.66:0/2122173811 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd54013980 con 0x7fbd48004010 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd5b400640 1 -- 172.21.15.66:0/2122173811 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd54017030 con 0x7fbd48004010 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd5b400640 1 -- 172.21.15.66:0/2122173811 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd54013de0 con 0x7fbd48004010 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd6a600640 1 -- 172.21.15.66:0/2122173811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd640b7540 con 0x7fbd48004010 2024-09-10T22:12:56.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd6a600640 1 -- 172.21.15.66:0/2122173811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd641f7f70 con 0x7fbd48004010 2024-09-10T22:12:56.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.407+0000 7fbd6a600640 1 -- 172.21.15.66:0/2122173811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd640ae960 con 0x7fbd48004010 2024-09-10T22:12:56.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.411+0000 7fbd5b400640 1 -- 172.21.15.66:0/2122173811 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7fbd54010440 con 0x7fbd48004010 2024-09-10T22:12:56.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.411+0000 7fbd5b400640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd400743e0 0x7fbd400768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:56.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.411+0000 7fbd5b400640 1 -- 172.21.15.66:0/2122173811 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5055+0+0 (secure 0 0 0) 0x7fbd54095c50 con 0x7fbd48004010 2024-09-10T22:12:56.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.411+0000 7fbd5be00640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd400743e0 0x7fbd400768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:56.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.411+0000 7fbd5be00640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd400743e0 0x7fbd400768a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0aa000 tx=0x7fbd4c019540 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.415+0000 7fbd5b400640 1 -- 172.21.15.66:0/2122173811 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbd54062740 con 0x7fbd48004010 2024-09-10T22:12:56.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.591+0000 7fbd6a600640 1 -- 172.21.15.66:0/2122173811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd640b1910 con 0x7fbd48004010 2024-09-10T22:12:56.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.591+0000 7fbd5b400640 1 -- 172.21.15.66:0/2122173811 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+13148 (secure 0 0 0) 0x7fbd54067170 con 0x7fbd48004010 2024-09-10T22:12:56.596 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:56.596 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":118,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:12:55.226661+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6837/3818206155":"2024-09-11T22:12:52.174557+0000","172.21.15.94:6835/3818206155":"2024-09-11T22:12:52.174557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:12:56.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.595+0000 7fbd6a600640 1 -- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd400743e0 msgr2=0x7fbd400768a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:56.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.595+0000 7fbd6a600640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd400743e0 0x7fbd400768a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0aa000 tx=0x7fbd4c019540 comp rx=0 tx=0).stop 2024-09-10T22:12:56.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.595+0000 7fbd6a600640 1 -- 172.21.15.66:0/2122173811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48004010 msgr2=0x7fbd641f8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:56.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.595+0000 7fbd6a600640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48004010 0x7fbd641f8770 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fbd54012480 tx=0x7fbd54012940 comp rx=0 tx=0).stop 2024-09-10T22:12:56.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.595+0000 7fbd6a600640 1 -- 172.21.15.66:0/2122173811 shutdown_connections 2024-09-10T22:12:56.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.595+0000 7fbd6a600640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbd400743e0 0x7fbd400768a0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.595+0000 7fbd6a600640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd6414ed90 0x7fbd6414f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.595+0000 7fbd6a600640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd6414c2b0 0x7fbd64070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.595+0000 7fbd6a600640 1 --2- 172.21.15.66:0/2122173811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48004010 0x7fbd641f8770 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.595+0000 7fbd6a600640 1 -- 172.21.15.66:0/2122173811 >> 172.21.15.66:0/2122173811 conn(0x7fbd640d1c50 msgr2=0x7fbd640d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:56.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.595+0000 7fbd6a600640 1 -- 172.21.15.66:0/2122173811 shutdown_connections 2024-09-10T22:12:56.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.595+0000 7fbd6a600640 1 -- 172.21.15.66:0/2122173811 wait complete. 2024-09-10T22:12:56.618 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:00:22.340318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:12:56.618 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:12:56.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.763+0000 7f3573000640 1 -- 172.21.15.66:0/168776913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0c4b10 msgr2=0x7f356c06e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:56.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.763+0000 7f3573000640 1 --2- 172.21.15.66:0/168776913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0c4b10 0x7f356c06e960 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f356c0c6a10 tx=0x7f355c01c8c0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.763+0000 7f3573000640 1 -- 172.21.15.66:0/168776913 shutdown_connections 2024-09-10T22:12:56.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.763+0000 7f3573000640 1 --2- 172.21.15.66:0/168776913 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f356c0c4ef0 0x7f356c0c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.763+0000 7f3573000640 1 --2- 172.21.15.66:0/168776913 >> v2:172.21.15.94:3300/0 conn(0x7f356c0ae620 0x7f356c0725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.763+0000 7f3573000640 1 --2- 172.21.15.66:0/168776913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0c4b10 0x7f356c06e960 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.763+0000 7f3573000640 1 -- 172.21.15.66:0/168776913 >> 172.21.15.66:0/168776913 conn(0x7f356c0cfa30 msgr2=0x7f356c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:56.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.763+0000 7f3573000640 1 -- 172.21.15.66:0/168776913 shutdown_connections 2024-09-10T22:12:56.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.763+0000 7f3573000640 1 -- 172.21.15.66:0/168776913 wait complete. 2024-09-10T22:12:56.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.763+0000 7f3573000640 1 Processor -- start 2024-09-10T22:12:56.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.763+0000 7f3573000640 1 -- start start 2024-09-10T22:12:56.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3573000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae620 0x7f356c0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:56.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3573000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f356c0c4b10 0x7f356c14cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:56.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3573000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f356c0c4ef0 0x7f356c0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:56.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3573000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f356c166cd0 con 0x7f356c0ae620 2024-09-10T22:12:56.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3573000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f356c166b50 con 0x7f356c0c4b10 2024-09-10T22:12:56.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3573000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f356c166e50 con 0x7f356c0c4ef0 2024-09-10T22:12:56.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f356be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae620 0x7f356c0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:56.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f356be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae620 0x7f356c0c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59434/0 (socket says 172.21.15.66:59434) 2024-09-10T22:12:56.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f356be00640 1 -- 172.21.15.66:0/2561553210 learned_addr learned my addr 172.21.15.66:0/2561553210 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:56.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3571000640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f356c0c4ef0 0x7f356c0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:56.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3563400640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f356c0c4b10 0x7f356c14cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:56.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f356be00640 1 -- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f356c0c4ef0 msgr2=0x7f356c0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:56.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f356be00640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f356c0c4ef0 0x7f356c0cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f356be00640 1 -- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f356c0c4b10 msgr2=0x7f356c14cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:56.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f356be00640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f356c0c4b10 0x7f356c14cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f356be00640 1 -- 172.21.15.66:0/2561553210 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356c0ce2a0 con 0x7f356c0ae620 2024-09-10T22:12:56.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3563400640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f356c0c4b10 0x7f356c14cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:56.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f356be00640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae620 0x7f356c0c7d20 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f355c01af70 tx=0x7f355c0a6f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:56.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3568c00640 1 -- 172.21.15.66:0/2561553210 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c0b2040 con 0x7f356c0ae620 2024-09-10T22:12:56.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3568c00640 1 -- 172.21.15.66:0/2561553210 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f355c0044a0 con 0x7f356c0ae620 2024-09-10T22:12:56.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3568c00640 1 -- 172.21.15.66:0/2561553210 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c0052d0 con 0x7f356c0ae620 2024-09-10T22:12:56.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3573000640 1 -- 172.21.15.66:0/2561553210 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356c0ce580 con 0x7f356c0ae620 2024-09-10T22:12:56.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3573000640 1 -- 172.21.15.66:0/2561553210 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356c0b3cb0 con 0x7f356c0ae620 2024-09-10T22:12:56.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3573000640 1 -- 172.21.15.66:0/2561553210 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f356c06e960 con 0x7f356c0ae620 2024-09-10T22:12:56.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.767+0000 7f3568c00640 1 -- 172.21.15.66:0/2561553210 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f355c005470 con 0x7f356c0ae620 2024-09-10T22:12:56.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.771+0000 7f3568c00640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35480743b0 0x7f3548076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:56.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.771+0000 7f3568c00640 1 -- 172.21.15.66:0/2561553210 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5055+0+0 (secure 0 0 0) 0x7f355c12b920 con 0x7f356c0ae620 2024-09-10T22:12:56.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.771+0000 7f3563400640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35480743b0 0x7f3548076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:56.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.771+0000 7f3563400640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35480743b0 0x7f3548076870 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3554005fd0 tx=0x7f3554005f20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.775+0000 7f3568c00640 1 -- 172.21.15.66:0/2561553210 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f355c0164b0 con 0x7f356c0ae620 2024-09-10T22:12:56.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.947+0000 7f3573000640 1 -- 172.21.15.66:0/2561553210 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f356c0b3ef0 con 0x7f356c0ae620 2024-09-10T22:12:56.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3568c00640 1 -- 172.21.15.66:0/2561553210 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7f355c101070 con 0x7f356c0ae620 2024-09-10T22:12:56.955 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-09-10T22:12:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3573000640 1 -- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35480743b0 msgr2=0x7f3548076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3573000640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35480743b0 0x7f3548076870 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3554005fd0 tx=0x7f3554005f20 comp rx=0 tx=0).stop 2024-09-10T22:12:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3573000640 1 -- 172.21.15.66:0/2561553210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae620 msgr2=0x7f356c0c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3573000640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae620 0x7f356c0c7d20 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f355c01af70 tx=0x7f355c0a6f60 comp rx=0 tx=0).stop 2024-09-10T22:12:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3573000640 1 -- 172.21.15.66:0/2561553210 shutdown_connections 2024-09-10T22:12:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3573000640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f35480743b0 0x7f3548076870 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3573000640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f356c0c4ef0 0x7f356c0cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3573000640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f356c0c4b10 0x7f356c14cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3573000640 1 --2- 172.21.15.66:0/2561553210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae620 0x7f356c0c7d20 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3573000640 1 -- 172.21.15.66:0/2561553210 >> 172.21.15.66:0/2561553210 conn(0x7f356c0cfa30 msgr2=0x7f356c15d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3573000640 1 -- 172.21.15.66:0/2561553210 shutdown_connections 2024-09-10T22:12:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:56.951+0000 7f3573000640 1 -- 172.21.15.66:0/2561553210 wait complete. 2024-09-10T22:12:56.977 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-10T22:12:57.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.119+0000 7fb382600640 1 -- 172.21.15.66:0/1233014225 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37c15f450 msgr2=0x7fb37c0b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:57.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.119+0000 7fb382600640 1 --2- 172.21.15.66:0/1233014225 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37c15f450 0x7fb37c0b3170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009870 tx=0x7fb36c01d200 comp rx=0 tx=0).stop 2024-09-10T22:12:57.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.119+0000 7fb382600640 1 -- 172.21.15.66:0/1233014225 shutdown_connections 2024-09-10T22:12:57.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.119+0000 7fb382600640 1 --2- 172.21.15.66:0/1233014225 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37c15f450 0x7fb37c0b3170 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:57.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.119+0000 7fb382600640 1 -- 172.21.15.66:0/1233014225 >> 172.21.15.66:0/1233014225 conn(0x7fb37c0d1c60 msgr2=0x7fb37c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:57.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.119+0000 7fb382600640 1 -- 172.21.15.66:0/1233014225 shutdown_connections 2024-09-10T22:12:57.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.119+0000 7fb382600640 1 -- 172.21.15.66:0/1233014225 wait complete. 2024-09-10T22:12:57.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.119+0000 7fb382600640 1 Processor -- start 2024-09-10T22:12:57.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.119+0000 7fb382600640 1 -- start start 2024-09-10T22:12:57.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb382600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb37c0b1e10 0x7fb37c1655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:57.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb382600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb37c15f450 0x7fb37c15af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:57.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb382600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37c160280 0x7fb37c158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:57.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb382600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c16a0d0 con 0x7fb37c0b1e10 2024-09-10T22:12:57.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb382600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c0afee0 con 0x7fb37c15f450 2024-09-10T22:12:57.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb382600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb37c16a250 con 0x7fb37c160280 2024-09-10T22:12:57.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb37b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb37c0b1e10 0x7fb37c1655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:57.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb37b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb37c0b1e10 0x7fb37c1655e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59438/0 (socket says 172.21.15.66:59438) 2024-09-10T22:12:57.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb37b400640 1 -- 172.21.15.66:0/3324845215 learned_addr learned my addr 172.21.15.66:0/3324845215 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:57.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb37be00640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37c160280 0x7fb37c158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb373e00640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb37c15f450 0x7fb37c15af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb37b400640 1 -- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37c160280 msgr2=0x7fb37c158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb37b400640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37c160280 0x7fb37c158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb37b400640 1 -- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb37c15f450 msgr2=0x7fb37c15af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb37b400640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb37c15f450 0x7fb37c15af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb37b400640 1 -- 172.21.15.66:0/3324845215 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c167730 con 0x7fb37c0b1e10 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb373e00640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb37c15f450 0x7fb37c15af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb37b400640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb37c0b1e10 0x7fb37c1655e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fb36c01ae50 tx=0x7fb36c005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb373400640 1 -- 172.21.15.66:0/3324845215 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0b2040 con 0x7fb37c0b1e10 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb373400640 1 -- 172.21.15.66:0/3324845215 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb36c004330 con 0x7fb37c0b1e10 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb373400640 1 -- 172.21.15.66:0/3324845215 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c003800 con 0x7fb37c0b1e10 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb382600640 1 -- 172.21.15.66:0/3324845215 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c1679d0 con 0x7fb37c0b1e10 2024-09-10T22:12:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb382600640 1 -- 172.21.15.66:0/3324845215 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c0b76e0 con 0x7fb37c0b1e10 2024-09-10T22:12:57.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.123+0000 7fb382600640 1 -- 172.21.15.66:0/3324845215 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb37c0b3170 con 0x7fb37c0b1e10 2024-09-10T22:12:57.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.127+0000 7fb373400640 1 -- 172.21.15.66:0/3324845215 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7fb36c0b5070 con 0x7fb37c0b1e10 2024-09-10T22:12:57.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.127+0000 7fb373400640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3580743b0 0x7fb358076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:57.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.127+0000 7fb373400640 1 -- 172.21.15.66:0/3324845215 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5055+0+0 (secure 0 0 0) 0x7fb36c12bb40 con 0x7fb37c0b1e10 2024-09-10T22:12:57.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.127+0000 7fb373e00640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3580743b0 0x7fb358076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:57.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.127+0000 7fb373e00640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3580743b0 0x7fb358076870 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb364001230 tx=0x7fb364009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:57.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.131+0000 7fb373400640 1 -- 172.21.15.66:0/3324845215 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb36c016450 con 0x7fb37c0b1e10 2024-09-10T22:12:57.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:57.367+0000 7fb382600640 1 -- 172.21.15.66:0/3324845215 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7fb37c0d2a40 con 0x7fb37c0b1e10 2024-09-10T22:12:57.782 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:12:57.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:12:58.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb373400640 1 -- 172.21.15.66:0/3324845215 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v568) ==== 189+0+0 (secure 0 0 0) 0x7fb36c0f8c80 con 0x7fb37c0b1e10 2024-09-10T22:12:58.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb382600640 1 -- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3580743b0 msgr2=0x7fb358076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:58.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb382600640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3580743b0 0x7fb358076870 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb364001230 tx=0x7fb364009040 comp rx=0 tx=0).stop 2024-09-10T22:12:58.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb382600640 1 -- 172.21.15.66:0/3324845215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb37c0b1e10 msgr2=0x7fb37c1655e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:58.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb382600640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb37c0b1e10 0x7fb37c1655e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fb36c01ae50 tx=0x7fb36c005790 comp rx=0 tx=0).stop 2024-09-10T22:12:58.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb382600640 1 -- 172.21.15.66:0/3324845215 shutdown_connections 2024-09-10T22:12:58.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb382600640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb3580743b0 0x7fb358076870 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb382600640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37c160280 0x7fb37c158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb382600640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb37c15f450 0x7fb37c15af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb382600640 1 --2- 172.21.15.66:0/3324845215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb37c0b1e10 0x7fb37c1655e0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb382600640 1 -- 172.21.15.66:0/3324845215 >> 172.21.15.66:0/3324845215 conn(0x7fb37c0d1c60 msgr2=0x7fb37c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:58.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb382600640 1 -- 172.21.15.66:0/3324845215 shutdown_connections 2024-09-10T22:12:58.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.239+0000 7fb382600640 1 -- 172.21.15.66:0/3324845215 wait complete. 2024-09-10T22:12:58.270 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T22:12:58.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.415+0000 7f82b6a00640 1 -- 172.21.15.66:0/3794451356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b00b2bf0 msgr2=0x7f82b00d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:58.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.415+0000 7f82b6a00640 1 --2- 172.21.15.66:0/3794451356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b00b2bf0 0x7f82b00d0090 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8298009840 tx=0x7f829801c7e0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.415+0000 7f82b6a00640 1 -- 172.21.15.66:0/3794451356 shutdown_connections 2024-09-10T22:12:58.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.415+0000 7f82b6a00640 1 --2- 172.21.15.66:0/3794451356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b00b2bf0 0x7f82b00d0090 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.415+0000 7f82b6a00640 1 --2- 172.21.15.66:0/3794451356 >> v2:172.21.15.94:3300/0 conn(0x7f82b015f340 0x7f82b00cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.415+0000 7f82b6a00640 1 -- 172.21.15.66:0/3794451356 >> 172.21.15.66:0/3794451356 conn(0x7f82b00d1c50 msgr2=0x7f82b00d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:58.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.415+0000 7f82b6a00640 1 -- 172.21.15.66:0/3794451356 shutdown_connections 2024-09-10T22:12:58.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.415+0000 7f82b6a00640 1 -- 172.21.15.66:0/3794451356 wait complete. 2024-09-10T22:12:58.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b6a00640 1 Processor -- start 2024-09-10T22:12:58.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b6a00640 1 -- start start 2024-09-10T22:12:58.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b00b2bf0 0x7f82b00cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b6a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f82b00d2770 0x7f82b00c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015f340 0x7f82b00cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b6a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0169190 con 0x7f82b015f340 2024-09-10T22:12:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0169010 con 0x7f82b00b2bf0 2024-09-10T22:12:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b6a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f82b0169310 con 0x7f82b00d2770 2024-09-10T22:12:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015f340 0x7f82b00cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015f340 0x7f82b00cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59458/0 (socket says 172.21.15.66:59458) 2024-09-10T22:12:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b4a00640 1 -- 172.21.15.66:0/149638587 learned_addr learned my addr 172.21.15.66:0/149638587 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82afe00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b00b2bf0 0x7f82b00cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82a7e00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f82b00d2770 0x7f82b00c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b4a00640 1 -- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f82b00d2770 msgr2=0x7f82b00c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:58.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b4a00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f82b00d2770 0x7f82b00c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b4a00640 1 -- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b00b2bf0 msgr2=0x7f82b00cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:58.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b4a00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b00b2bf0 0x7f82b00cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b4a00640 1 -- 172.21.15.66:0/149638587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b0157140 con 0x7f82b015f340 2024-09-10T22:12:58.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b4a00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015f340 0x7f82b00cdd60 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f82a800c770 tx=0x7f82a800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:58.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82acc00640 1 -- 172.21.15.66:0/149638587 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8017070 con 0x7f82b015f340 2024-09-10T22:12:58.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82acc00640 1 -- 172.21.15.66:0/149638587 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82a8005070 con 0x7f82b015f340 2024-09-10T22:12:58.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82acc00640 1 -- 172.21.15.66:0/149638587 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a80054c0 con 0x7f82b015f340 2024-09-10T22:12:58.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b6a00640 1 -- 172.21.15.66:0/149638587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b0157400 con 0x7f82b015f340 2024-09-10T22:12:58.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.419+0000 7f82b6a00640 1 -- 172.21.15.66:0/149638587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b00cc660 con 0x7f82b015f340 2024-09-10T22:12:58.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.423+0000 7f82b6a00640 1 -- 172.21.15.66:0/149638587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b015fac0 con 0x7f82b015f340 2024-09-10T22:12:58.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.423+0000 7f82acc00640 1 -- 172.21.15.66:0/149638587 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f82a8005660 con 0x7f82b015f340 2024-09-10T22:12:58.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.423+0000 7f82acc00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f828c0743b0 0x7f828c076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:58.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.423+0000 7f82acc00640 1 -- 172.21.15.66:0/149638587 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5055+0+0 (secure 0 0 0) 0x7f82a8091ab0 con 0x7f82b015f340 2024-09-10T22:12:58.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.423+0000 7f82afe00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f828c0743b0 0x7f828c076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:58.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.423+0000 7f82afe00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f828c0743b0 0x7f828c076870 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f82a0004630 tx=0x7f82a000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:58.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.427+0000 7f82acc00640 1 -- 172.21.15.66:0/149638587 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f82a805e640 con 0x7f82b015f340 2024-09-10T22:12:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.599+0000 7f82b6a00640 1 -- 172.21.15.66:0/149638587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f82b00d2310 con 0x7f82b015f340 2024-09-10T22:12:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.599+0000 7f82acc00640 1 -- 172.21.15.66:0/149638587 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v50) ==== 59+0+1998 (secure 0 0 0) 0x7f82a80673b0 con 0x7f82b015f340 2024-09-10T22:12:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T22:12:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T22:12:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T22:12:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T22:12:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4673.907365 2024-09-10T22:12:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4537.831230 2024-09-10T22:12:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 5706.341617 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4484.013907 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3862.285837 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3776.439661 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3820.270368 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3863.602758 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds 20/20 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_ms 2/2 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_frag true 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_scatterstat true 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_verify_scatter true 2024-09-10T22:12:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T22:12:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T22:12:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T22:12:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced client_mount_timeout 600 2024-09-10T22:12:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_client 20/20 2024-09-10T22:12:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_ms 1/1 2024-09-10T22:12:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T22:12:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T22:12:58.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.603+0000 7f82b6a00640 1 -- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f828c0743b0 msgr2=0x7f828c076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:58.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.603+0000 7f82b6a00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f828c0743b0 0x7f828c076870 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f82a0004630 tx=0x7f82a000b2e0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.603+0000 7f82b6a00640 1 -- 172.21.15.66:0/149638587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015f340 msgr2=0x7f82b00cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:58.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.603+0000 7f82b6a00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015f340 0x7f82b00cdd60 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f82a800c770 tx=0x7f82a800cc30 comp rx=0 tx=0).stop 2024-09-10T22:12:58.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.603+0000 7f82b6a00640 1 -- 172.21.15.66:0/149638587 shutdown_connections 2024-09-10T22:12:58.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.603+0000 7f82b6a00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f828c0743b0 0x7f828c076870 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.603+0000 7f82b6a00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015f340 0x7f82b00cdd60 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.603+0000 7f82b6a00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f82b00d2770 0x7f82b00c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.603+0000 7f82b6a00640 1 --2- 172.21.15.66:0/149638587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b00b2bf0 0x7f82b00cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.603+0000 7f82b6a00640 1 -- 172.21.15.66:0/149638587 >> 172.21.15.66:0/149638587 conn(0x7f82b00d1c50 msgr2=0x7f82b01588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:58.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.603+0000 7f82b6a00640 1 -- 172.21.15.66:0/149638587 shutdown_connections 2024-09-10T22:12:58.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.603+0000 7f82b6a00640 1 -- 172.21.15.66:0/149638587 wait complete. 2024-09-10T22:12:58.629 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:12:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.771+0000 7f299f000640 1 -- 172.21.15.66:0/2420894016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29980d0550 msgr2=0x7f29980c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.771+0000 7f299f000640 1 --2- 172.21.15.66:0/2420894016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29980d0550 0x7f29980c4df0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f299806c4f0 tx=0x7f298001c830 comp rx=0 tx=0).stop 2024-09-10T22:12:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.771+0000 7f299f000640 1 -- 172.21.15.66:0/2420894016 shutdown_connections 2024-09-10T22:12:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.771+0000 7f299f000640 1 --2- 172.21.15.66:0/2420894016 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29980aeec0 0x7f299815df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.771+0000 7f299f000640 1 --2- 172.21.15.66:0/2420894016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29980d0550 0x7f29980c4df0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.771+0000 7f299f000640 1 --2- 172.21.15.66:0/2420894016 >> v2:172.21.15.94:3300/0 conn(0x7f299815a310 0x7f29980c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.771+0000 7f299f000640 1 -- 172.21.15.66:0/2420894016 >> 172.21.15.66:0/2420894016 conn(0x7f29980cfa30 msgr2=0x7f29980cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:58.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f299f000640 1 -- 172.21.15.66:0/2420894016 shutdown_connections 2024-09-10T22:12:58.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f299f000640 1 -- 172.21.15.66:0/2420894016 wait complete. 2024-09-10T22:12:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f299f000640 1 Processor -- start 2024-09-10T22:12:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f299f000640 1 -- start start 2024-09-10T22:12:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f299f000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29980aeec0 0x7f29981f9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f299f000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f299815a310 0x7f29980b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f299f000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f299814adc0 0x7f299814b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f299f000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f299815ec90 con 0x7f29980aeec0 2024-09-10T22:12:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f299f000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f299815eb10 con 0x7f299815a310 2024-09-10T22:12:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f299f000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f29980ca1a0 con 0x7f299814adc0 2024-09-10T22:12:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f2997e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29980aeec0 0x7f29981f9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f2997e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29980aeec0 0x7f29981f9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59476/0 (socket says 172.21.15.66:59476) 2024-09-10T22:12:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f2997e00640 1 -- 172.21.15.66:0/2966176177 learned_addr learned my addr 172.21.15.66:0/2966176177 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f298f400640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f299815a310 0x7f29980b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f299d000640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f299814adc0 0x7f299814b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f2997e00640 1 -- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f299814adc0 msgr2=0x7f299814b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f2997e00640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f299814adc0 0x7f299814b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f2997e00640 1 -- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f299815a310 msgr2=0x7f29980b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f2997e00640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f299815a310 0x7f29980b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f2997e00640 1 -- 172.21.15.66:0/2966176177 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29981f9f90 con 0x7f29980aeec0 2024-09-10T22:12:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f298f400640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f299815a310 0x7f29980b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f2997e00640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29980aeec0 0x7f29981f9a50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f298800d750 tx=0x7f298800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f2994c00640 1 -- 172.21.15.66:0/2966176177 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988017070 con 0x7f29980aeec0 2024-09-10T22:12:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f2994c00640 1 -- 172.21.15.66:0/2966176177 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2988009d20 con 0x7f29980aeec0 2024-09-10T22:12:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f2994c00640 1 -- 172.21.15.66:0/2966176177 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298800de60 con 0x7f29980aeec0 2024-09-10T22:12:58.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.775+0000 7f299f000640 1 -- 172.21.15.66:0/2966176177 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299814bd80 con 0x7f29980aeec0 2024-09-10T22:12:58.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.779+0000 7f299f000640 1 -- 172.21.15.66:0/2966176177 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299814a5c0 con 0x7f29980aeec0 2024-09-10T22:12:58.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.779+0000 7f299f000640 1 -- 172.21.15.66:0/2966176177 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29980b02d0 con 0x7f29980aeec0 2024-09-10T22:12:58.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.779+0000 7f2994c00640 1 -- 172.21.15.66:0/2966176177 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f2988004280 con 0x7f29980aeec0 2024-09-10T22:12:58.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.779+0000 7f2994c00640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29740743e0 0x7f29740768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:58.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.779+0000 7f2994c00640 1 -- 172.21.15.66:0/2966176177 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5055+0+0 (secure 0 0 0) 0x7f2988092620 con 0x7f29980aeec0 2024-09-10T22:12:58.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.779+0000 7f298f400640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29740743e0 0x7f29740768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:58.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.783+0000 7f298f400640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29740743e0 0x7f29740768a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f29800ae000 tx=0x7f298001b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.783+0000 7f2994c00640 1 -- 172.21.15.66:0/2966176177 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2988020020 con 0x7f29980aeec0 2024-09-10T22:12:58.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.955+0000 7f299f000640 1 -- 172.21.15.66:0/2966176177 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2998076650 con 0x7f29980aeec0 2024-09-10T22:12:58.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.959+0000 7f2994c00640 1 -- 172.21.15.66:0/2966176177 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7f29981f9f90 con 0x7f29980aeec0 2024-09-10T22:12:58.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.963+0000 7f299f000640 1 -- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29740743e0 msgr2=0x7f29740768a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:58.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.963+0000 7f299f000640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29740743e0 0x7f29740768a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f29800ae000 tx=0x7f298001b070 comp rx=0 tx=0).stop 2024-09-10T22:12:58.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.963+0000 7f299f000640 1 -- 172.21.15.66:0/2966176177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29980aeec0 msgr2=0x7f29981f9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:58.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.963+0000 7f299f000640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29980aeec0 0x7f29981f9a50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f298800d750 tx=0x7f298800dc10 comp rx=0 tx=0).stop 2024-09-10T22:12:58.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.963+0000 7f299f000640 1 -- 172.21.15.66:0/2966176177 shutdown_connections 2024-09-10T22:12:58.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.963+0000 7f299f000640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29740743e0 0x7f29740768a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.963+0000 7f299f000640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f299814adc0 0x7f299814b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.963+0000 7f299f000640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f299815a310 0x7f29980b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.963+0000 7f299f000640 1 --2- 172.21.15.66:0/2966176177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29980aeec0 0x7f29981f9a50 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:58.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.963+0000 7f299f000640 1 -- 172.21.15.66:0/2966176177 >> 172.21.15.66:0/2966176177 conn(0x7f29980cfa30 msgr2=0x7f299814d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:58.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.963+0000 7f299f000640 1 -- 172.21.15.66:0/2966176177 shutdown_connections 2024-09-10T22:12:58.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:58.963+0000 7f299f000640 1 -- 172.21.15.66:0/2966176177 wait complete. 2024-09-10T22:12:58.987 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:12:58.987 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:12:59.092 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:12:59.181 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:12:59.181 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:12:59.181 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:12:59.190 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:12:59.190 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:12:59.190 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:12:59.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:12:59.199 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:12:59.199 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:12:59.242 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:12:59.242 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:12:59.242 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:12:59.253 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:12:59.253 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T22:12:59.253 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:12:59.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.427+0000 7fb83f600640 1 -- 172.21.15.66:0/4276092929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8380d06b0 msgr2=0x7fb8380ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:59.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.427+0000 7fb83f600640 1 --2- 172.21.15.66:0/4276092929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8380d06b0 0x7fb8380ce170 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb8380d1940 tx=0x7fb82401c830 comp rx=0 tx=0).stop 2024-09-10T22:12:59.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.427+0000 7fb83f600640 1 -- 172.21.15.66:0/4276092929 shutdown_connections 2024-09-10T22:12:59.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.427+0000 7fb83f600640 1 --2- 172.21.15.66:0/4276092929 >> v2:172.21.15.94:3300/0 conn(0x7fb8380afc20 0x7fb8380ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.427+0000 7fb83f600640 1 --2- 172.21.15.66:0/4276092929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8380d06b0 0x7fb8380ce170 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.427+0000 7fb83f600640 1 -- 172.21.15.66:0/4276092929 >> 172.21.15.66:0/4276092929 conn(0x7fb8380d1c60 msgr2=0x7fb8380d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:59.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.427+0000 7fb83f600640 1 -- 172.21.15.66:0/4276092929 shutdown_connections 2024-09-10T22:12:59.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.427+0000 7fb83f600640 1 -- 172.21.15.66:0/4276092929 wait complete. 2024-09-10T22:12:59.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.427+0000 7fb83f600640 1 Processor -- start 2024-09-10T22:12:59.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.427+0000 7fb83f600640 1 -- start start 2024-09-10T22:12:59.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83f600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8380afc20 0x7fb8380b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:59.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83f600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8380d06b0 0x7fb8380b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:59.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83f600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83815bd40 0x7fb8380b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:59.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83f600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb838169250 con 0x7fb8380afc20 2024-09-10T22:12:59.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83f600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381690d0 con 0x7fb83815bd40 2024-09-10T22:12:59.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83f600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb8381693d0 con 0x7fb8380d06b0 2024-09-10T22:12:59.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8380afc20 0x7fb8380b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:59.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8380afc20 0x7fb8380b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59504/0 (socket says 172.21.15.66:59504) 2024-09-10T22:12:59.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83cc00640 1 -- 172.21.15.66:0/2332451524 learned_addr learned my addr 172.21.15.66:0/2332451524 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:59.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb837e00640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8380d06b0 0x7fb8380b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:59.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83d600640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83815bd40 0x7fb8380b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:59.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83cc00640 1 -- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8380d06b0 msgr2=0x7fb8380b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:59.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83cc00640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8380d06b0 0x7fb8380b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83cc00640 1 -- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83815bd40 msgr2=0x7fb8380b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:59.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83cc00640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83815bd40 0x7fb8380b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83cc00640 1 -- 172.21.15.66:0/2332451524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8381fbdc0 con 0x7fb8380afc20 2024-09-10T22:12:59.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83cc00640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8380afc20 0x7fb8380b2350 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00c3c0 tx=0x7fb82c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:59.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb834c00640 1 -- 172.21.15.66:0/2332451524 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c017070 con 0x7fb8380afc20 2024-09-10T22:12:59.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb834c00640 1 -- 172.21.15.66:0/2332451524 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb82c0053c0 con 0x7fb8380afc20 2024-09-10T22:12:59.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb834c00640 1 -- 172.21.15.66:0/2332451524 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c003860 con 0x7fb8380afc20 2024-09-10T22:12:59.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83f600640 1 -- 172.21.15.66:0/2332451524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8381fc0a0 con 0x7fb8380afc20 2024-09-10T22:12:59.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83f600640 1 -- 172.21.15.66:0/2332451524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8381fc660 con 0x7fb8380afc20 2024-09-10T22:12:59.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb83f600640 1 -- 172.21.15.66:0/2332451524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8380cffa0 con 0x7fb8380afc20 2024-09-10T22:12:59.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.431+0000 7fb834c00640 1 -- 172.21.15.66:0/2332451524 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7fb82c003a00 con 0x7fb8380afc20 2024-09-10T22:12:59.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.435+0000 7fb834c00640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8180743b0 0x7fb818076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:59.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.435+0000 7fb834c00640 1 -- 172.21.15.66:0/2332451524 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5055+0+0 (secure 0 0 0) 0x7fb82c091750 con 0x7fb8380afc20 2024-09-10T22:12:59.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.435+0000 7fb837e00640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8180743b0 0x7fb818076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:59.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.435+0000 7fb837e00640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8180743b0 0x7fb818076870 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb8240af000 tx=0x7fb82401b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:59.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.439+0000 7fb834c00640 1 -- 172.21.15.66:0/2332451524 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb82c05e2e0 con 0x7fb8380afc20 2024-09-10T22:12:59.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.647+0000 7fb83f600640 1 -- 172.21.15.66:0/2332451524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8380b0df0 con 0x7fb8380afc20 2024-09-10T22:12:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.647+0000 7fb834c00640 1 -- 172.21.15.66:0/2332451524 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+3908 (secure 0 0 0) 0x7fb82c062d10 con 0x7fb8380afc20 2024-09-10T22:12:59.653 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:12:59.653 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":79,"btime":"2024-09-10T22:12:53:214437+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1769669365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1769669365},{"type":"v1","addr":"172.21.15.94:6836","nonce":1769669365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8119,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3617287726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3617287726},{"type":"v1","addr":"172.21.15.94:6837","nonce":3617287726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8301,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/298680158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":298680158},{"type":"v1","addr":"172.21.15.66:6835","nonce":298680158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1808370701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1808370701},{"type":"v1","addr":"172.21.15.66:6834","nonce":1808370701}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[]} 2024-09-10T22:12:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 79 2024-09-10T22:12:59.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.651+0000 7fb83f600640 1 -- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8180743b0 msgr2=0x7fb818076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.651+0000 7fb83f600640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8180743b0 0x7fb818076870 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb8240af000 tx=0x7fb82401b070 comp rx=0 tx=0).stop 2024-09-10T22:12:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.651+0000 7fb83f600640 1 -- 172.21.15.66:0/2332451524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8380afc20 msgr2=0x7fb8380b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.651+0000 7fb83f600640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8380afc20 0x7fb8380b2350 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00c3c0 tx=0x7fb82c00c880 comp rx=0 tx=0).stop 2024-09-10T22:12:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.651+0000 7fb83f600640 1 -- 172.21.15.66:0/2332451524 shutdown_connections 2024-09-10T22:12:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.651+0000 7fb83f600640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb8180743b0 0x7fb818076870 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.651+0000 7fb83f600640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83815bd40 0x7fb8380b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.651+0000 7fb83f600640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8380d06b0 0x7fb8380b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.651+0000 7fb83f600640 1 --2- 172.21.15.66:0/2332451524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8380afc20 0x7fb8380b2350 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.651+0000 7fb83f600640 1 -- 172.21.15.66:0/2332451524 >> 172.21.15.66:0/2332451524 conn(0x7fb8380d1c60 msgr2=0x7fb8380713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.651+0000 7fb83f600640 1 -- 172.21.15.66:0/2332451524 shutdown_connections 2024-09-10T22:12:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.651+0000 7fb83f600640 1 -- 172.21.15.66:0/2332451524 wait complete. 2024-09-10T22:12:59.675 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:12:59.675 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:12:59.675 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:12:59.675 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:12:59.675 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:12:59.676 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:12:59.676 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:12:59.676 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:12:59.676 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:12:59.676 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:12:59.676 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:12:59.676 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:12:59.677 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:12:59.677 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:12:59.677 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:12:59.677 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:12:59.688 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:12:59.688 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:12:59.689 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:12:59.689 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:12:59.693 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:12:59.693 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:12:59.694 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:12:59.694 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:12:59.737 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:12:59.737 DEBUG:teuthology.parallel:result is None 2024-09-10T22:12:59.738 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:12:59.738 DEBUG:teuthology.parallel:result is None 2024-09-10T22:12:59.739 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:12:59.739 DEBUG:teuthology.parallel:result is None 2024-09-10T22:12:59.739 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:12:59.739 DEBUG:teuthology.parallel:result is None 2024-09-10T22:12:59.740 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T22:12:59.824 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:12:59.825 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:12:59.832 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:12:59.832 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:12:59.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.887+0000 7facf7c00640 1 -- 172.21.15.66:0/1902058607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf00c4fc0 msgr2=0x7facf006e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.887+0000 7facf7c00640 1 --2- 172.21.15.66:0/1902058607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf00c4fc0 0x7facf006e630 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7facdc009900 tx=0x7facdc01c8b0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.887+0000 7facf7c00640 1 -- 172.21.15.66:0/1902058607 shutdown_connections 2024-09-10T22:12:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.887+0000 7facf7c00640 1 --2- 172.21.15.66:0/1902058607 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facf015e130 0x7facf006eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.887+0000 7facf7c00640 1 --2- 172.21.15.66:0/1902058607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf00c4fc0 0x7facf006e630 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.887+0000 7facf7c00640 1 -- 172.21.15.66:0/1902058607 >> 172.21.15.66:0/1902058607 conn(0x7facf00cfa30 msgr2=0x7facf00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.887+0000 7facf7c00640 1 -- 172.21.15.66:0/1902058607 shutdown_connections 2024-09-10T22:12:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.887+0000 7facf7c00640 1 -- 172.21.15.66:0/1902058607 wait complete. 2024-09-10T22:12:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.887+0000 7facf7c00640 1 Processor -- start 2024-09-10T22:12:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.887+0000 7facf7c00640 1 -- start start 2024-09-10T22:12:59.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf7c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facf00c4fc0 0x7facf0162a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:59.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf7c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facf015dd50 0x7facf0142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:59.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf7c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf015e130 0x7facf0151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:59.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf7c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7facf0166c70 con 0x7facf015e130 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf7c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7facf0166af0 con 0x7facf00c4fc0 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf7c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7facf0166df0 con 0x7facf015dd50 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf5c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf015e130 0x7facf0151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf5c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf015e130 0x7facf0151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59556/0 (socket says 172.21.15.66:59556) 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf5c00640 1 -- 172.21.15.66:0/979081419 learned_addr learned my addr 172.21.15.66:0/979081419 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facefe00640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facf015dd50 0x7facf0142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf5200640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facf00c4fc0 0x7facf0162a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf5c00640 1 -- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facf015dd50 msgr2=0x7facf0142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf5c00640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facf015dd50 0x7facf0142ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf5c00640 1 -- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facf00c4fc0 msgr2=0x7facf0162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf5c00640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facf00c4fc0 0x7facf0162a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf5c00640 1 -- 172.21.15.66:0/979081419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf0151ad0 con 0x7facf015e130 2024-09-10T22:12:59.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf5c00640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf015e130 0x7facf0151590 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7face8009d10 tx=0x7face800c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facecc00640 1 -- 172.21.15.66:0/979081419 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face800ce00 con 0x7facf015e130 2024-09-10T22:12:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facecc00640 1 -- 172.21.15.66:0/979081419 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7face8004b30 con 0x7facf015e130 2024-09-10T22:12:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf7c00640 1 -- 172.21.15.66:0/979081419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf015ed10 con 0x7facf015e130 2024-09-10T22:12:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facecc00640 1 -- 172.21.15.66:0/979081419 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8005020 con 0x7facf015e130 2024-09-10T22:12:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf7c00640 1 -- 172.21.15.66:0/979081419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf0151d50 con 0x7facf015e130 2024-09-10T22:12:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.891+0000 7facf7c00640 1 -- 172.21.15.66:0/979081419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf00c48b0 con 0x7facf015e130 2024-09-10T22:12:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.895+0000 7facecc00640 1 -- 172.21.15.66:0/979081419 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7face8013070 con 0x7facf015e130 2024-09-10T22:12:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.895+0000 7facecc00640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7facd00743e0 0x7facd00768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.895+0000 7facecc00640 1 -- 172.21.15.66:0/979081419 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5055+0+0 (secure 0 0 0) 0x7face80925f0 con 0x7facf015e130 2024-09-10T22:12:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.895+0000 7facf5200640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7facd00743e0 0x7facd00768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.895+0000 7facf5200640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7facd00743e0 0x7facd00768a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7face4000f70 tx=0x7face4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:12:59.899+0000 7facecc00640 1 -- 172.21.15.66:0/979081419 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7facf0151ad0 con 0x7facf015e130 2024-09-10T22:13:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.083+0000 7facf7c00640 1 -- 172.21.15.66:0/979081419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7facf00cffb0 con 0x7facf015e130 2024-09-10T22:13:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facecc00640 1 -- 172.21.15.66:0/979081419 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) ==== 97+0+0 (secure 0 0 0) 0x7face8011070 con 0x7facf015e130 2024-09-10T22:13:00.259 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-09-10T22:13:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facf7c00640 1 -- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7facd00743e0 msgr2=0x7facd00768a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facf7c00640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7facd00743e0 0x7facd00768a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7face4000f70 tx=0x7face4009040 comp rx=0 tx=0).stop 2024-09-10T22:13:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facf7c00640 1 -- 172.21.15.66:0/979081419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf015e130 msgr2=0x7facf0151590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facf7c00640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf015e130 0x7facf0151590 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7face8009d10 tx=0x7face800c560 comp rx=0 tx=0).stop 2024-09-10T22:13:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facf7c00640 1 -- 172.21.15.66:0/979081419 shutdown_connections 2024-09-10T22:13:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facf7c00640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7facd00743e0 0x7facd00768a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facf7c00640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf015e130 0x7facf0151590 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facf7c00640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facf015dd50 0x7facf0142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facf7c00640 1 --2- 172.21.15.66:0/979081419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facf00c4fc0 0x7facf0162a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facf7c00640 1 -- 172.21.15.66:0/979081419 >> 172.21.15.66:0/979081419 conn(0x7facf00cfa30 msgr2=0x7facf015c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facf7c00640 1 -- 172.21.15.66:0/979081419 shutdown_connections 2024-09-10T22:13:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.255+0000 7facf7c00640 1 -- 172.21.15.66:0/979081419 wait complete. 2024-09-10T22:13:00.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T22:13:00.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 -- 172.21.15.66:0/3843749275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaf80c4b10 msgr2=0x7fcaf806e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:00.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 --2- 172.21.15.66:0/3843749275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaf80c4b10 0x7fcaf806e960 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8072560 tx=0x7fcae801c8c0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 -- 172.21.15.66:0/3843749275 shutdown_connections 2024-09-10T22:13:00.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 --2- 172.21.15.66:0/3843749275 >> v2:172.21.15.94:3300/0 conn(0x7fcaf80c4ef0 0x7fcaf80c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 --2- 172.21.15.66:0/3843749275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaf80c4b10 0x7fcaf806e960 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 -- 172.21.15.66:0/3843749275 >> 172.21.15.66:0/3843749275 conn(0x7fcaf80cfa30 msgr2=0x7fcaf80cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:00.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 -- 172.21.15.66:0/3843749275 shutdown_connections 2024-09-10T22:13:00.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 -- 172.21.15.66:0/3843749275 wait complete. 2024-09-10T22:13:00.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 Processor -- start 2024-09-10T22:13:00.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 -- start start 2024-09-10T22:13:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf80ae620 0x7fcaf80b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcaf80c4b10 0x7fcaf8158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaf80c4ef0 0x7fcaf814e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8167f00 con 0x7fcaf80c4ef0 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8167d80 con 0x7fcaf80ae620 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.427+0000 7fcafe400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf8168080 con 0x7fcaf80c4b10 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaf7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf80ae620 0x7fcaf80b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaf7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaf80c4ef0 0x7fcaf814e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaf7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf80ae620 0x7fcaf80b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:48922/0 (socket says 172.21.15.66:48922) 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaf7400640 1 -- 172.21.15.66:0/609059049 learned_addr learned my addr 172.21.15.66:0/609059049 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaefe00640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcaf80c4b10 0x7fcaf8158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaf7400640 1 -- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcaf80c4b10 msgr2=0x7fcaf8158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaf7400640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcaf80c4b10 0x7fcaf8158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaf7400640 1 -- 172.21.15.66:0/609059049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaf80c4ef0 msgr2=0x7fcaf814e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaf7400640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaf80c4ef0 0x7fcaf814e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaf7400640 1 -- 172.21.15.66:0/609059049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf815c4e0 con 0x7fcaf80ae620 2024-09-10T22:13:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaf7e00640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaf80c4ef0 0x7fcaf814e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:13:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaf7400640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf80ae620 0x7fcaf80b2940 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcae80b0040 tx=0x7fcae8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaef400640 1 -- 172.21.15.66:0/609059049 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80bb070 con 0x7fcaf80ae620 2024-09-10T22:13:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaef400640 1 -- 172.21.15.66:0/609059049 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcae8005070 con 0x7fcaf80ae620 2024-09-10T22:13:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcafe400640 1 -- 172.21.15.66:0/609059049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf815c780 con 0x7fcaf80ae620 2024-09-10T22:13:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaef400640 1 -- 172.21.15.66:0/609059049 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80b2040 con 0x7fcaf80ae620 2024-09-10T22:13:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcafe400640 1 -- 172.21.15.66:0/609059049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf815cd40 con 0x7fcaf80ae620 2024-09-10T22:13:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcafe400640 1 -- 172.21.15.66:0/609059049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf806e960 con 0x7fcaf80ae620 2024-09-10T22:13:00.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.431+0000 7fcaef400640 1 -- 172.21.15.66:0/609059049 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7fcae80054c0 con 0x7fcaf80ae620 2024-09-10T22:13:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.435+0000 7fcaef400640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcad40743b0 0x7fcad4076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.435+0000 7fcaef400640 1 -- 172.21.15.66:0/609059049 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4969+0+0 (secure 0 0 0) 0x7fcae812b8f0 con 0x7fcaf80ae620 2024-09-10T22:13:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.435+0000 7fcaefe00640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcad40743b0 0x7fcad4076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.435+0000 7fcaefe00640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcad40743b0 0x7fcad4076870 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcaf815e1a0 tx=0x7fcae0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:00.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.439+0000 7fcaef400640 1 -- 172.21.15.66:0/609059049 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcae80164b0 con 0x7fcaf80ae620 2024-09-10T22:13:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.671+0000 7fcafe400640 1 -- 172.21.15.66:0/609059049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcaf8072fc0 con 0x7fcaf80ae620 2024-09-10T22:13:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.671+0000 7fcaef400640 1 -- 172.21.15.66:0/609059049 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v33) ==== 80+0+5304 (secure 0 0 0) 0x7fcae80f8460 con 0x7fcaf80ae620 2024-09-10T22:13:00.677 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:00.677 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:13:00.677 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmxQD7CBAAttZP0KtdjTI7QWbD7CVinQ==", 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmI7ufFRAANelhvqylKyfGPVfBuY40kg==", 2024-09-10T22:13:00.678 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQnVoEBAAvsZ/TSOOP/JgJBr/aHwI/g==", 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:13:00.679 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmRQkTHRAAHPIdwqtkkyh31D7IO2vxPA==", 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBCweBmDJkxJBAAVSQNiMAQl5bR3RUwVOuGYw==", 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBFweBmwcdsOxAAPPQyzktcfiF8pJNP8uAjwg==", 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.681 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBJweBmXQhZExAAnEf8mjGX6nnYwWygKjo/2A==", 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBNweBmlauHAxAA/k9xRqbHZsqJq9OYbJKfNQ==", 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBRweBmHnFDJRAAgF1ZAPvmoNzeUftZQyL0eg==", 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.683 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:13:00.684 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:13:00.684 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.684 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.684 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.684 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-09-10T22:13:00.684 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBUweBm/hXpORAA6jpOI/I+QyB2mZsEoAWF0w==", 2024-09-10T22:13:00.684 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.684 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:13:00.684 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBYweBmYu2MExAAPD9WcxrBLzwwJQm19z7YcQ==", 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.685 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbweBmXfJ9KRAA/AbWlli+Qfg7gy0SblJ7fg==", 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw==", 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-09-10T22:13:00.686 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmbS6IKhAAaEmK9irxMipALgkAct/+OQ==", 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQ47KJhAAI50ZaSQIhH6FxRM48JZICA==", 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmUcLCMBAAdf4tfcpVaZoaKjS1UzrxhA==", 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.688 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA+weBmPIAzKBAAkyVV875PNhBmEoKsrAkg8w==", 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:13:00.689 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmLiy1BBAADnraML8IUet7Sw0j83TUVQ==", 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-09-10T22:13:00.690 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmItWpABAAlJ+6dFjvtBV3g+jYmUh0tQ==", 2024-09-10T22:13:00.691 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:13:00.691 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:13:00.691 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:13:00.691 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:13:00.691 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.691 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:00.691 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T22:13:00.691 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:13:00.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.675+0000 7fcafe400640 1 -- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcad40743b0 msgr2=0x7fcad4076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:00.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.675+0000 7fcafe400640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcad40743b0 0x7fcad4076870 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcaf815e1a0 tx=0x7fcae0009040 comp rx=0 tx=0).stop 2024-09-10T22:13:00.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.675+0000 7fcafe400640 1 -- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf80ae620 msgr2=0x7fcaf80b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.675+0000 7fcafe400640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf80ae620 0x7fcaf80b2940 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcae80b0040 tx=0x7fcae8004040 comp rx=0 tx=0).stop 2024-09-10T22:13:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.675+0000 7fcafe400640 1 -- 172.21.15.66:0/609059049 shutdown_connections 2024-09-10T22:13:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.675+0000 7fcafe400640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcad40743b0 0x7fcad4076870 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.675+0000 7fcafe400640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaf80c4ef0 0x7fcaf814e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.675+0000 7fcafe400640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcaf80c4b10 0x7fcaf8158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.675+0000 7fcafe400640 1 --2- 172.21.15.66:0/609059049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf80ae620 0x7fcaf80b2940 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.675+0000 7fcafe400640 1 -- 172.21.15.66:0/609059049 >> 172.21.15.66:0/609059049 conn(0x7fcaf80cfa30 msgr2=0x7fcaf80cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.675+0000 7fcafe400640 1 -- 172.21.15.66:0/609059049 shutdown_connections 2024-09-10T22:13:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.675+0000 7fcafe400640 1 -- 172.21.15.66:0/609059049 wait complete. 2024-09-10T22:13:00.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:13:00.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.839+0000 7f37f3800640 1 -- 172.21.15.66:0/1745810965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec0d0550 msgr2=0x7f37ec0ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.839+0000 7f37f3800640 1 --2- 172.21.15.66:0/1745810965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec0d0550 0x7f37ec0ca1a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009900 tx=0x7f37d801c850 comp rx=0 tx=0).stop 2024-09-10T22:13:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.839+0000 7f37f3800640 1 -- 172.21.15.66:0/1745810965 shutdown_connections 2024-09-10T22:13:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.839+0000 7f37f3800640 1 --2- 172.21.15.66:0/1745810965 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ec0c4fc0 0x7f37ec0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.839+0000 7f37f3800640 1 --2- 172.21.15.66:0/1745810965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec0d0550 0x7f37ec0ca1a0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.839+0000 7f37f3800640 1 -- 172.21.15.66:0/1745810965 >> 172.21.15.66:0/1745810965 conn(0x7f37ec0cfa30 msgr2=0x7f37ec0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:00.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.839+0000 7f37f3800640 1 -- 172.21.15.66:0/1745810965 shutdown_connections 2024-09-10T22:13:00.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.839+0000 7f37f3800640 1 -- 172.21.15.66:0/1745810965 wait complete. 2024-09-10T22:13:00.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f3800640 1 Processor -- start 2024-09-10T22:13:00.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f3800640 1 -- start start 2024-09-10T22:13:00.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f3800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ec0c4fc0 0x7f37ec0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:00.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f3800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec15a310 0x7f37ec0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:00.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f3800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec14a9b0 0x7f37ec1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:00.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f3800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec167480 con 0x7f37ec14a9b0 2024-09-10T22:13:00.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f3800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec167300 con 0x7f37ec15a310 2024-09-10T22:13:00.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f3800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f37ec167600 con 0x7f37ec0c4fc0 2024-09-10T22:13:00.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec14a9b0 0x7f37ec1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:00.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f0e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ec0c4fc0 0x7f37ec0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:00.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37ebe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec15a310 0x7f37ec0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:00.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec14a9b0 0x7f37ec1585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59590/0 (socket says 172.21.15.66:59590) 2024-09-10T22:13:00.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f1800640 1 -- 172.21.15.66:0/1932733088 learned_addr learned my addr 172.21.15.66:0/1932733088 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:00.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f1800640 1 -- 172.21.15.66:0/1932733088 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ec0c4fc0 msgr2=0x7f37ec0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:00.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f1800640 1 --2- 172.21.15.66:0/1932733088 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ec0c4fc0 0x7f37ec0b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f1800640 1 -- 172.21.15.66:0/1932733088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec15a310 msgr2=0x7f37ec0b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:00.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f1800640 1 --2- 172.21.15.66:0/1932733088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec15a310 0x7f37ec0b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:00.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f1800640 1 -- 172.21.15.66:0/1932733088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec14ff30 con 0x7f37ec14a9b0 2024-09-10T22:13:00.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f1800640 1 --2- 172.21.15.66:0/1932733088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec14a9b0 0x7f37ec1585d0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f37e400ddf0 tx=0x7f37e400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:00.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37e8c00640 1 -- 172.21.15.66:0/1932733088 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4004d30 con 0x7f37ec14a9b0 2024-09-10T22:13:00.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37e8c00640 1 -- 172.21.15.66:0/1932733088 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37e4005020 con 0x7f37ec14a9b0 2024-09-10T22:13:00.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37e8c00640 1 -- 172.21.15.66:0/1932733088 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4005430 con 0x7f37ec14a9b0 2024-09-10T22:13:00.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f3800640 1 -- 172.21.15.66:0/1932733088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec1501d0 con 0x7f37ec14a9b0 2024-09-10T22:13:00.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.843+0000 7f37f3800640 1 -- 172.21.15.66:0/1932733088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec0b3ed0 con 0x7f37ec14a9b0 2024-09-10T22:13:00.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.847+0000 7f37f3800640 1 -- 172.21.15.66:0/1932733088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec070420 con 0x7f37ec14a9b0 2024-09-10T22:13:00.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.847+0000 7f37e8c00640 1 -- 172.21.15.66:0/1932733088 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f37e400b040 con 0x7f37ec14a9b0 2024-09-10T22:13:00.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.847+0000 7f37e8c00640 1 --2- 172.21.15.66:0/1932733088 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f37cc0743b0 0x7f37cc076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:00.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.847+0000 7f37e8c00640 1 -- 172.21.15.66:0/1932733088 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4969+0+0 (secure 0 0 0) 0x7f37e4092940 con 0x7f37ec14a9b0 2024-09-10T22:13:00.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.847+0000 7f37f0e00640 1 --2- 172.21.15.66:0/1932733088 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f37cc0743b0 0x7f37cc076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:00.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.847+0000 7f37f0e00640 1 --2- 172.21.15.66:0/1932733088 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f37cc0743b0 0x7f37cc076870 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f37e0007fd0 tx=0x7f37e0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:00.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:00.851+0000 7f37e8c00640 1 -- 172.21.15.66:0/1932733088 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f37e405f4d0 con 0x7f37ec14a9b0 2024-09-10T22:13:01.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.027+0000 7f37f3800640 1 -- 172.21.15.66:0/1932733088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f37ec076650 con 0x7f37ec14a9b0 2024-09-10T22:13:01.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37e8c00640 1 -- 172.21.15.66:0/1932733088 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v122) ==== 84+0+65 (secure 0 0 0) 0x7f37e4068240 con 0x7f37ec14a9b0 2024-09-10T22:13:01.035 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:01.035 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:13:01.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:01.035 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:13:01.035 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:13:01.035 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:01.035 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:13:01.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37f3800640 1 -- 172.21.15.66:0/1932733088 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f37cc0743b0 msgr2=0x7f37cc076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:01.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37f3800640 1 --2- 172.21.15.66:0/1932733088 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f37cc0743b0 0x7f37cc076870 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f37e0007fd0 tx=0x7f37e0009040 comp rx=0 tx=0).stop 2024-09-10T22:13:01.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37f3800640 1 -- 172.21.15.66:0/1932733088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec14a9b0 msgr2=0x7f37ec1585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:01.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37f3800640 1 --2- 172.21.15.66:0/1932733088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec14a9b0 0x7f37ec1585d0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f37e400ddf0 tx=0x7f37e400b730 comp rx=0 tx=0).stop 2024-09-10T22:13:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37f3800640 1 -- 172.21.15.66:0/1932733088 shutdown_connections 2024-09-10T22:13:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37f3800640 1 --2- 172.21.15.66:0/1932733088 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f37cc0743b0 0x7f37cc076870 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37f3800640 1 --2- 172.21.15.66:0/1932733088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec14a9b0 0x7f37ec1585d0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37f3800640 1 --2- 172.21.15.66:0/1932733088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec15a310 0x7f37ec0b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37f3800640 1 --2- 172.21.15.66:0/1932733088 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ec0c4fc0 0x7f37ec0b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37f3800640 1 -- 172.21.15.66:0/1932733088 >> 172.21.15.66:0/1932733088 conn(0x7f37ec0cfa30 msgr2=0x7f37ec072680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37f3800640 1 -- 172.21.15.66:0/1932733088 shutdown_connections 2024-09-10T22:13:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.031+0000 7f37f3800640 1 -- 172.21.15.66:0/1932733088 wait complete. 2024-09-10T22:13:01.056 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:13:01.057 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:13:01.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.203+0000 7f5a8e200640 1 -- 172.21.15.66:0/3187942912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a880b1e10 msgr2=0x7f5a880b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:01.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.203+0000 7f5a8e200640 1 --2- 172.21.15.66:0/3187942912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a880b1e10 0x7f5a880b3170 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009bc0 tx=0x7f5a8001d460 comp rx=0 tx=0).stop 2024-09-10T22:13:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.203+0000 7f5a8e200640 1 -- 172.21.15.66:0/3187942912 shutdown_connections 2024-09-10T22:13:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.203+0000 7f5a8e200640 1 --2- 172.21.15.66:0/3187942912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a880b1e10 0x7f5a880b3170 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.203+0000 7f5a8e200640 1 -- 172.21.15.66:0/3187942912 >> 172.21.15.66:0/3187942912 conn(0x7f5a880d1c60 msgr2=0x7f5a880d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.203+0000 7f5a8e200640 1 -- 172.21.15.66:0/3187942912 shutdown_connections 2024-09-10T22:13:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.203+0000 7f5a8e200640 1 -- 172.21.15.66:0/3187942912 wait complete. 2024-09-10T22:13:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.203+0000 7f5a8e200640 1 Processor -- start 2024-09-10T22:13:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.203+0000 7f5a8e200640 1 -- start start 2024-09-10T22:13:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a8e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a880b1e10 0x7f5a8815dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a8e200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a8815f450 0x7f5a88154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a8e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a88160280 0x7f5a8806f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a8e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8816a040 con 0x7f5a880b1e10 2024-09-10T22:13:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a8e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88169ec0 con 0x7f5a88160280 2024-09-10T22:13:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a8e200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5a8816a1c0 con 0x7f5a8815f450 2024-09-10T22:13:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a87400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a880b1e10 0x7f5a8815dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a87400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a880b1e10 0x7f5a8815dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59596/0 (socket says 172.21.15.66:59596) 2024-09-10T22:13:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a87400640 1 -- 172.21.15.66:0/845048732 learned_addr learned my addr 172.21.15.66:0/845048732 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a87e00640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a88160280 0x7f5a8806f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a86a00640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a8815f450 0x7f5a88154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a87400640 1 -- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a8815f450 msgr2=0x7f5a88154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a87400640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a8815f450 0x7f5a88154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a87400640 1 -- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a88160280 msgr2=0x7f5a8806f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a87400640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a88160280 0x7f5a8806f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a87400640 1 -- 172.21.15.66:0/845048732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a88158830 con 0x7f5a880b1e10 2024-09-10T22:13:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a86a00640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a8815f450 0x7f5a88154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a87400640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a880b1e10 0x7f5a8815dd70 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5a78001230 tx=0x7f5a7800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:01.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a7f400640 1 -- 172.21.15.66:0/845048732 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78016020 con 0x7f5a880b1e10 2024-09-10T22:13:01.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a7f400640 1 -- 172.21.15.66:0/845048732 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a7800a930 con 0x7f5a880b1e10 2024-09-10T22:13:01.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a7f400640 1 -- 172.21.15.66:0/845048732 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800ad80 con 0x7f5a880b1e10 2024-09-10T22:13:01.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a8e200640 1 -- 172.21.15.66:0/845048732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8815e400 con 0x7f5a880b1e10 2024-09-10T22:13:01.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a8e200640 1 -- 172.21.15.66:0/845048732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a88165160 con 0x7f5a880b1e10 2024-09-10T22:13:01.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.207+0000 7f5a8e200640 1 -- 172.21.15.66:0/845048732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a880d2a40 con 0x7f5a880b1e10 2024-09-10T22:13:01.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.211+0000 7f5a7f400640 1 -- 172.21.15.66:0/845048732 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f5a7801b400 con 0x7f5a880b1e10 2024-09-10T22:13:01.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.211+0000 7f5a7f400640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5a680743b0 0x7f5a68076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:01.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.211+0000 7f5a7f400640 1 -- 172.21.15.66:0/845048732 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4969+0+0 (secure 0 0 0) 0x7f5a78092550 con 0x7f5a880b1e10 2024-09-10T22:13:01.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.211+0000 7f5a86a00640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5a680743b0 0x7f5a68076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:01.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.211+0000 7f5a86a00640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5a680743b0 0x7f5a68076870 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5a8815db30 tx=0x7f5a74009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:01.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.215+0000 7f5a7f400640 1 -- 172.21.15.66:0/845048732 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5a78020020 con 0x7f5a880b1e10 2024-09-10T22:13:01.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:01.391+0000 7f5a8e200640 1 -- 172.21.15.66:0/845048732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5a88074a80 con 0x7f5a880b1e10 2024-09-10T22:13:02.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.267+0000 7f5a7f400640 1 -- 172.21.15.66:0/845048732 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v123) ==== 138+0+0 (secure 0 0 0) 0x7f5a7805f730 con 0x7f5a880b1e10 2024-09-10T22:13:02.271 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:13:02.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.271+0000 7f5a8e200640 1 -- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5a680743b0 msgr2=0x7f5a68076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:02.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.271+0000 7f5a8e200640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5a680743b0 0x7f5a68076870 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5a8815db30 tx=0x7f5a74009040 comp rx=0 tx=0).stop 2024-09-10T22:13:02.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.271+0000 7f5a8e200640 1 -- 172.21.15.66:0/845048732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a880b1e10 msgr2=0x7f5a8815dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:02.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.271+0000 7f5a8e200640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a880b1e10 0x7f5a8815dd70 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5a78001230 tx=0x7f5a7800cb90 comp rx=0 tx=0).stop 2024-09-10T22:13:02.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.271+0000 7f5a8e200640 1 -- 172.21.15.66:0/845048732 shutdown_connections 2024-09-10T22:13:02.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.271+0000 7f5a8e200640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5a680743b0 0x7f5a68076870 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:02.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.271+0000 7f5a8e200640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a88160280 0x7f5a8806f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:02.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.271+0000 7f5a8e200640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a8815f450 0x7f5a88154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:02.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.271+0000 7f5a8e200640 1 --2- 172.21.15.66:0/845048732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a880b1e10 0x7f5a8815dd70 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:02.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.271+0000 7f5a8e200640 1 -- 172.21.15.66:0/845048732 >> 172.21.15.66:0/845048732 conn(0x7f5a880d1c60 msgr2=0x7f5a88072050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:02.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.271+0000 7f5a8e200640 1 -- 172.21.15.66:0/845048732 shutdown_connections 2024-09-10T22:13:02.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.271+0000 7f5a8e200640 1 -- 172.21.15.66:0/845048732 wait complete. 2024-09-10T22:13:02.304 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:13:02.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.455+0000 7f1f89e00640 1 -- 172.21.15.66:0/55110097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8415e460 msgr2=0x7f1f840ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.455+0000 7f1f89e00640 1 --2- 172.21.15.66:0/55110097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8415e460 0x7f1f840ccab0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009900 tx=0x7f1f6c01c810 comp rx=0 tx=0).stop 2024-09-10T22:13:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 -- 172.21.15.66:0/55110097 shutdown_connections 2024-09-10T22:13:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 --2- 172.21.15.66:0/55110097 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f8406ebc0 0x7f1f8406f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 --2- 172.21.15.66:0/55110097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8415e460 0x7f1f840ccab0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 --2- 172.21.15.66:0/55110097 >> v2:172.21.15.94:3300/0 conn(0x7f1f840ce660 0x7f1f840cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 -- 172.21.15.66:0/55110097 >> 172.21.15.66:0/55110097 conn(0x7f1f840d1c50 msgr2=0x7f1f840d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 -- 172.21.15.66:0/55110097 shutdown_connections 2024-09-10T22:13:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 -- 172.21.15.66:0/55110097 wait complete. 2024-09-10T22:13:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 Processor -- start 2024-09-10T22:13:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 -- start start 2024-09-10T22:13:02.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f8406ebc0 0x7f1f84154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:02.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f840ce660 0x7f1f8415a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8415e460 0x7f1f840af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f841625d0 con 0x7f1f8415e460 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f84162450 con 0x7f1f8406ebc0 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f89e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1f84162750 con 0x7f1f840ce660 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f83400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8415e460 0x7f1f840af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f83400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8415e460 0x7f1f840af9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39982/0 (socket says 172.21.15.66:39982) 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f83400640 1 -- 172.21.15.66:0/2193461332 learned_addr learned my addr 172.21.15.66:0/2193461332 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f82a00640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f8406ebc0 0x7f1f84154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f7be00640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f840ce660 0x7f1f8415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f83400640 1 -- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f840ce660 msgr2=0x7f1f8415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f83400640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f840ce660 0x7f1f8415a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f83400640 1 -- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f8406ebc0 msgr2=0x7f1f84154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f83400640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f8406ebc0 0x7f1f84154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:02.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f83400640 1 -- 172.21.15.66:0/2193461332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f84154fa0 con 0x7f1f8415e460 2024-09-10T22:13:02.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f83400640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8415e460 0x7f1f840af9b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00dac0 tx=0x7f1f7c00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:02.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.459+0000 7f1f7b400640 1 -- 172.21.15.66:0/2193461332 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00bdd0 con 0x7f1f8415e460 2024-09-10T22:13:02.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.463+0000 7f1f7b400640 1 -- 172.21.15.66:0/2193461332 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f7c003980 con 0x7f1f8415e460 2024-09-10T22:13:02.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.463+0000 7f1f89e00640 1 -- 172.21.15.66:0/2193461332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f84158fe0 con 0x7f1f8415e460 2024-09-10T22:13:02.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.463+0000 7f1f7b400640 1 -- 172.21.15.66:0/2193461332 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c003dd0 con 0x7f1f8415e460 2024-09-10T22:13:02.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.463+0000 7f1f89e00640 1 -- 172.21.15.66:0/2193461332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f841595a0 con 0x7f1f8415e460 2024-09-10T22:13:02.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.463+0000 7f1f89e00640 1 -- 172.21.15.66:0/2193461332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f840cea40 con 0x7f1f8415e460 2024-09-10T22:13:02.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.463+0000 7f1f7b400640 1 -- 172.21.15.66:0/2193461332 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f1f7c003f70 con 0x7f1f8415e460 2024-09-10T22:13:02.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.467+0000 7f1f7b400640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1f600743b0 0x7f1f60076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:02.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.467+0000 7f1f7b400640 1 -- 172.21.15.66:0/2193461332 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5339+0+0 (secure 0 0 0) 0x7f1f7c092de0 con 0x7f1f8415e460 2024-09-10T22:13:02.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.467+0000 7f1f82a00640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1f600743b0 0x7f1f60076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:02.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.467+0000 7f1f82a00640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1f600743b0 0x7f1f60076870 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009be0 tx=0x7f1f740091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:02.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.471+0000 7f1f7b400640 1 -- 172.21.15.66:0/2193461332 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1f7c020030 con 0x7f1f8415e460 2024-09-10T22:13:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:02.651+0000 7f1f89e00640 1 -- 172.21.15.66:0/2193461332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1f840ccaa0 con 0x7f1f8415e460 2024-09-10T22:13:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.271+0000 7f1f7b400640 1 -- 172.21.15.66:0/2193461332 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v124) ==== 170+0+0 (secure 0 0 0) 0x7f1f7c017030 con 0x7f1f8415e460 2024-09-10T22:13:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:13:03.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.275+0000 7f1f89e00640 1 -- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1f600743b0 msgr2=0x7f1f60076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:03.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.275+0000 7f1f89e00640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1f600743b0 0x7f1f60076870 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009be0 tx=0x7f1f740091e0 comp rx=0 tx=0).stop 2024-09-10T22:13:03.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.275+0000 7f1f89e00640 1 -- 172.21.15.66:0/2193461332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8415e460 msgr2=0x7f1f840af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:03.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.275+0000 7f1f89e00640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8415e460 0x7f1f840af9b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00dac0 tx=0x7f1f7c00df80 comp rx=0 tx=0).stop 2024-09-10T22:13:03.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.275+0000 7f1f89e00640 1 -- 172.21.15.66:0/2193461332 shutdown_connections 2024-09-10T22:13:03.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.275+0000 7f1f89e00640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1f600743b0 0x7f1f60076870 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:03.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.275+0000 7f1f89e00640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8415e460 0x7f1f840af9b0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:03.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.275+0000 7f1f89e00640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f840ce660 0x7f1f8415a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:03.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.275+0000 7f1f89e00640 1 --2- 172.21.15.66:0/2193461332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f8406ebc0 0x7f1f84154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:03.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.275+0000 7f1f89e00640 1 -- 172.21.15.66:0/2193461332 >> 172.21.15.66:0/2193461332 conn(0x7f1f840d1c50 msgr2=0x7f1f841606b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:03.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.275+0000 7f1f89e00640 1 -- 172.21.15.66:0/2193461332 shutdown_connections 2024-09-10T22:13:03.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.275+0000 7f1f89e00640 1 -- 172.21.15.66:0/2193461332 wait complete. 2024-09-10T22:13:03.315 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:13:03.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.459+0000 7ff044800640 1 -- 172.21.15.66:0/1349417630 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff03c15a310 msgr2=0x7ff03c0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:03.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.459+0000 7ff044800640 1 --2- 172.21.15.66:0/1349417630 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff03c15a310 0x7ff03c0c48b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff03000a980 tx=0x7ff03001db00 comp rx=0 tx=0).stop 2024-09-10T22:13:03.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.459+0000 7ff044800640 1 -- 172.21.15.66:0/1349417630 shutdown_connections 2024-09-10T22:13:03.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.459+0000 7ff044800640 1 --2- 172.21.15.66:0/1349417630 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff03c15a310 0x7ff03c0c48b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:03.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.459+0000 7ff044800640 1 -- 172.21.15.66:0/1349417630 >> 172.21.15.66:0/1349417630 conn(0x7ff03c0cfa30 msgr2=0x7ff03c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:03.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.459+0000 7ff044800640 1 -- 172.21.15.66:0/1349417630 shutdown_connections 2024-09-10T22:13:03.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.459+0000 7ff044800640 1 -- 172.21.15.66:0/1349417630 wait complete. 2024-09-10T22:13:03.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff044800640 1 Processor -- start 2024-09-10T22:13:03.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff044800640 1 -- start start 2024-09-10T22:13:03.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff044800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff03c0aeec0 0x7ff03c0c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:03.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff044800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff03c15a310 0x7ff03c1455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:03.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff044800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff03c0ad8c0 0x7ff03c0add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:03.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff044800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1686f0 con 0x7ff03c0aeec0 2024-09-10T22:13:03.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff044800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c168570 con 0x7ff03c0ad8c0 2024-09-10T22:13:03.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff044800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff03c168870 con 0x7ff03c15a310 2024-09-10T22:13:03.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff041e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff03c0aeec0 0x7ff03c0c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:03.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff041e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff03c0aeec0 0x7ff03c0c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39986/0 (socket says 172.21.15.66:39986) 2024-09-10T22:13:03.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff041e00640 1 -- 172.21.15.66:0/1403967481 learned_addr learned my addr 172.21.15.66:0/1403967481 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:03.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff041400640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff03c15a310 0x7ff03c1455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:03.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff042800640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff03c0ad8c0 0x7ff03c0add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:03.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff041e00640 1 -- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff03c15a310 msgr2=0x7ff03c1455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:03.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff041e00640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff03c15a310 0x7ff03c1455a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:03.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff041e00640 1 -- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff03c0ad8c0 msgr2=0x7ff03c0add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:03.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff041e00640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff03c0ad8c0 0x7ff03c0add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:03.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff041e00640 1 -- 172.21.15.66:0/1403967481 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c0727a0 con 0x7ff03c0aeec0 2024-09-10T22:13:03.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff041400640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff03c15a310 0x7ff03c1455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:03.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff042800640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff03c0ad8c0 0x7ff03c0add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:03.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff041e00640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff03c0aeec0 0x7ff03c0c6b00 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff03001b730 tx=0x7ff030004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:03.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff036000640 1 -- 172.21.15.66:0/1403967481 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0300b3040 con 0x7ff03c0aeec0 2024-09-10T22:13:03.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff036000640 1 -- 172.21.15.66:0/1403967481 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff030005230 con 0x7ff03c0aeec0 2024-09-10T22:13:03.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff036000640 1 -- 172.21.15.66:0/1403967481 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0300046d0 con 0x7ff03c0aeec0 2024-09-10T22:13:03.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff044800640 1 -- 172.21.15.66:0/1403967481 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03c072a80 con 0x7ff03c0aeec0 2024-09-10T22:13:03.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.463+0000 7ff044800640 1 -- 172.21.15.66:0/1403967481 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03c073040 con 0x7ff03c0aeec0 2024-09-10T22:13:03.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.467+0000 7ff044800640 1 -- 172.21.15.66:0/1403967481 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff03c0c48b0 con 0x7ff03c0aeec0 2024-09-10T22:13:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.467+0000 7ff036000640 1 -- 172.21.15.66:0/1403967481 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7ff03000b040 con 0x7ff03c0aeec0 2024-09-10T22:13:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.467+0000 7ff036000640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff0200743e0 0x7ff0200768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.467+0000 7ff036000640 1 -- 172.21.15.66:0/1403967481 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5721+0+0 (secure 0 0 0) 0x7ff03012c360 con 0x7ff03c0aeec0 2024-09-10T22:13:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.467+0000 7ff041400640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff0200743e0 0x7ff0200768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.467+0000 7ff041400640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff0200743e0 0x7ff0200768a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff03c1fb9a0 tx=0x7ff02c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:03.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.471+0000 7ff036000640 1 -- 172.21.15.66:0/1403967481 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff0300c5060 con 0x7ff03c0aeec0 2024-09-10T22:13:03.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:03.683+0000 7ff044800640 1 -- 172.21.15.66:0/1403967481 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff03c070420 con 0x7ff03c0aeec0 2024-09-10T22:13:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.275+0000 7ff036000640 1 -- 172.21.15.66:0/1403967481 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v82) ==== 174+0+0 (secure 0 0 0) 0x7ff030017780 con 0x7ff03c0aeec0 2024-09-10T22:13:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 18 and data pool 19 2024-09-10T22:13:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.279+0000 7ff044800640 1 -- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff0200743e0 msgr2=0x7ff0200768a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.279+0000 7ff044800640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff0200743e0 0x7ff0200768a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff03c1fb9a0 tx=0x7ff02c009040 comp rx=0 tx=0).stop 2024-09-10T22:13:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.279+0000 7ff044800640 1 -- 172.21.15.66:0/1403967481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff03c0aeec0 msgr2=0x7ff03c0c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.279+0000 7ff044800640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff03c0aeec0 0x7ff03c0c6b00 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff03001b730 tx=0x7ff030004600 comp rx=0 tx=0).stop 2024-09-10T22:13:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.279+0000 7ff044800640 1 -- 172.21.15.66:0/1403967481 shutdown_connections 2024-09-10T22:13:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.279+0000 7ff044800640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff0200743e0 0x7ff0200768a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.279+0000 7ff044800640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff03c0ad8c0 0x7ff03c0add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.279+0000 7ff044800640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff03c15a310 0x7ff03c1455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.279+0000 7ff044800640 1 --2- 172.21.15.66:0/1403967481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff03c0aeec0 0x7ff03c0c6b00 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.279+0000 7ff044800640 1 -- 172.21.15.66:0/1403967481 >> 172.21.15.66:0/1403967481 conn(0x7ff03c0cfa30 msgr2=0x7ff03c14eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.279+0000 7ff044800640 1 -- 172.21.15.66:0/1403967481 shutdown_connections 2024-09-10T22:13:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.279+0000 7ff044800640 1 -- 172.21.15.66:0/1403967481 wait complete. 2024-09-10T22:13:04.328 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:13:04.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.467+0000 7f8c22000640 1 -- 172.21.15.66:0/1447961679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c06e960 msgr2=0x7f8c1c0cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:04.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.467+0000 7f8c22000640 1 --2- 172.21.15.66:0/1447961679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c06e960 0x7f8c1c0cb8a0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009c80 tx=0x7f8c1401cda0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.467+0000 7f8c22000640 1 -- 172.21.15.66:0/1447961679 shutdown_connections 2024-09-10T22:13:04.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.467+0000 7f8c22000640 1 --2- 172.21.15.66:0/1447961679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c06e960 0x7f8c1c0cb8a0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.467+0000 7f8c22000640 1 -- 172.21.15.66:0/1447961679 >> 172.21.15.66:0/1447961679 conn(0x7f8c1c0cfa30 msgr2=0x7f8c1c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:04.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.467+0000 7f8c22000640 1 -- 172.21.15.66:0/1447961679 shutdown_connections 2024-09-10T22:13:04.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.467+0000 7f8c22000640 1 -- 172.21.15.66:0/1447961679 wait complete. 2024-09-10T22:13:04.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c22000640 1 Processor -- start 2024-09-10T22:13:04.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c22000640 1 -- start start 2024-09-10T22:13:04.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c22000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c06e960 0x7f8c1c140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:04.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c22000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c1c0ae620 0x7f8c1c1408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:04.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c22000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0c4b10 0x7f8c1c140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:04.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c22000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1697b0 con 0x7f8c1c06e960 2024-09-10T22:13:04.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c22000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c169630 con 0x7f8c1c0c4b10 2024-09-10T22:13:04.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c22000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1c169930 con 0x7f8c1c0ae620 2024-09-10T22:13:04.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c1b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c06e960 0x7f8c1c140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:04.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c1b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c06e960 0x7f8c1c140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39996/0 (socket says 172.21.15.66:39996) 2024-09-10T22:13:04.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c1b400640 1 -- 172.21.15.66:0/2644091306 learned_addr learned my addr 172.21.15.66:0/2644091306 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:04.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c12a00640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c1c0ae620 0x7f8c1c1408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c1be00640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0c4b10 0x7f8c1c140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c1b400640 1 -- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c1c0ae620 msgr2=0x7f8c1c1408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c1b400640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c1c0ae620 0x7f8c1c1408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c1b400640 1 -- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0c4b10 msgr2=0x7f8c1c140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c1b400640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0c4b10 0x7f8c1c140e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c1b400640 1 -- 172.21.15.66:0/2644091306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c15e450 con 0x7f8c1c06e960 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c12a00640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c1c0ae620 0x7f8c1c1408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c1be00640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0c4b10 0x7f8c1c140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c1b400640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c06e960 0x7f8c1c140380 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00c500 tx=0x7f8c0c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c13e00640 1 -- 172.21.15.66:0/2644091306 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c017070 con 0x7f8c1c06e960 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c13e00640 1 -- 172.21.15.66:0/2644091306 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c0c0053c0 con 0x7f8c1c06e960 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c13e00640 1 -- 172.21.15.66:0/2644091306 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c012890 con 0x7f8c1c06e960 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c22000640 1 -- 172.21.15.66:0/2644091306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c15d3b0 con 0x7f8c1c06e960 2024-09-10T22:13:04.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.471+0000 7f8c22000640 1 -- 172.21.15.66:0/2644091306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c15d970 con 0x7f8c1c06e960 2024-09-10T22:13:04.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.475+0000 7f8c22000640 1 -- 172.21.15.66:0/2644091306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1c0cb8a0 con 0x7f8c1c06e960 2024-09-10T22:13:04.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.475+0000 7f8c13e00640 1 -- 172.21.15.66:0/2644091306 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f8c0c012a30 con 0x7f8c1c06e960 2024-09-10T22:13:04.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.475+0000 7f8c13e00640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8bf80743b0 0x7f8bf8076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:04.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.475+0000 7f8c13e00640 1 -- 172.21.15.66:0/2644091306 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5821+0+0 (secure 0 0 0) 0x7f8c0c091ac0 con 0x7f8c1c06e960 2024-09-10T22:13:04.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.475+0000 7f8c12a00640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8bf80743b0 0x7f8bf8076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:04.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.479+0000 7f8c12a00640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8bf80743b0 0x7f8bf8076870 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c1541b0 tx=0x7f8c04009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:04.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.479+0000 7f8c13e00640 1 -- 172.21.15.66:0/2644091306 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8c0c021080 con 0x7f8c1c06e960 2024-09-10T22:13:04.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.655+0000 7f8c22000640 1 -- 172.21.15.66:0/2644091306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c1c0cafb0 con 0x7f8c1c06e960 2024-09-10T22:13:04.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.655+0000 7f8c13e00640 1 -- 172.21.15.66:0/2644091306 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16858 (secure 0 0 0) 0x7f8c0c05ec00 con 0x7f8c1c06e960 2024-09-10T22:13:04.661 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:04.661 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":125,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:13:03.687900+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:13:01.405668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T22:13:02.664579+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:13:04.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.659+0000 7f8c22000640 1 -- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8bf80743b0 msgr2=0x7f8bf8076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:04.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.659+0000 7f8c22000640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8bf80743b0 0x7f8bf8076870 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c1541b0 tx=0x7f8c04009040 comp rx=0 tx=0).stop 2024-09-10T22:13:04.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.659+0000 7f8c22000640 1 -- 172.21.15.66:0/2644091306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c06e960 msgr2=0x7f8c1c140380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:04.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.659+0000 7f8c22000640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c06e960 0x7f8c1c140380 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00c500 tx=0x7f8c0c00c9c0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.659+0000 7f8c22000640 1 -- 172.21.15.66:0/2644091306 shutdown_connections 2024-09-10T22:13:04.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.659+0000 7f8c22000640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8bf80743b0 0x7f8bf8076870 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.659+0000 7f8c22000640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0c4b10 0x7f8c1c140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.659+0000 7f8c22000640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c1c0ae620 0x7f8c1c1408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.659+0000 7f8c22000640 1 --2- 172.21.15.66:0/2644091306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c06e960 0x7f8c1c140380 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.659+0000 7f8c22000640 1 -- 172.21.15.66:0/2644091306 >> 172.21.15.66:0/2644091306 conn(0x7f8c1c0cfa30 msgr2=0x7f8c1c0b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:04.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.659+0000 7f8c22000640 1 -- 172.21.15.66:0/2644091306 shutdown_connections 2024-09-10T22:13:04.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.659+0000 7f8c22000640 1 -- 172.21.15.66:0/2644091306 wait complete. 2024-09-10T22:13:04.684 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:13:04.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.823+0000 7fe70de00640 1 -- 172.21.15.66:0/377725238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70815f460 msgr2=0x7fe7080cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:04.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.823+0000 7fe70de00640 1 --2- 172.21.15.66:0/377725238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70815f460 0x7fe7080cfc00 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fe708073ef0 tx=0x7fe6f801c770 comp rx=0 tx=0).stop 2024-09-10T22:13:04.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 -- 172.21.15.66:0/377725238 shutdown_connections 2024-09-10T22:13:04.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 --2- 172.21.15.66:0/377725238 >> v2:172.21.15.94:3300/0 conn(0x7fe7080b32a0 0x7fe7080af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 --2- 172.21.15.66:0/377725238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70815f460 0x7fe7080cfc00 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 -- 172.21.15.66:0/377725238 >> 172.21.15.66:0/377725238 conn(0x7fe7080d1d00 msgr2=0x7fe7080d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:04.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 -- 172.21.15.66:0/377725238 shutdown_connections 2024-09-10T22:13:04.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 -- 172.21.15.66:0/377725238 wait complete. 2024-09-10T22:13:04.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 Processor -- start 2024-09-10T22:13:04.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 -- start start 2024-09-10T22:13:04.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7080b0f60 0x7fe7081f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:04.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7080b32a0 0x7fe708149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe70815f460 0x7fe70806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe7080b0520 con 0x7fe7080b32a0 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7080b03a0 con 0x7fe7080b0f60 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe70de00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe7080b06a0 con 0x7fe70815f460 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe706000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7080b32a0 0x7fe708149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe706000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7080b32a0 0x7fe708149a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40004/0 (socket says 172.21.15.66:40004) 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe706000640 1 -- 172.21.15.66:0/3016313198 learned_addr learned my addr 172.21.15.66:0/3016313198 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe706a00640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7080b0f60 0x7fe7081f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe707400640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe70815f460 0x7fe70806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe706000640 1 -- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe70815f460 msgr2=0x7fe70806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe706000640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe70815f460 0x7fe70806e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe706000640 1 -- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7080b0f60 msgr2=0x7fe7081f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe706000640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7080b0f60 0x7fe7081f7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:04.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe706000640 1 -- 172.21.15.66:0/3016313198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe708149fd0 con 0x7fe7080b32a0 2024-09-10T22:13:04.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe706000640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7080b32a0 0x7fe708149a90 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c500 tx=0x7fe6f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:04.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe6ff400640 1 -- 172.21.15.66:0/3016313198 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4017070 con 0x7fe7080b32a0 2024-09-10T22:13:04.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe6ff400640 1 -- 172.21.15.66:0/3016313198 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6f4003600 con 0x7fe7080b32a0 2024-09-10T22:13:04.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.827+0000 7fe6ff400640 1 -- 172.21.15.66:0/3016313198 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4003a90 con 0x7fe7080b32a0 2024-09-10T22:13:04.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.831+0000 7fe70de00640 1 -- 172.21.15.66:0/3016313198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70806f120 con 0x7fe7080b32a0 2024-09-10T22:13:04.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.831+0000 7fe70de00640 1 -- 172.21.15.66:0/3016313198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70806ded0 con 0x7fe7080b32a0 2024-09-10T22:13:04.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.831+0000 7fe70de00640 1 -- 172.21.15.66:0/3016313198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7080af390 con 0x7fe7080b32a0 2024-09-10T22:13:04.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.831+0000 7fe6ff400640 1 -- 172.21.15.66:0/3016313198 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7fe6f4003c30 con 0x7fe7080b32a0 2024-09-10T22:13:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.831+0000 7fe6ff400640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe6e80743b0 0x7fe6e8076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.831+0000 7fe6ff400640 1 -- 172.21.15.66:0/3016313198 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5821+0+0 (secure 0 0 0) 0x7fe6f4091980 con 0x7fe7080b32a0 2024-09-10T22:13:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.831+0000 7fe706a00640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe6e80743b0 0x7fe6e8076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:04.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.835+0000 7fe706a00640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe6e80743b0 0x7fe6e8076870 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80af000 tx=0x7fe6f80a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:04.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:04.835+0000 7fe6ff400640 1 -- 172.21.15.66:0/3016313198 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe708149fd0 con 0x7fe7080b32a0 2024-09-10T22:13:05.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.011+0000 7fe70de00640 1 -- 172.21.15.66:0/3016313198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe708074aa0 con 0x7fe7080b32a0 2024-09-10T22:13:05.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe6ff400640 1 -- 172.21.15.66:0/3016313198 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16858 (secure 0 0 0) 0x7fe6f40672a0 con 0x7fe7080b32a0 2024-09-10T22:13:05.019 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:05.019 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":125,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:13:03.687900+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:13:01.405668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T22:13:02.664579+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:13:05.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe70de00640 1 -- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe6e80743b0 msgr2=0x7fe6e8076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:05.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe70de00640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe6e80743b0 0x7fe6e8076870 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80af000 tx=0x7fe6f80a7040 comp rx=0 tx=0).stop 2024-09-10T22:13:05.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe70de00640 1 -- 172.21.15.66:0/3016313198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7080b32a0 msgr2=0x7fe708149a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:05.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe70de00640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7080b32a0 0x7fe708149a90 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c500 tx=0x7fe6f400c9c0 comp rx=0 tx=0).stop 2024-09-10T22:13:05.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe70de00640 1 -- 172.21.15.66:0/3016313198 shutdown_connections 2024-09-10T22:13:05.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe70de00640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe6e80743b0 0x7fe6e8076870 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:05.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe70de00640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe70815f460 0x7fe70806e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:05.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe70de00640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7080b32a0 0x7fe708149a90 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:05.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe70de00640 1 --2- 172.21.15.66:0/3016313198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7080b0f60 0x7fe7081f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:05.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe70de00640 1 -- 172.21.15.66:0/3016313198 >> 172.21.15.66:0/3016313198 conn(0x7fe7080d1d00 msgr2=0x7fe70815cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:05.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe70de00640 1 -- 172.21.15.66:0/3016313198 shutdown_connections 2024-09-10T22:13:05.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.015+0000 7fe70de00640 1 -- 172.21.15.66:0/3016313198 wait complete. 2024-09-10T22:13:05.041 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:13:05.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 -- 172.21.15.66:0/709535430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03880d0260 msgr2=0x7f03880ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:05.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 --2- 172.21.15.66:0/709535430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03880d0260 0x7f03880ae960 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f038806d6f0 tx=0x7f037001c8c0 comp rx=0 tx=0).stop 2024-09-10T22:13:05.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 -- 172.21.15.66:0/709535430 shutdown_connections 2024-09-10T22:13:05.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 --2- 172.21.15.66:0/709535430 >> v2:172.21.15.94:3300/0 conn(0x7f03880b2bf0 0x7f03880aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:05.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 --2- 172.21.15.66:0/709535430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03880d0260 0x7f03880ae960 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:05.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 -- 172.21.15.66:0/709535430 >> 172.21.15.66:0/709535430 conn(0x7f03880d1c50 msgr2=0x7f03880d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:05.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 -- 172.21.15.66:0/709535430 shutdown_connections 2024-09-10T22:13:05.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 -- 172.21.15.66:0/709535430 wait complete. 2024-09-10T22:13:05.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 Processor -- start 2024-09-10T22:13:05.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 -- start start 2024-09-10T22:13:05.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03880b2bf0 0x7f03881f8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:05.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03880d0260 0x7f03880ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:05.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f038815f340 0x7f038815ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f03881691e0 con 0x7f03880b2bf0 2024-09-10T22:13:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0388169060 con 0x7f03880d0260 2024-09-10T22:13:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.187+0000 7f038d600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0388169360 con 0x7f038815f340 2024-09-10T22:13:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f0386a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03880b2bf0 0x7f03881f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f0386a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03880b2bf0 0x7f03881f8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40018/0 (socket says 172.21.15.66:40018) 2024-09-10T22:13:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f0386a00640 1 -- 172.21.15.66:0/198106310 learned_addr learned my addr 172.21.15.66:0/198106310 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f037fe00640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03880d0260 0x7f03880ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f0387400640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f038815f340 0x7f038815ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f0386a00640 1 -- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f038815f340 msgr2=0x7f038815ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:05.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f0386a00640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f038815f340 0x7f038815ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:05.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f0386a00640 1 -- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03880d0260 msgr2=0x7f03880ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:05.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f0386a00640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03880d0260 0x7f03880ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:05.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f0386a00640 1 -- 172.21.15.66:0/198106310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03880ca1c0 con 0x7f03880b2bf0 2024-09-10T22:13:05.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f0386a00640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03880b2bf0 0x7f03881f8250 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f037800c3c0 tx=0x7f037800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:05.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f037ea00640 1 -- 172.21.15.66:0/198106310 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378017070 con 0x7f03880b2bf0 2024-09-10T22:13:05.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f037ea00640 1 -- 172.21.15.66:0/198106310 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03780053c0 con 0x7f03880b2bf0 2024-09-10T22:13:05.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f037ea00640 1 -- 172.21.15.66:0/198106310 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378003860 con 0x7f03880b2bf0 2024-09-10T22:13:05.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f038d600640 1 -- 172.21.15.66:0/198106310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03880ca4a0 con 0x7f03880b2bf0 2024-09-10T22:13:05.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f038d600640 1 -- 172.21.15.66:0/198106310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03880caa60 con 0x7f03880b2bf0 2024-09-10T22:13:05.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f038d600640 1 -- 172.21.15.66:0/198106310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03880cfb50 con 0x7f03880b2bf0 2024-09-10T22:13:05.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.191+0000 7f037ea00640 1 -- 172.21.15.66:0/198106310 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f0378003a00 con 0x7f03880b2bf0 2024-09-10T22:13:05.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.195+0000 7f037ea00640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03640743b0 0x7f0364076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:05.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.195+0000 7f037ea00640 1 -- 172.21.15.66:0/198106310 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5821+0+0 (secure 0 0 0) 0x7f03780918a0 con 0x7f03880b2bf0 2024-09-10T22:13:05.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.195+0000 7f037fe00640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03640743b0 0x7f0364076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.195+0000 7f037fe00640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03640743b0 0x7f0364076870 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f03700af000 tx=0x7f037001b100 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:05.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.199+0000 7f037ea00640 1 -- 172.21.15.66:0/198106310 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f037805e9e0 con 0x7f03880b2bf0 2024-09-10T22:13:05.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:05.415+0000 7f038d600640 1 -- 172.21.15.66:0/198106310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f03880d08e0 con 0x7f03880b2bf0 2024-09-10T22:13:06.268 INFO:tasks.ceph.mds.b.smithi094.stderr:2024-09-10T22:13:06.268+0000 7f1309000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:13:06.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.299+0000 7f037ea00640 1 -- 172.21.15.66:0/198106310 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) ==== 118+0+0 (secure 0 0 0) 0x7f037805e2e0 con 0x7f03880b2bf0 2024-09-10T22:13:06.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.303+0000 7f038d600640 1 -- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03640743b0 msgr2=0x7f0364076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:06.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.303+0000 7f038d600640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03640743b0 0x7f0364076870 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f03700af000 tx=0x7f037001b100 comp rx=0 tx=0).stop 2024-09-10T22:13:06.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.303+0000 7f038d600640 1 -- 172.21.15.66:0/198106310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03880b2bf0 msgr2=0x7f03881f8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:06.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.303+0000 7f038d600640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03880b2bf0 0x7f03881f8250 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f037800c3c0 tx=0x7f037800c880 comp rx=0 tx=0).stop 2024-09-10T22:13:06.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.303+0000 7f038d600640 1 -- 172.21.15.66:0/198106310 shutdown_connections 2024-09-10T22:13:06.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.303+0000 7f038d600640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03640743b0 0x7f0364076870 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:06.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.303+0000 7f038d600640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f038815f340 0x7f038815ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:06.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.303+0000 7f038d600640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03880d0260 0x7f03880ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:06.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.303+0000 7f038d600640 1 --2- 172.21.15.66:0/198106310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03880b2bf0 0x7f03881f8250 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:06.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.303+0000 7f038d600640 1 -- 172.21.15.66:0/198106310 >> 172.21.15.66:0/198106310 conn(0x7f03880d1c50 msgr2=0x7f03881666d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:06.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.303+0000 7f038d600640 1 -- 172.21.15.66:0/198106310 shutdown_connections 2024-09-10T22:13:06.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.303+0000 7f038d600640 1 -- 172.21.15.66:0/198106310 wait complete. 2024-09-10T22:13:06.334 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:13:06.334 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:13:06.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.483+0000 7f0bf6600640 1 -- 172.21.15.66:0/873835079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bf00aee20 msgr2=0x7f0bf00cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:06.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.483+0000 7f0bf6600640 1 --2- 172.21.15.66:0/873835079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bf00aee20 0x7f0bf00cd930 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009bc0 tx=0x7f0be801cb10 comp rx=0 tx=0).stop 2024-09-10T22:13:06.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.483+0000 7f0bf6600640 1 -- 172.21.15.66:0/873835079 shutdown_connections 2024-09-10T22:13:06.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.483+0000 7f0bf6600640 1 --2- 172.21.15.66:0/873835079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bf00aee20 0x7f0bf00cd930 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:06.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.483+0000 7f0bf6600640 1 --2- 172.21.15.66:0/873835079 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0bf0159900 0x7f0bf00ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:06.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.483+0000 7f0bf6600640 1 -- 172.21.15.66:0/873835079 >> 172.21.15.66:0/873835079 conn(0x7f0bf00cfa30 msgr2=0x7f0bf00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:06.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.483+0000 7f0bf6600640 1 -- 172.21.15.66:0/873835079 shutdown_connections 2024-09-10T22:13:06.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.483+0000 7f0bf6600640 1 -- 172.21.15.66:0/873835079 wait complete. 2024-09-10T22:13:06.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.483+0000 7f0bf6600640 1 Processor -- start 2024-09-10T22:13:06.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.483+0000 7f0bf6600640 1 -- start start 2024-09-10T22:13:06.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bf6600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bf00aee20 0x7f0bf00c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:06.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bf6600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0bf0159900 0x7f0bf014ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:06.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bf6600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bf015cc80 0x7f0bf01fb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:06.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bf6600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf0168f70 con 0x7f0bf00aee20 2024-09-10T22:13:06.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bf6600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf0168df0 con 0x7f0bf015cc80 2024-09-10T22:13:06.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bf6600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf01690f0 con 0x7f0bf0159900 2024-09-10T22:13:06.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bef400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bf00aee20 0x7f0bf00c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:06.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bef400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bf00aee20 0x7f0bf00c4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40028/0 (socket says 172.21.15.66:40028) 2024-09-10T22:13:06.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bef400640 1 -- 172.21.15.66:0/3939865424 learned_addr learned my addr 172.21.15.66:0/3939865424 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:06.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0befe00640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bf015cc80 0x7f0bf01fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:06.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0be7e00640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0bf0159900 0x7f0bf014ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:06.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bef400640 1 -- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0bf0159900 msgr2=0x7f0bf014ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:06.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bef400640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0bf0159900 0x7f0bf014ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:06.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bef400640 1 -- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bf015cc80 msgr2=0x7f0bf01fb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:06.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bef400640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bf015cc80 0x7f0bf01fb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:06.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bef400640 1 -- 172.21.15.66:0/3939865424 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf014f270 con 0x7f0bf00aee20 2024-09-10T22:13:06.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bef400640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bf00aee20 0x7f0bf00c4f70 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f0be000d980 tx=0x7f0be000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:06.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0be7400640 1 -- 172.21.15.66:0/3939865424 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0017070 con 0x7f0bf00aee20 2024-09-10T22:13:06.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0be7400640 1 -- 172.21.15.66:0/3939865424 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0be0009d20 con 0x7f0bf00aee20 2024-09-10T22:13:06.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0be7400640 1 -- 172.21.15.66:0/3939865424 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0004280 con 0x7f0bf00aee20 2024-09-10T22:13:06.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bf6600640 1 -- 172.21.15.66:0/3939865424 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf01fbd90 con 0x7f0bf00aee20 2024-09-10T22:13:06.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bf6600640 1 -- 172.21.15.66:0/3939865424 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf0157a40 con 0x7f0bf00aee20 2024-09-10T22:13:06.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.487+0000 7f0bf6600640 1 -- 172.21.15.66:0/3939865424 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf00cd930 con 0x7f0bf00aee20 2024-09-10T22:13:06.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.491+0000 7f0be7400640 1 -- 172.21.15.66:0/3939865424 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f0be0003600 con 0x7f0bf00aee20 2024-09-10T22:13:06.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.491+0000 7f0be7400640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0bcc0743b0 0x7f0bcc076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:06.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.491+0000 7f0be7400640 1 -- 172.21.15.66:0/3939865424 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f0be00926d0 con 0x7f0bf00aee20 2024-09-10T22:13:06.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.491+0000 7f0be7e00640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0bcc0743b0 0x7f0bcc076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:06.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.491+0000 7f0be7e00640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0bcc0743b0 0x7f0bcc076870 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0bf015c020 tx=0x7f0bd8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:06.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.495+0000 7f0be7400640 1 -- 172.21.15.66:0/3939865424 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0be005f8b0 con 0x7f0bf00aee20 2024-09-10T22:13:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:06.703+0000 7f0bf6600640 1 -- 172.21.15.66:0/3939865424 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0bf0070470 con 0x7f0bf00aee20 2024-09-10T22:13:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.307+0000 7f0be7400640 1 -- 172.21.15.66:0/3939865424 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) ==== 122+0+0 (secure 0 0 0) 0x7f0be0064e20 con 0x7f0bf00aee20 2024-09-10T22:13:07.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.307+0000 7f0bf6600640 1 -- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0bcc0743b0 msgr2=0x7f0bcc076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:07.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.307+0000 7f0bf6600640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0bcc0743b0 0x7f0bcc076870 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0bf015c020 tx=0x7f0bd8009040 comp rx=0 tx=0).stop 2024-09-10T22:13:07.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.307+0000 7f0bf6600640 1 -- 172.21.15.66:0/3939865424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bf00aee20 msgr2=0x7f0bf00c4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:07.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.307+0000 7f0bf6600640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bf00aee20 0x7f0bf00c4f70 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f0be000d980 tx=0x7f0be000de40 comp rx=0 tx=0).stop 2024-09-10T22:13:07.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.307+0000 7f0bf6600640 1 -- 172.21.15.66:0/3939865424 shutdown_connections 2024-09-10T22:13:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.307+0000 7f0bf6600640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0bcc0743b0 0x7f0bcc076870 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.307+0000 7f0bf6600640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bf015cc80 0x7f0bf01fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.307+0000 7f0bf6600640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0bf0159900 0x7f0bf014ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.307+0000 7f0bf6600640 1 --2- 172.21.15.66:0/3939865424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bf00aee20 0x7f0bf00c4f70 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.307+0000 7f0bf6600640 1 -- 172.21.15.66:0/3939865424 >> 172.21.15.66:0/3939865424 conn(0x7f0bf00cfa30 msgr2=0x7f0bf0072580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.311+0000 7f0bf6600640 1 -- 172.21.15.66:0/3939865424 shutdown_connections 2024-09-10T22:13:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.311+0000 7f0bf6600640 1 -- 172.21.15.66:0/3939865424 wait complete. 2024-09-10T22:13:07.336 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:13:07.336 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.479+0000 7f135f600640 1 -- 172.21.15.66:0/4048896386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1340004810 msgr2=0x7f1340004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.479+0000 7f135f600640 1 --2- 172.21.15.66:0/4048896386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1340004810 0x7f1340004c90 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f1344006f90 tx=0x7f134401cee0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 -- 172.21.15.66:0/4048896386 shutdown_connections 2024-09-10T22:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 --2- 172.21.15.66:0/4048896386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1340004810 0x7f1340004c90 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 -- 172.21.15.66:0/4048896386 >> 172.21.15.66:0/4048896386 conn(0x7f13580d1c60 msgr2=0x7f13580d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 -- 172.21.15.66:0/4048896386 shutdown_connections 2024-09-10T22:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 -- 172.21.15.66:0/4048896386 wait complete. 2024-09-10T22:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 Processor -- start 2024-09-10T22:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 -- start start 2024-09-10T22:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1340004810 0x7f13581fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f135815c2c0 0x7f13580b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f135815df70 0x7f135815e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f13580cedc0 con 0x7f135815c2c0 2024-09-10T22:13:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f13580cec40 con 0x7f135815df70 2024-09-10T22:13:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135f600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f13580cef40 con 0x7f1340004810 2024-09-10T22:13:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135cc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1340004810 0x7f13581fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f1357e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f135815c2c0 0x7f13580b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135cc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1340004810 0x7f13581fbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:60604/0 (socket says 172.21.15.66:60604) 2024-09-10T22:13:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135cc00640 1 -- 172.21.15.66:0/4222333123 learned_addr learned my addr 172.21.15.66:0/4222333123 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f135d600640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f135815df70 0x7f135815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:07.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f1357e00640 1 -- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1340004810 msgr2=0x7f13581fbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:07.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f1357e00640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1340004810 0x7f13581fbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f1357e00640 1 -- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f135815df70 msgr2=0x7f135815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f1357e00640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f135815df70 0x7f135815e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.483+0000 7f1357e00640 1 -- 172.21.15.66:0/4222333123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135815ebf0 con 0x7f135815c2c0 2024-09-10T22:13:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.487+0000 7f1357e00640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f135815c2c0 0x7f13580b2770 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f13440a8000 tx=0x7f1344006be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.487+0000 7f1354c00640 1 -- 172.21.15.66:0/4222333123 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13440015c0 con 0x7f135815c2c0 2024-09-10T22:13:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.487+0000 7f1354c00640 1 -- 172.21.15.66:0/4222333123 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1344006280 con 0x7f135815c2c0 2024-09-10T22:13:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.487+0000 7f135f600640 1 -- 172.21.15.66:0/4222333123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135815a7e0 con 0x7f135815c2c0 2024-09-10T22:13:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.487+0000 7f1354c00640 1 -- 172.21.15.66:0/4222333123 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344003600 con 0x7f135815c2c0 2024-09-10T22:13:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.487+0000 7f135f600640 1 -- 172.21.15.66:0/4222333123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135815ee60 con 0x7f135815c2c0 2024-09-10T22:13:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.487+0000 7f135f600640 1 -- 172.21.15.66:0/4222333123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13580aef90 con 0x7f135815c2c0 2024-09-10T22:13:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.487+0000 7f1354c00640 1 -- 172.21.15.66:0/4222333123 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f13440c1030 con 0x7f135815c2c0 2024-09-10T22:13:07.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.487+0000 7f1354c00640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f13380743b0 0x7f1338076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:07.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.487+0000 7f1354c00640 1 -- 172.21.15.66:0/4222333123 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f134412b860 con 0x7f135815c2c0 2024-09-10T22:13:07.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.487+0000 7f135cc00640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f13380743b0 0x7f1338076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:07.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.491+0000 7f135cc00640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f13380743b0 0x7f1338076870 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f134c009c10 tx=0x7f134c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:07.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.491+0000 7f1354c00640 1 -- 172.21.15.66:0/4222333123 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f135815a7e0 con 0x7f135815c2c0 2024-09-10T22:13:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.699+0000 7f135f600640 1 -- 172.21.15.66:0/4222333123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13580c6d50 con 0x7f135815c2c0 2024-09-10T22:13:07.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.699+0000 7f1354c00640 1 -- 172.21.15.66:0/4222333123 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5222 (secure 0 0 0) 0x7f1344016c90 con 0x7f135815c2c0 2024-09-10T22:13:07.706 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:07.706 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":86,"btime":"2024-09-10T22:13:07:305043+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1352582369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1352582369},{"type":"v1","addr":"172.21.15.94:6837","nonce":1352582369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8805,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/441263936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":441263936},{"type":"v1","addr":"172.21.15.66:6834","nonce":441263936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2554522797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2554522797},{"type":"v1","addr":"172.21.15.66:6835","nonce":2554522797}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:13:03.687866+0000","modified":"2024-09-10T22:13:06.708613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8811":{"gid":8811,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6836/1762705824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1762705824},{"type":"v1","addr":"172.21.15.94:6836","nonce":1762705824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8811,"qdb_cluster":[8811]},"id":9}]} 2024-09-10T22:13:07.706 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 86 2024-09-10T22:13:07.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.703+0000 7f135f600640 1 -- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f13380743b0 msgr2=0x7f1338076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:07.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.703+0000 7f135f600640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f13380743b0 0x7f1338076870 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f134c009c10 tx=0x7f134c0091e0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.703+0000 7f135f600640 1 -- 172.21.15.66:0/4222333123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f135815c2c0 msgr2=0x7f13580b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:07.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.703+0000 7f135f600640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f135815c2c0 0x7f13580b2770 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f13440a8000 tx=0x7f1344006be0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.703+0000 7f135f600640 1 -- 172.21.15.66:0/4222333123 shutdown_connections 2024-09-10T22:13:07.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.703+0000 7f135f600640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f13380743b0 0x7f1338076870 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.703+0000 7f135f600640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f135815df70 0x7f135815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.703+0000 7f135f600640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f135815c2c0 0x7f13580b2770 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.703+0000 7f135f600640 1 --2- 172.21.15.66:0/4222333123 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1340004810 0x7f13581fbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.703+0000 7f135f600640 1 -- 172.21.15.66:0/4222333123 >> 172.21.15.66:0/4222333123 conn(0x7f13580d1c60 msgr2=0x7f13580d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:07.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.703+0000 7f135f600640 1 -- 172.21.15.66:0/4222333123 shutdown_connections 2024-09-10T22:13:07.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.703+0000 7f135f600640 1 -- 172.21.15.66:0/4222333123 wait complete. 2024-09-10T22:13:07.728 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:13:07.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 -- 172.21.15.66:0/1397287979 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f52d8070b00 msgr2=0x7f52d80ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:07.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 --2- 172.21.15.66:0/1397287979 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f52d8070b00 0x7f52d80ca4d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009870 tx=0x7f52c001c7a0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 -- 172.21.15.66:0/1397287979 shutdown_connections 2024-09-10T22:13:07.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 --2- 172.21.15.66:0/1397287979 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f52d8070b00 0x7f52d80ca4d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 -- 172.21.15.66:0/1397287979 >> 172.21.15.66:0/1397287979 conn(0x7f52d80cc900 msgr2=0x7f52d80ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:07.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 -- 172.21.15.66:0/1397287979 shutdown_connections 2024-09-10T22:13:07.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 -- 172.21.15.66:0/1397287979 wait complete. 2024-09-10T22:13:07.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 Processor -- start 2024-09-10T22:13:07.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 -- start start 2024-09-10T22:13:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f52d8070b00 0x7f52d81590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d80738d0 0x7f52d815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52d80caa10 0x7f52d80c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81690b0 con 0x7f52d80738d0 2024-09-10T22:13:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8168f30 con 0x7f52d80caa10 2024-09-10T22:13:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52dd400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f52d8169230 con 0x7f52d8070b00 2024-09-10T22:13:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52cde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d80738d0 0x7f52d815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52cde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d80738d0 0x7f52d815a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40066/0 (socket says 172.21.15.66:40066) 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52cde00640 1 -- 172.21.15.66:0/2657429387 learned_addr learned my addr 172.21.15.66:0/2657429387 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52d6800640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f52d8070b00 0x7f52d81590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.871+0000 7f52d7200640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52d80caa10 0x7f52d80c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52cde00640 1 -- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f52d8070b00 msgr2=0x7f52d81590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52cde00640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f52d8070b00 0x7f52d81590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52cde00640 1 -- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52d80caa10 msgr2=0x7f52d80c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52cde00640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52d80caa10 0x7f52d80c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52cde00640 1 -- 172.21.15.66:0/2657429387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d8159620 con 0x7f52d80738d0 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52d7200640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52d80caa10 0x7f52d80c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52cde00640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d80738d0 0x7f52d815a500 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f52c0000c10 tx=0x7f52c0005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52cf400640 1 -- 172.21.15.66:0/2657429387 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c00b2040 con 0x7f52d80738d0 2024-09-10T22:13:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52cf400640 1 -- 172.21.15.66:0/2657429387 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52c0004240 con 0x7f52d80738d0 2024-09-10T22:13:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52dd400640 1 -- 172.21.15.66:0/2657429387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d80c76b0 con 0x7f52d80738d0 2024-09-10T22:13:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52cf400640 1 -- 172.21.15.66:0/2657429387 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0005020 con 0x7f52d80738d0 2024-09-10T22:13:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52dd400640 1 -- 172.21.15.66:0/2657429387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d80c7c70 con 0x7f52d80738d0 2024-09-10T22:13:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52dd400640 1 -- 172.21.15.66:0/2657429387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d80ca4d0 con 0x7f52d80738d0 2024-09-10T22:13:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52cf400640 1 -- 172.21.15.66:0/2657429387 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f52c00b5070 con 0x7f52d80738d0 2024-09-10T22:13:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.875+0000 7f52cf400640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f52b40743b0 0x7f52b4076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.879+0000 7f52cf400640 1 -- 172.21.15.66:0/2657429387 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f52c012b8e0 con 0x7f52d80738d0 2024-09-10T22:13:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.879+0000 7f52d6800640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f52b40743b0 0x7f52b4076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:07.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.879+0000 7f52d6800640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f52b40743b0 0x7f52b4076870 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f52c8001820 tx=0x7f52c8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:07.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:07.879+0000 7f52cf400640 1 -- 172.21.15.66:0/2657429387 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f52c0016450 con 0x7f52d80738d0 2024-09-10T22:13:08.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.059+0000 7f52dd400640 1 -- 172.21.15.66:0/2657429387 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52d80704b0 con 0x7f52d80738d0 2024-09-10T22:13:08.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.059+0000 7f52cf400640 1 -- 172.21.15.66:0/2657429387 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16845 (secure 0 0 0) 0x7f52c00f89a0 con 0x7f52d80738d0 2024-09-10T22:13:08.065 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:08.065 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":126,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:13:05.290906+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:13:01.405668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T22:13:02.664579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:13:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.063+0000 7f52dd400640 1 -- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f52b40743b0 msgr2=0x7f52b4076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.063+0000 7f52dd400640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f52b40743b0 0x7f52b4076870 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f52c8001820 tx=0x7f52c8009040 comp rx=0 tx=0).stop 2024-09-10T22:13:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.063+0000 7f52dd400640 1 -- 172.21.15.66:0/2657429387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d80738d0 msgr2=0x7f52d815a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:08.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.063+0000 7f52dd400640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d80738d0 0x7f52d815a500 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f52c0000c10 tx=0x7f52c0005d40 comp rx=0 tx=0).stop 2024-09-10T22:13:08.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.063+0000 7f52dd400640 1 -- 172.21.15.66:0/2657429387 shutdown_connections 2024-09-10T22:13:08.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.063+0000 7f52dd400640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f52b40743b0 0x7f52b4076870 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.063+0000 7f52dd400640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52d80caa10 0x7f52d80c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.063+0000 7f52dd400640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d80738d0 0x7f52d815a500 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.063+0000 7f52dd400640 1 --2- 172.21.15.66:0/2657429387 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f52d8070b00 0x7f52d81590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.063+0000 7f52dd400640 1 -- 172.21.15.66:0/2657429387 >> 172.21.15.66:0/2657429387 conn(0x7f52d80cc900 msgr2=0x7f52d8073cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:08.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.063+0000 7f52dd400640 1 -- 172.21.15.66:0/2657429387 shutdown_connections 2024-09-10T22:13:08.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.063+0000 7f52dd400640 1 -- 172.21.15.66:0/2657429387 wait complete. 2024-09-10T22:13:08.088 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:13:08.089 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:13:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.231+0000 7fc4e8200640 1 -- 172.21.15.66:0/166399381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e0070ac0 msgr2=0x7fc4e0070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.231+0000 7fc4e8200640 1 --2- 172.21.15.66:0/166399381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e0070ac0 0x7fc4e0070ea0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00c8d60 tx=0x7fc4d401c7a0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.231+0000 7fc4e8200640 1 -- 172.21.15.66:0/166399381 shutdown_connections 2024-09-10T22:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.231+0000 7fc4e8200640 1 --2- 172.21.15.66:0/166399381 >> v2:172.21.15.94:3300/0 conn(0x7fc4e00b0ef0 0x7fc4e00d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.231+0000 7fc4e8200640 1 --2- 172.21.15.66:0/166399381 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4e0071700 0x7fc4e00b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.231+0000 7fc4e8200640 1 --2- 172.21.15.66:0/166399381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e0070ac0 0x7fc4e0070ea0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.231+0000 7fc4e8200640 1 -- 172.21.15.66:0/166399381 >> 172.21.15.66:0/166399381 conn(0x7fc4e00d1d00 msgr2=0x7fc4e00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.231+0000 7fc4e8200640 1 -- 172.21.15.66:0/166399381 shutdown_connections 2024-09-10T22:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.231+0000 7fc4e8200640 1 -- 172.21.15.66:0/166399381 wait complete. 2024-09-10T22:13:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.231+0000 7fc4e8200640 1 Processor -- start 2024-09-10T22:13:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.231+0000 7fc4e8200640 1 -- start start 2024-09-10T22:13:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e8200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4e0070ac0 0x7fc4e015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e8200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4e0071700 0x7fc4e0157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e8200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e00b0ef0 0x7fc4e00b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e8200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e00af590 con 0x7fc4e00b0ef0 2024-09-10T22:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e8200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e00af410 con 0x7fc4e0071700 2024-09-10T22:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e8200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e00af710 con 0x7fc4e0070ac0 2024-09-10T22:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e00b0ef0 0x7fc4e00b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e00b0ef0 0x7fc4e00b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40088/0 (socket says 172.21.15.66:40088) 2024-09-10T22:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e6200640 1 -- 172.21.15.66:0/2872271995 learned_addr learned my addr 172.21.15.66:0/2872271995 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e5800640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4e0070ac0 0x7fc4e015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e4e00640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4e0071700 0x7fc4e0157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e6200640 1 -- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4e0070ac0 msgr2=0x7fc4e015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e6200640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4e0070ac0 0x7fc4e015ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e6200640 1 -- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4e0071700 msgr2=0x7fc4e0157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e6200640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4e0071700 0x7fc4e0157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e6200640 1 -- 172.21.15.66:0/2872271995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e01fba60 con 0x7fc4e00b0ef0 2024-09-10T22:13:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e6200640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e00b0ef0 0x7fc4e00b2d90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800cce0 tx=0x7fc4d800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4dd600640 1 -- 172.21.15.66:0/2872271995 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800f450 con 0x7fc4e00b0ef0 2024-09-10T22:13:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4dd600640 1 -- 172.21.15.66:0/2872271995 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4d800f6e0 con 0x7fc4e00b0ef0 2024-09-10T22:13:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4dd600640 1 -- 172.21.15.66:0/2872271995 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800fb30 con 0x7fc4e00b0ef0 2024-09-10T22:13:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e8200640 1 -- 172.21.15.66:0/2872271995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e015a060 con 0x7fc4e00b0ef0 2024-09-10T22:13:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e8200640 1 -- 172.21.15.66:0/2872271995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e01fb6a0 con 0x7fc4e00b0ef0 2024-09-10T22:13:08.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.235+0000 7fc4e8200640 1 -- 172.21.15.66:0/2872271995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e00b0410 con 0x7fc4e00b0ef0 2024-09-10T22:13:08.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.239+0000 7fc4dd600640 1 -- 172.21.15.66:0/2872271995 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7fc4d800fcd0 con 0x7fc4e00b0ef0 2024-09-10T22:13:08.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.239+0000 7fc4dd600640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc4c00743e0 0x7fc4c00768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:08.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.239+0000 7fc4dd600640 1 -- 172.21.15.66:0/2872271995 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7fc4d8092c30 con 0x7fc4e00b0ef0 2024-09-10T22:13:08.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.239+0000 7fc4e5800640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc4c00743e0 0x7fc4c00768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:08.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.239+0000 7fc4e5800640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc4c00743e0 0x7fc4c00768a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40ae000 tx=0x7fc4d401afe0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:08.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.243+0000 7fc4dd600640 1 -- 172.21.15.66:0/2872271995 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc4d8014030 con 0x7fc4e00b0ef0 2024-09-10T22:13:08.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.415+0000 7fc4e8200640 1 -- 172.21.15.66:0/2872271995 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4e00b4700 con 0x7fc4c00743e0 2024-09-10T22:13:08.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.419+0000 7fc4dd600640 1 -- 172.21.15.66:0/2872271995 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241171 (secure 0 0 0) 0x7fc4e00b4700 con 0x7fc4c00743e0 2024-09-10T22:13:08.428 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:08.429 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:13:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.423+0000 7fc4e8200640 1 -- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc4c00743e0 msgr2=0x7fc4c00768a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.423+0000 7fc4e8200640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc4c00743e0 0x7fc4c00768a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40ae000 tx=0x7fc4d401afe0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.423+0000 7fc4e8200640 1 -- 172.21.15.66:0/2872271995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e00b0ef0 msgr2=0x7fc4e00b2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.423+0000 7fc4e8200640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e00b0ef0 0x7fc4e00b2d90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800cce0 tx=0x7fc4d800a620 comp rx=0 tx=0).stop 2024-09-10T22:13:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.423+0000 7fc4e8200640 1 -- 172.21.15.66:0/2872271995 shutdown_connections 2024-09-10T22:13:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.423+0000 7fc4e8200640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc4c00743e0 0x7fc4c00768a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.423+0000 7fc4e8200640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e00b0ef0 0x7fc4e00b2d90 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.423+0000 7fc4e8200640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4e0071700 0x7fc4e0157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.423+0000 7fc4e8200640 1 --2- 172.21.15.66:0/2872271995 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4e0070ac0 0x7fc4e015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.423+0000 7fc4e8200640 1 -- 172.21.15.66:0/2872271995 >> 172.21.15.66:0/2872271995 conn(0x7fc4e00d1d00 msgr2=0x7fc4e01648b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:08.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.423+0000 7fc4e8200640 1 -- 172.21.15.66:0/2872271995 shutdown_connections 2024-09-10T22:13:08.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.423+0000 7fc4e8200640 1 -- 172.21.15.66:0/2872271995 wait complete. 2024-09-10T22:13:08.450 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-10T22:13:06.582639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":68,"acting":68,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":1065876,"kb_used_data":13056,"kb_used_omap":12,"kb_used_meta":1052595,"kb_avail":753908844,"statfs":{"total":773094113280,"available":772002656256,"internally_reserved":0,"allocated":13369344,"data_stored":6485983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1077857844},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.307342"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353797+0000","last_change":"2024-09-10T22:13:04.307228+0000","last_active":"2024-09-10T22:13:05.353797+0000","last_peered":"2024-09-10T22:13:05.353797+0000","last_clean":"2024-09-10T22:13:05.353797+0000","last_became_active":"2024-09-10T22:13:04.306866+0000","last_became_peered":"2024-09-10T22:13:04.306866+0000","last_unstale":"2024-09-10T22:13:05.353797+0000","last_undegraded":"2024-09-10T22:13:05.353797+0000","last_fullsized":"2024-09-10T22:13:05.353797+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:00.169255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354094+0000","last_change":"2024-09-10T22:13:04.297191+0000","last_active":"2024-09-10T22:13:05.354094+0000","last_peered":"2024-09-10T22:13:05.354094+0000","last_clean":"2024-09-10T22:13:05.354094+0000","last_became_active":"2024-09-10T22:13:04.296922+0000","last_became_peered":"2024-09-10T22:13:04.296922+0000","last_unstale":"2024-09-10T22:13:05.354094+0000","last_undegraded":"2024-09-10T22:13:05.354094+0000","last_fullsized":"2024-09-10T22:13:05.354094+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:12.004698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353458+0000","last_change":"2024-09-10T22:13:04.305839+0000","last_active":"2024-09-10T22:13:05.353458+0000","last_peered":"2024-09-10T22:13:05.353458+0000","last_clean":"2024-09-10T22:13:05.353458+0000","last_became_active":"2024-09-10T22:13:04.305564+0000","last_became_peered":"2024-09-10T22:13:04.305564+0000","last_unstale":"2024-09-10T22:13:05.353458+0000","last_undegraded":"2024-09-10T22:13:05.353458+0000","last_fullsized":"2024-09-10T22:13:05.353458+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:11.940828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.352894+0000","last_change":"2024-09-10T22:13:04.307792+0000","last_active":"2024-09-10T22:13:05.352894+0000","last_peered":"2024-09-10T22:13:05.352894+0000","last_clean":"2024-09-10T22:13:05.352894+0000","last_became_active":"2024-09-10T22:13:04.307499+0000","last_became_peered":"2024-09-10T22:13:04.307499+0000","last_unstale":"2024-09-10T22:13:05.352894+0000","last_undegraded":"2024-09-10T22:13:05.352894+0000","last_fullsized":"2024-09-10T22:13:05.352894+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:36.777792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353548+0000","last_change":"2024-09-10T22:13:04.308108+0000","last_active":"2024-09-10T22:13:05.353548+0000","last_peered":"2024-09-10T22:13:05.353548+0000","last_clean":"2024-09-10T22:13:05.353548+0000","last_became_active":"2024-09-10T22:13:04.307823+0000","last_became_peered":"2024-09-10T22:13:04.307823+0000","last_unstale":"2024-09-10T22:13:05.353548+0000","last_undegraded":"2024-09-10T22:13:05.353548+0000","last_fullsized":"2024-09-10T22:13:05.353548+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:03.333787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353494+0000","last_change":"2024-09-10T22:13:04.309277+0000","last_active":"2024-09-10T22:13:05.353494+0000","last_peered":"2024-09-10T22:13:05.353494+0000","last_clean":"2024-09-10T22:13:05.353494+0000","last_became_active":"2024-09-10T22:13:04.307644+0000","last_became_peered":"2024-09-10T22:13:04.307644+0000","last_unstale":"2024-09-10T22:13:05.353494+0000","last_undegraded":"2024-09-10T22:13:05.353494+0000","last_fullsized":"2024-09-10T22:13:05.353494+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:30.854268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353856+0000","last_change":"2024-09-10T22:13:04.307932+0000","last_active":"2024-09-10T22:13:05.353856+0000","last_peered":"2024-09-10T22:13:05.353856+0000","last_clean":"2024-09-10T22:13:05.353856+0000","last_became_active":"2024-09-10T22:13:04.307658+0000","last_became_peered":"2024-09-10T22:13:04.307658+0000","last_unstale":"2024-09-10T22:13:05.353856+0000","last_undegraded":"2024-09-10T22:13:05.353856+0000","last_fullsized":"2024-09-10T22:13:05.353856+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:07.546337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.280536+0000","last_change":"2024-09-10T22:13:02.280536+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.280536+0000","last_undegraded":"2024-09-10T22:13:02.280536+0000","last_fullsized":"2024-09-10T22:13:02.280536+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.283241+0000","last_change":"2024-09-10T22:13:02.283241+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.283241+0000","last_undegraded":"2024-09-10T22:13:02.283241+0000","last_fullsized":"2024-09-10T22:13:02.283241+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:02.267589+0000","last_change":"2024-09-10T22:13:02.267589+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.267589+0000","last_undegraded":"2024-09-10T22:13:02.267589+0000","last_fullsized":"2024-09-10T22:13:02.267589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.281091+0000","last_change":"2024-09-10T22:13:02.281091+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.281091+0000","last_undegraded":"2024-09-10T22:13:02.281091+0000","last_fullsized":"2024-09-10T22:13:02.281091+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.281979+0000","last_change":"2024-09-10T22:13:02.281979+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.281979+0000","last_undegraded":"2024-09-10T22:13:02.281979+0000","last_fullsized":"2024-09-10T22:13:02.281979+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:02.267589+0000","last_change":"2024-09-10T22:13:02.267589+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.267589+0000","last_undegraded":"2024-09-10T22:13:02.267589+0000","last_fullsized":"2024-09-10T22:13:02.267589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.277379+0000","last_change":"2024-09-10T22:13:02.277379+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.277379+0000","last_undegraded":"2024-09-10T22:13:02.277379+0000","last_fullsized":"2024-09-10T22:13:02.277379+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.279363+0000","last_change":"2024-09-10T22:13:02.279363+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.279363+0000","last_undegraded":"2024-09-10T22:13:02.279363+0000","last_fullsized":"2024-09-10T22:13:02.279363+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353704+0000","last_change":"2024-09-10T22:13:04.302884+0000","last_active":"2024-09-10T22:13:05.353704+0000","last_peered":"2024-09-10T22:13:05.353704+0000","last_clean":"2024-09-10T22:13:05.353704+0000","last_became_active":"2024-09-10T22:13:04.300050+0000","last_became_peered":"2024-09-10T22:13:04.300050+0000","last_unstale":"2024-09-10T22:13:05.353704+0000","last_undegraded":"2024-09-10T22:13:05.353704+0000","last_fullsized":"2024-09-10T22:13:05.353704+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:35.721001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.283928+0000","last_change":"2024-09-10T22:13:02.283928+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.283928+0000","last_undegraded":"2024-09-10T22:13:02.283928+0000","last_fullsized":"2024-09-10T22:13:02.283928+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:02.267589+0000","last_change":"2024-09-10T22:13:02.267589+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.267589+0000","last_undegraded":"2024-09-10T22:13:02.267589+0000","last_fullsized":"2024-09-10T22:13:02.267589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.283094+0000","last_change":"2024-09-10T22:13:02.283094+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.283094+0000","last_undegraded":"2024-09-10T22:13:02.283094+0000","last_fullsized":"2024-09-10T22:13:02.283094+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.277697+0000","last_change":"2024-09-10T22:13:02.277697+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.277697+0000","last_undegraded":"2024-09-10T22:13:02.277697+0000","last_fullsized":"2024-09-10T22:13:02.277697+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354042+0000","last_change":"2024-09-10T22:13:04.300726+0000","last_active":"2024-09-10T22:13:05.354042+0000","last_peered":"2024-09-10T22:13:05.354042+0000","last_clean":"2024-09-10T22:13:05.354042+0000","last_became_active":"2024-09-10T22:13:04.300465+0000","last_became_peered":"2024-09-10T22:13:04.300465+0000","last_unstale":"2024-09-10T22:13:05.354042+0000","last_undegraded":"2024-09-10T22:13:05.354042+0000","last_fullsized":"2024-09-10T22:13:05.354042+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:28.540714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353881+0000","last_change":"2024-09-10T22:13:03.295996+0000","last_active":"2024-09-10T22:13:05.353881+0000","last_peered":"2024-09-10T22:13:05.353881+0000","last_clean":"2024-09-10T22:13:05.353881+0000","last_became_active":"2024-09-10T22:13:03.295495+0000","last_became_peered":"2024-09-10T22:13:03.295495+0000","last_unstale":"2024-09-10T22:13:05.353881+0000","last_undegraded":"2024-09-10T22:13:05.353881+0000","last_fullsized":"2024-09-10T22:13:05.353881+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:34:54.520382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.281428+0000","last_change":"2024-09-10T22:13:02.281428+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.281428+0000","last_undegraded":"2024-09-10T22:13:02.281428+0000","last_fullsized":"2024-09-10T22:13:02.281428+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.277849+0000","last_change":"2024-09-10T22:13:02.277849+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.277849+0000","last_undegraded":"2024-09-10T22:13:02.277849+0000","last_fullsized":"2024-09-10T22:13:02.277849+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.283433+0000","last_change":"2024-09-10T22:13:02.283433+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.283433+0000","last_undegraded":"2024-09-10T22:13:02.283433+0000","last_fullsized":"2024-09-10T22:13:02.283433+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:02.267589+0000","last_change":"2024-09-10T22:13:02.267589+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.267589+0000","last_undegraded":"2024-09-10T22:13:02.267589+0000","last_fullsized":"2024-09-10T22:13:02.267589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.277367+0000","last_change":"2024-09-10T22:13:02.277367+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.277367+0000","last_undegraded":"2024-09-10T22:13:02.277367+0000","last_fullsized":"2024-09-10T22:13:02.277367+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.283144+0000","last_change":"2024-09-10T22:13:02.283144+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.283144+0000","last_undegraded":"2024-09-10T22:13:02.283144+0000","last_fullsized":"2024-09-10T22:13:02.283144+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353779+0000","last_change":"2024-09-10T22:13:04.307084+0000","last_active":"2024-09-10T22:13:05.353779+0000","last_peered":"2024-09-10T22:13:05.353779+0000","last_clean":"2024-09-10T22:13:05.353779+0000","last_became_active":"2024-09-10T22:13:04.306577+0000","last_became_peered":"2024-09-10T22:13:04.306577+0000","last_unstale":"2024-09-10T22:13:05.353779+0000","last_undegraded":"2024-09-10T22:13:05.353779+0000","last_fullsized":"2024-09-10T22:13:05.353779+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:07.936676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.283083+0000","last_change":"2024-09-10T22:13:02.283083+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.283083+0000","last_undegraded":"2024-09-10T22:13:02.283083+0000","last_fullsized":"2024-09-10T22:13:02.283083+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:02.267589+0000","last_change":"2024-09-10T22:13:02.267589+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.267589+0000","last_undegraded":"2024-09-10T22:13:02.267589+0000","last_fullsized":"2024-09-10T22:13:02.267589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:02.267589+0000","last_change":"2024-09-10T22:13:02.267589+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.267589+0000","last_undegraded":"2024-09-10T22:13:02.267589+0000","last_fullsized":"2024-09-10T22:13:02.267589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:02.267589+0000","last_change":"2024-09-10T22:13:02.267589+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.267589+0000","last_undegraded":"2024-09-10T22:13:02.267589+0000","last_fullsized":"2024-09-10T22:13:02.267589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"125'2","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353932+0000","last_change":"2024-09-10T22:13:03.302829+0000","last_active":"2024-09-10T22:13:05.353932+0000","last_peered":"2024-09-10T22:13:05.353932+0000","last_clean":"2024-09-10T22:13:05.353932+0000","last_became_active":"2024-09-10T22:13:03.302579+0000","last_became_peered":"2024-09-10T22:13:03.302579+0000","last_unstale":"2024-09-10T22:13:05.353932+0000","last_undegraded":"2024-09-10T22:13:05.353932+0000","last_fullsized":"2024-09-10T22:13:05.353932+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:35.384612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353716+0000","last_change":"2024-09-10T22:13:04.316440+0000","last_active":"2024-09-10T22:13:05.353716+0000","last_peered":"2024-09-10T22:13:05.353716+0000","last_clean":"2024-09-10T22:13:05.353716+0000","last_became_active":"2024-09-10T22:13:04.316162+0000","last_became_peered":"2024-09-10T22:13:04.316162+0000","last_unstale":"2024-09-10T22:13:05.353716+0000","last_undegraded":"2024-09-10T22:13:05.353716+0000","last_fullsized":"2024-09-10T22:13:05.353716+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:32.587807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.279798+0000","last_change":"2024-09-10T22:13:02.279798+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.279798+0000","last_undegraded":"2024-09-10T22:13:02.279798+0000","last_fullsized":"2024-09-10T22:13:02.279798+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.280339+0000","last_change":"2024-09-10T22:13:02.280339+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.280339+0000","last_undegraded":"2024-09-10T22:13:02.280339+0000","last_fullsized":"2024-09-10T22:13:02.280339+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:02.267589+0000","last_change":"2024-09-10T22:13:02.267589+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.267589+0000","last_undegraded":"2024-09-10T22:13:02.267589+0000","last_fullsized":"2024-09-10T22:13:02.267589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.281459+0000","last_change":"2024-09-10T22:13:02.281459+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.281459+0000","last_undegraded":"2024-09-10T22:13:02.281459+0000","last_fullsized":"2024-09-10T22:13:02.281459+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.279744+0000","last_change":"2024-09-10T22:13:02.279744+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.279744+0000","last_undegraded":"2024-09-10T22:13:02.279744+0000","last_fullsized":"2024-09-10T22:13:02.279744+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:02.267589+0000","last_change":"2024-09-10T22:13:02.267589+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.267589+0000","last_undegraded":"2024-09-10T22:13:02.267589+0000","last_fullsized":"2024-09-10T22:13:02.267589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:02.267589+0000","last_change":"2024-09-10T22:13:02.267589+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.267589+0000","last_undegraded":"2024-09-10T22:13:02.267589+0000","last_fullsized":"2024-09-10T22:13:02.267589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:13:03.270077+0000","last_change":"2024-09-10T22:13:03.270077+0000","last_active":"2024-09-10T22:13:03.270077+0000","last_peered":"2024-09-10T22:13:03.270077+0000","last_clean":"2024-09-10T22:13:03.270077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:03.270077+0000","last_undegraded":"2024-09-10T22:13:03.270077+0000","last_fullsized":"2024-09-10T22:13:03.270077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-09-10T22:13:02.279770+0000","last_change":"2024-09-10T22:13:02.279770+0000","last_active":"2024-09-10T22:13:02.267589+0000","last_peered":"2024-09-10T22:13:02.267589+0000","last_clean":"2024-09-10T22:13:02.267589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:13:02.279770+0000","last_undegraded":"2024-09-10T22:13:02.279770+0000","last_fullsized":"2024-09-10T22:13:02.279770+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":305,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T22:13:02.281010+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:13:02.281010+0000","last_peered":"2024-09-10T22:13:02.281010+0000","last_clean":"2024-09-10T22:13:02.281010+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:13:02.281010+0000","last_undegraded":"2024-09-10T22:13:02.281010+0000","last_fullsized":"2024-09-10T22:13:02.281010+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":1},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":44,"acting":44,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131752,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240088,"statfs":{"total":96636764160,"available":96501850112,"internally_reserved":0,"allocated":2109440,"data_stored":1250245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122852,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248988,"statfs":{"total":96636764160,"available":96510963712,"internally_reserved":0,"allocated":1515520,"data_stored":659877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":124254662},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509595,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108968,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262872,"statfs":{"total":96636764160,"available":96525180928,"internally_reserved":0,"allocated":1515520,"data_stored":659877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153376,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218464,"statfs":{"total":96636764160,"available":96479707136,"internally_reserved":0,"allocated":1515520,"data_stored":659877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509594,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149216,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222624,"statfs":{"total":96636764160,"available":96483966976,"internally_reserved":0,"allocated":1515520,"data_stored":659877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509595,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118692,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253148,"statfs":{"total":96636764160,"available":96515223552,"internally_reserved":0,"allocated":1515520,"data_stored":659877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509595,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131744,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240096,"statfs":{"total":96636764160,"available":96501858304,"internally_reserved":0,"allocated":2109440,"data_stored":1250245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":132774345},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542301,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149276,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222564,"statfs":{"total":96636764160,"available":96483905536,"internally_reserved":0,"allocated":1572864,"data_stored":686108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:13:08.455 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:13:08.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.635+0000 7fda66800640 1 -- 172.21.15.66:0/1962531849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda6015bd40 msgr2=0x7fda600cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:08.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.635+0000 7fda66800640 1 --2- 172.21.15.66:0/1962531849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda6015bd40 0x7fda600cffa0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fda50009840 tx=0x7fda5001c790 comp rx=0 tx=0).stop 2024-09-10T22:13:08.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.635+0000 7fda66800640 1 -- 172.21.15.66:0/1962531849 shutdown_connections 2024-09-10T22:13:08.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.635+0000 7fda66800640 1 --2- 172.21.15.66:0/1962531849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda6015bd40 0x7fda600cffa0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.635+0000 7fda66800640 1 -- 172.21.15.66:0/1962531849 >> 172.21.15.66:0/1962531849 conn(0x7fda600d1c60 msgr2=0x7fda600d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:08.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.635+0000 7fda66800640 1 -- 172.21.15.66:0/1962531849 shutdown_connections 2024-09-10T22:13:08.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.635+0000 7fda66800640 1 -- 172.21.15.66:0/1962531849 wait complete. 2024-09-10T22:13:08.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda66800640 1 Processor -- start 2024-09-10T22:13:08.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda66800640 1 -- start start 2024-09-10T22:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda66800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda600cf180 0x7fda6014b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda66800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda600cf830 0x7fda6015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda66800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda6015bd40 0x7fda6015acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda66800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fda6016a4d0 con 0x7fda600cf180 2024-09-10T22:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda66800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda6016a350 con 0x7fda600cf830 2024-09-10T22:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda66800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fda6016a650 con 0x7fda6015bd40 2024-09-10T22:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda5f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda600cf180 0x7fda6014b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda5f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda600cf180 0x7fda6014b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40106/0 (socket says 172.21.15.66:40106) 2024-09-10T22:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda5f400640 1 -- 172.21.15.66:0/3324363455 learned_addr learned my addr 172.21.15.66:0/3324363455 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda5f400640 1 -- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda6015bd40 msgr2=0x7fda6015acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda56a00640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda600cf830 0x7fda6015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda5fe00640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda6015bd40 0x7fda6015acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda5f400640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda6015bd40 0x7fda6015acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda5f400640 1 -- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda600cf830 msgr2=0x7fda6015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda5f400640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda600cf830 0x7fda6015e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda5f400640 1 -- 172.21.15.66:0/3324363455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6015ea80 con 0x7fda600cf180 2024-09-10T22:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda56a00640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda600cf830 0x7fda6015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda5fe00640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda6015bd40 0x7fda6015acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda5f400640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda600cf180 0x7fda6014b160 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fda5001cc70 tx=0x7fda500a5560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda57e00640 1 -- 172.21.15.66:0/3324363455 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda500b2040 con 0x7fda600cf180 2024-09-10T22:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda57e00640 1 -- 172.21.15.66:0/3324363455 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda50004270 con 0x7fda600cf180 2024-09-10T22:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda57e00640 1 -- 172.21.15.66:0/3324363455 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50003600 con 0x7fda600cf180 2024-09-10T22:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda66800640 1 -- 172.21.15.66:0/3324363455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda60163cd0 con 0x7fda600cf180 2024-09-10T22:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda66800640 1 -- 172.21.15.66:0/3324363455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda60164290 con 0x7fda600cf180 2024-09-10T22:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.639+0000 7fda66800640 1 -- 172.21.15.66:0/3324363455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda600cffa0 con 0x7fda600cf180 2024-09-10T22:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.643+0000 7fda57e00640 1 -- 172.21.15.66:0/3324363455 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7fda500b5070 con 0x7fda600cf180 2024-09-10T22:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.643+0000 7fda57e00640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda3c0743e0 0x7fda3c0768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.643+0000 7fda57e00640 1 -- 172.21.15.66:0/3324363455 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7fda5012bcf0 con 0x7fda600cf180 2024-09-10T22:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.643+0000 7fda56a00640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda3c0743e0 0x7fda3c0768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.643+0000 7fda56a00640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda3c0743e0 0x7fda3c0768a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fda6015f120 tx=0x7fda48009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.647+0000 7fda57e00640 1 -- 172.21.15.66:0/3324363455 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fda500c4060 con 0x7fda600cf180 2024-09-10T22:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.815+0000 7fda66800640 1 -- 172.21.15.66:0/3324363455 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda600b0df0 con 0x7fda3c0743e0 2024-09-10T22:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.819+0000 7fda57e00640 1 -- 172.21.15.66:0/3324363455 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252498 (secure 0 0 0) 0x7fda600b0df0 con 0x7fda3c0743e0 2024-09-10T22:13:08.828 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:08.830 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:13:08.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.823+0000 7fda66800640 1 -- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda3c0743e0 msgr2=0x7fda3c0768a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:08.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.823+0000 7fda66800640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda3c0743e0 0x7fda3c0768a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fda6015f120 tx=0x7fda48009040 comp rx=0 tx=0).stop 2024-09-10T22:13:08.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.823+0000 7fda66800640 1 -- 172.21.15.66:0/3324363455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda600cf180 msgr2=0x7fda6014b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:08.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.823+0000 7fda66800640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda600cf180 0x7fda6014b160 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fda5001cc70 tx=0x7fda500a5560 comp rx=0 tx=0).stop 2024-09-10T22:13:08.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.823+0000 7fda66800640 1 -- 172.21.15.66:0/3324363455 shutdown_connections 2024-09-10T22:13:08.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.823+0000 7fda66800640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda3c0743e0 0x7fda3c0768a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.823+0000 7fda66800640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda6015bd40 0x7fda6015acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.823+0000 7fda66800640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda600cf830 0x7fda6015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.823+0000 7fda66800640 1 --2- 172.21.15.66:0/3324363455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda600cf180 0x7fda6014b160 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:08.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.823+0000 7fda66800640 1 -- 172.21.15.66:0/3324363455 >> 172.21.15.66:0/3324363455 conn(0x7fda600d1c60 msgr2=0x7fda600b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:08.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.823+0000 7fda66800640 1 -- 172.21.15.66:0/3324363455 shutdown_connections 2024-09-10T22:13:08.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:08.823+0000 7fda66800640 1 -- 172.21.15.66:0/3324363455 wait complete. 2024-09-10T22:13:08.849 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-09-10T22:13:08.583145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1066252,"kb_used_data":13448,"kb_used_omap":12,"kb_used_meta":1052595,"kb_avail":753908468,"statfs":{"total":773094113280,"available":772002271232,"internally_reserved":0,"allocated":13770752,"data_stored":6669492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1077857844},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001547"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353797+0000","last_change":"2024-09-10T22:13:04.307228+0000","last_active":"2024-09-10T22:13:05.353797+0000","last_peered":"2024-09-10T22:13:05.353797+0000","last_clean":"2024-09-10T22:13:05.353797+0000","last_became_active":"2024-09-10T22:13:04.306866+0000","last_became_peered":"2024-09-10T22:13:04.306866+0000","last_unstale":"2024-09-10T22:13:05.353797+0000","last_undegraded":"2024-09-10T22:13:05.353797+0000","last_fullsized":"2024-09-10T22:13:05.353797+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:00.169255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354389+0000","last_change":"2024-09-10T22:13:04.298682+0000","last_active":"2024-09-10T22:13:05.354389+0000","last_peered":"2024-09-10T22:13:05.354389+0000","last_clean":"2024-09-10T22:13:05.354389+0000","last_became_active":"2024-09-10T22:13:04.298181+0000","last_became_peered":"2024-09-10T22:13:04.298181+0000","last_unstale":"2024-09-10T22:13:05.354389+0000","last_undegraded":"2024-09-10T22:13:05.354389+0000","last_fullsized":"2024-09-10T22:13:05.354389+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.870997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.359063+0000","last_change":"2024-09-10T22:13:04.312462+0000","last_active":"2024-09-10T22:13:05.359063+0000","last_peered":"2024-09-10T22:13:05.359063+0000","last_clean":"2024-09-10T22:13:05.359063+0000","last_became_active":"2024-09-10T22:13:04.311671+0000","last_became_peered":"2024-09-10T22:13:04.311671+0000","last_unstale":"2024-09-10T22:13:05.359063+0000","last_undegraded":"2024-09-10T22:13:05.359063+0000","last_fullsized":"2024-09-10T22:13:05.359063+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:16.773921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354094+0000","last_change":"2024-09-10T22:13:04.297191+0000","last_active":"2024-09-10T22:13:05.354094+0000","last_peered":"2024-09-10T22:13:05.354094+0000","last_clean":"2024-09-10T22:13:05.354094+0000","last_became_active":"2024-09-10T22:13:04.296922+0000","last_became_peered":"2024-09-10T22:13:04.296922+0000","last_unstale":"2024-09-10T22:13:05.354094+0000","last_undegraded":"2024-09-10T22:13:05.354094+0000","last_fullsized":"2024-09-10T22:13:05.354094+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:12.004698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358104+0000","last_change":"2024-09-10T22:13:04.309867+0000","last_active":"2024-09-10T22:13:05.358104+0000","last_peered":"2024-09-10T22:13:05.358104+0000","last_clean":"2024-09-10T22:13:05.358104+0000","last_became_active":"2024-09-10T22:13:04.309332+0000","last_became_peered":"2024-09-10T22:13:04.309332+0000","last_unstale":"2024-09-10T22:13:05.358104+0000","last_undegraded":"2024-09-10T22:13:05.358104+0000","last_fullsized":"2024-09-10T22:13:05.358104+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:22.268406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.306220+0000","last_change":"2024-09-10T22:13:04.310375+0000","last_active":"2024-09-10T22:13:05.306220+0000","last_peered":"2024-09-10T22:13:05.306220+0000","last_clean":"2024-09-10T22:13:05.306220+0000","last_became_active":"2024-09-10T22:13:04.309901+0000","last_became_peered":"2024-09-10T22:13:04.309901+0000","last_unstale":"2024-09-10T22:13:05.306220+0000","last_undegraded":"2024-09-10T22:13:05.306220+0000","last_fullsized":"2024-09-10T22:13:05.306220+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:26.331574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.304754+0000","last_change":"2024-09-10T22:13:04.312047+0000","last_active":"2024-09-10T22:13:05.304754+0000","last_peered":"2024-09-10T22:13:05.304754+0000","last_clean":"2024-09-10T22:13:05.304754+0000","last_became_active":"2024-09-10T22:13:04.311778+0000","last_became_peered":"2024-09-10T22:13:04.311778+0000","last_unstale":"2024-09-10T22:13:05.304754+0000","last_undegraded":"2024-09-10T22:13:05.304754+0000","last_fullsized":"2024-09-10T22:13:05.304754+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:11.548966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358377+0000","last_change":"2024-09-10T22:13:04.320234+0000","last_active":"2024-09-10T22:13:05.358377+0000","last_peered":"2024-09-10T22:13:05.358377+0000","last_clean":"2024-09-10T22:13:05.358377+0000","last_became_active":"2024-09-10T22:13:04.319918+0000","last_became_peered":"2024-09-10T22:13:04.319918+0000","last_unstale":"2024-09-10T22:13:05.358377+0000","last_undegraded":"2024-09-10T22:13:05.358377+0000","last_fullsized":"2024-09-10T22:13:05.358377+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:11:45.352717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.359048+0000","last_change":"2024-09-10T22:13:04.320889+0000","last_active":"2024-09-10T22:13:05.359048+0000","last_peered":"2024-09-10T22:13:05.359048+0000","last_clean":"2024-09-10T22:13:05.359048+0000","last_became_active":"2024-09-10T22:13:04.320621+0000","last_became_peered":"2024-09-10T22:13:04.320621+0000","last_unstale":"2024-09-10T22:13:05.359048+0000","last_undegraded":"2024-09-10T22:13:05.359048+0000","last_fullsized":"2024-09-10T22:13:05.359048+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:21.800686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354460+0000","last_change":"2024-09-10T22:13:04.298424+0000","last_active":"2024-09-10T22:13:05.354460+0000","last_peered":"2024-09-10T22:13:05.354460+0000","last_clean":"2024-09-10T22:13:05.354460+0000","last_became_active":"2024-09-10T22:13:04.297789+0000","last_became_peered":"2024-09-10T22:13:04.297789+0000","last_unstale":"2024-09-10T22:13:05.354460+0000","last_undegraded":"2024-09-10T22:13:05.354460+0000","last_fullsized":"2024-09-10T22:13:05.354460+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:20.664133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.299163+0000","last_change":"2024-09-10T22:13:04.294669+0000","last_active":"2024-09-10T22:13:05.299163+0000","last_peered":"2024-09-10T22:13:05.299163+0000","last_clean":"2024-09-10T22:13:05.299163+0000","last_became_active":"2024-09-10T22:13:04.294419+0000","last_became_peered":"2024-09-10T22:13:04.294419+0000","last_unstale":"2024-09-10T22:13:05.299163+0000","last_undegraded":"2024-09-10T22:13:05.299163+0000","last_fullsized":"2024-09-10T22:13:05.299163+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:35.852671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358873+0000","last_change":"2024-09-10T22:13:04.312086+0000","last_active":"2024-09-10T22:13:05.358873+0000","last_peered":"2024-09-10T22:13:05.358873+0000","last_clean":"2024-09-10T22:13:05.358873+0000","last_became_active":"2024-09-10T22:13:04.311781+0000","last_became_peered":"2024-09-10T22:13:04.311781+0000","last_unstale":"2024-09-10T22:13:05.358873+0000","last_undegraded":"2024-09-10T22:13:05.358873+0000","last_fullsized":"2024-09-10T22:13:05.358873+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:28:49.110733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353458+0000","last_change":"2024-09-10T22:13:04.305839+0000","last_active":"2024-09-10T22:13:05.353458+0000","last_peered":"2024-09-10T22:13:05.353458+0000","last_clean":"2024-09-10T22:13:05.353458+0000","last_became_active":"2024-09-10T22:13:04.305564+0000","last_became_peered":"2024-09-10T22:13:04.305564+0000","last_unstale":"2024-09-10T22:13:05.353458+0000","last_undegraded":"2024-09-10T22:13:05.353458+0000","last_fullsized":"2024-09-10T22:13:05.353458+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:11.940828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354288+0000","last_change":"2024-09-10T22:13:04.295549+0000","last_active":"2024-09-10T22:13:05.354288+0000","last_peered":"2024-09-10T22:13:05.354288+0000","last_clean":"2024-09-10T22:13:05.354288+0000","last_became_active":"2024-09-10T22:13:04.295104+0000","last_became_peered":"2024-09-10T22:13:04.295104+0000","last_unstale":"2024-09-10T22:13:05.354288+0000","last_undegraded":"2024-09-10T22:13:05.354288+0000","last_fullsized":"2024-09-10T22:13:05.354288+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:41.852294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.352894+0000","last_change":"2024-09-10T22:13:04.307792+0000","last_active":"2024-09-10T22:13:05.352894+0000","last_peered":"2024-09-10T22:13:05.352894+0000","last_clean":"2024-09-10T22:13:05.352894+0000","last_became_active":"2024-09-10T22:13:04.307499+0000","last_became_peered":"2024-09-10T22:13:04.307499+0000","last_unstale":"2024-09-10T22:13:05.352894+0000","last_undegraded":"2024-09-10T22:13:05.352894+0000","last_fullsized":"2024-09-10T22:13:05.352894+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:36.777792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.352919+0000","last_change":"2024-09-10T22:13:04.300118+0000","last_active":"2024-09-10T22:13:05.352919+0000","last_peered":"2024-09-10T22:13:05.352919+0000","last_clean":"2024-09-10T22:13:05.352919+0000","last_became_active":"2024-09-10T22:13:04.297650+0000","last_became_peered":"2024-09-10T22:13:04.297650+0000","last_unstale":"2024-09-10T22:13:05.352919+0000","last_undegraded":"2024-09-10T22:13:05.352919+0000","last_fullsized":"2024-09-10T22:13:05.352919+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:06.094339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.306900+0000","last_change":"2024-09-10T22:13:04.304273+0000","last_active":"2024-09-10T22:13:05.306900+0000","last_peered":"2024-09-10T22:13:05.306900+0000","last_clean":"2024-09-10T22:13:05.306900+0000","last_became_active":"2024-09-10T22:13:04.303978+0000","last_became_peered":"2024-09-10T22:13:04.303978+0000","last_unstale":"2024-09-10T22:13:05.306900+0000","last_undegraded":"2024-09-10T22:13:05.306900+0000","last_fullsized":"2024-09-10T22:13:05.306900+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:07.278355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353206+0000","last_change":"2024-09-10T22:13:04.297226+0000","last_active":"2024-09-10T22:13:05.353206+0000","last_peered":"2024-09-10T22:13:05.353206+0000","last_clean":"2024-09-10T22:13:05.353206+0000","last_became_active":"2024-09-10T22:13:04.296963+0000","last_became_peered":"2024-09-10T22:13:04.296963+0000","last_unstale":"2024-09-10T22:13:05.353206+0000","last_undegraded":"2024-09-10T22:13:05.353206+0000","last_fullsized":"2024-09-10T22:13:05.353206+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:47.577521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.299178+0000","last_change":"2024-09-10T22:13:04.298054+0000","last_active":"2024-09-10T22:13:05.299178+0000","last_peered":"2024-09-10T22:13:05.299178+0000","last_clean":"2024-09-10T22:13:05.299178+0000","last_became_active":"2024-09-10T22:13:04.297805+0000","last_became_peered":"2024-09-10T22:13:04.297805+0000","last_unstale":"2024-09-10T22:13:05.299178+0000","last_undegraded":"2024-09-10T22:13:05.299178+0000","last_fullsized":"2024-09-10T22:13:05.299178+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:30.788282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.352745+0000","last_change":"2024-09-10T22:13:04.295096+0000","last_active":"2024-09-10T22:13:05.352745+0000","last_peered":"2024-09-10T22:13:05.352745+0000","last_clean":"2024-09-10T22:13:05.352745+0000","last_became_active":"2024-09-10T22:13:04.294759+0000","last_became_peered":"2024-09-10T22:13:04.294759+0000","last_unstale":"2024-09-10T22:13:05.352745+0000","last_undegraded":"2024-09-10T22:13:05.352745+0000","last_fullsized":"2024-09-10T22:13:05.352745+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:09.306430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.357888+0000","last_change":"2024-09-10T22:13:04.308971+0000","last_active":"2024-09-10T22:13:05.357888+0000","last_peered":"2024-09-10T22:13:05.357888+0000","last_clean":"2024-09-10T22:13:05.357888+0000","last_became_active":"2024-09-10T22:13:04.308697+0000","last_became_peered":"2024-09-10T22:13:04.308697+0000","last_unstale":"2024-09-10T22:13:05.357888+0000","last_undegraded":"2024-09-10T22:13:05.357888+0000","last_fullsized":"2024-09-10T22:13:05.357888+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:54.871345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.299111+0000","last_change":"2024-09-10T22:13:04.292950+0000","last_active":"2024-09-10T22:13:05.299111+0000","last_peered":"2024-09-10T22:13:05.299111+0000","last_clean":"2024-09-10T22:13:05.299111+0000","last_became_active":"2024-09-10T22:13:04.292683+0000","last_became_peered":"2024-09-10T22:13:04.292683+0000","last_unstale":"2024-09-10T22:13:05.299111+0000","last_undegraded":"2024-09-10T22:13:05.299111+0000","last_fullsized":"2024-09-10T22:13:05.299111+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:20.930325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.359315+0000","last_change":"2024-09-10T22:13:04.311834+0000","last_active":"2024-09-10T22:13:05.359315+0000","last_peered":"2024-09-10T22:13:05.359315+0000","last_clean":"2024-09-10T22:13:05.359315+0000","last_became_active":"2024-09-10T22:13:04.311470+0000","last_became_peered":"2024-09-10T22:13:04.311470+0000","last_unstale":"2024-09-10T22:13:05.359315+0000","last_undegraded":"2024-09-10T22:13:05.359315+0000","last_fullsized":"2024-09-10T22:13:05.359315+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:34.289144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.298808+0000","last_change":"2024-09-10T22:13:04.302032+0000","last_active":"2024-09-10T22:13:05.298808+0000","last_peered":"2024-09-10T22:13:05.298808+0000","last_clean":"2024-09-10T22:13:05.298808+0000","last_became_active":"2024-09-10T22:13:04.301734+0000","last_became_peered":"2024-09-10T22:13:04.301734+0000","last_unstale":"2024-09-10T22:13:05.298808+0000","last_undegraded":"2024-09-10T22:13:05.298808+0000","last_fullsized":"2024-09-10T22:13:05.298808+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:57.493736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354638+0000","last_change":"2024-09-10T22:13:04.292323+0000","last_active":"2024-09-10T22:13:05.354638+0000","last_peered":"2024-09-10T22:13:05.354638+0000","last_clean":"2024-09-10T22:13:05.354638+0000","last_became_active":"2024-09-10T22:13:04.292008+0000","last_became_peered":"2024-09-10T22:13:04.292008+0000","last_unstale":"2024-09-10T22:13:05.354638+0000","last_undegraded":"2024-09-10T22:13:05.354638+0000","last_fullsized":"2024-09-10T22:13:05.354638+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:52.727856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353396+0000","last_change":"2024-09-10T22:13:04.297258+0000","last_active":"2024-09-10T22:13:05.353396+0000","last_peered":"2024-09-10T22:13:05.353396+0000","last_clean":"2024-09-10T22:13:05.353396+0000","last_became_active":"2024-09-10T22:13:04.296668+0000","last_became_peered":"2024-09-10T22:13:04.296668+0000","last_unstale":"2024-09-10T22:13:05.353396+0000","last_undegraded":"2024-09-10T22:13:05.353396+0000","last_fullsized":"2024-09-10T22:13:05.353396+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:02.727363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353548+0000","last_change":"2024-09-10T22:13:04.308108+0000","last_active":"2024-09-10T22:13:05.353548+0000","last_peered":"2024-09-10T22:13:05.353548+0000","last_clean":"2024-09-10T22:13:05.353548+0000","last_became_active":"2024-09-10T22:13:04.307823+0000","last_became_peered":"2024-09-10T22:13:04.307823+0000","last_unstale":"2024-09-10T22:13:05.353548+0000","last_undegraded":"2024-09-10T22:13:05.353548+0000","last_fullsized":"2024-09-10T22:13:05.353548+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:03.333787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.304976+0000","last_change":"2024-09-10T22:13:04.304454+0000","last_active":"2024-09-10T22:13:05.304976+0000","last_peered":"2024-09-10T22:13:05.304976+0000","last_clean":"2024-09-10T22:13:05.304976+0000","last_became_active":"2024-09-10T22:13:04.304145+0000","last_became_peered":"2024-09-10T22:13:04.304145+0000","last_unstale":"2024-09-10T22:13:05.304976+0000","last_undegraded":"2024-09-10T22:13:05.304976+0000","last_fullsized":"2024-09-10T22:13:05.304976+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:09.459785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353494+0000","last_change":"2024-09-10T22:13:04.309277+0000","last_active":"2024-09-10T22:13:05.353494+0000","last_peered":"2024-09-10T22:13:05.353494+0000","last_clean":"2024-09-10T22:13:05.353494+0000","last_became_active":"2024-09-10T22:13:04.307644+0000","last_became_peered":"2024-09-10T22:13:04.307644+0000","last_unstale":"2024-09-10T22:13:05.353494+0000","last_undegraded":"2024-09-10T22:13:05.353494+0000","last_fullsized":"2024-09-10T22:13:05.353494+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:30.854268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353856+0000","last_change":"2024-09-10T22:13:04.307932+0000","last_active":"2024-09-10T22:13:05.353856+0000","last_peered":"2024-09-10T22:13:05.353856+0000","last_clean":"2024-09-10T22:13:05.353856+0000","last_became_active":"2024-09-10T22:13:04.307658+0000","last_became_peered":"2024-09-10T22:13:04.307658+0000","last_unstale":"2024-09-10T22:13:05.353856+0000","last_undegraded":"2024-09-10T22:13:05.353856+0000","last_fullsized":"2024-09-10T22:13:05.353856+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:07.546337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.357881+0000","last_change":"2024-09-10T22:13:04.315873+0000","last_active":"2024-09-10T22:13:05.357881+0000","last_peered":"2024-09-10T22:13:05.357881+0000","last_clean":"2024-09-10T22:13:05.357881+0000","last_became_active":"2024-09-10T22:13:04.315599+0000","last_became_peered":"2024-09-10T22:13:04.315599+0000","last_unstale":"2024-09-10T22:13:05.357881+0000","last_undegraded":"2024-09-10T22:13:05.357881+0000","last_fullsized":"2024-09-10T22:13:05.357881+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:49.667765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.305362+0000","last_change":"2024-09-10T22:13:04.310875+0000","last_active":"2024-09-10T22:13:05.305362+0000","last_peered":"2024-09-10T22:13:05.305362+0000","last_clean":"2024-09-10T22:13:05.305362+0000","last_became_active":"2024-09-10T22:13:04.309855+0000","last_became_peered":"2024-09-10T22:13:04.309855+0000","last_unstale":"2024-09-10T22:13:05.305362+0000","last_undegraded":"2024-09-10T22:13:05.305362+0000","last_fullsized":"2024-09-10T22:13:05.305362+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:29.575473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353592+0000","last_change":"2024-09-10T22:13:04.296920+0000","last_active":"2024-09-10T22:13:05.353592+0000","last_peered":"2024-09-10T22:13:05.353592+0000","last_clean":"2024-09-10T22:13:05.353592+0000","last_became_active":"2024-09-10T22:13:04.296593+0000","last_became_peered":"2024-09-10T22:13:04.296593+0000","last_unstale":"2024-09-10T22:13:05.353592+0000","last_undegraded":"2024-09-10T22:13:05.353592+0000","last_fullsized":"2024-09-10T22:13:05.353592+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:46.309028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"125'4","reported_seq":19,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358378+0000","last_change":"2024-09-10T22:13:03.308357+0000","last_active":"2024-09-10T22:13:05.358378+0000","last_peered":"2024-09-10T22:13:05.358378+0000","last_clean":"2024-09-10T22:13:05.358378+0000","last_became_active":"2024-09-10T22:13:03.308101+0000","last_became_peered":"2024-09-10T22:13:03.308101+0000","last_unstale":"2024-09-10T22:13:05.358378+0000","last_undegraded":"2024-09-10T22:13:05.358378+0000","last_fullsized":"2024-09-10T22:13:05.358378+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:40.996306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.359221+0000","last_change":"2024-09-10T22:13:04.311870+0000","last_active":"2024-09-10T22:13:05.359221+0000","last_peered":"2024-09-10T22:13:05.359221+0000","last_clean":"2024-09-10T22:13:05.359221+0000","last_became_active":"2024-09-10T22:13:04.311467+0000","last_became_peered":"2024-09-10T22:13:04.311467+0000","last_unstale":"2024-09-10T22:13:05.359221+0000","last_undegraded":"2024-09-10T22:13:05.359221+0000","last_fullsized":"2024-09-10T22:13:05.359221+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:20.380331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"125'1","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.305721+0000","last_change":"2024-09-10T22:13:03.297559+0000","last_active":"2024-09-10T22:13:05.305721+0000","last_peered":"2024-09-10T22:13:05.305721+0000","last_clean":"2024-09-10T22:13:05.305721+0000","last_became_active":"2024-09-10T22:13:03.297067+0000","last_became_peered":"2024-09-10T22:13:03.297067+0000","last_unstale":"2024-09-10T22:13:05.305721+0000","last_undegraded":"2024-09-10T22:13:05.305721+0000","last_fullsized":"2024-09-10T22:13:05.305721+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:01.337755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358335+0000","last_change":"2024-09-10T22:13:04.304914+0000","last_active":"2024-09-10T22:13:05.358335+0000","last_peered":"2024-09-10T22:13:05.358335+0000","last_clean":"2024-09-10T22:13:05.358335+0000","last_became_active":"2024-09-10T22:13:04.304523+0000","last_became_peered":"2024-09-10T22:13:04.304523+0000","last_unstale":"2024-09-10T22:13:05.358335+0000","last_undegraded":"2024-09-10T22:13:05.358335+0000","last_fullsized":"2024-09-10T22:13:05.358335+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:54.877542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"125'2","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.299232+0000","last_change":"2024-09-10T22:13:03.289610+0000","last_active":"2024-09-10T22:13:05.299232+0000","last_peered":"2024-09-10T22:13:05.299232+0000","last_clean":"2024-09-10T22:13:05.299232+0000","last_became_active":"2024-09-10T22:13:03.289295+0000","last_became_peered":"2024-09-10T22:13:03.289295+0000","last_unstale":"2024-09-10T22:13:05.299232+0000","last_undegraded":"2024-09-10T22:13:05.299232+0000","last_fullsized":"2024-09-10T22:13:05.299232+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:16.483500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358215+0000","last_change":"2024-09-10T22:13:04.307991+0000","last_active":"2024-09-10T22:13:05.358215+0000","last_peered":"2024-09-10T22:13:05.358215+0000","last_clean":"2024-09-10T22:13:05.358215+0000","last_became_active":"2024-09-10T22:13:04.306506+0000","last_became_peered":"2024-09-10T22:13:04.306506+0000","last_unstale":"2024-09-10T22:13:05.358215+0000","last_undegraded":"2024-09-10T22:13:05.358215+0000","last_fullsized":"2024-09-10T22:13:05.358215+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:11:37.531809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"125'2","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354541+0000","last_change":"2024-09-10T22:13:03.296977+0000","last_active":"2024-09-10T22:13:05.354541+0000","last_peered":"2024-09-10T22:13:05.354541+0000","last_clean":"2024-09-10T22:13:05.354541+0000","last_became_active":"2024-09-10T22:13:03.296696+0000","last_became_peered":"2024-09-10T22:13:03.296696+0000","last_unstale":"2024-09-10T22:13:05.354541+0000","last_undegraded":"2024-09-10T22:13:05.354541+0000","last_fullsized":"2024-09-10T22:13:05.354541+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:29.464284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.352791+0000","last_change":"2024-09-10T22:13:04.300807+0000","last_active":"2024-09-10T22:13:05.352791+0000","last_peered":"2024-09-10T22:13:05.352791+0000","last_clean":"2024-09-10T22:13:05.352791+0000","last_became_active":"2024-09-10T22:13:04.295824+0000","last_became_peered":"2024-09-10T22:13:04.295824+0000","last_unstale":"2024-09-10T22:13:05.352791+0000","last_undegraded":"2024-09-10T22:13:05.352791+0000","last_fullsized":"2024-09-10T22:13:05.352791+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:08.473657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"125'1","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354667+0000","last_change":"2024-09-10T22:13:03.296322+0000","last_active":"2024-09-10T22:13:05.354667+0000","last_peered":"2024-09-10T22:13:05.354667+0000","last_clean":"2024-09-10T22:13:05.354667+0000","last_became_active":"2024-09-10T22:13:03.294933+0000","last_became_peered":"2024-09-10T22:13:03.294933+0000","last_unstale":"2024-09-10T22:13:05.354667+0000","last_undegraded":"2024-09-10T22:13:05.354667+0000","last_fullsized":"2024-09-10T22:13:05.354667+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:24:36.658470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.306531+0000","last_change":"2024-09-10T22:13:04.307050+0000","last_active":"2024-09-10T22:13:05.306531+0000","last_peered":"2024-09-10T22:13:05.306531+0000","last_clean":"2024-09-10T22:13:05.306531+0000","last_became_active":"2024-09-10T22:13:04.306755+0000","last_became_peered":"2024-09-10T22:13:04.306755+0000","last_unstale":"2024-09-10T22:13:05.306531+0000","last_undegraded":"2024-09-10T22:13:05.306531+0000","last_fullsized":"2024-09-10T22:13:05.306531+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:00.041249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"125'1","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.352903+0000","last_change":"2024-09-10T22:13:03.301959+0000","last_active":"2024-09-10T22:13:05.352903+0000","last_peered":"2024-09-10T22:13:05.352903+0000","last_clean":"2024-09-10T22:13:05.352903+0000","last_became_active":"2024-09-10T22:13:03.301666+0000","last_became_peered":"2024-09-10T22:13:03.301666+0000","last_unstale":"2024-09-10T22:13:05.352903+0000","last_undegraded":"2024-09-10T22:13:05.352903+0000","last_fullsized":"2024-09-10T22:13:05.352903+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:14.815306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358471+0000","last_change":"2024-09-10T22:13:04.318173+0000","last_active":"2024-09-10T22:13:05.358471+0000","last_peered":"2024-09-10T22:13:05.358471+0000","last_clean":"2024-09-10T22:13:05.358471+0000","last_became_active":"2024-09-10T22:13:04.317478+0000","last_became_peered":"2024-09-10T22:13:04.317478+0000","last_unstale":"2024-09-10T22:13:05.358471+0000","last_undegraded":"2024-09-10T22:13:05.358471+0000","last_fullsized":"2024-09-10T22:13:05.358471+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:54.077422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.304748+0000","last_change":"2024-09-10T22:13:03.293516+0000","last_active":"2024-09-10T22:13:05.304748+0000","last_peered":"2024-09-10T22:13:05.304748+0000","last_clean":"2024-09-10T22:13:05.304748+0000","last_became_active":"2024-09-10T22:13:03.293168+0000","last_became_peered":"2024-09-10T22:13:03.293168+0000","last_unstale":"2024-09-10T22:13:05.304748+0000","last_undegraded":"2024-09-10T22:13:05.304748+0000","last_fullsized":"2024-09-10T22:13:05.304748+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:36.164847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353031+0000","last_change":"2024-09-10T22:13:04.300252+0000","last_active":"2024-09-10T22:13:05.353031+0000","last_peered":"2024-09-10T22:13:05.353031+0000","last_clean":"2024-09-10T22:13:05.353031+0000","last_became_active":"2024-09-10T22:13:04.299997+0000","last_became_peered":"2024-09-10T22:13:04.299997+0000","last_unstale":"2024-09-10T22:13:05.353031+0000","last_undegraded":"2024-09-10T22:13:05.353031+0000","last_fullsized":"2024-09-10T22:13:05.353031+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:32.536456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.352802+0000","last_change":"2024-09-10T22:13:03.302822+0000","last_active":"2024-09-10T22:13:05.352802+0000","last_peered":"2024-09-10T22:13:05.352802+0000","last_clean":"2024-09-10T22:13:05.352802+0000","last_became_active":"2024-09-10T22:13:03.302545+0000","last_became_peered":"2024-09-10T22:13:03.302545+0000","last_unstale":"2024-09-10T22:13:05.352802+0000","last_undegraded":"2024-09-10T22:13:05.352802+0000","last_fullsized":"2024-09-10T22:13:05.352802+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:39.909968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353704+0000","last_change":"2024-09-10T22:13:04.302884+0000","last_active":"2024-09-10T22:13:05.353704+0000","last_peered":"2024-09-10T22:13:05.353704+0000","last_clean":"2024-09-10T22:13:05.353704+0000","last_became_active":"2024-09-10T22:13:04.300050+0000","last_became_peered":"2024-09-10T22:13:04.300050+0000","last_unstale":"2024-09-10T22:13:05.353704+0000","last_undegraded":"2024-09-10T22:13:05.353704+0000","last_fullsized":"2024-09-10T22:13:05.353704+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:35.721001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.305467+0000","last_change":"2024-09-10T22:13:03.307776+0000","last_active":"2024-09-10T22:13:05.305467+0000","last_peered":"2024-09-10T22:13:05.305467+0000","last_clean":"2024-09-10T22:13:05.305467+0000","last_became_active":"2024-09-10T22:13:03.307421+0000","last_became_peered":"2024-09-10T22:13:03.307421+0000","last_unstale":"2024-09-10T22:13:05.305467+0000","last_undegraded":"2024-09-10T22:13:05.305467+0000","last_fullsized":"2024-09-10T22:13:05.305467+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:42.757842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.305385+0000","last_change":"2024-09-10T22:13:04.307583+0000","last_active":"2024-09-10T22:13:05.305385+0000","last_peered":"2024-09-10T22:13:05.305385+0000","last_clean":"2024-09-10T22:13:05.305385+0000","last_became_active":"2024-09-10T22:13:04.307323+0000","last_became_peered":"2024-09-10T22:13:04.307323+0000","last_unstale":"2024-09-10T22:13:05.305385+0000","last_undegraded":"2024-09-10T22:13:05.305385+0000","last_fullsized":"2024-09-10T22:13:05.305385+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:59.555127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353692+0000","last_change":"2024-09-10T22:13:03.302385+0000","last_active":"2024-09-10T22:13:05.353692+0000","last_peered":"2024-09-10T22:13:05.353692+0000","last_clean":"2024-09-10T22:13:05.353692+0000","last_became_active":"2024-09-10T22:13:03.302098+0000","last_became_peered":"2024-09-10T22:13:03.302098+0000","last_unstale":"2024-09-10T22:13:05.353692+0000","last_undegraded":"2024-09-10T22:13:05.353692+0000","last_fullsized":"2024-09-10T22:13:05.353692+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:57.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358193+0000","last_change":"2024-09-10T22:13:04.305085+0000","last_active":"2024-09-10T22:13:05.358193+0000","last_peered":"2024-09-10T22:13:05.358193+0000","last_clean":"2024-09-10T22:13:05.358193+0000","last_became_active":"2024-09-10T22:13:04.304813+0000","last_became_peered":"2024-09-10T22:13:04.304813+0000","last_unstale":"2024-09-10T22:13:05.358193+0000","last_undegraded":"2024-09-10T22:13:05.358193+0000","last_fullsized":"2024-09-10T22:13:05.358193+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:28.233859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"125'3","reported_seq":18,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.305086+0000","last_change":"2024-09-10T22:13:03.297844+0000","last_active":"2024-09-10T22:13:05.305086+0000","last_peered":"2024-09-10T22:13:05.305086+0000","last_clean":"2024-09-10T22:13:05.305086+0000","last_became_active":"2024-09-10T22:13:03.297558+0000","last_became_peered":"2024-09-10T22:13:03.297558+0000","last_unstale":"2024-09-10T22:13:05.305086+0000","last_undegraded":"2024-09-10T22:13:05.305086+0000","last_fullsized":"2024-09-10T22:13:05.305086+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:22.921721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358413+0000","last_change":"2024-09-10T22:13:04.322234+0000","last_active":"2024-09-10T22:13:05.358413+0000","last_peered":"2024-09-10T22:13:05.358413+0000","last_clean":"2024-09-10T22:13:05.358413+0000","last_became_active":"2024-09-10T22:13:04.321830+0000","last_became_peered":"2024-09-10T22:13:04.321830+0000","last_unstale":"2024-09-10T22:13:05.358413+0000","last_undegraded":"2024-09-10T22:13:05.358413+0000","last_fullsized":"2024-09-10T22:13:05.358413+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:31.250264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"125'6","reported_seq":21,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358782+0000","last_change":"2024-09-10T22:13:03.302296+0000","last_active":"2024-09-10T22:13:05.358782+0000","last_peered":"2024-09-10T22:13:05.358782+0000","last_clean":"2024-09-10T22:13:05.358782+0000","last_became_active":"2024-09-10T22:13:03.301986+0000","last_became_peered":"2024-09-10T22:13:03.301986+0000","last_unstale":"2024-09-10T22:13:05.358782+0000","last_undegraded":"2024-09-10T22:13:05.358782+0000","last_fullsized":"2024-09-10T22:13:05.358782+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:39.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354042+0000","last_change":"2024-09-10T22:13:04.300726+0000","last_active":"2024-09-10T22:13:05.354042+0000","last_peered":"2024-09-10T22:13:05.354042+0000","last_clean":"2024-09-10T22:13:05.354042+0000","last_became_active":"2024-09-10T22:13:04.300465+0000","last_became_peered":"2024-09-10T22:13:04.300465+0000","last_unstale":"2024-09-10T22:13:05.354042+0000","last_undegraded":"2024-09-10T22:13:05.354042+0000","last_fullsized":"2024-09-10T22:13:05.354042+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:28.540714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353881+0000","last_change":"2024-09-10T22:13:03.295996+0000","last_active":"2024-09-10T22:13:05.353881+0000","last_peered":"2024-09-10T22:13:05.353881+0000","last_clean":"2024-09-10T22:13:05.353881+0000","last_became_active":"2024-09-10T22:13:03.295495+0000","last_became_peered":"2024-09-10T22:13:03.295495+0000","last_unstale":"2024-09-10T22:13:05.353881+0000","last_undegraded":"2024-09-10T22:13:05.353881+0000","last_fullsized":"2024-09-10T22:13:05.353881+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:34:54.520382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.352666+0000","last_change":"2024-09-10T22:13:04.314652+0000","last_active":"2024-09-10T22:13:05.352666+0000","last_peered":"2024-09-10T22:13:05.352666+0000","last_clean":"2024-09-10T22:13:05.352666+0000","last_became_active":"2024-09-10T22:13:04.314302+0000","last_became_peered":"2024-09-10T22:13:04.314302+0000","last_unstale":"2024-09-10T22:13:05.352666+0000","last_undegraded":"2024-09-10T22:13:05.352666+0000","last_fullsized":"2024-09-10T22:13:05.352666+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:39.423485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354183+0000","last_change":"2024-09-10T22:13:03.298374+0000","last_active":"2024-09-10T22:13:05.354183+0000","last_peered":"2024-09-10T22:13:05.354183+0000","last_clean":"2024-09-10T22:13:05.354183+0000","last_became_active":"2024-09-10T22:13:03.298093+0000","last_became_peered":"2024-09-10T22:13:03.298093+0000","last_unstale":"2024-09-10T22:13:05.354183+0000","last_undegraded":"2024-09-10T22:13:05.354183+0000","last_fullsized":"2024-09-10T22:13:05.354183+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:21.370807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358760+0000","last_change":"2024-09-10T22:13:04.312907+0000","last_active":"2024-09-10T22:13:05.358760+0000","last_peered":"2024-09-10T22:13:05.358760+0000","last_clean":"2024-09-10T22:13:05.358760+0000","last_became_active":"2024-09-10T22:13:04.311853+0000","last_became_peered":"2024-09-10T22:13:04.311853+0000","last_unstale":"2024-09-10T22:13:05.358760+0000","last_undegraded":"2024-09-10T22:13:05.358760+0000","last_fullsized":"2024-09-10T22:13:05.358760+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:37.167336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"125'8","reported_seq":23,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358548+0000","last_change":"2024-09-10T22:13:03.303728+0000","last_active":"2024-09-10T22:13:05.358548+0000","last_peered":"2024-09-10T22:13:05.358548+0000","last_clean":"2024-09-10T22:13:05.358548+0000","last_became_active":"2024-09-10T22:13:03.302954+0000","last_became_peered":"2024-09-10T22:13:03.302954+0000","last_unstale":"2024-09-10T22:13:05.358548+0000","last_undegraded":"2024-09-10T22:13:05.358548+0000","last_fullsized":"2024-09-10T22:13:05.358548+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:29.597432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358977+0000","last_change":"2024-09-10T22:13:04.315711+0000","last_active":"2024-09-10T22:13:05.358977+0000","last_peered":"2024-09-10T22:13:05.358977+0000","last_clean":"2024-09-10T22:13:05.358977+0000","last_became_active":"2024-09-10T22:13:04.315258+0000","last_became_peered":"2024-09-10T22:13:04.315258+0000","last_unstale":"2024-09-10T22:13:05.358977+0000","last_undegraded":"2024-09-10T22:13:05.358977+0000","last_fullsized":"2024-09-10T22:13:05.358977+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:37.267333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"125'1","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.305500+0000","last_change":"2024-09-10T22:13:03.297750+0000","last_active":"2024-09-10T22:13:05.305500+0000","last_peered":"2024-09-10T22:13:05.305500+0000","last_clean":"2024-09-10T22:13:05.305500+0000","last_became_active":"2024-09-10T22:13:03.297426+0000","last_became_peered":"2024-09-10T22:13:03.297426+0000","last_unstale":"2024-09-10T22:13:05.305500+0000","last_undegraded":"2024-09-10T22:13:05.305500+0000","last_fullsized":"2024-09-10T22:13:05.305500+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:07.996552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.305400+0000","last_change":"2024-09-10T22:13:04.300618+0000","last_active":"2024-09-10T22:13:05.305400+0000","last_peered":"2024-09-10T22:13:05.305400+0000","last_clean":"2024-09-10T22:13:05.305400+0000","last_became_active":"2024-09-10T22:13:04.298905+0000","last_became_peered":"2024-09-10T22:13:04.298905+0000","last_unstale":"2024-09-10T22:13:05.305400+0000","last_undegraded":"2024-09-10T22:13:05.305400+0000","last_fullsized":"2024-09-10T22:13:05.305400+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:53.571160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"125'2","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.298916+0000","last_change":"2024-09-10T22:13:03.300748+0000","last_active":"2024-09-10T22:13:05.298916+0000","last_peered":"2024-09-10T22:13:05.298916+0000","last_clean":"2024-09-10T22:13:05.298916+0000","last_became_active":"2024-09-10T22:13:03.298252+0000","last_became_peered":"2024-09-10T22:13:03.298252+0000","last_unstale":"2024-09-10T22:13:05.298916+0000","last_undegraded":"2024-09-10T22:13:05.298916+0000","last_fullsized":"2024-09-10T22:13:05.298916+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:47.208844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.352961+0000","last_change":"2024-09-10T22:13:04.302392+0000","last_active":"2024-09-10T22:13:05.352961+0000","last_peered":"2024-09-10T22:13:05.352961+0000","last_clean":"2024-09-10T22:13:05.352961+0000","last_became_active":"2024-09-10T22:13:04.300375+0000","last_became_peered":"2024-09-10T22:13:04.300375+0000","last_unstale":"2024-09-10T22:13:05.352961+0000","last_undegraded":"2024-09-10T22:13:05.352961+0000","last_fullsized":"2024-09-10T22:13:05.352961+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:23.874250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"125'1","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.306365+0000","last_change":"2024-09-10T22:13:03.293968+0000","last_active":"2024-09-10T22:13:05.306365+0000","last_peered":"2024-09-10T22:13:05.306365+0000","last_clean":"2024-09-10T22:13:05.306365+0000","last_became_active":"2024-09-10T22:13:03.293690+0000","last_became_peered":"2024-09-10T22:13:03.293690+0000","last_unstale":"2024-09-10T22:13:05.306365+0000","last_undegraded":"2024-09-10T22:13:05.306365+0000","last_fullsized":"2024-09-10T22:13:05.306365+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:28:48.108244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354506+0000","last_change":"2024-09-10T22:13:04.300010+0000","last_active":"2024-09-10T22:13:05.354506+0000","last_peered":"2024-09-10T22:13:05.354506+0000","last_clean":"2024-09-10T22:13:05.354506+0000","last_became_active":"2024-09-10T22:13:04.299719+0000","last_became_peered":"2024-09-10T22:13:04.299719+0000","last_unstale":"2024-09-10T22:13:05.354506+0000","last_undegraded":"2024-09-10T22:13:05.354506+0000","last_fullsized":"2024-09-10T22:13:05.354506+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:46.355493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.304914+0000","last_change":"2024-09-10T22:13:03.296135+0000","last_active":"2024-09-10T22:13:05.304914+0000","last_peered":"2024-09-10T22:13:05.304914+0000","last_clean":"2024-09-10T22:13:05.304914+0000","last_became_active":"2024-09-10T22:13:03.295778+0000","last_became_peered":"2024-09-10T22:13:03.295778+0000","last_unstale":"2024-09-10T22:13:05.304914+0000","last_undegraded":"2024-09-10T22:13:05.304914+0000","last_fullsized":"2024-09-10T22:13:05.304914+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:14.560879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353779+0000","last_change":"2024-09-10T22:13:04.307084+0000","last_active":"2024-09-10T22:13:05.353779+0000","last_peered":"2024-09-10T22:13:05.353779+0000","last_clean":"2024-09-10T22:13:05.353779+0000","last_became_active":"2024-09-10T22:13:04.306577+0000","last_became_peered":"2024-09-10T22:13:04.306577+0000","last_unstale":"2024-09-10T22:13:05.353779+0000","last_undegraded":"2024-09-10T22:13:05.353779+0000","last_fullsized":"2024-09-10T22:13:05.353779+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:07.936676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"125'2","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.304932+0000","last_change":"2024-09-10T22:13:03.307784+0000","last_active":"2024-09-10T22:13:05.304932+0000","last_peered":"2024-09-10T22:13:05.304932+0000","last_clean":"2024-09-10T22:13:05.304932+0000","last_became_active":"2024-09-10T22:13:03.307469+0000","last_became_peered":"2024-09-10T22:13:03.307469+0000","last_unstale":"2024-09-10T22:13:05.304932+0000","last_undegraded":"2024-09-10T22:13:05.304932+0000","last_fullsized":"2024-09-10T22:13:05.304932+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:57.558458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.306370+0000","last_change":"2024-09-10T22:13:04.318396+0000","last_active":"2024-09-10T22:13:05.306370+0000","last_peered":"2024-09-10T22:13:05.306370+0000","last_clean":"2024-09-10T22:13:05.306370+0000","last_became_active":"2024-09-10T22:13:04.318096+0000","last_became_peered":"2024-09-10T22:13:04.318096+0000","last_unstale":"2024-09-10T22:13:05.306370+0000","last_undegraded":"2024-09-10T22:13:05.306370+0000","last_fullsized":"2024-09-10T22:13:05.306370+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:09.233924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353275+0000","last_change":"2024-09-10T22:13:03.301812+0000","last_active":"2024-09-10T22:13:05.353275+0000","last_peered":"2024-09-10T22:13:05.353275+0000","last_clean":"2024-09-10T22:13:05.353275+0000","last_became_active":"2024-09-10T22:13:03.301346+0000","last_became_peered":"2024-09-10T22:13:03.301346+0000","last_unstale":"2024-09-10T22:13:05.353275+0000","last_undegraded":"2024-09-10T22:13:05.353275+0000","last_fullsized":"2024-09-10T22:13:05.353275+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:42.316632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.352773+0000","last_change":"2024-09-10T22:13:04.293567+0000","last_active":"2024-09-10T22:13:05.352773+0000","last_peered":"2024-09-10T22:13:05.352773+0000","last_clean":"2024-09-10T22:13:05.352773+0000","last_became_active":"2024-09-10T22:13:04.290504+0000","last_became_peered":"2024-09-10T22:13:04.290504+0000","last_unstale":"2024-09-10T22:13:05.352773+0000","last_undegraded":"2024-09-10T22:13:05.352773+0000","last_fullsized":"2024-09-10T22:13:05.352773+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:36.273579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.352482+0000","last_change":"2024-09-10T22:13:03.293919+0000","last_active":"2024-09-10T22:13:05.352482+0000","last_peered":"2024-09-10T22:13:05.352482+0000","last_clean":"2024-09-10T22:13:05.352482+0000","last_became_active":"2024-09-10T22:13:03.293657+0000","last_became_peered":"2024-09-10T22:13:03.293657+0000","last_unstale":"2024-09-10T22:13:05.352482+0000","last_undegraded":"2024-09-10T22:13:05.352482+0000","last_fullsized":"2024-09-10T22:13:05.352482+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:50.498092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.357860+0000","last_change":"2024-09-10T22:13:04.313158+0000","last_active":"2024-09-10T22:13:05.357860+0000","last_peered":"2024-09-10T22:13:05.357860+0000","last_clean":"2024-09-10T22:13:05.357860+0000","last_became_active":"2024-09-10T22:13:04.310905+0000","last_became_peered":"2024-09-10T22:13:04.310905+0000","last_unstale":"2024-09-10T22:13:05.357860+0000","last_undegraded":"2024-09-10T22:13:05.357860+0000","last_fullsized":"2024-09-10T22:13:05.357860+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:27.071060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.298673+0000","last_change":"2024-09-10T22:13:03.299144+0000","last_active":"2024-09-10T22:13:05.298673+0000","last_peered":"2024-09-10T22:13:05.298673+0000","last_clean":"2024-09-10T22:13:05.298673+0000","last_became_active":"2024-09-10T22:13:03.298863+0000","last_became_peered":"2024-09-10T22:13:03.298863+0000","last_unstale":"2024-09-10T22:13:05.298673+0000","last_undegraded":"2024-09-10T22:13:05.298673+0000","last_fullsized":"2024-09-10T22:13:05.298673+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:52.221857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.306899+0000","last_change":"2024-09-10T22:13:04.309229+0000","last_active":"2024-09-10T22:13:05.306899+0000","last_peered":"2024-09-10T22:13:05.306899+0000","last_clean":"2024-09-10T22:13:05.306899+0000","last_became_active":"2024-09-10T22:13:04.308903+0000","last_became_peered":"2024-09-10T22:13:04.308903+0000","last_unstale":"2024-09-10T22:13:05.306899+0000","last_undegraded":"2024-09-10T22:13:05.306899+0000","last_fullsized":"2024-09-10T22:13:05.306899+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:58.673515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"125'2","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353932+0000","last_change":"2024-09-10T22:13:03.302829+0000","last_active":"2024-09-10T22:13:05.353932+0000","last_peered":"2024-09-10T22:13:05.353932+0000","last_clean":"2024-09-10T22:13:05.353932+0000","last_became_active":"2024-09-10T22:13:03.302579+0000","last_became_peered":"2024-09-10T22:13:03.302579+0000","last_unstale":"2024-09-10T22:13:05.353932+0000","last_undegraded":"2024-09-10T22:13:05.353932+0000","last_fullsized":"2024-09-10T22:13:05.353932+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:35.384612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353716+0000","last_change":"2024-09-10T22:13:04.316440+0000","last_active":"2024-09-10T22:13:05.353716+0000","last_peered":"2024-09-10T22:13:05.353716+0000","last_clean":"2024-09-10T22:13:05.353716+0000","last_became_active":"2024-09-10T22:13:04.316162+0000","last_became_peered":"2024-09-10T22:13:04.316162+0000","last_unstale":"2024-09-10T22:13:05.353716+0000","last_undegraded":"2024-09-10T22:13:05.353716+0000","last_fullsized":"2024-09-10T22:13:05.353716+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:32.587807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"126'1","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.358155+0000","last_change":"2024-09-10T22:13:03.305387+0000","last_active":"2024-09-10T22:13:05.358155+0000","last_peered":"2024-09-10T22:13:05.358155+0000","last_clean":"2024-09-10T22:13:05.358155+0000","last_became_active":"2024-09-10T22:13:03.304974+0000","last_became_peered":"2024-09-10T22:13:03.304974+0000","last_unstale":"2024-09-10T22:13:05.358155+0000","last_undegraded":"2024-09-10T22:13:05.358155+0000","last_fullsized":"2024-09-10T22:13:05.358155+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:36.452122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354417+0000","last_change":"2024-09-10T22:13:04.290966+0000","last_active":"2024-09-10T22:13:05.354417+0000","last_peered":"2024-09-10T22:13:05.354417+0000","last_clean":"2024-09-10T22:13:05.354417+0000","last_became_active":"2024-09-10T22:13:04.289892+0000","last_became_peered":"2024-09-10T22:13:04.289892+0000","last_unstale":"2024-09-10T22:13:05.354417+0000","last_undegraded":"2024-09-10T22:13:05.354417+0000","last_fullsized":"2024-09-10T22:13:05.354417+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:44.322467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354426+0000","last_change":"2024-09-10T22:13:03.299056+0000","last_active":"2024-09-10T22:13:05.354426+0000","last_peered":"2024-09-10T22:13:05.354426+0000","last_clean":"2024-09-10T22:13:05.354426+0000","last_became_active":"2024-09-10T22:13:03.298843+0000","last_became_peered":"2024-09-10T22:13:03.298843+0000","last_unstale":"2024-09-10T22:13:05.354426+0000","last_undegraded":"2024-09-10T22:13:05.354426+0000","last_fullsized":"2024-09-10T22:13:05.354426+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:06.539219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.306528+0000","last_change":"2024-09-10T22:13:04.305579+0000","last_active":"2024-09-10T22:13:05.306528+0000","last_peered":"2024-09-10T22:13:05.306528+0000","last_clean":"2024-09-10T22:13:05.306528+0000","last_became_active":"2024-09-10T22:13:04.304695+0000","last_became_peered":"2024-09-10T22:13:04.304695+0000","last_unstale":"2024-09-10T22:13:05.306528+0000","last_undegraded":"2024-09-10T22:13:05.306528+0000","last_fullsized":"2024-09-10T22:13:05.306528+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:32.155558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.299026+0000","last_change":"2024-09-10T22:13:03.296838+0000","last_active":"2024-09-10T22:13:05.299026+0000","last_peered":"2024-09-10T22:13:05.299026+0000","last_clean":"2024-09-10T22:13:05.299026+0000","last_became_active":"2024-09-10T22:13:03.296570+0000","last_became_peered":"2024-09-10T22:13:03.296570+0000","last_unstale":"2024-09-10T22:13:05.299026+0000","last_undegraded":"2024-09-10T22:13:05.299026+0000","last_fullsized":"2024-09-10T22:13:05.299026+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:11:54.049523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.357957+0000","last_change":"2024-09-10T22:13:04.307455+0000","last_active":"2024-09-10T22:13:05.357957+0000","last_peered":"2024-09-10T22:13:05.357957+0000","last_clean":"2024-09-10T22:13:05.357957+0000","last_became_active":"2024-09-10T22:13:04.307204+0000","last_became_peered":"2024-09-10T22:13:04.307204+0000","last_unstale":"2024-09-10T22:13:05.357957+0000","last_undegraded":"2024-09-10T22:13:05.357957+0000","last_fullsized":"2024-09-10T22:13:05.357957+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:17.195771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354692+0000","last_change":"2024-09-10T22:13:03.300510+0000","last_active":"2024-09-10T22:13:05.354692+0000","last_peered":"2024-09-10T22:13:05.354692+0000","last_clean":"2024-09-10T22:13:05.354692+0000","last_became_active":"2024-09-10T22:13:03.300230+0000","last_became_peered":"2024-09-10T22:13:03.300230+0000","last_unstale":"2024-09-10T22:13:05.354692+0000","last_undegraded":"2024-09-10T22:13:05.354692+0000","last_fullsized":"2024-09-10T22:13:05.354692+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:28:49.363237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.305928+0000","last_change":"2024-09-10T22:13:04.301634+0000","last_active":"2024-09-10T22:13:05.305928+0000","last_peered":"2024-09-10T22:13:05.305928+0000","last_clean":"2024-09-10T22:13:05.305928+0000","last_became_active":"2024-09-10T22:13:04.301337+0000","last_became_peered":"2024-09-10T22:13:04.301337+0000","last_unstale":"2024-09-10T22:13:05.305928+0000","last_undegraded":"2024-09-10T22:13:05.305928+0000","last_fullsized":"2024-09-10T22:13:05.305928+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:12.453148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.357961+0000","last_change":"2024-09-10T22:13:03.302364+0000","last_active":"2024-09-10T22:13:05.357961+0000","last_peered":"2024-09-10T22:13:05.357961+0000","last_clean":"2024-09-10T22:13:05.357961+0000","last_became_active":"2024-09-10T22:13:03.302067+0000","last_became_peered":"2024-09-10T22:13:03.302067+0000","last_unstale":"2024-09-10T22:13:05.357961+0000","last_undegraded":"2024-09-10T22:13:05.357961+0000","last_fullsized":"2024-09-10T22:13:05.357961+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:49.568058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.353073+0000","last_change":"2024-09-10T22:13:04.314002+0000","last_active":"2024-09-10T22:13:05.353073+0000","last_peered":"2024-09-10T22:13:05.353073+0000","last_clean":"2024-09-10T22:13:05.353073+0000","last_became_active":"2024-09-10T22:13:04.313731+0000","last_became_peered":"2024-09-10T22:13:04.313731+0000","last_unstale":"2024-09-10T22:13:05.353073+0000","last_undegraded":"2024-09-10T22:13:05.353073+0000","last_fullsized":"2024-09-10T22:13:05.353073+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:43.198508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.352573+0000","last_change":"2024-09-10T22:13:03.301206+0000","last_active":"2024-09-10T22:13:05.352573+0000","last_peered":"2024-09-10T22:13:05.352573+0000","last_clean":"2024-09-10T22:13:05.352573+0000","last_became_active":"2024-09-10T22:13:03.300938+0000","last_became_peered":"2024-09-10T22:13:03.300938+0000","last_unstale":"2024-09-10T22:13:05.352573+0000","last_undegraded":"2024-09-10T22:13:05.352573+0000","last_fullsized":"2024-09-10T22:13:05.352573+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:47:35.471901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.354174+0000","last_change":"2024-09-10T22:13:04.290188+0000","last_active":"2024-09-10T22:13:05.354174+0000","last_peered":"2024-09-10T22:13:05.354174+0000","last_clean":"2024-09-10T22:13:05.354174+0000","last_became_active":"2024-09-10T22:13:04.289697+0000","last_became_peered":"2024-09-10T22:13:04.289697+0000","last_unstale":"2024-09-10T22:13:05.354174+0000","last_undegraded":"2024-09-10T22:13:05.354174+0000","last_fullsized":"2024-09-10T22:13:05.354174+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:43.351718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.298779+0000","last_change":"2024-09-10T22:13:03.284695+0000","last_active":"2024-09-10T22:13:05.298779+0000","last_peered":"2024-09-10T22:13:05.298779+0000","last_clean":"2024-09-10T22:13:05.298779+0000","last_became_active":"2024-09-10T22:13:03.284433+0000","last_became_peered":"2024-09-10T22:13:03.284433+0000","last_unstale":"2024-09-10T22:13:05.298779+0000","last_undegraded":"2024-09-10T22:13:05.298779+0000","last_fullsized":"2024-09-10T22:13:05.298779+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:27.231370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.357705+0000","last_change":"2024-09-10T22:13:04.311061+0000","last_active":"2024-09-10T22:13:05.357705+0000","last_peered":"2024-09-10T22:13:05.357705+0000","last_clean":"2024-09-10T22:13:05.357705+0000","last_became_active":"2024-09-10T22:13:04.310780+0000","last_became_peered":"2024-09-10T22:13:04.310780+0000","last_unstale":"2024-09-10T22:13:05.357705+0000","last_undegraded":"2024-09-10T22:13:05.357705+0000","last_fullsized":"2024-09-10T22:13:05.357705+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:03.270077+0000","last_clean_scrub_stamp":"2024-09-10T22:13:03.270077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:15.483212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"125'2","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.357987+0000","last_change":"2024-09-10T22:13:03.307598+0000","last_active":"2024-09-10T22:13:05.357987+0000","last_peered":"2024-09-10T22:13:05.357987+0000","last_clean":"2024-09-10T22:13:05.357987+0000","last_became_active":"2024-09-10T22:13:03.307306+0000","last_became_peered":"2024-09-10T22:13:03.307306+0000","last_unstale":"2024-09-10T22:13:05.357987+0000","last_undegraded":"2024-09-10T22:13:05.357987+0000","last_fullsized":"2024-09-10T22:13:05.357987+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:13:02.267589+0000","last_clean_scrub_stamp":"2024-09-10T22:13:02.267589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:55.701671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":311,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T22:13:05.357446+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:13:05.357446+0000","last_peered":"2024-09-10T22:13:05.357446+0000","last_clean":"2024-09-10T22:13:05.357446+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:13:05.357446+0000","last_undegraded":"2024-09-10T22:13:05.357446+0000","last_fullsized":"2024-09-10T22:13:05.357446+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509596,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131816,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240024,"statfs":{"total":96636764160,"available":96501784576,"internally_reserved":0,"allocated":2183168,"data_stored":1278110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509597,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122904,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248936,"statfs":{"total":96636764160,"available":96510910464,"internally_reserved":0,"allocated":1576960,"data_stored":687104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":124254662},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509596,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109016,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262824,"statfs":{"total":96636764160,"available":96525131776,"internally_reserved":0,"allocated":1572864,"data_stored":685604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509597,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153432,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218408,"statfs":{"total":96636764160,"available":96479649792,"internally_reserved":0,"allocated":1568768,"data_stored":685562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509595,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149276,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222564,"statfs":{"total":96636764160,"available":96483905536,"internally_reserved":0,"allocated":1572864,"data_stored":685652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509596,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118732,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253108,"statfs":{"total":96636764160,"available":96515182592,"internally_reserved":0,"allocated":1564672,"data_stored":685492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131800,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240040,"statfs":{"total":96636764160,"available":96501800960,"internally_reserved":0,"allocated":2158592,"data_stored":1275860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":132774345},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542301,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149276,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222564,"statfs":{"total":96636764160,"available":96483905536,"internally_reserved":0,"allocated":1572864,"data_stored":686108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:13:08.853 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:13:08.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T22:13:09.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.039+0000 7f663fa00640 1 -- 172.21.15.66:0/3813253260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66380c4fc0 msgr2=0x7f66380ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:09.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.039+0000 7f663fa00640 1 --2- 172.21.15.66:0/3813253260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66380c4fc0 0x7f66380ca8b0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f663000ad90 tx=0x7f663001dc20 comp rx=0 tx=0).stop 2024-09-10T22:13:09.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.039+0000 7f663fa00640 1 -- 172.21.15.66:0/3813253260 shutdown_connections 2024-09-10T22:13:09.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.039+0000 7f663fa00640 1 --2- 172.21.15.66:0/3813253260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66380c4fc0 0x7f66380ca8b0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.039+0000 7f663fa00640 1 -- 172.21.15.66:0/3813253260 >> 172.21.15.66:0/3813253260 conn(0x7f66380cfa30 msgr2=0x7f66380cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:09.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.039+0000 7f663fa00640 1 -- 172.21.15.66:0/3813253260 shutdown_connections 2024-09-10T22:13:09.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.039+0000 7f663fa00640 1 -- 172.21.15.66:0/3813253260 wait complete. 2024-09-10T22:13:09.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663fa00640 1 Processor -- start 2024-09-10T22:13:09.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663fa00640 1 -- start start 2024-09-10T22:13:09.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663fa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66380c4fc0 0x7f66381505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:09.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663fa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f663815a310 0x7f6638153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:09.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663fa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f663814cf70 0x7f66380af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:09.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663fa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f66381686a0 con 0x7f66380c4fc0 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663fa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6638168520 con 0x7f663815a310 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663fa00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6638168820 con 0x7f663814cf70 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66380c4fc0 0x7f66381505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66380c4fc0 0x7f66381505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40126/0 (socket says 172.21.15.66:40126) 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663d000640 1 -- 172.21.15.66:0/1755812205 learned_addr learned my addr 172.21.15.66:0/1755812205 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663da00640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f663814cf70 0x7f66380af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f662fe00640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f663815a310 0x7f6638153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663d000640 1 -- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f663814cf70 msgr2=0x7f66380af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663d000640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f663814cf70 0x7f66380af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663d000640 1 -- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f663815a310 msgr2=0x7f6638153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663d000640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f663815a310 0x7f6638153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663d000640 1 -- 172.21.15.66:0/1755812205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663814d4e0 con 0x7f66380c4fc0 2024-09-10T22:13:09.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663d000640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66380c4fc0 0x7f66381505c0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f662800c3c0 tx=0x7f662800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:09.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f6635600640 1 -- 172.21.15.66:0/1755812205 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628017070 con 0x7f66380c4fc0 2024-09-10T22:13:09.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f6635600640 1 -- 172.21.15.66:0/1755812205 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66280053c0 con 0x7f66380c4fc0 2024-09-10T22:13:09.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f6635600640 1 -- 172.21.15.66:0/1755812205 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628003860 con 0x7f66380c4fc0 2024-09-10T22:13:09.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663fa00640 1 -- 172.21.15.66:0/1755812205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663814d7c0 con 0x7f66380c4fc0 2024-09-10T22:13:09.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663fa00640 1 -- 172.21.15.66:0/1755812205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6638155b60 con 0x7f66380c4fc0 2024-09-10T22:13:09.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.043+0000 7f663fa00640 1 -- 172.21.15.66:0/1755812205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66380c48b0 con 0x7f66380c4fc0 2024-09-10T22:13:09.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.047+0000 7f6635600640 1 -- 172.21.15.66:0/1755812205 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f6628003a00 con 0x7f66380c4fc0 2024-09-10T22:13:09.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.047+0000 7f6635600640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f66140743b0 0x7f6614076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:09.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.047+0000 7f6635600640 1 -- 172.21.15.66:0/1755812205 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f6628091830 con 0x7f66380c4fc0 2024-09-10T22:13:09.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.047+0000 7f662fe00640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f66140743b0 0x7f6614076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:09.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.047+0000 7f662fe00640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f66140743b0 0x7f6614076870 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f66381fb860 tx=0x7f6620009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:09.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.051+0000 7f6635600640 1 -- 172.21.15.66:0/1755812205 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f662805e320 con 0x7f66380c4fc0 2024-09-10T22:13:09.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.287+0000 7f663fa00640 1 -- 172.21.15.66:0/1755812205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f66380b58c0 con 0x7f66380c4fc0 2024-09-10T22:13:09.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.287+0000 7f6635600640 1 -- 172.21.15.66:0/1755812205 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v34) ==== 190+0+147 (secure 0 0 0) 0x7f6628067090 con 0x7f66380c4fc0 2024-09-10T22:13:09.293 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T22:13:09.293 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw== 2024-09-10T22:13:09.293 INFO:teuthology.orchestra.run.smithi066.stdout: caps mds = "allow" 2024-09-10T22:13:09.293 INFO:teuthology.orchestra.run.smithi066.stdout: caps mon = "allow r" 2024-09-10T22:13:09.293 INFO:teuthology.orchestra.run.smithi066.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T22:13:09.294 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-09-10T22:13:09.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.291+0000 7f663fa00640 1 -- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f66140743b0 msgr2=0x7f6614076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:09.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.291+0000 7f663fa00640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f66140743b0 0x7f6614076870 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f66381fb860 tx=0x7f6620009040 comp rx=0 tx=0).stop 2024-09-10T22:13:09.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.291+0000 7f663fa00640 1 -- 172.21.15.66:0/1755812205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66380c4fc0 msgr2=0x7f66381505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:09.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.291+0000 7f663fa00640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66380c4fc0 0x7f66381505c0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f662800c3c0 tx=0x7f662800c880 comp rx=0 tx=0).stop 2024-09-10T22:13:09.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.291+0000 7f663fa00640 1 -- 172.21.15.66:0/1755812205 shutdown_connections 2024-09-10T22:13:09.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.291+0000 7f663fa00640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f66140743b0 0x7f6614076870 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.291+0000 7f663fa00640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f663814cf70 0x7f66380af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.291+0000 7f663fa00640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f663815a310 0x7f6638153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.291+0000 7f663fa00640 1 --2- 172.21.15.66:0/1755812205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66380c4fc0 0x7f66381505c0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.291+0000 7f663fa00640 1 -- 172.21.15.66:0/1755812205 >> 172.21.15.66:0/1755812205 conn(0x7f66380cfa30 msgr2=0x7f6638073d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.291+0000 7f663fa00640 1 -- 172.21.15.66:0/1755812205 shutdown_connections 2024-09-10T22:13:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.291+0000 7f663fa00640 1 -- 172.21.15.66:0/1755812205 wait complete. 2024-09-10T22:13:09.315 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:13:09.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.455+0000 7f55f0400640 1 -- 172.21.15.66:0/2181581246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e80d0260 msgr2=0x7f55e80ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:09.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.455+0000 7f55f0400640 1 --2- 172.21.15.66:0/2181581246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e80d0260 0x7f55e80ae960 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009840 tx=0x7f55d401c820 comp rx=0 tx=0).stop 2024-09-10T22:13:09.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.455+0000 7f55f0400640 1 -- 172.21.15.66:0/2181581246 shutdown_connections 2024-09-10T22:13:09.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.455+0000 7f55f0400640 1 --2- 172.21.15.66:0/2181581246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e80d0260 0x7f55e80ae960 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.455+0000 7f55f0400640 1 --2- 172.21.15.66:0/2181581246 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55e80b2bf0 0x7f55e80aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.455+0000 7f55f0400640 1 -- 172.21.15.66:0/2181581246 >> 172.21.15.66:0/2181581246 conn(0x7f55e80d1c50 msgr2=0x7f55e80d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:09.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.455+0000 7f55f0400640 1 -- 172.21.15.66:0/2181581246 shutdown_connections 2024-09-10T22:13:09.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.455+0000 7f55f0400640 1 -- 172.21.15.66:0/2181581246 wait complete. 2024-09-10T22:13:09.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55f0400640 1 Processor -- start 2024-09-10T22:13:09.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55f0400640 1 -- start start 2024-09-10T22:13:09.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55f0400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55e80b2bf0 0x7f55e80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:09.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55f0400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e80d0260 0x7f55e80cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:09.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55f0400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55e815f340 0x7f55e80cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:09.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55f0400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55e81692a0 con 0x7f55e80d0260 2024-09-10T22:13:09.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55f0400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55e8169120 con 0x7f55e80b2bf0 2024-09-10T22:13:09.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55f0400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f55e8169420 con 0x7f55e815f340 2024-09-10T22:13:09.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55ed000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e80d0260 0x7f55e80cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:09.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55ed000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e80d0260 0x7f55e80cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40136/0 (socket says 172.21.15.66:40136) 2024-09-10T22:13:09.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55ed000640 1 -- 172.21.15.66:0/25031235 learned_addr learned my addr 172.21.15.66:0/25031235 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:09.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55eda00640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55e80b2bf0 0x7f55e80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:09.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55ee400640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55e815f340 0x7f55e80cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55ed000640 1 -- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55e815f340 msgr2=0x7f55e80cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55ed000640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55e815f340 0x7f55e80cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55ed000640 1 -- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55e80b2bf0 msgr2=0x7f55e80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55ed000640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55e80b2bf0 0x7f55e80ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55ed000640 1 -- 172.21.15.66:0/25031235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e80c91c0 con 0x7f55e80d0260 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55ee400640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55e815f340 0x7f55e80cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55eda00640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55e80b2bf0 0x7f55e80ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55ed000640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e80d0260 0x7f55e80cb8e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009790 tx=0x7f55d40a5910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55e5600640 1 -- 172.21.15.66:0/25031235 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d40b2040 con 0x7f55e80d0260 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55e5600640 1 -- 172.21.15.66:0/25031235 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55d40a5df0 con 0x7f55e80d0260 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55e5600640 1 -- 172.21.15.66:0/25031235 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4005280 con 0x7f55e80d0260 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55f0400640 1 -- 172.21.15.66:0/25031235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e80b4cb0 con 0x7f55e80d0260 2024-09-10T22:13:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.459+0000 7f55f0400640 1 -- 172.21.15.66:0/25031235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e80b5270 con 0x7f55e80d0260 2024-09-10T22:13:09.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.463+0000 7f55f0400640 1 -- 172.21.15.66:0/25031235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e80cfb50 con 0x7f55e80d0260 2024-09-10T22:13:09.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.463+0000 7f55e5600640 1 -- 172.21.15.66:0/25031235 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f55d40b5070 con 0x7f55e80d0260 2024-09-10T22:13:09.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.463+0000 7f55e5600640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55c80743b0 0x7f55c8076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:09.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.463+0000 7f55e5600640 1 -- 172.21.15.66:0/25031235 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f55d412bb10 con 0x7f55e80d0260 2024-09-10T22:13:09.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.463+0000 7f55eda00640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55c80743b0 0x7f55c8076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:09.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.463+0000 7f55eda00640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55c80743b0 0x7f55c8076870 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f55dc000f70 tx=0x7f55dc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:09.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.467+0000 7f55e5600640 1 -- 172.21.15.66:0/25031235 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f55d4016410 con 0x7f55e80d0260 2024-09-10T22:13:09.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.675+0000 7f55f0400640 1 -- 172.21.15.66:0/25031235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55e8072320 con 0x7f55e80d0260 2024-09-10T22:13:09.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.675+0000 7f55e5600640 1 -- 172.21.15.66:0/25031235 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5222 (secure 0 0 0) 0x7f55d40f8c50 con 0x7f55e80d0260 2024-09-10T22:13:09.680 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:09.681 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":86,"btime":"2024-09-10T22:13:07:305043+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1352582369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1352582369},{"type":"v1","addr":"172.21.15.94:6837","nonce":1352582369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8805,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/441263936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":441263936},{"type":"v1","addr":"172.21.15.66:6834","nonce":441263936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2554522797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2554522797},{"type":"v1","addr":"172.21.15.66:6835","nonce":2554522797}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:13:03.687866+0000","modified":"2024-09-10T22:13:06.708613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8811":{"gid":8811,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6836/1762705824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1762705824},{"type":"v1","addr":"172.21.15.94:6836","nonce":1762705824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8811,"qdb_cluster":[8811]},"id":9}]} 2024-09-10T22:13:09.681 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 86 2024-09-10T22:13:09.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.679+0000 7f55f0400640 1 -- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55c80743b0 msgr2=0x7f55c8076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:09.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.679+0000 7f55f0400640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55c80743b0 0x7f55c8076870 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f55dc000f70 tx=0x7f55dc009040 comp rx=0 tx=0).stop 2024-09-10T22:13:09.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.679+0000 7f55f0400640 1 -- 172.21.15.66:0/25031235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e80d0260 msgr2=0x7f55e80cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:09.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.679+0000 7f55f0400640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e80d0260 0x7f55e80cb8e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009790 tx=0x7f55d40a5910 comp rx=0 tx=0).stop 2024-09-10T22:13:09.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.679+0000 7f55f0400640 1 -- 172.21.15.66:0/25031235 shutdown_connections 2024-09-10T22:13:09.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.679+0000 7f55f0400640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f55c80743b0 0x7f55c8076870 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.679+0000 7f55f0400640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55e815f340 0x7f55e80cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.679+0000 7f55f0400640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e80d0260 0x7f55e80cb8e0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.679+0000 7f55f0400640 1 --2- 172.21.15.66:0/25031235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55e80b2bf0 0x7f55e80ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:09.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.679+0000 7f55f0400640 1 -- 172.21.15.66:0/25031235 >> 172.21.15.66:0/25031235 conn(0x7f55e80d1c50 msgr2=0x7f55e80aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:09.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.679+0000 7f55f0400640 1 -- 172.21.15.66:0/25031235 shutdown_connections 2024-09-10T22:13:09.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:09.679+0000 7f55f0400640 1 -- 172.21.15.66:0/25031235 wait complete. 2024-09-10T22:13:10.104 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:13:03.687866+0000', 'modified': '2024-09-10T22:13:06.708613+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8811}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8811': {'gid': 8811, 'name': 'b', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6836/1762705824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6834', 'nonce': 1762705824}, {'type': 'v1', 'addr': '172.21.15.94:6836', 'nonce': 1762705824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8811, 'qdb_cluster': [8811]} 2024-09-10T22:13:10.104 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:13:10.104 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T22:13:10.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.251+0000 7f9a63000640 1 -- 172.21.15.66:0/1506745487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a5c0ae620 msgr2=0x7f9a5c0725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.251+0000 7f9a63000640 1 --2- 172.21.15.66:0/1506745487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a5c0ae620 0x7f9a5c0725d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009900 tx=0x7f9a4401c880 comp rx=0 tx=0).stop 2024-09-10T22:13:10.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.251+0000 7f9a63000640 1 -- 172.21.15.66:0/1506745487 shutdown_connections 2024-09-10T22:13:10.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.251+0000 7f9a63000640 1 --2- 172.21.15.66:0/1506745487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a5c0ae620 0x7f9a5c0725d0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.251+0000 7f9a63000640 1 --2- 172.21.15.66:0/1506745487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a5c0c4b10 0x7f9a5c06e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.251+0000 7f9a63000640 1 -- 172.21.15.66:0/1506745487 >> 172.21.15.66:0/1506745487 conn(0x7f9a5c0cfa30 msgr2=0x7f9a5c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:10.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.251+0000 7f9a63000640 1 -- 172.21.15.66:0/1506745487 shutdown_connections 2024-09-10T22:13:10.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.251+0000 7f9a63000640 1 -- 172.21.15.66:0/1506745487 wait complete. 2024-09-10T22:13:10.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.251+0000 7f9a63000640 1 Processor -- start 2024-09-10T22:13:10.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.251+0000 7f9a63000640 1 -- start start 2024-09-10T22:13:10.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a63000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a5c0ae620 0x7f9a5c14eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a63000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a5c0c4b10 0x7f9a5c1fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a63000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a5c0c4ef0 0x7f9a5c0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a63000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c168440 con 0x7f9a5c0c4b10 2024-09-10T22:13:10.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a63000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c1682c0 con 0x7f9a5c0c4ef0 2024-09-10T22:13:10.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a63000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5c1685c0 con 0x7f9a5c0ae620 2024-09-10T22:13:10.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a53400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a5c0c4b10 0x7f9a5c1fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a53400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a5c0c4b10 0x7f9a5c1fbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40156/0 (socket says 172.21.15.66:40156) 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a53400640 1 -- 172.21.15.66:0/1415914586 learned_addr learned my addr 172.21.15.66:0/1415914586 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a5be00640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a5c0ae620 0x7f9a5c14eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a61000640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a5c0c4ef0 0x7f9a5c0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a53400640 1 -- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a5c0ae620 msgr2=0x7f9a5c14eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a53400640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a5c0ae620 0x7f9a5c14eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a53400640 1 -- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a5c0c4ef0 msgr2=0x7f9a5c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a53400640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a5c0c4ef0 0x7f9a5c0ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a53400640 1 -- 172.21.15.66:0/1415914586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5c154a30 con 0x7f9a5c0c4b10 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a61000640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a5c0c4ef0 0x7f9a5c0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a53400640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a5c0c4b10 0x7f9a5c1fbae0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009850 tx=0x7f9a44005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a58c00640 1 -- 172.21.15.66:0/1415914586 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440bb070 con 0x7f9a5c0c4b10 2024-09-10T22:13:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a58c00640 1 -- 172.21.15.66:0/1415914586 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a440043f0 con 0x7f9a5c0c4b10 2024-09-10T22:13:10.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a63000640 1 -- 172.21.15.66:0/1415914586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5c154d10 con 0x7f9a5c0c4b10 2024-09-10T22:13:10.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a58c00640 1 -- 172.21.15.66:0/1415914586 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440b2040 con 0x7f9a5c0c4b10 2024-09-10T22:13:10.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.255+0000 7f9a63000640 1 -- 172.21.15.66:0/1415914586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5c1fb5c0 con 0x7f9a5c0c4b10 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a58c00640 1 -- 172.21.15.66:0/1415914586 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f9a44005290 con 0x7f9a5c0c4b10 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a58c00640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a380743b0 0x7f9a38076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a58c00640 1 -- 172.21.15.66:0/1415914586 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f9a4412b950 con 0x7f9a5c0c4b10 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a663db000 1 Processor -- start 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a663db000 1 -- start start 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a5be00640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a380743b0 0x7f9a38076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a663db000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d69d0 0x55b98d5d6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a663db000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 0x55b98d58a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a663db000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d58ae10 0x55b98d58b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a663db000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b98d4ab3f0 con 0x55b98d58ae10 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a663db000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b98d3e8b20 con 0x55b98d5d69d0 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a663db000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55b98d3d7f50 con 0x55b98d5d72f0 2024-09-10T22:13:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a5be00640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a380743b0 0x7f9a38076870 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009be0 tx=0x7f9a4c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a61000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d58ae10 0x55b98d58b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a5be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d69d0 0x55b98d5d6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a5be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d69d0 0x55b98d5d6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:40860/0 (socket says 172.21.15.66:40860) 2024-09-10T22:13:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a5be00640 1 -- 172.21.15.66:0/3615024946 learned_addr learned my addr 172.21.15.66:0/3615024946 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a53400640 1 --2- 172.21.15.66:0/3615024946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 0x55b98d58a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a61000640 1 -- 172.21.15.66:0/3615024946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 msgr2=0x55b98d58a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a61000640 1 --2- 172.21.15.66:0/3615024946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 0x55b98d58a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a61000640 1 -- 172.21.15.66:0/3615024946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d69d0 msgr2=0x55b98d5d6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a61000640 1 --2- 172.21.15.66:0/3615024946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d69d0 0x55b98d5d6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a61000640 1 -- 172.21.15.66:0/3615024946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98d39f590 con 0x55b98d58ae10 2024-09-10T22:13:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a5be00640 1 --2- 172.21.15.66:0/3615024946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d69d0 0x55b98d5d6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:13:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a61000640 1 --2- 172.21.15.66:0/3615024946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d58ae10 0x55b98d58b1f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9a54001c30 tx=0x7f9a540020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9722aca16e5ef1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a50c00640 1 -- 172.21.15.66:0/3615024946 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54019070 con 0x55b98d58ae10 2024-09-10T22:13:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a50c00640 1 -- 172.21.15.66:0/3615024946 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a54003c80 con 0x55b98d58ae10 2024-09-10T22:13:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.259+0000 7f9a50c00640 1 -- 172.21.15.66:0/3615024946 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5400eac0 con 0x55b98d58ae10 2024-09-10T22:13:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 -- 172.21.15.66:0/3615024946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d58ae10 msgr2=0x55b98d58b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 --2- 172.21.15.66:0/3615024946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d58ae10 0x55b98d58b1f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9a54001c30 tx=0x7f9a540020f0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 -- 172.21.15.66:0/3615024946 shutdown_connections 2024-09-10T22:13:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 --2- 172.21.15.66:0/3615024946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d58ae10 0x55b98d58b1f0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 --2- 172.21.15.66:0/3615024946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 0x55b98d58a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 --2- 172.21.15.66:0/3615024946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d69d0 0x55b98d5d6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 -- 172.21.15.66:0/3615024946 >> 172.21.15.66:0/3615024946 conn(0x55b98d474130 msgr2=0x55b98d5b9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:10.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 -- 172.21.15.66:0/3615024946 shutdown_connections 2024-09-10T22:13:10.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 -- 172.21.15.66:0/3615024946 wait complete. 2024-09-10T22:13:10.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 -- start start 2024-09-10T22:13:10.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 Processor -- start 2024-09-10T22:13:10.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d58ae10 0x55b98d5b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d5b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 0x55b98d5b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b98d3725d0 con 0x55b98d5d69d0 2024-09-10T22:13:10.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b98d3dbd60 con 0x55b98d58ae10 2024-09-10T22:13:10.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55b98d37e500 con 0x55b98d5d72f0 2024-09-10T22:13:10.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 10 client.0 _ll_register_callbacks cb 0x55b98d382810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:13:10.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a53400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d5b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a53400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d5b7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40178/0 (socket says 172.21.15.66:40178) 2024-09-10T22:13:10.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a53400640 1 -- 172.21.15.66:0/1666823853 learned_addr learned my addr 172.21.15.66:0/1666823853 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:10.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a5be00640 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d58ae10 0x55b98d5b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a61000640 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 0x55b98d5b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a53400640 1 -- 172.21.15.66:0/1666823853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 msgr2=0x55b98d5b83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:10.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a53400640 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 0x55b98d5b83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a53400640 1 -- 172.21.15.66:0/1666823853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d58ae10 msgr2=0x55b98d5b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:10.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a53400640 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d58ae10 0x55b98d5b76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a53400640 1 -- 172.21.15.66:0/1666823853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98d376170 con 0x55b98d5d69d0 2024-09-10T22:13:10.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a53400640 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d5b7e70 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9a4401cd60 tx=0x7f9a440f7ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f44a26028f3d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:10.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a49600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a49600640 1 -- 172.21.15.66:0/1666823853 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44101600 con 0x55b98d5d69d0 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 -- 172.21.15.66:0/1666823853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b98d3700f0 con 0x55b98d5d69d0 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a49600640 1 -- 172.21.15.66:0/1666823853 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f9a4411d1b0 con 0x55b98d5d69d0 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a49600640 1 -- 172.21.15.66:0/1666823853 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9a440f8440 con 0x55b98d5d69d0 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 10 client.8943 fetch_fsmap learned FSMap version 86 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a663db000 1 -- 172.21.15.66:0/1666823853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x55b98d45cef0 con 0x55b98d5d69d0 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.263+0000 7f9a49600640 1 -- 172.21.15.66:0/1666823853 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b98d3700f0 con 0x55b98d5d69d0 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a49600640 1 -- 172.21.15.66:0/1666823853 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7f9a440ff0c0 con 0x55b98d5d69d0 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 10 client.8943 fetch_fsmap finished waiting for FSMap version 86 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 10 client.8943 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 20 client.8943 populate_metadata read hostname 'smithi066' 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.94:6834/1762705824,v1:172.21.15.94:6836/1762705824] conn(0x55b98d5b90f0 0x55b98d5e8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 4 client.8943 mds_command: new command op to 8811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 -- 172.21.15.66:0/1666823853 --> [v2:172.21.15.94:6834/1762705824,v1:172.21.15.94:6836/1762705824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b98d3eaa70 con 0x55b98d5b90f0 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a5be00640 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.94:6834/1762705824,v1:172.21.15.94:6836/1762705824] conn(0x55b98d5b90f0 0x55b98d5e8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a5be00640 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.94:6834/1762705824,v1:172.21.15.94:6836/1762705824] conn(0x55b98d5b90f0 0x55b98d5e8c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0844dd29a14eec8 server_cookie=bf35f643e2a0ccfb in_seq=0 out_seq=0 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a49600640 10 client.8943 ms_handle_connect on v2:172.21.15.94:6834/1762705824 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a49600640 1 -- 172.21.15.66:0/1666823853 <== mds.0 v2:172.21.15.94:6834/1762705824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b98d3eaa70 con 0x55b98d5b90f0 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a49600640 10 client.8943 handle_command_reply: tid=0 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 client.8943 shutdown 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 -- 172.21.15.66:0/1666823853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 msgr2=0x55b98d5b7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d5b7e70 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9a4401cd60 tx=0x7f9a440f7ec0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 -- 172.21.15.66:0/1666823853 shutdown_connections 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.94:6834/1762705824,v1:172.21.15.94:6836/1762705824] conn(0x55b98d5b90f0 0x55b98d5e8c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 0x55b98d5b83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d5b7e70 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a49600640 0 client.8943 ms_handle_reset on v2:172.21.15.94:6834/1762705824 2024-09-10T22:13:10.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 --2- 172.21.15.66:0/1666823853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d58ae10 0x55b98d5b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 -- 172.21.15.66:0/1666823853 >> 172.21.15.66:0/1666823853 conn(0x55b98d474130 msgr2=0x55b98d589fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:10.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 -- 172.21.15.66:0/1666823853 shutdown_connections 2024-09-10T22:13:10.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 1 -- 172.21.15.66:0/1666823853 wait complete. 2024-09-10T22:13:10.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.267+0000 7f9a663db000 20 client.8943 trim_cache size 0 max 16384 2024-09-10T22:13:10.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a663db000 1 Processor -- start 2024-09-10T22:13:10.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a663db000 1 -- start start 2024-09-10T22:13:10.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a663db000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d58ae10 0x55b98d648dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a663db000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d649310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a663db000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 0x55b98d649850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a663db000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c2047f0 con 0x55b98d5d69d0 2024-09-10T22:13:10.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a663db000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c203870 con 0x55b98d58ae10 2024-09-10T22:13:10.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a663db000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5c15a8a0 con 0x55b98d5d72f0 2024-09-10T22:13:10.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a53400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d649310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a53400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d649310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40190/0 (socket says 172.21.15.66:40190) 2024-09-10T22:13:10.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a53400640 1 -- 172.21.15.66:0/427766824 learned_addr learned my addr 172.21.15.66:0/427766824 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:10.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a5be00640 1 --2- 172.21.15.66:0/427766824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d58ae10 0x55b98d648dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a61000640 1 --2- 172.21.15.66:0/427766824 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 0x55b98d649850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a53400640 1 -- 172.21.15.66:0/427766824 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 msgr2=0x55b98d649850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:10.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a53400640 1 --2- 172.21.15.66:0/427766824 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 0x55b98d649850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a53400640 1 -- 172.21.15.66:0/427766824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d58ae10 msgr2=0x55b98d648dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:10.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a53400640 1 --2- 172.21.15.66:0/427766824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d58ae10 0x55b98d648dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a53400640 1 -- 172.21.15.66:0/427766824 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98d433d90 con 0x55b98d5d69d0 2024-09-10T22:13:10.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a53400640 1 --2- 172.21.15.66:0/427766824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d649310 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9a44098260 tx=0x7f9a440a5400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6948b1bea1fd27f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:10.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a4be00640 1 -- 172.21.15.66:0/427766824 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44112020 con 0x55b98d5d69d0 2024-09-10T22:13:10.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a4be00640 1 -- 172.21.15.66:0/427766824 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a44101890 con 0x55b98d5d69d0 2024-09-10T22:13:10.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.287+0000 7f9a4be00640 1 -- 172.21.15.66:0/427766824 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44101660 con 0x55b98d5d69d0 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 -- 172.21.15.66:0/427766824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 msgr2=0x55b98d649310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 --2- 172.21.15.66:0/427766824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d649310 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9a44098260 tx=0x7f9a440a5400 comp rx=0 tx=0).stop 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 -- 172.21.15.66:0/427766824 shutdown_connections 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 --2- 172.21.15.66:0/427766824 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d5d72f0 0x55b98d649850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 --2- 172.21.15.66:0/427766824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d649310 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 --2- 172.21.15.66:0/427766824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d58ae10 0x55b98d648dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 -- 172.21.15.66:0/427766824 >> 172.21.15.66:0/427766824 conn(0x55b98d474130 msgr2=0x55b98d592a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 -- 172.21.15.66:0/427766824 shutdown_connections 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 -- 172.21.15.66:0/427766824 wait complete. 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 -- start start 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 Processor -- start 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d58ae10 0x55b98d64d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d64d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d72f0 0x55b98d64dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c150640 con 0x55b98d5d69d0 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b98d3dbd60 con 0x55b98d5d72f0 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55b98d5edce0 con 0x55b98d58ae10 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 10 client.0 _ll_register_callbacks cb 0x7f9a440f4a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a53400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d64d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a53400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d64d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40206/0 (socket says 172.21.15.66:40206) 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a53400640 1 -- 172.21.15.66:0/155516854 learned_addr learned my addr 172.21.15.66:0/155516854 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a61000640 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d72f0 0x55b98d64dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a5be00640 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d58ae10 0x55b98d64d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a53400640 1 -- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d58ae10 msgr2=0x55b98d64d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a53400640 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d58ae10 0x55b98d64d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a53400640 1 -- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d72f0 msgr2=0x55b98d64dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a53400640 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d72f0 0x55b98d64dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a53400640 1 -- 172.21.15.66:0/155516854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98d39ea90 con 0x55b98d5d69d0 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a61000640 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d72f0 0x55b98d64dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a53400640 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d64d770 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f9a440f56b0 tx=0x7f9a4411d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483c1a8022f079b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a49600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a49600640 1 -- 172.21.15.66:0/155516854 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44112020 con 0x55b98d5d69d0 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a49600640 1 -- 172.21.15.66:0/155516854 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f9a440fade0 con 0x55b98d5d69d0 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 -- 172.21.15.66:0/155516854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a44101890 con 0x55b98d5d69d0 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a49600640 1 -- 172.21.15.66:0/155516854 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9a440f3d90 con 0x55b98d5d69d0 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 10 client.8949 fetch_fsmap learned FSMap version 86 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 -- 172.21.15.66:0/155516854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x55b98d37d050 con 0x55b98d5d69d0 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a49600640 1 -- 172.21.15.66:0/155516854 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a44101890 con 0x55b98d5d69d0 2024-09-10T22:13:10.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a49600640 1 -- 172.21.15.66:0/155516854 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7f9a440f3f30 con 0x55b98d5d69d0 2024-09-10T22:13:10.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 10 client.8949 fetch_fsmap finished waiting for FSMap version 86 2024-09-10T22:13:10.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 10 client.8949 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:13:10.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 20 client.8949 populate_metadata read hostname 'smithi066' 2024-09-10T22:13:10.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:6834/1762705824,v1:172.21.15.94:6836/1762705824] conn(0x55b98d5eb450 0x55b98d5eb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.291+0000 7f9a663db000 4 client.8949 mds_command: new command op to 8811 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:13:10.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 1 -- 172.21.15.66:0/155516854 --> [v2:172.21.15.94:6834/1762705824,v1:172.21.15.94:6836/1762705824] -- command(tid 0: {"prefix": "status"}) -- 0x55b98d3e6520 con 0x55b98d5eb450 2024-09-10T22:13:10.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a5be00640 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:6834/1762705824,v1:172.21.15.94:6836/1762705824] conn(0x55b98d5eb450 0x55b98d5eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a5be00640 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:6834/1762705824,v1:172.21.15.94:6836/1762705824] conn(0x55b98d5eb450 0x55b98d5eb830 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75dfb7b80db00457 server_cookie=d61b7829f271a640 in_seq=0 out_seq=0 2024-09-10T22:13:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a49600640 10 client.8949 ms_handle_connect on v2:172.21.15.94:6834/1762705824 2024-09-10T22:13:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a49600640 1 -- 172.21.15.66:0/155516854 <== mds.0 v2:172.21.15.94:6834/1762705824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55b98d3e6520 con 0x55b98d5eb450 2024-09-10T22:13:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a49600640 10 client.8949 handle_command_reply: tid=0 2024-09-10T22:13:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 1 client.8949 shutdown 2024-09-10T22:13:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 1 -- 172.21.15.66:0/155516854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 msgr2=0x55b98d64d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d64d770 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f9a440f56b0 tx=0x7f9a4411d3e0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 1 -- 172.21.15.66:0/155516854 shutdown_connections 2024-09-10T22:13:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:6834/1762705824,v1:172.21.15.94:6836/1762705824] conn(0x55b98d5eb450 0x55b98d5eb830 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b98d5d72f0 0x55b98d64dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b98d5d69d0 0x55b98d64d770 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 1 --2- 172.21.15.66:0/155516854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b98d58ae10 0x55b98d64d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a49600640 0 client.8949 ms_handle_reset on v2:172.21.15.94:6834/1762705824 2024-09-10T22:13:10.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 1 -- 172.21.15.66:0/155516854 >> 172.21.15.66:0/155516854 conn(0x55b98d474130 msgr2=0x55b98d5d55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:10.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 1 -- 172.21.15.66:0/155516854 shutdown_connections 2024-09-10T22:13:10.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 1 -- 172.21.15.66:0/155516854 wait complete. 2024-09-10T22:13:10.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a663db000 20 client.8949 trim_cache size 0 max 16384 2024-09-10T22:13:10.305 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:13:10.305 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:13:10.305 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:13:10.305 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 8811, 2024-09-10T22:13:10.305 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:13:10.306 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:13:10.306 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:13:10.306 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 6.0195532629999997, 2024-09-10T22:13:10.306 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 86, 2024-09-10T22:13:10.306 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 126, 2024-09-10T22:13:10.306 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 126, 2024-09-10T22:13:10.306 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 10.465034774999999 2024-09-10T22:13:10.306 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:13:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a63000640 1 -- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a380743b0 msgr2=0x7f9a38076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a63000640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a380743b0 0x7f9a38076870 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009be0 tx=0x7f9a4c0091e0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a63000640 1 -- 172.21.15.66:0/1415914586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a5c0c4b10 msgr2=0x7f9a5c1fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a63000640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a5c0c4b10 0x7f9a5c1fbae0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009850 tx=0x7f9a44005e70 comp rx=0 tx=0).stop 2024-09-10T22:13:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a63000640 1 -- 172.21.15.66:0/1415914586 shutdown_connections 2024-09-10T22:13:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a63000640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a380743b0 0x7f9a38076870 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a63000640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a5c0c4ef0 0x7f9a5c0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a63000640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a5c0c4b10 0x7f9a5c1fbae0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a63000640 1 --2- 172.21.15.66:0/1415914586 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a5c0ae620 0x7f9a5c14eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a63000640 1 -- 172.21.15.66:0/1415914586 >> 172.21.15.66:0/1415914586 conn(0x7f9a5c0cfa30 msgr2=0x7f9a5c0afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a63000640 1 -- 172.21.15.66:0/1415914586 shutdown_connections 2024-09-10T22:13:10.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.295+0000 7f9a63000640 1 -- 172.21.15.66:0/1415914586 wait complete. 2024-09-10T22:13:10.320 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:13:10.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.463+0000 7f1266400640 1 -- 172.21.15.66:0/2411456516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1244003a60 msgr2=0x7f1244003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.463+0000 7f1266400640 1 --2- 172.21.15.66:0/2411456516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1244003a60 0x7f1244003e80 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f12500016c0 tx=0x7f125001c750 comp rx=0 tx=0).stop 2024-09-10T22:13:10.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.463+0000 7f1266400640 1 -- 172.21.15.66:0/2411456516 shutdown_connections 2024-09-10T22:13:10.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.463+0000 7f1266400640 1 --2- 172.21.15.66:0/2411456516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1244003a60 0x7f1244003e80 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.463+0000 7f1266400640 1 -- 172.21.15.66:0/2411456516 >> 172.21.15.66:0/2411456516 conn(0x7f12600cfa30 msgr2=0x7f12600cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:10.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.463+0000 7f1266400640 1 -- 172.21.15.66:0/2411456516 shutdown_connections 2024-09-10T22:13:10.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.463+0000 7f1266400640 1 -- 172.21.15.66:0/2411456516 wait complete. 2024-09-10T22:13:10.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.463+0000 7f1266400640 1 Processor -- start 2024-09-10T22:13:10.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.463+0000 7f1266400640 1 -- start start 2024-09-10T22:13:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.463+0000 7f1266400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1244003a60 0x7f1260152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1266400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1260153850 0x7f12600c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1266400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12600c6640 0x7f12600ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1266400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12601603e0 con 0x7f1260153850 2024-09-10T22:13:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1266400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1260160260 con 0x7f12600c6640 2024-09-10T22:13:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1266400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1260160560 con 0x7f1244003a60 2024-09-10T22:13:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1256a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1260153850 0x7f12600c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f125f400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1244003a60 0x7f1260152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1256a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1260153850 0x7f12600c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40228/0 (socket says 172.21.15.66:40228) 2024-09-10T22:13:10.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1256a00640 1 -- 172.21.15.66:0/328603206 learned_addr learned my addr 172.21.15.66:0/328603206 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:10.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f125fe00640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12600c6640 0x7f12600ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1256a00640 1 -- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1244003a60 msgr2=0x7f1260152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1256a00640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1244003a60 0x7f1260152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1256a00640 1 -- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12600c6640 msgr2=0x7f12600ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1256a00640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12600c6640 0x7f12600ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1256a00640 1 -- 172.21.15.66:0/328603206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1260153100 con 0x7f1260153850 2024-09-10T22:13:10.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1256a00640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1260153850 0x7f12600c6100 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1248011ae0 tx=0x7f1248009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:10.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1257e00640 1 -- 172.21.15.66:0/328603206 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124800fcd0 con 0x7f1260153850 2024-09-10T22:13:10.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1257e00640 1 -- 172.21.15.66:0/328603206 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f124800fe70 con 0x7f1260153850 2024-09-10T22:13:10.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1257e00640 1 -- 172.21.15.66:0/328603206 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248004260 con 0x7f1260153850 2024-09-10T22:13:10.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1266400640 1 -- 172.21.15.66:0/328603206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12600b41b0 con 0x7f1260153850 2024-09-10T22:13:10.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1266400640 1 -- 172.21.15.66:0/328603206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12600c5340 con 0x7f1260153850 2024-09-10T22:13:10.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1266400640 1 -- 172.21.15.66:0/328603206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12600cdb10 con 0x7f1260153850 2024-09-10T22:13:10.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.467+0000 7f1257e00640 1 -- 172.21.15.66:0/328603206 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f1248003600 con 0x7f1260153850 2024-09-10T22:13:10.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.471+0000 7f1257e00640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f123c0743b0 0x7f123c076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:10.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.471+0000 7f1257e00640 1 -- 172.21.15.66:0/328603206 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f1248092c50 con 0x7f1260153850 2024-09-10T22:13:10.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.471+0000 7f125f400640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f123c0743b0 0x7f123c076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:10.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.471+0000 7f125f400640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f123c0743b0 0x7f123c076870 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f12440057f0 tx=0x7f1250019540 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:10.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.475+0000 7f1257e00640 1 -- 172.21.15.66:0/328603206 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1260153100 con 0x7f1260153850 2024-09-10T22:13:10.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.683+0000 7f1266400640 1 -- 172.21.15.66:0/328603206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f126015f510 con 0x7f1260153850 2024-09-10T22:13:10.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1257e00640 1 -- 172.21.15.66:0/328603206 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) ==== 53+0+73 (secure 0 0 0) 0x7f124805f740 con 0x7f1260153850 2024-09-10T22:13:10.691 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:13:10.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1266400640 1 -- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f123c0743b0 msgr2=0x7f123c076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1266400640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f123c0743b0 0x7f123c076870 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f12440057f0 tx=0x7f1250019540 comp rx=0 tx=0).stop 2024-09-10T22:13:10.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1266400640 1 -- 172.21.15.66:0/328603206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1260153850 msgr2=0x7f12600c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:10.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1266400640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1260153850 0x7f12600c6100 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1248011ae0 tx=0x7f1248009f20 comp rx=0 tx=0).stop 2024-09-10T22:13:10.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1266400640 1 -- 172.21.15.66:0/328603206 shutdown_connections 2024-09-10T22:13:10.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1266400640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f123c0743b0 0x7f123c076870 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1266400640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12600c6640 0x7f12600ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1266400640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1260153850 0x7f12600c6100 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1266400640 1 --2- 172.21.15.66:0/328603206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1244003a60 0x7f1260152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:10.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1266400640 1 -- 172.21.15.66:0/328603206 >> 172.21.15.66:0/328603206 conn(0x7f12600cfa30 msgr2=0x7f12600af470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:10.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1266400640 1 -- 172.21.15.66:0/328603206 shutdown_connections 2024-09-10T22:13:10.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10.687+0000 7f1266400640 1 -- 172.21.15.66:0/328603206 wait complete. 2024-09-10T22:13:10.712 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:13:10.712 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:13:10.712 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:13:10.712 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:13:10.712 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:13:10.712 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:13:10.713 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:13:10.713 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:13:10.713 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:13:10.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:10.713 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:13:10.723 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:13:10.723 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:13:10.723 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:13:10.723 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:13:10.723 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:13:10.723 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:13:10.723 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:13:10.723 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:13:10.724 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:13:10.724 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:13:10.724 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:13:10.724 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:13:10.724 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:13:10.724 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:13:10.724 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:13:10.724 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:13:10.724 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:13:10.724 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41684sec preferred_lft 41684sec 2024-09-10T22:13:10.724 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-09-10T22:13:10.725 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:13:10.725 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:13:10.725 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:81:fc:4c:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:13:10.725 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:13:10.725 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:13:10.725 INFO:teuthology.orchestra.run.smithi066.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:13:10.725 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T22:13:10.725 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:13:10.725 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:13:10.725 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T22:13:10.726 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:13:10.726 INFO:teuthology.orchestra.run.smithi066.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:13:10.726 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:13:10.726 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T22:13:10.726 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:13:10.726 INFO:teuthology.orchestra.run.smithi066.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:13:10.726 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:13:10.726 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T22:13:10.726 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:13:10.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:10.727 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:13:10.787 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:13:10.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:10.790 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:13:10.845 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:13:10.845 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:13:10.845 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:13:10.845 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:13:10.845 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:13:10.845 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:13:10.846 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:13:10.846 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:13:10.846 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:13:10.846 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:13:10.846 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:13:10.846 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:13:10.846 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:13:10.846 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:13:10.846 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:13:10.846 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:13:10.847 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:13:10.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:13:10.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:13:10.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:10.982 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:13:11.039 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:13:11.039 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:13:11.039 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:13:11.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:11.039 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:13:11.102 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T22:13:11.102 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:13:11.102 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:13:11.154 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:13:11.154 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:13:11.154 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:13:11.205 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:13:11.266 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:13:11.324 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:13:11.324 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:13:11.324 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:13:11.324 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:13:11.324 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:13:11.324 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:13:11.324 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:13:11.324 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:13:11.325 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:13:11.326 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:13:11.326 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:13:11.326 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:13:11.326 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:13:11.326 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:13:11.326 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:13:11.326 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:13:11.326 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:13:11.326 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:13:11.326 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:13:11.326 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:13:11.326 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:13:11.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:11.327 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:13:11.378 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:13:11.378 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T22:13:11.424 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:13:11.445 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:13:11.495 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:13:11.495 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:13:11.496 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:13:11.497 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:13:11.498 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:13:11.498 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:13:11.498 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:13:11.498 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:13:11.498 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:13:11.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:11.499 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:13:11.521 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-09-10T22:13:11.515+0000 7ff3d790f580 -1 init, newargv = 0x557b5eb0b9a0 newargc=15 2024-09-10T22:13:11.521 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[52005]: starting ceph client 2024-09-10T22:13:11.528 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[52005]: starting fuse 2024-09-10T22:13:11.529 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-09-10T22:13:11.529 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T22:13:11.530 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:13:11.530 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-09-10T22:13:11.530 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-09-10T22:13:11.530 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:13:11.530 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-09-10T22:13:11.530 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:13:11.530 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-09-10T22:13:11.530 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T22:13:11.530 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-09-10T22:13:11.530 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T22:13:11.530 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:13:11.530 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-09-10T22:13:11.531 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:13:11.656 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.52005.asok 2024-09-10T22:13:11.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:13:11.668 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52005.asok 2024-09-10T22:13:11.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:11.668 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52005.asok status 2024-09-10T22:13:11.853 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "52005", 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 8961, 2024-09-10T22:13:11.854 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-09-10T22:13:11.855 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-09-10T22:13:11.855 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-09-10T22:13:11.855 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 8961 2024-09-10T22:13:11.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:11.855 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:13:11.855 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:13:11.855 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:13:11.855 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3943970644 2024-09-10T22:13:11.855 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3943970644 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.8961 192.168.0.1:0/3943970644", 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/3943970644", 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 86, 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 126, 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:13:11.856 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-09-10T22:13:11.857 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-09-10T22:13:11.857 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:13:11.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:11.869 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-09-10T22:13:11.869 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:13:11.878 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-09-10T22:13:11.878 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T22:13:11.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:11.878 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:13:11.947 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T22:13:12.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.139+0000 7f02cd200640 1 -- 172.21.15.66:0/68623709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02c80b2320 msgr2=0x7f02c80cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.139+0000 7f02cd200640 1 --2- 172.21.15.66:0/68623709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02c80b2320 0x7f02c80cdd60 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009c80 tx=0x7f02c001cd40 comp rx=0 tx=0).stop 2024-09-10T22:13:12.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.139+0000 7f02cd200640 1 -- 172.21.15.66:0/68623709 shutdown_connections 2024-09-10T22:13:12.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.139+0000 7f02cd200640 1 --2- 172.21.15.66:0/68623709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02c80b2320 0x7f02c80cdd60 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.139+0000 7f02cd200640 1 --2- 172.21.15.66:0/68623709 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02c815a040 0x7f02c80cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.139+0000 7f02cd200640 1 -- 172.21.15.66:0/68623709 >> 172.21.15.66:0/68623709 conn(0x7f02c80ccc10 msgr2=0x7f02c80cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:12.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.139+0000 7f02cd200640 1 -- 172.21.15.66:0/68623709 shutdown_connections 2024-09-10T22:13:12.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.139+0000 7f02cd200640 1 -- 172.21.15.66:0/68623709 wait complete. 2024-09-10T22:13:12.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02cd200640 1 Processor -- start 2024-09-10T22:13:12.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02cd200640 1 -- start start 2024-09-10T22:13:12.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02cd200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c80b2320 0x7f02c80b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:12.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02cd200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02c80cc450 0x7f02c80c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:12.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02cd200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02c815a040 0x7f02c80b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:12.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02cd200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f02c80cec80 con 0x7f02c80cc450 2024-09-10T22:13:12.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02cd200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f02c80ceb00 con 0x7f02c80b2320 2024-09-10T22:13:12.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02cd200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f02c80cee00 con 0x7f02c815a040 2024-09-10T22:13:12.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02bfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02c80cc450 0x7f02c80c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:12.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02bfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02c80cc450 0x7f02c80c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40246/0 (socket says 172.21.15.66:40246) 2024-09-10T22:13:12.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02bfe00640 1 -- 172.21.15.66:0/4184834258 learned_addr learned my addr 172.21.15.66:0/4184834258 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:12.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02c7e00640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02c815a040 0x7f02c80b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:12.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02c7400640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c80b2320 0x7f02c80b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:12.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02bfe00640 1 -- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02c815a040 msgr2=0x7f02c80b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02bfe00640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02c815a040 0x7f02c80b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02bfe00640 1 -- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c80b2320 msgr2=0x7f02c80b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02bfe00640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c80b2320 0x7f02c80b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02bfe00640 1 -- 172.21.15.66:0/4184834258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c81faaf0 con 0x7f02c80cc450 2024-09-10T22:13:12.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02c7400640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c80b2320 0x7f02c80b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:12.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02bfe00640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02c80cc450 0x7f02c80c9c40 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f02b000c490 tx=0x7f02b000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:12.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02bf400640 1 -- 172.21.15.66:0/4184834258 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b00052d0 con 0x7f02c80cc450 2024-09-10T22:13:12.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02bf400640 1 -- 172.21.15.66:0/4184834258 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02b0005490 con 0x7f02c80cc450 2024-09-10T22:13:12.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02bf400640 1 -- 172.21.15.66:0/4184834258 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0012850 con 0x7f02c80cc450 2024-09-10T22:13:12.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02cd200640 1 -- 172.21.15.66:0/4184834258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c81fadd0 con 0x7f02c80cc450 2024-09-10T22:13:12.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.143+0000 7f02cd200640 1 -- 172.21.15.66:0/4184834258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c81fb390 con 0x7f02c80cc450 2024-09-10T22:13:12.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.147+0000 7f02cd200640 1 -- 172.21.15.66:0/4184834258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c80cdd60 con 0x7f02c80cc450 2024-09-10T22:13:12.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.147+0000 7f02bf400640 1 -- 172.21.15.66:0/4184834258 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f02b00129f0 con 0x7f02c80cc450 2024-09-10T22:13:12.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.147+0000 7f02bf400640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f02a40743b0 0x7f02a4076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:12.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.147+0000 7f02bf400640 1 -- 172.21.15.66:0/4184834258 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f02b0091d40 con 0x7f02c80cc450 2024-09-10T22:13:12.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.147+0000 7f02c7400640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f02a40743b0 0x7f02a4076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:12.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.151+0000 7f02c7400640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f02a40743b0 0x7f02a4076870 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009b90 tx=0x7f02b80091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:12.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.151+0000 7f02bf400640 1 -- 172.21.15.66:0/4184834258 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f02b005ef40 con 0x7f02c80cc450 2024-09-10T22:13:12.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.367+0000 7f02cd200640 1 -- 172.21.15.66:0/4184834258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f02c80b0ac0 con 0x7f02c80cc450 2024-09-10T22:13:12.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02bf400640 1 -- 172.21.15.66:0/4184834258 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v87) ==== 136+0+0 (secure 0 0 0) 0x7f02b005e840 con 0x7f02c80cc450 2024-09-10T22:13:12.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02cd200640 1 -- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f02a40743b0 msgr2=0x7f02a4076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02cd200640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f02a40743b0 0x7f02a4076870 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009b90 tx=0x7f02b80091e0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02cd200640 1 -- 172.21.15.66:0/4184834258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02c80cc450 msgr2=0x7f02c80c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02cd200640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02c80cc450 0x7f02c80c9c40 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f02b000c490 tx=0x7f02b000c950 comp rx=0 tx=0).stop 2024-09-10T22:13:12.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02cd200640 1 -- 172.21.15.66:0/4184834258 shutdown_connections 2024-09-10T22:13:12.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02cd200640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f02a40743b0 0x7f02a4076870 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02cd200640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02c815a040 0x7f02c80b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02cd200640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02c80cc450 0x7f02c80c9c40 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02cd200640 1 --2- 172.21.15.66:0/4184834258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c80b2320 0x7f02c80b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02cd200640 1 -- 172.21.15.66:0/4184834258 >> 172.21.15.66:0/4184834258 conn(0x7f02c80ccc10 msgr2=0x7f02c8074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:12.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02cd200640 1 -- 172.21.15.66:0/4184834258 shutdown_connections 2024-09-10T22:13:12.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.423+0000 7f02cd200640 1 -- 172.21.15.66:0/4184834258 wait complete. 2024-09-10T22:13:12.450 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:13:12.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 -- 172.21.15.66:0/2318777381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03740b1e10 msgr2=0x7f037415fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 --2- 172.21.15.66:0/2318777381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03740b1e10 0x7f037415fb70 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f037406a2d0 tx=0x7f035c01d290 comp rx=0 tx=0).stop 2024-09-10T22:13:12.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 -- 172.21.15.66:0/2318777381 shutdown_connections 2024-09-10T22:13:12.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 --2- 172.21.15.66:0/2318777381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03740b1e10 0x7f037415fb70 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 --2- 172.21.15.66:0/2318777381 >> v2:172.21.15.94:3300/0 conn(0x7f037415f450 0x7f03740b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 -- 172.21.15.66:0/2318777381 >> 172.21.15.66:0/2318777381 conn(0x7f03740d1c60 msgr2=0x7f03740d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:12.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 -- 172.21.15.66:0/2318777381 shutdown_connections 2024-09-10T22:13:12.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 -- 172.21.15.66:0/2318777381 wait complete. 2024-09-10T22:13:12.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 Processor -- start 2024-09-10T22:13:12.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 -- start start 2024-09-10T22:13:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03740b1e10 0x7f0374166710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f037415f450 0x7f0374159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03741600b0 0x7f037415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0374169240 con 0x7f03741600b0 2024-09-10T22:13:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f03741690c0 con 0x7f03740b1e10 2024-09-10T22:13:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f037a600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f03741693c0 con 0x7f037415f450 2024-09-10T22:13:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f0373e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03741600b0 0x7f037415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f0373e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03741600b0 0x7f037415aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52888/0 (socket says 172.21.15.66:52888) 2024-09-10T22:13:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f0373e00640 1 -- 172.21.15.66:0/3249158002 learned_addr learned my addr 172.21.15.66:0/3249158002 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f0373400640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03740b1e10 0x7f0374166710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.595+0000 7f036aa00640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f037415f450 0x7f0374159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:12.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f0373e00640 1 -- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f037415f450 msgr2=0x7f0374159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f0373e00640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f037415f450 0x7f0374159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f0373e00640 1 -- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03740b1e10 msgr2=0x7f0374166710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f0373e00640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03740b1e10 0x7f0374166710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f0373e00640 1 -- 172.21.15.66:0/3249158002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0374166c50 con 0x7f03741600b0 2024-09-10T22:13:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f0373e00640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03741600b0 0x7f037415aac0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f036c00c710 tx=0x7f036c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f036be00640 1 -- 172.21.15.66:0/3249158002 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c003b80 con 0x7f03741600b0 2024-09-10T22:13:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f036be00640 1 -- 172.21.15.66:0/3249158002 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f036c003d20 con 0x7f03741600b0 2024-09-10T22:13:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f036be00640 1 -- 172.21.15.66:0/3249158002 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c00aab0 con 0x7f03741600b0 2024-09-10T22:13:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f037a600640 1 -- 172.21.15.66:0/3249158002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03741f7ff0 con 0x7f03741600b0 2024-09-10T22:13:12.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f037a600640 1 -- 172.21.15.66:0/3249158002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03741f85b0 con 0x7f03741600b0 2024-09-10T22:13:12.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f037a600640 1 -- 172.21.15.66:0/3249158002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f037415fb70 con 0x7f03741600b0 2024-09-10T22:13:12.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.599+0000 7f036be00640 1 -- 172.21.15.66:0/3249158002 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f036c00ac50 con 0x7f03741600b0 2024-09-10T22:13:12.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.603+0000 7f036be00640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03500743b0 0x7f0350076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:12.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.603+0000 7f036be00640 1 -- 172.21.15.66:0/3249158002 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f036c092930 con 0x7f03741600b0 2024-09-10T22:13:12.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.603+0000 7f0373400640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03500743b0 0x7f0350076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:12.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.603+0000 7f0373400640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03500743b0 0x7f0350076870 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f036400bc00 tx=0x7f036400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:12.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.603+0000 7f036be00640 1 -- 172.21.15.66:0/3249158002 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f036c00ce20 con 0x7f03741600b0 2024-09-10T22:13:12.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.803+0000 7f037a600640 1 -- 172.21.15.66:0/3249158002 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f03740c7400 con 0x7f03500743b0 2024-09-10T22:13:12.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.803+0000 7f036be00640 1 -- 172.21.15.66:0/3249158002 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f03740c7400 con 0x7f03500743b0 2024-09-10T22:13:12.809 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:13:12.810 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:12.810 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:13:12.810 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:12.810 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:13:12.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.807+0000 7f037a600640 1 -- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03500743b0 msgr2=0x7f0350076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.807+0000 7f037a600640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03500743b0 0x7f0350076870 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f036400bc00 tx=0x7f036400b390 comp rx=0 tx=0).stop 2024-09-10T22:13:12.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.807+0000 7f037a600640 1 -- 172.21.15.66:0/3249158002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03741600b0 msgr2=0x7f037415aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.807+0000 7f037a600640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03741600b0 0x7f037415aac0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f036c00c710 tx=0x7f036c00cbd0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.807+0000 7f037a600640 1 -- 172.21.15.66:0/3249158002 shutdown_connections 2024-09-10T22:13:12.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.807+0000 7f037a600640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03500743b0 0x7f0350076870 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.807+0000 7f037a600640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03741600b0 0x7f037415aac0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.807+0000 7f037a600640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f037415f450 0x7f0374159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.807+0000 7f037a600640 1 --2- 172.21.15.66:0/3249158002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03740b1e10 0x7f0374166710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.807+0000 7f037a600640 1 -- 172.21.15.66:0/3249158002 >> 172.21.15.66:0/3249158002 conn(0x7f03740d1c60 msgr2=0x7f037406e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:12.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.807+0000 7f037a600640 1 -- 172.21.15.66:0/3249158002 shutdown_connections 2024-09-10T22:13:12.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.807+0000 7f037a600640 1 -- 172.21.15.66:0/3249158002 wait complete. 2024-09-10T22:13:12.831 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:13:12.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.975+0000 7ff230c00640 1 -- 172.21.15.66:0/3275693608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff22c0c4b10 msgr2=0x7ff22c06e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.975+0000 7ff230c00640 1 --2- 172.21.15.66:0/3275693608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff22c0c4b10 0x7ff22c06e960 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009900 tx=0x7ff21c01c8e0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.975+0000 7ff230c00640 1 -- 172.21.15.66:0/3275693608 shutdown_connections 2024-09-10T22:13:12.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.975+0000 7ff230c00640 1 --2- 172.21.15.66:0/3275693608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff22c0c4b10 0x7ff22c06e960 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.975+0000 7ff230c00640 1 -- 172.21.15.66:0/3275693608 >> 172.21.15.66:0/3275693608 conn(0x7ff22c0cfa30 msgr2=0x7ff22c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:12.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff230c00640 1 -- 172.21.15.66:0/3275693608 shutdown_connections 2024-09-10T22:13:12.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff230c00640 1 -- 172.21.15.66:0/3275693608 wait complete. 2024-09-10T22:13:12.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff230c00640 1 Processor -- start 2024-09-10T22:13:12.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff230c00640 1 -- start start 2024-09-10T22:13:12.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff230c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff22c0ae620 0x7ff22c13bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:12.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff230c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff22c0c4b10 0x7ff22c13c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:12.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff230c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff22c0c4ef0 0x7ff22c14fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:12.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff230c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c169610 con 0x7ff22c0c4ef0 2024-09-10T22:13:12.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff230c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c169490 con 0x7ff22c0ae620 2024-09-10T22:13:12.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff230c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff22c169790 con 0x7ff22c0c4b10 2024-09-10T22:13:12.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff22a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff22c0c4ef0 0x7ff22c14fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:12.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff22a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff22c0c4ef0 0x7ff22c14fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52904/0 (socket says 172.21.15.66:52904) 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff22a800640 1 -- 172.21.15.66:0/134496019 learned_addr learned my addr 172.21.15.66:0/134496019 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff229e00640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff22c0ae620 0x7ff22c13bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff223e00640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff22c0c4b10 0x7ff22c13c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff22a800640 1 -- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff22c0c4b10 msgr2=0x7ff22c13c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff22a800640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff22c0c4b10 0x7ff22c13c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff22a800640 1 -- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff22c0ae620 msgr2=0x7ff22c13bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff22a800640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff22c0ae620 0x7ff22c13bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff22a800640 1 -- 172.21.15.66:0/134496019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c150590 con 0x7ff22c0c4ef0 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff22a800640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff22c0c4ef0 0x7ff22c14fa00 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff22400c710 tx=0x7ff22400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff222000640 1 -- 172.21.15.66:0/134496019 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224003b80 con 0x7ff22c0c4ef0 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff222000640 1 -- 172.21.15.66:0/134496019 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff224003d20 con 0x7ff22c0c4ef0 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff222000640 1 -- 172.21.15.66:0/134496019 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22400aab0 con 0x7ff22c0c4ef0 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.979+0000 7ff230c00640 1 -- 172.21.15.66:0/134496019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22c150870 con 0x7ff22c0c4ef0 2024-09-10T22:13:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.983+0000 7ff230c00640 1 -- 172.21.15.66:0/134496019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22c073210 con 0x7ff22c0c4ef0 2024-09-10T22:13:12.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.983+0000 7ff230c00640 1 -- 172.21.15.66:0/134496019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff22c073410 con 0x7ff22c0c4ef0 2024-09-10T22:13:12.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.983+0000 7ff222000640 1 -- 172.21.15.66:0/134496019 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7ff22400ac50 con 0x7ff22c0c4ef0 2024-09-10T22:13:12.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.983+0000 7ff222000640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff2080743b0 0x7ff208076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:12.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.983+0000 7ff222000640 1 -- 172.21.15.66:0/134496019 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7ff224091fc0 con 0x7ff22c0c4ef0 2024-09-10T22:13:12.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.983+0000 7ff229e00640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff2080743b0 0x7ff208076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:12.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.987+0000 7ff229e00640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff2080743b0 0x7ff208076870 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0af000 tx=0x7ff21c01b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:12.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:12.987+0000 7ff222000640 1 -- 172.21.15.66:0/134496019 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff22c073410 con 0x7ff22c0c4ef0 2024-09-10T22:13:13.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 -- 172.21.15.66:0/134496019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff22c06e960 con 0x7ff22c0c4ef0 2024-09-10T22:13:13.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff222000640 1 -- 172.21.15.66:0/134496019 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7ff22400ce20 con 0x7ff22c0c4ef0 2024-09-10T22:13:13.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 -- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff2080743b0 msgr2=0x7ff208076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:13.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff2080743b0 0x7ff208076870 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0af000 tx=0x7ff21c01b120 comp rx=0 tx=0).stop 2024-09-10T22:13:13.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 -- 172.21.15.66:0/134496019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff22c0c4ef0 msgr2=0x7ff22c14fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:13.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff22c0c4ef0 0x7ff22c14fa00 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff22400c710 tx=0x7ff22400cbd0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 -- 172.21.15.66:0/134496019 shutdown_connections 2024-09-10T22:13:13.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff2080743b0 0x7ff208076870 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff22c0c4ef0 0x7ff22c14fa00 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff22c0c4b10 0x7ff22c13c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 --2- 172.21.15.66:0/134496019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff22c0ae620 0x7ff22c13bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 -- 172.21.15.66:0/134496019 >> 172.21.15.66:0/134496019 conn(0x7ff22c0cfa30 msgr2=0x7ff22c0af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:13.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 -- 172.21.15.66:0/134496019 shutdown_connections 2024-09-10T22:13:13.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.163+0000 7ff230c00640 1 -- 172.21.15.66:0/134496019 wait complete. 2024-09-10T22:13:13.191 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-10T22:13:13.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.335+0000 7f6ec3400640 1 -- 172.21.15.66:0/234708652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea4005190 msgr2=0x7f6ea400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:13.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.335+0000 7f6ec3400640 1 --2- 172.21.15.66:0/234708652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea4005190 0x7f6ea400b9d0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009f10 tx=0x7f6eb401d610 comp rx=0 tx=0).stop 2024-09-10T22:13:13.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.335+0000 7f6ec1400640 1 -- 172.21.15.66:0/234708652 reap_dead start 2024-09-10T22:13:13.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.335+0000 7f6ec3400640 1 -- 172.21.15.66:0/234708652 shutdown_connections 2024-09-10T22:13:13.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.335+0000 7f6ec3400640 1 -- 172.21.15.66:0/234708652 >> 172.21.15.66:0/234708652 conn(0x7f6ebc0cfa30 msgr2=0x7f6ebc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:13.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.335+0000 7f6ec3400640 1 -- 172.21.15.66:0/234708652 shutdown_connections 2024-09-10T22:13:13.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.335+0000 7f6ec3400640 1 -- 172.21.15.66:0/234708652 wait complete. 2024-09-10T22:13:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec3400640 1 Processor -- start 2024-09-10T22:13:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec3400640 1 -- start start 2024-09-10T22:13:13.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ebc0ad8c0 0x7f6ebc0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec3400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ebc1458f0 0x7f6ebc0adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ebc0c8ba0 0x7f6ebc0c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec3400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc0c5180 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec3400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc0c5000 con 0x7f6ebc0ad8c0 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec3400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6ebc15a310 con 0x7f6ebc1458f0 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ebc0c8ba0 0x7f6ebc0c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ebc0c8ba0 0x7f6ebc0c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52912/0 (socket says 172.21.15.66:52912) 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec1400640 1 -- 172.21.15.66:0/2373423173 learned_addr learned my addr 172.21.15.66:0/2373423173 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec0a00640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ebc0ad8c0 0x7f6ebc0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ebbe00640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ebc1458f0 0x7f6ebc0adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec1400640 1 -- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ebc1458f0 msgr2=0x7f6ebc0adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec1400640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ebc1458f0 0x7f6ebc0adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec1400640 1 -- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ebc0ad8c0 msgr2=0x7f6ebc0b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:13.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec1400640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ebc0ad8c0 0x7f6ebc0b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec1400640 1 -- 172.21.15.66:0/2373423173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ebc0ae1e0 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec0a00640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ebc0ad8c0 0x7f6ebc0b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:13.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec1400640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ebc0c8ba0 0x7f6ebc0c9020 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4099140 tx=0x7f6eb4009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:13.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6eb8c00640 1 -- 172.21.15.66:0/2373423173 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb40ad030 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6eb8c00640 1 -- 172.21.15.66:0/2373423173 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6eb4003980 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6eb8c00640 1 -- 172.21.15.66:0/2373423173 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4009200 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec3400640 1 -- 172.21.15.66:0/2373423173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ebc145e40 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.339+0000 7f6ec3400640 1 -- 172.21.15.66:0/2373423173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ebc146400 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.343+0000 7f6ec3400640 1 -- 172.21.15.66:0/2373423173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ebc070420 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.343+0000 7f6eb8c00640 1 -- 172.21.15.66:0/2373423173 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f6eb40a76f0 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.343+0000 7f6eb8c00640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6e9c0743b0 0x7f6e9c076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:13.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.343+0000 7f6eb8c00640 1 -- 172.21.15.66:0/2373423173 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f6eb412c8e0 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.343+0000 7f6ec0a00640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6e9c0743b0 0x7f6e9c076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:13.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.343+0000 7f6ec0a00640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6e9c0743b0 0x7f6e9c076870 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009c60 tx=0x7f6eb00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:13.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.347+0000 7f6eb8c00640 1 -- 172.21.15.66:0/2373423173 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6eb40b7070 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.523+0000 7f6ec3400640 1 -- 172.21.15.66:0/2373423173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f6ebc0744b0 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.523+0000 7f6eb8c00640 1 -- 172.21.15.66:0/2373423173 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v51) ==== 116+0+2 (secure 0 0 0) 0x7f6eb4017360 con 0x7f6ebc0c8ba0 2024-09-10T22:13:13.529 INFO:teuthology.orchestra.run.smithi066.stdout:4 2024-09-10T22:13:13.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.523+0000 7f6ec3400640 1 -- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6e9c0743b0 msgr2=0x7f6e9c076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:13.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.523+0000 7f6ec3400640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6e9c0743b0 0x7f6e9c076870 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009c60 tx=0x7f6eb00091e0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.523+0000 7f6ec3400640 1 -- 172.21.15.66:0/2373423173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ebc0c8ba0 msgr2=0x7f6ebc0c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:13.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.523+0000 7f6ec3400640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ebc0c8ba0 0x7f6ebc0c9020 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4099140 tx=0x7f6eb4009de0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.527+0000 7f6ec3400640 1 -- 172.21.15.66:0/2373423173 shutdown_connections 2024-09-10T22:13:13.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.527+0000 7f6ec3400640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6e9c0743b0 0x7f6e9c076870 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.527+0000 7f6ec3400640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ebc0c8ba0 0x7f6ebc0c9020 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.527+0000 7f6ec3400640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ebc1458f0 0x7f6ebc0adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.527+0000 7f6ec3400640 1 --2- 172.21.15.66:0/2373423173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ebc0ad8c0 0x7f6ebc0b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.527+0000 7f6ec3400640 1 -- 172.21.15.66:0/2373423173 >> 172.21.15.66:0/2373423173 conn(0x7f6ebc0cfa30 msgr2=0x7f6ebc0cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:13.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.527+0000 7f6ec3400640 1 -- 172.21.15.66:0/2373423173 shutdown_connections 2024-09-10T22:13:13.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.527+0000 7f6ec3400640 1 -- 172.21.15.66:0/2373423173 wait complete. 2024-09-10T22:13:13.550 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:13:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.695+0000 7fdd11c00640 1 -- 172.21.15.66:0/1323610633 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd0c15f340 msgr2=0x7fdd0c0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.695+0000 7fdd11c00640 1 --2- 172.21.15.66:0/1323610633 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd0c15f340 0x7fdd0c0cfb50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009840 tx=0x7fdcfc01c820 comp rx=0 tx=0).stop 2024-09-10T22:13:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.695+0000 7fdd11c00640 1 -- 172.21.15.66:0/1323610633 shutdown_connections 2024-09-10T22:13:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.695+0000 7fdd11c00640 1 --2- 172.21.15.66:0/1323610633 >> v2:172.21.15.94:3300/0 conn(0x7fdd0c0af970 0x7fdd0c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.695+0000 7fdd11c00640 1 --2- 172.21.15.66:0/1323610633 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd0c15f340 0x7fdd0c0cfb50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.695+0000 7fdd11c00640 1 -- 172.21.15.66:0/1323610633 >> 172.21.15.66:0/1323610633 conn(0x7fdd0c0d1c50 msgr2=0x7fdd0c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.695+0000 7fdd11c00640 1 -- 172.21.15.66:0/1323610633 shutdown_connections 2024-09-10T22:13:13.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.695+0000 7fdd11c00640 1 -- 172.21.15.66:0/1323610633 wait complete. 2024-09-10T22:13:13.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd11c00640 1 Processor -- start 2024-09-10T22:13:13.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd11c00640 1 -- start start 2024-09-10T22:13:13.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd11c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd0c0af970 0x7fdd0c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:13.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd11c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd0c0b2bf0 0x7fdd0c15dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:13.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd11c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c15f340 0x7fdd0c15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:13.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd11c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c169190 con 0x7fdd0c15f340 2024-09-10T22:13:13.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd11c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c169010 con 0x7fdd0c0b2bf0 2024-09-10T22:13:13.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd11c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0c169310 con 0x7fdd0c0af970 2024-09-10T22:13:13.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd0b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c15f340 0x7fdd0c15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:13.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd0b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c15f340 0x7fdd0c15e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52914/0 (socket says 172.21.15.66:52914) 2024-09-10T22:13:13.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd0b400640 1 -- 172.21.15.66:0/3311249959 learned_addr learned my addr 172.21.15.66:0/3311249959 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:13.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd0aa00640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd0c0af970 0x7fdd0c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:13.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd0a000640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd0c0b2bf0 0x7fdd0c15dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:13.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd0b400640 1 -- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd0c0af970 msgr2=0x7fdd0c15bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:13.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd0b400640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd0c0af970 0x7fdd0c15bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd0b400640 1 -- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd0c0b2bf0 msgr2=0x7fdd0c15dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:13.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd0b400640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd0c0b2bf0 0x7fdd0c15dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd0b400640 1 -- 172.21.15.66:0/3311249959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c0c98f0 con 0x7fdd0c15f340 2024-09-10T22:13:13.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd0aa00640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd0c0af970 0x7fdd0c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:13.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd0b400640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c15f340 0x7fdd0c15e320 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400eaf0 tx=0x7fdd0400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:13.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd02a00640 1 -- 172.21.15.66:0/3311249959 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04004b30 con 0x7fdd0c15f340 2024-09-10T22:13:13.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd02a00640 1 -- 172.21.15.66:0/3311249959 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd04004cf0 con 0x7fdd0c15f340 2024-09-10T22:13:13.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd02a00640 1 -- 172.21.15.66:0/3311249959 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04005280 con 0x7fdd0c15f340 2024-09-10T22:13:13.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd11c00640 1 -- 172.21.15.66:0/3311249959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c0c9bd0 con 0x7fdd0c15f340 2024-09-10T22:13:13.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd11c00640 1 -- 172.21.15.66:0/3311249959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c0c85e0 con 0x7fdd0c15f340 2024-09-10T22:13:13.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.699+0000 7fdd11c00640 1 -- 172.21.15.66:0/3311249959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0c15fac0 con 0x7fdd0c15f340 2024-09-10T22:13:13.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.703+0000 7fdd02a00640 1 -- 172.21.15.66:0/3311249959 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7fdd04005420 con 0x7fdd0c15f340 2024-09-10T22:13:13.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.703+0000 7fdd02a00640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdcec0743b0 0x7fdcec076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:13.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.703+0000 7fdd02a00640 1 -- 172.21.15.66:0/3311249959 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7fdd04092f20 con 0x7fdd0c15f340 2024-09-10T22:13:13.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.703+0000 7fdd0aa00640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdcec0743b0 0x7fdcec076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:13.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.703+0000 7fdd0aa00640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdcec0743b0 0x7fdcec076870 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0af000 tx=0x7fdcfc01b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:13.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.707+0000 7fdd02a00640 1 -- 172.21.15.66:0/3311249959 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdd04060060 con 0x7fdd0c15f340 2024-09-10T22:13:13.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.879+0000 7fdd11c00640 1 -- 172.21.15.66:0/3311249959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fdd0c071a90 con 0x7fdd0c15f340 2024-09-10T22:13:13.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.883+0000 7fdd02a00640 1 -- 172.21.15.66:0/3311249959 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v52)=0 v52) ==== 131+0+0 (secure 0 0 0) 0x7fdd0405f960 con 0x7fdd0c15f340 2024-09-10T22:13:13.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.883+0000 7fdd11c00640 1 -- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdcec0743b0 msgr2=0x7fdcec076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:13.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.883+0000 7fdd11c00640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdcec0743b0 0x7fdcec076870 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0af000 tx=0x7fdcfc01b060 comp rx=0 tx=0).stop 2024-09-10T22:13:13.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.883+0000 7fdd11c00640 1 -- 172.21.15.66:0/3311249959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c15f340 msgr2=0x7fdd0c15e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:13.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.883+0000 7fdd11c00640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c15f340 0x7fdd0c15e320 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400eaf0 tx=0x7fdd0400c2d0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.883+0000 7fdd11c00640 1 -- 172.21.15.66:0/3311249959 shutdown_connections 2024-09-10T22:13:13.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.883+0000 7fdd11c00640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdcec0743b0 0x7fdcec076870 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.883+0000 7fdd11c00640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c15f340 0x7fdd0c15e320 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.883+0000 7fdd11c00640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd0c0b2bf0 0x7fdd0c15dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.883+0000 7fdd11c00640 1 --2- 172.21.15.66:0/3311249959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd0c0af970 0x7fdd0c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:13.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.883+0000 7fdd11c00640 1 -- 172.21.15.66:0/3311249959 >> 172.21.15.66:0/3311249959 conn(0x7fdd0c0d1c50 msgr2=0x7fdd0c071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:13.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.887+0000 7fdd11c00640 1 -- 172.21.15.66:0/3311249959 shutdown_connections 2024-09-10T22:13:13.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:13.887+0000 7fdd11c00640 1 -- 172.21.15.66:0/3311249959 wait complete. 2024-09-10T22:13:13.908 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:13.908+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:13.908 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:13.908+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:13.908 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:13.908+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:13.908 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:13.908+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:13.909 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:13.908+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:13.910 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-10T22:13:14.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.055+0000 7f0854600640 1 -- 172.21.15.66:0/4031992474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c144ac0 msgr2=0x7f084c06e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.055+0000 7f0854600640 1 --2- 172.21.15.66:0/4031992474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c144ac0 0x7f084c06e6a0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f084c0cd630 tx=0x7f084001c830 comp rx=0 tx=0).stop 2024-09-10T22:13:14.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.055+0000 7f0854600640 1 -- 172.21.15.66:0/4031992474 shutdown_connections 2024-09-10T22:13:14.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.055+0000 7f0854600640 1 --2- 172.21.15.66:0/4031992474 >> v2:172.21.15.94:3300/0 conn(0x7f084c145050 0x7f084c0ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.055+0000 7f0854600640 1 --2- 172.21.15.66:0/4031992474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c144ac0 0x7f084c06e6a0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.055+0000 7f0854600640 1 -- 172.21.15.66:0/4031992474 >> 172.21.15.66:0/4031992474 conn(0x7f084c0cfaa0 msgr2=0x7f084c0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:14.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.055+0000 7f0854600640 1 -- 172.21.15.66:0/4031992474 shutdown_connections 2024-09-10T22:13:14.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.055+0000 7f0854600640 1 -- 172.21.15.66:0/4031992474 wait complete. 2024-09-10T22:13:14.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0854600640 1 Processor -- start 2024-09-10T22:13:14.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0854600640 1 -- start start 2024-09-10T22:13:14.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0854600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f084c0cf080 0x7f084c15b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:14.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0854600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c144ac0 0x7f084c13d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:14.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0854600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f084c145050 0x7f084c164ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:14.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0854600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f084c168f50 con 0x7f084c144ac0 2024-09-10T22:13:14.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0854600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f084c168dd0 con 0x7f084c0cf080 2024-09-10T22:13:14.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0854600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f084c1690d0 con 0x7f084c145050 2024-09-10T22:13:14.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0851200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c144ac0 0x7f084c13d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:14.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0851200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c144ac0 0x7f084c13d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52942/0 (socket says 172.21.15.66:52942) 2024-09-10T22:13:14.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0851200640 1 -- 172.21.15.66:0/2347329748 learned_addr learned my addr 172.21.15.66:0/2347329748 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:14.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0851c00640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f084c0cf080 0x7f084c15b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0852600640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f084c145050 0x7f084c164ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0851200640 1 -- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f084c145050 msgr2=0x7f084c164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0851200640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f084c145050 0x7f084c164ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0851200640 1 -- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f084c0cf080 msgr2=0x7f084c15b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0851200640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f084c0cf080 0x7f084c15b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0851200640 1 -- 172.21.15.66:0/2347329748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c15af00 con 0x7f084c144ac0 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0851c00640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f084c0cf080 0x7f084c15b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0851200640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c144ac0 0x7f084c13d8b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f083800e7a0 tx=0x7f083800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0849600640 1 -- 172.21.15.66:0/2347329748 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083800ccc0 con 0x7f084c144ac0 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0849600640 1 -- 172.21.15.66:0/2347329748 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0838010430 con 0x7f084c144ac0 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0849600640 1 -- 172.21.15.66:0/2347329748 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838010860 con 0x7f084c144ac0 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0854600640 1 -- 172.21.15.66:0/2347329748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c15b1e0 con 0x7f084c144ac0 2024-09-10T22:13:14.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0854600640 1 -- 172.21.15.66:0/2347329748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c15f8f0 con 0x7f084c144ac0 2024-09-10T22:13:14.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.059+0000 7f0854600640 1 -- 172.21.15.66:0/2347329748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f084c0d0160 con 0x7f084c144ac0 2024-09-10T22:13:14.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.063+0000 7f0849600640 1 -- 172.21.15.66:0/2347329748 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f0838010a00 con 0x7f084c144ac0 2024-09-10T22:13:14.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.063+0000 7f0849600640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f082c0743b0 0x7f082c076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:14.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.063+0000 7f0849600640 1 -- 172.21.15.66:0/2347329748 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f0838092c90 con 0x7f084c144ac0 2024-09-10T22:13:14.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.063+0000 7f0851c00640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f082c0743b0 0x7f082c076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:14.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.063+0000 7f0851c00640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f082c0743b0 0x7f082c076870 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f08400af000 tx=0x7f084001b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:14.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.067+0000 7f0849600640 1 -- 172.21.15.66:0/2347329748 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f083805fdd0 con 0x7f084c144ac0 2024-09-10T22:13:14.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.267+0000 7f0854600640 1 -- 172.21.15.66:0/2347329748 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f084c160750 con 0x7f082c0743b0 2024-09-10T22:13:14.282 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:14.284+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:14.282 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:14.284+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:14.282 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:14.284+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:14.283 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:14.284+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:14.283 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:14.284+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:14.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.311+0000 7f0849600640 1 -- 172.21.15.66:0/2347329748 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f084c160750 con 0x7f082c0743b0 2024-09-10T22:13:14.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.315+0000 7f0854600640 1 -- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f082c0743b0 msgr2=0x7f082c076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.315+0000 7f0854600640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f082c0743b0 0x7f082c076870 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f08400af000 tx=0x7f084001b070 comp rx=0 tx=0).stop 2024-09-10T22:13:14.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.315+0000 7f0854600640 1 -- 172.21.15.66:0/2347329748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c144ac0 msgr2=0x7f084c13d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.315+0000 7f0854600640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c144ac0 0x7f084c13d8b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f083800e7a0 tx=0x7f083800ec60 comp rx=0 tx=0).stop 2024-09-10T22:13:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.315+0000 7f0854600640 1 -- 172.21.15.66:0/2347329748 shutdown_connections 2024-09-10T22:13:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.315+0000 7f0854600640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f082c0743b0 0x7f082c076870 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.315+0000 7f0854600640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f084c145050 0x7f084c164ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.315+0000 7f0854600640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c144ac0 0x7f084c13d8b0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.315+0000 7f0854600640 1 --2- 172.21.15.66:0/2347329748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f084c0cf080 0x7f084c15b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.315+0000 7f0854600640 1 -- 172.21.15.66:0/2347329748 >> 172.21.15.66:0/2347329748 conn(0x7f084c0cfaa0 msgr2=0x7f084c13e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.315+0000 7f0854600640 1 -- 172.21.15.66:0/2347329748 shutdown_connections 2024-09-10T22:13:14.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.315+0000 7f0854600640 1 -- 172.21.15.66:0/2347329748 wait complete. 2024-09-10T22:13:14.339 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:13:14.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.483+0000 7f8b6ca00640 1 -- 172.21.15.66:0/2250491213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b680b0ef0 msgr2=0x7f8b680d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.483+0000 7f8b6ca00640 1 --2- 172.21.15.66:0/2250491213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b680b0ef0 0x7f8b680d02c0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f8b68073480 tx=0x7f8b6001cc60 comp rx=0 tx=0).stop 2024-09-10T22:13:14.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.483+0000 7f8b6ca00640 1 -- 172.21.15.66:0/2250491213 shutdown_connections 2024-09-10T22:13:14.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.483+0000 7f8b6ca00640 1 --2- 172.21.15.66:0/2250491213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b680b0ef0 0x7f8b680d02c0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.483+0000 7f8b6ca00640 1 --2- 172.21.15.66:0/2250491213 >> v2:172.21.15.94:3300/0 conn(0x7f8b68070ac0 0x7f8b68070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.483+0000 7f8b6ca00640 1 -- 172.21.15.66:0/2250491213 >> 172.21.15.66:0/2250491213 conn(0x7f8b680d1d00 msgr2=0x7f8b680d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:14.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.483+0000 7f8b6ca00640 1 -- 172.21.15.66:0/2250491213 shutdown_connections 2024-09-10T22:13:14.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.483+0000 7f8b6ca00640 1 -- 172.21.15.66:0/2250491213 wait complete. 2024-09-10T22:13:14.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b6ca00640 1 Processor -- start 2024-09-10T22:13:14.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b6ca00640 1 -- start start 2024-09-10T22:13:14.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b6ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b68070ac0 0x7f8b680b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:14.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b6ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b68071700 0x7f8b6815af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:14.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b6ca00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b680b0ef0 0x7f8b68154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:14.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b6ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68160b50 con 0x7f8b68071700 2024-09-10T22:13:14.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b6ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b681609d0 con 0x7f8b68070ac0 2024-09-10T22:13:14.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b6ca00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8b68160cd0 con 0x7f8b680b0ef0 2024-09-10T22:13:14.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b5fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b68071700 0x7f8b6815af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:14.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b5fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b68071700 0x7f8b6815af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52970/0 (socket says 172.21.15.66:52970) 2024-09-10T22:13:14.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b5fe00640 1 -- 172.21.15.66:0/3783600583 learned_addr learned my addr 172.21.15.66:0/3783600583 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:14.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b65e00640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b68070ac0 0x7f8b680b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:14.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b66800640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b680b0ef0 0x7f8b68154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:14.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b5fe00640 1 -- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b680b0ef0 msgr2=0x7f8b68154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b5fe00640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b680b0ef0 0x7f8b68154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b5fe00640 1 -- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b68070ac0 msgr2=0x7f8b680b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b5fe00640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b68070ac0 0x7f8b680b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b5fe00640 1 -- 172.21.15.66:0/3783600583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b68154880 con 0x7f8b68071700 2024-09-10T22:13:14.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b5fe00640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b68071700 0x7f8b6815af30 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000c530 tx=0x7f8b5000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:14.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b5e000640 1 -- 172.21.15.66:0/3783600583 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50017070 con 0x7f8b68071700 2024-09-10T22:13:14.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b5e000640 1 -- 172.21.15.66:0/3783600583 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b50003600 con 0x7f8b68071700 2024-09-10T22:13:14.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b5e000640 1 -- 172.21.15.66:0/3783600583 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50003a50 con 0x7f8b68071700 2024-09-10T22:13:14.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b6ca00640 1 -- 172.21.15.66:0/3783600583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b680b1700 con 0x7f8b68071700 2024-09-10T22:13:14.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.487+0000 7f8b6ca00640 1 -- 172.21.15.66:0/3783600583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b680b25a0 con 0x7f8b68071700 2024-09-10T22:13:14.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.491+0000 7f8b6ca00640 1 -- 172.21.15.66:0/3783600583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b68070ea0 con 0x7f8b68071700 2024-09-10T22:13:14.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.491+0000 7f8b5e000640 1 -- 172.21.15.66:0/3783600583 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f8b50003bf0 con 0x7f8b68071700 2024-09-10T22:13:14.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.491+0000 7f8b5e000640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8b440743b0 0x7f8b44076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:14.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.491+0000 7f8b5e000640 1 -- 172.21.15.66:0/3783600583 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5821+0+0 (secure 0 0 0) 0x7f8b50091910 con 0x7f8b68071700 2024-09-10T22:13:14.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.491+0000 7f8b65e00640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8b440743b0 0x7f8b44076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:14.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.495+0000 7f8b65e00640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8b440743b0 0x7f8b44076870 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8b580058d0 tx=0x7f8b58005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:14.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.495+0000 7f8b5e000640 1 -- 172.21.15.66:0/3783600583 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8b5005eb10 con 0x7f8b68071700 2024-09-10T22:13:14.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.699+0000 7f8b6ca00640 1 -- 172.21.15.66:0/3783600583 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f8b680b1440 con 0x7f8b440743b0 2024-09-10T22:13:14.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b5e000640 1 -- 172.21.15.66:0/3783600583 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f8b680b1440 con 0x7f8b440743b0 2024-09-10T22:13:14.711 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17 2024-09-10T22:13:14.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b6ca00640 1 -- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8b440743b0 msgr2=0x7f8b44076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b6ca00640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8b440743b0 0x7f8b44076870 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8b580058d0 tx=0x7f8b58005860 comp rx=0 tx=0).stop 2024-09-10T22:13:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b6ca00640 1 -- 172.21.15.66:0/3783600583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b68071700 msgr2=0x7f8b6815af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b6ca00640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b68071700 0x7f8b6815af30 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000c530 tx=0x7f8b5000c9f0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b6ca00640 1 -- 172.21.15.66:0/3783600583 shutdown_connections 2024-09-10T22:13:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b6ca00640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8b440743b0 0x7f8b44076870 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b6ca00640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b680b0ef0 0x7f8b68154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b6ca00640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b68071700 0x7f8b6815af30 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b6ca00640 1 --2- 172.21.15.66:0/3783600583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b68070ac0 0x7f8b680b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b6ca00640 1 -- 172.21.15.66:0/3783600583 >> 172.21.15.66:0/3783600583 conn(0x7f8b680d1d00 msgr2=0x7f8b6806ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b6ca00640 1 -- 172.21.15.66:0/3783600583 shutdown_connections 2024-09-10T22:13:14.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.707+0000 7f8b6ca00640 1 -- 172.21.15.66:0/3783600583 wait complete. 2024-09-10T22:13:14.733 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:13:14.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.879+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1379567701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cec0d0550 msgr2=0x7f7cec0c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.879+0000 7f7cf3e00640 1 --2- 172.21.15.66:0/1379567701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cec0d0550 0x7f7cec0c4df0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009840 tx=0x7f7cd801c790 comp rx=0 tx=0).stop 2024-09-10T22:13:14.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.879+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1379567701 shutdown_connections 2024-09-10T22:13:14.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.879+0000 7f7cf3e00640 1 --2- 172.21.15.66:0/1379567701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cec0d0550 0x7f7cec0c4df0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.879+0000 7f7cf3e00640 1 --2- 172.21.15.66:0/1379567701 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cec15a310 0x7f7cec0c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.879+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1379567701 >> 172.21.15.66:0/1379567701 conn(0x7f7cec0cfa30 msgr2=0x7f7cec0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:14.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.879+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1379567701 shutdown_connections 2024-09-10T22:13:14.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.879+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1379567701 wait complete. 2024-09-10T22:13:14.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.879+0000 7f7cf3e00640 1 Processor -- start 2024-09-10T22:13:14.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.879+0000 7f7cf3e00640 1 -- start start 2024-09-10T22:13:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cec0aeec0 0x7f7cec1f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cec15a310 0x7f7cec0cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cec14aed0 0x7f7cec1fb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec167a10 con 0x7f7cec15a310 2024-09-10T22:13:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec167890 con 0x7f7cec0aeec0 2024-09-10T22:13:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf3e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7cec167b90 con 0x7f7cec14aed0 2024-09-10T22:13:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cec15a310 0x7f7cec0cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cec15a310 0x7f7cec0cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52996/0 (socket says 172.21.15.66:52996) 2024-09-10T22:13:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf0a00640 1 -- 172.21.15.66:0/1343233323 learned_addr learned my addr 172.21.15.66:0/1343233323 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf1e00640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cec14aed0 0x7f7cec1fb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf1400640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cec0aeec0 0x7f7cec1f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:14.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf0a00640 1 -- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cec14aed0 msgr2=0x7f7cec1fb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf0a00640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cec14aed0 0x7f7cec1fb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf0a00640 1 -- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cec0aeec0 msgr2=0x7f7cec1f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:14.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf0a00640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cec0aeec0 0x7f7cec1f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:14.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf0a00640 1 -- 172.21.15.66:0/1343233323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec1598e0 con 0x7f7cec15a310 2024-09-10T22:13:14.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf1400640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cec0aeec0 0x7f7cec1f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:13:14.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf1e00640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cec14aed0 0x7f7cec1fb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:13:14.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf0a00640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cec15a310 0x7f7cec0cefb0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009790 tx=0x7f7cd80a65f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:14.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7ce9600640 1 -- 172.21.15.66:0/1343233323 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80ac070 con 0x7f7cec15a310 2024-09-10T22:13:14.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7ce9600640 1 -- 172.21.15.66:0/1343233323 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cd8004310 con 0x7f7cec15a310 2024-09-10T22:13:14.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1343233323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec0afe40 con 0x7f7cec15a310 2024-09-10T22:13:14.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7ce9600640 1 -- 172.21.15.66:0/1343233323 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80a6bf0 con 0x7f7cec15a310 2024-09-10T22:13:14.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1343233323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec1f8030 con 0x7f7cec15a310 2024-09-10T22:13:14.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.883+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1343233323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec1f8470 con 0x7f7cec15a310 2024-09-10T22:13:14.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.887+0000 7f7ce9600640 1 -- 172.21.15.66:0/1343233323 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 42) ==== 99097+0+0 (secure 0 0 0) 0x7f7cd80046d0 con 0x7f7cec15a310 2024-09-10T22:13:14.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.891+0000 7f7ce9600640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ccc0743b0 0x7f7ccc076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:14.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.891+0000 7f7ce9600640 1 -- 172.21.15.66:0/1343233323 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5879+0+0 (secure 0 0 0) 0x7f7cd812cd40 con 0x7f7cec15a310 2024-09-10T22:13:14.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.891+0000 7f7cf1400640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ccc0743b0 0x7f7ccc076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:14.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.891+0000 7f7cf1400640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ccc0743b0 0x7f7ccc076870 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0005fd0 tx=0x7f7ce0005d00 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:14.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:14.891+0000 7f7ce9600640 1 -- 172.21.15.66:0/1343233323 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7cd80b6070 con 0x7f7cec15a310 2024-09-10T22:13:15.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.095+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1343233323 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f7cec15df60 con 0x7f7ccc0743b0 2024-09-10T22:13:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7ce9600640 1 -- 172.21.15.66:0/1343233323 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f7cec15df60 con 0x7f7ccc0743b0 2024-09-10T22:13:15.107 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17 2024-09-10T22:13:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ccc0743b0 msgr2=0x7f7ccc076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7cf3e00640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ccc0743b0 0x7f7ccc076870 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0005fd0 tx=0x7f7ce0005d00 comp rx=0 tx=0).stop 2024-09-10T22:13:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1343233323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cec15a310 msgr2=0x7f7cec0cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7cf3e00640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cec15a310 0x7f7cec0cefb0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009790 tx=0x7f7cd80a65f0 comp rx=0 tx=0).stop 2024-09-10T22:13:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1343233323 shutdown_connections 2024-09-10T22:13:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7cf3e00640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7ccc0743b0 0x7f7ccc076870 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7cf3e00640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cec14aed0 0x7f7cec1fb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7cf3e00640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cec15a310 0x7f7cec0cefb0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7cf3e00640 1 --2- 172.21.15.66:0/1343233323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cec0aeec0 0x7f7cec1f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1343233323 >> 172.21.15.66:0/1343233323 conn(0x7f7cec0cfa30 msgr2=0x7f7cec14ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1343233323 shutdown_connections 2024-09-10T22:13:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:15.103+0000 7f7cf3e00640 1 -- 172.21.15.66:0/1343233323 wait complete. 2024-09-10T22:13:15.129 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17 2024-09-10T22:13:15.130 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:13:15.130 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:13:15.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:15.139 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:13:22.244 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:13:22.244 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:13:22.244 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.03933 s, 153 MB/s 2024-09-10T22:13:22.245 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:13:22.246 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:13:22.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:22.255 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:13:27.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:13:27.857 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:13:29.221 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:13:29.221 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:13:29.221 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.90812 s, 155 MB/s 2024-09-10T22:13:29.223 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:13:29.223 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:13:29.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:29.232 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:13:36.130 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:13:36.130 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:13:36.130 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.83918 s, 157 MB/s 2024-09-10T22:13:36.132 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-10T22:13:36.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.279+0000 7f811e200640 1 -- 172.21.15.66:0/3720285605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81180ae550 msgr2=0x7f81180ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:36.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.279+0000 7f811e200640 1 --2- 172.21.15.66:0/3720285605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81180ae550 0x7f81180ae990 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f8118069cf0 tx=0x7f810401c800 comp rx=0 tx=0).stop 2024-09-10T22:13:36.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.279+0000 7f811e200640 1 -- 172.21.15.66:0/3720285605 shutdown_connections 2024-09-10T22:13:36.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.279+0000 7f811e200640 1 --2- 172.21.15.66:0/3720285605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81180ae550 0x7f81180ae990 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.279+0000 7f811e200640 1 --2- 172.21.15.66:0/3720285605 >> v2:172.21.15.94:3300/0 conn(0x7f811815b8d0 0x7f81180b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.279+0000 7f811e200640 1 -- 172.21.15.66:0/3720285605 >> 172.21.15.66:0/3720285605 conn(0x7f81180b58b0 msgr2=0x7f81180b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:36.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.279+0000 7f811e200640 1 -- 172.21.15.66:0/3720285605 shutdown_connections 2024-09-10T22:13:36.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.279+0000 7f811e200640 1 -- 172.21.15.66:0/3720285605 wait complete. 2024-09-10T22:13:36.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.279+0000 7f811e200640 1 Processor -- start 2024-09-10T22:13:36.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.279+0000 7f811e200640 1 -- start start 2024-09-10T22:13:36.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81180ae550 0x7f81181fbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:36.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811e200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81180aeed0 0x7f81181528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:36.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811815b8d0 0x7f8118152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:36.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8118168f20 con 0x7f81180ae550 2024-09-10T22:13:36.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8118168da0 con 0x7f811815b8d0 2024-09-10T22:13:36.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811e200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f81181690a0 con 0x7f81180aeed0 2024-09-10T22:13:36.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81180ae550 0x7f81181fbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:36.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81180ae550 0x7f81181fbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56844/0 (socket says 172.21.15.66:56844) 2024-09-10T22:13:36.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811ce00640 1 -- 172.21.15.66:0/362070130 learned_addr learned my addr 172.21.15.66:0/362070130 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:36.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f8117e00640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81180aeed0 0x7f81181528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:36.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811d800640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811815b8d0 0x7f8118152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:36.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811ce00640 1 -- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81180aeed0 msgr2=0x7f81181528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:36.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811ce00640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81180aeed0 0x7f81181528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811ce00640 1 -- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811815b8d0 msgr2=0x7f8118152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:36.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811ce00640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811815b8d0 0x7f8118152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811ce00640 1 -- 172.21.15.66:0/362070130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81181534f0 con 0x7f81180ae550 2024-09-10T22:13:36.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811ce00640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81180ae550 0x7f81181fbff0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f810800d950 tx=0x7f810800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:36.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f8114c00640 1 -- 172.21.15.66:0/362070130 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108017070 con 0x7f81180ae550 2024-09-10T22:13:36.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f8114c00640 1 -- 172.21.15.66:0/362070130 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8108009d20 con 0x7f81180ae550 2024-09-10T22:13:36.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f8114c00640 1 -- 172.21.15.66:0/362070130 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81080106c0 con 0x7f81180ae550 2024-09-10T22:13:36.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811e200640 1 -- 172.21.15.66:0/362070130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81181538f0 con 0x7f81180ae550 2024-09-10T22:13:36.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811e200640 1 -- 172.21.15.66:0/362070130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81181f8070 con 0x7f81180ae550 2024-09-10T22:13:36.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.283+0000 7f811e200640 1 -- 172.21.15.66:0/362070130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81180ae990 con 0x7f81180ae550 2024-09-10T22:13:36.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.287+0000 7f8114c00640 1 -- 172.21.15.66:0/362070130 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 43) ==== 99199+0+0 (secure 0 0 0) 0x7f8108010860 con 0x7f81180ae550 2024-09-10T22:13:36.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.287+0000 7f8114c00640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f80f80744d0 0x7f80f8076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:36.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.287+0000 7f8114c00640 1 -- 172.21.15.66:0/362070130 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f8108092740 con 0x7f81180ae550 2024-09-10T22:13:36.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.287+0000 7f8117e00640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f80f80744d0 0x7f80f8076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:36.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.287+0000 7f8117e00640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f80f80744d0 0x7f80f8076990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f81040af000 tx=0x7f810401b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:36.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.291+0000 7f8114c00640 1 -- 172.21.15.66:0/362070130 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f810805f1a0 con 0x7f81180ae550 2024-09-10T22:13:36.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.495+0000 7f811e200640 1 -- 172.21.15.66:0/362070130 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f81180cfee0 con 0x7f80f80744d0 2024-09-10T22:13:36.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.507+0000 7f8114c00640 1 -- 172.21.15.66:0/362070130 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f81180cfee0 con 0x7f80f80744d0 2024-09-10T22:13:36.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.511+0000 7f811e200640 1 -- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f80f80744d0 msgr2=0x7f80f8076990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:36.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.511+0000 7f811e200640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f80f80744d0 0x7f80f8076990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f81040af000 tx=0x7f810401b040 comp rx=0 tx=0).stop 2024-09-10T22:13:36.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.511+0000 7f811e200640 1 -- 172.21.15.66:0/362070130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81180ae550 msgr2=0x7f81181fbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:36.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.511+0000 7f811e200640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81180ae550 0x7f81181fbff0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f810800d950 tx=0x7f810800de10 comp rx=0 tx=0).stop 2024-09-10T22:13:36.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.511+0000 7f811e200640 1 -- 172.21.15.66:0/362070130 shutdown_connections 2024-09-10T22:13:36.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.511+0000 7f811e200640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f80f80744d0 0x7f80f8076990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.511+0000 7f811e200640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811815b8d0 0x7f8118152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.511+0000 7f811e200640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81180aeed0 0x7f81181528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.511+0000 7f811e200640 1 --2- 172.21.15.66:0/362070130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81180ae550 0x7f81181fbff0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.511+0000 7f811e200640 1 -- 172.21.15.66:0/362070130 >> 172.21.15.66:0/362070130 conn(0x7f81180b58b0 msgr2=0x7f811815a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:36.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.511+0000 7f811e200640 1 -- 172.21.15.66:0/362070130 shutdown_connections 2024-09-10T22:13:36.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.511+0000 7f811e200640 1 -- 172.21.15.66:0/362070130 wait complete. 2024-09-10T22:13:36.534 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:13:36.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.679+0000 7faabbe00640 1 -- 172.21.15.66:0/1108455386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc15ef20 msgr2=0x7faabc0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:36.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.679+0000 7faabbe00640 1 --2- 172.21.15.66:0/1108455386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc15ef20 0x7faabc0cecc0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7faaac009c80 tx=0x7faaac01cd60 comp rx=0 tx=0).stop 2024-09-10T22:13:36.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.679+0000 7faabbe00640 1 -- 172.21.15.66:0/1108455386 shutdown_connections 2024-09-10T22:13:36.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.679+0000 7faabbe00640 1 --2- 172.21.15.66:0/1108455386 >> v2:172.21.15.94:3300/0 conn(0x7faabc0cd7a0 0x7faabc0cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.679+0000 7faabbe00640 1 --2- 172.21.15.66:0/1108455386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc15ef20 0x7faabc0cecc0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.679+0000 7faabbe00640 1 -- 172.21.15.66:0/1108455386 >> 172.21.15.66:0/1108455386 conn(0x7faabc0d1b10 msgr2=0x7faabc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:36.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.679+0000 7faabbe00640 1 -- 172.21.15.66:0/1108455386 shutdown_connections 2024-09-10T22:13:36.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.679+0000 7faabbe00640 1 -- 172.21.15.66:0/1108455386 wait complete. 2024-09-10T22:13:36.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.679+0000 7faabbe00640 1 Processor -- start 2024-09-10T22:13:36.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.679+0000 7faabbe00640 1 -- start start 2024-09-10T22:13:36.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faabbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faabc0cd7a0 0x7faabc15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:36.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faabbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc15ef20 0x7faabc071e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:36.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faabbe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faabc15f300 0x7faabc0af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:36.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faabbe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faabc168ee0 con 0x7faabc15ef20 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faabbe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faabc168d60 con 0x7faabc0cd7a0 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faabbe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7faabc169060 con 0x7faabc15f300 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faab2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc15ef20 0x7faabc071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faab2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc15ef20 0x7faabc071e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56858/0 (socket says 172.21.15.66:56858) 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faab2000640 1 -- 172.21.15.66:0/3363258493 learned_addr learned my addr 172.21.15.66:0/3363258493 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faabaa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faabc0cd7a0 0x7faabc15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faabb400640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faabc15f300 0x7faabc0af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faab2000640 1 -- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faabc15f300 msgr2=0x7faabc0af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faab2000640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faabc15f300 0x7faabc0af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faab2000640 1 -- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faabc0cd7a0 msgr2=0x7faabc15afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faab2000640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faabc0cd7a0 0x7faabc15afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faab2000640 1 -- 172.21.15.66:0/3363258493 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc072390 con 0x7faabc15ef20 2024-09-10T22:13:36.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faabaa00640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faabc0cd7a0 0x7faabc15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:13:36.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faab2000640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc15ef20 0x7faabc071e50 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7faaa400b3e0 tx=0x7faaa400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:36.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faab3e00640 1 -- 172.21.15.66:0/3363258493 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4015020 con 0x7faabc15ef20 2024-09-10T22:13:36.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faab3e00640 1 -- 172.21.15.66:0/3363258493 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaa4011920 con 0x7faabc15ef20 2024-09-10T22:13:36.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faab3e00640 1 -- 172.21.15.66:0/3363258493 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4011db0 con 0x7faabc15ef20 2024-09-10T22:13:36.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faabbe00640 1 -- 172.21.15.66:0/3363258493 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc1f8180 con 0x7faabc15ef20 2024-09-10T22:13:36.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faabbe00640 1 -- 172.21.15.66:0/3363258493 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc1f8740 con 0x7faabc15ef20 2024-09-10T22:13:36.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.683+0000 7faabbe00640 1 -- 172.21.15.66:0/3363258493 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faabc0cdc10 con 0x7faabc15ef20 2024-09-10T22:13:36.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.687+0000 7faab3e00640 1 -- 172.21.15.66:0/3363258493 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 43) ==== 99199+0+0 (secure 0 0 0) 0x7faaa401a400 con 0x7faabc15ef20 2024-09-10T22:13:36.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.687+0000 7faab3e00640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa980744d0 0x7faa98076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:36.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.687+0000 7faab3e00640 1 -- 172.21.15.66:0/3363258493 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7faaa4092060 con 0x7faabc15ef20 2024-09-10T22:13:36.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.687+0000 7faabaa00640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa980744d0 0x7faa98076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:36.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.687+0000 7faabaa00640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa980744d0 0x7faa98076990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7faaac0af000 tx=0x7faaac01b5a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.691+0000 7faab3e00640 1 -- 172.21.15.66:0/3363258493 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faaa401f030 con 0x7faabc15ef20 2024-09-10T22:13:36.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.891+0000 7faabbe00640 1 -- 172.21.15.66:0/3363258493 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7faabc15b4e0 con 0x7faa980744d0 2024-09-10T22:13:36.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.895+0000 7faab3e00640 1 -- 172.21.15.66:0/3363258493 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7faabc15b4e0 con 0x7faa980744d0 2024-09-10T22:13:36.901 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17 2024-09-10T22:13:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.899+0000 7faabbe00640 1 -- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa980744d0 msgr2=0x7faa98076990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.899+0000 7faabbe00640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa980744d0 0x7faa98076990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7faaac0af000 tx=0x7faaac01b5a0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.899+0000 7faabbe00640 1 -- 172.21.15.66:0/3363258493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc15ef20 msgr2=0x7faabc071e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.899+0000 7faabbe00640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc15ef20 0x7faabc071e50 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7faaa400b3e0 tx=0x7faaa400b7e0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.899+0000 7faabbe00640 1 -- 172.21.15.66:0/3363258493 shutdown_connections 2024-09-10T22:13:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.899+0000 7faabbe00640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa980744d0 0x7faa98076990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.899+0000 7faabbe00640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faabc15f300 0x7faabc0af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.899+0000 7faabbe00640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc15ef20 0x7faabc071e50 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.899+0000 7faabbe00640 1 --2- 172.21.15.66:0/3363258493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faabc0cd7a0 0x7faabc15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.899+0000 7faabbe00640 1 -- 172.21.15.66:0/3363258493 >> 172.21.15.66:0/3363258493 conn(0x7faabc0d1b10 msgr2=0x7faabc06e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.899+0000 7faabbe00640 1 -- 172.21.15.66:0/3363258493 shutdown_connections 2024-09-10T22:13:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:36.899+0000 7faabbe00640 1 -- 172.21.15.66:0/3363258493 wait complete. 2024-09-10T22:13:36.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:36.923 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17 2024-09-10T22:13:36.923 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:13:36.937 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17 2024-09-10T22:13:36.937 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3138256896" 2024-09-10T22:13:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:36.938 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-09-10T22:13:38.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:38.939 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17 2024-09-10T22:13:38.939 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:13:38.954 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17 2024-09-10T22:13:38.954 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3138256896" 2024-09-10T22:13:38.955 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:38.955 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-09-10T22:13:40.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:40.956 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17 2024-09-10T22:13:40.957 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:13:40.973 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/a45ef0e0-341d-419a-a988-a57967b39d17 2024-09-10T22:13:40.974 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3221225472" 2024-09-10T22:13:40.974 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:40.975 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-10T22:13:40.975 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-10T22:13:41.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.171+0000 7f3cfac00640 1 -- 172.21.15.66:0/2879940236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3cf4070ac0 msgr2=0x7f3cf4070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:41.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.171+0000 7f3cfac00640 1 --2- 172.21.15.66:0/2879940236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3cf4070ac0 0x7f3cf4070ea0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4073480 tx=0x7f3ce401c7a0 comp rx=0 tx=0).stop 2024-09-10T22:13:41.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.171+0000 7f3cfac00640 1 -- 172.21.15.66:0/2879940236 shutdown_connections 2024-09-10T22:13:41.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.171+0000 7f3cfac00640 1 --2- 172.21.15.66:0/2879940236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3cf40b0ef0 0x7f3cf40d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:41.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.171+0000 7f3cfac00640 1 --2- 172.21.15.66:0/2879940236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3cf4070ac0 0x7f3cf4070ea0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:41.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.171+0000 7f3cfac00640 1 -- 172.21.15.66:0/2879940236 >> 172.21.15.66:0/2879940236 conn(0x7f3cf40d1d00 msgr2=0x7f3cf40d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:41.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.171+0000 7f3cfac00640 1 -- 172.21.15.66:0/2879940236 shutdown_connections 2024-09-10T22:13:41.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.171+0000 7f3cfac00640 1 -- 172.21.15.66:0/2879940236 wait complete. 2024-09-10T22:13:41.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.171+0000 7f3cfac00640 1 Processor -- start 2024-09-10T22:13:41.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.171+0000 7f3cfac00640 1 -- start start 2024-09-10T22:13:41.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cfac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3cf4070ac0 0x7f3cf41518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:41.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cfac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cf4071700 0x7f3cf406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:41.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cfac00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3cf40b0ef0 0x7f3cf41fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:41.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cfac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4160b50 con 0x7f3cf4070ac0 2024-09-10T22:13:41.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cfac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf41609d0 con 0x7f3cf4071700 2024-09-10T22:13:41.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cfac00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf4160cd0 con 0x7f3cf40b0ef0 2024-09-10T22:13:41.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cf3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3cf4070ac0 0x7f3cf41518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:41.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cf3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3cf4070ac0 0x7f3cf41518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56888/0 (socket says 172.21.15.66:56888) 2024-09-10T22:13:41.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cf3e00640 1 -- 172.21.15.66:0/3967624105 learned_addr learned my addr 172.21.15.66:0/3967624105 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:41.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cf3400640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cf4071700 0x7f3cf406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:41.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cf8c00640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3cf40b0ef0 0x7f3cf41fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:41.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cf3e00640 1 -- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3cf40b0ef0 msgr2=0x7f3cf41fb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:41.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cf3e00640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3cf40b0ef0 0x7f3cf41fb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:41.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cf3e00640 1 -- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cf4071700 msgr2=0x7f3cf406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:41.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cf3e00640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cf4071700 0x7f3cf406dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:41.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cf3e00640 1 -- 172.21.15.66:0/3967624105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf41589f0 con 0x7f3cf4070ac0 2024-09-10T22:13:41.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cf3e00640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3cf4070ac0 0x7f3cf41518b0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009850 tx=0x7f3ce4005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:41.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cebe00640 1 -- 172.21.15.66:0/3967624105 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40043d0 con 0x7f3cf4070ac0 2024-09-10T22:13:41.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cebe00640 1 -- 172.21.15.66:0/3967624105 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ce4003600 con 0x7f3cf4070ac0 2024-09-10T22:13:41.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cebe00640 1 -- 172.21.15.66:0/3967624105 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4003a50 con 0x7f3cf4070ac0 2024-09-10T22:13:41.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cfac00640 1 -- 172.21.15.66:0/3967624105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf4158cd0 con 0x7f3cf4070ac0 2024-09-10T22:13:41.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cfac00640 1 -- 172.21.15.66:0/3967624105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf41fc070 con 0x7f3cf4070ac0 2024-09-10T22:13:41.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cfac00640 1 -- 172.21.15.66:0/3967624105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf4070ea0 con 0x7f3cf4070ac0 2024-09-10T22:13:41.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.175+0000 7f3cebe00640 1 -- 172.21.15.66:0/3967624105 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 43) ==== 99199+0+0 (secure 0 0 0) 0x7f3ce4003bf0 con 0x7f3cf4070ac0 2024-09-10T22:13:41.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.179+0000 7f3cebe00640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3cd40744d0 0x7f3cd4076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:41.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.179+0000 7f3cebe00640 1 -- 172.21.15.66:0/3967624105 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f3ce412ccf0 con 0x7f3cf4070ac0 2024-09-10T22:13:41.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.179+0000 7f3cf3400640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3cd40744d0 0x7f3cd4076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:41.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.179+0000 7f3cf3400640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3cd40744d0 0x7f3cd4076990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40b74e0 tx=0x7f3ce0007e40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:41.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.183+0000 7f3cebe00640 1 -- 172.21.15.66:0/3967624105 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3ce40ac030 con 0x7f3cf4070ac0 2024-09-10T22:13:41.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.387+0000 7f3cfac00640 1 -- 172.21.15.66:0/3967624105 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f3cf4165f30 con 0x7f3cd40744d0 2024-09-10T22:13:41.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.447+0000 7f3cebe00640 1 -- 172.21.15.66:0/3967624105 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3cf4165f30 con 0x7f3cd40744d0 2024-09-10T22:13:41.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.447+0000 7f3cfac00640 1 -- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3cd40744d0 msgr2=0x7f3cd4076990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:41.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.447+0000 7f3cfac00640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3cd40744d0 0x7f3cd4076990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40b74e0 tx=0x7f3ce0007e40 comp rx=0 tx=0).stop 2024-09-10T22:13:41.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.447+0000 7f3cfac00640 1 -- 172.21.15.66:0/3967624105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3cf4070ac0 msgr2=0x7f3cf41518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:41.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.447+0000 7f3cfac00640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3cf4070ac0 0x7f3cf41518b0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009850 tx=0x7f3ce4005e70 comp rx=0 tx=0).stop 2024-09-10T22:13:41.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.451+0000 7f3cfac00640 1 -- 172.21.15.66:0/3967624105 shutdown_connections 2024-09-10T22:13:41.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.451+0000 7f3cfac00640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3cd40744d0 0x7f3cd4076990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:41.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.451+0000 7f3cfac00640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3cf40b0ef0 0x7f3cf41fb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:41.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.451+0000 7f3cfac00640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cf4071700 0x7f3cf406dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:41.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.451+0000 7f3cfac00640 1 --2- 172.21.15.66:0/3967624105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3cf4070ac0 0x7f3cf41518b0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:41.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.451+0000 7f3cfac00640 1 -- 172.21.15.66:0/3967624105 >> 172.21.15.66:0/3967624105 conn(0x7f3cf40d1d00 msgr2=0x7f3cf40b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:41.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.451+0000 7f3cfac00640 1 -- 172.21.15.66:0/3967624105 shutdown_connections 2024-09-10T22:13:41.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:41.451+0000 7f3cfac00640 1 -- 172.21.15.66:0/3967624105 wait complete. 2024-09-10T22:13:41.462 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:41.464+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:41.462 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:41.464+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:41.462 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:41.464+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:41.462 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:41.464+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:41.463 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:41.464+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:41.478 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:41.480+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:41.478 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:41.480+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:41.479 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:41.480+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:41.479 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:41.480+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:41.479 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:41.480+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:42.461 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:42.460+0000 7f5f95600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:42.461 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:42.460+0000 7f5f95600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:42.461 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:42.460+0000 7f5f95600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:42.461 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:42.460+0000 7f5f95600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:42.461 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:42.460+0000 7f5f95600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:42.475 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-10T22:13:42.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 -- 172.21.15.66:0/293816328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8694004810 msgr2=0x7f8694004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 --2- 172.21.15.66:0/293816328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8694004810 0x7f8694004c90 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f869c009b60 tx=0x7f869c01d270 comp rx=0 tx=0).stop 2024-09-10T22:13:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b4e00640 1 -- 172.21.15.66:0/293816328 reap_dead start 2024-09-10T22:13:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 -- 172.21.15.66:0/293816328 shutdown_connections 2024-09-10T22:13:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 -- 172.21.15.66:0/293816328 >> 172.21.15.66:0/293816328 conn(0x7f86b00d1c50 msgr2=0x7f86b00d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 -- 172.21.15.66:0/293816328 shutdown_connections 2024-09-10T22:13:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 -- 172.21.15.66:0/293816328 wait complete. 2024-09-10T22:13:42.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 Processor -- start 2024-09-10T22:13:42.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 -- start start 2024-09-10T22:13:42.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8694004810 0x7f86b0155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:42.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86b00b6300 0x7f86b00b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:42.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b00c7e40 0x7f86b00b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:42.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f86b00d28f0 con 0x7f86b00b6300 2024-09-10T22:13:42.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86b00d2770 con 0x7f86b00c7e40 2024-09-10T22:13:42.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86b6e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f86b00d2a70 con 0x7f8694004810 2024-09-10T22:13:42.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86af400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86b00b6300 0x7f86b00b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:42.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86af400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86b00b6300 0x7f86b00b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39810/0 (socket says 172.21.15.66:39810) 2024-09-10T22:13:42.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86af400640 1 -- 172.21.15.66:0/3497280941 learned_addr learned my addr 172.21.15.66:0/3497280941 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:42.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86afe00640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8694004810 0x7f86b0155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86af400640 1 -- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8694004810 msgr2=0x7f86b0155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86af400640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8694004810 0x7f86b0155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86af400640 1 -- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b00c7e40 msgr2=0x7f86b00b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86af400640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b00c7e40 0x7f86b00b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.675+0000 7f86af400640 1 -- 172.21.15.66:0/3497280941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b00b4fc0 con 0x7f86b00b6300 2024-09-10T22:13:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.679+0000 7f86af400640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86b00b6300 0x7f86b00b4a80 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f869c01d0a0 tx=0x7f869c005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.679+0000 7f86a7e00640 1 -- 172.21.15.66:0/3497280941 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c0b1040 con 0x7f86b00b6300 2024-09-10T22:13:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.679+0000 7f86a7e00640 1 -- 172.21.15.66:0/3497280941 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f869c0ac400 con 0x7f86b00b6300 2024-09-10T22:13:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.679+0000 7f86b6e00640 1 -- 172.21.15.66:0/3497280941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b00b5260 con 0x7f86b00b6300 2024-09-10T22:13:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.679+0000 7f86a7e00640 1 -- 172.21.15.66:0/3497280941 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c0aca10 con 0x7f86b00b6300 2024-09-10T22:13:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.679+0000 7f86b6e00640 1 -- 172.21.15.66:0/3497280941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b00b5b00 con 0x7f86b00b6300 2024-09-10T22:13:42.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.679+0000 7f86b6e00640 1 -- 172.21.15.66:0/3497280941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b00b5650 con 0x7f86b00b6300 2024-09-10T22:13:42.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.679+0000 7f86a7e00640 1 -- 172.21.15.66:0/3497280941 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 43) ==== 99199+0+0 (secure 0 0 0) 0x7f869c0acbb0 con 0x7f86b00b6300 2024-09-10T22:13:42.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.683+0000 7f86a7e00640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8690074500 0x7f86900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:42.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.683+0000 7f86a7e00640 1 -- 172.21.15.66:0/3497280941 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f869c12ccf0 con 0x7f86b00b6300 2024-09-10T22:13:42.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.687+0000 7f86afe00640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8690074500 0x7f86900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:42.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.687+0000 7f86a7e00640 1 -- 172.21.15.66:0/3497280941 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f869c0c5020 con 0x7f86b00b6300 2024-09-10T22:13:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:42.691+0000 7f86afe00640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8690074500 0x7f86900769c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f86a0000f70 tx=0x7f86a0006b70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:43.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.011+0000 7f86b6e00640 1 -- 172.21.15.66:0/3497280941 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f86b0160560 con 0x7f8690074500 2024-09-10T22:13:43.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.379+0000 7f86a7e00640 1 -- 172.21.15.66:0/3497280941 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 44) ==== 99352+0+0 (secure 0 0 0) 0x7f869c0f59a0 con 0x7f86b00b6300 2024-09-10T22:13:43.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.435+0000 7f86a7e00640 1 -- 172.21.15.66:0/3497280941 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86b0160560 con 0x7f8690074500 2024-09-10T22:13:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.439+0000 7f86b6e00640 1 -- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8690074500 msgr2=0x7f86900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.439+0000 7f86b6e00640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8690074500 0x7f86900769c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f86a0000f70 tx=0x7f86a0006b70 comp rx=0 tx=0).stop 2024-09-10T22:13:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.439+0000 7f86b6e00640 1 -- 172.21.15.66:0/3497280941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86b00b6300 msgr2=0x7f86b00b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.439+0000 7f86b6e00640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86b00b6300 0x7f86b00b4a80 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f869c01d0a0 tx=0x7f869c005640 comp rx=0 tx=0).stop 2024-09-10T22:13:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.439+0000 7f86b6e00640 1 -- 172.21.15.66:0/3497280941 shutdown_connections 2024-09-10T22:13:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.439+0000 7f86b6e00640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8690074500 0x7f86900769c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.439+0000 7f86b6e00640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86b00c7e40 0x7f86b00b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.439+0000 7f86b6e00640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86b00b6300 0x7f86b00b4a80 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.439+0000 7f86b6e00640 1 --2- 172.21.15.66:0/3497280941 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8694004810 0x7f86b0155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.439+0000 7f86b6e00640 1 -- 172.21.15.66:0/3497280941 >> 172.21.15.66:0/3497280941 conn(0x7f86b00d1c50 msgr2=0x7f86b00d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.439+0000 7f86b6e00640 1 -- 172.21.15.66:0/3497280941 shutdown_connections 2024-09-10T22:13:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.439+0000 7f86b6e00640 1 -- 172.21.15.66:0/3497280941 wait complete. 2024-09-10T22:13:43.465 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-10T22:13:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.611+0000 7f952b200640 1 -- 172.21.15.66:0/3723757382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f952c15ef20 msgr2=0x7f952c0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.611+0000 7f952b200640 1 --2- 172.21.15.66:0/3723757382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f952c15ef20 0x7f952c0cecc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f951c009870 tx=0x7f951c01c7f0 comp rx=0 tx=0).stop 2024-09-10T22:13:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.611+0000 7f952b200640 1 -- 172.21.15.66:0/3723757382 shutdown_connections 2024-09-10T22:13:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.611+0000 7f952b200640 1 --2- 172.21.15.66:0/3723757382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f952c15ef20 0x7f952c0cecc0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.611+0000 7f952b200640 1 -- 172.21.15.66:0/3723757382 >> 172.21.15.66:0/3723757382 conn(0x7f952c0d1b10 msgr2=0x7f952c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.611+0000 7f952b200640 1 -- 172.21.15.66:0/3723757382 shutdown_connections 2024-09-10T22:13:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.611+0000 7f952b200640 1 -- 172.21.15.66:0/3723757382 wait complete. 2024-09-10T22:13:43.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.611+0000 7f952b200640 1 Processor -- start 2024-09-10T22:13:43.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.611+0000 7f952b200640 1 -- start start 2024-09-10T22:13:43.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f952b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f952c0cd7a0 0x7f952c15bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:43.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f952b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f952c15ef20 0x7f952c15c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:43.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f952b200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f952c15f300 0x7f952c0c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f952b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f952c0cdd30 con 0x7f952c0cd7a0 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f952b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f952c0cdbb0 con 0x7f952c15ef20 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f952b200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f952c0cdeb0 con 0x7f952c15f300 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9529e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f952c0cd7a0 0x7f952c15bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9529e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f952c0cd7a0 0x7f952c15bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39814/0 (socket says 172.21.15.66:39814) 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9529e00640 1 -- 172.21.15.66:0/194529561 learned_addr learned my addr 172.21.15.66:0/194529561 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f952a800640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f952c15f300 0x7f952c0c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9523e00640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f952c15ef20 0x7f952c15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9529e00640 1 -- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f952c15f300 msgr2=0x7f952c0c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9529e00640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f952c15f300 0x7f952c0c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9529e00640 1 -- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f952c15ef20 msgr2=0x7f952c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9529e00640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f952c15ef20 0x7f952c15c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9529e00640 1 -- 172.21.15.66:0/194529561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952c165040 con 0x7f952c0cd7a0 2024-09-10T22:13:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9529e00640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f952c0cd7a0 0x7f952c15bb20 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f951c01ccd0 tx=0x7f951c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9522000640 1 -- 172.21.15.66:0/194529561 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c0b2040 con 0x7f952c0cd7a0 2024-09-10T22:13:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f952b200640 1 -- 172.21.15.66:0/194529561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952c166ae0 con 0x7f952c0cd7a0 2024-09-10T22:13:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9522000640 1 -- 172.21.15.66:0/194529561 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f951c0041f0 con 0x7f952c0cd7a0 2024-09-10T22:13:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9522000640 1 -- 172.21.15.66:0/194529561 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c003600 con 0x7f952c0cd7a0 2024-09-10T22:13:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f952b200640 1 -- 172.21.15.66:0/194529561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f952c0c9e70 con 0x7f952c0cd7a0 2024-09-10T22:13:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f952b200640 1 -- 172.21.15.66:0/194529561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f952c0cecc0 con 0x7f952c0cd7a0 2024-09-10T22:13:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.615+0000 7f9522000640 1 -- 172.21.15.66:0/194529561 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 44) ==== 99352+0+0 (secure 0 0 0) 0x7f951c0b5070 con 0x7f952c0cd7a0 2024-09-10T22:13:43.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.619+0000 7f9522000640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9508074680 0x7f9508076b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:43.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.619+0000 7f9522000640 1 -- 172.21.15.66:0/194529561 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f951c12bf60 con 0x7f952c0cd7a0 2024-09-10T22:13:43.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.619+0000 7f9523e00640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9508074680 0x7f9508076b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:43.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.619+0000 7f9523e00640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9508074680 0x7f9508076b40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f952c15c9c0 tx=0x7f9514009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.623+0000 7f9522000640 1 -- 172.21.15.66:0/194529561 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f951c0c4060 con 0x7f952c0cd7a0 2024-09-10T22:13:43.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:43.827+0000 7f952b200640 1 -- 172.21.15.66:0/194529561 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f952c072e00 con 0x7f9508074680 2024-09-10T22:13:44.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.843+0000 7f9522000640 1 -- 172.21.15.66:0/194529561 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f952c072e00 con 0x7f9508074680 2024-09-10T22:13:44.850 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:13:44.851 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:13:44.851 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "canceled", 2024-09-10T22:13:44.851 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:13:44.851 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:13:44.851 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:13:44.851 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:13:44.851 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:44.851 INFO:teuthology.orchestra.run.smithi066.stdout: "failure": { 2024-09-10T22:13:44.851 INFO:teuthology.orchestra.run.smithi066.stdout: "errno": "4", 2024-09-10T22:13:44.851 INFO:teuthology.orchestra.run.smithi066.stdout: "error_msg": "user interrupted clone operation" 2024-09-10T22:13:44.851 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:44.851 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:44.852 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:13:44.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.847+0000 7f952b200640 1 -- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9508074680 msgr2=0x7f9508076b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:44.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.847+0000 7f952b200640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9508074680 0x7f9508076b40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f952c15c9c0 tx=0x7f9514009040 comp rx=0 tx=0).stop 2024-09-10T22:13:44.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.847+0000 7f952b200640 1 -- 172.21.15.66:0/194529561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f952c0cd7a0 msgr2=0x7f952c15bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:44.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.847+0000 7f952b200640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f952c0cd7a0 0x7f952c15bb20 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f951c01ccd0 tx=0x7f951c005db0 comp rx=0 tx=0).stop 2024-09-10T22:13:44.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.847+0000 7f952b200640 1 -- 172.21.15.66:0/194529561 shutdown_connections 2024-09-10T22:13:44.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.847+0000 7f952b200640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9508074680 0x7f9508076b40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:44.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.847+0000 7f952b200640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f952c15f300 0x7f952c0c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:44.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.847+0000 7f952b200640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f952c15ef20 0x7f952c15c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:44.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.847+0000 7f952b200640 1 --2- 172.21.15.66:0/194529561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f952c0cd7a0 0x7f952c15bb20 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:44.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.847+0000 7f952b200640 1 -- 172.21.15.66:0/194529561 >> 172.21.15.66:0/194529561 conn(0x7f952c0d1b10 msgr2=0x7f952c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:44.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.847+0000 7f952b200640 1 -- 172.21.15.66:0/194529561 shutdown_connections 2024-09-10T22:13:44.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:44.847+0000 7f952b200640 1 -- 172.21.15.66:0/194529561 wait complete. 2024-09-10T22:13:44.877 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-10T22:13:44.877 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:13:45.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.027+0000 7f298f400640 1 -- 172.21.15.66:0/3224548512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2970003ab0 msgr2=0x7f2970003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.027+0000 7f298f400640 1 --2- 172.21.15.66:0/3224548512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2970003ab0 0x7f2970003ed0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f297c009b60 tx=0x7f297c01d270 comp rx=0 tx=0).stop 2024-09-10T22:13:45.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.027+0000 7f298f400640 1 -- 172.21.15.66:0/3224548512 shutdown_connections 2024-09-10T22:13:45.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.027+0000 7f298f400640 1 --2- 172.21.15.66:0/3224548512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2970003ab0 0x7f2970003ed0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.027+0000 7f298f400640 1 -- 172.21.15.66:0/3224548512 >> 172.21.15.66:0/3224548512 conn(0x7f29880cfa30 msgr2=0x7f29880cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:45.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.027+0000 7f298f400640 1 -- 172.21.15.66:0/3224548512 shutdown_connections 2024-09-10T22:13:45.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.027+0000 7f298f400640 1 -- 172.21.15.66:0/3224548512 wait complete. 2024-09-10T22:13:45.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.027+0000 7f298f400640 1 Processor -- start 2024-09-10T22:13:45.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.027+0000 7f298f400640 1 -- start start 2024-09-10T22:13:45.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2970003ab0 0x7f29880c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880b3350 0x7f2988073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298f400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29880b4670 0x7f29881490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29880ca410 con 0x7f2970003ab0 2024-09-10T22:13:45.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298f400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29880ca290 con 0x7f29880b3350 2024-09-10T22:13:45.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298f400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f29880ae710 con 0x7f29880b4670 2024-09-10T22:13:45.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2970003ab0 0x7f29880c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2970003ab0 0x7f29880c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39826/0 (socket says 172.21.15.66:39826) 2024-09-10T22:13:45.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298ca00640 1 -- 172.21.15.66:0/2740212974 learned_addr learned my addr 172.21.15.66:0/2740212974 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:45.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298ca00640 1 -- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29880b4670 msgr2=0x7f29881490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298d400640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29880b4670 0x7f29881490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f2987e00640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880b3350 0x7f2988073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298ca00640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29880b4670 0x7f29881490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298ca00640 1 -- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880b3350 msgr2=0x7f2988073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298ca00640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880b3350 0x7f2988073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298ca00640 1 -- 172.21.15.66:0/2740212974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29880b38b0 con 0x7f2970003ab0 2024-09-10T22:13:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298d400640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29880b4670 0x7f29881490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:13:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f2987e00640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880b3350 0x7f2988073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:13:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298ca00640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2970003ab0 0x7f29880c7860 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f297c0af040 tx=0x7f297c0055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f2984c00640 1 -- 172.21.15.66:0/2740212974 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0b1040 con 0x7f2970003ab0 2024-09-10T22:13:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f2984c00640 1 -- 172.21.15.66:0/2740212974 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f297c0bb070 con 0x7f2970003ab0 2024-09-10T22:13:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298f400640 1 -- 172.21.15.66:0/2740212974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29880b3b90 con 0x7f2970003ab0 2024-09-10T22:13:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f2984c00640 1 -- 172.21.15.66:0/2740212974 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c004290 con 0x7f2970003ab0 2024-09-10T22:13:45.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298f400640 1 -- 172.21.15.66:0/2740212974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f298815a060 con 0x7f2970003ab0 2024-09-10T22:13:45.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f298f400640 1 -- 172.21.15.66:0/2740212974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2988070470 con 0x7f2970003ab0 2024-09-10T22:13:45.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.031+0000 7f2984c00640 1 -- 172.21.15.66:0/2740212974 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 44) ==== 99352+0+0 (secure 0 0 0) 0x7f297c0046d0 con 0x7f2970003ab0 2024-09-10T22:13:45.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.035+0000 7f2984c00640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29680746b0 0x7f2968076b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.035+0000 7f2984c00640 1 -- 172.21.15.66:0/2740212974 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f297c12d110 con 0x7f2970003ab0 2024-09-10T22:13:45.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.035+0000 7f2987e00640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29680746b0 0x7f2968076b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.035+0000 7f2987e00640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29680746b0 0x7f2968076b70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f29880afe70 tx=0x7f2974006730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:45.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.039+0000 7f2984c00640 1 -- 172.21.15.66:0/2740212974 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f297c016f80 con 0x7f2970003ab0 2024-09-10T22:13:45.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.275+0000 7f298f400640 1 -- 172.21.15.66:0/2740212974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2988155970 con 0x7f2970003ab0 2024-09-10T22:13:45.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.275+0000 7f2984c00640 1 -- 172.21.15.66:0/2740212974 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1915 (secure 0 0 0) 0x7f297c0fa1d0 con 0x7f2970003ab0 2024-09-10T22:13:45.281 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:45.282 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:13:45.282 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:13:45.282 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:13:45.282 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:13:45.282 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:13:45.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:13:45.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:45.282 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:13:45.282 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:13:45.282 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:13:45.283 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:13:45.283 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:13:45.283 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:13:45.283 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:13:45.283 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:13:45.283 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:13:45.283 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:13:45.283 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:13:45.283 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 818, 2024-09-10T22:13:45.283 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:13:45.284 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:13:45.284 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:13:45.284 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:13:45.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:45.284 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:13:45.284 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 128, 2024-09-10T22:13:45.284 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:13:45.284 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:13:45.284 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:13:45.284 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:13:45.284 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:13:45.284 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 799, 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 3234666074, 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 7554863104, 2024-09-10T22:13:45.285 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 765539250176, 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "read_bytes_sec": 2795594, 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 64300888, 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 0, 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 18 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 87, 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:13:12:423173+0000", 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 9, 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:13:45.286 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 9, 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "b", 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 8811 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:13:45.287 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 22, 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:12:52.579565+0000", 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": {} 2024-09-10T22:13:45.288 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:13:45.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.279+0000 7f298f400640 1 -- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29680746b0 msgr2=0x7f2968076b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.279+0000 7f298f400640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29680746b0 0x7f2968076b70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f29880afe70 tx=0x7f2974006730 comp rx=0 tx=0).stop 2024-09-10T22:13:45.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.279+0000 7f298f400640 1 -- 172.21.15.66:0/2740212974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2970003ab0 msgr2=0x7f29880c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.279+0000 7f298f400640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2970003ab0 0x7f29880c7860 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f297c0af040 tx=0x7f297c0055d0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.279+0000 7f298f400640 1 -- 172.21.15.66:0/2740212974 shutdown_connections 2024-09-10T22:13:45.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.279+0000 7f298f400640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f29680746b0 0x7f2968076b70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.279+0000 7f298f400640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29880b4670 0x7f29881490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.279+0000 7f298f400640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880b3350 0x7f2988073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.279+0000 7f298f400640 1 --2- 172.21.15.66:0/2740212974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2970003ab0 0x7f29880c7860 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.279+0000 7f298f400640 1 -- 172.21.15.66:0/2740212974 >> 172.21.15.66:0/2740212974 conn(0x7f29880cfa30 msgr2=0x7f29880cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:45.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.279+0000 7f298f400640 1 -- 172.21.15.66:0/2740212974 shutdown_connections 2024-09-10T22:13:45.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.279+0000 7f298f400640 1 -- 172.21.15.66:0/2740212974 wait complete. 2024-09-10T22:13:45.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:45.303 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/ss1clone1/a45ef0e0-341d-419a-a988-a57967b39d17 2024-09-10T22:13:45.303 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:13:45.318 INFO:teuthology.orchestra.run.smithi066.stderr:ls: cannot access 'volumes/_nogroup/ss1clone1/a45ef0e0-341d-419a-a988-a57967b39d17': No such file or directory 2024-09-10T22:13:45.319 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T22:13:45.319 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:13:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.503+0000 7f760f200640 1 -- 172.21.15.66:0/2962065517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76100d0bf0 msgr2=0x7f76100ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.503+0000 7f760f200640 1 --2- 172.21.15.66:0/2962065517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76100d0bf0 0x7f76100ccf30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7608009c80 tx=0x7f760801cce0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.503+0000 7f760f200640 1 -- 172.21.15.66:0/2962065517 shutdown_connections 2024-09-10T22:13:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.503+0000 7f760f200640 1 --2- 172.21.15.66:0/2962065517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76100d0bf0 0x7f76100ccf30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.503+0000 7f760f200640 1 --2- 172.21.15.66:0/2962065517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f761015a6c0 0x7f76100d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.503+0000 7f760f200640 1 -- 172.21.15.66:0/2962065517 >> 172.21.15.66:0/2962065517 conn(0x7f7610074060 msgr2=0x7f7610074470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:45.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.503+0000 7f760f200640 1 -- 172.21.15.66:0/2962065517 shutdown_connections 2024-09-10T22:13:45.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.503+0000 7f760f200640 1 -- 172.21.15.66:0/2962065517 wait complete. 2024-09-10T22:13:45.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760f200640 1 Processor -- start 2024-09-10T22:13:45.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760f200640 1 -- start start 2024-09-10T22:13:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76100ccb50 0x7f76101f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760f200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76100d0bf0 0x7f76101f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f761015a6c0 0x7f761014b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7610154520 con 0x7f76100ccb50 2024-09-10T22:13:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76100ad9a0 con 0x7f761015a6c0 2024-09-10T22:13:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760f200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f76101546a0 con 0x7f76100d0bf0 2024-09-10T22:13:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76100ccb50 0x7f76101f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76100ccb50 0x7f76101f7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39834/0 (socket says 172.21.15.66:39834) 2024-09-10T22:13:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760de00640 1 -- 172.21.15.66:0/3917809096 learned_addr learned my addr 172.21.15.66:0/3917809096 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f7605400640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76100d0bf0 0x7f76101f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760e800640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f761015a6c0 0x7f761014b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760de00640 1 -- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76100d0bf0 msgr2=0x7f76101f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760de00640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76100d0bf0 0x7f76101f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760de00640 1 -- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f761015a6c0 msgr2=0x7f761014b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760de00640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f761015a6c0 0x7f761014b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760de00640 1 -- 172.21.15.66:0/3917809096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76101f8a10 con 0x7f76100ccb50 2024-09-10T22:13:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760de00640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76100ccb50 0x7f76101f7aa0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f760000c500 tx=0x7f760000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f7606a00640 1 -- 172.21.15.66:0/3917809096 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600017070 con 0x7f76100ccb50 2024-09-10T22:13:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f7606a00640 1 -- 172.21.15.66:0/3917809096 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76000053c0 con 0x7f76100ccb50 2024-09-10T22:13:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760f200640 1 -- 172.21.15.66:0/3917809096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76100b3730 con 0x7f76100ccb50 2024-09-10T22:13:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f7606a00640 1 -- 172.21.15.66:0/3917809096 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600003860 con 0x7f76100ccb50 2024-09-10T22:13:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.507+0000 7f760f200640 1 -- 172.21.15.66:0/3917809096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76100b3cf0 con 0x7f76100ccb50 2024-09-10T22:13:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.511+0000 7f760f200640 1 -- 172.21.15.66:0/3917809096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76100ccf30 con 0x7f76100ccb50 2024-09-10T22:13:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.511+0000 7f7606a00640 1 -- 172.21.15.66:0/3917809096 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 44) ==== 99352+0+0 (secure 0 0 0) 0x7f7600003a00 con 0x7f76100ccb50 2024-09-10T22:13:45.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.511+0000 7f7606a00640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75ec0746b0 0x7f75ec076b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.511+0000 7f7606a00640 1 -- 172.21.15.66:0/3917809096 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f7600091a70 con 0x7f76100ccb50 2024-09-10T22:13:45.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.515+0000 7f7605400640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75ec0746b0 0x7f75ec076b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.515+0000 7f7605400640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75ec0746b0 0x7f75ec076b70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f76100d0990 tx=0x7f75f80091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.515+0000 7f7606a00640 1 -- 172.21.15.66:0/3917809096 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f760005ebd0 con 0x7f76100ccb50 2024-09-10T22:13:45.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.711+0000 7f760f200640 1 -- 172.21.15.66:0/3917809096 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7610074e30 con 0x7f75ec0746b0 2024-09-10T22:13:45.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f7606a00640 1 -- 172.21.15.66:0/3917809096 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7610074e30 con 0x7f75ec0746b0 2024-09-10T22:13:45.719 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:13:45.719 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:45.719 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:13:45.719 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:45.720 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:13:45.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f760f200640 1 -- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75ec0746b0 msgr2=0x7f75ec076b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f760f200640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75ec0746b0 0x7f75ec076b70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f76100d0990 tx=0x7f75f80091e0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f760f200640 1 -- 172.21.15.66:0/3917809096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76100ccb50 msgr2=0x7f76101f7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f760f200640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76100ccb50 0x7f76101f7aa0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f760000c500 tx=0x7f760000c9c0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f760f200640 1 -- 172.21.15.66:0/3917809096 shutdown_connections 2024-09-10T22:13:45.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f760f200640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f75ec0746b0 0x7f75ec076b70 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f760f200640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f761015a6c0 0x7f761014b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f760f200640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76100d0bf0 0x7f76101f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f760f200640 1 --2- 172.21.15.66:0/3917809096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76100ccb50 0x7f76101f7aa0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f760f200640 1 -- 172.21.15.66:0/3917809096 >> 172.21.15.66:0/3917809096 conn(0x7f7610074060 msgr2=0x7f7610070450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:45.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f760f200640 1 -- 172.21.15.66:0/3917809096 shutdown_connections 2024-09-10T22:13:45.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.715+0000 7f760f200640 1 -- 172.21.15.66:0/3917809096 wait complete. 2024-09-10T22:13:45.741 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-10T22:13:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 -- 172.21.15.66:0/2913396855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0500156430 msgr2=0x7f050006d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 --2- 172.21.15.66:0/2913396855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0500156430 0x7f050006d630 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f001c850 comp rx=0 tx=0).stop 2024-09-10T22:13:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 -- 172.21.15.66:0/2913396855 shutdown_connections 2024-09-10T22:13:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 --2- 172.21.15.66:0/2913396855 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0500160370 0x7f05000cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 --2- 172.21.15.66:0/2913396855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0500156430 0x7f050006d630 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 -- 172.21.15.66:0/2913396855 >> 172.21.15.66:0/2913396855 conn(0x7f05000d0b30 msgr2=0x7f05000d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:45.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 -- 172.21.15.66:0/2913396855 shutdown_connections 2024-09-10T22:13:45.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 -- 172.21.15.66:0/2913396855 wait complete. 2024-09-10T22:13:45.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 Processor -- start 2024-09-10T22:13:45.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 -- start start 2024-09-10T22:13:45.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0500070b20 0x7f0500073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0500156430 0x7f050014b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0500160370 0x7f0500164d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0500168d10 con 0x7f0500160370 2024-09-10T22:13:45.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0500168b90 con 0x7f0500070b20 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f0505200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0500168e90 con 0x7f0500156430 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f04fe800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0500160370 0x7f0500164d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f04fe800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0500160370 0x7f0500164d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39846/0 (socket says 172.21.15.66:39846) 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.887+0000 7f04fe800640 1 -- 172.21.15.66:0/1606786520 learned_addr learned my addr 172.21.15.66:0/1606786520 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f04fde00640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0500070b20 0x7f0500073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f04f7e00640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0500156430 0x7f050014b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f04fe800640 1 -- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0500156430 msgr2=0x7f050014b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f04fe800640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0500156430 0x7f050014b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f04fe800640 1 -- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0500070b20 msgr2=0x7f0500073ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f04fe800640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0500070b20 0x7f0500073ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f04fe800640 1 -- 172.21.15.66:0/1606786520 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0500154060 con 0x7f0500160370 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f04fe800640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0500160370 0x7f0500164d30 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f04f800c7c0 tx=0x7f04f800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:45.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f04f6000640 1 -- 172.21.15.66:0/1606786520 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8003b80 con 0x7f0500160370 2024-09-10T22:13:45.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f04f6000640 1 -- 172.21.15.66:0/1606786520 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04f8003d20 con 0x7f0500160370 2024-09-10T22:13:45.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f0505200640 1 -- 172.21.15.66:0/1606786520 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0500154340 con 0x7f0500160370 2024-09-10T22:13:45.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f04f6000640 1 -- 172.21.15.66:0/1606786520 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f800aab0 con 0x7f0500160370 2024-09-10T22:13:45.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f0505200640 1 -- 172.21.15.66:0/1606786520 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0500154900 con 0x7f0500160370 2024-09-10T22:13:45.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f0505200640 1 -- 172.21.15.66:0/1606786520 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f050006d630 con 0x7f0500160370 2024-09-10T22:13:45.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.891+0000 7f04f6000640 1 -- 172.21.15.66:0/1606786520 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 44) ==== 99352+0+0 (secure 0 0 0) 0x7f04f800ac50 con 0x7f0500160370 2024-09-10T22:13:45.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.895+0000 7f04f6000640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04dc074680 0x7f04dc076b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.895+0000 7f04f6000640 1 -- 172.21.15.66:0/1606786520 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f04f8092060 con 0x7f0500160370 2024-09-10T22:13:45.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.895+0000 7f04fde00640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04dc074680 0x7f04dc076b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.895+0000 7f04fde00640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04dc074680 0x7f04dc076b40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f04f00af000 tx=0x7f04f001b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:45.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:45.899+0000 7f04f6000640 1 -- 172.21.15.66:0/1606786520 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f04f805ebf0 con 0x7f0500160370 2024-09-10T22:13:46.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.099+0000 7f0505200640 1 -- 172.21.15.66:0/1606786520 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0500154ad0 con 0x7f04dc074680 2024-09-10T22:13:46.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f04f6000640 1 -- 172.21.15.66:0/1606786520 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f0500154ad0 con 0x7f04dc074680 2024-09-10T22:13:46.110 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:46.110 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:13:46.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:46.110 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sv1" 2024-09-10T22:13:46.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:13:46.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:46.111 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ss1clone1" 2024-09-10T22:13:46.111 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:46.111 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:13:46.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f0505200640 1 -- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04dc074680 msgr2=0x7f04dc076b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f0505200640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04dc074680 0x7f04dc076b40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f04f00af000 tx=0x7f04f001b090 comp rx=0 tx=0).stop 2024-09-10T22:13:46.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f0505200640 1 -- 172.21.15.66:0/1606786520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0500160370 msgr2=0x7f0500164d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f0505200640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0500160370 0x7f0500164d30 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f04f800c7c0 tx=0x7f04f800cc80 comp rx=0 tx=0).stop 2024-09-10T22:13:46.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f0505200640 1 -- 172.21.15.66:0/1606786520 shutdown_connections 2024-09-10T22:13:46.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f0505200640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04dc074680 0x7f04dc076b40 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f0505200640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0500160370 0x7f0500164d30 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f0505200640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0500156430 0x7f050014b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f0505200640 1 --2- 172.21.15.66:0/1606786520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0500070b20 0x7f0500073ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f0505200640 1 -- 172.21.15.66:0/1606786520 >> 172.21.15.66:0/1606786520 conn(0x7f05000d0b30 msgr2=0x7f050015b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:46.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f0505200640 1 -- 172.21.15.66:0/1606786520 shutdown_connections 2024-09-10T22:13:46.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.107+0000 7f0505200640 1 -- 172.21.15.66:0/1606786520 wait complete. 2024-09-10T22:13:46.132 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-10T22:13:46.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.275+0000 7f45cee00640 1 -- 172.21.15.66:0/2133537822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c80b2b10 msgr2=0x7f45c80b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.275+0000 7f45cee00640 1 --2- 172.21.15.66:0/2133537822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c80b2b10 0x7f45c80b1b60 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009840 tx=0x7f45b001c7c0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.275+0000 7f45cee00640 1 -- 172.21.15.66:0/2133537822 shutdown_connections 2024-09-10T22:13:46.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.275+0000 7f45cee00640 1 --2- 172.21.15.66:0/2133537822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c80b2b10 0x7f45c80b1b60 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.275+0000 7f45cee00640 1 --2- 172.21.15.66:0/2133537822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45c81605c0 0x7f45c80b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.275+0000 7f45cee00640 1 -- 172.21.15.66:0/2133537822 >> 172.21.15.66:0/2133537822 conn(0x7f45c80d1ce0 msgr2=0x7f45c80d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cee00640 1 -- 172.21.15.66:0/2133537822 shutdown_connections 2024-09-10T22:13:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cee00640 1 -- 172.21.15.66:0/2133537822 wait complete. 2024-09-10T22:13:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cee00640 1 Processor -- start 2024-09-10T22:13:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cee00640 1 -- start start 2024-09-10T22:13:46.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cee00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45c80b2b10 0x7f45c81f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:46.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45c81605c0 0x7f45c81fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8162c80 0x7f45c8156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45c8168fb0 con 0x7f45c8162c80 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f45c80b1400 con 0x7f45c81605c0 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cee00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f45c8169130 con 0x7f45c80b2b10 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8162c80 0x7f45c8156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8162c80 0x7f45c8156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39856/0 (socket says 172.21.15.66:39856) 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cce00640 1 -- 172.21.15.66:0/157882178 learned_addr learned my addr 172.21.15.66:0/157882178 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45bfe00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45c81605c0 0x7f45c81fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45c7e00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45c80b2b10 0x7f45c81f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cce00640 1 -- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45c80b2b10 msgr2=0x7f45c81f86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cce00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45c80b2b10 0x7f45c81f86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cce00640 1 -- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45c81605c0 msgr2=0x7f45c81fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cce00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45c81605c0 0x7f45c81fbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cce00640 1 -- 172.21.15.66:0/157882178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c806e910 con 0x7f45c8162c80 2024-09-10T22:13:46.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45cce00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8162c80 0x7f45c8156340 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f45c000cae0 tx=0x7f45c000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:46.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45c4c00640 1 -- 172.21.15.66:0/157882178 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0016020 con 0x7f45c8162c80 2024-09-10T22:13:46.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45c4c00640 1 -- 172.21.15.66:0/157882178 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45c000ab40 con 0x7f45c8162c80 2024-09-10T22:13:46.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.279+0000 7f45c4c00640 1 -- 172.21.15.66:0/157882178 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c000ce20 con 0x7f45c8162c80 2024-09-10T22:13:46.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.283+0000 7f45cee00640 1 -- 172.21.15.66:0/157882178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c806ebf0 con 0x7f45c8162c80 2024-09-10T22:13:46.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.283+0000 7f45cee00640 1 -- 172.21.15.66:0/157882178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c81fb390 con 0x7f45c8162c80 2024-09-10T22:13:46.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.283+0000 7f45cee00640 1 -- 172.21.15.66:0/157882178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c80b1620 con 0x7f45c8162c80 2024-09-10T22:13:46.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.283+0000 7f45c4c00640 1 -- 172.21.15.66:0/157882178 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 44) ==== 99352+0+0 (secure 0 0 0) 0x7f45c001b400 con 0x7f45c8162c80 2024-09-10T22:13:46.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.287+0000 7f45c4c00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45a4074680 0x7f45a4076b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:46.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.287+0000 7f45c4c00640 1 -- 172.21.15.66:0/157882178 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f45c0092c10 con 0x7f45c8162c80 2024-09-10T22:13:46.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.287+0000 7f45c7e00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45a4074680 0x7f45a4076b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:46.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.287+0000 7f45c7e00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45a4074680 0x7f45a4076b40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f45b800bc00 tx=0x7f45b800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.287+0000 7f45c4c00640 1 -- 172.21.15.66:0/157882178 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f45c0020030 con 0x7f45c8162c80 2024-09-10T22:13:46.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.491+0000 7f45cee00640 1 -- 172.21.15.66:0/157882178 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45c8166570 con 0x7f45a4074680 2024-09-10T22:13:46.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.507+0000 7f45c4c00640 1 -- 172.21.15.66:0/157882178 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f45c8166570 con 0x7f45a4074680 2024-09-10T22:13:46.513 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:46.513 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:13:46.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:13:46.513 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ss1" 2024-09-10T22:13:46.513 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:13:46.513 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:13:46.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.511+0000 7f45cee00640 1 -- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45a4074680 msgr2=0x7f45a4076b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.511+0000 7f45cee00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45a4074680 0x7f45a4076b40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f45b800bc00 tx=0x7f45b800b390 comp rx=0 tx=0).stop 2024-09-10T22:13:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.511+0000 7f45cee00640 1 -- 172.21.15.66:0/157882178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8162c80 msgr2=0x7f45c8156340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.511+0000 7f45cee00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8162c80 0x7f45c8156340 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f45c000cae0 tx=0x7f45c000cbd0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.511+0000 7f45cee00640 1 -- 172.21.15.66:0/157882178 shutdown_connections 2024-09-10T22:13:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.511+0000 7f45cee00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f45a4074680 0x7f45a4076b40 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.511+0000 7f45cee00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45c8162c80 0x7f45c8156340 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.511+0000 7f45cee00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45c81605c0 0x7f45c81fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.511+0000 7f45cee00640 1 --2- 172.21.15.66:0/157882178 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f45c80b2b10 0x7f45c81f86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.511+0000 7f45cee00640 1 -- 172.21.15.66:0/157882178 >> 172.21.15.66:0/157882178 conn(0x7f45c80d1ce0 msgr2=0x7f45c8162380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.511+0000 7f45cee00640 1 -- 172.21.15.66:0/157882178 shutdown_connections 2024-09-10T22:13:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.511+0000 7f45cee00640 1 -- 172.21.15.66:0/157882178 wait complete. 2024-09-10T22:13:46.534 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-10T22:13:46.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 -- 172.21.15.66:0/958949339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f851015ada0 msgr2=0x7f85100caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 --2- 172.21.15.66:0/958949339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f851015ada0 0x7f85100caa90 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f85100c3e00 tx=0x7f850001c8c0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 -- 172.21.15.66:0/958949339 shutdown_connections 2024-09-10T22:13:46.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 --2- 172.21.15.66:0/958949339 >> v2:172.21.15.94:3300/0 conn(0x7f85100ce940 0x7f85100b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 --2- 172.21.15.66:0/958949339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f851015ada0 0x7f85100caa90 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 -- 172.21.15.66:0/958949339 >> 172.21.15.66:0/958949339 conn(0x7f8510074240 msgr2=0x7f8510074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:46.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 -- 172.21.15.66:0/958949339 shutdown_connections 2024-09-10T22:13:46.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 -- 172.21.15.66:0/958949339 wait complete. 2024-09-10T22:13:46.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 Processor -- start 2024-09-10T22:13:46.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 -- start start 2024-09-10T22:13:46.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85100cbb40 0x7f85100c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:46.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85100ce940 0x7f8510154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:46.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f851015ada0 0x7f85100ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:46.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8510167b90 con 0x7f85100cbb40 2024-09-10T22:13:46.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8510167a10 con 0x7f85100ce940 2024-09-10T22:13:46.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8514e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8510167d10 con 0x7f851015ada0 2024-09-10T22:13:46.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f850de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85100cbb40 0x7f85100c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:46.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f850de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85100cbb40 0x7f85100c7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39874/0 (socket says 172.21.15.66:39874) 2024-09-10T22:13:46.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f850de00640 1 -- 172.21.15.66:0/3190693697 learned_addr learned my addr 172.21.15.66:0/3190693697 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:46.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f8507e00640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85100ce940 0x7f8510154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:46.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f850e800640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f851015ada0 0x7f85100ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:46.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f850de00640 1 -- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f851015ada0 msgr2=0x7f85100ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f850de00640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f851015ada0 0x7f85100ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f850de00640 1 -- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85100ce940 msgr2=0x7f8510154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f850de00640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85100ce940 0x7f8510154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f850de00640 1 -- 172.21.15.66:0/3190693697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85100c8480 con 0x7f85100cbb40 2024-09-10T22:13:46.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.683+0000 7f850de00640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85100cbb40 0x7f85100c7da0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f85000b0040 tx=0x7f85000a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:46.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.687+0000 7f8506000640 1 -- 172.21.15.66:0/3190693697 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85000b2040 con 0x7f85100cbb40 2024-09-10T22:13:46.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.687+0000 7f8514e00640 1 -- 172.21.15.66:0/3190693697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8510147430 con 0x7f85100cbb40 2024-09-10T22:13:46.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.687+0000 7f8506000640 1 -- 172.21.15.66:0/3190693697 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8500004340 con 0x7f85100cbb40 2024-09-10T22:13:46.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.687+0000 7f8506000640 1 -- 172.21.15.66:0/3190693697 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500003860 con 0x7f85100cbb40 2024-09-10T22:13:46.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.687+0000 7f8514e00640 1 -- 172.21.15.66:0/3190693697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85100cace0 con 0x7f85100cbb40 2024-09-10T22:13:46.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.687+0000 7f8514e00640 1 -- 172.21.15.66:0/3190693697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85100b0270 con 0x7f85100cbb40 2024-09-10T22:13:46.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.687+0000 7f8506000640 1 -- 172.21.15.66:0/3190693697 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 44) ==== 99352+0+0 (secure 0 0 0) 0x7f8500003a00 con 0x7f85100cbb40 2024-09-10T22:13:46.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.687+0000 7f8506000640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84ec074680 0x7f84ec076b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:46.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.687+0000 7f8506000640 1 -- 172.21.15.66:0/3190693697 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f850012c4f0 con 0x7f85100cbb40 2024-09-10T22:13:46.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.687+0000 7f8507e00640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84ec074680 0x7f84ec076b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:46.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.691+0000 7f8507e00640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84ec074680 0x7f84ec076b40 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f85101fabd0 tx=0x7f84f8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:46.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.691+0000 7f8506000640 1 -- 172.21.15.66:0/3190693697 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f85000164b0 con 0x7f85100cbb40 2024-09-10T22:13:46.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.899+0000 7f8514e00640 1 -- 172.21.15.66:0/3190693697 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85100af2f0 con 0x7f84ec074680 2024-09-10T22:13:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.927+0000 7f8506000640 1 -- 172.21.15.66:0/3190693697 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f85100af2f0 con 0x7f84ec074680 2024-09-10T22:13:46.932 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:46.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.931+0000 7f8514e00640 1 -- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84ec074680 msgr2=0x7f84ec076b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.931+0000 7f8514e00640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84ec074680 0x7f84ec076b40 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f85101fabd0 tx=0x7f84f8009040 comp rx=0 tx=0).stop 2024-09-10T22:13:46.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.931+0000 7f8514e00640 1 -- 172.21.15.66:0/3190693697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85100cbb40 msgr2=0x7f85100c7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.931+0000 7f8514e00640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85100cbb40 0x7f85100c7da0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f85000b0040 tx=0x7f85000a5690 comp rx=0 tx=0).stop 2024-09-10T22:13:46.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.931+0000 7f8514e00640 1 -- 172.21.15.66:0/3190693697 shutdown_connections 2024-09-10T22:13:46.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.931+0000 7f8514e00640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f84ec074680 0x7f84ec076b40 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.931+0000 7f8514e00640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f851015ada0 0x7f85100ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.931+0000 7f8514e00640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85100ce940 0x7f8510154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.931+0000 7f8514e00640 1 --2- 172.21.15.66:0/3190693697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85100cbb40 0x7f85100c7da0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.931+0000 7f8514e00640 1 -- 172.21.15.66:0/3190693697 >> 172.21.15.66:0/3190693697 conn(0x7f8510074240 msgr2=0x7f85100cd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:46.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.931+0000 7f8514e00640 1 -- 172.21.15.66:0/3190693697 shutdown_connections 2024-09-10T22:13:46.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:46.931+0000 7f8514e00640 1 -- 172.21.15.66:0/3190693697 wait complete. 2024-09-10T22:13:46.954 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-10T22:13:47.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 -- 172.21.15.66:0/1673254729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f795415bfb0 msgr2=0x7f79540ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:47.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 --2- 172.21.15.66:0/1673254729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f795415bfb0 0x7f79540ccd40 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f7944009840 tx=0x7f794401c790 comp rx=0 tx=0).stop 2024-09-10T22:13:47.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 -- 172.21.15.66:0/1673254729 shutdown_connections 2024-09-10T22:13:47.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 --2- 172.21.15.66:0/1673254729 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79540b0590 0x7f79540cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 --2- 172.21.15.66:0/1673254729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f795415bfb0 0x7f79540ccd40 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 -- 172.21.15.66:0/1673254729 >> 172.21.15.66:0/1673254729 conn(0x7f7954074240 msgr2=0x7f7954074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:47.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 -- 172.21.15.66:0/1673254729 shutdown_connections 2024-09-10T22:13:47.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 -- 172.21.15.66:0/1673254729 wait complete. 2024-09-10T22:13:47.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 Processor -- start 2024-09-10T22:13:47.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 -- start start 2024-09-10T22:13:47.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79540b0590 0x7f795415aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:47.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79540cd7c0 0x7f79541f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:47.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f795415bfb0 0x7f79540925b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:47.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7954168e20 con 0x7f79540b0590 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7954168ca0 con 0x7f79540cd7c0 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7958a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7954168fa0 con 0x7f795415bfb0 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7951e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79540b0590 0x7f795415aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7951e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79540b0590 0x7f795415aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39904/0 (socket says 172.21.15.66:39904) 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7951e00640 1 -- 172.21.15.66:0/3956483924 learned_addr learned my addr 172.21.15.66:0/3956483924 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f794be00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79540cd7c0 0x7f79541f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7952800640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f795415bfb0 0x7f79540925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7951e00640 1 -- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f795415bfb0 msgr2=0x7f79540925b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7951e00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f795415bfb0 0x7f79540925b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7951e00640 1 -- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79540cd7c0 msgr2=0x7f79541f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7951e00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79540cd7c0 0x7f79541f5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7951e00640 1 -- 172.21.15.66:0/3956483924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79540978c0 con 0x7f79540b0590 2024-09-10T22:13:47.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f794be00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79540cd7c0 0x7f79541f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:13:47.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7952800640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f795415bfb0 0x7f79540925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:13:47.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.103+0000 7f7951e00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79540b0590 0x7f795415aaf0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f7944009790 tx=0x7f79440a55e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:47.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.107+0000 7f794a000640 1 -- 172.21.15.66:0/3956483924 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79440bb070 con 0x7f79540b0590 2024-09-10T22:13:47.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.107+0000 7f794a000640 1 -- 172.21.15.66:0/3956483924 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7944004470 con 0x7f79540b0590 2024-09-10T22:13:47.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.107+0000 7f7958a00640 1 -- 172.21.15.66:0/3956483924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7954093590 con 0x7f79540b0590 2024-09-10T22:13:47.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.107+0000 7f794a000640 1 -- 172.21.15.66:0/3956483924 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79440b2040 con 0x7f79540b0590 2024-09-10T22:13:47.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.107+0000 7f7958a00640 1 -- 172.21.15.66:0/3956483924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79540b3160 con 0x7f79540b0590 2024-09-10T22:13:47.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.107+0000 7f7958a00640 1 -- 172.21.15.66:0/3956483924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79540b0ae0 con 0x7f79540b0590 2024-09-10T22:13:47.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.107+0000 7f794a000640 1 -- 172.21.15.66:0/3956483924 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 44) ==== 99352+0+0 (secure 0 0 0) 0x7f79440b5030 con 0x7f79540b0590 2024-09-10T22:13:47.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.107+0000 7f794a000640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7930074680 0x7f7930076b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:47.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.107+0000 7f794a000640 1 -- 172.21.15.66:0/3956483924 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f794412b930 con 0x7f79540b0590 2024-09-10T22:13:47.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.107+0000 7f794be00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7930074680 0x7f7930076b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:47.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.111+0000 7f794be00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7930074680 0x7f7930076b40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f795409e490 tx=0x7f793c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:47.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.111+0000 7f794a000640 1 -- 172.21.15.66:0/3956483924 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7944016410 con 0x7f79540b0590 2024-09-10T22:13:47.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.311+0000 7f7958a00640 1 -- 172.21.15.66:0/3956483924 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f79540c38b0 con 0x7f7930074680 2024-09-10T22:13:47.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.331+0000 7f794a000640 1 -- 172.21.15.66:0/3956483924 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f79540c38b0 con 0x7f7930074680 2024-09-10T22:13:47.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.335+0000 7f7958a00640 1 -- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7930074680 msgr2=0x7f7930076b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:47.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.335+0000 7f7958a00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7930074680 0x7f7930076b40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f795409e490 tx=0x7f793c005f90 comp rx=0 tx=0).stop 2024-09-10T22:13:47.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.335+0000 7f7958a00640 1 -- 172.21.15.66:0/3956483924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79540b0590 msgr2=0x7f795415aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:47.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.335+0000 7f7958a00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79540b0590 0x7f795415aaf0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f7944009790 tx=0x7f79440a55e0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.335+0000 7f7958a00640 1 -- 172.21.15.66:0/3956483924 shutdown_connections 2024-09-10T22:13:47.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.335+0000 7f7958a00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7930074680 0x7f7930076b40 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.335+0000 7f7958a00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f795415bfb0 0x7f79540925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.335+0000 7f7958a00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79540cd7c0 0x7f79541f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.335+0000 7f7958a00640 1 --2- 172.21.15.66:0/3956483924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79540b0590 0x7f795415aaf0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.335+0000 7f7958a00640 1 -- 172.21.15.66:0/3956483924 >> 172.21.15.66:0/3956483924 conn(0x7f7954074240 msgr2=0x7f79540ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:47.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.335+0000 7f7958a00640 1 -- 172.21.15.66:0/3956483924 shutdown_connections 2024-09-10T22:13:47.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.335+0000 7f7958a00640 1 -- 172.21.15.66:0/3956483924 wait complete. 2024-09-10T22:13:47.350 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:47.347+0000 7f5feba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:47.350 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:47.347+0000 7f5feba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:47.350 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:47.347+0000 7f5feba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:47.350 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:47.347+0000 7f5feba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:47.350 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:13:47.347+0000 7f5feba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:13:47.359 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-10T22:13:47.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.507+0000 7fca6ae00640 1 -- 172.21.15.66:0/2291472131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640b4d20 msgr2=0x7fca64073e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:47.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.507+0000 7fca6ae00640 1 --2- 172.21.15.66:0/2291472131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640b4d20 0x7fca64073e60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009c80 tx=0x7fca5c01d800 comp rx=0 tx=0).stop 2024-09-10T22:13:47.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.507+0000 7fca6ae00640 1 -- 172.21.15.66:0/2291472131 shutdown_connections 2024-09-10T22:13:47.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.507+0000 7fca6ae00640 1 --2- 172.21.15.66:0/2291472131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640b4d20 0x7fca64073e60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.507+0000 7fca6ae00640 1 --2- 172.21.15.66:0/2291472131 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca6406ff10 0x7fca640b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.507+0000 7fca6ae00640 1 -- 172.21.15.66:0/2291472131 >> 172.21.15.66:0/2291472131 conn(0x7fca640b1e20 msgr2=0x7fca640b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:47.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.507+0000 7fca6ae00640 1 -- 172.21.15.66:0/2291472131 shutdown_connections 2024-09-10T22:13:47.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.507+0000 7fca6ae00640 1 -- 172.21.15.66:0/2291472131 wait complete. 2024-09-10T22:13:47.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.507+0000 7fca6ae00640 1 Processor -- start 2024-09-10T22:13:47.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.507+0000 7fca6ae00640 1 -- start start 2024-09-10T22:13:47.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca6ae00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca6406ff10 0x7fca6414b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:47.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca6ae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640b4d20 0x7fca640af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:47.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca6ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca641518a0 0x7fca6414f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:47.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca6ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fca641679b0 con 0x7fca641518a0 2024-09-10T22:13:47.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca6ae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca64167830 con 0x7fca640b4d20 2024-09-10T22:13:47.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca6ae00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fca64167b30 con 0x7fca6406ff10 2024-09-10T22:13:47.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca68e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca641518a0 0x7fca6414f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:47.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca68e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca641518a0 0x7fca6414f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39918/0 (socket says 172.21.15.66:39918) 2024-09-10T22:13:47.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca68e00640 1 -- 172.21.15.66:0/1145759506 learned_addr learned my addr 172.21.15.66:0/1145759506 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:47.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca63400640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640b4d20 0x7fca640af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:47.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca63e00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca6406ff10 0x7fca6414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca68e00640 1 -- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca6406ff10 msgr2=0x7fca6414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca68e00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca6406ff10 0x7fca6414b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca68e00640 1 -- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640b4d20 msgr2=0x7fca640af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca68e00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640b4d20 0x7fca640af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca68e00640 1 -- 172.21.15.66:0/1145759506 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6406a370 con 0x7fca641518a0 2024-09-10T22:13:47.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca68e00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca641518a0 0x7fca6414f780 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fca5c01a9d0 tx=0x7fca5c0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:47.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca5be00640 1 -- 172.21.15.66:0/1145759506 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0bb070 con 0x7fca641518a0 2024-09-10T22:13:47.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca5be00640 1 -- 172.21.15.66:0/1145759506 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca5c004d40 con 0x7fca641518a0 2024-09-10T22:13:47.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca5be00640 1 -- 172.21.15.66:0/1145759506 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0b2040 con 0x7fca641518a0 2024-09-10T22:13:47.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca6ae00640 1 -- 172.21.15.66:0/1145759506 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca641f7540 con 0x7fca641518a0 2024-09-10T22:13:47.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca6ae00640 1 -- 172.21.15.66:0/1145759506 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca641f7b00 con 0x7fca641518a0 2024-09-10T22:13:47.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca6ae00640 1 -- 172.21.15.66:0/1145759506 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca640b47e0 con 0x7fca641518a0 2024-09-10T22:13:47.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.511+0000 7fca5be00640 1 -- 172.21.15.66:0/1145759506 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 44) ==== 99352+0+0 (secure 0 0 0) 0x7fca5c0b5030 con 0x7fca641518a0 2024-09-10T22:13:47.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.515+0000 7fca5be00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fca44074680 0x7fca44076b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:47.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.515+0000 7fca5be00640 1 -- 172.21.15.66:0/1145759506 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7fca5c12be20 con 0x7fca641518a0 2024-09-10T22:13:47.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.515+0000 7fca63e00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fca44074680 0x7fca44076b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:47.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.515+0000 7fca63e00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fca44074680 0x7fca44076b40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fca54007fd0 tx=0x7fca54009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:47.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.519+0000 7fca5be00640 1 -- 172.21.15.66:0/1145759506 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fca5c016990 con 0x7fca641518a0 2024-09-10T22:13:47.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.719+0000 7fca6ae00640 1 -- 172.21.15.66:0/1145759506 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fca6406dc30 con 0x7fca44074680 2024-09-10T22:13:47.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.735+0000 7fca5be00640 1 -- 172.21.15.66:0/1145759506 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fca6406dc30 con 0x7fca44074680 2024-09-10T22:13:47.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.739+0000 7fca6ae00640 1 -- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fca44074680 msgr2=0x7fca44076b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:47.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.739+0000 7fca6ae00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fca44074680 0x7fca44076b40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fca54007fd0 tx=0x7fca54009040 comp rx=0 tx=0).stop 2024-09-10T22:13:47.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.739+0000 7fca6ae00640 1 -- 172.21.15.66:0/1145759506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca641518a0 msgr2=0x7fca6414f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:47.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.739+0000 7fca6ae00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca641518a0 0x7fca6414f780 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fca5c01a9d0 tx=0x7fca5c0092c0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.739+0000 7fca6ae00640 1 -- 172.21.15.66:0/1145759506 shutdown_connections 2024-09-10T22:13:47.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.739+0000 7fca6ae00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fca44074680 0x7fca44076b40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.739+0000 7fca6ae00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca641518a0 0x7fca6414f780 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.739+0000 7fca6ae00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640b4d20 0x7fca640af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.739+0000 7fca6ae00640 1 --2- 172.21.15.66:0/1145759506 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca6406ff10 0x7fca6414b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:47.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.739+0000 7fca6ae00640 1 -- 172.21.15.66:0/1145759506 >> 172.21.15.66:0/1145759506 conn(0x7fca640b1e20 msgr2=0x7fca64150b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:47.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.739+0000 7fca6ae00640 1 -- 172.21.15.66:0/1145759506 shutdown_connections 2024-09-10T22:13:47.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47.739+0000 7fca6ae00640 1 -- 172.21.15.66:0/1145759506 wait complete. 2024-09-10T22:13:47.764 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T22:13:47.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:13:47.821 INFO:teuthology.orchestra.run.smithi066.stdout:2 2024-09-10T22:13:47.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:13:47.828 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:13:48.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.015+0000 7faa71a00640 1 -- 172.21.15.66:0/1173298574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa6c15bd40 msgr2=0x7faa6c0cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.015+0000 7faa71a00640 1 --2- 172.21.15.66:0/1173298574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa6c15bd40 0x7faa6c0cffa0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7faa6c073290 tx=0x7faa5c01c830 comp rx=0 tx=0).stop 2024-09-10T22:13:48.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.015+0000 7faa71a00640 1 -- 172.21.15.66:0/1173298574 shutdown_connections 2024-09-10T22:13:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.015+0000 7faa71a00640 1 --2- 172.21.15.66:0/1173298574 >> v2:172.21.15.94:3300/0 conn(0x7faa6c0aec10 0x7faa6c0af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.015+0000 7faa71a00640 1 --2- 172.21.15.66:0/1173298574 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa6c0cf830 0x7faa6c0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.015+0000 7faa71a00640 1 --2- 172.21.15.66:0/1173298574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa6c15bd40 0x7faa6c0cffa0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.015+0000 7faa71a00640 1 -- 172.21.15.66:0/1173298574 >> 172.21.15.66:0/1173298574 conn(0x7faa6c0d1c60 msgr2=0x7faa6c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.015+0000 7faa71a00640 1 -- 172.21.15.66:0/1173298574 shutdown_connections 2024-09-10T22:13:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.015+0000 7faa71a00640 1 -- 172.21.15.66:0/1173298574 wait complete. 2024-09-10T22:13:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.015+0000 7faa71a00640 1 Processor -- start 2024-09-10T22:13:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.015+0000 7faa71a00640 1 -- start start 2024-09-10T22:13:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa71a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa6c0aec10 0x7faa6c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa71a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa6c0cf830 0x7faa6c06f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa71a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c15bd40 0x7faa6c06f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa71a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c0cea60 con 0x7faa6c0cf830 2024-09-10T22:13:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa71a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c0ce8e0 con 0x7faa6c15bd40 2024-09-10T22:13:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa71a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c0cebe0 con 0x7faa6c0aec10 2024-09-10T22:13:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa6a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa6c0cf830 0x7faa6c06f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa6a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa6c0cf830 0x7faa6c06f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39938/0 (socket says 172.21.15.66:39938) 2024-09-10T22:13:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa6a000640 1 -- 172.21.15.66:0/1608477674 learned_addr learned my addr 172.21.15.66:0/1608477674 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa6aa00640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa6c0aec10 0x7faa6c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa6b400640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c15bd40 0x7faa6c06f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa6a000640 1 -- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa6c0aec10 msgr2=0x7faa6c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa6a000640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa6c0aec10 0x7faa6c15b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa6a000640 1 -- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c15bd40 msgr2=0x7faa6c06f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa6a000640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c15bd40 0x7faa6c06f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa6a000640 1 -- 172.21.15.66:0/1608477674 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c06ee70 con 0x7faa6c0cf830 2024-09-10T22:13:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa6aa00640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa6c0aec10 0x7faa6c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:13:48.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa6a000640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa6c0cf830 0x7faa6c06f0c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7faa5800c970 tx=0x7faa5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:48.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa62a00640 1 -- 172.21.15.66:0/1608477674 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58007c70 con 0x7faa6c0cf830 2024-09-10T22:13:48.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa62a00640 1 -- 172.21.15.66:0/1608477674 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa5800f450 con 0x7faa6c0cf830 2024-09-10T22:13:48.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa71a00640 1 -- 172.21.15.66:0/1608477674 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c15b6e0 con 0x7faa6c0cf830 2024-09-10T22:13:48.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa62a00640 1 -- 172.21.15.66:0/1608477674 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5800f8a0 con 0x7faa6c0cf830 2024-09-10T22:13:48.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa71a00640 1 -- 172.21.15.66:0/1608477674 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c06e0b0 con 0x7faa6c0cf830 2024-09-10T22:13:48.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.019+0000 7faa71a00640 1 -- 172.21.15.66:0/1608477674 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6c0b4070 con 0x7faa6c0cf830 2024-09-10T22:13:48.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.023+0000 7faa62a00640 1 -- 172.21.15.66:0/1608477674 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 44) ==== 99352+0+0 (secure 0 0 0) 0x7faa58020030 con 0x7faa6c0cf830 2024-09-10T22:13:48.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.023+0000 7faa62a00640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa4c074680 0x7faa4c076b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.023+0000 7faa62a00640 1 -- 172.21.15.66:0/1608477674 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7faa58092790 con 0x7faa6c0cf830 2024-09-10T22:13:48.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.023+0000 7faa6aa00640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa4c074680 0x7faa4c076b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.023+0000 7faa6aa00640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa4c074680 0x7faa4c076b40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0af000 tx=0x7faa5c01b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:48.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.027+0000 7faa62a00640 1 -- 172.21.15.66:0/1608477674 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faa6c0b4070 con 0x7faa6c0cf830 2024-09-10T22:13:48.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.207+0000 7faa71a00640 1 -- 172.21.15.66:0/1608477674 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7faa6c0b0df0 con 0x7faa6c0cf830 2024-09-10T22:13:48.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.207+0000 7faa62a00640 1 -- 172.21.15.66:0/1608477674 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v52)=0 v52) ==== 131+0+0 (secure 0 0 0) 0x7faa58014030 con 0x7faa6c0cf830 2024-09-10T22:13:48.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.211+0000 7faa71a00640 1 -- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa4c074680 msgr2=0x7faa4c076b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.211+0000 7faa71a00640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa4c074680 0x7faa4c076b40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0af000 tx=0x7faa5c01b070 comp rx=0 tx=0).stop 2024-09-10T22:13:48.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.211+0000 7faa71a00640 1 -- 172.21.15.66:0/1608477674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa6c0cf830 msgr2=0x7faa6c06f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.211+0000 7faa71a00640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa6c0cf830 0x7faa6c06f0c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7faa5800c970 tx=0x7faa5800ce30 comp rx=0 tx=0).stop 2024-09-10T22:13:48.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.211+0000 7faa71a00640 1 -- 172.21.15.66:0/1608477674 shutdown_connections 2024-09-10T22:13:48.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.211+0000 7faa71a00640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa4c074680 0x7faa4c076b40 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.211+0000 7faa71a00640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c15bd40 0x7faa6c06f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.211+0000 7faa71a00640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa6c0cf830 0x7faa6c06f0c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.211+0000 7faa71a00640 1 --2- 172.21.15.66:0/1608477674 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa6c0aec10 0x7faa6c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.211+0000 7faa71a00640 1 -- 172.21.15.66:0/1608477674 >> 172.21.15.66:0/1608477674 conn(0x7faa6c0d1c60 msgr2=0x7faa6c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:48.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.211+0000 7faa71a00640 1 -- 172.21.15.66:0/1608477674 shutdown_connections 2024-09-10T22:13:48.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.211+0000 7faa71a00640 1 -- 172.21.15.66:0/1608477674 wait complete. 2024-09-10T22:13:48.235 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:13:48.236 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:13:48.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:13:48.409 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:13:48.409 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:13:48.409 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:13:48.409 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:13:48.418 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:13:48.418 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:13:48.474 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:13:48.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.659+0000 7f9015200640 1 -- 172.21.15.66:0/443215898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90100cd7a0 msgr2=0x7f90100cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.659+0000 7f9015200640 1 --2- 172.21.15.66:0/443215898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90100cd7a0 0x7f90100cdc10 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9008009bf0 tx=0x7f900801cba0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.659+0000 7f9015200640 1 -- 172.21.15.66:0/443215898 shutdown_connections 2024-09-10T22:13:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.659+0000 7f9015200640 1 --2- 172.21.15.66:0/443215898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90100cd7a0 0x7f90100cdc10 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.659+0000 7f9015200640 1 -- 172.21.15.66:0/443215898 >> 172.21.15.66:0/443215898 conn(0x7f90100d1b10 msgr2=0x7f90100d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.659+0000 7f9015200640 1 -- 172.21.15.66:0/443215898 shutdown_connections 2024-09-10T22:13:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.659+0000 7f9015200640 1 -- 172.21.15.66:0/443215898 wait complete. 2024-09-10T22:13:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9015200640 1 Processor -- start 2024-09-10T22:13:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9015200640 1 -- start start 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9015200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90100cd7a0 0x7f901014d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9015200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f901015ef20 0x7f90100b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9015200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f901015f300 0x7f90100b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9015200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f901016a190 con 0x7f901015ef20 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9015200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f901016a010 con 0x7f90100cd7a0 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9015200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f901016a310 con 0x7f901015f300 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9007e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f901015ef20 0x7f90100b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9007e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f901015ef20 0x7f90100b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39952/0 (socket says 172.21.15.66:39952) 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9007e00640 1 -- 172.21.15.66:0/3497477118 learned_addr learned my addr 172.21.15.66:0/3497477118 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f900f400640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90100cd7a0 0x7f901014d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9007e00640 1 -- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f901015f300 msgr2=0x7f90100b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9007e00640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f901015f300 0x7f90100b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9007e00640 1 -- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90100cd7a0 msgr2=0x7f901014d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9007e00640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90100cd7a0 0x7f901014d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9007e00640 1 -- 172.21.15.66:0/3497477118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90101fb4c0 con 0x7f901015ef20 2024-09-10T22:13:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f900f400640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90100cd7a0 0x7f901014d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:13:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9007e00640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f901015ef20 0x7f90100b5990 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c3c0 tx=0x7f8ff800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9007400640 1 -- 172.21.15.66:0/3497477118 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8017070 con 0x7f901015ef20 2024-09-10T22:13:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9007400640 1 -- 172.21.15.66:0/3497477118 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ff80053c0 con 0x7f901015ef20 2024-09-10T22:13:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9015200640 1 -- 172.21.15.66:0/3497477118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90101fb7a0 con 0x7f901015ef20 2024-09-10T22:13:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9007400640 1 -- 172.21.15.66:0/3497477118 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8003860 con 0x7f901015ef20 2024-09-10T22:13:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.663+0000 7f9015200640 1 -- 172.21.15.66:0/3497477118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90101fbd60 con 0x7f901015ef20 2024-09-10T22:13:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.667+0000 7f9015200640 1 -- 172.21.15.66:0/3497477118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f901006f3d0 con 0x7f901015ef20 2024-09-10T22:13:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.667+0000 7f9007400640 1 -- 172.21.15.66:0/3497477118 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 44) ==== 99352+0+0 (secure 0 0 0) 0x7f8ff8003a00 con 0x7f901015ef20 2024-09-10T22:13:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.667+0000 7f9007400640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8fec074680 0x7f8fec076b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.667+0000 7f9007400640 1 -- 172.21.15.66:0/3497477118 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f8ff80919f0 con 0x7f901015ef20 2024-09-10T22:13:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.667+0000 7f900f400640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8fec074680 0x7f8fec076b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.667+0000 7f900f400640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8fec074680 0x7f8fec076b40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9000001230 tx=0x7f9000009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.671+0000 7f9007400640 1 -- 172.21.15.66:0/3497477118 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ff805eab0 con 0x7f901015ef20 2024-09-10T22:13:48.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.855+0000 7f9015200640 1 -- 172.21.15.66:0/3497477118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f90101fc140 con 0x7f901015ef20 2024-09-10T22:13:48.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.855+0000 7f9007400640 1 -- 172.21.15.66:0/3497477118 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v128) ==== 80+0+0 (secure 0 0 0) 0x7f90101fc140 con 0x7f901015ef20 2024-09-10T22:13:48.862 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:13:48.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.859+0000 7f9015200640 1 -- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8fec074680 msgr2=0x7f8fec076b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.859+0000 7f9015200640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8fec074680 0x7f8fec076b40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9000001230 tx=0x7f9000009040 comp rx=0 tx=0).stop 2024-09-10T22:13:48.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.859+0000 7f9015200640 1 -- 172.21.15.66:0/3497477118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f901015ef20 msgr2=0x7f90100b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.859+0000 7f9015200640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f901015ef20 0x7f90100b5990 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c3c0 tx=0x7f8ff800c880 comp rx=0 tx=0).stop 2024-09-10T22:13:48.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.859+0000 7f9015200640 1 -- 172.21.15.66:0/3497477118 shutdown_connections 2024-09-10T22:13:48.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.859+0000 7f9015200640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8fec074680 0x7f8fec076b40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.859+0000 7f9015200640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f901015f300 0x7f90100b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.859+0000 7f9015200640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f901015ef20 0x7f90100b5990 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.859+0000 7f9015200640 1 --2- 172.21.15.66:0/3497477118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90100cd7a0 0x7f901014d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.859+0000 7f9015200640 1 -- 172.21.15.66:0/3497477118 >> 172.21.15.66:0/3497477118 conn(0x7f90100d1b10 msgr2=0x7f901015dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:48.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.859+0000 7f9015200640 1 -- 172.21.15.66:0/3497477118 shutdown_connections 2024-09-10T22:13:48.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:48.859+0000 7f9015200640 1 -- 172.21.15.66:0/3497477118 wait complete. 2024-09-10T22:13:48.884 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-09-10T22:13:48.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:48.884 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T22:13:48.902 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[52005]: fuse finished with error 0 and tester_r 0 2024-09-10T22:13:48.904 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:13:48.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:13:48.904 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:13:49.072 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:13:49.073 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:13:49.073 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:13:49.082 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:13:49.082 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:13:49.082 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:13:49.082 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:13:49.126 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:13:49.126 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:13:49.126 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:13:49.126 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:13:49.137 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:13:49.138 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:13:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.315+0000 7f7142200640 1 -- 172.21.15.66:0/859077294 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f713c0b0ef0 msgr2=0x7f713c0d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.315+0000 7f7142200640 1 --2- 172.21.15.66:0/859077294 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f713c0b0ef0 0x7f713c0d02c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7134009c80 tx=0x7f713401cc80 comp rx=0 tx=0).stop 2024-09-10T22:13:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.315+0000 7f7142200640 1 -- 172.21.15.66:0/859077294 shutdown_connections 2024-09-10T22:13:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.315+0000 7f7142200640 1 --2- 172.21.15.66:0/859077294 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f713c0b0ef0 0x7f713c0d02c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.315+0000 7f7142200640 1 --2- 172.21.15.66:0/859077294 >> v2:172.21.15.94:3300/0 conn(0x7f713c071700 0x7f713c070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.315+0000 7f7142200640 1 -- 172.21.15.66:0/859077294 >> 172.21.15.66:0/859077294 conn(0x7f713c0d1d00 msgr2=0x7f713c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.315+0000 7f7142200640 1 -- 172.21.15.66:0/859077294 shutdown_connections 2024-09-10T22:13:49.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.315+0000 7f7142200640 1 -- 172.21.15.66:0/859077294 wait complete. 2024-09-10T22:13:49.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.315+0000 7f7142200640 1 Processor -- start 2024-09-10T22:13:49.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.315+0000 7f7142200640 1 -- start start 2024-09-10T22:13:49.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f7142200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f713c070ac0 0x7f713c14b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:49.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f7142200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f713c071700 0x7f713c0b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:49.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f7142200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f713c0b0ef0 0x7f713c0c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:49.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f7142200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f713c160b50 con 0x7f713c070ac0 2024-09-10T22:13:49.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f7142200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f713c1609d0 con 0x7f713c0b0ef0 2024-09-10T22:13:49.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f7142200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f713c160cd0 con 0x7f713c071700 2024-09-10T22:13:49.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f713b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f713c070ac0 0x7f713c14b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:49.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f713b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f713c070ac0 0x7f713c14b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39956/0 (socket says 172.21.15.66:39956) 2024-09-10T22:13:49.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f713b400640 1 -- 172.21.15.66:0/3699812366 learned_addr learned my addr 172.21.15.66:0/3699812366 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f7133e00640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f713c071700 0x7f713c0b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f713be00640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f713c0b0ef0 0x7f713c0c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f713b400640 1 -- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f713c071700 msgr2=0x7f713c0b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f713b400640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f713c071700 0x7f713c0b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f713b400640 1 -- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f713c0b0ef0 msgr2=0x7f713c0c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f713b400640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f713c0b0ef0 0x7f713c0c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f713b400640 1 -- 172.21.15.66:0/3699812366 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713c071bf0 con 0x7f713c070ac0 2024-09-10T22:13:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f713be00640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f713c0b0ef0 0x7f713c0c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f713b400640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f713c070ac0 0x7f713c14b0b0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c430 tx=0x7f712c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.319+0000 7f7133400640 1 -- 172.21.15.66:0/3699812366 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c011790 con 0x7f713c070ac0 2024-09-10T22:13:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.323+0000 7f7133400640 1 -- 172.21.15.66:0/3699812366 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f712c017070 con 0x7f713c070ac0 2024-09-10T22:13:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.323+0000 7f7133400640 1 -- 172.21.15.66:0/3699812366 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c011c60 con 0x7f713c070ac0 2024-09-10T22:13:49.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.323+0000 7f7142200640 1 -- 172.21.15.66:0/3699812366 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713c14b5f0 con 0x7f713c070ac0 2024-09-10T22:13:49.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.323+0000 7f7142200640 1 -- 172.21.15.66:0/3699812366 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713c06fd40 con 0x7f713c070ac0 2024-09-10T22:13:49.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.323+0000 7f7142200640 1 -- 172.21.15.66:0/3699812366 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f713c070ea0 con 0x7f713c070ac0 2024-09-10T22:13:49.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.327+0000 7f7133400640 1 -- 172.21.15.66:0/3699812366 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f712c01c460 con 0x7f713c070ac0 2024-09-10T22:13:49.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.327+0000 7f7133400640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7118074710 0x7f7118076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:49.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.327+0000 7f7133400640 1 -- 172.21.15.66:0/3699812366 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f712c0921e0 con 0x7f713c070ac0 2024-09-10T22:13:49.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.331+0000 7f7133e00640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7118074710 0x7f7118076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:49.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.331+0000 7f7133e00640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7118074710 0x7f7118076bd0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7124009600 tx=0x7f71240091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:49.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.335+0000 7f7133400640 1 -- 172.21.15.66:0/3699812366 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f712c05f3a0 con 0x7f713c070ac0 2024-09-10T22:13:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.699+0000 7f7142200640 1 -- 172.21.15.66:0/3699812366 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f713c0c8a90 con 0x7f713c070ac0 2024-09-10T22:13:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.699+0000 7f7133400640 1 -- 172.21.15.66:0/3699812366 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5222 (secure 0 0 0) 0x7f712c05eca0 con 0x7f713c070ac0 2024-09-10T22:13:49.706 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:49.706 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":87,"btime":"2024-09-10T22:13:12:423173+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1352582369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1352582369},{"type":"v1","addr":"172.21.15.94:6837","nonce":1352582369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8805,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/441263936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":441263936},{"type":"v1","addr":"172.21.15.66:6834","nonce":441263936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2554522797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2554522797},{"type":"v1","addr":"172.21.15.66:6835","nonce":2554522797}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:13:03.687866+0000","modified":"2024-09-10T22:13:06.708613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8811":{"gid":8811,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6836/1762705824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1762705824},{"type":"v1","addr":"172.21.15.94:6836","nonce":1762705824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8811,"qdb_cluster":[8811]},"id":9}]} 2024-09-10T22:13:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 87 2024-09-10T22:13:49.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.703+0000 7f7142200640 1 -- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7118074710 msgr2=0x7f7118076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:49.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.703+0000 7f7142200640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7118074710 0x7f7118076bd0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7124009600 tx=0x7f71240091e0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.703+0000 7f7142200640 1 -- 172.21.15.66:0/3699812366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f713c070ac0 msgr2=0x7f713c14b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:49.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.703+0000 7f7142200640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f713c070ac0 0x7f713c14b0b0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c430 tx=0x7f712c00c8f0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.703+0000 7f7142200640 1 -- 172.21.15.66:0/3699812366 shutdown_connections 2024-09-10T22:13:49.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.703+0000 7f7142200640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7118074710 0x7f7118076bd0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.703+0000 7f7142200640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f713c0b0ef0 0x7f713c0c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.703+0000 7f7142200640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f713c071700 0x7f713c0b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.703+0000 7f7142200640 1 --2- 172.21.15.66:0/3699812366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f713c070ac0 0x7f713c14b0b0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.703+0000 7f7142200640 1 -- 172.21.15.66:0/3699812366 >> 172.21.15.66:0/3699812366 conn(0x7f713c0d1d00 msgr2=0x7f713c0b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:49.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.703+0000 7f7142200640 1 -- 172.21.15.66:0/3699812366 shutdown_connections 2024-09-10T22:13:49.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.703+0000 7f7142200640 1 -- 172.21.15.66:0/3699812366 wait complete. 2024-09-10T22:13:49.728 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:13:49.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.867+0000 7f130d200640 1 -- 172.21.15.66:0/2903199288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13080d0260 msgr2=0x7f13080ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.867+0000 7f130d200640 1 --2- 172.21.15.66:0/2903199288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13080d0260 0x7f13080ae960 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f130806d6f0 tx=0x7f12f001c800 comp rx=0 tx=0).stop 2024-09-10T22:13:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.867+0000 7f130d200640 1 -- 172.21.15.66:0/2903199288 shutdown_connections 2024-09-10T22:13:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.867+0000 7f130d200640 1 --2- 172.21.15.66:0/2903199288 >> v2:172.21.15.94:3300/0 conn(0x7f13080b2bf0 0x7f13080aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.867+0000 7f130d200640 1 --2- 172.21.15.66:0/2903199288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13080d0260 0x7f13080ae960 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.867+0000 7f130d200640 1 -- 172.21.15.66:0/2903199288 >> 172.21.15.66:0/2903199288 conn(0x7f13080d1c50 msgr2=0x7f13080d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.867+0000 7f130d200640 1 -- 172.21.15.66:0/2903199288 shutdown_connections 2024-09-10T22:13:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.867+0000 7f130d200640 1 -- 172.21.15.66:0/2903199288 wait complete. 2024-09-10T22:13:49.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f130d200640 1 Processor -- start 2024-09-10T22:13:49.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f130d200640 1 -- start start 2024-09-10T22:13:49.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f130d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13080b2bf0 0x7f1308150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:49.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f130d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13080d0260 0x7f13080ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:49.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f130d200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f130815f340 0x7f13080ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:49.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f130d200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f13081692a0 con 0x7f13080b2bf0 2024-09-10T22:13:49.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f130d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1308169120 con 0x7f13080d0260 2024-09-10T22:13:49.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f130d200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1308169420 con 0x7f130815f340 2024-09-10T22:13:49.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f1305e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13080b2bf0 0x7f1308150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:49.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f1305e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13080b2bf0 0x7f1308150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39976/0 (socket says 172.21.15.66:39976) 2024-09-10T22:13:49.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f1305e00640 1 -- 172.21.15.66:0/1776305178 learned_addr learned my addr 172.21.15.66:0/1776305178 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:49.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f12fd400640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13080d0260 0x7f13080ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:49.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f1306800640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f130815f340 0x7f13080ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:49.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f1305e00640 1 -- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f130815f340 msgr2=0x7f13080ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:49.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f1305e00640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f130815f340 0x7f13080ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f1305e00640 1 -- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13080d0260 msgr2=0x7f13080ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:49.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f1305e00640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13080d0260 0x7f13080ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:49.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f1305e00640 1 -- 172.21.15.66:0/1776305178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1308151400 con 0x7f13080b2bf0 2024-09-10T22:13:49.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f1305e00640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13080b2bf0 0x7f1308150ec0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f12f8001ec0 tx=0x7f12f800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:49.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f12fea00640 1 -- 172.21.15.66:0/1776305178 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8017070 con 0x7f13080b2bf0 2024-09-10T22:13:49.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f12fea00640 1 -- 172.21.15.66:0/1776305178 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12f8003600 con 0x7f13080b2bf0 2024-09-10T22:13:49.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f12fea00640 1 -- 172.21.15.66:0/1776305178 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8003a50 con 0x7f13080b2bf0 2024-09-10T22:13:49.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f130d200640 1 -- 172.21.15.66:0/1776305178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1308151680 con 0x7f13080b2bf0 2024-09-10T22:13:49.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.871+0000 7f130d200640 1 -- 172.21.15.66:0/1776305178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1308151c40 con 0x7f13080b2bf0 2024-09-10T22:13:49.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.875+0000 7f130d200640 1 -- 172.21.15.66:0/1776305178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13080cbca0 con 0x7f13080b2bf0 2024-09-10T22:13:49.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.875+0000 7f12fea00640 1 -- 172.21.15.66:0/1776305178 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f12f8003bf0 con 0x7f13080b2bf0 2024-09-10T22:13:49.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.875+0000 7f12fea00640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12e4074710 0x7f12e4076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:49.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.875+0000 7f12fea00640 1 -- 172.21.15.66:0/1776305178 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5879+0+0 (secure 0 0 0) 0x7f12f8091920 con 0x7f13080b2bf0 2024-09-10T22:13:49.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.875+0000 7f12fd400640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12e4074710 0x7f12e4076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:49.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.875+0000 7f12fd400640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12e4074710 0x7f12e4076bd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f12f00af000 tx=0x7f12f001b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:49.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:49.879+0000 7f12fea00640 1 -- 172.21.15.66:0/1776305178 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f12f809a030 con 0x7f13080b2bf0 2024-09-10T22:13:50.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.091+0000 7f130d200640 1 -- 172.21.15.66:0/1776305178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f130815ccc0 con 0x7f13080b2bf0 2024-09-10T22:13:50.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.091+0000 7f12fea00640 1 -- 172.21.15.66:0/1776305178 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5222 (secure 0 0 0) 0x7f1308151400 con 0x7f13080b2bf0 2024-09-10T22:13:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:50.098 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":87,"btime":"2024-09-10T22:13:12:423173+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1352582369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1352582369},{"type":"v1","addr":"172.21.15.94:6837","nonce":1352582369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8805,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/441263936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":441263936},{"type":"v1","addr":"172.21.15.66:6834","nonce":441263936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2554522797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2554522797},{"type":"v1","addr":"172.21.15.66:6835","nonce":2554522797}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:13:03.687866+0000","modified":"2024-09-10T22:13:06.708613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8811":{"gid":8811,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6836/1762705824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1762705824},{"type":"v1","addr":"172.21.15.94:6836","nonce":1762705824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8811,"qdb_cluster":[8811]},"id":9}]} 2024-09-10T22:13:50.098 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 87 2024-09-10T22:13:50.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.095+0000 7f130d200640 1 -- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12e4074710 msgr2=0x7f12e4076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:50.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.095+0000 7f130d200640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12e4074710 0x7f12e4076bd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f12f00af000 tx=0x7f12f001b040 comp rx=0 tx=0).stop 2024-09-10T22:13:50.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.095+0000 7f130d200640 1 -- 172.21.15.66:0/1776305178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13080b2bf0 msgr2=0x7f1308150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:50.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.095+0000 7f130d200640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13080b2bf0 0x7f1308150ec0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f12f8001ec0 tx=0x7f12f800c890 comp rx=0 tx=0).stop 2024-09-10T22:13:50.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.095+0000 7f130d200640 1 -- 172.21.15.66:0/1776305178 shutdown_connections 2024-09-10T22:13:50.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.095+0000 7f130d200640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f12e4074710 0x7f12e4076bd0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.095+0000 7f130d200640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f130815f340 0x7f13080ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.095+0000 7f130d200640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13080d0260 0x7f13080ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.095+0000 7f130d200640 1 --2- 172.21.15.66:0/1776305178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13080b2bf0 0x7f1308150ec0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.095+0000 7f130d200640 1 -- 172.21.15.66:0/1776305178 >> 172.21.15.66:0/1776305178 conn(0x7f13080d1c50 msgr2=0x7f1308071760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:50.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.095+0000 7f130d200640 1 -- 172.21.15.66:0/1776305178 shutdown_connections 2024-09-10T22:13:50.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.095+0000 7f130d200640 1 -- 172.21.15.66:0/1776305178 wait complete. 2024-09-10T22:13:50.121 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:13:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 -- 172.21.15.66:0/1117299340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f912815e010 msgr2=0x7f91280cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 --2- 172.21.15.66:0/1117299340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f912815e010 0x7f91280cfea0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9118009840 tx=0x7f911801c790 comp rx=0 tx=0).stop 2024-09-10T22:13:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 -- 172.21.15.66:0/1117299340 shutdown_connections 2024-09-10T22:13:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 --2- 172.21.15.66:0/1117299340 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91280d03e0 0x7f91280d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 --2- 172.21.15.66:0/1117299340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f912815e010 0x7f91280cfea0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 -- 172.21.15.66:0/1117299340 >> 172.21.15.66:0/1117299340 conn(0x7f91280d1c60 msgr2=0x7f91280d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 -- 172.21.15.66:0/1117299340 shutdown_connections 2024-09-10T22:13:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 -- 172.21.15.66:0/1117299340 wait complete. 2024-09-10T22:13:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 Processor -- start 2024-09-10T22:13:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 -- start start 2024-09-10T22:13:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91280d03e0 0x7f91281fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91280d0d60 0x7f912814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f912815e010 0x7f91281f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f912816ae90 con 0x7f91280d0d60 2024-09-10T22:13:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f912816ad10 con 0x7f91280d03e0 2024-09-10T22:13:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f912e200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f912816b010 con 0x7f912815e010 2024-09-10T22:13:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f911fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91280d0d60 0x7f912814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f911fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91280d0d60 0x7f912814fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39988/0 (socket says 172.21.15.66:39988) 2024-09-10T22:13:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f911fe00640 1 -- 172.21.15.66:0/3268860981 learned_addr learned my addr 172.21.15.66:0/3268860981 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f9127400640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91280d03e0 0x7f91281fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f9127e00640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f912815e010 0x7f91281f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f911fe00640 1 -- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f912815e010 msgr2=0x7f91281f7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f911fe00640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f912815e010 0x7f91281f7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f911fe00640 1 -- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91280d03e0 msgr2=0x7f91281fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f911fe00640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91280d03e0 0x7f91281fb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f911fe00640 1 -- 172.21.15.66:0/3268860981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91281f8400 con 0x7f91280d0d60 2024-09-10T22:13:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.271+0000 7f9127400640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91280d03e0 0x7f91281fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:13:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.275+0000 7f911fe00640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91280d0d60 0x7f912814fa20 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f911000e9f0 tx=0x7f911000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.275+0000 7f911f400640 1 -- 172.21.15.66:0/3268860981 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911000cd80 con 0x7f91280d0d60 2024-09-10T22:13:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.275+0000 7f911f400640 1 -- 172.21.15.66:0/3268860981 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9110010430 con 0x7f91280d0d60 2024-09-10T22:13:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.275+0000 7f911f400640 1 -- 172.21.15.66:0/3268860981 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110010860 con 0x7f91280d0d60 2024-09-10T22:13:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.275+0000 7f912e200640 1 -- 172.21.15.66:0/3268860981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91281f86e0 con 0x7f91280d0d60 2024-09-10T22:13:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.275+0000 7f912e200640 1 -- 172.21.15.66:0/3268860981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91281fb420 con 0x7f91280d0d60 2024-09-10T22:13:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.275+0000 7f912e200640 1 -- 172.21.15.66:0/3268860981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91280d0820 con 0x7f91280d0d60 2024-09-10T22:13:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.275+0000 7f911f400640 1 -- 172.21.15.66:0/3268860981 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f9110010a00 con 0x7f91280d0d60 2024-09-10T22:13:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.279+0000 7f911f400640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9104074710 0x7f9104076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.279+0000 7f911f400640 1 -- 172.21.15.66:0/3268860981 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5935+0+0 (secure 0 0 0) 0x7f9110092e30 con 0x7f91280d0d60 2024-09-10T22:13:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.279+0000 7f9127400640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9104074710 0x7f9104076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.279+0000 7f9127400640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9104074710 0x7f9104076bd0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f91180af000 tx=0x7f911801afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:50.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.283+0000 7f911f400640 1 -- 172.21.15.66:0/3268860981 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f911005fa80 con 0x7f91280d0d60 2024-09-10T22:13:50.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.471+0000 7f912e200640 1 -- 172.21.15.66:0/3268860981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91280cbef0 con 0x7f91280d0d60 2024-09-10T22:13:50.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.471+0000 7f911f400640 1 -- 172.21.15.66:0/3268860981 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16998 (secure 0 0 0) 0x7f91100644b0 con 0x7f91280d0d60 2024-09-10T22:13:50.476 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:50.477 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":129,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:13:50.004674+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:13:01.405668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T22:13:02.664579+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":129,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.a","mappings":[{"from":2,"to":0}]},{"pgid":"19.31","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:13:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.475+0000 7f912e200640 1 -- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9104074710 msgr2=0x7f9104076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.475+0000 7f912e200640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9104074710 0x7f9104076bd0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f91180af000 tx=0x7f911801afd0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.475+0000 7f912e200640 1 -- 172.21.15.66:0/3268860981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91280d0d60 msgr2=0x7f912814fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.475+0000 7f912e200640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91280d0d60 0x7f912814fa20 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f911000e9f0 tx=0x7f911000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.475+0000 7f912e200640 1 -- 172.21.15.66:0/3268860981 shutdown_connections 2024-09-10T22:13:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.475+0000 7f912e200640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9104074710 0x7f9104076bd0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.475+0000 7f912e200640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f912815e010 0x7f91281f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.475+0000 7f912e200640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91280d0d60 0x7f912814fa20 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.475+0000 7f912e200640 1 --2- 172.21.15.66:0/3268860981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91280d03e0 0x7f91281fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.475+0000 7f912e200640 1 -- 172.21.15.66:0/3268860981 >> 172.21.15.66:0/3268860981 conn(0x7f91280d1c60 msgr2=0x7f91280d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.475+0000 7f912e200640 1 -- 172.21.15.66:0/3268860981 shutdown_connections 2024-09-10T22:13:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.475+0000 7f912e200640 1 -- 172.21.15.66:0/3268860981 wait complete. 2024-09-10T22:13:50.499 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:13:50.499 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:13:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.639+0000 7f63bac00640 1 -- 172.21.15.66:0/2106889629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b415bd40 msgr2=0x7f63b40cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.639+0000 7f63bac00640 1 --2- 172.21.15.66:0/2106889629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b415bd40 0x7f63b40cffa0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f63b40cc2e0 tx=0x7f63a401c770 comp rx=0 tx=0).stop 2024-09-10T22:13:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.639+0000 7f63bac00640 1 -- 172.21.15.66:0/2106889629 shutdown_connections 2024-09-10T22:13:50.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.639+0000 7f63bac00640 1 --2- 172.21.15.66:0/2106889629 >> v2:172.21.15.94:3300/0 conn(0x7f63b40cf180 0x7f63b40d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.639+0000 7f63bac00640 1 --2- 172.21.15.66:0/2106889629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b415bd40 0x7f63b40cffa0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.639+0000 7f63bac00640 1 -- 172.21.15.66:0/2106889629 >> 172.21.15.66:0/2106889629 conn(0x7f63b40d1c60 msgr2=0x7f63b40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:50.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.639+0000 7f63bac00640 1 -- 172.21.15.66:0/2106889629 shutdown_connections 2024-09-10T22:13:50.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.639+0000 7f63bac00640 1 -- 172.21.15.66:0/2106889629 wait complete. 2024-09-10T22:13:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63bac00640 1 Processor -- start 2024-09-10T22:13:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63bac00640 1 -- start start 2024-09-10T22:13:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63bac00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63b40cf180 0x7f63b41648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63bac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b40cf830 0x7f63b4157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63bac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63b415bd40 0x7f63b415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63bac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f63b4168a20 con 0x7f63b40cf830 2024-09-10T22:13:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63bac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f63b40aff20 con 0x7f63b415bd40 2024-09-10T22:13:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63bac00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f63b4168ba0 con 0x7f63b40cf180 2024-09-10T22:13:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63b3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b40cf830 0x7f63b4157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63b8c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63b415bd40 0x7f63b415cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63b3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b40cf830 0x7f63b4157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40002/0 (socket says 172.21.15.66:40002) 2024-09-10T22:13:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63b3400640 1 -- 172.21.15.66:0/942119068 learned_addr learned my addr 172.21.15.66:0/942119068 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63b3e00640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63b40cf180 0x7f63b41648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63b3400640 1 -- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63b40cf180 msgr2=0x7f63b41648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63b3400640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63b40cf180 0x7f63b41648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63b3400640 1 -- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63b415bd40 msgr2=0x7f63b415cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:50.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63b3400640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63b415bd40 0x7f63b415cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63b3400640 1 -- 172.21.15.66:0/942119068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b415c2c0 con 0x7f63b40cf830 2024-09-10T22:13:50.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63b3400640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b40cf830 0x7f63b4157560 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f63a000c4c0 tx=0x7f63a000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:50.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63abe00640 1 -- 172.21.15.66:0/942119068 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0016020 con 0x7f63b40cf830 2024-09-10T22:13:50.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63abe00640 1 -- 172.21.15.66:0/942119068 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63a0012940 con 0x7f63b40cf830 2024-09-10T22:13:50.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63abe00640 1 -- 172.21.15.66:0/942119068 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0012d90 con 0x7f63b40cf830 2024-09-10T22:13:50.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63bac00640 1 -- 172.21.15.66:0/942119068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b41657f0 con 0x7f63b40cf830 2024-09-10T22:13:50.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63bac00640 1 -- 172.21.15.66:0/942119068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b415c560 con 0x7f63b40cf830 2024-09-10T22:13:50.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.643+0000 7f63bac00640 1 -- 172.21.15.66:0/942119068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b40aec10 con 0x7f63b40cf830 2024-09-10T22:13:50.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.647+0000 7f63abe00640 1 -- 172.21.15.66:0/942119068 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f63a001b400 con 0x7f63b40cf830 2024-09-10T22:13:50.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.647+0000 7f63abe00640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6394074710 0x7f6394076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:50.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.647+0000 7f63abe00640 1 -- 172.21.15.66:0/942119068 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5935+0+0 (secure 0 0 0) 0x7f63a0092480 con 0x7f63b40cf830 2024-09-10T22:13:50.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.647+0000 7f63b3e00640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6394074710 0x7f6394076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:50.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.647+0000 7f63b3e00640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6394074710 0x7f6394076bd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f63a40af000 tx=0x7f63a401afb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:50.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.651+0000 7f63abe00640 1 -- 172.21.15.66:0/942119068 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f63a0020020 con 0x7f63b40cf830 2024-09-10T22:13:50.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.859+0000 7f63bac00640 1 -- 172.21.15.66:0/942119068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63b40b0df0 con 0x7f63b40cf830 2024-09-10T22:13:50.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.859+0000 7f63abe00640 1 -- 172.21.15.66:0/942119068 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5222 (secure 0 0 0) 0x7f63a005f0d0 con 0x7f63b40cf830 2024-09-10T22:13:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:50.867 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":87,"btime":"2024-09-10T22:13:12:423173+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1352582369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1352582369},{"type":"v1","addr":"172.21.15.94:6837","nonce":1352582369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8805,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/441263936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":441263936},{"type":"v1","addr":"172.21.15.66:6834","nonce":441263936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2554522797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2554522797},{"type":"v1","addr":"172.21.15.66:6835","nonce":2554522797}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:13:03.687866+0000","modified":"2024-09-10T22:13:06.708613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8811":{"gid":8811,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6836/1762705824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1762705824},{"type":"v1","addr":"172.21.15.94:6836","nonce":1762705824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8811,"qdb_cluster":[8811]},"id":9}]} 2024-09-10T22:13:50.867 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 87 2024-09-10T22:13:50.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.863+0000 7f63bac00640 1 -- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6394074710 msgr2=0x7f6394076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.863+0000 7f63bac00640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6394074710 0x7f6394076bd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f63a40af000 tx=0x7f63a401afb0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.863+0000 7f63bac00640 1 -- 172.21.15.66:0/942119068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b40cf830 msgr2=0x7f63b4157560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.863+0000 7f63bac00640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b40cf830 0x7f63b4157560 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f63a000c4c0 tx=0x7f63a000c980 comp rx=0 tx=0).stop 2024-09-10T22:13:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.863+0000 7f63bac00640 1 -- 172.21.15.66:0/942119068 shutdown_connections 2024-09-10T22:13:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.863+0000 7f63bac00640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6394074710 0x7f6394076bd0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.863+0000 7f63bac00640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63b415bd40 0x7f63b415cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.863+0000 7f63bac00640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b40cf830 0x7f63b4157560 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.863+0000 7f63bac00640 1 --2- 172.21.15.66:0/942119068 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63b40cf180 0x7f63b41648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.863+0000 7f63bac00640 1 -- 172.21.15.66:0/942119068 >> 172.21.15.66:0/942119068 conn(0x7f63b40d1c60 msgr2=0x7f63b415acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.863+0000 7f63bac00640 1 -- 172.21.15.66:0/942119068 shutdown_connections 2024-09-10T22:13:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:50.863+0000 7f63bac00640 1 -- 172.21.15.66:0/942119068 wait complete. 2024-09-10T22:13:50.889 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:13:51.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.031+0000 7f8ae8c00640 1 -- 172.21.15.66:0/1064766904 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ae40ce3a0 msgr2=0x7f8ae40ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.031+0000 7f8ae8c00640 1 --2- 172.21.15.66:0/1064766904 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ae40ce3a0 0x7f8ae40ce830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009bc0 tx=0x7f8adc01f3b0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.031+0000 7f8ae8c00640 1 -- 172.21.15.66:0/1064766904 shutdown_connections 2024-09-10T22:13:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.031+0000 7f8ae8c00640 1 --2- 172.21.15.66:0/1064766904 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ae40ce3a0 0x7f8ae40ce830 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.031+0000 7f8ae8c00640 1 --2- 172.21.15.66:0/1064766904 >> v2:172.21.15.94:3300/0 conn(0x7f8ae40cc5e0 0x7f8ae415e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.031+0000 7f8ae8c00640 1 -- 172.21.15.66:0/1064766904 >> 172.21.15.66:0/1064766904 conn(0x7f8ae40d0120 msgr2=0x7f8ae40d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.031+0000 7f8ae8c00640 1 -- 172.21.15.66:0/1064766904 shutdown_connections 2024-09-10T22:13:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.031+0000 7f8ae8c00640 1 -- 172.21.15.66:0/1064766904 wait complete. 2024-09-10T22:13:51.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae8c00640 1 Processor -- start 2024-09-10T22:13:51.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae8c00640 1 -- start start 2024-09-10T22:13:51.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ae40cc5e0 0x7f8ae4154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:51.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae8c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ae40ce3a0 0x7f8ae40c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:51.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae8c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae415a200 0x7f8ae40c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:51.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae8c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae4068990 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae8c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae415ea70 con 0x7f8ae415a200 2024-09-10T22:13:51.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae8c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae4068b10 con 0x7f8ae40ce3a0 2024-09-10T22:13:51.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ae40cc5e0 0x7f8ae4154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:51.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ae40cc5e0 0x7f8ae4154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40006/0 (socket says 172.21.15.66:40006) 2024-09-10T22:13:51.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae3400640 1 -- 172.21.15.66:0/2915030104 learned_addr learned my addr 172.21.15.66:0/2915030104 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:51.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8adbe00640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ae40ce3a0 0x7f8ae40c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:51.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae3e00640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae415a200 0x7f8ae40c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:51.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae3400640 1 -- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ae40ce3a0 msgr2=0x7f8ae40c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae3400640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ae40ce3a0 0x7f8ae40c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae3400640 1 -- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae415a200 msgr2=0x7f8ae40c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae3400640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae415a200 0x7f8ae40c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae3400640 1 -- 172.21.15.66:0/2915030104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae40c7290 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae3400640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ae40cc5e0 0x7f8ae4154200 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c430 tx=0x7f8ad400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:51.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8adb400640 1 -- 172.21.15.66:0/2915030104 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4011790 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae8c00640 1 -- 172.21.15.66:0/2915030104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae40c7500 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8adb400640 1 -- 172.21.15.66:0/2915030104 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ad4017070 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae8c00640 1 -- 172.21.15.66:0/2915030104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae40c6bd0 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8adb400640 1 -- 172.21.15.66:0/2915030104 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4011c80 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.035+0000 7f8ae8c00640 1 -- 172.21.15.66:0/2915030104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae40c7740 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.039+0000 7f8adb400640 1 -- 172.21.15.66:0/2915030104 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f8ad401c460 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.039+0000 7f8adb400640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8ac0074710 0x7f8ac0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:51.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.039+0000 7f8adb400640 1 -- 172.21.15.66:0/2915030104 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5935+0+0 (secure 0 0 0) 0x7f8ad4092090 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.039+0000 7f8adbe00640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8ac0074710 0x7f8ac0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:51.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.039+0000 7f8adbe00640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8ac0074710 0x7f8ac0076bd0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8ae415e830 tx=0x7f8acc010040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:51.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.043+0000 7f8adb400640 1 -- 172.21.15.66:0/2915030104 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ae40c7740 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.251+0000 7f8ae8c00640 1 -- 172.21.15.66:0/2915030104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ae40c66e0 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.251+0000 7f8adb400640 1 -- 172.21.15.66:0/2915030104 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5222 (secure 0 0 0) 0x7f8ad4011e20 con 0x7f8ae40cc5e0 2024-09-10T22:13:51.257 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:51.258 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":87,"btime":"2024-09-10T22:13:12:423173+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1352582369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1352582369},{"type":"v1","addr":"172.21.15.94:6837","nonce":1352582369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8805,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/441263936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":441263936},{"type":"v1","addr":"172.21.15.66:6834","nonce":441263936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2554522797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2554522797},{"type":"v1","addr":"172.21.15.66:6835","nonce":2554522797}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:13:03.687866+0000","modified":"2024-09-10T22:13:06.708613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8811":{"gid":8811,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6836/1762705824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1762705824},{"type":"v1","addr":"172.21.15.94:6836","nonce":1762705824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8811,"qdb_cluster":[8811]},"id":9}]} 2024-09-10T22:13:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 87 2024-09-10T22:13:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.255+0000 7f8ae8c00640 1 -- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8ac0074710 msgr2=0x7f8ac0076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.255+0000 7f8ae8c00640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8ac0074710 0x7f8ac0076bd0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8ae415e830 tx=0x7f8acc010040 comp rx=0 tx=0).stop 2024-09-10T22:13:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.255+0000 7f8ae8c00640 1 -- 172.21.15.66:0/2915030104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ae40cc5e0 msgr2=0x7f8ae4154200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.255+0000 7f8ae8c00640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ae40cc5e0 0x7f8ae4154200 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c430 tx=0x7f8ad400c8f0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.255+0000 7f8ae8c00640 1 -- 172.21.15.66:0/2915030104 shutdown_connections 2024-09-10T22:13:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.255+0000 7f8ae8c00640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8ac0074710 0x7f8ac0076bd0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.255+0000 7f8ae8c00640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae415a200 0x7f8ae40c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.255+0000 7f8ae8c00640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8ae40ce3a0 0x7f8ae40c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.255+0000 7f8ae8c00640 1 --2- 172.21.15.66:0/2915030104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ae40cc5e0 0x7f8ae4154200 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.255+0000 7f8ae8c00640 1 -- 172.21.15.66:0/2915030104 >> 172.21.15.66:0/2915030104 conn(0x7f8ae40d0120 msgr2=0x7f8ae40b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:51.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.255+0000 7f8ae8c00640 1 -- 172.21.15.66:0/2915030104 shutdown_connections 2024-09-10T22:13:51.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.255+0000 7f8ae8c00640 1 -- 172.21.15.66:0/2915030104 wait complete. 2024-09-10T22:13:51.280 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:13:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.423+0000 7f485a000640 1 -- 172.21.15.66:0/1562898143 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f485415a310 msgr2=0x7f48540c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.423+0000 7f485a000640 1 --2- 172.21.15.66:0/1562898143 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f485415a310 0x7f48540c48b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4844009870 tx=0x7f484401c6f0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.423+0000 7f485a000640 1 -- 172.21.15.66:0/1562898143 shutdown_connections 2024-09-10T22:13:51.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.423+0000 7f485a000640 1 --2- 172.21.15.66:0/1562898143 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f485415a310 0x7f48540c48b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.423+0000 7f485a000640 1 -- 172.21.15.66:0/1562898143 >> 172.21.15.66:0/1562898143 conn(0x7f48540cfa30 msgr2=0x7f48540cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:51.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.423+0000 7f485a000640 1 -- 172.21.15.66:0/1562898143 shutdown_connections 2024-09-10T22:13:51.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.423+0000 7f485a000640 1 -- 172.21.15.66:0/1562898143 wait complete. 2024-09-10T22:13:51.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.423+0000 7f485a000640 1 Processor -- start 2024-09-10T22:13:51.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.423+0000 7f485a000640 1 -- start start 2024-09-10T22:13:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f485a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48540aeec0 0x7f4854134d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f485a000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f485415a310 0x7f48541fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f485a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4854149e90 0x7f4854147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f485a000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f48541687b0 con 0x7f48540aeec0 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f485a000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4854168630 con 0x7f4854149e90 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f485a000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4854168930 con 0x7f485415a310 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f4853400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48540aeec0 0x7f4854134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f4853400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48540aeec0 0x7f4854134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40020/0 (socket says 172.21.15.66:40020) 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f4853400640 1 -- 172.21.15.66:0/3409936837 learned_addr learned my addr 172.21.15.66:0/3409936837 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f484be00640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f485415a310 0x7f48541fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f4853e00640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4854149e90 0x7f4854147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f4853400640 1 -- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f485415a310 msgr2=0x7f48541fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f4853400640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f485415a310 0x7f48541fc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f4853400640 1 -- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4854149e90 msgr2=0x7f4854147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f4853400640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4854149e90 0x7f4854147e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f4853400640 1 -- 172.21.15.66:0/3409936837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48541fc630 con 0x7f48540aeec0 2024-09-10T22:13:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f4853400640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48540aeec0 0x7f4854134d20 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f484401cf90 tx=0x7f4844005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f484b400640 1 -- 172.21.15.66:0/3409936837 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48440ad030 con 0x7f48540aeec0 2024-09-10T22:13:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f484b400640 1 -- 172.21.15.66:0/3409936837 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4844004400 con 0x7f48540aeec0 2024-09-10T22:13:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f485a000640 1 -- 172.21.15.66:0/3409936837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485414a780 con 0x7f48540aeec0 2024-09-10T22:13:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f484b400640 1 -- 172.21.15.66:0/3409936837 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48440aa990 con 0x7f48540aeec0 2024-09-10T22:13:51.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f485a000640 1 -- 172.21.15.66:0/3409936837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485414ad40 con 0x7f48540aeec0 2024-09-10T22:13:51.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.427+0000 7f485a000640 1 -- 172.21.15.66:0/3409936837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48540c48b0 con 0x7f48540aeec0 2024-09-10T22:13:51.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.431+0000 7f484b400640 1 -- 172.21.15.66:0/3409936837 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f48440aab30 con 0x7f48540aeec0 2024-09-10T22:13:51.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.431+0000 7f484b400640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4830074710 0x7f4830076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:51.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.431+0000 7f484b400640 1 -- 172.21.15.66:0/3409936837 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5935+0+0 (secure 0 0 0) 0x7f484412c950 con 0x7f48540aeec0 2024-09-10T22:13:51.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.431+0000 7f484be00640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4830074710 0x7f4830076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:51.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.431+0000 7f484be00640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4830074710 0x7f4830076bd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f483c000fc0 tx=0x7f483c001d00 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:51.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.435+0000 7f484b400640 1 -- 172.21.15.66:0/3409936837 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f48541fc630 con 0x7f48540aeec0 2024-09-10T22:13:51.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.607+0000 7f485a000640 1 -- 172.21.15.66:0/3409936837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4854074270 con 0x7f48540aeec0 2024-09-10T22:13:51.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f484b400640 1 -- 172.21.15.66:0/3409936837 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16998 (secure 0 0 0) 0x7f48440b8070 con 0x7f48540aeec0 2024-09-10T22:13:51.615 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:51.615 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":129,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:13:50.004674+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:13:01.405668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T22:13:02.664579+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":129,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.a","mappings":[{"from":2,"to":0}]},{"pgid":"19.31","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:13:51.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f485a000640 1 -- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4830074710 msgr2=0x7f4830076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f485a000640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4830074710 0x7f4830076bd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f483c000fc0 tx=0x7f483c001d00 comp rx=0 tx=0).stop 2024-09-10T22:13:51.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f485a000640 1 -- 172.21.15.66:0/3409936837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48540aeec0 msgr2=0x7f4854134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f485a000640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48540aeec0 0x7f4854134d20 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f484401cf90 tx=0x7f4844005750 comp rx=0 tx=0).stop 2024-09-10T22:13:51.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f485a000640 1 -- 172.21.15.66:0/3409936837 shutdown_connections 2024-09-10T22:13:51.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f485a000640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4830074710 0x7f4830076bd0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f485a000640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4854149e90 0x7f4854147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f485a000640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f485415a310 0x7f48541fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f485a000640 1 --2- 172.21.15.66:0/3409936837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48540aeec0 0x7f4854134d20 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f485a000640 1 -- 172.21.15.66:0/3409936837 >> 172.21.15.66:0/3409936837 conn(0x7f48540cfa30 msgr2=0x7f4854073050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:51.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f485a000640 1 -- 172.21.15.66:0/3409936837 shutdown_connections 2024-09-10T22:13:51.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.611+0000 7f485a000640 1 -- 172.21.15.66:0/3409936837 wait complete. 2024-09-10T22:13:51.637 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:13:51.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.779+0000 7fd70e800640 1 -- 172.21.15.66:0/1791650801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7080b1e10 msgr2=0x7fd7080b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.779+0000 7fd70e800640 1 --2- 172.21.15.66:0/1791650801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7080b1e10 0x7fd7080b3170 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fd700009bc0 tx=0x7fd70001d540 comp rx=0 tx=0).stop 2024-09-10T22:13:51.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.779+0000 7fd70e800640 1 -- 172.21.15.66:0/1791650801 shutdown_connections 2024-09-10T22:13:51.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.779+0000 7fd70e800640 1 --2- 172.21.15.66:0/1791650801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7080b1e10 0x7fd7080b3170 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.779+0000 7fd70e800640 1 -- 172.21.15.66:0/1791650801 >> 172.21.15.66:0/1791650801 conn(0x7fd7080d1c60 msgr2=0x7fd7080d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:51.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.779+0000 7fd70e800640 1 -- 172.21.15.66:0/1791650801 shutdown_connections 2024-09-10T22:13:51.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.779+0000 7fd70e800640 1 -- 172.21.15.66:0/1791650801 wait complete. 2024-09-10T22:13:51.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.779+0000 7fd70e800640 1 Processor -- start 2024-09-10T22:13:51.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.779+0000 7fd70e800640 1 -- start start 2024-09-10T22:13:51.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd70e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7080b1e10 0x7fd708155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:51.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd70e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70815f450 0x7fd7080b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:51.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd70e800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd708160280 0x7fd7081540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:51.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd70e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd70816a040 con 0x7fd7080b1e10 2024-09-10T22:13:51.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd70e800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd708169ec0 con 0x7fd70815f450 2024-09-10T22:13:51.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd70e800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd70816a1c0 con 0x7fd708160280 2024-09-10T22:13:51.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd707400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7080b1e10 0x7fd708155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:51.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd707400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7080b1e10 0x7fd708155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40038/0 (socket says 172.21.15.66:40038) 2024-09-10T22:13:51.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd707400640 1 -- 172.21.15.66:0/942607492 learned_addr learned my addr 172.21.15.66:0/942607492 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd707e00640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd708160280 0x7fd7081540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd6ffe00640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70815f450 0x7fd7080b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd707400640 1 -- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd708160280 msgr2=0x7fd7081540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd707400640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd708160280 0x7fd7081540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd707400640 1 -- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70815f450 msgr2=0x7fd7080b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd707400640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70815f450 0x7fd7080b7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd707400640 1 -- 172.21.15.66:0/942607492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7081545e0 con 0x7fd7080b1e10 2024-09-10T22:13:51.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd707400640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7080b1e10 0x7fd708155c50 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c500 tx=0x7fd6f800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd6ff400640 1 -- 172.21.15.66:0/942607492 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8017070 con 0x7fd7080b1e10 2024-09-10T22:13:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd6ff400640 1 -- 172.21.15.66:0/942607492 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6f80053c0 con 0x7fd7080b1e10 2024-09-10T22:13:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd6ff400640 1 -- 172.21.15.66:0/942607492 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8012890 con 0x7fd7080b1e10 2024-09-10T22:13:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd70e800640 1 -- 172.21.15.66:0/942607492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7081553b0 con 0x7fd7080b1e10 2024-09-10T22:13:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd70e800640 1 -- 172.21.15.66:0/942607492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd708155970 con 0x7fd7080b1e10 2024-09-10T22:13:51.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd70e800640 1 -- 172.21.15.66:0/942607492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7080b3170 con 0x7fd7080b1e10 2024-09-10T22:13:51.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.783+0000 7fd6ff400640 1 -- 172.21.15.66:0/942607492 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fd6f8003600 con 0x7fd7080b1e10 2024-09-10T22:13:51.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.787+0000 7fd6ff400640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd6e4074710 0x7fd6e4076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:51.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.787+0000 7fd6ff400640 1 -- 172.21.15.66:0/942607492 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5935+0+0 (secure 0 0 0) 0x7fd6f8091960 con 0x7fd7080b1e10 2024-09-10T22:13:51.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.787+0000 7fd6ffe00640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd6e4074710 0x7fd6e4076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:51.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.787+0000 7fd6ffe00640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd6e4074710 0x7fd6e4076bd0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd708156280 tx=0x7fd6f0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:51.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:51.791+0000 7fd6ff400640 1 -- 172.21.15.66:0/942607492 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd6f805eac0 con 0x7fd7080b1e10 2024-09-10T22:13:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.003+0000 7fd70e800640 1 -- 172.21.15.66:0/942607492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd7080cbff0 con 0x7fd7080b1e10 2024-09-10T22:13:52.007 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:13:52.003+0000 7fcc93a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:13:52.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.003+0000 7fd6ff400640 1 -- 172.21.15.66:0/942607492 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) ==== 161+0+0 (secure 0 0 0) 0x7fd6f80671e0 con 0x7fd7080b1e10 2024-09-10T22:13:52.009 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:13:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.007+0000 7fd70e800640 1 -- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd6e4074710 msgr2=0x7fd6e4076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.007+0000 7fd70e800640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd6e4074710 0x7fd6e4076bd0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd708156280 tx=0x7fd6f0009040 comp rx=0 tx=0).stop 2024-09-10T22:13:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.007+0000 7fd70e800640 1 -- 172.21.15.66:0/942607492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7080b1e10 msgr2=0x7fd708155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.007+0000 7fd70e800640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7080b1e10 0x7fd708155c50 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c500 tx=0x7fd6f800c9c0 comp rx=0 tx=0).stop 2024-09-10T22:13:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.007+0000 7fd70e800640 1 -- 172.21.15.66:0/942607492 shutdown_connections 2024-09-10T22:13:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.007+0000 7fd70e800640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd6e4074710 0x7fd6e4076bd0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.007+0000 7fd70e800640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd708160280 0x7fd7081540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.007+0000 7fd70e800640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70815f450 0x7fd7080b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.007+0000 7fd70e800640 1 --2- 172.21.15.66:0/942607492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7080b1e10 0x7fd708155c50 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.007+0000 7fd70e800640 1 -- 172.21.15.66:0/942607492 >> 172.21.15.66:0/942607492 conn(0x7fd7080d1c60 msgr2=0x7fd7080b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.007+0000 7fd70e800640 1 -- 172.21.15.66:0/942607492 shutdown_connections 2024-09-10T22:13:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.007+0000 7fd70e800640 1 -- 172.21.15.66:0/942607492 wait complete. 2024-09-10T22:13:52.033 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:13:52.078 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:13:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.175+0000 7f9677e00640 1 -- 172.21.15.66:0/3762542449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96700cb620 msgr2=0x7f96700ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.175+0000 7f9677e00640 1 --2- 172.21.15.66:0/3762542449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96700cb620 0x7f96700ceff0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9668009bc0 tx=0x7f966801cb70 comp rx=0 tx=0).stop 2024-09-10T22:13:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.175+0000 7f9677e00640 1 -- 172.21.15.66:0/3762542449 shutdown_connections 2024-09-10T22:13:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.175+0000 7f9677e00640 1 --2- 172.21.15.66:0/3762542449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96700cb620 0x7f96700ceff0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.175+0000 7f9677e00640 1 --2- 172.21.15.66:0/3762542449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f967015ac80 0x7f96700cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.175+0000 7f9677e00640 1 -- 172.21.15.66:0/3762542449 >> 172.21.15.66:0/3762542449 conn(0x7f96700b26c0 msgr2=0x7f96700b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.175+0000 7f9677e00640 1 -- 172.21.15.66:0/3762542449 shutdown_connections 2024-09-10T22:13:52.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.175+0000 7f9677e00640 1 -- 172.21.15.66:0/3762542449 wait complete. 2024-09-10T22:13:52.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9677e00640 1 Processor -- start 2024-09-10T22:13:52.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9677e00640 1 -- start start 2024-09-10T22:13:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9677e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96700cb620 0x7f96700b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9677e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f967015a8a0 0x7f96700ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9677e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f967015ac80 0x7f9670151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9677e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9670167cc0 con 0x7f967015a8a0 2024-09-10T22:13:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9677e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9670167b40 con 0x7f96700cb620 2024-09-10T22:13:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9677e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9670167e40 con 0x7f967015ac80 2024-09-10T22:13:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9674a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f967015a8a0 0x7f96700ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9674a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f967015a8a0 0x7f96700ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40048/0 (socket says 172.21.15.66:40048) 2024-09-10T22:13:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9674a00640 1 -- 172.21.15.66:0/2394561403 learned_addr learned my addr 172.21.15.66:0/2394561403 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9675e00640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f967015ac80 0x7f9670151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9675400640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96700cb620 0x7f96700b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9674a00640 1 -- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f967015ac80 msgr2=0x7f9670151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9674a00640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f967015ac80 0x7f9670151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9674a00640 1 -- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96700cb620 msgr2=0x7f96700b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9674a00640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96700cb620 0x7f96700b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9674a00640 1 -- 172.21.15.66:0/2394561403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96700c9330 con 0x7f967015a8a0 2024-09-10T22:13:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9675400640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96700cb620 0x7f96700b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:13:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9674a00640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f967015a8a0 0x7f96700ca360 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f965c00e7a0 tx=0x7f965c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f966d600640 1 -- 172.21.15.66:0/2394561403 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c00ccc0 con 0x7f967015a8a0 2024-09-10T22:13:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f966d600640 1 -- 172.21.15.66:0/2394561403 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f965c010430 con 0x7f967015a8a0 2024-09-10T22:13:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9677e00640 1 -- 172.21.15.66:0/2394561403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96700c95a0 con 0x7f967015a8a0 2024-09-10T22:13:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f966d600640 1 -- 172.21.15.66:0/2394561403 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c010860 con 0x7f967015a8a0 2024-09-10T22:13:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.179+0000 7f9677e00640 1 -- 172.21.15.66:0/2394561403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96700c9b60 con 0x7f967015a8a0 2024-09-10T22:13:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.183+0000 7f9677e00640 1 -- 172.21.15.66:0/2394561403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96700ceff0 con 0x7f967015a8a0 2024-09-10T22:13:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.183+0000 7f966d600640 1 -- 172.21.15.66:0/2394561403 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f965c010a00 con 0x7f967015a8a0 2024-09-10T22:13:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.183+0000 7f966d600640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9650074740 0x7f9650076c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.183+0000 7f966d600640 1 -- 172.21.15.66:0/2394561403 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5993+0+0 (secure 0 0 0) 0x7f965c092dc0 con 0x7f967015a8a0 2024-09-10T22:13:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.183+0000 7f9675400640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9650074740 0x7f9650076c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.183+0000 7f9675400640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9650074740 0x7f9650076c00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9664001230 tx=0x7f9664009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.187+0000 7f966d600640 1 -- 172.21.15.66:0/2394561403 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f965c05ff00 con 0x7f967015a8a0 2024-09-10T22:13:52.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:52.403+0000 7f9677e00640 1 -- 172.21.15.66:0/2394561403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9670157990 con 0x7f967015a8a0 2024-09-10T22:13:53.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.023+0000 7f966d600640 1 -- 172.21.15.66:0/2394561403 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v89) ==== 104+0+0 (secure 0 0 0) 0x7f965c05f800 con 0x7f967015a8a0 2024-09-10T22:13:53.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.027+0000 7f9677e00640 1 -- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9650074740 msgr2=0x7f9650076c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:53.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.027+0000 7f9677e00640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9650074740 0x7f9650076c00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9664001230 tx=0x7f9664009040 comp rx=0 tx=0).stop 2024-09-10T22:13:53.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.027+0000 7f9677e00640 1 -- 172.21.15.66:0/2394561403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f967015a8a0 msgr2=0x7f96700ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:53.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.027+0000 7f9677e00640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f967015a8a0 0x7f96700ca360 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f965c00e7a0 tx=0x7f965c00ec60 comp rx=0 tx=0).stop 2024-09-10T22:13:53.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.027+0000 7f9677e00640 1 -- 172.21.15.66:0/2394561403 shutdown_connections 2024-09-10T22:13:53.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.027+0000 7f9677e00640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9650074740 0x7f9650076c00 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:53.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.027+0000 7f9677e00640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f967015ac80 0x7f9670151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:53.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.027+0000 7f9677e00640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f967015a8a0 0x7f96700ca360 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:53.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.027+0000 7f9677e00640 1 --2- 172.21.15.66:0/2394561403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96700cb620 0x7f96700b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:53.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.027+0000 7f9677e00640 1 -- 172.21.15.66:0/2394561403 >> 172.21.15.66:0/2394561403 conn(0x7f96700b26c0 msgr2=0x7f9670153bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:53.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.027+0000 7f9677e00640 1 -- 172.21.15.66:0/2394561403 shutdown_connections 2024-09-10T22:13:53.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.027+0000 7f9677e00640 1 -- 172.21.15.66:0/2394561403 wait complete. 2024-09-10T22:13:53.053 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:13:53.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.199+0000 7fd590000640 1 -- 172.21.15.66:0/734555773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5880b2b10 msgr2=0x7fd5881673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:53.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.199+0000 7fd590000640 1 --2- 172.21.15.66:0/734555773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5880b2b10 0x7fd5881673c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd580009bc0 tx=0x7fd58001cba0 comp rx=0 tx=0).stop 2024-09-10T22:13:53.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.199+0000 7fd590000640 1 -- 172.21.15.66:0/734555773 shutdown_connections 2024-09-10T22:13:53.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.199+0000 7fd590000640 1 --2- 172.21.15.66:0/734555773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5880b2b10 0x7fd5881673c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:53.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.199+0000 7fd590000640 1 -- 172.21.15.66:0/734555773 >> 172.21.15.66:0/734555773 conn(0x7fd5880d1ce0 msgr2=0x7fd5880d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:53.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.199+0000 7fd590000640 1 -- 172.21.15.66:0/734555773 shutdown_connections 2024-09-10T22:13:53.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.199+0000 7fd590000640 1 -- 172.21.15.66:0/734555773 wait complete. 2024-09-10T22:13:53.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd590000640 1 Processor -- start 2024-09-10T22:13:53.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd590000640 1 -- start start 2024-09-10T22:13:53.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd590000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1d30 0x7fd588165860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd590000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5880b2b10 0x7fd5880b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd590000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5881605c0 0x7fd5880b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd590000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd58816a6b0 con 0x7fd5880b1d30 2024-09-10T22:13:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd590000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd58816a530 con 0x7fd5881605c0 2024-09-10T22:13:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd590000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd58816a830 con 0x7fd5880b2b10 2024-09-10T22:13:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd58d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1d30 0x7fd588165860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd58d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1d30 0x7fd588165860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45164/0 (socket says 172.21.15.66:45164) 2024-09-10T22:13:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd58d600640 1 -- 172.21.15.66:0/2527587875 learned_addr learned my addr 172.21.15.66:0/2527587875 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd58e000640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5881605c0 0x7fd5880b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd58cc00640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5880b2b10 0x7fd5880b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd58d600640 1 -- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5880b2b10 msgr2=0x7fd5880b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd58d600640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5880b2b10 0x7fd5880b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd58d600640 1 -- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5881605c0 msgr2=0x7fd5880b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd58d600640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5881605c0 0x7fd5880b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd58d600640 1 -- 172.21.15.66:0/2527587875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5880af2a0 con 0x7fd5880b1d30 2024-09-10T22:13:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd58d600640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1d30 0x7fd588165860 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd57c001ec0 tx=0x7fd57c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd585600640 1 -- 172.21.15.66:0/2527587875 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c017070 con 0x7fd5880b1d30 2024-09-10T22:13:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd585600640 1 -- 172.21.15.66:0/2527587875 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd57c003600 con 0x7fd5880b1d30 2024-09-10T22:13:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd585600640 1 -- 172.21.15.66:0/2527587875 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c003a50 con 0x7fd5880b1d30 2024-09-10T22:13:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd590000640 1 -- 172.21.15.66:0/2527587875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd588165da0 con 0x7fd5880b1d30 2024-09-10T22:13:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.203+0000 7fd590000640 1 -- 172.21.15.66:0/2527587875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd588166040 con 0x7fd5880b1d30 2024-09-10T22:13:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.207+0000 7fd590000640 1 -- 172.21.15.66:0/2527587875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5880b1620 con 0x7fd5880b1d30 2024-09-10T22:13:53.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.207+0000 7fd585600640 1 -- 172.21.15.66:0/2527587875 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fd57c003bf0 con 0x7fd5880b1d30 2024-09-10T22:13:53.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.207+0000 7fd585600640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd568074710 0x7fd568076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:53.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.207+0000 7fd585600640 1 -- 172.21.15.66:0/2527587875 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5993+0+0 (secure 0 0 0) 0x7fd57c091980 con 0x7fd5880b1d30 2024-09-10T22:13:53.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.207+0000 7fd58cc00640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd568074710 0x7fd568076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:53.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.207+0000 7fd58cc00640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd568074710 0x7fd568076bd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd5880c8d20 tx=0x7fd574001420 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:53.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.211+0000 7fd585600640 1 -- 172.21.15.66:0/2527587875 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd57c05e470 con 0x7fd5880b1d30 2024-09-10T22:13:53.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:53.387+0000 7fd590000640 1 -- 172.21.15.66:0/2527587875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd5880d00a0 con 0x7fd5880b1d30 2024-09-10T22:13:54.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.027+0000 7fd585600640 1 -- 172.21.15.66:0/2527587875 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) ==== 181+0+0 (secure 0 0 0) 0x7fd57c0671e0 con 0x7fd5880b1d30 2024-09-10T22:13:54.032 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:13:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.031+0000 7fd590000640 1 -- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd568074710 msgr2=0x7fd568076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.031+0000 7fd590000640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd568074710 0x7fd568076bd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd5880c8d20 tx=0x7fd574001420 comp rx=0 tx=0).stop 2024-09-10T22:13:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.031+0000 7fd590000640 1 -- 172.21.15.66:0/2527587875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1d30 msgr2=0x7fd588165860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.031+0000 7fd590000640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1d30 0x7fd588165860 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd57c001ec0 tx=0x7fd57c00c890 comp rx=0 tx=0).stop 2024-09-10T22:13:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.031+0000 7fd590000640 1 -- 172.21.15.66:0/2527587875 shutdown_connections 2024-09-10T22:13:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.031+0000 7fd590000640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd568074710 0x7fd568076bd0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.031+0000 7fd590000640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5881605c0 0x7fd5880b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:54.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.031+0000 7fd590000640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5880b2b10 0x7fd5880b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:54.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.031+0000 7fd590000640 1 --2- 172.21.15.66:0/2527587875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880b1d30 0x7fd588165860 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:54.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.031+0000 7fd590000640 1 -- 172.21.15.66:0/2527587875 >> 172.21.15.66:0/2527587875 conn(0x7fd5880d1ce0 msgr2=0x7fd5880b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:54.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.031+0000 7fd590000640 1 -- 172.21.15.66:0/2527587875 shutdown_connections 2024-09-10T22:13:54.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.031+0000 7fd590000640 1 -- 172.21.15.66:0/2527587875 wait complete. 2024-09-10T22:13:54.064 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:13:54.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.215+0000 7f2a77400640 1 -- 172.21.15.66:0/3571744436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a54003a60 msgr2=0x7f2a54003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:54.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.215+0000 7f2a77400640 1 --2- 172.21.15.66:0/3571744436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a54003a60 0x7f2a54003e80 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009b60 tx=0x7f2a6001e3d0 comp rx=0 tx=0).stop 2024-09-10T22:13:54.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.215+0000 7f2a77400640 1 -- 172.21.15.66:0/3571744436 shutdown_connections 2024-09-10T22:13:54.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.215+0000 7f2a77400640 1 --2- 172.21.15.66:0/3571744436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a54003a60 0x7f2a54003e80 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:54.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.215+0000 7f2a77400640 1 -- 172.21.15.66:0/3571744436 >> 172.21.15.66:0/3571744436 conn(0x7f2a700cfaa0 msgr2=0x7f2a700cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:54.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.215+0000 7f2a77400640 1 -- 172.21.15.66:0/3571744436 shutdown_connections 2024-09-10T22:13:54.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.215+0000 7f2a77400640 1 -- 172.21.15.66:0/3571744436 wait complete. 2024-09-10T22:13:54.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a77400640 1 Processor -- start 2024-09-10T22:13:54.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a77400640 1 -- start start 2024-09-10T22:13:54.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a77400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a54003a60 0x7f2a7014bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:54.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a77400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a70149100 0x7f2a701f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:54.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a77400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a700ad460 0x7f2a700ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:54.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a77400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a700cf5a0 con 0x7f2a54003a60 2024-09-10T22:13:54.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a77400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a700cf420 con 0x7f2a70149100 2024-09-10T22:13:54.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a77400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2a70149560 con 0x7f2a700ad460 2024-09-10T22:13:54.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a74a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a54003a60 0x7f2a7014bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:54.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a74a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a54003a60 0x7f2a7014bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45182/0 (socket says 172.21.15.66:45182) 2024-09-10T22:13:54.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a74a00640 1 -- 172.21.15.66:0/2303949274 learned_addr learned my addr 172.21.15.66:0/2303949274 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:54.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a75400640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a700ad460 0x7f2a700ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:54.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a67e00640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a70149100 0x7f2a701f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:54.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a74a00640 1 -- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a700ad460 msgr2=0x7f2a700ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:54.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a74a00640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a700ad460 0x7f2a700ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:54.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a74a00640 1 -- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a70149100 msgr2=0x7f2a701f4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:54.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a74a00640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a70149100 0x7f2a701f4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:54.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a74a00640 1 -- 172.21.15.66:0/2303949274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a701f4dd0 con 0x7f2a54003a60 2024-09-10T22:13:54.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a74a00640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a54003a60 0x7f2a7014bf70 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f2a600b1040 tx=0x7f2a60005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:54.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a6d600640 1 -- 172.21.15.66:0/2303949274 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600be070 con 0x7f2a54003a60 2024-09-10T22:13:54.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a6d600640 1 -- 172.21.15.66:0/2303949274 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a60004120 con 0x7f2a54003a60 2024-09-10T22:13:54.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a6d600640 1 -- 172.21.15.66:0/2303949274 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600b3040 con 0x7f2a54003a60 2024-09-10T22:13:54.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a77400640 1 -- 172.21.15.66:0/2303949274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a701f5070 con 0x7f2a54003a60 2024-09-10T22:13:54.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.219+0000 7f2a77400640 1 -- 172.21.15.66:0/2303949274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a701f5630 con 0x7f2a54003a60 2024-09-10T22:13:54.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.223+0000 7f2a77400640 1 -- 172.21.15.66:0/2303949274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7015e600 con 0x7f2a54003a60 2024-09-10T22:13:54.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.223+0000 7f2a6d600640 1 -- 172.21.15.66:0/2303949274 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f2a600b6030 con 0x7f2a54003a60 2024-09-10T22:13:54.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.223+0000 7f2a6d600640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a4c074740 0x7f2a4c076c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:54.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.223+0000 7f2a6d600640 1 -- 172.21.15.66:0/2303949274 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5555+0+0 (secure 0 0 0) 0x7f2a6012c5c0 con 0x7f2a54003a60 2024-09-10T22:13:54.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.223+0000 7f2a67e00640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a4c074740 0x7f2a4c076c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:54.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.227+0000 7f2a67e00640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a4c074740 0x7f2a4c076c00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2a58006c70 tx=0x7f2a580067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:54.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.227+0000 7f2a6d600640 1 -- 172.21.15.66:0/2303949274 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2a600180e0 con 0x7f2a54003a60 2024-09-10T22:13:54.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:54.407+0000 7f2a77400640 1 -- 172.21.15.66:0/2303949274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2a700d07c0 con 0x7f2a54003a60 2024-09-10T22:13:55.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.039+0000 7f2a6d600640 1 -- 172.21.15.66:0/2303949274 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) ==== 169+0+0 (secure 0 0 0) 0x7f2a600f9020 con 0x7f2a54003a60 2024-09-10T22:13:55.046 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:13:55.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.047+0000 7f2a77400640 1 -- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a4c074740 msgr2=0x7f2a4c076c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:55.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.047+0000 7f2a77400640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a4c074740 0x7f2a4c076c00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2a58006c70 tx=0x7f2a580067b0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.047+0000 7f2a77400640 1 -- 172.21.15.66:0/2303949274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a54003a60 msgr2=0x7f2a7014bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:55.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.047+0000 7f2a77400640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a54003a60 0x7f2a7014bf70 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f2a600b1040 tx=0x7f2a60005520 comp rx=0 tx=0).stop 2024-09-10T22:13:55.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.047+0000 7f2a77400640 1 -- 172.21.15.66:0/2303949274 shutdown_connections 2024-09-10T22:13:55.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.047+0000 7f2a77400640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2a4c074740 0x7f2a4c076c00 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.047+0000 7f2a77400640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a700ad460 0x7f2a700ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.047+0000 7f2a77400640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a70149100 0x7f2a701f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.047+0000 7f2a77400640 1 --2- 172.21.15.66:0/2303949274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a54003a60 0x7f2a7014bf70 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.047+0000 7f2a77400640 1 -- 172.21.15.66:0/2303949274 >> 172.21.15.66:0/2303949274 conn(0x7f2a700cfaa0 msgr2=0x7f2a700b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:55.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.047+0000 7f2a77400640 1 -- 172.21.15.66:0/2303949274 shutdown_connections 2024-09-10T22:13:55.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.047+0000 7f2a77400640 1 -- 172.21.15.66:0/2303949274 wait complete. 2024-09-10T22:13:55.077 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T22:13:55.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 -- 172.21.15.66:0/2222172856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448c0b2bf0 msgr2=0x7f448c0af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 --2- 172.21.15.66:0/2222172856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448c0b2bf0 0x7f448c0af070 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f4484009bc0 tx=0x7f448401cba0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 -- 172.21.15.66:0/2222172856 shutdown_connections 2024-09-10T22:13:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 --2- 172.21.15.66:0/2222172856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448c0b2bf0 0x7f448c0af070 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 -- 172.21.15.66:0/2222172856 >> 172.21.15.66:0/2222172856 conn(0x7f448c0d1c50 msgr2=0x7f448c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 -- 172.21.15.66:0/2222172856 shutdown_connections 2024-09-10T22:13:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 -- 172.21.15.66:0/2222172856 wait complete. 2024-09-10T22:13:55.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 Processor -- start 2024-09-10T22:13:55.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 -- start start 2024-09-10T22:13:55.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448c0b2bf0 0x7f448c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:55.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f448c0d0260 0x7f448c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:55.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f448c15f340 0x7f448c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:55.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f448c16a2b0 con 0x7f448c0b2bf0 2024-09-10T22:13:55.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f448c16a130 con 0x7f448c0d0260 2024-09-10T22:13:55.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4494600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f448c16a430 con 0x7f448c15f340 2024-09-10T22:13:55.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4491c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448c0b2bf0 0x7f448c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:55.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4491c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448c0b2bf0 0x7f448c06f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45198/0 (socket says 172.21.15.66:45198) 2024-09-10T22:13:55.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4491c00640 1 -- 172.21.15.66:0/3358924725 learned_addr learned my addr 172.21.15.66:0/3358924725 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:55.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4491200640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f448c0d0260 0x7f448c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:55.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4492600640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f448c15f340 0x7f448c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:55.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4491c00640 1 -- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f448c15f340 msgr2=0x7f448c15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:55.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4491c00640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f448c15f340 0x7f448c15b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4491c00640 1 -- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f448c0d0260 msgr2=0x7f448c158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:55.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4491c00640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f448c0d0260 0x7f448c158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.235+0000 7f4491c00640 1 -- 172.21.15.66:0/3358924725 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c0b6160 con 0x7f448c0b2bf0 2024-09-10T22:13:55.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.239+0000 7f4491c00640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448c0b2bf0 0x7f448c06f9d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f448000b410 tx=0x7f448000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:55.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.239+0000 7f4489600640 1 -- 172.21.15.66:0/3358924725 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480015030 con 0x7f448c0b2bf0 2024-09-10T22:13:55.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.239+0000 7f4489600640 1 -- 172.21.15.66:0/3358924725 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4480004020 con 0x7f448c0b2bf0 2024-09-10T22:13:55.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.239+0000 7f4494600640 1 -- 172.21.15.66:0/3358924725 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448c0b6440 con 0x7f448c0b2bf0 2024-09-10T22:13:55.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.239+0000 7f4489600640 1 -- 172.21.15.66:0/3358924725 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44800044b0 con 0x7f448c0b2bf0 2024-09-10T22:13:55.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.239+0000 7f4494600640 1 -- 172.21.15.66:0/3358924725 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448c0b6a00 con 0x7f448c0b2bf0 2024-09-10T22:13:55.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.239+0000 7f4494600640 1 -- 172.21.15.66:0/3358924725 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f448c0cfb50 con 0x7f448c0b2bf0 2024-09-10T22:13:55.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.239+0000 7f4489600640 1 -- 172.21.15.66:0/3358924725 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f4480022060 con 0x7f448c0b2bf0 2024-09-10T22:13:55.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.243+0000 7f4489600640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f446c074710 0x7f446c076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:55.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.243+0000 7f4489600640 1 -- 172.21.15.66:0/3358924725 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5083+0+0 (secure 0 0 0) 0x7f4480091ea0 con 0x7f448c0b2bf0 2024-09-10T22:13:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.243+0000 7f4491200640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f446c074710 0x7f446c076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.243+0000 7f4491200640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f446c074710 0x7f446c076bd0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4478007fd0 tx=0x7f4478007ea0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.247+0000 7f4489600640 1 -- 172.21.15.66:0/3358924725 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f448005e910 con 0x7f448c0b2bf0 2024-09-10T22:13:55.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.419+0000 7f4494600640 1 -- 172.21.15.66:0/3358924725 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f448c167be0 con 0x7f448c0b2bf0 2024-09-10T22:13:55.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.423+0000 7f4489600640 1 -- 172.21.15.66:0/3358924725 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v53) ==== 71+0+0 (secure 0 0 0) 0x7f4480063340 con 0x7f448c0b2bf0 2024-09-10T22:13:55.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.427+0000 7f4494600640 1 -- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f446c074710 msgr2=0x7f446c076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.427+0000 7f4494600640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f446c074710 0x7f446c076bd0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4478007fd0 tx=0x7f4478007ea0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.427+0000 7f4494600640 1 -- 172.21.15.66:0/3358924725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448c0b2bf0 msgr2=0x7f448c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.427+0000 7f4494600640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448c0b2bf0 0x7f448c06f9d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f448000b410 tx=0x7f448000b8d0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.427+0000 7f4494600640 1 -- 172.21.15.66:0/3358924725 shutdown_connections 2024-09-10T22:13:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.427+0000 7f4494600640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f446c074710 0x7f446c076bd0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.427+0000 7f4494600640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f448c15f340 0x7f448c15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.427+0000 7f4494600640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f448c0d0260 0x7f448c158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.427+0000 7f4494600640 1 --2- 172.21.15.66:0/3358924725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448c0b2bf0 0x7f448c06f9d0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.427+0000 7f4494600640 1 -- 172.21.15.66:0/3358924725 >> 172.21.15.66:0/3358924725 conn(0x7f448c0d1c50 msgr2=0x7f448c15d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.427+0000 7f4494600640 1 -- 172.21.15.66:0/3358924725 shutdown_connections 2024-09-10T22:13:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.427+0000 7f4494600640 1 -- 172.21.15.66:0/3358924725 wait complete. 2024-09-10T22:13:55.451 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-10T22:13:55.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.591+0000 7fcdc3000640 1 -- 172.21.15.66:0/1196125034 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdbc15f300 msgr2=0x7fcdbc0b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:55.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.591+0000 7fcdc3000640 1 --2- 172.21.15.66:0/1196125034 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdbc15f300 0x7fcdbc0b1090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009840 tx=0x7fcda801c7f0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.591+0000 7fcdc3000640 1 -- 172.21.15.66:0/1196125034 shutdown_connections 2024-09-10T22:13:55.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.591+0000 7fcdc3000640 1 --2- 172.21.15.66:0/1196125034 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdbc15f300 0x7fcdbc0b1090 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.591+0000 7fcdc3000640 1 --2- 172.21.15.66:0/1196125034 >> v2:172.21.15.94:3300/0 conn(0x7fcdbc15ef20 0x7fcdbc0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.591+0000 7fcdc3000640 1 -- 172.21.15.66:0/1196125034 >> 172.21.15.66:0/1196125034 conn(0x7fcdbc0d1b10 msgr2=0x7fcdbc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.591+0000 7fcdc3000640 1 -- 172.21.15.66:0/1196125034 shutdown_connections 2024-09-10T22:13:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.591+0000 7fcdc3000640 1 -- 172.21.15.66:0/1196125034 wait complete. 2024-09-10T22:13:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc3000640 1 Processor -- start 2024-09-10T22:13:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc3000640 1 -- start start 2024-09-10T22:13:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc3000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcdbc0b15d0 0x7fcdbc0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc3000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc15ef20 0x7fcdbc0b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc3000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdbc15f300 0x7fcdbc0cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc3000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc168ee0 con 0x7fcdbc15ef20 2024-09-10T22:13:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc3000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc168d60 con 0x7fcdbc0b15d0 2024-09-10T22:13:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc3000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcdbc169060 con 0x7fcdbc15f300 2024-09-10T22:13:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc1200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc15ef20 0x7fcdbc0b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc1200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc15ef20 0x7fcdbc0b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45206/0 (socket says 172.21.15.66:45206) 2024-09-10T22:13:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc1200640 1 -- 172.21.15.66:0/2983835182 learned_addr learned my addr 172.21.15.66:0/2983835182 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc1c00640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcdbc0b15d0 0x7fcdbc0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:55.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc2600640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdbc15f300 0x7fcdbc0cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:55.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc1200640 1 -- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdbc15f300 msgr2=0x7fcdbc0cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:55.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc1200640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdbc15f300 0x7fcdbc0cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc1200640 1 -- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcdbc0b15d0 msgr2=0x7fcdbc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:55.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc1200640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcdbc0b15d0 0x7fcdbc0c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:55.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc1200640 1 -- 172.21.15.66:0/2983835182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc15d7c0 con 0x7fcdbc15ef20 2024-09-10T22:13:55.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc1200640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc15ef20 0x7fcdbc0b5270 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009790 tx=0x7fcda8005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:55.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdb9600640 1 -- 172.21.15.66:0/2983835182 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda80b2040 con 0x7fcdbc15ef20 2024-09-10T22:13:55.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdb9600640 1 -- 172.21.15.66:0/2983835182 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcda8004330 con 0x7fcdbc15ef20 2024-09-10T22:13:55.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc3000640 1 -- 172.21.15.66:0/2983835182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc0c9a40 con 0x7fcdbc15ef20 2024-09-10T22:13:55.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdb9600640 1 -- 172.21.15.66:0/2983835182 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8003820 con 0x7fcdbc15ef20 2024-09-10T22:13:55.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc3000640 1 -- 172.21.15.66:0/2983835182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc0b4a70 con 0x7fcdbc15ef20 2024-09-10T22:13:55.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.595+0000 7fcdc3000640 1 -- 172.21.15.66:0/2983835182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdbc0cecc0 con 0x7fcdbc15ef20 2024-09-10T22:13:55.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.599+0000 7fcdb9600640 1 -- 172.21.15.66:0/2983835182 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fcda80b5070 con 0x7fcdbc15ef20 2024-09-10T22:13:55.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.599+0000 7fcdb9600640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd9c074710 0x7fcd9c076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:55.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.599+0000 7fcdb9600640 1 -- 172.21.15.66:0/2983835182 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5083+0+0 (secure 0 0 0) 0x7fcda812ba10 con 0x7fcdbc15ef20 2024-09-10T22:13:55.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.599+0000 7fcdc1c00640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd9c074710 0x7fcd9c076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:55.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.599+0000 7fcdc1c00640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd9c074710 0x7fcd9c076bd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0b7b80 tx=0x7fcdb00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:55.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.603+0000 7fcdb9600640 1 -- 172.21.15.66:0/2983835182 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcdbc15d7c0 con 0x7fcdbc15ef20 2024-09-10T22:13:55.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:55.839+0000 7fcdc3000640 1 -- 172.21.15.66:0/2983835182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7fcdbc072e00 con 0x7fcdbc15ef20 2024-09-10T22:13:56.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.043+0000 7fcdb9600640 1 -- 172.21.15.66:0/2983835182 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v613) ==== 186+0+0 (secure 0 0 0) 0x7fcda8016410 con 0x7fcdbc15ef20 2024-09-10T22:13:56.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.047+0000 7fcdc3000640 1 -- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd9c074710 msgr2=0x7fcd9c076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.047+0000 7fcdc3000640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd9c074710 0x7fcd9c076bd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0b7b80 tx=0x7fcdb00091e0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.047+0000 7fcdc3000640 1 -- 172.21.15.66:0/2983835182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc15ef20 msgr2=0x7fcdbc0b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.047+0000 7fcdc3000640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc15ef20 0x7fcdbc0b5270 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009790 tx=0x7fcda8005d20 comp rx=0 tx=0).stop 2024-09-10T22:13:56.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.047+0000 7fcdc3000640 1 -- 172.21.15.66:0/2983835182 shutdown_connections 2024-09-10T22:13:56.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.047+0000 7fcdc3000640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd9c074710 0x7fcd9c076bd0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.047+0000 7fcdc3000640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcdbc15f300 0x7fcdbc0cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.047+0000 7fcdc3000640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdbc15ef20 0x7fcdbc0b5270 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.047+0000 7fcdc3000640 1 --2- 172.21.15.66:0/2983835182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcdbc0b15d0 0x7fcdbc0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.047+0000 7fcdc3000640 1 -- 172.21.15.66:0/2983835182 >> 172.21.15.66:0/2983835182 conn(0x7fcdbc0d1b10 msgr2=0x7fcdbc06e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:56.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.047+0000 7fcdc3000640 1 -- 172.21.15.66:0/2983835182 shutdown_connections 2024-09-10T22:13:56.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.047+0000 7fcdc3000640 1 -- 172.21.15.66:0/2983835182 wait complete. 2024-09-10T22:13:56.071 INFO:tasks.cephfs_test_runner:Test that the progress bar that is printed for 1 ongoing clone job is ... ok 2024-09-10T22:13:56.071 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that both the progress bars, that are printed for all 7 clone 2024-09-10T22:13:56.071 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-10T22:13:56.071 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:13:56.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.211+0000 7fc452e00640 1 -- 172.21.15.66:0/4012776040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c15f340 msgr2=0x7fc44c0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.211+0000 7fc452e00640 1 --2- 172.21.15.66:0/4012776040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c15f340 0x7fc44c0cfb50 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009900 tx=0x7fc43c01c850 comp rx=0 tx=0).stop 2024-09-10T22:13:56.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.211+0000 7fc452e00640 1 -- 172.21.15.66:0/4012776040 shutdown_connections 2024-09-10T22:13:56.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.211+0000 7fc452e00640 1 --2- 172.21.15.66:0/4012776040 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc44c0b2bf0 0x7fc44c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.211+0000 7fc452e00640 1 --2- 172.21.15.66:0/4012776040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c15f340 0x7fc44c0cfb50 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.211+0000 7fc452e00640 1 -- 172.21.15.66:0/4012776040 >> 172.21.15.66:0/4012776040 conn(0x7fc44c0d1c50 msgr2=0x7fc44c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:56.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.211+0000 7fc452e00640 1 -- 172.21.15.66:0/4012776040 shutdown_connections 2024-09-10T22:13:56.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.211+0000 7fc452e00640 1 -- 172.21.15.66:0/4012776040 wait complete. 2024-09-10T22:13:56.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc452e00640 1 Processor -- start 2024-09-10T22:13:56.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc452e00640 1 -- start start 2024-09-10T22:13:56.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc452e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c0b2bf0 0x7fc44c1f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:56.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc452e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc44c0d2770 0x7fc44c15b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:56.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc452e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc44c15f340 0x7fc44c0cf940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:56.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc452e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c169190 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc452e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c169010 con 0x7fc44c15f340 2024-09-10T22:13:56.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc452e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc44c169310 con 0x7fc44c0d2770 2024-09-10T22:13:56.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc44be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c0b2bf0 0x7fc44c1f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:56.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc44be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c0b2bf0 0x7fc44c1f8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45224/0 (socket says 172.21.15.66:45224) 2024-09-10T22:13:56.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc44be00640 1 -- 172.21.15.66:0/3244900167 learned_addr learned my addr 172.21.15.66:0/3244900167 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:56.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc443e00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc44c0d2770 0x7fc44c15b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:56.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc450e00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc44c15f340 0x7fc44c0cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:56.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc44be00640 1 -- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc44c0d2770 msgr2=0x7fc44c15b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc44be00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc44c0d2770 0x7fc44c15b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc44be00640 1 -- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc44c15f340 msgr2=0x7fc44c0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc44be00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc44c15f340 0x7fc44c0cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc44be00640 1 -- 172.21.15.66:0/3244900167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44c157ea0 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc44be00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c0b2bf0 0x7fc44c1f8440 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009850 tx=0x7fc43c0a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:56.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc448c00640 1 -- 172.21.15.66:0/3244900167 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0bb070 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc448c00640 1 -- 172.21.15.66:0/3244900167 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc43c0a6cd0 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc448c00640 1 -- 172.21.15.66:0/3244900167 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0b2040 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.215+0000 7fc452e00640 1 -- 172.21.15.66:0/3244900167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44c158180 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.219+0000 7fc452e00640 1 -- 172.21.15.66:0/3244900167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc44c158740 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.219+0000 7fc452e00640 1 -- 172.21.15.66:0/3244900167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc44c15fac0 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.219+0000 7fc448c00640 1 -- 172.21.15.66:0/3244900167 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fc43c003860 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.219+0000 7fc448c00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc428074740 0x7fc428076c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:56.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.219+0000 7fc448c00640 1 -- 172.21.15.66:0/3244900167 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5083+0+0 (secure 0 0 0) 0x7fc43c12be00 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.219+0000 7fc443e00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc428074740 0x7fc428076c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:56.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.223+0000 7fc443e00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc428074740 0x7fc428076c00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc44c06f6e0 tx=0x7fc43400a2d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:56.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.223+0000 7fc448c00640 1 -- 172.21.15.66:0/3244900167 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc43c0164d0 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.399+0000 7fc452e00640 1 -- 172.21.15.66:0/3244900167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc44c15be00 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.399+0000 7fc448c00640 1 -- 172.21.15.66:0/3244900167 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+13193 (secure 0 0 0) 0x7fc43c0f8930 con 0x7fc44c0b2bf0 2024-09-10T22:13:56.404 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:56.404 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":132,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:13:55.038982+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6836/1762705824":"2024-09-11T22:13:52.006587+0000","172.21.15.94:6834/1762705824":"2024-09-11T22:13:52.006587+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:13:56.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.403+0000 7fc452e00640 1 -- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc428074740 msgr2=0x7fc428076c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.403+0000 7fc452e00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc428074740 0x7fc428076c00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc44c06f6e0 tx=0x7fc43400a2d0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.403+0000 7fc452e00640 1 -- 172.21.15.66:0/3244900167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c0b2bf0 msgr2=0x7fc44c1f8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.403+0000 7fc452e00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c0b2bf0 0x7fc44c1f8440 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009850 tx=0x7fc43c0a5620 comp rx=0 tx=0).stop 2024-09-10T22:13:56.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.403+0000 7fc452e00640 1 -- 172.21.15.66:0/3244900167 shutdown_connections 2024-09-10T22:13:56.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.403+0000 7fc452e00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc428074740 0x7fc428076c00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.403+0000 7fc452e00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc44c15f340 0x7fc44c0cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.403+0000 7fc452e00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc44c0d2770 0x7fc44c15b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.403+0000 7fc452e00640 1 --2- 172.21.15.66:0/3244900167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c0b2bf0 0x7fc44c1f8440 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.403+0000 7fc452e00640 1 -- 172.21.15.66:0/3244900167 >> 172.21.15.66:0/3244900167 conn(0x7fc44c0d1c50 msgr2=0x7fc44c15c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:56.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.403+0000 7fc452e00640 1 -- 172.21.15.66:0/3244900167 shutdown_connections 2024-09-10T22:13:56.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.403+0000 7fc452e00640 1 -- 172.21.15.66:0/3244900167 wait complete. 2024-09-10T22:13:56.427 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:00:22.340318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:13:56.427 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:13:56.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.571+0000 7f9aa0200640 1 -- 172.21.15.66:0/688612273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a80004810 msgr2=0x7f9a80004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.571+0000 7f9aa0200640 1 --2- 172.21.15.66:0/688612273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a80004810 0x7f9a80004c90 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f9a84001ec0 tx=0x7f9a8401f5d0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.571+0000 7f9aa0200640 1 -- 172.21.15.66:0/688612273 shutdown_connections 2024-09-10T22:13:56.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.571+0000 7f9aa0200640 1 --2- 172.21.15.66:0/688612273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a80004810 0x7f9a80004c90 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.571+0000 7f9aa0200640 1 -- 172.21.15.66:0/688612273 >> 172.21.15.66:0/688612273 conn(0x7f9a980cfa30 msgr2=0x7f9a980cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:56.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.571+0000 7f9aa0200640 1 -- 172.21.15.66:0/688612273 shutdown_connections 2024-09-10T22:13:56.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.571+0000 7f9aa0200640 1 -- 172.21.15.66:0/688612273 wait complete. 2024-09-10T22:13:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9aa0200640 1 Processor -- start 2024-09-10T22:13:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9aa0200640 1 -- start start 2024-09-10T22:13:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9aa0200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a80004810 0x7f9a9814c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9aa0200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a980c9650 0x7f9a980c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9aa0200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a980c9ad0 0x7f9a980c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9aa0200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a980729b0 con 0x7f9a980c9ad0 2024-09-10T22:13:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9aa0200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98072830 con 0x7f9a80004810 2024-09-10T22:13:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9aa0200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9a98072b30 con 0x7f9a980c9650 2024-09-10T22:13:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a9e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a980c9ad0 0x7f9a980c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a9e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a980c9ad0 0x7f9a980c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45244/0 (socket says 172.21.15.66:45244) 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a9e200640 1 -- 172.21.15.66:0/3305877888 learned_addr learned my addr 172.21.15.66:0/3305877888 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a9d800640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a80004810 0x7f9a9814c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a9ce00640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a980c9650 0x7f9a980c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a9e200640 1 -- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a980c9650 msgr2=0x7f9a980c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a9e200640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a980c9650 0x7f9a980c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a9e200640 1 -- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a80004810 msgr2=0x7f9a9814c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a9e200640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a80004810 0x7f9a9814c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a9e200640 1 -- 172.21.15.66:0/3305877888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a981f4f60 con 0x7f9a980c9ad0 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a9e200640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a980c9ad0 0x7f9a980c7040 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f9a98071f70 tx=0x7f9a90010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a95600640 1 -- 172.21.15.66:0/3305877888 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000ede0 con 0x7f9a980c9ad0 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a95600640 1 -- 172.21.15.66:0/3305877888 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a90015070 con 0x7f9a980c9ad0 2024-09-10T22:13:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9a95600640 1 -- 172.21.15.66:0/3305877888 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90003d10 con 0x7f9a980c9ad0 2024-09-10T22:13:56.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9aa0200640 1 -- 172.21.15.66:0/3305877888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a981f5240 con 0x7f9a980c9ad0 2024-09-10T22:13:56.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.575+0000 7f9aa0200640 1 -- 172.21.15.66:0/3305877888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a980c8e50 con 0x7f9a980c9ad0 2024-09-10T22:13:56.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.579+0000 7f9aa0200640 1 -- 172.21.15.66:0/3305877888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a980c5170 con 0x7f9a980c9ad0 2024-09-10T22:13:56.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.579+0000 7f9a95600640 1 -- 172.21.15.66:0/3305877888 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f9a90024060 con 0x7f9a980c9ad0 2024-09-10T22:13:56.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.579+0000 7f9a95600640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a78074710 0x7f9a78076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:56.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.579+0000 7f9a95600640 1 -- 172.21.15.66:0/3305877888 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5083+0+0 (secure 0 0 0) 0x7f9a90092810 con 0x7f9a980c9ad0 2024-09-10T22:13:56.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.579+0000 7f9a9d800640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a78074710 0x7f9a78076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:56.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.579+0000 7f9a9d800640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a78074710 0x7f9a78076bd0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c000c10 tx=0x7f9a8c006b80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:56.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.583+0000 7f9a95600640 1 -- 172.21.15.66:0/3305877888 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9a9005f300 con 0x7f9a980c9ad0 2024-09-10T22:13:56.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 -- 172.21.15.66:0/3305877888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9a9814ca50 con 0x7f9a980c9ad0 2024-09-10T22:13:56.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9a95600640 1 -- 172.21.15.66:0/3305877888 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) ==== 93+0+10 (secure 0 0 0) 0x7f9a90063d30 con 0x7f9a980c9ad0 2024-09-10T22:13:56.763 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-09-10T22:13:56.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 -- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a78074710 msgr2=0x7f9a78076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a78074710 0x7f9a78076bd0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c000c10 tx=0x7f9a8c006b80 comp rx=0 tx=0).stop 2024-09-10T22:13:56.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 -- 172.21.15.66:0/3305877888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a980c9ad0 msgr2=0x7f9a980c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a980c9ad0 0x7f9a980c7040 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f9a98071f70 tx=0x7f9a90010f20 comp rx=0 tx=0).stop 2024-09-10T22:13:56.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 -- 172.21.15.66:0/3305877888 shutdown_connections 2024-09-10T22:13:56.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9a78074710 0x7f9a78076bd0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a980c9ad0 0x7f9a980c7040 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9a980c9650 0x7f9a980c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 --2- 172.21.15.66:0/3305877888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a80004810 0x7f9a9814c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 -- 172.21.15.66:0/3305877888 >> 172.21.15.66:0/3305877888 conn(0x7f9a980cfa30 msgr2=0x7f9a980af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:56.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 -- 172.21.15.66:0/3305877888 shutdown_connections 2024-09-10T22:13:56.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.759+0000 7f9aa0200640 1 -- 172.21.15.66:0/3305877888 wait complete. 2024-09-10T22:13:56.785 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-10T22:13:56.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.927+0000 7f57df000640 1 -- 172.21.15.66:0/1929255306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57bc003c40 msgr2=0x7f57bc004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.927+0000 7f57df000640 1 --2- 172.21.15.66:0/1929255306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57bc003c40 0x7f57bc004040 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f57c8006fb0 tx=0x7f57c801cf20 comp rx=0 tx=0).stop 2024-09-10T22:13:56.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.927+0000 7f57df000640 1 -- 172.21.15.66:0/1929255306 shutdown_connections 2024-09-10T22:13:56.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.927+0000 7f57df000640 1 --2- 172.21.15.66:0/1929255306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57bc003c40 0x7f57bc004040 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.927+0000 7f57df000640 1 -- 172.21.15.66:0/1929255306 >> 172.21.15.66:0/1929255306 conn(0x7f57d80d3e70 msgr2=0x7f57d80d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:56.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.927+0000 7f57df000640 1 -- 172.21.15.66:0/1929255306 shutdown_connections 2024-09-10T22:13:56.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.927+0000 7f57df000640 1 -- 172.21.15.66:0/1929255306 wait complete. 2024-09-10T22:13:56.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.927+0000 7f57df000640 1 Processor -- start 2024-09-10T22:13:56.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57df000640 1 -- start start 2024-09-10T22:13:56.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57df000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57bc003c40 0x7f57d80cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:56.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57df000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d81544b0 0x7f57d815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57df000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d81f7f00 0x7f57d81f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57df000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57d80d0940 con 0x7f57d81544b0 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57df000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f57d80d07c0 con 0x7f57d81f7f00 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57df000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f57d80d0ac0 con 0x7f57bc003c40 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d81544b0 0x7f57d815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d81544b0 0x7f57d815add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45268/0 (socket says 172.21.15.66:45268) 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57cfe00640 1 -- 172.21.15.66:0/3383138817 learned_addr learned my addr 172.21.15.66:0/3383138817 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57dd000640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d81f7f00 0x7f57d81f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57d7e00640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57bc003c40 0x7f57d80cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57cfe00640 1 -- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57bc003c40 msgr2=0x7f57d80cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57cfe00640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57bc003c40 0x7f57d80cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57cfe00640 1 -- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d81f7f00 msgr2=0x7f57d81f8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:56.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57cfe00640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d81f7f00 0x7f57d81f8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57cfe00640 1 -- 172.21.15.66:0/3383138817 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d81f8f30 con 0x7f57d81544b0 2024-09-10T22:13:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57cfe00640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d81544b0 0x7f57d815add0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f57c0004c60 tx=0x7f57c0001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57d4c00640 1 -- 172.21.15.66:0/3383138817 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0015020 con 0x7f57d81544b0 2024-09-10T22:13:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57d4c00640 1 -- 172.21.15.66:0/3383138817 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57c0011980 con 0x7f57d81544b0 2024-09-10T22:13:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57d4c00640 1 -- 172.21.15.66:0/3383138817 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0011dd0 con 0x7f57d81544b0 2024-09-10T22:13:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57df000640 1 -- 172.21.15.66:0/3383138817 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d81f8970 con 0x7f57d81544b0 2024-09-10T22:13:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57df000640 1 -- 172.21.15.66:0/3383138817 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d8156aa0 con 0x7f57d81544b0 2024-09-10T22:13:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.931+0000 7f57df000640 1 -- 172.21.15.66:0/3383138817 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d80d1020 con 0x7f57d81544b0 2024-09-10T22:13:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.935+0000 7f57d4c00640 1 -- 172.21.15.66:0/3383138817 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f57c00103c0 con 0x7f57d81544b0 2024-09-10T22:13:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.935+0000 7f57d4c00640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f57b4074710 0x7f57b4076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.935+0000 7f57d4c00640 1 -- 172.21.15.66:0/3383138817 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5083+0+0 (secure 0 0 0) 0x7f57c0092cf0 con 0x7f57d81544b0 2024-09-10T22:13:56.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.935+0000 7f57d7e00640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f57b4074710 0x7f57b4076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:56.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.935+0000 7f57d7e00640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f57b4074710 0x7f57b4076bd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f57c80ac000 tx=0x7f57c8019d60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:56.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:56.939+0000 7f57d4c00640 1 -- 172.21.15.66:0/3383138817 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f57c001f030 con 0x7f57d81544b0 2024-09-10T22:13:57.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:57.179+0000 7f57df000640 1 -- 172.21.15.66:0/3383138817 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7f57d80d3c50 con 0x7f57d81544b0 2024-09-10T22:13:57.914 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:13:57.919 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:13:58.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.051+0000 7f57d4c00640 1 -- 172.21.15.66:0/3383138817 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v615) ==== 189+0+0 (secure 0 0 0) 0x7f57c005feb0 con 0x7f57d81544b0 2024-09-10T22:13:58.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.055+0000 7f57df000640 1 -- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f57b4074710 msgr2=0x7f57b4076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:58.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.055+0000 7f57df000640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f57b4074710 0x7f57b4076bd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f57c80ac000 tx=0x7f57c8019d60 comp rx=0 tx=0).stop 2024-09-10T22:13:58.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.055+0000 7f57df000640 1 -- 172.21.15.66:0/3383138817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d81544b0 msgr2=0x7f57d815add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:58.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.055+0000 7f57df000640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d81544b0 0x7f57d815add0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f57c0004c60 tx=0x7f57c0001810 comp rx=0 tx=0).stop 2024-09-10T22:13:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.055+0000 7f57df000640 1 -- 172.21.15.66:0/3383138817 shutdown_connections 2024-09-10T22:13:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.055+0000 7f57df000640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f57b4074710 0x7f57b4076bd0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.055+0000 7f57df000640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d81f7f00 0x7f57d81f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.055+0000 7f57df000640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d81544b0 0x7f57d815add0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.055+0000 7f57df000640 1 --2- 172.21.15.66:0/3383138817 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57bc003c40 0x7f57d80cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.055+0000 7f57df000640 1 -- 172.21.15.66:0/3383138817 >> 172.21.15.66:0/3383138817 conn(0x7f57d80d3e70 msgr2=0x7f57d806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.055+0000 7f57df000640 1 -- 172.21.15.66:0/3383138817 shutdown_connections 2024-09-10T22:13:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.055+0000 7f57df000640 1 -- 172.21.15.66:0/3383138817 wait complete. 2024-09-10T22:13:58.081 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T22:13:58.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 -- 172.21.15.66:0/3186693744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5934159900 msgr2=0x7f59340ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 --2- 172.21.15.66:0/3186693744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5934159900 0x7f59340ca080 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5924009840 tx=0x7f592401c790 comp rx=0 tx=0).stop 2024-09-10T22:13:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 -- 172.21.15.66:0/3186693744 shutdown_connections 2024-09-10T22:13:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 --2- 172.21.15.66:0/3186693744 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f59340aee20 0x7f59340cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 --2- 172.21.15.66:0/3186693744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5934159900 0x7f59340ca080 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 -- 172.21.15.66:0/3186693744 >> 172.21.15.66:0/3186693744 conn(0x7f59340cfa30 msgr2=0x7f59340cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 -- 172.21.15.66:0/3186693744 shutdown_connections 2024-09-10T22:13:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 -- 172.21.15.66:0/3186693744 wait complete. 2024-09-10T22:13:58.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 Processor -- start 2024-09-10T22:13:58.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 -- start start 2024-09-10T22:13:58.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59340aee20 0x7f593414b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:58.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5934159900 0x7f59340b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:58.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593415cc80 0x7f593415b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:58.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5934168f70 con 0x7f593415cc80 2024-09-10T22:13:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5934168df0 con 0x7f59340aee20 2024-09-10T22:13:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5939200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f59341690f0 con 0x7f5934159900 2024-09-10T22:13:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5932800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593415cc80 0x7f593415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5932800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593415cc80 0x7f593415b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45276/0 (socket says 172.21.15.66:45276) 2024-09-10T22:13:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5932800640 1 -- 172.21.15.66:0/1322771574 learned_addr learned my addr 172.21.15.66:0/1322771574 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f592be00640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5934159900 0x7f59340b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5931e00640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59340aee20 0x7f593414b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5932800640 1 -- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5934159900 msgr2=0x7f59340b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5932800640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5934159900 0x7f59340b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5932800640 1 -- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59340aee20 msgr2=0x7f593414b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5932800640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59340aee20 0x7f593414b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5932800640 1 -- 172.21.15.66:0/1322771574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59340b0c30 con 0x7f593415cc80 2024-09-10T22:13:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5931e00640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59340aee20 0x7f593414b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.227+0000 7f5932800640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593415cc80 0x7f593415b630 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f592c00eb20 tx=0x7f592c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.231+0000 7f592a000640 1 -- 172.21.15.66:0/1322771574 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c004b30 con 0x7f593415cc80 2024-09-10T22:13:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.231+0000 7f592a000640 1 -- 172.21.15.66:0/1322771574 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f592c004cf0 con 0x7f593415cc80 2024-09-10T22:13:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.231+0000 7f592a000640 1 -- 172.21.15.66:0/1322771574 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c005280 con 0x7f593415cc80 2024-09-10T22:13:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.231+0000 7f5939200640 1 -- 172.21.15.66:0/1322771574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59340b0f10 con 0x7f593415cc80 2024-09-10T22:13:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.231+0000 7f5939200640 1 -- 172.21.15.66:0/1322771574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5934154310 con 0x7f593415cc80 2024-09-10T22:13:58.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.231+0000 7f5939200640 1 -- 172.21.15.66:0/1322771574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5934154510 con 0x7f593415cc80 2024-09-10T22:13:58.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.231+0000 7f592a000640 1 -- 172.21.15.66:0/1322771574 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f592c005420 con 0x7f593415cc80 2024-09-10T22:13:58.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.231+0000 7f592a000640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5910074710 0x7f5910076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:58.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.231+0000 7f592a000640 1 -- 172.21.15.66:0/1322771574 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5083+0+0 (secure 0 0 0) 0x7f592c092f90 con 0x7f593415cc80 2024-09-10T22:13:58.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.231+0000 7f5931e00640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5910074710 0x7f5910076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:58.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.235+0000 7f5931e00640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5910074710 0x7f5910076bd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f59240ae000 tx=0x7f592401afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:58.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.235+0000 7f592a000640 1 -- 172.21.15.66:0/1322771574 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5934154510 con 0x7f593415cc80 2024-09-10T22:13:58.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.415+0000 7f5939200640 1 -- 172.21.15.66:0/1322771574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f59340cd930 con 0x7f593415cc80 2024-09-10T22:13:58.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.415+0000 7f592a000640 1 -- 172.21.15.66:0/1322771574 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v53) ==== 59+0+1998 (secure 0 0 0) 0x7f592c060170 con 0x7f593415cc80 2024-09-10T22:13:58.421 INFO:teuthology.orchestra.run.smithi066.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T22:13:58.421 INFO:teuthology.orchestra.run.smithi066.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4673.907365 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4537.831230 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 5706.341617 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4484.013907 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3862.285837 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3776.439661 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3820.270368 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3863.602758 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds 20/20 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T22:13:58.422 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_ms 2/2 2024-09-10T22:13:58.423 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_frag true 2024-09-10T22:13:58.423 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_scatterstat true 2024-09-10T22:13:58.423 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T22:13:58.423 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_verify_scatter true 2024-09-10T22:13:58.423 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T22:13:58.423 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T22:13:58.423 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T22:13:58.423 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced client_mount_timeout 600 2024-09-10T22:13:58.423 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_client 20/20 2024-09-10T22:13:58.423 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_ms 1/1 2024-09-10T22:13:58.423 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T22:13:58.423 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T22:13:58.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.419+0000 7f5939200640 1 -- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5910074710 msgr2=0x7f5910076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:58.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.419+0000 7f5939200640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5910074710 0x7f5910076bd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f59240ae000 tx=0x7f592401afd0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.419+0000 7f5939200640 1 -- 172.21.15.66:0/1322771574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593415cc80 msgr2=0x7f593415b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.419+0000 7f5939200640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593415cc80 0x7f593415b630 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f592c00eb20 tx=0x7f592c00c2d0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.419+0000 7f5939200640 1 -- 172.21.15.66:0/1322771574 shutdown_connections 2024-09-10T22:13:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.419+0000 7f5939200640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5910074710 0x7f5910076bd0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.419+0000 7f5939200640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593415cc80 0x7f593415b630 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.419+0000 7f5939200640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5934159900 0x7f59340b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.419+0000 7f5939200640 1 --2- 172.21.15.66:0/1322771574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59340aee20 0x7f593414b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.419+0000 7f5939200640 1 -- 172.21.15.66:0/1322771574 >> 172.21.15.66:0/1322771574 conn(0x7f59340cfa30 msgr2=0x7f59340725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.419+0000 7f5939200640 1 -- 172.21.15.66:0/1322771574 shutdown_connections 2024-09-10T22:13:58.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.419+0000 7f5939200640 1 -- 172.21.15.66:0/1322771574 wait complete. 2024-09-10T22:13:58.443 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:13:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.587+0000 7f899c000640 1 -- 172.21.15.66:0/1962475318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89940cf830 msgr2=0x7f89940d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.587+0000 7f899c000640 1 --2- 172.21.15.66:0/1962475318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89940cf830 0x7f89940d04e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8980009840 tx=0x7f898001c790 comp rx=0 tx=0).stop 2024-09-10T22:13:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.587+0000 7f899c000640 1 -- 172.21.15.66:0/1962475318 shutdown_connections 2024-09-10T22:13:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.587+0000 7f899c000640 1 --2- 172.21.15.66:0/1962475318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89940cf830 0x7f89940d04e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.587+0000 7f899c000640 1 --2- 172.21.15.66:0/1962475318 >> v2:172.21.15.94:3300/0 conn(0x7f899415bd40 0x7f89940cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.587+0000 7f899c000640 1 -- 172.21.15.66:0/1962475318 >> 172.21.15.66:0/1962475318 conn(0x7f89940d1c60 msgr2=0x7f89940d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.587+0000 7f899c000640 1 -- 172.21.15.66:0/1962475318 shutdown_connections 2024-09-10T22:13:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.587+0000 7f899c000640 1 -- 172.21.15.66:0/1962475318 wait complete. 2024-09-10T22:13:58.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899c000640 1 Processor -- start 2024-09-10T22:13:58.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899c000640 1 -- start start 2024-09-10T22:13:58.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899c000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89940aec10 0x7f89940b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:58.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899c000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89940cf830 0x7f89940b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:58.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899c000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f899415bd40 0x7f899414c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:58.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899c000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89941690c0 con 0x7f899415bd40 2024-09-10T22:13:58.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899c000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8994168f40 con 0x7f89940cf830 2024-09-10T22:13:58.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899c000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8994169240 con 0x7f89940aec10 2024-09-10T22:13:58.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f899415bd40 0x7f899414c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:58.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f899415bd40 0x7f899414c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45280/0 (socket says 172.21.15.66:45280) 2024-09-10T22:13:58.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899a000640 1 -- 172.21.15.66:0/3008918949 learned_addr learned my addr 172.21.15.66:0/3008918949 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:58.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f8998c00640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89940cf830 0x7f89940b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:58.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f8999600640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89940aec10 0x7f89940b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:58.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899a000640 1 -- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89940aec10 msgr2=0x7f89940b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:58.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899a000640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89940aec10 0x7f89940b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899a000640 1 -- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89940cf830 msgr2=0x7f89940b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:58.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899a000640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89940cf830 0x7f89940b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899a000640 1 -- 172.21.15.66:0/3008918949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89940b6e60 con 0x7f899415bd40 2024-09-10T22:13:58.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899a000640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f899415bd40 0x7f899414c7c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f898c00cbc0 tx=0x7f898c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:58.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f8991600640 1 -- 172.21.15.66:0/3008918949 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c016020 con 0x7f899415bd40 2024-09-10T22:13:58.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f8991600640 1 -- 172.21.15.66:0/3008918949 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f898c00ab00 con 0x7f899415bd40 2024-09-10T22:13:58.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f8991600640 1 -- 172.21.15.66:0/3008918949 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c012420 con 0x7f899415bd40 2024-09-10T22:13:58.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899c000640 1 -- 172.21.15.66:0/3008918949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89940b7140 con 0x7f899415bd40 2024-09-10T22:13:58.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.591+0000 7f899c000640 1 -- 172.21.15.66:0/3008918949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89940b79e0 con 0x7f899415bd40 2024-09-10T22:13:58.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.595+0000 7f899c000640 1 -- 172.21.15.66:0/3008918949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89940b7530 con 0x7f899415bd40 2024-09-10T22:13:58.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.595+0000 7f8991600640 1 -- 172.21.15.66:0/3008918949 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f898c0125c0 con 0x7f899415bd40 2024-09-10T22:13:58.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.595+0000 7f8991600640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8974074710 0x7f8974076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:58.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.595+0000 7f8991600640 1 -- 172.21.15.66:0/3008918949 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5083+0+0 (secure 0 0 0) 0x7f898c092840 con 0x7f899415bd40 2024-09-10T22:13:58.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.595+0000 7f8999600640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8974074710 0x7f8974076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:58.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.595+0000 7f8999600640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8974074710 0x7f8974076bd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8988009b40 tx=0x7f89880091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.599+0000 7f8991600640 1 -- 172.21.15.66:0/3008918949 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f898c020020 con 0x7f899415bd40 2024-09-10T22:13:58.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.775+0000 7f899c000640 1 -- 172.21.15.66:0/3008918949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f89940af010 con 0x7f899415bd40 2024-09-10T22:13:58.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.779+0000 7f8991600640 1 -- 172.21.15.66:0/3008918949 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7f898c05f330 con 0x7f899415bd40 2024-09-10T22:13:58.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.783+0000 7f899c000640 1 -- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8974074710 msgr2=0x7f8974076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:58.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.783+0000 7f899c000640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8974074710 0x7f8974076bd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8988009b40 tx=0x7f89880091e0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.783+0000 7f899c000640 1 -- 172.21.15.66:0/3008918949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f899415bd40 msgr2=0x7f899414c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.783+0000 7f899c000640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f899415bd40 0x7f899414c7c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f898c00cbc0 tx=0x7f898c00ccb0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.783+0000 7f899c000640 1 -- 172.21.15.66:0/3008918949 shutdown_connections 2024-09-10T22:13:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.783+0000 7f899c000640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8974074710 0x7f8974076bd0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.783+0000 7f899c000640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f899415bd40 0x7f899414c7c0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.783+0000 7f899c000640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89940cf830 0x7f89940b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.783+0000 7f899c000640 1 --2- 172.21.15.66:0/3008918949 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89940aec10 0x7f89940b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.783+0000 7f899c000640 1 -- 172.21.15.66:0/3008918949 >> 172.21.15.66:0/3008918949 conn(0x7f89940d1c60 msgr2=0x7f899415acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.783+0000 7f899c000640 1 -- 172.21.15.66:0/3008918949 shutdown_connections 2024-09-10T22:13:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:58.783+0000 7f899c000640 1 -- 172.21.15.66:0/3008918949 wait complete. 2024-09-10T22:13:58.807 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:13:58.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:13:58.892 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:13:58.980 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:13:58.981 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:13:58.981 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:13:58.990 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:13:58.990 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:13:58.990 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:13:58.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:13:58.999 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:13:58.999 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:13:59.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:13:59.046 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:13:59.046 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:13:59.055 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:13:59.055 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T22:13:59.055 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:13:59.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.231+0000 7f5f9ba00640 1 -- 172.21.15.66:0/3226088357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f940aeec0 msgr2=0x7f5f9415df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:59.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.231+0000 7f5f9ba00640 1 --2- 172.21.15.66:0/3226088357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f940aeec0 0x7f5f9415df60 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f5f94069da0 tx=0x7f5f8c01ccf0 comp rx=0 tx=0).stop 2024-09-10T22:13:59.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.231+0000 7f5f9ba00640 1 -- 172.21.15.66:0/3226088357 shutdown_connections 2024-09-10T22:13:59.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.231+0000 7f5f9ba00640 1 --2- 172.21.15.66:0/3226088357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f940aeec0 0x7f5f9415df60 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:59.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.231+0000 7f5f9ba00640 1 --2- 172.21.15.66:0/3226088357 >> v2:172.21.15.94:3300/0 conn(0x7f5f940d0550 0x7f5f940c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:59.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.231+0000 7f5f9ba00640 1 --2- 172.21.15.66:0/3226088357 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f9415a310 0x7f5f940c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:59.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.231+0000 7f5f9ba00640 1 -- 172.21.15.66:0/3226088357 >> 172.21.15.66:0/3226088357 conn(0x7f5f940cfa30 msgr2=0x7f5f940cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:59.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.231+0000 7f5f9ba00640 1 -- 172.21.15.66:0/3226088357 shutdown_connections 2024-09-10T22:13:59.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.231+0000 7f5f9ba00640 1 -- 172.21.15.66:0/3226088357 wait complete. 2024-09-10T22:13:59.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.231+0000 7f5f9ba00640 1 Processor -- start 2024-09-10T22:13:59.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.231+0000 7f5f9ba00640 1 -- start start 2024-09-10T22:13:59.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f9ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f940aeec0 0x7f5f940744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:59.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f9ba00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f9415a310 0x7f5f94072590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:59.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f9ba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f94147dc0 0x7f5f9414ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:59.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f9ba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9415eb00 con 0x7f5f940aeec0 2024-09-10T22:13:59.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f9ba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9415e980 con 0x7f5f94147dc0 2024-09-10T22:13:59.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f9ba00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9415ec80 con 0x7f5f9415a310 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f99000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f940aeec0 0x7f5f940744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f99000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f940aeec0 0x7f5f940744a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45306/0 (socket says 172.21.15.66:45306) 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f99000640 1 -- 172.21.15.66:0/1883521991 learned_addr learned my addr 172.21.15.66:0/1883521991 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f93e00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f9415a310 0x7f5f94072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f99a00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f94147dc0 0x7f5f9414ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f99000640 1 -- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f9415a310 msgr2=0x7f5f94072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f99000640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f9415a310 0x7f5f94072590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f99000640 1 -- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f94147dc0 msgr2=0x7f5f9414ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f99000640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f94147dc0 0x7f5f9414ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f99000640 1 -- 172.21.15.66:0/1883521991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f940b2a70 con 0x7f5f940aeec0 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f99a00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f94147dc0 0x7f5f9414ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f93e00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f9415a310 0x7f5f94072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:13:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f99000640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f940aeec0 0x7f5f940744a0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800d750 tx=0x7f5f8800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f90c00640 1 -- 172.21.15.66:0/1883521991 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88017070 con 0x7f5f940aeec0 2024-09-10T22:13:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f90c00640 1 -- 172.21.15.66:0/1883521991 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f88009d20 con 0x7f5f940aeec0 2024-09-10T22:13:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f90c00640 1 -- 172.21.15.66:0/1883521991 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8800de60 con 0x7f5f940aeec0 2024-09-10T22:13:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f9ba00640 1 -- 172.21.15.66:0/1883521991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f940b2d50 con 0x7f5f940aeec0 2024-09-10T22:13:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f9ba00640 1 -- 172.21.15.66:0/1883521991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f940b3310 con 0x7f5f940aeec0 2024-09-10T22:13:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f9ba00640 1 -- 172.21.15.66:0/1883521991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f94070420 con 0x7f5f940aeec0 2024-09-10T22:13:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.235+0000 7f5f90c00640 1 -- 172.21.15.66:0/1883521991 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f5f88004280 con 0x7f5f940aeec0 2024-09-10T22:13:59.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.239+0000 7f5f90c00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f74074710 0x7f5f74076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:59.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.239+0000 7f5f90c00640 1 -- 172.21.15.66:0/1883521991 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5083+0+0 (secure 0 0 0) 0x7f5f88092790 con 0x7f5f940aeec0 2024-09-10T22:13:59.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.239+0000 7f5f93e00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f74074710 0x7f5f74076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:59.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.239+0000 7f5f93e00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f74074710 0x7f5f74076bd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005e10 tx=0x7f5f80005d60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:59.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.243+0000 7f5f90c00640 1 -- 172.21.15.66:0/1883521991 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f88020020 con 0x7f5f940aeec0 2024-09-10T22:13:59.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.447+0000 7f5f9ba00640 1 -- 172.21.15.66:0/1883521991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f940b08d0 con 0x7f5f940aeec0 2024-09-10T22:13:59.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f90c00640 1 -- 172.21.15.66:0/1883521991 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+3908 (secure 0 0 0) 0x7f5f8805f8d0 con 0x7f5f940aeec0 2024-09-10T22:13:59.454 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:13:59.455 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":89,"btime":"2024-09-10T22:13:53:020876+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1352582369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":1352582369},{"type":"v1","addr":"172.21.15.94:6837","nonce":1352582369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/2682475345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2682475345},{"type":"v1","addr":"172.21.15.94:6836","nonce":2682475345}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8805,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/441263936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":441263936},{"type":"v1","addr":"172.21.15.66:6834","nonce":441263936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2554522797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2554522797},{"type":"v1","addr":"172.21.15.66:6835","nonce":2554522797}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[]} 2024-09-10T22:13:59.455 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 89 2024-09-10T22:13:59.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f9ba00640 1 -- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f74074710 msgr2=0x7f5f74076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:59.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f9ba00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f74074710 0x7f5f74076bd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005e10 tx=0x7f5f80005d60 comp rx=0 tx=0).stop 2024-09-10T22:13:59.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f9ba00640 1 -- 172.21.15.66:0/1883521991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f940aeec0 msgr2=0x7f5f940744a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:59.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f9ba00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f940aeec0 0x7f5f940744a0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800d750 tx=0x7f5f8800dc10 comp rx=0 tx=0).stop 2024-09-10T22:13:59.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f9ba00640 1 -- 172.21.15.66:0/1883521991 shutdown_connections 2024-09-10T22:13:59.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f9ba00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f74074710 0x7f5f74076bd0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:59.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f9ba00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f94147dc0 0x7f5f9414ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:59.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f9ba00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f9415a310 0x7f5f94072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:59.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f9ba00640 1 --2- 172.21.15.66:0/1883521991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f940aeec0 0x7f5f940744a0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:59.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f9ba00640 1 -- 172.21.15.66:0/1883521991 >> 172.21.15.66:0/1883521991 conn(0x7f5f940cfa30 msgr2=0x7f5f940af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:59.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f9ba00640 1 -- 172.21.15.66:0/1883521991 shutdown_connections 2024-09-10T22:13:59.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:13:59.451+0000 7f5f9ba00640 1 -- 172.21.15.66:0/1883521991 wait complete. 2024-09-10T22:13:59.478 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:13:59.478 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:13:59.478 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:13:59.478 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:13:59.478 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:13:59.478 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:13:59.479 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:13:59.479 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:13:59.479 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:13:59.479 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:13:59.479 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:13:59.479 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:13:59.479 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:13:59.479 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:13:59.480 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:13:59.480 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:13:59.491 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:13:59.492 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:13:59.500 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:13:59.500 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:13:59.501 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:13:59.501 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:13:59.540 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:13:59.541 DEBUG:teuthology.parallel:result is None 2024-09-10T22:13:59.544 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:13:59.544 DEBUG:teuthology.parallel:result is None 2024-09-10T22:13:59.546 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:13:59.546 DEBUG:teuthology.parallel:result is None 2024-09-10T22:13:59.625 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:13:59.636 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:13:59.637 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:14:02.853 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-10T22:14:05.491 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:14:05.492 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:14:05.495 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:14:05.496 DEBUG:teuthology.parallel:result is None 2024-09-10T22:14:05.496 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T22:14:05.591 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:14:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 -- 172.21.15.66:0/3594917003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb34815f450 msgr2=0x7fb3480b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 --2- 172.21.15.66:0/3594917003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb34815f450 0x7fb3480b3170 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009900 tx=0x7fb33c01d2b0 comp rx=0 tx=0).stop 2024-09-10T22:14:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 -- 172.21.15.66:0/3594917003 shutdown_connections 2024-09-10T22:14:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 --2- 172.21.15.66:0/3594917003 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3480b1e10 0x7fb34815fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 --2- 172.21.15.66:0/3594917003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb34815f450 0x7fb3480b3170 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 -- 172.21.15.66:0/3594917003 >> 172.21.15.66:0/3594917003 conn(0x7fb3480d1c60 msgr2=0x7fb3480d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:05.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 -- 172.21.15.66:0/3594917003 shutdown_connections 2024-09-10T22:14:05.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 -- 172.21.15.66:0/3594917003 wait complete. 2024-09-10T22:14:05.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 Processor -- start 2024-09-10T22:14:05.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 -- start start 2024-09-10T22:14:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3480b1e10 0x7fb3480cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb34815f450 0x7fb3481fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3481600b0 0x7fb348154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb348169380 con 0x7fb34815f450 2024-09-10T22:14:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb348169200 con 0x7fb3480b1e10 2024-09-10T22:14:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34fe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb348169500 con 0x7fb3481600b0 2024-09-10T22:14:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb34815f450 0x7fb3481fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb34815f450 0x7fb3481fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60876/0 (socket says 172.21.15.66:60876) 2024-09-10T22:14:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34ca00640 1 -- 172.21.15.66:0/2157275505 learned_addr learned my addr 172.21.15.66:0/2157275505 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34d400640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3480b1e10 0x7fb3480cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.651+0000 7fb34de00640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3481600b0 0x7fb348154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.655+0000 7fb34ca00640 1 -- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3481600b0 msgr2=0x7fb348154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.655+0000 7fb34ca00640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3481600b0 0x7fb348154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.655+0000 7fb34ca00640 1 -- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3480b1e10 msgr2=0x7fb3480cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.655+0000 7fb34ca00640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3480b1e10 0x7fb3480cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.655+0000 7fb34ca00640 1 -- 172.21.15.66:0/2157275505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3481fb400 con 0x7fb34815f450 2024-09-10T22:14:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.655+0000 7fb34ca00640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb34815f450 0x7fb3481fbea0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fb33400e9f0 tx=0x7fb33400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.655+0000 7fb345600640 1 -- 172.21.15.66:0/2157275505 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33400cd80 con 0x7fb34815f450 2024-09-10T22:14:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.655+0000 7fb345600640 1 -- 172.21.15.66:0/2157275505 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb334005020 con 0x7fb34815f450 2024-09-10T22:14:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.655+0000 7fb345600640 1 -- 172.21.15.66:0/2157275505 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334005450 con 0x7fb34815f450 2024-09-10T22:14:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.655+0000 7fb34fe00640 1 -- 172.21.15.66:0/2157275505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3481549c0 con 0x7fb34815f450 2024-09-10T22:14:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.655+0000 7fb34fe00640 1 -- 172.21.15.66:0/2157275505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3481fb6c0 con 0x7fb34815f450 2024-09-10T22:14:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.655+0000 7fb34fe00640 1 -- 172.21.15.66:0/2157275505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb34815fb70 con 0x7fb34815f450 2024-09-10T22:14:05.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.659+0000 7fb345600640 1 -- 172.21.15.66:0/2157275505 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fb3340047b0 con 0x7fb34815f450 2024-09-10T22:14:05.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.659+0000 7fb345600640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb328074710 0x7fb328076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:05.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.659+0000 7fb345600640 1 -- 172.21.15.66:0/2157275505 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5083+0+0 (secure 0 0 0) 0x7fb334092db0 con 0x7fb34815f450 2024-09-10T22:14:05.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.659+0000 7fb34d400640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb328074710 0x7fb328076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:05.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.659+0000 7fb34d400640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb328074710 0x7fb328076bd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0af000 tx=0x7fb33c0a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.663+0000 7fb345600640 1 -- 172.21.15.66:0/2157275505 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb33405fa20 con 0x7fb34815f450 2024-09-10T22:14:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:05.847+0000 7fb34fe00640 1 -- 172.21.15.66:0/2157275505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb34815ea70 con 0x7fb34815f450 2024-09-10T22:14:07.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.087+0000 7fb345600640 1 -- 172.21.15.66:0/2157275505 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v137) ==== 97+0+0 (secure 0 0 0) 0x7fb334064450 con 0x7fb34815f450 2024-09-10T22:14:07.094 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-09-10T22:14:07.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.091+0000 7fb34fe00640 1 -- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb328074710 msgr2=0x7fb328076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:07.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.091+0000 7fb34fe00640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb328074710 0x7fb328076bd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0af000 tx=0x7fb33c0a7040 comp rx=0 tx=0).stop 2024-09-10T22:14:07.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.091+0000 7fb34fe00640 1 -- 172.21.15.66:0/2157275505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb34815f450 msgr2=0x7fb3481fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:07.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.091+0000 7fb34fe00640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb34815f450 0x7fb3481fbea0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fb33400e9f0 tx=0x7fb33400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.091+0000 7fb34fe00640 1 -- 172.21.15.66:0/2157275505 shutdown_connections 2024-09-10T22:14:07.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.091+0000 7fb34fe00640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb328074710 0x7fb328076bd0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.091+0000 7fb34fe00640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3481600b0 0x7fb348154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.091+0000 7fb34fe00640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb34815f450 0x7fb3481fbea0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.091+0000 7fb34fe00640 1 --2- 172.21.15.66:0/2157275505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3480b1e10 0x7fb3480cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.091+0000 7fb34fe00640 1 -- 172.21.15.66:0/2157275505 >> 172.21.15.66:0/2157275505 conn(0x7fb3480d1c60 msgr2=0x7fb3480cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:07.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.091+0000 7fb34fe00640 1 -- 172.21.15.66:0/2157275505 shutdown_connections 2024-09-10T22:14:07.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.091+0000 7fb34fe00640 1 -- 172.21.15.66:0/2157275505 wait complete. 2024-09-10T22:14:07.123 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T22:14:07.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 -- 172.21.15.66:0/4249622993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47ac003c20 msgr2=0x7f47ac004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:07.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 --2- 172.21.15.66:0/4249622993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47ac003c20 0x7f47ac004020 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f47b8006fb0 tx=0x7f47b801cf20 comp rx=0 tx=0).stop 2024-09-10T22:14:07.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cd200640 1 -- 172.21.15.66:0/4249622993 reap_dead start 2024-09-10T22:14:07.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 -- 172.21.15.66:0/4249622993 shutdown_connections 2024-09-10T22:14:07.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 -- 172.21.15.66:0/4249622993 >> 172.21.15.66:0/4249622993 conn(0x7f47c80d1c50 msgr2=0x7f47c80d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:07.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 -- 172.21.15.66:0/4249622993 shutdown_connections 2024-09-10T22:14:07.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 -- 172.21.15.66:0/4249622993 wait complete. 2024-09-10T22:14:07.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 Processor -- start 2024-09-10T22:14:07.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 -- start start 2024-09-10T22:14:07.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47ac003c20 0x7f47c8159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:07.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f47c80b67d0 0x7f47c815dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:07.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c80b6c50 0x7f47c815e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:07.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8167e00 con 0x7f47c80b6c50 2024-09-10T22:14:07.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47c80afba0 con 0x7f47ac003c20 2024-09-10T22:14:07.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cf200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f47c8167f80 con 0x7f47c80b67d0 2024-09-10T22:14:07.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cd200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c80b6c50 0x7f47c815e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cd200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c80b6c50 0x7f47c815e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60898/0 (socket says 172.21.15.66:60898) 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.271+0000 7f47cd200640 1 -- 172.21.15.66:0/3267456275 learned_addr learned my addr 172.21.15.66:0/3267456275 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47c7e00640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47ac003c20 0x7f47c8159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47bfe00640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f47c80b67d0 0x7f47c815dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47cd200640 1 -- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f47c80b67d0 msgr2=0x7f47c815dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47cd200640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f47c80b67d0 0x7f47c815dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47cd200640 1 -- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47ac003c20 msgr2=0x7f47c8159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47cd200640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47ac003c20 0x7f47c8159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47cd200640 1 -- 172.21.15.66:0/3267456275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c8159790 con 0x7f47c80b6c50 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47cd200640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c80b6c50 0x7f47c815e0f0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f47c80b2fd0 tx=0x7f47c000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47c4c00640 1 -- 172.21.15.66:0/3267456275 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0017070 con 0x7f47c80b6c50 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47c4c00640 1 -- 172.21.15.66:0/3267456275 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47c0015070 con 0x7f47c80b6c50 2024-09-10T22:14:07.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47c4c00640 1 -- 172.21.15.66:0/3267456275 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0003be0 con 0x7f47c80b6c50 2024-09-10T22:14:07.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47cf200640 1 -- 172.21.15.66:0/3267456275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c80b4bb0 con 0x7f47c80b6c50 2024-09-10T22:14:07.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47cf200640 1 -- 172.21.15.66:0/3267456275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c80b5170 con 0x7f47c80b6c50 2024-09-10T22:14:07.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47cf200640 1 -- 172.21.15.66:0/3267456275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c81657e0 con 0x7f47c80b6c50 2024-09-10T22:14:07.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.275+0000 7f47c4c00640 1 -- 172.21.15.66:0/3267456275 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f47c0011440 con 0x7f47c80b6c50 2024-09-10T22:14:07.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.279+0000 7f47c4c00640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f47a4074740 0x7f47a4076c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:07.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.279+0000 7f47c4c00640 1 -- 172.21.15.66:0/3267456275 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4997+0+0 (secure 0 0 0) 0x7f47c0092980 con 0x7f47c80b6c50 2024-09-10T22:14:07.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.279+0000 7f47c7e00640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f47a4074740 0x7f47a4076c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:07.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.279+0000 7f47c7e00640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f47a4074740 0x7f47a4076c00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f47b8006fb0 tx=0x7f47ac005840 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.283+0000 7f47c4c00640 1 -- 172.21.15.66:0/3267456275 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f47c005f470 con 0x7f47c80b6c50 2024-09-10T22:14:07.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.515+0000 7f47cf200640 1 -- 172.21.15.66:0/3267456275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f47c815c300 con 0x7f47c80b6c50 2024-09-10T22:14:07.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.515+0000 7f47c4c00640 1 -- 172.21.15.66:0/3267456275 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) ==== 80+0+5304 (secure 0 0 0) 0x7f47c0063ea0 con 0x7f47c80b6c50 2024-09-10T22:14:07.521 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:14:07.521 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:14:07.521 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-09-10T22:14:07.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.521 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-09-10T22:14:07.521 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmxQD7CBAAttZP0KtdjTI7QWbD7CVinQ==", 2024-09-10T22:14:07.521 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmI7ufFRAANelhvqylKyfGPVfBuY40kg==", 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:14:07.522 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQnVoEBAAvsZ/TSOOP/JgJBr/aHwI/g==", 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.524 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-09-10T22:14:07.524 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmRQkTHRAAHPIdwqtkkyh31D7IO2vxPA==", 2024-09-10T22:14:07.524 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.524 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:14:07.524 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:14:07.524 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:14:07.524 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.524 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.524 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-09-10T22:14:07.524 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBCweBmDJkxJBAAVSQNiMAQl5bR3RUwVOuGYw==", 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBFweBmwcdsOxAAPPQyzktcfiF8pJNP8uAjwg==", 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:14:07.525 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBJweBmXQhZExAAnEf8mjGX6nnYwWygKjo/2A==", 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.526 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBNweBmlauHAxAA/k9xRqbHZsqJq9OYbJKfNQ==", 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBRweBmHnFDJRAAgF1ZAPvmoNzeUftZQyL0eg==", 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:14:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBUweBm/hXpORAA6jpOI/I+QyB2mZsEoAWF0w==", 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.528 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBYweBmYu2MExAAPD9WcxrBLzwwJQm19z7YcQ==", 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbweBmXfJ9KRAA/AbWlli+Qfg7gy0SblJ7fg==", 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:14:07.529 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw==", 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.530 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmbS6IKhAAaEmK9irxMipALgkAct/+OQ==", 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmQ47KJhAAI50ZaSQIhH6FxRM48JZICA==", 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.531 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmUcLCMBAAdf4tfcpVaZoaKjS1UzrxhA==", 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:14:07.532 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA+weBmPIAzKBAAkyVV875PNhBmEoKsrAkg8w==", 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmLiy1BBAADnraML8IUet7Sw0j83TUVQ==", 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA/weBmItWpABAAlJ+6dFjvtBV3g+jYmUh0tQ==", 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:14:07.534 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:14:07.535 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:14:07.535 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.535 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.535 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T22:14:07.535 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:14:07.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.519+0000 7f47cf200640 1 -- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f47a4074740 msgr2=0x7f47a4076c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:07.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.519+0000 7f47cf200640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f47a4074740 0x7f47a4076c00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f47b8006fb0 tx=0x7f47ac005840 comp rx=0 tx=0).stop 2024-09-10T22:14:07.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.519+0000 7f47cf200640 1 -- 172.21.15.66:0/3267456275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c80b6c50 msgr2=0x7f47c815e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:07.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.519+0000 7f47cf200640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c80b6c50 0x7f47c815e0f0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f47c80b2fd0 tx=0x7f47c000e720 comp rx=0 tx=0).stop 2024-09-10T22:14:07.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.519+0000 7f47cf200640 1 -- 172.21.15.66:0/3267456275 shutdown_connections 2024-09-10T22:14:07.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.519+0000 7f47cf200640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f47a4074740 0x7f47a4076c00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.519+0000 7f47cf200640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c80b6c50 0x7f47c815e0f0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.519+0000 7f47cf200640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f47c80b67d0 0x7f47c815dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.519+0000 7f47cf200640 1 --2- 172.21.15.66:0/3267456275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47ac003c20 0x7f47c8159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.519+0000 7f47cf200640 1 -- 172.21.15.66:0/3267456275 >> 172.21.15.66:0/3267456275 conn(0x7f47c80d1c50 msgr2=0x7f47c80afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:07.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.519+0000 7f47cf200640 1 -- 172.21.15.66:0/3267456275 shutdown_connections 2024-09-10T22:14:07.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.519+0000 7f47cf200640 1 -- 172.21.15.66:0/3267456275 wait complete. 2024-09-10T22:14:07.543 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:14:07.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.683+0000 7fca37e00640 1 -- 172.21.15.66:0/137922772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca300cff10 msgr2=0x7fca300d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:07.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.683+0000 7fca37e00640 1 --2- 172.21.15.66:0/137922772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca300cff10 0x7fca300d0340 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fca30072de0 tx=0x7fca2801cac0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.683+0000 7fca37e00640 1 -- 172.21.15.66:0/137922772 shutdown_connections 2024-09-10T22:14:07.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.683+0000 7fca37e00640 1 --2- 172.21.15.66:0/137922772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca300cff10 0x7fca300d0340 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.683+0000 7fca37e00640 1 --2- 172.21.15.66:0/137922772 >> v2:172.21.15.94:3300/0 conn(0x7fca300b0590 0x7fca300cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.683+0000 7fca37e00640 1 -- 172.21.15.66:0/137922772 >> 172.21.15.66:0/137922772 conn(0x7fca30074240 msgr2=0x7fca30074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:07.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.683+0000 7fca37e00640 1 -- 172.21.15.66:0/137922772 shutdown_connections 2024-09-10T22:14:07.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.683+0000 7fca37e00640 1 -- 172.21.15.66:0/137922772 wait complete. 2024-09-10T22:14:07.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca37e00640 1 Processor -- start 2024-09-10T22:14:07.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca37e00640 1 -- start start 2024-09-10T22:14:07.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca37e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca300b0590 0x7fca300c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:07.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca37e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca300cff10 0x7fca30153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:07.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca37e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca3015bfb0 0x7fca300ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:07.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca37e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fca300cd990 con 0x7fca3015bfb0 2024-09-10T22:14:07.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca37e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca300d0ea0 con 0x7fca300b0590 2024-09-10T22:14:07.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca37e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fca300cdb10 con 0x7fca300cff10 2024-09-10T22:14:07.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca35e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca3015bfb0 0x7fca300ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:07.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca35e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca3015bfb0 0x7fca300ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60906/0 (socket says 172.21.15.66:60906) 2024-09-10T22:14:07.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca35e00640 1 -- 172.21.15.66:0/3907318120 learned_addr learned my addr 172.21.15.66:0/3907318120 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:07.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca35400640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca300b0590 0x7fca300c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:07.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca34a00640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca300cff10 0x7fca30153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:07.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca35e00640 1 -- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca300cff10 msgr2=0x7fca30153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:07.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca35e00640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca300cff10 0x7fca30153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca35e00640 1 -- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca300b0590 msgr2=0x7fca300c1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:07.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca35e00640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca300b0590 0x7fca300c1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca35e00640 1 -- 172.21.15.66:0/3907318120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca301f6bc0 con 0x7fca3015bfb0 2024-09-10T22:14:07.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca35e00640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca3015bfb0 0x7fca300ae430 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fca2801a6f0 tx=0x7fca280a5a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:07.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca2d600640 1 -- 172.21.15.66:0/3907318120 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca280bb070 con 0x7fca3015bfb0 2024-09-10T22:14:07.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca2d600640 1 -- 172.21.15.66:0/3907318120 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca28004e00 con 0x7fca3015bfb0 2024-09-10T22:14:07.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca2d600640 1 -- 172.21.15.66:0/3907318120 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca280b2040 con 0x7fca3015bfb0 2024-09-10T22:14:07.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca37e00640 1 -- 172.21.15.66:0/3907318120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca301f5e00 con 0x7fca3015bfb0 2024-09-10T22:14:07.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.687+0000 7fca37e00640 1 -- 172.21.15.66:0/3907318120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca301f63c0 con 0x7fca3015bfb0 2024-09-10T22:14:07.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.691+0000 7fca37e00640 1 -- 172.21.15.66:0/3907318120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca300d0340 con 0x7fca3015bfb0 2024-09-10T22:14:07.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.691+0000 7fca2d600640 1 -- 172.21.15.66:0/3907318120 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fca280b5030 con 0x7fca3015bfb0 2024-09-10T22:14:07.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.691+0000 7fca2d600640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fca10074710 0x7fca10076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:07.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.691+0000 7fca2d600640 1 -- 172.21.15.66:0/3907318120 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4997+0+0 (secure 0 0 0) 0x7fca2812c370 con 0x7fca3015bfb0 2024-09-10T22:14:07.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.691+0000 7fca35400640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fca10074710 0x7fca10076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:07.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.691+0000 7fca35400640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fca10074710 0x7fca10076bd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fca240015f0 tx=0x7fca24009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:07.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.695+0000 7fca2d600640 1 -- 172.21.15.66:0/3907318120 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fca28016770 con 0x7fca3015bfb0 2024-09-10T22:14:07.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 -- 172.21.15.66:0/3907318120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fca300c38b0 con 0x7fca3015bfb0 2024-09-10T22:14:07.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca2d600640 1 -- 172.21.15.66:0/3907318120 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v137) ==== 84+0+65 (secure 0 0 0) 0x7fca280f94b0 con 0x7fca3015bfb0 2024-09-10T22:14:07.875 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:14:07.876 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:14:07.876 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:07.876 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:14:07.876 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:14:07.876 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:07.876 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:14:07.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 -- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fca10074710 msgr2=0x7fca10076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fca10074710 0x7fca10076bd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fca240015f0 tx=0x7fca24009040 comp rx=0 tx=0).stop 2024-09-10T22:14:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 -- 172.21.15.66:0/3907318120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca3015bfb0 msgr2=0x7fca300ae430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca3015bfb0 0x7fca300ae430 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fca2801a6f0 tx=0x7fca280a5a80 comp rx=0 tx=0).stop 2024-09-10T22:14:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 -- 172.21.15.66:0/3907318120 shutdown_connections 2024-09-10T22:14:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fca10074710 0x7fca10076bd0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca3015bfb0 0x7fca300ae430 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca300cff10 0x7fca30153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 --2- 172.21.15.66:0/3907318120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca300b0590 0x7fca300c1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 -- 172.21.15.66:0/3907318120 >> 172.21.15.66:0/3907318120 conn(0x7fca30074240 msgr2=0x7fca3006f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 -- 172.21.15.66:0/3907318120 shutdown_connections 2024-09-10T22:14:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:07.871+0000 7fca37e00640 1 -- 172.21.15.66:0/3907318120 wait complete. 2024-09-10T22:14:07.898 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:14:07.898 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:14:08.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.043+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2188384425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbe40cdf90 msgr2=0x7ffbe415fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:08.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.043+0000 7ffbe8e00640 1 --2- 172.21.15.66:0/2188384425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbe40cdf90 0x7ffbe415fd50 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7ffbe40d0b10 tx=0x7ffbcc01c8c0 comp rx=0 tx=0).stop 2024-09-10T22:14:08.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.043+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2188384425 shutdown_connections 2024-09-10T22:14:08.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.043+0000 7ffbe8e00640 1 --2- 172.21.15.66:0/2188384425 >> v2:172.21.15.94:3300/0 conn(0x7ffbe40b0e20 0x7ffbe4160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:08.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.043+0000 7ffbe8e00640 1 --2- 172.21.15.66:0/2188384425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbe40cdf90 0x7ffbe415fd50 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:08.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.043+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2188384425 >> 172.21.15.66:0/2188384425 conn(0x7ffbe40d1c60 msgr2=0x7ffbe40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:08.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.043+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2188384425 shutdown_connections 2024-09-10T22:14:08.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.043+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2188384425 wait complete. 2024-09-10T22:14:08.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe8e00640 1 Processor -- start 2024-09-10T22:14:08.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe8e00640 1 -- start start 2024-09-10T22:14:08.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe8e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffbe40b0e20 0x7ffbe4151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:08.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe8e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbe40cdf90 0x7ffbe40c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:08.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe40ce8d0 0x7ffbe40c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:08.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe8e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41691a0 con 0x7ffbe40cdf90 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe8e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe4169020 con 0x7ffbe40ce8d0 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe8e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe4169320 con 0x7ffbe40b0e20 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbdbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbe40cdf90 0x7ffbe40c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbdbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbe40cdf90 0x7ffbe40c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60934/0 (socket says 172.21.15.66:60934) 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbdbe00640 1 -- 172.21.15.66:0/2635089105 learned_addr learned my addr 172.21.15.66:0/2635089105 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe1e00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffbe40b0e20 0x7ffbe4151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe2800640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe40ce8d0 0x7ffbe40c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbdbe00640 1 -- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffbe40b0e20 msgr2=0x7ffbe4151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbdbe00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffbe40b0e20 0x7ffbe4151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbdbe00640 1 -- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe40ce8d0 msgr2=0x7ffbe40c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbdbe00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe40ce8d0 0x7ffbe40c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbdbe00640 1 -- 172.21.15.66:0/2635089105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe4071eb0 con 0x7ffbe40cdf90 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe1e00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffbe40b0e20 0x7ffbe4151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:14:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbdbe00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbe40cdf90 0x7ffbe40c7ed0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0194d0 tx=0x7ffbcc0a6bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbda000640 1 -- 172.21.15.66:0/2635089105 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b2040 con 0x7ffbe40cdf90 2024-09-10T22:14:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbda000640 1 -- 172.21.15.66:0/2635089105 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbcc004310 con 0x7ffbe40cdf90 2024-09-10T22:14:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbda000640 1 -- 172.21.15.66:0/2635089105 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0052d0 con 0x7ffbe40cdf90 2024-09-10T22:14:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2635089105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe40b5de0 con 0x7ffbe40cdf90 2024-09-10T22:14:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2635089105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe4152df0 con 0x7ffbe40cdf90 2024-09-10T22:14:08.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.047+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2635089105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe40cecb0 con 0x7ffbe40cdf90 2024-09-10T22:14:08.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.051+0000 7ffbda000640 1 -- 172.21.15.66:0/2635089105 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7ffbcc005470 con 0x7ffbe40cdf90 2024-09-10T22:14:08.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.051+0000 7ffbda000640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffbc0074710 0x7ffbc0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:08.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.051+0000 7ffbda000640 1 -- 172.21.15.66:0/2635089105 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4997+0+0 (secure 0 0 0) 0x7ffbcc12bfc0 con 0x7ffbe40cdf90 2024-09-10T22:14:08.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.051+0000 7ffbe1e00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffbc0074710 0x7ffbc0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:08.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.051+0000 7ffbe1e00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffbc0074710 0x7ffbc0076bd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4166fb0 tx=0x7ffbd4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:08.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.055+0000 7ffbda000640 1 -- 172.21.15.66:0/2635089105 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffbcc135070 con 0x7ffbe40cdf90 2024-09-10T22:14:08.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:08.239+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2635089105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ffbe4070550 con 0x7ffbe40cdf90 2024-09-10T22:14:09.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbda000640 1 -- 172.21.15.66:0/2635089105 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v138) ==== 138+0+0 (secure 0 0 0) 0x7ffbcc0164b0 con 0x7ffbe40cdf90 2024-09-10T22:14:09.110 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:14:09.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffbc0074710 msgr2=0x7ffbc0076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:09.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbe8e00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffbc0074710 0x7ffbc0076bd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4166fb0 tx=0x7ffbd4009040 comp rx=0 tx=0).stop 2024-09-10T22:14:09.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2635089105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbe40cdf90 msgr2=0x7ffbe40c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:09.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbe8e00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbe40cdf90 0x7ffbe40c7ed0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0194d0 tx=0x7ffbcc0a6bc0 comp rx=0 tx=0).stop 2024-09-10T22:14:09.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2635089105 shutdown_connections 2024-09-10T22:14:09.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbe8e00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ffbc0074710 0x7ffbc0076bd0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:09.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbe8e00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbe40ce8d0 0x7ffbe40c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:09.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbe8e00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbe40cdf90 0x7ffbe40c7ed0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:09.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbe8e00640 1 --2- 172.21.15.66:0/2635089105 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffbe40b0e20 0x7ffbe4151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:09.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2635089105 >> 172.21.15.66:0/2635089105 conn(0x7ffbe40d1c60 msgr2=0x7ffbe40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:09.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2635089105 shutdown_connections 2024-09-10T22:14:09.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.107+0000 7ffbe8e00640 1 -- 172.21.15.66:0/2635089105 wait complete. 2024-09-10T22:14:09.141 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:14:09.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.295+0000 7fba4a800640 1 -- 172.21.15.66:0/4007666910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4415a310 msgr2=0x7fba440c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:09.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.295+0000 7fba4a800640 1 --2- 172.21.15.66:0/4007666910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4415a310 0x7fba440c48b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fba3400a950 tx=0x7fba3401dae0 comp rx=0 tx=0).stop 2024-09-10T22:14:09.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.295+0000 7fba4a800640 1 -- 172.21.15.66:0/4007666910 shutdown_connections 2024-09-10T22:14:09.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.295+0000 7fba4a800640 1 --2- 172.21.15.66:0/4007666910 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba440d0550 0x7fba440c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:09.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.295+0000 7fba4a800640 1 --2- 172.21.15.66:0/4007666910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4415a310 0x7fba440c48b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:09.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.295+0000 7fba4a800640 1 -- 172.21.15.66:0/4007666910 >> 172.21.15.66:0/4007666910 conn(0x7fba440cfa30 msgr2=0x7fba440cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:09.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.295+0000 7fba4a800640 1 -- 172.21.15.66:0/4007666910 shutdown_connections 2024-09-10T22:14:09.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.295+0000 7fba4a800640 1 -- 172.21.15.66:0/4007666910 wait complete. 2024-09-10T22:14:09.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.295+0000 7fba4a800640 1 Processor -- start 2024-09-10T22:14:09.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.295+0000 7fba4a800640 1 -- start start 2024-09-10T22:14:09.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba4a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba440aeec0 0x7fba440c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:09.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba4a800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba4415a310 0x7fba4414b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:09.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba4a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba441fc150 0x7fba440ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:09.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba4a800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fba44167300 con 0x7fba441fc150 2024-09-10T22:14:09.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba4a800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba4415ec90 con 0x7fba440aeec0 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba4a800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fba44167480 con 0x7fba4415a310 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba43400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba440aeec0 0x7fba440c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba43e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba441fc150 0x7fba440ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba3be00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba4415a310 0x7fba4414b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba43400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba440aeec0 0x7fba440c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:52160/0 (socket says 172.21.15.66:52160) 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba43400640 1 -- 172.21.15.66:0/3155221899 learned_addr learned my addr 172.21.15.66:0/3155221899 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba3be00640 1 -- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba440aeec0 msgr2=0x7fba440c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba3be00640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba440aeec0 0x7fba440c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba3be00640 1 -- 172.21.15.66:0/3155221899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba441fc150 msgr2=0x7fba440ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba3be00640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba441fc150 0x7fba440ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba3be00640 1 -- 172.21.15.66:0/3155221899 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba44074650 con 0x7fba4415a310 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba43400640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba440aeec0 0x7fba440c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:14:09.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba3be00640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba4415a310 0x7fba4414b1d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00e9f0 tx=0x7fba2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:09.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba3b400640 1 -- 172.21.15.66:0/3155221899 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c00cd80 con 0x7fba4415a310 2024-09-10T22:14:09.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba3b400640 1 -- 172.21.15.66:0/3155221899 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba2c004020 con 0x7fba4415a310 2024-09-10T22:14:09.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba4a800640 1 -- 172.21.15.66:0/3155221899 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba441fb670 con 0x7fba4415a310 2024-09-10T22:14:09.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba3b400640 1 -- 172.21.15.66:0/3155221899 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c004450 con 0x7fba4415a310 2024-09-10T22:14:09.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba4a800640 1 -- 172.21.15.66:0/3155221899 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fba441fbc30 con 0x7fba4415a310 2024-09-10T22:14:09.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.299+0000 7fba4a800640 1 -- 172.21.15.66:0/3155221899 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba4415df60 con 0x7fba4415a310 2024-09-10T22:14:09.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.303+0000 7fba3b400640 1 -- 172.21.15.66:0/3155221899 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fba2c003600 con 0x7fba4415a310 2024-09-10T22:14:09.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.303+0000 7fba3b400640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fba20074710 0x7fba20076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:09.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.303+0000 7fba3b400640 1 -- 172.21.15.66:0/3155221899 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(138..138 src has 1..138) ==== 5367+0+0 (secure 0 0 0) 0x7fba2c092bd0 con 0x7fba4415a310 2024-09-10T22:14:09.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.303+0000 7fba43400640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fba20074710 0x7fba20076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:09.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.303+0000 7fba43400640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fba20074710 0x7fba20076bd0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fba3401b710 tx=0x7fba3401c320 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:09.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.307+0000 7fba3b400640 1 -- 172.21.15.66:0/3155221899 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fba2c05f6c0 con 0x7fba4415a310 2024-09-10T22:14:09.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:09.483+0000 7fba4a800640 1 -- 172.21.15.66:0/3155221899 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fba440b1210 con 0x7fba4415a310 2024-09-10T22:14:09.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-10T22:14:10.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.115+0000 7fba3b400640 1 -- 172.21.15.66:0/3155221899 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v139) ==== 170+0+0 (secure 0 0 0) 0x7fba2c0640f0 con 0x7fba4415a310 2024-09-10T22:14:10.119 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:14:10.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.119+0000 7fba4a800640 1 -- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fba20074710 msgr2=0x7fba20076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:10.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.119+0000 7fba4a800640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fba20074710 0x7fba20076bd0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fba3401b710 tx=0x7fba3401c320 comp rx=0 tx=0).stop 2024-09-10T22:14:10.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.119+0000 7fba4a800640 1 -- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba4415a310 msgr2=0x7fba4414b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:10.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.119+0000 7fba4a800640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba4415a310 0x7fba4414b1d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00e9f0 tx=0x7fba2c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:14:10.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.123+0000 7fba4a800640 1 -- 172.21.15.66:0/3155221899 shutdown_connections 2024-09-10T22:14:10.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.123+0000 7fba4a800640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fba20074710 0x7fba20076bd0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:10.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.123+0000 7fba4a800640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba441fc150 0x7fba440ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:10.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.123+0000 7fba4a800640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba4415a310 0x7fba4414b1d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:10.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.123+0000 7fba4a800640 1 --2- 172.21.15.66:0/3155221899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba440aeec0 0x7fba440c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:10.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.123+0000 7fba4a800640 1 -- 172.21.15.66:0/3155221899 >> 172.21.15.66:0/3155221899 conn(0x7fba440cfa30 msgr2=0x7fba4414e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:10.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.123+0000 7fba4a800640 1 -- 172.21.15.66:0/3155221899 shutdown_connections 2024-09-10T22:14:10.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.123+0000 7fba4a800640 1 -- 172.21.15.66:0/3155221899 wait complete. 2024-09-10T22:14:10.162 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:14:10.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.307+0000 7f7c6f600640 1 -- 172.21.15.66:0/1457889426 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c680d0550 msgr2=0x7f7c680c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:10.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.307+0000 7f7c6f600640 1 --2- 172.21.15.66:0/1457889426 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c680d0550 0x7f7c680c4df0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009840 tx=0x7f7c5401c790 comp rx=0 tx=0).stop 2024-09-10T22:14:10.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.307+0000 7f7c6f600640 1 -- 172.21.15.66:0/1457889426 shutdown_connections 2024-09-10T22:14:10.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.307+0000 7f7c6f600640 1 --2- 172.21.15.66:0/1457889426 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c680d0550 0x7f7c680c4df0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:10.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.307+0000 7f7c6f600640 1 --2- 172.21.15.66:0/1457889426 >> v2:172.21.15.94:3300/0 conn(0x7f7c6815a310 0x7f7c680c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:10.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.307+0000 7f7c6f600640 1 -- 172.21.15.66:0/1457889426 >> 172.21.15.66:0/1457889426 conn(0x7f7c680cfa30 msgr2=0x7f7c680cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:10.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.307+0000 7f7c6f600640 1 -- 172.21.15.66:0/1457889426 shutdown_connections 2024-09-10T22:14:10.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.307+0000 7f7c6f600640 1 -- 172.21.15.66:0/1457889426 wait complete. 2024-09-10T22:14:10.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6f600640 1 Processor -- start 2024-09-10T22:14:10.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6f600640 1 -- start start 2024-09-10T22:14:10.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6f600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c680aeec0 0x7f7c680744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:10.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6f600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6815a310 0x7f7c6814b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:10.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6f600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c680b54e0 0x7f7c68072a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:10.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6f600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6815ed60 con 0x7f7c6815a310 2024-09-10T22:14:10.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6f600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6815ebe0 con 0x7f7c680b54e0 2024-09-10T22:14:10.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6f600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7c681652f0 con 0x7f7c680aeec0 2024-09-10T22:14:10.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6cc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c680aeec0 0x7f7c680744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:10.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c67e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6815a310 0x7f7c6814b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:10.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6cc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c680aeec0 0x7f7c680744a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:39538/0 (socket says 172.21.15.66:39538) 2024-09-10T22:14:10.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6cc00640 1 -- 172.21.15.66:0/3186203859 learned_addr learned my addr 172.21.15.66:0/3186203859 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:10.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6d600640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c680b54e0 0x7f7c68072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:10.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6cc00640 1 -- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c680b54e0 msgr2=0x7f7c68072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:10.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6cc00640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c680b54e0 0x7f7c68072a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:10.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6cc00640 1 -- 172.21.15.66:0/3186203859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6815a310 msgr2=0x7f7c6814b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:10.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6cc00640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6815a310 0x7f7c6814b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:10.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6cc00640 1 -- 172.21.15.66:0/3186203859 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c68073400 con 0x7f7c680aeec0 2024-09-10T22:14:10.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6cc00640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c680aeec0 0x7f7c680744a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00e9a0 tx=0x7f7c5c00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:10.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c64c00640 1 -- 172.21.15.66:0/3186203859 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c00cd80 con 0x7f7c680aeec0 2024-09-10T22:14:10.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c64c00640 1 -- 172.21.15.66:0/3186203859 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c5c004020 con 0x7f7c680aeec0 2024-09-10T22:14:10.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6f600640 1 -- 172.21.15.66:0/3186203859 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c680736e0 con 0x7f7c680aeec0 2024-09-10T22:14:10.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c64c00640 1 -- 172.21.15.66:0/3186203859 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c004450 con 0x7f7c680aeec0 2024-09-10T22:14:10.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.311+0000 7f7c6f600640 1 -- 172.21.15.66:0/3186203859 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c68073ca0 con 0x7f7c680aeec0 2024-09-10T22:14:10.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.315+0000 7f7c6f600640 1 -- 172.21.15.66:0/3186203859 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c68070420 con 0x7f7c680aeec0 2024-09-10T22:14:10.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.315+0000 7f7c64c00640 1 -- 172.21.15.66:0/3186203859 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f7c5c003600 con 0x7f7c680aeec0 2024-09-10T22:14:10.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.315+0000 7f7c64c00640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7c48074710 0x7f7c48076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:10.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.315+0000 7f7c64c00640 1 -- 172.21.15.66:0/3186203859 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(139..139 src has 1..139) ==== 5749+0+0 (secure 0 0 0) 0x7f7c5c092bd0 con 0x7f7c680aeec0 2024-09-10T22:14:10.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.315+0000 7f7c67e00640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7c48074710 0x7f7c48076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:10.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.319+0000 7f7c67e00640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7c48074710 0x7f7c48076bd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7c540af000 tx=0x7f7c5401afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:10.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.319+0000 7f7c64c00640 1 -- 172.21.15.66:0/3186203859 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7c5c05f6c0 con 0x7f7c680aeec0 2024-09-10T22:14:10.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:10.527+0000 7f7c6f600640 1 -- 172.21.15.66:0/3186203859 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7c6806f1a0 con 0x7f7c680aeec0 2024-09-10T22:14:11.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.143+0000 7f7c64c00640 1 -- 172.21.15.66:0/3186203859 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v93) ==== 174+0+0 (secure 0 0 0) 0x7f7c5c0640f0 con 0x7f7c680aeec0 2024-09-10T22:14:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 20 and data pool 21 2024-09-10T22:14:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.159+0000 7f7c6f600640 1 -- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7c48074710 msgr2=0x7f7c48076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.159+0000 7f7c6f600640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7c48074710 0x7f7c48076bd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7c540af000 tx=0x7f7c5401afd0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.159+0000 7f7c6f600640 1 -- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c680aeec0 msgr2=0x7f7c680744a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.159+0000 7f7c6f600640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c680aeec0 0x7f7c680744a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00e9a0 tx=0x7f7c5c00ee60 comp rx=0 tx=0).stop 2024-09-10T22:14:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.159+0000 7f7c6f600640 1 -- 172.21.15.66:0/3186203859 shutdown_connections 2024-09-10T22:14:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.159+0000 7f7c6f600640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7c48074710 0x7f7c48076bd0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.159+0000 7f7c6f600640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c680b54e0 0x7f7c68072a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.159+0000 7f7c6f600640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6815a310 0x7f7c6814b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.159+0000 7f7c6f600640 1 --2- 172.21.15.66:0/3186203859 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c680aeec0 0x7f7c680744a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.159+0000 7f7c6f600640 1 -- 172.21.15.66:0/3186203859 >> 172.21.15.66:0/3186203859 conn(0x7f7c680cfa30 msgr2=0x7f7c680adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:11.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.159+0000 7f7c6f600640 1 -- 172.21.15.66:0/3186203859 shutdown_connections 2024-09-10T22:14:11.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.159+0000 7f7c6f600640 1 -- 172.21.15.66:0/3186203859 wait complete. 2024-09-10T22:14:11.187 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:14:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 -- 172.21.15.66:0/3072236652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8cc15a310 msgr2=0x7fd8cc0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 --2- 172.21.15.66:0/3072236652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8cc15a310 0x7fd8cc0c48b0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009900 tx=0x7fd8bc01c850 comp rx=0 tx=0).stop 2024-09-10T22:14:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 -- 172.21.15.66:0/3072236652 shutdown_connections 2024-09-10T22:14:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 --2- 172.21.15.66:0/3072236652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8cc15a310 0x7fd8cc0c48b0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 -- 172.21.15.66:0/3072236652 >> 172.21.15.66:0/3072236652 conn(0x7fd8cc0cfa30 msgr2=0x7fd8cc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:11.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 -- 172.21.15.66:0/3072236652 shutdown_connections 2024-09-10T22:14:11.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 -- 172.21.15.66:0/3072236652 wait complete. 2024-09-10T22:14:11.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 Processor -- start 2024-09-10T22:14:11.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 -- start start 2024-09-10T22:14:11.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8cc0aeec0 0x7fd8cc14b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:11.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8cc15a310 0x7fd8cc14fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:11.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8cc0b5520 0x7fd8cc0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:11.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc168fc0 con 0x7fd8cc0b5520 2024-09-10T22:14:11.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc168e40 con 0x7fd8cc0aeec0 2024-09-10T22:14:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8d2400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd8cc169140 con 0x7fd8cc15a310 2024-09-10T22:14:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8cbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8cc0b5520 0x7fd8cc0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8cbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8cc0b5520 0x7fd8cc0c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60960/0 (socket says 172.21.15.66:60960) 2024-09-10T22:14:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8cbe00640 1 -- 172.21.15.66:0/3780039892 learned_addr learned my addr 172.21.15.66:0/3780039892 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8cb400640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8cc0aeec0 0x7fd8cc14b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.331+0000 7fd8c3e00640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8cc15a310 0x7fd8cc14fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8cbe00640 1 -- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8cc15a310 msgr2=0x7fd8cc14fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8cbe00640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8cc15a310 0x7fd8cc14fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8cbe00640 1 -- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8cc0aeec0 msgr2=0x7fd8cc14b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8cbe00640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8cc0aeec0 0x7fd8cc14b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8cbe00640 1 -- 172.21.15.66:0/3780039892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc1502e0 con 0x7fd8cc0b5520 2024-09-10T22:14:11.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8cbe00640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8cc0b5520 0x7fd8cc0c8a90 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400c7f0 tx=0x7fd8c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:11.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8c3400640 1 -- 172.21.15.66:0/3780039892 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4003b80 con 0x7fd8cc0b5520 2024-09-10T22:14:11.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8c3400640 1 -- 172.21.15.66:0/3780039892 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8c4003d20 con 0x7fd8cc0b5520 2024-09-10T22:14:11.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8c3400640 1 -- 172.21.15.66:0/3780039892 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400aab0 con 0x7fd8cc0b5520 2024-09-10T22:14:11.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8d2400640 1 -- 172.21.15.66:0/3780039892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc0c8340 con 0x7fd8cc0b5520 2024-09-10T22:14:11.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8d2400640 1 -- 172.21.15.66:0/3780039892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc0c6ca0 con 0x7fd8cc0b5520 2024-09-10T22:14:11.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8d2400640 1 -- 172.21.15.66:0/3780039892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc0c8860 con 0x7fd8cc0b5520 2024-09-10T22:14:11.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.335+0000 7fd8c3400640 1 -- 172.21.15.66:0/3780039892 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fd8c400ac50 con 0x7fd8cc0b5520 2024-09-10T22:14:11.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.339+0000 7fd8c3400640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd8a8074710 0x7fd8a8076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:11.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.339+0000 7fd8c3400640 1 -- 172.21.15.66:0/3780039892 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5849+0+0 (secure 0 0 0) 0x7fd8c4092b70 con 0x7fd8cc0b5520 2024-09-10T22:14:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.339+0000 7fd8cb400640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd8a8074710 0x7fd8a8076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.339+0000 7fd8cb400640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd8a8074710 0x7fd8a8076bd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0af000 tx=0x7fd8bc01b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:11.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.339+0000 7fd8c3400640 1 -- 172.21.15.66:0/3780039892 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd8c40c2de0 con 0x7fd8cc0b5520 2024-09-10T22:14:11.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.515+0000 7fd8d2400640 1 -- 172.21.15.66:0/3780039892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8cc0c48b0 con 0x7fd8cc0b5520 2024-09-10T22:14:11.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8c3400640 1 -- 172.21.15.66:0/3780039892 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+16903 (secure 0 0 0) 0x7fd8cc0c8860 con 0x7fd8cc0b5520 2024-09-10T22:14:11.523 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:14:11.523 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":140,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:14:10.533929+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:14:08.253835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T22:14:09.497667+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:14:11.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8d2400640 1 -- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd8a8074710 msgr2=0x7fd8a8076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:11.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8d2400640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd8a8074710 0x7fd8a8076bd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0af000 tx=0x7fd8bc01b090 comp rx=0 tx=0).stop 2024-09-10T22:14:11.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8d2400640 1 -- 172.21.15.66:0/3780039892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8cc0b5520 msgr2=0x7fd8cc0c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:11.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8d2400640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8cc0b5520 0x7fd8cc0c8a90 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400c7f0 tx=0x7fd8c400ccb0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8d2400640 1 -- 172.21.15.66:0/3780039892 shutdown_connections 2024-09-10T22:14:11.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8d2400640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd8a8074710 0x7fd8a8076bd0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8d2400640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8cc0b5520 0x7fd8cc0c8a90 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8d2400640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd8cc15a310 0x7fd8cc14fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8d2400640 1 --2- 172.21.15.66:0/3780039892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8cc0aeec0 0x7fd8cc14b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8d2400640 1 -- 172.21.15.66:0/3780039892 >> 172.21.15.66:0/3780039892 conn(0x7fd8cc0cfa30 msgr2=0x7fd8cc0af990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:11.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8d2400640 1 -- 172.21.15.66:0/3780039892 shutdown_connections 2024-09-10T22:14:11.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.519+0000 7fd8d2400640 1 -- 172.21.15.66:0/3780039892 wait complete. 2024-09-10T22:14:11.544 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:14:11.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 -- 172.21.15.66:0/2706759149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d80c4fc0 msgr2=0x7f76d80ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:11.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 --2- 172.21.15.66:0/2706759149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d80c4fc0 0x7f76d80ca8b0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f76d000acd0 tx=0x7f76d001de60 comp rx=0 tx=0).stop 2024-09-10T22:14:11.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 -- 172.21.15.66:0/2706759149 shutdown_connections 2024-09-10T22:14:11.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 --2- 172.21.15.66:0/2706759149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d80c4fc0 0x7f76d80ca8b0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 -- 172.21.15.66:0/2706759149 >> 172.21.15.66:0/2706759149 conn(0x7f76d80cfa30 msgr2=0x7f76d80cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:11.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 -- 172.21.15.66:0/2706759149 shutdown_connections 2024-09-10T22:14:11.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 -- 172.21.15.66:0/2706759149 wait complete. 2024-09-10T22:14:11.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 Processor -- start 2024-09-10T22:14:11.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 -- start start 2024-09-10T22:14:11.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76d80c4fc0 0x7f76d81f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:11.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d815a310 0x7f76d80ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:11.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76d81553e0 0x7f76d8153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:11.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76d81686a0 con 0x7f76d815a310 2024-09-10T22:14:11.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76d8168520 con 0x7f76d81553e0 2024-09-10T22:14:11.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76df000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f76d8168820 con 0x7f76d80c4fc0 2024-09-10T22:14:11.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d815a310 0x7f76d80ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:11.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d815a310 0x7f76d80ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60972/0 (socket says 172.21.15.66:60972) 2024-09-10T22:14:11.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76cfe00640 1 -- 172.21.15.66:0/1230165406 learned_addr learned my addr 172.21.15.66:0/1230165406 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:11.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76d7e00640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76d80c4fc0 0x7f76d81f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:11.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76dd000640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76d81553e0 0x7f76d8153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:11.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76cfe00640 1 -- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76d80c4fc0 msgr2=0x7f76d81f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:11.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76cfe00640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76d80c4fc0 0x7f76d81f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76cfe00640 1 -- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76d81553e0 msgr2=0x7f76d8153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:11.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76cfe00640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76d81553e0 0x7f76d8153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.687+0000 7f76cfe00640 1 -- 172.21.15.66:0/1230165406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d8156d10 con 0x7f76d815a310 2024-09-10T22:14:11.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.691+0000 7f76cfe00640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d815a310 0x7f76d80ca5b0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f76c000c500 tx=0x7f76c000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:11.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.691+0000 7f76d4c00640 1 -- 172.21.15.66:0/1230165406 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0017070 con 0x7f76d815a310 2024-09-10T22:14:11.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.691+0000 7f76d4c00640 1 -- 172.21.15.66:0/1230165406 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76c00053c0 con 0x7f76d815a310 2024-09-10T22:14:11.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.691+0000 7f76d4c00640 1 -- 172.21.15.66:0/1230165406 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0012890 con 0x7f76d815a310 2024-09-10T22:14:11.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.691+0000 7f76df000640 1 -- 172.21.15.66:0/1230165406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d8155c70 con 0x7f76d815a310 2024-09-10T22:14:11.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.691+0000 7f76df000640 1 -- 172.21.15.66:0/1230165406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d8156230 con 0x7f76d815a310 2024-09-10T22:14:11.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.691+0000 7f76df000640 1 -- 172.21.15.66:0/1230165406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d80c48b0 con 0x7f76d815a310 2024-09-10T22:14:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.691+0000 7f76d4c00640 1 -- 172.21.15.66:0/1230165406 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f76c0003600 con 0x7f76d815a310 2024-09-10T22:14:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.691+0000 7f76d4c00640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f76b4074740 0x7f76b4076c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.691+0000 7f76d4c00640 1 -- 172.21.15.66:0/1230165406 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5849+0+0 (secure 0 0 0) 0x7f76c0091940 con 0x7f76d815a310 2024-09-10T22:14:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.691+0000 7f76d7e00640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f76b4074740 0x7f76b4076c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.695+0000 7f76d7e00640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f76b4074740 0x7f76b4076c00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f76c80046c0 tx=0x7f76c8007ea0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.695+0000 7f76d4c00640 1 -- 172.21.15.66:0/1230165406 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f76c009a030 con 0x7f76d815a310 2024-09-10T22:14:11.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.871+0000 7f76df000640 1 -- 172.21.15.66:0/1230165406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76d8165fe0 con 0x7f76d815a310 2024-09-10T22:14:11.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.871+0000 7f76d4c00640 1 -- 172.21.15.66:0/1230165406 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+16903 (secure 0 0 0) 0x7f76c0067120 con 0x7f76d815a310 2024-09-10T22:14:11.877 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:14:11.877 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":140,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:14:10.533929+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:14:08.253835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T22:14:09.497667+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:14:11.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.875+0000 7f76df000640 1 -- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f76b4074740 msgr2=0x7f76b4076c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:11.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.875+0000 7f76df000640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f76b4074740 0x7f76b4076c00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f76c80046c0 tx=0x7f76c8007ea0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.875+0000 7f76df000640 1 -- 172.21.15.66:0/1230165406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d815a310 msgr2=0x7f76d80ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:11.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.875+0000 7f76df000640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d815a310 0x7f76d80ca5b0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f76c000c500 tx=0x7f76c000c9c0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.875+0000 7f76df000640 1 -- 172.21.15.66:0/1230165406 shutdown_connections 2024-09-10T22:14:11.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.875+0000 7f76df000640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f76b4074740 0x7f76b4076c00 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.875+0000 7f76df000640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76d81553e0 0x7f76d8153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.875+0000 7f76df000640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d815a310 0x7f76d80ca5b0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.875+0000 7f76df000640 1 --2- 172.21.15.66:0/1230165406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76d80c4fc0 0x7f76d81f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:11.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.875+0000 7f76df000640 1 -- 172.21.15.66:0/1230165406 >> 172.21.15.66:0/1230165406 conn(0x7f76d80cfa30 msgr2=0x7f76d8072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:11.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.875+0000 7f76df000640 1 -- 172.21.15.66:0/1230165406 shutdown_connections 2024-09-10T22:14:11.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:11.875+0000 7f76df000640 1 -- 172.21.15.66:0/1230165406 wait complete. 2024-09-10T22:14:11.899 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:14:12.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.043+0000 7fce2a200640 1 -- 172.21.15.66:0/907394937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce2415a310 msgr2=0x7fce240c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.043+0000 7fce2a200640 1 --2- 172.21.15.66:0/907394937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce2415a310 0x7fce240c48b0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fce14009900 tx=0x7fce1401c850 comp rx=0 tx=0).stop 2024-09-10T22:14:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.043+0000 7fce2a200640 1 -- 172.21.15.66:0/907394937 shutdown_connections 2024-09-10T22:14:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.043+0000 7fce2a200640 1 --2- 172.21.15.66:0/907394937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce2415a310 0x7fce240c48b0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.043+0000 7fce2a200640 1 -- 172.21.15.66:0/907394937 >> 172.21.15.66:0/907394937 conn(0x7fce240cfa30 msgr2=0x7fce240cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.043+0000 7fce2a200640 1 -- 172.21.15.66:0/907394937 shutdown_connections 2024-09-10T22:14:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.043+0000 7fce2a200640 1 -- 172.21.15.66:0/907394937 wait complete. 2024-09-10T22:14:12.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce2a200640 1 Processor -- start 2024-09-10T22:14:12.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce2a200640 1 -- start start 2024-09-10T22:14:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce2a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce240aeec0 0x7fce241fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce2a200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce2415a310 0x7fce241f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce2a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce241f8b50 0x7fce240c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce2a200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce2406eb00 con 0x7fce241f8b50 2024-09-10T22:14:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce2a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce2406e980 con 0x7fce240aeec0 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce2a200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fce2406ec80 con 0x7fce2415a310 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce23e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce241f8b50 0x7fce240c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce23e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce241f8b50 0x7fce240c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60988/0 (socket says 172.21.15.66:60988) 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce23e00640 1 -- 172.21.15.66:0/4095589964 learned_addr learned my addr 172.21.15.66:0/4095589964 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce23400640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce240aeec0 0x7fce241fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce1be00640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce2415a310 0x7fce241f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce23e00640 1 -- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce2415a310 msgr2=0x7fce241f8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce23e00640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce2415a310 0x7fce241f8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce23e00640 1 -- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce240aeec0 msgr2=0x7fce241fb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce23e00640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce240aeec0 0x7fce241fb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce23e00640 1 -- 172.21.15.66:0/4095589964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce241fbb60 con 0x7fce241f8b50 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce1be00640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce2415a310 0x7fce241f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce23e00640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce241f8b50 0x7fce240c6e60 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00c770 tx=0x7fce1c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce1b400640 1 -- 172.21.15.66:0/4095589964 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c017070 con 0x7fce241f8b50 2024-09-10T22:14:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce1b400640 1 -- 172.21.15.66:0/4095589964 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce1c005070 con 0x7fce241f8b50 2024-09-10T22:14:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce2a200640 1 -- 172.21.15.66:0/4095589964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce241fbe40 con 0x7fce241f8b50 2024-09-10T22:14:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce1b400640 1 -- 172.21.15.66:0/4095589964 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c0054c0 con 0x7fce241f8b50 2024-09-10T22:14:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.047+0000 7fce2a200640 1 -- 172.21.15.66:0/4095589964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2414b380 con 0x7fce241f8b50 2024-09-10T22:14:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.051+0000 7fce2a200640 1 -- 172.21.15.66:0/4095589964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce241fc360 con 0x7fce241f8b50 2024-09-10T22:14:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.051+0000 7fce1b400640 1 -- 172.21.15.66:0/4095589964 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fce1c005660 con 0x7fce241f8b50 2024-09-10T22:14:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.051+0000 7fce1b400640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce00074710 0x7fce00076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.051+0000 7fce1b400640 1 -- 172.21.15.66:0/4095589964 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5849+0+0 (secure 0 0 0) 0x7fce1c091cf0 con 0x7fce241f8b50 2024-09-10T22:14:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.051+0000 7fce23400640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce00074710 0x7fce00076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.051+0000 7fce23400640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce00074710 0x7fce00076bd0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fce140af000 tx=0x7fce1401b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.055+0000 7fce1b400640 1 -- 172.21.15.66:0/4095589964 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fce241fc360 con 0x7fce241f8b50 2024-09-10T22:14:12.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:12.271+0000 7fce2a200640 1 -- 172.21.15.66:0/4095589964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fce240c48b0 con 0x7fce241f8b50 2024-09-10T22:14:13.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.147+0000 7fce1b400640 1 -- 172.21.15.66:0/4095589964 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v95) ==== 118+0+0 (secure 0 0 0) 0x7fce1c05e760 con 0x7fce241f8b50 2024-09-10T22:14:13.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.151+0000 7fce2a200640 1 -- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce00074710 msgr2=0x7fce00076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:13.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.151+0000 7fce2a200640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce00074710 0x7fce00076bd0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fce140af000 tx=0x7fce1401b090 comp rx=0 tx=0).stop 2024-09-10T22:14:13.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.151+0000 7fce2a200640 1 -- 172.21.15.66:0/4095589964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce241f8b50 msgr2=0x7fce240c6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:13.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.151+0000 7fce2a200640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce241f8b50 0x7fce240c6e60 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00c770 tx=0x7fce1c00cc30 comp rx=0 tx=0).stop 2024-09-10T22:14:13.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.151+0000 7fce2a200640 1 -- 172.21.15.66:0/4095589964 shutdown_connections 2024-09-10T22:14:13.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.151+0000 7fce2a200640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fce00074710 0x7fce00076bd0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:13.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.151+0000 7fce2a200640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce241f8b50 0x7fce240c6e60 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:13.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.151+0000 7fce2a200640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce2415a310 0x7fce241f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:13.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.151+0000 7fce2a200640 1 --2- 172.21.15.66:0/4095589964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce240aeec0 0x7fce241fb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:13.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.151+0000 7fce2a200640 1 -- 172.21.15.66:0/4095589964 >> 172.21.15.66:0/4095589964 conn(0x7fce240cfa30 msgr2=0x7fce240cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:13.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.151+0000 7fce2a200640 1 -- 172.21.15.66:0/4095589964 shutdown_connections 2024-09-10T22:14:13.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.151+0000 7fce2a200640 1 -- 172.21.15.66:0/4095589964 wait complete. 2024-09-10T22:14:13.180 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:14:13.180 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:14:13.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 -- 172.21.15.66:0/2213097286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0740b0ef0 msgr2=0x7ff0740d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:13.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 --2- 172.21.15.66:0/2213097286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0740b0ef0 0x7ff0740d02c0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009bc0 tx=0x7ff06c01ca80 comp rx=0 tx=0).stop 2024-09-10T22:14:13.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 -- 172.21.15.66:0/2213097286 shutdown_connections 2024-09-10T22:14:13.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 --2- 172.21.15.66:0/2213097286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0740b0ef0 0x7ff0740d02c0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:13.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 --2- 172.21.15.66:0/2213097286 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff074071700 0x7ff074070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 -- 172.21.15.66:0/2213097286 >> 172.21.15.66:0/2213097286 conn(0x7ff0740d1d00 msgr2=0x7ff0740d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 -- 172.21.15.66:0/2213097286 shutdown_connections 2024-09-10T22:14:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 -- 172.21.15.66:0/2213097286 wait complete. 2024-09-10T22:14:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 Processor -- start 2024-09-10T22:14:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 -- start start 2024-09-10T22:14:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff074070ac0 0x7ff07414e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff074071700 0x7ff07415cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0740b0ef0 0x7ff07414f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff074160b50 con 0x7ff074071700 2024-09-10T22:14:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741609d0 con 0x7ff074070ac0 2024-09-10T22:14:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff07c600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff074160cd0 con 0x7ff0740b0ef0 2024-09-10T22:14:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff079200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff074071700 0x7ff07415cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff079200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff074071700 0x7ff07415cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52984/0 (socket says 172.21.15.66:52984) 2024-09-10T22:14:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.331+0000 7ff079200640 1 -- 172.21.15.66:0/699592096 learned_addr learned my addr 172.21.15.66:0/699592096 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff079c00640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff074070ac0 0x7ff07414e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff07a600640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0740b0ef0 0x7ff07414f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff079200640 1 -- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0740b0ef0 msgr2=0x7ff07414f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff079200640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0740b0ef0 0x7ff07414f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff079200640 1 -- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff074070ac0 msgr2=0x7ff07414e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff079200640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff074070ac0 0x7ff07414e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff079200640 1 -- 172.21.15.66:0/699592096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07415bca0 con 0x7ff074071700 2024-09-10T22:14:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff07a600640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0740b0ef0 0x7ff07414f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:14:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff079200640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff074071700 0x7ff07415cd40 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7ff06000d980 tx=0x7ff06000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff071600640 1 -- 172.21.15.66:0/699592096 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060017070 con 0x7ff074071700 2024-09-10T22:14:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff071600640 1 -- 172.21.15.66:0/699592096 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff060009d20 con 0x7ff074071700 2024-09-10T22:14:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff071600640 1 -- 172.21.15.66:0/699592096 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0600106c0 con 0x7ff074071700 2024-09-10T22:14:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff07c600640 1 -- 172.21.15.66:0/699592096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07415bf80 con 0x7ff074071700 2024-09-10T22:14:13.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff07c600640 1 -- 172.21.15.66:0/699592096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07415c540 con 0x7ff074071700 2024-09-10T22:14:13.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff07c600640 1 -- 172.21.15.66:0/699592096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff074070ea0 con 0x7ff074071700 2024-09-10T22:14:13.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff071600640 1 -- 172.21.15.66:0/699592096 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7ff060010860 con 0x7ff074071700 2024-09-10T22:14:13.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.335+0000 7ff071600640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff054074740 0x7ff054076c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:13.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.339+0000 7ff071600640 1 -- 172.21.15.66:0/699592096 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5849+0+0 (secure 0 0 0) 0x7ff060092780 con 0x7ff074071700 2024-09-10T22:14:13.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.339+0000 7ff079c00640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff054074740 0x7ff054076c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:13.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.339+0000 7ff079c00640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff054074740 0x7ff054076c00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff068000f70 tx=0x7ff068009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:13.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.339+0000 7ff071600640 1 -- 172.21.15.66:0/699592096 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff06005f8c0 con 0x7ff074071700 2024-09-10T22:14:13.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:13.547+0000 7ff07c600640 1 -- 172.21.15.66:0/699592096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff0740b1440 con 0x7ff074071700 2024-09-10T22:14:14.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.147+0000 7ff071600640 1 -- 172.21.15.66:0/699592096 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v96) ==== 122+0+0 (secure 0 0 0) 0x7ff06005f1c0 con 0x7ff074071700 2024-09-10T22:14:14.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.151+0000 7ff07c600640 1 -- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff054074740 msgr2=0x7ff054076c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:14.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.151+0000 7ff07c600640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff054074740 0x7ff054076c00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff068000f70 tx=0x7ff068009040 comp rx=0 tx=0).stop 2024-09-10T22:14:14.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.151+0000 7ff07c600640 1 -- 172.21.15.66:0/699592096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff074071700 msgr2=0x7ff07415cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:14.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.151+0000 7ff07c600640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff074071700 0x7ff07415cd40 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7ff06000d980 tx=0x7ff06000de40 comp rx=0 tx=0).stop 2024-09-10T22:14:14.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.151+0000 7ff07c600640 1 -- 172.21.15.66:0/699592096 shutdown_connections 2024-09-10T22:14:14.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.151+0000 7ff07c600640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff054074740 0x7ff054076c00 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.151+0000 7ff07c600640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0740b0ef0 0x7ff07414f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.151+0000 7ff07c600640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff074071700 0x7ff07415cd40 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.151+0000 7ff07c600640 1 --2- 172.21.15.66:0/699592096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff074070ac0 0x7ff07414e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.151+0000 7ff07c600640 1 -- 172.21.15.66:0/699592096 >> 172.21.15.66:0/699592096 conn(0x7ff0740d1d00 msgr2=0x7ff0740b7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:14.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.151+0000 7ff07c600640 1 -- 172.21.15.66:0/699592096 shutdown_connections 2024-09-10T22:14:14.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.151+0000 7ff07c600640 1 -- 172.21.15.66:0/699592096 wait complete. 2024-09-10T22:14:14.179 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:14:14.179 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:14:14.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.323+0000 7fad64a00640 1 -- 172.21.15.66:0/2837681763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad44004d50 msgr2=0x7fad440051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:14.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.323+0000 7fad64a00640 1 --2- 172.21.15.66:0/2837681763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad44004d50 0x7fad440051d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fad58001f50 tx=0x7fad5801f3b0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.323+0000 7fad5e800640 1 -- 172.21.15.66:0/2837681763 reap_dead start 2024-09-10T22:14:14.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.323+0000 7fad64a00640 1 -- 172.21.15.66:0/2837681763 shutdown_connections 2024-09-10T22:14:14.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.323+0000 7fad64a00640 1 -- 172.21.15.66:0/2837681763 >> 172.21.15.66:0/2837681763 conn(0x7fad600d1c60 msgr2=0x7fad600d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.323+0000 7fad64a00640 1 -- 172.21.15.66:0/2837681763 shutdown_connections 2024-09-10T22:14:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.323+0000 7fad64a00640 1 -- 172.21.15.66:0/2837681763 wait complete. 2024-09-10T22:14:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad64a00640 1 Processor -- start 2024-09-10T22:14:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad64a00640 1 -- start start 2024-09-10T22:14:14.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad64a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad44004d50 0x7fad600ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:14.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad64a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad600c8d30 0x7fad600cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:14.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad64a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad60158a60 0x7fad601f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:14.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad64a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad600afdc0 con 0x7fad44004d50 2024-09-10T22:14:14.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad64a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad600afc40 con 0x7fad600c8d30 2024-09-10T22:14:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad64a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fad600aff40 con 0x7fad60158a60 2024-09-10T22:14:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad5de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad44004d50 0x7fad600ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad5de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad44004d50 0x7fad600ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52990/0 (socket says 172.21.15.66:52990) 2024-09-10T22:14:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad5de00640 1 -- 172.21.15.66:0/1184914802 learned_addr learned my addr 172.21.15.66:0/1184914802 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad55400640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad600c8d30 0x7fad600cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad5e800640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad60158a60 0x7fad601f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad5de00640 1 -- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad60158a60 msgr2=0x7fad601f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad5de00640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad60158a60 0x7fad601f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad5de00640 1 -- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad600c8d30 msgr2=0x7fad600cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad5de00640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad600c8d30 0x7fad600cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad5de00640 1 -- 172.21.15.66:0/1184914802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad600c9280 con 0x7fad44004d50 2024-09-10T22:14:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad5de00640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad44004d50 0x7fad600ca320 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fad50011ae0 tx=0x7fad50009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:14.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad56a00640 1 -- 172.21.15.66:0/1184914802 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5001e070 con 0x7fad44004d50 2024-09-10T22:14:14.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad56a00640 1 -- 172.21.15.66:0/1184914802 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad5000fe40 con 0x7fad44004d50 2024-09-10T22:14:14.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad56a00640 1 -- 172.21.15.66:0/1184914802 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad500042b0 con 0x7fad44004d50 2024-09-10T22:14:14.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad64a00640 1 -- 172.21.15.66:0/1184914802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad600c9560 con 0x7fad44004d50 2024-09-10T22:14:14.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.327+0000 7fad64a00640 1 -- 172.21.15.66:0/1184914802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad600c9b20 con 0x7fad44004d50 2024-09-10T22:14:14.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.331+0000 7fad64a00640 1 -- 172.21.15.66:0/1184914802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad600d0320 con 0x7fad44004d50 2024-09-10T22:14:14.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.331+0000 7fad56a00640 1 -- 172.21.15.66:0/1184914802 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fad50003600 con 0x7fad44004d50 2024-09-10T22:14:14.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.331+0000 7fad56a00640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fad3c074710 0x7fad3c076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:14.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.331+0000 7fad56a00640 1 -- 172.21.15.66:0/1184914802 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5849+0+0 (secure 0 0 0) 0x7fad50093160 con 0x7fad44004d50 2024-09-10T22:14:14.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.331+0000 7fad55400640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fad3c074710 0x7fad3c076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:14.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.335+0000 7fad55400640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fad3c074710 0x7fad3c076bd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fad48001cb0 tx=0x7fad48006040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:14.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.335+0000 7fad56a00640 1 -- 172.21.15.66:0/1184914802 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fad5005fd50 con 0x7fad44004d50 2024-09-10T22:14:14.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.543+0000 7fad64a00640 1 -- 172.21.15.66:0/1184914802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad6015a470 con 0x7fad44004d50 2024-09-10T22:14:14.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad56a00640 1 -- 172.21.15.66:0/1184914802 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5221 (secure 0 0 0) 0x7fad50064780 con 0x7fad44004d50 2024-09-10T22:14:14.550 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:14:14.550 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":96,"btime":"2024-09-10T22:14:14:150270+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/401729631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":401729631},{"type":"v1","addr":"172.21.15.94:6836","nonce":401729631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/996427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":996427549},{"type":"v1","addr":"172.21.15.94:6837","nonce":996427549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2947602030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2947602030},{"type":"v1","addr":"172.21.15.66:6833","nonce":2947602030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:14:10.533890+0000","modified":"2024-09-10T22:14:13.552506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/3920857628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3920857628},{"type":"v1","addr":"172.21.15.66:6835","nonce":3920857628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9282,"qdb_cluster":[9282]},"id":10}]} 2024-09-10T22:14:14.550 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 96 2024-09-10T22:14:14.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad64a00640 1 -- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fad3c074710 msgr2=0x7fad3c076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:14.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad64a00640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fad3c074710 0x7fad3c076bd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fad48001cb0 tx=0x7fad48006040 comp rx=0 tx=0).stop 2024-09-10T22:14:14.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad64a00640 1 -- 172.21.15.66:0/1184914802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad44004d50 msgr2=0x7fad600ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:14.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad64a00640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad44004d50 0x7fad600ca320 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fad50011ae0 tx=0x7fad50009ea0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad64a00640 1 -- 172.21.15.66:0/1184914802 shutdown_connections 2024-09-10T22:14:14.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad64a00640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fad3c074710 0x7fad3c076bd0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad64a00640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad60158a60 0x7fad601f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad64a00640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad600c8d30 0x7fad600cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad64a00640 1 --2- 172.21.15.66:0/1184914802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad44004d50 0x7fad600ca320 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad64a00640 1 -- 172.21.15.66:0/1184914802 >> 172.21.15.66:0/1184914802 conn(0x7fad600d1c60 msgr2=0x7fad600b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:14.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad64a00640 1 -- 172.21.15.66:0/1184914802 shutdown_connections 2024-09-10T22:14:14.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.547+0000 7fad64a00640 1 -- 172.21.15.66:0/1184914802 wait complete. 2024-09-10T22:14:14.573 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:14:14.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.715+0000 7f3781a00640 1 -- 172.21.15.66:0/4139876156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c0cf830 msgr2=0x7f377c0d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:14.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.715+0000 7f3781a00640 1 --2- 172.21.15.66:0/4139876156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c0cf830 0x7f377c0d04e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f377c073840 tx=0x7f376401c830 comp rx=0 tx=0).stop 2024-09-10T22:14:14.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.715+0000 7f3781a00640 1 -- 172.21.15.66:0/4139876156 shutdown_connections 2024-09-10T22:14:14.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.715+0000 7f3781a00640 1 --2- 172.21.15.66:0/4139876156 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f377c0aec10 0x7f377c0af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.715+0000 7f3781a00640 1 --2- 172.21.15.66:0/4139876156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c0cf830 0x7f377c0d04e0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.715+0000 7f3781a00640 1 --2- 172.21.15.66:0/4139876156 >> v2:172.21.15.94:3300/0 conn(0x7f377c15bd40 0x7f377c0cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.715+0000 7f3781a00640 1 -- 172.21.15.66:0/4139876156 >> 172.21.15.66:0/4139876156 conn(0x7f377c0d1c60 msgr2=0x7f377c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:14.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.715+0000 7f3781a00640 1 -- 172.21.15.66:0/4139876156 shutdown_connections 2024-09-10T22:14:14.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.715+0000 7f3781a00640 1 -- 172.21.15.66:0/4139876156 wait complete. 2024-09-10T22:14:14.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3781a00640 1 Processor -- start 2024-09-10T22:14:14.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3781a00640 1 -- start start 2024-09-10T22:14:14.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3781a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f377c0aec10 0x7f377c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:14.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3781a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f377c0cf830 0x7f377c159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:14.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3781a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c15bd40 0x7f377c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:14.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3781a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f377c0ce9e0 con 0x7f377c15bd40 2024-09-10T22:14:14.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3781a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f377c0ce860 con 0x7f377c0cf830 2024-09-10T22:14:14.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3781a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f377c0ceb60 con 0x7f377c0aec10 2024-09-10T22:14:14.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f377b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c15bd40 0x7f377c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:14.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f377aa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f377c0aec10 0x7f377c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:14.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f377b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c15bd40 0x7f377c159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52998/0 (socket says 172.21.15.66:52998) 2024-09-10T22:14:14.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f377b400640 1 -- 172.21.15.66:0/886220091 learned_addr learned my addr 172.21.15.66:0/886220091 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:14.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3773e00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f377c0cf830 0x7f377c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:14.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f377aa00640 1 -- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f377c0cf830 msgr2=0x7f377c159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:14.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f377aa00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f377c0cf830 0x7f377c159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f377aa00640 1 -- 172.21.15.66:0/886220091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c15bd40 msgr2=0x7f377c159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:14.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f377aa00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c15bd40 0x7f377c159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f377aa00640 1 -- 172.21.15.66:0/886220091 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c156950 con 0x7f377c0aec10 2024-09-10T22:14:14.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f377b400640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c15bd40 0x7f377c159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:14:14.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f377aa00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f377c0aec10 0x7f377c0cb7c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f376c00e6c0 tx=0x7f376c00eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:14.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3772a00640 1 -- 172.21.15.66:0/886220091 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c00cab0 con 0x7f377c0aec10 2024-09-10T22:14:14.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3781a00640 1 -- 172.21.15.66:0/886220091 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377c156c30 con 0x7f377c0aec10 2024-09-10T22:14:14.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3772a00640 1 -- 172.21.15.66:0/886220091 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f376c00cd40 con 0x7f377c0aec10 2024-09-10T22:14:14.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3772a00640 1 -- 172.21.15.66:0/886220091 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c010690 con 0x7f377c0aec10 2024-09-10T22:14:14.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.719+0000 7f3781a00640 1 -- 172.21.15.66:0/886220091 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f377c1571f0 con 0x7f377c0aec10 2024-09-10T22:14:14.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.723+0000 7f3781a00640 1 -- 172.21.15.66:0/886220091 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f377c0af0a0 con 0x7f377c0aec10 2024-09-10T22:14:14.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.723+0000 7f3772a00640 1 -- 172.21.15.66:0/886220091 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f376c029080 con 0x7f377c0aec10 2024-09-10T22:14:14.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.723+0000 7f3772a00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3758074710 0x7f3758076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:14.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.723+0000 7f3772a00640 1 -- 172.21.15.66:0/886220091 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(141..141 src has 1..141) ==== 5849+0+0 (secure 0 0 0) 0x7f376c0931f0 con 0x7f377c0aec10 2024-09-10T22:14:14.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.723+0000 7f3773e00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3758074710 0x7f3758076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:14.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.723+0000 7f3773e00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3758074710 0x7f3758076bd0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f37640af000 tx=0x7f376401b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:14.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.727+0000 7f3772a00640 1 -- 172.21.15.66:0/886220091 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f376c05fce0 con 0x7f377c0aec10 2024-09-10T22:14:14.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.899+0000 7f3781a00640 1 -- 172.21.15.66:0/886220091 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f377c0b0df0 con 0x7f377c0aec10 2024-09-10T22:14:14.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3772a00640 1 -- 172.21.15.66:0/886220091 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16890 (secure 0 0 0) 0x7f376c064710 con 0x7f377c0aec10 2024-09-10T22:14:14.908 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:14:14.908 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":141,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:14:12.134795+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:14:08.253835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T22:14:09.497667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:14:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3781a00640 1 -- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3758074710 msgr2=0x7f3758076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3781a00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3758074710 0x7f3758076bd0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f37640af000 tx=0x7f376401b070 comp rx=0 tx=0).stop 2024-09-10T22:14:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3781a00640 1 -- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f377c0aec10 msgr2=0x7f377c0cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3781a00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f377c0aec10 0x7f377c0cb7c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f376c00e6c0 tx=0x7f376c00eb80 comp rx=0 tx=0).stop 2024-09-10T22:14:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3781a00640 1 -- 172.21.15.66:0/886220091 shutdown_connections 2024-09-10T22:14:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3781a00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3758074710 0x7f3758076bd0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3781a00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c15bd40 0x7f377c159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3781a00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f377c0cf830 0x7f377c159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3781a00640 1 --2- 172.21.15.66:0/886220091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f377c0aec10 0x7f377c0cb7c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3781a00640 1 -- 172.21.15.66:0/886220091 >> 172.21.15.66:0/886220091 conn(0x7f377c0d1c60 msgr2=0x7f377c0cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:14.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3781a00640 1 -- 172.21.15.66:0/886220091 shutdown_connections 2024-09-10T22:14:14.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:14.903+0000 7f3781a00640 1 -- 172.21.15.66:0/886220091 wait complete. 2024-09-10T22:14:14.930 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:14:14.930 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:14:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.071+0000 7f17b4600640 1 -- 172.21.15.66:0/2420872206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1794009f60 msgr2=0x7f179400c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.071+0000 7f17b4600640 1 --2- 172.21.15.66:0/2420872206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1794009f60 0x7f179400c3f0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f17a4001e60 tx=0x7f17a401cbe0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.071+0000 7f17b2600640 1 -- 172.21.15.66:0/2420872206 reap_dead start 2024-09-10T22:14:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.071+0000 7f17b4600640 1 -- 172.21.15.66:0/2420872206 shutdown_connections 2024-09-10T22:14:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.071+0000 7f17b4600640 1 -- 172.21.15.66:0/2420872206 >> 172.21.15.66:0/2420872206 conn(0x7f17ac0cfa30 msgr2=0x7f17ac0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.071+0000 7f17b4600640 1 -- 172.21.15.66:0/2420872206 shutdown_connections 2024-09-10T22:14:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.071+0000 7f17b4600640 1 -- 172.21.15.66:0/2420872206 wait complete. 2024-09-10T22:14:15.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b4600640 1 Processor -- start 2024-09-10T22:14:15.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b4600640 1 -- start start 2024-09-10T22:14:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b4600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1794009f60 0x7f17ac159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b4600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17ac1f7ca0 0x7f17ac0c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b4600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17ac0ad8f0 0x7f17ac0add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b4600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac06e9c0 con 0x7f1794009f60 2024-09-10T22:14:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b4600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac06e840 con 0x7f17ac0ad8f0 2024-09-10T22:14:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b4600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f17ac06eb40 con 0x7f17ac1f7ca0 2024-09-10T22:14:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1794009f60 0x7f17ac159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1794009f60 0x7f17ac159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53012/0 (socket says 172.21.15.66:53012) 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b1c00640 1 -- 172.21.15.66:0/4167868234 learned_addr learned my addr 172.21.15.66:0/4167868234 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b1200640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17ac1f7ca0 0x7f17ac0c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b2600640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17ac0ad8f0 0x7f17ac0add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b1c00640 1 -- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17ac1f7ca0 msgr2=0x7f17ac0c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b1c00640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17ac1f7ca0 0x7f17ac0c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b1c00640 1 -- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17ac0ad8f0 msgr2=0x7f17ac0add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b1c00640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17ac0ad8f0 0x7f17ac0add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b1c00640 1 -- 172.21.15.66:0/4167868234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ac1f8210 con 0x7f1794009f60 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b1200640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17ac1f7ca0 0x7f17ac0c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b1c00640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1794009f60 0x7f17ac159270 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f17a00126c0 tx=0x7f17a0012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17a9600640 1 -- 172.21.15.66:0/4167868234 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0010b40 con 0x7f1794009f60 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17a9600640 1 -- 172.21.15.66:0/4167868234 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17a0017030 con 0x7f1794009f60 2024-09-10T22:14:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17a9600640 1 -- 172.21.15.66:0/4167868234 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0013450 con 0x7f1794009f60 2024-09-10T22:14:15.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b4600640 1 -- 172.21.15.66:0/4167868234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ac1f84f0 con 0x7f1794009f60 2024-09-10T22:14:15.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.075+0000 7f17b4600640 1 -- 172.21.15.66:0/4167868234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ac1f8ab0 con 0x7f1794009f60 2024-09-10T22:14:15.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.079+0000 7f17b4600640 1 -- 172.21.15.66:0/4167868234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17ac06f0d0 con 0x7f1794009f60 2024-09-10T22:14:15.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.079+0000 7f17a9600640 1 -- 172.21.15.66:0/4167868234 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f17a00135f0 con 0x7f1794009f60 2024-09-10T22:14:15.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.079+0000 7f17a9600640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f178c074710 0x7f178c076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:15.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.079+0000 7f17a9600640 1 -- 172.21.15.66:0/4167868234 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5849+0+0 (secure 0 0 0) 0x7f17a0095f30 con 0x7f1794009f60 2024-09-10T22:14:15.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.079+0000 7f17b1200640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f178c074710 0x7f178c076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:15.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.083+0000 7f17b1200640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f178c074710 0x7f178c076bd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0ae4d0 tx=0x7f17980091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:15.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.083+0000 7f17a9600640 1 -- 172.21.15.66:0/4167868234 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f17a0062a20 con 0x7f1794009f60 2024-09-10T22:14:15.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.259+0000 7f17b4600640 1 -- 172.21.15.66:0/4167868234 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17ac159b30 con 0x7f178c074710 2024-09-10T22:14:15.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.263+0000 7f17a9600640 1 -- 172.21.15.66:0/4167868234 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252502 (secure 0 0 0) 0x7f179805ea70 con 0x7f178c074710 2024-09-10T22:14:15.270 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:14:15.272 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:14:15.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.267+0000 7f17b4600640 1 -- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f178c074710 msgr2=0x7f178c076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.267+0000 7f17b4600640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f178c074710 0x7f178c076bd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0ae4d0 tx=0x7f17980091e0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.267+0000 7f17b4600640 1 -- 172.21.15.66:0/4167868234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1794009f60 msgr2=0x7f17ac159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.267+0000 7f17b4600640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1794009f60 0x7f17ac159270 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f17a00126c0 tx=0x7f17a0012b80 comp rx=0 tx=0).stop 2024-09-10T22:14:15.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.267+0000 7f17b4600640 1 -- 172.21.15.66:0/4167868234 shutdown_connections 2024-09-10T22:14:15.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.267+0000 7f17b4600640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f178c074710 0x7f178c076bd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.267+0000 7f17b4600640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17ac0ad8f0 0x7f17ac0add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.267+0000 7f17b4600640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17ac1f7ca0 0x7f17ac0c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.267+0000 7f17b4600640 1 --2- 172.21.15.66:0/4167868234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1794009f60 0x7f17ac159270 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.267+0000 7f17b4600640 1 -- 172.21.15.66:0/4167868234 >> 172.21.15.66:0/4167868234 conn(0x7f17ac0cfa30 msgr2=0x7f17ac142e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:15.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.267+0000 7f17b4600640 1 -- 172.21.15.66:0/4167868234 shutdown_connections 2024-09-10T22:14:15.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.267+0000 7f17b4600640 1 -- 172.21.15.66:0/4167868234 wait complete. 2024-09-10T22:14:15.291 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":552,"stamp":"2024-09-10T22:14:15.169154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1074604,"kb_used_data":14840,"kb_used_omap":12,"kb_used_meta":1059571,"kb_avail":753900116,"statfs":{"total":773094113280,"available":771993718784,"internally_reserved":0,"allocated":15196160,"data_stored":7328799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1085001268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.050652"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139619+0000","last_change":"2024-09-10T22:14:11.156782+0000","last_active":"2024-09-10T22:14:12.139619+0000","last_peered":"2024-09-10T22:14:12.139619+0000","last_clean":"2024-09-10T22:14:12.139619+0000","last_became_active":"2024-09-10T22:14:11.156441+0000","last_became_peered":"2024-09-10T22:14:11.156441+0000","last_unstale":"2024-09-10T22:14:12.139619+0000","last_undegraded":"2024-09-10T22:14:12.139619+0000","last_fullsized":"2024-09-10T22:14:12.139619+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:39.168936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145834+0000","last_change":"2024-09-10T22:14:11.140653+0000","last_active":"2024-09-10T22:14:12.145834+0000","last_peered":"2024-09-10T22:14:12.145834+0000","last_clean":"2024-09-10T22:14:12.145834+0000","last_became_active":"2024-09-10T22:14:11.140338+0000","last_became_peered":"2024-09-10T22:14:11.140338+0000","last_unstale":"2024-09-10T22:14:12.145834+0000","last_undegraded":"2024-09-10T22:14:12.145834+0000","last_fullsized":"2024-09-10T22:14:12.145834+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:16.106926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150740+0000","last_change":"2024-09-10T22:14:11.154656+0000","last_active":"2024-09-10T22:14:12.150740+0000","last_peered":"2024-09-10T22:14:12.150740+0000","last_clean":"2024-09-10T22:14:12.150740+0000","last_became_active":"2024-09-10T22:14:11.154387+0000","last_became_peered":"2024-09-10T22:14:11.154387+0000","last_unstale":"2024-09-10T22:14:12.150740+0000","last_undegraded":"2024-09-10T22:14:12.150740+0000","last_fullsized":"2024-09-10T22:14:12.150740+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:17.167373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150908+0000","last_change":"2024-09-10T22:14:11.156234+0000","last_active":"2024-09-10T22:14:12.150908+0000","last_peered":"2024-09-10T22:14:12.150908+0000","last_clean":"2024-09-10T22:14:12.150908+0000","last_became_active":"2024-09-10T22:14:11.155964+0000","last_became_peered":"2024-09-10T22:14:11.155964+0000","last_unstale":"2024-09-10T22:14:12.150908+0000","last_undegraded":"2024-09-10T22:14:12.150908+0000","last_fullsized":"2024-09-10T22:14:12.150908+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:46.284534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145278+0000","last_change":"2024-09-10T22:14:11.155027+0000","last_active":"2024-09-10T22:14:12.145278+0000","last_peered":"2024-09-10T22:14:12.145278+0000","last_clean":"2024-09-10T22:14:12.145278+0000","last_became_active":"2024-09-10T22:14:11.154497+0000","last_became_peered":"2024-09-10T22:14:11.154497+0000","last_unstale":"2024-09-10T22:14:12.145278+0000","last_undegraded":"2024-09-10T22:14:12.145278+0000","last_fullsized":"2024-09-10T22:14:12.145278+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:12.933247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139113+0000","last_change":"2024-09-10T22:14:11.152457+0000","last_active":"2024-09-10T22:14:12.139113+0000","last_peered":"2024-09-10T22:14:12.139113+0000","last_clean":"2024-09-10T22:14:12.139113+0000","last_became_active":"2024-09-10T22:14:11.152149+0000","last_became_peered":"2024-09-10T22:14:11.152149+0000","last_unstale":"2024-09-10T22:14:12.139113+0000","last_undegraded":"2024-09-10T22:14:12.139113+0000","last_fullsized":"2024-09-10T22:14:12.139113+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:12.933247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.143804+0000","last_change":"2024-09-10T22:14:11.144548+0000","last_active":"2024-09-10T22:14:12.143804+0000","last_peered":"2024-09-10T22:14:12.143804+0000","last_clean":"2024-09-10T22:14:12.143804+0000","last_became_active":"2024-09-10T22:14:11.144290+0000","last_became_peered":"2024-09-10T22:14:11.144290+0000","last_unstale":"2024-09-10T22:14:12.143804+0000","last_undegraded":"2024-09-10T22:14:12.143804+0000","last_fullsized":"2024-09-10T22:14:12.143804+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:32.440842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.138947+0000","last_change":"2024-09-10T22:14:11.140584+0000","last_active":"2024-09-10T22:14:12.138947+0000","last_peered":"2024-09-10T22:14:12.138947+0000","last_clean":"2024-09-10T22:14:12.138947+0000","last_became_active":"2024-09-10T22:14:11.140218+0000","last_became_peered":"2024-09-10T22:14:11.140218+0000","last_unstale":"2024-09-10T22:14:12.138947+0000","last_undegraded":"2024-09-10T22:14:12.138947+0000","last_fullsized":"2024-09-10T22:14:12.138947+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:16.145337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145789+0000","last_change":"2024-09-10T22:14:11.137821+0000","last_active":"2024-09-10T22:14:12.145789+0000","last_peered":"2024-09-10T22:14:12.145789+0000","last_clean":"2024-09-10T22:14:12.145789+0000","last_became_active":"2024-09-10T22:14:11.137555+0000","last_became_peered":"2024-09-10T22:14:11.137555+0000","last_unstale":"2024-09-10T22:14:12.145789+0000","last_undegraded":"2024-09-10T22:14:12.145789+0000","last_fullsized":"2024-09-10T22:14:12.145789+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:16.041501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146672+0000","last_change":"2024-09-10T22:14:11.160296+0000","last_active":"2024-09-10T22:14:12.146672+0000","last_peered":"2024-09-10T22:14:12.146672+0000","last_clean":"2024-09-10T22:14:12.146672+0000","last_became_active":"2024-09-10T22:14:11.158717+0000","last_became_peered":"2024-09-10T22:14:11.158717+0000","last_unstale":"2024-09-10T22:14:12.146672+0000","last_undegraded":"2024-09-10T22:14:12.146672+0000","last_fullsized":"2024-09-10T22:14:12.146672+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:49.348318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139298+0000","last_change":"2024-09-10T22:14:11.145655+0000","last_active":"2024-09-10T22:14:12.139298+0000","last_peered":"2024-09-10T22:14:12.139298+0000","last_clean":"2024-09-10T22:14:12.139298+0000","last_became_active":"2024-09-10T22:14:11.140070+0000","last_became_peered":"2024-09-10T22:14:11.140070+0000","last_unstale":"2024-09-10T22:14:12.139298+0000","last_undegraded":"2024-09-10T22:14:12.139298+0000","last_fullsized":"2024-09-10T22:14:12.139298+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:50.037416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147807+0000","last_change":"2024-09-10T22:14:11.160833+0000","last_active":"2024-09-10T22:14:12.147807+0000","last_peered":"2024-09-10T22:14:12.147807+0000","last_clean":"2024-09-10T22:14:12.147807+0000","last_became_active":"2024-09-10T22:14:11.159759+0000","last_became_peered":"2024-09-10T22:14:11.159759+0000","last_unstale":"2024-09-10T22:14:12.147807+0000","last_undegraded":"2024-09-10T22:14:12.147807+0000","last_fullsized":"2024-09-10T22:14:12.147807+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:44.861904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145498+0000","last_change":"2024-09-10T22:14:11.154755+0000","last_active":"2024-09-10T22:14:12.145498+0000","last_peered":"2024-09-10T22:14:12.145498+0000","last_clean":"2024-09-10T22:14:12.145498+0000","last_became_active":"2024-09-10T22:14:11.154155+0000","last_became_peered":"2024-09-10T22:14:11.154155+0000","last_unstale":"2024-09-10T22:14:12.145498+0000","last_undegraded":"2024-09-10T22:14:12.145498+0000","last_fullsized":"2024-09-10T22:14:12.145498+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:50.037416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149505+0000","last_change":"2024-09-10T22:14:11.151653+0000","last_active":"2024-09-10T22:14:12.149505+0000","last_peered":"2024-09-10T22:14:12.149505+0000","last_clean":"2024-09-10T22:14:12.149505+0000","last_became_active":"2024-09-10T22:14:11.151318+0000","last_became_peered":"2024-09-10T22:14:11.151318+0000","last_unstale":"2024-09-10T22:14:12.149505+0000","last_undegraded":"2024-09-10T22:14:12.149505+0000","last_fullsized":"2024-09-10T22:14:12.149505+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:07.718745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146620+0000","last_change":"2024-09-10T22:14:11.154969+0000","last_active":"2024-09-10T22:14:12.146620+0000","last_peered":"2024-09-10T22:14:12.146620+0000","last_clean":"2024-09-10T22:14:12.146620+0000","last_became_active":"2024-09-10T22:14:11.154005+0000","last_became_peered":"2024-09-10T22:14:11.154005+0000","last_unstale":"2024-09-10T22:14:12.146620+0000","last_undegraded":"2024-09-10T22:14:12.146620+0000","last_fullsized":"2024-09-10T22:14:12.146620+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:08.016417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.156480+0000","last_change":"2024-09-10T22:14:11.151532+0000","last_active":"2024-09-10T22:14:11.156480+0000","last_peered":"2024-09-10T22:14:11.156480+0000","last_clean":"2024-09-10T22:14:11.156480+0000","last_became_active":"2024-09-10T22:14:11.151250+0000","last_became_peered":"2024-09-10T22:14:11.151250+0000","last_unstale":"2024-09-10T22:14:11.156480+0000","last_undegraded":"2024-09-10T22:14:11.156480+0000","last_fullsized":"2024-09-10T22:14:11.156480+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:04.487186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147564+0000","last_change":"2024-09-10T22:14:11.166179+0000","last_active":"2024-09-10T22:14:12.147564+0000","last_peered":"2024-09-10T22:14:12.147564+0000","last_clean":"2024-09-10T22:14:12.147564+0000","last_became_active":"2024-09-10T22:14:11.165644+0000","last_became_peered":"2024-09-10T22:14:11.165644+0000","last_unstale":"2024-09-10T22:14:12.147564+0000","last_undegraded":"2024-09-10T22:14:12.147564+0000","last_fullsized":"2024-09-10T22:14:12.147564+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:16.411855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150379+0000","last_change":"2024-09-10T22:14:11.152176+0000","last_active":"2024-09-10T22:14:12.150379+0000","last_peered":"2024-09-10T22:14:12.150379+0000","last_clean":"2024-09-10T22:14:12.150379+0000","last_became_active":"2024-09-10T22:14:11.151499+0000","last_became_peered":"2024-09-10T22:14:11.151499+0000","last_unstale":"2024-09-10T22:14:12.150379+0000","last_undegraded":"2024-09-10T22:14:12.150379+0000","last_fullsized":"2024-09-10T22:14:12.150379+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:52.932120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139590+0000","last_change":"2024-09-10T22:14:11.150200+0000","last_active":"2024-09-10T22:14:12.139590+0000","last_peered":"2024-09-10T22:14:12.139590+0000","last_clean":"2024-09-10T22:14:12.139590+0000","last_became_active":"2024-09-10T22:14:11.149920+0000","last_became_peered":"2024-09-10T22:14:11.149920+0000","last_unstale":"2024-09-10T22:14:12.139590+0000","last_undegraded":"2024-09-10T22:14:12.139590+0000","last_fullsized":"2024-09-10T22:14:12.139590+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:56.027031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.158650+0000","last_change":"2024-09-10T22:14:11.153324+0000","last_active":"2024-09-10T22:14:11.158650+0000","last_peered":"2024-09-10T22:14:11.158650+0000","last_clean":"2024-09-10T22:14:11.158650+0000","last_became_active":"2024-09-10T22:14:11.153038+0000","last_became_peered":"2024-09-10T22:14:11.153038+0000","last_unstale":"2024-09-10T22:14:11.158650+0000","last_undegraded":"2024-09-10T22:14:11.158650+0000","last_fullsized":"2024-09-10T22:14:11.158650+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:14.507062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147275+0000","last_change":"2024-09-10T22:14:11.156650+0000","last_active":"2024-09-10T22:14:12.147275+0000","last_peered":"2024-09-10T22:14:12.147275+0000","last_clean":"2024-09-10T22:14:12.147275+0000","last_became_active":"2024-09-10T22:14:11.156387+0000","last_became_peered":"2024-09-10T22:14:11.156387+0000","last_unstale":"2024-09-10T22:14:12.147275+0000","last_undegraded":"2024-09-10T22:14:12.147275+0000","last_fullsized":"2024-09-10T22:14:12.147275+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:24.593483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149736+0000","last_change":"2024-09-10T22:14:11.140124+0000","last_active":"2024-09-10T22:14:12.149736+0000","last_peered":"2024-09-10T22:14:12.149736+0000","last_clean":"2024-09-10T22:14:12.149736+0000","last_became_active":"2024-09-10T22:14:11.139638+0000","last_became_peered":"2024-09-10T22:14:11.139638+0000","last_unstale":"2024-09-10T22:14:12.149736+0000","last_undegraded":"2024-09-10T22:14:12.149736+0000","last_fullsized":"2024-09-10T22:14:12.149736+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:15.767299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149632+0000","last_change":"2024-09-10T22:14:11.144529+0000","last_active":"2024-09-10T22:14:12.149632+0000","last_peered":"2024-09-10T22:14:12.149632+0000","last_clean":"2024-09-10T22:14:12.149632+0000","last_became_active":"2024-09-10T22:14:11.144245+0000","last_became_peered":"2024-09-10T22:14:11.144245+0000","last_unstale":"2024-09-10T22:14:12.149632+0000","last_undegraded":"2024-09-10T22:14:12.149632+0000","last_fullsized":"2024-09-10T22:14:12.149632+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:55.019694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147752+0000","last_change":"2024-09-10T22:14:11.154833+0000","last_active":"2024-09-10T22:14:12.147752+0000","last_peered":"2024-09-10T22:14:12.147752+0000","last_clean":"2024-09-10T22:14:12.147752+0000","last_became_active":"2024-09-10T22:14:11.154571+0000","last_became_peered":"2024-09-10T22:14:11.154571+0000","last_unstale":"2024-09-10T22:14:12.147752+0000","last_undegraded":"2024-09-10T22:14:12.147752+0000","last_fullsized":"2024-09-10T22:14:12.147752+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:01.295873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.156698+0000","last_change":"2024-09-10T22:14:11.151352+0000","last_active":"2024-09-10T22:14:11.156698+0000","last_peered":"2024-09-10T22:14:11.156698+0000","last_clean":"2024-09-10T22:14:11.156698+0000","last_became_active":"2024-09-10T22:14:11.150860+0000","last_became_peered":"2024-09-10T22:14:11.150860+0000","last_unstale":"2024-09-10T22:14:11.156698+0000","last_undegraded":"2024-09-10T22:14:11.156698+0000","last_fullsized":"2024-09-10T22:14:11.156698+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:55.355103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139478+0000","last_change":"2024-09-10T22:14:11.146866+0000","last_active":"2024-09-10T22:14:12.139478+0000","last_peered":"2024-09-10T22:14:12.139478+0000","last_clean":"2024-09-10T22:14:12.139478+0000","last_became_active":"2024-09-10T22:14:11.146597+0000","last_became_peered":"2024-09-10T22:14:11.146597+0000","last_unstale":"2024-09-10T22:14:12.139478+0000","last_undegraded":"2024-09-10T22:14:12.139478+0000","last_fullsized":"2024-09-10T22:14:12.139478+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:09.566270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.158405+0000","last_change":"2024-09-10T22:14:11.151775+0000","last_active":"2024-09-10T22:14:11.158405+0000","last_peered":"2024-09-10T22:14:11.158405+0000","last_clean":"2024-09-10T22:14:11.158405+0000","last_became_active":"2024-09-10T22:14:11.151397+0000","last_became_peered":"2024-09-10T22:14:11.151397+0000","last_unstale":"2024-09-10T22:14:11.158405+0000","last_undegraded":"2024-09-10T22:14:11.158405+0000","last_fullsized":"2024-09-10T22:14:11.158405+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:33.228563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.144093+0000","last_change":"2024-09-10T22:14:11.152661+0000","last_active":"2024-09-10T22:14:12.144093+0000","last_peered":"2024-09-10T22:14:12.144093+0000","last_clean":"2024-09-10T22:14:12.144093+0000","last_became_active":"2024-09-10T22:14:11.152341+0000","last_became_peered":"2024-09-10T22:14:11.152341+0000","last_unstale":"2024-09-10T22:14:12.144093+0000","last_undegraded":"2024-09-10T22:14:12.144093+0000","last_fullsized":"2024-09-10T22:14:12.144093+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:08.016417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.138990+0000","last_change":"2024-09-10T22:14:11.151730+0000","last_active":"2024-09-10T22:14:12.138990+0000","last_peered":"2024-09-10T22:14:12.138990+0000","last_clean":"2024-09-10T22:14:12.138990+0000","last_became_active":"2024-09-10T22:14:11.151475+0000","last_became_peered":"2024-09-10T22:14:11.151475+0000","last_unstale":"2024-09-10T22:14:12.138990+0000","last_undegraded":"2024-09-10T22:14:12.138990+0000","last_fullsized":"2024-09-10T22:14:12.138990+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:15.312565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149745+0000","last_change":"2024-09-10T22:14:11.145130+0000","last_active":"2024-09-10T22:14:12.149745+0000","last_peered":"2024-09-10T22:14:12.149745+0000","last_clean":"2024-09-10T22:14:12.149745+0000","last_became_active":"2024-09-10T22:14:11.143987+0000","last_became_peered":"2024-09-10T22:14:11.143987+0000","last_unstale":"2024-09-10T22:14:12.149745+0000","last_undegraded":"2024-09-10T22:14:12.149745+0000","last_fullsized":"2024-09-10T22:14:12.149745+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:31.934705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145591+0000","last_change":"2024-09-10T22:14:11.152403+0000","last_active":"2024-09-10T22:14:12.145591+0000","last_peered":"2024-09-10T22:14:12.145591+0000","last_clean":"2024-09-10T22:14:12.145591+0000","last_became_active":"2024-09-10T22:14:11.152128+0000","last_became_peered":"2024-09-10T22:14:11.152128+0000","last_unstale":"2024-09-10T22:14:12.145591+0000","last_undegraded":"2024-09-10T22:14:12.145591+0000","last_fullsized":"2024-09-10T22:14:12.145591+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:28.222948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.150330+0000","last_change":"2024-09-10T22:14:11.147854+0000","last_active":"2024-09-10T22:14:11.150330+0000","last_peered":"2024-09-10T22:14:11.150330+0000","last_clean":"2024-09-10T22:14:11.150330+0000","last_became_active":"2024-09-10T22:14:11.147581+0000","last_became_peered":"2024-09-10T22:14:11.147581+0000","last_unstale":"2024-09-10T22:14:11.150330+0000","last_undegraded":"2024-09-10T22:14:11.150330+0000","last_fullsized":"2024-09-10T22:14:11.150330+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:39.564127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146948+0000","last_change":"2024-09-10T22:14:11.160273+0000","last_active":"2024-09-10T22:14:12.146948+0000","last_peered":"2024-09-10T22:14:12.146948+0000","last_clean":"2024-09-10T22:14:12.146948+0000","last_became_active":"2024-09-10T22:14:11.159258+0000","last_became_peered":"2024-09-10T22:14:11.159258+0000","last_unstale":"2024-09-10T22:14:12.146948+0000","last_undegraded":"2024-09-10T22:14:12.146948+0000","last_fullsized":"2024-09-10T22:14:12.146948+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:08.757824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"140'4","reported_seq":19,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147344+0000","last_change":"2024-09-10T22:14:10.143281+0000","last_active":"2024-09-10T22:14:12.147344+0000","last_peered":"2024-09-10T22:14:12.147344+0000","last_clean":"2024-09-10T22:14:12.147344+0000","last_became_active":"2024-09-10T22:14:10.142965+0000","last_became_peered":"2024-09-10T22:14:10.142965+0000","last_unstale":"2024-09-10T22:14:12.147344+0000","last_undegraded":"2024-09-10T22:14:12.147344+0000","last_fullsized":"2024-09-10T22:14:12.147344+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:15.034996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146116+0000","last_change":"2024-09-10T22:14:11.141664+0000","last_active":"2024-09-10T22:14:12.146116+0000","last_peered":"2024-09-10T22:14:12.146116+0000","last_clean":"2024-09-10T22:14:12.146116+0000","last_became_active":"2024-09-10T22:14:11.141382+0000","last_became_peered":"2024-09-10T22:14:11.141382+0000","last_unstale":"2024-09-10T22:14:12.146116+0000","last_undegraded":"2024-09-10T22:14:12.146116+0000","last_fullsized":"2024-09-10T22:14:12.146116+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:13.908833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"140'1","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145957+0000","last_change":"2024-09-10T22:14:10.130630+0000","last_active":"2024-09-10T22:14:12.145957+0000","last_peered":"2024-09-10T22:14:12.145957+0000","last_clean":"2024-09-10T22:14:12.145957+0000","last_became_active":"2024-09-10T22:14:10.129926+0000","last_became_peered":"2024-09-10T22:14:10.129926+0000","last_unstale":"2024-09-10T22:14:12.145957+0000","last_undegraded":"2024-09-10T22:14:12.145957+0000","last_fullsized":"2024-09-10T22:14:12.145957+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:14.760794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139239+0000","last_change":"2024-09-10T22:14:11.141345+0000","last_active":"2024-09-10T22:14:12.139239+0000","last_peered":"2024-09-10T22:14:12.139239+0000","last_clean":"2024-09-10T22:14:12.139239+0000","last_became_active":"2024-09-10T22:14:11.141064+0000","last_became_peered":"2024-09-10T22:14:11.141064+0000","last_unstale":"2024-09-10T22:14:12.139239+0000","last_undegraded":"2024-09-10T22:14:12.139239+0000","last_fullsized":"2024-09-10T22:14:12.139239+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.713157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"140'2","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150658+0000","last_change":"2024-09-10T22:14:10.133778+0000","last_active":"2024-09-10T22:14:12.150658+0000","last_peered":"2024-09-10T22:14:12.150658+0000","last_clean":"2024-09-10T22:14:12.150658+0000","last_became_active":"2024-09-10T22:14:10.133530+0000","last_became_peered":"2024-09-10T22:14:10.133530+0000","last_unstale":"2024-09-10T22:14:12.150658+0000","last_undegraded":"2024-09-10T22:14:12.150658+0000","last_fullsized":"2024-09-10T22:14:12.150658+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:54.043735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147395+0000","last_change":"2024-09-10T22:14:11.156934+0000","last_active":"2024-09-10T22:14:12.147395+0000","last_peered":"2024-09-10T22:14:12.147395+0000","last_clean":"2024-09-10T22:14:12.147395+0000","last_became_active":"2024-09-10T22:14:11.156701+0000","last_became_peered":"2024-09-10T22:14:11.156701+0000","last_unstale":"2024-09-10T22:14:12.147395+0000","last_undegraded":"2024-09-10T22:14:12.147395+0000","last_fullsized":"2024-09-10T22:14:12.147395+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:35.925135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"140'2","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150929+0000","last_change":"2024-09-10T22:14:10.131170+0000","last_active":"2024-09-10T22:14:12.150929+0000","last_peered":"2024-09-10T22:14:12.150929+0000","last_clean":"2024-09-10T22:14:12.150929+0000","last_became_active":"2024-09-10T22:14:10.130790+0000","last_became_peered":"2024-09-10T22:14:10.130790+0000","last_unstale":"2024-09-10T22:14:12.150929+0000","last_undegraded":"2024-09-10T22:14:12.150929+0000","last_fullsized":"2024-09-10T22:14:12.150929+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:00.709944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145759+0000","last_change":"2024-09-10T22:14:11.151864+0000","last_active":"2024-09-10T22:14:12.145759+0000","last_peered":"2024-09-10T22:14:12.145759+0000","last_clean":"2024-09-10T22:14:12.145759+0000","last_became_active":"2024-09-10T22:14:11.151558+0000","last_became_peered":"2024-09-10T22:14:11.151558+0000","last_unstale":"2024-09-10T22:14:12.145759+0000","last_undegraded":"2024-09-10T22:14:12.145759+0000","last_fullsized":"2024-09-10T22:14:12.145759+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:47.604624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"140'1","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.163647+0000","last_change":"2024-09-10T22:14:10.134766+0000","last_active":"2024-09-10T22:14:11.163647+0000","last_peered":"2024-09-10T22:14:11.163647+0000","last_clean":"2024-09-10T22:14:11.163647+0000","last_became_active":"2024-09-10T22:14:10.134449+0000","last_became_peered":"2024-09-10T22:14:10.134449+0000","last_unstale":"2024-09-10T22:14:11.163647+0000","last_undegraded":"2024-09-10T22:14:11.163647+0000","last_fullsized":"2024-09-10T22:14:11.163647+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:29.038997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150067+0000","last_change":"2024-09-10T22:14:11.156415+0000","last_active":"2024-09-10T22:14:12.150067+0000","last_peered":"2024-09-10T22:14:12.150067+0000","last_clean":"2024-09-10T22:14:12.150067+0000","last_became_active":"2024-09-10T22:14:11.156075+0000","last_became_peered":"2024-09-10T22:14:11.156075+0000","last_unstale":"2024-09-10T22:14:12.150067+0000","last_undegraded":"2024-09-10T22:14:12.150067+0000","last_fullsized":"2024-09-10T22:14:12.150067+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:28.688741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"140'1","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145701+0000","last_change":"2024-09-10T22:14:10.139434+0000","last_active":"2024-09-10T22:14:12.145701+0000","last_peered":"2024-09-10T22:14:12.145701+0000","last_clean":"2024-09-10T22:14:12.145701+0000","last_became_active":"2024-09-10T22:14:10.139128+0000","last_became_peered":"2024-09-10T22:14:10.139128+0000","last_unstale":"2024-09-10T22:14:12.145701+0000","last_undegraded":"2024-09-10T22:14:12.145701+0000","last_fullsized":"2024-09-10T22:14:12.145701+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:17.381369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145588+0000","last_change":"2024-09-10T22:14:11.135507+0000","last_active":"2024-09-10T22:14:12.145588+0000","last_peered":"2024-09-10T22:14:12.145588+0000","last_clean":"2024-09-10T22:14:12.145588+0000","last_became_active":"2024-09-10T22:14:11.135225+0000","last_became_peered":"2024-09-10T22:14:11.135225+0000","last_unstale":"2024-09-10T22:14:12.145588+0000","last_undegraded":"2024-09-10T22:14:12.145588+0000","last_fullsized":"2024-09-10T22:14:12.145588+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:11.250908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.144909+0000","last_change":"2024-09-10T22:14:10.135568+0000","last_active":"2024-09-10T22:14:11.144909+0000","last_peered":"2024-09-10T22:14:11.144909+0000","last_clean":"2024-09-10T22:14:11.144909+0000","last_became_active":"2024-09-10T22:14:10.133987+0000","last_became_peered":"2024-09-10T22:14:10.133987+0000","last_unstale":"2024-09-10T22:14:11.144909+0000","last_undegraded":"2024-09-10T22:14:11.144909+0000","last_fullsized":"2024-09-10T22:14:11.144909+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:39.742924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145084+0000","last_change":"2024-09-10T22:14:11.145109+0000","last_active":"2024-09-10T22:14:12.145084+0000","last_peered":"2024-09-10T22:14:12.145084+0000","last_clean":"2024-09-10T22:14:12.145084+0000","last_became_active":"2024-09-10T22:14:11.144845+0000","last_became_peered":"2024-09-10T22:14:11.144845+0000","last_unstale":"2024-09-10T22:14:12.145084+0000","last_undegraded":"2024-09-10T22:14:12.145084+0000","last_fullsized":"2024-09-10T22:14:12.145084+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:16.145337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149137+0000","last_change":"2024-09-10T22:14:10.133113+0000","last_active":"2024-09-10T22:14:12.149137+0000","last_peered":"2024-09-10T22:14:12.149137+0000","last_clean":"2024-09-10T22:14:12.149137+0000","last_became_active":"2024-09-10T22:14:10.132815+0000","last_became_peered":"2024-09-10T22:14:10.132815+0000","last_unstale":"2024-09-10T22:14:12.149137+0000","last_undegraded":"2024-09-10T22:14:12.149137+0000","last_fullsized":"2024-09-10T22:14:12.149137+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:00.884414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146126+0000","last_change":"2024-09-10T22:14:11.153743+0000","last_active":"2024-09-10T22:14:12.146126+0000","last_peered":"2024-09-10T22:14:12.146126+0000","last_clean":"2024-09-10T22:14:12.146126+0000","last_became_active":"2024-09-10T22:14:11.153204+0000","last_became_peered":"2024-09-10T22:14:11.153204+0000","last_unstale":"2024-09-10T22:14:12.146126+0000","last_undegraded":"2024-09-10T22:14:12.146126+0000","last_fullsized":"2024-09-10T22:14:12.146126+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:50.063184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145988+0000","last_change":"2024-09-10T22:14:10.137055+0000","last_active":"2024-09-10T22:14:12.145988+0000","last_peered":"2024-09-10T22:14:12.145988+0000","last_clean":"2024-09-10T22:14:12.145988+0000","last_became_active":"2024-09-10T22:14:10.136715+0000","last_became_peered":"2024-09-10T22:14:10.136715+0000","last_unstale":"2024-09-10T22:14:12.145988+0000","last_undegraded":"2024-09-10T22:14:12.145988+0000","last_fullsized":"2024-09-10T22:14:12.145988+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:37.770852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147190+0000","last_change":"2024-09-10T22:14:11.165901+0000","last_active":"2024-09-10T22:14:12.147190+0000","last_peered":"2024-09-10T22:14:12.147190+0000","last_clean":"2024-09-10T22:14:12.147190+0000","last_became_active":"2024-09-10T22:14:11.165277+0000","last_became_peered":"2024-09-10T22:14:11.165277+0000","last_unstale":"2024-09-10T22:14:12.147190+0000","last_undegraded":"2024-09-10T22:14:12.147190+0000","last_fullsized":"2024-09-10T22:14:12.147190+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:58.682027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147163+0000","last_change":"2024-09-10T22:14:10.139594+0000","last_active":"2024-09-10T22:14:12.147163+0000","last_peered":"2024-09-10T22:14:12.147163+0000","last_clean":"2024-09-10T22:14:12.147163+0000","last_became_active":"2024-09-10T22:14:10.139324+0000","last_became_peered":"2024-09-10T22:14:10.139324+0000","last_unstale":"2024-09-10T22:14:12.147163+0000","last_undegraded":"2024-09-10T22:14:12.147163+0000","last_fullsized":"2024-09-10T22:14:12.147163+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:04.219100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.158431+0000","last_change":"2024-09-10T22:14:11.154367+0000","last_active":"2024-09-10T22:14:11.158431+0000","last_peered":"2024-09-10T22:14:11.158431+0000","last_clean":"2024-09-10T22:14:11.158431+0000","last_became_active":"2024-09-10T22:14:11.154027+0000","last_became_peered":"2024-09-10T22:14:11.154027+0000","last_unstale":"2024-09-10T22:14:11.158431+0000","last_undegraded":"2024-09-10T22:14:11.158431+0000","last_fullsized":"2024-09-10T22:14:11.158431+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:22.194910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"140'6","reported_seq":21,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.144200+0000","last_change":"2024-09-10T22:14:10.139555+0000","last_active":"2024-09-10T22:14:12.144200+0000","last_peered":"2024-09-10T22:14:12.144200+0000","last_clean":"2024-09-10T22:14:12.144200+0000","last_became_active":"2024-09-10T22:14:10.139172+0000","last_became_peered":"2024-09-10T22:14:10.139172+0000","last_unstale":"2024-09-10T22:14:12.144200+0000","last_undegraded":"2024-09-10T22:14:12.144200+0000","last_fullsized":"2024-09-10T22:14:12.144200+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:19.487900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150889+0000","last_change":"2024-09-10T22:14:11.151806+0000","last_active":"2024-09-10T22:14:12.150889+0000","last_peered":"2024-09-10T22:14:12.150889+0000","last_clean":"2024-09-10T22:14:12.150889+0000","last_became_active":"2024-09-10T22:14:11.151386+0000","last_became_peered":"2024-09-10T22:14:11.151386+0000","last_unstale":"2024-09-10T22:14:12.150889+0000","last_undegraded":"2024-09-10T22:14:12.150889+0000","last_fullsized":"2024-09-10T22:14:12.150889+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:04.332643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"140'2","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150636+0000","last_change":"2024-09-10T22:14:10.132968+0000","last_active":"2024-09-10T22:14:12.150636+0000","last_peered":"2024-09-10T22:14:12.150636+0000","last_clean":"2024-09-10T22:14:12.150636+0000","last_became_active":"2024-09-10T22:14:10.131997+0000","last_became_peered":"2024-09-10T22:14:10.131997+0000","last_unstale":"2024-09-10T22:14:12.150636+0000","last_undegraded":"2024-09-10T22:14:12.150636+0000","last_fullsized":"2024-09-10T22:14:12.150636+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:28.767939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145626+0000","last_change":"2024-09-10T22:14:11.139527+0000","last_active":"2024-09-10T22:14:12.145626+0000","last_peered":"2024-09-10T22:14:12.145626+0000","last_clean":"2024-09-10T22:14:12.145626+0000","last_became_active":"2024-09-10T22:14:11.138083+0000","last_became_peered":"2024-09-10T22:14:11.138083+0000","last_unstale":"2024-09-10T22:14:12.145626+0000","last_undegraded":"2024-09-10T22:14:12.145626+0000","last_fullsized":"2024-09-10T22:14:12.145626+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:54.135329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150789+0000","last_change":"2024-09-10T22:14:10.139618+0000","last_active":"2024-09-10T22:14:12.150789+0000","last_peered":"2024-09-10T22:14:12.150789+0000","last_clean":"2024-09-10T22:14:12.150789+0000","last_became_active":"2024-09-10T22:14:10.138960+0000","last_became_peered":"2024-09-10T22:14:10.138960+0000","last_unstale":"2024-09-10T22:14:12.150789+0000","last_undegraded":"2024-09-10T22:14:12.150789+0000","last_fullsized":"2024-09-10T22:14:12.150789+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:21.315614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145397+0000","last_change":"2024-09-10T22:14:11.154479+0000","last_active":"2024-09-10T22:14:12.145397+0000","last_peered":"2024-09-10T22:14:12.145397+0000","last_clean":"2024-09-10T22:14:12.145397+0000","last_became_active":"2024-09-10T22:14:11.154061+0000","last_became_peered":"2024-09-10T22:14:11.154061+0000","last_unstale":"2024-09-10T22:14:12.145397+0000","last_undegraded":"2024-09-10T22:14:12.145397+0000","last_fullsized":"2024-09-10T22:14:12.145397+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.713157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.138924+0000","last_change":"2024-09-10T22:14:10.126544+0000","last_active":"2024-09-10T22:14:12.138924+0000","last_peered":"2024-09-10T22:14:12.138924+0000","last_clean":"2024-09-10T22:14:12.138924+0000","last_became_active":"2024-09-10T22:14:10.126235+0000","last_became_peered":"2024-09-10T22:14:10.126235+0000","last_unstale":"2024-09-10T22:14:12.138924+0000","last_undegraded":"2024-09-10T22:14:12.138924+0000","last_fullsized":"2024-09-10T22:14:12.138924+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:41.794864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149538+0000","last_change":"2024-09-10T22:14:11.157646+0000","last_active":"2024-09-10T22:14:12.149538+0000","last_peered":"2024-09-10T22:14:12.149538+0000","last_clean":"2024-09-10T22:14:12.149538+0000","last_became_active":"2024-09-10T22:14:11.155287+0000","last_became_peered":"2024-09-10T22:14:11.155287+0000","last_unstale":"2024-09-10T22:14:12.149538+0000","last_undegraded":"2024-09-10T22:14:12.149538+0000","last_fullsized":"2024-09-10T22:14:12.149538+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:01.890920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"140'8","reported_seq":21,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.164736+0000","last_change":"2024-09-10T22:14:10.134829+0000","last_active":"2024-09-10T22:14:11.164736+0000","last_peered":"2024-09-10T22:14:11.164736+0000","last_clean":"2024-09-10T22:14:11.164736+0000","last_became_active":"2024-09-10T22:14:10.134543+0000","last_became_peered":"2024-09-10T22:14:10.134543+0000","last_unstale":"2024-09-10T22:14:11.164736+0000","last_undegraded":"2024-09-10T22:14:11.164736+0000","last_fullsized":"2024-09-10T22:14:11.164736+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:01.838543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146932+0000","last_change":"2024-09-10T22:14:11.164344+0000","last_active":"2024-09-10T22:14:12.146932+0000","last_peered":"2024-09-10T22:14:12.146932+0000","last_clean":"2024-09-10T22:14:12.146932+0000","last_became_active":"2024-09-10T22:14:11.163968+0000","last_became_peered":"2024-09-10T22:14:11.163968+0000","last_unstale":"2024-09-10T22:14:12.146932+0000","last_undegraded":"2024-09-10T22:14:12.146932+0000","last_fullsized":"2024-09-10T22:14:12.146932+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:37.799310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"140'1","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146249+0000","last_change":"2024-09-10T22:14:10.136669+0000","last_active":"2024-09-10T22:14:12.146249+0000","last_peered":"2024-09-10T22:14:12.146249+0000","last_clean":"2024-09-10T22:14:12.146249+0000","last_became_active":"2024-09-10T22:14:10.136374+0000","last_became_peered":"2024-09-10T22:14:10.136374+0000","last_unstale":"2024-09-10T22:14:12.146249+0000","last_undegraded":"2024-09-10T22:14:12.146249+0000","last_fullsized":"2024-09-10T22:14:12.146249+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:35.407875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.158022+0000","last_change":"2024-09-10T22:14:11.155195+0000","last_active":"2024-09-10T22:14:11.158022+0000","last_peered":"2024-09-10T22:14:11.158022+0000","last_clean":"2024-09-10T22:14:11.158022+0000","last_became_active":"2024-09-10T22:14:11.154945+0000","last_became_peered":"2024-09-10T22:14:11.154945+0000","last_unstale":"2024-09-10T22:14:11.158022+0000","last_undegraded":"2024-09-10T22:14:11.158022+0000","last_fullsized":"2024-09-10T22:14:11.158022+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:02.158638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"140'2","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146070+0000","last_change":"2024-09-10T22:14:10.136406+0000","last_active":"2024-09-10T22:14:12.146070+0000","last_peered":"2024-09-10T22:14:12.146070+0000","last_clean":"2024-09-10T22:14:12.146070+0000","last_became_active":"2024-09-10T22:14:10.135227+0000","last_became_peered":"2024-09-10T22:14:10.135227+0000","last_unstale":"2024-09-10T22:14:12.146070+0000","last_undegraded":"2024-09-10T22:14:12.146070+0000","last_fullsized":"2024-09-10T22:14:12.146070+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:30.928200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150280+0000","last_change":"2024-09-10T22:14:11.150099+0000","last_active":"2024-09-10T22:14:12.150280+0000","last_peered":"2024-09-10T22:14:12.150280+0000","last_clean":"2024-09-10T22:14:12.150280+0000","last_became_active":"2024-09-10T22:14:11.149837+0000","last_became_peered":"2024-09-10T22:14:11.149837+0000","last_unstale":"2024-09-10T22:14:12.150280+0000","last_undegraded":"2024-09-10T22:14:12.150280+0000","last_fullsized":"2024-09-10T22:14:12.150280+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:26.272238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"140'1","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.163295+0000","last_change":"2024-09-10T22:14:10.133929+0000","last_active":"2024-09-10T22:14:11.163295+0000","last_peered":"2024-09-10T22:14:11.163295+0000","last_clean":"2024-09-10T22:14:11.163295+0000","last_became_active":"2024-09-10T22:14:10.133326+0000","last_became_peered":"2024-09-10T22:14:10.133326+0000","last_unstale":"2024-09-10T22:14:11.163295+0000","last_undegraded":"2024-09-10T22:14:11.163295+0000","last_fullsized":"2024-09-10T22:14:11.163295+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:25.987026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139369+0000","last_change":"2024-09-10T22:14:11.151134+0000","last_active":"2024-09-10T22:14:12.139369+0000","last_peered":"2024-09-10T22:14:12.139369+0000","last_clean":"2024-09-10T22:14:12.139369+0000","last_became_active":"2024-09-10T22:14:11.150863+0000","last_became_peered":"2024-09-10T22:14:11.150863+0000","last_unstale":"2024-09-10T22:14:12.139369+0000","last_undegraded":"2024-09-10T22:14:12.139369+0000","last_fullsized":"2024-09-10T22:14:12.139369+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:27:54.416428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146174+0000","last_change":"2024-09-10T22:14:10.135580+0000","last_active":"2024-09-10T22:14:12.146174+0000","last_peered":"2024-09-10T22:14:12.146174+0000","last_clean":"2024-09-10T22:14:12.146174+0000","last_became_active":"2024-09-10T22:14:10.134413+0000","last_became_peered":"2024-09-10T22:14:10.134413+0000","last_unstale":"2024-09-10T22:14:12.146174+0000","last_undegraded":"2024-09-10T22:14:12.146174+0000","last_fullsized":"2024-09-10T22:14:12.146174+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:06.712240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149967+0000","last_change":"2024-09-10T22:14:11.144790+0000","last_active":"2024-09-10T22:14:12.149967+0000","last_peered":"2024-09-10T22:14:12.149967+0000","last_clean":"2024-09-10T22:14:12.149967+0000","last_became_active":"2024-09-10T22:14:11.143920+0000","last_became_peered":"2024-09-10T22:14:11.143920+0000","last_unstale":"2024-09-10T22:14:12.149967+0000","last_undegraded":"2024-09-10T22:14:12.149967+0000","last_fullsized":"2024-09-10T22:14:12.149967+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:38.777357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"140'2","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.143817+0000","last_change":"2024-09-10T22:14:10.131729+0000","last_active":"2024-09-10T22:14:12.143817+0000","last_peered":"2024-09-10T22:14:12.143817+0000","last_clean":"2024-09-10T22:14:12.143817+0000","last_became_active":"2024-09-10T22:14:10.129611+0000","last_became_peered":"2024-09-10T22:14:10.129611+0000","last_unstale":"2024-09-10T22:14:12.143817+0000","last_undegraded":"2024-09-10T22:14:12.143817+0000","last_fullsized":"2024-09-10T22:14:12.143817+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:11:36.178767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.143885+0000","last_change":"2024-09-10T22:14:11.142721+0000","last_active":"2024-09-10T22:14:12.143885+0000","last_peered":"2024-09-10T22:14:12.143885+0000","last_clean":"2024-09-10T22:14:12.143885+0000","last_became_active":"2024-09-10T22:14:11.142410+0000","last_became_peered":"2024-09-10T22:14:11.142410+0000","last_unstale":"2024-09-10T22:14:12.143885+0000","last_undegraded":"2024-09-10T22:14:12.143885+0000","last_fullsized":"2024-09-10T22:14:12.143885+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:48.174482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147632+0000","last_change":"2024-09-10T22:14:10.142440+0000","last_active":"2024-09-10T22:14:12.147632+0000","last_peered":"2024-09-10T22:14:12.147632+0000","last_clean":"2024-09-10T22:14:12.147632+0000","last_became_active":"2024-09-10T22:14:10.142117+0000","last_became_peered":"2024-09-10T22:14:10.142117+0000","last_unstale":"2024-09-10T22:14:12.147632+0000","last_undegraded":"2024-09-10T22:14:12.147632+0000","last_fullsized":"2024-09-10T22:14:12.147632+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:15.100421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149220+0000","last_change":"2024-09-10T22:14:11.143555+0000","last_active":"2024-09-10T22:14:12.149220+0000","last_peered":"2024-09-10T22:14:12.149220+0000","last_clean":"2024-09-10T22:14:12.149220+0000","last_became_active":"2024-09-10T22:14:11.143273+0000","last_became_peered":"2024-09-10T22:14:11.143273+0000","last_unstale":"2024-09-10T22:14:12.149220+0000","last_undegraded":"2024-09-10T22:14:12.149220+0000","last_fullsized":"2024-09-10T22:14:12.149220+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:05.768167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146753+0000","last_change":"2024-09-10T22:14:10.142483+0000","last_active":"2024-09-10T22:14:12.146753+0000","last_peered":"2024-09-10T22:14:12.146753+0000","last_clean":"2024-09-10T22:14:12.146753+0000","last_became_active":"2024-09-10T22:14:10.141397+0000","last_became_peered":"2024-09-10T22:14:10.141397+0000","last_unstale":"2024-09-10T22:14:12.146753+0000","last_undegraded":"2024-09-10T22:14:12.146753+0000","last_fullsized":"2024-09-10T22:14:12.146753+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:46.598119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.143474+0000","last_change":"2024-09-10T22:14:11.142115+0000","last_active":"2024-09-10T22:14:12.143474+0000","last_peered":"2024-09-10T22:14:12.143474+0000","last_clean":"2024-09-10T22:14:12.143474+0000","last_became_active":"2024-09-10T22:14:11.141739+0000","last_became_peered":"2024-09-10T22:14:11.141739+0000","last_unstale":"2024-09-10T22:14:12.143474+0000","last_undegraded":"2024-09-10T22:14:12.143474+0000","last_fullsized":"2024-09-10T22:14:12.143474+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:33.081442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149838+0000","last_change":"2024-09-10T22:14:10.143014+0000","last_active":"2024-09-10T22:14:12.149838+0000","last_peered":"2024-09-10T22:14:12.149838+0000","last_clean":"2024-09-10T22:14:12.149838+0000","last_became_active":"2024-09-10T22:14:10.142701+0000","last_became_peered":"2024-09-10T22:14:10.142701+0000","last_unstale":"2024-09-10T22:14:12.149838+0000","last_undegraded":"2024-09-10T22:14:12.149838+0000","last_fullsized":"2024-09-10T22:14:12.149838+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:02.251766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146360+0000","last_change":"2024-09-10T22:14:11.153458+0000","last_active":"2024-09-10T22:14:12.146360+0000","last_peered":"2024-09-10T22:14:12.146360+0000","last_clean":"2024-09-10T22:14:12.146360+0000","last_became_active":"2024-09-10T22:14:11.152885+0000","last_became_peered":"2024-09-10T22:14:11.152885+0000","last_unstale":"2024-09-10T22:14:12.146360+0000","last_undegraded":"2024-09-10T22:14:12.146360+0000","last_fullsized":"2024-09-10T22:14:12.146360+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:10.283455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"140'2","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147639+0000","last_change":"2024-09-10T22:14:10.144525+0000","last_active":"2024-09-10T22:14:12.147639+0000","last_peered":"2024-09-10T22:14:12.147639+0000","last_clean":"2024-09-10T22:14:12.147639+0000","last_became_active":"2024-09-10T22:14:10.144290+0000","last_became_peered":"2024-09-10T22:14:10.144290+0000","last_unstale":"2024-09-10T22:14:12.147639+0000","last_undegraded":"2024-09-10T22:14:12.147639+0000","last_fullsized":"2024-09-10T22:14:12.147639+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:12.902328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150533+0000","last_change":"2024-09-10T22:14:11.153510+0000","last_active":"2024-09-10T22:14:12.150533+0000","last_peered":"2024-09-10T22:14:12.150533+0000","last_clean":"2024-09-10T22:14:12.150533+0000","last_became_active":"2024-09-10T22:14:11.152647+0000","last_became_peered":"2024-09-10T22:14:11.152647+0000","last_unstale":"2024-09-10T22:14:12.150533+0000","last_undegraded":"2024-09-10T22:14:12.150533+0000","last_fullsized":"2024-09-10T22:14:12.150533+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:02.793500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"141'1","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.151132+0000","last_change":"2024-09-10T22:14:10.136755+0000","last_active":"2024-09-10T22:14:12.151132+0000","last_peered":"2024-09-10T22:14:12.151132+0000","last_clean":"2024-09-10T22:14:12.151132+0000","last_became_active":"2024-09-10T22:14:10.136454+0000","last_became_peered":"2024-09-10T22:14:10.136454+0000","last_unstale":"2024-09-10T22:14:12.151132+0000","last_undegraded":"2024-09-10T22:14:12.151132+0000","last_fullsized":"2024-09-10T22:14:12.151132+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:57.332982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145921+0000","last_change":"2024-09-10T22:14:11.135153+0000","last_active":"2024-09-10T22:14:12.145921+0000","last_peered":"2024-09-10T22:14:12.145921+0000","last_clean":"2024-09-10T22:14:12.145921+0000","last_became_active":"2024-09-10T22:14:11.134856+0000","last_became_peered":"2024-09-10T22:14:11.134856+0000","last_unstale":"2024-09-10T22:14:12.145921+0000","last_undegraded":"2024-09-10T22:14:12.145921+0000","last_fullsized":"2024-09-10T22:14:12.145921+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:43.862488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139620+0000","last_change":"2024-09-10T22:14:10.138413+0000","last_active":"2024-09-10T22:14:12.139620+0000","last_peered":"2024-09-10T22:14:12.139620+0000","last_clean":"2024-09-10T22:14:12.139620+0000","last_became_active":"2024-09-10T22:14:10.138155+0000","last_became_peered":"2024-09-10T22:14:10.138155+0000","last_unstale":"2024-09-10T22:14:12.139620+0000","last_undegraded":"2024-09-10T22:14:12.139620+0000","last_fullsized":"2024-09-10T22:14:12.139620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:21.650197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139514+0000","last_change":"2024-09-10T22:14:11.139594+0000","last_active":"2024-09-10T22:14:12.139514+0000","last_peered":"2024-09-10T22:14:12.139514+0000","last_clean":"2024-09-10T22:14:12.139514+0000","last_became_active":"2024-09-10T22:14:11.139309+0000","last_became_peered":"2024-09-10T22:14:11.139309+0000","last_unstale":"2024-09-10T22:14:12.139514+0000","last_undegraded":"2024-09-10T22:14:12.139514+0000","last_fullsized":"2024-09-10T22:14:12.139514+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:53.147936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139139+0000","last_change":"2024-09-10T22:14:10.139717+0000","last_active":"2024-09-10T22:14:12.139139+0000","last_peered":"2024-09-10T22:14:12.139139+0000","last_clean":"2024-09-10T22:14:12.139139+0000","last_became_active":"2024-09-10T22:14:10.139455+0000","last_became_peered":"2024-09-10T22:14:10.139455+0000","last_unstale":"2024-09-10T22:14:12.139139+0000","last_undegraded":"2024-09-10T22:14:12.139139+0000","last_fullsized":"2024-09-10T22:14:12.139139+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:42.306792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.152649+0000","last_change":"2024-09-10T22:14:11.147393+0000","last_active":"2024-09-10T22:14:11.152649+0000","last_peered":"2024-09-10T22:14:11.152649+0000","last_clean":"2024-09-10T22:14:11.152649+0000","last_became_active":"2024-09-10T22:14:11.146195+0000","last_became_peered":"2024-09-10T22:14:11.146195+0000","last_unstale":"2024-09-10T22:14:11.152649+0000","last_undegraded":"2024-09-10T22:14:11.152649+0000","last_fullsized":"2024-09-10T22:14:11.152649+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:04.264645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.143909+0000","last_change":"2024-09-10T22:14:10.127456+0000","last_active":"2024-09-10T22:14:12.143909+0000","last_peered":"2024-09-10T22:14:12.143909+0000","last_clean":"2024-09-10T22:14:12.143909+0000","last_became_active":"2024-09-10T22:14:10.127211+0000","last_became_peered":"2024-09-10T22:14:10.127211+0000","last_unstale":"2024-09-10T22:14:12.143909+0000","last_undegraded":"2024-09-10T22:14:12.143909+0000","last_fullsized":"2024-09-10T22:14:12.143909+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:08.076582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145735+0000","last_change":"2024-09-10T22:14:11.160004+0000","last_active":"2024-09-10T22:14:12.145735+0000","last_peered":"2024-09-10T22:14:12.145735+0000","last_clean":"2024-09-10T22:14:12.145735+0000","last_became_active":"2024-09-10T22:14:11.159452+0000","last_became_peered":"2024-09-10T22:14:11.159452+0000","last_unstale":"2024-09-10T22:14:12.145735+0000","last_undegraded":"2024-09-10T22:14:12.145735+0000","last_fullsized":"2024-09-10T22:14:12.145735+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:30.086459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147553+0000","last_change":"2024-09-10T22:14:10.141753+0000","last_active":"2024-09-10T22:14:12.147553+0000","last_peered":"2024-09-10T22:14:12.147553+0000","last_clean":"2024-09-10T22:14:12.147553+0000","last_became_active":"2024-09-10T22:14:10.141066+0000","last_became_peered":"2024-09-10T22:14:10.141066+0000","last_unstale":"2024-09-10T22:14:12.147553+0000","last_undegraded":"2024-09-10T22:14:12.147553+0000","last_fullsized":"2024-09-10T22:14:12.147553+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:42.855982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146744+0000","last_change":"2024-09-10T22:14:11.155949+0000","last_active":"2024-09-10T22:14:12.146744+0000","last_peered":"2024-09-10T22:14:12.146744+0000","last_clean":"2024-09-10T22:14:12.146744+0000","last_became_active":"2024-09-10T22:14:11.155695+0000","last_became_peered":"2024-09-10T22:14:11.155695+0000","last_unstale":"2024-09-10T22:14:12.146744+0000","last_undegraded":"2024-09-10T22:14:12.146744+0000","last_fullsized":"2024-09-10T22:14:12.146744+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:19.732547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149959+0000","last_change":"2024-09-10T22:14:10.133689+0000","last_active":"2024-09-10T22:14:12.149959+0000","last_peered":"2024-09-10T22:14:12.149959+0000","last_clean":"2024-09-10T22:14:12.149959+0000","last_became_active":"2024-09-10T22:14:10.133389+0000","last_became_peered":"2024-09-10T22:14:10.133389+0000","last_unstale":"2024-09-10T22:14:12.149959+0000","last_undegraded":"2024-09-10T22:14:12.149959+0000","last_fullsized":"2024-09-10T22:14:12.149959+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:32.903462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145220+0000","last_change":"2024-09-10T22:14:11.153696+0000","last_active":"2024-09-10T22:14:12.145220+0000","last_peered":"2024-09-10T22:14:12.145220+0000","last_clean":"2024-09-10T22:14:12.145220+0000","last_became_active":"2024-09-10T22:14:11.153424+0000","last_became_peered":"2024-09-10T22:14:11.153424+0000","last_unstale":"2024-09-10T22:14:12.145220+0000","last_undegraded":"2024-09-10T22:14:12.145220+0000","last_fullsized":"2024-09-10T22:14:12.145220+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:15.312565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145527+0000","last_change":"2024-09-10T22:14:10.135791+0000","last_active":"2024-09-10T22:14:12.145527+0000","last_peered":"2024-09-10T22:14:12.145527+0000","last_clean":"2024-09-10T22:14:12.145527+0000","last_became_active":"2024-09-10T22:14:10.135446+0000","last_became_peered":"2024-09-10T22:14:10.135446+0000","last_unstale":"2024-09-10T22:14:12.145527+0000","last_undegraded":"2024-09-10T22:14:12.145527+0000","last_fullsized":"2024-09-10T22:14:12.145527+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:54.013189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.144166+0000","last_change":"2024-09-10T22:14:11.145265+0000","last_active":"2024-09-10T22:14:12.144166+0000","last_peered":"2024-09-10T22:14:12.144166+0000","last_clean":"2024-09-10T22:14:12.144166+0000","last_became_active":"2024-09-10T22:14:11.144952+0000","last_became_peered":"2024-09-10T22:14:11.144952+0000","last_unstale":"2024-09-10T22:14:12.144166+0000","last_undegraded":"2024-09-10T22:14:12.144166+0000","last_fullsized":"2024-09-10T22:14:12.144166+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:49.348318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"140'3","reported_seq":18,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150501+0000","last_change":"2024-09-10T22:14:10.142497+0000","last_active":"2024-09-10T22:14:12.150501+0000","last_peered":"2024-09-10T22:14:12.150501+0000","last_clean":"2024-09-10T22:14:12.150501+0000","last_became_active":"2024-09-10T22:14:10.142198+0000","last_became_peered":"2024-09-10T22:14:10.142198+0000","last_unstale":"2024-09-10T22:14:12.150501+0000","last_undegraded":"2024-09-10T22:14:12.150501+0000","last_fullsized":"2024-09-10T22:14:12.150501+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:33.805508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":339,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146826+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:14:12.146826+0000","last_peered":"2024-09-10T22:14:12.146826+0000","last_clean":"2024-09-10T22:14:12.146826+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:14:12.146826+0000","last_undegraded":"2024-09-10T22:14:12.146826+0000","last_fullsized":"2024-09-10T22:14:12.146826+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509609,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136136,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235704,"statfs":{"total":96636764160,"available":96497360896,"internally_reserved":0,"allocated":2347008,"data_stored":1359195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":137034186},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509610,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145288,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226552,"statfs":{"total":96636764160,"available":96487989248,"internally_reserved":0,"allocated":1757184,"data_stored":768321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509609,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109184,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262656,"statfs":{"total":96636764160,"available":96524959744,"internally_reserved":0,"allocated":1748992,"data_stored":768197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509610,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136964,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234876,"statfs":{"total":96636764160,"available":96496513024,"internally_reserved":0,"allocated":1744896,"data_stored":768175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509608,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136980,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234860,"statfs":{"total":96636764160,"available":96496496640,"internally_reserved":0,"allocated":1761280,"data_stored":774705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":138475975},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509609,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141120,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230720,"statfs":{"total":96636764160,"available":96492257280,"internally_reserved":0,"allocated":1748992,"data_stored":768757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509609,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131988,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239852,"statfs":{"total":96636764160,"available":96501608448,"internally_reserved":0,"allocated":2355200,"data_stored":1360189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":132774345},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542314,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136944,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234896,"statfs":{"total":96636764160,"available":96496533504,"internally_reserved":0,"allocated":1732608,"data_stored":761260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:14:15.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:14:15.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.479+0000 7fc338a00640 1 -- 172.21.15.66:0/2713393099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc334160370 msgr2=0x7fc3340cfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.479+0000 7fc338a00640 1 --2- 172.21.15.66:0/2713393099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc334160370 0x7fc3340cfe80 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fc33406c4f0 tx=0x7fc32001c830 comp rx=0 tx=0).stop 2024-09-10T22:14:15.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.479+0000 7fc338a00640 1 -- 172.21.15.66:0/2713393099 shutdown_connections 2024-09-10T22:14:15.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.479+0000 7fc338a00640 1 --2- 172.21.15.66:0/2713393099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc334160370 0x7fc3340cfe80 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.479+0000 7fc338a00640 1 --2- 172.21.15.66:0/2713393099 >> v2:172.21.15.94:3300/0 conn(0x7fc334156430 0x7fc33406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.479+0000 7fc338a00640 1 -- 172.21.15.66:0/2713393099 >> 172.21.15.66:0/2713393099 conn(0x7fc3340d0b30 msgr2=0x7fc3340d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:15.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.479+0000 7fc338a00640 1 -- 172.21.15.66:0/2713393099 shutdown_connections 2024-09-10T22:14:15.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.479+0000 7fc338a00640 1 -- 172.21.15.66:0/2713393099 wait complete. 2024-09-10T22:14:15.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.479+0000 7fc338a00640 1 Processor -- start 2024-09-10T22:14:15.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.479+0000 7fc338a00640 1 -- start start 2024-09-10T22:14:15.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc338a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc334070b20 0x7fc33414d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:15.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc338a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc334156430 0x7fc33414b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:15.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc338a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc334160370 0x7fc33414da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:15.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc338a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc334168d10 con 0x7fc334160370 2024-09-10T22:14:15.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc338a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc334168b90 con 0x7fc334156430 2024-09-10T22:14:15.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc338a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc334168e90 con 0x7fc334070b20 2024-09-10T22:14:15.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc332800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc334160370 0x7fc33414da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:15.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc332800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc334160370 0x7fc33414da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53018/0 (socket says 172.21.15.66:53018) 2024-09-10T22:14:15.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc332800640 1 -- 172.21.15.66:0/645241640 learned_addr learned my addr 172.21.15.66:0/645241640 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:15.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc331400640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc334156430 0x7fc33414b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:15.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc331e00640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc334070b20 0x7fc33414d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:15.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc332800640 1 -- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc334070b20 msgr2=0x7fc33414d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc332800640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc334070b20 0x7fc33414d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc332800640 1 -- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc334156430 msgr2=0x7fc33414b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc332800640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc334156430 0x7fc33414b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc332800640 1 -- 172.21.15.66:0/645241640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33415f590 con 0x7fc334160370 2024-09-10T22:14:15.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc331e00640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc334070b20 0x7fc33414d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:14:15.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc332800640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc334160370 0x7fc33414da40 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00c7b0 tx=0x7fc32c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:15.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc32a000640 1 -- 172.21.15.66:0/645241640 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c004b70 con 0x7fc334160370 2024-09-10T22:14:15.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc32a000640 1 -- 172.21.15.66:0/645241640 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc32c004e00 con 0x7fc334160370 2024-09-10T22:14:15.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc32a000640 1 -- 172.21.15.66:0/645241640 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c00aab0 con 0x7fc334160370 2024-09-10T22:14:15.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc338a00640 1 -- 172.21.15.66:0/645241640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33414c2b0 con 0x7fc334160370 2024-09-10T22:14:15.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc338a00640 1 -- 172.21.15.66:0/645241640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc33414c870 con 0x7fc334160370 2024-09-10T22:14:15.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.483+0000 7fc338a00640 1 -- 172.21.15.66:0/645241640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc33406d630 con 0x7fc334160370 2024-09-10T22:14:15.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.487+0000 7fc32a000640 1 -- 172.21.15.66:0/645241640 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fc32c00ac50 con 0x7fc334160370 2024-09-10T22:14:15.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.487+0000 7fc32a000640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc314074710 0x7fc314076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:15.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.487+0000 7fc32a000640 1 -- 172.21.15.66:0/645241640 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5878+0+0 (secure 0 0 0) 0x7fc32c092500 con 0x7fc334160370 2024-09-10T22:14:15.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.487+0000 7fc331e00640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc314074710 0x7fc314076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:15.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.487+0000 7fc331e00640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc314074710 0x7fc314076bd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc3240058d0 tx=0x7fc324005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:15.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.491+0000 7fc32a000640 1 -- 172.21.15.66:0/645241640 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc32c020070 con 0x7fc334160370 2024-09-10T22:14:15.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.667+0000 7fc338a00640 1 -- 172.21.15.66:0/645241640 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc33415b690 con 0x7fc314074710 2024-09-10T22:14:15.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.671+0000 7fc32a000640 1 -- 172.21.15.66:0/645241640 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252502 (secure 0 0 0) 0x7fc33415b690 con 0x7fc314074710 2024-09-10T22:14:15.678 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:14:15.680 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:14:15.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.675+0000 7fc338a00640 1 -- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc314074710 msgr2=0x7fc314076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.675+0000 7fc338a00640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc314074710 0x7fc314076bd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc3240058d0 tx=0x7fc324005860 comp rx=0 tx=0).stop 2024-09-10T22:14:15.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.675+0000 7fc338a00640 1 -- 172.21.15.66:0/645241640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc334160370 msgr2=0x7fc33414da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.675+0000 7fc338a00640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc334160370 0x7fc33414da40 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00c7b0 tx=0x7fc32c00cc70 comp rx=0 tx=0).stop 2024-09-10T22:14:15.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.675+0000 7fc338a00640 1 -- 172.21.15.66:0/645241640 shutdown_connections 2024-09-10T22:14:15.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.675+0000 7fc338a00640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc314074710 0x7fc314076bd0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.675+0000 7fc338a00640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc334160370 0x7fc33414da40 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.675+0000 7fc338a00640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc334156430 0x7fc33414b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.675+0000 7fc338a00640 1 --2- 172.21.15.66:0/645241640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc334070b20 0x7fc33414d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.675+0000 7fc338a00640 1 -- 172.21.15.66:0/645241640 >> 172.21.15.66:0/645241640 conn(0x7fc3340d0b30 msgr2=0x7fc3340740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:15.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.675+0000 7fc338a00640 1 -- 172.21.15.66:0/645241640 shutdown_connections 2024-09-10T22:14:15.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.675+0000 7fc338a00640 1 -- 172.21.15.66:0/645241640 wait complete. 2024-09-10T22:14:15.699 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":552,"stamp":"2024-09-10T22:14:15.169154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1074604,"kb_used_data":14840,"kb_used_omap":12,"kb_used_meta":1059571,"kb_avail":753900116,"statfs":{"total":773094113280,"available":771993718784,"internally_reserved":0,"allocated":15196160,"data_stored":7328799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1085001268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.050652"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139619+0000","last_change":"2024-09-10T22:14:11.156782+0000","last_active":"2024-09-10T22:14:12.139619+0000","last_peered":"2024-09-10T22:14:12.139619+0000","last_clean":"2024-09-10T22:14:12.139619+0000","last_became_active":"2024-09-10T22:14:11.156441+0000","last_became_peered":"2024-09-10T22:14:11.156441+0000","last_unstale":"2024-09-10T22:14:12.139619+0000","last_undegraded":"2024-09-10T22:14:12.139619+0000","last_fullsized":"2024-09-10T22:14:12.139619+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:39.168936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145834+0000","last_change":"2024-09-10T22:14:11.140653+0000","last_active":"2024-09-10T22:14:12.145834+0000","last_peered":"2024-09-10T22:14:12.145834+0000","last_clean":"2024-09-10T22:14:12.145834+0000","last_became_active":"2024-09-10T22:14:11.140338+0000","last_became_peered":"2024-09-10T22:14:11.140338+0000","last_unstale":"2024-09-10T22:14:12.145834+0000","last_undegraded":"2024-09-10T22:14:12.145834+0000","last_fullsized":"2024-09-10T22:14:12.145834+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:16.106926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150740+0000","last_change":"2024-09-10T22:14:11.154656+0000","last_active":"2024-09-10T22:14:12.150740+0000","last_peered":"2024-09-10T22:14:12.150740+0000","last_clean":"2024-09-10T22:14:12.150740+0000","last_became_active":"2024-09-10T22:14:11.154387+0000","last_became_peered":"2024-09-10T22:14:11.154387+0000","last_unstale":"2024-09-10T22:14:12.150740+0000","last_undegraded":"2024-09-10T22:14:12.150740+0000","last_fullsized":"2024-09-10T22:14:12.150740+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:17.167373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150908+0000","last_change":"2024-09-10T22:14:11.156234+0000","last_active":"2024-09-10T22:14:12.150908+0000","last_peered":"2024-09-10T22:14:12.150908+0000","last_clean":"2024-09-10T22:14:12.150908+0000","last_became_active":"2024-09-10T22:14:11.155964+0000","last_became_peered":"2024-09-10T22:14:11.155964+0000","last_unstale":"2024-09-10T22:14:12.150908+0000","last_undegraded":"2024-09-10T22:14:12.150908+0000","last_fullsized":"2024-09-10T22:14:12.150908+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:46.284534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145278+0000","last_change":"2024-09-10T22:14:11.155027+0000","last_active":"2024-09-10T22:14:12.145278+0000","last_peered":"2024-09-10T22:14:12.145278+0000","last_clean":"2024-09-10T22:14:12.145278+0000","last_became_active":"2024-09-10T22:14:11.154497+0000","last_became_peered":"2024-09-10T22:14:11.154497+0000","last_unstale":"2024-09-10T22:14:12.145278+0000","last_undegraded":"2024-09-10T22:14:12.145278+0000","last_fullsized":"2024-09-10T22:14:12.145278+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:12.933247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139113+0000","last_change":"2024-09-10T22:14:11.152457+0000","last_active":"2024-09-10T22:14:12.139113+0000","last_peered":"2024-09-10T22:14:12.139113+0000","last_clean":"2024-09-10T22:14:12.139113+0000","last_became_active":"2024-09-10T22:14:11.152149+0000","last_became_peered":"2024-09-10T22:14:11.152149+0000","last_unstale":"2024-09-10T22:14:12.139113+0000","last_undegraded":"2024-09-10T22:14:12.139113+0000","last_fullsized":"2024-09-10T22:14:12.139113+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:12.933247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.143804+0000","last_change":"2024-09-10T22:14:11.144548+0000","last_active":"2024-09-10T22:14:12.143804+0000","last_peered":"2024-09-10T22:14:12.143804+0000","last_clean":"2024-09-10T22:14:12.143804+0000","last_became_active":"2024-09-10T22:14:11.144290+0000","last_became_peered":"2024-09-10T22:14:11.144290+0000","last_unstale":"2024-09-10T22:14:12.143804+0000","last_undegraded":"2024-09-10T22:14:12.143804+0000","last_fullsized":"2024-09-10T22:14:12.143804+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:32.440842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.138947+0000","last_change":"2024-09-10T22:14:11.140584+0000","last_active":"2024-09-10T22:14:12.138947+0000","last_peered":"2024-09-10T22:14:12.138947+0000","last_clean":"2024-09-10T22:14:12.138947+0000","last_became_active":"2024-09-10T22:14:11.140218+0000","last_became_peered":"2024-09-10T22:14:11.140218+0000","last_unstale":"2024-09-10T22:14:12.138947+0000","last_undegraded":"2024-09-10T22:14:12.138947+0000","last_fullsized":"2024-09-10T22:14:12.138947+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:16.145337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145789+0000","last_change":"2024-09-10T22:14:11.137821+0000","last_active":"2024-09-10T22:14:12.145789+0000","last_peered":"2024-09-10T22:14:12.145789+0000","last_clean":"2024-09-10T22:14:12.145789+0000","last_became_active":"2024-09-10T22:14:11.137555+0000","last_became_peered":"2024-09-10T22:14:11.137555+0000","last_unstale":"2024-09-10T22:14:12.145789+0000","last_undegraded":"2024-09-10T22:14:12.145789+0000","last_fullsized":"2024-09-10T22:14:12.145789+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:16.041501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146672+0000","last_change":"2024-09-10T22:14:11.160296+0000","last_active":"2024-09-10T22:14:12.146672+0000","last_peered":"2024-09-10T22:14:12.146672+0000","last_clean":"2024-09-10T22:14:12.146672+0000","last_became_active":"2024-09-10T22:14:11.158717+0000","last_became_peered":"2024-09-10T22:14:11.158717+0000","last_unstale":"2024-09-10T22:14:12.146672+0000","last_undegraded":"2024-09-10T22:14:12.146672+0000","last_fullsized":"2024-09-10T22:14:12.146672+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:49.348318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139298+0000","last_change":"2024-09-10T22:14:11.145655+0000","last_active":"2024-09-10T22:14:12.139298+0000","last_peered":"2024-09-10T22:14:12.139298+0000","last_clean":"2024-09-10T22:14:12.139298+0000","last_became_active":"2024-09-10T22:14:11.140070+0000","last_became_peered":"2024-09-10T22:14:11.140070+0000","last_unstale":"2024-09-10T22:14:12.139298+0000","last_undegraded":"2024-09-10T22:14:12.139298+0000","last_fullsized":"2024-09-10T22:14:12.139298+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:50.037416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147807+0000","last_change":"2024-09-10T22:14:11.160833+0000","last_active":"2024-09-10T22:14:12.147807+0000","last_peered":"2024-09-10T22:14:12.147807+0000","last_clean":"2024-09-10T22:14:12.147807+0000","last_became_active":"2024-09-10T22:14:11.159759+0000","last_became_peered":"2024-09-10T22:14:11.159759+0000","last_unstale":"2024-09-10T22:14:12.147807+0000","last_undegraded":"2024-09-10T22:14:12.147807+0000","last_fullsized":"2024-09-10T22:14:12.147807+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:44.861904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145498+0000","last_change":"2024-09-10T22:14:11.154755+0000","last_active":"2024-09-10T22:14:12.145498+0000","last_peered":"2024-09-10T22:14:12.145498+0000","last_clean":"2024-09-10T22:14:12.145498+0000","last_became_active":"2024-09-10T22:14:11.154155+0000","last_became_peered":"2024-09-10T22:14:11.154155+0000","last_unstale":"2024-09-10T22:14:12.145498+0000","last_undegraded":"2024-09-10T22:14:12.145498+0000","last_fullsized":"2024-09-10T22:14:12.145498+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:50.037416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149505+0000","last_change":"2024-09-10T22:14:11.151653+0000","last_active":"2024-09-10T22:14:12.149505+0000","last_peered":"2024-09-10T22:14:12.149505+0000","last_clean":"2024-09-10T22:14:12.149505+0000","last_became_active":"2024-09-10T22:14:11.151318+0000","last_became_peered":"2024-09-10T22:14:11.151318+0000","last_unstale":"2024-09-10T22:14:12.149505+0000","last_undegraded":"2024-09-10T22:14:12.149505+0000","last_fullsized":"2024-09-10T22:14:12.149505+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:07.718745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146620+0000","last_change":"2024-09-10T22:14:11.154969+0000","last_active":"2024-09-10T22:14:12.146620+0000","last_peered":"2024-09-10T22:14:12.146620+0000","last_clean":"2024-09-10T22:14:12.146620+0000","last_became_active":"2024-09-10T22:14:11.154005+0000","last_became_peered":"2024-09-10T22:14:11.154005+0000","last_unstale":"2024-09-10T22:14:12.146620+0000","last_undegraded":"2024-09-10T22:14:12.146620+0000","last_fullsized":"2024-09-10T22:14:12.146620+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:08.016417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.156480+0000","last_change":"2024-09-10T22:14:11.151532+0000","last_active":"2024-09-10T22:14:11.156480+0000","last_peered":"2024-09-10T22:14:11.156480+0000","last_clean":"2024-09-10T22:14:11.156480+0000","last_became_active":"2024-09-10T22:14:11.151250+0000","last_became_peered":"2024-09-10T22:14:11.151250+0000","last_unstale":"2024-09-10T22:14:11.156480+0000","last_undegraded":"2024-09-10T22:14:11.156480+0000","last_fullsized":"2024-09-10T22:14:11.156480+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:04.487186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147564+0000","last_change":"2024-09-10T22:14:11.166179+0000","last_active":"2024-09-10T22:14:12.147564+0000","last_peered":"2024-09-10T22:14:12.147564+0000","last_clean":"2024-09-10T22:14:12.147564+0000","last_became_active":"2024-09-10T22:14:11.165644+0000","last_became_peered":"2024-09-10T22:14:11.165644+0000","last_unstale":"2024-09-10T22:14:12.147564+0000","last_undegraded":"2024-09-10T22:14:12.147564+0000","last_fullsized":"2024-09-10T22:14:12.147564+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:16.411855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150379+0000","last_change":"2024-09-10T22:14:11.152176+0000","last_active":"2024-09-10T22:14:12.150379+0000","last_peered":"2024-09-10T22:14:12.150379+0000","last_clean":"2024-09-10T22:14:12.150379+0000","last_became_active":"2024-09-10T22:14:11.151499+0000","last_became_peered":"2024-09-10T22:14:11.151499+0000","last_unstale":"2024-09-10T22:14:12.150379+0000","last_undegraded":"2024-09-10T22:14:12.150379+0000","last_fullsized":"2024-09-10T22:14:12.150379+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:52.932120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139590+0000","last_change":"2024-09-10T22:14:11.150200+0000","last_active":"2024-09-10T22:14:12.139590+0000","last_peered":"2024-09-10T22:14:12.139590+0000","last_clean":"2024-09-10T22:14:12.139590+0000","last_became_active":"2024-09-10T22:14:11.149920+0000","last_became_peered":"2024-09-10T22:14:11.149920+0000","last_unstale":"2024-09-10T22:14:12.139590+0000","last_undegraded":"2024-09-10T22:14:12.139590+0000","last_fullsized":"2024-09-10T22:14:12.139590+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:56.027031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.158650+0000","last_change":"2024-09-10T22:14:11.153324+0000","last_active":"2024-09-10T22:14:11.158650+0000","last_peered":"2024-09-10T22:14:11.158650+0000","last_clean":"2024-09-10T22:14:11.158650+0000","last_became_active":"2024-09-10T22:14:11.153038+0000","last_became_peered":"2024-09-10T22:14:11.153038+0000","last_unstale":"2024-09-10T22:14:11.158650+0000","last_undegraded":"2024-09-10T22:14:11.158650+0000","last_fullsized":"2024-09-10T22:14:11.158650+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:14.507062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147275+0000","last_change":"2024-09-10T22:14:11.156650+0000","last_active":"2024-09-10T22:14:12.147275+0000","last_peered":"2024-09-10T22:14:12.147275+0000","last_clean":"2024-09-10T22:14:12.147275+0000","last_became_active":"2024-09-10T22:14:11.156387+0000","last_became_peered":"2024-09-10T22:14:11.156387+0000","last_unstale":"2024-09-10T22:14:12.147275+0000","last_undegraded":"2024-09-10T22:14:12.147275+0000","last_fullsized":"2024-09-10T22:14:12.147275+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:24.593483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149736+0000","last_change":"2024-09-10T22:14:11.140124+0000","last_active":"2024-09-10T22:14:12.149736+0000","last_peered":"2024-09-10T22:14:12.149736+0000","last_clean":"2024-09-10T22:14:12.149736+0000","last_became_active":"2024-09-10T22:14:11.139638+0000","last_became_peered":"2024-09-10T22:14:11.139638+0000","last_unstale":"2024-09-10T22:14:12.149736+0000","last_undegraded":"2024-09-10T22:14:12.149736+0000","last_fullsized":"2024-09-10T22:14:12.149736+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:15.767299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149632+0000","last_change":"2024-09-10T22:14:11.144529+0000","last_active":"2024-09-10T22:14:12.149632+0000","last_peered":"2024-09-10T22:14:12.149632+0000","last_clean":"2024-09-10T22:14:12.149632+0000","last_became_active":"2024-09-10T22:14:11.144245+0000","last_became_peered":"2024-09-10T22:14:11.144245+0000","last_unstale":"2024-09-10T22:14:12.149632+0000","last_undegraded":"2024-09-10T22:14:12.149632+0000","last_fullsized":"2024-09-10T22:14:12.149632+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:55.019694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147752+0000","last_change":"2024-09-10T22:14:11.154833+0000","last_active":"2024-09-10T22:14:12.147752+0000","last_peered":"2024-09-10T22:14:12.147752+0000","last_clean":"2024-09-10T22:14:12.147752+0000","last_became_active":"2024-09-10T22:14:11.154571+0000","last_became_peered":"2024-09-10T22:14:11.154571+0000","last_unstale":"2024-09-10T22:14:12.147752+0000","last_undegraded":"2024-09-10T22:14:12.147752+0000","last_fullsized":"2024-09-10T22:14:12.147752+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:01.295873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.156698+0000","last_change":"2024-09-10T22:14:11.151352+0000","last_active":"2024-09-10T22:14:11.156698+0000","last_peered":"2024-09-10T22:14:11.156698+0000","last_clean":"2024-09-10T22:14:11.156698+0000","last_became_active":"2024-09-10T22:14:11.150860+0000","last_became_peered":"2024-09-10T22:14:11.150860+0000","last_unstale":"2024-09-10T22:14:11.156698+0000","last_undegraded":"2024-09-10T22:14:11.156698+0000","last_fullsized":"2024-09-10T22:14:11.156698+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:55.355103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139478+0000","last_change":"2024-09-10T22:14:11.146866+0000","last_active":"2024-09-10T22:14:12.139478+0000","last_peered":"2024-09-10T22:14:12.139478+0000","last_clean":"2024-09-10T22:14:12.139478+0000","last_became_active":"2024-09-10T22:14:11.146597+0000","last_became_peered":"2024-09-10T22:14:11.146597+0000","last_unstale":"2024-09-10T22:14:12.139478+0000","last_undegraded":"2024-09-10T22:14:12.139478+0000","last_fullsized":"2024-09-10T22:14:12.139478+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:09.566270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.158405+0000","last_change":"2024-09-10T22:14:11.151775+0000","last_active":"2024-09-10T22:14:11.158405+0000","last_peered":"2024-09-10T22:14:11.158405+0000","last_clean":"2024-09-10T22:14:11.158405+0000","last_became_active":"2024-09-10T22:14:11.151397+0000","last_became_peered":"2024-09-10T22:14:11.151397+0000","last_unstale":"2024-09-10T22:14:11.158405+0000","last_undegraded":"2024-09-10T22:14:11.158405+0000","last_fullsized":"2024-09-10T22:14:11.158405+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:33.228563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.144093+0000","last_change":"2024-09-10T22:14:11.152661+0000","last_active":"2024-09-10T22:14:12.144093+0000","last_peered":"2024-09-10T22:14:12.144093+0000","last_clean":"2024-09-10T22:14:12.144093+0000","last_became_active":"2024-09-10T22:14:11.152341+0000","last_became_peered":"2024-09-10T22:14:11.152341+0000","last_unstale":"2024-09-10T22:14:12.144093+0000","last_undegraded":"2024-09-10T22:14:12.144093+0000","last_fullsized":"2024-09-10T22:14:12.144093+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:08.016417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.138990+0000","last_change":"2024-09-10T22:14:11.151730+0000","last_active":"2024-09-10T22:14:12.138990+0000","last_peered":"2024-09-10T22:14:12.138990+0000","last_clean":"2024-09-10T22:14:12.138990+0000","last_became_active":"2024-09-10T22:14:11.151475+0000","last_became_peered":"2024-09-10T22:14:11.151475+0000","last_unstale":"2024-09-10T22:14:12.138990+0000","last_undegraded":"2024-09-10T22:14:12.138990+0000","last_fullsized":"2024-09-10T22:14:12.138990+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:15.312565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149745+0000","last_change":"2024-09-10T22:14:11.145130+0000","last_active":"2024-09-10T22:14:12.149745+0000","last_peered":"2024-09-10T22:14:12.149745+0000","last_clean":"2024-09-10T22:14:12.149745+0000","last_became_active":"2024-09-10T22:14:11.143987+0000","last_became_peered":"2024-09-10T22:14:11.143987+0000","last_unstale":"2024-09-10T22:14:12.149745+0000","last_undegraded":"2024-09-10T22:14:12.149745+0000","last_fullsized":"2024-09-10T22:14:12.149745+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:31.934705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145591+0000","last_change":"2024-09-10T22:14:11.152403+0000","last_active":"2024-09-10T22:14:12.145591+0000","last_peered":"2024-09-10T22:14:12.145591+0000","last_clean":"2024-09-10T22:14:12.145591+0000","last_became_active":"2024-09-10T22:14:11.152128+0000","last_became_peered":"2024-09-10T22:14:11.152128+0000","last_unstale":"2024-09-10T22:14:12.145591+0000","last_undegraded":"2024-09-10T22:14:12.145591+0000","last_fullsized":"2024-09-10T22:14:12.145591+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:28.222948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.150330+0000","last_change":"2024-09-10T22:14:11.147854+0000","last_active":"2024-09-10T22:14:11.150330+0000","last_peered":"2024-09-10T22:14:11.150330+0000","last_clean":"2024-09-10T22:14:11.150330+0000","last_became_active":"2024-09-10T22:14:11.147581+0000","last_became_peered":"2024-09-10T22:14:11.147581+0000","last_unstale":"2024-09-10T22:14:11.150330+0000","last_undegraded":"2024-09-10T22:14:11.150330+0000","last_fullsized":"2024-09-10T22:14:11.150330+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:39.564127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146948+0000","last_change":"2024-09-10T22:14:11.160273+0000","last_active":"2024-09-10T22:14:12.146948+0000","last_peered":"2024-09-10T22:14:12.146948+0000","last_clean":"2024-09-10T22:14:12.146948+0000","last_became_active":"2024-09-10T22:14:11.159258+0000","last_became_peered":"2024-09-10T22:14:11.159258+0000","last_unstale":"2024-09-10T22:14:12.146948+0000","last_undegraded":"2024-09-10T22:14:12.146948+0000","last_fullsized":"2024-09-10T22:14:12.146948+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:08.757824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"140'4","reported_seq":19,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147344+0000","last_change":"2024-09-10T22:14:10.143281+0000","last_active":"2024-09-10T22:14:12.147344+0000","last_peered":"2024-09-10T22:14:12.147344+0000","last_clean":"2024-09-10T22:14:12.147344+0000","last_became_active":"2024-09-10T22:14:10.142965+0000","last_became_peered":"2024-09-10T22:14:10.142965+0000","last_unstale":"2024-09-10T22:14:12.147344+0000","last_undegraded":"2024-09-10T22:14:12.147344+0000","last_fullsized":"2024-09-10T22:14:12.147344+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:15.034996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146116+0000","last_change":"2024-09-10T22:14:11.141664+0000","last_active":"2024-09-10T22:14:12.146116+0000","last_peered":"2024-09-10T22:14:12.146116+0000","last_clean":"2024-09-10T22:14:12.146116+0000","last_became_active":"2024-09-10T22:14:11.141382+0000","last_became_peered":"2024-09-10T22:14:11.141382+0000","last_unstale":"2024-09-10T22:14:12.146116+0000","last_undegraded":"2024-09-10T22:14:12.146116+0000","last_fullsized":"2024-09-10T22:14:12.146116+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:13.908833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"140'1","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145957+0000","last_change":"2024-09-10T22:14:10.130630+0000","last_active":"2024-09-10T22:14:12.145957+0000","last_peered":"2024-09-10T22:14:12.145957+0000","last_clean":"2024-09-10T22:14:12.145957+0000","last_became_active":"2024-09-10T22:14:10.129926+0000","last_became_peered":"2024-09-10T22:14:10.129926+0000","last_unstale":"2024-09-10T22:14:12.145957+0000","last_undegraded":"2024-09-10T22:14:12.145957+0000","last_fullsized":"2024-09-10T22:14:12.145957+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:14.760794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139239+0000","last_change":"2024-09-10T22:14:11.141345+0000","last_active":"2024-09-10T22:14:12.139239+0000","last_peered":"2024-09-10T22:14:12.139239+0000","last_clean":"2024-09-10T22:14:12.139239+0000","last_became_active":"2024-09-10T22:14:11.141064+0000","last_became_peered":"2024-09-10T22:14:11.141064+0000","last_unstale":"2024-09-10T22:14:12.139239+0000","last_undegraded":"2024-09-10T22:14:12.139239+0000","last_fullsized":"2024-09-10T22:14:12.139239+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.713157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"140'2","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150658+0000","last_change":"2024-09-10T22:14:10.133778+0000","last_active":"2024-09-10T22:14:12.150658+0000","last_peered":"2024-09-10T22:14:12.150658+0000","last_clean":"2024-09-10T22:14:12.150658+0000","last_became_active":"2024-09-10T22:14:10.133530+0000","last_became_peered":"2024-09-10T22:14:10.133530+0000","last_unstale":"2024-09-10T22:14:12.150658+0000","last_undegraded":"2024-09-10T22:14:12.150658+0000","last_fullsized":"2024-09-10T22:14:12.150658+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:54.043735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147395+0000","last_change":"2024-09-10T22:14:11.156934+0000","last_active":"2024-09-10T22:14:12.147395+0000","last_peered":"2024-09-10T22:14:12.147395+0000","last_clean":"2024-09-10T22:14:12.147395+0000","last_became_active":"2024-09-10T22:14:11.156701+0000","last_became_peered":"2024-09-10T22:14:11.156701+0000","last_unstale":"2024-09-10T22:14:12.147395+0000","last_undegraded":"2024-09-10T22:14:12.147395+0000","last_fullsized":"2024-09-10T22:14:12.147395+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:35.925135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"140'2","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150929+0000","last_change":"2024-09-10T22:14:10.131170+0000","last_active":"2024-09-10T22:14:12.150929+0000","last_peered":"2024-09-10T22:14:12.150929+0000","last_clean":"2024-09-10T22:14:12.150929+0000","last_became_active":"2024-09-10T22:14:10.130790+0000","last_became_peered":"2024-09-10T22:14:10.130790+0000","last_unstale":"2024-09-10T22:14:12.150929+0000","last_undegraded":"2024-09-10T22:14:12.150929+0000","last_fullsized":"2024-09-10T22:14:12.150929+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:00.709944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145759+0000","last_change":"2024-09-10T22:14:11.151864+0000","last_active":"2024-09-10T22:14:12.145759+0000","last_peered":"2024-09-10T22:14:12.145759+0000","last_clean":"2024-09-10T22:14:12.145759+0000","last_became_active":"2024-09-10T22:14:11.151558+0000","last_became_peered":"2024-09-10T22:14:11.151558+0000","last_unstale":"2024-09-10T22:14:12.145759+0000","last_undegraded":"2024-09-10T22:14:12.145759+0000","last_fullsized":"2024-09-10T22:14:12.145759+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:47.604624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"140'1","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.163647+0000","last_change":"2024-09-10T22:14:10.134766+0000","last_active":"2024-09-10T22:14:11.163647+0000","last_peered":"2024-09-10T22:14:11.163647+0000","last_clean":"2024-09-10T22:14:11.163647+0000","last_became_active":"2024-09-10T22:14:10.134449+0000","last_became_peered":"2024-09-10T22:14:10.134449+0000","last_unstale":"2024-09-10T22:14:11.163647+0000","last_undegraded":"2024-09-10T22:14:11.163647+0000","last_fullsized":"2024-09-10T22:14:11.163647+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:29.038997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150067+0000","last_change":"2024-09-10T22:14:11.156415+0000","last_active":"2024-09-10T22:14:12.150067+0000","last_peered":"2024-09-10T22:14:12.150067+0000","last_clean":"2024-09-10T22:14:12.150067+0000","last_became_active":"2024-09-10T22:14:11.156075+0000","last_became_peered":"2024-09-10T22:14:11.156075+0000","last_unstale":"2024-09-10T22:14:12.150067+0000","last_undegraded":"2024-09-10T22:14:12.150067+0000","last_fullsized":"2024-09-10T22:14:12.150067+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:28.688741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"140'1","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145701+0000","last_change":"2024-09-10T22:14:10.139434+0000","last_active":"2024-09-10T22:14:12.145701+0000","last_peered":"2024-09-10T22:14:12.145701+0000","last_clean":"2024-09-10T22:14:12.145701+0000","last_became_active":"2024-09-10T22:14:10.139128+0000","last_became_peered":"2024-09-10T22:14:10.139128+0000","last_unstale":"2024-09-10T22:14:12.145701+0000","last_undegraded":"2024-09-10T22:14:12.145701+0000","last_fullsized":"2024-09-10T22:14:12.145701+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:17.381369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145588+0000","last_change":"2024-09-10T22:14:11.135507+0000","last_active":"2024-09-10T22:14:12.145588+0000","last_peered":"2024-09-10T22:14:12.145588+0000","last_clean":"2024-09-10T22:14:12.145588+0000","last_became_active":"2024-09-10T22:14:11.135225+0000","last_became_peered":"2024-09-10T22:14:11.135225+0000","last_unstale":"2024-09-10T22:14:12.145588+0000","last_undegraded":"2024-09-10T22:14:12.145588+0000","last_fullsized":"2024-09-10T22:14:12.145588+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:11.250908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.144909+0000","last_change":"2024-09-10T22:14:10.135568+0000","last_active":"2024-09-10T22:14:11.144909+0000","last_peered":"2024-09-10T22:14:11.144909+0000","last_clean":"2024-09-10T22:14:11.144909+0000","last_became_active":"2024-09-10T22:14:10.133987+0000","last_became_peered":"2024-09-10T22:14:10.133987+0000","last_unstale":"2024-09-10T22:14:11.144909+0000","last_undegraded":"2024-09-10T22:14:11.144909+0000","last_fullsized":"2024-09-10T22:14:11.144909+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:39.742924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145084+0000","last_change":"2024-09-10T22:14:11.145109+0000","last_active":"2024-09-10T22:14:12.145084+0000","last_peered":"2024-09-10T22:14:12.145084+0000","last_clean":"2024-09-10T22:14:12.145084+0000","last_became_active":"2024-09-10T22:14:11.144845+0000","last_became_peered":"2024-09-10T22:14:11.144845+0000","last_unstale":"2024-09-10T22:14:12.145084+0000","last_undegraded":"2024-09-10T22:14:12.145084+0000","last_fullsized":"2024-09-10T22:14:12.145084+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:16.145337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149137+0000","last_change":"2024-09-10T22:14:10.133113+0000","last_active":"2024-09-10T22:14:12.149137+0000","last_peered":"2024-09-10T22:14:12.149137+0000","last_clean":"2024-09-10T22:14:12.149137+0000","last_became_active":"2024-09-10T22:14:10.132815+0000","last_became_peered":"2024-09-10T22:14:10.132815+0000","last_unstale":"2024-09-10T22:14:12.149137+0000","last_undegraded":"2024-09-10T22:14:12.149137+0000","last_fullsized":"2024-09-10T22:14:12.149137+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:00.884414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146126+0000","last_change":"2024-09-10T22:14:11.153743+0000","last_active":"2024-09-10T22:14:12.146126+0000","last_peered":"2024-09-10T22:14:12.146126+0000","last_clean":"2024-09-10T22:14:12.146126+0000","last_became_active":"2024-09-10T22:14:11.153204+0000","last_became_peered":"2024-09-10T22:14:11.153204+0000","last_unstale":"2024-09-10T22:14:12.146126+0000","last_undegraded":"2024-09-10T22:14:12.146126+0000","last_fullsized":"2024-09-10T22:14:12.146126+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:50.063184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145988+0000","last_change":"2024-09-10T22:14:10.137055+0000","last_active":"2024-09-10T22:14:12.145988+0000","last_peered":"2024-09-10T22:14:12.145988+0000","last_clean":"2024-09-10T22:14:12.145988+0000","last_became_active":"2024-09-10T22:14:10.136715+0000","last_became_peered":"2024-09-10T22:14:10.136715+0000","last_unstale":"2024-09-10T22:14:12.145988+0000","last_undegraded":"2024-09-10T22:14:12.145988+0000","last_fullsized":"2024-09-10T22:14:12.145988+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:37.770852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147190+0000","last_change":"2024-09-10T22:14:11.165901+0000","last_active":"2024-09-10T22:14:12.147190+0000","last_peered":"2024-09-10T22:14:12.147190+0000","last_clean":"2024-09-10T22:14:12.147190+0000","last_became_active":"2024-09-10T22:14:11.165277+0000","last_became_peered":"2024-09-10T22:14:11.165277+0000","last_unstale":"2024-09-10T22:14:12.147190+0000","last_undegraded":"2024-09-10T22:14:12.147190+0000","last_fullsized":"2024-09-10T22:14:12.147190+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:58.682027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147163+0000","last_change":"2024-09-10T22:14:10.139594+0000","last_active":"2024-09-10T22:14:12.147163+0000","last_peered":"2024-09-10T22:14:12.147163+0000","last_clean":"2024-09-10T22:14:12.147163+0000","last_became_active":"2024-09-10T22:14:10.139324+0000","last_became_peered":"2024-09-10T22:14:10.139324+0000","last_unstale":"2024-09-10T22:14:12.147163+0000","last_undegraded":"2024-09-10T22:14:12.147163+0000","last_fullsized":"2024-09-10T22:14:12.147163+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:04.219100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.158431+0000","last_change":"2024-09-10T22:14:11.154367+0000","last_active":"2024-09-10T22:14:11.158431+0000","last_peered":"2024-09-10T22:14:11.158431+0000","last_clean":"2024-09-10T22:14:11.158431+0000","last_became_active":"2024-09-10T22:14:11.154027+0000","last_became_peered":"2024-09-10T22:14:11.154027+0000","last_unstale":"2024-09-10T22:14:11.158431+0000","last_undegraded":"2024-09-10T22:14:11.158431+0000","last_fullsized":"2024-09-10T22:14:11.158431+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:22.194910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"140'6","reported_seq":21,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.144200+0000","last_change":"2024-09-10T22:14:10.139555+0000","last_active":"2024-09-10T22:14:12.144200+0000","last_peered":"2024-09-10T22:14:12.144200+0000","last_clean":"2024-09-10T22:14:12.144200+0000","last_became_active":"2024-09-10T22:14:10.139172+0000","last_became_peered":"2024-09-10T22:14:10.139172+0000","last_unstale":"2024-09-10T22:14:12.144200+0000","last_undegraded":"2024-09-10T22:14:12.144200+0000","last_fullsized":"2024-09-10T22:14:12.144200+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:19.487900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150889+0000","last_change":"2024-09-10T22:14:11.151806+0000","last_active":"2024-09-10T22:14:12.150889+0000","last_peered":"2024-09-10T22:14:12.150889+0000","last_clean":"2024-09-10T22:14:12.150889+0000","last_became_active":"2024-09-10T22:14:11.151386+0000","last_became_peered":"2024-09-10T22:14:11.151386+0000","last_unstale":"2024-09-10T22:14:12.150889+0000","last_undegraded":"2024-09-10T22:14:12.150889+0000","last_fullsized":"2024-09-10T22:14:12.150889+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:04.332643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"140'2","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150636+0000","last_change":"2024-09-10T22:14:10.132968+0000","last_active":"2024-09-10T22:14:12.150636+0000","last_peered":"2024-09-10T22:14:12.150636+0000","last_clean":"2024-09-10T22:14:12.150636+0000","last_became_active":"2024-09-10T22:14:10.131997+0000","last_became_peered":"2024-09-10T22:14:10.131997+0000","last_unstale":"2024-09-10T22:14:12.150636+0000","last_undegraded":"2024-09-10T22:14:12.150636+0000","last_fullsized":"2024-09-10T22:14:12.150636+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:28.767939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145626+0000","last_change":"2024-09-10T22:14:11.139527+0000","last_active":"2024-09-10T22:14:12.145626+0000","last_peered":"2024-09-10T22:14:12.145626+0000","last_clean":"2024-09-10T22:14:12.145626+0000","last_became_active":"2024-09-10T22:14:11.138083+0000","last_became_peered":"2024-09-10T22:14:11.138083+0000","last_unstale":"2024-09-10T22:14:12.145626+0000","last_undegraded":"2024-09-10T22:14:12.145626+0000","last_fullsized":"2024-09-10T22:14:12.145626+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:54.135329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150789+0000","last_change":"2024-09-10T22:14:10.139618+0000","last_active":"2024-09-10T22:14:12.150789+0000","last_peered":"2024-09-10T22:14:12.150789+0000","last_clean":"2024-09-10T22:14:12.150789+0000","last_became_active":"2024-09-10T22:14:10.138960+0000","last_became_peered":"2024-09-10T22:14:10.138960+0000","last_unstale":"2024-09-10T22:14:12.150789+0000","last_undegraded":"2024-09-10T22:14:12.150789+0000","last_fullsized":"2024-09-10T22:14:12.150789+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:21.315614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145397+0000","last_change":"2024-09-10T22:14:11.154479+0000","last_active":"2024-09-10T22:14:12.145397+0000","last_peered":"2024-09-10T22:14:12.145397+0000","last_clean":"2024-09-10T22:14:12.145397+0000","last_became_active":"2024-09-10T22:14:11.154061+0000","last_became_peered":"2024-09-10T22:14:11.154061+0000","last_unstale":"2024-09-10T22:14:12.145397+0000","last_undegraded":"2024-09-10T22:14:12.145397+0000","last_fullsized":"2024-09-10T22:14:12.145397+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.713157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.138924+0000","last_change":"2024-09-10T22:14:10.126544+0000","last_active":"2024-09-10T22:14:12.138924+0000","last_peered":"2024-09-10T22:14:12.138924+0000","last_clean":"2024-09-10T22:14:12.138924+0000","last_became_active":"2024-09-10T22:14:10.126235+0000","last_became_peered":"2024-09-10T22:14:10.126235+0000","last_unstale":"2024-09-10T22:14:12.138924+0000","last_undegraded":"2024-09-10T22:14:12.138924+0000","last_fullsized":"2024-09-10T22:14:12.138924+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:41.794864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149538+0000","last_change":"2024-09-10T22:14:11.157646+0000","last_active":"2024-09-10T22:14:12.149538+0000","last_peered":"2024-09-10T22:14:12.149538+0000","last_clean":"2024-09-10T22:14:12.149538+0000","last_became_active":"2024-09-10T22:14:11.155287+0000","last_became_peered":"2024-09-10T22:14:11.155287+0000","last_unstale":"2024-09-10T22:14:12.149538+0000","last_undegraded":"2024-09-10T22:14:12.149538+0000","last_fullsized":"2024-09-10T22:14:12.149538+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:01.890920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"140'8","reported_seq":21,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.164736+0000","last_change":"2024-09-10T22:14:10.134829+0000","last_active":"2024-09-10T22:14:11.164736+0000","last_peered":"2024-09-10T22:14:11.164736+0000","last_clean":"2024-09-10T22:14:11.164736+0000","last_became_active":"2024-09-10T22:14:10.134543+0000","last_became_peered":"2024-09-10T22:14:10.134543+0000","last_unstale":"2024-09-10T22:14:11.164736+0000","last_undegraded":"2024-09-10T22:14:11.164736+0000","last_fullsized":"2024-09-10T22:14:11.164736+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:01.838543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146932+0000","last_change":"2024-09-10T22:14:11.164344+0000","last_active":"2024-09-10T22:14:12.146932+0000","last_peered":"2024-09-10T22:14:12.146932+0000","last_clean":"2024-09-10T22:14:12.146932+0000","last_became_active":"2024-09-10T22:14:11.163968+0000","last_became_peered":"2024-09-10T22:14:11.163968+0000","last_unstale":"2024-09-10T22:14:12.146932+0000","last_undegraded":"2024-09-10T22:14:12.146932+0000","last_fullsized":"2024-09-10T22:14:12.146932+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:37.799310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"140'1","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146249+0000","last_change":"2024-09-10T22:14:10.136669+0000","last_active":"2024-09-10T22:14:12.146249+0000","last_peered":"2024-09-10T22:14:12.146249+0000","last_clean":"2024-09-10T22:14:12.146249+0000","last_became_active":"2024-09-10T22:14:10.136374+0000","last_became_peered":"2024-09-10T22:14:10.136374+0000","last_unstale":"2024-09-10T22:14:12.146249+0000","last_undegraded":"2024-09-10T22:14:12.146249+0000","last_fullsized":"2024-09-10T22:14:12.146249+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:35.407875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.158022+0000","last_change":"2024-09-10T22:14:11.155195+0000","last_active":"2024-09-10T22:14:11.158022+0000","last_peered":"2024-09-10T22:14:11.158022+0000","last_clean":"2024-09-10T22:14:11.158022+0000","last_became_active":"2024-09-10T22:14:11.154945+0000","last_became_peered":"2024-09-10T22:14:11.154945+0000","last_unstale":"2024-09-10T22:14:11.158022+0000","last_undegraded":"2024-09-10T22:14:11.158022+0000","last_fullsized":"2024-09-10T22:14:11.158022+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:02.158638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"140'2","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146070+0000","last_change":"2024-09-10T22:14:10.136406+0000","last_active":"2024-09-10T22:14:12.146070+0000","last_peered":"2024-09-10T22:14:12.146070+0000","last_clean":"2024-09-10T22:14:12.146070+0000","last_became_active":"2024-09-10T22:14:10.135227+0000","last_became_peered":"2024-09-10T22:14:10.135227+0000","last_unstale":"2024-09-10T22:14:12.146070+0000","last_undegraded":"2024-09-10T22:14:12.146070+0000","last_fullsized":"2024-09-10T22:14:12.146070+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:30.928200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150280+0000","last_change":"2024-09-10T22:14:11.150099+0000","last_active":"2024-09-10T22:14:12.150280+0000","last_peered":"2024-09-10T22:14:12.150280+0000","last_clean":"2024-09-10T22:14:12.150280+0000","last_became_active":"2024-09-10T22:14:11.149837+0000","last_became_peered":"2024-09-10T22:14:11.149837+0000","last_unstale":"2024-09-10T22:14:12.150280+0000","last_undegraded":"2024-09-10T22:14:12.150280+0000","last_fullsized":"2024-09-10T22:14:12.150280+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:26.272238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"140'1","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.163295+0000","last_change":"2024-09-10T22:14:10.133929+0000","last_active":"2024-09-10T22:14:11.163295+0000","last_peered":"2024-09-10T22:14:11.163295+0000","last_clean":"2024-09-10T22:14:11.163295+0000","last_became_active":"2024-09-10T22:14:10.133326+0000","last_became_peered":"2024-09-10T22:14:10.133326+0000","last_unstale":"2024-09-10T22:14:11.163295+0000","last_undegraded":"2024-09-10T22:14:11.163295+0000","last_fullsized":"2024-09-10T22:14:11.163295+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:25.987026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139369+0000","last_change":"2024-09-10T22:14:11.151134+0000","last_active":"2024-09-10T22:14:12.139369+0000","last_peered":"2024-09-10T22:14:12.139369+0000","last_clean":"2024-09-10T22:14:12.139369+0000","last_became_active":"2024-09-10T22:14:11.150863+0000","last_became_peered":"2024-09-10T22:14:11.150863+0000","last_unstale":"2024-09-10T22:14:12.139369+0000","last_undegraded":"2024-09-10T22:14:12.139369+0000","last_fullsized":"2024-09-10T22:14:12.139369+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:27:54.416428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146174+0000","last_change":"2024-09-10T22:14:10.135580+0000","last_active":"2024-09-10T22:14:12.146174+0000","last_peered":"2024-09-10T22:14:12.146174+0000","last_clean":"2024-09-10T22:14:12.146174+0000","last_became_active":"2024-09-10T22:14:10.134413+0000","last_became_peered":"2024-09-10T22:14:10.134413+0000","last_unstale":"2024-09-10T22:14:12.146174+0000","last_undegraded":"2024-09-10T22:14:12.146174+0000","last_fullsized":"2024-09-10T22:14:12.146174+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:06.712240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149967+0000","last_change":"2024-09-10T22:14:11.144790+0000","last_active":"2024-09-10T22:14:12.149967+0000","last_peered":"2024-09-10T22:14:12.149967+0000","last_clean":"2024-09-10T22:14:12.149967+0000","last_became_active":"2024-09-10T22:14:11.143920+0000","last_became_peered":"2024-09-10T22:14:11.143920+0000","last_unstale":"2024-09-10T22:14:12.149967+0000","last_undegraded":"2024-09-10T22:14:12.149967+0000","last_fullsized":"2024-09-10T22:14:12.149967+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:38.777357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"140'2","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.143817+0000","last_change":"2024-09-10T22:14:10.131729+0000","last_active":"2024-09-10T22:14:12.143817+0000","last_peered":"2024-09-10T22:14:12.143817+0000","last_clean":"2024-09-10T22:14:12.143817+0000","last_became_active":"2024-09-10T22:14:10.129611+0000","last_became_peered":"2024-09-10T22:14:10.129611+0000","last_unstale":"2024-09-10T22:14:12.143817+0000","last_undegraded":"2024-09-10T22:14:12.143817+0000","last_fullsized":"2024-09-10T22:14:12.143817+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:11:36.178767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.143885+0000","last_change":"2024-09-10T22:14:11.142721+0000","last_active":"2024-09-10T22:14:12.143885+0000","last_peered":"2024-09-10T22:14:12.143885+0000","last_clean":"2024-09-10T22:14:12.143885+0000","last_became_active":"2024-09-10T22:14:11.142410+0000","last_became_peered":"2024-09-10T22:14:11.142410+0000","last_unstale":"2024-09-10T22:14:12.143885+0000","last_undegraded":"2024-09-10T22:14:12.143885+0000","last_fullsized":"2024-09-10T22:14:12.143885+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:48.174482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147632+0000","last_change":"2024-09-10T22:14:10.142440+0000","last_active":"2024-09-10T22:14:12.147632+0000","last_peered":"2024-09-10T22:14:12.147632+0000","last_clean":"2024-09-10T22:14:12.147632+0000","last_became_active":"2024-09-10T22:14:10.142117+0000","last_became_peered":"2024-09-10T22:14:10.142117+0000","last_unstale":"2024-09-10T22:14:12.147632+0000","last_undegraded":"2024-09-10T22:14:12.147632+0000","last_fullsized":"2024-09-10T22:14:12.147632+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:15.100421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149220+0000","last_change":"2024-09-10T22:14:11.143555+0000","last_active":"2024-09-10T22:14:12.149220+0000","last_peered":"2024-09-10T22:14:12.149220+0000","last_clean":"2024-09-10T22:14:12.149220+0000","last_became_active":"2024-09-10T22:14:11.143273+0000","last_became_peered":"2024-09-10T22:14:11.143273+0000","last_unstale":"2024-09-10T22:14:12.149220+0000","last_undegraded":"2024-09-10T22:14:12.149220+0000","last_fullsized":"2024-09-10T22:14:12.149220+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:05.768167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146753+0000","last_change":"2024-09-10T22:14:10.142483+0000","last_active":"2024-09-10T22:14:12.146753+0000","last_peered":"2024-09-10T22:14:12.146753+0000","last_clean":"2024-09-10T22:14:12.146753+0000","last_became_active":"2024-09-10T22:14:10.141397+0000","last_became_peered":"2024-09-10T22:14:10.141397+0000","last_unstale":"2024-09-10T22:14:12.146753+0000","last_undegraded":"2024-09-10T22:14:12.146753+0000","last_fullsized":"2024-09-10T22:14:12.146753+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:46.598119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.143474+0000","last_change":"2024-09-10T22:14:11.142115+0000","last_active":"2024-09-10T22:14:12.143474+0000","last_peered":"2024-09-10T22:14:12.143474+0000","last_clean":"2024-09-10T22:14:12.143474+0000","last_became_active":"2024-09-10T22:14:11.141739+0000","last_became_peered":"2024-09-10T22:14:11.141739+0000","last_unstale":"2024-09-10T22:14:12.143474+0000","last_undegraded":"2024-09-10T22:14:12.143474+0000","last_fullsized":"2024-09-10T22:14:12.143474+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:33.081442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149838+0000","last_change":"2024-09-10T22:14:10.143014+0000","last_active":"2024-09-10T22:14:12.149838+0000","last_peered":"2024-09-10T22:14:12.149838+0000","last_clean":"2024-09-10T22:14:12.149838+0000","last_became_active":"2024-09-10T22:14:10.142701+0000","last_became_peered":"2024-09-10T22:14:10.142701+0000","last_unstale":"2024-09-10T22:14:12.149838+0000","last_undegraded":"2024-09-10T22:14:12.149838+0000","last_fullsized":"2024-09-10T22:14:12.149838+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:02.251766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146360+0000","last_change":"2024-09-10T22:14:11.153458+0000","last_active":"2024-09-10T22:14:12.146360+0000","last_peered":"2024-09-10T22:14:12.146360+0000","last_clean":"2024-09-10T22:14:12.146360+0000","last_became_active":"2024-09-10T22:14:11.152885+0000","last_became_peered":"2024-09-10T22:14:11.152885+0000","last_unstale":"2024-09-10T22:14:12.146360+0000","last_undegraded":"2024-09-10T22:14:12.146360+0000","last_fullsized":"2024-09-10T22:14:12.146360+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:10.283455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"140'2","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147639+0000","last_change":"2024-09-10T22:14:10.144525+0000","last_active":"2024-09-10T22:14:12.147639+0000","last_peered":"2024-09-10T22:14:12.147639+0000","last_clean":"2024-09-10T22:14:12.147639+0000","last_became_active":"2024-09-10T22:14:10.144290+0000","last_became_peered":"2024-09-10T22:14:10.144290+0000","last_unstale":"2024-09-10T22:14:12.147639+0000","last_undegraded":"2024-09-10T22:14:12.147639+0000","last_fullsized":"2024-09-10T22:14:12.147639+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:12.902328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150533+0000","last_change":"2024-09-10T22:14:11.153510+0000","last_active":"2024-09-10T22:14:12.150533+0000","last_peered":"2024-09-10T22:14:12.150533+0000","last_clean":"2024-09-10T22:14:12.150533+0000","last_became_active":"2024-09-10T22:14:11.152647+0000","last_became_peered":"2024-09-10T22:14:11.152647+0000","last_unstale":"2024-09-10T22:14:12.150533+0000","last_undegraded":"2024-09-10T22:14:12.150533+0000","last_fullsized":"2024-09-10T22:14:12.150533+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:02.793500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"141'1","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.151132+0000","last_change":"2024-09-10T22:14:10.136755+0000","last_active":"2024-09-10T22:14:12.151132+0000","last_peered":"2024-09-10T22:14:12.151132+0000","last_clean":"2024-09-10T22:14:12.151132+0000","last_became_active":"2024-09-10T22:14:10.136454+0000","last_became_peered":"2024-09-10T22:14:10.136454+0000","last_unstale":"2024-09-10T22:14:12.151132+0000","last_undegraded":"2024-09-10T22:14:12.151132+0000","last_fullsized":"2024-09-10T22:14:12.151132+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:57.332982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145921+0000","last_change":"2024-09-10T22:14:11.135153+0000","last_active":"2024-09-10T22:14:12.145921+0000","last_peered":"2024-09-10T22:14:12.145921+0000","last_clean":"2024-09-10T22:14:12.145921+0000","last_became_active":"2024-09-10T22:14:11.134856+0000","last_became_peered":"2024-09-10T22:14:11.134856+0000","last_unstale":"2024-09-10T22:14:12.145921+0000","last_undegraded":"2024-09-10T22:14:12.145921+0000","last_fullsized":"2024-09-10T22:14:12.145921+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:43.862488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139620+0000","last_change":"2024-09-10T22:14:10.138413+0000","last_active":"2024-09-10T22:14:12.139620+0000","last_peered":"2024-09-10T22:14:12.139620+0000","last_clean":"2024-09-10T22:14:12.139620+0000","last_became_active":"2024-09-10T22:14:10.138155+0000","last_became_peered":"2024-09-10T22:14:10.138155+0000","last_unstale":"2024-09-10T22:14:12.139620+0000","last_undegraded":"2024-09-10T22:14:12.139620+0000","last_fullsized":"2024-09-10T22:14:12.139620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:21.650197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139514+0000","last_change":"2024-09-10T22:14:11.139594+0000","last_active":"2024-09-10T22:14:12.139514+0000","last_peered":"2024-09-10T22:14:12.139514+0000","last_clean":"2024-09-10T22:14:12.139514+0000","last_became_active":"2024-09-10T22:14:11.139309+0000","last_became_peered":"2024-09-10T22:14:11.139309+0000","last_unstale":"2024-09-10T22:14:12.139514+0000","last_undegraded":"2024-09-10T22:14:12.139514+0000","last_fullsized":"2024-09-10T22:14:12.139514+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:53.147936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.139139+0000","last_change":"2024-09-10T22:14:10.139717+0000","last_active":"2024-09-10T22:14:12.139139+0000","last_peered":"2024-09-10T22:14:12.139139+0000","last_clean":"2024-09-10T22:14:12.139139+0000","last_became_active":"2024-09-10T22:14:10.139455+0000","last_became_peered":"2024-09-10T22:14:10.139455+0000","last_unstale":"2024-09-10T22:14:12.139139+0000","last_undegraded":"2024-09-10T22:14:12.139139+0000","last_fullsized":"2024-09-10T22:14:12.139139+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:42.306792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-10T22:14:11.152649+0000","last_change":"2024-09-10T22:14:11.147393+0000","last_active":"2024-09-10T22:14:11.152649+0000","last_peered":"2024-09-10T22:14:11.152649+0000","last_clean":"2024-09-10T22:14:11.152649+0000","last_became_active":"2024-09-10T22:14:11.146195+0000","last_became_peered":"2024-09-10T22:14:11.146195+0000","last_unstale":"2024-09-10T22:14:11.152649+0000","last_undegraded":"2024-09-10T22:14:11.152649+0000","last_fullsized":"2024-09-10T22:14:11.152649+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:04.264645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.143909+0000","last_change":"2024-09-10T22:14:10.127456+0000","last_active":"2024-09-10T22:14:12.143909+0000","last_peered":"2024-09-10T22:14:12.143909+0000","last_clean":"2024-09-10T22:14:12.143909+0000","last_became_active":"2024-09-10T22:14:10.127211+0000","last_became_peered":"2024-09-10T22:14:10.127211+0000","last_unstale":"2024-09-10T22:14:12.143909+0000","last_undegraded":"2024-09-10T22:14:12.143909+0000","last_fullsized":"2024-09-10T22:14:12.143909+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:08.076582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145735+0000","last_change":"2024-09-10T22:14:11.160004+0000","last_active":"2024-09-10T22:14:12.145735+0000","last_peered":"2024-09-10T22:14:12.145735+0000","last_clean":"2024-09-10T22:14:12.145735+0000","last_became_active":"2024-09-10T22:14:11.159452+0000","last_became_peered":"2024-09-10T22:14:11.159452+0000","last_unstale":"2024-09-10T22:14:12.145735+0000","last_undegraded":"2024-09-10T22:14:12.145735+0000","last_fullsized":"2024-09-10T22:14:12.145735+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:30.086459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.147553+0000","last_change":"2024-09-10T22:14:10.141753+0000","last_active":"2024-09-10T22:14:12.147553+0000","last_peered":"2024-09-10T22:14:12.147553+0000","last_clean":"2024-09-10T22:14:12.147553+0000","last_became_active":"2024-09-10T22:14:10.141066+0000","last_became_peered":"2024-09-10T22:14:10.141066+0000","last_unstale":"2024-09-10T22:14:12.147553+0000","last_undegraded":"2024-09-10T22:14:12.147553+0000","last_fullsized":"2024-09-10T22:14:12.147553+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:42.855982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146744+0000","last_change":"2024-09-10T22:14:11.155949+0000","last_active":"2024-09-10T22:14:12.146744+0000","last_peered":"2024-09-10T22:14:12.146744+0000","last_clean":"2024-09-10T22:14:12.146744+0000","last_became_active":"2024-09-10T22:14:11.155695+0000","last_became_peered":"2024-09-10T22:14:11.155695+0000","last_unstale":"2024-09-10T22:14:12.146744+0000","last_undegraded":"2024-09-10T22:14:12.146744+0000","last_fullsized":"2024-09-10T22:14:12.146744+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:19.732547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.149959+0000","last_change":"2024-09-10T22:14:10.133689+0000","last_active":"2024-09-10T22:14:12.149959+0000","last_peered":"2024-09-10T22:14:12.149959+0000","last_clean":"2024-09-10T22:14:12.149959+0000","last_became_active":"2024-09-10T22:14:10.133389+0000","last_became_peered":"2024-09-10T22:14:10.133389+0000","last_unstale":"2024-09-10T22:14:12.149959+0000","last_undegraded":"2024-09-10T22:14:12.149959+0000","last_fullsized":"2024-09-10T22:14:12.149959+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:32.903462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145220+0000","last_change":"2024-09-10T22:14:11.153696+0000","last_active":"2024-09-10T22:14:12.145220+0000","last_peered":"2024-09-10T22:14:12.145220+0000","last_clean":"2024-09-10T22:14:12.145220+0000","last_became_active":"2024-09-10T22:14:11.153424+0000","last_became_peered":"2024-09-10T22:14:11.153424+0000","last_unstale":"2024-09-10T22:14:12.145220+0000","last_undegraded":"2024-09-10T22:14:12.145220+0000","last_fullsized":"2024-09-10T22:14:12.145220+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:15.312565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.145527+0000","last_change":"2024-09-10T22:14:10.135791+0000","last_active":"2024-09-10T22:14:12.145527+0000","last_peered":"2024-09-10T22:14:12.145527+0000","last_clean":"2024-09-10T22:14:12.145527+0000","last_became_active":"2024-09-10T22:14:10.135446+0000","last_became_peered":"2024-09-10T22:14:10.135446+0000","last_unstale":"2024-09-10T22:14:12.145527+0000","last_undegraded":"2024-09-10T22:14:12.145527+0000","last_fullsized":"2024-09-10T22:14:12.145527+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:54.013189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.144166+0000","last_change":"2024-09-10T22:14:11.145265+0000","last_active":"2024-09-10T22:14:12.144166+0000","last_peered":"2024-09-10T22:14:12.144166+0000","last_clean":"2024-09-10T22:14:12.144166+0000","last_became_active":"2024-09-10T22:14:11.144952+0000","last_became_peered":"2024-09-10T22:14:11.144952+0000","last_unstale":"2024-09-10T22:14:12.144166+0000","last_undegraded":"2024-09-10T22:14:12.144166+0000","last_fullsized":"2024-09-10T22:14:12.144166+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:10.108985+0000","last_clean_scrub_stamp":"2024-09-10T22:14:10.108985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:49.348318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"140'3","reported_seq":18,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.150501+0000","last_change":"2024-09-10T22:14:10.142497+0000","last_active":"2024-09-10T22:14:12.150501+0000","last_peered":"2024-09-10T22:14:12.150501+0000","last_clean":"2024-09-10T22:14:12.150501+0000","last_became_active":"2024-09-10T22:14:10.142198+0000","last_became_peered":"2024-09-10T22:14:10.142198+0000","last_unstale":"2024-09-10T22:14:12.150501+0000","last_undegraded":"2024-09-10T22:14:12.150501+0000","last_fullsized":"2024-09-10T22:14:12.150501+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:14:09.102480+0000","last_clean_scrub_stamp":"2024-09-10T22:14:09.102480+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:33.805508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":339,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T22:14:12.146826+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:14:12.146826+0000","last_peered":"2024-09-10T22:14:12.146826+0000","last_clean":"2024-09-10T22:14:12.146826+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:14:12.146826+0000","last_undegraded":"2024-09-10T22:14:12.146826+0000","last_fullsized":"2024-09-10T22:14:12.146826+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509609,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136136,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235704,"statfs":{"total":96636764160,"available":96497360896,"internally_reserved":0,"allocated":2347008,"data_stored":1359195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":137034186},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509610,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145288,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226552,"statfs":{"total":96636764160,"available":96487989248,"internally_reserved":0,"allocated":1757184,"data_stored":768321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509609,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109184,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262656,"statfs":{"total":96636764160,"available":96524959744,"internally_reserved":0,"allocated":1748992,"data_stored":768197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509610,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136964,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234876,"statfs":{"total":96636764160,"available":96496513024,"internally_reserved":0,"allocated":1744896,"data_stored":768175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509608,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136980,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234860,"statfs":{"total":96636764160,"available":96496496640,"internally_reserved":0,"allocated":1761280,"data_stored":774705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":138475975},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509609,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141120,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230720,"statfs":{"total":96636764160,"available":96492257280,"internally_reserved":0,"allocated":1748992,"data_stored":768757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509609,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131988,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239852,"statfs":{"total":96636764160,"available":96501608448,"internally_reserved":0,"allocated":2355200,"data_stored":1360189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":132774345},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542314,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136944,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234896,"statfs":{"total":96636764160,"available":96496533504,"internally_reserved":0,"allocated":1732608,"data_stored":761260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:14:15.703 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:14:15.703 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T22:14:15.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.891+0000 7fd87ba00640 1 -- 172.21.15.66:0/1134937275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd85c003ae0 msgr2=0x7fd85c003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.891+0000 7fd87ba00640 1 --2- 172.21.15.66:0/1134937275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd85c003ae0 0x7fd85c003f00 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd868006fc0 tx=0x7fd86801cf00 comp rx=0 tx=0).stop 2024-09-10T22:14:15.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.891+0000 7fd879a00640 1 -- 172.21.15.66:0/1134937275 reap_dead start 2024-09-10T22:14:15.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.891+0000 7fd87ba00640 1 -- 172.21.15.66:0/1134937275 shutdown_connections 2024-09-10T22:14:15.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.891+0000 7fd87ba00640 1 -- 172.21.15.66:0/1134937275 >> 172.21.15.66:0/1134937275 conn(0x7fd8740cfa30 msgr2=0x7fd8740cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:15.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.891+0000 7fd87ba00640 1 -- 172.21.15.66:0/1134937275 shutdown_connections 2024-09-10T22:14:15.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.891+0000 7fd87ba00640 1 -- 172.21.15.66:0/1134937275 wait complete. 2024-09-10T22:14:15.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.891+0000 7fd87ba00640 1 Processor -- start 2024-09-10T22:14:15.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd87ba00640 1 -- start start 2024-09-10T22:14:15.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd87ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd85c003ae0 0x7fd874154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd87ba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8741faec0 0x7fd8740adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd87ba00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd874159090 0x7fd87414d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd87ba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741667f0 con 0x7fd85c003ae0 2024-09-10T22:14:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd87ba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd874166970 con 0x7fd8741faec0 2024-09-10T22:14:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd87ba00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd874166af0 con 0x7fd874159090 2024-09-10T22:14:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd879000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd85c003ae0 0x7fd874154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd879000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd85c003ae0 0x7fd874154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53038/0 (socket says 172.21.15.66:53038) 2024-09-10T22:14:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd879000640 1 -- 172.21.15.66:0/28879793 learned_addr learned my addr 172.21.15.66:0/28879793 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd879000640 1 -- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd874159090 msgr2=0x7fd87414d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd879a00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd874159090 0x7fd87414d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd873e00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8741faec0 0x7fd8740adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:15.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd879000640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd874159090 0x7fd87414d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd879000640 1 -- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8741faec0 msgr2=0x7fd8740adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:15.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd879000640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8741faec0 0x7fd8740adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:15.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd879000640 1 -- 172.21.15.66:0/28879793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8741fb350 con 0x7fd85c003ae0 2024-09-10T22:14:15.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd879a00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd874159090 0x7fd87414d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:14:15.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd873e00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8741faec0 0x7fd8740adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:14:15.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd879000640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd85c003ae0 0x7fd874154f30 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fd86801c9c0 tx=0x7fd868006c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:15.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd870c00640 1 -- 172.21.15.66:0/28879793 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868001440 con 0x7fd85c003ae0 2024-09-10T22:14:15.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd870c00640 1 -- 172.21.15.66:0/28879793 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8680015e0 con 0x7fd85c003ae0 2024-09-10T22:14:15.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd870c00640 1 -- 172.21.15.66:0/28879793 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8680064e0 con 0x7fd85c003ae0 2024-09-10T22:14:15.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd87ba00640 1 -- 172.21.15.66:0/28879793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8741fb630 con 0x7fd85c003ae0 2024-09-10T22:14:15.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd87ba00640 1 -- 172.21.15.66:0/28879793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8741fbbf0 con 0x7fd85c003ae0 2024-09-10T22:14:15.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.895+0000 7fd87ba00640 1 -- 172.21.15.66:0/28879793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8740c4ef0 con 0x7fd85c003ae0 2024-09-10T22:14:15.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.899+0000 7fd870c00640 1 -- 172.21.15.66:0/28879793 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fd8680c1030 con 0x7fd85c003ae0 2024-09-10T22:14:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.899+0000 7fd870c00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd854074710 0x7fd854076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.899+0000 7fd870c00640 1 -- 172.21.15.66:0/28879793 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5878+0+0 (secure 0 0 0) 0x7fd86812ba50 con 0x7fd85c003ae0 2024-09-10T22:14:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.899+0000 7fd873e00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd854074710 0x7fd854076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.899+0000 7fd873e00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd854074710 0x7fd854076bd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd860000c10 tx=0x7fd860006b80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:15.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:15.903+0000 7fd870c00640 1 -- 172.21.15.66:0/28879793 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd868016cb0 con 0x7fd85c003ae0 2024-09-10T22:14:16.099 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-09-10T22:14:16.095+0000 7f1e7fa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:14:16.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.139+0000 7fd87ba00640 1 -- 172.21.15.66:0/28879793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd87406c7a0 con 0x7fd85c003ae0 2024-09-10T22:14:16.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.139+0000 7fd870c00640 1 -- 172.21.15.66:0/28879793 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v35) ==== 190+0+147 (secure 0 0 0) 0x7fd8680f8b90 con 0x7fd85c003ae0 2024-09-10T22:14:16.146 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T22:14:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQA/weBmVY7fIBAA7mxJekpI3vmyIs4ov62EBw== 2024-09-10T22:14:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: caps mds = "allow" 2024-09-10T22:14:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: caps mon = "allow r" 2024-09-10T22:14:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T22:14:16.147 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-09-10T22:14:16.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.143+0000 7fd87ba00640 1 -- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd854074710 msgr2=0x7fd854076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:16.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.143+0000 7fd87ba00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd854074710 0x7fd854076bd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd860000c10 tx=0x7fd860006b80 comp rx=0 tx=0).stop 2024-09-10T22:14:16.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.143+0000 7fd87ba00640 1 -- 172.21.15.66:0/28879793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd85c003ae0 msgr2=0x7fd874154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:16.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.143+0000 7fd87ba00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd85c003ae0 0x7fd874154f30 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fd86801c9c0 tx=0x7fd868006c70 comp rx=0 tx=0).stop 2024-09-10T22:14:16.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.143+0000 7fd87ba00640 1 -- 172.21.15.66:0/28879793 shutdown_connections 2024-09-10T22:14:16.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.143+0000 7fd87ba00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd854074710 0x7fd854076bd0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:16.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.143+0000 7fd87ba00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd874159090 0x7fd87414d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:16.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.143+0000 7fd87ba00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8741faec0 0x7fd8740adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:16.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.143+0000 7fd87ba00640 1 --2- 172.21.15.66:0/28879793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd85c003ae0 0x7fd874154f30 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:16.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.143+0000 7fd87ba00640 1 -- 172.21.15.66:0/28879793 >> 172.21.15.66:0/28879793 conn(0x7fd8740cfa30 msgr2=0x7fd87415da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:16.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.143+0000 7fd87ba00640 1 -- 172.21.15.66:0/28879793 shutdown_connections 2024-09-10T22:14:16.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.143+0000 7fd87ba00640 1 -- 172.21.15.66:0/28879793 wait complete. 2024-09-10T22:14:16.170 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:14:16.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.311+0000 7fb21a600640 1 -- 172.21.15.66:0/116254423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2140d0550 msgr2=0x7fb2140c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:16.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.311+0000 7fb21a600640 1 --2- 172.21.15.66:0/116254423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2140d0550 0x7fb2140c4df0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009900 tx=0x7fb1fc01c850 comp rx=0 tx=0).stop 2024-09-10T22:14:16.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.311+0000 7fb21a600640 1 -- 172.21.15.66:0/116254423 shutdown_connections 2024-09-10T22:14:16.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.311+0000 7fb21a600640 1 --2- 172.21.15.66:0/116254423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2140d0550 0x7fb2140c4df0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:16.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.311+0000 7fb21a600640 1 --2- 172.21.15.66:0/116254423 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb21415a310 0x7fb2140c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:16.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.311+0000 7fb21a600640 1 -- 172.21.15.66:0/116254423 >> 172.21.15.66:0/116254423 conn(0x7fb2140cfa30 msgr2=0x7fb2140cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:16.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.311+0000 7fb21a600640 1 -- 172.21.15.66:0/116254423 shutdown_connections 2024-09-10T22:14:16.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.311+0000 7fb21a600640 1 -- 172.21.15.66:0/116254423 wait complete. 2024-09-10T22:14:16.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb21a600640 1 Processor -- start 2024-09-10T22:14:16.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb21a600640 1 -- start start 2024-09-10T22:14:16.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb21a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2140aeec0 0x7fb2140743d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:16.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb21a600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb21415a310 0x7fb2140c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:16.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb21a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2140c7a50 0x7fb21414b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:16.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb21a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb214167300 con 0x7fb2140aeec0 2024-09-10T22:14:16.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb21a600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb21406ebf0 con 0x7fb2140c7a50 2024-09-10T22:14:16.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb21a600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb214167480 con 0x7fb21415a310 2024-09-10T22:14:16.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb213400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2140aeec0 0x7fb2140743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:16.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb213400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2140aeec0 0x7fb2140743d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53058/0 (socket says 172.21.15.66:53058) 2024-09-10T22:14:16.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb213400640 1 -- 172.21.15.66:0/4229439145 learned_addr learned my addr 172.21.15.66:0/4229439145 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:16.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb20aa00640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb21415a310 0x7fb2140c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:16.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb213e00640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2140c7a50 0x7fb21414b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:16.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb213400640 1 -- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb21415a310 msgr2=0x7fb2140c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:16.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb213400640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb21415a310 0x7fb2140c7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:16.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb213400640 1 -- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2140c7a50 msgr2=0x7fb21414b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:16.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb213400640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2140c7a50 0x7fb21414b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:16.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb213400640 1 -- 172.21.15.66:0/4229439145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2140c7f40 con 0x7fb2140aeec0 2024-09-10T22:14:16.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb213e00640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2140c7a50 0x7fb21414b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:14:16.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb213400640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2140aeec0 0x7fb2140743d0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb20400b440 tx=0x7fb20400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:16.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb20be00640 1 -- 172.21.15.66:0/4229439145 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204015020 con 0x7fb2140aeec0 2024-09-10T22:14:16.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb20be00640 1 -- 172.21.15.66:0/4229439145 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2040119a0 con 0x7fb2140aeec0 2024-09-10T22:14:16.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb21a600640 1 -- 172.21.15.66:0/4229439145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2140c8220 con 0x7fb2140aeec0 2024-09-10T22:14:16.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb20be00640 1 -- 172.21.15.66:0/4229439145 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204011df0 con 0x7fb2140aeec0 2024-09-10T22:14:16.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.315+0000 7fb21a600640 1 -- 172.21.15.66:0/4229439145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2140c87e0 con 0x7fb2140aeec0 2024-09-10T22:14:16.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.319+0000 7fb21a600640 1 -- 172.21.15.66:0/4229439145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb21415df60 con 0x7fb2140aeec0 2024-09-10T22:14:16.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.319+0000 7fb20be00640 1 -- 172.21.15.66:0/4229439145 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fb20401a400 con 0x7fb2140aeec0 2024-09-10T22:14:16.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.319+0000 7fb20be00640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1f0074710 0x7fb1f0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:16.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.319+0000 7fb20be00640 1 -- 172.21.15.66:0/4229439145 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7fb204092770 con 0x7fb2140aeec0 2024-09-10T22:14:16.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.319+0000 7fb20aa00640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1f0074710 0x7fb1f0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:16.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.323+0000 7fb20aa00640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1f0074710 0x7fb1f0076bd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0ae000 tx=0x7fb1fc01b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:16.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.323+0000 7fb20be00640 1 -- 172.21.15.66:0/4229439145 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb20401f020 con 0x7fb2140aeec0 2024-09-10T22:14:16.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.535+0000 7fb21a600640 1 -- 172.21.15.66:0/4229439145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb214072640 con 0x7fb2140aeec0 2024-09-10T22:14:16.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.535+0000 7fb20be00640 1 -- 172.21.15.66:0/4229439145 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5221 (secure 0 0 0) 0x7fb204064e20 con 0x7fb2140aeec0 2024-09-10T22:14:16.541 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:14:16.541 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":96,"btime":"2024-09-10T22:14:14:150270+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/401729631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":401729631},{"type":"v1","addr":"172.21.15.94:6836","nonce":401729631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/996427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":996427549},{"type":"v1","addr":"172.21.15.94:6837","nonce":996427549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2947602030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2947602030},{"type":"v1","addr":"172.21.15.66:6833","nonce":2947602030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:14:10.533890+0000","modified":"2024-09-10T22:14:13.552506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/3920857628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3920857628},{"type":"v1","addr":"172.21.15.66:6835","nonce":3920857628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9282,"qdb_cluster":[9282]},"id":10}]} 2024-09-10T22:14:16.542 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 96 2024-09-10T22:14:16.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.539+0000 7fb21a600640 1 -- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1f0074710 msgr2=0x7fb1f0076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:16.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.539+0000 7fb21a600640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1f0074710 0x7fb1f0076bd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0ae000 tx=0x7fb1fc01b090 comp rx=0 tx=0).stop 2024-09-10T22:14:16.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.539+0000 7fb21a600640 1 -- 172.21.15.66:0/4229439145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2140aeec0 msgr2=0x7fb2140743d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.539+0000 7fb21a600640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2140aeec0 0x7fb2140743d0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb20400b440 tx=0x7fb20400b900 comp rx=0 tx=0).stop 2024-09-10T22:14:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.539+0000 7fb21a600640 1 -- 172.21.15.66:0/4229439145 shutdown_connections 2024-09-10T22:14:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.539+0000 7fb21a600640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb1f0074710 0x7fb1f0076bd0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.539+0000 7fb21a600640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2140c7a50 0x7fb21414b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.539+0000 7fb21a600640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb21415a310 0x7fb2140c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.539+0000 7fb21a600640 1 --2- 172.21.15.66:0/4229439145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2140aeec0 0x7fb2140743d0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.539+0000 7fb21a600640 1 -- 172.21.15.66:0/4229439145 >> 172.21.15.66:0/4229439145 conn(0x7fb2140cfa30 msgr2=0x7fb2140ca130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.539+0000 7fb21a600640 1 -- 172.21.15.66:0/4229439145 shutdown_connections 2024-09-10T22:14:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:16.539+0000 7fb21a600640 1 -- 172.21.15.66:0/4229439145 wait complete. 2024-09-10T22:14:16.966 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:14:10.533890+0000', 'modified': '2024-09-10T22:14:13.552506+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9282': {'gid': 9282, 'name': 'c', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/3920857628', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 3920857628}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 3920857628}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9282, 'qdb_cluster': [9282]} 2024-09-10T22:14:16.967 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:14:16.967 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T22:14:17.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.111+0000 7f0662400640 1 -- 172.21.15.66:0/957356539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065c15a310 msgr2=0x7f065c0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.111+0000 7f0662400640 1 --2- 172.21.15.66:0/957356539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065c15a310 0x7f065c0c48b0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f065c0af550 tx=0x7f064c01daa0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.111+0000 7f0662400640 1 -- 172.21.15.66:0/957356539 shutdown_connections 2024-09-10T22:14:17.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.111+0000 7f0662400640 1 --2- 172.21.15.66:0/957356539 >> v2:172.21.15.94:3300/0 conn(0x7f065c0aeec0 0x7f065c0c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.111+0000 7f0662400640 1 --2- 172.21.15.66:0/957356539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065c15a310 0x7f065c0c48b0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.111+0000 7f0662400640 1 -- 172.21.15.66:0/957356539 >> 172.21.15.66:0/957356539 conn(0x7f065c0cfa30 msgr2=0x7f065c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:17.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.111+0000 7f0662400640 1 -- 172.21.15.66:0/957356539 shutdown_connections 2024-09-10T22:14:17.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.111+0000 7f0662400640 1 -- 172.21.15.66:0/957356539 wait complete. 2024-09-10T22:14:17.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0662400640 1 Processor -- start 2024-09-10T22:14:17.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0662400640 1 -- start start 2024-09-10T22:14:17.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0662400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f065c0aeec0 0x7f065c0c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0662400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065c15a310 0x7f065c0c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0662400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f065c14dfc0 0x7f065c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0662400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f065c15edb0 con 0x7f065c15a310 2024-09-10T22:14:17.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0662400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f065c15ec30 con 0x7f065c14dfc0 2024-09-10T22:14:17.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0662400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f065c167300 con 0x7f065c0aeec0 2024-09-10T22:14:17.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f065aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065c15a310 0x7f065c0c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f065aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065c15a310 0x7f065c0c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53086/0 (socket says 172.21.15.66:53086) 2024-09-10T22:14:17.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f065aa00640 1 -- 172.21.15.66:0/1659010714 learned_addr learned my addr 172.21.15.66:0/1659010714 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:17.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f065aa00640 1 -- 172.21.15.66:0/1659010714 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f065c0aeec0 msgr2=0x7f065c0c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f065aa00640 1 --2- 172.21.15.66:0/1659010714 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f065c0aeec0 0x7f065c0c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f065aa00640 1 -- 172.21.15.66:0/1659010714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f065c14dfc0 msgr2=0x7f065c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f065aa00640 1 --2- 172.21.15.66:0/1659010714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f065c14dfc0 0x7f065c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f065aa00640 1 -- 172.21.15.66:0/1659010714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c1fb700 con 0x7f065c15a310 2024-09-10T22:14:17.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f065aa00640 1 --2- 172.21.15.66:0/1659010714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065c15a310 0x7f065c0c7b50 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f064800c490 tx=0x7f064800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:17.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0653400640 1 -- 172.21.15.66:0/1659010714 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648016020 con 0x7f065c15a310 2024-09-10T22:14:17.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0653400640 1 -- 172.21.15.66:0/1659010714 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0648012940 con 0x7f065c15a310 2024-09-10T22:14:17.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0653400640 1 -- 172.21.15.66:0/1659010714 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648012d90 con 0x7f065c15a310 2024-09-10T22:14:17.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0662400640 1 -- 172.21.15.66:0/1659010714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c1fb9e0 con 0x7f065c15a310 2024-09-10T22:14:17.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.115+0000 7f0662400640 1 -- 172.21.15.66:0/1659010714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c1fbfa0 con 0x7f065c15a310 2024-09-10T22:14:17.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f0653400640 1 -- 172.21.15.66:0/1659010714 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f064801b400 con 0x7f065c15a310 2024-09-10T22:14:17.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f0653400640 1 --2- 172.21.15.66:0/1659010714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f063c074740 0x7f063c076c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f0653400640 1 -- 172.21.15.66:0/1659010714 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f0648092600 con 0x7f065c15a310 2024-09-10T22:14:17.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f0665899000 1 Processor -- start 2024-09-10T22:14:17.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f0665899000 1 -- start start 2024-09-10T22:14:17.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065b400640 1 --2- 172.21.15.66:0/1659010714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f063c074740 0x7f063c076c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f0665899000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 0x55a9d36dadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f0665899000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36db2f0 0x55a9d368e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f0665899000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d368ee10 0x55a9d368f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f0665899000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9d35af3f0 con 0x55a9d36da9d0 2024-09-10T22:14:17.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f0665899000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a9d34ecb20 con 0x55a9d368ee10 2024-09-10T22:14:17.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f0665899000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55a9d34dbf50 con 0x55a9d36db2f0 2024-09-10T22:14:17.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 0x55a9d36dadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 0x55a9d36dadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53102/0 (socket says 172.21.15.66:53102) 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065b400640 1 -- 172.21.15.66:0/1621255549 learned_addr learned my addr 172.21.15.66:0/1621255549 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065b400640 1 --2- 172.21.15.66:0/1659010714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f063c074740 0x7f063c076c00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f064c0b0000 tx=0x7f064c01a840 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065be00640 1 --2- 172.21.15.66:0/1621255549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d368ee10 0x55a9d368f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065aa00640 1 --2- 172.21.15.66:0/1621255549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36db2f0 0x55a9d368e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065b400640 1 -- 172.21.15.66:0/1621255549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36db2f0 msgr2=0x55a9d368e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065b400640 1 --2- 172.21.15.66:0/1621255549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36db2f0 0x55a9d368e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065b400640 1 -- 172.21.15.66:0/1621255549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d368ee10 msgr2=0x55a9d368f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065b400640 1 --2- 172.21.15.66:0/1621255549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d368ee10 0x55a9d368f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065b400640 1 -- 172.21.15.66:0/1621255549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d34a3590 con 0x55a9d36da9d0 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f065b400640 1 --2- 172.21.15.66:0/1621255549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 0x55a9d36dadb0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f064c00a8a0 tx=0x7f064c0b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b9a3f7fc93b1c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.119+0000 7f0647e00640 1 -- 172.21.15.66:0/1621255549 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c0c7070 con 0x55a9d36da9d0 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0647e00640 1 -- 172.21.15.66:0/1621255549 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f064c0a7ac0 con 0x55a9d36da9d0 2024-09-10T22:14:17.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0647e00640 1 -- 172.21.15.66:0/1621255549 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c0b3040 con 0x55a9d36da9d0 2024-09-10T22:14:17.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 -- 172.21.15.66:0/1621255549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 msgr2=0x55a9d36dadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 --2- 172.21.15.66:0/1621255549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 0x55a9d36dadb0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f064c00a8a0 tx=0x7f064c0b4000 comp rx=0 tx=0).stop 2024-09-10T22:14:17.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 -- 172.21.15.66:0/1621255549 shutdown_connections 2024-09-10T22:14:17.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 --2- 172.21.15.66:0/1621255549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d368ee10 0x55a9d368f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 --2- 172.21.15.66:0/1621255549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36db2f0 0x55a9d368e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 --2- 172.21.15.66:0/1621255549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 0x55a9d36dadb0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 -- 172.21.15.66:0/1621255549 >> 172.21.15.66:0/1621255549 conn(0x55a9d3578130 msgr2=0x55a9d36bde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:17.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 -- 172.21.15.66:0/1621255549 shutdown_connections 2024-09-10T22:14:17.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 -- 172.21.15.66:0/1621255549 wait complete. 2024-09-10T22:14:17.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 -- start start 2024-09-10T22:14:17.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 Processor -- start 2024-09-10T22:14:17.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d368ee10 0x55a9d36bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36da9d0 0x55a9d36bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d36db2f0 0x55a9d36bc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9d34765d0 con 0x55a9d368ee10 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a9d34dfd60 con 0x55a9d36db2f0 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55a9d3482500 con 0x55a9d36da9d0 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 10 client.0 _ll_register_callbacks cb 0x55a9d3486810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d368ee10 0x55a9d36bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d368ee10 0x55a9d36bb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53108/0 (socket says 172.21.15.66:53108) 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065b400640 1 -- 172.21.15.66:0/4094343487 learned_addr learned my addr 172.21.15.66:0/4094343487 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065aa00640 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36da9d0 0x55a9d36bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065be00640 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d36db2f0 0x55a9d36bc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065b400640 1 -- 172.21.15.66:0/4094343487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36da9d0 msgr2=0x55a9d36bbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:17.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065b400640 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36da9d0 0x55a9d36bbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065b400640 1 -- 172.21.15.66:0/4094343487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d36db2f0 msgr2=0x55a9d36bc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065b400640 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d36db2f0 0x55a9d36bc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065b400640 1 -- 172.21.15.66:0/4094343487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d347a170 con 0x55a9d368ee10 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065aa00640 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36da9d0 0x55a9d36bbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065be00640 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d36db2f0 0x55a9d36bc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f065b400640 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d368ee10 0x55a9d36bb6a0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f064c01f180 tx=0x7f064c0a8860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e4e4eef3522ded server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0644c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0644c00640 1 -- 172.21.15.66:0/4094343487 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c0b3070 con 0x55a9d368ee10 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0665899000 1 -- 172.21.15.66:0/4094343487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9d34740f0 con 0x55a9d368ee10 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.123+0000 7f0644c00640 1 -- 172.21.15.66:0/4094343487 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f064c0a8d10 con 0x55a9d368ee10 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0644c00640 1 -- 172.21.15.66:0/4094343487 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f064c0d4050 con 0x55a9d368ee10 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 10 client.9402 fetch_fsmap learned FSMap version 96 2024-09-10T22:14:17.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 -- 172.21.15.66:0/4094343487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55a9d3560ef0 con 0x55a9d368ee10 2024-09-10T22:14:17.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0644c00640 1 -- 172.21.15.66:0/4094343487 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a9d34740f0 con 0x55a9d368ee10 2024-09-10T22:14:17.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0644c00640 1 -- 172.21.15.66:0/4094343487 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 96) ==== 3502+0+0 (secure 0 0 0) 0x7f064c0d7020 con 0x55a9d368ee10 2024-09-10T22:14:17.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 10 client.9402 fetch_fsmap finished waiting for FSMap version 96 2024-09-10T22:14:17.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 10 client.9402 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:14:17.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 20 client.9402 populate_metadata read hostname 'smithi066' 2024-09-10T22:14:17.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.66:6834/3920857628,v1:172.21.15.66:6835/3920857628] conn(0x55a9d36bd0f0 0x55a9d36ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 4 client.9402 mds_command: new command op to 9282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:14:17.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 -- 172.21.15.66:0/4094343487 --> [v2:172.21.15.66:6834/3920857628,v1:172.21.15.66:6835/3920857628] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9d34eea70 con 0x55a9d36bd0f0 2024-09-10T22:14:17.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f065aa00640 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.66:6834/3920857628,v1:172.21.15.66:6835/3920857628] conn(0x55a9d36bd0f0 0x55a9d36ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0644c00640 10 client.9402 ms_handle_connect on v2:172.21.15.66:6834/3920857628 2024-09-10T22:14:17.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f065aa00640 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.66:6834/3920857628,v1:172.21.15.66:6835/3920857628] conn(0x55a9d36bd0f0 0x55a9d36ecc90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23c30dd8a6787210 server_cookie=5f7164e01d6772e5 in_seq=0 out_seq=0 2024-09-10T22:14:17.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0644c00640 1 -- 172.21.15.66:0/4094343487 <== mds.0 v2:172.21.15.66:6834/3920857628 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a9d34eea70 con 0x55a9d36bd0f0 2024-09-10T22:14:17.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0644c00640 10 client.9402 handle_command_reply: tid=0 2024-09-10T22:14:17.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 client.9402 shutdown 2024-09-10T22:14:17.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 -- 172.21.15.66:0/4094343487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d368ee10 msgr2=0x55a9d36bb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d368ee10 0x55a9d36bb6a0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f064c01f180 tx=0x7f064c0a8860 comp rx=0 tx=0).stop 2024-09-10T22:14:17.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 -- 172.21.15.66:0/4094343487 shutdown_connections 2024-09-10T22:14:17.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.66:6834/3920857628,v1:172.21.15.66:6835/3920857628] conn(0x55a9d36bd0f0 0x55a9d36ecc90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d36db2f0 0x55a9d36bc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36da9d0 0x55a9d36bbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 --2- 172.21.15.66:0/4094343487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d368ee10 0x55a9d36bb6a0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 -- 172.21.15.66:0/4094343487 >> 172.21.15.66:0/4094343487 conn(0x55a9d3578130 msgr2=0x55a9d368dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:17.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0644c00640 0 client.9402 ms_handle_reset on v2:172.21.15.66:6834/3920857628 2024-09-10T22:14:17.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 -- 172.21.15.66:0/4094343487 shutdown_connections 2024-09-10T22:14:17.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 1 -- 172.21.15.66:0/4094343487 wait complete. 2024-09-10T22:14:17.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.127+0000 7f0665899000 20 client.9402 trim_cache size 0 max 16384 2024-09-10T22:14:17.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f0665899000 1 Processor -- start 2024-09-10T22:14:17.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f0665899000 1 -- start start 2024-09-10T22:14:17.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f0665899000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d368ee10 0x55a9d374cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f0665899000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 0x55a9d374d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f0665899000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36db2f0 0x55a9d374d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f0665899000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f065c204390 con 0x55a9d36da9d0 2024-09-10T22:14:17.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f0665899000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f065c0b35e0 con 0x55a9d368ee10 2024-09-10T22:14:17.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f0665899000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f065c0b2940 con 0x55a9d36db2f0 2024-09-10T22:14:17.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f065aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 0x55a9d374d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f065aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 0x55a9d374d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53114/0 (socket says 172.21.15.66:53114) 2024-09-10T22:14:17.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f065aa00640 1 -- 172.21.15.66:0/1394467548 learned_addr learned my addr 172.21.15.66:0/1394467548 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:17.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f065b400640 1 --2- 172.21.15.66:0/1394467548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d368ee10 0x55a9d374cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f065be00640 1 --2- 172.21.15.66:0/1394467548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36db2f0 0x55a9d374d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f065aa00640 1 -- 172.21.15.66:0/1394467548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36db2f0 msgr2=0x55a9d374d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:17.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f065aa00640 1 --2- 172.21.15.66:0/1394467548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36db2f0 0x55a9d374d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f065aa00640 1 -- 172.21.15.66:0/1394467548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d368ee10 msgr2=0x55a9d374cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:17.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f065aa00640 1 --2- 172.21.15.66:0/1394467548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d368ee10 0x55a9d374cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f065aa00640 1 -- 172.21.15.66:0/1394467548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d3537d90 con 0x55a9d36da9d0 2024-09-10T22:14:17.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f065aa00640 1 --2- 172.21.15.66:0/1394467548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 0x55a9d374d310 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f064800ce50 tx=0x7f06480929b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1b9e353ff29488 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:17.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f0647400640 1 -- 172.21.15.66:0/1394467548 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648090070 con 0x55a9d36da9d0 2024-09-10T22:14:17.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f0647400640 1 -- 172.21.15.66:0/1394467548 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0648068600 con 0x55a9d36da9d0 2024-09-10T22:14:17.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.147+0000 7f0647400640 1 -- 172.21.15.66:0/1394467548 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064805b370 con 0x55a9d36da9d0 2024-09-10T22:14:17.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 -- 172.21.15.66:0/1394467548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 msgr2=0x55a9d374d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 --2- 172.21.15.66:0/1394467548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 0x55a9d374d310 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f064800ce50 tx=0x7f06480929b0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 -- 172.21.15.66:0/1394467548 shutdown_connections 2024-09-10T22:14:17.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 --2- 172.21.15.66:0/1394467548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d36db2f0 0x55a9d374d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 --2- 172.21.15.66:0/1394467548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36da9d0 0x55a9d374d310 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 --2- 172.21.15.66:0/1394467548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d368ee10 0x55a9d374cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 -- 172.21.15.66:0/1394467548 >> 172.21.15.66:0/1394467548 conn(0x55a9d3578130 msgr2=0x55a9d3696ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:17.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 -- 172.21.15.66:0/1394467548 shutdown_connections 2024-09-10T22:14:17.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 -- 172.21.15.66:0/1394467548 wait complete. 2024-09-10T22:14:17.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 -- start start 2024-09-10T22:14:17.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 Processor -- start 2024-09-10T22:14:17.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d368ee10 0x55a9d36ee790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d36da9d0 0x55a9d36eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36db2f0 0x55a9d36ef210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9d34a7ec0 con 0x55a9d36db2f0 2024-09-10T22:14:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f065c0b2f90 con 0x55a9d36da9d0 2024-09-10T22:14:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55a9d34765d0 con 0x55a9d368ee10 2024-09-10T22:14:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 10 client.0 _ll_register_callbacks cb 0x7f064c0c5070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:14:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f065be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36db2f0 0x55a9d36ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f065be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36db2f0 0x55a9d36ef210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53116/0 (socket says 172.21.15.66:53116) 2024-09-10T22:14:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f065be00640 1 -- 172.21.15.66:0/748273221 learned_addr learned my addr 172.21.15.66:0/748273221 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:17.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f065b400640 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d368ee10 0x55a9d36ee790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f065aa00640 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d36da9d0 0x55a9d36eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f065be00640 1 -- 172.21.15.66:0/748273221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d368ee10 msgr2=0x55a9d36ee790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:17.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f065be00640 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d368ee10 0x55a9d36ee790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f065be00640 1 -- 172.21.15.66:0/748273221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d36da9d0 msgr2=0x55a9d36eecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:17.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f065be00640 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d36da9d0 0x55a9d36eecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f065be00640 1 -- 172.21.15.66:0/748273221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d34a2a90 con 0x55a9d36db2f0 2024-09-10T22:14:17.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f065be00640 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36db2f0 0x55a9d36ef210 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f0654013a20 tx=0x7f0654013ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4764688504839546 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:17.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0644c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:14:17.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0644c00640 1 -- 172.21.15.66:0/748273221 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654021060 con 0x55a9d36db2f0 2024-09-10T22:14:17.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0644c00640 1 -- 172.21.15.66:0/748273221 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f0654011cb0 con 0x55a9d36db2f0 2024-09-10T22:14:17.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 -- 172.21.15.66:0/748273221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0648068600 con 0x55a9d36db2f0 2024-09-10T22:14:17.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0644c00640 1 -- 172.21.15.66:0/748273221 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f0654027050 con 0x55a9d36db2f0 2024-09-10T22:14:17.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 10 client.9408 fetch_fsmap learned FSMap version 96 2024-09-10T22:14:17.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.151+0000 7f0665899000 1 -- 172.21.15.66:0/748273221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55a9d3481050 con 0x55a9d36db2f0 2024-09-10T22:14:17.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0644c00640 1 -- 172.21.15.66:0/748273221 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0648068600 con 0x55a9d36db2f0 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0644c00640 1 -- 172.21.15.66:0/748273221 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 96) ==== 3502+0+0 (secure 0 0 0) 0x7f065402a020 con 0x55a9d36db2f0 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 10 client.9408 fetch_fsmap finished waiting for FSMap version 96 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 10 client.9408 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 20 client.9408 populate_metadata read hostname 'smithi066' 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.66:6834/3920857628,v1:172.21.15.66:6835/3920857628] conn(0x55a9d3756e40 0x55a9d3759230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 4 client.9408 mds_command: new command op to 9282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 -- 172.21.15.66:0/748273221 --> [v2:172.21.15.66:6834/3920857628,v1:172.21.15.66:6835/3920857628] -- command(tid 0: {"prefix": "status"}) -- 0x55a9d34ea520 con 0x55a9d3756e40 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f065b400640 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.66:6834/3920857628,v1:172.21.15.66:6835/3920857628] conn(0x55a9d3756e40 0x55a9d3759230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f065b400640 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.66:6834/3920857628,v1:172.21.15.66:6835/3920857628] conn(0x55a9d3756e40 0x55a9d3759230 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736dd4c991a45d64 server_cookie=6cee32a5e0c74df7 in_seq=0 out_seq=0 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0644c00640 10 client.9408 ms_handle_connect on v2:172.21.15.66:6834/3920857628 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0644c00640 1 -- 172.21.15.66:0/748273221 <== mds.0 v2:172.21.15.66:6834/3920857628 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a9d34ea520 con 0x55a9d3756e40 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0644c00640 10 client.9408 handle_command_reply: tid=0 2024-09-10T22:14:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 client.9408 shutdown 2024-09-10T22:14:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 -- 172.21.15.66:0/748273221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36db2f0 msgr2=0x55a9d36ef210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36db2f0 0x55a9d36ef210 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f0654013a20 tx=0x7f0654013ee0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 -- 172.21.15.66:0/748273221 shutdown_connections 2024-09-10T22:14:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.66:6834/3920857628,v1:172.21.15.66:6835/3920857628] conn(0x55a9d3756e40 0x55a9d3759230 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9d36db2f0 0x55a9d36ef210 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d36da9d0 0x55a9d36eecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 --2- 172.21.15.66:0/748273221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a9d368ee10 0x55a9d36ee790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0644c00640 0 client.9408 ms_handle_reset on v2:172.21.15.66:6834/3920857628 2024-09-10T22:14:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 -- 172.21.15.66:0/748273221 >> 172.21.15.66:0/748273221 conn(0x55a9d3578130 msgr2=0x55a9d3696ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 -- 172.21.15.66:0/748273221 shutdown_connections 2024-09-10T22:14:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 1 -- 172.21.15.66:0/748273221 wait complete. 2024-09-10T22:14:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0665899000 20 client.9408 trim_cache size 0 max 16384 2024-09-10T22:14:17.168 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:14:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:14:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:14:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 9282, 2024-09-10T22:14:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:14:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:14:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:14:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 6.0219482280000003, 2024-09-10T22:14:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 96, 2024-09-10T22:14:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 143, 2024-09-10T22:14:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 141, 2024-09-10T22:14:17.169 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 11.565066217 2024-09-10T22:14:17.169 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:14:17.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0662400640 1 -- 172.21.15.66:0/1659010714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f063c074740 msgr2=0x7f063c076c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0662400640 1 --2- 172.21.15.66:0/1659010714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f063c074740 0x7f063c076c00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f064c0b0000 tx=0x7f064c01a840 comp rx=0 tx=0).stop 2024-09-10T22:14:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0662400640 1 -- 172.21.15.66:0/1659010714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065c15a310 msgr2=0x7f065c0c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0662400640 1 --2- 172.21.15.66:0/1659010714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065c15a310 0x7f065c0c7b50 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f064800c490 tx=0x7f064800c950 comp rx=0 tx=0).stop 2024-09-10T22:14:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0662400640 1 -- 172.21.15.66:0/1659010714 shutdown_connections 2024-09-10T22:14:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0662400640 1 --2- 172.21.15.66:0/1659010714 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f063c074740 0x7f063c076c00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0662400640 1 --2- 172.21.15.66:0/1659010714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f065c14dfc0 0x7f065c0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0662400640 1 --2- 172.21.15.66:0/1659010714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065c15a310 0x7f065c0c7b50 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0662400640 1 --2- 172.21.15.66:0/1659010714 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f065c0aeec0 0x7f065c0c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0662400640 1 -- 172.21.15.66:0/1659010714 >> 172.21.15.66:0/1659010714 conn(0x7f065c0cfa30 msgr2=0x7f065c14e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0662400640 1 -- 172.21.15.66:0/1659010714 shutdown_connections 2024-09-10T22:14:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.155+0000 7f0662400640 1 -- 172.21.15.66:0/1659010714 wait complete. 2024-09-10T22:14:17.182 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:14:17.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.323+0000 7f503cc00640 1 -- 172.21.15.66:0/3916594077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c004810 msgr2=0x7f501c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.323+0000 7f503cc00640 1 --2- 172.21.15.66:0/3916594077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c004810 0x7f501c004c90 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f5024009b60 tx=0x7f502401d270 comp rx=0 tx=0).stop 2024-09-10T22:14:17.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 -- 172.21.15.66:0/3916594077 shutdown_connections 2024-09-10T22:14:17.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 --2- 172.21.15.66:0/3916594077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c004810 0x7f501c004c90 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 -- 172.21.15.66:0/3916594077 >> 172.21.15.66:0/3916594077 conn(0x7f50380d1c50 msgr2=0x7f50380d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 -- 172.21.15.66:0/3916594077 shutdown_connections 2024-09-10T22:14:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 -- 172.21.15.66:0/3916594077 wait complete. 2024-09-10T22:14:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 Processor -- start 2024-09-10T22:14:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 -- start start 2024-09-10T22:14:17.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f501c004810 0x7f50380b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50380cb200 0x7f503815d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50380cb680 0x7f5038155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f50380d28f0 con 0x7f50380cb200 2024-09-10T22:14:17.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50380d2770 con 0x7f501c004810 2024-09-10T22:14:17.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f503cc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f50380d2a70 con 0x7f50380cb680 2024-09-10T22:14:17.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f5035400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50380cb200 0x7f503815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f5035400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50380cb200 0x7f503815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53132/0 (socket says 172.21.15.66:53132) 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f5035400640 1 -- 172.21.15.66:0/412161840 learned_addr learned my addr 172.21.15.66:0/412161840 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f5035e00640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f501c004810 0x7f50380b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f5036800640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50380cb680 0x7f5038155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f5035400640 1 -- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50380cb680 msgr2=0x7f5038155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f5035400640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50380cb680 0x7f5038155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f5035400640 1 -- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f501c004810 msgr2=0x7f50380b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f5035400640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f501c004810 0x7f50380b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f5035400640 1 -- 172.21.15.66:0/412161840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50380cedc0 con 0x7f50380cb200 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.327+0000 7f5035400640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50380cb200 0x7f503815d120 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f5024009a80 tx=0x7f50240055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.331+0000 7f502e000640 1 -- 172.21.15.66:0/412161840 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50240b1040 con 0x7f50380cb200 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.331+0000 7f502e000640 1 -- 172.21.15.66:0/412161840 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50240bb070 con 0x7f50380cb200 2024-09-10T22:14:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.331+0000 7f502e000640 1 -- 172.21.15.66:0/412161840 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024004290 con 0x7f50380cb200 2024-09-10T22:14:17.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.331+0000 7f503cc00640 1 -- 172.21.15.66:0/412161840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50380cf0a0 con 0x7f50380cb200 2024-09-10T22:14:17.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.331+0000 7f503cc00640 1 -- 172.21.15.66:0/412161840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50381f9110 con 0x7f50380cb200 2024-09-10T22:14:17.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.331+0000 7f503cc00640 1 -- 172.21.15.66:0/412161840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50380cfc70 con 0x7f50380cb200 2024-09-10T22:14:17.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.331+0000 7f502e000640 1 -- 172.21.15.66:0/412161840 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f50240046d0 con 0x7f50380cb200 2024-09-10T22:14:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.335+0000 7f502e000640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5018074740 0x7f5018076c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.335+0000 7f502e000640 1 -- 172.21.15.66:0/412161840 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f502412d260 con 0x7f50380cb200 2024-09-10T22:14:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.335+0000 7f5035e00640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5018074740 0x7f5018076c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:17.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.335+0000 7f5035e00640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5018074740 0x7f5018076c00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5028001870 tx=0x7f5028001120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:17.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.339+0000 7f502e000640 1 -- 172.21.15.66:0/412161840 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5024016f80 con 0x7f50380cb200 2024-09-10T22:14:17.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.547+0000 7f503cc00640 1 -- 172.21.15.66:0/412161840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f503806c7c0 con 0x7f50380cb200 2024-09-10T22:14:17.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.547+0000 7f502e000640 1 -- 172.21.15.66:0/412161840 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) ==== 53+0+73 (secure 0 0 0) 0x7f50240f9d50 con 0x7f50380cb200 2024-09-10T22:14:17.552 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:14:17.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.551+0000 7f503cc00640 1 -- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5018074740 msgr2=0x7f5018076c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.551+0000 7f503cc00640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5018074740 0x7f5018076c00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5028001870 tx=0x7f5028001120 comp rx=0 tx=0).stop 2024-09-10T22:14:17.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.551+0000 7f503cc00640 1 -- 172.21.15.66:0/412161840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50380cb200 msgr2=0x7f503815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:17.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.551+0000 7f503cc00640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50380cb200 0x7f503815d120 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f5024009a80 tx=0x7f50240055d0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.551+0000 7f503cc00640 1 -- 172.21.15.66:0/412161840 shutdown_connections 2024-09-10T22:14:17.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.551+0000 7f503cc00640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5018074740 0x7f5018076c00 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.551+0000 7f503cc00640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50380cb680 0x7f5038155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.551+0000 7f503cc00640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50380cb200 0x7f503815d120 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.551+0000 7f503cc00640 1 --2- 172.21.15.66:0/412161840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f501c004810 0x7f50380b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:17.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.551+0000 7f503cc00640 1 -- 172.21.15.66:0/412161840 >> 172.21.15.66:0/412161840 conn(0x7f50380d1c50 msgr2=0x7f50380afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:17.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.551+0000 7f503cc00640 1 -- 172.21.15.66:0/412161840 shutdown_connections 2024-09-10T22:14:17.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17.551+0000 7f503cc00640 1 -- 172.21.15.66:0/412161840 wait complete. 2024-09-10T22:14:17.574 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:14:17.574 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:14:17.574 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:14:17.574 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:14:17.574 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:14:17.574 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:14:17.574 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:14:17.575 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:14:17.575 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:14:17.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:17.575 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:14:17.584 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T22:14:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41617sec preferred_lft 41617sec 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:81:fc:4c:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:14:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:14:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T22:14:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:14:17.587 INFO:teuthology.orchestra.run.smithi066.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:14:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:14:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T22:14:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:14:17.587 INFO:teuthology.orchestra.run.smithi066.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:14:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:14:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T22:14:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:14:17.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:17.588 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:14:17.652 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:14:17.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:17.654 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:14:17.713 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:14:17.713 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:14:17.713 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T22:14:17.713 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:14:17.713 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T22:14:17.713 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T22:14:17.713 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:14:17.714 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:14:17.714 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:14:17.714 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T22:14:17.714 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:14:17.714 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:14:17.714 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:14:17.714 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:14:17.714 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:14:17.714 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:14:17.714 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:14:17.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:14:17.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:14:17.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:17.850 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:14:17.859 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:14:17.859 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:14:17.860 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:14:17.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:17.860 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:14:17.922 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T22:14:17.922 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:14:17.922 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:14:17.973 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:14:17.974 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:14:17.974 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:14:18.025 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:14:18.090 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:14:18.147 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:14:18.147 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:14:18.147 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:14:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:14:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:14:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:14:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:14:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:14:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:14:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:14:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:14:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:14:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:14:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:14:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:14:18.149 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:14:18.150 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:14:18.150 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:14:18.150 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:14:18.150 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:14:18.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:18.151 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:14:18.202 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:14:18.202 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T22:14:18.248 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-09-10T22:14:18.269 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:14:18.323 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-09-10T22:14:18.323 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-09-10T22:14:18.323 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-09-10T22:14:18.323 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-09-10T22:14:18.323 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-09-10T22:14:18.323 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/run/lock 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/fs/nfsd 2024-09-10T22:14:18.324 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-09-10T22:14:18.325 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-09-10T22:14:18.325 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T22:14:18.325 INFO:teuthology.orchestra.run.smithi066.stdout:/run/rpc_pipefs 2024-09-10T22:14:18.325 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-09-10T22:14:18.325 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-09-10T22:14:18.325 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T22:14:18.325 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T22:14:18.325 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T22:14:18.325 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T22:14:18.325 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-09-10T22:14:18.325 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:14:18.325 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:14:18.326 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:14:18.326 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:14:18.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:18.327 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-09-10T22:14:18.345 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-09-10T22:14:18.339+0000 7f5eeb58c580 -1 init, newargv = 0x55d40dbdaaf0 newargc=15 2024-09-10T22:14:18.345 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[53727]: starting ceph client 2024-09-10T22:14:18.351 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[53727]: starting fuse 2024-09-10T22:14:18.354 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-09-10T22:14:18.354 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T22:14:18.354 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:14:18.354 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-09-10T22:14:18.354 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-09-10T22:14:18.354 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:14:18.354 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-09-10T22:14:18.354 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-09-10T22:14:18.355 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-09-10T22:14:18.356 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-09-10T22:14:18.356 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:14:18.356 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:14:18.356 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-09-10T22:14:18.356 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:14:18.484 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.53727.asok 2024-09-10T22:14:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:14:18.495 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53727.asok 2024-09-10T22:14:18.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:18.496 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53727.asok status 2024-09-10T22:14:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:14:18.669 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "53727", 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 9420, 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-09-10T22:14:18.670 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 9420 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1407563352 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1407563352 2024-09-10T22:14:18.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:14:18.672 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.9420 192.168.0.1:0/1407563352", 2024-09-10T22:14:18.672 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/1407563352", 2024-09-10T22:14:18.672 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-09-10T22:14:18.672 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 96, 2024-09-10T22:14:18.672 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 143, 2024-09-10T22:14:18.672 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:14:18.672 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-09-10T22:14:18.672 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-09-10T22:14:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:14:18.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:18.685 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-09-10T22:14:18.685 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:14:18.697 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-09-10T22:14:18.697 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T22:14:18.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:18.697 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:14:18.758 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T22:14:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.947+0000 7f405a200640 1 -- 172.21.15.66:0/5051329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405415a3b0 msgr2=0x7f40540b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.947+0000 7f405a200640 1 --2- 172.21.15.66:0/5051329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405415a3b0 0x7f40540b2cc0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4044009900 tx=0x7f404401c8b0 comp rx=0 tx=0).stop 2024-09-10T22:14:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.947+0000 7f405a200640 1 -- 172.21.15.66:0/5051329 shutdown_connections 2024-09-10T22:14:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.947+0000 7f405a200640 1 --2- 172.21.15.66:0/5051329 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4054072d50 0x7f4054164be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.947+0000 7f405a200640 1 --2- 172.21.15.66:0/5051329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405415a3b0 0x7f40540b2cc0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.947+0000 7f405a200640 1 -- 172.21.15.66:0/5051329 >> 172.21.15.66:0/5051329 conn(0x7f40540cfaa0 msgr2=0x7f40540cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.947+0000 7f405a200640 1 -- 172.21.15.66:0/5051329 shutdown_connections 2024-09-10T22:14:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.947+0000 7f405a200640 1 -- 172.21.15.66:0/5051329 wait complete. 2024-09-10T22:14:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f405a200640 1 Processor -- start 2024-09-10T22:14:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f405a200640 1 -- start start 2024-09-10T22:14:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f405a200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4054070730 0x7f40541f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f405a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4054072d50 0x7f40541f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f405a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405415a3b0 0x7f405414c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f405a200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4054168030 con 0x7f405415a3b0 2024-09-10T22:14:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f405a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4054167eb0 con 0x7f4054072d50 2024-09-10T22:14:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f405a200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f40541681b0 con 0x7f4054070730 2024-09-10T22:14:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f4053e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405415a3b0 0x7f405414c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f4053e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405415a3b0 0x7f405414c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53146/0 (socket says 172.21.15.66:53146) 2024-09-10T22:14:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f4053e00640 1 -- 172.21.15.66:0/689240876 learned_addr learned my addr 172.21.15.66:0/689240876 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f4053400640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4054070730 0x7f40541f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f404be00640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4054072d50 0x7f40541f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f4053e00640 1 -- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4054070730 msgr2=0x7f40541f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f4053e00640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4054070730 0x7f40541f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:18.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f4053e00640 1 -- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4054072d50 msgr2=0x7f40541f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:18.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f4053e00640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4054072d50 0x7f40541f5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:18.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f4053e00640 1 -- 172.21.15.66:0/689240876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40541f8df0 con 0x7f405415a3b0 2024-09-10T22:14:18.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f4053400640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4054070730 0x7f40541f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:14:18.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f404be00640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4054072d50 0x7f40541f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:14:18.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f4053e00640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405415a3b0 0x7f405414c090 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f404c00eab0 tx=0x7f404c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:18.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f404b400640 1 -- 172.21.15.66:0/689240876 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c00ce40 con 0x7f405415a3b0 2024-09-10T22:14:18.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f404b400640 1 -- 172.21.15.66:0/689240876 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f404c004b30 con 0x7f405415a3b0 2024-09-10T22:14:18.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f405a200640 1 -- 172.21.15.66:0/689240876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40540ad120 con 0x7f405415a3b0 2024-09-10T22:14:18.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f404b400640 1 -- 172.21.15.66:0/689240876 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c005020 con 0x7f405415a3b0 2024-09-10T22:14:18.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f405a200640 1 -- 172.21.15.66:0/689240876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40540ad6e0 con 0x7f405415a3b0 2024-09-10T22:14:18.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.951+0000 7f405a200640 1 -- 172.21.15.66:0/689240876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40540b2cc0 con 0x7f405415a3b0 2024-09-10T22:14:18.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.955+0000 7f404b400640 1 -- 172.21.15.66:0/689240876 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f404c0051c0 con 0x7f405415a3b0 2024-09-10T22:14:18.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.955+0000 7f404b400640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4030074710 0x7f4030076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:18.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.955+0000 7f404b400640 1 -- 172.21.15.66:0/689240876 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f404c092f80 con 0x7f405415a3b0 2024-09-10T22:14:18.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.955+0000 7f4053400640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4030074710 0x7f4030076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:18.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.955+0000 7f4053400640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4030074710 0x7f4030076bd0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f40440af000 tx=0x7f404401b0f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:18.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:18.959+0000 7f404b400640 1 -- 172.21.15.66:0/689240876 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f404c0600c0 con 0x7f405415a3b0 2024-09-10T22:14:19.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:19.175+0000 7f405a200640 1 -- 172.21.15.66:0/689240876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f40540d0700 con 0x7f405415a3b0 2024-09-10T22:14:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.183+0000 7f404b400640 1 -- 172.21.15.66:0/689240876 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v97) ==== 136+0+0 (secure 0 0 0) 0x7f404c05f9c0 con 0x7f405415a3b0 2024-09-10T22:14:20.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.187+0000 7f405a200640 1 -- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4030074710 msgr2=0x7f4030076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:20.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.187+0000 7f405a200640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4030074710 0x7f4030076bd0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f40440af000 tx=0x7f404401b0f0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.187+0000 7f405a200640 1 -- 172.21.15.66:0/689240876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405415a3b0 msgr2=0x7f405414c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:20.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.187+0000 7f405a200640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405415a3b0 0x7f405414c090 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f404c00eab0 tx=0x7f404c00ef70 comp rx=0 tx=0).stop 2024-09-10T22:14:20.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.187+0000 7f405a200640 1 -- 172.21.15.66:0/689240876 shutdown_connections 2024-09-10T22:14:20.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.187+0000 7f405a200640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4030074710 0x7f4030076bd0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.187+0000 7f405a200640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405415a3b0 0x7f405414c090 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.187+0000 7f405a200640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4054072d50 0x7f40541f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.187+0000 7f405a200640 1 --2- 172.21.15.66:0/689240876 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4054070730 0x7f40541f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.187+0000 7f405a200640 1 -- 172.21.15.66:0/689240876 >> 172.21.15.66:0/689240876 conn(0x7f40540cfaa0 msgr2=0x7f405414aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:20.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.187+0000 7f405a200640 1 -- 172.21.15.66:0/689240876 shutdown_connections 2024-09-10T22:14:20.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.187+0000 7f405a200640 1 -- 172.21.15.66:0/689240876 wait complete. 2024-09-10T22:14:20.215 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:14:20.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.363+0000 7fd270400640 1 -- 172.21.15.66:0/105214139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2500052e0 msgr2=0x7fd25000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:20.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.363+0000 7fd270400640 1 --2- 172.21.15.66:0/105214139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2500052e0 0x7fd25000b9d0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fd268073860 tx=0x7fd26001d750 comp rx=0 tx=0).stop 2024-09-10T22:14:20.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.363+0000 7fd270400640 1 -- 172.21.15.66:0/105214139 shutdown_connections 2024-09-10T22:14:20.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.363+0000 7fd270400640 1 --2- 172.21.15.66:0/105214139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2500052e0 0x7fd25000b9d0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.363+0000 7fd270400640 1 -- 172.21.15.66:0/105214139 >> 172.21.15.66:0/105214139 conn(0x7fd2680d1c60 msgr2=0x7fd2680d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:20.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.363+0000 7fd270400640 1 -- 172.21.15.66:0/105214139 shutdown_connections 2024-09-10T22:14:20.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.363+0000 7fd270400640 1 -- 172.21.15.66:0/105214139 wait complete. 2024-09-10T22:14:20.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.363+0000 7fd270400640 1 Processor -- start 2024-09-10T22:14:20.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.363+0000 7fd270400640 1 -- start start 2024-09-10T22:14:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd270400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2500052e0 0x7fd2681f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd270400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd2680b7370 0x7fd26806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd270400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2680b6300 0x7fd26815b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd270400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd2680b32f0 con 0x7fd2500052e0 2024-09-10T22:14:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd270400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd2680b3170 con 0x7fd2680b6300 2024-09-10T22:14:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd270400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd2680b3470 con 0x7fd2680b7370 2024-09-10T22:14:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd26da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2500052e0 0x7fd2681f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd26da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2500052e0 0x7fd2681f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53168/0 (socket says 172.21.15.66:53168) 2024-09-10T22:14:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd26da00640 1 -- 172.21.15.66:0/4004919746 learned_addr learned my addr 172.21.15.66:0/4004919746 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd26d000640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd2680b7370 0x7fd26806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd26e400640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2680b6300 0x7fd26815b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd26da00640 1 -- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd2680b7370 msgr2=0x7fd26806f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd26da00640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd2680b7370 0x7fd26806f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd26da00640 1 -- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2680b6300 msgr2=0x7fd26815b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd26da00640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2680b6300 0x7fd26815b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd26da00640 1 -- 172.21.15.66:0/4004919746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2681f5090 con 0x7fd2500052e0 2024-09-10T22:14:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd26da00640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2500052e0 0x7fd2681f47f0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fd25c005e70 tx=0x7fd25c011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd265600640 1 -- 172.21.15.66:0/4004919746 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c005320 con 0x7fd2500052e0 2024-09-10T22:14:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd265600640 1 -- 172.21.15.66:0/4004919746 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd25c018020 con 0x7fd2500052e0 2024-09-10T22:14:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd270400640 1 -- 172.21.15.66:0/4004919746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2681f5370 con 0x7fd2500052e0 2024-09-10T22:14:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd265600640 1 -- 172.21.15.66:0/4004919746 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c003860 con 0x7fd2500052e0 2024-09-10T22:14:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd270400640 1 -- 172.21.15.66:0/4004919746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2680b5de0 con 0x7fd2500052e0 2024-09-10T22:14:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.367+0000 7fd270400640 1 -- 172.21.15.66:0/4004919746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26815fd80 con 0x7fd2500052e0 2024-09-10T22:14:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.371+0000 7fd265600640 1 -- 172.21.15.66:0/4004919746 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fd25c010040 con 0x7fd2500052e0 2024-09-10T22:14:20.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.371+0000 7fd265600640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd248074710 0x7fd248076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:20.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.371+0000 7fd265600640 1 -- 172.21.15.66:0/4004919746 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7fd25c0957a0 con 0x7fd2500052e0 2024-09-10T22:14:20.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.371+0000 7fd26d000640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd248074710 0x7fd248076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:20.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.371+0000 7fd26d000640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd248074710 0x7fd248076bd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd254000c10 tx=0x7fd254001df0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:20.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.375+0000 7fd265600640 1 -- 172.21.15.66:0/4004919746 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd25c022020 con 0x7fd2500052e0 2024-09-10T22:14:20.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.571+0000 7fd270400640 1 -- 172.21.15.66:0/4004919746 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fd2680b5fb0 con 0x7fd248074710 2024-09-10T22:14:20.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd265600640 1 -- 172.21.15.66:0/4004919746 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fd2680b5fb0 con 0x7fd248074710 2024-09-10T22:14:20.579 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:14:20.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:14:20.579 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:14:20.579 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:14:20.579 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:14:20.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd270400640 1 -- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd248074710 msgr2=0x7fd248076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:20.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd270400640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd248074710 0x7fd248076bd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd254000c10 tx=0x7fd254001df0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd270400640 1 -- 172.21.15.66:0/4004919746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2500052e0 msgr2=0x7fd2681f47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:20.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd270400640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2500052e0 0x7fd2681f47f0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fd25c005e70 tx=0x7fd25c011740 comp rx=0 tx=0).stop 2024-09-10T22:14:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd270400640 1 -- 172.21.15.66:0/4004919746 shutdown_connections 2024-09-10T22:14:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd270400640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd248074710 0x7fd248076bd0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd270400640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2680b6300 0x7fd26815b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd270400640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd2680b7370 0x7fd26806f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd270400640 1 --2- 172.21.15.66:0/4004919746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2500052e0 0x7fd2681f47f0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd270400640 1 -- 172.21.15.66:0/4004919746 >> 172.21.15.66:0/4004919746 conn(0x7fd2680d1c60 msgr2=0x7fd268072060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd270400640 1 -- 172.21.15.66:0/4004919746 shutdown_connections 2024-09-10T22:14:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.575+0000 7fd270400640 1 -- 172.21.15.66:0/4004919746 wait complete. 2024-09-10T22:14:20.601 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:14:20.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 -- 172.21.15.66:0/4258987611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08073f40 msgr2=0x7fad080743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:20.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 --2- 172.21.15.66:0/4258987611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08073f40 0x7fad080743f0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fad080cd670 tx=0x7fad0001cb60 comp rx=0 tx=0).stop 2024-09-10T22:14:20.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 -- 172.21.15.66:0/4258987611 shutdown_connections 2024-09-10T22:14:20.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 --2- 172.21.15.66:0/4258987611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08073f40 0x7fad080743f0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 --2- 172.21.15.66:0/4258987611 >> v2:172.21.15.94:3300/0 conn(0x7fad08152b90 0x7fad0806f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 -- 172.21.15.66:0/4258987611 >> 172.21.15.66:0/4258987611 conn(0x7fad080b26c0 msgr2=0x7fad080b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:20.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 -- 172.21.15.66:0/4258987611 shutdown_connections 2024-09-10T22:14:20.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 -- 172.21.15.66:0/4258987611 wait complete. 2024-09-10T22:14:20.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 Processor -- start 2024-09-10T22:14:20.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 -- start start 2024-09-10T22:14:20.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad080737c0 0x7fad08153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:20.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08073f40 0x7fad0814c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:20.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad08152b90 0x7fad080c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:20.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad081689e0 con 0x7fad08073f40 2024-09-10T22:14:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad080ce3c0 con 0x7fad08152b90 2024-09-10T22:14:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad0fc00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fad08168b60 con 0x7fad080737c0 2024-09-10T22:14:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad07e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08073f40 0x7fad0814c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad07e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08073f40 0x7fad0814c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53176/0 (socket says 172.21.15.66:53176) 2024-09-10T22:14:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.747+0000 7fad07e00640 1 -- 172.21.15.66:0/2124864768 learned_addr learned my addr 172.21.15.66:0/2124864768 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad0dc00640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad08152b90 0x7fad080c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad0d200640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad080737c0 0x7fad08153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad07e00640 1 -- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad080737c0 msgr2=0x7fad08153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad07e00640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad080737c0 0x7fad08153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad07e00640 1 -- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad08152b90 msgr2=0x7fad080c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:20.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad07e00640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad08152b90 0x7fad080c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad07e00640 1 -- 172.21.15.66:0/2124864768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad08155c20 con 0x7fad08073f40 2024-09-10T22:14:20.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad07e00640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08073f40 0x7fad0814c6a0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7facf400c500 tx=0x7facf400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:20.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad04c00640 1 -- 172.21.15.66:0/2124864768 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4017070 con 0x7fad08073f40 2024-09-10T22:14:20.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad04c00640 1 -- 172.21.15.66:0/2124864768 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facf4003600 con 0x7fad08073f40 2024-09-10T22:14:20.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad04c00640 1 -- 172.21.15.66:0/2124864768 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4003a90 con 0x7fad08073f40 2024-09-10T22:14:20.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad0fc00640 1 -- 172.21.15.66:0/2124864768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad08155ec0 con 0x7fad08073f40 2024-09-10T22:14:20.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad0fc00640 1 -- 172.21.15.66:0/2124864768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad08156160 con 0x7fad08073f40 2024-09-10T22:14:20.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad0fc00640 1 -- 172.21.15.66:0/2124864768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad080743f0 con 0x7fad08073f40 2024-09-10T22:14:20.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.751+0000 7fad04c00640 1 -- 172.21.15.66:0/2124864768 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7facf4003c30 con 0x7fad08073f40 2024-09-10T22:14:20.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.755+0000 7fad04c00640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7face8074710 0x7face8076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:20.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.755+0000 7fad04c00640 1 -- 172.21.15.66:0/2124864768 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7facf4091ad0 con 0x7fad08073f40 2024-09-10T22:14:20.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.755+0000 7fad0d200640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7face8074710 0x7face8076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.755+0000 7fad0d200640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7face8074710 0x7face8076bd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fad080cac70 tx=0x7facfc0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.759+0000 7fad04c00640 1 -- 172.21.15.66:0/2124864768 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7facf405e680 con 0x7fad08073f40 2024-09-10T22:14:20.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.935+0000 7fad0fc00640 1 -- 172.21.15.66:0/2124864768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fad0806d1f0 con 0x7fad08073f40 2024-09-10T22:14:20.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.935+0000 7fad04c00640 1 -- 172.21.15.66:0/2124864768 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7facf40630b0 con 0x7fad08073f40 2024-09-10T22:14:20.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.939+0000 7fad0fc00640 1 -- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7face8074710 msgr2=0x7face8076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:20.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.939+0000 7fad0fc00640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7face8074710 0x7face8076bd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fad080cac70 tx=0x7facfc0091e0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.939+0000 7fad0fc00640 1 -- 172.21.15.66:0/2124864768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08073f40 msgr2=0x7fad0814c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:20.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.939+0000 7fad0fc00640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08073f40 0x7fad0814c6a0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7facf400c500 tx=0x7facf400c9c0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.939+0000 7fad0fc00640 1 -- 172.21.15.66:0/2124864768 shutdown_connections 2024-09-10T22:14:20.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.939+0000 7fad0fc00640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7face8074710 0x7face8076bd0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.939+0000 7fad0fc00640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad08152b90 0x7fad080c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.939+0000 7fad0fc00640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08073f40 0x7fad0814c6a0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.939+0000 7fad0fc00640 1 --2- 172.21.15.66:0/2124864768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad080737c0 0x7fad08153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:20.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.939+0000 7fad0fc00640 1 -- 172.21.15.66:0/2124864768 >> 172.21.15.66:0/2124864768 conn(0x7fad080b26c0 msgr2=0x7fad0815ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:20.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.939+0000 7fad0fc00640 1 -- 172.21.15.66:0/2124864768 shutdown_connections 2024-09-10T22:14:20.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:20.939+0000 7fad0fc00640 1 -- 172.21.15.66:0/2124864768 wait complete. 2024-09-10T22:14:20.963 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-10T22:14:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.107+0000 7f4401400640 1 -- 172.21.15.66:0/2750589732 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43fc0d06b0 msgr2=0x7f43fc0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.107+0000 7f4401400640 1 --2- 172.21.15.66:0/2750589732 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43fc0d06b0 0x7f43fc0ce170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009900 tx=0x7f43e401c850 comp rx=0 tx=0).stop 2024-09-10T22:14:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.107+0000 7f4401400640 1 -- 172.21.15.66:0/2750589732 shutdown_connections 2024-09-10T22:14:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.107+0000 7f4401400640 1 --2- 172.21.15.66:0/2750589732 >> v2:172.21.15.94:3300/0 conn(0x7f43fc0afc20 0x7f43fc0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.107+0000 7f4401400640 1 --2- 172.21.15.66:0/2750589732 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43fc0d06b0 0x7f43fc0ce170 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.107+0000 7f4401400640 1 -- 172.21.15.66:0/2750589732 >> 172.21.15.66:0/2750589732 conn(0x7f43fc0d1c60 msgr2=0x7f43fc0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:21.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.107+0000 7f4401400640 1 -- 172.21.15.66:0/2750589732 shutdown_connections 2024-09-10T22:14:21.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.107+0000 7f4401400640 1 -- 172.21.15.66:0/2750589732 wait complete. 2024-09-10T22:14:21.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f4401400640 1 Processor -- start 2024-09-10T22:14:21.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f4401400640 1 -- start start 2024-09-10T22:14:21.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f4401400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43fc0afc20 0x7f43fc0b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f4401400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43fc0d06b0 0x7f43fc06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f4401400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43fc15bd40 0x7f43fc0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f4401400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc169230 con 0x7f43fc0d06b0 2024-09-10T22:14:21.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f4401400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc1690b0 con 0x7f43fc0afc20 2024-09-10T22:14:21.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f4401400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f43fc1693b0 con 0x7f43fc15bd40 2024-09-10T22:14:21.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43f1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43fc0d06b0 0x7f43fc06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43f1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43fc0d06b0 0x7f43fc06e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53184/0 (socket says 172.21.15.66:53184) 2024-09-10T22:14:21.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43f1e00640 1 -- 172.21.15.66:0/3547189459 learned_addr learned my addr 172.21.15.66:0/3547189459 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:21.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43fa800640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43fc0afc20 0x7f43fc0b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43fb200640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43fc15bd40 0x7f43fc0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43f1e00640 1 -- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43fc15bd40 msgr2=0x7f43fc0b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43f1e00640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43fc15bd40 0x7f43fc0b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43f1e00640 1 -- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43fc0afc20 msgr2=0x7f43fc0b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43f1e00640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43fc0afc20 0x7f43fc0b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43f1e00640 1 -- 172.21.15.66:0/3547189459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc0b72a0 con 0x7f43fc0d06b0 2024-09-10T22:14:21.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43f1e00640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43fc0d06b0 0x7f43fc06e6d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009850 tx=0x7f43e40a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:21.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43f3400640 1 -- 172.21.15.66:0/3547189459 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e40b2040 con 0x7f43fc0d06b0 2024-09-10T22:14:21.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43f3400640 1 -- 172.21.15.66:0/3547189459 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43e4004380 con 0x7f43fc0d06b0 2024-09-10T22:14:21.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f43f3400640 1 -- 172.21.15.66:0/3547189459 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4005290 con 0x7f43fc0d06b0 2024-09-10T22:14:21.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f4401400640 1 -- 172.21.15.66:0/3547189459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43fc0b7580 con 0x7f43fc0d06b0 2024-09-10T22:14:21.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.111+0000 7f4401400640 1 -- 172.21.15.66:0/3547189459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43fc06e1b0 con 0x7f43fc0d06b0 2024-09-10T22:14:21.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.115+0000 7f4401400640 1 -- 172.21.15.66:0/3547189459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43fc0b77c0 con 0x7f43fc0d06b0 2024-09-10T22:14:21.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.115+0000 7f43f3400640 1 -- 172.21.15.66:0/3547189459 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f43e4005430 con 0x7f43fc0d06b0 2024-09-10T22:14:21.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.115+0000 7f43f3400640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f43d8074710 0x7f43d8076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.115+0000 7f43f3400640 1 -- 172.21.15.66:0/3547189459 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f43e412bd90 con 0x7f43fc0d06b0 2024-09-10T22:14:21.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.115+0000 7f43fa800640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f43d8074710 0x7f43d8076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.119+0000 7f43fa800640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f43d8074710 0x7f43d8076bd0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f43ec001650 tx=0x7f43ec009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:21.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.119+0000 7f43f3400640 1 -- 172.21.15.66:0/3547189459 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f43fc0b77c0 con 0x7f43fc0d06b0 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.295+0000 7f4401400640 1 -- 172.21.15.66:0/3547189459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f43fc0cffa0 con 0x7f43fc0d06b0 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.295+0000 7f43f3400640 1 -- 172.21.15.66:0/3547189459 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v54) ==== 116+0+2 (secure 0 0 0) 0x7f43e40164d0 con 0x7f43fc0d06b0 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi066.stdout:4 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.299+0000 7f4401400640 1 -- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f43d8074710 msgr2=0x7f43d8076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.299+0000 7f4401400640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f43d8074710 0x7f43d8076bd0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f43ec001650 tx=0x7f43ec009040 comp rx=0 tx=0).stop 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.299+0000 7f4401400640 1 -- 172.21.15.66:0/3547189459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43fc0d06b0 msgr2=0x7f43fc06e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.299+0000 7f4401400640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43fc0d06b0 0x7f43fc06e6d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009850 tx=0x7f43e40a56a0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.299+0000 7f4401400640 1 -- 172.21.15.66:0/3547189459 shutdown_connections 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.299+0000 7f4401400640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f43d8074710 0x7f43d8076bd0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.299+0000 7f4401400640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43fc15bd40 0x7f43fc0b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.299+0000 7f4401400640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43fc0d06b0 0x7f43fc06e6d0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.299+0000 7f4401400640 1 --2- 172.21.15.66:0/3547189459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43fc0afc20 0x7f43fc0b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.299+0000 7f4401400640 1 -- 172.21.15.66:0/3547189459 >> 172.21.15.66:0/3547189459 conn(0x7f43fc0d1c60 msgr2=0x7f43fc071d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.299+0000 7f4401400640 1 -- 172.21.15.66:0/3547189459 shutdown_connections 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.299+0000 7f4401400640 1 -- 172.21.15.66:0/3547189459 wait complete. 2024-09-10T22:14:21.324 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:14:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.467+0000 7f6d1ac00640 1 -- 172.21.15.66:0/74594660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d140cff10 msgr2=0x7f6d140d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.467+0000 7f6d1ac00640 1 --2- 172.21.15.66:0/74594660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d140cff10 0x7f6d140d03a0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009c80 tx=0x7f6d0c01cce0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.467+0000 7f6d1ac00640 1 -- 172.21.15.66:0/74594660 shutdown_connections 2024-09-10T22:14:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.467+0000 7f6d1ac00640 1 --2- 172.21.15.66:0/74594660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d140cff10 0x7f6d140d03a0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.467+0000 7f6d1ac00640 1 --2- 172.21.15.66:0/74594660 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6d140b0590 0x7f6d140cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.467+0000 7f6d1ac00640 1 -- 172.21.15.66:0/74594660 >> 172.21.15.66:0/74594660 conn(0x7f6d14074240 msgr2=0x7f6d14074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.467+0000 7f6d1ac00640 1 -- 172.21.15.66:0/74594660 shutdown_connections 2024-09-10T22:14:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.467+0000 7f6d1ac00640 1 -- 172.21.15.66:0/74594660 wait complete. 2024-09-10T22:14:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d1ac00640 1 Processor -- start 2024-09-10T22:14:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d1ac00640 1 -- start start 2024-09-10T22:14:21.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d1ac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140b0590 0x7f6d1415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d1ac00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6d140cff10 0x7f6d140c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d1ac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d1415bfb0 0x7f6d141fb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d1ac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140cd990 con 0x7f6d1415bfb0 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d1ac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140cdb10 con 0x7f6d140b0590 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d1ac00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6d141692a0 con 0x7f6d140cff10 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d18c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d1415bfb0 0x7f6d141fb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d18c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d1415bfb0 0x7f6d141fb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53196/0 (socket says 172.21.15.66:53196) 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d18c00640 1 -- 172.21.15.66:0/1127116352 learned_addr learned my addr 172.21.15.66:0/1127116352 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d0be00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6d140cff10 0x7f6d140c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d13e00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140b0590 0x7f6d1415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d18c00640 1 -- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6d140cff10 msgr2=0x7f6d140c2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d18c00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6d140cff10 0x7f6d140c2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d18c00640 1 -- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140b0590 msgr2=0x7f6d1415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d18c00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140b0590 0x7f6d1415adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d18c00640 1 -- 172.21.15.66:0/1127116352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d140b3160 con 0x7f6d1415bfb0 2024-09-10T22:14:21.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d18c00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d1415bfb0 0x7f6d141fb2b0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009bd0 tx=0x7f6d0c009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:21.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d10c00640 1 -- 172.21.15.66:0/1127116352 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0bb070 con 0x7f6d1415bfb0 2024-09-10T22:14:21.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d10c00640 1 -- 172.21.15.66:0/1127116352 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d0c004e00 con 0x7f6d1415bfb0 2024-09-10T22:14:21.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d10c00640 1 -- 172.21.15.66:0/1127116352 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0b2040 con 0x7f6d1415bfb0 2024-09-10T22:14:21.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d1ac00640 1 -- 172.21.15.66:0/1127116352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d141fc0c0 con 0x7f6d1415bfb0 2024-09-10T22:14:21.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.471+0000 7f6d1ac00640 1 -- 172.21.15.66:0/1127116352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d140c35a0 con 0x7f6d1415bfb0 2024-09-10T22:14:21.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.475+0000 7f6d1ac00640 1 -- 172.21.15.66:0/1127116352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d140d03a0 con 0x7f6d1415bfb0 2024-09-10T22:14:21.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.475+0000 7f6d10c00640 1 -- 172.21.15.66:0/1127116352 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f6d0c0b5030 con 0x7f6d1415bfb0 2024-09-10T22:14:21.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.475+0000 7f6d10c00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6cf0074710 0x7f6cf0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.475+0000 7f6d10c00640 1 -- 172.21.15.66:0/1127116352 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f6d0c12c2e0 con 0x7f6d1415bfb0 2024-09-10T22:14:21.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.475+0000 7f6d13e00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6cf0074710 0x7f6cf0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.479+0000 7f6d13e00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6cf0074710 0x7f6cf0076bd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6d140cfce0 tx=0x7f6d04009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:21.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.479+0000 7f6d10c00640 1 -- 172.21.15.66:0/1127116352 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6d0c016990 con 0x7f6d1415bfb0 2024-09-10T22:14:21.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.651+0000 7f6d1ac00640 1 -- 172.21.15.66:0/1127116352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f6d14096530 con 0x7f6d1415bfb0 2024-09-10T22:14:21.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.655+0000 7f6d10c00640 1 -- 172.21.15.66:0/1127116352 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v55)=0 v55) ==== 131+0+0 (secure 0 0 0) 0x7f6d0c0f8dd0 con 0x7f6d1415bfb0 2024-09-10T22:14:21.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.659+0000 7f6d1ac00640 1 -- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6cf0074710 msgr2=0x7f6cf0076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.659+0000 7f6d1ac00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6cf0074710 0x7f6cf0076bd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6d140cfce0 tx=0x7f6d04009040 comp rx=0 tx=0).stop 2024-09-10T22:14:21.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.659+0000 7f6d1ac00640 1 -- 172.21.15.66:0/1127116352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d1415bfb0 msgr2=0x7f6d141fb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.659+0000 7f6d1ac00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d1415bfb0 0x7f6d141fb2b0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009bd0 tx=0x7f6d0c009390 comp rx=0 tx=0).stop 2024-09-10T22:14:21.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.659+0000 7f6d1ac00640 1 -- 172.21.15.66:0/1127116352 shutdown_connections 2024-09-10T22:14:21.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.659+0000 7f6d1ac00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6cf0074710 0x7f6cf0076bd0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.659+0000 7f6d1ac00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d1415bfb0 0x7f6d141fb2b0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.659+0000 7f6d1ac00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6d140cff10 0x7f6d140c2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.659+0000 7f6d1ac00640 1 --2- 172.21.15.66:0/1127116352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140b0590 0x7f6d1415adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.659+0000 7f6d1ac00640 1 -- 172.21.15.66:0/1127116352 >> 172.21.15.66:0/1127116352 conn(0x7f6d14074240 msgr2=0x7f6d14158030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:21.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.659+0000 7f6d1ac00640 1 -- 172.21.15.66:0/1127116352 shutdown_connections 2024-09-10T22:14:21.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.659+0000 7f6d1ac00640 1 -- 172.21.15.66:0/1127116352 wait complete. 2024-09-10T22:14:21.680 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:21.679+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:21.680 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:21.679+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:21.680 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:21.679+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:21.680 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:21.679+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:21.680 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:21.679+0000 7f5fec400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:21.682 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-10T22:14:21.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.827+0000 7f7c67600640 1 -- 172.21.15.66:0/4088404373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6015e460 msgr2=0x7f7c600ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.827+0000 7f7c67600640 1 --2- 172.21.15.66:0/4088404373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6015e460 0x7f7c600ccab0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009900 tx=0x7f7c4c01c810 comp rx=0 tx=0).stop 2024-09-10T22:14:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.827+0000 7f7c67600640 1 -- 172.21.15.66:0/4088404373 shutdown_connections 2024-09-10T22:14:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.827+0000 7f7c67600640 1 --2- 172.21.15.66:0/4088404373 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c6006ebc0 0x7f7c6006f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.827+0000 7f7c67600640 1 --2- 172.21.15.66:0/4088404373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6015e460 0x7f7c600ccab0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.827+0000 7f7c67600640 1 --2- 172.21.15.66:0/4088404373 >> v2:172.21.15.94:3300/0 conn(0x7f7c600ce660 0x7f7c600cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.827+0000 7f7c67600640 1 -- 172.21.15.66:0/4088404373 >> 172.21.15.66:0/4088404373 conn(0x7f7c600d1c50 msgr2=0x7f7c600d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.827+0000 7f7c67600640 1 -- 172.21.15.66:0/4088404373 shutdown_connections 2024-09-10T22:14:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.827+0000 7f7c67600640 1 -- 172.21.15.66:0/4088404373 wait complete. 2024-09-10T22:14:21.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.827+0000 7f7c67600640 1 Processor -- start 2024-09-10T22:14:21.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.827+0000 7f7c67600640 1 -- start start 2024-09-10T22:14:21.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c67600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6006ebc0 0x7f7c60154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c67600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c600ce660 0x7f7c601547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c67600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c6015e460 0x7f7c600b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c67600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601625d0 con 0x7f7c600ce660 2024-09-10T22:14:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c67600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60162450 con 0x7f7c6006ebc0 2024-09-10T22:14:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c67600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7c60162750 con 0x7f7c6015e460 2024-09-10T22:14:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c5fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c600ce660 0x7f7c601547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c5fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c600ce660 0x7f7c601547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53212/0 (socket says 172.21.15.66:53212) 2024-09-10T22:14:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c5fe00640 1 -- 172.21.15.66:0/933216194 learned_addr learned my addr 172.21.15.66:0/933216194 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c64c00640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6006ebc0 0x7f7c60154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c65600640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c6015e460 0x7f7c600b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c5fe00640 1 -- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c6015e460 msgr2=0x7f7c600b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c5fe00640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c6015e460 0x7f7c600b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c5fe00640 1 -- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6006ebc0 msgr2=0x7f7c60154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c5fe00640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6006ebc0 0x7f7c60154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c5fe00640 1 -- 172.21.15.66:0/933216194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c60155360 con 0x7f7c600ce660 2024-09-10T22:14:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c65600640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c6015e460 0x7f7c600b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:14:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c5fe00640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c600ce660 0x7f7c601547e0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c01aec0 tx=0x7f7c4c0a6520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c5cc00640 1 -- 172.21.15.66:0/933216194 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c0ac070 con 0x7f7c600ce660 2024-09-10T22:14:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c5cc00640 1 -- 172.21.15.66:0/933216194 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c4c004320 con 0x7f7c600ce660 2024-09-10T22:14:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c5cc00640 1 -- 172.21.15.66:0/933216194 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c005320 con 0x7f7c600ce660 2024-09-10T22:14:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c67600640 1 -- 172.21.15.66:0/933216194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c60155640 con 0x7f7c600ce660 2024-09-10T22:14:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c67600640 1 -- 172.21.15.66:0/933216194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c60155c00 con 0x7f7c600ce660 2024-09-10T22:14:21.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.831+0000 7f7c67600640 1 -- 172.21.15.66:0/933216194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c600cea40 con 0x7f7c600ce660 2024-09-10T22:14:21.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.835+0000 7f7c5cc00640 1 -- 172.21.15.66:0/933216194 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f7c4c0054c0 con 0x7f7c600ce660 2024-09-10T22:14:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.835+0000 7f7c5cc00640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7c40074710 0x7f7c40076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.835+0000 7f7c5cc00640 1 -- 172.21.15.66:0/933216194 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f7c4c12c8e0 con 0x7f7c600ce660 2024-09-10T22:14:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.835+0000 7f7c64c00640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7c40074710 0x7f7c40076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.835+0000 7f7c64c00640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7c40074710 0x7f7c40076bd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7c54005d90 tx=0x7f7c54005c60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:21.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:21.839+0000 7f7c5cc00640 1 -- 172.21.15.66:0/933216194 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7c4c016490 con 0x7f7c600ce660 2024-09-10T22:14:22.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.015+0000 7f7c67600640 1 -- 172.21.15.66:0/933216194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f7c6006fd20 con 0x7f7c600ce660 2024-09-10T22:14:22.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.019+0000 7f7c5cc00640 1 -- 172.21.15.66:0/933216194 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v56)=0 v56) ==== 136+0+0 (secure 0 0 0) 0x7f7c4c0f9a40 con 0x7f7c600ce660 2024-09-10T22:14:22.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.023+0000 7f7c67600640 1 -- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7c40074710 msgr2=0x7f7c40076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:22.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.023+0000 7f7c67600640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7c40074710 0x7f7c40076bd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7c54005d90 tx=0x7f7c54005c60 comp rx=0 tx=0).stop 2024-09-10T22:14:22.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.023+0000 7f7c67600640 1 -- 172.21.15.66:0/933216194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c600ce660 msgr2=0x7f7c601547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:22.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.023+0000 7f7c67600640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c600ce660 0x7f7c601547e0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c01aec0 tx=0x7f7c4c0a6520 comp rx=0 tx=0).stop 2024-09-10T22:14:22.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.023+0000 7f7c67600640 1 -- 172.21.15.66:0/933216194 shutdown_connections 2024-09-10T22:14:22.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.023+0000 7f7c67600640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7c40074710 0x7f7c40076bd0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.023+0000 7f7c67600640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c6015e460 0x7f7c600b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.023+0000 7f7c67600640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c600ce660 0x7f7c601547e0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.023+0000 7f7c67600640 1 --2- 172.21.15.66:0/933216194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6006ebc0 0x7f7c60154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.023+0000 7f7c67600640 1 -- 172.21.15.66:0/933216194 >> 172.21.15.66:0/933216194 conn(0x7f7c600d1c50 msgr2=0x7f7c600b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:22.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.023+0000 7f7c67600640 1 -- 172.21.15.66:0/933216194 shutdown_connections 2024-09-10T22:14:22.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.023+0000 7f7c67600640 1 -- 172.21.15.66:0/933216194 wait complete. 2024-09-10T22:14:22.048 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-10T22:14:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 -- 172.21.15.66:0/3218445504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a01600b0 msgr2=0x7f37a00cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 --2- 172.21.15.66:0/3218445504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a01600b0 0x7f37a00cfad0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f37a0073ca0 tx=0x7f379801d490 comp rx=0 tx=0).stop 2024-09-10T22:14:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 -- 172.21.15.66:0/3218445504 shutdown_connections 2024-09-10T22:14:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 --2- 172.21.15.66:0/3218445504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a01600b0 0x7f37a00cfad0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 --2- 172.21.15.66:0/3218445504 >> v2:172.21.15.94:3300/0 conn(0x7f37a00b1e10 0x7f37a015fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 --2- 172.21.15.66:0/3218445504 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37a015f450 0x7f37a00b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 -- 172.21.15.66:0/3218445504 >> 172.21.15.66:0/3218445504 conn(0x7f37a00d1c60 msgr2=0x7f37a00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 -- 172.21.15.66:0/3218445504 shutdown_connections 2024-09-10T22:14:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 -- 172.21.15.66:0/3218445504 wait complete. 2024-09-10T22:14:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 Processor -- start 2024-09-10T22:14:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 -- start start 2024-09-10T22:14:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37a00b1e10 0x7f37a0154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37a015f450 0x7f37a0157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a01600b0 0x7f37a01583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f37a00af170 con 0x7f37a01600b0 2024-09-10T22:14:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f37a00aeff0 con 0x7f37a00b1e10 2024-09-10T22:14:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a8400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f37a00af2f0 con 0x7f37a015f450 2024-09-10T22:14:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a6400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a01600b0 0x7f37a01583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a6400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a01600b0 0x7f37a01583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53224/0 (socket says 172.21.15.66:53224) 2024-09-10T22:14:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a6400640 1 -- 172.21.15.66:0/1466723326 learned_addr learned my addr 172.21.15.66:0/1466723326 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a5a00640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37a00b1e10 0x7f37a0154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a5000640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37a015f450 0x7f37a0157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a6400640 1 -- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37a015f450 msgr2=0x7f37a0157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a6400640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37a015f450 0x7f37a0157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a6400640 1 -- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37a00b1e10 msgr2=0x7f37a0154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a6400640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37a00b1e10 0x7f37a0154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.195+0000 7f37a6400640 1 -- 172.21.15.66:0/1466723326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a01554a0 con 0x7f37a01600b0 2024-09-10T22:14:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.199+0000 7f37a6400640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a01600b0 0x7f37a01583c0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f3798009bd0 tx=0x7f3798003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:22.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.199+0000 7f379d600640 1 -- 172.21.15.66:0/1466723326 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37980bb070 con 0x7f37a01600b0 2024-09-10T22:14:22.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.199+0000 7f379d600640 1 -- 172.21.15.66:0/1466723326 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3798004b40 con 0x7f37a01600b0 2024-09-10T22:14:22.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.199+0000 7f379d600640 1 -- 172.21.15.66:0/1466723326 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37980b2040 con 0x7f37a01600b0 2024-09-10T22:14:22.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.199+0000 7f37a8400640 1 -- 172.21.15.66:0/1466723326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a014bd10 con 0x7f37a01600b0 2024-09-10T22:14:22.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.199+0000 7f37a8400640 1 -- 172.21.15.66:0/1466723326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a0157900 con 0x7f37a01600b0 2024-09-10T22:14:22.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.199+0000 7f37a8400640 1 -- 172.21.15.66:0/1466723326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a0157b40 con 0x7f37a01600b0 2024-09-10T22:14:22.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.199+0000 7f379d600640 1 -- 172.21.15.66:0/1466723326 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f37980b5070 con 0x7f37a01600b0 2024-09-10T22:14:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.199+0000 7f379d600640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3780074710 0x7f3780076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.203+0000 7f379d600640 1 -- 172.21.15.66:0/1466723326 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f379812bcb0 con 0x7f37a01600b0 2024-09-10T22:14:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.203+0000 7f37a5a00640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3780074710 0x7f3780076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:22.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.203+0000 7f37a5a00640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3780074710 0x7f3780076bd0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3794009be0 tx=0x7f37940091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:22.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.207+0000 7f379d600640 1 -- 172.21.15.66:0/1466723326 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f37a0157b40 con 0x7f37a01600b0 2024-09-10T22:14:22.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.403+0000 7f37a8400640 1 -- 172.21.15.66:0/1466723326 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f37a014bff0 con 0x7f3780074710 2024-09-10T22:14:22.419 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:22.419+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:22.419 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:22.419+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:22.419 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:22.419+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:22.419 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:22.419+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:22.419 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:22.419+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:22.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f379d600640 1 -- 172.21.15.66:0/1466723326 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f37a014bff0 con 0x7f3780074710 2024-09-10T22:14:22.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f37a8400640 1 -- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3780074710 msgr2=0x7f3780076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:22.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f37a8400640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3780074710 0x7f3780076bd0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3794009be0 tx=0x7f37940091e0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f37a8400640 1 -- 172.21.15.66:0/1466723326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a01600b0 msgr2=0x7f37a01583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:22.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f37a8400640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a01600b0 0x7f37a01583c0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f3798009bd0 tx=0x7f3798003f60 comp rx=0 tx=0).stop 2024-09-10T22:14:22.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f37a8400640 1 -- 172.21.15.66:0/1466723326 shutdown_connections 2024-09-10T22:14:22.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f37a8400640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3780074710 0x7f3780076bd0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f37a8400640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a01600b0 0x7f37a01583c0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f37a8400640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37a015f450 0x7f37a0157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f37a8400640 1 --2- 172.21.15.66:0/1466723326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37a00b1e10 0x7f37a0154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f37a8400640 1 -- 172.21.15.66:0/1466723326 >> 172.21.15.66:0/1466723326 conn(0x7f37a00d1c60 msgr2=0x7f37a00cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:22.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f37a8400640 1 -- 172.21.15.66:0/1466723326 shutdown_connections 2024-09-10T22:14:22.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.451+0000 7f37a8400640 1 -- 172.21.15.66:0/1466723326 wait complete. 2024-09-10T22:14:22.477 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:14:22.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.619+0000 7f1359000640 1 -- 172.21.15.66:0/4168591718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f135415f450 msgr2=0x7f13540b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:22.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.619+0000 7f1359000640 1 --2- 172.21.15.66:0/4168591718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f135415f450 0x7f13540b3170 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f1344009840 tx=0x7f134401d1f0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.619+0000 7f1359000640 1 -- 172.21.15.66:0/4168591718 shutdown_connections 2024-09-10T22:14:22.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.619+0000 7f1359000640 1 --2- 172.21.15.66:0/4168591718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13540b1e10 0x7f135415fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.619+0000 7f1359000640 1 --2- 172.21.15.66:0/4168591718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f135415f450 0x7f13540b3170 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.619+0000 7f1359000640 1 -- 172.21.15.66:0/4168591718 >> 172.21.15.66:0/4168591718 conn(0x7f13540d1c60 msgr2=0x7f13540d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:22.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.619+0000 7f1359000640 1 -- 172.21.15.66:0/4168591718 shutdown_connections 2024-09-10T22:14:22.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.619+0000 7f1359000640 1 -- 172.21.15.66:0/4168591718 wait complete. 2024-09-10T22:14:22.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1359000640 1 Processor -- start 2024-09-10T22:14:22.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1359000640 1 -- start start 2024-09-10T22:14:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1359000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13540b1e10 0x7f1354150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1359000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f135415f450 0x7f13540ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1359000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13541600b0 0x7f135406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1359000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f13541693a0 con 0x7f13540b1e10 2024-09-10T22:14:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1359000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1354169220 con 0x7f13541600b0 2024-09-10T22:14:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1359000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1354169520 con 0x7f135415f450 2024-09-10T22:14:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1351e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13540b1e10 0x7f1354150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1351e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13540b1e10 0x7f1354150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41050/0 (socket says 172.21.15.66:41050) 2024-09-10T22:14:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1351e00640 1 -- 172.21.15.66:0/2339150723 learned_addr learned my addr 172.21.15.66:0/2339150723 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f134be00640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f135415f450 0x7f13540ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1352800640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13541600b0 0x7f135406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1351e00640 1 -- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f135415f450 msgr2=0x7f13540ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1351e00640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f135415f450 0x7f13540ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1351e00640 1 -- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13541600b0 msgr2=0x7f135406f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1351e00640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13541600b0 0x7f135406f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1351e00640 1 -- 172.21.15.66:0/2339150723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13541521e0 con 0x7f13540b1e10 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f134be00640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f135415f450 0x7f13540ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1351e00640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13540b1e10 0x7f1354150290 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f134401ae40 tx=0x7f1344005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f134a000640 1 -- 172.21.15.66:0/2339150723 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13440bb070 con 0x7f13540b1e10 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f134a000640 1 -- 172.21.15.66:0/2339150723 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13440a5cc0 con 0x7f13540b1e10 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f134a000640 1 -- 172.21.15.66:0/2339150723 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13440b2040 con 0x7f13540b1e10 2024-09-10T22:14:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1359000640 1 -- 172.21.15.66:0/2339150723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13541524c0 con 0x7f13540b1e10 2024-09-10T22:14:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.623+0000 7f1359000640 1 -- 172.21.15.66:0/2339150723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1354152a80 con 0x7f13540b1e10 2024-09-10T22:14:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.627+0000 7f1359000640 1 -- 172.21.15.66:0/2339150723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f135415fb70 con 0x7f13540b1e10 2024-09-10T22:14:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.627+0000 7f134a000640 1 -- 172.21.15.66:0/2339150723 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7f13440b5030 con 0x7f13540b1e10 2024-09-10T22:14:22.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.627+0000 7f134a000640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1330074710 0x7f1330076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:22.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.627+0000 7f134a000640 1 -- 172.21.15.66:0/2339150723 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f134412be40 con 0x7f13540b1e10 2024-09-10T22:14:22.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.627+0000 7f134be00640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1330074710 0x7f1330076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.631+0000 7f134be00640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1330074710 0x7f1330076bd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f133c0058d0 tx=0x7f133c005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:22.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.631+0000 7f134a000640 1 -- 172.21.15.66:0/2339150723 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1344016410 con 0x7f13540b1e10 2024-09-10T22:14:22.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.831+0000 7f1359000640 1 -- 172.21.15.66:0/2339150723 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f1354152e20 con 0x7f1330074710 2024-09-10T22:14:22.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.835+0000 7f134a000640 1 -- 172.21.15.66:0/2339150723 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f1354152e20 con 0x7f1330074710 2024-09-10T22:14:22.841 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/9c08db3e-b973-4f8c-ac9b-4fd7276890c6 2024-09-10T22:14:22.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.839+0000 7f1359000640 1 -- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1330074710 msgr2=0x7f1330076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.839+0000 7f1359000640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1330074710 0x7f1330076bd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f133c0058d0 tx=0x7f133c005860 comp rx=0 tx=0).stop 2024-09-10T22:14:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.839+0000 7f1359000640 1 -- 172.21.15.66:0/2339150723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13540b1e10 msgr2=0x7f1354150290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.839+0000 7f1359000640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13540b1e10 0x7f1354150290 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f134401ae40 tx=0x7f1344005dd0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.839+0000 7f1359000640 1 -- 172.21.15.66:0/2339150723 shutdown_connections 2024-09-10T22:14:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.839+0000 7f1359000640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1330074710 0x7f1330076bd0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.839+0000 7f1359000640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13541600b0 0x7f135406f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.839+0000 7f1359000640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f135415f450 0x7f13540ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.839+0000 7f1359000640 1 --2- 172.21.15.66:0/2339150723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13540b1e10 0x7f1354150290 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.839+0000 7f1359000640 1 -- 172.21.15.66:0/2339150723 >> 172.21.15.66:0/2339150723 conn(0x7f13540d1c60 msgr2=0x7f13540708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.839+0000 7f1359000640 1 -- 172.21.15.66:0/2339150723 shutdown_connections 2024-09-10T22:14:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:22.839+0000 7f1359000640 1 -- 172.21.15.66:0/2339150723 wait complete. 2024-09-10T22:14:22.864 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:14:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.007+0000 7fd37f000640 1 -- 172.21.15.66:0/1910937464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd37815f340 msgr2=0x7fd3780cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.007+0000 7fd37f000640 1 --2- 172.21.15.66:0/1910937464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd37815f340 0x7fd3780cfb50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd368009870 tx=0x7fd36801c820 comp rx=0 tx=0).stop 2024-09-10T22:14:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.007+0000 7fd37f000640 1 -- 172.21.15.66:0/1910937464 shutdown_connections 2024-09-10T22:14:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.007+0000 7fd37f000640 1 --2- 172.21.15.66:0/1910937464 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3780af970 0x7fd3780d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.007+0000 7fd37f000640 1 --2- 172.21.15.66:0/1910937464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd37815f340 0x7fd3780cfb50 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.007+0000 7fd37f000640 1 -- 172.21.15.66:0/1910937464 >> 172.21.15.66:0/1910937464 conn(0x7fd3780d1c50 msgr2=0x7fd3780d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:23.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.007+0000 7fd37f000640 1 -- 172.21.15.66:0/1910937464 shutdown_connections 2024-09-10T22:14:23.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.007+0000 7fd37f000640 1 -- 172.21.15.66:0/1910937464 wait complete. 2024-09-10T22:14:23.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd37f000640 1 Processor -- start 2024-09-10T22:14:23.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd37f000640 1 -- start start 2024-09-10T22:14:23.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd37f000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3780af970 0x7fd3780c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:23.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd37f000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3780b2bf0 0x7fd37815beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:23.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd37f000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd37815f340 0x7fd378155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:23.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd37f000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd378169190 con 0x7fd3780af970 2024-09-10T22:14:23.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd37f000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd378169010 con 0x7fd37815f340 2024-09-10T22:14:23.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd37f000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd378169310 con 0x7fd3780b2bf0 2024-09-10T22:14:23.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd377e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3780af970 0x7fd3780c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:23.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd377e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3780af970 0x7fd3780c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41060/0 (socket says 172.21.15.66:41060) 2024-09-10T22:14:23.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd377e00640 1 -- 172.21.15.66:0/3584298524 learned_addr learned my addr 172.21.15.66:0/3584298524 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:23.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd36f400640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3780b2bf0 0x7fd37815beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:23.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd37d000640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd37815f340 0x7fd378155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:23.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd377e00640 1 -- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3780b2bf0 msgr2=0x7fd37815beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:23.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd377e00640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3780b2bf0 0x7fd37815beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:23.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd377e00640 1 -- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd37815f340 msgr2=0x7fd378155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:23.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd377e00640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd37815f340 0x7fd378155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:23.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd377e00640 1 -- 172.21.15.66:0/3584298524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd378159900 con 0x7fd3780af970 2024-09-10T22:14:23.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd36f400640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3780b2bf0 0x7fd37815beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:14:23.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd377e00640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3780af970 0x7fd3780c9fa0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd368009790 tx=0x7fd368004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:23.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd374c00640 1 -- 172.21.15.66:0/3584298524 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3680bb070 con 0x7fd3780af970 2024-09-10T22:14:23.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd374c00640 1 -- 172.21.15.66:0/3584298524 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3680a5c90 con 0x7fd3780af970 2024-09-10T22:14:23.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd374c00640 1 -- 172.21.15.66:0/3584298524 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3680b2040 con 0x7fd3780af970 2024-09-10T22:14:23.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd37f000640 1 -- 172.21.15.66:0/3584298524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3780ca4e0 con 0x7fd3780af970 2024-09-10T22:14:23.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd37f000640 1 -- 172.21.15.66:0/3584298524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3780b77f0 con 0x7fd3780af970 2024-09-10T22:14:23.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.011+0000 7fd37f000640 1 -- 172.21.15.66:0/3584298524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd378159d70 con 0x7fd3780af970 2024-09-10T22:14:23.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.015+0000 7fd374c00640 1 -- 172.21.15.66:0/3584298524 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 45) ==== 99403+0+0 (secure 0 0 0) 0x7fd3680b5030 con 0x7fd3780af970 2024-09-10T22:14:23.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.015+0000 7fd374c00640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd354074710 0x7fd354076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:23.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.015+0000 7fd374c00640 1 -- 172.21.15.66:0/3584298524 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7fd36812b9f0 con 0x7fd3780af970 2024-09-10T22:14:23.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.015+0000 7fd36f400640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd354074710 0x7fd354076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:23.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.015+0000 7fd36f400640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd354074710 0x7fd354076bd0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd360007a60 tx=0x7fd360009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:23.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.019+0000 7fd374c00640 1 -- 172.21.15.66:0/3584298524 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd368016410 con 0x7fd3780af970 2024-09-10T22:14:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.199+0000 7fd374c00640 1 -- 172.21.15.66:0/3584298524 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 46) ==== 99505+0+0 (secure 0 0 0) 0x7fd3680f46a0 con 0x7fd3780af970 2024-09-10T22:14:23.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.223+0000 7fd37f000640 1 -- 172.21.15.66:0/3584298524 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fd37815fac0 con 0x7fd354074710 2024-09-10T22:14:23.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.227+0000 7fd374c00640 1 -- 172.21.15.66:0/3584298524 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fd37815fac0 con 0x7fd354074710 2024-09-10T22:14:23.232 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/9c08db3e-b973-4f8c-ac9b-4fd7276890c6 2024-09-10T22:14:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.231+0000 7fd37f000640 1 -- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd354074710 msgr2=0x7fd354076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.231+0000 7fd37f000640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd354074710 0x7fd354076bd0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd360007a60 tx=0x7fd360009040 comp rx=0 tx=0).stop 2024-09-10T22:14:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.231+0000 7fd37f000640 1 -- 172.21.15.66:0/3584298524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3780af970 msgr2=0x7fd3780c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.231+0000 7fd37f000640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3780af970 0x7fd3780c9fa0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd368009790 tx=0x7fd368004620 comp rx=0 tx=0).stop 2024-09-10T22:14:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.231+0000 7fd37f000640 1 -- 172.21.15.66:0/3584298524 shutdown_connections 2024-09-10T22:14:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.231+0000 7fd37f000640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd354074710 0x7fd354076bd0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.231+0000 7fd37f000640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd37815f340 0x7fd378155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.231+0000 7fd37f000640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3780b2bf0 0x7fd37815beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:23.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.231+0000 7fd37f000640 1 --2- 172.21.15.66:0/3584298524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3780af970 0x7fd3780c9fa0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.231+0000 7fd37f000640 1 -- 172.21.15.66:0/3584298524 >> 172.21.15.66:0/3584298524 conn(0x7fd3780d1c50 msgr2=0x7fd3780c5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.231+0000 7fd37f000640 1 -- 172.21.15.66:0/3584298524 shutdown_connections 2024-09-10T22:14:23.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:23.231+0000 7fd37f000640 1 -- 172.21.15.66:0/3584298524 wait complete. 2024-09-10T22:14:23.254 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/9c08db3e-b973-4f8c-ac9b-4fd7276890c6 2024-09-10T22:14:23.255 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:14:23.255 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:14:23.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:23.264 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9c08db3e-b973-4f8c-ac9b-4fd7276890c6/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:14:27.975 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:14:27.978 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:14:30.410 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:14:30.410 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:14:30.410 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.08431 s, 152 MB/s 2024-09-10T22:14:30.412 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:14:30.412 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:14:30.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:30.421 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9c08db3e-b973-4f8c-ac9b-4fd7276890c6/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:14:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:14:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:14:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.80552 s, 158 MB/s 2024-09-10T22:14:37.281 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:14:37.281 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:14:37.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:37.290 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9c08db3e-b973-4f8c-ac9b-4fd7276890c6/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-10T22:14:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records in 2024-09-10T22:14:44.106 INFO:teuthology.orchestra.run.smithi066.stderr:1024+0 records out 2024-09-10T22:14:44.106 INFO:teuthology.orchestra.run.smithi066.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.7601 s, 159 MB/s 2024-09-10T22:14:44.107 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-10T22:14:44.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 -- 172.21.15.66:0/3728709107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e7c15b9e0 msgr2=0x7f8e7c0d1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:44.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 --2- 172.21.15.66:0/3728709107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e7c15b9e0 0x7f8e7c0d1940 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0753d0 tx=0x7f8e6c01c600 comp rx=0 tx=0).stop 2024-09-10T22:14:44.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 -- 172.21.15.66:0/3728709107 shutdown_connections 2024-09-10T22:14:44.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 --2- 172.21.15.66:0/3728709107 >> v2:172.21.15.94:3300/0 conn(0x7f8e7c0b1170 0x7f8e7c0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 --2- 172.21.15.66:0/3728709107 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e7c15bdc0 0x7f8e7c0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 --2- 172.21.15.66:0/3728709107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e7c15b9e0 0x7f8e7c0d1940 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 -- 172.21.15.66:0/3728709107 >> 172.21.15.66:0/3728709107 conn(0x7f8e7c09aa50 msgr2=0x7f8e7c09ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:44.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 -- 172.21.15.66:0/3728709107 shutdown_connections 2024-09-10T22:14:44.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 -- 172.21.15.66:0/3728709107 wait complete. 2024-09-10T22:14:44.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 Processor -- start 2024-09-10T22:14:44.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 -- start start 2024-09-10T22:14:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e7c0b1170 0x7f8e7c0c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e7c15b9e0 0x7f8e7c15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e7c15bdc0 0x7f8e7c0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c0b1fe0 con 0x7f8e7c15bdc0 2024-09-10T22:14:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c0b1e60 con 0x7f8e7c15b9e0 2024-09-10T22:14:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7b200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8e7c0b2160 con 0x7f8e7c0b1170 2024-09-10T22:14:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e7c15bdc0 0x7f8e7c0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e79e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e7c0b1170 0x7f8e7c0c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e7c15bdc0 0x7f8e7c0b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58530/0 (socket says 172.21.15.66:58530) 2024-09-10T22:14:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e7a800640 1 -- 172.21.15.66:0/4079088626 learned_addr learned my addr 172.21.15.66:0/4079088626 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.254+0000 7f8e73e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e7c15b9e0 0x7f8e7c15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:44.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e79e00640 1 -- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e7c15b9e0 msgr2=0x7f8e7c15a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:44.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e79e00640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e7c15b9e0 0x7f8e7c15a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e79e00640 1 -- 172.21.15.66:0/4079088626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e7c15bdc0 msgr2=0x7f8e7c0b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:44.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e79e00640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e7c15bdc0 0x7f8e7c0b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e79e00640 1 -- 172.21.15.66:0/4079088626 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c0b7680 con 0x7f8e7c0b1170 2024-09-10T22:14:44.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e7a800640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e7c15bdc0 0x7f8e7c0b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:14:44.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e79e00640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e7c0b1170 0x7f8e7c0c5950 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0ac000 tx=0x7f8e6c0b0a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:44.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e72000640 1 -- 172.21.15.66:0/4079088626 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c004040 con 0x7f8e7c0b1170 2024-09-10T22:14:44.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e72000640 1 -- 172.21.15.66:0/4079088626 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e6c0041e0 con 0x7f8e7c0b1170 2024-09-10T22:14:44.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e72000640 1 -- 172.21.15.66:0/4079088626 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c005020 con 0x7f8e7c0b1170 2024-09-10T22:14:44.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e7b200640 1 -- 172.21.15.66:0/4079088626 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7c15ae40 con 0x7f8e7c0b1170 2024-09-10T22:14:44.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e7b200640 1 -- 172.21.15.66:0/4079088626 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7c0c75c0 con 0x7f8e7c0b1170 2024-09-10T22:14:44.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e7b200640 1 -- 172.21.15.66:0/4079088626 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e7c0ce540 con 0x7f8e7c0b1170 2024-09-10T22:14:44.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.258+0000 7f8e72000640 1 -- 172.21.15.66:0/4079088626 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 46) ==== 99505+0+0 (secure 0 0 0) 0x7f8e6c0051c0 con 0x7f8e7c0b1170 2024-09-10T22:14:44.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.262+0000 7f8e72000640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8e58074830 0x7f8e58076cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:44.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.262+0000 7f8e72000640 1 -- 172.21.15.66:0/4079088626 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f8e6c12bb10 con 0x7f8e7c0b1170 2024-09-10T22:14:44.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.262+0000 7f8e73e00640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8e58074830 0x7f8e58076cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:44.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.262+0000 7f8e73e00640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8e58074830 0x7f8e58076cf0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c165870 tx=0x7f8e6400a2d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.266+0000 7f8e72000640 1 -- 172.21.15.66:0/4079088626 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8e6c0ac030 con 0x7f8e7c0b1170 2024-09-10T22:14:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.470+0000 7f8e7b200640 1 -- 172.21.15.66:0/4079088626 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f8e7c15b540 con 0x7f8e58074830 2024-09-10T22:14:44.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e72000640 1 -- 172.21.15.66:0/4079088626 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8e7c15b540 con 0x7f8e58074830 2024-09-10T22:14:44.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e7b200640 1 -- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8e58074830 msgr2=0x7f8e58076cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:44.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e7b200640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8e58074830 0x7f8e58076cf0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c165870 tx=0x7f8e6400a2d0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e7b200640 1 -- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e7c0b1170 msgr2=0x7f8e7c0c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:44.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e7b200640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e7c0b1170 0x7f8e7c0c5950 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0ac000 tx=0x7f8e6c0b0a40 comp rx=0 tx=0).stop 2024-09-10T22:14:44.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e7b200640 1 -- 172.21.15.66:0/4079088626 shutdown_connections 2024-09-10T22:14:44.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e7b200640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8e58074830 0x7f8e58076cf0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e7b200640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e7c15bdc0 0x7f8e7c0b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e7b200640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e7c15b9e0 0x7f8e7c15a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e7b200640 1 --2- 172.21.15.66:0/4079088626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8e7c0b1170 0x7f8e7c0c5950 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e7b200640 1 -- 172.21.15.66:0/4079088626 >> 172.21.15.66:0/4079088626 conn(0x7f8e7c09aa50 msgr2=0x7f8e7c0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:44.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e7b200640 1 -- 172.21.15.66:0/4079088626 shutdown_connections 2024-09-10T22:14:44.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.482+0000 7f8e7b200640 1 -- 172.21.15.66:0/4079088626 wait complete. 2024-09-10T22:14:44.507 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-10T22:14:44.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 -- 172.21.15.66:0/824217834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c8c15f300 msgr2=0x7f4c8c0b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:44.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 --2- 172.21.15.66:0/824217834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c8c15f300 0x7f4c8c0b1090 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009840 tx=0x7f4c7401c7e0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 -- 172.21.15.66:0/824217834 shutdown_connections 2024-09-10T22:14:44.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 --2- 172.21.15.66:0/824217834 >> v2:172.21.15.94:3300/0 conn(0x7f4c8c0b15d0 0x7f4c8c0cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 --2- 172.21.15.66:0/824217834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c8c15f300 0x7f4c8c0b1090 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 -- 172.21.15.66:0/824217834 >> 172.21.15.66:0/824217834 conn(0x7f4c8c0d1b10 msgr2=0x7f4c8c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:44.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 -- 172.21.15.66:0/824217834 shutdown_connections 2024-09-10T22:14:44.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 -- 172.21.15.66:0/824217834 wait complete. 2024-09-10T22:14:44.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 Processor -- start 2024-09-10T22:14:44.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 -- start start 2024-09-10T22:14:44.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c8c0b15d0 0x7f4c8c15a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:44.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4c8c15ef20 0x7f4c8c070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:44.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c8c15f300 0x7f4c8c070b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c1692c0 con 0x7f4c8c0b15d0 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c169140 con 0x7f4c8c15f300 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c91600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f4c8c169440 con 0x7f4c8c15ef20 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c8be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c8c0b15d0 0x7f4c8c15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c8be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c8c0b15d0 0x7f4c8c15a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58554/0 (socket says 172.21.15.66:58554) 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c8be00640 1 -- 172.21.15.66:0/3405011996 learned_addr learned my addr 172.21.15.66:0/3405011996 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c90c00640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c8c15f300 0x7f4c8c070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c83e00640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4c8c15ef20 0x7f4c8c070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c8be00640 1 -- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4c8c15ef20 msgr2=0x7f4c8c070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c8be00640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4c8c15ef20 0x7f4c8c070620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c8be00640 1 -- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c8c15f300 msgr2=0x7f4c8c070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c8be00640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c8c15f300 0x7f4c8c070b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c8be00640 1 -- 172.21.15.66:0/3405011996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c06fbd0 con 0x7f4c8c0b15d0 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.654+0000 7f4c8be00640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c8c0b15d0 0x7f4c8c15a500 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00c4c0 tx=0x7f4c7c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.658+0000 7f4c88c00640 1 -- 172.21.15.66:0/3405011996 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c016020 con 0x7f4c8c0b15d0 2024-09-10T22:14:44.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.658+0000 7f4c88c00640 1 -- 172.21.15.66:0/3405011996 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c7c012940 con 0x7f4c8c0b15d0 2024-09-10T22:14:44.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.658+0000 7f4c88c00640 1 -- 172.21.15.66:0/3405011996 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c012d90 con 0x7f4c8c0b15d0 2024-09-10T22:14:44.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.658+0000 7f4c91600640 1 -- 172.21.15.66:0/3405011996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c06df80 con 0x7f4c8c0b15d0 2024-09-10T22:14:44.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.658+0000 7f4c91600640 1 -- 172.21.15.66:0/3405011996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c06e540 con 0x7f4c8c0b15d0 2024-09-10T22:14:44.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.658+0000 7f4c91600640 1 -- 172.21.15.66:0/3405011996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c06e740 con 0x7f4c8c0b15d0 2024-09-10T22:14:44.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.658+0000 7f4c88c00640 1 -- 172.21.15.66:0/3405011996 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 46) ==== 99505+0+0 (secure 0 0 0) 0x7f4c7c01b400 con 0x7f4c8c0b15d0 2024-09-10T22:14:44.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.658+0000 7f4c88c00640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4c68074830 0x7f4c68076cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:44.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.658+0000 7f4c88c00640 1 -- 172.21.15.66:0/3405011996 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f4c7c092760 con 0x7f4c8c0b15d0 2024-09-10T22:14:44.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.658+0000 7f4c83e00640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4c68074830 0x7f4c68076cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:44.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.662+0000 7f4c83e00640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4c68074830 0x7f4c68076cf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4c740af000 tx=0x7f4c7401b020 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:44.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.662+0000 7f4c88c00640 1 -- 172.21.15.66:0/3405011996 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4c7c020020 con 0x7f4c8c0b15d0 2024-09-10T22:14:44.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.862+0000 7f4c91600640 1 -- 172.21.15.66:0/3405011996 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f4c8c072e00 con 0x7f4c68074830 2024-09-10T22:14:44.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c88c00640 1 -- 172.21.15.66:0/3405011996 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f4c8c072e00 con 0x7f4c68074830 2024-09-10T22:14:44.875 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv1/9c08db3e-b973-4f8c-ac9b-4fd7276890c6 2024-09-10T22:14:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c91600640 1 -- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4c68074830 msgr2=0x7f4c68076cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c91600640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4c68074830 0x7f4c68076cf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4c740af000 tx=0x7f4c7401b020 comp rx=0 tx=0).stop 2024-09-10T22:14:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c91600640 1 -- 172.21.15.66:0/3405011996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c8c0b15d0 msgr2=0x7f4c8c15a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c91600640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c8c0b15d0 0x7f4c8c15a500 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00c4c0 tx=0x7f4c7c00c980 comp rx=0 tx=0).stop 2024-09-10T22:14:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c91600640 1 -- 172.21.15.66:0/3405011996 shutdown_connections 2024-09-10T22:14:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c91600640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4c68074830 0x7f4c68076cf0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c91600640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c8c15f300 0x7f4c8c070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c91600640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4c8c15ef20 0x7f4c8c070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c91600640 1 --2- 172.21.15.66:0/3405011996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c8c0b15d0 0x7f4c8c15a500 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:44.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c91600640 1 -- 172.21.15.66:0/3405011996 >> 172.21.15.66:0/3405011996 conn(0x7f4c8c0d1b10 msgr2=0x7f4c8c15cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:44.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c91600640 1 -- 172.21.15.66:0/3405011996 shutdown_connections 2024-09-10T22:14:44.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:44.870+0000 7f4c91600640 1 -- 172.21.15.66:0/3405011996 wait complete. 2024-09-10T22:14:44.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:44.896 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9c08db3e-b973-4f8c-ac9b-4fd7276890c6 2024-09-10T22:14:44.896 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:14:44.909 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/9c08db3e-b973-4f8c-ac9b-4fd7276890c6 2024-09-10T22:14:44.909 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3138256896" 2024-09-10T22:14:44.909 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:14:44.910 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-09-10T22:14:46.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:14:46.911 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9c08db3e-b973-4f8c-ac9b-4fd7276890c6 2024-09-10T22:14:46.911 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:14:46.925 INFO:teuthology.orchestra.run.smithi066.stdout:# file: volumes/_nogroup/sv1/9c08db3e-b973-4f8c-ac9b-4fd7276890c6 2024-09-10T22:14:46.925 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.rbytes="3221225472" 2024-09-10T22:14:46.925 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:14:46.926 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-10T22:14:46.927 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone36130 2024-09-10T22:14:47.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.126+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3952999476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac015ef20 msgr2=0x7f6ac00cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:47.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.126+0000 7f6ac4a00640 1 --2- 172.21.15.66:0/3952999476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac015ef20 0x7f6ac00cecc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009840 tx=0x7f6ab001c7f0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.126+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3952999476 shutdown_connections 2024-09-10T22:14:47.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.126+0000 7f6ac4a00640 1 --2- 172.21.15.66:0/3952999476 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ac015f300 0x7f6ac00b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.126+0000 7f6ac4a00640 1 --2- 172.21.15.66:0/3952999476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac015ef20 0x7f6ac00cecc0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.126+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3952999476 >> 172.21.15.66:0/3952999476 conn(0x7f6ac00d1b10 msgr2=0x7f6ac00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:47.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.126+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3952999476 shutdown_connections 2024-09-10T22:14:47.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.126+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3952999476 wait complete. 2024-09-10T22:14:47.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ac4a00640 1 Processor -- start 2024-09-10T22:14:47.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ac4a00640 1 -- start start 2024-09-10T22:14:47.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ac4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ac00b15d0 0x7f6ac006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:47.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ac4a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ac015ef20 0x7f6ac014de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:47.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ac4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac015f300 0x7f6ac014e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:47.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ac4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac0168d60 con 0x7f6ac00b15d0 2024-09-10T22:14:47.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ac4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac0168ee0 con 0x7f6ac015f300 2024-09-10T22:14:47.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ac4a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac0169060 con 0x7f6ac015ef20 2024-09-10T22:14:47.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6abf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ac00b15d0 0x7f6ac006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:47.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6abf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ac00b15d0 0x7f6ac006e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58566/0 (socket says 172.21.15.66:58566) 2024-09-10T22:14:47.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6abf400640 1 -- 172.21.15.66:0/3505796383 learned_addr learned my addr 172.21.15.66:0/3505796383 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:47.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6abfe00640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac015f300 0x7f6ac014e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:47.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ab7e00640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ac015ef20 0x7f6ac014de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:47.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6abf400640 1 -- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ac015ef20 msgr2=0x7f6ac014de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:47.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6abf400640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ac015ef20 0x7f6ac014de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6abf400640 1 -- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac015f300 msgr2=0x7f6ac014e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:47.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6abf400640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac015f300 0x7f6ac014e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6abf400640 1 -- 172.21.15.66:0/3505796383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac01fc310 con 0x7f6ac00b15d0 2024-09-10T22:14:47.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6abf400640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ac00b15d0 0x7f6ac006e3f0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009790 tx=0x7f6ab0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:47.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ab7400640 1 -- 172.21.15.66:0/3505796383 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00b2040 con 0x7f6ac00b15d0 2024-09-10T22:14:47.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ab7400640 1 -- 172.21.15.66:0/3505796383 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ab0004240 con 0x7f6ac00b15d0 2024-09-10T22:14:47.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ab7400640 1 -- 172.21.15.66:0/3505796383 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0005020 con 0x7f6ac00b15d0 2024-09-10T22:14:47.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3505796383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac01f9350 con 0x7f6ac00b15d0 2024-09-10T22:14:47.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.130+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3505796383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac006e930 con 0x7f6ac00b15d0 2024-09-10T22:14:47.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.134+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3505796383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac00cecc0 con 0x7f6ac00b15d0 2024-09-10T22:14:47.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.134+0000 7f6ab7400640 1 -- 172.21.15.66:0/3505796383 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 46) ==== 99505+0+0 (secure 0 0 0) 0x7f6ab00b5070 con 0x7f6ac00b15d0 2024-09-10T22:14:47.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.134+0000 7f6ab7400640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a9c074830 0x7f6a9c076cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:47.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.134+0000 7f6ab7400640 1 -- 172.21.15.66:0/3505796383 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f6ab012bb40 con 0x7f6ac00b15d0 2024-09-10T22:14:47.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.134+0000 7f6ab7e00640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a9c074830 0x7f6a9c076cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:47.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.134+0000 7f6ab7e00640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a9c074830 0x7f6a9c076cf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6ac015f770 tx=0x7f6aa80091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:47.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.138+0000 7f6ab7400640 1 -- 172.21.15.66:0/3505796383 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6ab0016410 con 0x7f6ac00b15d0 2024-09-10T22:14:47.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.346+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3505796383 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone36130", "target": ["mon-mgr", ""]}) -- 0x7f6ac00ce0d0 con 0x7f6a9c074830 2024-09-10T22:14:47.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.406+0000 7f6ab7400640 1 -- 172.21.15.66:0/3505796383 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6aa8009720 con 0x7f6a9c074830 2024-09-10T22:14:47.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.410+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a9c074830 msgr2=0x7f6a9c076cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:47.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.410+0000 7f6ac4a00640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a9c074830 0x7f6a9c076cf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6ac015f770 tx=0x7f6aa80091e0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.410+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3505796383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ac00b15d0 msgr2=0x7f6ac006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:47.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.410+0000 7f6ac4a00640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ac00b15d0 0x7f6ac006e3f0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009790 tx=0x7f6ab0005db0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.410+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3505796383 shutdown_connections 2024-09-10T22:14:47.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.410+0000 7f6ac4a00640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6a9c074830 0x7f6a9c076cf0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.410+0000 7f6ac4a00640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac015f300 0x7f6ac014e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.410+0000 7f6ac4a00640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ac015ef20 0x7f6ac014de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.410+0000 7f6ac4a00640 1 --2- 172.21.15.66:0/3505796383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ac00b15d0 0x7f6ac006e3f0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.410+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3505796383 >> 172.21.15.66:0/3505796383 conn(0x7f6ac00d1b10 msgr2=0x7f6ac01662d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:47.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.410+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3505796383 shutdown_connections 2024-09-10T22:14:47.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.410+0000 7f6ac4a00640 1 -- 172.21.15.66:0/3505796383 wait complete. 2024-09-10T22:14:47.419 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:47.418+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:47.419 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:47.418+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:47.419 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:47.418+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:47.420 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:47.418+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:47.420 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:47.418+0000 7f5feec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:47.437 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99941 2024-09-10T22:14:47.437 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:47.434+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:47.438 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:47.434+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:47.438 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:47.434+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:47.438 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:47.434+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:47.438 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:47.434+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:47.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 -- 172.21.15.66:0/4035346749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f050415f450 msgr2=0x7f05040b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:47.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 --2- 172.21.15.66:0/4035346749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f050415f450 0x7f05040b3170 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f05040b3080 tx=0x7f04f401d240 comp rx=0 tx=0).stop 2024-09-10T22:14:47.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 -- 172.21.15.66:0/4035346749 shutdown_connections 2024-09-10T22:14:47.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 --2- 172.21.15.66:0/4035346749 >> v2:172.21.15.94:3300/0 conn(0x7f0504160280 0x7f05040aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 --2- 172.21.15.66:0/4035346749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f050415f450 0x7f05040b3170 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 -- 172.21.15.66:0/4035346749 >> 172.21.15.66:0/4035346749 conn(0x7f05040d1c60 msgr2=0x7f05040d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:47.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 -- 172.21.15.66:0/4035346749 shutdown_connections 2024-09-10T22:14:47.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 -- 172.21.15.66:0/4035346749 wait complete. 2024-09-10T22:14:47.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 Processor -- start 2024-09-10T22:14:47.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 -- start start 2024-09-10T22:14:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05040b1e10 0x7f0504155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f050415f450 0x7f05040ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0504160280 0x7f05040b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05040aff60 con 0x7f0504160280 2024-09-10T22:14:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05040afde0 con 0x7f050415f450 2024-09-10T22:14:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f050ac00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0504168eb0 con 0x7f05040b1e10 2024-09-10T22:14:47.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f0508c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0504160280 0x7f05040b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f0508c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0504160280 0x7f05040b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58576/0 (socket says 172.21.15.66:58576) 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.590+0000 7f0508c00640 1 -- 172.21.15.66:0/386566206 learned_addr learned my addr 172.21.15.66:0/386566206 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f0503e00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05040b1e10 0x7f0504155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f04fbe00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f050415f450 0x7f05040ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f0508c00640 1 -- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05040b1e10 msgr2=0x7f0504155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f0508c00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05040b1e10 0x7f0504155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f0508c00640 1 -- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f050415f450 msgr2=0x7f05040ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f0508c00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f050415f450 0x7f05040ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f0508c00640 1 -- 172.21.15.66:0/386566206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05040b66f0 con 0x7f0504160280 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f04fbe00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f050415f450 0x7f05040ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f0508c00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0504160280 0x7f05040b5c30 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00e910 tx=0x7f04fc00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f0500c00640 1 -- 172.21.15.66:0/386566206 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc004d30 con 0x7f0504160280 2024-09-10T22:14:47.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f0500c00640 1 -- 172.21.15.66:0/386566206 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04fc00cb20 con 0x7f0504160280 2024-09-10T22:14:47.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f0500c00640 1 -- 172.21.15.66:0/386566206 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc005020 con 0x7f0504160280 2024-09-10T22:14:47.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f050ac00640 1 -- 172.21.15.66:0/386566206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05040b69d0 con 0x7f0504160280 2024-09-10T22:14:47.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f050ac00640 1 -- 172.21.15.66:0/386566206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05041561e0 con 0x7f0504160280 2024-09-10T22:14:47.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f050ac00640 1 -- 172.21.15.66:0/386566206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05040b3170 con 0x7f0504160280 2024-09-10T22:14:47.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.594+0000 7f0500c00640 1 -- 172.21.15.66:0/386566206 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 46) ==== 99505+0+0 (secure 0 0 0) 0x7f04fc022060 con 0x7f0504160280 2024-09-10T22:14:47.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.598+0000 7f0500c00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04e0074830 0x7f04e0076cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:47.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.598+0000 7f0500c00640 1 -- 172.21.15.66:0/386566206 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f04fc092820 con 0x7f0504160280 2024-09-10T22:14:47.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.598+0000 7f0503e00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04e0074830 0x7f04e0076cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:47.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.598+0000 7f0503e00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04e0074830 0x7f04e0076cf0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f04f40af000 tx=0x7f04f40a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:47.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.602+0000 7f0500c00640 1 -- 172.21.15.66:0/386566206 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f04fc016030 con 0x7f0504160280 2024-09-10T22:14:47.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:47.806+0000 7f050ac00640 1 -- 172.21.15.66:0/386566206 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99941", "target": ["mon-mgr", ""]}) -- 0x7f05040d2a40 con 0x7f04e0074830 2024-09-10T22:14:48.419 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:48.418+0000 7f5f71c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:48.419 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:48.418+0000 7f5f71c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:48.419 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:48.418+0000 7f5f71c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:48.419 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:48.418+0000 7f5f71c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:48.420 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:48.418+0000 7f5f71c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:48.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.486+0000 7f0500c00640 1 -- 172.21.15.66:0/386566206 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f05040d2a40 con 0x7f04e0074830 2024-09-10T22:14:48.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.490+0000 7f050ac00640 1 -- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04e0074830 msgr2=0x7f04e0076cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:48.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.490+0000 7f050ac00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04e0074830 0x7f04e0076cf0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f04f40af000 tx=0x7f04f40a7040 comp rx=0 tx=0).stop 2024-09-10T22:14:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.490+0000 7f050ac00640 1 -- 172.21.15.66:0/386566206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0504160280 msgr2=0x7f05040b5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.490+0000 7f050ac00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0504160280 0x7f05040b5c30 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00e910 tx=0x7f04fc00edd0 comp rx=0 tx=0).stop 2024-09-10T22:14:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.490+0000 7f050ac00640 1 -- 172.21.15.66:0/386566206 shutdown_connections 2024-09-10T22:14:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.490+0000 7f050ac00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f04e0074830 0x7f04e0076cf0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.490+0000 7f050ac00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0504160280 0x7f05040b5c30 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.490+0000 7f050ac00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f050415f450 0x7f05040ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.490+0000 7f050ac00640 1 --2- 172.21.15.66:0/386566206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05040b1e10 0x7f0504155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.490+0000 7f050ac00640 1 -- 172.21.15.66:0/386566206 >> 172.21.15.66:0/386566206 conn(0x7f05040d1c60 msgr2=0x7f05040cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.490+0000 7f050ac00640 1 -- 172.21.15.66:0/386566206 shutdown_connections 2024-09-10T22:14:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.490+0000 7f050ac00640 1 -- 172.21.15.66:0/386566206 wait complete. 2024-09-10T22:14:48.516 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone53292 2024-09-10T22:14:48.524 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:48.522+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:48.524 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:48.522+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:48.524 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:48.522+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:48.524 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:48.522+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:48.524 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:48.522+0000 7f5fef600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:48.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.762+0000 7f49ea200640 1 -- 172.21.15.66:0/2062048245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49c8004810 msgr2=0x7f49c8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:48.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.762+0000 7f49ea200640 1 --2- 172.21.15.66:0/2062048245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49c8004810 0x7f49c8004c90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0016c0 tx=0x7f49cc01c700 comp rx=0 tx=0).stop 2024-09-10T22:14:48.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.762+0000 7f49ea200640 1 -- 172.21.15.66:0/2062048245 shutdown_connections 2024-09-10T22:14:48.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.762+0000 7f49ea200640 1 --2- 172.21.15.66:0/2062048245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49c8004810 0x7f49c8004c90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:48.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.762+0000 7f49ea200640 1 -- 172.21.15.66:0/2062048245 >> 172.21.15.66:0/2062048245 conn(0x7f49e40cfa30 msgr2=0x7f49e40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:48.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.762+0000 7f49ea200640 1 -- 172.21.15.66:0/2062048245 shutdown_connections 2024-09-10T22:14:48.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.762+0000 7f49ea200640 1 -- 172.21.15.66:0/2062048245 wait complete. 2024-09-10T22:14:48.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.770+0000 7f49ea200640 1 Processor -- start 2024-09-10T22:14:48.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.770+0000 7f49ea200640 1 -- start start 2024-09-10T22:14:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.770+0000 7f49ea200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49c8004810 0x7f49e414f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.770+0000 7f49ea200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e4154050 0x7f49e40adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.770+0000 7f49ea200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e414d190 0x7f49e414d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.770+0000 7f49ea200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f49e415a890 con 0x7f49c8004810 2024-09-10T22:14:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.770+0000 7f49ea200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f49e415a710 con 0x7f49e414d190 2024-09-10T22:14:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.770+0000 7f49ea200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f49e415aa10 con 0x7f49e4154050 2024-09-10T22:14:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49e3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49c8004810 0x7f49e414f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49e3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49c8004810 0x7f49e414f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58602/0 (socket says 172.21.15.66:58602) 2024-09-10T22:14:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49e3400640 1 -- 172.21.15.66:0/597240655 learned_addr learned my addr 172.21.15.66:0/597240655 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49e3e00640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e414d190 0x7f49e414d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:48.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49dbe00640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e4154050 0x7f49e40adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:48.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49e3400640 1 -- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e4154050 msgr2=0x7f49e40adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:48.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49e3400640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e4154050 0x7f49e40adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:48.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49e3400640 1 -- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e414d190 msgr2=0x7f49e414d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:48.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49e3400640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e414d190 0x7f49e414d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:48.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49e3400640 1 -- 172.21.15.66:0/597240655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e414db10 con 0x7f49c8004810 2024-09-10T22:14:48.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49e3400640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49c8004810 0x7f49e414f5f0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f49d4006430 tx=0x7f49d40039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49db400640 1 -- 172.21.15.66:0/597240655 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4015070 con 0x7f49c8004810 2024-09-10T22:14:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49db400640 1 -- 172.21.15.66:0/597240655 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49d4011900 con 0x7f49c8004810 2024-09-10T22:14:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49ea200640 1 -- 172.21.15.66:0/597240655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e4152680 con 0x7f49c8004810 2024-09-10T22:14:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49db400640 1 -- 172.21.15.66:0/597240655 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4011d50 con 0x7f49c8004810 2024-09-10T22:14:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.774+0000 7f49ea200640 1 -- 172.21.15.66:0/597240655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e4152c40 con 0x7f49c8004810 2024-09-10T22:14:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.778+0000 7f49ea200640 1 -- 172.21.15.66:0/597240655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e415afa0 con 0x7f49c8004810 2024-09-10T22:14:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.778+0000 7f49db400640 1 -- 172.21.15.66:0/597240655 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 46) ==== 99505+0+0 (secure 0 0 0) 0x7f49d4024440 con 0x7f49c8004810 2024-09-10T22:14:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.782+0000 7f49db400640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f49c0074830 0x7f49c0076cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.782+0000 7f49db400640 1 -- 172.21.15.66:0/597240655 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f49d40934d0 con 0x7f49c8004810 2024-09-10T22:14:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.782+0000 7f49dbe00640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f49c0074830 0x7f49c0076cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.782+0000 7f49dbe00640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f49c0074830 0x7f49c0076cf0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f49c8004790 tx=0x7f49cc0a8040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:48.786+0000 7f49db400640 1 -- 172.21.15.66:0/597240655 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f49d4060140 con 0x7f49c8004810 2024-09-10T22:14:49.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.154+0000 7f49ea200640 1 -- 172.21.15.66:0/597240655 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone53292", "target": ["mon-mgr", ""]}) -- 0x7f49e40ccef0 con 0x7f49c0074830 2024-09-10T22:14:49.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.290+0000 7f49db400640 1 -- 172.21.15.66:0/597240655 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 47) ==== 99709+0+0 (secure 0 0 0) 0x7f49d405c3b0 con 0x7f49c8004810 2024-09-10T22:14:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.766+0000 7f49db400640 1 -- 172.21.15.66:0/597240655 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f49e40ccef0 con 0x7f49c0074830 2024-09-10T22:14:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.770+0000 7f49ea200640 1 -- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f49c0074830 msgr2=0x7f49c0076cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.770+0000 7f49ea200640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f49c0074830 0x7f49c0076cf0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f49c8004790 tx=0x7f49cc0a8040 comp rx=0 tx=0).stop 2024-09-10T22:14:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.770+0000 7f49ea200640 1 -- 172.21.15.66:0/597240655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49c8004810 msgr2=0x7f49e414f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.770+0000 7f49ea200640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49c8004810 0x7f49e414f5f0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f49d4006430 tx=0x7f49d40039d0 comp rx=0 tx=0).stop 2024-09-10T22:14:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.770+0000 7f49ea200640 1 -- 172.21.15.66:0/597240655 shutdown_connections 2024-09-10T22:14:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.770+0000 7f49ea200640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f49c0074830 0x7f49c0076cf0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.770+0000 7f49ea200640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e414d190 0x7f49e414d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.770+0000 7f49ea200640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49e4154050 0x7f49e40adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.770+0000 7f49ea200640 1 --2- 172.21.15.66:0/597240655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49c8004810 0x7f49e414f5f0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.770+0000 7f49ea200640 1 -- 172.21.15.66:0/597240655 >> 172.21.15.66:0/597240655 conn(0x7f49e40cfa30 msgr2=0x7f49e40af120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.770+0000 7f49ea200640 1 -- 172.21.15.66:0/597240655 shutdown_connections 2024-09-10T22:14:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:49.770+0000 7f49ea200640 1 -- 172.21.15.66:0/597240655 wait complete. 2024-09-10T22:14:49.821 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone71863 2024-09-10T22:14:50.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.098+0000 7fcd26800640 1 -- 172.21.15.66:0/4248839234 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd1800a4b0 msgr2=0x7fcd1800a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:50.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.098+0000 7fcd26800640 1 --2- 172.21.15.66:0/4248839234 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd1800a4b0 0x7fcd1800a890 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009920 tx=0x7fcd0c01c660 comp rx=0 tx=0).stop 2024-09-10T22:14:50.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.102+0000 7fcd26800640 1 -- 172.21.15.66:0/4248839234 shutdown_connections 2024-09-10T22:14:50.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.102+0000 7fcd26800640 1 --2- 172.21.15.66:0/4248839234 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd1800a4b0 0x7fcd1800a890 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:50.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.102+0000 7fcd26800640 1 -- 172.21.15.66:0/4248839234 >> 172.21.15.66:0/4248839234 conn(0x7fcd180545a0 msgr2=0x7fcd180549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:50.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.102+0000 7fcd26800640 1 -- 172.21.15.66:0/4248839234 shutdown_connections 2024-09-10T22:14:50.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.102+0000 7fcd26800640 1 -- 172.21.15.66:0/4248839234 wait complete. 2024-09-10T22:14:50.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.106+0000 7fcd26800640 1 Processor -- start 2024-09-10T22:14:50.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.106+0000 7fcd26800640 1 -- start start 2024-09-10T22:14:50.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd26800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1800a4b0 0x7fcd180477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:50.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd26800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd1800ba40 0x7fcd18180800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:50.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd26800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd18036730 0x7fcd1817f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:50.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd26800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd180ee2f0 con 0x7fcd1800a4b0 2024-09-10T22:14:50.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd26800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd180ee170 con 0x7fcd18036730 2024-09-10T22:14:50.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd26800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcd180ee470 con 0x7fcd1800ba40 2024-09-10T22:14:50.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd1ea00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd1800ba40 0x7fcd18180800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:50.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd1ea00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd1800ba40 0x7fcd18180800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:49466/0 (socket says 172.21.15.66:49466) 2024-09-10T22:14:50.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd1ea00640 1 -- 172.21.15.66:0/2302598374 learned_addr learned my addr 172.21.15.66:0/2302598374 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:50.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd1fe00640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd18036730 0x7fcd1817f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:50.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd1fe00640 1 -- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd1800ba40 msgr2=0x7fcd18180800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:50.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd1fe00640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd1800ba40 0x7fcd18180800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:50.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd1fe00640 1 -- 172.21.15.66:0/2302598374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1800a4b0 msgr2=0x7fcd180477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:50.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd1fe00640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1800a4b0 0x7fcd180477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:50.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd1fe00640 1 -- 172.21.15.66:0/2302598374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1804aae0 con 0x7fcd18036730 2024-09-10T22:14:50.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd1ea00640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd1800ba40 0x7fcd18180800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:14:50.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd1fe00640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd18036730 0x7fcd1817f5c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fcd200773a0 tx=0x7fcd2006c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:50.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.110+0000 7fcd13400640 1 -- 172.21.15.66:0/2302598374 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2008f070 con 0x7fcd18036730 2024-09-10T22:14:50.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.114+0000 7fcd13400640 1 -- 172.21.15.66:0/2302598374 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd200936c0 con 0x7fcd18036730 2024-09-10T22:14:50.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.114+0000 7fcd13400640 1 -- 172.21.15.66:0/2302598374 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20093b50 con 0x7fcd18036730 2024-09-10T22:14:50.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.114+0000 7fcd26800640 1 -- 172.21.15.66:0/2302598374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1804ad80 con 0x7fcd18036730 2024-09-10T22:14:50.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.114+0000 7fcd26800640 1 -- 172.21.15.66:0/2302598374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd18047d20 con 0x7fcd18036730 2024-09-10T22:14:50.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.118+0000 7fcd26800640 1 -- 172.21.15.66:0/2302598374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccfc0052a0 con 0x7fcd18036730 2024-09-10T22:14:50.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.118+0000 7fcd13400640 1 -- 172.21.15.66:0/2302598374 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 47) ==== 99709+0+0 (secure 0 0 0) 0x7fcd20093cf0 con 0x7fcd18036730 2024-09-10T22:14:50.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.118+0000 7fcd13400640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd00074b40 0x7fcd00077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:50.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.122+0000 7fcd13400640 1 -- 172.21.15.66:0/2302598374 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7fcd2010d2f0 con 0x7fcd18036730 2024-09-10T22:14:50.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.126+0000 7fcd1f400640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd00074b40 0x7fcd00077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:50.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.130+0000 7fcd13400640 1 -- 172.21.15.66:0/2302598374 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcd200d9e60 con 0x7fcd18036730 2024-09-10T22:14:50.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.138+0000 7fcd1f400640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd00074b40 0x7fcd00077000 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0ae000 tx=0x7fcd0c00b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:50.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:50.530+0000 7fcd26800640 1 -- 172.21.15.66:0/2302598374 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone71863", "target": ["mon-mgr", ""]}) -- 0x7fccfc000c90 con 0x7fcd00074b40 2024-09-10T22:14:51.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.194+0000 7fcd13400640 1 -- 172.21.15.66:0/2302598374 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fccfc000c90 con 0x7fcd00074b40 2024-09-10T22:14:51.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.202+0000 7fcd26800640 1 -- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd00074b40 msgr2=0x7fcd00077000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:51.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.202+0000 7fcd26800640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd00074b40 0x7fcd00077000 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0ae000 tx=0x7fcd0c00b040 comp rx=0 tx=0).stop 2024-09-10T22:14:51.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.202+0000 7fcd26800640 1 -- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd18036730 msgr2=0x7fcd1817f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:51.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.202+0000 7fcd26800640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd18036730 0x7fcd1817f5c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fcd200773a0 tx=0x7fcd2006c7e0 comp rx=0 tx=0).stop 2024-09-10T22:14:51.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.202+0000 7fcd26800640 1 -- 172.21.15.66:0/2302598374 shutdown_connections 2024-09-10T22:14:51.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.202+0000 7fcd26800640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fcd00074b40 0x7fcd00077000 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:51.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.202+0000 7fcd26800640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd18036730 0x7fcd1817f5c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:51.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.202+0000 7fcd26800640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd1800ba40 0x7fcd18180800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:51.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.202+0000 7fcd26800640 1 --2- 172.21.15.66:0/2302598374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1800a4b0 0x7fcd180477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:51.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.202+0000 7fcd26800640 1 -- 172.21.15.66:0/2302598374 >> 172.21.15.66:0/2302598374 conn(0x7fcd180545a0 msgr2=0x7fcd18033f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:51.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.202+0000 7fcd26800640 1 -- 172.21.15.66:0/2302598374 shutdown_connections 2024-09-10T22:14:51.216 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:51.214+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:51.216 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:51.214+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:51.216 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:51.214+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:51.216 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:51.214+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:51.216 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:14:51.214+0000 7f5ff0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:14:51.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.210+0000 7fcd26800640 1 -- 172.21.15.66:0/2302598374 wait complete. 2024-09-10T22:14:51.274 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99264 2024-09-10T22:14:51.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.538+0000 7fb05d800640 1 -- 172.21.15.66:0/1657460226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb03c004810 msgr2=0x7fb03c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:51.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.538+0000 7fb05d800640 1 --2- 172.21.15.66:0/1657460226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb03c004810 0x7fb03c004c90 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fb040009940 tx=0x7fb04001c720 comp rx=0 tx=0).stop 2024-09-10T22:14:51.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.538+0000 7fb05d800640 1 -- 172.21.15.66:0/1657460226 shutdown_connections 2024-09-10T22:14:51.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.538+0000 7fb05d800640 1 --2- 172.21.15.66:0/1657460226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb03c004810 0x7fb03c004c90 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:51.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.538+0000 7fb05d800640 1 -- 172.21.15.66:0/1657460226 >> 172.21.15.66:0/1657460226 conn(0x7fb0580cfa30 msgr2=0x7fb0580cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:51.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.538+0000 7fb05d800640 1 -- 172.21.15.66:0/1657460226 shutdown_connections 2024-09-10T22:14:51.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.538+0000 7fb05d800640 1 -- 172.21.15.66:0/1657460226 wait complete. 2024-09-10T22:14:51.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.538+0000 7fb05d800640 1 Processor -- start 2024-09-10T22:14:51.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.538+0000 7fb05d800640 1 -- start start 2024-09-10T22:14:51.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb05d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb03c004810 0x7fb05815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:51.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb056a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb03c004810 0x7fb05815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:51.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb05d800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0580b3990 0x7fb0580ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:51.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb056a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb03c004810 0x7fb05815c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58624/0 (socket says 172.21.15.66:58624) 2024-09-10T22:14:51.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb056a00640 1 -- 172.21.15.66:0/3250663346 learned_addr learned my addr 172.21.15.66:0/3250663346 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:51.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb05d800640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0580ade00 0x7fb0580c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:51.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb0580c5360 con 0x7fb03c004810 2024-09-10T22:14:51.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb05806ebc0 con 0x7fb0580ade00 2024-09-10T22:14:51.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb0580c54e0 con 0x7fb0580b3990 2024-09-10T22:14:51.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb056a00640 1 -- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0580b3990 msgr2=0x7fb0580ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb056a00640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0580b3990 0x7fb0580ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb056a00640 1 -- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0580ade00 msgr2=0x7fb0580c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb056a00640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0580ade00 0x7fb0580c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb056a00640 1 -- 172.21.15.66:0/3250663346 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0580b3e10 con 0x7fb03c004810 2024-09-10T22:14:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb056a00640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb03c004810 0x7fb05815c3c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fb048010c90 tx=0x7fb04800e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb04f400640 1 -- 172.21.15.66:0/3250663346 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048010e40 con 0x7fb03c004810 2024-09-10T22:14:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb04f400640 1 -- 172.21.15.66:0/3250663346 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb048018020 con 0x7fb03c004810 2024-09-10T22:14:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0580b40c0 con 0x7fb03c004810 2024-09-10T22:14:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb04f400640 1 -- 172.21.15.66:0/3250663346 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0480137d0 con 0x7fb03c004810 2024-09-10T22:14:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.542+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0580b4680 con 0x7fb03c004810 2024-09-10T22:14:51.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.546+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0580b49c0 con 0x7fb03c004810 2024-09-10T22:14:51.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.546+0000 7fb04f400640 1 -- 172.21.15.66:0/3250663346 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 47) ==== 99709+0+0 (secure 0 0 0) 0x7fb048013970 con 0x7fb03c004810 2024-09-10T22:14:51.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.550+0000 7fb04f400640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb034074aa0 0x7fb034076f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:51.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.550+0000 7fb04f400640 1 -- 172.21.15.66:0/3250663346 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7fb0480968c0 con 0x7fb03c004810 2024-09-10T22:14:51.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.554+0000 7fb04f400640 1 -- 172.21.15.66:0/3250663346 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb0580b49c0 con 0x7fb03c004810 2024-09-10T22:14:51.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.558+0000 7fb04e000640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb034074aa0 0x7fb034076f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.590+0000 7fb04e000640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb034074aa0 0x7fb034076f60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb0400ae000 tx=0x7fb03c005840 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:51.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:51.890+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99264", "target": ["mon-mgr", ""]}) -- 0x7fb0580cda60 con 0x7fb034074aa0 2024-09-10T22:14:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.594+0000 7fb04f400640 1 -- 172.21.15.66:0/3250663346 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb0580cda60 con 0x7fb034074aa0 2024-09-10T22:14:52.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.598+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb034074aa0 msgr2=0x7fb034076f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:52.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.598+0000 7fb05d800640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb034074aa0 0x7fb034076f60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb0400ae000 tx=0x7fb03c005840 comp rx=0 tx=0).stop 2024-09-10T22:14:52.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.598+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb03c004810 msgr2=0x7fb05815c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:52.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.598+0000 7fb05d800640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb03c004810 0x7fb05815c3c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fb048010c90 tx=0x7fb04800e650 comp rx=0 tx=0).stop 2024-09-10T22:14:52.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.598+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 shutdown_connections 2024-09-10T22:14:52.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.598+0000 7fb05d800640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb034074aa0 0x7fb034076f60 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:52.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.598+0000 7fb05d800640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0580ade00 0x7fb0580c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.598+0000 7fb05d800640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0580b3990 0x7fb0580ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.598+0000 7fb05d800640 1 --2- 172.21.15.66:0/3250663346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb03c004810 0x7fb05815c3c0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.598+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 >> 172.21.15.66:0/3250663346 conn(0x7fb0580cfa30 msgr2=0x7fb0580afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.598+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 shutdown_connections 2024-09-10T22:14:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.598+0000 7fb05d800640 1 -- 172.21.15.66:0/3250663346 wait complete. 2024-09-10T22:14:52.641 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone98955 2024-09-10T22:14:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.926+0000 7fe3a4000640 1 -- 172.21.15.66:0/3939790004 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe39c0b4270 msgr2=0x7fe39c06ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.926+0000 7fe3a4000640 1 --2- 172.21.15.66:0/3939790004 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe39c0b4270 0x7fe39c06ffc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe390009900 tx=0x7fe39001c900 comp rx=0 tx=0).stop 2024-09-10T22:14:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.926+0000 7fe3a4000640 1 -- 172.21.15.66:0/3939790004 shutdown_connections 2024-09-10T22:14:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.926+0000 7fe3a4000640 1 --2- 172.21.15.66:0/3939790004 >> v2:172.21.15.94:3300/0 conn(0x7fe39c162ce0 0x7fe39c163150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.926+0000 7fe3a4000640 1 --2- 172.21.15.66:0/3939790004 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe39c0b4270 0x7fe39c06ffc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.926+0000 7fe3a4000640 1 -- 172.21.15.66:0/3939790004 >> 172.21.15.66:0/3939790004 conn(0x7fe39c06cc20 msgr2=0x7fe39c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.926+0000 7fe3a4000640 1 -- 172.21.15.66:0/3939790004 shutdown_connections 2024-09-10T22:14:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.926+0000 7fe3a4000640 1 -- 172.21.15.66:0/3939790004 wait complete. 2024-09-10T22:14:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.930+0000 7fe3a4000640 1 Processor -- start 2024-09-10T22:14:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.930+0000 7fe3a4000640 1 -- start start 2024-09-10T22:14:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.930+0000 7fe3a4000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe39c0b4270 0x7fe39c161c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.930+0000 7fe3a4000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe39c162ce0 0x7fe39c09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.930+0000 7fe3a4000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe39c165be0 0x7fe39c156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.930+0000 7fe3a4000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c159190 con 0x7fe39c165be0 2024-09-10T22:14:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.930+0000 7fe3a4000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c159010 con 0x7fe39c162ce0 2024-09-10T22:14:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.930+0000 7fe3a4000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe39c159310 con 0x7fe39c0b4270 2024-09-10T22:14:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.930+0000 7fe3a2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe39c165be0 0x7fe39c156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.930+0000 7fe3a2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe39c165be0 0x7fe39c156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48330/0 (socket says 172.21.15.66:48330) 2024-09-10T22:14:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.930+0000 7fe3a2000640 1 -- 172.21.15.66:0/662173735 learned_addr learned my addr 172.21.15.66:0/662173735 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:52.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe3a2000640 1 -- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe39c0b4270 msgr2=0x7fe39c161c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:52.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe3a2000640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe39c0b4270 0x7fe39c161c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:52.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe3a2000640 1 -- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe39c162ce0 msgr2=0x7fe39c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:52.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe3a0c00640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe39c162ce0 0x7fe39c09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:52.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe3a2000640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe39c162ce0 0x7fe39c09aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:52.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe3a2000640 1 -- 172.21.15.66:0/662173735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c16dba0 con 0x7fe39c165be0 2024-09-10T22:14:52.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe3a2000640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe39c165be0 0x7fe39c156680 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe39400b990 tx=0x7fe39400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:52.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe399600640 1 -- 172.21.15.66:0/662173735 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394019070 con 0x7fe39c165be0 2024-09-10T22:14:52.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe399600640 1 -- 172.21.15.66:0/662173735 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe39400cc90 con 0x7fe39c165be0 2024-09-10T22:14:52.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe399600640 1 -- 172.21.15.66:0/662173735 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3940146a0 con 0x7fe39c165be0 2024-09-10T22:14:52.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe3a4000640 1 -- 172.21.15.66:0/662173735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39c09af90 con 0x7fe39c165be0 2024-09-10T22:14:52.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe3a4000640 1 -- 172.21.15.66:0/662173735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39c09d720 con 0x7fe39c165be0 2024-09-10T22:14:52.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.934+0000 7fe3a4000640 1 -- 172.21.15.66:0/662173735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39c156eb0 con 0x7fe39c165be0 2024-09-10T22:14:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.938+0000 7fe399600640 1 -- 172.21.15.66:0/662173735 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 47) ==== 99709+0+0 (secure 0 0 0) 0x7fe39401d3b0 con 0x7fe39c165be0 2024-09-10T22:14:52.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.938+0000 7fe399600640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe37c074a70 0x7fe37c076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:52.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.938+0000 7fe399600640 1 -- 172.21.15.66:0/662173735 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7fe3940925e0 con 0x7fe39c165be0 2024-09-10T22:14:52.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.946+0000 7fe399600640 1 -- 172.21.15.66:0/662173735 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe39405f720 con 0x7fe39c165be0 2024-09-10T22:14:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.962+0000 7fe3a1600640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe37c074a70 0x7fe37c076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:52.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:52.978+0000 7fe3a1600640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe37c074a70 0x7fe37c076f30 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe3900af000 tx=0x7fe3900196a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:53.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:53.302+0000 7fe3a4000640 1 -- 172.21.15.66:0/662173735 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone98955", "target": ["mon-mgr", ""]}) -- 0x7fe39c06ff40 con 0x7fe37c074a70 2024-09-10T22:14:53.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:53.466+0000 7fe399600640 1 -- 172.21.15.66:0/662173735 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 48) ==== 99760+0+0 (secure 0 0 0) 0x7fe39405b290 con 0x7fe39c165be0 2024-09-10T22:14:56.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.234+0000 7fe399600640 1 -- 172.21.15.66:0/662173735 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe39c06ff40 con 0x7fe37c074a70 2024-09-10T22:14:56.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.242+0000 7fe3a4000640 1 -- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe37c074a70 msgr2=0x7fe37c076f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:56.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.242+0000 7fe3a4000640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe37c074a70 0x7fe37c076f30 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe3900af000 tx=0x7fe3900196a0 comp rx=0 tx=0).stop 2024-09-10T22:14:56.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.242+0000 7fe3a4000640 1 -- 172.21.15.66:0/662173735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe39c165be0 msgr2=0x7fe39c156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:56.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.242+0000 7fe3a4000640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe39c165be0 0x7fe39c156680 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe39400b990 tx=0x7fe39400be50 comp rx=0 tx=0).stop 2024-09-10T22:14:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.242+0000 7fe3a4000640 1 -- 172.21.15.66:0/662173735 shutdown_connections 2024-09-10T22:14:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.242+0000 7fe3a4000640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe37c074a70 0x7fe37c076f30 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.242+0000 7fe3a4000640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe39c165be0 0x7fe39c156680 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.242+0000 7fe3a4000640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe39c162ce0 0x7fe39c09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.242+0000 7fe3a4000640 1 --2- 172.21.15.66:0/662173735 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe39c0b4270 0x7fe39c161c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.242+0000 7fe3a4000640 1 -- 172.21.15.66:0/662173735 >> 172.21.15.66:0/662173735 conn(0x7fe39c06cc20 msgr2=0x7fe39c165070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.246+0000 7fe3a4000640 1 -- 172.21.15.66:0/662173735 shutdown_connections 2024-09-10T22:14:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.246+0000 7fe3a4000640 1 -- 172.21.15.66:0/662173735 wait complete. 2024-09-10T22:14:56.299 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone55126 2024-09-10T22:14:56.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.594+0000 7ff2abe00640 1 -- 172.21.15.66:0/1854992769 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff2ac0cb200 msgr2=0x7ff290005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:56.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.594+0000 7ff2abe00640 1 --2- 172.21.15.66:0/1854992769 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff2ac0cb200 0x7ff290005280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4001d90 tx=0x7ff2a401ca40 comp rx=0 tx=0).stop 2024-09-10T22:14:56.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.594+0000 7ff2ab400640 1 -- 172.21.15.66:0/1854992769 reap_dead start 2024-09-10T22:14:56.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.594+0000 7ff2abe00640 1 -- 172.21.15.66:0/1854992769 shutdown_connections 2024-09-10T22:14:56.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.594+0000 7ff2abe00640 1 -- 172.21.15.66:0/1854992769 >> 172.21.15.66:0/1854992769 conn(0x7ff2ac0d01a0 msgr2=0x7ff2ac0d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:56.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.594+0000 7ff2abe00640 1 -- 172.21.15.66:0/1854992769 shutdown_connections 2024-09-10T22:14:56.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.594+0000 7ff2abe00640 1 -- 172.21.15.66:0/1854992769 wait complete. 2024-09-10T22:14:56.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2abe00640 1 Processor -- start 2024-09-10T22:14:56.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2abe00640 1 -- start start 2024-09-10T22:14:56.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2ac0cb200 0x7ff2ac159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:56.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ac0b4940 0x7ff2ac06ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:56.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff2ac06f1b0 0x7ff2ac15a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:56.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2abe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac15e210 con 0x7ff2ac0b4940 2024-09-10T22:14:56.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2abe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac15e090 con 0x7ff2ac0cb200 2024-09-10T22:14:56.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2abe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff2ac073700 con 0x7ff2ac06f1b0 2024-09-10T22:14:56.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2aa000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ac0b4940 0x7ff2ac06ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:56.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2aa000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ac0b4940 0x7ff2ac06ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48344/0 (socket says 172.21.15.66:48344) 2024-09-10T22:14:56.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2aa000640 1 -- 172.21.15.66:0/1920163280 learned_addr learned my addr 172.21.15.66:0/1920163280 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2aa000640 1 -- 172.21.15.66:0/1920163280 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff2ac06f1b0 msgr2=0x7ff2ac15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2aa000640 1 --2- 172.21.15.66:0/1920163280 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff2ac06f1b0 0x7ff2ac15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2aa000640 1 -- 172.21.15.66:0/1920163280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2ac0cb200 msgr2=0x7ff2ac159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2aa000640 1 --2- 172.21.15.66:0/1920163280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2ac0cb200 0x7ff2ac159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2aa000640 1 -- 172.21.15.66:0/1920163280 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ac0b4e20 con 0x7ff2ac0b4940 2024-09-10T22:14:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2aa000640 1 --2- 172.21.15.66:0/1920163280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ac0b4940 0x7ff2ac06ec70 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7ff29c011ae0 tx=0x7ff29c009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2a2a00640 1 -- 172.21.15.66:0/1920163280 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c01e070 con 0x7ff2ac0b4940 2024-09-10T22:14:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2a2a00640 1 -- 172.21.15.66:0/1920163280 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff29c00fb90 con 0x7ff2ac0b4940 2024-09-10T22:14:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2abe00640 1 -- 172.21.15.66:0/1920163280 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ac0b5100 con 0x7ff2ac0b4940 2024-09-10T22:14:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2a2a00640 1 -- 172.21.15.66:0/1920163280 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c019410 con 0x7ff2ac0b4940 2024-09-10T22:14:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.598+0000 7ff2abe00640 1 -- 172.21.15.66:0/1920163280 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ac0b56c0 con 0x7ff2ac0b4940 2024-09-10T22:14:56.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.602+0000 7ff2abe00640 1 -- 172.21.15.66:0/1920163280 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2ac070160 con 0x7ff2ac0b4940 2024-09-10T22:14:56.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.602+0000 7ff2a2a00640 1 -- 172.21.15.66:0/1920163280 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 48) ==== 99760+0+0 (secure 0 0 0) 0x7ff29c0195b0 con 0x7ff2ac0b4940 2024-09-10T22:14:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.602+0000 7ff2a2a00640 1 --2- 172.21.15.66:0/1920163280 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff28c074b30 0x7ff28c076ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.602+0000 7ff2a2a00640 1 -- 172.21.15.66:0/1920163280 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7ff29c092f00 con 0x7ff2ac0b4940 2024-09-10T22:14:56.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.610+0000 7ff2a2a00640 1 -- 172.21.15.66:0/1920163280 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff29c05fa90 con 0x7ff2ac0b4940 2024-09-10T22:14:56.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.618+0000 7ff2aaa00640 1 --2- 172.21.15.66:0/1920163280 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff28c074b30 0x7ff28c076ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:56.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:56.658+0000 7ff2aaa00640 1 --2- 172.21.15.66:0/1920163280 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff28c074b30 0x7ff28c076ff0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff298001870 tx=0x7ff298000f80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:57.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:57.038+0000 7ff2abe00640 1 -- 172.21.15.66:0/1920163280 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone55126", "target": ["mon-mgr", ""]}) -- 0x7ff2ac0d10b0 con 0x7ff28c074b30 2024-09-10T22:14:58.032 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:14:58.040 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:14:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.030+0000 7ff2a2a00640 1 -- 172.21.15.66:0/1920163280 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff2ac0d10b0 con 0x7ff28c074b30 2024-09-10T22:14:58.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.046+0000 7ff2abe00640 1 -- 172.21.15.66:0/1920163280 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff28c074b30 msgr2=0x7ff28c076ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:58.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.046+0000 7ff2abe00640 1 --2- 172.21.15.66:0/1920163280 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff28c074b30 0x7ff28c076ff0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff298001870 tx=0x7ff298000f80 comp rx=0 tx=0).stop 2024-09-10T22:14:58.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.046+0000 7ff2abe00640 1 -- 172.21.15.66:0/1920163280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ac0b4940 msgr2=0x7ff2ac06ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:58.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.046+0000 7ff2abe00640 1 --2- 172.21.15.66:0/1920163280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ac0b4940 0x7ff2ac06ec70 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7ff29c011ae0 tx=0x7ff29c009e70 comp rx=0 tx=0).stop 2024-09-10T22:14:58.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.050+0000 7ff2abe00640 1 -- 172.21.15.66:0/1920163280 shutdown_connections 2024-09-10T22:14:58.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.050+0000 7ff2abe00640 1 --2- 172.21.15.66:0/1920163280 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff28c074b30 0x7ff28c076ff0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:58.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.050+0000 7ff2abe00640 1 --2- 172.21.15.66:0/1920163280 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff2ac06f1b0 0x7ff2ac15a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:58.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.050+0000 7ff2abe00640 1 --2- 172.21.15.66:0/1920163280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ac0b4940 0x7ff2ac06ec70 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:58.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.050+0000 7ff2abe00640 1 --2- 172.21.15.66:0/1920163280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2ac0cb200 0x7ff2ac159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:58.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.050+0000 7ff2abe00640 1 -- 172.21.15.66:0/1920163280 >> 172.21.15.66:0/1920163280 conn(0x7ff2ac0d01a0 msgr2=0x7ff2ac0ce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:58.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.050+0000 7ff2abe00640 1 -- 172.21.15.66:0/1920163280 shutdown_connections 2024-09-10T22:14:58.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:58.050+0000 7ff2abe00640 1 -- 172.21.15.66:0/1920163280 wait complete. 2024-09-10T22:14:59.106 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone36130 2024-09-10T22:14:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.346+0000 7f7e64c00640 1 -- 172.21.15.66:0/88803855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e60162440 msgr2=0x7f7e6006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.346+0000 7f7e64c00640 1 --2- 172.21.15.66:0/88803855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e60162440 0x7f7e6006e440 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009bc0 tx=0x7f7e5801cba0 comp rx=0 tx=0).stop 2024-09-10T22:14:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.346+0000 7f7e64c00640 1 -- 172.21.15.66:0/88803855 shutdown_connections 2024-09-10T22:14:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.346+0000 7f7e64c00640 1 --2- 172.21.15.66:0/88803855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e60162440 0x7f7e6006e440 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.346+0000 7f7e64c00640 1 --2- 172.21.15.66:0/88803855 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e60161c00 0x7f7e600d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.346+0000 7f7e64c00640 1 -- 172.21.15.66:0/88803855 >> 172.21.15.66:0/88803855 conn(0x7f7e600d1b10 msgr2=0x7f7e600d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.346+0000 7f7e64c00640 1 -- 172.21.15.66:0/88803855 shutdown_connections 2024-09-10T22:14:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.346+0000 7f7e64c00640 1 -- 172.21.15.66:0/88803855 wait complete. 2024-09-10T22:14:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.346+0000 7f7e64c00640 1 Processor -- start 2024-09-10T22:14:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.346+0000 7f7e64c00640 1 -- start start 2024-09-10T22:14:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e64c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e60161c00 0x7f7e6016dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e64c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e60161fe0 0x7f7e601feed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e64c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e60162440 0x7f7e600b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e64c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7e600b2970 con 0x7f7e60162440 2024-09-10T22:14:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e64c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e600b27f0 con 0x7f7e60161c00 2024-09-10T22:14:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e64c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7e6015f9c0 con 0x7f7e60161fe0 2024-09-10T22:14:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e5fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e60162440 0x7f7e600b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e5fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e60162440 0x7f7e600b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48350/0 (socket says 172.21.15.66:48350) 2024-09-10T22:14:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e5fe00640 1 -- 172.21.15.66:0/4205256958 learned_addr learned my addr 172.21.15.66:0/4205256958 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:14:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e5ea00640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e60161fe0 0x7f7e601feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e5fe00640 1 -- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e60161fe0 msgr2=0x7f7e601feed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e5fe00640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e60161fe0 0x7f7e601feed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e5fe00640 1 -- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e60161c00 msgr2=0x7f7e6016dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e5f400640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e60161c00 0x7f7e6016dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e5fe00640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e60161c00 0x7f7e6016dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e5fe00640 1 -- 172.21.15.66:0/4205256958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e601ff410 con 0x7f7e60162440 2024-09-10T22:14:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e5f400640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e60161c00 0x7f7e6016dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:14:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e5fe00640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e60162440 0x7f7e600b4940 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f7e5801d440 tx=0x7f7e580a5b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e57400640 1 -- 172.21.15.66:0/4205256958 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e580bb070 con 0x7f7e60162440 2024-09-10T22:14:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e57400640 1 -- 172.21.15.66:0/4205256958 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e58004d40 con 0x7f7e60162440 2024-09-10T22:14:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e64c00640 1 -- 172.21.15.66:0/4205256958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e601ff6b0 con 0x7f7e60162440 2024-09-10T22:14:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e57400640 1 -- 172.21.15.66:0/4205256958 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e580b2040 con 0x7f7e60162440 2024-09-10T22:14:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.350+0000 7f7e64c00640 1 -- 172.21.15.66:0/4205256958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e601ffc70 con 0x7f7e60162440 2024-09-10T22:14:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.354+0000 7f7e64c00640 1 -- 172.21.15.66:0/4205256958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e600d12c0 con 0x7f7e60162440 2024-09-10T22:14:59.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.354+0000 7f7e57400640 1 -- 172.21.15.66:0/4205256958 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 48) ==== 99760+0+0 (secure 0 0 0) 0x7f7e580b5030 con 0x7f7e60162440 2024-09-10T22:14:59.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.358+0000 7f7e57400640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7e40074b00 0x7f7e40076fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:59.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.358+0000 7f7e57400640 1 -- 172.21.15.66:0/4205256958 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f7e5812c550 con 0x7f7e60162440 2024-09-10T22:14:59.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.362+0000 7f7e57400640 1 -- 172.21.15.66:0/4205256958 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7e58016790 con 0x7f7e60162440 2024-09-10T22:14:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.374+0000 7f7e5f400640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7e40074b00 0x7f7e40076fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:59.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.450+0000 7f7e5f400640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7e40074b00 0x7f7e40076fc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7e6015c1d0 tx=0x7f7e50008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:59.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:14:59.754+0000 7f7e64c00640 1 -- 172.21.15.66:0/4205256958 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone36130", "target": ["mon-mgr", ""]}) -- 0x7f7e601588a0 con 0x7f7e40074b00 2024-09-10T22:15:00.227 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:15:00.226+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:15:00.227 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:15:00.226+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:15:00.228 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:15:00.226+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:15:00.228 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:15:00.226+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:15:00.228 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:15:00.226+0000 7f5fe8800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:15:00.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:00.822+0000 7f7e57400640 1 -- 172.21.15.66:0/4205256958 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f7e5812c8a0 con 0x7f7e60162440 2024-09-10T22:15:03.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.042+0000 7f7e57400640 1 -- 172.21.15.66:0/4205256958 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e601588a0 con 0x7f7e40074b00 2024-09-10T22:15:03.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.046+0000 7f7e64c00640 1 -- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7e40074b00 msgr2=0x7f7e40076fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:03.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.046+0000 7f7e64c00640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7e40074b00 0x7f7e40076fc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7e6015c1d0 tx=0x7f7e50008040 comp rx=0 tx=0).stop 2024-09-10T22:15:03.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.046+0000 7f7e64c00640 1 -- 172.21.15.66:0/4205256958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e60162440 msgr2=0x7f7e600b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:03.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.046+0000 7f7e64c00640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e60162440 0x7f7e600b4940 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f7e5801d440 tx=0x7f7e580a5b70 comp rx=0 tx=0).stop 2024-09-10T22:15:03.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.046+0000 7f7e64c00640 1 -- 172.21.15.66:0/4205256958 shutdown_connections 2024-09-10T22:15:03.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.046+0000 7f7e64c00640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7e40074b00 0x7f7e40076fc0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:03.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.046+0000 7f7e64c00640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e60162440 0x7f7e600b4940 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:03.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.046+0000 7f7e64c00640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e60161fe0 0x7f7e601feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:03.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.046+0000 7f7e64c00640 1 --2- 172.21.15.66:0/4205256958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e60161c00 0x7f7e6016dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:03.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.046+0000 7f7e64c00640 1 -- 172.21.15.66:0/4205256958 >> 172.21.15.66:0/4205256958 conn(0x7f7e600d1b10 msgr2=0x7f7e6016ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:03.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.046+0000 7f7e64c00640 1 -- 172.21.15.66:0/4205256958 shutdown_connections 2024-09-10T22:15:03.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.046+0000 7f7e64c00640 1 -- 172.21.15.66:0/4205256958 wait complete. 2024-09-10T22:15:03.097 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99941 2024-09-10T22:15:03.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.354+0000 7f1cf2c00640 1 -- 172.21.15.66:0/1300615011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd0004d50 msgr2=0x7f1cd00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:03.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.354+0000 7f1cf2c00640 1 --2- 172.21.15.66:0/1300615011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd0004d50 0x7f1cd00051d0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0cf050 tx=0x7f1ce401d5d0 comp rx=0 tx=0).stop 2024-09-10T22:15:03.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.354+0000 7f1cf0c00640 1 -- 172.21.15.66:0/1300615011 reap_dead start 2024-09-10T22:15:03.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.354+0000 7f1cf2c00640 1 -- 172.21.15.66:0/1300615011 shutdown_connections 2024-09-10T22:15:03.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.354+0000 7f1cf2c00640 1 -- 172.21.15.66:0/1300615011 >> 172.21.15.66:0/1300615011 conn(0x7f1cec0d1c60 msgr2=0x7f1cec0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:03.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.354+0000 7f1cf2c00640 1 -- 172.21.15.66:0/1300615011 shutdown_connections 2024-09-10T22:15:03.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.354+0000 7f1cf2c00640 1 -- 172.21.15.66:0/1300615011 wait complete. 2024-09-10T22:15:03.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.354+0000 7f1cf2c00640 1 Processor -- start 2024-09-10T22:15:03.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf2c00640 1 -- start start 2024-09-10T22:15:03.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf2c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cd0004d50 0x7f1cec0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:03.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf2c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec1f7db0 0x7f1cec15be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:03.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf2c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec1f8230 0x7f1cec15c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:03.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf2c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec06e0c0 con 0x7f1cec1f8230 2024-09-10T22:15:03.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf2c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec06df40 con 0x7f1cec1f7db0 2024-09-10T22:15:03.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf2c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1cec06e240 con 0x7f1cd0004d50 2024-09-10T22:15:03.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec1f8230 0x7f1cec15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:03.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec1f8230 0x7f1cec15c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52514/0 (socket says 172.21.15.66:52514) 2024-09-10T22:15:03.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf0c00640 1 -- 172.21.15.66:0/4138503884 learned_addr learned my addr 172.21.15.66:0/4138503884 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:03.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1ce3e00640 1 --2- 172.21.15.66:0/4138503884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec1f7db0 0x7f1cec15be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:03.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf0c00640 1 -- 172.21.15.66:0/4138503884 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cd0004d50 msgr2=0x7f1cec0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:03.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf0c00640 1 --2- 172.21.15.66:0/4138503884 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1cd0004d50 0x7f1cec0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:03.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf0c00640 1 -- 172.21.15.66:0/4138503884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec1f7db0 msgr2=0x7f1cec15be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:03.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf0c00640 1 --2- 172.21.15.66:0/4138503884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec1f7db0 0x7f1cec15be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:03.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf0c00640 1 -- 172.21.15.66:0/4138503884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec15c8b0 con 0x7f1cec1f8230 2024-09-10T22:15:03.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1ce3e00640 1 --2- 172.21.15.66:0/4138503884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cec1f7db0 0x7f1cec15be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:03.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.358+0000 7f1cf0c00640 1 --2- 172.21.15.66:0/4138503884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cec1f8230 0x7f1cec15c370 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4002240 tx=0x7f1cd0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:03.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.362+0000 7f1ce8c00640 1 -- 172.21.15.66:0/4138503884 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4009ab0 con 0x7f1cec1f8230 2024-09-10T22:15:03.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.362+0000 7f1ce8c00640 1 -- 172.21.15.66:0/4138503884 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ce40a9510 con 0x7f1cec1f8230 2024-09-10T22:15:03.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.362+0000 7f1ce8c00640 1 -- 172.21.15.66:0/4138503884 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4009200 con 0x7f1cec1f8230 2024-09-10T22:15:03.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.362+0000 7f1cf2c00640 1 -- 172.21.15.66:0/4138503884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec09f8f0 con 0x7f1cec1f8230 2024-09-10T22:15:03.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.362+0000 7f1cf2c00640 1 -- 172.21.15.66:0/4138503884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec1f5a10 con 0x7f1cec1f8230 2024-09-10T22:15:03.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.362+0000 7f1cf2c00640 1 -- 172.21.15.66:0/4138503884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cec0cbef0 con 0x7f1cec1f8230 2024-09-10T22:15:03.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.366+0000 7f1ce8c00640 1 -- 172.21.15.66:0/4138503884 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f1ce4004a50 con 0x7f1cec1f8230 2024-09-10T22:15:03.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.366+0000 7f1ce8c00640 1 --2- 172.21.15.66:0/4138503884 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1cc8074b90 0x7f1cc8077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:03.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.366+0000 7f1ce8c00640 1 -- 172.21.15.66:0/4138503884 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f1ce412d570 con 0x7f1cec1f8230 2024-09-10T22:15:03.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.370+0000 7f1ce8c00640 1 -- 172.21.15.66:0/4138503884 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ce40ad030 con 0x7f1cec1f8230 2024-09-10T22:15:03.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.374+0000 7f1cebe00640 1 --2- 172.21.15.66:0/4138503884 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1cc8074b90 0x7f1cc8077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:03.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.378+0000 7f1cebe00640 1 --2- 172.21.15.66:0/4138503884 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1cc8074b90 0x7f1cc8077050 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009710 tx=0x7f1cdc006d10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.782+0000 7f1cf2c00640 1 -- 172.21.15.66:0/4138503884 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99941", "target": ["mon-mgr", ""]}) -- 0x7f1cec0d23a0 con 0x7f1cc8074b90 2024-09-10T22:15:03.831 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:15:03.830+0000 7f5fe8800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-10T22:15:03.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:03.826+0000 7f1ce8c00640 1 -- 172.21.15.66:0/4138503884 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f1cec0d23a0 con 0x7f1cc8074b90 2024-09-10T22:15:03.831 INFO:teuthology.orchestra.run.smithi066.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-10T22:15:03.844 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T22:15:03.845 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone53292 2024-09-10T22:15:04.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.126+0000 7f0d3fe00640 1 -- 172.21.15.66:0/1071934365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d380b2e60 msgr2=0x7f0d38160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.126+0000 7f0d3fe00640 1 --2- 172.21.15.66:0/1071934365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d380b2e60 0x7f0d38160a20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0d380d11d0 tx=0x7f0d3001cd60 comp rx=0 tx=0).stop 2024-09-10T22:15:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.126+0000 7f0d3fe00640 1 -- 172.21.15.66:0/1071934365 shutdown_connections 2024-09-10T22:15:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.126+0000 7f0d3fe00640 1 --2- 172.21.15.66:0/1071934365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d380b2e60 0x7f0d38160a20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.126+0000 7f0d3fe00640 1 -- 172.21.15.66:0/1071934365 >> 172.21.15.66:0/1071934365 conn(0x7f0d380d3e90 msgr2=0x7f0d380d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.126+0000 7f0d3fe00640 1 -- 172.21.15.66:0/1071934365 shutdown_connections 2024-09-10T22:15:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.126+0000 7f0d3fe00640 1 -- 172.21.15.66:0/1071934365 wait complete. 2024-09-10T22:15:04.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.130+0000 7f0d3fe00640 1 Processor -- start 2024-09-10T22:15:04.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.130+0000 7f0d3fe00640 1 -- start start 2024-09-10T22:15:04.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.130+0000 7f0d3fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d380b2e60 0x7f0d38156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:04.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.130+0000 7f0d3fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d380d49b0 0x7f0d38159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:04.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.130+0000 7f0d3fe00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d381611d0 0x7f0d380b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:04.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.130+0000 7f0d3fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3816c120 con 0x7f0d380b2e60 2024-09-10T22:15:04.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.130+0000 7f0d3fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3816bfa0 con 0x7f0d380d49b0 2024-09-10T22:15:04.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.130+0000 7f0d3fe00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3816c2a0 con 0x7f0d381611d0 2024-09-10T22:15:04.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d3d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d380b2e60 0x7f0d38156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:04.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d3d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d380b2e60 0x7f0d38156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52516/0 (socket says 172.21.15.66:52516) 2024-09-10T22:15:04.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d3d400640 1 -- 172.21.15.66:0/4184406450 learned_addr learned my addr 172.21.15.66:0/4184406450 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:04.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d3d400640 1 -- 172.21.15.66:0/4184406450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d381611d0 msgr2=0x7f0d380b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:04.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d3d400640 1 --2- 172.21.15.66:0/4184406450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d381611d0 0x7f0d380b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:04.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d3d400640 1 -- 172.21.15.66:0/4184406450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d380d49b0 msgr2=0x7f0d38159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:04.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d3d400640 1 --2- 172.21.15.66:0/4184406450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d380d49b0 0x7f0d38159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:04.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d3d400640 1 -- 172.21.15.66:0/4184406450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d380ce1b0 con 0x7f0d380b2e60 2024-09-10T22:15:04.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d3d400640 1 --2- 172.21.15.66:0/4184406450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d380b2e60 0x7f0d38156560 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c3c0 tx=0x7f0d2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:04.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d35600640 1 -- 172.21.15.66:0/4184406450 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c017070 con 0x7f0d380b2e60 2024-09-10T22:15:04.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d35600640 1 -- 172.21.15.66:0/4184406450 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d2c0053c0 con 0x7f0d380b2e60 2024-09-10T22:15:04.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d35600640 1 -- 172.21.15.66:0/4184406450 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c003860 con 0x7f0d380b2e60 2024-09-10T22:15:04.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d3fe00640 1 -- 172.21.15.66:0/4184406450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3815a2f0 con 0x7f0d380b2e60 2024-09-10T22:15:04.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.134+0000 7f0d3fe00640 1 -- 172.21.15.66:0/4184406450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d381fc770 con 0x7f0d380b2e60 2024-09-10T22:15:04.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.138+0000 7f0d3fe00640 1 -- 172.21.15.66:0/4184406450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d381615b0 con 0x7f0d380b2e60 2024-09-10T22:15:04.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.138+0000 7f0d35600640 1 -- 172.21.15.66:0/4184406450 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f0d2c003a00 con 0x7f0d380b2e60 2024-09-10T22:15:04.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.142+0000 7f0d35600640 1 --2- 172.21.15.66:0/4184406450 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d18074be0 0x7f0d180770a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:04.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.142+0000 7f0d35600640 1 -- 172.21.15.66:0/4184406450 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f0d2c091a60 con 0x7f0d380b2e60 2024-09-10T22:15:04.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.150+0000 7f0d3ca00640 1 --2- 172.21.15.66:0/4184406450 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d18074be0 0x7f0d180770a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:04.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.150+0000 7f0d35600640 1 -- 172.21.15.66:0/4184406450 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0d2c05ed20 con 0x7f0d380b2e60 2024-09-10T22:15:04.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.154+0000 7f0d3ca00640 1 --2- 172.21.15.66:0/4184406450 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d18074be0 0x7f0d180770a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0d24007fd0 tx=0x7f0d240079a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:04.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:04.518+0000 7f0d3fe00640 1 -- 172.21.15.66:0/4184406450 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone53292", "target": ["mon-mgr", ""]}) -- 0x7f0d380b7d40 con 0x7f0d18074be0 2024-09-10T22:15:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.330+0000 7f0d35600640 1 -- 172.21.15.66:0/4184406450 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d380b7d40 con 0x7f0d18074be0 2024-09-10T22:15:06.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.334+0000 7f0d3fe00640 1 -- 172.21.15.66:0/4184406450 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d18074be0 msgr2=0x7f0d180770a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.334+0000 7f0d3fe00640 1 --2- 172.21.15.66:0/4184406450 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d18074be0 0x7f0d180770a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0d24007fd0 tx=0x7f0d240079a0 comp rx=0 tx=0).stop 2024-09-10T22:15:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.334+0000 7f0d3fe00640 1 -- 172.21.15.66:0/4184406450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d380b2e60 msgr2=0x7f0d38156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.334+0000 7f0d3fe00640 1 --2- 172.21.15.66:0/4184406450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d380b2e60 0x7f0d38156560 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c3c0 tx=0x7f0d2c00c880 comp rx=0 tx=0).stop 2024-09-10T22:15:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.334+0000 7f0d3fe00640 1 -- 172.21.15.66:0/4184406450 shutdown_connections 2024-09-10T22:15:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.334+0000 7f0d3fe00640 1 --2- 172.21.15.66:0/4184406450 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0d18074be0 0x7f0d180770a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.334+0000 7f0d3fe00640 1 --2- 172.21.15.66:0/4184406450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d381611d0 0x7f0d380b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.334+0000 7f0d3fe00640 1 --2- 172.21.15.66:0/4184406450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d380d49b0 0x7f0d38159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.334+0000 7f0d3fe00640 1 --2- 172.21.15.66:0/4184406450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d380b2e60 0x7f0d38156560 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.334+0000 7f0d3fe00640 1 -- 172.21.15.66:0/4184406450 >> 172.21.15.66:0/4184406450 conn(0x7f0d380d3e90 msgr2=0x7f0d380d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.334+0000 7f0d3fe00640 1 -- 172.21.15.66:0/4184406450 shutdown_connections 2024-09-10T22:15:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.334+0000 7f0d3fe00640 1 -- 172.21.15.66:0/4184406450 wait complete. 2024-09-10T22:15:06.383 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone71863 2024-09-10T22:15:06.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.646+0000 7f1fc6c00640 1 -- 172.21.15.66:0/2363602987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fa80051d0 msgr2=0x7f1fa800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:06.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.646+0000 7f1fc6c00640 1 --2- 172.21.15.66:0/2363602987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fa80051d0 0x7f1fa800b9d0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009ee0 tx=0x7f1fb801d630 comp rx=0 tx=0).stop 2024-09-10T22:15:06.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.646+0000 7f1fc6c00640 1 -- 172.21.15.66:0/2363602987 shutdown_connections 2024-09-10T22:15:06.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.646+0000 7f1fc6c00640 1 --2- 172.21.15.66:0/2363602987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fa80051d0 0x7f1fa800b9d0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:06.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.646+0000 7f1fc6c00640 1 -- 172.21.15.66:0/2363602987 >> 172.21.15.66:0/2363602987 conn(0x7f1fc009d300 msgr2=0x7f1fc00d3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:06.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.646+0000 7f1fc6c00640 1 -- 172.21.15.66:0/2363602987 shutdown_connections 2024-09-10T22:15:06.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.646+0000 7f1fc6c00640 1 -- 172.21.15.66:0/2363602987 wait complete. 2024-09-10T22:15:06.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6c00640 1 Processor -- start 2024-09-10T22:15:06.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6c00640 1 -- start start 2024-09-10T22:15:06.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa80051d0 0x7f1fc00c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:06.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1fc00b9940 0x7f1fc01f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:06.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc00c9da0 0x7f1fc00ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:06.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc00b2180 con 0x7f1fc00c9da0 2024-09-10T22:15:06.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc00b2000 con 0x7f1fa80051d0 2024-09-10T22:15:06.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc00b2300 con 0x7f1fc00b9940 2024-09-10T22:15:06.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc00c9da0 0x7f1fc00ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:06.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc00c9da0 0x7f1fc00ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52532/0 (socket says 172.21.15.66:52532) 2024-09-10T22:15:06.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6200640 1 -- 172.21.15.66:0/671601480 learned_addr learned my addr 172.21.15.66:0/671601480 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:06.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6200640 1 -- 172.21.15.66:0/671601480 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1fc00b9940 msgr2=0x7f1fc01f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:06.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6200640 1 --2- 172.21.15.66:0/671601480 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1fc00b9940 0x7f1fc01f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:06.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6200640 1 -- 172.21.15.66:0/671601480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa80051d0 msgr2=0x7f1fc00c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:06.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6200640 1 --2- 172.21.15.66:0/671601480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa80051d0 0x7f1fc00c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:06.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.650+0000 7f1fc6200640 1 -- 172.21.15.66:0/671601480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc00b3300 con 0x7f1fc00c9da0 2024-09-10T22:15:06.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.654+0000 7f1fc6200640 1 --2- 172.21.15.66:0/671601480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc00c9da0 0x7f1fc00ca230 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f1fb801d460 tx=0x7f1fb8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:06.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.654+0000 7f1fbd600640 1 -- 172.21.15.66:0/671601480 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80ac030 con 0x7f1fc00c9da0 2024-09-10T22:15:06.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.654+0000 7f1fbd600640 1 -- 172.21.15.66:0/671601480 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fb8003f40 con 0x7f1fc00c9da0 2024-09-10T22:15:06.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.654+0000 7f1fc6c00640 1 -- 172.21.15.66:0/671601480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc01f4770 con 0x7f1fc00c9da0 2024-09-10T22:15:06.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.654+0000 7f1fbd600640 1 -- 172.21.15.66:0/671601480 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8009ab0 con 0x7f1fc00c9da0 2024-09-10T22:15:06.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.654+0000 7f1fc6c00640 1 -- 172.21.15.66:0/671601480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc00b8100 con 0x7f1fc00c9da0 2024-09-10T22:15:06.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.654+0000 7f1fc6c00640 1 -- 172.21.15.66:0/671601480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc00b27f0 con 0x7f1fc00c9da0 2024-09-10T22:15:06.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.658+0000 7f1fbd600640 1 -- 172.21.15.66:0/671601480 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f1fb80bd030 con 0x7f1fc00c9da0 2024-09-10T22:15:06.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.658+0000 7f1fbd600640 1 --2- 172.21.15.66:0/671601480 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1fa0074b90 0x7f1fa0077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:06.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.658+0000 7f1fbd600640 1 -- 172.21.15.66:0/671601480 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f1fb8002020 con 0x7f1fc00c9da0 2024-09-10T22:15:06.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.662+0000 7f1fbd600640 1 -- 172.21.15.66:0/671601480 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1fb80b1070 con 0x7f1fc00c9da0 2024-09-10T22:15:06.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.682+0000 7f1fc5800640 1 --2- 172.21.15.66:0/671601480 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1fa0074b90 0x7f1fa0077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:06.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:06.734+0000 7f1fc5800640 1 --2- 172.21.15.66:0/671601480 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1fa0074b90 0x7f1fa0077050 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4001340 tx=0x7f1fb40012d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:07.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.062+0000 7f1fc6c00640 1 -- 172.21.15.66:0/671601480 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone71863", "target": ["mon-mgr", ""]}) -- 0x7f1fc00cf120 con 0x7f1fa0074b90 2024-09-10T22:15:07.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.726+0000 7f1fbd600640 1 -- 172.21.15.66:0/671601480 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1fc00cf120 con 0x7f1fa0074b90 2024-09-10T22:15:07.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.730+0000 7f1fc6c00640 1 -- 172.21.15.66:0/671601480 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1fa0074b90 msgr2=0x7f1fa0077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:07.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.730+0000 7f1fc6c00640 1 --2- 172.21.15.66:0/671601480 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1fa0074b90 0x7f1fa0077050 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4001340 tx=0x7f1fb40012d0 comp rx=0 tx=0).stop 2024-09-10T22:15:07.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.730+0000 7f1fc6c00640 1 -- 172.21.15.66:0/671601480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc00c9da0 msgr2=0x7f1fc00ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:07.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.730+0000 7f1fc6c00640 1 --2- 172.21.15.66:0/671601480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc00c9da0 0x7f1fc00ca230 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f1fb801d460 tx=0x7f1fb8009220 comp rx=0 tx=0).stop 2024-09-10T22:15:07.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.730+0000 7f1fc6c00640 1 -- 172.21.15.66:0/671601480 shutdown_connections 2024-09-10T22:15:07.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.730+0000 7f1fc6c00640 1 --2- 172.21.15.66:0/671601480 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1fa0074b90 0x7f1fa0077050 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:07.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.730+0000 7f1fc6c00640 1 --2- 172.21.15.66:0/671601480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc00c9da0 0x7f1fc00ca230 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:07.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.730+0000 7f1fc6c00640 1 --2- 172.21.15.66:0/671601480 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1fc00b9940 0x7f1fc01f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:07.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.730+0000 7f1fc6c00640 1 --2- 172.21.15.66:0/671601480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fa80051d0 0x7f1fc00c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:07.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.730+0000 7f1fc6c00640 1 -- 172.21.15.66:0/671601480 >> 172.21.15.66:0/671601480 conn(0x7f1fc009d300 msgr2=0x7f1fc00b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:07.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.730+0000 7f1fc6c00640 1 -- 172.21.15.66:0/671601480 shutdown_connections 2024-09-10T22:15:07.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.730+0000 7f1fc6c00640 1 -- 172.21.15.66:0/671601480 wait complete. 2024-09-10T22:15:07.756 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99264 2024-09-10T22:15:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.966+0000 7f53b8200640 1 -- 172.21.15.66:0/2491367016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b01447b0 msgr2=0x7f53b00ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.966+0000 7f53b8200640 1 --2- 172.21.15.66:0/2491367016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b01447b0 0x7f53b00ca080 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009840 tx=0x7f53a401c7f0 comp rx=0 tx=0).stop 2024-09-10T22:15:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.966+0000 7f53b8200640 1 -- 172.21.15.66:0/2491367016 shutdown_connections 2024-09-10T22:15:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.966+0000 7f53b8200640 1 --2- 172.21.15.66:0/2491367016 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b00ae7b0 0x7f53b00cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.966+0000 7f53b8200640 1 --2- 172.21.15.66:0/2491367016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b01447b0 0x7f53b00ca080 secure :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009840 tx=0x7f53a401c7f0 comp rx=0 tx=0).stop 2024-09-10T22:15:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.966+0000 7f53b8200640 1 -- 172.21.15.66:0/2491367016 >> 172.21.15.66:0/2491367016 conn(0x7f53b00cfa30 msgr2=0x7f53b00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.970+0000 7f53b8200640 1 -- 172.21.15.66:0/2491367016 shutdown_connections 2024-09-10T22:15:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.970+0000 7f53b8200640 1 -- 172.21.15.66:0/2491367016 wait complete. 2024-09-10T22:15:07.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.970+0000 7f53b8200640 1 Processor -- start 2024-09-10T22:15:07.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.970+0000 7f53b8200640 1 -- start start 2024-09-10T22:15:07.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b8200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b00ae7b0 0x7f53b00b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b8200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b00aebf0 0x7f53b00b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b8200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53b015d060 0x7f53b0146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b8200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53b016b630 con 0x7f53b015d060 2024-09-10T22:15:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b8200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53b00adbf0 con 0x7f53b00ae7b0 2024-09-10T22:15:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b8200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f53b016b7b0 con 0x7f53b00aebf0 2024-09-10T22:15:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53b015d060 0x7f53b0146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53b015d060 0x7f53b0146f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52544/0 (socket says 172.21.15.66:52544) 2024-09-10T22:15:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b6200640 1 -- 172.21.15.66:0/2737408752 learned_addr learned my addr 172.21.15.66:0/2737408752 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b5800640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b00ae7b0 0x7f53b00b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b6200640 1 -- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b00aebf0 msgr2=0x7f53b00b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b6200640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b00aebf0 0x7f53b00b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b6200640 1 -- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b00ae7b0 msgr2=0x7f53b00b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b6200640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b00ae7b0 0x7f53b00b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:07.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b6200640 1 -- 172.21.15.66:0/2737408752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b00b1340 con 0x7f53b015d060 2024-09-10T22:15:07.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b6200640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53b015d060 0x7f53b0146f40 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f53a800b9c0 tx=0x7f53a800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:07.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53ad600640 1 -- 172.21.15.66:0/2737408752 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a8019070 con 0x7f53b015d060 2024-09-10T22:15:07.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53b8200640 1 -- 172.21.15.66:0/2737408752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b00b2fd0 con 0x7f53b015d060 2024-09-10T22:15:07.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53ad600640 1 -- 172.21.15.66:0/2737408752 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53a800cc90 con 0x7f53b015d060 2024-09-10T22:15:07.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.974+0000 7f53ad600640 1 -- 172.21.15.66:0/2737408752 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a80146a0 con 0x7f53b015d060 2024-09-10T22:15:07.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.978+0000 7f53b8200640 1 -- 172.21.15.66:0/2737408752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b015c860 con 0x7f53b015d060 2024-09-10T22:15:07.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.978+0000 7f53b8200640 1 -- 172.21.15.66:0/2737408752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b00ca080 con 0x7f53b015d060 2024-09-10T22:15:07.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.978+0000 7f53ad600640 1 -- 172.21.15.66:0/2737408752 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f53a801d3b0 con 0x7f53b015d060 2024-09-10T22:15:07.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.982+0000 7f53ad600640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5390074b90 0x7f5390077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:07.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.982+0000 7f53ad600640 1 -- 172.21.15.66:0/2737408752 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f53a8092620 con 0x7f53b015d060 2024-09-10T22:15:07.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.982+0000 7f53b5800640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5390074b90 0x7f5390077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:07.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.986+0000 7f53b5800640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5390074b90 0x7f5390077050 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f53a40af000 tx=0x7f53a401b030 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:07.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:07.986+0000 7f53ad600640 1 -- 172.21.15.66:0/2737408752 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f53a805f6e0 con 0x7f53b015d060 2024-09-10T22:15:08.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:08.338+0000 7f53b8200640 1 -- 172.21.15.66:0/2737408752 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99264", "target": ["mon-mgr", ""]}) -- 0x7f53b00d0190 con 0x7f5390074b90 2024-09-10T22:15:09.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.354+0000 7f53ad600640 1 -- 172.21.15.66:0/2737408752 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f53b00d0190 con 0x7f5390074b90 2024-09-10T22:15:09.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.358+0000 7f53b8200640 1 -- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5390074b90 msgr2=0x7f5390077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:09.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.358+0000 7f53b8200640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5390074b90 0x7f5390077050 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f53a40af000 tx=0x7f53a401b030 comp rx=0 tx=0).stop 2024-09-10T22:15:09.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.358+0000 7f53b8200640 1 -- 172.21.15.66:0/2737408752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53b015d060 msgr2=0x7f53b0146f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:09.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.358+0000 7f53b8200640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53b015d060 0x7f53b0146f40 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f53a800b9c0 tx=0x7f53a800be80 comp rx=0 tx=0).stop 2024-09-10T22:15:09.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.358+0000 7f53b8200640 1 -- 172.21.15.66:0/2737408752 shutdown_connections 2024-09-10T22:15:09.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.358+0000 7f53b8200640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5390074b90 0x7f5390077050 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:09.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.358+0000 7f53b8200640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53b015d060 0x7f53b0146f40 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:09.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.358+0000 7f53b8200640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b00aebf0 0x7f53b00b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:09.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.358+0000 7f53b8200640 1 --2- 172.21.15.66:0/2737408752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b00ae7b0 0x7f53b00b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:09.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.358+0000 7f53b8200640 1 -- 172.21.15.66:0/2737408752 >> 172.21.15.66:0/2737408752 conn(0x7f53b00cfa30 msgr2=0x7f53b006e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:09.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.358+0000 7f53b8200640 1 -- 172.21.15.66:0/2737408752 shutdown_connections 2024-09-10T22:15:09.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.358+0000 7f53b8200640 1 -- 172.21.15.66:0/2737408752 wait complete. 2024-09-10T22:15:09.400 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone98955 2024-09-10T22:15:09.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.622+0000 7ff063e00640 1 -- 172.21.15.66:0/743150467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048004da0 msgr2=0x7ff048005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:09.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.622+0000 7ff063e00640 1 --2- 172.21.15.66:0/743150467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048004da0 0x7ff048005220 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00c6b0 tx=0x7ff05c01d460 comp rx=0 tx=0).stop 2024-09-10T22:15:09.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.622+0000 7ff063400640 1 -- 172.21.15.66:0/743150467 reap_dead start 2024-09-10T22:15:09.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.622+0000 7ff063e00640 1 -- 172.21.15.66:0/743150467 shutdown_connections 2024-09-10T22:15:09.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.622+0000 7ff063e00640 1 -- 172.21.15.66:0/743150467 >> 172.21.15.66:0/743150467 conn(0x7ff0640b2730 msgr2=0x7ff0640b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:09.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.626+0000 7ff063e00640 1 -- 172.21.15.66:0/743150467 shutdown_connections 2024-09-10T22:15:09.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.626+0000 7ff063e00640 1 -- 172.21.15.66:0/743150467 wait complete. 2024-09-10T22:15:09.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.626+0000 7ff063e00640 1 Processor -- start 2024-09-10T22:15:09.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.626+0000 7ff063e00640 1 -- start start 2024-09-10T22:15:09.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff048004da0 0x7ff064070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:09.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0640cf940 0x7ff06414b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:09.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff06415c390 0x7ff0640cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:09.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff0640b3a30 con 0x7ff06415c390 2024-09-10T22:15:09.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff0640b4f50 con 0x7ff0640cf940 2024-09-10T22:15:09.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff0640b3bb0 con 0x7ff048004da0 2024-09-10T22:15:09.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff06415c390 0x7ff0640cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:09.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff06415c390 0x7ff0640cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52570/0 (socket says 172.21.15.66:52570) 2024-09-10T22:15:09.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063400640 1 -- 172.21.15.66:0/647077814 learned_addr learned my addr 172.21.15.66:0/647077814 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:09.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063400640 1 -- 172.21.15.66:0/647077814 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff048004da0 msgr2=0x7ff064070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:09.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063400640 1 --2- 172.21.15.66:0/647077814 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff048004da0 0x7ff064070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:09.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063400640 1 -- 172.21.15.66:0/647077814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0640cf940 msgr2=0x7ff06414b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:09.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063400640 1 --2- 172.21.15.66:0/647077814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0640cf940 0x7ff06414b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:09.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063400640 1 -- 172.21.15.66:0/647077814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff064071320 con 0x7ff06415c390 2024-09-10T22:15:09.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063400640 1 --2- 172.21.15.66:0/647077814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff06415c390 0x7ff0640cecf0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7ff05c01ced0 tx=0x7ff05c006460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:09.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff05b400640 1 -- 172.21.15.66:0/647077814 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0bd070 con 0x7ff06415c390 2024-09-10T22:15:09.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff05b400640 1 -- 172.21.15.66:0/647077814 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff05c004b50 con 0x7ff06415c390 2024-09-10T22:15:09.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff05b400640 1 -- 172.21.15.66:0/647077814 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c00c300 con 0x7ff06415c390 2024-09-10T22:15:09.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063e00640 1 -- 172.21.15.66:0/647077814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0640cdf70 con 0x7ff06415c390 2024-09-10T22:15:09.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.630+0000 7ff063e00640 1 -- 172.21.15.66:0/647077814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0640ce210 con 0x7ff06415c390 2024-09-10T22:15:09.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.634+0000 7ff063e00640 1 -- 172.21.15.66:0/647077814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0640b4030 con 0x7ff06415c390 2024-09-10T22:15:09.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.634+0000 7ff05b400640 1 -- 172.21.15.66:0/647077814 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7ff05c001880 con 0x7ff06415c390 2024-09-10T22:15:09.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.638+0000 7ff05b400640 1 --2- 172.21.15.66:0/647077814 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff040074b90 0x7ff040077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:09.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.638+0000 7ff05b400640 1 -- 172.21.15.66:0/647077814 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7ff05c12ca90 con 0x7ff06415c390 2024-09-10T22:15:09.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.642+0000 7ff05b400640 1 -- 172.21.15.66:0/647077814 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff05c017170 con 0x7ff06415c390 2024-09-10T22:15:09.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.642+0000 7ff062a00640 1 --2- 172.21.15.66:0/647077814 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff040074b90 0x7ff040077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:09.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.646+0000 7ff062a00640 1 --2- 172.21.15.66:0/647077814 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff040074b90 0x7ff040077050 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff054005fd0 tx=0x7ff054005bf0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:09.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:09.974+0000 7ff063e00640 1 -- 172.21.15.66:0/647077814 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone98955", "target": ["mon-mgr", ""]}) -- 0x7ff06415e2d0 con 0x7ff040074b90 2024-09-10T22:15:10.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.566+0000 7ff05b400640 1 -- 172.21.15.66:0/647077814 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff06415e2d0 con 0x7ff040074b90 2024-09-10T22:15:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.570+0000 7ff063e00640 1 -- 172.21.15.66:0/647077814 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff040074b90 msgr2=0x7ff040077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.570+0000 7ff063e00640 1 --2- 172.21.15.66:0/647077814 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff040074b90 0x7ff040077050 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff054005fd0 tx=0x7ff054005bf0 comp rx=0 tx=0).stop 2024-09-10T22:15:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.570+0000 7ff063e00640 1 -- 172.21.15.66:0/647077814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff06415c390 msgr2=0x7ff0640cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.570+0000 7ff063e00640 1 --2- 172.21.15.66:0/647077814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff06415c390 0x7ff0640cecf0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7ff05c01ced0 tx=0x7ff05c006460 comp rx=0 tx=0).stop 2024-09-10T22:15:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.570+0000 7ff063e00640 1 -- 172.21.15.66:0/647077814 shutdown_connections 2024-09-10T22:15:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.570+0000 7ff063e00640 1 --2- 172.21.15.66:0/647077814 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff040074b90 0x7ff040077050 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.570+0000 7ff063e00640 1 --2- 172.21.15.66:0/647077814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff06415c390 0x7ff0640cecf0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.570+0000 7ff063e00640 1 --2- 172.21.15.66:0/647077814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0640cf940 0x7ff06414b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.570+0000 7ff063e00640 1 --2- 172.21.15.66:0/647077814 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff048004da0 0x7ff064070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.570+0000 7ff063e00640 1 -- 172.21.15.66:0/647077814 >> 172.21.15.66:0/647077814 conn(0x7ff0640b2730 msgr2=0x7ff06415ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:10.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.570+0000 7ff063e00640 1 -- 172.21.15.66:0/647077814 shutdown_connections 2024-09-10T22:15:10.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.570+0000 7ff063e00640 1 -- 172.21.15.66:0/647077814 wait complete. 2024-09-10T22:15:10.610 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone55126 2024-09-10T22:15:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.818+0000 7f9b61c00640 1 -- 172.21.15.66:0/3955765240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b5c0b50f0 msgr2=0x7f9b5c15af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:10.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.818+0000 7f9b61c00640 1 --2- 172.21.15.66:0/3955765240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b5c0b50f0 0x7f9b5c15af80 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c073e30 tx=0x7f9b4401c8c0 comp rx=0 tx=0).stop 2024-09-10T22:15:10.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.818+0000 7f9b61c00640 1 -- 172.21.15.66:0/3955765240 shutdown_connections 2024-09-10T22:15:10.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.818+0000 7f9b61c00640 1 --2- 172.21.15.66:0/3955765240 >> v2:172.21.15.94:3300/0 conn(0x7f9b5c068990 0x7f9b5c15b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:10.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.818+0000 7f9b61c00640 1 --2- 172.21.15.66:0/3955765240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b5c0b50f0 0x7f9b5c15af80 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:10.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.818+0000 7f9b61c00640 1 -- 172.21.15.66:0/3955765240 >> 172.21.15.66:0/3955765240 conn(0x7f9b5c074060 msgr2=0x7f9b5c074470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:10.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.822+0000 7f9b61c00640 1 -- 172.21.15.66:0/3955765240 shutdown_connections 2024-09-10T22:15:10.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.822+0000 7f9b61c00640 1 -- 172.21.15.66:0/3955765240 wait complete. 2024-09-10T22:15:10.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.822+0000 7f9b61c00640 1 Processor -- start 2024-09-10T22:15:10.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.822+0000 7f9b61c00640 1 -- start start 2024-09-10T22:15:10.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b61c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b5c068990 0x7f9b5c0c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:10.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b61c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b5c15aba0 0x7f9b5c0c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:10.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b61c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c0b0bc0 0x7f9b5c155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:10.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b61c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c167fc0 con 0x7f9b5c15aba0 2024-09-10T22:15:10.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b61c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c167e40 con 0x7f9b5c0b0bc0 2024-09-10T22:15:10.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b61c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9b5c168140 con 0x7f9b5c068990 2024-09-10T22:15:10.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b53400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b5c15aba0 0x7f9b5c0c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:10.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b53400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b5c15aba0 0x7f9b5c0c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52588/0 (socket says 172.21.15.66:52588) 2024-09-10T22:15:10.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b53400640 1 -- 172.21.15.66:0/353220275 learned_addr learned my addr 172.21.15.66:0/353220275 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:10.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b5be00640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b5c068990 0x7f9b5c0c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:10.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b61200640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c0b0bc0 0x7f9b5c155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:10.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b53400640 1 -- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b5c068990 msgr2=0x7f9b5c0c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:10.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b53400640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b5c068990 0x7f9b5c0c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:10.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b53400640 1 -- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c0b0bc0 msgr2=0x7f9b5c155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:10.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b53400640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c0b0bc0 0x7f9b5c155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:10.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b53400640 1 -- 172.21.15.66:0/353220275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c155980 con 0x7f9b5c15aba0 2024-09-10T22:15:10.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b53400640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b5c15aba0 0x7f9b5c0c6f90 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f9b440194d0 tx=0x7f9b440a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:10.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b58c00640 1 -- 172.21.15.66:0/353220275 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b440bc070 con 0x7f9b5c15aba0 2024-09-10T22:15:10.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b58c00640 1 -- 172.21.15.66:0/353220275 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b44004430 con 0x7f9b5c15aba0 2024-09-10T22:15:10.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b61c00640 1 -- 172.21.15.66:0/353220275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c0b1070 con 0x7f9b5c15aba0 2024-09-10T22:15:10.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b58c00640 1 -- 172.21.15.66:0/353220275 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b440b2040 con 0x7f9b5c15aba0 2024-09-10T22:15:10.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.826+0000 7f9b61c00640 1 -- 172.21.15.66:0/353220275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c0c5540 con 0x7f9b5c15aba0 2024-09-10T22:15:10.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.830+0000 7f9b61c00640 1 -- 172.21.15.66:0/353220275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b5c0b49e0 con 0x7f9b5c15aba0 2024-09-10T22:15:10.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.830+0000 7f9b58c00640 1 -- 172.21.15.66:0/353220275 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f9b440b5070 con 0x7f9b5c15aba0 2024-09-10T22:15:10.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.830+0000 7f9b58c00640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b38074bc0 0x7f9b38077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:10.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.830+0000 7f9b58c00640 1 -- 172.21.15.66:0/353220275 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f9b4412b980 con 0x7f9b5c15aba0 2024-09-10T22:15:10.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.830+0000 7f9b5be00640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b38074bc0 0x7f9b38077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:10.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.830+0000 7f9b5be00640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b38074bc0 0x7f9b38077080 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c007fd0 tx=0x7f9b4c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:10.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:10.834+0000 7f9b58c00640 1 -- 172.21.15.66:0/353220275 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9b440164b0 con 0x7f9b5c15aba0 2024-09-10T22:15:11.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.178+0000 7f9b61c00640 1 -- 172.21.15.66:0/353220275 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone55126", "target": ["mon-mgr", ""]}) -- 0x7f9b5c0746e0 con 0x7f9b38074bc0 2024-09-10T22:15:11.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.406+0000 7f9b58c00640 1 -- 172.21.15.66:0/353220275 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9b5c0746e0 con 0x7f9b38074bc0 2024-09-10T22:15:11.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.410+0000 7f9b61c00640 1 -- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b38074bc0 msgr2=0x7f9b38077080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:11.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.410+0000 7f9b61c00640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b38074bc0 0x7f9b38077080 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c007fd0 tx=0x7f9b4c009040 comp rx=0 tx=0).stop 2024-09-10T22:15:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.410+0000 7f9b61c00640 1 -- 172.21.15.66:0/353220275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b5c15aba0 msgr2=0x7f9b5c0c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.410+0000 7f9b61c00640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b5c15aba0 0x7f9b5c0c6f90 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f9b440194d0 tx=0x7f9b440a5780 comp rx=0 tx=0).stop 2024-09-10T22:15:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.410+0000 7f9b61c00640 1 -- 172.21.15.66:0/353220275 shutdown_connections 2024-09-10T22:15:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.410+0000 7f9b61c00640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b38074bc0 0x7f9b38077080 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.410+0000 7f9b61c00640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c0b0bc0 0x7f9b5c155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.410+0000 7f9b61c00640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b5c15aba0 0x7f9b5c0c6f90 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.410+0000 7f9b61c00640 1 --2- 172.21.15.66:0/353220275 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b5c068990 0x7f9b5c0c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.410+0000 7f9b61c00640 1 -- 172.21.15.66:0/353220275 >> 172.21.15.66:0/353220275 conn(0x7f9b5c074060 msgr2=0x7f9b5c15af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.410+0000 7f9b61c00640 1 -- 172.21.15.66:0/353220275 shutdown_connections 2024-09-10T22:15:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.410+0000 7f9b61c00640 1 -- 172.21.15.66:0/353220275 wait complete. 2024-09-10T22:15:11.435 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone36130 2024-09-10T22:15:11.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.578+0000 7f03e0e00640 1 -- 172.21.15.66:0/192050036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03dc0d0550 msgr2=0x7f03dc0c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:11.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.578+0000 7f03e0e00640 1 --2- 172.21.15.66:0/192050036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03dc0d0550 0x7f03dc0c4df0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f03dc06c4f0 tx=0x7f03c401c830 comp rx=0 tx=0).stop 2024-09-10T22:15:11.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.578+0000 7f03e0e00640 1 -- 172.21.15.66:0/192050036 shutdown_connections 2024-09-10T22:15:11.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.578+0000 7f03e0e00640 1 --2- 172.21.15.66:0/192050036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03dc0d0550 0x7f03dc0c4df0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.578+0000 7f03e0e00640 1 --2- 172.21.15.66:0/192050036 >> v2:172.21.15.94:3300/0 conn(0x7f03dc15a310 0x7f03dc0c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.578+0000 7f03e0e00640 1 -- 172.21.15.66:0/192050036 >> 172.21.15.66:0/192050036 conn(0x7f03dc0cfa30 msgr2=0x7f03dc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:11.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.578+0000 7f03e0e00640 1 -- 172.21.15.66:0/192050036 shutdown_connections 2024-09-10T22:15:11.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.578+0000 7f03e0e00640 1 -- 172.21.15.66:0/192050036 wait complete. 2024-09-10T22:15:11.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03e0e00640 1 Processor -- start 2024-09-10T22:15:11.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03e0e00640 1 -- start start 2024-09-10T22:15:11.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03e0e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03dc0aeec0 0x7f03dc152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:11.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03e0e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03dc15a310 0x7f03dc14b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:11.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03e0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03dc14c760 0x7f03dc14bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:11.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03e0e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc15edb0 con 0x7f03dc14c760 2024-09-10T22:15:11.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03e0e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc15ec30 con 0x7f03dc0aeec0 2024-09-10T22:15:11.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03e0e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f03dc1652f0 con 0x7f03dc15a310 2024-09-10T22:15:11.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03da800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03dc14c760 0x7f03dc14bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:11.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03da800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03dc14c760 0x7f03dc14bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52608/0 (socket says 172.21.15.66:52608) 2024-09-10T22:15:11.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03da800640 1 -- 172.21.15.66:0/1660637321 learned_addr learned my addr 172.21.15.66:0/1660637321 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:11.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03d1400640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03dc15a310 0x7f03dc14b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:11.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03d9e00640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03dc0aeec0 0x7f03dc152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:11.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03da800640 1 -- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03dc15a310 msgr2=0x7f03dc14b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:11.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03da800640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03dc15a310 0x7f03dc14b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03da800640 1 -- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03dc0aeec0 msgr2=0x7f03dc152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:11.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03da800640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03dc0aeec0 0x7f03dc152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03da800640 1 -- 172.21.15.66:0/1660637321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc14e390 con 0x7f03dc14c760 2024-09-10T22:15:11.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03d9e00640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03dc0aeec0 0x7f03dc152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:11.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03da800640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03dc14c760 0x7f03dc14bb90 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f03d400c7c0 tx=0x7f03d400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:11.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03d2a00640 1 -- 172.21.15.66:0/1660637321 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4003b80 con 0x7f03dc14c760 2024-09-10T22:15:11.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03d2a00640 1 -- 172.21.15.66:0/1660637321 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03d4003d20 con 0x7f03dc14c760 2024-09-10T22:15:11.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03e0e00640 1 -- 172.21.15.66:0/1660637321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03dc14e670 con 0x7f03dc14c760 2024-09-10T22:15:11.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03d2a00640 1 -- 172.21.15.66:0/1660637321 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d400aab0 con 0x7f03dc14c760 2024-09-10T22:15:11.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.582+0000 7f03e0e00640 1 -- 172.21.15.66:0/1660637321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03dc074540 con 0x7f03dc14c760 2024-09-10T22:15:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.586+0000 7f03e0e00640 1 -- 172.21.15.66:0/1660637321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03dc15df60 con 0x7f03dc14c760 2024-09-10T22:15:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.586+0000 7f03d2a00640 1 -- 172.21.15.66:0/1660637321 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f03d400ac50 con 0x7f03dc14c760 2024-09-10T22:15:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.586+0000 7f03d2a00640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03b8074b90 0x7f03b8077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.586+0000 7f03d2a00640 1 -- 172.21.15.66:0/1660637321 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f03d4092410 con 0x7f03dc14c760 2024-09-10T22:15:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.586+0000 7f03d9e00640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03b8074b90 0x7f03b8077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.590+0000 7f03d9e00640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03b8074b90 0x7f03b8077050 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009ac0 tx=0x7f03cc0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.590+0000 7f03d2a00640 1 -- 172.21.15.66:0/1660637321 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f03d405f550 con 0x7f03dc14c760 2024-09-10T22:15:11.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.798+0000 7f03e0e00640 1 -- 172.21.15.66:0/1660637321 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone36130", "target": ["mon-mgr", ""]}) -- 0x7f03dc070420 con 0x7f03b8074b90 2024-09-10T22:15:11.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.806+0000 7f03d2a00640 1 -- 172.21.15.66:0/1660637321 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f03dc070420 con 0x7f03b8074b90 2024-09-10T22:15:11.812 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:11.813 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:15:11.813 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "canceled", 2024-09-10T22:15:11.813 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:15:11.813 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:15:11.813 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:15:11.813 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:15:11.813 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:11.813 INFO:teuthology.orchestra.run.smithi066.stdout: "failure": { 2024-09-10T22:15:11.813 INFO:teuthology.orchestra.run.smithi066.stdout: "errno": "4", 2024-09-10T22:15:11.813 INFO:teuthology.orchestra.run.smithi066.stdout: "error_msg": "user interrupted clone operation" 2024-09-10T22:15:11.814 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:11.814 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:11.814 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:11.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.810+0000 7f03e0e00640 1 -- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03b8074b90 msgr2=0x7f03b8077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:11.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.810+0000 7f03e0e00640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03b8074b90 0x7f03b8077050 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009ac0 tx=0x7f03cc0091e0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.810+0000 7f03e0e00640 1 -- 172.21.15.66:0/1660637321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03dc14c760 msgr2=0x7f03dc14bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:11.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.810+0000 7f03e0e00640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03dc14c760 0x7f03dc14bb90 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f03d400c7c0 tx=0x7f03d400cc80 comp rx=0 tx=0).stop 2024-09-10T22:15:11.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.810+0000 7f03e0e00640 1 -- 172.21.15.66:0/1660637321 shutdown_connections 2024-09-10T22:15:11.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.810+0000 7f03e0e00640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f03b8074b90 0x7f03b8077050 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.810+0000 7f03e0e00640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03dc14c760 0x7f03dc14bb90 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.810+0000 7f03e0e00640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03dc15a310 0x7f03dc14b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.810+0000 7f03e0e00640 1 --2- 172.21.15.66:0/1660637321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03dc0aeec0 0x7f03dc152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.810+0000 7f03e0e00640 1 -- 172.21.15.66:0/1660637321 >> 172.21.15.66:0/1660637321 conn(0x7f03dc0cfa30 msgr2=0x7f03dc153b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:11.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.810+0000 7f03e0e00640 1 -- 172.21.15.66:0/1660637321 shutdown_connections 2024-09-10T22:15:11.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.810+0000 7f03e0e00640 1 -- 172.21.15.66:0/1660637321 wait complete. 2024-09-10T22:15:11.834 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-10T22:15:11.834 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99941 2024-09-10T22:15:11.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 -- 172.21.15.66:0/723547623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9a00051d0 msgr2=0x7fe9a000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:11.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 --2- 172.21.15.66:0/723547623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9a00051d0 0x7fe9a000b9d0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400c640 tx=0x7fe9b401f9c0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 -- 172.21.15.66:0/723547623 shutdown_connections 2024-09-10T22:15:11.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 --2- 172.21.15.66:0/723547623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9a00051d0 0x7fe9a000b9d0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 -- 172.21.15.66:0/723547623 >> 172.21.15.66:0/723547623 conn(0x7fe9bc0d3e90 msgr2=0x7fe9bc0d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:11.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 -- 172.21.15.66:0/723547623 shutdown_connections 2024-09-10T22:15:11.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 -- 172.21.15.66:0/723547623 wait complete. 2024-09-10T22:15:11.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 Processor -- start 2024-09-10T22:15:11.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 -- start start 2024-09-10T22:15:11.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9a00051d0 0x7fe9bc0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:11.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9bc1f8350 0x7fe9bc15d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:11.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9bc156e50 0x7fe9bc1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:11.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc161bb0 con 0x7fe9bc1f8350 2024-09-10T22:15:11.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc161a30 con 0x7fe9bc156e50 2024-09-10T22:15:11.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9c1800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe9bc161d30 con 0x7fe9a00051d0 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9b2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9bc1f8350 0x7fe9bc15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9b2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9bc1f8350 0x7fe9bc15d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52632/0 (socket says 172.21.15.66:52632) 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9b2000640 1 -- 172.21.15.66:0/620955197 learned_addr learned my addr 172.21.15.66:0/620955197 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9baa00640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9a00051d0 0x7fe9bc0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9bb400640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9bc156e50 0x7fe9bc1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9b2000640 1 -- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9a00051d0 msgr2=0x7fe9bc0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9b2000640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9a00051d0 0x7fe9bc0cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9b2000640 1 -- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9bc156e50 msgr2=0x7fe9bc1572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9b2000640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9bc156e50 0x7fe9bc1572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9b2000640 1 -- 172.21.15.66:0/620955197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc1f52b0 con 0x7fe9bc1f8350 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.982+0000 7fe9b2000640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9bc1f8350 0x7fe9bc15d550 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4011850 tx=0x7fe9a4011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.986+0000 7fe9b3400640 1 -- 172.21.15.66:0/620955197 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4004460 con 0x7fe9bc1f8350 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.986+0000 7fe9b3400640 1 -- 172.21.15.66:0/620955197 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9a4016070 con 0x7fe9bc1f8350 2024-09-10T22:15:11.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.986+0000 7fe9b3400640 1 -- 172.21.15.66:0/620955197 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a400fc50 con 0x7fe9bc1f8350 2024-09-10T22:15:11.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.986+0000 7fe9c1800640 1 -- 172.21.15.66:0/620955197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9bc0ce1c0 con 0x7fe9bc1f8350 2024-09-10T22:15:11.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.986+0000 7fe9c1800640 1 -- 172.21.15.66:0/620955197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9bc1f47d0 con 0x7fe9bc1f8350 2024-09-10T22:15:11.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.986+0000 7fe9c1800640 1 -- 172.21.15.66:0/620955197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9bc160600 con 0x7fe9bc1f8350 2024-09-10T22:15:11.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.986+0000 7fe9b3400640 1 -- 172.21.15.66:0/620955197 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7fe9a4003600 con 0x7fe9bc1f8350 2024-09-10T22:15:11.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.986+0000 7fe9b3400640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe998074bc0 0x7fe998077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:11.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.986+0000 7fe9b3400640 1 -- 172.21.15.66:0/620955197 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7fe9a4093020 con 0x7fe9bc1f8350 2024-09-10T22:15:11.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.986+0000 7fe9baa00640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe998074bc0 0x7fe998077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:11.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.990+0000 7fe9baa00640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe998074bc0 0x7fe998077080 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac000c10 tx=0x7fe9ac006b20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:11.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:11.990+0000 7fe9b3400640 1 -- 172.21.15.66:0/620955197 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe9a405fb90 con 0x7fe9bc1f8350 2024-09-10T22:15:12.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.198+0000 7fe9c1800640 1 -- 172.21.15.66:0/620955197 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99941", "target": ["mon-mgr", ""]}) -- 0x7fe9bc0b48d0 con 0x7fe998074bc0 2024-09-10T22:15:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9b3400640 1 -- 172.21.15.66:0/620955197 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fe9bc0b48d0 con 0x7fe998074bc0 2024-09-10T22:15:12.215 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:12.215 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:15:12.215 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "complete" 2024-09-10T22:15:12.215 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:12.215 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9c1800640 1 -- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe998074bc0 msgr2=0x7fe998077080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9c1800640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe998074bc0 0x7fe998077080 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac000c10 tx=0x7fe9ac006b20 comp rx=0 tx=0).stop 2024-09-10T22:15:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9c1800640 1 -- 172.21.15.66:0/620955197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9bc1f8350 msgr2=0x7fe9bc15d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9c1800640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9bc1f8350 0x7fe9bc15d550 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4011850 tx=0x7fe9a4011d10 comp rx=0 tx=0).stop 2024-09-10T22:15:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9c1800640 1 -- 172.21.15.66:0/620955197 shutdown_connections 2024-09-10T22:15:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9c1800640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe998074bc0 0x7fe998077080 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9c1800640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9bc156e50 0x7fe9bc1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9c1800640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9bc1f8350 0x7fe9bc15d550 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9c1800640 1 --2- 172.21.15.66:0/620955197 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9a00051d0 0x7fe9bc0cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9c1800640 1 -- 172.21.15.66:0/620955197 >> 172.21.15.66:0/620955197 conn(0x7fe9bc0d3e90 msgr2=0x7fe9bc0b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9c1800640 1 -- 172.21.15.66:0/620955197 shutdown_connections 2024-09-10T22:15:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.210+0000 7fe9c1800640 1 -- 172.21.15.66:0/620955197 wait complete. 2024-09-10T22:15:12.236 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-10T22:15:12.236 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone53292 2024-09-10T22:15:12.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 -- 172.21.15.66:0/1453113205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f3815ef20 msgr2=0x7f8f380cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:12.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 --2- 172.21.15.66:0/1453113205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f3815ef20 0x7f8f380cecc0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009930 tx=0x7f8f2801c8b0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 -- 172.21.15.66:0/1453113205 shutdown_connections 2024-09-10T22:15:12.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 --2- 172.21.15.66:0/1453113205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f3815ef20 0x7f8f380cecc0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 -- 172.21.15.66:0/1453113205 >> 172.21.15.66:0/1453113205 conn(0x7f8f380d1b10 msgr2=0x7f8f380d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:12.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 -- 172.21.15.66:0/1453113205 shutdown_connections 2024-09-10T22:15:12.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 -- 172.21.15.66:0/1453113205 wait complete. 2024-09-10T22:15:12.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 Processor -- start 2024-09-10T22:15:12.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 -- start start 2024-09-10T22:15:12.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f380cd7a0 0x7f8f3814d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:12.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f3815ef20 0x7f8f380b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:12.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f3815f300 0x7f8f380b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:12.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f380cdbb0 con 0x7f8f380cd7a0 2024-09-10T22:15:12.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8f380d0240 con 0x7f8f3815f300 2024-09-10T22:15:12.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.382+0000 7f8f3e200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8f380cdd30 con 0x7f8f3815ef20 2024-09-10T22:15:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f380cd7a0 0x7f8f3814d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f380cd7a0 0x7f8f3814d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46084/0 (socket says 172.21.15.66:46084) 2024-09-10T22:15:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3ce00640 1 -- 172.21.15.66:0/4066967128 learned_addr learned my addr 172.21.15.66:0/4066967128 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f2fe00640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f3815ef20 0x7f8f380b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3d800640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f3815f300 0x7f8f380b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3ce00640 1 -- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f3815ef20 msgr2=0x7f8f380b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3ce00640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f3815ef20 0x7f8f380b1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3ce00640 1 -- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f3815f300 msgr2=0x7f8f380b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3ce00640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f3815f300 0x7f8f380b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3ce00640 1 -- 172.21.15.66:0/4066967128 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f381fad50 con 0x7f8f380cd7a0 2024-09-10T22:15:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3ce00640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f380cd7a0 0x7f8f3814d050 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009850 tx=0x7f8f28004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f35600640 1 -- 172.21.15.66:0/4066967128 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f280b2040 con 0x7f8f380cd7a0 2024-09-10T22:15:12.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f35600640 1 -- 172.21.15.66:0/4066967128 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f28005310 con 0x7f8f380cd7a0 2024-09-10T22:15:12.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f35600640 1 -- 172.21.15.66:0/4066967128 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f280044d0 con 0x7f8f380cd7a0 2024-09-10T22:15:12.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3e200640 1 -- 172.21.15.66:0/4066967128 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3814d590 con 0x7f8f380cd7a0 2024-09-10T22:15:12.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3e200640 1 -- 172.21.15.66:0/4066967128 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f381faff0 con 0x7f8f380cd7a0 2024-09-10T22:15:12.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f3e200640 1 -- 172.21.15.66:0/4066967128 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f380cecc0 con 0x7f8f380cd7a0 2024-09-10T22:15:12.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.386+0000 7f8f35600640 1 -- 172.21.15.66:0/4066967128 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f8f280046d0 con 0x7f8f380cd7a0 2024-09-10T22:15:12.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.390+0000 7f8f35600640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f14074b90 0x7f8f14077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:12.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.390+0000 7f8f35600640 1 -- 172.21.15.66:0/4066967128 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f8f2812c1a0 con 0x7f8f380cd7a0 2024-09-10T22:15:12.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.390+0000 7f8f2fe00640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f14074b90 0x7f8f14077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:12.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.390+0000 7f8f2fe00640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f14074b90 0x7f8f14077050 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8f380ca160 tx=0x7f8f20009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:12.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.394+0000 7f8f35600640 1 -- 172.21.15.66:0/4066967128 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8f280164d0 con 0x7f8f380cd7a0 2024-09-10T22:15:12.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.602+0000 7f8f3e200640 1 -- 172.21.15.66:0/4066967128 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone53292", "target": ["mon-mgr", ""]}) -- 0x7f8f38070a80 con 0x7f8f14074b90 2024-09-10T22:15:12.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.610+0000 7f8f35600640 1 -- 172.21.15.66:0/4066967128 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f8f38070a80 con 0x7f8f14074b90 2024-09-10T22:15:12.617 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:12.617 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:15:12.617 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "canceled", 2024-09-10T22:15:12.617 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:15:12.617 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:15:12.617 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:15:12.617 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:15:12.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:12.617 INFO:teuthology.orchestra.run.smithi066.stdout: "failure": { 2024-09-10T22:15:12.617 INFO:teuthology.orchestra.run.smithi066.stdout: "errno": "4", 2024-09-10T22:15:12.617 INFO:teuthology.orchestra.run.smithi066.stdout: "error_msg": "user interrupted clone operation" 2024-09-10T22:15:12.617 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:12.618 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:12.618 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:12.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.614+0000 7f8f3e200640 1 -- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f14074b90 msgr2=0x7f8f14077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:12.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.614+0000 7f8f3e200640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f14074b90 0x7f8f14077050 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8f380ca160 tx=0x7f8f20009040 comp rx=0 tx=0).stop 2024-09-10T22:15:12.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.614+0000 7f8f3e200640 1 -- 172.21.15.66:0/4066967128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f380cd7a0 msgr2=0x7f8f3814d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:12.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.614+0000 7f8f3e200640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f380cd7a0 0x7f8f3814d050 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009850 tx=0x7f8f28004220 comp rx=0 tx=0).stop 2024-09-10T22:15:12.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.614+0000 7f8f3e200640 1 -- 172.21.15.66:0/4066967128 shutdown_connections 2024-09-10T22:15:12.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.614+0000 7f8f3e200640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8f14074b90 0x7f8f14077050 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.614+0000 7f8f3e200640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f3815f300 0x7f8f380b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.614+0000 7f8f3e200640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f3815ef20 0x7f8f380b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.614+0000 7f8f3e200640 1 --2- 172.21.15.66:0/4066967128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f380cd7a0 0x7f8f3814d050 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.614+0000 7f8f3e200640 1 -- 172.21.15.66:0/4066967128 >> 172.21.15.66:0/4066967128 conn(0x7f8f380d1b10 msgr2=0x7f8f380aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:12.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.614+0000 7f8f3e200640 1 -- 172.21.15.66:0/4066967128 shutdown_connections 2024-09-10T22:15:12.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.614+0000 7f8f3e200640 1 -- 172.21.15.66:0/4066967128 wait complete. 2024-09-10T22:15:12.638 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-10T22:15:12.638 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone71863 2024-09-10T22:15:12.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.782+0000 7f01f8c00640 1 -- 172.21.15.66:0/3316584251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01f415f300 msgr2=0x7f01f40b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:12.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.782+0000 7f01f8c00640 1 --2- 172.21.15.66:0/3316584251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01f415f300 0x7f01f40b1090 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009900 tx=0x7f01dc01c870 comp rx=0 tx=0).stop 2024-09-10T22:15:12.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.782+0000 7f01f8c00640 1 -- 172.21.15.66:0/3316584251 shutdown_connections 2024-09-10T22:15:12.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.782+0000 7f01f8c00640 1 --2- 172.21.15.66:0/3316584251 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01f40b15d0 0x7f01f40cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.782+0000 7f01f8c00640 1 --2- 172.21.15.66:0/3316584251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01f415f300 0x7f01f40b1090 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.782+0000 7f01f8c00640 1 --2- 172.21.15.66:0/3316584251 >> v2:172.21.15.94:3300/0 conn(0x7f01f415ef20 0x7f01f40cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.782+0000 7f01f8c00640 1 -- 172.21.15.66:0/3316584251 >> 172.21.15.66:0/3316584251 conn(0x7f01f40d1b10 msgr2=0x7f01f40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:12.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.782+0000 7f01f8c00640 1 -- 172.21.15.66:0/3316584251 shutdown_connections 2024-09-10T22:15:12.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.782+0000 7f01f8c00640 1 -- 172.21.15.66:0/3316584251 wait complete. 2024-09-10T22:15:12.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.782+0000 7f01f8c00640 1 Processor -- start 2024-09-10T22:15:12.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.782+0000 7f01f8c00640 1 -- start start 2024-09-10T22:15:12.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f8c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01f40b15d0 0x7f01f40c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:12.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01f415ef20 0x7f01f41fba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:12.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f8c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01f415f300 0x7f01f4070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:12.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f8c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f01f415fd90 con 0x7f01f415ef20 2024-09-10T22:15:12.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f8c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01f415fc10 con 0x7f01f415f300 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f8c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f01f415ff10 con 0x7f01f40b15d0 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01ebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01f415ef20 0x7f01f41fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01ebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01f415ef20 0x7f01f41fba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46090/0 (socket says 172.21.15.66:46090) 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01ebe00640 1 -- 172.21.15.66:0/2640863986 learned_addr learned my addr 172.21.15.66:0/2640863986 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f3e00640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01f415f300 0x7f01f4070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f3400640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01f40b15d0 0x7f01f40c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01ebe00640 1 -- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01f40b15d0 msgr2=0x7f01f40c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01ebe00640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01f40b15d0 0x7f01f40c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01ebe00640 1 -- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01f415f300 msgr2=0x7f01f4070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01ebe00640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01f415f300 0x7f01f4070b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01ebe00640 1 -- 172.21.15.66:0/2640863986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f40b4a50 con 0x7f01f415ef20 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f3e00640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01f415f300 0x7f01f4070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f3400640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01f40b15d0 0x7f01f40c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01ebe00640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01f415ef20 0x7f01f41fba70 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009850 tx=0x7f01dc0a5680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01eb400640 1 -- 172.21.15.66:0/2640863986 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0b2040 con 0x7f01f415ef20 2024-09-10T22:15:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01eb400640 1 -- 172.21.15.66:0/2640863986 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01dc0043b0 con 0x7f01f415ef20 2024-09-10T22:15:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01eb400640 1 -- 172.21.15.66:0/2640863986 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc005290 con 0x7f01f415ef20 2024-09-10T22:15:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f8c00640 1 -- 172.21.15.66:0/2640863986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f40b4d30 con 0x7f01f415ef20 2024-09-10T22:15:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f8c00640 1 -- 172.21.15.66:0/2640863986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f41fb270 con 0x7f01f415ef20 2024-09-10T22:15:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01f8c00640 1 -- 172.21.15.66:0/2640863986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f4160430 con 0x7f01f415ef20 2024-09-10T22:15:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.786+0000 7f01eb400640 1 -- 172.21.15.66:0/2640863986 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f01f40d0100 con 0x7f01f415ef20 2024-09-10T22:15:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.790+0000 7f01eb400640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f01d0074bc0 0x7f01d0077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.790+0000 7f01eb400640 1 -- 172.21.15.66:0/2640863986 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f01dc12b910 con 0x7f01f415ef20 2024-09-10T22:15:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.790+0000 7f01f3400640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f01d0074bc0 0x7f01d0077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.790+0000 7f01f3400640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f01d0074bc0 0x7f01d0077080 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009b40 tx=0x7f01e4005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:12.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:12.794+0000 7f01eb400640 1 -- 172.21.15.66:0/2640863986 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f01dc0bc060 con 0x7f01f415ef20 2024-09-10T22:15:13.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:13.002+0000 7f01f8c00640 1 -- 172.21.15.66:0/2640863986 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone71863", "target": ["mon-mgr", ""]}) -- 0x7f01f40720e0 con 0x7f01d0074bc0 2024-09-10T22:15:15.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.606+0000 7f01eb400640 1 -- 172.21.15.66:0/2640863986 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f01f40720e0 con 0x7f01d0074bc0 2024-09-10T22:15:15.614 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:15.614 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:15:15.614 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "complete" 2024-09-10T22:15:15.614 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:15.614 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:15.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.610+0000 7f01f8c00640 1 -- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f01d0074bc0 msgr2=0x7f01d0077080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:15.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.610+0000 7f01f8c00640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f01d0074bc0 0x7f01d0077080 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009b40 tx=0x7f01e4005f90 comp rx=0 tx=0).stop 2024-09-10T22:15:15.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.610+0000 7f01f8c00640 1 -- 172.21.15.66:0/2640863986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01f415ef20 msgr2=0x7f01f41fba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:15.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.610+0000 7f01f8c00640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01f415ef20 0x7f01f41fba70 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009850 tx=0x7f01dc0a5680 comp rx=0 tx=0).stop 2024-09-10T22:15:15.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.610+0000 7f01f8c00640 1 -- 172.21.15.66:0/2640863986 shutdown_connections 2024-09-10T22:15:15.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.610+0000 7f01f8c00640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f01d0074bc0 0x7f01d0077080 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:15.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.610+0000 7f01f8c00640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01f415f300 0x7f01f4070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:15.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.610+0000 7f01f8c00640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01f415ef20 0x7f01f41fba70 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:15.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.610+0000 7f01f8c00640 1 --2- 172.21.15.66:0/2640863986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01f40b15d0 0x7f01f40c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:15.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.610+0000 7f01f8c00640 1 -- 172.21.15.66:0/2640863986 >> 172.21.15.66:0/2640863986 conn(0x7f01f40d1b10 msgr2=0x7f01f415d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:15.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.610+0000 7f01f8c00640 1 -- 172.21.15.66:0/2640863986 shutdown_connections 2024-09-10T22:15:15.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.610+0000 7f01f8c00640 1 -- 172.21.15.66:0/2640863986 wait complete. 2024-09-10T22:15:15.641 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-10T22:15:15.641 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99264 2024-09-10T22:15:15.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.790+0000 7fc2e8600640 1 -- 172.21.15.66:0/4074869521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015a310 msgr2=0x7fc2e00c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.790+0000 7fc2e8600640 1 --2- 172.21.15.66:0/4074869521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015a310 0x7fc2e00c48b0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00cf9e0 tx=0x7fc2d401c830 comp rx=0 tx=0).stop 2024-09-10T22:15:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.790+0000 7fc2e8600640 1 -- 172.21.15.66:0/4074869521 shutdown_connections 2024-09-10T22:15:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.790+0000 7fc2e8600640 1 --2- 172.21.15.66:0/4074869521 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2e00aeec0 0x7fc2e015df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.790+0000 7fc2e8600640 1 --2- 172.21.15.66:0/4074869521 >> v2:172.21.15.94:3300/0 conn(0x7fc2e00d0550 0x7fc2e00c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.790+0000 7fc2e8600640 1 --2- 172.21.15.66:0/4074869521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015a310 0x7fc2e00c48b0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.790+0000 7fc2e8600640 1 -- 172.21.15.66:0/4074869521 >> 172.21.15.66:0/4074869521 conn(0x7fc2e00cfa30 msgr2=0x7fc2e00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.790+0000 7fc2e8600640 1 -- 172.21.15.66:0/4074869521 shutdown_connections 2024-09-10T22:15:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.790+0000 7fc2e8600640 1 -- 172.21.15.66:0/4074869521 wait complete. 2024-09-10T22:15:15.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e8600640 1 Processor -- start 2024-09-10T22:15:15.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e8600640 1 -- start start 2024-09-10T22:15:15.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e8600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2e00aeec0 0x7fc2e00b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:15.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e8600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015a310 0x7fc2e014f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:15.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e8600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2e015b750 0x7fc2e015bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:15.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e8600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e015ec90 con 0x7fc2e015a310 2024-09-10T22:15:15.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e8600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e015eb10 con 0x7fc2e015b750 2024-09-10T22:15:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e8600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e0167300 con 0x7fc2e00aeec0 2024-09-10T22:15:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015a310 0x7fc2e014f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015a310 0x7fc2e014f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46104/0 (socket says 172.21.15.66:46104) 2024-09-10T22:15:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e5200640 1 -- 172.21.15.66:0/61608000 learned_addr learned my addr 172.21.15.66:0/61608000 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e5200640 1 -- 172.21.15.66:0/61608000 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2e00aeec0 msgr2=0x7fc2e00b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e5200640 1 --2- 172.21.15.66:0/61608000 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2e00aeec0 0x7fc2e00b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e5200640 1 -- 172.21.15.66:0/61608000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2e015b750 msgr2=0x7fc2e015bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e5200640 1 --2- 172.21.15.66:0/61608000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2e015b750 0x7fc2e015bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e5200640 1 -- 172.21.15.66:0/61608000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e00b12d0 con 0x7fc2e015a310 2024-09-10T22:15:15.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e5200640 1 --2- 172.21.15.66:0/61608000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015a310 0x7fc2e014f560 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00d980 tx=0x7fc2cc00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:15.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2dd600640 1 -- 172.21.15.66:0/61608000 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc017070 con 0x7fc2e015a310 2024-09-10T22:15:15.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2dd600640 1 -- 172.21.15.66:0/61608000 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2cc009d20 con 0x7fc2e015a310 2024-09-10T22:15:15.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2dd600640 1 -- 172.21.15.66:0/61608000 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc004280 con 0x7fc2e015a310 2024-09-10T22:15:15.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e8600640 1 -- 172.21.15.66:0/61608000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e01fc140 con 0x7fc2e015a310 2024-09-10T22:15:15.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.794+0000 7fc2e8600640 1 -- 172.21.15.66:0/61608000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e01fc700 con 0x7fc2e015a310 2024-09-10T22:15:15.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.798+0000 7fc2e8600640 1 -- 172.21.15.66:0/61608000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e015df60 con 0x7fc2e015a310 2024-09-10T22:15:15.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.798+0000 7fc2dd600640 1 -- 172.21.15.66:0/61608000 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7fc2cc003600 con 0x7fc2e015a310 2024-09-10T22:15:15.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.798+0000 7fc2dd600640 1 --2- 172.21.15.66:0/61608000 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc2c0074bc0 0x7fc2c0077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:15.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.798+0000 7fc2dd600640 1 -- 172.21.15.66:0/61608000 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7fc2cc092630 con 0x7fc2e015a310 2024-09-10T22:15:15.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.798+0000 7fc2e5c00640 1 --2- 172.21.15.66:0/61608000 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc2c0074bc0 0x7fc2c0077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:15.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.798+0000 7fc2e5c00640 1 --2- 172.21.15.66:0/61608000 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc2c0074bc0 0x7fc2c0077080 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40ae000 tx=0x7fc2d40195d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:15.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:15.802+0000 7fc2dd600640 1 -- 172.21.15.66:0/61608000 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc2e00b12d0 con 0x7fc2e015a310 2024-09-10T22:15:16.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.010+0000 7fc2e8600640 1 -- 172.21.15.66:0/61608000 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99264", "target": ["mon-mgr", ""]}) -- 0x7fc2e014fdd0 con 0x7fc2c0074bc0 2024-09-10T22:15:16.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2dd600640 1 -- 172.21.15.66:0/61608000 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fc2e014fdd0 con 0x7fc2c0074bc0 2024-09-10T22:15:16.027 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:16.027 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:15:16.027 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "canceled", 2024-09-10T22:15:16.027 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:15:16.027 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:15:16.027 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:15:16.028 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:15:16.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:16.028 INFO:teuthology.orchestra.run.smithi066.stdout: "failure": { 2024-09-10T22:15:16.028 INFO:teuthology.orchestra.run.smithi066.stdout: "errno": "4", 2024-09-10T22:15:16.028 INFO:teuthology.orchestra.run.smithi066.stdout: "error_msg": "user interrupted clone operation" 2024-09-10T22:15:16.028 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:16.029 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:16.029 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:16.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2e8600640 1 -- 172.21.15.66:0/61608000 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc2c0074bc0 msgr2=0x7fc2c0077080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:16.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2e8600640 1 --2- 172.21.15.66:0/61608000 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc2c0074bc0 0x7fc2c0077080 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40ae000 tx=0x7fc2d40195d0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2e8600640 1 -- 172.21.15.66:0/61608000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015a310 msgr2=0x7fc2e014f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:16.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2e8600640 1 --2- 172.21.15.66:0/61608000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015a310 0x7fc2e014f560 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00d980 tx=0x7fc2cc00de40 comp rx=0 tx=0).stop 2024-09-10T22:15:16.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2e8600640 1 -- 172.21.15.66:0/61608000 shutdown_connections 2024-09-10T22:15:16.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2e8600640 1 --2- 172.21.15.66:0/61608000 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc2c0074bc0 0x7fc2c0077080 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2e8600640 1 --2- 172.21.15.66:0/61608000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2e015b750 0x7fc2e015bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2e8600640 1 --2- 172.21.15.66:0/61608000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2e015a310 0x7fc2e014f560 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2e8600640 1 --2- 172.21.15.66:0/61608000 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc2e00aeec0 0x7fc2e00b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2e8600640 1 -- 172.21.15.66:0/61608000 >> 172.21.15.66:0/61608000 conn(0x7fc2e00cfa30 msgr2=0x7fc2e0072e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:16.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2e8600640 1 -- 172.21.15.66:0/61608000 shutdown_connections 2024-09-10T22:15:16.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.022+0000 7fc2e8600640 1 -- 172.21.15.66:0/61608000 wait complete. 2024-09-10T22:15:16.048 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-10T22:15:16.048 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone98955 2024-09-10T22:15:16.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.190+0000 7fb51c800640 1 -- 172.21.15.66:0/318755579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5141611d0 msgr2=0x7fb5141615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:16.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.190+0000 7fb51c800640 1 --2- 172.21.15.66:0/318755579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5141611d0 0x7fb5141615b0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fb508009900 tx=0x7fb50801c8e0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 -- 172.21.15.66:0/318755579 shutdown_connections 2024-09-10T22:15:16.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 --2- 172.21.15.66:0/318755579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5141611d0 0x7fb5141615b0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 -- 172.21.15.66:0/318755579 >> 172.21.15.66:0/318755579 conn(0x7fb5140d3e90 msgr2=0x7fb5140d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:16.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 -- 172.21.15.66:0/318755579 shutdown_connections 2024-09-10T22:15:16.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 -- 172.21.15.66:0/318755579 wait complete. 2024-09-10T22:15:16.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 Processor -- start 2024-09-10T22:15:16.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 -- start start 2024-09-10T22:15:16.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5140b2e60 0x7fb5140b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:16.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5140d49b0 0x7fb5141575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:16.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5141611d0 0x7fb5141560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:16.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb51416c4a0 con 0x7fb5140b2e60 2024-09-10T22:15:16.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb51416c320 con 0x7fb5141611d0 2024-09-10T22:15:16.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51c800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb51416c620 con 0x7fb5140d49b0 2024-09-10T22:15:16.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb519e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5140b2e60 0x7fb5140b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:16.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb519e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5140b2e60 0x7fb5140b88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46116/0 (socket says 172.21.15.66:46116) 2024-09-10T22:15:16.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb519e00640 1 -- 172.21.15.66:0/2515372236 learned_addr learned my addr 172.21.15.66:0/2515372236 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:16.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb519400640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5140d49b0 0x7fb5141575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:16.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb51a800640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5141611d0 0x7fb5141560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:16.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb519e00640 1 -- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5140d49b0 msgr2=0x7fb5141575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:16.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb519e00640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5140d49b0 0x7fb5141575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb519e00640 1 -- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5141611d0 msgr2=0x7fb5141560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:16.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb519e00640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5141611d0 0x7fb5141560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb519e00640 1 -- 172.21.15.66:0/2515372236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb514157b00 con 0x7fb5140b2e60 2024-09-10T22:15:16.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.194+0000 7fb519e00640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5140b2e60 0x7fb5140b88e0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fb508009850 tx=0x7fb5080a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:16.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.198+0000 7fb50e000640 1 -- 172.21.15.66:0/2515372236 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5080b2040 con 0x7fb5140b2e60 2024-09-10T22:15:16.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.198+0000 7fb50e000640 1 -- 172.21.15.66:0/2515372236 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5080044b0 con 0x7fb5140b2e60 2024-09-10T22:15:16.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.198+0000 7fb50e000640 1 -- 172.21.15.66:0/2515372236 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508003860 con 0x7fb5140b2e60 2024-09-10T22:15:16.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.198+0000 7fb51c800640 1 -- 172.21.15.66:0/2515372236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5141559d0 con 0x7fb5140b2e60 2024-09-10T22:15:16.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.198+0000 7fb51c800640 1 -- 172.21.15.66:0/2515372236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5140b9330 con 0x7fb5140b2e60 2024-09-10T22:15:16.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.198+0000 7fb51c800640 1 -- 172.21.15.66:0/2515372236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5141615b0 con 0x7fb5140b2e60 2024-09-10T22:15:16.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.198+0000 7fb50e000640 1 -- 172.21.15.66:0/2515372236 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7fb508003a00 con 0x7fb5140b2e60 2024-09-10T22:15:16.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.198+0000 7fb50e000640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb4f8074bf0 0x7fb4f80770b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:16.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.198+0000 7fb50e000640 1 -- 172.21.15.66:0/2515372236 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7fb50812c6b0 con 0x7fb5140b2e60 2024-09-10T22:15:16.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.198+0000 7fb519400640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb4f8074bf0 0x7fb4f80770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:16.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.202+0000 7fb519400640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb4f8074bf0 0x7fb4f80770b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb51415cd20 tx=0x7fb504009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:16.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.202+0000 7fb50e000640 1 -- 172.21.15.66:0/2515372236 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb5080164d0 con 0x7fb5140b2e60 2024-09-10T22:15:16.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.414+0000 7fb51c800640 1 -- 172.21.15.66:0/2515372236 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone98955", "target": ["mon-mgr", ""]}) -- 0x7fb5140d2c20 con 0x7fb4f8074bf0 2024-09-10T22:15:16.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.426+0000 7fb50e000640 1 -- 172.21.15.66:0/2515372236 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fb5140d2c20 con 0x7fb4f8074bf0 2024-09-10T22:15:16.433 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:16.433 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:15:16.433 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "canceled", 2024-09-10T22:15:16.433 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:15:16.433 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:15:16.433 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:15:16.434 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:15:16.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:16.434 INFO:teuthology.orchestra.run.smithi066.stdout: "failure": { 2024-09-10T22:15:16.434 INFO:teuthology.orchestra.run.smithi066.stdout: "errno": "4", 2024-09-10T22:15:16.434 INFO:teuthology.orchestra.run.smithi066.stdout: "error_msg": "user interrupted clone operation" 2024-09-10T22:15:16.434 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:16.434 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:16.434 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:16.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.430+0000 7fb51c800640 1 -- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb4f8074bf0 msgr2=0x7fb4f80770b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:16.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.430+0000 7fb51c800640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb4f8074bf0 0x7fb4f80770b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb51415cd20 tx=0x7fb504009040 comp rx=0 tx=0).stop 2024-09-10T22:15:16.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.430+0000 7fb51c800640 1 -- 172.21.15.66:0/2515372236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5140b2e60 msgr2=0x7fb5140b88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:16.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.430+0000 7fb51c800640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5140b2e60 0x7fb5140b88e0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fb508009850 tx=0x7fb5080a5700 comp rx=0 tx=0).stop 2024-09-10T22:15:16.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.430+0000 7fb51c800640 1 -- 172.21.15.66:0/2515372236 shutdown_connections 2024-09-10T22:15:16.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.430+0000 7fb51c800640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb4f8074bf0 0x7fb4f80770b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.430+0000 7fb51c800640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5141611d0 0x7fb5141560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.430+0000 7fb51c800640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5140d49b0 0x7fb5141575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.430+0000 7fb51c800640 1 --2- 172.21.15.66:0/2515372236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5140b2e60 0x7fb5140b88e0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.430+0000 7fb51c800640 1 -- 172.21.15.66:0/2515372236 >> 172.21.15.66:0/2515372236 conn(0x7fb5140d3e90 msgr2=0x7fb5140d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:16.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.430+0000 7fb51c800640 1 -- 172.21.15.66:0/2515372236 shutdown_connections 2024-09-10T22:15:16.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.430+0000 7fb51c800640 1 -- 172.21.15.66:0/2515372236 wait complete. 2024-09-10T22:15:16.455 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-10T22:15:16.456 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone55126 2024-09-10T22:15:16.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.598+0000 7f236b000640 1 -- 172.21.15.66:0/769629011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23640cd7a0 msgr2=0x7f23640cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:16.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.598+0000 7f236b000640 1 --2- 172.21.15.66:0/769629011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23640cd7a0 0x7f23640cdc10 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f235c009c80 tx=0x7f235c01cd60 comp rx=0 tx=0).stop 2024-09-10T22:15:16.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.598+0000 7f236b000640 1 -- 172.21.15.66:0/769629011 shutdown_connections 2024-09-10T22:15:16.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.598+0000 7f236b000640 1 --2- 172.21.15.66:0/769629011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23640cd7a0 0x7f23640cdc10 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.598+0000 7f236b000640 1 --2- 172.21.15.66:0/769629011 >> v2:172.21.15.94:3300/0 conn(0x7f236415ef20 0x7f23640cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.598+0000 7f236b000640 1 -- 172.21.15.66:0/769629011 >> 172.21.15.66:0/769629011 conn(0x7f23640d1b10 msgr2=0x7f23640d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:16.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.598+0000 7f236b000640 1 -- 172.21.15.66:0/769629011 shutdown_connections 2024-09-10T22:15:16.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.598+0000 7f236b000640 1 -- 172.21.15.66:0/769629011 wait complete. 2024-09-10T22:15:16.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f236b000640 1 Processor -- start 2024-09-10T22:15:16.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f236b000640 1 -- start start 2024-09-10T22:15:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f236b000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23640cd7a0 0x7f2364153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f236b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f236415ef20 0x7f236415ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f236b000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f236415f300 0x7f23641649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f236b000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2364168ee0 con 0x7f236415ef20 2024-09-10T22:15:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f236b000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2364168d60 con 0x7f236415f300 2024-09-10T22:15:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f236b000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2364169060 con 0x7f23640cd7a0 2024-09-10T22:15:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2369200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f236415ef20 0x7f236415ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2369200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f236415ef20 0x7f236415ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46132/0 (socket says 172.21.15.66:46132) 2024-09-10T22:15:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2369200640 1 -- 172.21.15.66:0/2301446311 learned_addr learned my addr 172.21.15.66:0/2301446311 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2369c00640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23640cd7a0 0x7f2364153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f236a600640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f236415f300 0x7f23641649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2369200640 1 -- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23640cd7a0 msgr2=0x7f2364153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2369200640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23640cd7a0 0x7f2364153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2369200640 1 -- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f236415f300 msgr2=0x7f23641649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2369200640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f236415f300 0x7f23641649d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2369200640 1 -- 172.21.15.66:0/2301446311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236415cfd0 con 0x7f236415ef20 2024-09-10T22:15:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2369200640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f236415ef20 0x7f236415ca90 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f235000c3f0 tx=0x7f235000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2361600640 1 -- 172.21.15.66:0/2301446311 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350017070 con 0x7f236415ef20 2024-09-10T22:15:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2361600640 1 -- 172.21.15.66:0/2301446311 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23500053c0 con 0x7f236415ef20 2024-09-10T22:15:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f2361600640 1 -- 172.21.15.66:0/2301446311 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350003860 con 0x7f236415ef20 2024-09-10T22:15:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f236b000640 1 -- 172.21.15.66:0/2301446311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2364165200 con 0x7f236415ef20 2024-09-10T22:15:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f236b000640 1 -- 172.21.15.66:0/2301446311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236406e950 con 0x7f236415ef20 2024-09-10T22:15:16.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.602+0000 7f236b000640 1 -- 172.21.15.66:0/2301446311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23640cdc10 con 0x7f236415ef20 2024-09-10T22:15:16.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.606+0000 7f2361600640 1 -- 172.21.15.66:0/2301446311 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f2350003a00 con 0x7f236415ef20 2024-09-10T22:15:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.606+0000 7f2361600640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2344074b90 0x7f2344077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.606+0000 7f2361600640 1 -- 172.21.15.66:0/2301446311 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f2350091b00 con 0x7f236415ef20 2024-09-10T22:15:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.606+0000 7f2369c00640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2344074b90 0x7f2344077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.606+0000 7f2369c00640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2344074b90 0x7f2344077050 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f235800bc00 tx=0x7f235800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.610+0000 7f2361600640 1 -- 172.21.15.66:0/2301446311 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2350001090 con 0x7f236415ef20 2024-09-10T22:15:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.818+0000 7f236b000640 1 -- 172.21.15.66:0/2301446311 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone55126", "target": ["mon-mgr", ""]}) -- 0x7f2364072e00 con 0x7f2344074b90 2024-09-10T22:15:16.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.830+0000 7f2361600640 1 -- 172.21.15.66:0/2301446311 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f2364072e00 con 0x7f2344074b90 2024-09-10T22:15:16.836 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "status": { 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "canceled", 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "source": { 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "volume": "cephfs", 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "subvolume": "sv1", 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot": "ss1" 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "failure": { 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "errno": "4", 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "error_msg": "user interrupted clone operation" 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:16.837 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.834+0000 7f236b000640 1 -- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2344074b90 msgr2=0x7f2344077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.834+0000 7f236b000640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2344074b90 0x7f2344077050 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f235800bc00 tx=0x7f235800b390 comp rx=0 tx=0).stop 2024-09-10T22:15:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.834+0000 7f236b000640 1 -- 172.21.15.66:0/2301446311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f236415ef20 msgr2=0x7f236415ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.834+0000 7f236b000640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f236415ef20 0x7f236415ca90 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f235000c3f0 tx=0x7f235000c8b0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.834+0000 7f236b000640 1 -- 172.21.15.66:0/2301446311 shutdown_connections 2024-09-10T22:15:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.834+0000 7f236b000640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2344074b90 0x7f2344077050 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.834+0000 7f236b000640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f236415f300 0x7f23641649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.834+0000 7f236b000640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f236415ef20 0x7f236415ca90 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.834+0000 7f236b000640 1 --2- 172.21.15.66:0/2301446311 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23640cd7a0 0x7f2364153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.834+0000 7f236b000640 1 -- 172.21.15.66:0/2301446311 >> 172.21.15.66:0/2301446311 conn(0x7f23640d1b10 msgr2=0x7f2364158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.834+0000 7f236b000640 1 -- 172.21.15.66:0/2301446311 shutdown_connections 2024-09-10T22:15:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.834+0000 7f236b000640 1 -- 172.21.15.66:0/2301446311 wait complete. 2024-09-10T22:15:16.858 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-10T22:15:16.859 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-10T22:15:17.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.998+0000 7f93ca400640 1 -- 172.21.15.66:0/1670580422 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93c415f460 msgr2=0x7f93c40cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.998+0000 7f93ca400640 1 --2- 172.21.15.66:0/1670580422 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93c415f460 0x7f93c40cfc00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009840 tx=0x7f93b401c6d0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.998+0000 7f93ca400640 1 -- 172.21.15.66:0/1670580422 shutdown_connections 2024-09-10T22:15:17.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.998+0000 7f93ca400640 1 --2- 172.21.15.66:0/1670580422 >> v2:172.21.15.94:3300/0 conn(0x7f93c40b0f60 0x7f93c40d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.998+0000 7f93ca400640 1 --2- 172.21.15.66:0/1670580422 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93c415f460 0x7f93c40cfc00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.998+0000 7f93ca400640 1 -- 172.21.15.66:0/1670580422 >> 172.21.15.66:0/1670580422 conn(0x7f93c40d1d00 msgr2=0x7f93c40d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:17.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.998+0000 7f93ca400640 1 -- 172.21.15.66:0/1670580422 shutdown_connections 2024-09-10T22:15:17.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:16.998+0000 7f93ca400640 1 -- 172.21.15.66:0/1670580422 wait complete. 2024-09-10T22:15:17.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93ca400640 1 Processor -- start 2024-09-10T22:15:17.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93ca400640 1 -- start start 2024-09-10T22:15:17.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93ca400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93c40b0f60 0x7f93c406f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:17.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93ca400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93c40b32a0 0x7f93c4156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:17.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93ca400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93c415f460 0x7f93c41566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:17.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93ca400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4168ff0 con 0x7f93c40b32a0 2024-09-10T22:15:17.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93ca400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4168e70 con 0x7f93c40b0f60 2024-09-10T22:15:17.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93ca400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f93c4169170 con 0x7f93c415f460 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93c40b32a0 0x7f93c4156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93c40b32a0 0x7f93c4156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46146/0 (socket says 172.21.15.66:46146) 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93bbe00640 1 -- 172.21.15.66:0/992075928 learned_addr learned my addr 172.21.15.66:0/992075928 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93c3400640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93c40b0f60 0x7f93c406f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93c3e00640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93c415f460 0x7f93c41566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93bbe00640 1 -- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93c415f460 msgr2=0x7f93c41566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93bbe00640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93c415f460 0x7f93c41566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93bbe00640 1 -- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93c40b0f60 msgr2=0x7f93c406f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93bbe00640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93c40b0f60 0x7f93c406f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93bbe00640 1 -- 172.21.15.66:0/992075928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c406e660 con 0x7f93c40b32a0 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93c3e00640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93c415f460 0x7f93c41566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93bbe00640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93c40b32a0 0x7f93c4156190 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e9f0 tx=0x7f93ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93bb400640 1 -- 172.21.15.66:0/992075928 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac00cc60 con 0x7f93c40b32a0 2024-09-10T22:15:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93ca400640 1 -- 172.21.15.66:0/992075928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c415b420 con 0x7f93c40b32a0 2024-09-10T22:15:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93bb400640 1 -- 172.21.15.66:0/992075928 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93ac010430 con 0x7f93c40b32a0 2024-09-10T22:15:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93bb400640 1 -- 172.21.15.66:0/992075928 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac010860 con 0x7f93c40b32a0 2024-09-10T22:15:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.002+0000 7f93ca400640 1 -- 172.21.15.66:0/992075928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c415b9e0 con 0x7f93c40b32a0 2024-09-10T22:15:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.006+0000 7f93ca400640 1 -- 172.21.15.66:0/992075928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c40b14b0 con 0x7f93c40b32a0 2024-09-10T22:15:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.006+0000 7f93bb400640 1 -- 172.21.15.66:0/992075928 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f93ac010a00 con 0x7f93c40b32a0 2024-09-10T22:15:17.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.006+0000 7f93bb400640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f93a0074bc0 0x7f93a0077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:17.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.006+0000 7f93bb400640 1 -- 172.21.15.66:0/992075928 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f93ac092f80 con 0x7f93c40b32a0 2024-09-10T22:15:17.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.006+0000 7f93c3400640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f93a0074bc0 0x7f93a0077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:17.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.006+0000 7f93c3400640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f93a0074bc0 0x7f93a0077080 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f93b40af000 tx=0x7f93b401af10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:17.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.010+0000 7f93bb400640 1 -- 172.21.15.66:0/992075928 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f93ac05f9f0 con 0x7f93c40b32a0 2024-09-10T22:15:17.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93ca400640 1 -- 172.21.15.66:0/992075928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f93c4074aa0 con 0x7f93c40b32a0 2024-09-10T22:15:17.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93bb400640 1 -- 172.21.15.66:0/992075928 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+3808 (secure 0 0 0) 0x7f93ac064420 con 0x7f93c40b32a0 2024-09-10T22:15:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:17.252 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "fsid": "944cb5ba-80f3-427e-9dcb-45be8cb10513", 2024-09-10T22:15:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "health": { 2024-09-10T22:15:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "HEALTH_OK", 2024-09-10T22:15:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "checks": {}, 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "mutes": [] 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "election_epoch": 8, 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum": [ 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: 1, 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: 2 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_names": [ 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "a", 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "b", 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "c" 2024-09-10T22:15:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "quorum_age": 910, 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "monmap": { 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 1, 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "min_mon_release_name": "squid", 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "num_mons": 3 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap": { 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 143, 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "num_osds": 8, 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "num_up_osds": 8, 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_up_since": 1726005621, 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "num_in_osds": 8, 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_in_since": 1726005611, 2024-09-10T22:15:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "num_remapped_pgs": 0 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap": { 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "pgs_by_state": [ 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "state_name": "active+clean", 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "count": 97 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pgs": 97, 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "num_pools": 3, 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "num_objects": 4837, 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "data_bytes": 20126459983, 2024-09-10T22:15:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_used": 41467707392, 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_avail": 731626405888, 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "bytes_total": 773094113280, 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "read_bytes_sec": 541296300, 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "write_bytes_sec": 539955774, 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "read_op_per_sec": 133, 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "write_op_per_sec": 147 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "fsmap": { 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 97, 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-09-10T22:14:20:178870+0000", 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 10, 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "up": 1, 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "in": 1, 2024-09-10T22:15:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1, 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "by_rank": [ 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "filesystem_id": 10, 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "rank": 0, 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "c", 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "up:active", 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 9282 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "up:standby": 3 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "mgrmap": { 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-09-10T22:15:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "num_standbys": 1, 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "services": {} 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "servicemap": { 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 27, 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "modified": "2024-09-10T22:15:16.640244+0000", 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "services": { 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": { 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "daemons": { 2024-09-10T22:15:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "summary": "", 2024-09-10T22:15:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "x": { 2024-09-10T22:15:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "start_epoch": 0, 2024-09-10T22:15:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "start_stamp": "0.000000", 2024-09-10T22:15:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 0, 2024-09-10T22:15:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-10T22:15:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": {}, 2024-09-10T22:15:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "task_status": {} 2024-09-10T22:15:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "y": { 2024-09-10T22:15:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "start_epoch": 0, 2024-09-10T22:15:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "start_stamp": "0.000000", 2024-09-10T22:15:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 0, 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": {}, 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "task_status": {} 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": { 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "daemons": { 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "summary": "", 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "a": { 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "start_epoch": 0, 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "start_stamp": "0.000000", 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 0, 2024-09-10T22:15:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": {}, 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "task_status": {} 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "b": { 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "start_epoch": 0, 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "start_stamp": "0.000000", 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 0, 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": {}, 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "task_status": {} 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "c": { 2024-09-10T22:15:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "start_epoch": 0, 2024-09-10T22:15:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "start_stamp": "0.000000", 2024-09-10T22:15:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 0, 2024-09-10T22:15:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-10T22:15:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": {}, 2024-09-10T22:15:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "task_status": {} 2024-09-10T22:15:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "progress_events": {} 2024-09-10T22:15:17.262 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:17.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93ca400640 1 -- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f93a0074bc0 msgr2=0x7f93a0077080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93ca400640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f93a0074bc0 0x7f93a0077080 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f93b40af000 tx=0x7f93b401af10 comp rx=0 tx=0).stop 2024-09-10T22:15:17.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93ca400640 1 -- 172.21.15.66:0/992075928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93c40b32a0 msgr2=0x7f93c4156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93ca400640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93c40b32a0 0x7f93c4156190 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e9f0 tx=0x7f93ac00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93ca400640 1 -- 172.21.15.66:0/992075928 shutdown_connections 2024-09-10T22:15:17.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93ca400640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f93a0074bc0 0x7f93a0077080 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93ca400640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93c415f460 0x7f93c41566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93ca400640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93c40b32a0 0x7f93c4156190 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93ca400640 1 --2- 172.21.15.66:0/992075928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93c40b0f60 0x7f93c406f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93ca400640 1 -- 172.21.15.66:0/992075928 >> 172.21.15.66:0/992075928 conn(0x7f93c40d1d00 msgr2=0x7f93c40b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:17.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.246+0000 7f93ca400640 1 -- 172.21.15.66:0/992075928 shutdown_connections 2024-09-10T22:15:17.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.250+0000 7f93ca400640 1 -- 172.21.15.66:0/992075928 wait complete. 2024-09-10T22:15:17.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:15:17.273 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone36130/9c08db3e-b973-4f8c-ac9b-4fd7276890c6 2024-09-10T22:15:17.273 DEBUG:teuthology.orchestra.run.smithi066:> ) 2024-09-10T22:15:17.286 INFO:teuthology.orchestra.run.smithi066.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone36130/9c08db3e-b973-4f8c-ac9b-4fd7276890c6': No such file or directory 2024-09-10T22:15:17.286 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T22:15:17.287 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T22:15:17.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.470+0000 7f6624a00640 1 -- 172.21.15.66:0/4248229417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f662015bd40 msgr2=0x7f66200cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.470+0000 7f6624a00640 1 --2- 172.21.15.66:0/4248229417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f662015bd40 0x7f66200cffa0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f6610009900 tx=0x7f661001c850 comp rx=0 tx=0).stop 2024-09-10T22:15:17.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.470+0000 7f6624a00640 1 -- 172.21.15.66:0/4248229417 shutdown_connections 2024-09-10T22:15:17.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.470+0000 7f6624a00640 1 --2- 172.21.15.66:0/4248229417 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66200cf830 0x7f66200d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.470+0000 7f6624a00640 1 --2- 172.21.15.66:0/4248229417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f662015bd40 0x7f66200cffa0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.470+0000 7f6624a00640 1 -- 172.21.15.66:0/4248229417 >> 172.21.15.66:0/4248229417 conn(0x7f66200d1c60 msgr2=0x7f66200d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:17.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.470+0000 7f6624a00640 1 -- 172.21.15.66:0/4248229417 shutdown_connections 2024-09-10T22:15:17.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.470+0000 7f6624a00640 1 -- 172.21.15.66:0/4248229417 wait complete. 2024-09-10T22:15:17.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6624a00640 1 Processor -- start 2024-09-10T22:15:17.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6624a00640 1 -- start start 2024-09-10T22:15:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6624a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66200aec10 0x7f6620163b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6624a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66200cf830 0x7f6620149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6624a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662015bd40 0x7f6620153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6624a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6620169070 con 0x7f66200aec10 2024-09-10T22:15:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6624a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6620168ef0 con 0x7f662015bd40 2024-09-10T22:15:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6624a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f66201691f0 con 0x7f66200cf830 2024-09-10T22:15:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f661de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66200aec10 0x7f6620163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f661de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66200aec10 0x7f6620163b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46164/0 (socket says 172.21.15.66:46164) 2024-09-10T22:15:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f661de00640 1 -- 172.21.15.66:0/2793914920 learned_addr learned my addr 172.21.15.66:0/2793914920 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f661e800640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662015bd40 0x7f6620153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6615400640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66200cf830 0x7f6620149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f661de00640 1 -- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66200cf830 msgr2=0x7f6620149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f661de00640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66200cf830 0x7f6620149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f661de00640 1 -- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662015bd40 msgr2=0x7f6620153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f661de00640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662015bd40 0x7f6620153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f661de00640 1 -- 172.21.15.66:0/2793914920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6620164160 con 0x7f66200aec10 2024-09-10T22:15:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f661de00640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66200aec10 0x7f6620163b60 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f6610009850 tx=0x7f6610005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6616a00640 1 -- 172.21.15.66:0/2793914920 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66100b2040 con 0x7f66200aec10 2024-09-10T22:15:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6616a00640 1 -- 172.21.15.66:0/2793914920 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66100a5c00 con 0x7f66200aec10 2024-09-10T22:15:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6624a00640 1 -- 172.21.15.66:0/2793914920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6620164440 con 0x7f66200aec10 2024-09-10T22:15:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6616a00640 1 -- 172.21.15.66:0/2793914920 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6610005020 con 0x7f66200aec10 2024-09-10T22:15:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6624a00640 1 -- 172.21.15.66:0/2793914920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6620164a00 con 0x7f66200aec10 2024-09-10T22:15:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.474+0000 7f6624a00640 1 -- 172.21.15.66:0/2793914920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66200af010 con 0x7f66200aec10 2024-09-10T22:15:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.478+0000 7f6616a00640 1 -- 172.21.15.66:0/2793914920 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f66100be030 con 0x7f66200aec10 2024-09-10T22:15:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.478+0000 7f6616a00640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65fc074b90 0x7f65fc077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.478+0000 7f6616a00640 1 -- 172.21.15.66:0/2793914920 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f661012ba80 con 0x7f66200aec10 2024-09-10T22:15:17.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.478+0000 7f6615400640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65fc074b90 0x7f65fc077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:17.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.478+0000 7f6615400640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65fc074b90 0x7f65fc077050 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f66080058d0 tx=0x7f6608005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:17.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.482+0000 7f6616a00640 1 -- 172.21.15.66:0/2793914920 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f66100164d0 con 0x7f66200aec10 2024-09-10T22:15:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.678+0000 7f6624a00640 1 -- 172.21.15.66:0/2793914920 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f66200b0df0 con 0x7f65fc074b90 2024-09-10T22:15:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6616a00640 1 -- 172.21.15.66:0/2793914920 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f66200b0df0 con 0x7f65fc074b90 2024-09-10T22:15:17.686 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:15:17.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:15:17.687 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephfs" 2024-09-10T22:15:17.687 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:17.687 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:15:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6624a00640 1 -- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65fc074b90 msgr2=0x7f65fc077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6624a00640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65fc074b90 0x7f65fc077050 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f66080058d0 tx=0x7f6608005860 comp rx=0 tx=0).stop 2024-09-10T22:15:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6624a00640 1 -- 172.21.15.66:0/2793914920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66200aec10 msgr2=0x7f6620163b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6624a00640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66200aec10 0x7f6620163b60 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f6610009850 tx=0x7f6610005760 comp rx=0 tx=0).stop 2024-09-10T22:15:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6624a00640 1 -- 172.21.15.66:0/2793914920 shutdown_connections 2024-09-10T22:15:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6624a00640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f65fc074b90 0x7f65fc077050 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6624a00640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662015bd40 0x7f6620153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6624a00640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66200cf830 0x7f6620149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6624a00640 1 --2- 172.21.15.66:0/2793914920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66200aec10 0x7f6620163b60 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6624a00640 1 -- 172.21.15.66:0/2793914920 >> 172.21.15.66:0/2793914920 conn(0x7f66200d1c60 msgr2=0x7f662014f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6624a00640 1 -- 172.21.15.66:0/2793914920 shutdown_connections 2024-09-10T22:15:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.682+0000 7f6624a00640 1 -- 172.21.15.66:0/2793914920 wait complete. 2024-09-10T22:15:17.708 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-10T22:15:17.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.850+0000 7f9113c00640 1 -- 172.21.15.66:0/1295022303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f910c15def0 msgr2=0x7f910c0adf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.850+0000 7f9113c00640 1 --2- 172.21.15.66:0/1295022303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f910c15def0 0x7f910c0adf60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9100009840 tx=0x7f910001c820 comp rx=0 tx=0).stop 2024-09-10T22:15:17.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.850+0000 7f9113c00640 1 -- 172.21.15.66:0/1295022303 shutdown_connections 2024-09-10T22:15:17.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.850+0000 7f9113c00640 1 --2- 172.21.15.66:0/1295022303 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f910c0d0360 0x7f910c0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.850+0000 7f9113c00640 1 --2- 172.21.15.66:0/1295022303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f910c15def0 0x7f910c0adf60 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.850+0000 7f9113c00640 1 -- 172.21.15.66:0/1295022303 >> 172.21.15.66:0/1295022303 conn(0x7f910c074240 msgr2=0x7f910c074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:17.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.850+0000 7f9113c00640 1 -- 172.21.15.66:0/1295022303 shutdown_connections 2024-09-10T22:15:17.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.850+0000 7f9113c00640 1 -- 172.21.15.66:0/1295022303 wait complete. 2024-09-10T22:15:17.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9113c00640 1 Processor -- start 2024-09-10T22:15:17.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9113c00640 1 -- start start 2024-09-10T22:15:17.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9113c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f910c0ae4a0 0x7f910c097c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:17.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9113c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f910c0d0360 0x7f910c099350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:17.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9113c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f910c15def0 0x7f910c099890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9113c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f910c16aeb0 con 0x7f910c15def0 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9113c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f910c16ad30 con 0x7f910c0ae4a0 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9113c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f910c16b030 con 0x7f910c0d0360 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9111c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f910c15def0 0x7f910c099890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9111200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f910c0ae4a0 0x7f910c097c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9111c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f910c15def0 0x7f910c099890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46172/0 (socket says 172.21.15.66:46172) 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9111c00640 1 -- 172.21.15.66:0/4240071211 learned_addr learned my addr 172.21.15.66:0/4240071211 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f910be00640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f910c0d0360 0x7f910c099350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9111c00640 1 -- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f910c0d0360 msgr2=0x7f910c099350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9111c00640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f910c0d0360 0x7f910c099350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9111c00640 1 -- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f910c0ae4a0 msgr2=0x7f910c097c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9111c00640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f910c0ae4a0 0x7f910c097c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9111c00640 1 -- 172.21.15.66:0/4240071211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c1f8210 con 0x7f910c15def0 2024-09-10T22:15:17.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9111c00640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f910c15def0 0x7f910c099890 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f910400efd0 tx=0x7f910400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:17.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9108c00640 1 -- 172.21.15.66:0/4240071211 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104003b80 con 0x7f910c15def0 2024-09-10T22:15:17.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9108c00640 1 -- 172.21.15.66:0/4240071211 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9104003d20 con 0x7f910c15def0 2024-09-10T22:15:17.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9108c00640 1 -- 172.21.15.66:0/4240071211 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104004280 con 0x7f910c15def0 2024-09-10T22:15:17.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9113c00640 1 -- 172.21.15.66:0/4240071211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910c098e70 con 0x7f910c15def0 2024-09-10T22:15:17.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.854+0000 7f9113c00640 1 -- 172.21.15.66:0/4240071211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910c0cdb60 con 0x7f910c15def0 2024-09-10T22:15:17.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.858+0000 7f9113c00640 1 -- 172.21.15.66:0/4240071211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910c1f7f90 con 0x7f910c15def0 2024-09-10T22:15:17.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.858+0000 7f9108c00640 1 -- 172.21.15.66:0/4240071211 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f9104004420 con 0x7f910c15def0 2024-09-10T22:15:17.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.858+0000 7f9108c00640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f90ec074b90 0x7f90ec077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:17.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.858+0000 7f9108c00640 1 -- 172.21.15.66:0/4240071211 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f9104093290 con 0x7f910c15def0 2024-09-10T22:15:17.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.858+0000 7f9111200640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f90ec074b90 0x7f90ec077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:17.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.858+0000 7f9111200640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f90ec074b90 0x7f90ec077050 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f91000af000 tx=0x7f910001b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:17.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:17.862+0000 7f9108c00640 1 -- 172.21.15.66:0/4240071211 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f910c1f7f90 con 0x7f910c15def0 2024-09-10T22:15:18.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.062+0000 7f9113c00640 1 -- 172.21.15.66:0/4240071211 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f910c0d07d0 con 0x7f90ec074b90 2024-09-10T22:15:18.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.070+0000 7f9108c00640 1 -- 172.21.15.66:0/4240071211 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7f910c0d07d0 con 0x7f90ec074b90 2024-09-10T22:15:18.076 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:18.076 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sv1" 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone98955" 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone99264" 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone36130" 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:15:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone55126" 2024-09-10T22:15:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:15:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone71863" 2024-09-10T22:15:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:15:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone53292" 2024-09-10T22:15:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:15:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:15:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subvol_clone99941" 2024-09-10T22:15:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:18.078 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:15:18.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.074+0000 7f9113c00640 1 -- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f90ec074b90 msgr2=0x7f90ec077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:18.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.074+0000 7f9113c00640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f90ec074b90 0x7f90ec077050 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f91000af000 tx=0x7f910001b060 comp rx=0 tx=0).stop 2024-09-10T22:15:18.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.074+0000 7f9113c00640 1 -- 172.21.15.66:0/4240071211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f910c15def0 msgr2=0x7f910c099890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:18.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.074+0000 7f9113c00640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f910c15def0 0x7f910c099890 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f910400efd0 tx=0x7f910400c6a0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.074+0000 7f9113c00640 1 -- 172.21.15.66:0/4240071211 shutdown_connections 2024-09-10T22:15:18.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.074+0000 7f9113c00640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f90ec074b90 0x7f90ec077050 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.074+0000 7f9113c00640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f910c15def0 0x7f910c099890 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.074+0000 7f9113c00640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f910c0d0360 0x7f910c099350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.074+0000 7f9113c00640 1 --2- 172.21.15.66:0/4240071211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f910c0ae4a0 0x7f910c097c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.074+0000 7f9113c00640 1 -- 172.21.15.66:0/4240071211 >> 172.21.15.66:0/4240071211 conn(0x7f910c074240 msgr2=0x7f910c157150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:18.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.074+0000 7f9113c00640 1 -- 172.21.15.66:0/4240071211 shutdown_connections 2024-09-10T22:15:18.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.074+0000 7f9113c00640 1 -- 172.21.15.66:0/4240071211 wait complete. 2024-09-10T22:15:18.099 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-10T22:15:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.242+0000 7fe29ae00640 1 -- 172.21.15.66:0/455790911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2941605c0 msgr2=0x7fe2940b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.242+0000 7fe29ae00640 1 --2- 172.21.15.66:0/455790911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2941605c0 0x7fe2940b1620 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fe284009900 tx=0x7fe28401c8e0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 -- 172.21.15.66:0/455790911 shutdown_connections 2024-09-10T22:15:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 --2- 172.21.15.66:0/455790911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2940b1b60 0x7fe2941663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 --2- 172.21.15.66:0/455790911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2941605c0 0x7fe2940b1620 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 -- 172.21.15.66:0/455790911 >> 172.21.15.66:0/455790911 conn(0x7fe2940d1ce0 msgr2=0x7fe2940d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 -- 172.21.15.66:0/455790911 shutdown_connections 2024-09-10T22:15:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 -- 172.21.15.66:0/455790911 wait complete. 2024-09-10T22:15:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 Processor -- start 2024-09-10T22:15:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 -- start start 2024-09-10T22:15:18.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2940b1b60 0x7fe29415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:18.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2940b2b10 0x7fe2941f90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2941605c0 0x7fe294156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe2940d0a80 con 0x7fe2940b2b10 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2940b1400 con 0x7fe2941605c0 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe29ae00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe294169110 con 0x7fe2940b1b60 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe28be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2940b2b10 0x7fe2941f90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe28be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2940b2b10 0x7fe2941f90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46184/0 (socket says 172.21.15.66:46184) 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe28be00640 1 -- 172.21.15.66:0/771536803 learned_addr learned my addr 172.21.15.66:0/771536803 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe293e00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2940b1b60 0x7fe29415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe298e00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2941605c0 0x7fe294156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe28be00640 1 -- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2940b1b60 msgr2=0x7fe29415cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe28be00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2940b1b60 0x7fe29415cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe28be00640 1 -- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2941605c0 msgr2=0x7fe294156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe28be00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2941605c0 0x7fe294156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe28be00640 1 -- 172.21.15.66:0/771536803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2940c8180 con 0x7fe2940b2b10 2024-09-10T22:15:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe28be00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2940b2b10 0x7fe2941f90e0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00c450 tx=0x7fe27c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.246+0000 7fe290c00640 1 -- 172.21.15.66:0/771536803 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c012830 con 0x7fe2940b2b10 2024-09-10T22:15:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.250+0000 7fe290c00640 1 -- 172.21.15.66:0/771536803 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe27c017070 con 0x7fe2940b2b10 2024-09-10T22:15:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.250+0000 7fe290c00640 1 -- 172.21.15.66:0/771536803 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c012d80 con 0x7fe2940b2b10 2024-09-10T22:15:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.250+0000 7fe29ae00640 1 -- 172.21.15.66:0/771536803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2940c5d90 con 0x7fe2940b2b10 2024-09-10T22:15:18.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.250+0000 7fe29ae00640 1 -- 172.21.15.66:0/771536803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2940c8420 con 0x7fe2940b2b10 2024-09-10T22:15:18.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.250+0000 7fe29ae00640 1 -- 172.21.15.66:0/771536803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2940b1620 con 0x7fe2940b2b10 2024-09-10T22:15:18.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.250+0000 7fe290c00640 1 -- 172.21.15.66:0/771536803 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7fe27c01c460 con 0x7fe2940b2b10 2024-09-10T22:15:18.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.250+0000 7fe290c00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe270074b90 0x7fe270077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:18.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.250+0000 7fe290c00640 1 -- 172.21.15.66:0/771536803 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7fe27c092270 con 0x7fe2940b2b10 2024-09-10T22:15:18.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.250+0000 7fe293e00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe270074b90 0x7fe270077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:18.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.254+0000 7fe293e00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe270074b90 0x7fe270077050 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe2840af000 tx=0x7fe28401b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:18.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.254+0000 7fe290c00640 1 -- 172.21.15.66:0/771536803 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe27c05ed60 con 0x7fe2940b2b10 2024-09-10T22:15:18.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.458+0000 7fe29ae00640 1 -- 172.21.15.66:0/771536803 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe2940d00a0 con 0x7fe270074b90 2024-09-10T22:15:18.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.470+0000 7fe290c00640 1 -- 172.21.15.66:0/771536803 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe2940d00a0 con 0x7fe270074b90 2024-09-10T22:15:18.477 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:18.478 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:15:18.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:15:18.478 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ss1" 2024-09-10T22:15:18.478 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:15:18.478 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:15:18.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.474+0000 7fe29ae00640 1 -- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe270074b90 msgr2=0x7fe270077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:18.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.474+0000 7fe29ae00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe270074b90 0x7fe270077050 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe2840af000 tx=0x7fe28401b120 comp rx=0 tx=0).stop 2024-09-10T22:15:18.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.474+0000 7fe29ae00640 1 -- 172.21.15.66:0/771536803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2940b2b10 msgr2=0x7fe2941f90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:18.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.474+0000 7fe29ae00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2940b2b10 0x7fe2941f90e0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00c450 tx=0x7fe27c00c910 comp rx=0 tx=0).stop 2024-09-10T22:15:18.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.474+0000 7fe29ae00640 1 -- 172.21.15.66:0/771536803 shutdown_connections 2024-09-10T22:15:18.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.474+0000 7fe29ae00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe270074b90 0x7fe270077050 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.474+0000 7fe29ae00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2941605c0 0x7fe294156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.474+0000 7fe29ae00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2940b2b10 0x7fe2941f90e0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.474+0000 7fe29ae00640 1 --2- 172.21.15.66:0/771536803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2940b1b60 0x7fe29415cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.474+0000 7fe29ae00640 1 -- 172.21.15.66:0/771536803 >> 172.21.15.66:0/771536803 conn(0x7fe2940d1ce0 msgr2=0x7fe2940b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:18.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.474+0000 7fe29ae00640 1 -- 172.21.15.66:0/771536803 shutdown_connections 2024-09-10T22:15:18.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.474+0000 7fe29ae00640 1 -- 172.21.15.66:0/771536803 wait complete. 2024-09-10T22:15:18.500 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-10T22:15:18.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.646+0000 7f7788e00640 1 -- 172.21.15.66:0/1249519900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778415f340 msgr2=0x7f77840cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:18.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.646+0000 7f7788e00640 1 --2- 172.21.15.66:0/1249519900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778415f340 0x7f77840cfb50 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f7774009930 tx=0x7f777401c900 comp rx=0 tx=0).stop 2024-09-10T22:15:18.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.646+0000 7f7788e00640 1 -- 172.21.15.66:0/1249519900 shutdown_connections 2024-09-10T22:15:18.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.646+0000 7f7788e00640 1 --2- 172.21.15.66:0/1249519900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f77840af970 0x7f77840d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.646+0000 7f7788e00640 1 --2- 172.21.15.66:0/1249519900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778415f340 0x7f77840cfb50 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.646+0000 7f7788e00640 1 -- 172.21.15.66:0/1249519900 >> 172.21.15.66:0/1249519900 conn(0x7f77840d1c50 msgr2=0x7f77840d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:18.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.646+0000 7f7788e00640 1 -- 172.21.15.66:0/1249519900 shutdown_connections 2024-09-10T22:15:18.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.646+0000 7f7788e00640 1 -- 172.21.15.66:0/1249519900 wait complete. 2024-09-10T22:15:18.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.646+0000 7f7788e00640 1 Processor -- start 2024-09-10T22:15:18.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7788e00640 1 -- start start 2024-09-10T22:15:18.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7788e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f77840af970 0x7f77840c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:18.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7788e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77840b2bf0 0x7f7784154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:18.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7788e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f778415f340 0x7f7784152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:18.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7788e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7784169190 con 0x7f77840b2bf0 2024-09-10T22:15:18.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7788e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7784169010 con 0x7f778415f340 2024-09-10T22:15:18.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7788e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7784169310 con 0x7f77840af970 2024-09-10T22:15:18.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7779400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77840b2bf0 0x7f7784154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:18.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7779400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77840b2bf0 0x7f7784154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46204/0 (socket says 172.21.15.66:46204) 2024-09-10T22:15:18.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7779400640 1 -- 172.21.15.66:0/1464074264 learned_addr learned my addr 172.21.15.66:0/1464074264 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:18.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7782800640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f778415f340 0x7f7784152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:18.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7781e00640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f77840af970 0x7f77840c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:18.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7779400640 1 -- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f77840af970 msgr2=0x7f77840c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:18.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7779400640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f77840af970 0x7f77840c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7779400640 1 -- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f778415f340 msgr2=0x7f7784152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:18.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7779400640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f778415f340 0x7f7784152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7779400640 1 -- 172.21.15.66:0/1464074264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778415ad90 con 0x7f77840b2bf0 2024-09-10T22:15:18.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7782800640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f778415f340 0x7f7784152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:18.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7781e00640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f77840af970 0x7f77840c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:18.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7779400640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77840b2bf0 0x7f7784154c20 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c500 tx=0x7f776c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:18.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f777aa00640 1 -- 172.21.15.66:0/1464074264 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c017070 con 0x7f77840b2bf0 2024-09-10T22:15:18.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f777aa00640 1 -- 172.21.15.66:0/1464074264 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f776c0053c0 con 0x7f77840b2bf0 2024-09-10T22:15:18.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f777aa00640 1 -- 172.21.15.66:0/1464074264 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c012890 con 0x7f77840b2bf0 2024-09-10T22:15:18.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7788e00640 1 -- 172.21.15.66:0/1464074264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778415b070 con 0x7f77840b2bf0 2024-09-10T22:15:18.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7788e00640 1 -- 172.21.15.66:0/1464074264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7784154130 con 0x7f77840b2bf0 2024-09-10T22:15:18.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.650+0000 7f7788e00640 1 -- 172.21.15.66:0/1464074264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f778415b460 con 0x7f77840b2bf0 2024-09-10T22:15:18.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.654+0000 7f777aa00640 1 -- 172.21.15.66:0/1464074264 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f776c003600 con 0x7f77840b2bf0 2024-09-10T22:15:18.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.654+0000 7f777aa00640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7760074b90 0x7f7760077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:18.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.654+0000 7f777aa00640 1 -- 172.21.15.66:0/1464074264 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f776c091b20 con 0x7f77840b2bf0 2024-09-10T22:15:18.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.654+0000 7f7781e00640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7760074b90 0x7f7760077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:18.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.654+0000 7f7781e00640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7760074b90 0x7f7760077050 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f77740af000 tx=0x7f777401b140 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:18.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.658+0000 7f777aa00640 1 -- 172.21.15.66:0/1464074264 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f776c05ec80 con 0x7f77840b2bf0 2024-09-10T22:15:18.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.862+0000 7f7788e00640 1 -- 172.21.15.66:0/1464074264 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f778415b240 con 0x7f7760074b90 2024-09-10T22:15:18.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.890+0000 7f777aa00640 1 -- 172.21.15.66:0/1464074264 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f778415b240 con 0x7f7760074b90 2024-09-10T22:15:18.896 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:18.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.894+0000 7f7788e00640 1 -- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7760074b90 msgr2=0x7f7760077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:18.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.894+0000 7f7788e00640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7760074b90 0x7f7760077050 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f77740af000 tx=0x7f777401b140 comp rx=0 tx=0).stop 2024-09-10T22:15:18.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.894+0000 7f7788e00640 1 -- 172.21.15.66:0/1464074264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77840b2bf0 msgr2=0x7f7784154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:18.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.894+0000 7f7788e00640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77840b2bf0 0x7f7784154c20 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c500 tx=0x7f776c00c9c0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.894+0000 7f7788e00640 1 -- 172.21.15.66:0/1464074264 shutdown_connections 2024-09-10T22:15:18.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.894+0000 7f7788e00640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f7760074b90 0x7f7760077050 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.894+0000 7f7788e00640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f778415f340 0x7f7784152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.894+0000 7f7788e00640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77840b2bf0 0x7f7784154c20 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.894+0000 7f7788e00640 1 --2- 172.21.15.66:0/1464074264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f77840af970 0x7f77840c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:18.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.894+0000 7f7788e00640 1 -- 172.21.15.66:0/1464074264 >> 172.21.15.66:0/1464074264 conn(0x7f77840d1c50 msgr2=0x7f7784072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:18.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.894+0000 7f7788e00640 1 -- 172.21.15.66:0/1464074264 shutdown_connections 2024-09-10T22:15:18.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:18.894+0000 7f7788e00640 1 -- 172.21.15.66:0/1464074264 wait complete. 2024-09-10T22:15:18.917 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-10T22:15:19.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.062+0000 7f2c6e000640 1 -- 172.21.15.66:0/4251992464 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c680b2bf0 msgr2=0x7f2c680d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.062+0000 7f2c6e000640 1 --2- 172.21.15.66:0/4251992464 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c680b2bf0 0x7f2c680d0090 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009900 tx=0x7f2c5401c850 comp rx=0 tx=0).stop 2024-09-10T22:15:19.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.062+0000 7f2c6e000640 1 -- 172.21.15.66:0/4251992464 shutdown_connections 2024-09-10T22:15:19.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.062+0000 7f2c6e000640 1 --2- 172.21.15.66:0/4251992464 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c680b2bf0 0x7f2c680d0090 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.062+0000 7f2c6e000640 1 --2- 172.21.15.66:0/4251992464 >> v2:172.21.15.94:3300/0 conn(0x7f2c6815f340 0x7f2c680cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.062+0000 7f2c6e000640 1 -- 172.21.15.66:0/4251992464 >> 172.21.15.66:0/4251992464 conn(0x7f2c680d1c50 msgr2=0x7f2c680d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:19.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.062+0000 7f2c6e000640 1 -- 172.21.15.66:0/4251992464 shutdown_connections 2024-09-10T22:15:19.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.062+0000 7f2c6e000640 1 -- 172.21.15.66:0/4251992464 wait complete. 2024-09-10T22:15:19.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.062+0000 7f2c6e000640 1 Processor -- start 2024-09-10T22:15:19.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c6e000640 1 -- start start 2024-09-10T22:15:19.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c6e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c680b2bf0 0x7f2c6815b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:19.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c6e000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c680d2770 0x7f2c680cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:19.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c6e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c6815f340 0x7f2c6815e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:19.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c6e000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68169190 con 0x7f2c6815f340 2024-09-10T22:15:19.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c6e000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68169010 con 0x7f2c680b2bf0 2024-09-10T22:15:19.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c6e000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2c68169310 con 0x7f2c680d2770 2024-09-10T22:15:19.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c67e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c6815f340 0x7f2c6815e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:19.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c67e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c6815f340 0x7f2c6815e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46208/0 (socket says 172.21.15.66:46208) 2024-09-10T22:15:19.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c67e00640 1 -- 172.21.15.66:0/3049243550 learned_addr learned my addr 172.21.15.66:0/3049243550 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:19.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c66a00640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c680d2770 0x7f2c680cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:19.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c67400640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c680b2bf0 0x7f2c6815b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:19.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c67e00640 1 -- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c680d2770 msgr2=0x7f2c680cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c67e00640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c680d2770 0x7f2c680cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c67e00640 1 -- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c680b2bf0 msgr2=0x7f2c6815b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c67e00640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c680b2bf0 0x7f2c6815b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c67e00640 1 -- 172.21.15.66:0/3049243550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c68148d70 con 0x7f2c6815f340 2024-09-10T22:15:19.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c67e00640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c6815f340 0x7f2c6815e130 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000c7c0 tx=0x7f2c6000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:19.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c5f400640 1 -- 172.21.15.66:0/3049243550 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60003b80 con 0x7f2c6815f340 2024-09-10T22:15:19.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c5f400640 1 -- 172.21.15.66:0/3049243550 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c60003d20 con 0x7f2c6815f340 2024-09-10T22:15:19.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c6e000640 1 -- 172.21.15.66:0/3049243550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c68149050 con 0x7f2c6815f340 2024-09-10T22:15:19.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c5f400640 1 -- 172.21.15.66:0/3049243550 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000aab0 con 0x7f2c6815f340 2024-09-10T22:15:19.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c6e000640 1 -- 172.21.15.66:0/3049243550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c68149610 con 0x7f2c6815f340 2024-09-10T22:15:19.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.066+0000 7f2c6e000640 1 -- 172.21.15.66:0/3049243550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c68149810 con 0x7f2c6815f340 2024-09-10T22:15:19.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.070+0000 7f2c5f400640 1 -- 172.21.15.66:0/3049243550 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f2c6000ac50 con 0x7f2c6815f340 2024-09-10T22:15:19.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.070+0000 7f2c5f400640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c48074b90 0x7f2c48077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:19.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.070+0000 7f2c5f400640 1 -- 172.21.15.66:0/3049243550 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f2c600921e0 con 0x7f2c6815f340 2024-09-10T22:15:19.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.070+0000 7f2c67400640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c48074b90 0x7f2c48077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:19.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.070+0000 7f2c67400640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c48074b90 0x7f2c48077050 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009b40 tx=0x7f2c580091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:19.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.074+0000 7f2c5f400640 1 -- 172.21.15.66:0/3049243550 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2c6005ed70 con 0x7f2c6815f340 2024-09-10T22:15:19.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.278+0000 7f2c6e000640 1 -- 172.21.15.66:0/3049243550 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f2c68146910 con 0x7f2c48074b90 2024-09-10T22:15:19.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.302+0000 7f2c5f400640 1 -- 172.21.15.66:0/3049243550 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2c68146910 con 0x7f2c48074b90 2024-09-10T22:15:19.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.306+0000 7f2c6e000640 1 -- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c48074b90 msgr2=0x7f2c48077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.306+0000 7f2c6e000640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c48074b90 0x7f2c48077050 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009b40 tx=0x7f2c580091e0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.306+0000 7f2c6e000640 1 -- 172.21.15.66:0/3049243550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c6815f340 msgr2=0x7f2c6815e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.306+0000 7f2c6e000640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c6815f340 0x7f2c6815e130 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000c7c0 tx=0x7f2c6000cc80 comp rx=0 tx=0).stop 2024-09-10T22:15:19.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.306+0000 7f2c6e000640 1 -- 172.21.15.66:0/3049243550 shutdown_connections 2024-09-10T22:15:19.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.306+0000 7f2c6e000640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f2c48074b90 0x7f2c48077050 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.306+0000 7f2c6e000640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c6815f340 0x7f2c6815e130 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.306+0000 7f2c6e000640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c680d2770 0x7f2c680cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.306+0000 7f2c6e000640 1 --2- 172.21.15.66:0/3049243550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c680b2bf0 0x7f2c6815b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.306+0000 7f2c6e000640 1 -- 172.21.15.66:0/3049243550 >> 172.21.15.66:0/3049243550 conn(0x7f2c680d1c50 msgr2=0x7f2c681663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:19.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.306+0000 7f2c6e000640 1 -- 172.21.15.66:0/3049243550 shutdown_connections 2024-09-10T22:15:19.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.306+0000 7f2c6e000640 1 -- 172.21.15.66:0/3049243550 wait complete. 2024-09-10T22:15:19.323 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:15:19.322+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:15:19.323 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:15:19.322+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:15:19.323 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:15:19.322+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:15:19.323 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:15:19.322+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:15:19.324 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:15:19.322+0000 7f5feb000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:15:19.336 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone98955 2024-09-10T22:15:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 -- 172.21.15.66:0/1062903903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919c0cdf90 msgr2=0x7f919c0cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 --2- 172.21.15.66:0/1062903903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919c0cdf90 0x7f919c0cf830 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f919c0b3700 tx=0x7f918401c8c0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 -- 172.21.15.66:0/1062903903 shutdown_connections 2024-09-10T22:15:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 --2- 172.21.15.66:0/1062903903 >> v2:172.21.15.94:3300/0 conn(0x7f919c0cf1f0 0x7f919c0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 --2- 172.21.15.66:0/1062903903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919c0cdf90 0x7f919c0cf830 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 --2- 172.21.15.66:0/1062903903 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f919c0ce8d0 0x7f919c0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 -- 172.21.15.66:0/1062903903 >> 172.21.15.66:0/1062903903 conn(0x7f919c0d1c60 msgr2=0x7f919c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 -- 172.21.15.66:0/1062903903 shutdown_connections 2024-09-10T22:15:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 -- 172.21.15.66:0/1062903903 wait complete. 2024-09-10T22:15:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 Processor -- start 2024-09-10T22:15:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 -- start start 2024-09-10T22:15:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f919c0cdf90 0x7f919c151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919c0ce8d0 0x7f919c15eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919c0cf1f0 0x7f919c151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f919c0b1850 con 0x7f919c0ce8d0 2024-09-10T22:15:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f919c0b16d0 con 0x7f919c0cf1f0 2024-09-10T22:15:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.486+0000 7f91a1200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f919c0b19d0 con 0x7f919c0cdf90 2024-09-10T22:15:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9193e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919c0ce8d0 0x7f919c15eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9199e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f919c0cdf90 0x7f919c151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9199e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f919c0cdf90 0x7f919c151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:34328/0 (socket says 172.21.15.66:34328) 2024-09-10T22:15:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9193e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919c0ce8d0 0x7f919c15eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46222/0 (socket says 172.21.15.66:46222) 2024-09-10T22:15:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9199e00640 1 -- 172.21.15.66:0/1240091712 learned_addr learned my addr 172.21.15.66:0/1240091712 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f919a800640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919c0cf1f0 0x7f919c151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9193e00640 1 -- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f919c0cdf90 msgr2=0x7f919c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9193e00640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f919c0cdf90 0x7f919c151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9193e00640 1 -- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919c0cf1f0 msgr2=0x7f919c151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9193e00640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919c0cf1f0 0x7f919c151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9193e00640 1 -- 172.21.15.66:0/1240091712 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c06fd40 con 0x7f919c0ce8d0 2024-09-10T22:15:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9193e00640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919c0ce8d0 0x7f919c15eca0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f91840ac000 tx=0x7f9184004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9192000640 1 -- 172.21.15.66:0/1240091712 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91840b2040 con 0x7f919c0ce8d0 2024-09-10T22:15:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9192000640 1 -- 172.21.15.66:0/1240091712 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9184003600 con 0x7f919c0ce8d0 2024-09-10T22:15:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9192000640 1 -- 172.21.15.66:0/1240091712 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184003a50 con 0x7f919c0ce8d0 2024-09-10T22:15:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f91a1200640 1 -- 172.21.15.66:0/1240091712 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919c156580 con 0x7f919c0ce8d0 2024-09-10T22:15:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f91a1200640 1 -- 172.21.15.66:0/1240091712 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919c159b70 con 0x7f919c0ce8d0 2024-09-10T22:15:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f91a1200640 1 -- 172.21.15.66:0/1240091712 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f919c156a60 con 0x7f919c0ce8d0 2024-09-10T22:15:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.490+0000 7f9192000640 1 -- 172.21.15.66:0/1240091712 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f9184003bf0 con 0x7f919c0ce8d0 2024-09-10T22:15:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.494+0000 7f9192000640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9178074b90 0x7f9178077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.494+0000 7f9192000640 1 -- 172.21.15.66:0/1240091712 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f918412bde0 con 0x7f919c0ce8d0 2024-09-10T22:15:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.494+0000 7f9199e00640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9178074b90 0x7f9178077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.494+0000 7f9199e00640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9178074b90 0x7f9178077050 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f919c1fb310 tx=0x7f918c00a380 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.498+0000 7f9192000640 1 -- 172.21.15.66:0/1240091712 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f919c156a60 con 0x7f919c0ce8d0 2024-09-10T22:15:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.698+0000 7f91a1200640 1 -- 172.21.15.66:0/1240091712 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone98955", "target": ["mon-mgr", ""]}) -- 0x7f919c0cecb0 con 0x7f9178074b90 2024-09-10T22:15:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.714+0000 7f9192000640 1 -- 172.21.15.66:0/1240091712 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f919c0cecb0 con 0x7f9178074b90 2024-09-10T22:15:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.718+0000 7f91a1200640 1 -- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9178074b90 msgr2=0x7f9178077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.718+0000 7f91a1200640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9178074b90 0x7f9178077050 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f919c1fb310 tx=0x7f918c00a380 comp rx=0 tx=0).stop 2024-09-10T22:15:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.718+0000 7f91a1200640 1 -- 172.21.15.66:0/1240091712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919c0ce8d0 msgr2=0x7f919c15eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.718+0000 7f91a1200640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919c0ce8d0 0x7f919c15eca0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f91840ac000 tx=0x7f9184004040 comp rx=0 tx=0).stop 2024-09-10T22:15:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.718+0000 7f91a1200640 1 -- 172.21.15.66:0/1240091712 shutdown_connections 2024-09-10T22:15:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.718+0000 7f91a1200640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9178074b90 0x7f9178077050 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.718+0000 7f91a1200640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919c0cf1f0 0x7f919c151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.718+0000 7f91a1200640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919c0ce8d0 0x7f919c15eca0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.718+0000 7f91a1200640 1 --2- 172.21.15.66:0/1240091712 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f919c0cdf90 0x7f919c151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.718+0000 7f91a1200640 1 -- 172.21.15.66:0/1240091712 >> 172.21.15.66:0/1240091712 conn(0x7f919c0d1c60 msgr2=0x7f919c06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.718+0000 7f91a1200640 1 -- 172.21.15.66:0/1240091712 shutdown_connections 2024-09-10T22:15:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.718+0000 7f91a1200640 1 -- 172.21.15.66:0/1240091712 wait complete. 2024-09-10T22:15:19.748 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99264 2024-09-10T22:15:19.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.894+0000 7fd565400640 1 -- 172.21.15.66:0/2934128174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd56015b730 msgr2=0x7fd56015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.894+0000 7fd565400640 1 --2- 172.21.15.66:0/2934128174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd56015b730 0x7fd56015bb60 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fd560074af0 tx=0x7fd55801f580 comp rx=0 tx=0).stop 2024-09-10T22:15:19.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.894+0000 7fd565400640 1 -- 172.21.15.66:0/2934128174 shutdown_connections 2024-09-10T22:15:19.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.894+0000 7fd565400640 1 --2- 172.21.15.66:0/2934128174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd56015b730 0x7fd56015bb60 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.894+0000 7fd565400640 1 --2- 172.21.15.66:0/2934128174 >> v2:172.21.15.94:3300/0 conn(0x7fd5600cee30 0x7fd5600cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.894+0000 7fd565400640 1 -- 172.21.15.66:0/2934128174 >> 172.21.15.66:0/2934128174 conn(0x7fd5600cfaa0 msgr2=0x7fd5600cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:19.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.894+0000 7fd565400640 1 -- 172.21.15.66:0/2934128174 shutdown_connections 2024-09-10T22:15:19.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.894+0000 7fd565400640 1 -- 172.21.15.66:0/2934128174 wait complete. 2024-09-10T22:15:19.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.894+0000 7fd565400640 1 Processor -- start 2024-09-10T22:15:19.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.894+0000 7fd565400640 1 -- start start 2024-09-10T22:15:19.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.894+0000 7fd565400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5600cee30 0x7fd5601f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:19.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.894+0000 7fd565400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd56015b730 0x7fd560159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd565400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd56015ddd0 0x7fd5601f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd565400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd56015c220 con 0x7fd5600cee30 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd565400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd56015c0a0 con 0x7fd56015ddd0 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd565400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd56015c3a0 con 0x7fd56015b730 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd55e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5600cee30 0x7fd5601f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd55e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5600cee30 0x7fd5601f3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46254/0 (socket says 172.21.15.66:46254) 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd55e800640 1 -- 172.21.15.66:0/4268246727 learned_addr learned my addr 172.21.15.66:0/4268246727 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd557e00640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd56015b730 0x7fd560159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd55f200640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd56015ddd0 0x7fd5601f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd55e800640 1 -- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd56015b730 msgr2=0x7fd560159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd55e800640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd56015b730 0x7fd560159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd55e800640 1 -- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd56015ddd0 msgr2=0x7fd5601f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd55e800640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd56015ddd0 0x7fd5601f3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:19.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd55e800640 1 -- 172.21.15.66:0/4268246727 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56006c6a0 con 0x7fd5600cee30 2024-09-10T22:15:19.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd55f200640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd56015ddd0 0x7fd5601f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:19.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd55e800640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5600cee30 0x7fd5601f3310 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fd55000b550 tx=0x7fd55000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:19.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd556a00640 1 -- 172.21.15.66:0/4268246727 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550014070 con 0x7fd5600cee30 2024-09-10T22:15:19.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd556a00640 1 -- 172.21.15.66:0/4268246727 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd550010980 con 0x7fd5600cee30 2024-09-10T22:15:19.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd556a00640 1 -- 172.21.15.66:0/4268246727 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550010dd0 con 0x7fd5600cee30 2024-09-10T22:15:19.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd565400640 1 -- 172.21.15.66:0/4268246727 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd560155160 con 0x7fd5600cee30 2024-09-10T22:15:19.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd565400640 1 -- 172.21.15.66:0/4268246727 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd560155720 con 0x7fd5600cee30 2024-09-10T22:15:19.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd565400640 1 -- 172.21.15.66:0/4268246727 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd56015c850 con 0x7fd5600cee30 2024-09-10T22:15:19.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.898+0000 7fd556a00640 1 -- 172.21.15.66:0/4268246727 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7fd55000f3c0 con 0x7fd5600cee30 2024-09-10T22:15:19.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.902+0000 7fd556a00640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd53c074bc0 0x7fd53c077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.902+0000 7fd556a00640 1 -- 172.21.15.66:0/4268246727 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7fd5500924b0 con 0x7fd5600cee30 2024-09-10T22:15:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.902+0000 7fd557e00640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd53c074bc0 0x7fd53c077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.902+0000 7fd557e00640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd53c074bc0 0x7fd53c077080 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd5600cbc50 tx=0x7fd548009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:19.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:19.906+0000 7fd556a00640 1 -- 172.21.15.66:0/4268246727 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd55005f040 con 0x7fd5600cee30 2024-09-10T22:15:20.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.106+0000 7fd565400640 1 -- 172.21.15.66:0/4268246727 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99264", "target": ["mon-mgr", ""]}) -- 0x7fd5600d07c0 con 0x7fd53c074bc0 2024-09-10T22:15:20.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.126+0000 7fd556a00640 1 -- 172.21.15.66:0/4268246727 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd5600d07c0 con 0x7fd53c074bc0 2024-09-10T22:15:20.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.126+0000 7fd565400640 1 -- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd53c074bc0 msgr2=0x7fd53c077080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:20.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.126+0000 7fd565400640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd53c074bc0 0x7fd53c077080 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd5600cbc50 tx=0x7fd548009040 comp rx=0 tx=0).stop 2024-09-10T22:15:20.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.130+0000 7fd565400640 1 -- 172.21.15.66:0/4268246727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5600cee30 msgr2=0x7fd5601f3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:20.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.130+0000 7fd565400640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5600cee30 0x7fd5601f3310 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fd55000b550 tx=0x7fd55000ba10 comp rx=0 tx=0).stop 2024-09-10T22:15:20.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.130+0000 7fd565400640 1 -- 172.21.15.66:0/4268246727 shutdown_connections 2024-09-10T22:15:20.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.130+0000 7fd565400640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd53c074bc0 0x7fd53c077080 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.130+0000 7fd565400640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd56015ddd0 0x7fd5601f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.130+0000 7fd565400640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd56015b730 0x7fd560159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.130+0000 7fd565400640 1 --2- 172.21.15.66:0/4268246727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5600cee30 0x7fd5601f3310 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.130+0000 7fd565400640 1 -- 172.21.15.66:0/4268246727 >> 172.21.15.66:0/4268246727 conn(0x7fd5600cfaa0 msgr2=0x7fd5600b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:20.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.130+0000 7fd565400640 1 -- 172.21.15.66:0/4268246727 shutdown_connections 2024-09-10T22:15:20.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.130+0000 7fd565400640 1 -- 172.21.15.66:0/4268246727 wait complete. 2024-09-10T22:15:20.155 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone36130 2024-09-10T22:15:20.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 -- 172.21.15.66:0/2577032300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda2c06e6a0 msgr2=0x7fda2c1667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:20.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 --2- 172.21.15.66:0/2577032300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda2c06e6a0 0x7fda2c1667a0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fda24009c80 tx=0x7fda2401cd10 comp rx=0 tx=0).stop 2024-09-10T22:15:20.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 -- 172.21.15.66:0/2577032300 shutdown_connections 2024-09-10T22:15:20.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 --2- 172.21.15.66:0/2577032300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda2c06e6a0 0x7fda2c1667a0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 -- 172.21.15.66:0/2577032300 >> 172.21.15.66:0/2577032300 conn(0x7fda2c0cfaa0 msgr2=0x7fda2c0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:20.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 -- 172.21.15.66:0/2577032300 shutdown_connections 2024-09-10T22:15:20.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 -- 172.21.15.66:0/2577032300 wait complete. 2024-09-10T22:15:20.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 Processor -- start 2024-09-10T22:15:20.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 -- start start 2024-09-10T22:15:20.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda2c06e6a0 0x7fda2c131f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:20.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda2c144ac0 0x7fda2c1355c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda2c145050 0x7fda2c1fade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c169da0 con 0x7fda2c145050 2024-09-10T22:15:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c169c20 con 0x7fda2c144ac0 2024-09-10T22:15:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda31400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fda2c169f20 con 0x7fda2c06e6a0 2024-09-10T22:15:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda2b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda2c145050 0x7fda2c1fade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda2b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda2c145050 0x7fda2c1fade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46274/0 (socket says 172.21.15.66:46274) 2024-09-10T22:15:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.302+0000 7fda2b200640 1 -- 172.21.15.66:0/2861498102 learned_addr learned my addr 172.21.15.66:0/2861498102 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda2b200640 1 -- 172.21.15.66:0/2861498102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda2c06e6a0 msgr2=0x7fda2c131f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda2b200640 1 --2- 172.21.15.66:0/2861498102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda2c06e6a0 0x7fda2c131f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda2b200640 1 -- 172.21.15.66:0/2861498102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda2c144ac0 msgr2=0x7fda2c1355c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda2b200640 1 --2- 172.21.15.66:0/2861498102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda2c144ac0 0x7fda2c1355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda2b200640 1 -- 172.21.15.66:0/2861498102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c138910 con 0x7fda2c145050 2024-09-10T22:15:20.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda2b200640 1 --2- 172.21.15.66:0/2861498102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda2c145050 0x7fda2c1fade0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fda24009bd0 tx=0x7fda24003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:20.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda22a00640 1 -- 172.21.15.66:0/2861498102 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240bb070 con 0x7fda2c145050 2024-09-10T22:15:20.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda22a00640 1 -- 172.21.15.66:0/2861498102 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda24004d40 con 0x7fda2c145050 2024-09-10T22:15:20.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda22a00640 1 -- 172.21.15.66:0/2861498102 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240b2040 con 0x7fda2c145050 2024-09-10T22:15:20.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda31400640 1 -- 172.21.15.66:0/2861498102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c138bf0 con 0x7fda2c145050 2024-09-10T22:15:20.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda31400640 1 -- 172.21.15.66:0/2861498102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c1391b0 con 0x7fda2c145050 2024-09-10T22:15:20.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda31400640 1 -- 172.21.15.66:0/2861498102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda2c0ceb40 con 0x7fda2c145050 2024-09-10T22:15:20.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda22a00640 1 -- 172.21.15.66:0/2861498102 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7fda240b5030 con 0x7fda2c145050 2024-09-10T22:15:20.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda22a00640 1 --2- 172.21.15.66:0/2861498102 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda08074b90 0x7fda08077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:20.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.306+0000 7fda22a00640 1 -- 172.21.15.66:0/2861498102 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7fda2412c320 con 0x7fda2c145050 2024-09-10T22:15:20.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.310+0000 7fda2a800640 1 --2- 172.21.15.66:0/2861498102 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda08074b90 0x7fda08077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:20.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.310+0000 7fda2a800640 1 --2- 172.21.15.66:0/2861498102 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda08074b90 0x7fda08077050 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fda1c000f70 tx=0x7fda1c0079c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:20.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.310+0000 7fda22a00640 1 -- 172.21.15.66:0/2861498102 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fda240c4020 con 0x7fda2c145050 2024-09-10T22:15:20.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.510+0000 7fda31400640 1 -- 172.21.15.66:0/2861498102 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone36130", "target": ["mon-mgr", ""]}) -- 0x7fda2c13f860 con 0x7fda08074b90 2024-09-10T22:15:20.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda22a00640 1 -- 172.21.15.66:0/2861498102 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda2c13f860 con 0x7fda08074b90 2024-09-10T22:15:20.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda31400640 1 -- 172.21.15.66:0/2861498102 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda08074b90 msgr2=0x7fda08077050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:20.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda31400640 1 --2- 172.21.15.66:0/2861498102 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda08074b90 0x7fda08077050 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fda1c000f70 tx=0x7fda1c0079c0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda31400640 1 -- 172.21.15.66:0/2861498102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda2c145050 msgr2=0x7fda2c1fade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:20.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda31400640 1 --2- 172.21.15.66:0/2861498102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda2c145050 0x7fda2c1fade0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fda24009bd0 tx=0x7fda24003f80 comp rx=0 tx=0).stop 2024-09-10T22:15:20.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda31400640 1 -- 172.21.15.66:0/2861498102 shutdown_connections 2024-09-10T22:15:20.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda31400640 1 --2- 172.21.15.66:0/2861498102 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda08074b90 0x7fda08077050 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda31400640 1 --2- 172.21.15.66:0/2861498102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda2c145050 0x7fda2c1fade0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda31400640 1 --2- 172.21.15.66:0/2861498102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda2c144ac0 0x7fda2c1355c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda31400640 1 --2- 172.21.15.66:0/2861498102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda2c06e6a0 0x7fda2c131f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda31400640 1 -- 172.21.15.66:0/2861498102 >> 172.21.15.66:0/2861498102 conn(0x7fda2c0cfaa0 msgr2=0x7fda2c072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:20.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda31400640 1 -- 172.21.15.66:0/2861498102 shutdown_connections 2024-09-10T22:15:20.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.534+0000 7fda31400640 1 -- 172.21.15.66:0/2861498102 wait complete. 2024-09-10T22:15:20.567 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone55126 2024-09-10T22:15:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.718+0000 7f4218000640 1 -- 172.21.15.66:0/1188804428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f8003a60 msgr2=0x7f41f8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.718+0000 7f4218000640 1 --2- 172.21.15.66:0/1188804428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f8003a60 0x7f41f8003e80 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f42040016f0 tx=0x7f420401c5b0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.718+0000 7f4218000640 1 -- 172.21.15.66:0/1188804428 shutdown_connections 2024-09-10T22:15:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.718+0000 7f4218000640 1 --2- 172.21.15.66:0/1188804428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f8003a60 0x7f41f8003e80 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.718+0000 7f4218000640 1 -- 172.21.15.66:0/1188804428 >> 172.21.15.66:0/1188804428 conn(0x7f42100d1c60 msgr2=0x7f42100d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.718+0000 7f4218000640 1 -- 172.21.15.66:0/1188804428 shutdown_connections 2024-09-10T22:15:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.718+0000 7f4218000640 1 -- 172.21.15.66:0/1188804428 wait complete. 2024-09-10T22:15:20.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4218000640 1 Processor -- start 2024-09-10T22:15:20.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4218000640 1 -- start start 2024-09-10T22:15:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4218000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f8003a60 0x7f42100b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4218000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42101f5540 0x7f42100b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4218000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42101f1940 0x7f42101f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4218000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f42100b3060 con 0x7f42101f5540 2024-09-10T22:15:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4218000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42100b31e0 con 0x7f42101f1940 2024-09-10T22:15:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4218000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f42100b3360 con 0x7f41f8003a60 2024-09-10T22:15:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4214c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42101f5540 0x7f42100b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4214c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42101f5540 0x7f42100b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46290/0 (socket says 172.21.15.66:46290) 2024-09-10T22:15:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4214c00640 1 -- 172.21.15.66:0/1195257031 learned_addr learned my addr 172.21.15.66:0/1195257031 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4215600640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f8003a60 0x7f42100b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4216000640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42101f1940 0x7f42101f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4214c00640 1 -- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f8003a60 msgr2=0x7f42100b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4214c00640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f8003a60 0x7f42100b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4214c00640 1 -- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42101f1940 msgr2=0x7f42101f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4214c00640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42101f1940 0x7f42101f1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4214c00640 1 -- 172.21.15.66:0/1195257031 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42101f2300 con 0x7f42101f5540 2024-09-10T22:15:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4216000640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42101f1940 0x7f42101f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4214c00640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42101f5540 0x7f42100b4eb0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f41fc011780 tx=0x7f41fc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f420d600640 1 -- 172.21.15.66:0/1195257031 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc01f070 con 0x7f42101f5540 2024-09-10T22:15:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f420d600640 1 -- 172.21.15.66:0/1195257031 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41fc011e90 con 0x7f42101f5540 2024-09-10T22:15:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f420d600640 1 -- 172.21.15.66:0/1195257031 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0126b0 con 0x7f42101f5540 2024-09-10T22:15:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4218000640 1 -- 172.21.15.66:0/1195257031 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42100b7590 con 0x7f42101f5540 2024-09-10T22:15:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.722+0000 7f4218000640 1 -- 172.21.15.66:0/1195257031 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42100cd530 con 0x7f42101f5540 2024-09-10T22:15:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.726+0000 7f4218000640 1 -- 172.21.15.66:0/1195257031 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421006e570 con 0x7f42101f5540 2024-09-10T22:15:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.726+0000 7f420d600640 1 -- 172.21.15.66:0/1195257031 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 49) ==== 99811+0+0 (secure 0 0 0) 0x7f41fc012850 con 0x7f42101f5540 2024-09-10T22:15:20.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.726+0000 7f420d600640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41f0074bc0 0x7f41f0077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:20.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.726+0000 7f420d600640 1 -- 172.21.15.66:0/1195257031 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5878+0+0 (secure 0 0 0) 0x7f41fc093050 con 0x7f42101f5540 2024-09-10T22:15:20.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.726+0000 7f4215600640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41f0074bc0 0x7f41f0077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:20.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.730+0000 7f4215600640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41f0074bc0 0x7f41f0077080 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f42040016c0 tx=0x7f42040193a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.730+0000 7f420d600640 1 -- 172.21.15.66:0/1195257031 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f42101f2300 con 0x7f42101f5540 2024-09-10T22:15:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.930+0000 7f4218000640 1 -- 172.21.15.66:0/1195257031 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone55126", "target": ["mon-mgr", ""]}) -- 0x7f42100b7a40 con 0x7f41f0074bc0 2024-09-10T22:15:20.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.950+0000 7f420d600640 1 -- 172.21.15.66:0/1195257031 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f42100b7a40 con 0x7f41f0074bc0 2024-09-10T22:15:20.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.954+0000 7f4218000640 1 -- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41f0074bc0 msgr2=0x7f41f0077080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:20.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.954+0000 7f4218000640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41f0074bc0 0x7f41f0077080 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f42040016c0 tx=0x7f42040193a0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.954+0000 7f4218000640 1 -- 172.21.15.66:0/1195257031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42101f5540 msgr2=0x7f42100b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:20.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.954+0000 7f4218000640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42101f5540 0x7f42100b4eb0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f41fc011780 tx=0x7f41fc011c40 comp rx=0 tx=0).stop 2024-09-10T22:15:20.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.954+0000 7f4218000640 1 -- 172.21.15.66:0/1195257031 shutdown_connections 2024-09-10T22:15:20.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.954+0000 7f4218000640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f41f0074bc0 0x7f41f0077080 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.954+0000 7f4218000640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42101f1940 0x7f42101f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.954+0000 7f4218000640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42101f5540 0x7f42100b4eb0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.954+0000 7f4218000640 1 --2- 172.21.15.66:0/1195257031 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41f8003a60 0x7f42100b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:20.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.954+0000 7f4218000640 1 -- 172.21.15.66:0/1195257031 >> 172.21.15.66:0/1195257031 conn(0x7f42100d1c60 msgr2=0x7f421015f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:20.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.954+0000 7f4218000640 1 -- 172.21.15.66:0/1195257031 shutdown_connections 2024-09-10T22:15:20.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:20.954+0000 7f4218000640 1 -- 172.21.15.66:0/1195257031 wait complete. 2024-09-10T22:15:20.988 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone71863 2024-09-10T22:15:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.138+0000 7f77ef800640 1 -- 172.21.15.66:0/2266612089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e80b2450 msgr2=0x7f77e8166a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.138+0000 7f77ef800640 1 --2- 172.21.15.66:0/2266612089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e80b2450 0x7f77e8166a90 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009cb0 tx=0x7f77e001cda0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.138+0000 7f77ef800640 1 -- 172.21.15.66:0/2266612089 shutdown_connections 2024-09-10T22:15:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.138+0000 7f77ef800640 1 --2- 172.21.15.66:0/2266612089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e80b2450 0x7f77e8166a90 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.138+0000 7f77ef800640 1 -- 172.21.15.66:0/2266612089 >> 172.21.15.66:0/2266612089 conn(0x7f77e80d1c50 msgr2=0x7f77e80d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.138+0000 7f77ef800640 1 -- 172.21.15.66:0/2266612089 shutdown_connections 2024-09-10T22:15:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.138+0000 7f77ef800640 1 -- 172.21.15.66:0/2266612089 wait complete. 2024-09-10T22:15:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77ef800640 1 Processor -- start 2024-09-10T22:15:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77ef800640 1 -- start start 2024-09-10T22:15:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77ef800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f77e80b2450 0x7f77e815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77ef800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e80cd020 0x7f77e80b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77ef800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77e815f240 0x7f77e80b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77ef800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8169c30 con 0x7f77e80cd020 2024-09-10T22:15:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77ef800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8169ab0 con 0x7f77e815f240 2024-09-10T22:15:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77ef800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f77e8169db0 con 0x7f77e80b2450 2024-09-10T22:15:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e80cd020 0x7f77e80b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e80cd020 0x7f77e80b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46314/0 (socket says 172.21.15.66:46314) 2024-09-10T22:15:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77e7e00640 1 -- 172.21.15.66:0/3292492651 learned_addr learned my addr 172.21.15.66:0/3292492651 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77ed800640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77e815f240 0x7f77e80b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77ece00640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f77e80b2450 0x7f77e815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77e7e00640 1 -- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f77e80b2450 msgr2=0x7f77e815b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77e7e00640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f77e80b2450 0x7f77e815b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77e7e00640 1 -- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77e815f240 msgr2=0x7f77e80b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77e7e00640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77e815f240 0x7f77e80b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77e7e00640 1 -- 172.21.15.66:0/3292492651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e814f450 con 0x7f77e80cd020 2024-09-10T22:15:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77e7e00640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e80cd020 0x7f77e80b00a0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f77d400c4b0 tx=0x7f77d400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77e4c00640 1 -- 172.21.15.66:0/3292492651 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d40052d0 con 0x7f77e80cd020 2024-09-10T22:15:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77e4c00640 1 -- 172.21.15.66:0/3292492651 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77d4005470 con 0x7f77e80cd020 2024-09-10T22:15:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77e4c00640 1 -- 172.21.15.66:0/3292492651 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4012870 con 0x7f77e80cd020 2024-09-10T22:15:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77ef800640 1 -- 172.21.15.66:0/3292492651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e815a530 con 0x7f77e80cd020 2024-09-10T22:15:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.142+0000 7f77ef800640 1 -- 172.21.15.66:0/3292492651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e815aab0 con 0x7f77e80cd020 2024-09-10T22:15:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.146+0000 7f77ef800640 1 -- 172.21.15.66:0/3292492651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e80cec40 con 0x7f77e80cd020 2024-09-10T22:15:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.146+0000 7f77e4c00640 1 -- 172.21.15.66:0/3292492651 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f77d4003600 con 0x7f77e80cd020 2024-09-10T22:15:21.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.146+0000 7f77e4c00640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f77c8074c20 0x7f77c80770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:21.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.146+0000 7f77e4c00640 1 -- 172.21.15.66:0/3292492651 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5934+0+0 (secure 0 0 0) 0x7f77d4092000 con 0x7f77e80cd020 2024-09-10T22:15:21.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.146+0000 7f77ece00640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f77c8074c20 0x7f77c80770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:21.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.146+0000 7f77ece00640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f77c8074c20 0x7f77c80770e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f77dc007fd0 tx=0x7f77dc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:21.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.150+0000 7f77e4c00640 1 -- 172.21.15.66:0/3292492651 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f77d405eaf0 con 0x7f77e80cd020 2024-09-10T22:15:21.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.358+0000 7f77ef800640 1 -- 172.21.15.66:0/3292492651 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone71863", "target": ["mon-mgr", ""]}) -- 0x7f77e80cc110 con 0x7f77c8074c20 2024-09-10T22:15:21.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.378+0000 7f77e4c00640 1 -- 172.21.15.66:0/3292492651 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f77e80cc110 con 0x7f77c8074c20 2024-09-10T22:15:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.382+0000 7f77ef800640 1 -- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f77c8074c20 msgr2=0x7f77c80770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.382+0000 7f77ef800640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f77c8074c20 0x7f77c80770e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f77dc007fd0 tx=0x7f77dc009040 comp rx=0 tx=0).stop 2024-09-10T22:15:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.382+0000 7f77ef800640 1 -- 172.21.15.66:0/3292492651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e80cd020 msgr2=0x7f77e80b00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.382+0000 7f77ef800640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e80cd020 0x7f77e80b00a0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f77d400c4b0 tx=0x7f77d400c970 comp rx=0 tx=0).stop 2024-09-10T22:15:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.382+0000 7f77ef800640 1 -- 172.21.15.66:0/3292492651 shutdown_connections 2024-09-10T22:15:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.382+0000 7f77ef800640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f77c8074c20 0x7f77c80770e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.382+0000 7f77ef800640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77e815f240 0x7f77e80b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.382+0000 7f77ef800640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e80cd020 0x7f77e80b00a0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.382+0000 7f77ef800640 1 --2- 172.21.15.66:0/3292492651 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f77e80b2450 0x7f77e815b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.382+0000 7f77ef800640 1 -- 172.21.15.66:0/3292492651 >> 172.21.15.66:0/3292492651 conn(0x7f77e80d1c50 msgr2=0x7f77e8157520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:21.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.382+0000 7f77ef800640 1 -- 172.21.15.66:0/3292492651 shutdown_connections 2024-09-10T22:15:21.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.382+0000 7f77ef800640 1 -- 172.21.15.66:0/3292492651 wait complete. 2024-09-10T22:15:21.408 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone53292 2024-09-10T22:15:21.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.554+0000 7fd2f6e00640 1 -- 172.21.15.66:0/2416562234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2f015b9e0 msgr2=0x7fd2f00d1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.554+0000 7fd2f6e00640 1 --2- 172.21.15.66:0/2416562234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2f015b9e0 0x7fd2f00d1940 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00b7be0 tx=0x7fd2e401c8c0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.554+0000 7fd2f6e00640 1 -- 172.21.15.66:0/2416562234 shutdown_connections 2024-09-10T22:15:21.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.554+0000 7fd2f6e00640 1 --2- 172.21.15.66:0/2416562234 >> v2:172.21.15.94:3300/0 conn(0x7fd2f015bdc0 0x7fd2f00d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.554+0000 7fd2f6e00640 1 --2- 172.21.15.66:0/2416562234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2f015b9e0 0x7fd2f00d1940 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.554+0000 7fd2f6e00640 1 -- 172.21.15.66:0/2416562234 >> 172.21.15.66:0/2416562234 conn(0x7fd2f009aa50 msgr2=0x7fd2f009ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:21.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.554+0000 7fd2f6e00640 1 -- 172.21.15.66:0/2416562234 shutdown_connections 2024-09-10T22:15:21.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.554+0000 7fd2f6e00640 1 -- 172.21.15.66:0/2416562234 wait complete. 2024-09-10T22:15:21.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.554+0000 7fd2f6e00640 1 Processor -- start 2024-09-10T22:15:21.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.554+0000 7fd2f6e00640 1 -- start start 2024-09-10T22:15:21.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd2f00b1170 0x7fd2f00ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:21.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2f015b9e0 0x7fd2f01fb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:21.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2f015bdc0 0x7fd2f0070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:21.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01689f0 con 0x7fd2f015bdc0 2024-09-10T22:15:21.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f00b2c90 con 0x7fd2f015b9e0 2024-09-10T22:15:21.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd2f0168b70 con 0x7fd2f00b1170 2024-09-10T22:15:21.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2f015bdc0 0x7fd2f0070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:21.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2f015bdc0 0x7fd2f0070710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46334/0 (socket says 172.21.15.66:46334) 2024-09-10T22:15:21.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6400640 1 -- 172.21.15.66:0/1819873788 learned_addr learned my addr 172.21.15.66:0/1819873788 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:21.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6400640 1 -- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd2f00b1170 msgr2=0x7fd2f00ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6400640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd2f00b1170 0x7fd2f00ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f5000640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2f015b9e0 0x7fd2f01fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:21.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6400640 1 -- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2f015b9e0 msgr2=0x7fd2f01fb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6400640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2f015b9e0 0x7fd2f01fb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6400640 1 -- 172.21.15.66:0/1819873788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f0155960 con 0x7fd2f015bdc0 2024-09-10T22:15:21.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f5000640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2f015b9e0 0x7fd2f01fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:21.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6400640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2f015bdc0 0x7fd2f0070710 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800c7a0 tx=0x7fd2e800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:21.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2ed600640 1 -- 172.21.15.66:0/1819873788 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8003b80 con 0x7fd2f015bdc0 2024-09-10T22:15:21.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2ed600640 1 -- 172.21.15.66:0/1819873788 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2e8003d40 con 0x7fd2f015bdc0 2024-09-10T22:15:21.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2ed600640 1 -- 172.21.15.66:0/1819873788 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e800aab0 con 0x7fd2f015bdc0 2024-09-10T22:15:21.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6e00640 1 -- 172.21.15.66:0/1819873788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f0155bd0 con 0x7fd2f015bdc0 2024-09-10T22:15:21.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6e00640 1 -- 172.21.15.66:0/1819873788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f00cac00 con 0x7fd2f015bdc0 2024-09-10T22:15:21.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.558+0000 7fd2f6e00640 1 -- 172.21.15.66:0/1819873788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f0155fc0 con 0x7fd2f015bdc0 2024-09-10T22:15:21.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.562+0000 7fd2ed600640 1 -- 172.21.15.66:0/1819873788 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fd2e800ac50 con 0x7fd2f015bdc0 2024-09-10T22:15:21.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.562+0000 7fd2ed600640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd2d0074c20 0x7fd2d00770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:21.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.562+0000 7fd2ed600640 1 -- 172.21.15.66:0/1819873788 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5934+0+0 (secure 0 0 0) 0x7fd2e8092e50 con 0x7fd2f015bdc0 2024-09-10T22:15:21.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.562+0000 7fd2f5a00640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd2d0074c20 0x7fd2d00770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:21.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.562+0000 7fd2f5a00640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd2d0074c20 0x7fd2d00770e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40af000 tx=0x7fd2e4019660 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.566+0000 7fd2ed600640 1 -- 172.21.15.66:0/1819873788 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd2e8060010 con 0x7fd2f015bdc0 2024-09-10T22:15:21.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.762+0000 7fd2f6e00640 1 -- 172.21.15.66:0/1819873788 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone53292", "target": ["mon-mgr", ""]}) -- 0x7fd2f0155da0 con 0x7fd2d0074c20 2024-09-10T22:15:21.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.778+0000 7fd2ed600640 1 -- 172.21.15.66:0/1819873788 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd2f0155da0 con 0x7fd2d0074c20 2024-09-10T22:15:21.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.782+0000 7fd2f6e00640 1 -- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd2d0074c20 msgr2=0x7fd2d00770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.782+0000 7fd2f6e00640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd2d0074c20 0x7fd2d00770e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40af000 tx=0x7fd2e4019660 comp rx=0 tx=0).stop 2024-09-10T22:15:21.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.782+0000 7fd2f6e00640 1 -- 172.21.15.66:0/1819873788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2f015bdc0 msgr2=0x7fd2f0070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.782+0000 7fd2f6e00640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2f015bdc0 0x7fd2f0070710 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800c7a0 tx=0x7fd2e800cc60 comp rx=0 tx=0).stop 2024-09-10T22:15:21.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.782+0000 7fd2f6e00640 1 -- 172.21.15.66:0/1819873788 shutdown_connections 2024-09-10T22:15:21.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.782+0000 7fd2f6e00640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd2d0074c20 0x7fd2d00770e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.782+0000 7fd2f6e00640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2f015bdc0 0x7fd2f0070710 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.782+0000 7fd2f6e00640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2f015b9e0 0x7fd2f01fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.782+0000 7fd2f6e00640 1 --2- 172.21.15.66:0/1819873788 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd2f00b1170 0x7fd2f00ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.782+0000 7fd2f6e00640 1 -- 172.21.15.66:0/1819873788 >> 172.21.15.66:0/1819873788 conn(0x7fd2f009aa50 msgr2=0x7fd2f006e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:21.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.782+0000 7fd2f6e00640 1 -- 172.21.15.66:0/1819873788 shutdown_connections 2024-09-10T22:15:21.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.782+0000 7fd2f6e00640 1 -- 172.21.15.66:0/1819873788 wait complete. 2024-09-10T22:15:21.808 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99941 2024-09-10T22:15:21.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.954+0000 7ff470800640 1 -- 172.21.15.66:0/2704693686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff468145050 msgr2=0x7ff46806e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.954+0000 7ff470800640 1 --2- 172.21.15.66:0/2704693686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff468145050 0x7ff46806e6a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff458009840 tx=0x7ff45801c790 comp rx=0 tx=0).stop 2024-09-10T22:15:21.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.954+0000 7ff470800640 1 -- 172.21.15.66:0/2704693686 shutdown_connections 2024-09-10T22:15:21.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.954+0000 7ff470800640 1 --2- 172.21.15.66:0/2704693686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff468145050 0x7ff46806e6a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.954+0000 7ff470800640 1 -- 172.21.15.66:0/2704693686 >> 172.21.15.66:0/2704693686 conn(0x7ff4680cfaa0 msgr2=0x7ff4680cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:21.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.954+0000 7ff470800640 1 -- 172.21.15.66:0/2704693686 shutdown_connections 2024-09-10T22:15:21.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.954+0000 7ff470800640 1 -- 172.21.15.66:0/2704693686 wait complete. 2024-09-10T22:15:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff470800640 1 Processor -- start 2024-09-10T22:15:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff470800640 1 -- start start 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff470800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4680cf250 0x7ff468138e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff470800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff468144ac0 0x7ff4681f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff470800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff468145050 0x7ff46815ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff470800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff468169d90 con 0x7ff468145050 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff470800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff468169c10 con 0x7ff468144ac0 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff470800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff468169f10 con 0x7ff4680cf250 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff46e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff468145050 0x7ff46815ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff46d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff468144ac0 0x7ff4681f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff46e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff468145050 0x7ff46815ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46346/0 (socket says 172.21.15.66:46346) 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff46e800640 1 -- 172.21.15.66:0/1631367688 learned_addr learned my addr 172.21.15.66:0/1631367688 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff46de00640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4680cf250 0x7ff468138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff46e800640 1 -- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4680cf250 msgr2=0x7ff468138e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff46e800640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4680cf250 0x7ff468138e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff46e800640 1 -- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff468144ac0 msgr2=0x7ff4681f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:21.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff46e800640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff468144ac0 0x7ff4681f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:21.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff46e800640 1 -- 172.21.15.66:0/1631367688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4681f8450 con 0x7ff468145050 2024-09-10T22:15:21.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff46d400640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff468144ac0 0x7ff4681f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:15:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff46e800640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff468145050 0x7ff46815ac20 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff46400cb10 tx=0x7ff46400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff462000640 1 -- 172.21.15.66:0/1631367688 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464016020 con 0x7ff468145050 2024-09-10T22:15:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff462000640 1 -- 172.21.15.66:0/1631367688 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff46400ce50 con 0x7ff468145050 2024-09-10T22:15:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff462000640 1 -- 172.21.15.66:0/1631367688 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46400ada0 con 0x7ff468145050 2024-09-10T22:15:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff470800640 1 -- 172.21.15.66:0/1631367688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46815b160 con 0x7ff468145050 2024-09-10T22:15:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.958+0000 7ff470800640 1 -- 172.21.15.66:0/1631367688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4681faef0 con 0x7ff468145050 2024-09-10T22:15:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.962+0000 7ff470800640 1 -- 172.21.15.66:0/1631367688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4680ce6e0 con 0x7ff468145050 2024-09-10T22:15:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.962+0000 7ff462000640 1 -- 172.21.15.66:0/1631367688 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7ff464012420 con 0x7ff468145050 2024-09-10T22:15:21.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.962+0000 7ff462000640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff44c074c20 0x7ff44c0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:21.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.962+0000 7ff462000640 1 -- 172.21.15.66:0/1631367688 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5934+0+0 (secure 0 0 0) 0x7ff464092980 con 0x7ff468145050 2024-09-10T22:15:21.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.962+0000 7ff46de00640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff44c074c20 0x7ff44c0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:21.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.962+0000 7ff46de00640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff44c074c20 0x7ff44c0770e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff45c000c10 tx=0x7ff45c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:21.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:21.966+0000 7ff462000640 1 -- 172.21.15.66:0/1631367688 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff464020020 con 0x7ff468145050 2024-09-10T22:15:22.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.170+0000 7ff470800640 1 -- 172.21.15.66:0/1631367688 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99941", "target": ["mon-mgr", ""]}) -- 0x7ff46806f580 con 0x7ff44c074c20 2024-09-10T22:15:22.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.186+0000 7ff462000640 1 -- 172.21.15.66:0/1631367688 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff46806f580 con 0x7ff44c074c20 2024-09-10T22:15:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.190+0000 7ff470800640 1 -- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff44c074c20 msgr2=0x7ff44c0770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.190+0000 7ff470800640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff44c074c20 0x7ff44c0770e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff45c000c10 tx=0x7ff45c009040 comp rx=0 tx=0).stop 2024-09-10T22:15:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.190+0000 7ff470800640 1 -- 172.21.15.66:0/1631367688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff468145050 msgr2=0x7ff46815ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.190+0000 7ff470800640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff468145050 0x7ff46815ac20 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff46400cb10 tx=0x7ff46400cc00 comp rx=0 tx=0).stop 2024-09-10T22:15:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.190+0000 7ff470800640 1 -- 172.21.15.66:0/1631367688 shutdown_connections 2024-09-10T22:15:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.190+0000 7ff470800640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff44c074c20 0x7ff44c0770e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.190+0000 7ff470800640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff468145050 0x7ff46815ac20 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.190+0000 7ff470800640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff468144ac0 0x7ff4681f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.190+0000 7ff470800640 1 --2- 172.21.15.66:0/1631367688 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4680cf250 0x7ff468138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.190+0000 7ff470800640 1 -- 172.21.15.66:0/1631367688 >> 172.21.15.66:0/1631367688 conn(0x7ff4680cfaa0 msgr2=0x7ff46806d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.190+0000 7ff470800640 1 -- 172.21.15.66:0/1631367688 shutdown_connections 2024-09-10T22:15:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.190+0000 7ff470800640 1 -- 172.21.15.66:0/1631367688 wait complete. 2024-09-10T22:15:22.220 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T22:15:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:15:22.331 INFO:teuthology.orchestra.run.smithi066.stdout:2 2024-09-10T22:15:22.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:15:22.341 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-10T22:15:22.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.586+0000 7f5933a00640 1 -- 172.21.15.66:0/2211221295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f592c1447b0 msgr2=0x7f592c0ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:22.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.586+0000 7f5933a00640 1 --2- 172.21.15.66:0/2211221295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f592c1447b0 0x7f592c0ca080 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f592c147e10 tx=0x7f592401cd50 comp rx=0 tx=0).stop 2024-09-10T22:15:22.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.586+0000 7f5933a00640 1 -- 172.21.15.66:0/2211221295 shutdown_connections 2024-09-10T22:15:22.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.586+0000 7f5933a00640 1 --2- 172.21.15.66:0/2211221295 >> v2:172.21.15.94:3300/0 conn(0x7f592c0aebf0 0x7f592c0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:22.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.586+0000 7f5933a00640 1 --2- 172.21.15.66:0/2211221295 >> v2:172.21.15.66:3300/0 conn(0x7f592c0ae7b0 0x7f592c0cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:22.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.586+0000 7f5933a00640 1 --2- 172.21.15.66:0/2211221295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f592c1447b0 0x7f592c0ca080 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f592c147e10 tx=0x7f592401cd50 comp rx=0 tx=0).stop 2024-09-10T22:15:22.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.586+0000 7f5933a00640 1 -- 172.21.15.66:0/2211221295 >> 172.21.15.66:0/2211221295 conn(0x7f592c0cfa30 msgr2=0x7f592c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:22.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.586+0000 7f5933a00640 1 -- 172.21.15.66:0/2211221295 shutdown_connections 2024-09-10T22:15:22.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.586+0000 7f5933a00640 1 -- 172.21.15.66:0/2211221295 wait complete. 2024-09-10T22:15:22.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.586+0000 7f5933a00640 1 Processor -- start 2024-09-10T22:15:22.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5933a00640 1 -- start start 2024-09-10T22:15:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5933a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f592c0ae7b0 0x7f592c13f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5933a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f592c0aebf0 0x7f592c073600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5933a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f592c1400a0 0x7f592c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5933a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f592c161aa0 con 0x7f592c1400a0 2024-09-10T22:15:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5933a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f592c0c6270 con 0x7f592c0ae7b0 2024-09-10T22:15:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5933a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f592c161c20 con 0x7f592c0aebf0 2024-09-10T22:15:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f592be00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f592c0aebf0 0x7f592c073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:22.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5931000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f592c0ae7b0 0x7f592c13f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:22.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f592be00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f592c0aebf0 0x7f592c073600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:54512/0 (socket says 172.21.15.66:54512) 2024-09-10T22:15:22.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5931a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f592c1400a0 0x7f592c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:22.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f592be00640 1 -- 172.21.15.66:0/3787637956 learned_addr learned my addr 172.21.15.66:0/3787637956 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:22.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5931a00640 1 -- 172.21.15.66:0/3787637956 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f592c0aebf0 msgr2=0x7f592c073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:22.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5931a00640 1 --2- 172.21.15.66:0/3787637956 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f592c0aebf0 0x7f592c073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:22.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5931a00640 1 -- 172.21.15.66:0/3787637956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f592c0ae7b0 msgr2=0x7f592c13f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:22.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5931a00640 1 --2- 172.21.15.66:0/3787637956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f592c0ae7b0 0x7f592c13f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:22.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.590+0000 7f5931a00640 1 -- 172.21.15.66:0/3787637956 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c143d00 con 0x7f592c1400a0 2024-09-10T22:15:22.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.594+0000 7f5931a00640 1 --2- 172.21.15.66:0/3787637956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f592c1400a0 0x7f592c073b40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f591c00e670 tx=0x7f591c00eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.594+0000 7f5928c00640 1 -- 172.21.15.66:0/3787637956 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c004220 con 0x7f592c1400a0 2024-09-10T22:15:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.594+0000 7f5928c00640 1 -- 172.21.15.66:0/3787637956 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f591c00c8b0 con 0x7f592c1400a0 2024-09-10T22:15:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.594+0000 7f5928c00640 1 -- 172.21.15.66:0/3787637956 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c00cd00 con 0x7f592c1400a0 2024-09-10T22:15:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.594+0000 7f5933a00640 1 -- 172.21.15.66:0/3787637956 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592c143a20 con 0x7f592c1400a0 2024-09-10T22:15:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.594+0000 7f5933a00640 1 -- 172.21.15.66:0/3787637956 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f592c0b36d0 con 0x7f592c1400a0 2024-09-10T22:15:22.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.598+0000 7f5933a00640 1 -- 172.21.15.66:0/3787637956 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592c06f8d0 con 0x7f592c1400a0 2024-09-10T22:15:22.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.598+0000 7f5928c00640 1 -- 172.21.15.66:0/3787637956 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f591c003600 con 0x7f592c1400a0 2024-09-10T22:15:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.602+0000 7f5928c00640 1 --2- 172.21.15.66:0/3787637956 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f590c074c70 0x7f590c077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.602+0000 7f5928c00640 1 -- 172.21.15.66:0/3787637956 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5934+0+0 (secure 0 0 0) 0x7f591c0928b0 con 0x7f592c1400a0 2024-09-10T22:15:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.602+0000 7f5931000640 1 --2- 172.21.15.66:0/3787637956 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f590c074c70 0x7f590c077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:22.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.606+0000 7f5931000640 1 --2- 172.21.15.66:0/3787637956 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f590c074c70 0x7f590c077130 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f59240af000 tx=0x7f592401b590 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:22.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.606+0000 7f5928c00640 1 -- 172.21.15.66:0/3787637956 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f591c05f440 con 0x7f592c1400a0 2024-09-10T22:15:22.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 -- 172.21.15.66:0/3787637956 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f592c06f310 con 0x7f592c1400a0 2024-09-10T22:15:22.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5928c00640 1 -- 172.21.15.66:0/3787637956 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v56)=0 v56) ==== 131+0+0 (secure 0 0 0) 0x7f591c063e70 con 0x7f592c1400a0 2024-09-10T22:15:22.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 -- 172.21.15.66:0/3787637956 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f590c074c70 msgr2=0x7f590c077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:22.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 --2- 172.21.15.66:0/3787637956 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f590c074c70 0x7f590c077130 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f59240af000 tx=0x7f592401b590 comp rx=0 tx=0).stop 2024-09-10T22:15:22.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 -- 172.21.15.66:0/3787637956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f592c1400a0 msgr2=0x7f592c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:22.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 --2- 172.21.15.66:0/3787637956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f592c1400a0 0x7f592c073b40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f591c00e670 tx=0x7f591c00eb30 comp rx=0 tx=0).stop 2024-09-10T22:15:22.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 -- 172.21.15.66:0/3787637956 shutdown_connections 2024-09-10T22:15:22.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 --2- 172.21.15.66:0/3787637956 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f590c074c70 0x7f590c077130 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:22.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 --2- 172.21.15.66:0/3787637956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f592c1400a0 0x7f592c073b40 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:22.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 --2- 172.21.15.66:0/3787637956 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f592c0aebf0 0x7f592c073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:22.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 --2- 172.21.15.66:0/3787637956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f592c0ae7b0 0x7f592c13f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:22.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 -- 172.21.15.66:0/3787637956 >> 172.21.15.66:0/3787637956 conn(0x7f592c0cfa30 msgr2=0x7f592c1612b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:22.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 -- 172.21.15.66:0/3787637956 shutdown_connections 2024-09-10T22:15:22.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:22.914+0000 7f5933a00640 1 -- 172.21.15.66:0/3787637956 wait complete. 2024-09-10T22:15:22.942 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:15:22.942 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:15:23.025 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:15:23.108 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:15:23.108 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:15:23.108 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:15:23.109 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:15:23.117 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:15:23.117 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:15:23.166 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:15:23.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.454+0000 7fc96e800640 1 -- 172.21.15.66:0/2820115859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc96815cc80 msgr2=0x7fc9680cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:23.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.454+0000 7fc96e800640 1 --2- 172.21.15.66:0/2820115859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc96815cc80 0x7fc9680cd930 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fc960009bc0 tx=0x7fc96001cb30 comp rx=0 tx=0).stop 2024-09-10T22:15:23.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.454+0000 7fc96e800640 1 -- 172.21.15.66:0/2820115859 shutdown_connections 2024-09-10T22:15:23.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.454+0000 7fc96e800640 1 --2- 172.21.15.66:0/2820115859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc96815cc80 0x7fc9680cd930 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:23.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.454+0000 7fc96e800640 1 --2- 172.21.15.66:0/2820115859 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc9680aee20 0x7fc9680ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:23.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.454+0000 7fc96e800640 1 -- 172.21.15.66:0/2820115859 >> 172.21.15.66:0/2820115859 conn(0x7fc9680cfa30 msgr2=0x7fc9680cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:23.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.454+0000 7fc96e800640 1 -- 172.21.15.66:0/2820115859 shutdown_connections 2024-09-10T22:15:23.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.454+0000 7fc96e800640 1 -- 172.21.15.66:0/2820115859 wait complete. 2024-09-10T22:15:23.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.458+0000 7fc96e800640 1 Processor -- start 2024-09-10T22:15:23.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.458+0000 7fc96e800640 1 -- start start 2024-09-10T22:15:23.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.458+0000 7fc96e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9680aee20 0x7fc968158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:23.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.458+0000 7fc96e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc968159900 0x7fc968159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:23.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.458+0000 7fc96e800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc96815cc80 0x7fc968151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:23.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.458+0000 7fc96e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9680c4c40 con 0x7fc968159900 2024-09-10T22:15:23.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.458+0000 7fc96e800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9680c4ac0 con 0x7fc9680aee20 2024-09-10T22:15:23.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.458+0000 7fc96e800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc9680c4dc0 con 0x7fc96815cc80 2024-09-10T22:15:23.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc966a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc968159900 0x7fc968159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:23.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc966a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc968159900 0x7fc968159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49340/0 (socket says 172.21.15.66:49340) 2024-09-10T22:15:23.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc966a00640 1 -- 172.21.15.66:0/397441499 learned_addr learned my addr 172.21.15.66:0/397441499 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:23.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc967400640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9680aee20 0x7fc968158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:23.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc966a00640 1 -- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc96815cc80 msgr2=0x7fc968151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:23.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc966a00640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc96815cc80 0x7fc968151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:23.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc966a00640 1 -- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9680aee20 msgr2=0x7fc968158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:23.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc966a00640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9680aee20 0x7fc968158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:23.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc966a00640 1 -- 172.21.15.66:0/397441499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9680af590 con 0x7fc968159900 2024-09-10T22:15:23.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc966a00640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc968159900 0x7fc968159e70 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fc95400ea20 tx=0x7fc95400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc95f400640 1 -- 172.21.15.66:0/397441499 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95400cd80 con 0x7fc968159900 2024-09-10T22:15:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc95f400640 1 -- 172.21.15.66:0/397441499 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc954010430 con 0x7fc968159900 2024-09-10T22:15:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc95f400640 1 -- 172.21.15.66:0/397441499 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954010860 con 0x7fc968159900 2024-09-10T22:15:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc96e800640 1 -- 172.21.15.66:0/397441499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9680af870 con 0x7fc968159900 2024-09-10T22:15:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.462+0000 7fc96e800640 1 -- 172.21.15.66:0/397441499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9680afe30 con 0x7fc968159900 2024-09-10T22:15:23.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.466+0000 7fc96e800640 1 -- 172.21.15.66:0/397441499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc968070470 con 0x7fc968159900 2024-09-10T22:15:23.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.466+0000 7fc95f400640 1 -- 172.21.15.66:0/397441499 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fc954010a00 con 0x7fc968159900 2024-09-10T22:15:23.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.466+0000 7fc95f400640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc948074c20 0x7fc9480770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:23.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.466+0000 7fc95f400640 1 -- 172.21.15.66:0/397441499 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5934+0+0 (secure 0 0 0) 0x7fc954093070 con 0x7fc968159900 2024-09-10T22:15:23.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.470+0000 7fc967400640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc948074c20 0x7fc9480770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:23.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.470+0000 7fc967400640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc948074c20 0x7fc9480770e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc958007fd0 tx=0x7fc958009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:23.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.478+0000 7fc95f400640 1 -- 172.21.15.66:0/397441499 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc95405fc00 con 0x7fc968159900 2024-09-10T22:15:23.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.782+0000 7fc96e800640 1 -- 172.21.15.66:0/397441499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc96806fe80 con 0x7fc968159900 2024-09-10T22:15:23.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.782+0000 7fc95f400640 1 -- 172.21.15.66:0/397441499 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v144) ==== 80+0+0 (secure 0 0 0) 0x7fc954064630 con 0x7fc968159900 2024-09-10T22:15:23.789 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:15:23.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.786+0000 7fc96e800640 1 -- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc948074c20 msgr2=0x7fc9480770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:23.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.786+0000 7fc96e800640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc948074c20 0x7fc9480770e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc958007fd0 tx=0x7fc958009040 comp rx=0 tx=0).stop 2024-09-10T22:15:23.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.786+0000 7fc96e800640 1 -- 172.21.15.66:0/397441499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc968159900 msgr2=0x7fc968159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:23.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.786+0000 7fc96e800640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc968159900 0x7fc968159e70 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fc95400ea20 tx=0x7fc95400eee0 comp rx=0 tx=0).stop 2024-09-10T22:15:23.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.786+0000 7fc96e800640 1 -- 172.21.15.66:0/397441499 shutdown_connections 2024-09-10T22:15:23.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.786+0000 7fc96e800640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc948074c20 0x7fc9480770e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:23.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.786+0000 7fc96e800640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc96815cc80 0x7fc968151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:23.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.786+0000 7fc96e800640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc968159900 0x7fc968159e70 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:23.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.786+0000 7fc96e800640 1 --2- 172.21.15.66:0/397441499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9680aee20 0x7fc968158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:23.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.786+0000 7fc96e800640 1 -- 172.21.15.66:0/397441499 >> 172.21.15.66:0/397441499 conn(0x7fc9680cfa30 msgr2=0x7fc96806ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:23.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.786+0000 7fc96e800640 1 -- 172.21.15.66:0/397441499 shutdown_connections 2024-09-10T22:15:23.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:23.786+0000 7fc96e800640 1 -- 172.21.15.66:0/397441499 wait complete. 2024-09-10T22:15:23.812 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-09-10T22:15:23.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:15:23.812 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T22:15:23.833 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:15:23.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:15:23.833 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:15:23.844 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[53727]: fuse finished with error 0 and tester_r 0 2024-09-10T22:15:23.970 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:15:23.971 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:15:23.971 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:15:23.980 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:15:23.980 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:15:23.980 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:15:23.980 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:15:24.026 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:15:24.026 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:15:24.026 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:15:24.026 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:15:24.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:15:24.035 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:15:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.214+0000 7fd64de00640 1 -- 172.21.15.66:0/313292683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6480aee20 msgr2=0x7fd6480ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.214+0000 7fd64de00640 1 --2- 172.21.15.66:0/313292683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6480aee20 0x7fd6480ca080 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fd634009840 tx=0x7fd63401c790 comp rx=0 tx=0).stop 2024-09-10T22:15:24.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.214+0000 7fd64de00640 1 -- 172.21.15.66:0/313292683 shutdown_connections 2024-09-10T22:15:24.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.214+0000 7fd64de00640 1 --2- 172.21.15.66:0/313292683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6480aee20 0x7fd6480ca080 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:24.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.214+0000 7fd64de00640 1 -- 172.21.15.66:0/313292683 >> 172.21.15.66:0/313292683 conn(0x7fd6480cfa30 msgr2=0x7fd6480cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:24.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.214+0000 7fd64de00640 1 -- 172.21.15.66:0/313292683 shutdown_connections 2024-09-10T22:15:24.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.214+0000 7fd64de00640 1 -- 172.21.15.66:0/313292683 wait complete. 2024-09-10T22:15:24.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.218+0000 7fd64de00640 1 Processor -- start 2024-09-10T22:15:24.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.218+0000 7fd64de00640 1 -- start start 2024-09-10T22:15:24.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.218+0000 7fd64de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6480aee20 0x7fd64815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:24.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.218+0000 7fd64de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd648159900 0x7fd648146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:24.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.218+0000 7fd64de00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd64815cc80 0x7fd648073810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:24.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.218+0000 7fd64de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd64816a400 con 0x7fd648159900 2024-09-10T22:15:24.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.218+0000 7fd64de00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd64816a280 con 0x7fd6480aee20 2024-09-10T22:15:24.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.218+0000 7fd64de00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd64816a580 con 0x7fd64815cc80 2024-09-10T22:15:24.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.218+0000 7fd646a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6480aee20 0x7fd64815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:24.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.218+0000 7fd646a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6480aee20 0x7fd64815b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:51278/0 (socket says 172.21.15.66:51278) 2024-09-10T22:15:24.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.218+0000 7fd647400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd64815cc80 0x7fd648073810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:24.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.218+0000 7fd646a00640 1 -- 172.21.15.66:0/4035365535 learned_addr learned my addr 172.21.15.66:0/4035365535 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:24.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.222+0000 7fd646a00640 1 -- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd64815cc80 msgr2=0x7fd648073810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:24.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.222+0000 7fd646a00640 1 --2- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd64815cc80 0x7fd648073810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:24.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.222+0000 7fd646a00640 1 -- 172.21.15.66:0/4035365535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd648159900 msgr2=0x7fd648146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:24.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.222+0000 7fd646a00640 1 --2- 172.21.15.66:0/4035365535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd648159900 0x7fd648146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:24.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.222+0000 7fd646a00640 1 -- 172.21.15.66:0/4035365535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd648147260 con 0x7fd6480aee20 2024-09-10T22:15:24.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.222+0000 7fd646a00640 1 --2- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6480aee20 0x7fd64815b9d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd63800c4e0 tx=0x7fd63800c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:24.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.222+0000 7fd63f400640 1 -- 172.21.15.66:0/4035365535 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6380052d0 con 0x7fd6480aee20 2024-09-10T22:15:24.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.222+0000 7fd63f400640 1 -- 172.21.15.66:0/4035365535 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd638005470 con 0x7fd6480aee20 2024-09-10T22:15:24.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.222+0000 7fd63f400640 1 -- 172.21.15.66:0/4035365535 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd638012890 con 0x7fd6480aee20 2024-09-10T22:15:24.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.222+0000 7fd64de00640 1 -- 172.21.15.66:0/4035365535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64815a1e0 con 0x7fd6480aee20 2024-09-10T22:15:24.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.222+0000 7fd64de00640 1 -- 172.21.15.66:0/4035365535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd648150fe0 con 0x7fd6480aee20 2024-09-10T22:15:24.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.222+0000 7fd64de00640 1 -- 172.21.15.66:0/4035365535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6200052a0 con 0x7fd6480aee20 2024-09-10T22:15:24.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.226+0000 7fd63f400640 1 -- 172.21.15.66:0/4035365535 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fd638012a30 con 0x7fd6480aee20 2024-09-10T22:15:24.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.226+0000 7fd63f400640 1 --2- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd628074c50 0x7fd628077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:24.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.226+0000 7fd63f400640 1 -- 172.21.15.66:0/4035365535 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5934+0+0 (secure 0 0 0) 0x7fd638091df0 con 0x7fd6480aee20 2024-09-10T22:15:24.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.230+0000 7fd646000640 1 --2- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd628074c50 0x7fd628077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:24.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.234+0000 7fd646000640 1 --2- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd628074c50 0x7fd628077110 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd6340af000 tx=0x7fd634019530 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:24.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.234+0000 7fd63f400640 1 -- 172.21.15.66:0/4035365535 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd63805efd0 con 0x7fd6480aee20 2024-09-10T22:15:24.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.626+0000 7fd64de00640 1 -- 172.21.15.66:0/4035365535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd620005860 con 0x7fd6480aee20 2024-09-10T22:15:24.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.630+0000 7fd63f400640 1 -- 172.21.15.66:0/4035365535 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5221 (secure 0 0 0) 0x7fd63805e8d0 con 0x7fd6480aee20 2024-09-10T22:15:24.635 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:24.635 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":97,"btime":"2024-09-10T22:14:20:178870+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/401729631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":401729631},{"type":"v1","addr":"172.21.15.94:6836","nonce":401729631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/996427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":996427549},{"type":"v1","addr":"172.21.15.94:6837","nonce":996427549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2947602030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2947602030},{"type":"v1","addr":"172.21.15.66:6833","nonce":2947602030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:14:10.533890+0000","modified":"2024-09-10T22:14:13.552506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/3920857628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3920857628},{"type":"v1","addr":"172.21.15.66:6835","nonce":3920857628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9282,"qdb_cluster":[9282]},"id":10}]} 2024-09-10T22:15:24.636 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 97 2024-09-10T22:15:24.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.634+0000 7fd64de00640 1 -- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd628074c50 msgr2=0x7fd628077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:24.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.634+0000 7fd64de00640 1 --2- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd628074c50 0x7fd628077110 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd6340af000 tx=0x7fd634019530 comp rx=0 tx=0).stop 2024-09-10T22:15:24.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.634+0000 7fd64de00640 1 -- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6480aee20 msgr2=0x7fd64815b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:24.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.634+0000 7fd64de00640 1 --2- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6480aee20 0x7fd64815b9d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd63800c4e0 tx=0x7fd63800c9a0 comp rx=0 tx=0).stop 2024-09-10T22:15:24.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.634+0000 7fd64de00640 1 -- 172.21.15.66:0/4035365535 shutdown_connections 2024-09-10T22:15:24.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.634+0000 7fd64de00640 1 --2- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fd628074c50 0x7fd628077110 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:24.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.634+0000 7fd64de00640 1 --2- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd64815cc80 0x7fd648073810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:24.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.634+0000 7fd64de00640 1 --2- 172.21.15.66:0/4035365535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd648159900 0x7fd648146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:24.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.634+0000 7fd64de00640 1 --2- 172.21.15.66:0/4035365535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6480aee20 0x7fd64815b9d0 secure :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd63800c4e0 tx=0x7fd63800c9a0 comp rx=0 tx=0).stop 2024-09-10T22:15:24.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.634+0000 7fd64de00640 1 -- 172.21.15.66:0/4035365535 >> 172.21.15.66:0/4035365535 conn(0x7fd6480cfa30 msgr2=0x7fd6480cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:24.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.634+0000 7fd64de00640 1 -- 172.21.15.66:0/4035365535 shutdown_connections 2024-09-10T22:15:24.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.634+0000 7fd64de00640 1 -- 172.21.15.66:0/4035365535 wait complete. 2024-09-10T22:15:24.677 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:15:24.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.918+0000 7ff5c4e00640 1 -- 172.21.15.66:0/1211616712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c00c4fc0 msgr2=0x7ff5c006eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:24.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.918+0000 7ff5c4e00640 1 --2- 172.21.15.66:0/1211616712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c00c4fc0 0x7ff5c006eb70 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0143c40 tx=0x7ff5b801cd50 comp rx=0 tx=0).stop 2024-09-10T22:15:24.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.918+0000 7ff5c4e00640 1 -- 172.21.15.66:0/1211616712 shutdown_connections 2024-09-10T22:15:24.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.918+0000 7ff5c4e00640 1 --2- 172.21.15.66:0/1211616712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c00c4fc0 0x7ff5c006eb70 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:24.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.918+0000 7ff5c4e00640 1 --2- 172.21.15.66:0/1211616712 >> v2:172.21.15.94:3300/0 conn(0x7ff5c00706c0 0x7ff5c006e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:24.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.918+0000 7ff5c4e00640 1 -- 172.21.15.66:0/1211616712 >> 172.21.15.66:0/1211616712 conn(0x7ff5c00cfa30 msgr2=0x7ff5c00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:24.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.918+0000 7ff5c4e00640 1 -- 172.21.15.66:0/1211616712 shutdown_connections 2024-09-10T22:15:24.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.918+0000 7ff5c4e00640 1 -- 172.21.15.66:0/1211616712 wait complete. 2024-09-10T22:15:24.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.922+0000 7ff5c4e00640 1 Processor -- start 2024-09-10T22:15:24.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.922+0000 7ff5c4e00640 1 -- start start 2024-09-10T22:15:24.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.922+0000 7ff5c4e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c00706c0 0x7ff5c00b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:24.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.922+0000 7ff5c4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c00c4fc0 0x7ff5c01fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:24.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.922+0000 7ff5c4e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c015dd50 0x7ff5c01fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:24.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.922+0000 7ff5c4e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c0167e60 con 0x7ff5c00c4fc0 2024-09-10T22:15:24.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.922+0000 7ff5c4e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c00cf880 con 0x7ff5c00706c0 2024-09-10T22:15:24.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.922+0000 7ff5c4e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c0167fe0 con 0x7ff5c015dd50 2024-09-10T22:15:24.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.922+0000 7ff5be800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c015dd50 0x7ff5c01fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:24.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.922+0000 7ff5be800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c015dd50 0x7ff5c01fc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:54562/0 (socket says 172.21.15.66:54562) 2024-09-10T22:15:24.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.922+0000 7ff5be800640 1 -- 172.21.15.66:0/3776155706 learned_addr learned my addr 172.21.15.66:0/3776155706 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:24.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.926+0000 7ff5be800640 1 -- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c00706c0 msgr2=0x7ff5c00b3e20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:15:24.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.926+0000 7ff5be800640 1 --2- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c00706c0 0x7ff5c00b3e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:24.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.926+0000 7ff5be800640 1 -- 172.21.15.66:0/3776155706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c00c4fc0 msgr2=0x7ff5c01fbca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:15:24.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.926+0000 7ff5be800640 1 --2- 172.21.15.66:0/3776155706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c00c4fc0 0x7ff5c01fbca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:24.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.926+0000 7ff5be800640 1 -- 172.21.15.66:0/3776155706 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c01545d0 con 0x7ff5c015dd50 2024-09-10T22:15:24.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.926+0000 7ff5be800640 1 --2- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c015dd50 0x7ff5c01fc1e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8019960 tx=0x7ff5b8003f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:24.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.926+0000 7ff5b6000640 1 -- 172.21.15.66:0/3776155706 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80bb070 con 0x7ff5c015dd50 2024-09-10T22:15:24.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.926+0000 7ff5b6000640 1 -- 172.21.15.66:0/3776155706 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5b8004d10 con 0x7ff5c015dd50 2024-09-10T22:15:24.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.926+0000 7ff5c4e00640 1 -- 172.21.15.66:0/3776155706 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c00b4640 con 0x7ff5c015dd50 2024-09-10T22:15:24.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.926+0000 7ff5b6000640 1 -- 172.21.15.66:0/3776155706 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80b2040 con 0x7ff5c015dd50 2024-09-10T22:15:24.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.926+0000 7ff5c4e00640 1 -- 172.21.15.66:0/3776155706 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c00b4360 con 0x7ff5c015dd50 2024-09-10T22:15:24.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.930+0000 7ff5c4e00640 1 -- 172.21.15.66:0/3776155706 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c00c48b0 con 0x7ff5c015dd50 2024-09-10T22:15:24.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.930+0000 7ff5b6000640 1 -- 172.21.15.66:0/3776155706 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7ff5b80b5030 con 0x7ff5c015dd50 2024-09-10T22:15:24.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.930+0000 7ff5b6000640 1 --2- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff59c074cf0 0x7ff59c0771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:24.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.930+0000 7ff5b6000640 1 -- 172.21.15.66:0/3776155706 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(144..144 src has 1..144) ==== 5934+0+0 (secure 0 0 0) 0x7ff5b812c4e0 con 0x7ff5c015dd50 2024-09-10T22:15:24.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.934+0000 7ff5bde00640 1 --2- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff59c074cf0 0x7ff59c0771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:24.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.934+0000 7ff5bde00640 1 --2- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff59c074cf0 0x7ff59c0771b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00014b0 tx=0x7ff5b00079c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:24.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:24.938+0000 7ff5b6000640 1 -- 172.21.15.66:0/3776155706 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff5b8016970 con 0x7ff5c015dd50 2024-09-10T22:15:25.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.310+0000 7ff5c4e00640 1 -- 172.21.15.66:0/3776155706 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5c00d0190 con 0x7ff5c015dd50 2024-09-10T22:15:25.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.310+0000 7ff5b6000640 1 -- 172.21.15.66:0/3776155706 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5221 (secure 0 0 0) 0x7ff5b80f96a0 con 0x7ff5c015dd50 2024-09-10T22:15:25.317 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:25.317 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":97,"btime":"2024-09-10T22:14:20:178870+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/401729631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":401729631},{"type":"v1","addr":"172.21.15.94:6836","nonce":401729631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/996427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":996427549},{"type":"v1","addr":"172.21.15.94:6837","nonce":996427549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2947602030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2947602030},{"type":"v1","addr":"172.21.15.66:6833","nonce":2947602030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:14:10.533890+0000","modified":"2024-09-10T22:14:13.552506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/3920857628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3920857628},{"type":"v1","addr":"172.21.15.66:6835","nonce":3920857628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9282,"qdb_cluster":[9282]},"id":10}]} 2024-09-10T22:15:25.318 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 97 2024-09-10T22:15:25.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.314+0000 7ff5c4e00640 1 -- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff59c074cf0 msgr2=0x7ff59c0771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.314+0000 7ff5c4e00640 1 --2- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff59c074cf0 0x7ff59c0771b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00014b0 tx=0x7ff5b00079c0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.314+0000 7ff5c4e00640 1 -- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c015dd50 msgr2=0x7ff5c01fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.314+0000 7ff5c4e00640 1 --2- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c015dd50 0x7ff5c01fc1e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8019960 tx=0x7ff5b8003f60 comp rx=0 tx=0).stop 2024-09-10T22:15:25.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.314+0000 7ff5c4e00640 1 -- 172.21.15.66:0/3776155706 shutdown_connections 2024-09-10T22:15:25.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.314+0000 7ff5c4e00640 1 --2- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff59c074cf0 0x7ff59c0771b0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.314+0000 7ff5c4e00640 1 --2- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5c015dd50 0x7ff5c01fc1e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.314+0000 7ff5c4e00640 1 --2- 172.21.15.66:0/3776155706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c00c4fc0 0x7ff5c01fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.314+0000 7ff5c4e00640 1 --2- 172.21.15.66:0/3776155706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5c00706c0 0x7ff5c00b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.314+0000 7ff5c4e00640 1 -- 172.21.15.66:0/3776155706 >> 172.21.15.66:0/3776155706 conn(0x7ff5c00cfa30 msgr2=0x7ff5c00ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:25.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.314+0000 7ff5c4e00640 1 -- 172.21.15.66:0/3776155706 shutdown_connections 2024-09-10T22:15:25.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.314+0000 7ff5c4e00640 1 -- 172.21.15.66:0/3776155706 wait complete. 2024-09-10T22:15:25.340 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:15:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.482+0000 7fdab9a00640 1 -- 172.21.15.66:0/3213865801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdab40cf180 msgr2=0x7fdab40d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.482+0000 7fdab9a00640 1 --2- 172.21.15.66:0/3213865801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdab40cf180 0x7fdab40d04e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009c80 tx=0x7fdaac01cd10 comp rx=0 tx=0).stop 2024-09-10T22:15:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.482+0000 7fdab9a00640 1 -- 172.21.15.66:0/3213865801 shutdown_connections 2024-09-10T22:15:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.482+0000 7fdab9a00640 1 --2- 172.21.15.66:0/3213865801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdab40cf180 0x7fdab40d04e0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.482+0000 7fdab9a00640 1 --2- 172.21.15.66:0/3213865801 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdab415bd40 0x7fdab40cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.482+0000 7fdab9a00640 1 -- 172.21.15.66:0/3213865801 >> 172.21.15.66:0/3213865801 conn(0x7fdab40d1c60 msgr2=0x7fdab40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.482+0000 7fdab9a00640 1 -- 172.21.15.66:0/3213865801 shutdown_connections 2024-09-10T22:15:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.482+0000 7fdab9a00640 1 -- 172.21.15.66:0/3213865801 wait complete. 2024-09-10T22:15:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.482+0000 7fdab9a00640 1 Processor -- start 2024-09-10T22:15:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.482+0000 7fdab9a00640 1 -- start start 2024-09-10T22:15:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40cf180 0x7fdab4158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab9a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdab40cf830 0x7fdab4158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdab415bd40 0x7fdab4158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab9a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4168ba0 con 0x7fdab415bd40 2024-09-10T22:15:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab9a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4168a20 con 0x7fdab40cf180 2024-09-10T22:15:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab9a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdab4168d20 con 0x7fdab40cf830 2024-09-10T22:15:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdab415bd40 0x7fdab4158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:25.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40cf180 0x7fdab4158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdab415bd40 0x7fdab4158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49380/0 (socket says 172.21.15.66:49380) 2024-09-10T22:15:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab3400640 1 -- 172.21.15.66:0/1695196922 learned_addr learned my addr 172.21.15.66:0/1695196922 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdaabe00640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdab40cf830 0x7fdab4158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab3400640 1 -- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdab40cf830 msgr2=0x7fdab4158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab3400640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdab40cf830 0x7fdab4158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab3400640 1 -- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40cf180 msgr2=0x7fdab4158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab3400640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40cf180 0x7fdab4158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab3400640 1 -- 172.21.15.66:0/1695196922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab415ac10 con 0x7fdab415bd40 2024-09-10T22:15:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab3400640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdab415bd40 0x7fdab4158ec0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fdaac01c780 tx=0x7fdaac005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdaaaa00640 1 -- 172.21.15.66:0/1695196922 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0b2040 con 0x7fdab415bd40 2024-09-10T22:15:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdaaaa00640 1 -- 172.21.15.66:0/1695196922 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdaac0a5c00 con 0x7fdab415bd40 2024-09-10T22:15:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdaaaa00640 1 -- 172.21.15.66:0/1695196922 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac004240 con 0x7fdab415bd40 2024-09-10T22:15:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab9a00640 1 -- 172.21.15.66:0/1695196922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab415aef0 con 0x7fdab415bd40 2024-09-10T22:15:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab9a00640 1 -- 172.21.15.66:0/1695196922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab415b4b0 con 0x7fdab415bd40 2024-09-10T22:15:25.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.486+0000 7fdab9a00640 1 -- 172.21.15.66:0/1695196922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab40aec10 con 0x7fdab415bd40 2024-09-10T22:15:25.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.490+0000 7fdaaaa00640 1 -- 172.21.15.66:0/1695196922 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fdaac0b5070 con 0x7fdab415bd40 2024-09-10T22:15:25.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.490+0000 7fdaaaa00640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda90074c50 0x7fda90077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:25.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.490+0000 7fdaaaa00640 1 -- 172.21.15.66:0/1695196922 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5934+0+0 (secure 0 0 0) 0x7fdaac12c0c0 con 0x7fdab415bd40 2024-09-10T22:15:25.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.490+0000 7fdab2a00640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda90074c50 0x7fda90077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:25.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.490+0000 7fdab2a00640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda90074c50 0x7fda90077110 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009600 tx=0x7fdaa40091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:25.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.494+0000 7fdaaaa00640 1 -- 172.21.15.66:0/1695196922 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdaac0c4060 con 0x7fdab415bd40 2024-09-10T22:15:25.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.674+0000 7fdab9a00640 1 -- 172.21.15.66:0/1695196922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdab40b0df0 con 0x7fdab415bd40 2024-09-10T22:15:25.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.674+0000 7fdaaaa00640 1 -- 172.21.15.66:0/1695196922 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+17051 (secure 0 0 0) 0x7fdaac016990 con 0x7fdab415bd40 2024-09-10T22:15:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:25.680 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":144,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:15:21.004466+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:14:08.253835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T22:14:09.497667+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":144,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.8","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:15:25.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.674+0000 7fdab9a00640 1 -- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda90074c50 msgr2=0x7fda90077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.674+0000 7fdab9a00640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda90074c50 0x7fda90077110 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009600 tx=0x7fdaa40091e0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.674+0000 7fdab9a00640 1 -- 172.21.15.66:0/1695196922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdab415bd40 msgr2=0x7fdab4158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.674+0000 7fdab9a00640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdab415bd40 0x7fdab4158ec0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fdaac01c780 tx=0x7fdaac005c40 comp rx=0 tx=0).stop 2024-09-10T22:15:25.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.678+0000 7fdab9a00640 1 -- 172.21.15.66:0/1695196922 shutdown_connections 2024-09-10T22:15:25.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.678+0000 7fdab9a00640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fda90074c50 0x7fda90077110 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.678+0000 7fdab9a00640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdab415bd40 0x7fdab4158ec0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.678+0000 7fdab9a00640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdab40cf830 0x7fdab4158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.678+0000 7fdab9a00640 1 --2- 172.21.15.66:0/1695196922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40cf180 0x7fdab4158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.678+0000 7fdab9a00640 1 -- 172.21.15.66:0/1695196922 >> 172.21.15.66:0/1695196922 conn(0x7fdab40d1c60 msgr2=0x7fdab40b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:25.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.678+0000 7fdab9a00640 1 -- 172.21.15.66:0/1695196922 shutdown_connections 2024-09-10T22:15:25.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.678+0000 7fdab9a00640 1 -- 172.21.15.66:0/1695196922 wait complete. 2024-09-10T22:15:25.702 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:15:25.702 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:15:25.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.842+0000 7f5f9ec00640 1 -- 172.21.15.66:0/1766140023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9815cc80 msgr2=0x7f5f980ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.842+0000 7f5f9ec00640 1 --2- 172.21.15.66:0/1766140023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9815cc80 0x7f5f980ae710 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f5f980701e0 tx=0x7f5f8001c570 comp rx=0 tx=0).stop 2024-09-10T22:15:25.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.842+0000 7f5f9ec00640 1 -- 172.21.15.66:0/1766140023 shutdown_connections 2024-09-10T22:15:25.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.842+0000 7f5f9ec00640 1 --2- 172.21.15.66:0/1766140023 >> v2:172.21.15.94:3300/0 conn(0x7f5f980aec50 0x7f5f9815f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.842+0000 7f5f9ec00640 1 --2- 172.21.15.66:0/1766140023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9815cc80 0x7f5f980ae710 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.842+0000 7f5f9ec00640 1 --2- 172.21.15.66:0/1766140023 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f98159900 0x7f5f980ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.842+0000 7f5f9ec00640 1 -- 172.21.15.66:0/1766140023 >> 172.21.15.66:0/1766140023 conn(0x7f5f980cfa30 msgr2=0x7f5f980cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:25.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.842+0000 7f5f9ec00640 1 -- 172.21.15.66:0/1766140023 shutdown_connections 2024-09-10T22:15:25.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.842+0000 7f5f9ec00640 1 -- 172.21.15.66:0/1766140023 wait complete. 2024-09-10T22:15:25.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f9ec00640 1 Processor -- start 2024-09-10T22:15:25.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f9ec00640 1 -- start start 2024-09-10T22:15:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f9ec00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f980aec50 0x7f5f9814f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f9ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f98159900 0x7f5f9813bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f9ec00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f9815cc80 0x7f5f98073b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f9ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5f980cde50 con 0x7f5f98159900 2024-09-10T22:15:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f9ec00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f980cdcd0 con 0x7f5f980aec50 2024-09-10T22:15:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f9ec00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5f980cdfd0 con 0x7f5f9815cc80 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f8fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f98159900 0x7f5f9813bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f8fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f98159900 0x7f5f9813bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49400/0 (socket says 172.21.15.66:49400) 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f8fe00640 1 -- 172.21.15.66:0/3359534540 learned_addr learned my addr 172.21.15.66:0/3359534540 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f97e00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f980aec50 0x7f5f9814f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f9cc00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f9815cc80 0x7f5f98073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f8fe00640 1 -- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f9815cc80 msgr2=0x7f5f98073b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f8fe00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f9815cc80 0x7f5f98073b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f8fe00640 1 -- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f980aec50 msgr2=0x7f5f9814f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f8fe00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f980aec50 0x7f5f9814f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f8fe00640 1 -- 172.21.15.66:0/3359534540 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f980ca310 con 0x7f5f98159900 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f8fe00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f98159900 0x7f5f9813bb30 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5f800ad040 tx=0x7f5f80004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f94c00640 1 -- 172.21.15.66:0/3359534540 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f800ab8d0 con 0x7f5f98159900 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f94c00640 1 -- 172.21.15.66:0/3359534540 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f800b9070 con 0x7f5f98159900 2024-09-10T22:15:25.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f9ec00640 1 -- 172.21.15.66:0/3359534540 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f98153fa0 con 0x7f5f98159900 2024-09-10T22:15:25.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f94c00640 1 -- 172.21.15.66:0/3359534540 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f800abdc0 con 0x7f5f98159900 2024-09-10T22:15:25.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.846+0000 7f5f9ec00640 1 -- 172.21.15.66:0/3359534540 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9813eb10 con 0x7f5f98159900 2024-09-10T22:15:25.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.850+0000 7f5f9ec00640 1 -- 172.21.15.66:0/3359534540 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f98070470 con 0x7f5f98159900 2024-09-10T22:15:25.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.850+0000 7f5f94c00640 1 -- 172.21.15.66:0/3359534540 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f5f80003600 con 0x7f5f98159900 2024-09-10T22:15:25.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.850+0000 7f5f94c00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f74074c20 0x7f5f740770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:25.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.850+0000 7f5f94c00640 1 -- 172.21.15.66:0/3359534540 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5934+0+0 (secure 0 0 0) 0x7f5f8012cae0 con 0x7f5f98159900 2024-09-10T22:15:25.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.850+0000 7f5f97e00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f74074c20 0x7f5f740770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:25.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.850+0000 7f5f97e00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f74074c20 0x7f5f740770e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5f88005d90 tx=0x7f5f88005c60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:25.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:25.854+0000 7f5f94c00640 1 -- 172.21.15.66:0/3359534540 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f800164b0 con 0x7f5f98159900 2024-09-10T22:15:26.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 -- 172.21.15.66:0/3359534540 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f98156a70 con 0x7f5f98159900 2024-09-10T22:15:26.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f94c00640 1 -- 172.21.15.66:0/3359534540 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5221 (secure 0 0 0) 0x7f5f98156a70 con 0x7f5f98159900 2024-09-10T22:15:26.071 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:26.071 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":97,"btime":"2024-09-10T22:14:20:178870+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/401729631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":401729631},{"type":"v1","addr":"172.21.15.94:6836","nonce":401729631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/996427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":996427549},{"type":"v1","addr":"172.21.15.94:6837","nonce":996427549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2947602030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2947602030},{"type":"v1","addr":"172.21.15.66:6833","nonce":2947602030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:14:10.533890+0000","modified":"2024-09-10T22:14:13.552506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/3920857628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3920857628},{"type":"v1","addr":"172.21.15.66:6835","nonce":3920857628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9282,"qdb_cluster":[9282]},"id":10}]} 2024-09-10T22:15:26.071 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 97 2024-09-10T22:15:26.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 -- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f74074c20 msgr2=0x7f5f740770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f74074c20 0x7f5f740770e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5f88005d90 tx=0x7f5f88005c60 comp rx=0 tx=0).stop 2024-09-10T22:15:26.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 -- 172.21.15.66:0/3359534540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f98159900 msgr2=0x7f5f9813bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f98159900 0x7f5f9813bb30 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5f800ad040 tx=0x7f5f80004600 comp rx=0 tx=0).stop 2024-09-10T22:15:26.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 -- 172.21.15.66:0/3359534540 shutdown_connections 2024-09-10T22:15:26.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5f74074c20 0x7f5f740770e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5f9815cc80 0x7f5f98073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f98159900 0x7f5f9813bb30 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 --2- 172.21.15.66:0/3359534540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f980aec50 0x7f5f9814f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 -- 172.21.15.66:0/3359534540 >> 172.21.15.66:0/3359534540 conn(0x7f5f980cfa30 msgr2=0x7f5f9815bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:26.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 -- 172.21.15.66:0/3359534540 shutdown_connections 2024-09-10T22:15:26.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.066+0000 7f5f9ec00640 1 -- 172.21.15.66:0/3359534540 wait complete. 2024-09-10T22:15:26.094 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:15:26.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.234+0000 7f41b9000640 1 -- 172.21.15.66:0/4261723905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b40aee20 msgr2=0x7f41b40ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.234+0000 7f41b9000640 1 --2- 172.21.15.66:0/4261723905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b40aee20 0x7f41b40ca080 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f419c009840 tx=0x7f419c01c790 comp rx=0 tx=0).stop 2024-09-10T22:15:26.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.234+0000 7f41b9000640 1 -- 172.21.15.66:0/4261723905 shutdown_connections 2024-09-10T22:15:26.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.234+0000 7f41b9000640 1 --2- 172.21.15.66:0/4261723905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b40aee20 0x7f41b40ca080 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.234+0000 7f41b9000640 1 --2- 172.21.15.66:0/4261723905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41b415cc80 0x7f41b40cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.234+0000 7f41b9000640 1 -- 172.21.15.66:0/4261723905 >> 172.21.15.66:0/4261723905 conn(0x7f41b40cfa30 msgr2=0x7f41b40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:26.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.234+0000 7f41b9000640 1 -- 172.21.15.66:0/4261723905 shutdown_connections 2024-09-10T22:15:26.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.234+0000 7f41b9000640 1 -- 172.21.15.66:0/4261723905 wait complete. 2024-09-10T22:15:26.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b9000640 1 Processor -- start 2024-09-10T22:15:26.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b9000640 1 -- start start 2024-09-10T22:15:26.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b40aee20 0x7f41b415f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b9000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41b4159900 0x7f41b4140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b415cc80 0x7f41b4159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b9000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41b4168f70 con 0x7f41b40aee20 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b9000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41b4168df0 con 0x7f41b415cc80 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b9000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f41b41690f0 con 0x7f41b4159900 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b40aee20 0x7f41b415f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b40aee20 0x7f41b415f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49414/0 (socket says 172.21.15.66:49414) 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b1e00640 1 -- 172.21.15.66:0/20874288 learned_addr learned my addr 172.21.15.66:0/20874288 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b2800640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b415cc80 0x7f41b4159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41abe00640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41b4159900 0x7f41b4140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b1e00640 1 -- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41b4159900 msgr2=0x7f41b4140de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b1e00640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41b4159900 0x7f41b4140de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b1e00640 1 -- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b415cc80 msgr2=0x7f41b4159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b1e00640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b415cc80 0x7f41b4159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b1e00640 1 -- 172.21.15.66:0/20874288 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b4144480 con 0x7f41b40aee20 2024-09-10T22:15:26.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b1e00640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b40aee20 0x7f41b415f180 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f41a400c490 tx=0x7f41a400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:26.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41aa000640 1 -- 172.21.15.66:0/20874288 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4016020 con 0x7f41b40aee20 2024-09-10T22:15:26.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41aa000640 1 -- 172.21.15.66:0/20874288 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41a4012940 con 0x7f41b40aee20 2024-09-10T22:15:26.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b9000640 1 -- 172.21.15.66:0/20874288 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b41face0 con 0x7f41b40aee20 2024-09-10T22:15:26.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41aa000640 1 -- 172.21.15.66:0/20874288 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4012d90 con 0x7f41b40aee20 2024-09-10T22:15:26.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b9000640 1 -- 172.21.15.66:0/20874288 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b41faf80 con 0x7f41b40aee20 2024-09-10T22:15:26.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.238+0000 7f41b9000640 1 -- 172.21.15.66:0/20874288 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b4070470 con 0x7f41b40aee20 2024-09-10T22:15:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.242+0000 7f41aa000640 1 -- 172.21.15.66:0/20874288 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f41a401b400 con 0x7f41b40aee20 2024-09-10T22:15:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.242+0000 7f41aa000640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4190074c20 0x7f41900770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.242+0000 7f41aa000640 1 -- 172.21.15.66:0/20874288 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5934+0+0 (secure 0 0 0) 0x7f41a4092770 con 0x7f41b40aee20 2024-09-10T22:15:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.242+0000 7f41abe00640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4190074c20 0x7f41900770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:26.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.242+0000 7f41abe00640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4190074c20 0x7f41900770e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f419c0ae000 tx=0x7f419c01afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:26.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.246+0000 7f41aa000640 1 -- 172.21.15.66:0/20874288 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f41a4020020 con 0x7f41b40aee20 2024-09-10T22:15:26.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.454+0000 7f41b9000640 1 -- 172.21.15.66:0/20874288 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41b406fb50 con 0x7f41b40aee20 2024-09-10T22:15:26.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.454+0000 7f41aa000640 1 -- 172.21.15.66:0/20874288 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5221 (secure 0 0 0) 0x7f41a405f320 con 0x7f41b40aee20 2024-09-10T22:15:26.461 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:26.461 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":97,"btime":"2024-09-10T22:14:20:178870+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/401729631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":401729631},{"type":"v1","addr":"172.21.15.94:6836","nonce":401729631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/996427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":996427549},{"type":"v1","addr":"172.21.15.94:6837","nonce":996427549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2947602030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2947602030},{"type":"v1","addr":"172.21.15.66:6833","nonce":2947602030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:14:10.533890+0000","modified":"2024-09-10T22:14:13.552506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/3920857628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3920857628},{"type":"v1","addr":"172.21.15.66:6835","nonce":3920857628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9282,"qdb_cluster":[9282]},"id":10}]} 2024-09-10T22:15:26.461 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 97 2024-09-10T22:15:26.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.458+0000 7f41b9000640 1 -- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4190074c20 msgr2=0x7f41900770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.458+0000 7f41b9000640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4190074c20 0x7f41900770e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f419c0ae000 tx=0x7f419c01afd0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.458+0000 7f41b9000640 1 -- 172.21.15.66:0/20874288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b40aee20 msgr2=0x7f41b415f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.458+0000 7f41b9000640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b40aee20 0x7f41b415f180 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f41a400c490 tx=0x7f41a400c950 comp rx=0 tx=0).stop 2024-09-10T22:15:26.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.458+0000 7f41b9000640 1 -- 172.21.15.66:0/20874288 shutdown_connections 2024-09-10T22:15:26.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.458+0000 7f41b9000640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f4190074c20 0x7f41900770e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.458+0000 7f41b9000640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b415cc80 0x7f41b4159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.458+0000 7f41b9000640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41b4159900 0x7f41b4140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.458+0000 7f41b9000640 1 --2- 172.21.15.66:0/20874288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b40aee20 0x7f41b415f180 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.458+0000 7f41b9000640 1 -- 172.21.15.66:0/20874288 >> 172.21.15.66:0/20874288 conn(0x7f41b40cfa30 msgr2=0x7f41b4072c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:26.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.458+0000 7f41b9000640 1 -- 172.21.15.66:0/20874288 shutdown_connections 2024-09-10T22:15:26.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.458+0000 7f41b9000640 1 -- 172.21.15.66:0/20874288 wait complete. 2024-09-10T22:15:26.484 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:15:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 -- 172.21.15.66:0/1731639282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b58160280 msgr2=0x7f5b580b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 --2- 172.21.15.66:0/1731639282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b58160280 0x7f5b580b3170 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009840 tx=0x7f5b4401d1e0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 -- 172.21.15.66:0/1731639282 shutdown_connections 2024-09-10T22:15:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 --2- 172.21.15.66:0/1731639282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b58160280 0x7f5b580b3170 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 -- 172.21.15.66:0/1731639282 >> 172.21.15.66:0/1731639282 conn(0x7f5b580d1c60 msgr2=0x7f5b580d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 -- 172.21.15.66:0/1731639282 shutdown_connections 2024-09-10T22:15:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 -- 172.21.15.66:0/1731639282 wait complete. 2024-09-10T22:15:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 Processor -- start 2024-09-10T22:15:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 -- start start 2024-09-10T22:15:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b580b1e10 0x7f5b580b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b5815f450 0x7f5b580b7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b58160280 0x7f5b581669f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5816a040 con 0x7f5b58160280 2024-09-10T22:15:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b58169ec0 con 0x7f5b5815f450 2024-09-10T22:15:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b60400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5816a1c0 con 0x7f5b580b1e10 2024-09-10T22:15:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b5e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b58160280 0x7f5b581669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b5e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b58160280 0x7f5b581669f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49418/0 (socket says 172.21.15.66:49418) 2024-09-10T22:15:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b5e400640 1 -- 172.21.15.66:0/2244405556 learned_addr learned my addr 172.21.15.66:0/2244405556 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b5da00640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b580b1e10 0x7f5b580b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b5d000640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b5815f450 0x7f5b580b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b5e400640 1 -- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b580b1e10 msgr2=0x7f5b580b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b5e400640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b580b1e10 0x7f5b580b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b5e400640 1 -- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b5815f450 msgr2=0x7f5b580b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b5e400640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b5815f450 0x7f5b580b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.626+0000 7f5b5e400640 1 -- 172.21.15.66:0/2244405556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b580b5090 con 0x7f5b58160280 2024-09-10T22:15:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.630+0000 7f5b5e400640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b58160280 0x7f5b581669f0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000c690 tx=0x7f5b5000cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.630+0000 7f5b55600640 1 -- 172.21.15.66:0/2244405556 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50003b80 con 0x7f5b58160280 2024-09-10T22:15:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.630+0000 7f5b60400640 1 -- 172.21.15.66:0/2244405556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b580b6a70 con 0x7f5b58160280 2024-09-10T22:15:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.630+0000 7f5b55600640 1 -- 172.21.15.66:0/2244405556 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b50003d40 con 0x7f5b58160280 2024-09-10T22:15:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.630+0000 7f5b55600640 1 -- 172.21.15.66:0/2244405556 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b500052d0 con 0x7f5b58160280 2024-09-10T22:15:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.630+0000 7f5b60400640 1 -- 172.21.15.66:0/2244405556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b580b7030 con 0x7f5b58160280 2024-09-10T22:15:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.630+0000 7f5b60400640 1 -- 172.21.15.66:0/2244405556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b580b3170 con 0x7f5b58160280 2024-09-10T22:15:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.630+0000 7f5b55600640 1 -- 172.21.15.66:0/2244405556 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f5b50005470 con 0x7f5b58160280 2024-09-10T22:15:26.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.634+0000 7f5b55600640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b38074c20 0x7f5b380770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:26.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.634+0000 7f5b55600640 1 -- 172.21.15.66:0/2244405556 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5934+0+0 (secure 0 0 0) 0x7f5b50092320 con 0x7f5b58160280 2024-09-10T22:15:26.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.634+0000 7f5b5da00640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b38074c20 0x7f5b380770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.634+0000 7f5b5da00640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b38074c20 0x7f5b380770e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0016e0 tx=0x7f5b4c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:26.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.638+0000 7f5b55600640 1 -- 172.21.15.66:0/2244405556 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5b5005f480 con 0x7f5b58160280 2024-09-10T22:15:26.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.810+0000 7f5b60400640 1 -- 172.21.15.66:0/2244405556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b580d2a40 con 0x7f5b58160280 2024-09-10T22:15:26.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b55600640 1 -- 172.21.15.66:0/2244405556 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+17051 (secure 0 0 0) 0x7f5b5005ed80 con 0x7f5b58160280 2024-09-10T22:15:26.819 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:26.819 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":144,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:15:21.004466+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:14:08.253835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T22:14:09.497667+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":144,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.8","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:15:26.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b60400640 1 -- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b38074c20 msgr2=0x7f5b380770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b60400640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b38074c20 0x7f5b380770e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0016e0 tx=0x7f5b4c009040 comp rx=0 tx=0).stop 2024-09-10T22:15:26.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b60400640 1 -- 172.21.15.66:0/2244405556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b58160280 msgr2=0x7f5b581669f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b60400640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b58160280 0x7f5b581669f0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000c690 tx=0x7f5b5000cb50 comp rx=0 tx=0).stop 2024-09-10T22:15:26.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b60400640 1 -- 172.21.15.66:0/2244405556 shutdown_connections 2024-09-10T22:15:26.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b60400640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5b38074c20 0x7f5b380770e0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b60400640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b58160280 0x7f5b581669f0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b60400640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b5815f450 0x7f5b580b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b60400640 1 --2- 172.21.15.66:0/2244405556 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b580b1e10 0x7f5b580b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b60400640 1 -- 172.21.15.66:0/2244405556 >> 172.21.15.66:0/2244405556 conn(0x7f5b580d1c60 msgr2=0x7f5b58071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b60400640 1 -- 172.21.15.66:0/2244405556 shutdown_connections 2024-09-10T22:15:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.814+0000 7f5b60400640 1 -- 172.21.15.66:0/2244405556 wait complete. 2024-09-10T22:15:26.841 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:15:26.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.982+0000 7fad26a00640 1 -- 172.21.15.66:0/2602599965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad201611d0 msgr2=0x7fad201615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.982+0000 7fad26a00640 1 --2- 172.21.15.66:0/2602599965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad201611d0 0x7fad201615b0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fad200d02f0 tx=0x7fad1001c8c0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.982+0000 7fad26a00640 1 -- 172.21.15.66:0/2602599965 shutdown_connections 2024-09-10T22:15:26.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.982+0000 7fad26a00640 1 --2- 172.21.15.66:0/2602599965 >> v2:172.21.15.94:3300/0 conn(0x7fad200b2e60 0x7fad20160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.982+0000 7fad26a00640 1 --2- 172.21.15.66:0/2602599965 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad200d49b0 0x7fad20160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.982+0000 7fad26a00640 1 --2- 172.21.15.66:0/2602599965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad201611d0 0x7fad201615b0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.982+0000 7fad26a00640 1 -- 172.21.15.66:0/2602599965 >> 172.21.15.66:0/2602599965 conn(0x7fad200d3e90 msgr2=0x7fad200d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:26.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.982+0000 7fad26a00640 1 -- 172.21.15.66:0/2602599965 shutdown_connections 2024-09-10T22:15:26.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.982+0000 7fad26a00640 1 -- 172.21.15.66:0/2602599965 wait complete. 2024-09-10T22:15:26.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad26a00640 1 Processor -- start 2024-09-10T22:15:26.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad26a00640 1 -- start start 2024-09-10T22:15:26.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad26a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad200b2e60 0x7fad2014f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:26.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad26a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad200d49b0 0x7fad200c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:26.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad26a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad201611d0 0x7fad2006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:26.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad26a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad200d2b40 con 0x7fad201611d0 2024-09-10T22:15:26.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad26a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad200d29c0 con 0x7fad200b2e60 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad26a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fad200ceae0 con 0x7fad200d49b0 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad24a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad201611d0 0x7fad2006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad1fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad200b2e60 0x7fad2014f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad17400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad200d49b0 0x7fad200c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad24a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad201611d0 0x7fad2006e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49438/0 (socket says 172.21.15.66:49438) 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad1fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad200b2e60 0x7fad2014f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:51346/0 (socket says 172.21.15.66:51346) 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad17400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad200d49b0 0x7fad200c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:54628/0 (socket says 172.21.15.66:54628) 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad1fe00640 1 -- 172.21.15.66:0/2214792946 learned_addr learned my addr 172.21.15.66:0/2214792946 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad24a00640 1 -- 172.21.15.66:0/2214792946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad200d49b0 msgr2=0x7fad200c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad24a00640 1 --2- 172.21.15.66:0/2214792946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad200d49b0 0x7fad200c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad24a00640 1 -- 172.21.15.66:0/2214792946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad200b2e60 msgr2=0x7fad2014f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad24a00640 1 --2- 172.21.15.66:0/2214792946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad200b2e60 0x7fad2014f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:26.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad24a00640 1 -- 172.21.15.66:0/2214792946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad200c7ff0 con 0x7fad201611d0 2024-09-10T22:15:26.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad24a00640 1 --2- 172.21.15.66:0/2214792946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad201611d0 0x7fad2006e6b0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fad1800e960 tx=0x7fad1800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:26.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad1cc00640 1 -- 172.21.15.66:0/2214792946 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18004d30 con 0x7fad201611d0 2024-09-10T22:15:26.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad1cc00640 1 -- 172.21.15.66:0/2214792946 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad1800ca60 con 0x7fad201611d0 2024-09-10T22:15:26.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad1cc00640 1 -- 172.21.15.66:0/2214792946 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18005020 con 0x7fad201611d0 2024-09-10T22:15:26.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad26a00640 1 -- 172.21.15.66:0/2214792946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad200b7c20 con 0x7fad201611d0 2024-09-10T22:15:26.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.986+0000 7fad26a00640 1 -- 172.21.15.66:0/2214792946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad200c8290 con 0x7fad201611d0 2024-09-10T22:15:26.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.990+0000 7fad26a00640 1 -- 172.21.15.66:0/2214792946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad201615b0 con 0x7fad201611d0 2024-09-10T22:15:26.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.990+0000 7fad1cc00640 1 -- 172.21.15.66:0/2214792946 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fad1800f040 con 0x7fad201611d0 2024-09-10T22:15:26.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.990+0000 7fad1cc00640 1 --2- 172.21.15.66:0/2214792946 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7facfc074c20 0x7facfc0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:26.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.990+0000 7fad1cc00640 1 -- 172.21.15.66:0/2214792946 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5934+0+0 (secure 0 0 0) 0x7fad18092a70 con 0x7fad201611d0 2024-09-10T22:15:26.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.990+0000 7fad1fe00640 1 --2- 172.21.15.66:0/2214792946 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7facfc074c20 0x7facfc0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:26.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.990+0000 7fad1fe00640 1 --2- 172.21.15.66:0/2214792946 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7facfc074c20 0x7facfc0770e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fad200ca9c0 tx=0x7fad1001b100 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:27.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:26.994+0000 7fad1cc00640 1 -- 172.21.15.66:0/2214792946 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fad18016030 con 0x7fad201611d0 2024-09-10T22:15:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.210+0000 7fad26a00640 1 -- 172.21.15.66:0/2214792946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fad200b23c0 con 0x7fad201611d0 2024-09-10T22:15:27.215 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:15:27.210+0000 7fcc93a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:15:27.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad1cc00640 1 -- 172.21.15.66:0/2214792946 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v98) ==== 161+0+0 (secure 0 0 0) 0x7fad180682d0 con 0x7fad201611d0 2024-09-10T22:15:27.219 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:15:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad26a00640 1 -- 172.21.15.66:0/2214792946 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7facfc074c20 msgr2=0x7facfc0770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad26a00640 1 --2- 172.21.15.66:0/2214792946 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7facfc074c20 0x7facfc0770e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fad200ca9c0 tx=0x7fad1001b100 comp rx=0 tx=0).stop 2024-09-10T22:15:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad26a00640 1 -- 172.21.15.66:0/2214792946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad201611d0 msgr2=0x7fad2006e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad26a00640 1 --2- 172.21.15.66:0/2214792946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad201611d0 0x7fad2006e6b0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fad1800e960 tx=0x7fad1800ed60 comp rx=0 tx=0).stop 2024-09-10T22:15:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad26a00640 1 -- 172.21.15.66:0/2214792946 shutdown_connections 2024-09-10T22:15:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad26a00640 1 --2- 172.21.15.66:0/2214792946 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7facfc074c20 0x7facfc0770e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad26a00640 1 --2- 172.21.15.66:0/2214792946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad201611d0 0x7fad2006e6b0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad26a00640 1 --2- 172.21.15.66:0/2214792946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad200d49b0 0x7fad200c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad26a00640 1 --2- 172.21.15.66:0/2214792946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad200b2e60 0x7fad2014f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad26a00640 1 -- 172.21.15.66:0/2214792946 >> 172.21.15.66:0/2214792946 conn(0x7fad200d3e90 msgr2=0x7fad2015a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad26a00640 1 -- 172.21.15.66:0/2214792946 shutdown_connections 2024-09-10T22:15:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.214+0000 7fad26a00640 1 -- 172.21.15.66:0/2214792946 wait complete. 2024-09-10T22:15:27.243 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:15:27.289 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:15:27.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 -- 172.21.15.66:0/2290139547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b6015f340 msgr2=0x7f0b600cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 --2- 172.21.15.66:0/2290139547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b6015f340 0x7f0b600cfb50 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009900 tx=0x7f0b5001c850 comp rx=0 tx=0).stop 2024-09-10T22:15:27.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 -- 172.21.15.66:0/2290139547 shutdown_connections 2024-09-10T22:15:27.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 --2- 172.21.15.66:0/2290139547 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b600b2bf0 0x7f0b600d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 --2- 172.21.15.66:0/2290139547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b6015f340 0x7f0b600cfb50 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 -- 172.21.15.66:0/2290139547 >> 172.21.15.66:0/2290139547 conn(0x7f0b600d1c50 msgr2=0x7f0b600d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:27.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 -- 172.21.15.66:0/2290139547 shutdown_connections 2024-09-10T22:15:27.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 -- 172.21.15.66:0/2290139547 wait complete. 2024-09-10T22:15:27.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 Processor -- start 2024-09-10T22:15:27.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 -- start start 2024-09-10T22:15:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b600b2bf0 0x7f0b60157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b600d2770 0x7f0b6015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b6015f340 0x7f0b6015c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60169190 con 0x7f0b6015f340 2024-09-10T22:15:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60169010 con 0x7f0b600b2bf0 2024-09-10T22:15:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b66e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0b60169310 con 0x7f0b600d2770 2024-09-10T22:15:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b64e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b6015f340 0x7f0b6015c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b5fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b600b2bf0 0x7f0b60157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b57e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b600d2770 0x7f0b6015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b64e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b6015f340 0x7f0b6015c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49446/0 (socket says 172.21.15.66:49446) 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b64e00640 1 -- 172.21.15.66:0/2084914348 learned_addr learned my addr 172.21.15.66:0/2084914348 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.394+0000 7f0b5fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b600b2bf0 0x7f0b60157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:51372/0 (socket says 172.21.15.66:51372) 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b64e00640 1 -- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b600d2770 msgr2=0x7f0b6015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b64e00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b600d2770 0x7f0b6015bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b64e00640 1 -- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b600b2bf0 msgr2=0x7f0b60157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b64e00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b600b2bf0 0x7f0b60157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b64e00640 1 -- 172.21.15.66:0/2084914348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b600d2570 con 0x7f0b6015f340 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b57e00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b600d2770 0x7f0b6015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b64e00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b6015f340 0x7f0b6015c270 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800c7c0 tx=0x7f0b5800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b5cc00640 1 -- 172.21.15.66:0/2084914348 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58003b80 con 0x7f0b6015f340 2024-09-10T22:15:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b5cc00640 1 -- 172.21.15.66:0/2084914348 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b58003d20 con 0x7f0b6015f340 2024-09-10T22:15:27.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b5cc00640 1 -- 172.21.15.66:0/2084914348 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5800aab0 con 0x7f0b6015f340 2024-09-10T22:15:27.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b66e00640 1 -- 172.21.15.66:0/2084914348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b601f9580 con 0x7f0b6015f340 2024-09-10T22:15:27.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b66e00640 1 -- 172.21.15.66:0/2084914348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b601f9b40 con 0x7f0b6015f340 2024-09-10T22:15:27.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b66e00640 1 -- 172.21.15.66:0/2084914348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6015fac0 con 0x7f0b6015f340 2024-09-10T22:15:27.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.398+0000 7f0b5cc00640 1 -- 172.21.15.66:0/2084914348 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f0b5800ac50 con 0x7f0b6015f340 2024-09-10T22:15:27.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.402+0000 7f0b5cc00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b3c074c50 0x7f0b3c077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.402+0000 7f0b5cc00640 1 -- 172.21.15.66:0/2084914348 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5964+0+0 (secure 0 0 0) 0x7f0b58092300 con 0x7f0b6015f340 2024-09-10T22:15:27.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.402+0000 7f0b5fe00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b3c074c50 0x7f0b3c077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.402+0000 7f0b5fe00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b3c074c50 0x7f0b3c077110 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0b500af000 tx=0x7f0b5001b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:27.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.406+0000 7f0b5cc00640 1 -- 172.21.15.66:0/2084914348 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0b5805edf0 con 0x7f0b6015f340 2024-09-10T22:15:27.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:27.618+0000 7f0b66e00640 1 -- 172.21.15.66:0/2084914348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0b600b43c0 con 0x7f0b6015f340 2024-09-10T22:15:28.136 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:15:28.140 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:15:28.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.214+0000 7f0b5cc00640 1 -- 172.21.15.66:0/2084914348 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v99) ==== 104+0+0 (secure 0 0 0) 0x7f0b58063820 con 0x7f0b6015f340 2024-09-10T22:15:28.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.218+0000 7f0b66e00640 1 -- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b3c074c50 msgr2=0x7f0b3c077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:28.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.218+0000 7f0b66e00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b3c074c50 0x7f0b3c077110 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0b500af000 tx=0x7f0b5001b090 comp rx=0 tx=0).stop 2024-09-10T22:15:28.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.218+0000 7f0b66e00640 1 -- 172.21.15.66:0/2084914348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b6015f340 msgr2=0x7f0b6015c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:28.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.218+0000 7f0b66e00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b6015f340 0x7f0b6015c270 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800c7c0 tx=0x7f0b5800cc80 comp rx=0 tx=0).stop 2024-09-10T22:15:28.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.218+0000 7f0b66e00640 1 -- 172.21.15.66:0/2084914348 shutdown_connections 2024-09-10T22:15:28.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.218+0000 7f0b66e00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0b3c074c50 0x7f0b3c077110 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:28.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.218+0000 7f0b66e00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b6015f340 0x7f0b6015c270 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:28.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.218+0000 7f0b66e00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b600d2770 0x7f0b6015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:28.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.218+0000 7f0b66e00640 1 --2- 172.21.15.66:0/2084914348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b600b2bf0 0x7f0b60157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:28.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.218+0000 7f0b66e00640 1 -- 172.21.15.66:0/2084914348 >> 172.21.15.66:0/2084914348 conn(0x7f0b600d1c50 msgr2=0x7f0b600b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:28.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.218+0000 7f0b66e00640 1 -- 172.21.15.66:0/2084914348 shutdown_connections 2024-09-10T22:15:28.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.218+0000 7f0b66e00640 1 -- 172.21.15.66:0/2084914348 wait complete. 2024-09-10T22:15:28.249 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:15:28.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.442+0000 7f30fd000640 1 -- 172.21.15.66:0/986147944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f815f450 msgr2=0x7f30f80b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:28.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.442+0000 7f30fd000640 1 --2- 172.21.15.66:0/986147944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f815f450 0x7f30f80b3170 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f30f80d13f0 tx=0x7f30e801d290 comp rx=0 tx=0).stop 2024-09-10T22:15:28.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.442+0000 7f30fd000640 1 -- 172.21.15.66:0/986147944 shutdown_connections 2024-09-10T22:15:28.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.442+0000 7f30fd000640 1 --2- 172.21.15.66:0/986147944 >> v2:172.21.15.94:3300/0 conn(0x7f30f80b1e10 0x7f30f815fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:28.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.442+0000 7f30fd000640 1 --2- 172.21.15.66:0/986147944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f815f450 0x7f30f80b3170 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:28.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.442+0000 7f30fd000640 1 -- 172.21.15.66:0/986147944 >> 172.21.15.66:0/986147944 conn(0x7f30f80d1c60 msgr2=0x7f30f80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:28.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.442+0000 7f30fd000640 1 -- 172.21.15.66:0/986147944 shutdown_connections 2024-09-10T22:15:28.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.442+0000 7f30fd000640 1 -- 172.21.15.66:0/986147944 wait complete. 2024-09-10T22:15:28.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.442+0000 7f30fd000640 1 Processor -- start 2024-09-10T22:15:28.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.442+0000 7f30fd000640 1 -- start start 2024-09-10T22:15:28.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30fd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f80b1e10 0x7f30f806e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:28.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30fd000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f30f815f450 0x7f30f81f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:28.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30fd000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30f81600b0 0x7f30f80b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:28.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30fd000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30f81692a0 con 0x7f30f80b1e10 2024-09-10T22:15:28.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30fd000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f30f8169120 con 0x7f30f81600b0 2024-09-10T22:15:28.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30fd000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f30f8169420 con 0x7f30f815f450 2024-09-10T22:15:28.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f80b1e10 0x7f30f806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:28.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f80b1e10 0x7f30f806e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49466/0 (socket says 172.21.15.66:49466) 2024-09-10T22:15:28.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30f5e00640 1 -- 172.21.15.66:0/1388200098 learned_addr learned my addr 172.21.15.66:0/1388200098 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:28.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30efe00640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f30f815f450 0x7f30f81f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:28.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30f6800640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30f81600b0 0x7f30f80b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:28.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30f5e00640 1 -- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f30f815f450 msgr2=0x7f30f81f8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:28.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30f5e00640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f30f815f450 0x7f30f81f8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:28.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30f5e00640 1 -- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30f81600b0 msgr2=0x7f30f80b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:28.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30f5e00640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30f81600b0 0x7f30f80b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:28.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30f5e00640 1 -- 172.21.15.66:0/1388200098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f81f8880 con 0x7f30f80b1e10 2024-09-10T22:15:28.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30f5e00640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f80b1e10 0x7f30f806e950 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f30e80b0040 tx=0x7f30e8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:28.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30ee000640 1 -- 172.21.15.66:0/1388200098 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e80bb070 con 0x7f30f80b1e10 2024-09-10T22:15:28.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30ee000640 1 -- 172.21.15.66:0/1388200098 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30e80043f0 con 0x7f30f80b1e10 2024-09-10T22:15:28.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30ee000640 1 -- 172.21.15.66:0/1388200098 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e80b2040 con 0x7f30f80b1e10 2024-09-10T22:15:28.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30fd000640 1 -- 172.21.15.66:0/1388200098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f81f8b60 con 0x7f30f80b1e10 2024-09-10T22:15:28.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30fd000640 1 -- 172.21.15.66:0/1388200098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f81f8580 con 0x7f30f80b1e10 2024-09-10T22:15:28.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30fd000640 1 -- 172.21.15.66:0/1388200098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f815fb70 con 0x7f30f80b1e10 2024-09-10T22:15:28.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.446+0000 7f30ee000640 1 -- 172.21.15.66:0/1388200098 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f30e80052d0 con 0x7f30f80b1e10 2024-09-10T22:15:28.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.450+0000 7f30ee000640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f30d4074c20 0x7f30d40770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:28.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.450+0000 7f30ee000640 1 -- 172.21.15.66:0/1388200098 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5964+0+0 (secure 0 0 0) 0x7f30e812bb50 con 0x7f30f80b1e10 2024-09-10T22:15:28.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.450+0000 7f30efe00640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f30d4074c20 0x7f30d40770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:28.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.450+0000 7f30efe00640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f30d4074c20 0x7f30d40770e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f30f815b0d0 tx=0x7f30e00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:28.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.454+0000 7f30ee000640 1 -- 172.21.15.66:0/1388200098 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f30e80164b0 con 0x7f30f80b1e10 2024-09-10T22:15:28.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:28.630+0000 7f30fd000640 1 -- 172.21.15.66:0/1388200098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f30f80d2a40 con 0x7f30f80b1e10 2024-09-10T22:15:29.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.222+0000 7f30ee000640 1 -- 172.21.15.66:0/1388200098 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) ==== 181+0+0 (secure 0 0 0) 0x7f30e80f8640 con 0x7f30f80b1e10 2024-09-10T22:15:29.228 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:15:29.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.226+0000 7f30fd000640 1 -- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f30d4074c20 msgr2=0x7f30d40770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:29.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.226+0000 7f30fd000640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f30d4074c20 0x7f30d40770e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f30f815b0d0 tx=0x7f30e00091e0 comp rx=0 tx=0).stop 2024-09-10T22:15:29.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.226+0000 7f30fd000640 1 -- 172.21.15.66:0/1388200098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f80b1e10 msgr2=0x7f30f806e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:29.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.226+0000 7f30fd000640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f80b1e10 0x7f30f806e950 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f30e80b0040 tx=0x7f30e8005e70 comp rx=0 tx=0).stop 2024-09-10T22:15:29.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.226+0000 7f30fd000640 1 -- 172.21.15.66:0/1388200098 shutdown_connections 2024-09-10T22:15:29.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.226+0000 7f30fd000640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f30d4074c20 0x7f30d40770e0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:29.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.226+0000 7f30fd000640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30f81600b0 0x7f30f80b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:29.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.226+0000 7f30fd000640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f30f815f450 0x7f30f81f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:29.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.226+0000 7f30fd000640 1 --2- 172.21.15.66:0/1388200098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30f80b1e10 0x7f30f806e950 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:29.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.226+0000 7f30fd000640 1 -- 172.21.15.66:0/1388200098 >> 172.21.15.66:0/1388200098 conn(0x7f30f80d1c60 msgr2=0x7f30f815a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:29.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.226+0000 7f30fd000640 1 -- 172.21.15.66:0/1388200098 shutdown_connections 2024-09-10T22:15:29.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.226+0000 7f30fd000640 1 -- 172.21.15.66:0/1388200098 wait complete. 2024-09-10T22:15:29.257 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:15:29.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.406+0000 7f6dc2800640 1 -- 172.21.15.66:0/4072509485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dbc1605c0 msgr2=0x7f6dbc0b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:29.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.406+0000 7f6dc2800640 1 --2- 172.21.15.66:0/4072509485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dbc1605c0 0x7f6dbc0b1620 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009840 tx=0x7f6dac01c740 comp rx=0 tx=0).stop 2024-09-10T22:15:29.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.406+0000 7f6dc2800640 1 -- 172.21.15.66:0/4072509485 shutdown_connections 2024-09-10T22:15:29.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.406+0000 7f6dc2800640 1 --2- 172.21.15.66:0/4072509485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dbc1605c0 0x7f6dbc0b1620 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:29.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.406+0000 7f6dc2800640 1 -- 172.21.15.66:0/4072509485 >> 172.21.15.66:0/4072509485 conn(0x7f6dbc0d1ce0 msgr2=0x7f6dbc0d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:29.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.406+0000 7f6dc2800640 1 -- 172.21.15.66:0/4072509485 shutdown_connections 2024-09-10T22:15:29.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.406+0000 7f6dc2800640 1 -- 172.21.15.66:0/4072509485 wait complete. 2024-09-10T22:15:29.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dc2800640 1 Processor -- start 2024-09-10T22:15:29.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dc2800640 1 -- start start 2024-09-10T22:15:29.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dc2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dbc0b1b60 0x7f6dbc14fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:29.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dc2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dbc0b2b10 0x7f6dbc0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:29.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dc2800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6dbc1605c0 0x7f6dbc0af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:29.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dc2800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc16a700 con 0x7f6dbc0b2b10 2024-09-10T22:15:29.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dc2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc0b1400 con 0x7f6dbc0b1b60 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dc2800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6dbc16a880 con 0x7f6dbc1605c0 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6db3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dbc0b2b10 0x7f6dbc0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6db3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dbc0b2b10 0x7f6dbc0aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49468/0 (socket says 172.21.15.66:49468) 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6db3e00640 1 -- 172.21.15.66:0/3148358532 learned_addr learned my addr 172.21.15.66:0/3148358532 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dbb400640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dbc0b1b60 0x7f6dbc14fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dbbe00640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6dbc1605c0 0x7f6dbc0af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6db3e00640 1 -- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6dbc1605c0 msgr2=0x7f6dbc0af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6db3e00640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6dbc1605c0 0x7f6dbc0af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6db3e00640 1 -- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dbc0b1b60 msgr2=0x7f6dbc14fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6db3e00640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dbc0b1b60 0x7f6dbc14fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6db3e00640 1 -- 172.21.15.66:0/3148358532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc0b73b0 con 0x7f6dbc0b2b10 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dbb400640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dbc0b1b60 0x7f6dbc14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dbbe00640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6dbc1605c0 0x7f6dbc0af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:29.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6db3e00640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dbc0b2b10 0x7f6dbc0aeb10 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f6da400c4b0 tx=0x7f6da400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:29.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6db3400640 1 -- 172.21.15.66:0/3148358532 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da40052d0 con 0x7f6dbc0b2b10 2024-09-10T22:15:29.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6db3400640 1 -- 172.21.15.66:0/3148358532 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6da4005470 con 0x7f6dbc0b2b10 2024-09-10T22:15:29.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6db3400640 1 -- 172.21.15.66:0/3148358532 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4012870 con 0x7f6dbc0b2b10 2024-09-10T22:15:29.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dc2800640 1 -- 172.21.15.66:0/3148358532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dbc155b50 con 0x7f6dbc0b2b10 2024-09-10T22:15:29.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.410+0000 7f6dc2800640 1 -- 172.21.15.66:0/3148358532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dbc0b7930 con 0x7f6dbc0b2b10 2024-09-10T22:15:29.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.414+0000 7f6dc2800640 1 -- 172.21.15.66:0/3148358532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dbc0b1620 con 0x7f6dbc0b2b10 2024-09-10T22:15:29.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.414+0000 7f6db3400640 1 -- 172.21.15.66:0/3148358532 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f6da4003600 con 0x7f6dbc0b2b10 2024-09-10T22:15:29.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.414+0000 7f6db3400640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6d98074c20 0x7f6d980770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:29.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.414+0000 7f6db3400640 1 -- 172.21.15.66:0/3148358532 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5526+0+0 (secure 0 0 0) 0x7f6da4092170 con 0x7f6dbc0b2b10 2024-09-10T22:15:29.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.414+0000 7f6dbb400640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6d98074c20 0x7f6d980770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:29.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.414+0000 7f6dbb400640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6d98074c20 0x7f6d980770e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0af000 tx=0x7f6dac01af80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.418+0000 7f6db3400640 1 -- 172.21.15.66:0/3148358532 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6da405f230 con 0x7f6dbc0b2b10 2024-09-10T22:15:29.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:29.594+0000 7f6dc2800640 1 -- 172.21.15.66:0/3148358532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6dbc0c87e0 con 0x7f6dbc0b2b10 2024-09-10T22:15:30.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.222+0000 7f6db3400640 1 -- 172.21.15.66:0/3148358532 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) ==== 169+0+0 (secure 0 0 0) 0x7f6da405eb30 con 0x7f6dbc0b2b10 2024-09-10T22:15:30.230 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:15:30.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.226+0000 7f6dc2800640 1 -- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6d98074c20 msgr2=0x7f6d980770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:30.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.226+0000 7f6dc2800640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6d98074c20 0x7f6d980770e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0af000 tx=0x7f6dac01af80 comp rx=0 tx=0).stop 2024-09-10T22:15:30.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.226+0000 7f6dc2800640 1 -- 172.21.15.66:0/3148358532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dbc0b2b10 msgr2=0x7f6dbc0aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:30.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.226+0000 7f6dc2800640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dbc0b2b10 0x7f6dbc0aeb10 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f6da400c4b0 tx=0x7f6da400c970 comp rx=0 tx=0).stop 2024-09-10T22:15:30.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.226+0000 7f6dc2800640 1 -- 172.21.15.66:0/3148358532 shutdown_connections 2024-09-10T22:15:30.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.226+0000 7f6dc2800640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f6d98074c20 0x7f6d980770e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.226+0000 7f6dc2800640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6dbc1605c0 0x7f6dbc0af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.226+0000 7f6dc2800640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dbc0b2b10 0x7f6dbc0aeb10 secure :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f6da400c4b0 tx=0x7f6da400c970 comp rx=0 tx=0).stop 2024-09-10T22:15:30.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.226+0000 7f6dc2800640 1 --2- 172.21.15.66:0/3148358532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dbc0b1b60 0x7f6dbc14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.226+0000 7f6dc2800640 1 -- 172.21.15.66:0/3148358532 >> 172.21.15.66:0/3148358532 conn(0x7f6dbc0d1ce0 msgr2=0x7f6dbc150c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:30.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.230+0000 7f6dc2800640 1 -- 172.21.15.66:0/3148358532 shutdown_connections 2024-09-10T22:15:30.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.230+0000 7f6dc2800640 1 -- 172.21.15.66:0/3148358532 wait complete. 2024-09-10T22:15:30.264 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T22:15:30.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.410+0000 7fa001800640 1 -- 172.21.15.66:0/421322239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe0004e70 msgr2=0x7f9fe000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:30.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.410+0000 7fa001800640 1 --2- 172.21.15.66:0/421322239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe0004e70 0x7f9fe000b9d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400c590 tx=0x7f9ff401d2a0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.410+0000 7f9ffb400640 1 -- 172.21.15.66:0/421322239 reap_dead start 2024-09-10T22:15:30.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.410+0000 7fa001800640 1 -- 172.21.15.66:0/421322239 shutdown_connections 2024-09-10T22:15:30.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.410+0000 7fa001800640 1 -- 172.21.15.66:0/421322239 >> 172.21.15.66:0/421322239 conn(0x7f9ffc0d1c50 msgr2=0x7f9ffc0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:30.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.410+0000 7fa001800640 1 -- 172.21.15.66:0/421322239 shutdown_connections 2024-09-10T22:15:30.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.410+0000 7fa001800640 1 -- 172.21.15.66:0/421322239 wait complete. 2024-09-10T22:15:30.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.410+0000 7fa001800640 1 Processor -- start 2024-09-10T22:15:30.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.410+0000 7fa001800640 1 -- start start 2024-09-10T22:15:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7fa001800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe0004e70 0x7f9ffc14e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7fa001800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ffc1513f0 0x7f9ffc1f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7fa001800640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ffc0aebc0 0x7f9ffc1f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7fa001800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc15f8a0 con 0x7f9fe0004e70 2024-09-10T22:15:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7fa001800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc15f720 con 0x7f9ffc1513f0 2024-09-10T22:15:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7fa001800640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9ffc0d2770 con 0x7f9ffc0aebc0 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ffaa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe0004e70 0x7f9ffc14e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ffaa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe0004e70 0x7f9ffc14e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49482/0 (socket says 172.21.15.66:49482) 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ffaa00640 1 -- 172.21.15.66:0/2940981406 learned_addr learned my addr 172.21.15.66:0/2940981406 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ff3e00640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ffc1513f0 0x7f9ffc1f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ffb400640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ffc0aebc0 0x7f9ffc1f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ffaa00640 1 -- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ffc0aebc0 msgr2=0x7f9ffc1f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ffaa00640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ffc0aebc0 0x7f9ffc1f1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ffaa00640 1 -- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ffc1513f0 msgr2=0x7f9ffc1f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ffaa00640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ffc1513f0 0x7f9ffc1f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ffaa00640 1 -- 172.21.15.66:0/2940981406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc14e880 con 0x7f9fe0004e70 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ffaa00640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe0004e70 0x7f9ffc14e340 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f9fec003f90 tx=0x7f9fec00ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ff2a00640 1 -- 172.21.15.66:0/2940981406 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec005220 con 0x7f9fe0004e70 2024-09-10T22:15:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ff2a00640 1 -- 172.21.15.66:0/2940981406 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fec0053c0 con 0x7f9fe0004e70 2024-09-10T22:15:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7f9ff2a00640 1 -- 172.21.15.66:0/2940981406 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0047b0 con 0x7f9fe0004e70 2024-09-10T22:15:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7fa001800640 1 -- 172.21.15.66:0/2940981406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ffc0b7520 con 0x7f9fe0004e70 2024-09-10T22:15:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7fa001800640 1 -- 172.21.15.66:0/2940981406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ffc0b77c0 con 0x7f9fe0004e70 2024-09-10T22:15:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.414+0000 7fa001800640 1 -- 172.21.15.66:0/2940981406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ffc1f62b0 con 0x7f9fe0004e70 2024-09-10T22:15:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.418+0000 7f9ff2a00640 1 -- 172.21.15.66:0/2940981406 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f9fec022030 con 0x7f9fe0004e70 2024-09-10T22:15:30.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.418+0000 7f9ff2a00640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9fd8074c50 0x7f9fd8077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:30.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.418+0000 7f9ff2a00640 1 -- 172.21.15.66:0/2940981406 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f9fec092c80 con 0x7f9fe0004e70 2024-09-10T22:15:30.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.418+0000 7f9ff3e00640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9fd8074c50 0x7f9fd8077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:30.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.418+0000 7f9ff3e00640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9fd8074c50 0x7f9fd8077110 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0af5f0 tx=0x7f9fe4006b80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.422+0000 7f9ff2a00640 1 -- 172.21.15.66:0/2940981406 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9ffc14e880 con 0x7f9fe0004e70 2024-09-10T22:15:30.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.610+0000 7fa001800640 1 -- 172.21.15.66:0/2940981406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f9ffc15e8c0 con 0x7f9fe0004e70 2024-09-10T22:15:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.614+0000 7f9ff2a00640 1 -- 172.21.15.66:0/2940981406 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v57) ==== 71+0+0 (secure 0 0 0) 0x7f9fec05f770 con 0x7f9fe0004e70 2024-09-10T22:15:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.618+0000 7fa001800640 1 -- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9fd8074c50 msgr2=0x7f9fd8077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.618+0000 7fa001800640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9fd8074c50 0x7f9fd8077110 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0af5f0 tx=0x7f9fe4006b80 comp rx=0 tx=0).stop 2024-09-10T22:15:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.618+0000 7fa001800640 1 -- 172.21.15.66:0/2940981406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe0004e70 msgr2=0x7f9ffc14e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.618+0000 7fa001800640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe0004e70 0x7f9ffc14e340 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f9fec003f90 tx=0x7f9fec00ba80 comp rx=0 tx=0).stop 2024-09-10T22:15:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.618+0000 7fa001800640 1 -- 172.21.15.66:0/2940981406 shutdown_connections 2024-09-10T22:15:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.618+0000 7fa001800640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9fd8074c50 0x7f9fd8077110 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.618+0000 7fa001800640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ffc0aebc0 0x7f9ffc1f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.618+0000 7fa001800640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ffc1513f0 0x7f9ffc1f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.618+0000 7fa001800640 1 --2- 172.21.15.66:0/2940981406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe0004e70 0x7f9ffc14e340 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.618+0000 7fa001800640 1 -- 172.21.15.66:0/2940981406 >> 172.21.15.66:0/2940981406 conn(0x7f9ffc0d1c50 msgr2=0x7f9ffc0c8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:30.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.618+0000 7fa001800640 1 -- 172.21.15.66:0/2940981406 shutdown_connections 2024-09-10T22:15:30.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.618+0000 7fa001800640 1 -- 172.21.15.66:0/2940981406 wait complete. 2024-09-10T22:15:30.643 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-10T22:15:30.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.786+0000 7fbcec600640 1 -- 172.21.15.66:0/106693838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbce415f340 msgr2=0x7fbce40cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:30.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.786+0000 7fbcec600640 1 --2- 172.21.15.66:0/106693838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbce415f340 0x7fbce40cfb50 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009900 tx=0x7fbcd801c8a0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.786+0000 7fbcec600640 1 -- 172.21.15.66:0/106693838 shutdown_connections 2024-09-10T22:15:30.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.786+0000 7fbcec600640 1 --2- 172.21.15.66:0/106693838 >> v2:172.21.15.94:3300/0 conn(0x7fbce40b2bf0 0x7fbce40d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.786+0000 7fbcec600640 1 --2- 172.21.15.66:0/106693838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbce415f340 0x7fbce40cfb50 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.786+0000 7fbcec600640 1 -- 172.21.15.66:0/106693838 >> 172.21.15.66:0/106693838 conn(0x7fbce40d1c50 msgr2=0x7fbce40d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:30.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.786+0000 7fbcec600640 1 -- 172.21.15.66:0/106693838 shutdown_connections 2024-09-10T22:15:30.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.786+0000 7fbcec600640 1 -- 172.21.15.66:0/106693838 wait complete. 2024-09-10T22:15:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.786+0000 7fbcec600640 1 Processor -- start 2024-09-10T22:15:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.786+0000 7fbcec600640 1 -- start start 2024-09-10T22:15:30.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.786+0000 7fbcec600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbce40b2bf0 0x7fbce414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:30.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.786+0000 7fbcec600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce40d2770 0x7fbce4156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:30.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcec600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbce415f340 0x7fbce4156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:30.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcec600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbce4169190 con 0x7fbce415f340 2024-09-10T22:15:30.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcec600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbce4169010 con 0x7fbce40d2770 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcec600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbce4169310 con 0x7fbce40b2bf0 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcea600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbce415f340 0x7fbce4156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcea600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbce415f340 0x7fbce4156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49494/0 (socket says 172.21.15.66:49494) 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcea600640 1 -- 172.21.15.66:0/3556931582 learned_addr learned my addr 172.21.15.66:0/3556931582 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbce9200640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce40d2770 0x7fbce4156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbce9c00640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbce40b2bf0 0x7fbce414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcea600640 1 -- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbce40b2bf0 msgr2=0x7fbce414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcea600640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbce40b2bf0 0x7fbce414fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcea600640 1 -- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce40d2770 msgr2=0x7fbce4156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcea600640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce40d2770 0x7fbce4156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcea600640 1 -- 172.21.15.66:0/3556931582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce41fb440 con 0x7fbce415f340 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbce9200640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce40d2770 0x7fbce4156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:30.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcea600640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbce415f340 0x7fbce4156d60 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00c730 tx=0x7fbcdc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:30.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbce1600640 1 -- 172.21.15.66:0/3556931582 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc003db0 con 0x7fbce415f340 2024-09-10T22:15:30.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbce1600640 1 -- 172.21.15.66:0/3556931582 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcdc00ce40 con 0x7fbce415f340 2024-09-10T22:15:30.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbce1600640 1 -- 172.21.15.66:0/3556931582 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0052d0 con 0x7fbce415f340 2024-09-10T22:15:30.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcec600640 1 -- 172.21.15.66:0/3556931582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce41fb720 con 0x7fbce415f340 2024-09-10T22:15:30.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcec600640 1 -- 172.21.15.66:0/3556931582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce41fbce0 con 0x7fbce415f340 2024-09-10T22:15:30.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbcec600640 1 -- 172.21.15.66:0/3556931582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce41fc0c0 con 0x7fbce415f340 2024-09-10T22:15:30.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.790+0000 7fbce1600640 1 -- 172.21.15.66:0/3556931582 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fbcdc005470 con 0x7fbce415f340 2024-09-10T22:15:30.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.794+0000 7fbce1600640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbcc4074c50 0x7fbcc4077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:30.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.794+0000 7fbce1600640 1 -- 172.21.15.66:0/3556931582 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7fbcdc092340 con 0x7fbce415f340 2024-09-10T22:15:30.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.794+0000 7fbce9c00640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbcc4074c50 0x7fbcc4077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:30.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.794+0000 7fbce9c00640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbcc4074c50 0x7fbcc4077110 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80af000 tx=0x7fbcd801b0e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:30.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:30.798+0000 7fbce1600640 1 -- 172.21.15.66:0/3556931582 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbce41fc0c0 con 0x7fbce415f340 2024-09-10T22:15:31.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.034+0000 7fbcec600640 1 -- 172.21.15.66:0/3556931582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7fbce415fac0 con 0x7fbce415f340 2024-09-10T22:15:31.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbce1600640 1 -- 172.21.15.66:0/3556931582 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v685) ==== 186+0+0 (secure 0 0 0) 0x7fbcdc05f520 con 0x7fbce415f340 2024-09-10T22:15:31.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbcec600640 1 -- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbcc4074c50 msgr2=0x7fbcc4077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:31.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbcec600640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbcc4074c50 0x7fbcc4077110 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80af000 tx=0x7fbcd801b0e0 comp rx=0 tx=0).stop 2024-09-10T22:15:31.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbcec600640 1 -- 172.21.15.66:0/3556931582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbce415f340 msgr2=0x7fbce4156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:31.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbcec600640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbce415f340 0x7fbce4156d60 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00c730 tx=0x7fbcdc00cbf0 comp rx=0 tx=0).stop 2024-09-10T22:15:31.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbcec600640 1 -- 172.21.15.66:0/3556931582 shutdown_connections 2024-09-10T22:15:31.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbcec600640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fbcc4074c50 0x7fbcc4077110 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:31.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbcec600640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbce415f340 0x7fbce4156d60 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:31.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbcec600640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce40d2770 0x7fbce4156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:31.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbcec600640 1 --2- 172.21.15.66:0/3556931582 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbce40b2bf0 0x7fbce414fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:31.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbcec600640 1 -- 172.21.15.66:0/3556931582 >> 172.21.15.66:0/3556931582 conn(0x7fbce40d1c50 msgr2=0x7fbce40721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:31.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbcec600640 1 -- 172.21.15.66:0/3556931582 shutdown_connections 2024-09-10T22:15:31.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:31.234+0000 7fbcec600640 1 -- 172.21.15.66:0/3556931582 wait complete. 2024-09-10T22:15:31.262 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone ... ok 2024-09-10T22:15:31.263 INFO:tasks.cephfs_test_runner: 2024-09-10T22:15:31.263 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T22:15:31.263 INFO:tasks.cephfs_test_runner:Ran 9 tests in 871.715s 2024-09-10T22:15:31.263 INFO:tasks.cephfs_test_runner: 2024-09-10T22:15:31.263 INFO:tasks.cephfs_test_runner:OK 2024-09-10T22:15:31.263 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-10T22:15:31.272 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-10T22:15:31.360 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-10T22:15:31.361 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:15:31.361 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:15:31.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:15:31.370 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:15:31.370 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:15:31.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350176k,nr_inodes=4087544,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=164 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:15:31.422 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:15:31.422 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:15:31.432 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:15:31.432 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:15:31.432 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:15:31.482 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4161 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:15:31.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:15:31.482 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:15:31.490 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:15:31.490 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:15:31.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:15:31.492 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:15:31.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:15:31.558 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:15:31.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:15:31.932 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-09-10T22:15:32.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:15:32.072 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T22:15:32.082 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T22:15:32.082 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T22:15:32.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:15:32.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:15:32.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:15:32.148 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T22:15:32.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:15:32.622 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-09-10T22:15:32.752 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T22:15:32.762 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:15:32.762 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:15:32.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.906+0000 7fc1b3e00640 1 -- 172.21.15.66:0/987433328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b40cd7a0 msgr2=0x7fc1b40cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:32.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.906+0000 7fc1b3e00640 1 --2- 172.21.15.66:0/987433328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b40cd7a0 0x7fc1b40cdc30 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009c80 tx=0x7fc1ac01cda0 comp rx=0 tx=0).stop 2024-09-10T22:15:32.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.906+0000 7fc1b3e00640 1 -- 172.21.15.66:0/987433328 shutdown_connections 2024-09-10T22:15:32.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.906+0000 7fc1b3e00640 1 --2- 172.21.15.66:0/987433328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b40cd7a0 0x7fc1b40cdc30 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:32.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.906+0000 7fc1b3e00640 1 --2- 172.21.15.66:0/987433328 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1b415ef20 0x7fc1b40cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:32.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.906+0000 7fc1b3e00640 1 -- 172.21.15.66:0/987433328 >> 172.21.15.66:0/987433328 conn(0x7fc1b40d1b10 msgr2=0x7fc1b40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:32.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.906+0000 7fc1b3e00640 1 -- 172.21.15.66:0/987433328 shutdown_connections 2024-09-10T22:15:32.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.906+0000 7fc1b3e00640 1 -- 172.21.15.66:0/987433328 wait complete. 2024-09-10T22:15:32.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3e00640 1 Processor -- start 2024-09-10T22:15:32.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3e00640 1 -- start start 2024-09-10T22:15:32.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b40cd7a0 0x7fc1b41f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:32.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1b415ef20 0x7fc1b41660f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:32.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b415f300 0x7fc1b4155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:32.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4168ee0 con 0x7fc1b415f300 2024-09-10T22:15:32.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4168d60 con 0x7fc1b40cd7a0 2024-09-10T22:15:32.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b4169060 con 0x7fc1b415ef20 2024-09-10T22:15:32.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b415f300 0x7fc1b4155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b415f300 0x7fc1b4155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53448/0 (socket says 172.21.15.66:53448) 2024-09-10T22:15:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3400640 1 -- 172.21.15.66:0/2992519982 learned_addr learned my addr 172.21.15.66:0/2992519982 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b2a00640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b40cd7a0 0x7fc1b41f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1aa000640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1b415ef20 0x7fc1b41660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3400640 1 -- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1b415ef20 msgr2=0x7fc1b41660f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3400640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1b415ef20 0x7fc1b41660f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3400640 1 -- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b40cd7a0 msgr2=0x7fc1b41f8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3400640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b40cd7a0 0x7fc1b41f8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3400640 1 -- 172.21.15.66:0/2992519982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b4166630 con 0x7fc1b415f300 2024-09-10T22:15:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1aa000640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1b415ef20 0x7fc1b41660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3400640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b415f300 0x7fc1b4155be0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac01b450 tx=0x7fc1ac0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1ab400640 1 -- 172.21.15.66:0/2992519982 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0bb070 con 0x7fc1b415f300 2024-09-10T22:15:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1ab400640 1 -- 172.21.15.66:0/2992519982 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1ac004d40 con 0x7fc1b415f300 2024-09-10T22:15:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1ab400640 1 -- 172.21.15.66:0/2992519982 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0b2040 con 0x7fc1b415f300 2024-09-10T22:15:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3e00640 1 -- 172.21.15.66:0/2992519982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b406f910 con 0x7fc1b415f300 2024-09-10T22:15:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3e00640 1 -- 172.21.15.66:0/2992519982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b41f8140 con 0x7fc1b415f300 2024-09-10T22:15:32.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.910+0000 7fc1b3e00640 1 -- 172.21.15.66:0/2992519982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b40cdc30 con 0x7fc1b415f300 2024-09-10T22:15:32.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.914+0000 7fc1ab400640 1 -- 172.21.15.66:0/2992519982 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fc1ac0b5030 con 0x7fc1b415f300 2024-09-10T22:15:32.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.914+0000 7fc1ab400640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc190074c20 0x7fc1900770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:32.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.914+0000 7fc1ab400640 1 -- 172.21.15.66:0/2992519982 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7fc1ac12bfb0 con 0x7fc1b415f300 2024-09-10T22:15:32.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.914+0000 7fc1b2a00640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc190074c20 0x7fc1900770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:32.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.914+0000 7fc1b2a00640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc190074c20 0x7fc1900770e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4070eb0 tx=0x7fc1a40091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:32.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:32.918+0000 7fc1ab400640 1 -- 172.21.15.66:0/2992519982 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc1ac0f1de0 con 0x7fc1b415f300 2024-09-10T22:15:33.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.086+0000 7fc1b3e00640 1 -- 172.21.15.66:0/2992519982 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1b40d1380 con 0x7fc190074c20 2024-09-10T22:15:33.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1ab400640 1 -- 172.21.15.66:0/2992519982 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13555 (secure 0 0 0) 0x7fc1a401b3f0 con 0x7fc190074c20 2024-09-10T22:15:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:33.095 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:15:33.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1b3e00640 1 -- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc190074c20 msgr2=0x7fc1900770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1b3e00640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc190074c20 0x7fc1900770e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4070eb0 tx=0x7fc1a40091e0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1b3e00640 1 -- 172.21.15.66:0/2992519982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b415f300 msgr2=0x7fc1b4155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1b3e00640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b415f300 0x7fc1b4155be0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac01b450 tx=0x7fc1ac0092e0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1b3e00640 1 -- 172.21.15.66:0/2992519982 shutdown_connections 2024-09-10T22:15:33.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1b3e00640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc190074c20 0x7fc1900770e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1b3e00640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b415f300 0x7fc1b4155be0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1b3e00640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1b415ef20 0x7fc1b41660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1b3e00640 1 --2- 172.21.15.66:0/2992519982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b40cd7a0 0x7fc1b41f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1b3e00640 1 -- 172.21.15.66:0/2992519982 >> 172.21.15.66:0/2992519982 conn(0x7fc1b40d1b10 msgr2=0x7fc1b40aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:33.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1b3e00640 1 -- 172.21.15.66:0/2992519982 shutdown_connections 2024-09-10T22:15:33.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.090+0000 7fc1b3e00640 1 -- 172.21.15.66:0/2992519982 wait complete. 2024-09-10T22:15:33.116 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":597,"stamp":"2024-09-10T22:15:32.643794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1210040,"kb_used_data":15608,"kb_used_omap":12,"kb_used_meta":1194227,"kb_avail":753764680,"statfs":{"total":773094113280,"available":771855032320,"internally_reserved":0,"allocated":15982592,"data_stored":7716126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1222889012},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17685122517,"num_objects":-4245,"num_object_clones":0,"num_object_copies":-8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4245,"num_whiteouts":0,"num_read":-4146,"num_read_kb":-16597086,"num_write":-6112,"num_write_kb":-19654791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.630644"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":353,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T22:15:30.238266+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:15:30.238266+0000","last_peered":"2024-09-10T22:15:30.238266+0000","last_clean":"2024-09-10T22:15:30.238266+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:15:30.238266+0000","last_undegraded":"2024-09-10T22:15:30.238266+0000","last_fullsized":"2024-09-10T22:15:30.238266+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509625,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145964,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225876,"statfs":{"total":96636764160,"available":96487297024,"internally_reserved":0,"allocated":2445312,"data_stored":1408007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509626,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137064,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234776,"statfs":{"total":96636764160,"available":96496410624,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509625,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137064,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234776,"statfs":{"total":96636764160,"available":96496410624,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509626,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167592,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204248,"statfs":{"total":96636764160,"available":96465149952,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509624,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163432,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208408,"statfs":{"total":96636764160,"available":96469409792,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":165476807},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509625,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149544,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222296,"statfs":{"total":96636764160,"available":96483631104,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509625,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145964,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225876,"statfs":{"total":96636764160,"available":96487297024,"internally_reserved":0,"allocated":2445312,"data_stored":1408007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":146995657},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542330,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163416,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208424,"statfs":{"total":96636764160,"available":96469426176,"internally_reserved":0,"allocated":1835008,"data_stored":811917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:15:33.117 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:15:33.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.258+0000 7f9b76000640 1 -- 172.21.15.66:0/3878412347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7015a310 msgr2=0x7f9b700c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.258+0000 7f9b76000640 1 --2- 172.21.15.66:0/3878412347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7015a310 0x7f9b700c48b0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f9b700af550 tx=0x7f9b6001db60 comp rx=0 tx=0).stop 2024-09-10T22:15:33.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.258+0000 7f9b76000640 1 -- 172.21.15.66:0/3878412347 shutdown_connections 2024-09-10T22:15:33.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.258+0000 7f9b76000640 1 --2- 172.21.15.66:0/3878412347 >> v2:172.21.15.94:3300/0 conn(0x7f9b700aeec0 0x7f9b700c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.258+0000 7f9b76000640 1 --2- 172.21.15.66:0/3878412347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7015a310 0x7f9b700c48b0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.258+0000 7f9b76000640 1 -- 172.21.15.66:0/3878412347 >> 172.21.15.66:0/3878412347 conn(0x7f9b700cfa30 msgr2=0x7f9b700cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:33.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.258+0000 7f9b76000640 1 -- 172.21.15.66:0/3878412347 shutdown_connections 2024-09-10T22:15:33.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.258+0000 7f9b76000640 1 -- 172.21.15.66:0/3878412347 wait complete. 2024-09-10T22:15:33.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.258+0000 7f9b76000640 1 Processor -- start 2024-09-10T22:15:33.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.258+0000 7f9b76000640 1 -- start start 2024-09-10T22:15:33.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b76000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b700aeec0 0x7f9b700b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:33.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b76000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b7015a310 0x7f9b701fc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:33.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b76000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7015b690 0x7f9b701f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:33.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b76000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7015edd0 con 0x7f9b7015b690 2024-09-10T22:15:33.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b76000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7015ec50 con 0x7f9b700aeec0 2024-09-10T22:15:33.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b76000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9b70167300 con 0x7f9b7015a310 2024-09-10T22:15:33.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b6fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7015b690 0x7f9b701f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:33.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b6fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7015b690 0x7f9b701f8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53476/0 (socket says 172.21.15.66:53476) 2024-09-10T22:15:33.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b6fe00640 1 -- 172.21.15.66:0/1655479600 learned_addr learned my addr 172.21.15.66:0/1655479600 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:33.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b6f400640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b700aeec0 0x7f9b700b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:33.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b67e00640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b7015a310 0x7f9b701fc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:33.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b6fe00640 1 -- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b7015a310 msgr2=0x7f9b701fc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b6fe00640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b7015a310 0x7f9b701fc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b6fe00640 1 -- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b700aeec0 msgr2=0x7f9b700b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b6fe00640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b700aeec0 0x7f9b700b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b6fe00640 1 -- 172.21.15.66:0/1655479600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7015bb40 con 0x7f9b7015b690 2024-09-10T22:15:33.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b67e00640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b7015a310 0x7f9b701fc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:33.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b6fe00640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7015b690 0x7f9b701f8bb0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800b970 tx=0x7f9b6800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:33.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b67400640 1 -- 172.21.15.66:0/1655479600 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68019070 con 0x7f9b7015b690 2024-09-10T22:15:33.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b67400640 1 -- 172.21.15.66:0/1655479600 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b6800cc50 con 0x7f9b7015b690 2024-09-10T22:15:33.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b67400640 1 -- 172.21.15.66:0/1655479600 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680146a0 con 0x7f9b7015b690 2024-09-10T22:15:33.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b76000640 1 -- 172.21.15.66:0/1655479600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7015bdc0 con 0x7f9b7015b690 2024-09-10T22:15:33.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b76000640 1 -- 172.21.15.66:0/1655479600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b701f83b0 con 0x7f9b7015b690 2024-09-10T22:15:33.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b76000640 1 -- 172.21.15.66:0/1655479600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b701f81c0 con 0x7f9b7015b690 2024-09-10T22:15:33.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.262+0000 7f9b67400640 1 -- 172.21.15.66:0/1655479600 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f9b6801d3b0 con 0x7f9b7015b690 2024-09-10T22:15:33.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.266+0000 7f9b67400640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b4c074c50 0x7f9b4c077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:33.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.266+0000 7f9b67400640 1 -- 172.21.15.66:0/1655479600 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f9b680924d0 con 0x7f9b7015b690 2024-09-10T22:15:33.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.266+0000 7f9b6f400640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b4c074c50 0x7f9b4c077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:33.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.266+0000 7f9b6f400640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b4c074c50 0x7f9b4c077110 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9b600b0000 tx=0x7f9b6001c3a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:33.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.270+0000 7f9b67400640 1 -- 172.21.15.66:0/1655479600 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9b7015bb40 con 0x7f9b7015b690 2024-09-10T22:15:33.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.442+0000 7f9b76000640 1 -- 172.21.15.66:0/1655479600 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b7014bff0 con 0x7f9b4c074c50 2024-09-10T22:15:33.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.442+0000 7f9b67400640 1 -- 172.21.15.66:0/1655479600 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13555 (secure 0 0 0) 0x7f9b600c0510 con 0x7f9b4c074c50 2024-09-10T22:15:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:33.450 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:15:33.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.446+0000 7f9b76000640 1 -- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b4c074c50 msgr2=0x7f9b4c077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.446+0000 7f9b76000640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b4c074c50 0x7f9b4c077110 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9b600b0000 tx=0x7f9b6001c3a0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.446+0000 7f9b76000640 1 -- 172.21.15.66:0/1655479600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7015b690 msgr2=0x7f9b701f8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.446+0000 7f9b76000640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7015b690 0x7f9b701f8bb0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800b970 tx=0x7f9b6800be30 comp rx=0 tx=0).stop 2024-09-10T22:15:33.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.446+0000 7f9b76000640 1 -- 172.21.15.66:0/1655479600 shutdown_connections 2024-09-10T22:15:33.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.446+0000 7f9b76000640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f9b4c074c50 0x7f9b4c077110 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.446+0000 7f9b76000640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7015b690 0x7f9b701f8bb0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.446+0000 7f9b76000640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b7015a310 0x7f9b701fc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.446+0000 7f9b76000640 1 --2- 172.21.15.66:0/1655479600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b700aeec0 0x7f9b700b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.446+0000 7f9b76000640 1 -- 172.21.15.66:0/1655479600 >> 172.21.15.66:0/1655479600 conn(0x7f9b700cfa30 msgr2=0x7f9b700cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.446+0000 7f9b76000640 1 -- 172.21.15.66:0/1655479600 shutdown_connections 2024-09-10T22:15:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.446+0000 7f9b76000640 1 -- 172.21.15.66:0/1655479600 wait complete. 2024-09-10T22:15:33.471 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":597,"stamp":"2024-09-10T22:15:32.643794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1210040,"kb_used_data":15608,"kb_used_omap":12,"kb_used_meta":1194227,"kb_avail":753764680,"statfs":{"total":773094113280,"available":771855032320,"internally_reserved":0,"allocated":15982592,"data_stored":7716126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1222889012},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17685122517,"num_objects":-4245,"num_object_clones":0,"num_object_copies":-8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4245,"num_whiteouts":0,"num_read":-4146,"num_read_kb":-16597086,"num_write":-6112,"num_write_kb":-19654791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.630644"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":353,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T22:15:30.238266+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:15:30.238266+0000","last_peered":"2024-09-10T22:15:30.238266+0000","last_clean":"2024-09-10T22:15:30.238266+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:15:30.238266+0000","last_undegraded":"2024-09-10T22:15:30.238266+0000","last_fullsized":"2024-09-10T22:15:30.238266+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509625,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145964,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225876,"statfs":{"total":96636764160,"available":96487297024,"internally_reserved":0,"allocated":2445312,"data_stored":1408007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509626,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137064,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234776,"statfs":{"total":96636764160,"available":96496410624,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509625,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137064,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234776,"statfs":{"total":96636764160,"available":96496410624,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509626,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167592,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204248,"statfs":{"total":96636764160,"available":96465149952,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509624,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163432,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208408,"statfs":{"total":96636764160,"available":96469409792,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":165476807},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509625,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149544,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222296,"statfs":{"total":96636764160,"available":96483631104,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509625,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145964,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225876,"statfs":{"total":96636764160,"available":96487297024,"internally_reserved":0,"allocated":2445312,"data_stored":1408007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":146995657},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542330,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163416,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208424,"statfs":{"total":96636764160,"available":96469426176,"internally_reserved":0,"allocated":1835008,"data_stored":811917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:15:33.472 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:15:33.472 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:15:33.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.610+0000 7fafe4600640 1 -- 172.21.15.66:0/2952664597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafdc071700 msgr2=0x7fafdc0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.610+0000 7fafe4600640 1 --2- 172.21.15.66:0/2952664597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafdc071700 0x7fafdc0b0410 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009900 tx=0x7fafc801c7c0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.610+0000 7fafe4600640 1 -- 172.21.15.66:0/2952664597 shutdown_connections 2024-09-10T22:15:33.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.610+0000 7fafe4600640 1 --2- 172.21.15.66:0/2952664597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafdc071700 0x7fafdc0b0410 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.610+0000 7fafe4600640 1 --2- 172.21.15.66:0/2952664597 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafdc070ac0 0x7fafdc070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.610+0000 7fafe4600640 1 -- 172.21.15.66:0/2952664597 >> 172.21.15.66:0/2952664597 conn(0x7fafdc0d1d00 msgr2=0x7fafdc0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:33.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.610+0000 7fafe4600640 1 -- 172.21.15.66:0/2952664597 shutdown_connections 2024-09-10T22:15:33.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.610+0000 7fafe4600640 1 -- 172.21.15.66:0/2952664597 wait complete. 2024-09-10T22:15:33.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe4600640 1 Processor -- start 2024-09-10T22:15:33.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe4600640 1 -- start start 2024-09-10T22:15:33.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe4600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafdc070ac0 0x7fafdc156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:33.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe4600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafdc071700 0x7fafdc15f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:33.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe4600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc0b0ef0 0x7fafdc15f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:33.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe4600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc160b50 con 0x7fafdc070ac0 2024-09-10T22:15:33.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe4600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1609d0 con 0x7fafdc0b0ef0 2024-09-10T22:15:33.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe4600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fafdc160cd0 con 0x7fafdc071700 2024-09-10T22:15:33.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafdc070ac0 0x7fafdc156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:33.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafdc070ac0 0x7fafdc156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53492/0 (socket says 172.21.15.66:53492) 2024-09-10T22:15:33.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe1c00640 1 -- 172.21.15.66:0/1848587614 learned_addr learned my addr 172.21.15.66:0/1848587614 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:33.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe1c00640 1 -- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafdc071700 msgr2=0x7fafdc15f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe1200640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafdc071700 0x7fafdc15f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:33.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe2600640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc0b0ef0 0x7fafdc15f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:33.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe1c00640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafdc071700 0x7fafdc15f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe1c00640 1 -- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc0b0ef0 msgr2=0x7fafdc15f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe1c00640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc0b0ef0 0x7fafdc15f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe1c00640 1 -- 172.21.15.66:0/1848587614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc071c60 con 0x7fafdc070ac0 2024-09-10T22:15:33.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe2600640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc0b0ef0 0x7fafdc15f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:33.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe1c00640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafdc070ac0 0x7fafdc156ad0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fafd000d950 tx=0x7fafd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:33.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafd9600640 1 -- 172.21.15.66:0/1848587614 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0017070 con 0x7fafdc070ac0 2024-09-10T22:15:33.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafd9600640 1 -- 172.21.15.66:0/1848587614 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafd0009d20 con 0x7fafdc070ac0 2024-09-10T22:15:33.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe4600640 1 -- 172.21.15.66:0/1848587614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc166680 con 0x7fafdc070ac0 2024-09-10T22:15:33.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafd9600640 1 -- 172.21.15.66:0/1848587614 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd00106c0 con 0x7fafdc070ac0 2024-09-10T22:15:33.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.614+0000 7fafe4600640 1 -- 172.21.15.66:0/1848587614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc1663a0 con 0x7fafdc070ac0 2024-09-10T22:15:33.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.618+0000 7fafe4600640 1 -- 172.21.15.66:0/1848587614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafdc070ea0 con 0x7fafdc070ac0 2024-09-10T22:15:33.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.618+0000 7fafd9600640 1 -- 172.21.15.66:0/1848587614 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fafd0010860 con 0x7fafdc070ac0 2024-09-10T22:15:33.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.618+0000 7fafd9600640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fafbc074c50 0x7fafbc077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:33.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.618+0000 7fafd9600640 1 -- 172.21.15.66:0/1848587614 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7fafd00929b0 con 0x7fafdc070ac0 2024-09-10T22:15:33.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.618+0000 7fafe1200640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fafbc074c50 0x7fafbc077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:33.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.622+0000 7fafe1200640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fafbc074c50 0x7fafbc077110 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fafc80ae000 tx=0x7fafc801b000 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:33.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.622+0000 7fafd9600640 1 -- 172.21.15.66:0/1848587614 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fafd005faf0 con 0x7fafdc070ac0 2024-09-10T22:15:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafe4600640 1 -- 172.21.15.66:0/1848587614 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fafdc0b1420 con 0x7fafbc074c50 2024-09-10T22:15:33.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafd9600640 1 -- 172.21.15.66:0/1848587614 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13555 (secure 0 0 0) 0x7fafdc0b1420 con 0x7fafbc074c50 2024-09-10T22:15:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:33.800 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:15:33.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafe4600640 1 -- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fafbc074c50 msgr2=0x7fafbc077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafe4600640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fafbc074c50 0x7fafbc077110 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fafc80ae000 tx=0x7fafc801b000 comp rx=0 tx=0).stop 2024-09-10T22:15:33.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafe4600640 1 -- 172.21.15.66:0/1848587614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafdc070ac0 msgr2=0x7fafdc156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafe4600640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafdc070ac0 0x7fafdc156ad0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fafd000d950 tx=0x7fafd000de10 comp rx=0 tx=0).stop 2024-09-10T22:15:33.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafe4600640 1 -- 172.21.15.66:0/1848587614 shutdown_connections 2024-09-10T22:15:33.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafe4600640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fafbc074c50 0x7fafbc077110 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafe4600640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc0b0ef0 0x7fafdc15f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafe4600640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafdc071700 0x7fafdc15f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafe4600640 1 --2- 172.21.15.66:0/1848587614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafdc070ac0 0x7fafdc156ad0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafe4600640 1 -- 172.21.15.66:0/1848587614 >> 172.21.15.66:0/1848587614 conn(0x7fafdc0d1d00 msgr2=0x7fafdc0b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.794+0000 7fafe4600640 1 -- 172.21.15.66:0/1848587614 shutdown_connections 2024-09-10T22:15:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.798+0000 7fafe4600640 1 -- 172.21.15.66:0/1848587614 wait complete. 2024-09-10T22:15:33.822 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":597,"stamp":"2024-09-10T22:15:32.643794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1210040,"kb_used_data":15608,"kb_used_omap":12,"kb_used_meta":1194227,"kb_avail":753764680,"statfs":{"total":773094113280,"available":771855032320,"internally_reserved":0,"allocated":15982592,"data_stored":7716126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1222889012},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17685122517,"num_objects":-4245,"num_object_clones":0,"num_object_copies":-8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4245,"num_whiteouts":0,"num_read":-4146,"num_read_kb":-16597086,"num_write":-6112,"num_write_kb":-19654791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.630644"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":353,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T22:15:30.238266+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:15:30.238266+0000","last_peered":"2024-09-10T22:15:30.238266+0000","last_clean":"2024-09-10T22:15:30.238266+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:15:30.238266+0000","last_undegraded":"2024-09-10T22:15:30.238266+0000","last_fullsized":"2024-09-10T22:15:30.238266+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509625,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145964,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225876,"statfs":{"total":96636764160,"available":96487297024,"internally_reserved":0,"allocated":2445312,"data_stored":1408007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509626,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137064,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234776,"statfs":{"total":96636764160,"available":96496410624,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509625,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137064,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234776,"statfs":{"total":96636764160,"available":96496410624,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509626,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167592,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204248,"statfs":{"total":96636764160,"available":96465149952,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509624,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163432,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208408,"statfs":{"total":96636764160,"available":96469409792,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":165476807},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509625,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149544,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222296,"statfs":{"total":96636764160,"available":96483631104,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509625,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145964,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225876,"statfs":{"total":96636764160,"available":96487297024,"internally_reserved":0,"allocated":2445312,"data_stored":1408007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":146995657},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542330,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163416,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208424,"statfs":{"total":96636764160,"available":96469426176,"internally_reserved":0,"allocated":1835008,"data_stored":811917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:15:33.822 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:15:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.962+0000 7f1a9d400640 1 -- 172.21.15.66:0/4096376959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a980d0110 msgr2=0x7f1a980d1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.962+0000 7f1a9d400640 1 --2- 172.21.15.66:0/4096376959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a980d0110 0x7f1a980d1ea0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009900 tx=0x7f1a8401c8a0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.962+0000 7f1a9d400640 1 -- 172.21.15.66:0/4096376959 shutdown_connections 2024-09-10T22:15:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.962+0000 7f1a9d400640 1 --2- 172.21.15.66:0/4096376959 >> v2:172.21.15.94:3300/0 conn(0x7f1a980b3110 0x7f1a9815f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.962+0000 7f1a9d400640 1 --2- 172.21.15.66:0/4096376959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a980d0110 0x7f1a980d1ea0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.962+0000 7f1a9d400640 1 --2- 172.21.15.66:0/4096376959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1a9815f100 0x7f1a980d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.962+0000 7f1a9d400640 1 -- 172.21.15.66:0/4096376959 >> 172.21.15.66:0/4096376959 conn(0x7f1a9809aa50 msgr2=0x7f1a9809ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.962+0000 7f1a9d400640 1 -- 172.21.15.66:0/4096376959 shutdown_connections 2024-09-10T22:15:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.962+0000 7f1a9d400640 1 -- 172.21.15.66:0/4096376959 wait complete. 2024-09-10T22:15:33.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9d400640 1 Processor -- start 2024-09-10T22:15:33.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9d400640 1 -- start start 2024-09-10T22:15:33.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9d400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1a980b3110 0x7f1a98157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:33.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a980d0110 0x7f1a98158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:33.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a9815f100 0x7f1a98152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:33.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1a980af290 con 0x7f1a9815f100 2024-09-10T22:15:33.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9d400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a980af110 con 0x7f1a980d0110 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9d400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1a980af410 con 0x7f1a980b3110 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a9815f100 0x7f1a98152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a9815f100 0x7f1a98152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53516/0 (socket says 172.21.15.66:53516) 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9ca00640 1 -- 172.21.15.66:0/2836862374 learned_addr learned my addr 172.21.15.66:0/2836862374 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a97e00640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1a980b3110 0x7f1a98157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a97400640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a980d0110 0x7f1a98158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9ca00640 1 -- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1a980b3110 msgr2=0x7f1a98157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9ca00640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1a980b3110 0x7f1a98157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9ca00640 1 -- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a980d0110 msgr2=0x7f1a98158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9ca00640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a980d0110 0x7f1a98158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9ca00640 1 -- 172.21.15.66:0/2836862374 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a981582e0 con 0x7f1a9815f100 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a97400640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a980d0110 0x7f1a98158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:15:33.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9ca00640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a9815f100 0x7f1a98152fb0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000e990 tx=0x7f1a9000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:33.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a8fe00640 1 -- 172.21.15.66:0/2836862374 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90004d30 con 0x7f1a9815f100 2024-09-10T22:15:33.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a8fe00640 1 -- 172.21.15.66:0/2836862374 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a9000caa0 con 0x7f1a9815f100 2024-09-10T22:15:33.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a8fe00640 1 -- 172.21.15.66:0/2836862374 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90005020 con 0x7f1a9815f100 2024-09-10T22:15:33.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9d400640 1 -- 172.21.15.66:0/2836862374 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9815c810 con 0x7f1a9815f100 2024-09-10T22:15:33.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9d400640 1 -- 172.21.15.66:0/2836862374 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a9815cdd0 con 0x7f1a9815f100 2024-09-10T22:15:33.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.966+0000 7f1a9d400640 1 -- 172.21.15.66:0/2836862374 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a9815f5b0 con 0x7f1a9815f100 2024-09-10T22:15:33.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.970+0000 7f1a8fe00640 1 -- 172.21.15.66:0/2836862374 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f1a9000f040 con 0x7f1a9815f100 2024-09-10T22:15:33.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.970+0000 7f1a8fe00640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1a78074c20 0x7f1a780770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:33.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.970+0000 7f1a8fe00640 1 -- 172.21.15.66:0/2836862374 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f1a90092ab0 con 0x7f1a9815f100 2024-09-10T22:15:33.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.970+0000 7f1a97e00640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1a78074c20 0x7f1a780770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:33.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.970+0000 7f1a97e00640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1a78074c20 0x7f1a780770e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1a88004630 tx=0x7f1a8800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:33.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:33.974+0000 7f1a8fe00640 1 -- 172.21.15.66:0/2836862374 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1a90016030 con 0x7f1a9815f100 2024-09-10T22:15:34.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.146+0000 7f1a9d400640 1 -- 172.21.15.66:0/2836862374 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a980d2970 con 0x7f1a9815f100 2024-09-10T22:15:34.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a8fe00640 1 -- 172.21.15.66:0/2836862374 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+13193 (secure 0 0 0) 0x7f1a90068310 con 0x7f1a9815f100 2024-09-10T22:15:34.154 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:34.154 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":148,"fsid":"944cb5ba-80f3-427e-9dcb-45be8cb10513","created":"2024-09-10T22:00:06.937654+0000","modified":"2024-09-10T22:15:30.226595+0000","last_up_change":"2024-09-10T22:00:21.798463+0000","last_in_change":"2024-09-10T22:00:11.244473+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:00:22.340318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c2f6b27-f225-42a8-8c2a-34078bba9b13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6817","nonce":261900486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6819","nonce":261900486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6823","nonce":261900486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":261900486},{"type":"v1","addr":"172.21.15.66:6821","nonce":261900486}]},"public_addr":"172.21.15.66:6817/261900486","cluster_addr":"172.21.15.66:6819/261900486","heartbeat_back_addr":"172.21.15.66:6823/261900486","heartbeat_front_addr":"172.21.15.66:6821/261900486","state":["exists","up"]},{"osd":1,"uuid":"5dca926d-7e4e-423f-b413-f69dfcc207fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6809","nonce":1441775173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6811","nonce":1441775173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6815","nonce":1441775173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1441775173},{"type":"v1","addr":"172.21.15.66:6813","nonce":1441775173}]},"public_addr":"172.21.15.66:6809/1441775173","cluster_addr":"172.21.15.66:6811/1441775173","heartbeat_back_addr":"172.21.15.66:6815/1441775173","heartbeat_front_addr":"172.21.15.66:6813/1441775173","state":["exists","up"]},{"osd":2,"uuid":"440ebb37-778a-4842-acf2-159f17fd29d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6801","nonce":3422283261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6803","nonce":3422283261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6807","nonce":3422283261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":3422283261},{"type":"v1","addr":"172.21.15.66:6805","nonce":3422283261}]},"public_addr":"172.21.15.66:6801/3422283261","cluster_addr":"172.21.15.66:6803/3422283261","heartbeat_back_addr":"172.21.15.66:6807/3422283261","heartbeat_front_addr":"172.21.15.66:6805/3422283261","state":["exists","up"]},{"osd":3,"uuid":"a06c9eae-f712-4076-96e3-05a60e6c30b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6825","nonce":4172584223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6827","nonce":4172584223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6831","nonce":4172584223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":4172584223},{"type":"v1","addr":"172.21.15.66:6829","nonce":4172584223}]},"public_addr":"172.21.15.66:6825/4172584223","cluster_addr":"172.21.15.66:6827/4172584223","heartbeat_back_addr":"172.21.15.66:6831/4172584223","heartbeat_front_addr":"172.21.15.66:6829/4172584223","state":["exists","up"]},{"osd":4,"uuid":"0ba4966e-0371-445c-9660-1c8605713886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6819","nonce":496135754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6821","nonce":496135754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6825","nonce":496135754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":496135754},{"type":"v1","addr":"172.21.15.94:6823","nonce":496135754}]},"public_addr":"172.21.15.94:6819/496135754","cluster_addr":"172.21.15.94:6821/496135754","heartbeat_back_addr":"172.21.15.94:6825/496135754","heartbeat_front_addr":"172.21.15.94:6823/496135754","state":["exists","up"]},{"osd":5,"uuid":"e308cc33-fb34-4a88-a7e3-cdbb2929478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6803","nonce":1528499896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6805","nonce":1528499896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6809","nonce":1528499896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1528499896},{"type":"v1","addr":"172.21.15.94:6807","nonce":1528499896}]},"public_addr":"172.21.15.94:6803/1528499896","cluster_addr":"172.21.15.94:6805/1528499896","heartbeat_back_addr":"172.21.15.94:6809/1528499896","heartbeat_front_addr":"172.21.15.94:6807/1528499896","state":["exists","up"]},{"osd":6,"uuid":"b6dea904-68b7-41e4-a0f5-c68fa404e5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6811","nonce":47260656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6813","nonce":47260656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6817","nonce":47260656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":47260656},{"type":"v1","addr":"172.21.15.94:6815","nonce":47260656}]},"public_addr":"172.21.15.94:6811/47260656","cluster_addr":"172.21.15.94:6813/47260656","heartbeat_back_addr":"172.21.15.94:6817/47260656","heartbeat_front_addr":"172.21.15.94:6815/47260656","state":["exists","up"]},{"osd":7,"uuid":"a291da02-ca76-4af0-9b97-7ef8e2a97799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6827","nonce":1972820213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6829","nonce":1972820213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6833","nonce":1972820213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1972820213},{"type":"v1","addr":"172.21.15.94:6831","nonce":1972820213}]},"public_addr":"172.21.15.94:6827/1972820213","cluster_addr":"172.21.15.94:6829/1972820213","heartbeat_back_addr":"172.21.15.94:6833/1972820213","heartbeat_front_addr":"172.21.15.94:6831/1972820213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.118774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.048146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:17.946920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.382286+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.821983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.216434+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.911718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:00:18.469611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/3920857628":"2024-09-11T22:15:27.214170+0000","172.21.15.66:6834/3920857628":"2024-09-11T22:15:27.214170+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:15:34.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a9d400640 1 -- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1a78074c20 msgr2=0x7f1a780770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:34.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a9d400640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1a78074c20 0x7f1a780770e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1a88004630 tx=0x7f1a8800b2e0 comp rx=0 tx=0).stop 2024-09-10T22:15:34.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a9d400640 1 -- 172.21.15.66:0/2836862374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a9815f100 msgr2=0x7f1a98152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:34.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a9d400640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a9815f100 0x7f1a98152fb0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000e990 tx=0x7f1a9000ee50 comp rx=0 tx=0).stop 2024-09-10T22:15:34.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a9d400640 1 -- 172.21.15.66:0/2836862374 shutdown_connections 2024-09-10T22:15:34.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a9d400640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f1a78074c20 0x7f1a780770e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:34.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a9d400640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a9815f100 0x7f1a98152fb0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:34.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a9d400640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a980d0110 0x7f1a98158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:34.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a9d400640 1 --2- 172.21.15.66:0/2836862374 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1a980b3110 0x7f1a98157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:34.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a9d400640 1 -- 172.21.15.66:0/2836862374 >> 172.21.15.66:0/2836862374 conn(0x7f1a9809aa50 msgr2=0x7f1a9809ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:34.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a9d400640 1 -- 172.21.15.66:0/2836862374 shutdown_connections 2024-09-10T22:15:34.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:34.150+0000 7f1a9d400640 1 -- 172.21.15.66:0/2836862374 wait complete. 2024-09-10T22:15:35.177 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T22:15:35.178 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:15:35.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 -- 172.21.15.66:0/2224792832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f08160280 msgr2=0x7f0f080aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 --2- 172.21.15.66:0/2224792832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f08160280 0x7f0f080aec10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009c80 tx=0x7f0f0001d740 comp rx=0 tx=0).stop 2024-09-10T22:15:35.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 -- 172.21.15.66:0/2224792832 shutdown_connections 2024-09-10T22:15:35.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 --2- 172.21.15.66:0/2224792832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f08160280 0x7f0f080aec10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 --2- 172.21.15.66:0/2224792832 >> v2:172.21.15.94:3300/0 conn(0x7f0f0815f450 0x7f0f080b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 -- 172.21.15.66:0/2224792832 >> 172.21.15.66:0/2224792832 conn(0x7f0f080d1c60 msgr2=0x7f0f080d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:35.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 -- 172.21.15.66:0/2224792832 shutdown_connections 2024-09-10T22:15:35.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 -- 172.21.15.66:0/2224792832 wait complete. 2024-09-10T22:15:35.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 Processor -- start 2024-09-10T22:15:35.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 -- start start 2024-09-10T22:15:35.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f080b1e10 0x7f0f08150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f0815f450 0x7f0f081fac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f08160280 0x7f0f081fb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0f080aff30 con 0x7f0f080b1e10 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f080afdb0 con 0x7f0f0815f450 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f0e400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0f08168eb0 con 0x7f0f08160280 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f07400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f080b1e10 0x7f0f08150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f07400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f080b1e10 0x7f0f08150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53532/0 (socket says 172.21.15.66:53532) 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f07400640 1 -- 172.21.15.66:0/1551412821 learned_addr learned my addr 172.21.15.66:0/1551412821 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f07400640 1 -- 172.21.15.66:0/1551412821 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f08160280 msgr2=0x7f0f081fb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0efea00640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f0815f450 0x7f0f081fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f07400640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f08160280 0x7f0f081fb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f07400640 1 -- 172.21.15.66:0/1551412821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f0815f450 msgr2=0x7f0f081fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f07400640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f0815f450 0x7f0f081fac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.338+0000 7f0f07400640 1 -- 172.21.15.66:0/1551412821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f081511d0 con 0x7f0f080b1e10 2024-09-10T22:15:35.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.342+0000 7f0efea00640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f0815f450 0x7f0f081fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:35.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.342+0000 7f0f07400640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f080b1e10 0x7f0f08150c90 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800ea40 tx=0x7f0ef800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:35.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.342+0000 7f0effe00640 1 -- 172.21.15.66:0/1551412821 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef800ce40 con 0x7f0f080b1e10 2024-09-10T22:15:35.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.342+0000 7f0effe00640 1 -- 172.21.15.66:0/1551412821 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ef8004020 con 0x7f0f080b1e10 2024-09-10T22:15:35.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.342+0000 7f0effe00640 1 -- 172.21.15.66:0/1551412821 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8004470 con 0x7f0f080b1e10 2024-09-10T22:15:35.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.342+0000 7f0f0e400640 1 -- 172.21.15.66:0/1551412821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f08159540 con 0x7f0f080b1e10 2024-09-10T22:15:35.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.342+0000 7f0f0e400640 1 -- 172.21.15.66:0/1551412821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f08159b00 con 0x7f0f080b1e10 2024-09-10T22:15:35.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.342+0000 7f0f0e400640 1 -- 172.21.15.66:0/1551412821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f080b3170 con 0x7f0f080b1e10 2024-09-10T22:15:35.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.342+0000 7f0effe00640 1 -- 172.21.15.66:0/1551412821 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f0ef8003600 con 0x7f0f080b1e10 2024-09-10T22:15:35.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.342+0000 7f0effe00640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ee4074c50 0x7f0ee4077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.342+0000 7f0effe00640 1 -- 172.21.15.66:0/1551412821 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f0ef8093270 con 0x7f0f080b1e10 2024-09-10T22:15:35.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.346+0000 7f0effe00640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] conn(0x7f0ee407abd0 0x7f0ee407d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.346+0000 7f0effe00640 1 -- 172.21.15.66:0/1551412821 --> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ef8016ec0 con 0x7f0ee407abd0 2024-09-10T22:15:35.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.346+0000 7f0effe00640 1 -- 172.21.15.66:0/1551412821 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f0ef80938f0 con 0x7f0f080b1e10 2024-09-10T22:15:35.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.346+0000 7f0efea00640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ee4074c50 0x7f0ee4077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.346+0000 7f0f07e00640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] conn(0x7f0ee407abd0 0x7f0ee407d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.346+0000 7f0f07e00640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] conn(0x7f0ee407abd0 0x7f0ee407d070 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:35.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.346+0000 7f0efea00640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ee4074c50 0x7f0ee4077110 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0007fd0 tx=0x7f0ef0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:35.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.346+0000 7f0effe00640 1 -- 172.21.15.66:0/1551412821 <== osd.0 v2:172.21.15.66:6816/261900486 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ef8016ec0 con 0x7f0ee407abd0 2024-09-10T22:15:35.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.358+0000 7f0f0e400640 1 -- 172.21.15.66:0/1551412821 --> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0f080d2a40 con 0x7f0ee407abd0 2024-09-10T22:15:35.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.358+0000 7f0effe00640 1 -- 172.21.15.66:0/1551412821 <== osd.0 v2:172.21.15.66:6816/261900486 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0f080d2a40 con 0x7f0ee407abd0 2024-09-10T22:15:35.366 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:35.366 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:15:35.366 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:35.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.358+0000 7f0f0e400640 1 -- 172.21.15.66:0/1551412821 >> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] conn(0x7f0ee407abd0 msgr2=0x7f0ee407d070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.358+0000 7f0f0e400640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.66:6816/261900486,v1:172.21.15.66:6817/261900486] conn(0x7f0ee407abd0 0x7f0ee407d070 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.358+0000 7f0f0e400640 1 -- 172.21.15.66:0/1551412821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ee4074c50 msgr2=0x7f0ee4077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.358+0000 7f0f0e400640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f0ee4074c50 0x7f0ee4077110 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0007fd0 tx=0x7f0ef0009040 comp rx=0 tx=0).stop 2024-09-10T22:15:35.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.362+0000 7f0f0e400640 1 -- 172.21.15.66:0/1551412821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f080b1e10 msgr2=0x7f0f08150c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.362+0000 7f0f0e400640 1 --2- 172.21.15.66:0/1551412821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f080b1e10 0x7f0f08150c90 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800ea40 tx=0x7f0ef800ef00 comp rx=0 tx=0).stop 2024-09-10T22:15:35.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.362+0000 7f0f07e00640 1 -- 172.21.15.66:0/1551412821 reap_dead start 2024-09-10T22:15:35.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.362+0000 7f0f0e400640 1 -- 172.21.15.66:0/1551412821 shutdown_connections 2024-09-10T22:15:35.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.362+0000 7f0f0e400640 1 -- 172.21.15.66:0/1551412821 >> 172.21.15.66:0/1551412821 conn(0x7f0f080d1c60 msgr2=0x7f0f080708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:35.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.362+0000 7f0f0e400640 1 -- 172.21.15.66:0/1551412821 shutdown_connections 2024-09-10T22:15:35.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.362+0000 7f0f0e400640 1 -- 172.21.15.66:0/1551412821 wait complete. 2024-09-10T22:15:35.385 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T22:15:35.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.526+0000 7faa84400640 1 -- 172.21.15.66:0/3578848282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa7c15a310 msgr2=0x7faa7c0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.526+0000 7faa84400640 1 --2- 172.21.15.66:0/3578848282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa7c15a310 0x7faa7c0c48b0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7faa70009900 tx=0x7faa7001c850 comp rx=0 tx=0).stop 2024-09-10T22:15:35.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.526+0000 7faa84400640 1 -- 172.21.15.66:0/3578848282 shutdown_connections 2024-09-10T22:15:35.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.526+0000 7faa84400640 1 --2- 172.21.15.66:0/3578848282 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa7c0d0550 0x7faa7c0c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.526+0000 7faa84400640 1 --2- 172.21.15.66:0/3578848282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa7c15a310 0x7faa7c0c48b0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.526+0000 7faa84400640 1 -- 172.21.15.66:0/3578848282 >> 172.21.15.66:0/3578848282 conn(0x7faa7c0cfa30 msgr2=0x7faa7c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:35.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.526+0000 7faa84400640 1 -- 172.21.15.66:0/3578848282 shutdown_connections 2024-09-10T22:15:35.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.526+0000 7faa84400640 1 -- 172.21.15.66:0/3578848282 wait complete. 2024-09-10T22:15:35.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa84400640 1 Processor -- start 2024-09-10T22:15:35.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa84400640 1 -- start start 2024-09-10T22:15:35.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa84400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa7c0aeec0 0x7faa7c140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa84400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa7c15a310 0x7faa7c0b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa84400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c15d450 0x7faa7c15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa84400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c167a10 con 0x7faa7c15a310 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa84400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c167890 con 0x7faa7c15d450 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa84400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7faa7c167b90 con 0x7faa7c0aeec0 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa81000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa7c15a310 0x7faa7c0b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa81000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa7c15a310 0x7faa7c0b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53550/0 (socket says 172.21.15.66:53550) 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa81000640 1 -- 172.21.15.66:0/1972493548 learned_addr learned my addr 172.21.15.66:0/1972493548 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa81a00640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa7c0aeec0 0x7faa7c140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa82400640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c15d450 0x7faa7c15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa81000640 1 -- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa7c0aeec0 msgr2=0x7faa7c140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa81000640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa7c0aeec0 0x7faa7c140e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa81000640 1 -- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c15d450 msgr2=0x7faa7c15d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa81000640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c15d450 0x7faa7c15d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa81000640 1 -- 172.21.15.66:0/1972493548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7c1fae70 con 0x7faa7c15a310 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa82400640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c15d450 0x7faa7c15d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:35.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa81000640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa7c15a310 0x7faa7c0b50f0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7faa6800c9a0 tx=0x7faa6800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:35.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa79600640 1 -- 172.21.15.66:0/1972493548 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68007c10 con 0x7faa7c15a310 2024-09-10T22:15:35.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa79600640 1 -- 172.21.15.66:0/1972493548 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa68005020 con 0x7faa7c15a310 2024-09-10T22:15:35.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa79600640 1 -- 172.21.15.66:0/1972493548 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68005450 con 0x7faa7c15a310 2024-09-10T22:15:35.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa84400640 1 -- 172.21.15.66:0/1972493548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7c1fb3f0 con 0x7faa7c15a310 2024-09-10T22:15:35.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa84400640 1 -- 172.21.15.66:0/1972493548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa7c0b5930 con 0x7faa7c15a310 2024-09-10T22:15:35.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.530+0000 7faa84400640 1 -- 172.21.15.66:0/1972493548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa7c15df60 con 0x7faa7c15a310 2024-09-10T22:15:35.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.534+0000 7faa79600640 1 -- 172.21.15.66:0/1972493548 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7faa680047b0 con 0x7faa7c15a310 2024-09-10T22:15:35.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.534+0000 7faa79600640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa5c074c20 0x7faa5c0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.534+0000 7faa79600640 1 -- 172.21.15.66:0/1972493548 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7faa68092a60 con 0x7faa7c15a310 2024-09-10T22:15:35.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.534+0000 7faa81a00640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa5c074c20 0x7faa5c0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.534+0000 7faa81a00640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa5c074c20 0x7faa5c0770e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7faa700ae000 tx=0x7faa7001b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:35.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.538+0000 7faa79600640 1 -- 172.21.15.66:0/1972493548 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faa68014030 con 0x7faa7c15a310 2024-09-10T22:15:35.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.714+0000 7faa84400640 1 -- 172.21.15.66:0/1972493548 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7faa7c0b5630 con 0x7faa5c074c20 2024-09-10T22:15:35.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.714+0000 7faa79600640 1 -- 172.21.15.66:0/1972493548 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7faa7c0b5630 con 0x7faa5c074c20 2024-09-10T22:15:35.721 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T22:15:35.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.718+0000 7faa84400640 1 -- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa5c074c20 msgr2=0x7faa5c0770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.718+0000 7faa84400640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa5c074c20 0x7faa5c0770e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7faa700ae000 tx=0x7faa7001b090 comp rx=0 tx=0).stop 2024-09-10T22:15:35.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.718+0000 7faa84400640 1 -- 172.21.15.66:0/1972493548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa7c15a310 msgr2=0x7faa7c0b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.718+0000 7faa84400640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa7c15a310 0x7faa7c0b50f0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7faa6800c9a0 tx=0x7faa6800ce60 comp rx=0 tx=0).stop 2024-09-10T22:15:35.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.718+0000 7faa84400640 1 -- 172.21.15.66:0/1972493548 shutdown_connections 2024-09-10T22:15:35.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.718+0000 7faa84400640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7faa5c074c20 0x7faa5c0770e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.718+0000 7faa84400640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c15d450 0x7faa7c15d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.718+0000 7faa84400640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa7c15a310 0x7faa7c0b50f0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.718+0000 7faa84400640 1 --2- 172.21.15.66:0/1972493548 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa7c0aeec0 0x7faa7c140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.718+0000 7faa84400640 1 -- 172.21.15.66:0/1972493548 >> 172.21.15.66:0/1972493548 conn(0x7faa7c0cfa30 msgr2=0x7faa7c0cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:35.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.718+0000 7faa84400640 1 -- 172.21.15.66:0/1972493548 shutdown_connections 2024-09-10T22:15:35.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.718+0000 7faa84400640 1 -- 172.21.15.66:0/1972493548 wait complete. 2024-09-10T22:15:35.742 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T22:15:35.743 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:15:35.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.886+0000 7ff7da200640 1 -- 172.21.15.66:0/776265633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7d40cf250 msgr2=0x7ff7d415f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.886+0000 7ff7da200640 1 --2- 172.21.15.66:0/776265633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7d40cf250 0x7ff7d415f620 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009bc0 tx=0x7ff7cc01cae0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.886+0000 7ff7da200640 1 -- 172.21.15.66:0/776265633 shutdown_connections 2024-09-10T22:15:35.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.886+0000 7ff7da200640 1 --2- 172.21.15.66:0/776265633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7d40cf250 0x7ff7d415f620 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.886+0000 7ff7da200640 1 --2- 172.21.15.66:0/776265633 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7d4145050 0x7ff7d406e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.886+0000 7ff7da200640 1 -- 172.21.15.66:0/776265633 >> 172.21.15.66:0/776265633 conn(0x7ff7d40cfaa0 msgr2=0x7ff7d40cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:35.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.886+0000 7ff7da200640 1 -- 172.21.15.66:0/776265633 shutdown_connections 2024-09-10T22:15:35.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.886+0000 7ff7da200640 1 -- 172.21.15.66:0/776265633 wait complete. 2024-09-10T22:15:35.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7da200640 1 Processor -- start 2024-09-10T22:15:35.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7da200640 1 -- start start 2024-09-10T22:15:35.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7da200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7d40cf250 0x7ff7d4164b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7da200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7d4144ac0 0x7ff7d41650a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7da200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7d4145050 0x7ff7d415e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7da200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d4168de0 con 0x7ff7d40cf250 2024-09-10T22:15:35.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7da200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d4168c60 con 0x7ff7d4145050 2024-09-10T22:15:35.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7da200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d4168f60 con 0x7ff7d4144ac0 2024-09-10T22:15:35.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7d3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7d40cf250 0x7ff7d4164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7d3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7d40cf250 0x7ff7d4164b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53578/0 (socket says 172.21.15.66:53578) 2024-09-10T22:15:35.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7d3400640 1 -- 172.21.15.66:0/473919090 learned_addr learned my addr 172.21.15.66:0/473919090 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:35.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7cbe00640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7d4144ac0 0x7ff7d41650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7d3e00640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7d4145050 0x7ff7d415e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7d3400640 1 -- 172.21.15.66:0/473919090 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7d4144ac0 msgr2=0x7ff7d41650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7d3400640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7d4144ac0 0x7ff7d41650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7d3400640 1 -- 172.21.15.66:0/473919090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7d4145050 msgr2=0x7ff7d415e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7d3400640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7d4145050 0x7ff7d415e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7d3400640 1 -- 172.21.15.66:0/473919090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d40c4b70 con 0x7ff7d40cf250 2024-09-10T22:15:35.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7d3e00640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7d4145050 0x7ff7d415e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:35.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7d3400640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7d40cf250 0x7ff7d4164b60 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400c500 tx=0x7ff7c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:35.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7cb400640 1 -- 172.21.15.66:0/473919090 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4018070 con 0x7ff7d40cf250 2024-09-10T22:15:35.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7cb400640 1 -- 172.21.15.66:0/473919090 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7c4003600 con 0x7ff7d40cf250 2024-09-10T22:15:35.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7da200640 1 -- 172.21.15.66:0/473919090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d41fb660 con 0x7ff7d40cf250 2024-09-10T22:15:35.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7cb400640 1 -- 172.21.15.66:0/473919090 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4003ad0 con 0x7ff7d40cf250 2024-09-10T22:15:35.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.890+0000 7ff7da200640 1 -- 172.21.15.66:0/473919090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d415c750 con 0x7ff7d40cf250 2024-09-10T22:15:35.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.894+0000 7ff7da200640 1 -- 172.21.15.66:0/473919090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7d40c4920 con 0x7ff7d40cf250 2024-09-10T22:15:35.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.894+0000 7ff7cb400640 1 -- 172.21.15.66:0/473919090 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7ff7c4003c70 con 0x7ff7d40cf250 2024-09-10T22:15:35.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.894+0000 7ff7cb400640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff7b0074c20 0x7ff7b00770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.894+0000 7ff7cb400640 1 -- 172.21.15.66:0/473919090 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7ff7c40924c0 con 0x7ff7d40cf250 2024-09-10T22:15:35.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.894+0000 7ff7cbe00640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff7b0074c20 0x7ff7b00770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.894+0000 7ff7cb400640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] conn(0x7ff7b007aba0 0x7ff7b007d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:35.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.894+0000 7ff7cb400640 1 -- 172.21.15.66:0/473919090 --> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7c4020e80 con 0x7ff7b007aba0 2024-09-10T22:15:35.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.894+0000 7ff7cb400640 1 -- 172.21.15.66:0/473919090 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7ff7c4092b40 con 0x7ff7d40cf250 2024-09-10T22:15:35.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.894+0000 7ff7d3e00640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] conn(0x7ff7b007aba0 0x7ff7b007d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:35.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.894+0000 7ff7cbe00640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff7b0074c20 0x7ff7b00770e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff7d415b100 tx=0x7ff7bc005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:35.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.898+0000 7ff7d3e00640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] conn(0x7ff7b007aba0 0x7ff7b007d040 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:35.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.898+0000 7ff7cb400640 1 -- 172.21.15.66:0/473919090 <== osd.1 v2:172.21.15.66:6808/1441775173 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff7c4020e80 con 0x7ff7b007aba0 2024-09-10T22:15:35.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7da200640 1 -- 172.21.15.66:0/473919090 --> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff7d413df80 con 0x7ff7b007aba0 2024-09-10T22:15:35.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7cb400640 1 -- 172.21.15.66:0/473919090 <== osd.1 v2:172.21.15.66:6808/1441775173 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff7cc0a59b0 con 0x7ff7b007aba0 2024-09-10T22:15:35.916 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:35.917 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:15:35.917 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:35.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7da200640 1 -- 172.21.15.66:0/473919090 >> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] conn(0x7ff7b007aba0 msgr2=0x7ff7b007d040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7da200640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.66:6808/1441775173,v1:172.21.15.66:6809/1441775173] conn(0x7ff7b007aba0 0x7ff7b007d040 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7da200640 1 -- 172.21.15.66:0/473919090 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff7b0074c20 msgr2=0x7ff7b00770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7da200640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff7b0074c20 0x7ff7b00770e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff7d415b100 tx=0x7ff7bc005860 comp rx=0 tx=0).stop 2024-09-10T22:15:35.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7da200640 1 -- 172.21.15.66:0/473919090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7d40cf250 msgr2=0x7ff7d4164b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:35.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7da200640 1 --2- 172.21.15.66:0/473919090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7d40cf250 0x7ff7d4164b60 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400c500 tx=0x7ff7c400c9c0 comp rx=0 tx=0).stop 2024-09-10T22:15:35.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7d3e00640 1 -- 172.21.15.66:0/473919090 reap_dead start 2024-09-10T22:15:35.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7da200640 1 -- 172.21.15.66:0/473919090 shutdown_connections 2024-09-10T22:15:35.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7da200640 1 -- 172.21.15.66:0/473919090 >> 172.21.15.66:0/473919090 conn(0x7ff7d40cfaa0 msgr2=0x7ff7d4073990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:35.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7da200640 1 -- 172.21.15.66:0/473919090 shutdown_connections 2024-09-10T22:15:35.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:35.910+0000 7ff7da200640 1 -- 172.21.15.66:0/473919090 wait complete. 2024-09-10T22:15:35.935 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T22:15:36.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.074+0000 7f5737200640 1 -- 172.21.15.66:0/2056674942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730070b00 msgr2=0x7f57300ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.074+0000 7f5737200640 1 --2- 172.21.15.66:0/2056674942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730070b00 0x7f57300ca4d0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f5718009840 tx=0x7f571801c760 comp rx=0 tx=0).stop 2024-09-10T22:15:36.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.074+0000 7f5737200640 1 -- 172.21.15.66:0/2056674942 shutdown_connections 2024-09-10T22:15:36.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.074+0000 7f5737200640 1 --2- 172.21.15.66:0/2056674942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730070b00 0x7f57300ca4d0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.074+0000 7f5737200640 1 --2- 172.21.15.66:0/2056674942 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57300caa10 0x7f57300cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.074+0000 7f5737200640 1 -- 172.21.15.66:0/2056674942 >> 172.21.15.66:0/2056674942 conn(0x7f57300cc900 msgr2=0x7f57300ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:36.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5737200640 1 -- 172.21.15.66:0/2056674942 shutdown_connections 2024-09-10T22:15:36.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5737200640 1 -- 172.21.15.66:0/2056674942 wait complete. 2024-09-10T22:15:36.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5737200640 1 Processor -- start 2024-09-10T22:15:36.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5737200640 1 -- start start 2024-09-10T22:15:36.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5737200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730070b00 0x7f573015ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5737200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57300738d0 0x7f57301f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5737200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57300caa10 0x7f57301f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5737200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57300ae850 con 0x7f5730070b00 2024-09-10T22:15:36.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5737200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f57300ae6d0 con 0x7f57300738d0 2024-09-10T22:15:36.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5737200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f57300ae9d0 con 0x7f57300caa10 2024-09-10T22:15:36.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f572fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730070b00 0x7f573015ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f572fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730070b00 0x7f573015ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53598/0 (socket says 172.21.15.66:53598) 2024-09-10T22:15:36.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f572fe00640 1 -- 172.21.15.66:0/4055958236 learned_addr learned my addr 172.21.15.66:0/4055958236 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:36.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f572fe00640 1 -- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57300caa10 msgr2=0x7f57301f74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5735200640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57300caa10 0x7f57301f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5727e00640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57300738d0 0x7f57301f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f572fe00640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57300caa10 0x7f57301f74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f572fe00640 1 -- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57300738d0 msgr2=0x7f57301f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f572fe00640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57300738d0 0x7f57301f6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f572fe00640 1 -- 172.21.15.66:0/4055958236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573015b3a0 con 0x7f5730070b00 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5735200640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57300caa10 0x7f57301f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f572fe00640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730070b00 0x7f573015ab90 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f572000c500 tx=0x7f572000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f572cc00640 1 -- 172.21.15.66:0/4055958236 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720017070 con 0x7f5730070b00 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f572cc00640 1 -- 172.21.15.66:0/4055958236 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5720003600 con 0x7f5730070b00 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.078+0000 7f5737200640 1 -- 172.21.15.66:0/4055958236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57301f7e00 con 0x7f5730070b00 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.082+0000 7f572cc00640 1 -- 172.21.15.66:0/4055958236 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720003a50 con 0x7f5730070b00 2024-09-10T22:15:36.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.082+0000 7f5737200640 1 -- 172.21.15.66:0/4055958236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57300c3af0 con 0x7f5730070b00 2024-09-10T22:15:36.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.082+0000 7f5737200640 1 -- 172.21.15.66:0/4055958236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57300cc150 con 0x7f5730070b00 2024-09-10T22:15:36.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.082+0000 7f572cc00640 1 -- 172.21.15.66:0/4055958236 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f5720003bf0 con 0x7f5730070b00 2024-09-10T22:15:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.082+0000 7f572cc00640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f570c074c20 0x7f570c0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.082+0000 7f572cc00640 1 -- 172.21.15.66:0/4055958236 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f5720091cf0 con 0x7f5730070b00 2024-09-10T22:15:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.082+0000 7f5727e00640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f570c074c20 0x7f570c0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.086+0000 7f5727e00640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f570c074c20 0x7f570c0770e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f57180af000 tx=0x7f571801afa0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:36.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.086+0000 7f572cc00640 1 -- 172.21.15.66:0/4055958236 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f572005eef0 con 0x7f5730070b00 2024-09-10T22:15:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.262+0000 7f5737200640 1 -- 172.21.15.66:0/4055958236 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f57300c3d30 con 0x7f570c074c20 2024-09-10T22:15:36.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.262+0000 7f572cc00640 1 -- 172.21.15.66:0/4055958236 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f57300c3d30 con 0x7f570c074c20 2024-09-10T22:15:36.269 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T22:15:36.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.266+0000 7f5737200640 1 -- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f570c074c20 msgr2=0x7f570c0770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.266+0000 7f5737200640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f570c074c20 0x7f570c0770e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f57180af000 tx=0x7f571801afa0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.266+0000 7f5737200640 1 -- 172.21.15.66:0/4055958236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730070b00 msgr2=0x7f573015ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.266+0000 7f5737200640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730070b00 0x7f573015ab90 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f572000c500 tx=0x7f572000c9c0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.266+0000 7f5737200640 1 -- 172.21.15.66:0/4055958236 shutdown_connections 2024-09-10T22:15:36.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.266+0000 7f5737200640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f570c074c20 0x7f570c0770e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.266+0000 7f5737200640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57300caa10 0x7f57301f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.266+0000 7f5737200640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57300738d0 0x7f57301f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.266+0000 7f5737200640 1 --2- 172.21.15.66:0/4055958236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5730070b00 0x7f573015ab90 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.266+0000 7f5737200640 1 -- 172.21.15.66:0/4055958236 >> 172.21.15.66:0/4055958236 conn(0x7f57300cc900 msgr2=0x7f57300b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.266+0000 7f5737200640 1 -- 172.21.15.66:0/4055958236 shutdown_connections 2024-09-10T22:15:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.266+0000 7f5737200640 1 -- 172.21.15.66:0/4055958236 wait complete. 2024-09-10T22:15:36.292 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T22:15:36.292 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:15:36.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.438+0000 7fdc7e000640 1 -- 172.21.15.66:0/1551605065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc780b0e20 msgr2=0x7fdc78160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.438+0000 7fdc7e000640 1 --2- 172.21.15.66:0/1551605065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc780b0e20 0x7fdc78160290 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fdc780d2d30 tx=0x7fdc7001cd80 comp rx=0 tx=0).stop 2024-09-10T22:15:36.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.438+0000 7fdc7e000640 1 -- 172.21.15.66:0/1551605065 shutdown_connections 2024-09-10T22:15:36.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.438+0000 7fdc7e000640 1 --2- 172.21.15.66:0/1551605065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc780b0e20 0x7fdc78160290 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.438+0000 7fdc7e000640 1 --2- 172.21.15.66:0/1551605065 >> v2:172.21.15.94:3300/0 conn(0x7fdc780cdf90 0x7fdc7815fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.438+0000 7fdc7e000640 1 -- 172.21.15.66:0/1551605065 >> 172.21.15.66:0/1551605065 conn(0x7fdc780d1c60 msgr2=0x7fdc780d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:36.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.438+0000 7fdc7e000640 1 -- 172.21.15.66:0/1551605065 shutdown_connections 2024-09-10T22:15:36.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.438+0000 7fdc7e000640 1 -- 172.21.15.66:0/1551605065 wait complete. 2024-09-10T22:15:36.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc7e000640 1 Processor -- start 2024-09-10T22:15:36.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc7e000640 1 -- start start 2024-09-10T22:15:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc7e000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc780b0e20 0x7fdc7806e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc7e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc780cdf90 0x7fdc781f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc7e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc780ce8d0 0x7fdc781fb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc7e000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdc781691a0 con 0x7fdc780cdf90 2024-09-10T22:15:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc7e000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc78169020 con 0x7fdc780ce8d0 2024-09-10T22:15:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc7e000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdc78169320 con 0x7fdc780b0e20 2024-09-10T22:15:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc780cdf90 0x7fdc781f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc780cdf90 0x7fdc781f7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53608/0 (socket says 172.21.15.66:53608) 2024-09-10T22:15:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc6fe00640 1 -- 172.21.15.66:0/1652560011 learned_addr learned my addr 172.21.15.66:0/1652560011 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc77400640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc780b0e20 0x7fdc7806e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc77e00640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc780ce8d0 0x7fdc781fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc6fe00640 1 -- 172.21.15.66:0/1652560011 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc780b0e20 msgr2=0x7fdc7806e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc6fe00640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc780b0e20 0x7fdc7806e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc6fe00640 1 -- 172.21.15.66:0/1652560011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc780ce8d0 msgr2=0x7fdc781fb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc6fe00640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc780ce8d0 0x7fdc781fb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc6fe00640 1 -- 172.21.15.66:0/1652560011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7806f590 con 0x7fdc780cdf90 2024-09-10T22:15:36.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc6fe00640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc780cdf90 0x7fdc781f7960 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000c900 tx=0x7fdc6000cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:36.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc6f400640 1 -- 172.21.15.66:0/1652560011 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60007c70 con 0x7fdc780cdf90 2024-09-10T22:15:36.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc6f400640 1 -- 172.21.15.66:0/1652560011 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc6000f450 con 0x7fdc780cdf90 2024-09-10T22:15:36.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc7e000640 1 -- 172.21.15.66:0/1652560011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc78166220 con 0x7fdc780cdf90 2024-09-10T22:15:36.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc6f400640 1 -- 172.21.15.66:0/1652560011 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000f860 con 0x7fdc780cdf90 2024-09-10T22:15:36.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.442+0000 7fdc7e000640 1 -- 172.21.15.66:0/1652560011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc781667e0 con 0x7fdc780cdf90 2024-09-10T22:15:36.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.446+0000 7fdc7e000640 1 -- 172.21.15.66:0/1652560011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc780cecb0 con 0x7fdc780cdf90 2024-09-10T22:15:36.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.446+0000 7fdc6f400640 1 -- 172.21.15.66:0/1652560011 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fdc60020030 con 0x7fdc780cdf90 2024-09-10T22:15:36.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.446+0000 7fdc6f400640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc54074c20 0x7fdc540770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.446+0000 7fdc6f400640 1 -- 172.21.15.66:0/1652560011 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7fdc60092f70 con 0x7fdc780cdf90 2024-09-10T22:15:36.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.446+0000 7fdc6f400640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] conn(0x7fdc5407ac00 0x7fdc5407d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.446+0000 7fdc6f400640 1 -- 172.21.15.66:0/1652560011 --> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc5407d770 con 0x7fdc5407ac00 2024-09-10T22:15:36.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.446+0000 7fdc6f400640 1 -- 172.21.15.66:0/1652560011 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7fdc600935f0 con 0x7fdc780cdf90 2024-09-10T22:15:36.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.446+0000 7fdc77400640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc54074c20 0x7fdc540770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.446+0000 7fdc77e00640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] conn(0x7fdc5407ac00 0x7fdc5407d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.446+0000 7fdc77e00640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] conn(0x7fdc5407ac00 0x7fdc5407d0a0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:36.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.446+0000 7fdc77400640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc54074c20 0x7fdc540770e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdc680098e0 tx=0x7fdc68009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:36.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.450+0000 7fdc6f400640 1 -- 172.21.15.66:0/1652560011 <== osd.2 v2:172.21.15.66:6800/3422283261 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc5407d770 con 0x7fdc5407ac00 2024-09-10T22:15:36.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc7e000640 1 -- 172.21.15.66:0/1652560011 --> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdc780b4b20 con 0x7fdc5407ac00 2024-09-10T22:15:36.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc6f400640 1 -- 172.21.15.66:0/1652560011 <== osd.2 v2:172.21.15.66:6800/3422283261 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdc780b4b20 con 0x7fdc5407ac00 2024-09-10T22:15:36.468 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:36.469 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:15:36.469 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:36.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc7e000640 1 -- 172.21.15.66:0/1652560011 >> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] conn(0x7fdc5407ac00 msgr2=0x7fdc5407d0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc7e000640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.66:6800/3422283261,v1:172.21.15.66:6801/3422283261] conn(0x7fdc5407ac00 0x7fdc5407d0a0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc7e000640 1 -- 172.21.15.66:0/1652560011 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc54074c20 msgr2=0x7fdc540770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc7e000640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fdc54074c20 0x7fdc540770e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdc680098e0 tx=0x7fdc68009060 comp rx=0 tx=0).stop 2024-09-10T22:15:36.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc7e000640 1 -- 172.21.15.66:0/1652560011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc780cdf90 msgr2=0x7fdc781f7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc7e000640 1 --2- 172.21.15.66:0/1652560011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc780cdf90 0x7fdc781f7960 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000c900 tx=0x7fdc6000cdc0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc77e00640 1 -- 172.21.15.66:0/1652560011 reap_dead start 2024-09-10T22:15:36.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc7e000640 1 -- 172.21.15.66:0/1652560011 shutdown_connections 2024-09-10T22:15:36.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc7e000640 1 -- 172.21.15.66:0/1652560011 >> 172.21.15.66:0/1652560011 conn(0x7fdc780d1c60 msgr2=0x7fdc780afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:36.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc7e000640 1 -- 172.21.15.66:0/1652560011 shutdown_connections 2024-09-10T22:15:36.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.462+0000 7fdc7e000640 1 -- 172.21.15.66:0/1652560011 wait complete. 2024-09-10T22:15:36.487 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T22:15:36.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 -- 172.21.15.66:0/712206646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3840d0550 msgr2=0x7fe3840ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 --2- 172.21.15.66:0/712206646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3840d0550 0x7fe3840ca1a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009840 tx=0x7fe36c01c790 comp rx=0 tx=0).stop 2024-09-10T22:15:36.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 -- 172.21.15.66:0/712206646 shutdown_connections 2024-09-10T22:15:36.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 --2- 172.21.15.66:0/712206646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3840d0550 0x7fe3840ca1a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 -- 172.21.15.66:0/712206646 >> 172.21.15.66:0/712206646 conn(0x7fe3840cfa30 msgr2=0x7fe3840cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:36.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 -- 172.21.15.66:0/712206646 shutdown_connections 2024-09-10T22:15:36.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 -- 172.21.15.66:0/712206646 wait complete. 2024-09-10T22:15:36.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 Processor -- start 2024-09-10T22:15:36.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 -- start start 2024-09-10T22:15:36.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3840c4fc0 0x7fe38414f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe38415a310 0x7fe384156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe38414eb70 0x7fe384073c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3841686a0 con 0x7fe38414eb70 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe384168520 con 0x7fe38415a310 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe38ac00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe384168820 con 0x7fe3840c4fc0 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe388c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe38414eb70 0x7fe384073c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe388c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe38414eb70 0x7fe384073c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53618/0 (socket says 172.21.15.66:53618) 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe383e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3840c4fc0 0x7fe38414f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe388c00640 1 -- 172.21.15.66:0/3137691858 learned_addr learned my addr 172.21.15.66:0/3137691858 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.630+0000 7fe37be00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe38415a310 0x7fe384156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe388c00640 1 -- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3840c4fc0 msgr2=0x7fe38414f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe388c00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3840c4fc0 0x7fe38414f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe388c00640 1 -- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe38415a310 msgr2=0x7fe384156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe388c00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe38415a310 0x7fe384156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe388c00640 1 -- 172.21.15.66:0/3137691858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38414fbd0 con 0x7fe38414eb70 2024-09-10T22:15:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe388c00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe38414eb70 0x7fe384073c50 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00cbc0 tx=0x7fe37c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:36.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe380c00640 1 -- 172.21.15.66:0/3137691858 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c016020 con 0x7fe38414eb70 2024-09-10T22:15:36.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe38ac00640 1 -- 172.21.15.66:0/3137691858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe384156c00 con 0x7fe38414eb70 2024-09-10T22:15:36.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe380c00640 1 -- 172.21.15.66:0/3137691858 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe37c00ab60 con 0x7fe38414eb70 2024-09-10T22:15:36.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe380c00640 1 -- 172.21.15.66:0/3137691858 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c012420 con 0x7fe38414eb70 2024-09-10T22:15:36.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe38ac00640 1 -- 172.21.15.66:0/3137691858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3841571c0 con 0x7fe38414eb70 2024-09-10T22:15:36.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe38ac00640 1 -- 172.21.15.66:0/3137691858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3840c48b0 con 0x7fe38414eb70 2024-09-10T22:15:36.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.634+0000 7fe380c00640 1 -- 172.21.15.66:0/3137691858 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fe37c0125c0 con 0x7fe38414eb70 2024-09-10T22:15:36.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.638+0000 7fe380c00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe360074c20 0x7fe3600770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.638+0000 7fe380c00640 1 -- 172.21.15.66:0/3137691858 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7fe37c092a70 con 0x7fe38414eb70 2024-09-10T22:15:36.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.638+0000 7fe383e00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe360074c20 0x7fe3600770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.638+0000 7fe383e00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe360074c20 0x7fe3600770e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe374000c10 tx=0x7fe374009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:36.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.638+0000 7fe380c00640 1 -- 172.21.15.66:0/3137691858 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe37c020020 con 0x7fe38414eb70 2024-09-10T22:15:36.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.814+0000 7fe38ac00640 1 -- 172.21.15.66:0/3137691858 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fe3840cf6d0 con 0x7fe360074c20 2024-09-10T22:15:36.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.814+0000 7fe380c00640 1 -- 172.21.15.66:0/3137691858 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe3840cf6d0 con 0x7fe360074c20 2024-09-10T22:15:36.821 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T22:15:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.818+0000 7fe38ac00640 1 -- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe360074c20 msgr2=0x7fe3600770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.818+0000 7fe38ac00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe360074c20 0x7fe3600770e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe374000c10 tx=0x7fe374009040 comp rx=0 tx=0).stop 2024-09-10T22:15:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.818+0000 7fe38ac00640 1 -- 172.21.15.66:0/3137691858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe38414eb70 msgr2=0x7fe384073c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.818+0000 7fe38ac00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe38414eb70 0x7fe384073c50 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00cbc0 tx=0x7fe37c00ccb0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.818+0000 7fe38ac00640 1 -- 172.21.15.66:0/3137691858 shutdown_connections 2024-09-10T22:15:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.818+0000 7fe38ac00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fe360074c20 0x7fe3600770e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.818+0000 7fe38ac00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe38414eb70 0x7fe384073c50 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.818+0000 7fe38ac00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe38415a310 0x7fe384156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.818+0000 7fe38ac00640 1 --2- 172.21.15.66:0/3137691858 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3840c4fc0 0x7fe38414f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.818+0000 7fe38ac00640 1 -- 172.21.15.66:0/3137691858 >> 172.21.15.66:0/3137691858 conn(0x7fe3840cfa30 msgr2=0x7fe3840b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:36.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.818+0000 7fe38ac00640 1 -- 172.21.15.66:0/3137691858 shutdown_connections 2024-09-10T22:15:36.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.818+0000 7fe38ac00640 1 -- 172.21.15.66:0/3137691858 wait complete. 2024-09-10T22:15:36.843 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T22:15:36.843 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:15:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.986+0000 7fc760400640 1 -- 172.21.15.66:0/227446497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7580cf080 msgr2=0x7fc758072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.986+0000 7fc760400640 1 --2- 172.21.15.66:0/227446497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7580cf080 0x7fc758072640 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fc758076880 tx=0x7fc75001cca0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.986+0000 7fc760400640 1 -- 172.21.15.66:0/227446497 shutdown_connections 2024-09-10T22:15:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.986+0000 7fc760400640 1 --2- 172.21.15.66:0/227446497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7580cf080 0x7fc758072640 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.986+0000 7fc760400640 1 --2- 172.21.15.66:0/227446497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc758145050 0x7fc7580ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.986+0000 7fc760400640 1 --2- 172.21.15.66:0/227446497 >> v2:172.21.15.94:3300/0 conn(0x7fc758144ac0 0x7fc75806e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.986+0000 7fc760400640 1 -- 172.21.15.66:0/227446497 >> 172.21.15.66:0/227446497 conn(0x7fc7580cfaa0 msgr2=0x7fc7580cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.986+0000 7fc760400640 1 -- 172.21.15.66:0/227446497 shutdown_connections 2024-09-10T22:15:36.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.986+0000 7fc760400640 1 -- 172.21.15.66:0/227446497 wait complete. 2024-09-10T22:15:36.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc760400640 1 Processor -- start 2024-09-10T22:15:36.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc760400640 1 -- start start 2024-09-10T22:15:36.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc760400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7580cf080 0x7fc7580b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc760400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc758144ac0 0x7fc7581fa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc760400640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc758145050 0x7fc75815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc760400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc75815fdf0 con 0x7fc7580cf080 2024-09-10T22:15:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc760400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc75815fc70 con 0x7fc758144ac0 2024-09-10T22:15:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc760400640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc75815ff70 con 0x7fc758145050 2024-09-10T22:15:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc75da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7580cf080 0x7fc7580b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc75da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7580cf080 0x7fc7580b46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53628/0 (socket says 172.21.15.66:53628) 2024-09-10T22:15:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc75da00640 1 -- 172.21.15.66:0/2344544140 learned_addr learned my addr 172.21.15.66:0/2344544140 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc75e400640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc758145050 0x7fc75815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc75d000640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc758144ac0 0x7fc7581fa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc75da00640 1 -- 172.21.15.66:0/2344544140 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc758145050 msgr2=0x7fc75815b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc75da00640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc758145050 0x7fc75815b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc75da00640 1 -- 172.21.15.66:0/2344544140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc758144ac0 msgr2=0x7fc7581fa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc75da00640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc758144ac0 0x7fc7581fa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:36.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc75da00640 1 -- 172.21.15.66:0/2344544140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7581f73f0 con 0x7fc7580cf080 2024-09-10T22:15:36.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc75da00640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7580cf080 0x7fc7580b46f0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00ea20 tx=0x7fc74c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:36.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc755600640 1 -- 172.21.15.66:0/2344544140 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00cd80 con 0x7fc7580cf080 2024-09-10T22:15:36.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc755600640 1 -- 172.21.15.66:0/2344544140 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc74c010430 con 0x7fc7580cf080 2024-09-10T22:15:36.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc755600640 1 -- 172.21.15.66:0/2344544140 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c010840 con 0x7fc7580cf080 2024-09-10T22:15:36.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc760400640 1 -- 172.21.15.66:0/2344544140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7581fb180 con 0x7fc7580cf080 2024-09-10T22:15:36.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.990+0000 7fc760400640 1 -- 172.21.15.66:0/2344544140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7581fb740 con 0x7fc7580cf080 2024-09-10T22:15:36.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.994+0000 7fc760400640 1 -- 172.21.15.66:0/2344544140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc758160490 con 0x7fc7580cf080 2024-09-10T22:15:37.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.994+0000 7fc755600640 1 -- 172.21.15.66:0/2344544140 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fc74c0109e0 con 0x7fc7580cf080 2024-09-10T22:15:37.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.994+0000 7fc755600640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc738074c20 0x7fc7380770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.994+0000 7fc755600640 1 -- 172.21.15.66:0/2344544140 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7fc74c093080 con 0x7fc7580cf080 2024-09-10T22:15:37.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.994+0000 7fc755600640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] conn(0x7fc73807ac00 0x7fc73807d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.994+0000 7fc755600640 1 -- 172.21.15.66:0/2344544140 --> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc74c01ee50 con 0x7fc73807ac00 2024-09-10T22:15:37.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.994+0000 7fc755600640 1 -- 172.21.15.66:0/2344544140 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7fc74c09f050 con 0x7fc7580cf080 2024-09-10T22:15:37.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.994+0000 7fc75d000640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc738074c20 0x7fc7380770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.994+0000 7fc75e400640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] conn(0x7fc73807ac00 0x7fc73807d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.994+0000 7fc75d000640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc738074c20 0x7fc7380770e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc744005fd0 tx=0x7fc744005e40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:37.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.994+0000 7fc75e400640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] conn(0x7fc73807ac00 0x7fc73807d0a0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:37.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:36.998+0000 7fc755600640 1 -- 172.21.15.66:0/2344544140 <== osd.3 v2:172.21.15.66:6824/4172584223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc74c01ee50 con 0x7fc73807ac00 2024-09-10T22:15:37.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc760400640 1 -- 172.21.15.66:0/2344544140 --> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc7581f75c0 con 0x7fc73807ac00 2024-09-10T22:15:37.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc755600640 1 -- 172.21.15.66:0/2344544140 <== osd.3 v2:172.21.15.66:6824/4172584223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc7581f75c0 con 0x7fc73807ac00 2024-09-10T22:15:37.016 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:15:37.016 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:37.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc760400640 1 -- 172.21.15.66:0/2344544140 >> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] conn(0x7fc73807ac00 msgr2=0x7fc73807d0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc760400640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.66:6824/4172584223,v1:172.21.15.66:6825/4172584223] conn(0x7fc73807ac00 0x7fc73807d0a0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc760400640 1 -- 172.21.15.66:0/2344544140 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc738074c20 msgr2=0x7fc7380770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc760400640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fc738074c20 0x7fc7380770e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc744005fd0 tx=0x7fc744005e40 comp rx=0 tx=0).stop 2024-09-10T22:15:37.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc760400640 1 -- 172.21.15.66:0/2344544140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7580cf080 msgr2=0x7fc7580b46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc760400640 1 --2- 172.21.15.66:0/2344544140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7580cf080 0x7fc7580b46f0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00ea20 tx=0x7fc74c00eee0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc75e400640 1 -- 172.21.15.66:0/2344544140 reap_dead start 2024-09-10T22:15:37.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc760400640 1 -- 172.21.15.66:0/2344544140 shutdown_connections 2024-09-10T22:15:37.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc760400640 1 -- 172.21.15.66:0/2344544140 >> 172.21.15.66:0/2344544140 conn(0x7fc7580cfaa0 msgr2=0x7fc7581420c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:37.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc760400640 1 -- 172.21.15.66:0/2344544140 shutdown_connections 2024-09-10T22:15:37.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.010+0000 7fc760400640 1 -- 172.21.15.66:0/2344544140 wait complete. 2024-09-10T22:15:37.034 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T22:15:37.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 -- 172.21.15.66:0/3424798624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22080b1e10 msgr2=0x7f22080aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 --2- 172.21.15.66:0/3424798624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22080b1e10 0x7f22080aec10 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f2200009c80 tx=0x7f220001d740 comp rx=0 tx=0).stop 2024-09-10T22:15:37.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 -- 172.21.15.66:0/3424798624 shutdown_connections 2024-09-10T22:15:37.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 --2- 172.21.15.66:0/3424798624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22080b1e10 0x7f22080aec10 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 --2- 172.21.15.66:0/3424798624 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2208160280 0x7f22080b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 -- 172.21.15.66:0/3424798624 >> 172.21.15.66:0/3424798624 conn(0x7f22080d1c60 msgr2=0x7f22080d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:37.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 -- 172.21.15.66:0/3424798624 shutdown_connections 2024-09-10T22:15:37.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 -- 172.21.15.66:0/3424798624 wait complete. 2024-09-10T22:15:37.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 Processor -- start 2024-09-10T22:15:37.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 -- start start 2024-09-10T22:15:37.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22080b1e10 0x7f22080713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f220815f450 0x7f2208152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2208160280 0x7f220814ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2208168eb0 con 0x7f22080b1e10 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22080afe40 con 0x7f2208160280 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220f200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2208169030 con 0x7f220815f450 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f2207e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22080b1e10 0x7f22080713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f2207e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22080b1e10 0x7f22080713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53654/0 (socket says 172.21.15.66:53654) 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f2207e00640 1 -- 172.21.15.66:0/3628821538 learned_addr learned my addr 172.21.15.66:0/3628821538 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220d200640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2208160280 0x7f220814ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f21ff400640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f220815f450 0x7f2208152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f2207e00640 1 -- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f220815f450 msgr2=0x7f2208152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f2207e00640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f220815f450 0x7f2208152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f2207e00640 1 -- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2208160280 msgr2=0x7f220814ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f2207e00640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2208160280 0x7f220814ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f2207e00640 1 -- 172.21.15.66:0/3628821538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22081f9ea0 con 0x7f22080b1e10 2024-09-10T22:15:37.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f220d200640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2208160280 0x7f220814ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:37.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f21ff400640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f220815f450 0x7f2208152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:37.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.178+0000 7f2207e00640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22080b1e10 0x7f22080713f0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c530 tx=0x7f21f800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:37.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.182+0000 7f2204c00640 1 -- 172.21.15.66:0/3628821538 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8017070 con 0x7f22080b1e10 2024-09-10T22:15:37.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.182+0000 7f2204c00640 1 -- 172.21.15.66:0/3628821538 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21f80053c0 con 0x7f22080b1e10 2024-09-10T22:15:37.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.182+0000 7f2204c00640 1 -- 172.21.15.66:0/3628821538 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8012890 con 0x7f22080b1e10 2024-09-10T22:15:37.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.182+0000 7f220f200640 1 -- 172.21.15.66:0/3628821538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22081fa140 con 0x7f22080b1e10 2024-09-10T22:15:37.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.182+0000 7f220f200640 1 -- 172.21.15.66:0/3628821538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2208071930 con 0x7f22080b1e10 2024-09-10T22:15:37.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.182+0000 7f220f200640 1 -- 172.21.15.66:0/3628821538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22080aec10 con 0x7f22080b1e10 2024-09-10T22:15:37.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.182+0000 7f2204c00640 1 -- 172.21.15.66:0/3628821538 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f21f8003600 con 0x7f22080b1e10 2024-09-10T22:15:37.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.182+0000 7f2204c00640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21e4074c20 0x7f21e40770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.182+0000 7f2204c00640 1 -- 172.21.15.66:0/3628821538 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f21f8091ad0 con 0x7f22080b1e10 2024-09-10T22:15:37.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.182+0000 7f21ff400640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21e4074c20 0x7f21e40770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.186+0000 7f21ff400640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21e4074c20 0x7f21e40770e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f21f0005fd0 tx=0x7f21f0005f00 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:37.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.186+0000 7f2204c00640 1 -- 172.21.15.66:0/3628821538 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f21f805ec90 con 0x7f22080b1e10 2024-09-10T22:15:37.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.362+0000 7f220f200640 1 -- 172.21.15.66:0/3628821538 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f2208166c80 con 0x7f21e4074c20 2024-09-10T22:15:37.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f2204c00640 1 -- 172.21.15.66:0/3628821538 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2208166c80 con 0x7f21e4074c20 2024-09-10T22:15:37.371 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T22:15:37.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f220f200640 1 -- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21e4074c20 msgr2=0x7f21e40770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f220f200640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21e4074c20 0x7f21e40770e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f21f0005fd0 tx=0x7f21f0005f00 comp rx=0 tx=0).stop 2024-09-10T22:15:37.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f220f200640 1 -- 172.21.15.66:0/3628821538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22080b1e10 msgr2=0x7f22080713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f220f200640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22080b1e10 0x7f22080713f0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c530 tx=0x7f21f800c9f0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f220f200640 1 -- 172.21.15.66:0/3628821538 shutdown_connections 2024-09-10T22:15:37.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f220f200640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f21e4074c20 0x7f21e40770e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f220f200640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2208160280 0x7f220814ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f220f200640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f220815f450 0x7f2208152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f220f200640 1 --2- 172.21.15.66:0/3628821538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22080b1e10 0x7f22080713f0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f220f200640 1 -- 172.21.15.66:0/3628821538 >> 172.21.15.66:0/3628821538 conn(0x7f22080d1c60 msgr2=0x7f2208158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:37.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f220f200640 1 -- 172.21.15.66:0/3628821538 shutdown_connections 2024-09-10T22:15:37.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.366+0000 7f220f200640 1 -- 172.21.15.66:0/3628821538 wait complete. 2024-09-10T22:15:37.393 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T22:15:37.394 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:15:37.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.538+0000 7f8861a00640 1 -- 172.21.15.66:0/176900910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c0cee30 msgr2=0x7f885c0cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.538+0000 7f8861a00640 1 --2- 172.21.15.66:0/176900910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c0cee30 0x7f885c0cf210 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f884c009900 tx=0x7f884c01f0d0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.538+0000 7f8861a00640 1 -- 172.21.15.66:0/176900910 shutdown_connections 2024-09-10T22:15:37.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.538+0000 7f8861a00640 1 --2- 172.21.15.66:0/176900910 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f885c15ddd0 0x7f885c15e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.538+0000 7f8861a00640 1 --2- 172.21.15.66:0/176900910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c0cee30 0x7f885c0cf210 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.538+0000 7f8861a00640 1 -- 172.21.15.66:0/176900910 >> 172.21.15.66:0/176900910 conn(0x7f885c0cfaa0 msgr2=0x7f885c0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:37.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.538+0000 7f8861a00640 1 -- 172.21.15.66:0/176900910 shutdown_connections 2024-09-10T22:15:37.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.538+0000 7f8861a00640 1 -- 172.21.15.66:0/176900910 wait complete. 2024-09-10T22:15:37.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8861a00640 1 Processor -- start 2024-09-10T22:15:37.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8861a00640 1 -- start start 2024-09-10T22:15:37.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8861a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f885c06e6a0 0x7f885c0aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8861a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f885c0cee30 0x7f885c14f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8861a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c15ddd0 0x7f885c1f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8861a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f885c151490 con 0x7f885c15ddd0 2024-09-10T22:15:37.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8861a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f885c151310 con 0x7f885c0cee30 2024-09-10T22:15:37.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8861a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f885c06f240 con 0x7f885c06e6a0 2024-09-10T22:15:37.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f885b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c15ddd0 0x7f885c1f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f885aa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f885c06e6a0 0x7f885c0aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f885aa00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f885c06e6a0 0x7f885c0aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:37344/0 (socket says 172.21.15.66:37344) 2024-09-10T22:15:37.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f885aa00640 1 -- 172.21.15.66:0/971870778 learned_addr learned my addr 172.21.15.66:0/971870778 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:37.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8853e00640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f885c0cee30 0x7f885c14f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f885b400640 1 -- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f885c06e6a0 msgr2=0x7f885c0aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f885b400640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f885c06e6a0 0x7f885c0aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f885b400640 1 -- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f885c0cee30 msgr2=0x7f885c14f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f885b400640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f885c0cee30 0x7f885c14f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f885b400640 1 -- 172.21.15.66:0/971870778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c1f56b0 con 0x7f885c15ddd0 2024-09-10T22:15:37.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f885b400640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c15ddd0 0x7f885c1f7a40 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f885400c700 tx=0x7f885400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:37.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8852a00640 1 -- 172.21.15.66:0/971870778 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854003b80 con 0x7f885c15ddd0 2024-09-10T22:15:37.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8852a00640 1 -- 172.21.15.66:0/971870778 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8854003d20 con 0x7f885c15ddd0 2024-09-10T22:15:37.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8852a00640 1 -- 172.21.15.66:0/971870778 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854005330 con 0x7f885c15ddd0 2024-09-10T22:15:37.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8861a00640 1 -- 172.21.15.66:0/971870778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885c1f5990 con 0x7f885c15ddd0 2024-09-10T22:15:37.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8861a00640 1 -- 172.21.15.66:0/971870778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885c0aaa60 con 0x7f885c15ddd0 2024-09-10T22:15:37.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.542+0000 7f8861a00640 1 -- 172.21.15.66:0/971870778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f885c0cf210 con 0x7f885c15ddd0 2024-09-10T22:15:37.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.546+0000 7f8852a00640 1 -- 172.21.15.66:0/971870778 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f88540054d0 con 0x7f885c15ddd0 2024-09-10T22:15:37.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.546+0000 7f8852a00640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8838074c50 0x7f8838077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.546+0000 7f8852a00640 1 -- 172.21.15.66:0/971870778 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f8854092b40 con 0x7f885c15ddd0 2024-09-10T22:15:37.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.546+0000 7f8852a00640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] conn(0x7f883807abd0 0x7f883807d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.546+0000 7f8852a00640 1 -- 172.21.15.66:0/971870778 --> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8854023e50 con 0x7f883807abd0 2024-09-10T22:15:37.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.546+0000 7f8852a00640 1 -- 172.21.15.66:0/971870778 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f88540931c0 con 0x7f885c15ddd0 2024-09-10T22:15:37.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.546+0000 7f885aa00640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8838074c50 0x7f8838077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.546+0000 7f8853e00640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] conn(0x7f883807abd0 0x7f883807d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.546+0000 7f885aa00640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8838074c50 0x7f8838077110 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f884c01d780 tx=0x7f884c01d910 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:37.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.546+0000 7f8853e00640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] conn(0x7f883807abd0 0x7f883807d070 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:37.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.550+0000 7f8852a00640 1 -- 172.21.15.66:0/971870778 <== osd.4 v2:172.21.15.94:6818/496135754 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8854023e50 con 0x7f883807abd0 2024-09-10T22:15:37.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f8861a00640 1 -- 172.21.15.66:0/971870778 --> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f885c1f60a0 con 0x7f883807abd0 2024-09-10T22:15:37.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f8852a00640 1 -- 172.21.15.66:0/971870778 <== osd.4 v2:172.21.15.94:6818/496135754 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f885c1f60a0 con 0x7f883807abd0 2024-09-10T22:15:37.568 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:15:37.568 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:37.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f8861a00640 1 -- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] conn(0x7f883807abd0 msgr2=0x7f883807d070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f8861a00640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:6818/496135754,v1:172.21.15.94:6819/496135754] conn(0x7f883807abd0 0x7f883807d070 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f8861a00640 1 -- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8838074c50 msgr2=0x7f8838077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f8861a00640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8838074c50 0x7f8838077110 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f884c01d780 tx=0x7f884c01d910 comp rx=0 tx=0).stop 2024-09-10T22:15:37.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f8861a00640 1 -- 172.21.15.66:0/971870778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c15ddd0 msgr2=0x7f885c1f7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f8861a00640 1 --2- 172.21.15.66:0/971870778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f885c15ddd0 0x7f885c1f7a40 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f885400c700 tx=0x7f885400cb00 comp rx=0 tx=0).stop 2024-09-10T22:15:37.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f885b400640 1 -- 172.21.15.66:0/971870778 reap_dead start 2024-09-10T22:15:37.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f8861a00640 1 -- 172.21.15.66:0/971870778 shutdown_connections 2024-09-10T22:15:37.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f8861a00640 1 -- 172.21.15.66:0/971870778 >> 172.21.15.66:0/971870778 conn(0x7f885c0cfaa0 msgr2=0x7f885c0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:37.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f8861a00640 1 -- 172.21.15.66:0/971870778 shutdown_connections 2024-09-10T22:15:37.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.562+0000 7f8861a00640 1 -- 172.21.15.66:0/971870778 wait complete. 2024-09-10T22:15:37.587 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T22:15:37.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 -- 172.21.15.66:0/1120147394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0c4fc0 msgr2=0x7f8a6c0ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 --2- 172.21.15.66:0/1120147394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0c4fc0 0x7f8a6c0ca6e0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0cf9e0 tx=0x7f8a6401de20 comp rx=0 tx=0).stop 2024-09-10T22:15:37.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 -- 172.21.15.66:0/1120147394 shutdown_connections 2024-09-10T22:15:37.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 --2- 172.21.15.66:0/1120147394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0c4fc0 0x7f8a6c0ca6e0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 --2- 172.21.15.66:0/1120147394 >> v2:172.21.15.94:3300/0 conn(0x7f8a6c0d0550 0x7f8a6c0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 -- 172.21.15.66:0/1120147394 >> 172.21.15.66:0/1120147394 conn(0x7f8a6c0cfa30 msgr2=0x7f8a6c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:37.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 -- 172.21.15.66:0/1120147394 shutdown_connections 2024-09-10T22:15:37.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 -- 172.21.15.66:0/1120147394 wait complete. 2024-09-10T22:15:37.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 Processor -- start 2024-09-10T22:15:37.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 -- start start 2024-09-10T22:15:37.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0c4fc0 0x7f8a6c144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a6c15a310 0x7f8a6c0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8a6c1f7d00 0x7f8a6c0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c167480 con 0x7f8a6c0c4fc0 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c167300 con 0x7f8a6c15a310 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a70e00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8a6c167600 con 0x7f8a6c1f7d00 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a69e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0c4fc0 0x7f8a6c144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a69e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0c4fc0 0x7f8a6c144740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53698/0 (socket says 172.21.15.66:53698) 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a69e00640 1 -- 172.21.15.66:0/1138780904 learned_addr learned my addr 172.21.15.66:0/1138780904 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a63e00640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a6c15a310 0x7f8a6c0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a6a800640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8a6c1f7d00 0x7f8a6c0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a69e00640 1 -- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8a6c1f7d00 msgr2=0x7f8a6c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a69e00640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8a6c1f7d00 0x7f8a6c0ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a69e00640 1 -- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a6c15a310 msgr2=0x7f8a6c0ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a69e00640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a6c15a310 0x7f8a6c0ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a69e00640 1 -- 172.21.15.66:0/1138780904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6c1f81e0 con 0x7f8a6c0c4fc0 2024-09-10T22:15:37.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.730+0000 7f8a6a800640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8a6c1f7d00 0x7f8a6c0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:37.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.734+0000 7f8a69e00640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0c4fc0 0x7f8a6c144740 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c001ec0 tx=0x7f8a5c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:37.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.734+0000 7f8a62000640 1 -- 172.21.15.66:0/1138780904 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c017070 con 0x7f8a6c0c4fc0 2024-09-10T22:15:37.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.734+0000 7f8a62000640 1 -- 172.21.15.66:0/1138780904 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a5c003600 con 0x7f8a6c0c4fc0 2024-09-10T22:15:37.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.734+0000 7f8a70e00640 1 -- 172.21.15.66:0/1138780904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6c1f84c0 con 0x7f8a6c0c4fc0 2024-09-10T22:15:37.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.734+0000 7f8a62000640 1 -- 172.21.15.66:0/1138780904 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c003a50 con 0x7f8a6c0c4fc0 2024-09-10T22:15:37.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.734+0000 7f8a70e00640 1 -- 172.21.15.66:0/1138780904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6c1f8a80 con 0x7f8a6c0c4fc0 2024-09-10T22:15:37.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.734+0000 7f8a70e00640 1 -- 172.21.15.66:0/1138780904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a6c1f8cc0 con 0x7f8a6c0c4fc0 2024-09-10T22:15:37.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.734+0000 7f8a62000640 1 -- 172.21.15.66:0/1138780904 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f8a5c003bf0 con 0x7f8a6c0c4fc0 2024-09-10T22:15:37.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.734+0000 7f8a62000640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8a48074c20 0x7f8a480770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:37.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.734+0000 7f8a62000640 1 -- 172.21.15.66:0/1138780904 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f8a5c091af0 con 0x7f8a6c0c4fc0 2024-09-10T22:15:37.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.738+0000 7f8a63e00640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8a48074c20 0x7f8a480770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:37.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.738+0000 7f8a63e00640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8a48074c20 0x7f8a480770e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0b45b0 tx=0x7f8a540091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:37.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.738+0000 7f8a62000640 1 -- 172.21.15.66:0/1138780904 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8a6c1f8cc0 con 0x7f8a6c0c4fc0 2024-09-10T22:15:37.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.914+0000 7f8a70e00640 1 -- 172.21.15.66:0/1138780904 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f8a6c070420 con 0x7f8a48074c20 2024-09-10T22:15:37.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.914+0000 7f8a62000640 1 -- 172.21.15.66:0/1138780904 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8a6c070420 con 0x7f8a48074c20 2024-09-10T22:15:37.920 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T22:15:37.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.914+0000 7f8a70e00640 1 -- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8a48074c20 msgr2=0x7f8a480770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.914+0000 7f8a70e00640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8a48074c20 0x7f8a480770e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0b45b0 tx=0x7f8a540091e0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.914+0000 7f8a70e00640 1 -- 172.21.15.66:0/1138780904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0c4fc0 msgr2=0x7f8a6c144740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:37.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.914+0000 7f8a70e00640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0c4fc0 0x7f8a6c144740 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c001ec0 tx=0x7f8a5c00c890 comp rx=0 tx=0).stop 2024-09-10T22:15:37.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.914+0000 7f8a70e00640 1 -- 172.21.15.66:0/1138780904 shutdown_connections 2024-09-10T22:15:37.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.918+0000 7f8a70e00640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8a48074c20 0x7f8a480770e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.918+0000 7f8a70e00640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8a6c1f7d00 0x7f8a6c0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.918+0000 7f8a70e00640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a6c15a310 0x7f8a6c0ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.918+0000 7f8a70e00640 1 --2- 172.21.15.66:0/1138780904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0c4fc0 0x7f8a6c144740 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:37.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.918+0000 7f8a70e00640 1 -- 172.21.15.66:0/1138780904 >> 172.21.15.66:0/1138780904 conn(0x7f8a6c0cfa30 msgr2=0x7f8a6c159110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:37.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.918+0000 7f8a70e00640 1 -- 172.21.15.66:0/1138780904 shutdown_connections 2024-09-10T22:15:37.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:37.918+0000 7f8a70e00640 1 -- 172.21.15.66:0/1138780904 wait complete. 2024-09-10T22:15:37.943 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T22:15:37.943 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:15:38.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.086+0000 7f82f2000640 1 -- 172.21.15.66:0/2647764397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ec06f6a0 msgr2=0x7f82ec15f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.086+0000 7f82f2000640 1 --2- 172.21.15.66:0/2647764397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ec06f6a0 0x7f82ec15f6b0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009c80 tx=0x7f82e401cda0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.086+0000 7f82f2000640 1 -- 172.21.15.66:0/2647764397 shutdown_connections 2024-09-10T22:15:38.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.086+0000 7f82f2000640 1 --2- 172.21.15.66:0/2647764397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ec06f6a0 0x7f82ec15f6b0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.086+0000 7f82f2000640 1 --2- 172.21.15.66:0/2647764397 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f82ec0b4ed0 0x7f82ec15ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.086+0000 7f82f2000640 1 -- 172.21.15.66:0/2647764397 >> 172.21.15.66:0/2647764397 conn(0x7f82ec068990 msgr2=0x7f82ec068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:38.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.086+0000 7f82f2000640 1 -- 172.21.15.66:0/2647764397 shutdown_connections 2024-09-10T22:15:38.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.086+0000 7f82f2000640 1 -- 172.21.15.66:0/2647764397 wait complete. 2024-09-10T22:15:38.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82f2000640 1 Processor -- start 2024-09-10T22:15:38.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82f2000640 1 -- start start 2024-09-10T22:15:38.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82f2000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f82ec06f6a0 0x7f82ec073bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82f2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ec0b4ed0 0x7f82ec14d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82f2000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82ec0b52b0 0x7f82ec15ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82f2000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec152580 con 0x7f82ec0b4ed0 2024-09-10T22:15:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82f2000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec152400 con 0x7f82ec0b52b0 2024-09-10T22:15:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82f2000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f82ec152700 con 0x7f82ec06f6a0 2024-09-10T22:15:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82e3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ec0b4ed0 0x7f82ec14d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82e3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ec0b4ed0 0x7f82ec14d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53716/0 (socket says 172.21.15.66:53716) 2024-09-10T22:15:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82e3e00640 1 -- 172.21.15.66:0/2104672118 learned_addr learned my addr 172.21.15.66:0/2104672118 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82eb400640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f82ec06f6a0 0x7f82ec073bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82ebe00640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82ec0b52b0 0x7f82ec15ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82e3e00640 1 -- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f82ec06f6a0 msgr2=0x7f82ec073bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82e3e00640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f82ec06f6a0 0x7f82ec073bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82e3e00640 1 -- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82ec0b52b0 msgr2=0x7f82ec15ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82e3e00640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82ec0b52b0 0x7f82ec15ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82e3e00640 1 -- 172.21.15.66:0/2104672118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec14c4b0 con 0x7f82ec0b4ed0 2024-09-10T22:15:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82e3e00640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ec0b4ed0 0x7f82ec14d250 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f82d400c3a0 tx=0x7f82d400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82e3400640 1 -- 172.21.15.66:0/2104672118 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4017070 con 0x7f82ec0b4ed0 2024-09-10T22:15:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82e3400640 1 -- 172.21.15.66:0/2104672118 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82d4003600 con 0x7f82ec0b4ed0 2024-09-10T22:15:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82f2000640 1 -- 172.21.15.66:0/2104672118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ec1f8240 con 0x7f82ec0b4ed0 2024-09-10T22:15:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82e3400640 1 -- 172.21.15.66:0/2104672118 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4003a50 con 0x7f82ec0b4ed0 2024-09-10T22:15:38.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.090+0000 7f82f2000640 1 -- 172.21.15.66:0/2104672118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ec14ca50 con 0x7f82ec0b4ed0 2024-09-10T22:15:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.094+0000 7f82f2000640 1 -- 172.21.15.66:0/2104672118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82ec15f6b0 con 0x7f82ec0b4ed0 2024-09-10T22:15:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.094+0000 7f82e3400640 1 -- 172.21.15.66:0/2104672118 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f82d4003bf0 con 0x7f82ec0b4ed0 2024-09-10T22:15:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.094+0000 7f82e3400640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f82c8074c50 0x7f82c8077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.094+0000 7f82e3400640 1 -- 172.21.15.66:0/2104672118 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f82d4092220 con 0x7f82ec0b4ed0 2024-09-10T22:15:38.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.094+0000 7f82e3400640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] conn(0x7f82c807abd0 0x7f82c807d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.094+0000 7f82e3400640 1 -- 172.21.15.66:0/2104672118 --> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82d401be50 con 0x7f82c807abd0 2024-09-10T22:15:38.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.094+0000 7f82e3400640 1 -- 172.21.15.66:0/2104672118 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f82d40928a0 con 0x7f82ec0b4ed0 2024-09-10T22:15:38.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.094+0000 7f82eb400640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f82c8074c50 0x7f82c8077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.094+0000 7f82ebe00640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] conn(0x7f82c807abd0 0x7f82c807d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.094+0000 7f82eb400640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f82c8074c50 0x7f82c8077110 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009be0 tx=0x7f82dc0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:38.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.098+0000 7f82ebe00640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] conn(0x7f82c807abd0 0x7f82c807d070 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:38.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.098+0000 7f82e3400640 1 -- 172.21.15.66:0/2104672118 <== osd.5 v2:172.21.15.94:6802/1528499896 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f82d401be50 con 0x7f82c807abd0 2024-09-10T22:15:38.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82f2000640 1 -- 172.21.15.66:0/2104672118 --> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f82ec14f560 con 0x7f82c807abd0 2024-09-10T22:15:38.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82e3400640 1 -- 172.21.15.66:0/2104672118 <== osd.5 v2:172.21.15.94:6802/1528499896 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f82ec14f560 con 0x7f82c807abd0 2024-09-10T22:15:38.117 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:38.117 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:15:38.117 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:38.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82f2000640 1 -- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] conn(0x7f82c807abd0 msgr2=0x7f82c807d070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82f2000640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:6802/1528499896,v1:172.21.15.94:6803/1528499896] conn(0x7f82c807abd0 0x7f82c807d070 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82f2000640 1 -- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f82c8074c50 msgr2=0x7f82c8077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82f2000640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f82c8074c50 0x7f82c8077110 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009be0 tx=0x7f82dc0091e0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82f2000640 1 -- 172.21.15.66:0/2104672118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ec0b4ed0 msgr2=0x7f82ec14d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82f2000640 1 --2- 172.21.15.66:0/2104672118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82ec0b4ed0 0x7f82ec14d250 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f82d400c3a0 tx=0x7f82d400c860 comp rx=0 tx=0).stop 2024-09-10T22:15:38.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82ebe00640 1 -- 172.21.15.66:0/2104672118 reap_dead start 2024-09-10T22:15:38.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82f2000640 1 -- 172.21.15.66:0/2104672118 shutdown_connections 2024-09-10T22:15:38.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82f2000640 1 -- 172.21.15.66:0/2104672118 >> 172.21.15.66:0/2104672118 conn(0x7f82ec068990 msgr2=0x7f82ec068d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:38.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82f2000640 1 -- 172.21.15.66:0/2104672118 shutdown_connections 2024-09-10T22:15:38.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.110+0000 7f82f2000640 1 -- 172.21.15.66:0/2104672118 wait complete. 2024-09-10T22:15:38.137 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T22:15:38.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.278+0000 7f5788600640 1 -- 172.21.15.66:0/2042435159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800b32a0 msgr2=0x7f57800d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.278+0000 7f5788600640 1 --2- 172.21.15.66:0/2042435159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800b32a0 0x7f57800d0310 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f5778009bc0 tx=0x7f577801ca80 comp rx=0 tx=0).stop 2024-09-10T22:15:38.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.278+0000 7f5788600640 1 -- 172.21.15.66:0/2042435159 shutdown_connections 2024-09-10T22:15:38.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.278+0000 7f5788600640 1 --2- 172.21.15.66:0/2042435159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800b32a0 0x7f57800d0310 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.278+0000 7f5788600640 1 -- 172.21.15.66:0/2042435159 >> 172.21.15.66:0/2042435159 conn(0x7f57800d1d00 msgr2=0x7f57800d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:38.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.278+0000 7f5788600640 1 -- 172.21.15.66:0/2042435159 shutdown_connections 2024-09-10T22:15:38.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.278+0000 7f5788600640 1 -- 172.21.15.66:0/2042435159 wait complete. 2024-09-10T22:15:38.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5788600640 1 Processor -- start 2024-09-10T22:15:38.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5788600640 1 -- start start 2024-09-10T22:15:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5788600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800b0f60 0x7f57800b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5788600640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57800b32a0 0x7f5780148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5788600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f578015f460 0x7f578014ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5788600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5780169df0 con 0x7f57800b0f60 2024-09-10T22:15:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5788600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5780169c70 con 0x7f578015f460 2024-09-10T22:15:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5788600640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5780169f70 con 0x7f57800b32a0 2024-09-10T22:15:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5785c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800b0f60 0x7f57800b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5785c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800b0f60 0x7f57800b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53744/0 (socket says 172.21.15.66:53744) 2024-09-10T22:15:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5785c00640 1 -- 172.21.15.66:0/2243216514 learned_addr learned my addr 172.21.15.66:0/2243216514 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5785200640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57800b32a0 0x7f5780148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5786600640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f578015f460 0x7f578014ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5785c00640 1 -- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57800b32a0 msgr2=0x7f5780148a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5785c00640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57800b32a0 0x7f5780148a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5785c00640 1 -- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f578015f460 msgr2=0x7f578014ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5785c00640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f578015f460 0x7f578014ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5785c00640 1 -- 172.21.15.66:0/2243216514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57800c6ca0 con 0x7f57800b0f60 2024-09-10T22:15:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5785c00640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800b0f60 0x7f57800b3d30 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f577400c400 tx=0x7f577400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f577d600640 1 -- 172.21.15.66:0/2243216514 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774011790 con 0x7f57800b0f60 2024-09-10T22:15:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f577d600640 1 -- 172.21.15.66:0/2243216514 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5774017070 con 0x7f57800b0f60 2024-09-10T22:15:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f577d600640 1 -- 172.21.15.66:0/2243216514 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774011ce0 con 0x7f57800b0f60 2024-09-10T22:15:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5788600640 1 -- 172.21.15.66:0/2243216514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57800c6f80 con 0x7f57800b0f60 2024-09-10T22:15:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.282+0000 7f5788600640 1 -- 172.21.15.66:0/2243216514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57800c7540 con 0x7f57800b0f60 2024-09-10T22:15:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.286+0000 7f5788600640 1 -- 172.21.15.66:0/2243216514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57800af390 con 0x7f57800b0f60 2024-09-10T22:15:38.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.286+0000 7f577d600640 1 -- 172.21.15.66:0/2243216514 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f577401c460 con 0x7f57800b0f60 2024-09-10T22:15:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.286+0000 7f577d600640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5760074c50 0x7f5760077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.286+0000 7f577d600640 1 -- 172.21.15.66:0/2243216514 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f5774092210 con 0x7f57800b0f60 2024-09-10T22:15:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.286+0000 7f5785200640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5760074c50 0x7f5760077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.286+0000 7f5785200640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5760074c50 0x7f5760077110 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f576c000c10 tx=0x7f576c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:38.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.290+0000 7f577d600640 1 -- 172.21.15.66:0/2243216514 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f577405ed00 con 0x7f57800b0f60 2024-09-10T22:15:38.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 -- 172.21.15.66:0/2243216514 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f5780074aa0 con 0x7f5760074c50 2024-09-10T22:15:38.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f577d600640 1 -- 172.21.15.66:0/2243216514 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5780074aa0 con 0x7f5760074c50 2024-09-10T22:15:38.471 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T22:15:38.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 -- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5760074c50 msgr2=0x7f5760077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5760074c50 0x7f5760077110 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f576c000c10 tx=0x7f576c009040 comp rx=0 tx=0).stop 2024-09-10T22:15:38.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 -- 172.21.15.66:0/2243216514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800b0f60 msgr2=0x7f57800b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800b0f60 0x7f57800b3d30 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f577400c400 tx=0x7f577400c8c0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 -- 172.21.15.66:0/2243216514 shutdown_connections 2024-09-10T22:15:38.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f5760074c50 0x7f5760077110 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f578015f460 0x7f578014ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f57800b32a0 0x7f5780148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 --2- 172.21.15.66:0/2243216514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800b0f60 0x7f57800b3d30 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 -- 172.21.15.66:0/2243216514 >> 172.21.15.66:0/2243216514 conn(0x7f57800d1d00 msgr2=0x7f578014c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:38.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 -- 172.21.15.66:0/2243216514 shutdown_connections 2024-09-10T22:15:38.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.466+0000 7f5788600640 1 -- 172.21.15.66:0/2243216514 wait complete. 2024-09-10T22:15:38.493 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T22:15:38.493 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:15:38.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.638+0000 7fba24c00640 1 -- 172.21.15.66:0/3856733843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba2006e6a0 msgr2=0x7fba201667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.638+0000 7fba24c00640 1 --2- 172.21.15.66:0/3856733843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba2006e6a0 0x7fba201667a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fba18009bf0 tx=0x7fba1801cb50 comp rx=0 tx=0).stop 2024-09-10T22:15:38.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.638+0000 7fba24c00640 1 -- 172.21.15.66:0/3856733843 shutdown_connections 2024-09-10T22:15:38.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.638+0000 7fba24c00640 1 --2- 172.21.15.66:0/3856733843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba2006e6a0 0x7fba201667a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.638+0000 7fba24c00640 1 -- 172.21.15.66:0/3856733843 >> 172.21.15.66:0/3856733843 conn(0x7fba200cfaa0 msgr2=0x7fba200cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:38.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.638+0000 7fba24c00640 1 -- 172.21.15.66:0/3856733843 shutdown_connections 2024-09-10T22:15:38.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.638+0000 7fba24c00640 1 -- 172.21.15.66:0/3856733843 wait complete. 2024-09-10T22:15:38.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.638+0000 7fba24c00640 1 Processor -- start 2024-09-10T22:15:38.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.638+0000 7fba24c00640 1 -- start start 2024-09-10T22:15:38.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba24c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2006e6a0 0x7fba2013bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba24c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba20144ac0 0x7fba200b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba24c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba20145050 0x7fba201395d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba24c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fba20169d90 con 0x7fba2006e6a0 2024-09-10T22:15:38.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba24c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba20169c10 con 0x7fba20144ac0 2024-09-10T22:15:38.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba24c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fba20169f10 con 0x7fba20145050 2024-09-10T22:15:38.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba1de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2006e6a0 0x7fba2013bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba1de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2006e6a0 0x7fba2013bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53758/0 (socket says 172.21.15.66:53758) 2024-09-10T22:15:38.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba1de00640 1 -- 172.21.15.66:0/1704431429 learned_addr learned my addr 172.21.15.66:0/1704431429 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:38.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba1de00640 1 -- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba20145050 msgr2=0x7fba201395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba1de00640 1 --2- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba20145050 0x7fba201395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba1de00640 1 -- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba20144ac0 msgr2=0x7fba200b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba1de00640 1 --2- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba20144ac0 0x7fba200b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba1de00640 1 -- 172.21.15.66:0/1704431429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba201f7a40 con 0x7fba2006e6a0 2024-09-10T22:15:38.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba1de00640 1 --2- 172.21.15.66:0/1704431429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2006e6a0 0x7fba2013bf90 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fba1000c430 tx=0x7fba1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:38.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba16000640 1 -- 172.21.15.66:0/1704431429 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10017070 con 0x7fba2006e6a0 2024-09-10T22:15:38.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba16000640 1 -- 172.21.15.66:0/1704431429 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba100053c0 con 0x7fba2006e6a0 2024-09-10T22:15:38.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba16000640 1 -- 172.21.15.66:0/1704431429 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10003860 con 0x7fba2006e6a0 2024-09-10T22:15:38.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba24c00640 1 -- 172.21.15.66:0/1704431429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba201f7d20 con 0x7fba2006e6a0 2024-09-10T22:15:38.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba24c00640 1 -- 172.21.15.66:0/1704431429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba201f82e0 con 0x7fba2006e6a0 2024-09-10T22:15:38.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.642+0000 7fba24c00640 1 -- 172.21.15.66:0/1704431429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba2013c4d0 con 0x7fba2006e6a0 2024-09-10T22:15:38.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.646+0000 7fba16000640 1 -- 172.21.15.66:0/1704431429 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fba10003a00 con 0x7fba2006e6a0 2024-09-10T22:15:38.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.646+0000 7fba16000640 1 --2- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9fc074c20 0x7fb9fc0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.646+0000 7fba16000640 1 -- 172.21.15.66:0/1704431429 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7fba100923f0 con 0x7fba2006e6a0 2024-09-10T22:15:38.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.646+0000 7fba16000640 1 --2- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] conn(0x7fb9fc07aba0 0x7fb9fc07d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.646+0000 7fba16000640 1 -- 172.21.15.66:0/1704431429 --> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9fc07d710 con 0x7fb9fc07aba0 2024-09-10T22:15:38.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.646+0000 7fba16000640 1 -- 172.21.15.66:0/1704431429 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7fba10092a70 con 0x7fba2006e6a0 2024-09-10T22:15:38.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.646+0000 7fba17e00640 1 --2- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9fc074c20 0x7fb9fc0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.646+0000 7fba1e800640 1 --2- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] conn(0x7fb9fc07aba0 0x7fb9fc07d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.646+0000 7fba17e00640 1 --2- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9fc074c20 0x7fb9fc0770e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fba200b1680 tx=0x7fba080078b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:38.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.646+0000 7fba1e800640 1 --2- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] conn(0x7fb9fc07aba0 0x7fb9fc07d040 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:38.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.646+0000 7fba16000640 1 -- 172.21.15.66:0/1704431429 <== osd.6 v2:172.21.15.94:6810/47260656 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9fc07d710 con 0x7fb9fc07aba0 2024-09-10T22:15:38.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba24c00640 1 -- 172.21.15.66:0/1704431429 --> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fba200b3860 con 0x7fb9fc07aba0 2024-09-10T22:15:38.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba16000640 1 -- 172.21.15.66:0/1704431429 <== osd.6 v2:172.21.15.94:6810/47260656 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fba18003980 con 0x7fb9fc07aba0 2024-09-10T22:15:38.667 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:38.668 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:15:38.668 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:38.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba24c00640 1 -- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] conn(0x7fb9fc07aba0 msgr2=0x7fb9fc07d040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba24c00640 1 --2- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:6810/47260656,v1:172.21.15.94:6811/47260656] conn(0x7fb9fc07aba0 0x7fb9fc07d040 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba24c00640 1 -- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9fc074c20 msgr2=0x7fb9fc0770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba24c00640 1 --2- 172.21.15.66:0/1704431429 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb9fc074c20 0x7fb9fc0770e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fba200b1680 tx=0x7fba080078b0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba24c00640 1 -- 172.21.15.66:0/1704431429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2006e6a0 msgr2=0x7fba2013bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba24c00640 1 --2- 172.21.15.66:0/1704431429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2006e6a0 0x7fba2013bf90 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fba1000c430 tx=0x7fba1000c8f0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba1e800640 1 -- 172.21.15.66:0/1704431429 reap_dead start 2024-09-10T22:15:38.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba24c00640 1 -- 172.21.15.66:0/1704431429 shutdown_connections 2024-09-10T22:15:38.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba24c00640 1 -- 172.21.15.66:0/1704431429 >> 172.21.15.66:0/1704431429 conn(0x7fba200cfaa0 msgr2=0x7fba20072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:38.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba24c00640 1 -- 172.21.15.66:0/1704431429 shutdown_connections 2024-09-10T22:15:38.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.662+0000 7fba24c00640 1 -- 172.21.15.66:0/1704431429 wait complete. 2024-09-10T22:15:38.686 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T22:15:38.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 -- 172.21.15.66:0/4162273305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ba00b32a0 msgr2=0x7f3ba00af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 --2- 172.21.15.66:0/4162273305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ba00b32a0 0x7f3ba00af390 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0069970 tx=0x7f3b9801ca30 comp rx=0 tx=0).stop 2024-09-10T22:15:38.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 -- 172.21.15.66:0/4162273305 shutdown_connections 2024-09-10T22:15:38.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 --2- 172.21.15.66:0/4162273305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ba00b32a0 0x7f3ba00af390 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 --2- 172.21.15.66:0/4162273305 >> v2:172.21.15.94:3300/0 conn(0x7f3ba015f460 0x7f3ba00cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 -- 172.21.15.66:0/4162273305 >> 172.21.15.66:0/4162273305 conn(0x7f3ba00d1d00 msgr2=0x7f3ba00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:38.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 -- 172.21.15.66:0/4162273305 shutdown_connections 2024-09-10T22:15:38.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 -- 172.21.15.66:0/4162273305 wait complete. 2024-09-10T22:15:38.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 Processor -- start 2024-09-10T22:15:38.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 -- start start 2024-09-10T22:15:38.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba00b0f60 0x7f3ba0149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ba00b32a0 0x7f3ba01520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba015f460 0x7f3ba0071c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba00b0550 con 0x7f3ba00b32a0 2024-09-10T22:15:38.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba00b03d0 con 0x7f3ba015f460 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3ba4c00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba006e140 con 0x7f3ba00b0f60 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3b97e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ba00b32a0 0x7f3ba01520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3b97e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ba00b32a0 0x7f3ba01520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53776/0 (socket says 172.21.15.66:53776) 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3b97e00640 1 -- 172.21.15.66:0/708451788 learned_addr learned my addr 172.21.15.66:0/708451788 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3b9de00640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba00b0f60 0x7f3ba0149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3b9e800640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba015f460 0x7f3ba0071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3b97e00640 1 -- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba00b0f60 msgr2=0x7f3ba0149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3b97e00640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba00b0f60 0x7f3ba0149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3b97e00640 1 -- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba015f460 msgr2=0x7f3ba0071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3b97e00640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba015f460 0x7f3ba0071c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3b97e00640 1 -- 172.21.15.66:0/708451788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba01496d0 con 0x7f3ba00b32a0 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.830+0000 7f3b97e00640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ba00b32a0 0x7f3ba01520e0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800c500 tx=0x7f3b8800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.834+0000 7f3b96000640 1 -- 172.21.15.66:0/708451788 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b88017070 con 0x7f3ba00b32a0 2024-09-10T22:15:38.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.834+0000 7f3b96000640 1 -- 172.21.15.66:0/708451788 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b88003600 con 0x7f3ba00b32a0 2024-09-10T22:15:38.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.834+0000 7f3b96000640 1 -- 172.21.15.66:0/708451788 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b88003a90 con 0x7f3ba00b32a0 2024-09-10T22:15:38.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.834+0000 7f3ba4c00640 1 -- 172.21.15.66:0/708451788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba00cf2e0 con 0x7f3ba00b32a0 2024-09-10T22:15:38.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.834+0000 7f3ba4c00640 1 -- 172.21.15.66:0/708451788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba00cf000 con 0x7f3ba00b32a0 2024-09-10T22:15:38.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.834+0000 7f3ba4c00640 1 -- 172.21.15.66:0/708451788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba00af390 con 0x7f3ba00b32a0 2024-09-10T22:15:38.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.834+0000 7f3b96000640 1 -- 172.21.15.66:0/708451788 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f3b88003c30 con 0x7f3ba00b32a0 2024-09-10T22:15:38.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.834+0000 7f3b96000640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3b7c074c50 0x7f3b7c077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:38.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.834+0000 7f3b96000640 1 -- 172.21.15.66:0/708451788 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f3b88091ca0 con 0x7f3ba00b32a0 2024-09-10T22:15:38.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.834+0000 7f3b9de00640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3b7c074c50 0x7f3b7c077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:38.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.838+0000 7f3b9de00640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3b7c074c50 0x7f3b7c077110 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3b90009b40 tx=0x7f3b900091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:38.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:38.838+0000 7f3b96000640 1 -- 172.21.15.66:0/708451788 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3b8805e850 con 0x7f3ba00b32a0 2024-09-10T22:15:39.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 -- 172.21.15.66:0/708451788 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f3ba0074aa0 con 0x7f3b7c074c50 2024-09-10T22:15:39.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3b96000640 1 -- 172.21.15.66:0/708451788 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3ba0074aa0 con 0x7f3b7c074c50 2024-09-10T22:15:39.019 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T22:15:39.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 -- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3b7c074c50 msgr2=0x7f3b7c077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3b7c074c50 0x7f3b7c077110 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3b90009b40 tx=0x7f3b900091e0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 -- 172.21.15.66:0/708451788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ba00b32a0 msgr2=0x7f3ba01520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ba00b32a0 0x7f3ba01520e0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800c500 tx=0x7f3b8800c9c0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 -- 172.21.15.66:0/708451788 shutdown_connections 2024-09-10T22:15:39.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3b7c074c50 0x7f3b7c077110 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba015f460 0x7f3ba0071c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ba00b32a0 0x7f3ba01520e0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 --2- 172.21.15.66:0/708451788 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba00b0f60 0x7f3ba0149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 -- 172.21.15.66:0/708451788 >> 172.21.15.66:0/708451788 conn(0x7f3ba00d1d00 msgr2=0x7f3ba00d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:39.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 -- 172.21.15.66:0/708451788 shutdown_connections 2024-09-10T22:15:39.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.014+0000 7f3ba4c00640 1 -- 172.21.15.66:0/708451788 wait complete. 2024-09-10T22:15:39.041 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T22:15:39.041 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:15:39.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.186+0000 7f8358a00640 1 -- 172.21.15.66:0/732511507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540cd0a0 msgr2=0x7f83540d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.186+0000 7f8358a00640 1 --2- 172.21.15.66:0/732511507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540cd0a0 0x7f83540d0150 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f833c009900 tx=0x7f833c01c8b0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.186+0000 7f8358a00640 1 -- 172.21.15.66:0/732511507 shutdown_connections 2024-09-10T22:15:39.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.186+0000 7f8358a00640 1 --2- 172.21.15.66:0/732511507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540cd0a0 0x7f83540d0150 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.186+0000 7f8358a00640 1 -- 172.21.15.66:0/732511507 >> 172.21.15.66:0/732511507 conn(0x7f83540d1c60 msgr2=0x7f83540d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:39.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.186+0000 7f8358a00640 1 -- 172.21.15.66:0/732511507 shutdown_connections 2024-09-10T22:15:39.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.186+0000 7f8358a00640 1 -- 172.21.15.66:0/732511507 wait complete. 2024-09-10T22:15:39.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.186+0000 7f8358a00640 1 Processor -- start 2024-09-10T22:15:39.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.186+0000 7f8358a00640 1 -- start start 2024-09-10T22:15:39.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.186+0000 7f8358a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83540aebd0 0x7f83541f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.186+0000 7f8358a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540cd0a0 0x7f83541671c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.186+0000 7f8358a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f835415f470 0x7f83541551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f8358a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8354169d80 con 0x7f83540cd0a0 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f8358a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8354169c00 con 0x7f83540aebd0 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f8358a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8354169f00 con 0x7f835415f470 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540cd0a0 0x7f83541671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540cd0a0 0x7f83541671c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53800/0 (socket says 172.21.15.66:53800) 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834be00640 1 -- 172.21.15.66:0/1508072800 learned_addr learned my addr 172.21.15.66:0/1508072800 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f8352800640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f835415f470 0x7f83541551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f8351e00640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83540aebd0 0x7f83541f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834be00640 1 -- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f835415f470 msgr2=0x7f83541551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834be00640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f835415f470 0x7f83541551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834be00640 1 -- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83540aebd0 msgr2=0x7f83541f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834be00640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83540aebd0 0x7f83541f87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834be00640 1 -- 172.21.15.66:0/1508072800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83541f8cf0 con 0x7f83540cd0a0 2024-09-10T22:15:39.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834be00640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540cd0a0 0x7f83541671c0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f833c009850 tx=0x7f833c0040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:39.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834a000640 1 -- 172.21.15.66:0/1508072800 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c0b2040 con 0x7f83540cd0a0 2024-09-10T22:15:39.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834a000640 1 -- 172.21.15.66:0/1508072800 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f833c0044d0 con 0x7f83540cd0a0 2024-09-10T22:15:39.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834a000640 1 -- 172.21.15.66:0/1508072800 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c005280 con 0x7f83540cd0a0 2024-09-10T22:15:39.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f8358a00640 1 -- 172.21.15.66:0/1508072800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83540b4c10 con 0x7f83540cd0a0 2024-09-10T22:15:39.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f8358a00640 1 -- 172.21.15.66:0/1508072800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83540b51d0 con 0x7f83540cd0a0 2024-09-10T22:15:39.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f8358a00640 1 -- 172.21.15.66:0/1508072800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83540b5890 con 0x7f83540cd0a0 2024-09-10T22:15:39.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.190+0000 7f834a000640 1 -- 172.21.15.66:0/1508072800 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f833c005420 con 0x7f83540cd0a0 2024-09-10T22:15:39.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.194+0000 7f834a000640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8330074c20 0x7f83300770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.194+0000 7f834a000640 1 -- 172.21.15.66:0/1508072800 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f833c12c3b0 con 0x7f83540cd0a0 2024-09-10T22:15:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.194+0000 7f834a000640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] conn(0x7f833007aba0 0x7f833007d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.194+0000 7f834a000640 1 -- 172.21.15.66:0/1508072800 --> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f833c004e70 con 0x7f833007aba0 2024-09-10T22:15:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.194+0000 7f834a000640 1 -- 172.21.15.66:0/1508072800 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f833c12ca30 con 0x7f83540cd0a0 2024-09-10T22:15:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.194+0000 7f8351e00640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8330074c20 0x7f83300770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.194+0000 7f8352800640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] conn(0x7f833007aba0 0x7f833007d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.194+0000 7f8351e00640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8330074c20 0x7f83300770e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f83540cbf80 tx=0x7f8344009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:39.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.194+0000 7f8352800640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] conn(0x7f833007aba0 0x7f833007d040 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:39.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.194+0000 7f834a000640 1 -- 172.21.15.66:0/1508072800 <== osd.7 v2:172.21.15.94:6826/1972820213 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f833c004e70 con 0x7f833007aba0 2024-09-10T22:15:39.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f8358a00640 1 -- 172.21.15.66:0/1508072800 --> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f83540af020 con 0x7f833007aba0 2024-09-10T22:15:39.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f834a000640 1 -- 172.21.15.66:0/1508072800 <== osd.7 v2:172.21.15.94:6826/1972820213 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f83540af020 con 0x7f833007aba0 2024-09-10T22:15:39.215 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:15:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:15:39.215 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:15:39.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f8358a00640 1 -- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] conn(0x7f833007aba0 msgr2=0x7f833007d040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f8358a00640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:6826/1972820213,v1:172.21.15.94:6827/1972820213] conn(0x7f833007aba0 0x7f833007d040 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f8358a00640 1 -- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8330074c20 msgr2=0x7f83300770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f8358a00640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f8330074c20 0x7f83300770e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f83540cbf80 tx=0x7f8344009040 comp rx=0 tx=0).stop 2024-09-10T22:15:39.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f8358a00640 1 -- 172.21.15.66:0/1508072800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540cd0a0 msgr2=0x7f83541671c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f8358a00640 1 --2- 172.21.15.66:0/1508072800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83540cd0a0 0x7f83541671c0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f833c009850 tx=0x7f833c0040e0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f8352800640 1 -- 172.21.15.66:0/1508072800 reap_dead start 2024-09-10T22:15:39.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f8358a00640 1 -- 172.21.15.66:0/1508072800 shutdown_connections 2024-09-10T22:15:39.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f8358a00640 1 -- 172.21.15.66:0/1508072800 >> 172.21.15.66:0/1508072800 conn(0x7f83540d1c60 msgr2=0x7f8354071380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:39.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f8358a00640 1 -- 172.21.15.66:0/1508072800 shutdown_connections 2024-09-10T22:15:39.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.210+0000 7f8358a00640 1 -- 172.21.15.66:0/1508072800 wait complete. 2024-09-10T22:15:39.234 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T22:15:39.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.378+0000 7fb661000640 1 -- 172.21.15.66:0/1144810497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb65c0b0590 msgr2=0x7fb65c0cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.378+0000 7fb661000640 1 --2- 172.21.15.66:0/1144810497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb65c0b0590 0x7fb65c0cd280 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb644009900 tx=0x7fb64401c850 comp rx=0 tx=0).stop 2024-09-10T22:15:39.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.378+0000 7fb661000640 1 -- 172.21.15.66:0/1144810497 shutdown_connections 2024-09-10T22:15:39.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.378+0000 7fb661000640 1 --2- 172.21.15.66:0/1144810497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb65c0b0590 0x7fb65c0cd280 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.378+0000 7fb661000640 1 --2- 172.21.15.66:0/1144810497 >> v2:172.21.15.94:3300/0 conn(0x7fb65c15bfb0 0x7fb65c0ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.378+0000 7fb661000640 1 -- 172.21.15.66:0/1144810497 >> 172.21.15.66:0/1144810497 conn(0x7fb65c074240 msgr2=0x7fb65c074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:39.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.378+0000 7fb661000640 1 -- 172.21.15.66:0/1144810497 shutdown_connections 2024-09-10T22:15:39.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.378+0000 7fb661000640 1 -- 172.21.15.66:0/1144810497 wait complete. 2024-09-10T22:15:39.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb661000640 1 Processor -- start 2024-09-10T22:15:39.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb661000640 1 -- start start 2024-09-10T22:15:39.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb661000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb65c0b0590 0x7fb65c0c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb661000640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb65c0cd7c0 0x7fb65c0cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb661000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb65c15bfb0 0x7fb65c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb661000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c168e20 con 0x7fb65c15bfb0 2024-09-10T22:15:39.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb661000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c168ca0 con 0x7fb65c0b0590 2024-09-10T22:15:39.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb661000640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb65c168fa0 con 0x7fb65c0cd7c0 2024-09-10T22:15:39.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb65a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb65c15bfb0 0x7fb65c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb65a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb65c15bfb0 0x7fb65c0d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53802/0 (socket says 172.21.15.66:53802) 2024-09-10T22:15:39.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb65a800640 1 -- 172.21.15.66:0/223040420 learned_addr learned my addr 172.21.15.66:0/223040420 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:39.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb659e00640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb65c0b0590 0x7fb65c0c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb651400640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb65c0cd7c0 0x7fb65c0cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb65a800640 1 -- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb65c0cd7c0 msgr2=0x7fb65c0cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb65a800640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb65c0cd7c0 0x7fb65c0cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb65a800640 1 -- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb65c0b0590 msgr2=0x7fb65c0c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb65a800640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb65c0b0590 0x7fb65c0c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb65a800640 1 -- 172.21.15.66:0/223040420 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c0c65d0 con 0x7fb65c15bfb0 2024-09-10T22:15:39.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb65a800640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb65c15bfb0 0x7fb65c0d03c0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb65400c7a0 tx=0x7fb65400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:39.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb652a00640 1 -- 172.21.15.66:0/223040420 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654003b80 con 0x7fb65c15bfb0 2024-09-10T22:15:39.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb652a00640 1 -- 172.21.15.66:0/223040420 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb654003d40 con 0x7fb65c15bfb0 2024-09-10T22:15:39.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb652a00640 1 -- 172.21.15.66:0/223040420 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65400aab0 con 0x7fb65c15bfb0 2024-09-10T22:15:39.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb661000640 1 -- 172.21.15.66:0/223040420 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65c0c52d0 con 0x7fb65c15bfb0 2024-09-10T22:15:39.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.382+0000 7fb661000640 1 -- 172.21.15.66:0/223040420 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65c0c5890 con 0x7fb65c15bfb0 2024-09-10T22:15:39.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.386+0000 7fb661000640 1 -- 172.21.15.66:0/223040420 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb65c0b0ae0 con 0x7fb65c15bfb0 2024-09-10T22:15:39.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.386+0000 7fb652a00640 1 -- 172.21.15.66:0/223040420 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7fb65400ac50 con 0x7fb65c15bfb0 2024-09-10T22:15:39.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.386+0000 7fb652a00640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb638074c50 0x7fb638077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.386+0000 7fb652a00640 1 -- 172.21.15.66:0/223040420 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7fb6540921f0 con 0x7fb65c15bfb0 2024-09-10T22:15:39.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.386+0000 7fb659e00640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb638074c50 0x7fb638077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.386+0000 7fb659e00640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb638074c50 0x7fb638077110 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009b40 tx=0x7fb64c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:39.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.390+0000 7fb652a00640 1 -- 172.21.15.66:0/223040420 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb65405ece0 con 0x7fb65c15bfb0 2024-09-10T22:15:39.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 -- 172.21.15.66:0/223040420 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fb65c1568b0 con 0x7fb638074c50 2024-09-10T22:15:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb652a00640 1 -- 172.21.15.66:0/223040420 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb65c1568b0 con 0x7fb638074c50 2024-09-10T22:15:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T22:15:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 -- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb638074c50 msgr2=0x7fb638077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb638074c50 0x7fb638077110 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009b40 tx=0x7fb64c0091e0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 -- 172.21.15.66:0/223040420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb65c15bfb0 msgr2=0x7fb65c0d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb65c15bfb0 0x7fb65c0d03c0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb65400c7a0 tx=0x7fb65400cc60 comp rx=0 tx=0).stop 2024-09-10T22:15:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 -- 172.21.15.66:0/223040420 shutdown_connections 2024-09-10T22:15:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7fb638074c50 0x7fb638077110 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb65c15bfb0 0x7fb65c0d03c0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb65c0cd7c0 0x7fb65c0cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 --2- 172.21.15.66:0/223040420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb65c0b0590 0x7fb65c0c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 -- 172.21.15.66:0/223040420 >> 172.21.15.66:0/223040420 conn(0x7fb65c074240 msgr2=0x7fb65c0ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:39.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 -- 172.21.15.66:0/223040420 shutdown_connections 2024-09-10T22:15:39.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.566+0000 7fb661000640 1 -- 172.21.15.66:0/223040420 wait complete. 2024-09-10T22:15:39.593 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:15:39.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 -- 172.21.15.66:0/1539149499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6180b32a0 msgr2=0x7ff6180af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 --2- 172.21.15.66:0/1539149499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6180b32a0 0x7ff6180af390 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7ff610009bc0 tx=0x7ff61001c9f0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 -- 172.21.15.66:0/1539149499 shutdown_connections 2024-09-10T22:15:39.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 --2- 172.21.15.66:0/1539149499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6180b32a0 0x7ff6180af390 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 --2- 172.21.15.66:0/1539149499 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff6180b0f60 0x7ff6180d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 -- 172.21.15.66:0/1539149499 >> 172.21.15.66:0/1539149499 conn(0x7ff6180d1d00 msgr2=0x7ff6180d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:39.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 -- 172.21.15.66:0/1539149499 shutdown_connections 2024-09-10T22:15:39.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 -- 172.21.15.66:0/1539149499 wait complete. 2024-09-10T22:15:39.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 Processor -- start 2024-09-10T22:15:39.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 -- start start 2024-09-10T22:15:39.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6180b0f60 0x7ff6181537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6180b32a0 0x7ff61815d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff61815f460 0x7ff6181f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff6180b0600 con 0x7ff6180b32a0 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6180b0480 con 0x7ff6180b0f60 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.734+0000 7ff61e200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff618168c60 con 0x7ff61815f460 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6180b32a0 0x7ff61815d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6180b32a0 0x7ff61815d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53820/0 (socket says 172.21.15.66:53820) 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60fe00640 1 -- 172.21.15.66:0/1260590797 learned_addr learned my addr 172.21.15.66:0/1260590797 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff617e00640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff61815f460 0x7ff6181f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff617400640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6180b0f60 0x7ff6181537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60fe00640 1 -- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff61815f460 msgr2=0x7ff6181f8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60fe00640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff61815f460 0x7ff6181f8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60fe00640 1 -- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6180b0f60 msgr2=0x7ff6181537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60fe00640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6180b0f60 0x7ff6181537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60fe00640 1 -- 172.21.15.66:0/1260590797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61815d640 con 0x7ff6180b32a0 2024-09-10T22:15:39.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60fe00640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6180b32a0 0x7ff61815d100 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff60000d9b0 tx=0x7ff60000de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:39.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60f400640 1 -- 172.21.15.66:0/1260590797 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600017070 con 0x7ff6180b32a0 2024-09-10T22:15:39.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60f400640 1 -- 172.21.15.66:0/1260590797 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff600009cc0 con 0x7ff6180b32a0 2024-09-10T22:15:39.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff61e200640 1 -- 172.21.15.66:0/1260590797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6181f8fb0 con 0x7ff6180b32a0 2024-09-10T22:15:39.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60f400640 1 -- 172.21.15.66:0/1260590797 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6000106c0 con 0x7ff6180b32a0 2024-09-10T22:15:39.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff61e200640 1 -- 172.21.15.66:0/1260590797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6181f9570 con 0x7ff6180b32a0 2024-09-10T22:15:39.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff61e200640 1 -- 172.21.15.66:0/1260590797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6180af390 con 0x7ff6180b32a0 2024-09-10T22:15:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.738+0000 7ff60f400640 1 -- 172.21.15.66:0/1260590797 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7ff600010860 con 0x7ff6180b32a0 2024-09-10T22:15:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.742+0000 7ff60f400640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff5f4074c20 0x7ff5f40770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.742+0000 7ff60f400640 1 -- 172.21.15.66:0/1260590797 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7ff600092a90 con 0x7ff6180b32a0 2024-09-10T22:15:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.742+0000 7ff617400640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff5f4074c20 0x7ff5f40770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.742+0000 7ff617400640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff5f4074c20 0x7ff5f40770e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff608007fd0 tx=0x7ff608007ea0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:39.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.746+0000 7ff60f400640 1 -- 172.21.15.66:0/1260590797 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff61815d640 con 0x7ff6180b32a0 2024-09-10T22:15:39.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.918+0000 7ff61e200640 1 -- 172.21.15.66:0/1260590797 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff618074aa0 con 0x7ff5f4074c20 2024-09-10T22:15:39.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.918+0000 7ff60f400640 1 -- 172.21.15.66:0/1260590797 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13407 (secure 0 0 0) 0x7ff618074aa0 con 0x7ff5f4074c20 2024-09-10T22:15:39.926 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:15:39.926 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:15:39.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.922+0000 7ff61e200640 1 -- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff5f4074c20 msgr2=0x7ff5f40770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.922+0000 7ff61e200640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff5f4074c20 0x7ff5f40770e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff608007fd0 tx=0x7ff608007ea0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.922+0000 7ff61e200640 1 -- 172.21.15.66:0/1260590797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6180b32a0 msgr2=0x7ff61815d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:39.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.922+0000 7ff61e200640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6180b32a0 0x7ff61815d100 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff60000d9b0 tx=0x7ff60000de70 comp rx=0 tx=0).stop 2024-09-10T22:15:39.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.922+0000 7ff61e200640 1 -- 172.21.15.66:0/1260590797 shutdown_connections 2024-09-10T22:15:39.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.922+0000 7ff61e200640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7ff5f4074c20 0x7ff5f40770e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.922+0000 7ff61e200640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff61815f460 0x7ff6181f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.922+0000 7ff61e200640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6180b32a0 0x7ff61815d100 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.922+0000 7ff61e200640 1 --2- 172.21.15.66:0/1260590797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6180b0f60 0x7ff6181537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:39.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.922+0000 7ff61e200640 1 -- 172.21.15.66:0/1260590797 >> 172.21.15.66:0/1260590797 conn(0x7ff6180d1d00 msgr2=0x7ff61815e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:39.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.922+0000 7ff61e200640 1 -- 172.21.15.66:0/1260590797 shutdown_connections 2024-09-10T22:15:39.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:15:39.922+0000 7ff61e200640 1 -- 172.21.15.66:0/1260590797 wait complete. 2024-09-10T22:15:39.948 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":600,"stamp":"2024-09-10T22:15:38.644988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1210052,"kb_used_data":15624,"kb_used_omap":12,"kb_used_meta":1194227,"kb_avail":753764668,"statfs":{"total":773094113280,"available":771855020032,"internally_reserved":0,"allocated":15998976,"data_stored":7721848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1222889012},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-4114,"num_read_kb":-16597054,"num_write":-9684,"num_write_kb":-19652652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.412894"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":353,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T22:15:30.238266+0000","last_change":"2024-09-10T22:00:24.323869+0000","last_active":"2024-09-10T22:15:30.238266+0000","last_peered":"2024-09-10T22:15:30.238266+0000","last_clean":"2024-09-10T22:15:30.238266+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:15:30.238266+0000","last_undegraded":"2024-09-10T22:15:30.238266+0000","last_fullsized":"2024-09-10T22:15:30.238266+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:22.801753+0000","last_clean_scrub_stamp":"2024-09-10T22:00:22.801753+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:22.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509626,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145964,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225876,"statfs":{"total":96636764160,"available":96487297024,"internally_reserved":0,"allocated":2445312,"data_stored":1408007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509627,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137064,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234776,"statfs":{"total":96636764160,"available":96496410624,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509626,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137064,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234776,"statfs":{"total":96636764160,"available":96496410624,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509627,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167592,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204248,"statfs":{"total":96636764160,"available":96465149952,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509625,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163432,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208408,"statfs":{"total":96636764160,"available":96469409792,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":165476807},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509626,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149544,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222296,"statfs":{"total":96636764160,"available":96483631104,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509626,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145964,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225876,"statfs":{"total":96636764160,"available":96487297024,"internally_reserved":0,"allocated":2445312,"data_stored":1408007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":146995657},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542331,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163428,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208412,"statfs":{"total":96636764160,"available":96469413888,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:15:39.952 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T22:00:22.801753+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=22, tm_min=0, tm_sec=22, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=22, tm_min=15, tm_sec=34, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T22:15:39.952 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T22:15:58.195 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:15:58.199 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:15:59.953 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:16:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.106+0000 7f711ba00640 1 -- 172.21.15.66:0/659025261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711406e630 msgr2=0x7f711406ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.106+0000 7f711ba00640 1 --2- 172.21.15.66:0/659025261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711406e630 0x7f711406ea60 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f71140cd550 tx=0x7f710c01d7b0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.106+0000 7f711ba00640 1 -- 172.21.15.66:0/659025261 shutdown_connections 2024-09-10T22:16:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.106+0000 7f711ba00640 1 --2- 172.21.15.66:0/659025261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711406e630 0x7f711406ea60 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.106+0000 7f711ba00640 1 --2- 172.21.15.66:0/659025261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f711415e130 0x7f71140c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.106+0000 7f711ba00640 1 --2- 172.21.15.66:0/659025261 >> v2:172.21.15.94:3300/0 conn(0x7f711415dd50 0x7f71140c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.106+0000 7f711ba00640 1 -- 172.21.15.66:0/659025261 >> 172.21.15.66:0/659025261 conn(0x7f71140cfa30 msgr2=0x7f71140cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:00.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.106+0000 7f711ba00640 1 -- 172.21.15.66:0/659025261 shutdown_connections 2024-09-10T22:16:00.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.106+0000 7f711ba00640 1 -- 172.21.15.66:0/659025261 wait complete. 2024-09-10T22:16:00.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f711ba00640 1 Processor -- start 2024-09-10T22:16:00.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f711ba00640 1 -- start start 2024-09-10T22:16:00.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f711ba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f711406e630 0x7f71141f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:00.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f711ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711415dd50 0x7f71141fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:00.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f711ba00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f711415e130 0x7f71140b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:00.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f711ba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7114166d50 con 0x7f711415dd50 2024-09-10T22:16:00.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f711ba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f711415b280 con 0x7f711406e630 2024-09-10T22:16:00.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f711ba00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7114166ed0 con 0x7f711415e130 2024-09-10T22:16:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7113e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711415dd50 0x7f71141fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7113e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711415dd50 0x7f71141fa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36276/0 (socket says 172.21.15.66:36276) 2024-09-10T22:16:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7113e00640 1 -- 172.21.15.66:0/4116652159 learned_addr learned my addr 172.21.15.66:0/4116652159 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:16:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7119000640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f711406e630 0x7f71141f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7119a00640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f711415e130 0x7f71140b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7113e00640 1 -- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f711415e130 msgr2=0x7f71140b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7113e00640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f711415e130 0x7f71140b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7113e00640 1 -- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f711406e630 msgr2=0x7f71141f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7113e00640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f711406e630 0x7f71141f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7113e00640 1 -- 172.21.15.66:0/4116652159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71140b5600 con 0x7f711415dd50 2024-09-10T22:16:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7113e00640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711415dd50 0x7f71141fa2b0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f710000e7a0 tx=0x7f710000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7110c00640 1 -- 172.21.15.66:0/4116652159 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710000cc60 con 0x7f711415dd50 2024-09-10T22:16:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7110c00640 1 -- 172.21.15.66:0/4116652159 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7100010430 con 0x7f711415dd50 2024-09-10T22:16:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f7110c00640 1 -- 172.21.15.66:0/4116652159 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100010860 con 0x7f711415dd50 2024-09-10T22:16:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f711ba00640 1 -- 172.21.15.66:0/4116652159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71140b58e0 con 0x7f711415dd50 2024-09-10T22:16:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f711ba00640 1 -- 172.21.15.66:0/4116652159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71140b3910 con 0x7f711415dd50 2024-09-10T22:16:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.110+0000 7f711ba00640 1 -- 172.21.15.66:0/4116652159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71140cffb0 con 0x7f711415dd50 2024-09-10T22:16:00.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.114+0000 7f7110c00640 1 -- 172.21.15.66:0/4116652159 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f7100010a00 con 0x7f711415dd50 2024-09-10T22:16:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.114+0000 7f7110c00640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f70f4074c20 0x7f70f40770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.114+0000 7f7110c00640 1 -- 172.21.15.66:0/4116652159 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f7100092e90 con 0x7f711415dd50 2024-09-10T22:16:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.114+0000 7f7119000640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f70f4074c20 0x7f70f40770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:00.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.114+0000 7f7119000640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f70f4074c20 0x7f70f40770e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f71140cc200 tx=0x7f71080091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:00.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.118+0000 7f7110c00640 1 -- 172.21.15.66:0/4116652159 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f710005f980 con 0x7f711415dd50 2024-09-10T22:16:00.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 -- 172.21.15.66:0/4116652159 --> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7114069850 con 0x7f70f4074c20 2024-09-10T22:16:00.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f7110c00640 1 -- 172.21.15.66:0/4116652159 <== mgr.4123 v2:172.21.15.94:6800/2679227974 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13391 (secure 0 0 0) 0x7f7114069850 con 0x7f70f4074c20 2024-09-10T22:16:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:16:00.295 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:16:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 -- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f70f4074c20 msgr2=0x7f70f40770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f70f4074c20 0x7f70f40770e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f71140cc200 tx=0x7f71080091e0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 -- 172.21.15.66:0/4116652159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711415dd50 msgr2=0x7f71141fa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711415dd50 0x7f71141fa2b0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f710000e7a0 tx=0x7f710000ec60 comp rx=0 tx=0).stop 2024-09-10T22:16:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 -- 172.21.15.66:0/4116652159 shutdown_connections 2024-09-10T22:16:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f70f4074c20 0x7f70f40770e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f711415e130 0x7f71140b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f711415dd50 0x7f71141fa2b0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 --2- 172.21.15.66:0/4116652159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f711406e630 0x7f71141f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 -- 172.21.15.66:0/4116652159 >> 172.21.15.66:0/4116652159 conn(0x7f71140cfa30 msgr2=0x7f711415cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 -- 172.21.15.66:0/4116652159 shutdown_connections 2024-09-10T22:16:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.290+0000 7f711ba00640 1 -- 172.21.15.66:0/4116652159 wait complete. 2024-09-10T22:16:00.318 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":610,"stamp":"2024-09-10T22:15:58.649310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1210036,"kb_used_data":15624,"kb_used_omap":12,"kb_used_meta":1194227,"kb_avail":753764684,"statfs":{"total":773094113280,"available":771855036416,"internally_reserved":0,"allocated":15998976,"data_stored":7721848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1222889012},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002669"},"pg_stats":[{"pgid":"1.0","version":"148'74","reported_seq":363,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T22:15:40.348754+0000","last_change":"2024-09-10T22:15:40.348754+0000","last_active":"2024-09-10T22:15:40.348754+0000","last_peered":"2024-09-10T22:15:40.348754+0000","last_clean":"2024-09-10T22:15:40.348754+0000","last_became_active":"2024-09-10T22:00:24.322660+0000","last_became_peered":"2024-09-10T22:00:24.322660+0000","last_unstale":"2024-09-10T22:15:40.348754+0000","last_undegraded":"2024-09-10T22:15:40.348754+0000","last_fullsized":"2024-09-10T22:15:40.348754+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'74","last_scrub_stamp":"2024-09-10T22:15:40.348655+0000","last_deep_scrub":"148'74","last_deep_scrub_stamp":"2024-09-10T22:15:40.348655+0000","last_clean_scrub_stamp":"2024-09-10T22:15:40.348655+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:08.727357+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509630,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145956,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225884,"statfs":{"total":96636764160,"available":96487305216,"internally_reserved":0,"allocated":2445312,"data_stored":1408007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137064,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234776,"statfs":{"total":96636764160,"available":96496410624,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509630,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137064,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234776,"statfs":{"total":96636764160,"available":96496410624,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167592,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204248,"statfs":{"total":96636764160,"available":96465149952,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509629,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163432,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208408,"statfs":{"total":96636764160,"available":96469409792,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":165476807},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509630,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149544,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222296,"statfs":{"total":96636764160,"available":96483631104,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509630,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145956,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225884,"statfs":{"total":96636764160,"available":96487305216,"internally_reserved":0,"allocated":2445312,"data_stored":1408007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":146995657},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542335,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163428,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208412,"statfs":{"total":96636764160,"available":96469413888,"internally_reserved":0,"allocated":1851392,"data_stored":817639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:16:00.319 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T22:16:00.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.458+0000 7f31c1200640 1 -- 172.21.15.66:0/249938083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31bc160370 msgr2=0x7f31bc0cfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.458+0000 7f31c1200640 1 --2- 172.21.15.66:0/249938083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31bc160370 0x7f31bc0cfe80 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f31bc06c4f0 tx=0x7f31a401c830 comp rx=0 tx=0).stop 2024-09-10T22:16:00.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.458+0000 7f31c1200640 1 -- 172.21.15.66:0/249938083 shutdown_connections 2024-09-10T22:16:00.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.458+0000 7f31c1200640 1 --2- 172.21.15.66:0/249938083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31bc160370 0x7f31bc0cfe80 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.458+0000 7f31c1200640 1 --2- 172.21.15.66:0/249938083 >> v2:172.21.15.94:3300/0 conn(0x7f31bc156430 0x7f31bc06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.458+0000 7f31c1200640 1 -- 172.21.15.66:0/249938083 >> 172.21.15.66:0/249938083 conn(0x7f31bc0d0b30 msgr2=0x7f31bc0d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:00.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.458+0000 7f31c1200640 1 -- 172.21.15.66:0/249938083 shutdown_connections 2024-09-10T22:16:00.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.458+0000 7f31c1200640 1 -- 172.21.15.66:0/249938083 wait complete. 2024-09-10T22:16:00.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.458+0000 7f31c1200640 1 Processor -- start 2024-09-10T22:16:00.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31c1200640 1 -- start start 2024-09-10T22:16:00.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31c1200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31bc070b20 0x7f31bc1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31c1200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31bc156430 0x7f31bc0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31c1200640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31bc160370 0x7f31bc164d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31c1200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc168d10 con 0x7f31bc156430 2024-09-10T22:16:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31c1200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc168b90 con 0x7f31bc070b20 2024-09-10T22:16:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31c1200640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f31bc168e90 con 0x7f31bc160370 2024-09-10T22:16:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31bc156430 0x7f31bc0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31bc156430 0x7f31bc0b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36288/0 (socket says 172.21.15.66:36288) 2024-09-10T22:16:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b3e00640 1 -- 172.21.15.66:0/3599893593 learned_addr learned my addr 172.21.15.66:0/3599893593 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:16:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b9e00640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31bc070b20 0x7f31bc1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31ba800640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31bc160370 0x7f31bc164d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b3e00640 1 -- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31bc160370 msgr2=0x7f31bc164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b3e00640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31bc160370 0x7f31bc164d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b3e00640 1 -- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31bc070b20 msgr2=0x7f31bc1517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b3e00640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31bc070b20 0x7f31bc1517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b3e00640 1 -- 172.21.15.66:0/3599893593 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc1faf70 con 0x7f31bc156430 2024-09-10T22:16:00.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31ba800640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31bc160370 0x7f31bc164d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:16:00.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b3e00640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31bc156430 0x7f31bc0b6390 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009230 tx=0x7f31a40a6790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b2000640 1 -- 172.21.15.66:0/3599893593 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4004140 con 0x7f31bc156430 2024-09-10T22:16:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b2000640 1 -- 172.21.15.66:0/3599893593 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31a40043d0 con 0x7f31bc156430 2024-09-10T22:16:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31c1200640 1 -- 172.21.15.66:0/3599893593 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31bc1fb250 con 0x7f31bc156430 2024-09-10T22:16:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31b2000640 1 -- 172.21.15.66:0/3599893593 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a40052a0 con 0x7f31bc156430 2024-09-10T22:16:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31c1200640 1 -- 172.21.15.66:0/3599893593 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31bc1fb810 con 0x7f31bc156430 2024-09-10T22:16:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.462+0000 7f31c1200640 1 -- 172.21.15.66:0/3599893593 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31bc0b68d0 con 0x7f31bc156430 2024-09-10T22:16:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.466+0000 7f31b2000640 1 -- 172.21.15.66:0/3599893593 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f31a4005440 con 0x7f31bc156430 2024-09-10T22:16:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.466+0000 7f31b2000640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3198074c20 0x7f31980770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.466+0000 7f31b2000640 1 -- 172.21.15.66:0/3599893593 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f31a412cf70 con 0x7f31bc156430 2024-09-10T22:16:00.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.466+0000 7f31b9e00640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3198074c20 0x7f31980770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:00.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.466+0000 7f31b9e00640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3198074c20 0x7f31980770e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009be0 tx=0x7f31ac0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:00.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.470+0000 7f31b2000640 1 -- 172.21.15.66:0/3599893593 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f31a40164b0 con 0x7f31bc156430 2024-09-10T22:16:00.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.646+0000 7f31c1200640 1 -- 172.21.15.66:0/3599893593 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f31bc0c7850 con 0x7f31bc156430 2024-09-10T22:16:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.650+0000 7f31b2000640 1 -- 172.21.15.66:0/3599893593 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v58)=0 v58) ==== 123+0+0 (secure 0 0 0) 0x7f31a40fa0b0 con 0x7f31bc156430 2024-09-10T22:16:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.650+0000 7f31b2000640 1 -- 172.21.15.66:0/3599893593 <== mon.0 v2:172.21.15.66:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f31a40f94c0 con 0x7f31bc156430 2024-09-10T22:16:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.654+0000 7f31c1200640 1 -- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3198074c20 msgr2=0x7f31980770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.654+0000 7f31c1200640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3198074c20 0x7f31980770e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009be0 tx=0x7f31ac0091e0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.654+0000 7f31c1200640 1 -- 172.21.15.66:0/3599893593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31bc156430 msgr2=0x7f31bc0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.654+0000 7f31c1200640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31bc156430 0x7f31bc0b6390 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009230 tx=0x7f31a40a6790 comp rx=0 tx=0).stop 2024-09-10T22:16:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.654+0000 7f31c1200640 1 -- 172.21.15.66:0/3599893593 shutdown_connections 2024-09-10T22:16:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.654+0000 7f31c1200640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f3198074c20 0x7f31980770e0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.654+0000 7f31c1200640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31bc160370 0x7f31bc164d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.654+0000 7f31c1200640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31bc156430 0x7f31bc0b6390 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.654+0000 7f31c1200640 1 --2- 172.21.15.66:0/3599893593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31bc070b20 0x7f31bc1517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.654+0000 7f31c1200640 1 -- 172.21.15.66:0/3599893593 >> 172.21.15.66:0/3599893593 conn(0x7f31bc0d0b30 msgr2=0x7f31bc15ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:00.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.654+0000 7f31c1200640 1 -- 172.21.15.66:0/3599893593 shutdown_connections 2024-09-10T22:16:00.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.654+0000 7f31c1200640 1 -- 172.21.15.66:0/3599893593 wait complete. 2024-09-10T22:16:00.677 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:16:00.678 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:16:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 -- 172.21.15.66:0/2318446993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83cc15cc80 msgr2=0x7f83cc0cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 --2- 172.21.15.66:0/2318446993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83cc15cc80 0x7f83cc0cd930 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009c80 tx=0x7f83c401cd10 comp rx=0 tx=0).stop 2024-09-10T22:16:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 -- 172.21.15.66:0/2318446993 shutdown_connections 2024-09-10T22:16:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 --2- 172.21.15.66:0/2318446993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83cc15cc80 0x7f83cc0cd930 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 --2- 172.21.15.66:0/2318446993 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f83cc0aee20 0x7f83cc0ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 -- 172.21.15.66:0/2318446993 >> 172.21.15.66:0/2318446993 conn(0x7f83cc0cfa30 msgr2=0x7f83cc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 -- 172.21.15.66:0/2318446993 shutdown_connections 2024-09-10T22:16:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 -- 172.21.15.66:0/2318446993 wait complete. 2024-09-10T22:16:00.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 Processor -- start 2024-09-10T22:16:00.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 -- start start 2024-09-10T22:16:00.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83cc0aee20 0x7f83cc1544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:00.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f83cc159900 0x7f83cc154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:00.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83cc15cc80 0x7f83cc15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:00.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc168fc0 con 0x7f83cc15cc80 2024-09-10T22:16:00.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc168e40 con 0x7f83cc0aee20 2024-09-10T22:16:00.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83d1a00640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f83cc169140 con 0x7f83cc159900 2024-09-10T22:16:00.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83cb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83cc15cc80 0x7f83cc15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:00.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83cb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83cc15cc80 0x7f83cc15a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36306/0 (socket says 172.21.15.66:36306) 2024-09-10T22:16:00.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83cb400640 1 -- 172.21.15.66:0/1726128279 learned_addr learned my addr 172.21.15.66:0/1726128279 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:16:00.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83c3e00640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f83cc159900 0x7f83cc154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83caa00640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83cc0aee20 0x7f83cc1544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83cb400640 1 -- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f83cc159900 msgr2=0x7f83cc154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83cb400640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f83cc159900 0x7f83cc154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83cb400640 1 -- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83cc0aee20 msgr2=0x7f83cc1544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83cb400640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83cc0aee20 0x7f83cc1544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83cb400640 1 -- 172.21.15.66:0/1726128279 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83cc0adbb0 con 0x7f83cc15cc80 2024-09-10T22:16:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83cb400640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83cc15cc80 0x7f83cc15a9f0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009bd0 tx=0x7f83c40a5960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.822+0000 7f83c2a00640 1 -- 172.21.15.66:0/1726128279 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40bb070 con 0x7f83cc15cc80 2024-09-10T22:16:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.826+0000 7f83c2a00640 1 -- 172.21.15.66:0/1726128279 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f83c4004e20 con 0x7f83cc15cc80 2024-09-10T22:16:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.826+0000 7f83d1a00640 1 -- 172.21.15.66:0/1726128279 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83cc15bde0 con 0x7f83cc15cc80 2024-09-10T22:16:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.826+0000 7f83c2a00640 1 -- 172.21.15.66:0/1726128279 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40b2040 con 0x7f83cc15cc80 2024-09-10T22:16:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.826+0000 7f83d1a00640 1 -- 172.21.15.66:0/1726128279 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83cc0ade50 con 0x7f83cc15cc80 2024-09-10T22:16:00.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.826+0000 7f83d1a00640 1 -- 172.21.15.66:0/1726128279 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83cc0ca080 con 0x7f83cc15cc80 2024-09-10T22:16:00.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.826+0000 7f83c2a00640 1 -- 172.21.15.66:0/1726128279 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 50) ==== 99862+0+0 (secure 0 0 0) 0x7f83c40b5030 con 0x7f83cc15cc80 2024-09-10T22:16:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.830+0000 7f83c2a00640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f83a8074c20 0x7f83a80770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.830+0000 7f83c2a00640 1 -- 172.21.15.66:0/1726128279 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5083+0+0 (secure 0 0 0) 0x7f83c412c4c0 con 0x7f83cc15cc80 2024-09-10T22:16:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.830+0000 7f83caa00640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f83a8074c20 0x7f83a80770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:00.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.830+0000 7f83caa00640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f83a8074c20 0x7f83a80770e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f83bc004e30 tx=0x7f83bc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:00.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:00.834+0000 7f83c2a00640 1 -- 172.21.15.66:0/1726128279 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f83c4016990 con 0x7f83cc15cc80 2024-09-10T22:16:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 -- 172.21.15.66:0/1726128279 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83cc0ae950 con 0x7f83cc15cc80 2024-09-10T22:16:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83c2a00640 1 -- 172.21.15.66:0/1726128279 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+3905 (secure 0 0 0) 0x7f83c40f9580 con 0x7f83cc15cc80 2024-09-10T22:16:01.048 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:16:01.048 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":99,"btime":"2024-09-10T22:15:28:217717+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/401729631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":401729631},{"type":"v1","addr":"172.21.15.94:6836","nonce":401729631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/996427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":996427549},{"type":"v1","addr":"172.21.15.94:6837","nonce":996427549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2947602030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2947602030},{"type":"v1","addr":"172.21.15.66:6833","nonce":2947602030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9816,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2788007396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2788007396},{"type":"v1","addr":"172.21.15.66:6835","nonce":2788007396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[]} 2024-09-10T22:16:01.048 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 99 2024-09-10T22:16:01.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 -- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f83a8074c20 msgr2=0x7f83a80770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:01.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f83a8074c20 0x7f83a80770e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f83bc004e30 tx=0x7f83bc009040 comp rx=0 tx=0).stop 2024-09-10T22:16:01.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 -- 172.21.15.66:0/1726128279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83cc15cc80 msgr2=0x7f83cc15a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:01.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83cc15cc80 0x7f83cc15a9f0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009bd0 tx=0x7f83c40a5960 comp rx=0 tx=0).stop 2024-09-10T22:16:01.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 -- 172.21.15.66:0/1726128279 shutdown_connections 2024-09-10T22:16:01.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:6800/2679227974,v1:172.21.15.94:6801/2679227974] conn(0x7f83a8074c20 0x7f83a80770e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:01.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83cc15cc80 0x7f83cc15a9f0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:01.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f83cc159900 0x7f83cc154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:01.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 --2- 172.21.15.66:0/1726128279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83cc0aee20 0x7f83cc1544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:01.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 -- 172.21.15.66:0/1726128279 >> 172.21.15.66:0/1726128279 conn(0x7f83cc0cfa30 msgr2=0x7f83cc06f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:01.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 -- 172.21.15.66:0/1726128279 shutdown_connections 2024-09-10T22:16:01.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:16:01.042+0000 7f83d1a00640 1 -- 172.21.15.66:0/1726128279 wait complete. 2024-09-10T22:16:01.070 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-10T22:16:01.070 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T22:16:01.070 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:16:01.071 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:01.084 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:16:01.084 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:16:01.084 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:01.139 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:16:01.139 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:16:01.139 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:01.154 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:16:01.154 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:16:01.154 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:01.213 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:16:01.213 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T22:16:01.214 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T22:16:01.214 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:03.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-10T22:16:07.315 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T22:16:07.316 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T22:16:07.316 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:09.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-10T22:16:09.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-10T22:16:13.418 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T22:16:13.418 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T22:16:13.418 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:15.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-10T22:16:15.314 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-10T22:16:19.520 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T22:16:19.520 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T22:16:19.521 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:21.221 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-10T22:16:21.221 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-10T22:16:25.622 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T22:16:25.622 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T22:16:25.623 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:25.714 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T22:16:25.714 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T22:16:25.714 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:25.802 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T22:16:25.802 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T22:16:25.802 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:26.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-10T22:16:26.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-10T22:16:28.252 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:16:28.256 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:16:31.904 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T22:16:31.905 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T22:16:31.905 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:31.998 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T22:16:31.999 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T22:16:31.999 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T22:16:31.999 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:32.061 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T22:16:32.062 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T22:16:32.062 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:32.258 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~6s 2024-09-10T22:16:32.258 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-09-10T22:16:37.561 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~5s 2024-09-10T22:16:37.561 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-10T22:16:38.164 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T22:16:38.164 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T22:16:38.164 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T22:16:38.165 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:38.189 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T22:16:38.189 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T22:16:38.189 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:38.208 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T22:16:38.208 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T22:16:38.208 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:16:38.269 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T22:16:38.269 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T22:16:38.269 DEBUG:teuthology.orchestra.run.smithi066:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-10T22:16:38.314 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:16:38.314 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T22:16:38.549 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:16:38.549 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T22:16:38.763 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:16:38.763 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T22:16:39.087 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:16:39.088 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T22:16:39.287 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:16:39.287 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T22:16:39.528 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:16:39.528 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T22:16:39.748 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:16:39.748 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T22:16:39.921 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:16:39.921 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T22:16:40.091 INFO:tasks.ceph:Archiving mon data... 2024-09-10T22:16:40.092 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705/data/mon.a.tgz 2024-09-10T22:16:40.093 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-09-10T22:16:40.102 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.Yonc157SFI 2024-09-10T22:16:40.102 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Yonc157SFI 2024-09-10T22:16:40.458 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.Yonc157SFI 2024-09-10T22:16:40.535 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.Yonc157SFI is 2MB 2024-09-10T22:16:40.617 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.Yonc157SFI 2024-09-10T22:16:40.625 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705/data/mon.b.tgz 2024-09-10T22:16:40.626 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-10T22:16:40.635 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.ZZRYbwnc9D 2024-09-10T22:16:40.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ZZRYbwnc9D 2024-09-10T22:16:40.997 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.ZZRYbwnc9D 2024-09-10T22:16:41.069 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.ZZRYbwnc9D is 2MB 2024-09-10T22:16:41.157 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.ZZRYbwnc9D 2024-09-10T22:16:41.167 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705/data/mon.c.tgz 2024-09-10T22:16:41.167 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-10T22:16:41.221 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.V5QgbzDsHT 2024-09-10T22:16:41.221 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.V5QgbzDsHT 2024-09-10T22:16:41.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.V5QgbzDsHT 2024-09-10T22:16:41.653 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.V5QgbzDsHT is 2MB 2024-09-10T22:16:41.733 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.V5QgbzDsHT 2024-09-10T22:16:41.742 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T22:16:41.742 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T22:16:41.745 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T22:16:41.814 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T22:16:41.853 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T22:16:41.853 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T22:16:41.855 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705/remote/smithi066/crash 2024-09-10T22:16:41.856 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T22:16:41.884 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705/remote/smithi094/crash 2024-09-10T22:16:41.885 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T22:16:41.911 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T22:16:41.911 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:16:41.937 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:16:42.009 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T22:16:42.043 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T22:16:42.061 INFO:tasks.ceph:Compressing logs... 2024-09-10T22:16:42.062 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T22:16:42.088 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T22:16:42.110 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36392.log 2024-09-10T22:16:42.111 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44506.log 2024-09-10T22:16:42.111 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39725.log 2024-09-10T22:16:42.112 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36392.log.gz 2024-09-10T22:16:42.112 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42928.log 2024-09-10T22:16:42.113 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44506.log.gz 2024-09-10T22:16:42.113 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28880.log 2024-09-10T22:16:42.114 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39725.log.gz 2024-09-10T22:16:42.114 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41296.log 2024-09-10T22:16:42.114 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42928.log.gz 2024-09-10T22:16:42.115 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54021.log 2024-09-10T22:16:42.115 INFO:teuthology.orchestra.run.smithi066.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28880.log.gz 2024-09-10T22:16:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45330.log 2024-09-10T22:16:42.116 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41296.log.gz 2024-09-10T22:16:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48123.log 2024-09-10T22:16:42.117 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54021.log.gz 2024-09-10T22:16:42.117 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50496.log 2024-09-10T22:16:42.118 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45330.log.gz 2024-09-10T22:16:42.118 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43502.log 2024-09-10T22:16:42.119 INFO:teuthology.orchestra.run.smithi066.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48123.log.gz 2024-09-10T22:16:42.119 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53853.log 2024-09-10T22:16:42.119 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50496.log.gz 2024-09-10T22:16:42.120 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52209.log 2024-09-10T22:16:42.120 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43502.log.gz 2024-09-10T22:16:42.121 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38198.log 2024-09-10T22:16:42.121 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53853.log.gz 2024-09-10T22:16:42.122 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39533.log 2024-09-10T22:16:42.122 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52209.log.gz 2024-09-10T22:16:42.122 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40471.log 2024-09-10T22:16:42.123 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38198.log.gz 2024-09-10T22:16:42.123 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35601.log 2024-09-10T22:16:42.124 INFO:teuthology.orchestra.run.smithi066.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.39533.log.gz 2024-09-10T22:16:42.124 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51216.log 2024-09-10T22:16:42.124 INFO:teuthology.orchestra.run.smithi066.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40471.log.gz 2024-09-10T22:16:42.125 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38658.log 2024-09-10T22:16:42.125 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35601.log.gz 2024-09-10T22:16:42.126 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51216.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51216.log.gz 2024-09-10T22:16:42.126 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55091.log 2024-09-10T22:16:42.127 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54047.log 2024-09-10T22:16:42.127 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38658.log.gz 2024-09-10T22:16:42.128 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39299.log 2024-09-10T22:16:42.128 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55091.log.gz 2024-09-10T22:16:42.129 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48556.log 2024-09-10T22:16:42.129 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54047.log.gz 2024-09-10T22:16:42.130 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54891.log 2024-09-10T22:16:42.130 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39299.log.gz 2024-09-10T22:16:42.130 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48714.log 2024-09-10T22:16:42.131 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48556.log.gz 2024-09-10T22:16:42.131 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39165.log 2024-09-10T22:16:42.131 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54891.log.gz 2024-09-10T22:16:42.132 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28892.log 2024-09-10T22:16:42.132 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48714.log.gz 2024-09-10T22:16:42.133 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40841.log 2024-09-10T22:16:42.133 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39165.log.gz 2024-09-10T22:16:42.134 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28892.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28892.log.gz 2024-09-10T22:16:42.134 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45200.log 2024-09-10T22:16:42.135 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.40841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50165.log 2024-09-10T22:16:42.136 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55496.log 2024-09-10T22:16:42.136 INFO:teuthology.orchestra.run.smithi066.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.45200.log.gz 2024-09-10T22:16:42.136 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44558.log 2024-09-10T22:16:42.137 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50165.log.gz 2024-09-10T22:16:42.137 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46851.log 2024-09-10T22:16:42.138 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.55496.log.gz 2024-09-10T22:16:42.138 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54103.log 2024-09-10T22:16:42.139 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44558.log.gz 2024-09-10T22:16:42.139 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44376.log 2024-09-10T22:16:42.139 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46851.log.gz 2024-09-10T22:16:42.140 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54103.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T22:16:42.140 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54103.log.gz 2024-09-10T22:16:42.141 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41825.log 2024-09-10T22:16:42.141 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44376.log.gz 2024-09-10T22:16:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28996.log 2024-09-10T22:16:42.142 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51743.log 2024-09-10T22:16:42.143 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33554.log 2024-09-10T22:16:42.143 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52714.log 2024-09-10T22:16:42.143 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41825.log.gz 2024-09-10T22:16:42.144 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40012.log 2024-09-10T22:16:42.145 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51743.log.gz 2024-09-10T22:16:42.145 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.28996.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T22:16:42.145 INFO:teuthology.orchestra.run.smithi094.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.28996.log.gz 2024-09-10T22:16:42.146 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.1.33554.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T22:16:42.146 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.7.log: 87.7% -- replaced with /var/log/ceph/ceph-client.1.33554.log.gz 2024-09-10T22:16:42.146 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29051.log 2024-09-10T22:16:42.146 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28972.log 2024-09-10T22:16:42.149 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29075.log 2024-09-10T22:16:42.149 INFO:teuthology.orchestra.run.smithi094.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29051.log.gz 2024-09-10T22:16:42.149 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.28972.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T22:16:42.149 INFO:teuthology.orchestra.run.smithi094.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28972.log.gz 2024-09-10T22:16:42.149 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29075.log: gzip 86.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.29075.log.gz --verbose 2024-09-10T22:16:42.149 INFO:teuthology.orchestra.run.smithi094.stderr: -- /var/log/ceph/ceph-client.admin.29099.log 2024-09-10T22:16:42.149 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29123.log 2024-09-10T22:16:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38461.log 2024-09-10T22:16:42.150 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52714.log.gz 2024-09-10T22:16:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51374.log 2024-09-10T22:16:42.151 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40012.log.gz 2024-09-10T22:16:42.151 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40171.log 2024-09-10T22:16:42.151 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38461.log.gz 2024-09-10T22:16:42.151 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36366.log 2024-09-10T22:16:42.151 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51374.log.gz 2024-09-10T22:16:42.151 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47076.log 2024-09-10T22:16:42.151 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40171.log.gz 2024-09-10T22:16:42.151 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41426.log 2024-09-10T22:16:42.151 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36366.log.gz 2024-09-10T22:16:42.155 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29147.log 2024-09-10T22:16:42.155 INFO:teuthology.orchestra.run.smithi094.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29099.log.gz 2024-09-10T22:16:42.155 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29123.log: gzip -5 --verbose -- 86.8% /var/log/ceph/ceph-mgr.x.log -- replaced with /var/log/ceph/ceph-client.admin.29123.log.gz 2024-09-10T22:16:42.155 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-10T22:16:42.156 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29027.log 2024-09-10T22:16:42.156 INFO:teuthology.orchestra.run.smithi094.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29147.log.gz 2024-09-10T22:16:42.156 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T22:16:42.156 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29027.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T22:16:42.156 INFO:teuthology.orchestra.run.smithi094.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29027.log.gz 2024-09-10T22:16:42.156 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T22:16:42.157 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43927.log 87.4% 2024-09-10T22:16:42.157 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47076.log.gz 2024-09-10T22:16:42.157 INFO:teuthology.orchestra.run.smithi066.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T22:16:42.157 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46383.log 2024-09-10T22:16:42.157 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41426.log.gz 2024-09-10T22:16:42.157 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47011.log 2024-09-10T22:16:42.157 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43927.log.gz 2024-09-10T22:16:42.157 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43700.log 2024-09-10T22:16:42.157 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46383.log.gz 2024-09-10T22:16:42.157 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44532.log 2024-09-10T22:16:42.157 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47011.log.gz 2024-09-10T22:16:42.157 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43344.log 2024-09-10T22:16:42.158 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43700.log.gz 2024-09-10T22:16:42.158 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35696.log 2024-09-10T22:16:42.158 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44532.log.gz 2024-09-10T22:16:42.159 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T22:16:42.160 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49510.log 2024-09-10T22:16:42.160 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43344.log.gz 2024-09-10T22:16:42.160 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.2.35696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55626.log 2024-09-10T22:16:42.160 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49510.log: 89.2% -- replaced with /var/log/ceph/ceph-client.2.35696.log.gz 2024-09-10T22:16:42.160 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45000.log 2024-09-10T22:16:42.160 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49510.log.gz 2024-09-10T22:16:42.160 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34790.log 2024-09-10T22:16:42.162 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.d.log: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T22:16:42.162 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-10T22:16:42.162 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55626.log.gz 2024-09-10T22:16:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54472.log 2024-09-10T22:16:42.163 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45000.log.gz 2024-09-10T22:16:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43084.log 2024-09-10T22:16:42.163 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34790.log.gz 2024-09-10T22:16:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41322.log 2024-09-10T22:16:42.165 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T22:16:42.165 INFO:teuthology.orchestra.run.smithi094.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T22:16:42.166 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54472.log.gz 2024-09-10T22:16:42.166 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43084.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.43084.log.gz 2024-09-10T22:16:42.166 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39663.log 2024-09-10T22:16:42.166 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47494.log 2024-09-10T22:16:42.166 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41322.log.gz 2024-09-10T22:16:42.166 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46435.log 2024-09-10T22:16:42.166 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39663.log.gz 2024-09-10T22:16:42.166 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39273.log 2024-09-10T22:16:42.167 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47494.log.gz 2024-09-10T22:16:42.167 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36592.log 2024-09-10T22:16:42.167 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46435.log.gz 2024-09-10T22:16:42.167 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49905.log 2024-09-10T22:16:42.168 INFO:teuthology.orchestra.run.smithi066.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.39273.log.gz 2024-09-10T22:16:42.168 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45036.log 2024-09-10T22:16:42.169 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36592.log.gz 2024-09-10T22:16:42.169 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47416.log 2024-09-10T22:16:42.170 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49905.log.gz 2024-09-10T22:16:42.170 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45941.log 2024-09-10T22:16:42.170 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45036.log.gz 2024-09-10T22:16:42.171 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47468.log 2024-09-10T22:16:42.171 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47416.log.gz 2024-09-10T22:16:42.172 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48688.log 2024-09-10T22:16:42.172 INFO:teuthology.orchestra.run.smithi066.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.45941.log.gz 2024-09-10T22:16:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55808.log 2024-09-10T22:16:42.173 INFO:teuthology.orchestra.run.smithi066.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47468.log.gz 2024-09-10T22:16:42.174 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-10T22:16:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49094.log 2024-09-10T22:16:42.174 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48688.log.gz 2024-09-10T22:16:42.175 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52079.log 2024-09-10T22:16:42.175 INFO:teuthology.orchestra.run.smithi066.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55808.log.gz 2024-09-10T22:16:42.175 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33673.log 2024-09-10T22:16:42.176 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48608.log 2024-09-10T22:16:42.176 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49094.log.gz 2024-09-10T22:16:42.176 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39429.log 2024-09-10T22:16:42.176 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52079.log.gz 2024-09-10T22:16:42.177 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42166.log 2024-09-10T22:16:42.177 INFO:teuthology.orchestra.run.smithi066.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.48608.log.gz 2024-09-10T22:16:42.178 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54446.log 2024-09-10T22:16:42.178 INFO:teuthology.orchestra.run.smithi066.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.39429.log.gz 2024-09-10T22:16:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34987.log 2024-09-10T22:16:42.179 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42166.log.gz 2024-09-10T22:16:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54446.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T22:16:42.180 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54446.log.gz 2024-09-10T22:16:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55782.log 2024-09-10T22:16:42.181 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34987.log.gz 2024-09-10T22:16:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34785.log 2024-09-10T22:16:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43882.log 2024-09-10T22:16:42.182 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55782.log.gz 2024-09-10T22:16:42.183 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42311.log 2024-09-10T22:16:42.183 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34785.log.gz 2024-09-10T22:16:42.184 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43882.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43882.log.gz 2024-09-10T22:16:42.184 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42545.log 2024-09-10T22:16:42.185 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38524.log 2024-09-10T22:16:42.185 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42311.log.gz 2024-09-10T22:16:42.186 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.3.33673.log: 87.3% -- replaced with /var/log/ceph/ceph-client.3.33673.log.gz 2024-09-10T22:16:42.186 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42545.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42545.log.gz 2024-09-10T22:16:42.186 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41650.log 2024-09-10T22:16:42.187 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50384.log 2024-09-10T22:16:42.188 INFO:teuthology.orchestra.run.smithi066.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38524.log.gz 2024-09-10T22:16:42.188 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41650.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41650.log.gz 2024-09-10T22:16:42.188 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43726.log 2024-09-10T22:16:42.189 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47258.log 2024-09-10T22:16:42.190 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50384.log.gz 2024-09-10T22:16:42.190 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43726.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43726.log.gz 2024-09-10T22:16:42.190 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46513.log 2024-09-10T22:16:42.191 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44136.log 2024-09-10T22:16:42.192 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47258.log: /var/log/ceph/ceph-client.admin.46513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51639.log 2024-09-10T22:16:42.192 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47258.log.gz 2024-09-10T22:16:42.192 INFO:teuthology.orchestra.run.smithi066.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46513.log.gz 2024-09-10T22:16:42.193 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51112.log 2024-09-10T22:16:42.193 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44136.log.gz 2024-09-10T22:16:42.194 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51639.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51639.log.gz 2024-09-10T22:16:42.194 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47785.log 2024-09-10T22:16:42.195 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51821.log 2024-09-10T22:16:42.195 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51112.log.gz 2024-09-10T22:16:42.196 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53620.log 2024-09-10T22:16:42.196 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47785.log.gz 2024-09-10T22:16:42.197 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48497.log 2024-09-10T22:16:42.197 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51821.log.gz 2024-09-10T22:16:42.198 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52157.log 2024-09-10T22:16:42.198 INFO:teuthology.orchestra.run.smithi066.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53620.log.gz 2024-09-10T22:16:42.199 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53408.log 2024-09-10T22:16:42.199 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48497.log.gz 2024-09-10T22:16:42.200 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52157.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52157.log.gz 2024-09-10T22:16:42.200 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48990.log 2024-09-10T22:16:42.201 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54550.log 2024-09-10T22:16:42.201 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53408.log.gz 2024-09-10T22:16:42.202 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49619.log 2024-09-10T22:16:42.202 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48990.log.gz 2024-09-10T22:16:42.203 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45995.log 2024-09-10T22:16:42.203 INFO:teuthology.orchestra.run.smithi066.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.54550.log.gz 2024-09-10T22:16:42.204 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49619.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49619.log.gz 2024-09-10T22:16:42.204 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44454.log 2024-09-10T22:16:42.205 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45995.log: /var/log/ceph/ceph-client.admin.49565.log 2024-09-10T22:16:42.205 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44454.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.45995.log.gz 2024-09-10T22:16:42.205 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41929.log 2024-09-10T22:16:42.206 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44454.log.gz 2024-09-10T22:16:42.207 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43778.log 2024-09-10T22:16:42.207 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49565.log.gz 2024-09-10T22:16:42.207 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41270.log 2024-09-10T22:16:42.208 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41929.log.gz 2024-09-10T22:16:42.208 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54761.log 2024-09-10T22:16:42.208 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43778.log.gz 2024-09-10T22:16:42.209 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34432.log 2024-09-10T22:16:42.209 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41270.log.gz 2024-09-10T22:16:42.210 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54761.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54761.log.gz 2024-09-10T22:16:42.211 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50548.log 2024-09-10T22:16:42.211 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41244.log 2024-09-10T22:16:42.212 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34432.log.gz 2024-09-10T22:16:42.212 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55992.log 2024-09-10T22:16:42.212 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50548.log.gz 2024-09-10T22:16:42.213 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42255.log 2024-09-10T22:16:42.213 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41244.log.gz 2024-09-10T22:16:42.214 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45408.log 2024-09-10T22:16:42.214 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55992.log.gz 2024-09-10T22:16:42.215 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35930.log 2024-09-10T22:16:42.216 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42255.log.gz 2024-09-10T22:16:42.218 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50470.log 2024-09-10T22:16:42.218 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45408.log.gz 2024-09-10T22:16:42.218 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36894.log 2024-09-10T22:16:42.218 INFO:teuthology.orchestra.run.smithi066.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35930.log.gz 2024-09-10T22:16:42.219 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43318.log 2024-09-10T22:16:42.219 INFO:teuthology.orchestra.run.smithi066.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.50470.log.gz 2024-09-10T22:16:42.219 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37995.log 2024-09-10T22:16:42.219 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36894.log.gz 2024-09-10T22:16:42.220 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53957.log 2024-09-10T22:16:42.220 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43318.log.gz 2024-09-10T22:16:42.221 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52766.log 2024-09-10T22:16:42.221 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37995.log.gz 2024-09-10T22:16:42.222 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44922.log 2024-09-10T22:16:42.222 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53957.log.gz 2024-09-10T22:16:42.223 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34300.log 2024-09-10T22:16:42.224 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52766.log.gz 2024-09-10T22:16:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42281.log 2024-09-10T22:16:42.225 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44922.log.gz 2024-09-10T22:16:42.225 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34300.log: gzip -5 --verbose 87.6% -- -- replaced with /var/log/ceph/ceph-client.admin.34300.log.gz 2024-09-10T22:16:42.226 INFO:teuthology.orchestra.run.smithi066.stderr: /var/log/ceph/ceph-client.admin.41020.log 2024-09-10T22:16:42.226 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50217.log 2024-09-10T22:16:42.227 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41020.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42281.log.gz 2024-09-10T22:16:42.227 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42417.log 2024-09-10T22:16:42.227 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41020.log.gz 2024-09-10T22:16:42.228 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46409.log 2024-09-10T22:16:42.228 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50217.log.gz 2024-09-10T22:16:42.229 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50243.log 2024-09-10T22:16:42.229 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42417.log.gz 2024-09-10T22:16:42.230 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46409.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46409.log.gz 2024-09-10T22:16:42.230 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53983.log 2024-09-10T22:16:42.232 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46057.log 2024-09-10T22:16:42.232 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50243.log.gz 2024-09-10T22:16:42.234 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43606.log 2024-09-10T22:16:42.234 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53983.log.gz 2024-09-10T22:16:42.234 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40785.log 2024-09-10T22:16:42.235 INFO:teuthology.orchestra.run.smithi066.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46057.log.gz 2024-09-10T22:16:42.235 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51691.log 2024-09-10T22:16:42.235 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43606.log.gz 2024-09-10T22:16:42.236 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45864.log 2024-09-10T22:16:42.236 INFO:teuthology.orchestra.run.smithi066.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40785.log.gz 2024-09-10T22:16:42.236 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52274.log 2024-09-10T22:16:42.236 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51691.log.gz 2024-09-10T22:16:42.238 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34484.log 2024-09-10T22:16:42.238 INFO:teuthology.orchestra.run.smithi066.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.45864.log.gz 2024-09-10T22:16:42.238 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45763.log 87.0% 2024-09-10T22:16:42.238 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52274.log.gz 2024-09-10T22:16:42.239 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44032.log 2024-09-10T22:16:42.240 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34484.log.gz 2024-09-10T22:16:42.241 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45763.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45763.log.gz 2024-09-10T22:16:42.241 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38762.log 2024-09-10T22:16:42.241 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41218.log 2024-09-10T22:16:42.242 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44032.log.gz 2024-09-10T22:16:42.242 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41348.log 2024-09-10T22:16:42.242 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38762.log.gz 2024-09-10T22:16:42.243 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51949.log 2024-09-10T22:16:42.243 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41218.log.gz 2024-09-10T22:16:42.244 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45648.log 2024-09-10T22:16:42.244 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41348.log.gz 2024-09-10T22:16:42.245 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48381.log 2024-09-10T22:16:42.245 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51949.log.gz 2024-09-10T22:16:42.246 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36950.log 2024-09-10T22:16:42.246 INFO:teuthology.orchestra.run.smithi066.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.45648.log.gz 2024-09-10T22:16:42.247 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52740.log 2024-09-10T22:16:42.247 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48381.log.gz 2024-09-10T22:16:42.248 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42365.log 2024-09-10T22:16:42.248 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36950.log.gz 2024-09-10T22:16:42.249 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43398.log 2024-09-10T22:16:42.249 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52740.log.gz 2024-09-10T22:16:42.250 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36184.log 2024-09-10T22:16:42.250 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42365.log.gz 2024-09-10T22:16:42.251 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54944.log 2024-09-10T22:16:42.251 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43398.log.gz 2024-09-10T22:16:42.251 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42675.log 2024-09-10T22:16:42.252 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36184.log.gz 2024-09-10T22:16:42.252 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54944.log: gzip 87.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.54944.log.gz 2024-09-10T22:16:42.252 INFO:teuthology.orchestra.run.smithi066.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37444.log 2024-09-10T22:16:42.253 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46331.log 2024-09-10T22:16:42.254 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42675.log.gz 2024-09-10T22:16:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51439.log 2024-09-10T22:16:42.255 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37444.log.gz 2024-09-10T22:16:42.255 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50574.log 2024-09-10T22:16:42.255 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46331.log.gz 2024-09-10T22:16:42.256 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38838.log 2024-09-10T22:16:42.256 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51439.log.gz 2024-09-10T22:16:42.257 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55117.log 2024-09-10T22:16:42.257 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50574.log.gz 2024-09-10T22:16:42.258 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51008.log 2024-09-10T22:16:42.258 INFO:teuthology.orchestra.run.smithi066.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38838.log.gz 2024-09-10T22:16:42.259 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37640.log 2024-09-10T22:16:42.259 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55117.log.gz 2024-09-10T22:16:42.261 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52105.log 2024-09-10T22:16:42.261 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51008.log.gz 2024-09-10T22:16:42.261 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37028.log 2024-09-10T22:16:42.262 INFO:teuthology.orchestra.run.smithi066.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.37640.log.gz 2024-09-10T22:16:42.264 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54628.log 2024-09-10T22:16:42.264 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52105.log.gz 2024-09-10T22:16:42.264 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50035.log 2024-09-10T22:16:42.264 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37028.log.gz 2024-09-10T22:16:42.264 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54628.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54628.log.gz 2024-09-10T22:16:42.264 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T22:16:42.265 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39351.log 2024-09-10T22:16:42.265 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50035.log.gz 2024-09-10T22:16:42.265 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41192.log 2024-09-10T22:16:42.267 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40090.log 2024-09-10T22:16:42.267 INFO:teuthology.orchestra.run.smithi066.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.39351.log.gz 2024-09-10T22:16:42.267 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40915.log 2024-09-10T22:16:42.268 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41192.log.gz 2024-09-10T22:16:42.268 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42137.log 2024-09-10T22:16:42.269 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40090.log.gz 2024-09-10T22:16:42.269 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37532.log 2024-09-10T22:16:42.270 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40915.log.gz 2024-09-10T22:16:42.271 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34510.log 2024-09-10T22:16:42.271 INFO:teuthology.orchestra.run.smithi066.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.42137.log.gz 2024-09-10T22:16:42.272 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52438.log 2024-09-10T22:16:42.272 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37532.log.gz 2024-09-10T22:16:42.273 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43110.log 2024-09-10T22:16:42.273 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34510.log.gz 2024-09-10T22:16:42.274 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52438.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52438.log.gz 2024-09-10T22:16:42.274 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52005.log 2024-09-10T22:16:42.276 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55039.log 2024-09-10T22:16:42.276 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43110.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43110.log.gz 2024-09-10T22:16:42.276 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.52005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51138.log 2024-09-10T22:16:42.277 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51060.log 2024-09-10T22:16:42.277 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55039.log.gz 2024-09-10T22:16:42.278 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36698.log 2024-09-10T22:16:42.278 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51138.log.gz 2024-09-10T22:16:42.279 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55418.log 2024-09-10T22:16:42.279 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51060.log.gz 2024-09-10T22:16:42.280 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44006.log 2024-09-10T22:16:42.280 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36698.log.gz 2024-09-10T22:16:42.281 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39861.log 2024-09-10T22:16:42.281 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55418.log.gz 2024-09-10T22:16:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38710.log 2024-09-10T22:16:42.283 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44006.log.gz 2024-09-10T22:16:42.283 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39099.log 2024-09-10T22:16:42.284 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39861.log.gz 2024-09-10T22:16:42.285 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36340.log 2024-09-10T22:16:42.285 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38710.log.gz 2024-09-10T22:16:42.285 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48660.log 2024-09-10T22:16:42.286 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39099.log.gz 2024-09-10T22:16:42.287 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49419.log 2024-09-10T22:16:42.287 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36340.log.gz 2024-09-10T22:16:42.288 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54787.log 2024-09-10T22:16:42.288 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48660.log.gz 2024-09-10T22:16:42.289 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43292.log 2024-09-10T22:16:42.289 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49419.log.gz 2024-09-10T22:16:42.290 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50091.log 2024-09-10T22:16:42.291 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54787.log.gz 2024-09-10T22:16:42.291 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36016.log 2024-09-10T22:16:42.292 INFO:teuthology.orchestra.run.smithi066.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.43292.log.gz 2024-09-10T22:16:42.292 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.50091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43635.log 2024-09-10T22:16:42.294 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39072.log 2024-09-10T22:16:42.294 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36016.log.gz 2024-09-10T22:16:42.295 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43635.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43635.log.gz 2024-09-10T22:16:42.296 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39072.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39072.log.gz 2024-09-10T22:16:42.296 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40605.log 2024-09-10T22:16:42.297 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47154.log 2024-09-10T22:16:42.299 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52235.log 2024-09-10T22:16:42.299 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40605.log.gz 2024-09-10T22:16:42.300 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39757.log 2024-09-10T22:16:42.300 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47154.log.gz 2024-09-10T22:16:42.301 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52235.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52235.log.gz 2024-09-10T22:16:42.301 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54524.log 2024-09-10T22:16:42.302 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39757.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39757.log.gz 2024-09-10T22:16:42.303 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49775.log 2024-09-10T22:16:42.304 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40116.log 2024-09-10T22:16:42.304 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54524.log.gz 2024-09-10T22:16:42.305 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49775.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49775.log.gz 2024-09-10T22:16:42.306 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40116.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40116.log.gz 2024-09-10T22:16:42.307 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T22:16:42.308 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42339.log 2024-09-10T22:16:42.312 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41702.log 2024-09-10T22:16:42.312 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42339.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42339.log.gz 2024-09-10T22:16:42.312 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41702.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41702.log.gz 2024-09-10T22:16:42.312 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45916.log 2024-09-10T22:16:42.313 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54316.log 2024-09-10T22:16:42.315 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45916.log: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.45916.log.gz 2024-09-10T22:16:42.315 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54316.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54316.log.gz 2024-09-10T22:16:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52896.log 2024-09-10T22:16:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39046.log 2024-09-10T22:16:42.319 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52896.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52896.log.gz 2024-09-10T22:16:42.319 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39046.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39046.log.gz 2024-09-10T22:16:42.320 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41598.log 2024-09-10T22:16:42.321 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45226.log 2024-09-10T22:16:42.323 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41598.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41598.log.gz 2024-09-10T22:16:42.323 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45226.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45226.log.gz 2024-09-10T22:16:42.324 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37418.log 2024-09-10T22:16:42.325 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36210.log 2024-09-10T22:16:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37418.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37418.log.gz 2024-09-10T22:16:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36210.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36210.log.gz 2024-09-10T22:16:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34793.log 2024-09-10T22:16:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39377.log 2024-09-10T22:16:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34793.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34793.log.gz 2024-09-10T22:16:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39377.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39377.log.gz 2024-09-10T22:16:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38172.log 2024-09-10T22:16:42.333 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47915.log 2024-09-10T22:16:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38172.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38172.log.gz 2024-09-10T22:16:42.336 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47915.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47915.log.gz 2024-09-10T22:16:42.336 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42980.log 2024-09-10T22:16:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55143.log 2024-09-10T22:16:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42980.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.42980.log.gz 2024-09-10T22:16:42.340 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55143.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55143.log.gz 2024-09-10T22:16:42.340 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34718.log 2024-09-10T22:16:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48766.log 2024-09-10T22:16:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34718.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34718.log.gz 2024-09-10T22:16:42.344 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48766.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.48766.log.gz 2024-09-10T22:16:42.344 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39585.log 2024-09-10T22:16:42.346 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48634.log 2024-09-10T22:16:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39585.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.39585.log.gz 2024-09-10T22:16:42.348 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48634.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.48634.log.gz 2024-09-10T22:16:42.348 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34246.log 2024-09-10T22:16:42.350 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44948.log 2024-09-10T22:16:42.351 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34246.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34246.log.gz 2024-09-10T22:16:42.352 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44948.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44948.log.gz 2024-09-10T22:16:42.352 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36924.log 2024-09-10T22:16:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37288.log 2024-09-10T22:16:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36924.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36924.log.gz 2024-09-10T22:16:42.356 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37288.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37288.log.gz 2024-09-10T22:16:42.357 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54342.log 2024-09-10T22:16:42.358 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52131.log 2024-09-10T22:16:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54342.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.54342.log.gz 2024-09-10T22:16:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52131.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52131.log.gz 2024-09-10T22:16:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36158.log 2024-09-10T22:16:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43953.log 2024-09-10T22:16:42.364 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36158.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36158.log.gz 2024-09-10T22:16:42.364 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43953.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43953.log.gz 2024-09-10T22:16:42.365 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55966.log 2024-09-10T22:16:42.366 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47759.log 2024-09-10T22:16:42.368 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55966.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55966.log.gz 2024-09-10T22:16:42.368 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47759.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47759.log.gz 2024-09-10T22:16:42.369 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52490.log 2024-09-10T22:16:42.370 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46877.log 2024-09-10T22:16:42.372 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52490.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52490.log.gz 2024-09-10T22:16:42.372 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46877.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46877.log.gz 2024-09-10T22:16:42.373 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51086.log 2024-09-10T22:16:42.374 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46083.log 2024-09-10T22:16:42.376 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51086.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51086.log.gz 2024-09-10T22:16:42.376 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46083.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46083.log.gz 2024-09-10T22:16:42.377 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28906.log 2024-09-10T22:16:42.378 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35800.log 2024-09-10T22:16:42.380 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28906.log.gz 2024-09-10T22:16:42.381 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35800.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35800.log.gz 2024-09-10T22:16:42.381 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48964.log 2024-09-10T22:16:42.382 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39507.log 2024-09-10T22:16:42.384 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48964.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48964.log.gz 2024-09-10T22:16:42.385 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39507.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39507.log.gz 2024-09-10T22:16:42.385 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49801.log 2024-09-10T22:16:42.386 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52332.log 2024-09-10T22:16:42.388 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49801.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49801.log.gz 2024-09-10T22:16:42.389 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52332.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52332.log.gz 2024-09-10T22:16:42.389 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38550.log 2024-09-10T22:16:42.391 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47967.log 2024-09-10T22:16:42.392 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38550.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38550.log.gz 2024-09-10T22:16:42.393 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47967.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.47967.log.gz 2024-09-10T22:16:42.393 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55704.log 2024-09-10T22:16:42.395 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29312.log 2024-09-10T22:16:42.396 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55704.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55704.log.gz 2024-09-10T22:16:42.397 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.29312.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29312.log.gz 2024-09-10T22:16:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49827.log 2024-09-10T22:16:42.399 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48844.log 2024-09-10T22:16:42.400 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49827.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49827.log.gz 2024-09-10T22:16:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48844.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48844.log.gz 2024-09-10T22:16:42.402 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42701.log 2024-09-10T22:16:42.403 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50706.log 2024-09-10T22:16:42.405 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42701.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42701.log.gz 2024-09-10T22:16:42.405 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50706.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50706.log.gz 2024-09-10T22:16:42.406 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37366.log 2024-09-10T22:16:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38040.log 2024-09-10T22:16:42.409 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37366.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37366.log.gz 2024-09-10T22:16:42.409 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38040.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38040.log.gz 2024-09-10T22:16:42.410 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50680.log 2024-09-10T22:16:42.411 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42868.log 2024-09-10T22:16:42.413 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50680.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50680.log.gz 2024-09-10T22:16:42.413 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42868.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42868.log.gz 2024-09-10T22:16:42.414 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55346.log 2024-09-10T22:16:42.415 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45891.log 2024-09-10T22:16:42.417 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55346.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55346.log.gz 2024-09-10T22:16:42.417 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45891.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.45891.log.gz 2024-09-10T22:16:42.418 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53213.log 2024-09-10T22:16:42.419 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36842.log 2024-09-10T22:16:42.421 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53213.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53213.log.gz 2024-09-10T22:16:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36842.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36842.log.gz 2024-09-10T22:16:42.422 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53879.log 2024-09-10T22:16:42.423 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34640.log 2024-09-10T22:16:42.425 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53879.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53879.log.gz 2024-09-10T22:16:42.426 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34640.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34640.log.gz 2024-09-10T22:16:42.426 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44636.log 2024-09-10T22:16:42.427 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41851.log 2024-09-10T22:16:42.429 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44636.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44636.log.gz 2024-09-10T22:16:42.430 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41851.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41851.log.gz 2024-09-10T22:16:42.430 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43006.log 2024-09-10T22:16:42.431 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34614.log 2024-09-10T22:16:42.433 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43006.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.43006.log.gz 2024-09-10T22:16:42.434 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34614.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34614.log.gz 2024-09-10T22:16:42.434 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34744.log 2024-09-10T22:16:42.435 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46982.log 2024-09-10T22:16:42.437 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34744.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34744.log.gz 2024-09-10T22:16:42.438 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46982.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46982.log.gz 2024-09-10T22:16:42.438 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55652.log 2024-09-10T22:16:42.440 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38066.log 2024-09-10T22:16:42.441 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55652.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55652.log.gz 2024-09-10T22:16:42.442 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38066.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38066.log.gz 2024-09-10T22:16:42.442 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46799.log 2024-09-10T22:16:42.444 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40735.log 2024-09-10T22:16:42.445 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46799.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46799.log.gz 2024-09-10T22:16:42.446 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40735.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40735.log.gz 2024-09-10T22:16:42.447 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55548.log 2024-09-10T22:16:42.448 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39611.log 2024-09-10T22:16:42.450 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55548.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55548.log.gz 2024-09-10T22:16:42.450 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41520.log 2024-09-10T22:16:42.452 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39611.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39611.log.gz 2024-09-10T22:16:42.452 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55392.log 2024-09-10T22:16:42.453 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41520.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41520.log.gz 2024-09-10T22:16:42.454 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55392.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55392.log.gz 2024-09-10T22:16:42.455 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34994.log 2024-09-10T22:16:42.456 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50956.log 2024-09-10T22:16:42.457 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34994.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34994.log.gz 2024-09-10T22:16:42.458 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52662.log 2024-09-10T22:16:42.459 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50956.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50956.log.gz 2024-09-10T22:16:42.460 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40553.log 2024-09-10T22:16:42.461 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52662.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52662.log.gz 2024-09-10T22:16:42.462 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40553.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40553.log.gz 2024-09-10T22:16:42.462 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44818.log 2024-09-10T22:16:42.464 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37692.log 2024-09-10T22:16:42.465 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44818.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44818.log.gz 2024-09-10T22:16:42.466 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38120.log 2024-09-10T22:16:42.467 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37692.log.gz 2024-09-10T22:16:42.467 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35368.log 2024-09-10T22:16:42.468 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38120.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38120.log.gz 2024-09-10T22:16:42.470 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50626.log 2024-09-10T22:16:42.472 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35368.log: /var/log/ceph/ceph-client.admin.50626.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.35368.log.gz 2024-09-10T22:16:42.472 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53775.log 2024-09-10T22:16:42.472 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50626.log.gz 2024-09-10T22:16:42.473 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49302.log 2024-09-10T22:16:42.475 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53775.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53775.log.gz 2024-09-10T22:16:42.475 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49302.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49302.log.gz 2024-09-10T22:16:42.476 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39325.log 2024-09-10T22:16:42.477 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53028.log 2024-09-10T22:16:42.479 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39325.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39325.log.gz 2024-09-10T22:16:42.480 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53028.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53028.log.gz 2024-09-10T22:16:42.480 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46617.log 2024-09-10T22:16:42.481 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49146.log 2024-09-10T22:16:42.483 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46617.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46617.log.gz 2024-09-10T22:16:42.484 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49146.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49146.log.gz 2024-09-10T22:16:42.484 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45789.log 2024-09-10T22:16:42.485 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40379.log 2024-09-10T22:16:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45789.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.45789.log.gz 2024-09-10T22:16:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40379.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40379.log.gz 2024-09-10T22:16:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35264.log 2024-09-10T22:16:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38432.log 2024-09-10T22:16:42.491 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35264.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35264.log.gz 2024-09-10T22:16:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38432.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38432.log.gz 2024-09-10T22:16:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44292.log 2024-09-10T22:16:42.494 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47811.log 2024-09-10T22:16:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44292.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44292.log.gz 2024-09-10T22:16:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47811.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47811.log.gz 2024-09-10T22:16:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37214.log 2024-09-10T22:16:42.498 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34380.log 2024-09-10T22:16:42.501 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.37214.log: /var/log/ceph/ceph-client.admin.34380.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34380.log.gz 2024-09-10T22:16:42.501 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28964.log 2024-09-10T22:16:42.503 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28964.log.gz 2024-09-10T22:16:42.503 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50269.log 2024-09-10T22:16:42.506 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48906.log 2024-09-10T22:16:42.506 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50269.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50269.log.gz 2024-09-10T22:16:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48906.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48906.log.gz 2024-09-10T22:16:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50321.log 2024-09-10T22:16:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50321.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50321.log.gz 2024-09-10T22:16:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46487.log 2024-09-10T22:16:42.515 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46487.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46487.log.gz 2024-09-10T22:16:42.515 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43830.log 2024-09-10T22:16:42.521 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43830.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43830.log.gz 2024-09-10T22:16:42.521 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55007.log 2024-09-10T22:16:42.521 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55007.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55007.log.gz 2024-09-10T22:16:42.521 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55574.log 2024-09-10T22:16:42.524 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55574.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55574.log.gz 2024-09-10T22:16:42.524 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39835.log 2024-09-10T22:16:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38380.log 2024-09-10T22:16:42.527 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39835.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39835.log.gz 2024-09-10T22:16:42.528 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37262.log 2024-09-10T22:16:42.528 INFO:teuthology.orchestra.run.smithi066.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38380.log.gz 2024-09-10T22:16:42.530 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37262.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37262.log.gz 2024-09-10T22:16:42.530 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45148.log 2024-09-10T22:16:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45148.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45148.log.gz 2024-09-10T22:16:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42805.log 2024-09-10T22:16:42.536 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42805.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42805.log.gz 2024-09-10T22:16:42.536 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28904.log 2024-09-10T22:16:42.539 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28904.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28904.log.gz 2024-09-10T22:16:42.539 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43136.log 2024-09-10T22:16:42.541 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43136.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43136.log.gz 2024-09-10T22:16:42.542 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36724.log 2024-09-10T22:16:42.544 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37614.log 2024-09-10T22:16:42.544 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36724.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36724.log.gz 2024-09-10T22:16:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37614.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37614.log.gz 2024-09-10T22:16:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48045.log 2024-09-10T22:16:42.550 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48045.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.48045.log.gz 2024-09-10T22:16:42.550 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43476.log 2024-09-10T22:16:42.553 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37080.log 2024-09-10T22:16:42.553 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43476.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43476.log.gz 2024-09-10T22:16:42.556 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37080.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37080.log.gz 2024-09-10T22:16:42.556 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43752.log 2024-09-10T22:16:42.559 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43752.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43752.log.gz 2024-09-10T22:16:42.559 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50418.log 2024-09-10T22:16:42.561 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-09-10T22:16:42.562 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50418.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50418.log.gz 2024-09-10T22:16:42.562 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43424.log 2024-09-10T22:16:42.565 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34666.log 2024-09-10T22:16:42.565 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43424.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43424.log.gz 2024-09-10T22:16:42.567 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34666.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34666.log.gz 2024-09-10T22:16:42.568 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50836.log 2024-09-10T22:16:42.571 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52818.log 2024-09-10T22:16:42.571 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50836.log.gz 2024-09-10T22:16:42.572 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44058.log 2024-09-10T22:16:42.574 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52818.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52818.log.gz 2024-09-10T22:16:42.576 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39221.log 2024-09-10T22:16:42.576 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44058.log.gz 2024-09-10T22:16:42.579 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39221.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39221.log.gz 2024-09-10T22:16:42.580 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51795.log 2024-09-10T22:16:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51795.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51795.log.gz 2024-09-10T22:16:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34406.log 2024-09-10T22:16:42.585 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38146.log 2024-09-10T22:16:42.585 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34406.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34406.log.gz 2024-09-10T22:16:42.588 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38146.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38146.log.gz 2024-09-10T22:16:42.588 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36750.log 2024-09-10T22:16:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36750.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36750.log.gz 2024-09-10T22:16:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34988.log 2024-09-10T22:16:42.595 INFO:teuthology.orchestra.run.smithi094.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T22:16:42.598 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34988.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34988.log.gz 2024-09-10T22:16:42.598 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T22:16:42.598 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42391.log 2024-09-10T22:16:42.603 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51899.log 2024-09-10T22:16:42.604 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42391.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42391.log.gz 2024-09-10T22:16:42.609 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51899.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51899.log.gz 2024-09-10T22:16:42.609 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38632.log 2024-09-10T22:16:42.610 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53464.log 2024-09-10T22:16:42.611 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38632.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38632.log.gz 2024-09-10T22:16:42.616 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53464.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53464.log.gz 2024-09-10T22:16:42.616 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53905.log 2024-09-10T22:16:42.619 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39637.log 2024-09-10T22:16:42.621 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53905.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53905.log.gz 2024-09-10T22:16:42.622 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46032.log 2024-09-10T22:16:42.622 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37054.log 2024-09-10T22:16:42.623 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39637.log.gz 2024-09-10T22:16:42.627 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46032.log: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.46032.log.gz 2024-09-10T22:16:42.628 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49985.log 2024-09-10T22:16:42.628 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37054.log.gz 2024-09-10T22:16:42.632 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49985.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49985.log.gz 2024-09-10T22:16:42.633 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47527.log 2024-09-10T22:16:42.634 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37865.log 2024-09-10T22:16:42.635 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47527.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47527.log.gz 2024-09-10T22:16:42.637 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37865.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37865.log.gz 2024-09-10T22:16:42.638 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48471.log 2024-09-10T22:16:42.644 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48471.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48471.log.gz 2024-09-10T22:16:42.644 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38942.log 2024-09-10T22:16:42.647 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49042.log 2024-09-10T22:16:42.648 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51034.log 2024-09-10T22:16:42.648 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38942.log.gz 2024-09-10T22:16:42.651 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49042.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49042.log.gz 2024-09-10T22:16:42.651 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43580.log 2024-09-10T22:16:42.652 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51034.log.gz 2024-09-10T22:16:42.654 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45513.log 2024-09-10T22:16:42.656 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43580.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43580.log.gz 2024-09-10T22:16:42.662 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45513.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.45513.log.gz 2024-09-10T22:16:42.666 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42601.log 2024-09-10T22:16:42.671 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53167.log 2024-09-10T22:16:42.675 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.42601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44896.log 2024-09-10T22:16:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53167.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53167.log.gz 2024-09-10T22:16:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54208.log 2024-09-10T22:16:42.680 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44896.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44896.log.gz 2024-09-10T22:16:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51613.log 2024-09-10T22:16:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54208.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.54208.log.gz 2024-09-10T22:16:42.686 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36476.log 2024-09-10T22:16:42.688 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51613.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51613.log.gz 2024-09-10T22:16:42.691 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55221.log 2024-09-10T22:16:42.692 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36476.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36476.log.gz 2024-09-10T22:16:42.693 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36672.log 2024-09-10T22:16:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35878.log 2024-09-10T22:16:42.699 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55221.log.gz 2024-09-10T22:16:42.700 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36672.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36672.log.gz 2024-09-10T22:16:42.702 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41046.log 2024-09-10T22:16:42.704 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35878.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35878.log.gz 2024-09-10T22:16:42.706 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36620.log 2024-09-10T22:16:42.706 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41046.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41046.log.gz 2024-09-10T22:16:42.712 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55942.log 2024-09-10T22:16:42.712 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36620.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36620.log.gz 2024-09-10T22:16:42.715 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41799.log 2024-09-10T22:16:42.716 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55942.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55942.log.gz 2024-09-10T22:16:42.717 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40353.log 2024-09-10T22:16:42.719 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41799.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41799.log.gz 2024-09-10T22:16:42.720 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40353.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40353.log.gz 2024-09-10T22:16:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51717.log 2024-09-10T22:16:42.727 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54129.log 2024-09-10T22:16:42.728 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51717.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51717.log.gz 2024-09-10T22:16:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41988.log 2024-09-10T22:16:42.732 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54129.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.54129.log.gz 2024-09-10T22:16:42.733 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37506.log 2024-09-10T22:16:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46747.log 2024-09-10T22:16:42.738 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37506.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37506.log.gz 2024-09-10T22:16:42.738 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41988.log: gzip -5 87.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.41988.log.gz /var/log/ceph/ceph-client.admin.45814.log 2024-09-10T22:16:42.739 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:16:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46747.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46747.log.gz 2024-09-10T22:16:42.743 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45814.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.45814.log.gz 2024-09-10T22:16:42.743 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55470.log 2024-09-10T22:16:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55470.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55470.log.gz 2024-09-10T22:16:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52844.log 2024-09-10T22:16:42.755 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42469.log 2024-09-10T22:16:42.756 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52844.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52844.log.gz 2024-09-10T22:16:42.759 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49749.log 2024-09-10T22:16:42.760 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42469.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42469.log.gz 2024-09-10T22:16:42.762 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45278.log 2024-09-10T22:16:42.764 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49749.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49749.log.gz 2024-09-10T22:16:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42779.log 2024-09-10T22:16:42.769 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45278.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2024-09-10T22:16:42.769 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42779.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42779.log.gz 2024-09-10T22:16:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34328.log 2024-09-10T22:16:42.776 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34328.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34328.log.gz 2024-09-10T22:16:42.778 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46929.log 2024-09-10T22:16:42.783 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47338.log 2024-09-10T22:16:42.784 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46929.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46929.log.gz 2024-09-10T22:16:42.787 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51407.log 2024-09-10T22:16:42.788 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47338.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47338.log.gz 2024-09-10T22:16:42.791 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49328.log 2024-09-10T22:16:42.792 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51407.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51407.log.gz 2024-09-10T22:16:42.794 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28928.log 2024-09-10T22:16:42.796 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49328.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49328.log.gz 2024-09-10T22:16:42.797 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37392.log 2024-09-10T22:16:42.799 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28928.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28928.log.gz 2024-09-10T22:16:42.802 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49354.log 2024-09-10T22:16:42.804 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37392.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37392.log.gz 2024-09-10T22:16:42.807 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T22:16:42.808 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49354.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49354.log.gz 2024-09-10T22:16:42.810 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49484.log 2024-09-10T22:16:42.811 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51242.log 2024-09-10T22:16:42.812 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49484.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49484.log.gz 2024-09-10T22:16:42.816 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.51242.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51242.log.gz 2024-09-10T22:16:42.818 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50139.log 2024-09-10T22:16:42.823 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45434.log 2024-09-10T22:16:42.824 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50139.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50139.log.gz 2024-09-10T22:16:42.827 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53727.log 2024-09-10T22:16:42.828 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45434.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45434.log.gz 2024-09-10T22:16:42.831 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36314.log 2024-09-10T22:16:42.839 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.53727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34588.log 2024-09-10T22:16:42.840 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36314.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36314.log.gz 2024-09-10T22:16:42.843 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41112.log 2024-09-10T22:16:42.844 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34588.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34588.log.gz 2024-09-10T22:16:42.847 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52300.log 2024-09-10T22:16:42.848 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41112.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41112.log.gz 2024-09-10T22:16:42.852 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52300.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52300.log.gz 2024-09-10T22:16:42.854 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41955.log 2024-09-10T22:16:42.858 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45252.log 2024-09-10T22:16:42.860 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41955.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41955.log.gz 2024-09-10T22:16:42.862 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46956.log 2024-09-10T22:16:42.864 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45252.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45252.log.gz 2024-09-10T22:16:42.867 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48277.log 2024-09-10T22:16:42.868 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46956.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46956.log.gz 2024-09-10T22:16:42.869 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37747.log 2024-09-10T22:16:42.871 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48277.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48277.log.gz 2024-09-10T22:16:42.875 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48329.log 2024-09-10T22:16:42.876 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37747.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37747.log.gz 2024-09-10T22:16:42.879 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28952.log 2024-09-10T22:16:42.880 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48329.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48329.log.gz 2024-09-10T22:16:42.882 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38684.log 2024-09-10T22:16:42.887 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28952.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28952.log.gz 2024-09-10T22:16:42.887 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52410.log 2024-09-10T22:16:42.888 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38684.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38684.log.gz 2024-09-10T22:16:42.892 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52410.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52410.log.gz 2024-09-10T22:16:42.894 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29048.log 2024-09-10T22:16:42.899 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54182.log 2024-09-10T22:16:42.900 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.29048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29048.log.gz 2024-09-10T22:16:42.902 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55834.log 2024-09-10T22:16:42.904 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54182.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.54182.log.gz 2024-09-10T22:16:42.905 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49959.log 2024-09-10T22:16:42.910 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54498.log 2024-09-10T22:16:42.911 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55834.log.gz 2024-09-10T22:16:42.912 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49959.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49959.log.gz 2024-09-10T22:16:42.916 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54498.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54498.log.gz 2024-09-10T22:16:42.918 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41773.log 2024-09-10T22:16:42.923 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52384.log 2024-09-10T22:16:42.924 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41773.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41773.log.gz 2024-09-10T22:16:42.927 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53115.log 2024-09-10T22:16:42.928 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52384.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52384.log.gz 2024-09-10T22:16:42.929 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34220.log 2024-09-10T22:16:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53115.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53115.log.gz 2024-09-10T22:16:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54973.log 2024-09-10T22:16:42.933 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34220.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34220.log.gz 2024-09-10T22:16:42.935 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44974.log 2024-09-10T22:16:42.936 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54973.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54973.log.gz 2024-09-10T22:16:42.938 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52542.log 2024-09-10T22:16:42.939 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44974.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44974.log.gz 2024-09-10T22:16:42.941 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52542.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52542.log.gz 2024-09-10T22:16:42.942 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54262.log 2024-09-10T22:16:42.947 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55320.log 2024-09-10T22:16:42.948 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54262.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54262.log.gz 2024-09-10T22:16:42.950 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42085.log 2024-09-10T22:16:42.952 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55320.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55320.log.gz 2024-09-10T22:16:42.956 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42085.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42085.log.gz 2024-09-10T22:16:42.958 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35989.log 2024-09-10T22:16:42.963 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54813.log 2024-09-10T22:16:42.964 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35989.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35989.log.gz 2024-09-10T22:16:42.967 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51665.log 2024-09-10T22:16:42.968 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54813.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54813.log.gz 2024-09-10T22:16:42.970 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28858.log 2024-09-10T22:16:42.972 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51665.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51665.log.gz 2024-09-10T22:16:42.973 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53293.log 2024-09-10T22:16:42.974 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36976.log 2024-09-10T22:16:42.974 INFO:teuthology.orchestra.run.smithi066.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28858.log.gz 2024-09-10T22:16:42.975 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53293.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53293.log.gz 2024-09-10T22:16:42.978 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34154.log 2024-09-10T22:16:42.980 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36976.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36976.log.gz 2024-09-10T22:16:42.984 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34154.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34154.log.gz 2024-09-10T22:16:42.986 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49224.log 2024-09-10T22:16:42.991 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35006.log 2024-09-10T22:16:42.992 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49224.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49224.log.gz 2024-09-10T22:16:42.994 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35238.log 2024-09-10T22:16:42.996 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35006.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35006.log.gz 2024-09-10T22:16:43.000 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35238.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35238.log.gz 2024-09-10T22:16:43.000 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35342.log 2024-09-10T22:16:43.003 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46565.log 2024-09-10T22:16:43.005 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35342.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35342.log.gz 2024-09-10T22:16:43.007 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48433.log 2024-09-10T22:16:43.008 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46565.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46565.log.gz 2024-09-10T22:16:43.011 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53931.log 2024-09-10T22:16:43.011 INFO:teuthology.orchestra.run.smithi094.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T22:16:43.012 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48433.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48433.log.gz 2024-09-10T22:16:43.016 INFO:teuthology.orchestra.run.smithi066.stderr:gzip/var/log/ceph/ceph-client.admin.53931.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53931.log.gz 2024-09-10T22:16:43.018 INFO:teuthology.orchestra.run.smithi066.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34354.log 2024-09-10T22:16:43.024 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34354.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34354.log.gz 2024-09-10T22:16:43.024 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39783.log 2024-09-10T22:16:43.032 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39783.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39783.log.gz 2024-09-10T22:16:43.032 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51873.log 2024-09-10T22:16:43.034 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39020.log 2024-09-10T22:16:43.037 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51873.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51873.log.gz 2024-09-10T22:16:43.039 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48873.log 2024-09-10T22:16:43.040 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39020.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39020.log.gz 2024-09-10T22:16:43.044 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48873.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48873.log.gz 2024-09-10T22:16:43.044 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39403.log 2024-09-10T22:16:43.046 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49723.log 2024-09-10T22:16:43.050 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39403.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39403.log.gz 2024-09-10T22:16:43.052 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49723.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49723.log.gz 2024-09-10T22:16:43.052 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37588.log 2024-09-10T22:16:43.054 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43856.log 2024-09-10T22:16:43.059 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48229.log 2024-09-10T22:16:43.060 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43856.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43856.log.gz 2024-09-10T22:16:43.063 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37588.log.gz 2024-09-10T22:16:43.063 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47733.log 2024-09-10T22:16:43.067 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.48229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47390.log 2024-09-10T22:16:43.068 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47733.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47733.log.gz 2024-09-10T22:16:43.070 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38224.log 2024-09-10T22:16:43.072 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47390.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47390.log.gz 2024-09-10T22:16:43.076 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38224.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38224.log.gz 2024-09-10T22:16:43.078 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53568.log 2024-09-10T22:16:43.083 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45706.log 2024-09-10T22:16:43.084 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50444.log 2024-09-10T22:16:43.084 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53568.log.gz 2024-09-10T22:16:43.085 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45706.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45706.log.gz 2024-09-10T22:16:43.087 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40631.log 2024-09-10T22:16:43.091 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50444.log: 86.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47675.log 2024-09-10T22:16:43.092 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40631.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40631.log.gz 2024-09-10T22:16:43.094 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50444.log.gz 2024-09-10T22:16:43.096 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47675.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47675.log.gz 2024-09-10T22:16:43.096 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49593.log 2024-09-10T22:16:43.097 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38894.log 2024-09-10T22:16:43.099 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49593.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49593.log.gz 2024-09-10T22:16:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51504.log 2024-09-10T22:16:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.38894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36073.log 2024-09-10T22:16:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51504.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51504.log.gz 2024-09-10T22:16:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48019.log 2024-09-10T22:16:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36073.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36073.log.gz 2024-09-10T22:16:43.116 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49250.log 2024-09-10T22:16:43.116 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48019.log.gz 2024-09-10T22:16:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49250.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49250.log.gz 2024-09-10T22:16:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41903.log 2024-09-10T22:16:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35826.log 2024-09-10T22:16:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41903.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41903.log.gz 2024-09-10T22:16:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49016.log 2024-09-10T22:16:43.132 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35826.log.gz 2024-09-10T22:16:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44428.log 2024-09-10T22:16:43.140 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49016.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49016.log.gz 2024-09-10T22:16:43.140 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51585.log 2024-09-10T22:16:43.140 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44428.log.gz 2024-09-10T22:16:43.141 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50784.log 2024-09-10T22:16:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51585.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51585.log.gz 2024-09-10T22:16:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54576.log 2024-09-10T22:16:43.148 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50784.log.gz 2024-09-10T22:16:43.157 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54576.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54576.log.gz 2024-09-10T22:16:43.157 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54656.log 2024-09-10T22:16:43.163 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54656.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54656.log.gz 2024-09-10T22:16:43.163 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T22:16:43.167 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45674.log 2024-09-10T22:16:43.172 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.45674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35004.log 2024-09-10T22:16:43.176 INFO:teuthology.orchestra.run.smithi066.stderr: 86.2% 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T22:16:43.178 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45674.log.gz 2024-09-10T22:16:43.180 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35004.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35004.log.gz 2024-09-10T22:16:43.182 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52688.log 2024-09-10T22:16:43.187 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35212.log 2024-09-10T22:16:43.188 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52688.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52688.log.gz 2024-09-10T22:16:43.191 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48938.log 2024-09-10T22:16:43.192 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40657.log 2024-09-10T22:16:43.194 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35212.log.gz 2024-09-10T22:16:43.199 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48938.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.48938.log.gz 2024-09-10T22:16:43.200 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34272.log 2024-09-10T22:16:43.200 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40657.log.gz 2024-09-10T22:16:43.207 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37002.log 2024-09-10T22:16:43.208 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34272.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34272.log.gz 2024-09-10T22:16:43.209 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37002.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37002.log.gz 2024-09-10T22:16:43.214 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35551.log 2024-09-10T22:16:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41374.log 2024-09-10T22:16:43.221 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.35551.log: 89.8% -- replaced with /var/log/ceph/ceph-client.0.35551.log.gz 2024-09-10T22:16:43.223 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43804.log 2024-09-10T22:16:43.224 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47206.log 2024-09-10T22:16:43.226 INFO:teuthology.orchestra.run.smithi066.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.41374.log.gz 2024-09-10T22:16:43.231 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43804.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43804.log.gz 2024-09-10T22:16:43.232 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51164.log 2024-09-10T22:16:43.232 INFO:teuthology.orchestra.run.smithi066.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47206.log.gz 2024-09-10T22:16:43.240 INFO:teuthology.orchestra.run.smithi066.stderr:gzip/var/log/ceph/ceph-client.admin.51164.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51164.log.gz 2024-09-10T22:16:43.242 INFO:teuthology.orchestra.run.smithi066.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52630.log 2024-09-10T22:16:43.247 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37969.log 2024-09-10T22:16:43.248 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52630.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52630.log.gz 2024-09-10T22:16:43.251 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53239.log 2024-09-10T22:16:43.256 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37969.log: 87.2%/var/log/ceph/ceph-client.admin.53239.log: -- replaced with /var/log/ceph/ceph-client.admin.37969.log.gz 2024-09-10T22:16:43.256 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53239.log.gz 2024-09-10T22:16:43.256 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40683.log 2024-09-10T22:16:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50982.log 2024-09-10T22:16:43.264 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40683.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40683.log.gz 2024-09-10T22:16:43.268 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50982.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50982.log.gz 2024-09-10T22:16:43.268 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37340.log 2024-09-10T22:16:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53356.log 2024-09-10T22:16:43.276 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37340.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37340.log.gz 2024-09-10T22:16:43.281 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53356.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44870.log 2024-09-10T22:16:43.281 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53356.log.gz 2024-09-10T22:16:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43528.log 2024-09-10T22:16:43.289 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44870.log: /var/log/ceph/ceph-client.admin.43528.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51556.log 2024-09-10T22:16:43.289 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43528.log.gz 2024-09-10T22:16:43.291 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44870.log.gz 2024-09-10T22:16:43.292 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51556.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51556.log.gz 2024-09-10T22:16:43.294 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44318.log 2024-09-10T22:16:43.295 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37780.log 2024-09-10T22:16:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44318.log: /var/log/ceph/ceph-client.admin.37780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43980.log 2024-09-10T22:16:43.300 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37780.log.gz 2024-09-10T22:16:43.302 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44318.log.gz 2024-09-10T22:16:43.303 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49387.log 2024-09-10T22:16:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43980.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50758.log 2024-09-10T22:16:43.308 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49387.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49387.log.gz 2024-09-10T22:16:43.310 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43980.log.gz 2024-09-10T22:16:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50758.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50758.log.gz 2024-09-10T22:16:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35423.log 2024-09-10T22:16:43.319 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47286.log 2024-09-10T22:16:43.320 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35423.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35423.log.gz 2024-09-10T22:16:43.324 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47286.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47286.log.gz 2024-09-10T22:16:43.326 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44201.log 2024-09-10T22:16:43.333 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44201.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37562.log 2024-09-10T22:16:43.333 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44201.log.gz 2024-09-10T22:16:43.335 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52183.log 2024-09-10T22:16:43.339 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42727.log 2024-09-10T22:16:43.340 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47312.log 2024-09-10T22:16:43.340 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52183.log.gz 2024-09-10T22:16:43.342 INFO:teuthology.orchestra.run.smithi066.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37562.log.gz 2024-09-10T22:16:43.347 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47312.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47312.log.gz 2024-09-10T22:16:43.348 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42727.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42727.log.gz 2024-09-10T22:16:43.348 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39941.log 2024-09-10T22:16:43.352 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45607.log 2024-09-10T22:16:43.353 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39941.log: gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39941.log.gz 2024-09-10T22:16:43.353 INFO:teuthology.orchestra.run.smithi066.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47232.log 2024-09-10T22:16:43.353 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44348.log 2024-09-10T22:16:43.359 INFO:teuthology.orchestra.run.smithi066.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.45607.log.gz 2024-09-10T22:16:43.360 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47232.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47232.log.gz 2024-09-10T22:16:43.360 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43162.log 2024-09-10T22:16:43.360 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44348.log.gz 2024-09-10T22:16:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42831.log 2024-09-10T22:16:43.368 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40408.log 2024-09-10T22:16:43.369 INFO:teuthology.orchestra.run.smithi066.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.43162.log.gz 2024-09-10T22:16:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42831.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42831.log.gz 2024-09-10T22:16:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40889.log 2024-09-10T22:16:43.378 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40408.log.gz 2024-09-10T22:16:43.379 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55600.log 2024-09-10T22:16:43.380 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40889.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40889.log.gz 2024-09-10T22:16:43.384 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55600.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55600.log.gz 2024-09-10T22:16:43.386 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48740.log 2024-09-10T22:16:43.391 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54077.log 2024-09-10T22:16:43.392 INFO:teuthology.orchestra.run.smithi094.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T22:16:43.392 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48740.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48740.log.gz 2024-09-10T22:16:43.395 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39191.log 2024-09-10T22:16:43.396 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54077.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54077.log.gz 2024-09-10T22:16:43.399 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53670.log 2024-09-10T22:16:43.400 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39191.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39191.log.gz 2024-09-10T22:16:43.403 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44402.log 2024-09-10T22:16:43.405 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53670.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53670.log.gz 2024-09-10T22:16:43.409 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53542.log 2024-09-10T22:16:43.415 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44402.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44402.log.gz 2024-09-10T22:16:43.415 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49198.log 2024-09-10T22:16:43.415 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53542.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53542.log.gz 2024-09-10T22:16:43.415 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35186.log 2024-09-10T22:16:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49198.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49198.log.gz 2024-09-10T22:16:43.422 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35186.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35186.log.gz 2024-09-10T22:16:43.423 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36444.log 2024-09-10T22:16:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42229.log 2024-09-10T22:16:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36444.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36444.log.gz 2024-09-10T22:16:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53490.log 2024-09-10T22:16:43.432 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42229.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42229.log.gz 2024-09-10T22:16:43.439 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39986.log 2024-09-10T22:16:43.699 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53490.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53490.log.gz 2024-09-10T22:16:43.699 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44662.log 2024-09-10T22:16:43.699 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39986.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39986.log.gz 2024-09-10T22:16:43.699 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29036.log 2024-09-10T22:16:43.699 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44662.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44662.log.gz 2024-09-10T22:16:43.699 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45460.log 2024-09-10T22:16:43.699 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.29036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29036.log.gz 2024-09-10T22:16:43.700 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40230.log 2024-09-10T22:16:43.700 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45460.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.45460.log.gz 2024-09-10T22:16:43.700 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52948.log 2024-09-10T22:16:43.700 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40230.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40230.log.gz 2024-09-10T22:16:43.700 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52948.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52948.log.gz 2024-09-10T22:16:43.700 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37106.log 2024-09-10T22:16:43.700 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52922.log 2024-09-10T22:16:43.700 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37106.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.37106.log.gz 2024-09-10T22:16:43.700 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28976.log 2024-09-10T22:16:43.700 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52922.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52922.log.gz 2024-09-10T22:16:43.700 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34458.log 2024-09-10T22:16:43.700 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28976.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28976.log.gz 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47623.log 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34458.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34458.log.gz 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54735.log 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47623.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47623.log.gz 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35740.log 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54735.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54735.log.gz 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38736.log 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35740.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35740.log.gz 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38736.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38736.log.gz 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51294.log 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45539.log 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51294.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51294.log.gz 2024-09-10T22:16:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38250.log 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45539.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45539.log.gz 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55268.log 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38250.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38250.log.gz 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53516.log 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55268.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55268.log.gz 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46539.log 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53516.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.53516.log.gz 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49276.log 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46539.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46539.log.gz 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34778.log 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49276.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49276.log.gz 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34778.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34778.log.gz 2024-09-10T22:16:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44266.log 2024-09-10T22:16:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37666.log 2024-09-10T22:16:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44266.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44266.log.gz 2024-09-10T22:16:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47559.log 2024-09-10T22:16:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37666.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37666.log.gz 2024-09-10T22:16:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49172.log 2024-09-10T22:16:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47559.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47559.log.gz 2024-09-10T22:16:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38315.log 2024-09-10T22:16:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49172.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49172.log.gz 2024-09-10T22:16:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36128.log 2024-09-10T22:16:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38315.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38315.log.gz 2024-09-10T22:16:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28934.log 2024-09-10T22:16:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36128.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36128.log.gz 2024-09-10T22:16:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43450.log 2024-09-10T22:16:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28934.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28934.log.gz 2024-09-10T22:16:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43450.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43450.log.gz 2024-09-10T22:16:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48582.log 2024-09-10T22:16:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48582.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48582.log.gz 2024-09-10T22:16:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39809.log 2024-09-10T22:16:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38406.log 2024-09-10T22:16:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39809.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39809.log.gz 2024-09-10T22:16:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43554.log 2024-09-10T22:16:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38406.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38406.log.gz 2024-09-10T22:16:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43554.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43554.log.gz 2024-09-10T22:16:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50600.log 2024-09-10T22:16:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38788.log 2024-09-10T22:16:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38968.log 2024-09-10T22:16:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50600.log: /var/log/ceph/ceph-client.admin.38788.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50600.log.gz 2024-09-10T22:16:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50358.log 2024-09-10T22:16:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38968.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38788.log.gz 2024-09-10T22:16:43.705 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38968.log.gz 2024-09-10T22:16:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39455.log 2024-09-10T22:16:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50358.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50358.log.gz 2024-09-10T22:16:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39455.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39455.log.gz 2024-09-10T22:16:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37314.log 2024-09-10T22:16:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54839.log 2024-09-10T22:16:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37314.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37314.log.gz 2024-09-10T22:16:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53801.log 2024-09-10T22:16:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54839.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.54839.log.gz 2024-09-10T22:16:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50891.log 2024-09-10T22:16:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53801.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53801.log.gz 2024-09-10T22:16:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50891.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50891.log.gz 2024-09-10T22:16:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47941.log 2024-09-10T22:16:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42194.log 2024-09-10T22:16:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47941.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.47941.log.gz 2024-09-10T22:16:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46357.log 2024-09-10T22:16:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42194.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42194.log.gz 2024-09-10T22:16:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34792.log 2024-09-10T22:16:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46357.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46357.log.gz 2024-09-10T22:16:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34986.log 2024-09-10T22:16:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34792.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34792.log.gz 2024-09-10T22:16:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34986.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34986.log.gz 2024-09-10T22:16:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55860.log 2024-09-10T22:16:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45304.log 2024-09-10T22:16:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55860.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55860.log.gz 2024-09-10T22:16:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33821.log 2024-09-10T22:16:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45304.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45304.log.gz 2024-09-10T22:16:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45732.log 2024-09-10T22:16:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.33821.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33821.log.gz 2024-09-10T22:16:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29000.log 2024-09-10T22:16:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45732.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.45732.log.gz 2024-09-10T22:16:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52597.log 2024-09-10T22:16:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.29000.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29000.log.gz 2024-09-10T22:16:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52597.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52597.log.gz 2024-09-10T22:16:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54291.log 2024-09-10T22:16:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54291.log: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.54291.log.gz 2024-09-10T22:16:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49853.log 2024-09-10T22:16:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42894.log 2024-09-10T22:16:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49853.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49853.log.gz 2024-09-10T22:16:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53438.log 2024-09-10T22:16:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42894.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42894.log.gz 2024-09-10T22:16:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43266.log 2024-09-10T22:16:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53438.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53438.log.gz 2024-09-10T22:16:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37156.log 2024-09-10T22:16:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43266.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.43266.log.gz 2024-09-10T22:16:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34992.log 2024-09-10T22:16:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37156.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37156.log.gz 2024-09-10T22:16:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35005.log 2024-09-10T22:16:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34992.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34992.log.gz 2024-09-10T22:16:43.718 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47442.log 2024-09-10T22:16:43.720 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35005.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35005.log.gz 2024-09-10T22:16:43.724 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47442.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47442.log.gz 2024-09-10T22:16:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53265.log 2024-09-10T22:16:43.731 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47128.log 2024-09-10T22:16:43.732 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53265.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53265.log.gz 2024-09-10T22:16:43.737 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47128.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47128.log.gz 2024-09-10T22:16:43.738 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55522.log 2024-09-10T22:16:43.743 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45486.log 2024-09-10T22:16:43.744 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52358.log 2024-09-10T22:16:43.745 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55522.log.gz 2024-09-10T22:16:43.746 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45486.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.45486.log.gz 2024-09-10T22:16:43.749 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52358.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.52358.log.gz 2024-09-10T22:16:43.749 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39559.log 2024-09-10T22:16:43.755 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34797.log 2024-09-10T22:16:43.756 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39559.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39559.log.gz 2024-09-10T22:16:43.760 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34797.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34797.log.gz 2024-09-10T22:16:43.763 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52974.log 2024-09-10T22:16:43.763 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40497.log 2024-09-10T22:16:43.765 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52974.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52974.log.gz 2024-09-10T22:16:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45062.log 2024-09-10T22:16:43.768 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40497.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40497.log.gz 2024-09-10T22:16:43.771 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50862.log 2024-09-10T22:16:43.772 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45062.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45062.log.gz 2024-09-10T22:16:43.775 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40142.log 2024-09-10T22:16:43.777 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50862.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50862.log.gz 2024-09-10T22:16:43.779 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34798.log 2024-09-10T22:16:43.781 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40142.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40142.log.gz 2024-09-10T22:16:43.783 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45356.log 2024-09-10T22:16:43.784 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34798.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34798.log.gz 2024-09-10T22:16:43.787 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T22:16:43.788 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45356.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45356.log.gz 2024-09-10T22:16:43.794 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41877.log 2024-09-10T22:16:43.797 INFO:teuthology.orchestra.run.smithi066.stderr: 89.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48407.log 2024-09-10T22:16:43.799 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41877.log: 87.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T22:16:43.799 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41877.log.gz 2024-09-10T22:16:43.803 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35956.log 2024-09-10T22:16:43.804 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48407.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48407.log.gz 2024-09-10T22:16:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29367.log 2024-09-10T22:16:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35956.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35956.log.gz 2024-09-10T22:16:43.812 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.29367.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.29367.log.gz 2024-09-10T22:16:43.814 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41624.log 2024-09-10T22:16:43.818 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34782.log 2024-09-10T22:16:43.820 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41624.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41624.log.gz 2024-09-10T22:16:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34782.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34782.log.gz 2024-09-10T22:16:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-10T22:16:43.831 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54234.log 2024-09-10T22:16:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47837.log 2024-09-10T22:16:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52870.log 2024-09-10T22:16:43.836 INFO:teuthology.orchestra.run.smithi066.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.54234.log.gz 2024-09-10T22:16:43.844 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47837.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47837.log.gz 2024-09-10T22:16:43.844 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46643.log 2024-09-10T22:16:43.844 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52870.log.gz 2024-09-10T22:16:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48792.log 2024-09-10T22:16:43.850 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46643.log.gz 2024-09-10T22:16:43.851 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-10T22:16:43.852 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48792.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.48792.log.gz 2024-09-10T22:16:43.858 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36807.log 2024-09-10T22:16:43.864 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36807.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36807.log.gz 2024-09-10T22:16:43.866 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46150.log 2024-09-10T22:16:43.872 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42649.log 2024-09-10T22:16:43.872 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46150.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46150.log.gz 2024-09-10T22:16:43.880 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42649.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42649.log.gz 2024-09-10T22:16:43.882 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38606.log 2024-09-10T22:16:43.888 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37839.log 2024-09-10T22:16:43.888 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38606.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38606.log.gz 2024-09-10T22:16:43.896 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38283.log 2024-09-10T22:16:43.896 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37839.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37839.log.gz 2024-09-10T22:16:43.904 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38283.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38283.log.gz 2024-09-10T22:16:43.906 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28868.log 2024-09-10T22:16:43.912 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46204.log 2024-09-10T22:16:43.912 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28868.log: 74.7% -- replaced with /var/log/ceph/ceph-client.admin.28868.log.gz 2024-09-10T22:16:43.920 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41455.log 2024-09-10T22:16:43.921 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46204.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46204.log.gz 2024-09-10T22:16:43.928 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41455.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41455.log.gz 2024-09-10T22:16:43.930 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51268.log 2024-09-10T22:16:43.936 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35768.log 2024-09-10T22:16:43.937 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51268.log.gz 2024-09-10T22:16:43.943 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46461.log 2024-09-10T22:16:43.944 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35768.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35768.log.gz 2024-09-10T22:16:43.948 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51320.log 2024-09-10T22:16:43.954 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46461.log.gz 2024-09-10T22:16:43.956 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46903.log 2024-09-10T22:16:43.956 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51320.log.gz 2024-09-10T22:16:43.962 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35852.log 2024-09-10T22:16:43.963 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46903.log.gz 2024-09-10T22:16:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40197.log 2024-09-10T22:16:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35852.log: 87.4%/var/log/ceph/ceph-client.admin.40197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45120.log 2024-09-10T22:16:43.973 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40197.log.gz 2024-09-10T22:16:43.974 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35852.log.gz 2024-09-10T22:16:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45120.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.45120.log.gz 2024-09-10T22:16:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48097.log 2024-09-10T22:16:43.982 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48303.log 2024-09-10T22:16:43.988 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48097.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48097.log.gz 2024-09-10T22:16:43.988 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42111.log 2024-09-10T22:16:43.988 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48303.log.gz 2024-09-10T22:16:43.989 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51190.log 2024-09-10T22:16:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42111.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42111.log.gz 2024-09-10T22:16:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37891.log 2024-09-10T22:16:43.996 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51190.log.gz 2024-09-10T22:16:43.997 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51348.log 2024-09-10T22:16:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37891.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37891.log.gz 2024-09-10T22:16:44.004 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40262.log 2024-09-10T22:16:44.004 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51348.log.gz 2024-09-10T22:16:44.006 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51530.log 2024-09-10T22:16:44.010 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40262.log.gz 2024-09-10T22:16:44.016 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51530.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51530.log.gz 2024-09-10T22:16:44.017 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34121.log 2024-09-10T22:16:44.022 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34121.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34121.log.gz 2024-09-10T22:16:44.023 INFO:teuthology.orchestra.run.smithi066.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44084.log 2024-09-10T22:16:44.025 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43240.log 2024-09-10T22:16:44.030 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44084.log.gz 2024-09-10T22:16:44.032 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45174.log 2024-09-10T22:16:44.032 INFO:teuthology.orchestra.run.smithi066.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.43240.log.gz 2024-09-10T22:16:44.039 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37718.log 2024-09-10T22:16:44.039 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45174.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.45174.log.gz 2024-09-10T22:16:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37718.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37718.log.gz 2024-09-10T22:16:44.050 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37917.log 2024-09-10T22:16:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37917.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37917.log.gz 2024-09-10T22:16:44.058 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53082.log 2024-09-10T22:16:44.064 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44610.log 2024-09-10T22:16:44.064 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53082.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53082.log.gz 2024-09-10T22:16:44.072 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43032.log 2024-09-10T22:16:44.072 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44610.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44610.log.gz 2024-09-10T22:16:44.075 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51769.log 2024-09-10T22:16:44.078 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43032.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43032.log.gz 2024-09-10T22:16:44.079 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29458.log 2024-09-10T22:16:44.083 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46825.log 2024-09-10T22:16:44.084 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.29458.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29458.log.gz 2024-09-10T22:16:44.086 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51769.log.gz 2024-09-10T22:16:44.087 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47649.log 2024-09-10T22:16:44.092 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46825.log: 86.9%/var/log/ceph/ceph-client.admin.47649.log: -- replaced with /var/log/ceph/ceph-client.admin.46825.log.gz 2024-09-10T22:16:44.092 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47649.log.gz 2024-09-10T22:16:44.094 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53827.log 2024-09-10T22:16:44.100 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54918.log 2024-09-10T22:16:44.100 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53827.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53827.log.gz 2024-09-10T22:16:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35290.log 2024-09-10T22:16:44.109 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54918.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54918.log.gz 2024-09-10T22:16:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50924.log 2024-09-10T22:16:44.114 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35290.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35290.log.gz 2024-09-10T22:16:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47889.log 2024-09-10T22:16:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50924.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47044.log 2024-09-10T22:16:44.122 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50924.log.gz 2024-09-10T22:16:44.124 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47889.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47889.log.gz 2024-09-10T22:16:44.124 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55678.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47044.log.gz 2024-09-10T22:16:44.125 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:16:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41138.log 2024-09-10T22:16:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55678.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55678.log.gz 2024-09-10T22:16:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41400.log 2024-09-10T22:16:44.132 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41138.log.gz 2024-09-10T22:16:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46256.log 2024-09-10T22:16:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41400.log: 87.0%/var/log/ceph/ceph-client.admin.46256.log: -- replaced with /var/log/ceph/ceph-client.admin.41400.log.gz 2024-09-10T22:16:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46256.log.gz 2024-09-10T22:16:44.140 INFO:teuthology.orchestra.run.smithi066.stderr: -- /var/log/ceph/ceph-client.admin.45094.log 2024-09-10T22:16:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36418.log 2024-09-10T22:16:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45094.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45094.log.gz 2024-09-10T22:16:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36418.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36418.log.gz 2024-09-10T22:16:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51847.log 2024-09-10T22:16:44.155 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47102.log 2024-09-10T22:16:44.158 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51847.log.gz 2024-09-10T22:16:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36288.log 2024-09-10T22:16:44.174 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47102.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47102.log.gz 2024-09-10T22:16:44.174 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36288.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36288.log.gz 2024-09-10T22:16:44.174 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36778.log 2024-09-10T22:16:44.174 INFO:teuthology.orchestra.run.smithi066.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T22:16:44.177 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36778.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53382.log 2024-09-10T22:16:44.177 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36778.log.gz 2024-09-10T22:16:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55886.log 2024-09-10T22:16:44.183 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28988.log 2024-09-10T22:16:44.184 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53382.log.gz 2024-09-10T22:16:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55886.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55886.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54420.log 2024-09-10T22:16:44.187 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:16:44.190 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28988.log.gz 2024-09-10T22:16:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48530.log 2024-09-10T22:16:44.194 INFO:teuthology.orchestra.run.smithi066.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54420.log.gz 2024-09-10T22:16:44.200 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48530.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48530.log.gz 2024-09-10T22:16:44.202 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40994.log 2024-09-10T22:16:44.203 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48071.log 2024-09-10T22:16:44.208 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40994.log: /var/log/ceph/ceph-client.admin.48071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43188.log 2024-09-10T22:16:44.208 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48071.log.gz 2024-09-10T22:16:44.210 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40994.log.gz 2024-09-10T22:16:44.215 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.23993.log 2024-09-10T22:16:44.216 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43188.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43188.log.gz 2024-09-10T22:16:44.219 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.tmp-client.admin.23993.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.23993.log.gz 2024-09-10T22:16:44.220 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42753.log 2024-09-10T22:16:44.227 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41747.log 2024-09-10T22:16:44.228 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42753.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42753.log.gz 2024-09-10T22:16:44.230 INFO:teuthology.orchestra.run.smithi066.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.52005.log.gz 2024-09-10T22:16:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41747.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41747.log.gz 2024-09-10T22:16:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53594.log 2024-09-10T22:16:44.239 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53056.log 2024-09-10T22:16:44.240 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53594.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53594.log.gz 2024-09-10T22:16:44.244 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53056.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53056.log.gz 2024-09-10T22:16:44.246 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41676.log 2024-09-10T22:16:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36236.log 2024-09-10T22:16:44.252 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41676.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41676.log.gz 2024-09-10T22:16:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36236.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36236.log.gz 2024-09-10T22:16:44.258 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42954.log 2024-09-10T22:16:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37813.log 2024-09-10T22:16:44.264 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42954.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42954.log.gz 2024-09-10T22:16:44.267 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47993.log 2024-09-10T22:16:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37813.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37813.log.gz 2024-09-10T22:16:44.272 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47993.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47993.log.gz 2024-09-10T22:16:44.274 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39692.log 2024-09-10T22:16:44.279 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54682.log 2024-09-10T22:16:44.280 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39692.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39692.log.gz 2024-09-10T22:16:44.283 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44169.log 2024-09-10T22:16:44.284 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54682.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54682.log.gz 2024-09-10T22:16:44.286 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40523.log 2024-09-10T22:16:44.289 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44169.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44169.log.gz 2024-09-10T22:16:44.291 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39887.log 2024-09-10T22:16:44.292 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40523.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40523.log.gz 2024-09-10T22:16:44.295 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49697.log 2024-09-10T22:16:44.296 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39887.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39887.log.gz 2024-09-10T22:16:44.298 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44770.log 2024-09-10T22:16:44.300 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49697.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49697.log.gz 2024-09-10T22:16:44.306 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.44770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52464.log 2024-09-10T22:16:44.311 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48173.log 2024-09-10T22:16:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52464.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52464.log.gz 2024-09-10T22:16:44.320 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48173.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48173.log.gz 2024-09-10T22:16:44.320 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52792.log 2024-09-10T22:16:44.322 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38994.log 2024-09-10T22:16:44.324 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36101.log 2024-09-10T22:16:44.331 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38994.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38994.log.gz 2024-09-10T22:16:44.331 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52792.log.gz 2024-09-10T22:16:44.333 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36101.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36101.log.gz 2024-09-10T22:16:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34692.log 2024-09-10T22:16:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55444.log 2024-09-10T22:16:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34692.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34692.log.gz 2024-09-10T22:16:44.349 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55444.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55444.log.gz 2024-09-10T22:16:44.349 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44712.log 2024-09-10T22:16:44.351 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47364.log 2024-09-10T22:16:44.353 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44712.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44712.log.gz 2024-09-10T22:16:44.356 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47364.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47364.log.gz 2024-09-10T22:16:44.356 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50522.log 2024-09-10T22:16:44.364 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50522.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50522.log.gz 2024-09-10T22:16:44.366 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39247.log 2024-09-10T22:16:44.371 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44480.log 2024-09-10T22:16:44.373 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39247.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39247.log.gz 2024-09-10T22:16:44.378 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44480.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36868.log 2024-09-10T22:16:44.382 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44480.log.gz 2024-09-10T22:16:44.382 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55756.log 2024-09-10T22:16:44.388 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36868.log: /var/log/ceph/ceph-client.admin.55756.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55756.log.gz 2024-09-10T22:16:44.390 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54709.log 2024-09-10T22:16:44.390 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36868.log.gz 2024-09-10T22:16:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45581.log 2024-09-10T22:16:44.396 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54709.log: /var/log/ceph/ceph-client.admin.45581.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.45581.log.gz 2024-09-10T22:16:44.398 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54709.log.gz 2024-09-10T22:16:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45839.log 2024-09-10T22:16:44.403 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55294.log 2024-09-10T22:16:44.404 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45839.log: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.45839.log.gz 2024-09-10T22:16:44.412 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55294.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55294.log.gz 2024-09-10T22:16:44.413 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46721.log 2024-09-10T22:16:44.414 INFO:teuthology.orchestra.run.smithi066.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.50091.log.gz 2024-09-10T22:16:44.415 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55912.log 2024-09-10T22:16:44.419 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46721.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46721.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45382.log 2024-09-10T22:16:44.419 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:16:44.424 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55912.log: 87.3%/var/log/ceph/ceph-client.admin.45382.log: 86.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52568.log 2024-09-10T22:16:44.426 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55912.log.gz 2024-09-10T22:16:44.428 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52568.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52568.log.gz 2024-09-10T22:16:44.430 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45382.log.gz 2024-09-10T22:16:44.430 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36047.log 2024-09-10T22:16:44.435 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55065.log 2024-09-10T22:16:44.436 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36047.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36047.log.gz 2024-09-10T22:16:44.439 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49536.log 2024-09-10T22:16:44.441 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55065.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55065.log.gz 2024-09-10T22:16:44.443 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43058.log 2024-09-10T22:16:44.446 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49536.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49536.log.gz 2024-09-10T22:16:44.447 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47180.log 2024-09-10T22:16:44.450 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43058.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.43058.log.gz 2024-09-10T22:16:44.452 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43668.log 2024-09-10T22:16:44.453 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47180.log.gz 2024-09-10T22:16:44.455 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52053.log 2024-09-10T22:16:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43668.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46773.log 2024-09-10T22:16:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52053.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52053.log.gz 2024-09-10T22:16:44.462 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43668.log.gz 2024-09-10T22:16:44.464 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46773.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46773.log.gz 2024-09-10T22:16:44.464 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40436.log 2024-09-10T22:16:44.467 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46305.log 2024-09-10T22:16:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40436.log: /var/log/ceph/ceph-client.admin.46305.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46305.log.gz 2024-09-10T22:16:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42020.log 2024-09-10T22:16:44.474 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40436.log.gz 2024-09-10T22:16:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46669.log 2024-09-10T22:16:44.480 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42020.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42020.log.gz 2024-09-10T22:16:44.484 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46669.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46669.log.gz 2024-09-10T22:16:44.486 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47863.log 2024-09-10T22:16:44.491 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40038.log 2024-09-10T22:16:44.492 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47863.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47863.log.gz 2024-09-10T22:16:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50810.log 2024-09-10T22:16:44.496 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40038.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40038.log.gz 2024-09-10T22:16:44.499 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55730.log 2024-09-10T22:16:44.503 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48355.log 2024-09-10T22:16:44.504 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55730.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55730.log.gz 2024-09-10T22:16:44.507 INFO:teuthology.orchestra.run.smithi066.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.50810.log.gz 2024-09-10T22:16:44.508 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48355.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48355.log.gz 2024-09-10T22:16:44.509 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44584.log 2024-09-10T22:16:44.515 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42443.log 2024-09-10T22:16:44.516 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44584.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44584.log.gz 2024-09-10T22:16:44.520 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42443.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42443.log.gz 2024-09-10T22:16:44.522 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43214.log 2024-09-10T22:16:44.527 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41572.log 2024-09-10T22:16:44.528 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43214.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.43214.log.gz 2024-09-10T22:16:44.531 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54602.log 2024-09-10T22:16:44.532 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41572.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41572.log.gz 2024-09-10T22:16:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:gzip/var/log/ceph/ceph-client.admin.54602.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54602.log.gz 2024-09-10T22:16:44.538 INFO:teuthology.orchestra.run.smithi066.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49120.log 2024-09-10T22:16:44.543 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37943.log 2024-09-10T22:16:44.544 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49120.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49120.log.gz 2024-09-10T22:16:44.547 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50652.log 2024-09-10T22:16:44.548 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37943.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37943.log.gz 2024-09-10T22:16:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55195.log 2024-09-10T22:16:44.553 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50652.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50652.log.gz 2024-09-10T22:16:44.556 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55195.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55195.log.gz 2024-09-10T22:16:44.558 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40941.log 2024-09-10T22:16:44.559 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49671.log 2024-09-10T22:16:44.560 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40941.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40941.log.gz 2024-09-10T22:16:44.563 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46230.log 2024-09-10T22:16:44.564 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49671.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49671.log.gz 2024-09-10T22:16:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46230.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46230.log.gz 2024-09-10T22:16:44.570 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41546.log 2024-09-10T22:16:44.575 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40064.log 2024-09-10T22:16:44.576 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41546.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41546.log.gz 2024-09-10T22:16:44.579 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41166.log 2024-09-10T22:16:44.580 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40064.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40064.log.gz 2024-09-10T22:16:44.583 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50191.log 2024-09-10T22:16:44.584 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41166.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.41166.log.gz 2024-09-10T22:16:44.587 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38580.log 2024-09-10T22:16:44.588 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50191.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50191.log.gz 2024-09-10T22:16:44.592 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38580.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38580.log.gz 2024-09-10T22:16:44.594 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49931.log 2024-09-10T22:16:44.599 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34536.log 2024-09-10T22:16:44.600 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49931.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49931.log.gz 2024-09-10T22:16:44.603 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39915.log 2024-09-10T22:16:44.604 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34536.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34536.log.gz 2024-09-10T22:16:44.608 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39915.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39915.log.gz 2024-09-10T22:16:44.610 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55169.log 2024-09-10T22:16:44.615 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49645.log 2024-09-10T22:16:44.616 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55169.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55169.log.gz 2024-09-10T22:16:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38489.log 2024-09-10T22:16:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49645.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49645.log.gz 2024-09-10T22:16:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38489.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38489.log.gz 2024-09-10T22:16:44.626 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46695.log 2024-09-10T22:16:44.632 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34562.log 2024-09-10T22:16:44.632 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46695.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46695.log.gz 2024-09-10T22:16:44.635 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29012.log 2024-09-10T22:16:44.642 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.34562.log: 87.1%/var/log/ceph/ceph-client.admin.29012.log: -- replaced with /var/log/ceph/ceph-client.admin.34562.log.gz 2024-09-10T22:16:44.642 INFO:teuthology.orchestra.run.smithi066.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29012.log.gz 2024-09-10T22:16:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38094.log 2024-09-10T22:16:44.647 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49068.log 2024-09-10T22:16:44.648 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.38094.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38094.log.gz 2024-09-10T22:16:44.652 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49068.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49068.log.gz 2024-09-10T22:16:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28817.log 2024-09-10T22:16:44.660 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36566.log 2024-09-10T22:16:44.661 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.28817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28817.log.gz 2024-09-10T22:16:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36540.log 2024-09-10T22:16:44.666 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36566.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36566.log.gz 2024-09-10T22:16:44.667 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53321.log 2024-09-10T22:16:44.668 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36540.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36540.log.gz 2024-09-10T22:16:44.670 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49879.log 2024-09-10T22:16:44.675 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53321.log.gz 2024-09-10T22:16:44.675 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42495.log 2024-09-10T22:16:44.676 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49879.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49879.log.gz 2024-09-10T22:16:44.679 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29024.log 2024-09-10T22:16:44.681 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42495.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42495.log.gz 2024-09-10T22:16:44.681 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.29024.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29024.log.gz 2024-09-10T22:16:44.686 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50295.log 2024-09-10T22:16:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53002.log 2024-09-10T22:16:44.692 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40709.log 2024-09-10T22:16:44.693 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50295.log.gz 2024-09-10T22:16:44.695 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46124.log 2024-09-10T22:16:44.696 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40709.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53002.log.gz 2024-09-10T22:16:44.697 INFO:teuthology.orchestra.run.smithi066.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.42601.log.gz 2024-09-10T22:16:44.699 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40709.log.gz 2024-09-10T22:16:44.699 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44844.log 2024-09-10T22:16:44.701 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46124.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.46124.log.gz 2024-09-10T22:16:44.702 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36262.log 2024-09-10T22:16:44.704 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45970.log 2024-09-10T22:16:44.705 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36262.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36262.log.gz 2024-09-10T22:16:44.706 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44844.log.gz 2024-09-10T22:16:44.709 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45970.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.45970.log.gz 2024-09-10T22:16:44.710 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52516.log 2024-09-10T22:16:44.713 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52516.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.52516.log.gz 2024-09-10T22:16:44.713 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47705.log 2024-09-10T22:16:44.716 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47705.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47705.log.gz 2024-09-10T22:16:44.716 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35904.log 2024-09-10T22:16:44.722 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35904.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35904.log.gz 2024-09-10T22:16:44.722 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54865.log 2024-09-10T22:16:44.722 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54865.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54865.log.gz 2024-09-10T22:16:44.722 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39481.log 2024-09-10T22:16:44.726 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.39481.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39481.log.gz 2024-09-10T22:16:44.726 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50732.log 2024-09-10T22:16:44.727 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41488.log 2024-09-10T22:16:44.731 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.41488.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41488.log.gz 2024-09-10T22:16:44.731 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50732.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50732.log.gz 2024-09-10T22:16:44.732 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54156.log 2024-09-10T22:16:44.734 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54156.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54156.log.gz 2024-09-10T22:16:44.734 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48818.log 2024-09-10T22:16:44.740 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48818.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48818.log.gz 2024-09-10T22:16:44.741 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40327.log 2024-09-10T22:16:44.741 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40327.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40327.log.gz 2024-09-10T22:16:44.741 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43370.log 2024-09-10T22:16:44.742 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46591.log 2024-09-10T22:16:44.744 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43370.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43370.log.gz 2024-09-10T22:16:44.750 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46591.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46591.log.gz 2024-09-10T22:16:44.750 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46176.log 2024-09-10T22:16:44.750 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46176.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46176.log.gz 2024-09-10T22:16:44.750 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40968.log 2024-09-10T22:16:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.40968.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40968.log.gz 2024-09-10T22:16:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54368.log 2024-09-10T22:16:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54394.log 2024-09-10T22:16:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54368.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.54368.log.gz 2024-09-10T22:16:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36646.log 2024-09-10T22:16:44.758 INFO:teuthology.orchestra.run.smithi066.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.54394.log.gz 2024-09-10T22:16:44.761 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36646.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36646.log.gz 2024-09-10T22:16:44.762 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44110.log 2024-09-10T22:16:44.764 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40579.log 2024-09-10T22:16:44.764 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44110.log.gz 2024-09-10T22:16:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35316.log/var/log/ceph/ceph-client.admin.40579.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40579.log.gz 2024-09-10T22:16:44.768 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:16:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.35316.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35316.log.gz 2024-09-10T22:16:44.936 INFO:teuthology.orchestra.run.smithi066.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.53727.log.gz 2024-09-10T22:16:45.390 INFO:teuthology.orchestra.run.smithi066.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.48229.log.gz 2024-09-10T22:16:45.842 INFO:teuthology.orchestra.run.smithi094.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T22:16:46.102 INFO:teuthology.orchestra.run.smithi066.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.44770.log.gz 2024-09-10T22:16:46.398 INFO:teuthology.orchestra.run.smithi066.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T22:16:46.472 INFO:teuthology.orchestra.run.smithi094.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T22:16:46.600 INFO:teuthology.orchestra.run.smithi094.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T22:16:47.414 INFO:teuthology.orchestra.run.smithi094.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T22:16:47.436 INFO:teuthology.orchestra.run.smithi066.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.40841.log.gz 2024-09-10T22:16:47.577 INFO:teuthology.orchestra.run.smithi094.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T22:16:47.625 INFO:teuthology.orchestra.run.smithi066.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.37214.log.gz 2024-09-10T22:16:48.047 INFO:teuthology.orchestra.run.smithi066.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.38894.log.gz 2024-09-10T22:16:48.273 INFO:teuthology.orchestra.run.smithi066.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T22:16:48.321 INFO:teuthology.orchestra.run.smithi066.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T22:16:48.601 INFO:teuthology.orchestra.run.smithi094.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T22:16:48.603 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-10T22:16:48.603 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m6.492s 2024-09-10T22:16:48.603 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m29.388s 2024-09-10T22:16:48.603 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m2.346s 2024-09-10T22:16:48.687 INFO:teuthology.orchestra.run.smithi066.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T22:16:48.803 INFO:teuthology.orchestra.run.smithi066.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T22:16:49.450 INFO:teuthology.orchestra.run.smithi066.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T22:16:49.457 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:16:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m7.360s 2024-09-10T22:16:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:user 0m44.251s 2024-09-10T22:16:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m4.577s 2024-09-10T22:16:49.457 INFO:tasks.ceph:Archiving logs... 2024-09-10T22:16:49.458 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705/remote/smithi066/log 2024-09-10T22:16:49.458 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T22:16:53.402 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705/remote/smithi094/log 2024-09-10T22:16:53.403 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T22:16:56.296 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T22:16:56.305 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T22:16:56.306 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T22:16:56.309 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T22:16:56.410 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T22:16:56.423 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T22:16:56.424 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:16:56.428 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:16:56.441 INFO:teuthology.orchestra.run.smithi066.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T22:16:56.441 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================== 2024-09-10T22:16:56.441 INFO:teuthology.orchestra.run.smithi066.stdout:*hv01.front.sepi 67.205.162.81 3 u 36 64 377 0.085 +0.894 0.113 2024-09-10T22:16:56.441 INFO:teuthology.orchestra.run.smithi066.stdout:+hv02.front.sepi 63.231.80.2 3 u 22 64 377 0.065 -0.446 0.214 2024-09-10T22:16:56.441 INFO:teuthology.orchestra.run.smithi066.stdout:+hv03.front.sepi 74.6.168.72 3 u 33 64 377 0.081 +1.564 0.109 2024-09-10T22:16:56.441 INFO:teuthology.orchestra.run.smithi066.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-10T22:16:56.443 INFO:teuthology.orchestra.run.smithi094.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T22:16:56.444 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================== 2024-09-10T22:16:56.444 INFO:teuthology.orchestra.run.smithi094.stdout:*hv01.front.sepi 67.205.162.81 3 u 31 64 377 0.095 -2.742 0.701 2024-09-10T22:16:56.444 INFO:teuthology.orchestra.run.smithi094.stdout:+hv02.front.sepi 63.231.80.2 3 u 32 64 377 0.117 -4.663 0.329 2024-09-10T22:16:56.444 INFO:teuthology.orchestra.run.smithi094.stdout:+hv03.front.sepi 74.6.168.72 3 u 31 64 377 0.099 -3.066 0.507 2024-09-10T22:16:56.444 INFO:teuthology.orchestra.run.smithi094.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-10T22:16:56.445 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T22:16:56.510 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T22:16:56.510 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T22:16:56.618 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T22:16:56.665 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T22:16:56.743 INFO:teuthology.task.internal:Duration was 1717.492659 seconds 2024-09-10T22:16:56.744 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T22:16:56.805 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T22:16:56.805 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T22:16:56.809 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T22:16:56.901 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T22:16:56.901 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:16:56.901 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T22:16:56.921 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:16:56.922 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T22:16:56.943 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T22:16:56.943 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T22:16:56.969 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T22:16:57.028 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T22:16:57.028 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T22:16:57.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T22:16:57.147 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T22:16:57.156 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T22:16:57.157 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T22:16:57.177 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T22:16:57.196 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T22:16:57.244 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:16:57.247 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:16:57.262 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-09-10T22:16:57.264 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-09-10T22:16:57.282 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:16:57.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:16:57.333 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:16:57.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:16:57.340 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T22:16:57.349 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T22:16:57.350 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705/remote/smithi066 2024-09-10T22:16:57.350 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T22:16:57.406 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899705/remote/smithi094 2024-09-10T22:16:57.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T22:16:57.436 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T22:16:57.436 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T22:16:57.448 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T22:16:57.481 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T22:16:57.492 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T22:16:57.492 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T22:16:57.568 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T22:16:57.569 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T22:16:57.572 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T22:16:57.578 INFO:teuthology.orchestra.run.smithi066.stdout: 655462 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 22:16 /home/ubuntu/cephtest 2024-09-10T22:16:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: 658141 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 22:16 /home/ubuntu/cephtest 2024-09-10T22:16:57.582 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T22:16:57.624 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T22:16:57.768 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} duration: 1717.4926588535309 flavor: default mon.a-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a mon.b-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a owner: scheduled_hyelloji@teuthology success: true 2024-09-10T22:16:57.768 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T22:16:57.863 INFO:teuthology.run:pass